PnetCDF benchmarking results

[Peter Schmitt]

PnetCDF User's Group,

I had an internship over the summer of 2004 through the Division of 
Educational Programs at ANL. I worked with Mike Dvorak and the LCRC team 
(http://www.lcrc.anl.gov <http://www.lcrc.anl.gov/>) in MCS. We 
developed a comprehensive set of parallel programming tutorials. We 
expanded upon a tutorial from Indiana University written to solve the 
diffusion equation in parallel (tutorial available online: 
http://www.iu.edu/~rac/hpc/mpi_tutorial/9-0.html 
<http://www.iu.edu/%7Erac/hpc/mpi_tutorial/9-0.html>). Among other 
things, Four I/O implementations were developed in C: ASCII, binary, 
netCDF and PnetCDF. I ran some benchmarking to compare the four 
implementations. I have attached a portion of my SULI (Science 
Undergraduate Laboratory Internship) research paper as a PDF document.

A few quick notes:

<>-PnetCDF performed much better than the other I/O implementations we 
developed. Our results show that PnetCDF executed up to 13 times faster 
than serial NetCDF.
-Given a smaller data set, performance degraded significantly as the 
number of processors increased. This can be seen by the longer execution 
time on a 1500x1500 grid given more than 8 processors. As we increased 
the grid size, the performance hit occurred using larger numbers of 
processors. For instance, using PnetCDF on a 6000x6000 grid the 
32-processor job completed in less time than the 64-processor job. <>-If 
the graphs are too hard to read, I can print one graph per page in 
landscape. Due to time constraints, we used open office.org's chart 
wizard to display our results. Let me know if you want this.
-We were going to implement a parallel HDF5 I/O implementation but ran 
out of time.

Anyway, just thought I'd pass the information along. Feel free to 
contact me (pschmitt at students.depaul.edu 
<mailto:pschmitt at students.depaul.edu>) if you have any questions or if 
you would like a copy of my SULI paper. Many thanks to Rob Latham for 
answering some questions we had about PnetCDF!

-Peter Schmitt

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PnetCDF benchmarking results.pdf
Type: application/pdf
Size: 102755 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20040910/a062c366/attachment.pdf>