Performance study

[Jianwei Li]

Hi Christian,

There's one difference between current PnetCDF release and the one we 
used in our earlier develop stage. All the length arguments, including
attribute lengths and sizes/indices/strides/maps for each dimension, 
are now of type "MPI_Offset" instead of the original "int".
So for the flash_io_bench on the uchicago website, those arguments
need to be modified to MPI_Offset in order for the code to run
correctly (some type conversion may also need to be added in the code
to bridge the int's passed by FLASH and the MPI_Offset's required by
PnetCDF).

Please let us know if there're other problems.

Thanks!

Jianwei

On Mon, 2006-01-23 at 12:09 -0600, Christian Chilan wrote:
> Hi,
> 
> We are trying to reproduce the results of your comparison between HDF5 and 
> parallel netCDF on an IBM ps690 (AIX 5.3, GPFS). We downloaded and 
> installed parallel netCDF from http://www.mcs.anl.gov/parallel-netcdf/, and 
> flash benchmark from 
> http://flash.uchicago.edu/~jbgallag/io_bench/flash_io_bench.tar.gz.
> 
> After making some modifications to the makefile and to the flash source 
> code (such as changing netcdf.h for pnetcdf.h and nc_ prefixes for ncmpi_) 
> , we have the benchmark running. For now, we are executing flash using 1 
> through 4 processors.
> 
> We noted that there are some problems. For example, after dumping the 
> checkpoint files, we noticed that the file generated with HDF5 contains 
> non-zero values and its size (7MB through 36 MB approx.) depends on the 
> number of processors, but the file generated with parallel netCDF contains 
> mostly zero values and its very small size (1844 bytes) is the same for 
> different numbers of processors. In addition, the speedup resulting from 
> going from 1 to 2 processors is about 10 for both HDF5 and netCDF.
> 
> We'd appreciate that you let us know what tuning and modifications were 
> done in your comparison study.
> 
> Thanks,
> 
> Christian Chilan
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign