pnetCDF performance issues

[Rob Latham]

On Sun, Mar 06, 2011 at 05:34:31PM -0700, Russ Rew wrote:
> Hi Bill,
> 
> > We've been running various benchmarks, including IOR (I know - there  
> > are lots of problems with this benchmark) and found that pnetCDF tends  
> > to perform poorly on several of the systems we're using.  We tracked  
> > it down to the header  on the netCDF file - if this header is written  
> > and isn't a good size (like 4MB), performance can become very poor.   
> > This may be an artifact of the MPI-IO implementation (which includes  
> > ones that use ROMIO), but it impacts uses of pnetCDF.  Have you seen  
> > this problem?  What is the best way to fix this (e.g., can/should  
> > pnetCDF expect unaligned writes to be poor in MPI-IO, and work to  
> > force aligned writes?).  Note that HDF-5 appears to work around this  
> > problem; in any event, in our tests, we didn't see a serious  
> > performance impact.
> 
> A relatively straightforward fix for new files would be to force the
> header to always be a good size, such as a multiple of 4MB.  When
> writing a new file, all other variable offsets are computed relative to
> where the first variable begins.

in parallel-netcdf land we don't have an equivalent of nc__create
(note the double underscore).  We have to align things through a
different mechanism:

http://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/VariableAlignment

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA