parallel netCDF question

[Robert Latham]

On Wed, Dec 21, 2005 at 02:52:09PM -0800, Daniel S. Katz wrote:
> Assuming that this will not always work, is parallel netCDF the next 
> choice, even when the file system is question is not a parallel file system?

Hi Dan

It can be hard to get correct MPI-IO behavior on top of NFS and you'll
probably need your admins to make some changes to the NFS settings
(the changes are MPI-IO implementation dependent.  the ROMIO users
guide has recommended settings for ROMIO-based mpi-io
implementations). 

If that's the only file system you have access to, then
parallel-netcdf might still be a good solution.  

The other option is to forward data to a master node and have that guy
do all the I/O.  At least with parallel-netcdf you have less overall
data transfer (no copy to the master) and less memory pressure on the
master node (who no longer has to stage in memory the data set from
all other processors before writing out).  

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B