[MPICH] MPICH2 and PVFS2

[Peter Diamessis]

Hi folks,

I have a very general question. My parallel CFD solver has
been using MPI-2 non-contiguous parallel I/O which has turned
out to be extremely handy in terms of file management and running
on different number of processors of each time. I run my production
runs at the University of Southern California 1,000 node Linux cluster.

Until recently the parallel I/O performed beautifully. However, a few
weeks ago the cluster actually moved to a different location and the
storage device was also changed. To my understanding, they have now
shifted to PVFS2 from PVFS. I'm now experiencing this problem, where
the files containing 3-D flow field datasets, which
are dumped out at regular intervals during my CFD simulations,
are of different sizes, something they shouldn't be. Also the file size can
differ dramatically when running with a different number of processors. 
Normally, regardless of the time the files are dumped out and regardless of
the number of processors, the file sizes should be the same. This is how
things worked in the past and this is what I get when running on a workstation
with two procs.

Apparently, MPICH v1.2.7..15 is still being used at the cluster. Our system
manager has made an effort to build it to make ROMIO compatible with
PVFS2. Nonetheless, my problems persist. Would it simply be a better
idea that they shift the whole cluster to MPICH2, which I assume is more
naturally compatible with PVFS2 ?

Sorry if I sound too ignorant but my I/O troubles have driven me up the wall.

Many thanks,

Peter Diamessis