[mpich-discuss] MPI IO performance

[Rob Latham]

On Mon, Jul 13, 2009 at 04:56:08PM -0700, burlen wrote:
> My test array is 513Mb, I have tested on 2,4, and 8 nodes, with a single  
> process per node. File system is nfs. For my tests I am using  
> mpich2-1.0.8p1 and ch3:ssm.

When MPI-IO operates on an NFS file system, MPI-IO has to try very very
hard to make NFS behave correctly, and that can have an impact on
performance.  

NFS client side caching gives great performance for a single process,
but if multiple processes operate on a file, their client side caches
do not know about each other and you end up with a variant of "false
sharing". 

NFS should be the last choice for a parallel file system.  Instead,
consider PVFS, GPFS, Panasas, or Lustre.

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA