[mpich-discuss] Reading a large file

[Tabrez Ali]

MPICH Developers/Users

What is the fastest way for 1000+ ranks/cores to read a single 1+ GB 
file with irregular data.

Basically I have an unstructured FE code where all ranks need to read 
the mesh data they own from a single input file (based on the mesh 
partitioning info).

Right now all ranks simultaneously open the file (using 'open'  in 
Fortran), read in the values they own and skip the rest. For a problem 
with 16 million nodes (~1.5GB total file size) on 1024 cores of a linux 
cluster (with Lustre) this takes upto 2 mins (I/O part) before all ranks 
have (owned) node/element info.

Would MPI I/O routines perform better in my situation specially for 
larger problems?

Thanks in advance. I am new to MPI I/O.

Tabrez