[mpich-discuss] Question on MPI-IO

[Rob Latham]

On Mon, May 28, 2012 at 10:57:38AM -0300, Luiz Carlos da Costa Junior wrote:
> Hello,
> 
> I have an implementation in which I have to distribute the same data across
> all processes. Because of concurrent access issues, a master process is
> responsible for reading the file and, after, distribute its content using
> MPI_BCAST.
> Now I have a new file in the same situation.
> 
> I have read a little about MPI-IO and I am considering to use it.
> Is it suitable to situations like this? Can someone please point me out
> what I should expect comparing my current implementation with MPI-IO?

You can certainly use MPI-IO to request that all processes read the
same data.  I caution you though that this request might not be
handled particularly efficiently -- it's actually quite difficult for
the MPI-IO layer to detect "everybody read the same region" workloads.

Your "read and broadcast" approach will work well in an I/O context,
and probably deliver better performance.

If you have situations where your data is decomposed across
processors, such as each processor reading a different set of initial
conditions, or each processor contributing data to an output file,
then consider MPI-IO.

Even better, maybe a library like HDF5 or Parallel-NetCDF will be
better suited to your needs.  

==rob


-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA