[mpich-discuss] MPI IO on lustre

[Rob Latham]

On Thu, Mar 04, 2010 at 08:44:49AM -0600, Rob Latham wrote:
> I fear the authors of the studies you have read have reached the exact
> wrong conclusion.  It is not MPI-IO that has the defect.  Rather,
> Lustre's design makes it difficult to achieve high performance
> parallel I/O.  Fortunately, the task is merely difficult, not
> impossible, and the MPI-IO community has stepped up to the challenge.

I should have mentioned that the algorithm to achieve good performance
on Lustre is sufficiently complex that it got published at SC2008:

    Wei keng Liao and Alok Choudhary. "Dynamically adapting file
    domain partitioning methods for collective I/O based on underlying
    parallel file system locking protocols". In Proceedings of
    International Conference for High Performance Computing,
    Networking, Storage and Analysis (SC08), Austin, Texas, November
    2008.

    http://portal.acm.org/citation.cfm?id=1413370.1413374

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA