Hints on improving performance with WRF and Pnetcdf

[Craig Tierney]

On 9/8/10 8:01 AM, Rob Latham wrote:
> On Mon, Sep 06, 2010 at 08:47:15AM -0500, Gerry Creager wrote:
>> Wei-keng Liao wrote:
>>> I have built a ROMIO library with the recent optimizations for Lustre
>>> in my home directory and you are welcomed to give it a try. Below is
>>> the usage example of the library:
>>> %  mpif90 myprogram.o -L/share/home/00531/tg457823/ROMIO/lib -lmpio
>>>
>>> Please note that this library is built using mvapich2 on Ranger. Run the
>>> command below before compile/link your programs.
>>> %  module load mvapich2
>>>
>
>> Thanks. Useful information. I'll look at your ROMIO library later
>> today (about to go into a meeting for the rest of the morning).
>> Last time I set stripe-count to homething above 16, rsl files were
>> also "taking advantage" of that and shut down the LFS. Have you seen
>> this or do you address this in ROMIO?
>
> Hi Gerry, Craig:
>
> Have either of you tried Wei-keng's library?  We went on a bit of a
> tangent about MPI-IO tests, but that was just to demonstrate that his
> library does work, does provide performance benefits, and does not
> flatten the lustre servers :>
>
> We can talk about hints for improving performance, too, but Lustre
> needs special attention with respect to collective I/O, attention
> which Wei-keng's library (and MPICH2-1.3.1b1 and newer) do pay to
> Lustre.
>
> Do mind Wei-keng's suggestions to set the striping configuration on
> the output directory.  If Lustre really does fall over for you
> whenever you stripe across 16 or more servers, then your options for
> parallel I/O are going to be fairly limited.
>
> In addition to runs with Wei-keng's library (probably the easiest way
> to get the optimizations), can you post the output of 'ncdump -h' of
> one of the slowly-produced output files?
>
> ==rob
>

Rob,

Can I build mvapich2, without ROMIO support, then build mpich2-1.3b1 
with ROMIO support, then just include the link commands to pull the 
ROMIO from mpich2 to get at the new stuff?  Does Wei-keng have any other
patches beyond what is in the mpich2-1.3b1?

Craig