Hints on improving performance with WRF and Pnetcdf

[Rob Latham]

On Mon, Sep 06, 2010 at 08:47:15AM -0500, Gerry Creager wrote:
> Wei-keng Liao wrote:
> >I have built a ROMIO library with the recent optimizations for Lustre
> >in my home directory and you are welcomed to give it a try. Below is
> >the usage example of the library:
> >%  mpif90 myprogram.o -L/share/home/00531/tg457823/ROMIO/lib -lmpio
> >
> >Please note that this library is built using mvapich2 on Ranger. Run the
> >command below before compile/link your programs.
> >%  module load mvapich2
> >

> Thanks. Useful information. I'll look at your ROMIO library later
> today (about to go into a meeting for the rest of the morning).
> Last time I set stripe-count to homething above 16, rsl files were
> also "taking advantage" of that and shut down the LFS. Have you seen
> this or do you address this in ROMIO?

Hi Gerry, Craig:

Have either of you tried Wei-keng's library?  We went on a bit of a
tangent about MPI-IO tests, but that was just to demonstrate that his
library does work, does provide performance benefits, and does not
flatten the lustre servers :>

We can talk about hints for improving performance, too, but Lustre
needs special attention with respect to collective I/O, attention
which Wei-keng's library (and MPICH2-1.3.1b1 and newer) do pay to
Lustre. 

Do mind Wei-keng's suggestions to set the striping configuration on
the output directory.  If Lustre really does fall over for you
whenever you stripe across 16 or more servers, then your options for
parallel I/O are going to be fairly limited.  

In addition to runs with Wei-keng's library (probably the easiest way
to get the optimizations), can you post the output of 'ncdump -h' of
one of the slowly-produced output files?

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA