performance issue

[Wei-Keng Liao]

OK. I will test it myself on Perlmutter.
Do you have a small test program to reproduce or is it still pioperf?
If pioperf, are the build instructions on Perlmutter the same?

Please let me know how you run on Perlmutter, i.e. no. process, nodes,
Lustre striping, problem size, etc.

Does "1 16 64" in your results mean 16 I/O tasks and 64 variables,
and only 16 MPI processes out of total ? processes call PnetCDF APIs?

Wei-keng

On Aug 11, 2023, at 9:35 AM, Jim Edwards <jedwards at ucar.edu> wrote:

I tried on perlmutter and am seeing the same issue only maybe even worse:

RESULT: write    SUBSET         1        16        64     1261.0737058071       14.7176171500
RESULT: write    SUBSET         1        16        64       90.3736534450      205.3695882870


On Fri, Aug 11, 2023 at 8:17 AM Jim Edwards <jedwards at ucar.edu<mailto:jedwards at ucar.edu>> wrote:
Hi Wei-Keng,

I released that the numbers in this table are all showing the slow performing file and the fast file
(the one without the scalar variable) are not represented - I will rerun and present these numbers again.

Here are corrected numbers for a few cases:
GPFS (/glade/work on derecho):
RESULT: write    SUBSET         1        16        64     4570.2078677815        4.0610844270
RESULT: write    SUBSET         1        16        64     4470.3231494386        4.1518251320

Lustre, default PFL's:
RESULT: write    SUBSET         1        16        64     2808.6570137094        6.6081404420
RESULT: write    SUBSET         1        16        64     1025.1671656858       18.1043644600

LUSTRE, no PFL's and very wide stripe:
 RESULT: write    SUBSET         1        16        64     4687.6852437580        3.9593102000
 RESULT: write    SUBSET         1        16        64     3001.4741125579        6.1836282120

On Thu, Aug 10, 2023 at 11:34 AM Jim Edwards <jedwards at ucar.edu<mailto:jedwards at ucar.edu>> wrote:
the stripe settings
lfs setstripe -c 96 -S 128M

logs/c96_S128M/



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20230811/2e904f03/attachment.html>