performance issue

[Wei-Keng Liao]

From the file header dump you sent earlier, no PnetCDF hint is necessary.

What MPI-IO hints are you using?

One question. Did you delete the output file before each run?

Wei-keng

On Aug 10, 2023, at 11:35 AM, Jim Edwards <jedwards at ucar.edu> wrote:

We are having a lot of problems with darshan on our system - I'm working with the darshan developers to resolve.

I am setting MPIIO hints - but I see that there are some hints specific to pnetcdf - do you have any recommendations?

On Thu, Aug 10, 2023 at 10:15 AM Wei-Keng Liao <wkliao at northwestern.edu<mailto:wkliao at northwestern.edu>> wrote:
Hi, Jim

FYI. Darshan now can capture the I/O activities of PnetCDF,
in addition to the already MPI-IO and POSIX-IO.

Wei-keng

On Aug 9, 2023, at 6:22 PM, Wei-Keng Liao <wkliao at northwestern.edu<mailto:wkliao at northwestern.edu>> wrote:

In that case, I have the E3SM-IO benchmark that has a fairly complicate I/O
partitioning pattern. It used the decomposition maps generated from PIO.
https://github.com/Parallel-NetCDF/E3SM-IO<https://urldefense.com/v3/__https://github.com/Parallel-NetCDF/E3SM-IO__;!!Dq0X2DkFhyF93HkjWTBQKhk!UJAnknjDPQWY43szbAutlokjDrSHyBOcTlhEOChqYtThxnr812hxwvb8aSmHcEPiNOxNdOUHZi9Z2fyXbg2j6JH1HTo$>

Wei-keng

On Aug 9, 2023, at 6:17 PM, Jim Edwards <jedwards at ucar.edu<mailto:jedwards at ucar.edu>> wrote:

I think that your example case is too simple - it's doing a simple block decomposition.
In order to get the performance difference I am observing I need to do a more complicated
mapping.   I will work on a program that reproduces the problem without pio but it may take a
while.





--
Jim Edwards

CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20230810/4b4dcc3b/attachment-0001.html>