<div dir="ltr">Hi Phil and others,<div><br></div><div>in the end the problem was somewhere else and much more prosaic :/. The path where darshan was installed by default (lib folder specifically) was missing from the <span style="font-family:Calibri;font-size:11pt">LD_LIBRARY_PATH environment variable and therefore the library could not be found at runtime.</span></div><div><span style="font-family:Calibri;font-size:11pt"><br></span></div><div><span style="font-family:Calibri;font-size:11pt">BR,</span></div><div><span style="font-family:Calibri;font-size:11pt"><br></span></div><div><span style="font-family:Calibri;font-size:11pt">Jiri</span></div><div><span style="font-family:Calibri;font-size:11pt"></span><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">pá 6. 5. 2022 v 15:56 odesílatel Jiří Nádvorník <<a href="mailto:nadvornik.ji@gmail.com">nadvornik.ji@gmail.com</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Aha,<div><br></div><div>maybe I formulated it wrongly. The problem is that the version that is installed by make install is taken from the .lib folder and not from the darshan-util one. So when trying to run for example darshan-parser, it complains about ".libs/libdarshan-util.so.0" missing.</div><div><br></div><div>Cheers,</div><div><br></div><div>Jiri</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">st 4. 5. 2022 v 22:41 odesílatel Phil Carns <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Jiri,</p>
<p>For #2, are you talking about .libs/darshan-parser in the build
path? If so, I wouldn't expect to be able to run that directly
(in general, not particular to Darshan's build system).</p>
<p>Autotools and libtool create executables in .libs subdirs as part
of the build process, but those are intermediate executables that
don't have final library paths set. The copy installed in
<prefix>/bin should be fine (and should use corresponding
installed libraries), and you should also be able to run
darshan-parser one level up in the build tree (that's actually a
shell script wrapper created by libtool that will run the
.libs/darshan-parser with library paths set to the build tree).</p>
<p>Sometimes .libs/ executables might work, but it's a little dicey
what libraries they will pick up so it's usually not a good idea.<br>
</p>
<p>thanks,</p>
<p>-Phil<br>
</p>
<div>On 4/27/22 1:34 PM, Jiří Nádvorník
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>to reproduce the installation issue:</div>
<div>mkdir darshan_root</div>
<div>cd darshan_root</div>
<div>git clone <a href="https://github.com/darshan-hpc/darshan.git" target="_blank">https://github.com/darshan-hpc/darshan.git</a>
.<br>
<div><br>
</div>
<div>Then cd darshan-utils/ and then run:</div>
<div>autoconf</div>
<div>configure</div>
<div>make install</div>
<div><br>
</div>
<div>Then:</div>
<div>
<ol>
<li>If running darshan-parser within the same folder it
runs fine.</li>
<li>If running .lib/darshan-parser (which is installed by
make install) it crashes with the library not available,
see the previous email.</li>
</ol>
</div>
<div>Cheers,</div>
<div><br>
Jiri</div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">st 27. 4. 2022 v 18:23
odesílatel Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>>
napsal:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Great,
I'm glad that you were able to get the instrumentation
mostly working!</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">I
think it's sensible to ignore Python source/compiled code
for most cases -- I doubt there's any insight to gain and
you'll just end up trying to filter them out when
analyzing logs anyways.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">I'm
not sure what's going on with the installation issues you
mention. If you think something might be wrong with
Darshan's build, then would you mind sharing how you ran
configure, etc.? I could see if I'm able to reproduce
anything.
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">If
you wouldn't mind starting a new thread related to the
HDF5 issue, I think that would be helpful -- it might help
if other users ever want to search the list archive for
h5py/HDF5 related issues if you include those in the
title.<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">--Shane<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-9014243290627863420gmail-m_6519552878307446660gmail-m_-2273691217974330162divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri,
sans-serif" color="#000000"><b>From:</b> Jiří Nádvorník
<<a href="mailto:nadvornik.ji@gmail.com" target="_blank">nadvornik.ji@gmail.com</a>><br>
<b>Sent:</b> Wednesday, April 27, 2022 11:06 AM<br>
<b>To:</b> Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>><br>
<b>Cc:</b> <a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>
<<a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>><br>
<b>Subject:</b> Re: [Darshan-users] Module contains
incomplete data</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Hi,</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">yes, that NAMEMEM got it done. I also
excluded .py and .pyc files - those reads are only
loading of them, right? No data access itself (and no,
I'm not reading and manually interpreting my own
python files :), so I'm not interested into that ).
Actually, I'm reading thousands of small files which
I'm ingesting into HDF5 and I'm interested into how
many reads, etc. are happening. </div>
<div dir="ltr"><br>
</div>
<div dir="ltr">I'm trying to make some sense of what I
see but for now I'm just going to say it's very
valuable data for me. Pity I can't get the Hdf5
module, if it would give me more granularity it would
be very helpful.
<div><br>
</div>
<div>Regarding the darshan-utils you were right, I
didn't reinstall them. I actually ran into an
install problem - for some reason, the git
installation takes the .lib/darshan-parser when
installing it to /usr/local/... and that one throws:</div>
<div>darshan-parser: error while loading shared
libraries: libdarshan-util.so.0: cannot open shared
object file: No such file or directory</div>
<div><br>
</div>
<div>But if I run the darshan-parser within
darshan_root_folder/darshan-util/ then the error is
gone and --show-incomplete |grep incomplete prints
nothing.</div>
<div><br>
</div>
<div>Could we now focus on the HDF5 issue or should I
create a new thread for clarity?</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Jiri</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
<br>
<div>
<div dir="ltr">st 27. 4. 2022 v 17:21 odesílatel
Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>>
napsal:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Thanks
for working through the build issues and giving
this a shot.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">A
couple of things stand out to me (ignoring your
HDF5 issue for now):</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<ul>
<li>It looks like at least the MPI-IO module
is no longer reporting partial data? Small
progress...</li>
<li>There is a new warning about there being
no log utility handlers for a "null" module.
Are you perhaps parsing a log generated by
your prior Darshan install? Maybe you have
not completely re-installed a new
darshan-util? We should figure out what's
going on there, too, to be safe.<br>
</li>
</ul>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">I'd
also suggest two things for your config file:</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<ul>
<li>Dial back MODMEM and MAX_RECORDS values.
Your MODMEM value is asking Darshan to
allocate a GiB of memory (it is expressed in
MiB units and you set to 1024), which
Darshan will happily try to do, I'm not sure
it's a good idea though. I'd probably start
with a MODMEM value of 8 and MAX_RECORDS of
2000, and just double those again if needed
-- anything beyond that would be surprising
unless you know your workload is really
opening hundreds of thousands of files. You
might also have a look at the files Darshan
is currently instrumenting and see if you
really want it to -- I've noticed when
instrumenting Python frameworks that you can
get tons of records for things like shared
libraries, source files, etc. that can just
be ignored using NAME_EXCLUDE mechanisms.
<br>
</li>
<li>Add "NAMEMEM 2" to your config file to
force Darshan to allocate more memory (2
MiB) for storing the filenames associated
with each record. This might actually be the
main reason your log is reporting partial
data rather than actually running out of
module data, which is another reason not to
get too aggressive with the
MODMEM/MAX_RECORDS parameters. I should have
mentioned this setting originally as there
have been other users who have reported
exceeding it recently.<br>
</li>
</ul>
<div>Hopefully that gets you further along and
we can move onto the HDF5 issue you mention.<br>
</div>
<div><br>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Thanks,<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">--Shane<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-9014243290627863420gmail-m_6519552878307446660gmail-m_-2273691217974330162x_gmail-m_6650929377004058598divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
Jiří Nádvorník <<a href="mailto:nadvornik.ji@gmail.com" target="_blank">nadvornik.ji@gmail.com</a>><br>
<b>Sent:</b> Wednesday, April 27, 2022 6:37 AM<br>
<b>To:</b> Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>><br>
<b>Cc:</b> <a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>
<<a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>><br>
<b>Subject:</b> Re: [Darshan-users] Module
contains incomplete data</font>
<div> </div>
</div>
<div>
<div dir="ltr">Aha! I just realized there is an
obvious "prepare.sh" script that I didn't run,
I found out by trial and error though, could
be more documented :).
<div><br>
</div>
<div>Now I'm further.. With a config file:</div>
<div>MAX_RECORDS 102400
POSIX,MPI-IO,STDIO<br>
</div>
<div>MODMEM 1024<br>
</div>
<div>APP_EXCLUDE git,ls<br>
</div>
<div><br>
</div>
<div>I'm getting for:</div>
<div>darshan-parser --show-incomplete
caucau_python_id127447-127447_4-27-48556-1842455298968263838_1.darshan
|grep incomplete<br>
</div>
<div><br>
</div>
<div>output:</div>
<div># *WARNING*: The POSIX module contains
incomplete data!<br>
# *WARNING*: The STDIO module contains
incomplete data!<br>
Warning: no log utility handlers defined for
module (null), SKIPPING.<br>
</div>
<div><br>
</div>
<div>I don't think I have more than 100000
files to be touched by my poor tiny python
script, right?</div>
<div><br>
</div>
<div>By the way I've encountered another
problem, not sure whether to put it to
another thread. If I compile with HDF5 (the
results above are without it):</div>
<div>./configure
--with-log-path=/gpfs/raid/darshan-logs
--with-jobid-env=PBS_JOBID CC=mpicc
--enable-hdf5-mod
--with-hdf5=/gpfs/raid/SDSSCube/ext_lib//hdf5-1.12.0/hdf5/<br>
</div>
<div><br>
</div>
<div>It messes up my runtime and causes python
to crash:</div>
<div>mpirun -x
DARSHAN_CONFIG_PATH=/gpfs/raid/SDSSCube/darshan.conf
-x
LD_PRELOAD=/gpfs/raid/shared_libs/darshan/darshan-runtime/lib/.libs/libdarshan.so:/gpfs/raid/SDSSCube/ext_lib/hdf5-1.12.0/hdf5/lib/libhdf5.so
-np 65 --hostfile hosts --map-by node
/gpfs/raid/SDSSCube/venv_par/bin/python
hisscube.py --truncate ../sdss_data/
results/SDSS_cube_c_par.h5<br>
</div>
<div><br>
</div>
<div>Resulting in:</div>
<div><a>INFO:rank</a>[0]:Rank 0 pid: 137058<br>
Darshan HDF5 module error: runtime library
version (1.12) incompatible with Darshan
module (1.10-).<br>
Traceback (most recent call last):<br>
File "hisscube.py", line 74, in
<module><br>
writer.ingest(fits_image_path,
fits_spectra_path,
truncate_file=args.truncate)<br>
File
"/gpfs/raid/SDSSCube/hisscube/ParallelWriterMWMR.py",
line 45, in ingest<br>
self.process_metadata(image_path,
image_pattern, spectra_path,
spectra_pattern, truncate_file)<br>
File
"/gpfs/raid/SDSSCube/hisscube/CWriter.py",
line 150, in process_metadata<br>
h5_file =
self.open_h5_file_serial(truncate_file)<br>
File
"/gpfs/raid/SDSSCube/hisscube/CWriter.py",
line 170, in open_h5_file_serial<br>
return h5py.File(self.h5_path, 'w',
fs_strategy="page", fs_page_size=4096,
libver="latest")<br>
File
"/gpfs/raid/SDSSCube/venv_par/lib/python3.8/site-packages/h5py-3.6.0-py3.8-linux-x86_64.egg/h5py/_hl/files.py",
line 533, in __init__<br>
fid = make_fid(name, mode,
userblock_size, fapl, fcpl, swmr=swmr)<br>
File
"/gpfs/raid/SDSSCube/venv_par/lib/python3.8/site-packages/h5py-3.6.0-py3.8-linux-x86_64.egg/h5py/_hl/files.py",
line 232, in make_fid<br>
fid = h5f.create(name, h5f.ACC_TRUNC,
fapl=fapl, fcpl=fcpl)<br>
File "h5py/_objects.pyx", line 54, in
h5py._objects.with_phil.wrapper<br>
File "h5py/_objects.pyx", line 55, in
h5py._objects.with_phil.wrapper<br>
File "h5py/h5f.pyx", line 126, in
h5py.h5f.create<br>
File "h5py/defs.pyx", line 693, in
h5py.defs.H5Fcreate<br>
RuntimeError: Unspecified error in H5Fcreate
(return value <0)<br>
</div>
<div><br>
</div>
<div>You are saying that darshan should be
compatible with HDF5 > 1.8, which 1.12
should be, right?</div>
<div><br>
</div>
<div>Thanks for help!</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Jiri</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div>
<div dir="ltr">st 27. 4. 2022 v 8:43
odesílatel Jiří Nádvorník <<a href="mailto:nadvornik.ji@gmail.com" target="_blank">nadvornik.ji@gmail.com</a>>
napsal:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi,
<div><br>
</div>
<div>I think I will chew through the
documentation just fine but two things
are not clear:</div>
<div>
<ol>
<li>Does the darshan library provide
its own config file that I need to
change or do I need to always create
my own?</li>
<li>How can I build the git version? I
didn't find any instructions and the
usual autoconf just throws:</li>
<ol>
<li>root@kub-b1:/gpfs/raid/shared_libs/darshan/darshan-runtime#
autoconf<br>
<a href="http://configure.ac:19" target="_blank">configure.ac:19</a>:
error: possibly undefined macro:
AC_CONFIG_MACRO_DIRS<br>
If this token and others are
legitimate, please use
m4_pattern_allow.<br>
See the Autoconf
documentation.<br>
<a href="http://configure.ac:21" target="_blank">configure.ac:21</a>:
error: possibly undefined macro:
AM_INIT_AUTOMAKE<br>
<a href="http://configure.ac:22" target="_blank">configure.ac:22</a>:
error: possibly undefined macro:
AM_SILENT_RULES<br>
<a href="http://configure.ac:23" target="_blank">configure.ac:23</a>:
error: possibly undefined macro:
AM_MAINTAINER_MODE<br>
<a href="http://configure.ac:713" target="_blank">configure.ac:713</a>:
error: possibly undefined macro:
AM_CONDITIONAL<br>
root@kub-b1:/gpfs/raid/shared_libs/darshan/darshan-runtime# ./configure<br>
configure: error: cannot find
install-sh, install.sh, or shtool
in ../maint/scripts
"."/../maint/scripts<br>
</li>
</ol>
</ol>
<div>Thanks for help.</div>
</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Jiri</div>
</div>
<br>
<div>
<div dir="ltr">út 26. 4. 2022 v 17:43
odesílatel Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>>
napsal:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Hi
Jiri,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">For
some background, Darshan enforces
some internal memory limits to avoid
ballooning memory usage at runtime.
Specifically, all of our
instrumentation modules should
pre-allocate file records for up to
1,024 files opened by the app -- if
your app opens more than 1,024 files
per-process, Darshan stops
instrumenting and issues those
warning messages when parsing the
log file.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">We
have users hit this issue pretty
frequently now, and we actually just
wrapped up development of some new
mechanisms to help out with this.
They were just merged into our main
branch, and we will be formally
releasing a pre-release version of
this code in the next week or so.
For the time being, you should be
able to use the 'main' branch of our
repo (<a href="https://github.com/darshan-hpc/darshan" id="gmail-m_-9014243290627863420gmail-m_6519552878307446660gmail-m_-2273691217974330162x_gmail-m_6650929377004058598x_gmail-m_8980332545816639580gmail-m_8205458234077072596LPNoLPOWALinkPreview" target="_blank">https://github.com/darshan-hpc/darshan</a>)
to leverage this new functionality.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">There
are 2 new mechanisms that can help
out, both of which require you to
provide a configuration file to
Darshan at runtime:</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<ul style="list-style-position:inside">
<li>MAX_RECORDS setting can be
used to bump up the number of
pre-allocated records for
different modules. In your case,
you might try to bump up the
default number of records for
the POSIX, MPI-IO, and STDIO
modules by setting something
like this in your config file
(this would allow you to
instrument up to 4000 files
per-process for each of these
modules):</li>
<ul>
<li>MAX_RECORDS 4000
POSIX,MPI-IO,STDIO</li>
</ul>
<li>An alternative (or
complementary) approach to
bumping up the record limit is
to limit instrumentation to
particular files. You can use
the NAME_EXCLUDE setting to
avoid instrumenting specific
directory paths, file
extensions, etc by specifying
regular expressions. E.g, the
following settings would avoid
instrumenting files with .so
prefixes or files located in a
directory we don't care about
for all modules (* denotes all
modules):</li>
<ul>
<li>NAME_EXCLUDE .so$ *</li>
<li>NAME_EXCLUDE
^/path/to/avoid *</li>
</ul>
</ul>
<div>I'm attaching the updated
runtime documentation for Darshan
for your reference. Section 8
provides a ton of details on how
to provide a config file to
Darshan that should help clear up
any missing gaps in my description
above.</div>
<div><br>
</div>
<div>Please let us know if you have
any further questions or issues,
though!</div>
<div><br>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">Thanks,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">--Shane<br>
</div>
<div id="gmail-m_-9014243290627863420gmail-m_6519552878307446660gmail-m_-2273691217974330162x_gmail-m_6650929377004058598x_gmail-m_8980332545816639580gmail-m_8205458234077072596appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-9014243290627863420gmail-m_6519552878307446660gmail-m_-2273691217974330162x_gmail-m_6650929377004058598x_gmail-m_8980332545816639580gmail-m_8205458234077072596divRplyFwdMsg" dir="ltr">
<font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
Darshan-users <<a href="mailto:darshan-users-bounces@lists.mcs.anl.gov" target="_blank">darshan-users-bounces@lists.mcs.anl.gov</a>>
on behalf of Jiří Nádvorník <<a href="mailto:nadvornik.ji@gmail.com" target="_blank">nadvornik.ji@gmail.com</a>><br>
<b>Sent:</b> Sunday, April 24,
2022 3:00 PM<br>
<b>To:</b> <a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>
<<a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>><br>
<b>Subject:</b> [Darshan-users]
Module contains incomplete data</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi All,
<div><br>
</div>
<div>I just tried out Darshan and
the potential output seems
perfect for my HDF5 MPI
application! Although I'm not
able to get there :(.</div>
<div><br>
</div>
<div>I have a log that has a big
stamp "This darshan log contains
incomplete data".</div>
<div><br>
</div>
<div>When I run:</div>
<div>darshan-parser
--show-incomplete mylog.darshan
|grep incomplete<br>
</div>
<div>Output is:</div>
<div># *WARNING*: The POSIX module
contains incomplete data!<br>
# *WARNING*: The MPI-IO module
contains incomplete data!<br>
# *WARNING*: The STDIO module
contains incomplete data!<br>
</div>
<div><br>
</div>
<div>Would you be able to point me
to some setting that would
improve the measurements? Can I
actually rely on the profiling
results if it says the data is
incomplete in some of the
categories?</div>
<div><br>
</div>
<div>Thank you very much for your
help!</div>
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Jiri</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Darshan-users mailing list
<a href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.gov</a>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
Darshan-users mailing list<br>
<a href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
</blockquote></div>
</blockquote></div>