<div dir="ltr">Hi Phil,<div><br></div><div>Thanks for the suggestion. I solved this issue finally </div><div>by recompiling zlib.</div><div><br></div><div>Teng</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 21, 2017 at 1:05 PM, Phil Carns <span dir="ltr"><<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_-8621407278620756273moz-cite-prefix">It might be helpful to compare the
mpich configure/compile lines between your two configurations too,
in case there is some difference in mpich libraries that is
triggering an issue.<br>
<br>
thanks,<br>
-Phil<div><div class="h5"><br>
<br>
On 07/20/2017 05:35 PM, Shane Snyder wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<br>
<div class="m_-8621407278620756273moz-cite-prefix">On 07/20/2017 04:19 PM, Shane Snyder
wrote:<br>
</div>
<blockquote type="cite"> Hi
Teng,<br>
<br>
First of all, I actually just got darshan 3.1.4 to properly
generate a log when using static linking. I'm not sure what was
leading to the problem I had earlier, but after reconfiguring
and rebuilding the library, it works for me now. I also
confirmed the nightly tests were working correctly -- the log
files were going to another spot that I wasn't expecting, so
missed them the first time around. You may want to try something
similar -- just rebuilding everything from scratch and trying
again. I've also run into this problem in the past whenever I
was accidentally pointing to an mpich build that defaults to
using shared libraries rather than the static ones, so that
could be another thing to check. <br>
<br>
As for the error you are getting with Darshan 3.1.3, I've
actually never seen someone hit that before. That error would be
due to the libz compression functions (deflate) returning some
sort of error. Not exactly sure what could cause that, but is
the application opening a really large amount of files or
something like that? I suppose it's possible the compression
routines could be running out of memory if there is too much log
data they are trying to compress, but have never seen that
happen before. If you aren't already, it might be worth trying a
test program that does very basic I/O just to confirm that we
can get Darshan working. One that we frequently use is
mpi-io-test -- there is a version of this code in the darshan
source @ darshan-test/regression/test-<wbr>cases/src. You could try
just building that and running it (it doesn't take any command
line parameters) to see if that works?<br>
</blockquote>
<br>
Oh, actually I just noticed in your emails that you were using
mpi-io-test? That is pretty bizarre...<br>
<br>
What system are you running this on? A cluster? Your own personal
machine? If possible, you might try running somewhere else so we
can confirm the issue is some problem with the environment on the
system you have been using... I can't think of why the deflate
routines would fail when running something simple like
mpi-io-test. But if needed, we could probably get you a patch or
something that tries to give us more details on the libz error you
are seeing. <br>
<br>
--Shane<br>
<br>
<blockquote type="cite"> <br>
Thanks,<br>
--Shane<br>
<br>
<div class="m_-8621407278620756273moz-cite-prefix">On 07/20/2017 03:52 PM, Teng Wang
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Shane,
<div><br>
</div>
<div>Thanks for your help on this. I tried v3.1.3 and
compiled successfully. </div>
<div>But when I ran the application, there was no *.log
generated under the </div>
<div>log directory, and I also got a warning from the
application:</div>
<div>
<p style="margin:0px;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(175,173,36)"> </span><span style="font-variant-ligatures:no-common-ligatures">darshan
library warning: unable to compress job data</span></p>
<p style="margin:0px;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><br>
</p>
</div>
<div>May I know how to fix this?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Teng</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Jul 20, 2017 at 11:51 AM,
Shane Snyder <span dir="ltr"><<a href="mailto:ssnyder@mcs.anl.gov" target="_blank">ssnyder@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> Hi Teng,<br>
<br>
Thanks for reporting this. This is a really strange
issue, but I can actually reproduce it, too. I'm going
to keep digging and see if I can find the problem.
Will keep you updated.<br>
<br>
The strange thing is that I was able to use version
3.1.4 successfully right before making the release
available (i.e., the log @
darshan-test/example-output/mp<wbr>i-io-test-x86_64-3.1.4.darshan
was generated using version 3.1.4 with static
linking). Also strange is that our nightly testing is
not catching this issue...<br>
<br>
FYI, version 3.1.3 and earlier work fine for me if you
really need a quick resolution -- version 3.1.4 just
fixed a couple of non-critical bugs in some darshan
log parsing utilities, so there shouldn't be much
functionally different between the versions.<br>
<br>
--Shane
<div>
<div class="m_-8621407278620756273h5"><br>
<br>
<div class="m_-8621407278620756273m_6259078016213707887moz-cite-prefix">On
07/20/2017 12:43 PM, Teng Wang wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_-8621407278620756273h5">
<div dir="ltr">
<p>Hi,</p>
<p><br>
</p>
<p>I'm having trouble when trying to
statically link </p>
<p>Darshan with mpich3.3 applications
following the instructions </p>
<p>here. Could you give me any suggestion on
how to fix it? </p>
<p><br>
</p>
<p><span style="text-decoration:underline"><a href="http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html" target="_blank">http://www.mcs.anl.gov/researc<wbr>h/projects/darshan/docs/<wbr>darshan-runtime.html</a></span></p>
<p><br>
</p>
<p>Here is my steps:</p>
<p>1. Compile and install darshan 3.1.4</p>
<p>tar -xvzf darshan-3.1.4.tar.gz</p>
<p>cd darshan-3.1.4/darshan-runtime</p>
<p>./configure --with-mem-align=8
--with-log-path=<log directory>
--prefix=<install directory>
--with-jobid-env=PBS_JOBID --disable-cuserid
CC=mpicc</p>
<p>make</p>
<p>make install</p>
<p><br>
</p>
<p>2. Create log directory, which successfully
create the directory </p>
<p><span style="text-decoration:underline"><a href="http://darshan-mk-log-dirs.pl/" target="_blank">darshan-mk-log-dirs.pl</a></span></p>
<p><br>
</p>
<p>3. Generate wrapper</p>
<p><span style="text-decoration:underline;color:rgb(228,175,10)"><a href="http://darshan-gen-cc.pl/" target="_blank">darshan-gen-cc.pl</a></span>
`which mpicc` --output mpicc.darshan</p>
<p><br>
</p>
<p>4. Compile application (mpi-io-test.c)
using mpicc.darshan</p>
<p>mpicc.darshan mpi-io-test.c -o mpi-io-test</p>
<p><br>
</p>
<p>After step 4, the following issue happened:</p>
<p>initthread.c:(.text+0xc59d): undefined
reference to `__wrap_fprintf'</p>
<p>initthread.c:(.text+0xc5bd): undefined
reference to `__wrap_fputs'</p>
<p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-finalize<wbr>.o):
In function `MPIR_Add_finalize':</p>
<p>finalize.c:(.text+0x226e): undefined
reference to `__wrap_fputs'</p>
<p>finalize.c:(.text+0x227a): undefined
reference to `__wrap_fflush'</p>
<p>finalize.c:(.text+0x2286): undefined
reference to `__wrap_fflush'</p>
<p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-util.o):
In function `MPIDI_OFI_control_handler':</p>
<p>util.c:(.text+0x17bc4): undefined reference
to `__wrap_fprintf'</p>
<p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-ch4_glob<wbr>als.o):
In function `MPID_Abort':</p>
<p>ch4_globals.c:(.text+0x38b): undefined
reference to `__wrap_fputs'</p>
<p>ch4_globals.c:(.text+0x397): undefined
reference to `__wrap_fflush'</p>
<p>ch4_globals.c:(.text+0x3a3): undefined
reference to `__wrap_fflush'</p>
<p>/tmp/ccyntLoA.o: In function `main':</p>
<p>mpi-io-test.c:(.text+0x5ae): undefined
reference to `__wrap_printf'</p>
<p>mpi-io-test.c:(.text+0x819): undefined
reference to `__wrap_printf'</p>
<p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-contexti<wbr>d.o):
In function `MPIR_Free_contextid':</p>
<p>contextid.c:(.text+0x1c3f): undefined
reference to `__wrap_fputs'</p>
<p>contextid.c:(.text+0x1c4b): undefined
reference to `__wrap_fflush'</p>
<p>contextid.c:(.text+0x1c57): undefined
reference to `__wrap_fflush'</p>
<p><br>
</p>
<p><br>
</p>
<p>mpicc.darshan -show mpi-io-test.c -o
mpi-io-test gave:</p>
<p><br>
</p>
<p>gcc -L/usr/lib64/slurmpmi
-L/global/common/cori/software<wbr>/libfabric/1.4.1/gnu/lib
mpi-io-test.c -o mpi-io-test
-L/global/homes/t/user/softwar<wbr>e_install/darshan/lib
-ldarshan -lz -Wl,@/global/homes/t/user/soft<wbr>ware_install/darshan/share/ld-<wbr>opts/darshan-base-ld-opts
-I/global/homes/t/user/softwar<wbr>e_install/mpich3.3a-static/<wbr>include
-L/global/homes/t/user/softwar<wbr>e_install/mpich3.3a-static/lib
-lmpi -lpmi -lpmi -lpthread -lfabric -lrt
-lpmi -L/global/homes/t/user/softwar<wbr>e_install/darshan/lib
-Wl,--start-group -ldarshan -ldarshan-stubs
-Wl,--end-group -lz -lrt -lpthread</p>
<p><br>
</p>
<p>The strange thing is when I do </p>
<p>nm libdarshan.a, it did show the functions
(e.g. __wrap_printf) were linked to </p>
<p>libdarshan.a. </p>
<p><br>
</p>
<p><b>0000000000002bf0 T __wrap__IO_getc</b></p>
<p><b>0000000000002d80 T __wrap__IO_putc</b></p>
<p><b>00000000000030b0 T
__wrap___isoc99_fscanf</b></p>
<p><b>0000000000001990 T __wrap_fclose</b></p>
<p><b>00000000000011d0 T __wrap_fdopen</b></p>
<p><b>0000000000001810 T __wrap_fflush</b></p>
<p><b>0000000000002a60 T __wrap_fgetc</b></p>
<p><b>0000000000003690 T __wrap_fgets</b></p>
<p><b>0000000000000db0 T __wrap_fopen</b></p>
<p><b>0000000000000fc0 T __wrap_fopen64</b></p>
<p><b>00000000000026b0 T __wrap_fprintf</b></p>
<p><b>0000000000001c80 T __wrap_fputc</b></p>
<p><b>0000000000001fa0 T __wrap_fputs</b></p>
<p><b>00000000000028d0 T __wrap_fread</b></p>
<p><b>00000000000013d0 T __wrap_freopen</b></p>
<p><b>00000000000015f0 T __wrap_freopen64</b></p>
<p><b>00000000000032d0 T __wrap_fscanf</b></p>
<p><b>0000000000003920 T __wrap_fseek</b></p>
<p><b>0000000000003a30 T __wrap_fseeko</b></p>
<p><b>0000000000003b40 T __wrap_fseeko64</b></p>
<p><b>0000000000003c50 T __wrap_fsetpos</b></p>
<p><br>
</p>
<p>Thanks,</p>
<p>Teng</p>
</div>
<br>
<fieldset class="m_-8621407278620756273m_6259078016213707887mimeAttachmentHeader"></fieldset>
<br>
</div>
</div>
<pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273m_6259078016213707887moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.go<wbr>v</a>
<a class="m_-8621407278620756273m_6259078016213707887moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank">https://lists.mcs.anl.gov/mail<wbr>man/listinfo/darshan-users</a>
</pre>
</blockquote>
<br>
</div>
<br>
______________________________<wbr>_________________<br>
Darshan-users mailing list<br>
<a href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.go<wbr>v</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mail<wbr>man/listinfo/darshan-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<fieldset class="m_-8621407278620756273mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.<wbr>gov</a>
<a class="m_-8621407278620756273moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="m_-8621407278620756273mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank">Darshan-users@lists.mcs.anl.<wbr>gov</a>
<a class="m_-8621407278620756273moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a>
</pre>
</blockquote>
<p><br>
</p>
</div></div></div>
<br>______________________________<wbr>_________________<br>
Darshan-users mailing list<br>
<a href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.<wbr>gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a><br>
<br></blockquote></div><br></div>