<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">Great news- I'm glad it's resolved!<br>
      <br>
      On 07/21/2017 04:19 PM, Teng Wang wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CACSGo+qq6WVYY+eSMC64=EMwQfpjtcKDFKTptLpfuQxruOdUxw@mail.gmail.com">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <div dir="ltr">Hi Phil,
        <div><br>
        </div>
        <div>Thanks for the suggestion. I solved this issue finally </div>
        <div>by recompiling zlib.</div>
        <div><br>
        </div>
        <div>Teng</div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Fri, Jul 21, 2017 at 1:05 PM, Phil
          Carns <span dir="ltr"><<a href="mailto:carns@mcs.anl.gov"
              target="_blank" moz-do-not-send="true">carns@mcs.anl.gov</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <div class="m_-8621407278620756273moz-cite-prefix">It
                might be helpful to compare the mpich configure/compile
                lines between your two configurations too, in case there
                is some difference in mpich libraries that is triggering
                an issue.<br>
                <br>
                thanks,<br>
                -Phil
                <div>
                  <div class="h5"><br>
                    <br>
                    On 07/20/2017 05:35 PM, Shane Snyder wrote:<br>
                  </div>
                </div>
              </div>
              <div>
                <div class="h5">
                  <blockquote type="cite"> <br>
                    <div class="m_-8621407278620756273moz-cite-prefix">On
                      07/20/2017 04:19 PM, Shane Snyder wrote:<br>
                    </div>
                    <blockquote type="cite"> Hi Teng,<br>
                      <br>
                      First of all, I actually just got darshan 3.1.4 to
                      properly generate a log when using static linking.
                      I'm not sure what was leading to the problem I had
                      earlier, but after reconfiguring and rebuilding
                      the library, it works for me now. I also confirmed
                      the nightly tests were working correctly -- the
                      log files were going to another spot that I wasn't
                      expecting, so missed them the first time around.
                      You may want to try something similar -- just
                      rebuilding everything from scratch and trying
                      again. I've also run into this problem in the past
                      whenever I was accidentally pointing to an mpich
                      build that defaults to using shared libraries
                      rather than the static ones, so that could be
                      another thing to check. <br>
                      <br>
                      As for the error you are getting with Darshan
                      3.1.3, I've actually never seen someone hit that
                      before. That error would be due to the libz
                      compression functions (deflate) returning some
                      sort of error. Not exactly sure what could cause
                      that, but is the application opening a really
                      large amount of files or something like that? I
                      suppose it's possible the compression routines
                      could be running out of memory if there is too
                      much log data they are trying to compress, but
                      have never seen that happen before. If you aren't
                      already, it might be worth trying a test program
                      that does very basic I/O just to confirm that we
                      can get Darshan working. One that we frequently
                      use is mpi-io-test -- there is a version of this
                      code in the darshan source @
                      darshan-test/regression/test-<wbr>cases/src. You
                      could try just building that and running it (it
                      doesn't take any command line parameters) to see
                      if that works?<br>
                    </blockquote>
                    <br>
                    Oh, actually I just noticed in your emails that you
                    were using mpi-io-test? That is pretty bizarre...<br>
                    <br>
                    What system are you running this on? A cluster? Your
                    own personal machine? If possible, you might try
                    running somewhere else so we can confirm the issue
                    is some problem with the environment on the system
                    you have been using... I can't think of why the
                    deflate routines would fail when running something
                    simple like mpi-io-test. But if needed, we could
                    probably get you a patch or something that tries to
                    give us more details on the libz error you are
                    seeing. <br>
                    <br>
                    --Shane<br>
                    <br>
                    <blockquote type="cite"> <br>
                      Thanks,<br>
                      --Shane<br>
                      <br>
                      <div class="m_-8621407278620756273moz-cite-prefix">On
                        07/20/2017 03:52 PM, Teng Wang wrote:<br>
                      </div>
                      <blockquote type="cite">
                        <div dir="ltr">Hi Shane,
                          <div><br>
                          </div>
                          <div>Thanks for your help on this. I tried
                            v3.1.3 and compiled successfully. </div>
                          <div>But when I ran the application, there was
                            no *.log generated under the </div>
                          <div>log directory, and I also got a warning
                            from the application:</div>
                          <div>
                            <p
style="margin:0px;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures;color:rgb(175,173,36)"> </span><span
style="font-variant-ligatures:no-common-ligatures">darshan library
                                warning: unable to compress job data</span></p>
                            <p
style="margin:0px;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><br>
                            </p>
                          </div>
                          <div>May I know how to fix this?</div>
                          <div><br>
                          </div>
                          <div>Thanks,</div>
                          <div>Teng</div>
                        </div>
                        <div class="gmail_extra"><br>
                          <div class="gmail_quote">On Thu, Jul 20, 2017
                            at 11:51 AM, Shane Snyder <span dir="ltr"><<a
                                href="mailto:ssnyder@mcs.anl.gov"
                                target="_blank" moz-do-not-send="true">ssnyder@mcs.anl.gov</a>></span>
                            wrote:<br>
                            <blockquote class="gmail_quote"
                              style="margin:0 0 0 .8ex;border-left:1px
                              #ccc solid;padding-left:1ex">
                              <div text="#000000" bgcolor="#FFFFFF"> Hi
                                Teng,<br>
                                <br>
                                Thanks for reporting this. This is a
                                really strange issue, but I can actually
                                reproduce it, too. I'm going to keep
                                digging and see if I can find the
                                problem. Will keep you updated.<br>
                                <br>
                                The strange thing is that I was able to
                                use version 3.1.4 successfully right
                                before making the release available
                                (i.e., the log @
                                darshan-test/example-output/mp<wbr>i-io-test-x86_64-3.1.4.darshan
                                was generated using version 3.1.4 with
                                static linking). Also strange is that
                                our nightly testing is not catching this
                                issue...<br>
                                <br>
                                FYI, version 3.1.3 and earlier work fine
                                for me if you really need a quick
                                resolution -- version 3.1.4 just fixed a
                                couple of non-critical bugs in some
                                darshan log parsing utilities, so there
                                shouldn't be much functionally different
                                between the versions.<br>
                                <br>
                                --Shane
                                <div>
                                  <div class="m_-8621407278620756273h5"><br>
                                    <br>
                                    <div
                                      class="m_-8621407278620756273m_6259078016213707887moz-cite-prefix">On
                                      07/20/2017 12:43 PM, Teng Wang
                                      wrote:<br>
                                    </div>
                                  </div>
                                </div>
                                <blockquote type="cite">
                                  <div>
                                    <div
                                      class="m_-8621407278620756273h5">
                                      <div dir="ltr">
                                        <p>Hi,</p>
                                        <p><br>
                                        </p>
                                        <p>I'm having trouble when
                                          trying to statically link </p>
                                        <p>Darshan with mpich3.3
                                          applications following the
                                          instructions </p>
                                        <p>here. Could you give me any
                                          suggestion on how to fix it? </p>
                                        <p><br>
                                        </p>
                                        <p><span
                                            style="text-decoration:underline"><a
href="http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html"
                                              target="_blank"
                                              moz-do-not-send="true">http://www.mcs.anl.gov/researc<wbr>h/projects/darshan/docs/<wbr>darshan-runtime.html</a></span></p>
                                        <p><br>
                                        </p>
                                        <p>Here is my steps:</p>
                                        <p>1. Compile and install
                                          darshan 3.1.4</p>
                                        <p>tar -xvzf
                                          darshan-3.1.4.tar.gz</p>
                                        <p>cd
                                          darshan-3.1.4/darshan-runtime</p>
                                        <p>./configure
                                          --with-mem-align=8
                                          --with-log-path=<log
                                          directory>
                                          --prefix=<install
                                          directory>
                                          --with-jobid-env=PBS_JOBID
                                          --disable-cuserid CC=mpicc</p>
                                        <p>make</p>
                                        <p>make install</p>
                                        <p><br>
                                        </p>
                                        <p>2. Create log directory,
                                          which successfully create the
                                          directory </p>
                                        <p><span
                                            style="text-decoration:underline"><a
href="http://darshan-mk-log-dirs.pl/" target="_blank"
                                              moz-do-not-send="true">darshan-mk-log-dirs.pl</a></span></p>
                                        <p><br>
                                        </p>
                                        <p>3. Generate wrapper</p>
                                        <p><span
                                            style="text-decoration:underline;color:rgb(228,175,10)"><a
href="http://darshan-gen-cc.pl/" target="_blank" moz-do-not-send="true">darshan-gen-cc.pl</a></span>
                                          `which mpicc` --output
                                          mpicc.darshan</p>
                                        <p><br>
                                        </p>
                                        <p>4. Compile application
                                          (mpi-io-test.c) using
                                          mpicc.darshan</p>
                                        <p>mpicc.darshan mpi-io-test.c
                                          -o mpi-io-test</p>
                                        <p><br>
                                        </p>
                                        <p>After step 4, the following
                                          issue happened:</p>
                                        <p>initthread.c:(.text+0xc59d):
                                          undefined reference to
                                          `__wrap_fprintf'</p>
                                        <p>initthread.c:(.text+0xc5bd):
                                          undefined reference to
                                          `__wrap_fputs'</p>
                                        <p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-finalize<wbr>.o):
                                          In function
                                          `MPIR_Add_finalize':</p>
                                        <p>finalize.c:(.text+0x226e):
                                          undefined reference to
                                          `__wrap_fputs'</p>
                                        <p>finalize.c:(.text+0x227a):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p>finalize.c:(.text+0x2286):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-util.o):
                                          In function
                                          `MPIDI_OFI_control_handler':</p>
                                        <p>util.c:(.text+0x17bc4):
                                          undefined reference to
                                          `__wrap_fprintf'</p>
                                        <p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-ch4_glob<wbr>als.o):
                                          In function `MPID_Abort':</p>
                                        <p>ch4_globals.c:(.text+0x38b):
                                          undefined reference to
                                          `__wrap_fputs'</p>
                                        <p>ch4_globals.c:(.text+0x397):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p>ch4_globals.c:(.text+0x3a3):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p>/tmp/ccyntLoA.o: In function
                                          `main':</p>
                                        <p>mpi-io-test.c:(.text+0x5ae):
                                          undefined reference to
                                          `__wrap_printf'</p>
                                        <p>mpi-io-test.c:(.text+0x819):
                                          undefined reference to
                                          `__wrap_printf'</p>
                                        <p>/global/homes/t/user/software_<wbr>install/mpich3.3a-static/lib/l<wbr>ibmpi.a(lib_libmpi_la-contexti<wbr>d.o):
                                          In function
                                          `MPIR_Free_contextid':</p>
                                        <p>contextid.c:(.text+0x1c3f):
                                          undefined reference to
                                          `__wrap_fputs'</p>
                                        <p>contextid.c:(.text+0x1c4b):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p>contextid.c:(.text+0x1c57):
                                          undefined reference to
                                          `__wrap_fflush'</p>
                                        <p><br>
                                        </p>
                                        <p><br>
                                        </p>
                                        <p>mpicc.darshan -show
                                          mpi-io-test.c -o mpi-io-test
                                          gave:</p>
                                        <p><br>
                                        </p>
                                        <p>gcc -L/usr/lib64/slurmpmi
                                          -L/global/common/cori/software<wbr>/libfabric/1.4.1/gnu/lib
                                          mpi-io-test.c -o mpi-io-test
                                          -L/global/homes/t/user/softwar<wbr>e_install/darshan/lib
                                          -ldarshan -lz
                                          -Wl,@/global/homes/t/user/soft<wbr>ware_install/darshan/share/ld-<wbr>opts/darshan-base-ld-opts
                                          -I/global/homes/t/user/softwar<wbr>e_install/mpich3.3a-static/<wbr>include
                                          -L/global/homes/t/user/softwar<wbr>e_install/mpich3.3a-static/lib
                                          -lmpi -lpmi -lpmi -lpthread
                                          -lfabric -lrt -lpmi
                                          -L/global/homes/t/user/softwar<wbr>e_install/darshan/lib
                                          -Wl,--start-group -ldarshan
                                          -ldarshan-stubs
                                          -Wl,--end-group -lz -lrt
                                          -lpthread</p>
                                        <p><br>
                                        </p>
                                        <p>The strange thing is when I
                                          do </p>
                                        <p>nm libdarshan.a, it did show
                                          the functions (e.g.
                                          __wrap_printf) were linked to </p>
                                        <p>libdarshan.a. </p>
                                        <p><br>
                                        </p>
                                        <p><b>0000000000002bf0 T
                                            __wrap__IO_getc</b></p>
                                        <p><b>0000000000002d80 T
                                            __wrap__IO_putc</b></p>
                                        <p><b>00000000000030b0 T
                                            __wrap___isoc99_fscanf</b></p>
                                        <p><b>0000000000001990 T
                                            __wrap_fclose</b></p>
                                        <p><b>00000000000011d0 T
                                            __wrap_fdopen</b></p>
                                        <p><b>0000000000001810 T
                                            __wrap_fflush</b></p>
                                        <p><b>0000000000002a60 T
                                            __wrap_fgetc</b></p>
                                        <p><b>0000000000003690 T
                                            __wrap_fgets</b></p>
                                        <p><b>0000000000000db0 T
                                            __wrap_fopen</b></p>
                                        <p><b>0000000000000fc0 T
                                            __wrap_fopen64</b></p>
                                        <p><b>00000000000026b0 T
                                            __wrap_fprintf</b></p>
                                        <p><b>0000000000001c80 T
                                            __wrap_fputc</b></p>
                                        <p><b>0000000000001fa0 T
                                            __wrap_fputs</b></p>
                                        <p><b>00000000000028d0 T
                                            __wrap_fread</b></p>
                                        <p><b>00000000000013d0 T
                                            __wrap_freopen</b></p>
                                        <p><b>00000000000015f0 T
                                            __wrap_freopen64</b></p>
                                        <p><b>00000000000032d0 T
                                            __wrap_fscanf</b></p>
                                        <p><b>0000000000003920 T
                                            __wrap_fseek</b></p>
                                        <p><b>0000000000003a30 T
                                            __wrap_fseeko</b></p>
                                        <p><b>0000000000003b40 T
                                            __wrap_fseeko64</b></p>
                                        <p><b>0000000000003c50 T
                                            __wrap_fsetpos</b></p>
                                        <p><br>
                                        </p>
                                        <p>Thanks,</p>
                                        <p>Teng</p>
                                      </div>
                                      <br>
                                      <fieldset
                                        class="m_-8621407278620756273m_6259078016213707887mimeAttachmentHeader"></fieldset>
                                      <br>
                                    </div>
                                  </div>
                                  <pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273m_6259078016213707887moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank" moz-do-not-send="true">Darshan-users@lists.mcs.anl.go<wbr>v</a>
<a class="m_-8621407278620756273m_6259078016213707887moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank" moz-do-not-send="true">https://lists.mcs.anl.gov/mail<wbr>man/listinfo/darshan-users</a>
</pre>
                                </blockquote>
                                <br>
                              </div>
                              <br>
                              ______________________________<wbr>_________________<br>
                              Darshan-users mailing list<br>
                              <a
                                href="mailto:Darshan-users@lists.mcs.anl.gov"
                                target="_blank" moz-do-not-send="true">Darshan-users@lists.mcs.anl.go<wbr>v</a><br>
                              <a
                                href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users"
                                rel="noreferrer" target="_blank"
                                moz-do-not-send="true">https://lists.mcs.anl.gov/mail<wbr>man/listinfo/darshan-users</a><br>
                              <br>
                            </blockquote>
                          </div>
                          <br>
                        </div>
                      </blockquote>
                      <br>
                      <br>
                      <fieldset
                        class="m_-8621407278620756273mimeAttachmentHeader"></fieldset>
                      <br>
                      <pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank" moz-do-not-send="true">Darshan-users@lists.mcs.anl.<wbr>gov</a>
<a class="m_-8621407278620756273moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank" moz-do-not-send="true">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a>
</pre>
                    </blockquote>
                    <br>
                    <br>
                    <fieldset
                      class="m_-8621407278620756273mimeAttachmentHeader"></fieldset>
                    <br>
                    <pre>______________________________<wbr>_________________
Darshan-users mailing list
<a class="m_-8621407278620756273moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov" target="_blank" moz-do-not-send="true">Darshan-users@lists.mcs.anl.<wbr>gov</a>
<a class="m_-8621407278620756273moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" target="_blank" moz-do-not-send="true">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a>
</pre>
                  </blockquote>
                  <p><br>
                  </p>
                </div>
              </div>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            Darshan-users mailing list<br>
            <a href="mailto:Darshan-users@lists.mcs.anl.gov"
              moz-do-not-send="true">Darshan-users@lists.mcs.anl.<wbr>gov</a><br>
            <a
              href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users"
              rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.mcs.anl.gov/<wbr>mailman/listinfo/darshan-users</a><br>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <p><br>
    </p>
  </body>
</html>