<div dir="ltr">Hey Phil,<div>After exclusively using LD_PRELOAD, i get the following error:</div><div>Darshan failed to map symbol: H5FOpen</div><div>Darshan failed to map symbol: H5FOpen<br></div><div>Darshan failed to map symbol: H5FOpen<br></div><div>Darshan failed to map symbol: H5FOpen<br></div><div><br></div><div>Also you are correct; our application during runtime loads mpi4py which contains the MPI_Init and MPI_Finalize calls.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">Thanks, Adnan Haider<div>B.S Candidate, Computer Science</div><div>Illinois Institute of Technology </div></div></div></div>
<br><div class="gmail_quote">On Thu, Jul 2, 2015 at 10:00 AM, <span dir="ltr"><<a href="mailto:darshan-users-request@lists.mcs.anl.gov" target="_blank">darshan-users-request@lists.mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send Darshan-users mailing list submissions to<br>
<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:darshan-users-request@lists.mcs.anl.gov">darshan-users-request@lists.mcs.anl.gov</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:darshan-users-owner@lists.mcs.anl.gov">darshan-users-owner@lists.mcs.anl.gov</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Darshan-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: Darshan-users Digest, Vol 45, Issue 6 (Phil Carns)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 2 Jul 2015 09:40:10 -0400<br>
From: Phil Carns <<a href="mailto:carns@mcs.anl.gov">carns@mcs.anl.gov</a>><br>
To: <<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a>><br>
Subject: Re: [Darshan-users] Darshan-users Digest, Vol 45, Issue 6<br>
Message-ID: <<a href="mailto:55953F3A.9020005@mcs.anl.gov">55953F3A.9020005@mcs.anl.gov</a>><br>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"<br>
<br>
Hi Adnan,<br>
<br>
Python indeed might be a little trickier, but I think its possible.<br>
I'll see if I can find more information about this and report back.<br>
<br>
I *think* that you need to actually do the reverse and disable the<br>
static instrumentation and use the dynamic LD_PRELOAD approach instead<br>
in this case. I imagine that the Python MPI bindings are dynamically<br>
loading the MPI library at runtime even though the Python program itself<br>
doesn't show up as a dynamic executable.<br>
<br>
The critical point in getting a Darshan log to be generated is that it<br>
must intercept MPI_Init() and MPI_Finalize(). If it can achieve that<br>
much then it should at least produce a log file.<br>
<br>
Can you try that method again (disabling the static instrumentation and<br>
instead using LD_PRELOAD exclusively) and confirm the error?<br>
<br>
thanks,<br>
-Phil<br>
<br>
On 06/29/2015 02:47 PM, Adnan Haider wrote:<br>
> Hi Phil,<br>
><br>
> Sorry for the late reply, we were focusing on some other aspects but<br>
> now we are back to using Darshan for analyzing our application. Our<br>
> executable is statically linked; the previous error was caused by<br>
> LD_PRELOAD being set in our environment. After resetting LD_PRELOAD<br>
> (so that we are not dynamically instrumenting), the application runs<br>
> and there are no symbol mapping errors. However after job completion,<br>
> no trace file appears in the log directory. We used environment<br>
> variables with the latest MPICH release in an attempt to statically<br>
> instrument our application. Compiling a simple mpi application with<br>
> the environment variables set does produce a Darshan trace file, but<br>
> our application does not. Also for our executable, using ldd <python<br>
> script> we get: "not a dynamic executable"<br>
><br>
><br>
> Here is some more detail about our application and its dependencies.<br>
> Our application is written in python. The application uses mpi4py for<br>
> all MPI calls. Also we are reading and writing netcdf files but are<br>
> using PyNIO which breaks down python calls into standard netcdf I/O.<br>
> This site <<a href="http://www.alcf.anl.gov/user-guides/darshan" rel="noreferrer" target="_blank">http://www.alcf.anl.gov/user-guides/darshan</a>> says that<br>
> dealing with languages other than c,c++,and Fortran could cause<br>
> issues. I was thinking the python code could be causing the issue.<br>
> Possibly, the PyNIO package on top of netcdf or mpi4py are not being<br>
> detected by Darshan. Or more likely, I am not correctly compiling the<br>
> correct software with the environment variables. The only software<br>
> that directly uses the mpi compiler is mpi4py, and I did recompile<br>
> this after setting the environment variables. I also recompiled PyNIO<br>
> and our application for good measure.<br>
><br>
><br>
> Thanks, Adnan Haider<br>
> B.S Candidate, Computer Science<br>
> Illinois Institute of Technology<br>
><br>
> On Fri, May 29, 2015 at 10:00 AM,<br>
> <<a href="mailto:darshan-users-request@lists.mcs.anl.gov">darshan-users-request@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users-request@lists.mcs.anl.gov">darshan-users-request@lists.mcs.anl.gov</a>>> wrote:<br>
><br>
> Send Darshan-users mailing list submissions to<br>
> <a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a>><br>
><br>
> To subscribe or unsubscribe via the World Wide Web, visit<br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
> <<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a>><br>
> or, via email, send a message with subject or body 'help' to<br>
> <a href="mailto:darshan-users-request@lists.mcs.anl.gov">darshan-users-request@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users-request@lists.mcs.anl.gov">darshan-users-request@lists.mcs.anl.gov</a>><br>
><br>
> You can reach the person managing the list at<br>
> <a href="mailto:darshan-users-owner@lists.mcs.anl.gov">darshan-users-owner@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users-owner@lists.mcs.anl.gov">darshan-users-owner@lists.mcs.anl.gov</a>><br>
><br>
> When replying, please edit your Subject line so it is more specific<br>
> than "Re: Contents of Darshan-users digest..."<br>
><br>
><br>
> Today's Topics:<br>
><br>
> 1. Darshan error with HDF5 (Adnan Haider)<br>
> 2. Re: Darshan error with HDF5 (Phil Carns)<br>
><br>
><br>
> ----------------------------------------------------------------------<br>
><br>
> Message: 1<br>
> Date: Thu, 28 May 2015 14:20:07 -0600<br>
> From: Adnan Haider <<a href="mailto:ahaider3@hawk.iit.edu">ahaider3@hawk.iit.edu</a><br>
> <mailto:<a href="mailto:ahaider3@hawk.iit.edu">ahaider3@hawk.iit.edu</a>>><br>
> To: <a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a>><br>
> Subject: [Darshan-users] Darshan error with HDF5<br>
> Message-ID:<br>
><br>
> <<a href="mailto:CAM2fH%2ByEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA@mail.gmail.com">CAM2fH+yEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA@mail.gmail.com</a> <mailto:<a href="mailto:CAM2fH%252ByEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA@mail.gmail.com">CAM2fH%2ByEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA@mail.gmail.com</a>>><br>
> Content-Type: text/plain; charset="utf-8"<br>
><br>
> Hi,<br>
> I am hoping to use Darshan for analyzing our application's<br>
> performance. We<br>
> are using HDF5 and netcdf 4. I installed Darshan version 2.3.1 and<br>
> mvapich2<br>
> for our mpi implementation. Darshan works fine for a standard mpi<br>
> pingpong<br>
> application. When we run our application, however, it gives the<br>
> following<br>
> error:<br>
> Darshan failed to map symbol: H5Fopen<br>
> Darshan failed to map symbol: H5Fopen<br>
> Darshan failed to map symbol: H5Fopen<br>
> Darshan failed to map symbol: H5Fopen<br>
><br>
> The four instances of the error probably have to do with the 4 mpi<br>
> ranks<br>
> specified at runtime. Nothing is written into the log directory.<br>
> This is<br>
> the only output, unless there is some error file I am not aware<br>
> of. I used<br>
> the static instrumentation approach for enabling Darshan. Thanks<br>
> for the<br>
> help!<br>
><br>
> Thanks, Adnan Haider<br>
> B.S Candidate, Computer Science<br>
> Illinois Institute of Technology<br>
> -------------- next part --------------<br>
> An HTML attachment was scrubbed...<br>
> URL:<br>
> <<a href="http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/5ebdf005/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/5ebdf005/attachment-0001.html</a>><br>
><br>
> ------------------------------<br>
><br>
> Message: 2<br>
> Date: Thu, 28 May 2015 16:32:04 -0400<br>
> From: Phil Carns <<a href="mailto:carns@mcs.anl.gov">carns@mcs.anl.gov</a> <mailto:<a href="mailto:carns@mcs.anl.gov">carns@mcs.anl.gov</a>>><br>
> To: <<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a>>><br>
> Subject: Re: [Darshan-users] Darshan error with HDF5<br>
> Message-ID: <<a href="mailto:55677B44.7060107@mcs.anl.gov">55677B44.7060107@mcs.anl.gov</a><br>
> <mailto:<a href="mailto:55677B44.7060107@mcs.anl.gov">55677B44.7060107@mcs.anl.gov</a>>><br>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"<br>
><br>
> Hi Adnan!<br>
><br>
> Is everything in the application statically linked (maybe check<br>
> with ldd<br>
> <executable>)? That error message is supposed to only be generated by<br>
> the dynamic (ld-preloaded) darshan library, because it has to locate<br>
> underlying symbols at run time. The symbol name mapping should be set<br>
> at link time for a static executable.<br>
><br>
> thanks,<br>
> -Phil<br>
><br>
> On 05/28/2015 04:20 PM, Adnan Haider wrote:<br>
> > Hi,<br>
> > I am hoping to use Darshan for analyzing our application's<br>
> > performance. We are using HDF5 and netcdf 4. I installed Darshan<br>
> > version 2.3.1 and mvapich2 for our mpi implementation. Darshan works<br>
> > fine for a standard mpi pingpong application. When we run our<br>
> > application, however, it gives the following error:<br>
> > Darshan failed to map symbol: H5Fopen<br>
> > Darshan failed to map symbol: H5Fopen<br>
> > Darshan failed to map symbol: H5Fopen<br>
> > Darshan failed to map symbol: H5Fopen<br>
> ><br>
> > The four instances of the error probably have to do with the 4 mpi<br>
> > ranks specified at runtime. Nothing is written into the log<br>
> directory.<br>
> > This is the only output, unless there is some error file I am not<br>
> > aware of. I used the static instrumentation approach for enabling<br>
> > Darshan. Thanks for the help!<br>
> ><br>
> > Thanks, Adnan Haider<br>
> > B.S Candidate, Computer Science<br>
> > Illinois Institute of Technology<br>
><br>
><br>
> -------------- next part --------------<br>
> An HTML attachment was scrubbed...<br>
> URL:<br>
> <<a href="http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/bc72a5df/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/bc72a5df/attachment-0001.html</a>><br>
><br>
> ------------------------------<br>
><br>
> _______________________________________________<br>
> Darshan-users mailing list<br>
> <a href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><br>
> <mailto:<a href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a>><br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
><br>
><br>
> End of Darshan-users Digest, Vol 45, Issue 6<br>
> ********************************************<br>
><br>
><br>
<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150702/f7894a89/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150702/f7894a89/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Darshan-users mailing list<br>
<a href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users" rel="noreferrer" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
<br>
<br>
End of Darshan-users Digest, Vol 47, Issue 1<br>
********************************************<br>
</blockquote></div><br></div>