[Darshan-users] Darshan-users Digest, Vol 47, Issue 1
Adnan Haider
ahaider3 at hawk.iit.edu
Mon Jul 6 17:23:34 CDT 2015
Hey Phil,
After exclusively using LD_PRELOAD, i get the following error:
Darshan failed to map symbol: H5FOpen
Darshan failed to map symbol: H5FOpen
Darshan failed to map symbol: H5FOpen
Darshan failed to map symbol: H5FOpen
Also you are correct; our application during runtime loads mpi4py which
contains the MPI_Init and MPI_Finalize calls.
Thanks, Adnan Haider
B.S Candidate, Computer Science
Illinois Institute of Technology
On Thu, Jul 2, 2015 at 10:00 AM, <darshan-users-request at lists.mcs.anl.gov>
wrote:
> Send Darshan-users mailing list submissions to
> darshan-users at lists.mcs.anl.gov
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
> or, via email, send a message with subject or body 'help' to
> darshan-users-request at lists.mcs.anl.gov
>
> You can reach the person managing the list at
> darshan-users-owner at lists.mcs.anl.gov
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Darshan-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Darshan-users Digest, Vol 45, Issue 6 (Phil Carns)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 2 Jul 2015 09:40:10 -0400
> From: Phil Carns <carns at mcs.anl.gov>
> To: <darshan-users at lists.mcs.anl.gov>
> Subject: Re: [Darshan-users] Darshan-users Digest, Vol 45, Issue 6
> Message-ID: <55953F3A.9020005 at mcs.anl.gov>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hi Adnan,
>
> Python indeed might be a little trickier, but I think its possible.
> I'll see if I can find more information about this and report back.
>
> I *think* that you need to actually do the reverse and disable the
> static instrumentation and use the dynamic LD_PRELOAD approach instead
> in this case. I imagine that the Python MPI bindings are dynamically
> loading the MPI library at runtime even though the Python program itself
> doesn't show up as a dynamic executable.
>
> The critical point in getting a Darshan log to be generated is that it
> must intercept MPI_Init() and MPI_Finalize(). If it can achieve that
> much then it should at least produce a log file.
>
> Can you try that method again (disabling the static instrumentation and
> instead using LD_PRELOAD exclusively) and confirm the error?
>
> thanks,
> -Phil
>
> On 06/29/2015 02:47 PM, Adnan Haider wrote:
> > Hi Phil,
> >
> > Sorry for the late reply, we were focusing on some other aspects but
> > now we are back to using Darshan for analyzing our application. Our
> > executable is statically linked; the previous error was caused by
> > LD_PRELOAD being set in our environment. After resetting LD_PRELOAD
> > (so that we are not dynamically instrumenting), the application runs
> > and there are no symbol mapping errors. However after job completion,
> > no trace file appears in the log directory. We used environment
> > variables with the latest MPICH release in an attempt to statically
> > instrument our application. Compiling a simple mpi application with
> > the environment variables set does produce a Darshan trace file, but
> > our application does not. Also for our executable, using ldd <python
> > script> we get: "not a dynamic executable"
> >
> >
> > Here is some more detail about our application and its dependencies.
> > Our application is written in python. The application uses mpi4py for
> > all MPI calls. Also we are reading and writing netcdf files but are
> > using PyNIO which breaks down python calls into standard netcdf I/O.
> > This site <http://www.alcf.anl.gov/user-guides/darshan> says that
> > dealing with languages other than c,c++,and Fortran could cause
> > issues. I was thinking the python code could be causing the issue.
> > Possibly, the PyNIO package on top of netcdf or mpi4py are not being
> > detected by Darshan. Or more likely, I am not correctly compiling the
> > correct software with the environment variables. The only software
> > that directly uses the mpi compiler is mpi4py, and I did recompile
> > this after setting the environment variables. I also recompiled PyNIO
> > and our application for good measure.
> >
> >
> > Thanks, Adnan Haider
> > B.S Candidate, Computer Science
> > Illinois Institute of Technology
> >
> > On Fri, May 29, 2015 at 10:00 AM,
> > <darshan-users-request at lists.mcs.anl.gov
> > <mailto:darshan-users-request at lists.mcs.anl.gov>> wrote:
> >
> > Send Darshan-users mailing list submissions to
> > darshan-users at lists.mcs.anl.gov
> > <mailto:darshan-users at lists.mcs.anl.gov>
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
> > <https://lists.mcs.anl.gov/mailman/listinfo/darshan-users>
> > or, via email, send a message with subject or body 'help' to
> > darshan-users-request at lists.mcs.anl.gov
> > <mailto:darshan-users-request at lists.mcs.anl.gov>
> >
> > You can reach the person managing the list at
> > darshan-users-owner at lists.mcs.anl.gov
> > <mailto:darshan-users-owner at lists.mcs.anl.gov>
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Darshan-users digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Darshan error with HDF5 (Adnan Haider)
> > 2. Re: Darshan error with HDF5 (Phil Carns)
> >
> >
> >
> ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Thu, 28 May 2015 14:20:07 -0600
> > From: Adnan Haider <ahaider3 at hawk.iit.edu
> > <mailto:ahaider3 at hawk.iit.edu>>
> > To: darshan-users at lists.mcs.anl.gov
> > <mailto:darshan-users at lists.mcs.anl.gov>
> > Subject: [Darshan-users] Darshan error with HDF5
> > Message-ID:
> >
> > <CAM2fH+yEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA at mail.gmail.com
> <mailto:
> CAM2fH%2ByEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA at mail.gmail.com>>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Hi,
> > I am hoping to use Darshan for analyzing our application's
> > performance. We
> > are using HDF5 and netcdf 4. I installed Darshan version 2.3.1 and
> > mvapich2
> > for our mpi implementation. Darshan works fine for a standard mpi
> > pingpong
> > application. When we run our application, however, it gives the
> > following
> > error:
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> >
> > The four instances of the error probably have to do with the 4 mpi
> > ranks
> > specified at runtime. Nothing is written into the log directory.
> > This is
> > the only output, unless there is some error file I am not aware
> > of. I used
> > the static instrumentation approach for enabling Darshan. Thanks
> > for the
> > help!
> >
> > Thanks, Adnan Haider
> > B.S Candidate, Computer Science
> > Illinois Institute of Technology
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/5ebdf005/attachment-0001.html
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Thu, 28 May 2015 16:32:04 -0400
> > From: Phil Carns <carns at mcs.anl.gov <mailto:carns at mcs.anl.gov>>
> > To: <darshan-users at lists.mcs.anl.gov
> > <mailto:darshan-users at lists.mcs.anl.gov>>
> > Subject: Re: [Darshan-users] Darshan error with HDF5
> > Message-ID: <55677B44.7060107 at mcs.anl.gov
> > <mailto:55677B44.7060107 at mcs.anl.gov>>
> > Content-Type: text/plain; charset="windows-1252"; Format="flowed"
> >
> > Hi Adnan!
> >
> > Is everything in the application statically linked (maybe check
> > with ldd
> > <executable>)? That error message is supposed to only be generated
> by
> > the dynamic (ld-preloaded) darshan library, because it has to locate
> > underlying symbols at run time. The symbol name mapping should be
> set
> > at link time for a static executable.
> >
> > thanks,
> > -Phil
> >
> > On 05/28/2015 04:20 PM, Adnan Haider wrote:
> > > Hi,
> > > I am hoping to use Darshan for analyzing our application's
> > > performance. We are using HDF5 and netcdf 4. I installed Darshan
> > > version 2.3.1 and mvapich2 for our mpi implementation. Darshan
> works
> > > fine for a standard mpi pingpong application. When we run our
> > > application, however, it gives the following error:
> > > Darshan failed to map symbol: H5Fopen
> > > Darshan failed to map symbol: H5Fopen
> > > Darshan failed to map symbol: H5Fopen
> > > Darshan failed to map symbol: H5Fopen
> > >
> > > The four instances of the error probably have to do with the 4 mpi
> > > ranks specified at runtime. Nothing is written into the log
> > directory.
> > > This is the only output, unless there is some error file I am not
> > > aware of. I used the static instrumentation approach for enabling
> > > Darshan. Thanks for the help!
> > >
> > > Thanks, Adnan Haider
> > > B.S Candidate, Computer Science
> > > Illinois Institute of Technology
> >
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> > <
> http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/bc72a5df/attachment-0001.html
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > Darshan-users mailing list
> > Darshan-users at lists.mcs.anl.gov
> > <mailto:Darshan-users at lists.mcs.anl.gov>
> > https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
> >
> >
> > End of Darshan-users Digest, Vol 45, Issue 6
> > ********************************************
> >
> >
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150702/f7894a89/attachment-0001.html
> >
>
> ------------------------------
>
> _______________________________________________
> Darshan-users mailing list
> Darshan-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>
>
> End of Darshan-users Digest, Vol 47, Issue 1
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150706/9e69492c/attachment-0001.html>
More information about the Darshan-users
mailing list