[Darshan-users] Darshan-users Digest, Vol 45, Issue 6

[Adnan Haider]

Hi Phil,

Sorry for the late reply, we were focusing on some other aspects but now we
are back to using Darshan for analyzing our application. Our executable is
statically linked; the previous error was caused by LD_PRELOAD being set in
our environment. After resetting LD_PRELOAD (so that we are not dynamically
instrumenting), the application runs and there are no symbol mapping
errors. However after job completion, no trace file appears in the log
directory. We used environment variables with the latest MPICH release in
an attempt to statically instrument our application. Compiling a simple mpi
application with the environment variables set does produce a Darshan trace
file, but our application does not. Also for our executable, using ldd
<python script> we get: "not a dynamic executable"


Here is some more detail about our application and its dependencies. Our
application is written in python. The application uses mpi4py for all MPI
calls. Also we are reading and writing netcdf files but are using PyNIO
which breaks down python calls into standard netcdf I/O.  This site
<http://www.alcf.anl.gov/user-guides/darshan> says that dealing with
languages other than c,c++,and Fortran could cause issues. I was thinking
the python code could be causing the issue. Possibly, the PyNIO package on
top of netcdf or mpi4py are not being detected by Darshan. Or more likely,
I am not correctly compiling the correct software with the environment
variables. The only software that directly uses the mpi compiler is mpi4py,
and I did recompile this after setting the environment variables. I also
recompiled PyNIO and our application for good measure.


Thanks, Adnan Haider
B.S Candidate, Computer Science
Illinois Institute of Technology

On Fri, May 29, 2015 at 10:00 AM, <darshan-users-request at lists.mcs.anl.gov>
wrote:

> Send Darshan-users mailing list submissions to
>         darshan-users at lists.mcs.anl.gov
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
> or, via email, send a message with subject or body 'help' to
>         darshan-users-request at lists.mcs.anl.gov
>
> You can reach the person managing the list at
>         darshan-users-owner at lists.mcs.anl.gov
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Darshan-users digest..."
>
>
> Today's Topics:
>
>    1. Darshan error with HDF5 (Adnan Haider)
>    2. Re: Darshan error with HDF5 (Phil Carns)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 28 May 2015 14:20:07 -0600
> From: Adnan Haider <ahaider3 at hawk.iit.edu>
> To: darshan-users at lists.mcs.anl.gov
> Subject: [Darshan-users] Darshan error with HDF5
> Message-ID:
>         <
> CAM2fH+yEoqtPOnxVPvZNm853kCXEmhWQMrDpWWxZ8UEWC8jUcA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
> I am hoping to use Darshan for analyzing our application's performance. We
> are using HDF5 and netcdf 4. I installed Darshan version 2.3.1 and mvapich2
> for our mpi implementation. Darshan works fine for a standard mpi pingpong
> application. When we run our application, however, it gives the following
> error:
> Darshan failed to map symbol: H5Fopen
> Darshan failed to map symbol: H5Fopen
> Darshan failed to map symbol: H5Fopen
> Darshan failed to map symbol: H5Fopen
>
> The four instances of the error probably have to do with the 4 mpi ranks
> specified at runtime. Nothing is written into the log directory. This is
> the only output, unless there is some error file I am not aware of. I used
> the static instrumentation approach for enabling Darshan. Thanks for the
> help!
>
> Thanks, Adnan Haider
> B.S Candidate, Computer Science
> Illinois Institute of Technology
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/5ebdf005/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Thu, 28 May 2015 16:32:04 -0400
> From: Phil Carns <carns at mcs.anl.gov>
> To: <darshan-users at lists.mcs.anl.gov>
> Subject: Re: [Darshan-users] Darshan error with HDF5
> Message-ID: <55677B44.7060107 at mcs.anl.gov>
> Content-Type: text/plain; charset="windows-1252"; Format="flowed"
>
> Hi Adnan!
>
> Is everything in the application statically linked (maybe check with ldd
> <executable>)?  That error message is supposed to only be generated by
> the dynamic (ld-preloaded) darshan library, because it has to locate
> underlying symbols at run time.  The symbol name mapping should be set
> at link time for a static executable.
>
> thanks,
> -Phil
>
> On 05/28/2015 04:20 PM, Adnan Haider wrote:
> > Hi,
> > I am hoping to use Darshan for analyzing our application's
> > performance. We are using HDF5 and netcdf 4. I installed Darshan
> > version 2.3.1 and mvapich2 for our mpi implementation. Darshan works
> > fine for a standard mpi pingpong application. When we run our
> > application, however, it gives the following error:
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> > Darshan failed to map symbol: H5Fopen
> >
> > The four instances of the error probably have to do with the 4 mpi
> > ranks specified at runtime. Nothing is written into the log directory.
> > This is the only output, unless there is some error file I am not
> > aware of. I used the static instrumentation approach for enabling
> > Darshan. Thanks for the help!
> >
> > Thanks, Adnan Haider
> > B.S Candidate, Computer Science
> > Illinois Institute of Technology
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150528/bc72a5df/attachment-0001.html
> >
>
> ------------------------------
>
> _______________________________________________
> Darshan-users mailing list
> Darshan-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>
>
> End of Darshan-users Digest, Vol 45, Issue 6
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20150629/a6608c8b/attachment.html>