[Darshan-users] Profiling Python C/C++ extensions with Darshan
ivan.g.raikov at gmail.com
Mon Feb 27 17:50:36 CST 2017
The Blue Waters support staff worked out a solution, so I am posting
it here for the sake of posterity.
The following environment variable settings are necessary to work
around the socket error on MPI startup:
#both hdf5 and darshan are needed.
On Mon, Feb 27, 2017 at 11:09 AM, Ivan Raikov <ivan.g.raikov at gmail.com> wrote:
> I have a Python extension written in C++ which calls the HDF library,
> and I would like to use Darshan to profile an MPI-based Python
> application which invokes that extension. However, compiling the
> extension and running the application on Blue Waters (Cray-based
> system) with Darshan 2.3 results in no profile data being generated.
> Using LD_PRELOAD to load Darshan dynamically results in some kind of
> socket error when (I think) the MPI process manager is launching the
> Python executable. I have also tried the LD_PRELOAD mechanism with
> Darshan 3.1.3 built by the HDF group, but I get the same error. I was
> wondering if anyone here has done profiling with dynamically linked
> Python extensions, or if anyone has any advice how to approach this.
> I invoke the application with LD_PRELOAD thus:
> export LD_PRELOAD=/path/to/darshan/3.1.3/lib/libdarshan.so
> aprun -n 4096 python main.py ...
> aprun -n 4096 -e LD_PRELOAD=/path/to/darshan/3.1.3/.../lib/libdarshan.so
> The error I get with LD_PRELOAD is the following:
> [PE_3344]:inet_listen_socket_setup:inet_setup_listen_socket: bind
> failed port 1371 listen_sock = 88 Address already in use
> [PE_3344]:_pmi_inet_listen_socket_setup:socket setup failed
> [PE_3344]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
> [PE_208]:inet_listen_socket_setup:inet_setup_listen_socket: bind
> failed port 1371 listen_sock = 74 Address already in use
> [PE_208]:_pmi_inet_listen_socket_setup:socket setup failed
> [PE_208]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
> Thanks and best regards,
More information about the Darshan-users