[Darshan-users] Profiling Python C/C++ extensions with Darshan
harms at alcf.anl.gov
Mon Feb 27 13:18:41 CST 2017
I have not tried it myself but it should be possible. For your python application, are you using the Cray provided MPI library? Are you using the cluster compatibility mode (CCM)?
From: <darshan-users-bounces at lists.mcs.anl.gov> on behalf of Ivan Raikov <ivan.g.raikov at gmail.com>
Date: Monday, February 27, 2017 at 1:09 PM
To: "darshan-users at lists.mcs.anl.gov" <darshan-users at lists.mcs.anl.gov>
Subject: [Darshan-users] Profiling Python C/C++ extensions with Darshan
I have a Python extension written in C++ which calls the HDF library,
and I would like to use Darshan to profile an MPI-based Python
application which invokes that extension. However, compiling the
extension and running the application on Blue Waters (Cray-based
system) with Darshan 2.3 results in no profile data being generated.
Using LD_PRELOAD to load Darshan dynamically results in some kind of
socket error when (I think) the MPI process manager is launching the
Python executable. I have also tried the LD_PRELOAD mechanism with
Darshan 3.1.3 built by the HDF group, but I get the same error. I was
wondering if anyone here has done profiling with dynamically linked
Python extensions, or if anyone has any advice how to approach this.
I invoke the application with LD_PRELOAD thus:
aprun -n 4096 python main.py ...
aprun -n 4096 -e LD_PRELOAD=/path/to/darshan/3.1.3/.../lib/libdarshan.so
The error I get with LD_PRELOAD is the following:
failed port 1371 listen_sock = 88 Address already in use
[PE_3344]:_pmi_inet_listen_socket_setup:socket setup failed
[PE_3344]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
failed port 1371 listen_sock = 74 Address already in use
[PE_208]:_pmi_inet_listen_socket_setup:socket setup failed
[PE_208]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
Thanks and best regards,
Darshan-users mailing list
Darshan-users at lists.mcs.anl.gov
More information about the Darshan-users