[Darshan-users] Profiling Python C/C++ extensions with Darshan

Harms, Kevin harms at alcf.anl.gov
Mon Feb 27 13:18:41 CST 2017


  I have not tried it myself but it should be possible. For your python application, are you using the Cray provided MPI library? Are you using the cluster compatibility mode (CCM)?


-----Original Message-----
From: <darshan-users-bounces at lists.mcs.anl.gov> on behalf of Ivan Raikov <ivan.g.raikov at gmail.com>
Date: Monday, February 27, 2017 at 1:09 PM
To: "darshan-users at lists.mcs.anl.gov" <darshan-users at lists.mcs.anl.gov>
Subject: [Darshan-users] Profiling Python C/C++ extensions with Darshan

    I have a Python extension written in C++ which calls the HDF library,
    and I would like to use Darshan to profile an MPI-based Python
    application which invokes that extension. However, compiling the
    extension and running the application on Blue Waters (Cray-based
    system) with Darshan 2.3 results in no profile data being generated.
    Using LD_PRELOAD to load Darshan dynamically results in some kind of
    socket error when (I think) the MPI process manager is launching the
    Python executable. I have also tried the LD_PRELOAD mechanism with
    Darshan 3.1.3 built by the HDF group, but I  get the same error. I was
    wondering if anyone here has done profiling with dynamically linked
    Python extensions, or if anyone has any advice how to approach this.
    I invoke the application with LD_PRELOAD thus:
    export LD_PRELOAD=/path/to/darshan/3.1.3/lib/libdarshan.so
    aprun -n 4096 python main.py ...
    aprun -n 4096 -e LD_PRELOAD=/path/to/darshan/3.1.3/.../lib/libdarshan.so
    The error I get with LD_PRELOAD is the following:
    [PE_3344]:inet_listen_socket_setup:inet_setup_listen_socket: bind
    failed port 1371 listen_sock = 88 Address already in use
    [PE_3344]:_pmi_inet_listen_socket_setup:socket setup failed
    [PE_3344]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
    [PE_208]:inet_listen_socket_setup:inet_setup_listen_socket: bind
    failed port 1371 listen_sock = 74 Address already in use
    [PE_208]:_pmi_inet_listen_socket_setup:socket setup failed
    [PE_208]:_pmi_init:_pmi_inet_listen_socket_setup (full) returned -1
    Thanks and best regards,
    Darshan-users mailing list
    Darshan-users at lists.mcs.anl.gov

More information about the Darshan-users mailing list