[Darshan-users] trace files can not be created with Darshan

[Carns, Philip H.]

Hi Ayse,

I assume you aren't getting any stderr messages from Darshan in your job output, right?

I think maybe the LD_PRELOAD line needs an "export" to make sure it carries through to child processes?  It may also be the case that your mpirun needs an explicit argument to carry it along to the compute node environment.  I'm not sure if the default pulls the whole environment or not (and maybe it depends on the underlying launcher). It looks like mpirun has a -x argument for this purpose if you need to force it.

https://www.open-mpi.org/doc/v4.0/man1/mpirun.1.php

Another option if you are willing to rebuild Darshan is to try the new 3.2.0 release.  With that release there is another possibility for instrumentation: don't set LD_PRELOAD at all, but instead add `darshan-config --dyn-ld-flags` to your link options when compiling the application.  That one may be easier to debug because you could inspect if Darshan is linked in with tools like ldd and nm.

(technically that *might* work with 3.1.7 as well, but in 3.2.0 we more thoroughly vetted that approach and documented it)

thanks,
-Phil
________________________________
From: Darshan-users <darshan-users-bounces at lists.mcs.anl.gov> on behalf of Ayşe Arslan <aysearslan2491 at gmail.com>
Sent: Tuesday, May 12, 2020 2:31 PM
To: darshan-users at lists.mcs.anl.gov <darshan-users at lists.mcs.anl.gov>
Subject: [Darshan-users] trace files can not be created with Darshan

Hello,

I can not create trace files under darshan log directory,

I built in that way,
----------------------------------------
 DARSHAN_LOGPATH=/lustre/nec/ws2/ws/tuner/logs
 ./configure --with-mem-align=8 --with-log-path-by-env=DARSHAN_LOGPATH --with-jobid-env=NONE CC=mpicc --prefix=/lustre/nec/ws2/ws/tuner/darshan-3.1.7-ompi
make
make install
---------------------------------------
#! /bin/bash

#PBS -N a
#PBS -l nodes=1:ppn=24
#PBS -l walltime=00:40:00
#PBS -o a.txt
cd $PBS_O_WORKDIR

module load compiler/gnu/8.3.0
module load mpi/openmpi/4.0.1-gnu-8.3.0
module load tools/hdf5/1.10.5-openmpi-4.0.1-gnu-8.3.0

export DARSHAN_LOGPATH=/lustre/nec/ws2/ws/tuner/logs

cd /lustre/nec/ws2/tuner/benchmarks/Generators/mkTcTS

LD_PRELOAD=/lustre/nec/ws2/ws/tuner/darshan-3.1.7-ompi/lib/libdarshan.so
mpirun -n 24 ../../../src/MarDyn config.xml
-------------------------------------------------------------
I have 2020/5/12 folders under log directory, configuration was successful, I got no error after running script, but there is no trace file (.darshan files)

Any idea?

Thanks,
Ayse


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20200512/06bcdb48/attachment.html>