[Darshan-users] Darshan not creating trace file and not complaining

[Shane Snyder]

Hi Wadud,

It looks like you are trying to run one of the NAS benchmark codes, 
which I believe are Fortran programs, right?

This is due to a known "issue" with OpenMPI2 where the Fortran bindings 
are calling PMPI_ routines directly, bypassing Darshan's use of the MPI 
profiling interface (PMPI). Basically, Darshan is unable to intercept 
any MPI calls, including MPI_Init/Finalize which is where Darshan 
initializes its data structures and does the generation of the log file, 
respectively. No error is generated for the same reason: Darshan never 
initializes itself and doesn't think it should be instrumenting 
anything. If interested in more details, check out this github issue: 
https://github.com/open-mpi/ompi/issues/3954

Ignoring the philosophical question of whether the "issue" here lies 
with the MPI implementation or tools that use the MPI profiling 
interface (this issue affects all PMPI tools, not just Darshan), we've 
actually spent some time developing a workaround for this issue 
recently, so you are in luck!

I have not gone through with making a pre-release version of Darshan 
with this workaround publicly available, though I really have been 
meaning to. I will definitely make sure to do that today, and will send 
an update to this mailing list when its ready so you (and others) are 
aware. So, you have a couple of options:

1.) Wait for that pre-release to come out later this afternoon, or
2.) Check out the master branch of the Darshan git repo 
(https://xgitlab.cels.anl.gov/darshan/darshan)

Either should resolve the issue, but be sure to let us know if not.

Thanks,
--Shane

On 10/31/2017 10:51 AM, Wadud Miah wrote:
>
> Hi everyone,
>
>
> I built Darshan 3.1.4 with OpenMPI 2.0.2 and gcc 7.2.1 using the 
> following configure flags:
>
>
> ./configure --with-mem-align=8 --with-log-path-by-env=DARSHAN_LOGPATH 
> --with-jobid-env=NONE CC=mpicc --prefix=/usr/local/darshan-3.1.4-ompi
>
>
> I have also attached the config.log file. When I run my application 
> with LD_PRELOAD, Darshan does not create any trace file and neither 
> does it complain:
>
>
> [miahw at bedale bin]$ export DARSHAN_LOGPATH=.
> [miahw at bedale bin]$ 
> LD_PRELOAD=/usr/local/darshan-3.1.4-ompi/lib/libdarshan.so mpirun -n 4 
> ./bt.A.4.mpi_io_full
> [ ... lots of application output ... ]
> [miahw at bedale bin]$ ls
> bt.A.4  bt.A.4.mpi_io_full  btio.full.out  inputbt.data
>
> As you can see, there is no trace file in the current working 
> directory. Any help will be greatly appreciated.
>
> Thanks in advance,
> Wadud.
>
>
> ---------------------------------------
>
> Dr. Wadud Miah
>
> Numerical Algorithms Group
>
>
>
>
>
> *Disclaimer*
>
> The Numerical Algorithms Group Ltd is a company registered in England 
> and Wales with company number 1249803. The registered office is: 
> Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.
>
> This e-mail has been scanned for all viruses and malware, and may have 
> been automatically archived by Mimecast Ltd, an innovator in Software 
> as a Service (SaaS) for business.
>
>
>
> _______________________________________________
> Darshan-users mailing list
> Darshan-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20171031/21791fa4/attachment.html>