[Darshan-users] profiling DASK application using darshan

[Sandra A. Mendez]

Hi Razvan,
I have traced a python program by using Darshan. You only need to set up
the DARSHAN_ENABLE_NONMPI variable as follows:

export DARSHAN_ENABLE_NONMPI=1

Before running your application. Only a comment, the maximum number of
files to trace by Darshan is 1024 files per parallel task. In case of
python applications, Darshan traces all the files opened not only the
application files (but also the files opened by the python libraries). So
take care if your application overcomes that limit.
Thanks,
Sandra.-


On Sun, 18 Oct 2020 at 15:09, Razvan Stefanescu <razvan.stefanescu at spire.com>
wrote:

> Hello All,
>
> Following the documentation note saying that Darshan instrumentation of
> non-MPI applications is only possible with dynamically-linked applications,
> it seems I have to convert the multiprocessing and multithreading DASK code
> to a dynamically-linked code. I know some folks were able to profile Spark
> and TensorFlow codes with Darshan, so I wonder if you could provide some
> suggestions about creating such dynamically-linked code.
>
> Thank you,
>
> Razvan
>
> --
> *RAZVAN STEFANESCU *
> Head of Statistics and Machine Learning Branch
>
> Senior Data Assimilation and Data Scientist
>
> *Spire Global, Inc.*
>
> 1050 Walnut Street, Suite 402, Boulder, CO 80302 USA
>
> +1-720- <7206432245>*643-2231*
> +1-850-443-1718 <4178491948>
> _______________________________________________
> Darshan-users mailing list
> Darshan-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20201018/507f9d04/attachment.html>