[Darshan-users] darshan-parser output in json format

[George Papadimitriou]

Hi Shane,


I will take a look at the script. This seems to be helpful !!!


Thanks,

George

________________________________
From: Shane Snyder <ssnyder at mcs.anl.gov>
Sent: Wednesday, August 22, 2018 10:55:16 AM
To: George Papadimitriou; Harms, Kevin; darshan-users at lists.mcs.anl.gov
Subject: Re: [Darshan-users] darshan-parser output in json format

Hi George,

Adding to Kevin's comment -- we have found the ability to extract Darshan parser output in JSON format to be valuable in a couple of other external projects too, but it is not currently on any development roadmap for integration into Darshan proper. It is something we can consider in the future though, especially since we are working on other features that will probably make this easier (namely, implementing a Python interface to the darshan-utils library).

If it's useful in the meantime, I can point you at code from one of our projects (TOKIO) that implements a JSON parser for Darshan logs. TOKIO is a broader project concerned with holistic I/O analysis of HPC systems, and as part of it we have implemented a number of different Python tools for analyzing different I/O logs (including Darshan), all in the pytokio repo: https://github.com/NERSC/pytokio.

'bin/cache_darshan.py' is the script that can take a Darshan log file and spit out JSON data for all of the counters, metadata, etc.

Thanks,
--Shane

On 08/21/2018 06:16 PM, George Papadimitriou wrote:

Hi Kevin,


This is what we are currently doing. But I've found that parsing the ascii output can be a bit tedious and not an elegant solution to the problem.


George

________________________________
From: Harms, Kevin <harms at alcf.anl.gov><mailto:harms at alcf.anl.gov>
Sent: Tuesday, August 21, 2018 3:52:31 PM
To: George Papadimitriou; darshan-users at lists.mcs.anl.gov<mailto:darshan-users at lists.mcs.anl.gov>
Subject: Re: darshan-parser output in json format


  I don't think we have a plan for it right now. I think the easiest method would be to take the ASCII output and convert it to JSON using a separate tool.

kevin

________________________________________
From: Darshan-users <darshan-users-bounces at lists.mcs.anl.gov><mailto:darshan-users-bounces at lists.mcs.anl.gov> on behalf of George Papadimitriou <georgpap at isi.edu><mailto:georgpap at isi.edu>
Sent: Tuesday, August 21, 2018 12:13:42 PM
To: darshan-users at lists.mcs.anl.gov<mailto:darshan-users at lists.mcs.anl.gov>
Subject: [Darshan-users] darshan-parser output in json format

Hello,


I'm aware of an open ticket (https://xgitlab.cels.anl.gov/darshan/darshan/issues/242) about having an option in darshan-parser to enable json output.

Is this feature on darshan-parser's roadmap ?


Thanks,

George




_______________________________________________
Darshan-users mailing list
Darshan-users at lists.mcs.anl.gov<mailto:Darshan-users at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/darshan-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20180822/8ee1983b/attachment-0001.html>