<html lang='en'>

<head>

<meta content='text/html; charset=utf-8' http-equiv='Content-Type'>

<title>

GitLab

</title>

</meta>

</head>

<style>

  img {

    max-width: 100%;

    height: auto;

  }

  p.details {

    font-style:italic;

    color:#777

  }

  .footer p {

    font-size:small;

    color:#777

  }

  pre.commit-message {

    white-space: pre-wrap;

  }

  .file-stats a {

    text-decoration: none;

  }

  .file-stats .new-file {

    color: #090;

  }

  .file-stats .deleted-file {

    color: #B00;

  }

</style>

<body>

<div class='content'>

<h3>

Shane Snyder pushed to branch master

at <a href="https://xgitlab.cels.anl.gov/darshan/darshan">darshan / darshan</a>

</h3>

<h4>

Commits:

</h4>

<ul>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9879059d25f881af71d2c144c0a8a677dbc3deb2">9879059d</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2014-09-02T16:16:57-04:00</i>

</div>

<pre class='commit-message'>bump version to 3.0.0-pre1 in dev-modular branch</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/69611d265d9f5c533f411e42e6238515d7fc2cd9">69611d26</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2014-09-02T16:20:28-04:00</i>

</div>

<pre class='commit-message'>whitespace change for testing</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2db1f1ebfa37b028ea236ebd1dbc7d617609ce60">2db1f1eb</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2014-09-02T16:26:54-04:00</i>

</div>

<pre class='commit-message'>whitespace change for testing</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/aa56e9028038ce4c0c807bee4b0bcdd1797bb496">aa56e902</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2014-09-02T17:06:18-04:00</i>

</div>

<pre class='commit-message'>Merge remote-tracking branch 'origin/master' into dev-modular</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e60076aee7f5ebe7d7a0303c880a6cb61c8e5fcd">e60076ae</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2014-09-25T10:34:41-05:00</i>

</div>

<pre class='commit-message'>add whiteboard notes and text notes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/66a491402851fa417e01c8e571ddf68fe06ee5e9">66a49140</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-09-29T10:27:46-05:00</i>

</div>

<pre class='commit-message'>First commit for modularing darshan runtime

New darshan-core source files which intercept MPI init/finalize,

initialize darshan job information, and first attempt at a POSIX

module.</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/bd6ef3089a15eea6df6a0e1af9d85104aec27ec0">bd6ef308</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-09-29T14:09:50-05:00</i>

</div>

<pre class='commit-message'>Update with some TODO notes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0843817664ce9c4648fcbb63bed987f0a6aa7359">08438176</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-09-29T16:20:48-05:00</i>

</div>

<pre class='commit-message'>slight update to code structure/naming</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f9e5a24e9b089cd60ae3e9bd330d702c05a7884c">f9e5a24e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-09T13:14:15-05:00</i>

</div>

<pre class='commit-message'>updated code for darshan modularization</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/29cc3cd6b4a15555a3b5ad7c5d229594d420e64d">29cc3cd6</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-28T17:41:29-05:00</i>

</div>

<pre class='commit-message'>further refactoring of darshan code base</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/265328856281b37ae5c368db8be346239bb0fe55">26532885</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-30T13:30:29-05:00</i>

</div>

<pre class='commit-message'>simplified posix module and added core lookup</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b1ae796cb558e93b5de96bd3d46077319a4acac9">b1ae796c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-30T14:24:33-05:00</i>

</div>

<pre class='commit-message'>Convert posix module to use uthash</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/40c8ed2eb34f503dcfcc184890854412ef2b24ad">40c8ed2e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-31T12:22:35-05:00</i>

</div>

<pre class='commit-message'>updat todos, instrumentation for posix open/close</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0da31a588b438cc6905b94d6923ba90d51f3f964">0da31a58</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-10-31T16:05:27-05:00</i>

</div>

<pre class='commit-message'>initial darshan core shutdown logic

-also, darshan core uses utlist to keep track of modules</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c88a970e8fa565ca47228e25a4da2bdc104ead32">c88a970e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-11-04T15:36:42-06:00</i>

</div>

<pre class='commit-message'>Remove darshan-common from build temporarily</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/44a058c21369e5e069c87a243e5b59b7e47cbd1a">44a058c2</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-11-04T15:37:05-06:00</i>

</div>

<pre class='commit-message'>Initial darshan-core shutdown logic</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/37d68c1ea108118df1dfd259e96b7c6d33f32110">37d68c1e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-11-06T11:01:48-06:00</i>

</div>

<pre class='commit-message'>code cleanup and add darshan-common to build</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7496c279aeb8003a655ee581145ecec6b4d18ddc">7496c279</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-11-12T16:01:20-06:00</i>

</div>

<pre class='commit-message'>More updates to facilitate runtime core shutdown</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/da477e427153126228171de95ea8d9a3a78e0a1a">da477e42</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2014-11-20T15:41:50-06:00</i>

</div>

<pre class='commit-message'>Complete (first) implementation for darshan

-darshan-core:

    - initializes darshan job data structures

    - creates log file

    - writes job info header

    - reduces which modules were used locally/globally

    - shut down and write out individual modules one by one

-darshan-posix:

    - register w/ darshan-core and allocate allowed memory

    - track open counter and timestamp for opened files

    - provide file record buffer at shutdown

Major TODOs:

    - shared file reductions (prefer reusable across modules)

    - register handles with darshan file_ids (prefer reusable)

    - generic compression of module buffers (zlib/bzip2)

    - write out map of filenames to darshan ids

    - modify darshan-util and add darshan-posix parser to read logs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/abf424f2859b8ea43a4818a2c086531e39134dd4">abf424f2</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-01-16T11:54:03-06:00</i>

</div>

<pre class='commit-message'>More changes to support darshan_core shutdown</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/01d0a980101212371eb3fb6ccc76562c6f5eb857">01d0a980</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-01-22T10:57:50-06:00</i>

</div>

<pre class='commit-message'>Refactoring/bug fixes for darshan-runtime</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a0e8f8a87a9113013ae4460c32fa49b8257a0459">a0e8f8a8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-01-22T10:58:35-06:00</i>

</div>

<pre class='commit-message'>initial rewrite of logutils for new file format</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2936261cb3ad0146fac92846fff70e4c62a4fa75">2936261c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-06T14:16:24-06:00</i>

</div>

<pre class='commit-message'>more refactoring runtime side</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3d963df90711b53b02beeb649bfaf0a363d83328">3d963df9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-06T14:17:22-06:00</i>

</div>

<pre class='commit-message'>additions to log format</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7fdf26f08dfc4528bcf7397f3786ad96a2d6cbd2">7fdf26f0</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-06T14:17:51-06:00</i>

</div>

<pre class='commit-message'>First cut at logutils and posix parser</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/721e5e5ca6ca0e14a530cef57d9bca47dbc94bc4">721e5e5c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-09T14:49:37-06:00</i>

</div>

<pre class='commit-message'>some code/header reorg + minor edits</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5348600bd92eba9cf3d8935512b523a5232838f7">5348600b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-10T17:30:04-06:00</i>

</div>

<pre class='commit-message'>update runtime/util side with new index map def</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/018122117ee9f50abaa2b5fd73b7d3133c09373c">01812211</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-11T13:10:25-06:00</i>

</div>

<pre class='commit-message'>revamped error handling runtime side</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/756b640d97a13630c0e30e6b84b82e975e78d369">756b640d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-16T23:40:20-06:00</i>

</div>

<pre class='commit-message'>updated runtime/util to use exe/mount info</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/42a4b3b084b0aebf1a1a4b37d4b9b4270fd39967">42a4b3b0</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-17T21:13:30+00:00</i>

</div>

<pre class='commit-message'>darshan runtime makefile cleanup</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/bec14184625b9df85e56f60d8d3ca6b9feb4d013">bec14184</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-17T15:50:27-06:00</i>

</div>

<pre class='commit-message'>update configure to only wrap open for now</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7b412204c29e14502ee14a2f7f71b99f285c1fb9">7b412204</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-17T17:00:13-06:00</i>

</div>

<pre class='commit-message'>misc bug fixes and more documentation</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2de806226fd5c86dd9ecb4863054afb93e6cef55">2de80622</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-18T10:11:50-06:00</i>

</div>

<pre class='commit-message'>No longer store unique hash with mount info</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/601e5ef70169f6d00c271f53f2f0a8cd7fdfd86f">601e5ef7</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-18T10:13:41-06:00</i>

</div>

<pre class='commit-message'>more documentation for dev-modular</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/af20f42af9a38bf4d5258eb0bfa8ad2703158a3c">af20f42a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-18T11:33:21-06:00</i>

</div>

<pre class='commit-message'>Clean up variable names and source files</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/35f07b3803bd03fe2f7ec893c59b12970d700bbe">35f07b38</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-19T16:00:46-06:00</i>

</div>

<pre class='commit-message'>code cleanup / bug fixes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/efd500f045180f703b65ec537ed312c7bb13b40e">efd500f0</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-20T10:44:01-06:00</i>

</div>

<pre class='commit-message'>makefile bug in darshan-util</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8f85fd08a49f9c2862ae1636b3ed46f609806239">8f85fd08</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-20T16:56:59-06:00</i>

</div>

<pre class='commit-message'>log compression good to go, revamped logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/56b12ca2d8bcb72045dd70bdee2d5bf63d6857c9">56b12ca2</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-02-22T10:18:10-05:00</i>

</div>

<pre class='commit-message'>Merge remote-tracking branch 'origin/master' into dev-modular

Conflicts:

        ChangeLog

        darshan-log-format.h

        darshan-runtime/Makefile.in

        darshan-runtime/configure

        darshan-runtime/configure.in

        darshan-runtime/lib/darshan-mpi-io.c

        darshan-runtime/lib/darshan-posix.c

        darshan-util/configure

        darshan-util/configure.in

        darshan-util/darshan-logutils.c</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f999603f0a66c729d7a3163c3060a48012f4de55">f999603f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-02-22T23:00:29-06:00</i>

</div>

<pre class='commit-message'>darshan log decompression bug fix</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ddf51033c75168a7930d47e5a2353cf56aaef255">ddf51033</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-02T14:31:04-06:00</i>

</div>

<pre class='commit-message'>runtime support added for shared file reductions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e984a8b5ed52b9628040dcf28d33715241f235fb">e984a8b5</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-02T16:38:05-06:00</i>

</div>

<pre class='commit-message'>cleanup darshan runtime shutdown code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/cb17024a79ffee1c5521dabe1fcf67dc83ac2c26">cb17024a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-03T15:10:28-06:00</i>

</div>

<pre class='commit-message'>add core shutdown instrumentation</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1c3f36906e32f8acd563d09071a3b3e044ef7c81">1c3f3690</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-03T15:17:42-06:00</i>

</div>

<pre class='commit-message'>instrumentation bug fix</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/75eee95096b07874b7ad3e95be5c80eccad24d8f">75eee950</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-05T15:49:30-06:00</i>

</div>

<pre class='commit-message'>initial shared lib support + more TODOs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/acc05e4a78089a657525a8e6fc96e2e643aae45b">acc05e4a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-06T11:23:56-06:00</i>

</div>

<pre class='commit-message'>resolve minor TODO items</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/72380aad89dc2e3f215de57cbcfd880c79bbacb2">72380aad</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-12T17:53:46-05:00</i>

</div>

<pre class='commit-message'>more docs and added unregister functions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/35dcc69a6d66df480ad9b7ce81688f5b4297ea9f">35dcc69a</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-15T12:37:21-04:00</i>

</div>

<pre class='commit-message'>Merge remote-tracking branch 'origin/master' into dev-modular

Conflicts:

        ChangeLog

        darshan-runtime/Makefile.in

        darshan-runtime/configure

        darshan-runtime/configure.in

        darshan-util/configure

        darshan-util/configure.in</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/694c6a9ae8bf695a9a84e1e2c9f35535d1b6c9a7">694c6a9a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-17T15:09:11-05:00</i>

</div>

<pre class='commit-message'>bunch of bug fixes for gzip madness</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b2b9fcbef8fd7a0475eef712d0e96aac50d2c4c3">b2b9fcbe</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-17T17:27:39-05:00</i>

</div>

<pre class='commit-message'>forgot new util files</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/657e1b1a816abdb4ca7e2226a7c2573533ce0040">657e1b1a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-17T22:35:36-05:00</i>

</div>

<pre class='commit-message'>bug fix and makefile cleanup</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b4a0aed40c78cfa4904b3b0134defc8ed0c04181">b4a0aed4</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T12:53:31-05:00</i>

</div>

<pre class='commit-message'>first cut at complete dev-modular documentation</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e47a391ca48676c5a7e0f4392d023bab9d92fe12">e47a391c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T14:56:38-05:00</i>

</div>

<pre class='commit-message'>update docs to mention byte swap functionality</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f234f6623002245d75e3be3341fed24c1cbe88a9">f234f662</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T15:23:04-05:00</i>

</div>

<pre class='commit-message'>darshan-util out-of-tree build bug</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1fc4876f4ec4ca28f92b3f7ac3db92d5434180d3">1fc4876f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T16:22:49-05:00</i>

</div>

<pre class='commit-message'>no mpi counter check in regression tests for now</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b50b656dc36a3ff5102650d80ca923a8a91ec427">b50b656d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T16:44:25-05:00</i>

</div>

<pre class='commit-message'>update regr. testing to use darshan-posix-parser</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2a6ab02671323b9911f910ca3b257b2fc300fe94">2a6ab026</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-18T16:49:31-05:00</i>

</div>

<pre class='commit-message'>add cxx and fortran wrapper generators</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5e00d1cff81afd94d1705df7bb3a8d8effc4e368">5e00d1cf</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-23T12:45:08-05:00</i>

</div>

<pre class='commit-message'>bug in darshan init for fortran programs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/15289f25cfa2b3e6fe6f686f2ababedf34f34aab">15289f25</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-23T14:41:29-05:00</i>

</div>

<pre class='commit-message'>install mpi profiling config files</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5ae8039b11deea5fe88140bb9dba0c496521a62d">5ae8039b</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-24T20:17:59-04:00</i>

</div>

<pre class='commit-message'>stub in mpiio log format header</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/64d9f7962af76ea6782347d008f428bd92fd6fba">64d9f796</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-24T20:41:21-04:00</i>

</div>

<pre class='commit-message'>stub in mpiio module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/91a27d657ce9f8b2421762039d5308c4dbfbf1d2">91a27d65</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-24T20:41:52-04:00</i>

</div>

<pre class='commit-message'>comment out debug printf</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/43cfa9bf3b3c49cb32fd3cfba8a9b4324922eefe">43cfa9bf</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-24T20:55:56-04:00</i>

</div>

<pre class='commit-message'>stub in more mpiio module code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/165a27fc58a7791bf3cc0f6e9986a2ba9db311ab">165a27fc</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-03-24T20:59:19-04:00</i>

</div>

<pre class='commit-message'>call init fn</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0448ed3c6ad5cc81e5c7800d9b605690e560e2f9">0448ed3c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-25T09:21:26-05:00</i>

</div>

<pre class='commit-message'>code cleanup + posix read/write wrappers</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c8026d6b05d6c7548ba722a310b4a8ac9d739823">c8026d6b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-25T09:44:17-05:00</i>

</div>

<pre class='commit-message'>clean up compile warnings</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e644c976e99db1dffdd25548a24ca5fddb8bb3cf">e644c976</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-25T11:18:53-05:00</i>

</div>

<pre class='commit-message'>Updated regression testing README</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/969f0b8dc38ae7776b80843c8077028f1f47eb8e">969f0b8d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-25T11:19:06-05:00</i>

</div>

<pre class='commit-message'>Updated copyrights</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1a856178c65897ea196c36962133c2e1adf6cbba">1a856178</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-25T12:57:23-05:00</i>

</div>

<pre class='commit-message'>shared record reduction enhancements</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1592fd9f9c1e2a511fa9a4277f9a148d1b646688">1592fd9f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-26T10:48:20-05:00</i>

</div>

<pre class='commit-message'>modules now define ld options in file</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1110d3a68ddd47537460b6e07f32125188bf6391">1110d3a6</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-26T16:50:42-05:00</i>

</div>

<pre class='commit-message'>refactor exclusions list and macros out of modules</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/910d55f263a84e56b822e0c331f7f518fb832dee">910d55f2</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-27T12:11:35-05:00</i>

</div>

<pre class='commit-message'>makefile bug fix</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9801de536d56164d4df5c9a61002247d51366667">9801de53</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-30T13:45:50-05:00</i>

</div>

<pre class='commit-message'>more POSIX wrappers added and improved docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/dc00384c4238486f01236ddb7ea69cd86428b096">dc00384c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-30T16:21:17-05:00</i>

</div>

<pre class='commit-message'>bug fix in counter macros: forgot '+'</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/6af7ad398b4ba7ce8eb1c571cc00aeb20b6a4d1c">6af7ad39</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-31T13:16:28-05:00</i>

</div>

<pre class='commit-message'>more posix wrappers: sync*, seek*, & stream ops</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d72958deef26f9e8b81f83ecd8f0a627e20c5c9e">d72958de</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-03-31T17:05:04-05:00</i>

</div>

<pre class='commit-message'>more posix wrappers: stat family</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2f7e0da108c8fcb07b5246c79ead55e6e647ba25">2f7e0da1</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-01T09:52:06-05:00</i>

</div>

<pre class='commit-message'>extended comments on darshan_file_runtime struct</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/905e0ac4cccc3cee2bf5324b572799ce64a9c8ec">905e0ac4</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-01T09:54:54-05:00</i>

</div>

<pre class='commit-message'>extended comments on posix_file_runtime_ref</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8a1f77e3eb293eabe336194b279ee96c7130db6c">8a1f77e3</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-01T11:40:58-05:00</i>

</div>

<pre class='commit-message'>carry over file ref comments to mpiio mod</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1c701efff1a082d2bba582190da5ad9d2c5bbe86">1c701eff</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-01T11:44:55-05:00</i>

</div>

<pre class='commit-message'>mpi-io module works on file handles, not fds</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/6b4f41096ff819be3bbe77948b92f945f10a7ed2">6b4f4109</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-01T12:35:33-05:00</i>

</div>

<pre class='commit-message'>more posix wrappers: mmap</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c97d2f192f6c2e27bdd50f8fac007f5df4a07f90">c97d2f19</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-01T12:35:33-05:00</i>

</div>

<pre class='commit-message'>more posix wrappers: mkstemp family</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d9750b4949076c42d7adab1b556f1de416bfcbf9">d9750b49</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-01T17:20:08-05:00</i>

</div>

<pre class='commit-message'>ability to look up file records in mpiio mod</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/339ed2516fcee2d3bd1ae92bbe19804c73629f7d">339ed251</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-01T17:39:01-05:00</i>

</div>

<pre class='commit-message'>add support for slowest/fastest rank counters</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9ac0bd93aef0907e40850308c6467498f155cd03">9ac0bd93</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-01T17:39:01-05:00</i>

</div>

<pre class='commit-message'>depracate old counters & update mod_register</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/90c39ea6cdd1abebd83fe7916a8434c151e82a9e">90c39ea6</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-02T12:32:03-05:00</i>

</div>

<pre class='commit-message'>support for poisx mem and file alignment counters</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5c5918aa36ddb95896ed547b726ed1432e5ca794">5c5918aa</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-03T11:19:05-05:00</i>

</div>

<pre class='commit-message'>add LD_PRELOAD functionality</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/58ae2da29e36978ead086d05597df9c7dfbee614">58ae2da2</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T15:59:51-04:00</i>

</div>

<pre class='commit-message'>record open counters in mpiio module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b70229e4c9d745170efca03084c3b3b244792eb5">b70229e4</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T16:22:25-04:00</i>

</div>

<pre class='commit-message'>stub logutils for mpiio</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/13b92a1f8d0fc3f3e603424e1686cbbfab1b2282">13b92a1f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-06T15:36:25-05:00</i>

</div>

<pre class='commit-message'>adding posix common access/stride counters</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1e5a8cb74811c8d5beb93a3409617b3f5466c3f2">1e5a8cb7</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T17:03:14-04:00</i>

</div>

<pre class='commit-message'>add mpiio parser</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/37d5884986e9d072d61cb94bef8885f2482019b0">37d58849</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T17:09:29-04:00</i>

</div>

<pre class='commit-message'>remove old code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f18d0c3f6b375e058d404f686f4f76324c6f805a">f18d0c3f</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T17:12:24-04:00</i>

</div>

<pre class='commit-message'>fix missing rank in mpiio module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/cc000cb3d4f478a614c4115c193fc10ff0e83b59">cc000cb3</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T17:26:53-04:00</i>

</div>

<pre class='commit-message'>stub in reduction op for mpiio</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/db7b2f7b684d83d3f24cbca82f2dbeff9feb9b9d">db7b2f7b</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-06T17:34:18-04:00</i>

</div>

<pre class='commit-message'>stub in prepare for reduction

- doesn't work yet</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3fae94044d78b6954d1a4e574eb6ff02d533fd1a">3fae9404</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-06T16:34:37-05:00</i>

</div>

<pre class='commit-message'>darshan-base-parser added</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5a14a88b6155a9af96a8e86da50b420f12f076ab">5a14a88b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-07T11:02:25-05:00</i>

</div>

<pre class='commit-message'>shared file reduction bug fix</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/96df671a91a1f0e0dd867943da1fc39cdc002249">96df671a</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-07T12:46:41-04:00</i>

</div>

<pre class='commit-message'>check mpi counters in mpi-io-test regression</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/233f1607498e8197f127a5b2712f1593e752ae28">233f1607</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-04-07T12:50:07-04:00</i>

</div>

<pre class='commit-message'>mpi counter checks for fperf regression</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/5ecf4013b55c356441590ce97e4ef29cf03d3a1f">5ecf4013</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-08T16:00:43-05:00</i>

</div>

<pre class='commit-message'>more docs + "NULL" example module implementation</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/451114027638f9dac2a2a3c5282362dc26055539">45111402</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-09T13:17:45-05:00</i>

</div>

<pre class='commit-message'>updated dev-modular docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/6ef99ac9fec17bebfdabe8f4b47be25532e410fe">6ef99ac9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-09T15:31:51-05:00</i>

</div>

<pre class='commit-message'>revamp docs, including runtime block diagram</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3a01d053fa7a64b2a60c500e50987b52813f7658">3a01d053</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-15T10:04:45-05:00</i>

</div>

<pre class='commit-message'>Some dev-modular docs edits</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c3f4cafd7cbb8a4d6bf044de9660fc95be93c2e7">c3f4cafd</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-04-28T15:26:50-05:00</i>

</div>

<pre class='commit-message'>Initial commit for bgq module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/347a0744d26f300de58cd784556ee5ff67da39a9">347a0744</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-04-28T17:00:40-05:00</i>

</div>

<pre class='commit-message'>small edits to posix mod and docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/faa3101fd3c8aac2d7f3b51106b3a84f96a6b6f8">faa3101f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-06-29T14:39:19-05:00</i>

</div>

<pre class='commit-message'>X-macro mod identifiers for consistent ordering</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7dc249cbc6ed3f9ef7263270443067a744d5111f">7dc249cb</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-06-29T22:40:21+00:00</i>

</div>

<pre class='commit-message'>Initial parser code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/94efcc1729a753e7cdb2894f8219262c67ba3961">94efcc17</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-06-29T22:41:43+00:00</i>

</div>

<pre class='commit-message'>Updates to bgq module core</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/28f19402834f8b016441741b7dd87c0b4859e9f7">28f19402</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-06-30T22:27:40-05:00</i>

</div>

<pre class='commit-message'>add aio and lio wrappers to posix module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/628724ba033d37cad799ef52d95cbe74d2274776">628724ba</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-01T15:56:41-05:00</i>

</div>

<pre class='commit-message'>clean up & reorganize mpiio module source file</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f1ad34e8cf3f2aff0e12f83cd85e9f95726bd9e6">f1ad34e8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-02T15:44:03-05:00</i>

</div>

<pre class='commit-message'>add mpiio wrappers for close,sync,set_view</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c572aa446c79a9192b8d2c30d33485da8c6f4ef2">c572aa44</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-02T17:29:41-05:00</i>

</div>

<pre class='commit-message'>add mpiio r/w wrappers, remove combiner counters</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/53fa81e8bd7449c9caf48f78fb64e55829b9aa1d">53fa81e8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-07T16:42:13-05:00</i>

</div>

<pre class='commit-message'>add the remaining read/write wrappers to mpiio</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d2776592214b92e72cac8c3903823271ef541ba8">d2776592</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-14T18:52:10-05:00</i>

</div>

<pre class='commit-message'>more mpiio counters</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e0eb27be2b7827593b442e104af8b45cb585e876">e0eb27be</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-14T18:52:41-05:00</i>

</div>

<pre class='commit-message'>xmacro posix/mpiio counters to maintain ordering</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/6406c8726d1fe664f9de8147248da121c642ec6a">6406c872</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-20T11:43:42-05:00</i>

</div>

<pre class='commit-message'>header file cleanups & resolve some small todos</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/735415387319aa6e406800952327bf3aeef7e12c">73541538</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-21T13:44:08-05:00</i>

</div>

<pre class='commit-message'>refactor and add common access counts to mpiio</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a708b0ce36c2bc7fd25b44d1756624129bb870e0">a708b0ce</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-21T16:14:31-05:00</i>

</div>

<pre class='commit-message'>add new counters to mpiio reduction code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f4df41a355a7327706ed3a9fc8d65a0ab4496c0b">f4df41a3</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-21T16:42:41-05:00</i>

</div>

<pre class='commit-message'>add slowest/fastest rank counters to mpiio</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4e6d0c89355ee38b733e83d594b77590f379ecde">4e6d0c89</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-22T10:32:40-05:00</i>

</div>

<pre class='commit-message'>move shared file redux code to get_output func</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/aa46bf39a7c294be76267303f98f8d34a5da3975">aa46bf39</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-22T15:02:40-05:00</i>

</div>

<pre class='commit-message'>add time/size variance reductions to mpiio/posix</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a0b3f9cc1820fa2e7828ad0919886e9739e03187">a0b3f9cc</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-23T14:26:02-05:00</i>

</div>

<pre class='commit-message'>bug fix, add mpi dynamic headers to posix mod</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8eea697a4c9e6eaca6cc6021b29521bf23e9c3d3">8eea697a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-07-24T16:09:33-05:00</i>

</div>

<pre class='commit-message'>refactor logutil code, create base darshan-parser</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f1468f4943f8bba74afbd2a3f84ce6fc0071a556">f1468f49</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-04T10:13:23-05:00</i>

</div>

<pre class='commit-message'>stubbed out version of hdf5 module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/dafe379082b24c552f1285c30a3fcf4edd8db8de">dafe3790</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-04T13:41:29-05:00</i>

</div>

<pre class='commit-message'>fill in the hdf5 module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/019fba2263b7ab66fecde7298f4c6dd84a4e1e2f">019fba22</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-04T13:42:00-05:00</i>

</div>

<pre class='commit-message'>tiny misc. edits</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b6807fcc3b14ebb5e1323e219a72ddbfde2af64e">b6807fcc</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-04T14:31:56-05:00</i>

</div>

<pre class='commit-message'>forgot hdf5 log format header</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/158337c98e18596458cf2b3102c3f4a63c2ebfae">158337c9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-04T16:51:02-05:00</i>

</div>

<pre class='commit-message'>add code for the pnetcdf module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/328425b72ce7b211f940024719aa9aca7f4dbc38">328425b7</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-05T15:05:52-05:00</i>

</div>

<pre class='commit-message'>add hdf5 and pnetcdf stubs library</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/659b579db8c0086e707570feef45c89211fc5d16">659b579d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-05T17:48:12-05:00</i>

</div>

<pre class='commit-message'>add necessary ld options for hdf5/pnetcdf</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3a362ed5a6ae8e08f3e2edbdb5c9360e42940e12">3a362ed5</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-05T17:54:17-05:00</i>

</div>

<pre class='commit-message'>remove old parser tool</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3b0dacf7e5b8066fc7d911ed9e96f2f319da479e">3b0dacf7</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-06T09:51:54-05:00</i>

</div>

<pre class='commit-message'>logutil definitions for hdf5 module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f20198469abb49a27eef6a0d88e7fe8022266ac3">f2019846</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-06T11:48:44-05:00</i>

</div>

<pre class='commit-message'>logutil definitions for pnetcdf module</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a2ef6c6965fc2df514bd89728f69b2f205ec615c">a2ef6c69</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-06T13:14:39-05:00</i>

</div>

<pre class='commit-message'>bug fix in finding shared file records</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/6959ca27817bb471896df14308f5605b732bc11f">6959ca27</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-07T10:27:19-05:00</i>

</div>

<pre class='commit-message'>darshan-util makefile cleanup</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/30336460ceb8becd0dd219a2631f647b9f65ff6f">30336460</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-07T14:45:35-05:00</i>

</div>

<pre class='commit-message'>small fix for module logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4f5d233935e74193e7298efe688bb03169693457">4f5d2339</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-07T14:54:16-05:00</i>

</div>

<pre class='commit-message'>more makefile edits</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0d7fe6935fdccba8d604962228b5fdb0688f2656">0d7fe693</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-11T16:59:51-05:00</i>

</div>

<pre class='commit-message'>finish implementation of darshan logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b2268e7b10a7494ea7df2cfaae622f628bdf701a">b2268e7b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-12T15:54:00-05:00</i>

</div>

<pre class='commit-message'>log file format changes

* header is now raw uncompressed, rather than gzip format

* mapping info stored in header is now in uncompressed terms

* storing mapping info for job data now</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9f202697a19e98ae8822037260f1ec342033d8f4">9f202697</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-13T15:59:42-05:00</i>

</div>

<pre class='commit-message'>update logutils to use new compression format</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9e716a5770285d595ca6961ccf418242972c9a09">9e716a57</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-13T16:16:52-05:00</i>

</div>

<pre class='commit-message'>skip building analyzer/converter for now</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/718c949783d8f39699000b2002e884a9f2d2ac35">718c9497</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-13T16:23:16-05:00</i>

</div>

<pre class='commit-message'>forgot to update make install target</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/baa3f7bca57ca182c15462528f6de0a4776f8906">baa3f7bc</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-13T16:53:45-05:00</i>

</div>

<pre class='commit-message'>update regression tests to reflect recent updates</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/53c9e15b4b2df3b67e99ee61ace8b456e86a931f">53c9e15b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T14:58:43-05:00</i>

</div>

<pre class='commit-message'>change runtime to only write minimal job data</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d8b6e0dc9610db8dcec6a4dce64b12d906526239">d8b6e0dc</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T15:21:37-05:00</i>

</div>

<pre class='commit-message'>update darshan log header to show compression type</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/093b9f22baae4320fe61e8903b721d29e7a8feed">093b9f22</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T15:31:13-05:00</i>

</div>

<pre class='commit-message'>update log put functions for new log format</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d4db229887c798c5b3cb3d2f9ce7e6f531f9a050">d4db2298</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T15:31:44-05:00</i>

</div>

<pre class='commit-message'>update current utilities to build properly</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/eb0e5f5b95d4dda633ee97208b658e9d45a331df">eb0e5f5b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T21:18:36-05:00</i>

</div>

<pre class='commit-message'>log util bug fixes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/dcc9c7feafae1d129b5ce5e4d6db38b10344259a">dcc9c7fe</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-17T22:18:51-05:00</i>

</div>

<pre class='commit-message'>initial hooks for bzip2 compression</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a850ea7453abc28a3e46e554cd48ab49739ea156">a850ea74</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-18T14:39:20-05:00</i>

</div>

<pre class='commit-message'>add bzip2 comp/decomp functions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/a94832b33200c1dfbcbbc05a8be71a7219e30446">a94832b3</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-19T13:33:10-05:00</i>

</div>

<pre class='commit-message'>byte swap bug fix for reading the record hash</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e5bfa490dd6dd2d1318a352a20c34ca219951cf0">e5bfa490</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-19T15:33:07-05:00</i>

</div>

<pre class='commit-message'>slight api change for retrieving mod records</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/814b126671a39bcad4a3df69fc1273518f633168">814b1266</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-20T12:41:12-05:00</i>

</div>

<pre class='commit-message'>add macro to detect partial posix file recs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/88926470e0528fda0b182f247c0052c84276e3c0">88926470</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-21T15:08:24-05:00</i>

</div>

<pre class='commit-message'>fully implemented darshan-parser w/ all options</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8eb2f98d7ad53bfede836909ff9ba3956b2351a1">8eb2f98d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-21T16:46:02-05:00</i>

</div>

<pre class='commit-message'>small typo in posix macro</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/89a97a0f4de39f7ee149308af537390d5db0a8dc">89a97a0f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-24T12:49:46-05:00</i>

</div>

<pre class='commit-message'>darshan parser bug fixes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c39904ba0f3a690585b0d93b9cf902a425e3b5f6">c39904ba</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-24T16:03:23-05:00</i>

</div>

<pre class='commit-message'>fix a couple of memory leaks</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7f0cf8630fc86db60aaef68928ef9023a156bb36">7f0cf863</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T12:36:02-05:00</i>

</div>

<pre class='commit-message'>minor bug fixes in modput</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f42d933f17d04332649575a1206b2492c602d112">f42d933f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T12:37:15-05:00</i>

</div>

<pre class='commit-message'>add darshan-convert utility, hash gen exe</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/168fe00f798234d2da5bdd059939eb38f59f04c6">168fe00f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T14:10:56-05:00</i>

</div>

<pre class='commit-message'>port darshan-analyzer utility</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e5172a88ac2646efa6049bc41e2ccafe51ea33ab">e5172a88</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T14:17:39-05:00</i>

</div>

<pre class='commit-message'>remove load-mysql source</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/fab0224e40a04833418f345e4b391fa918156e44">fab0224e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T14:30:44-05:00</i>

</div>

<pre class='commit-message'>stubbed out version of job summary script</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/020d91b6f8d584236b1b8033afb724792b0e1014">020d91b6</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-08-25T22:36:44-05:00</i>

</div>

<pre class='commit-message'>bug fix in mpiio reduction code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/760e827e12c90f773ed0358a820e23cf45b432fe">760e827e</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-09-01T19:25:55+00:00</i>

</div>

<pre class='commit-message'>merge from dev-modular</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/835e28d98724d6685611fb5b485339aecd3417d0">835e28d9</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-09-02T01:44:21+00:00</i>

</div>

<pre class='commit-message'>Updates to align with latest modular code base</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7e0566b7f583000e7ae82a98b7a94a68c915dbd0">7e0566b7</a></strong>

<div>

<span>by Kevin Harms</span>

<i>at 2015-09-03T00:06:53+00:00</i>

</div>

<pre class='commit-message'>Bug fixes after testing</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/3e3d2df8f841083834789a2bc9dc78800d8b9823">3e3d2df8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-03T11:33:37-05:00</i>

</div>

<pre class='commit-message'>whole bunch of changes to job-summary scipts</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ac26379597807cf610c81a3945388d7f746f4c80">ac263795</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-03T16:36:06-05:00</i>

</div>

<pre class='commit-message'>Merge remote-tracking branch 'origin/dev-bgq-mod' into dev-modular</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/fc94307057726297593d0a5af8e5c4c8ebfc50f6">fc943070</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-04T14:56:10-05:00</i>

</div>

<pre class='commit-message'>final tweaks for darshan-job-summary graphs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/efd4d7e7bba6578d0d53e678dcceb8649cd038c9">efd4d7e7</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-09T21:33:59+00:00</i>

</div>

<pre class='commit-message'>add autoconf check for building bgq mod</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/10068d0076c8e93783ca2ebffcdd7725c4b4ce51">10068d00</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-09T22:42:52+00:00</i>

</div>

<pre class='commit-message'>only build bgq module if config check passes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/328f68bd9a8faa7128423f6589bb047d1ce4f6f8">328f68bd</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-10T03:43:45+00:00</i>

</div>

<pre class='commit-message'>allow modules to statically init</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/57cb76241543beb5fbbdd28b220940f068465a75">57cb7624</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-10T03:57:36+00:00</i>

</div>

<pre class='commit-message'>fix compiler warnings</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/22bc23f2782273ebe8c65eb559f67d9b4b8004e4">22bc23f2</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-10T09:56:08-05:00</i>

</div>

<pre class='commit-message'>drop CP_ prefixes in runtime code</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ae15d896720a92d229119944890da28a9f4a2038">ae15d896</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-10T15:17:08-05:00</i>

</div>

<pre class='commit-message'>cleanup runtime component code & docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/86d174406f55abe938044a28eca0ce5c52b33e04">86d17440</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-10T18:58:10-05:00</i>

</div>

<pre class='commit-message'>cleanup code in utility component</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8659e6a33068878916580d5f69bb1d19101d72c9">8659e6a3</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-18T15:36:02-05:00</i>

</div>

<pre class='commit-message'>new implementation of darshan-logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/cb808ec2ab449f156045621f0e3998a544387adc">cb808ec2</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-18T17:26:08-05:00</i>

</div>

<pre class='commit-message'>add bzip2 implementation to darshan-logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/38e0e4be33dc4a4761ed3915311043b71aedfbfb">38e0e4be</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-20T22:11:49-05:00</i>

</div>

<pre class='commit-message'>add include for PATH_MAX header</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/dbefada8fb0355f45b76124ae1513e375a059b71">dbefada8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-21T10:15:41-05:00</i>

</div>

<pre class='commit-message'>handle case of empty record map in log file</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/82b0a5951d254d389daac3fc7a077a37814e3e34">82b0a595</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-21T15:03:25-05:00</i>

</div>

<pre class='commit-message'>bug fix in obfuscating darshan file record names</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/200e9d71e04cd6bc361cff120f7eb26ef2321a04">200e9d71</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-21T15:04:06-05:00</i>

</div>

<pre class='commit-message'>add more base ld flags needed for cray systems</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/991aaa2fbf770a5e4c03848ce6a4e8aa7896f582">991aaa2f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-22T11:45:50-05:00</i>

</div>

<pre class='commit-message'>refactor file desciptor in logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/67b02ca5120f74d7a7134f2761bc4db0e6b2bead">67b02ca5</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-22T17:31:54-05:00</i>

</div>

<pre class='commit-message'>x-macro the logutil defs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/20ff15470f5264c9b8b3e02d86ac39030d2bea64">20ff1547</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-23T08:39:03-07:00</i>

</div>

<pre class='commit-message'>git configure working on cray</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/779162dfb241581839e0fdf32d0320c8546b8747">779162df</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-23T08:39:03-07:00</i>

</div>

<pre class='commit-message'>update runtime pkgconfig file</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/da0aaec641c2c4fa48d1079ff34679a3aec227ab">da0aaec6</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-23T13:45:10-05:00</i>

</div>

<pre class='commit-message'>update darshan-job-summary to be more clear</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/773a085a2c0623c0172e66db4658d4a8539ae311">773a085a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-24T09:39:32-05:00</i>

</div>

<pre class='commit-message'>clean up utilities output and minor bug fixes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/550977d1a68c464b873b2b6f1a321062c59abf71">550977d1</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-24T09:41:01-05:00</i>

</div>

<pre class='commit-message'>updated darshan-util documentation for dev-modular</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d0c50a3bf55e23d64e08fb37e53466e1e2c6ab63">d0c50a3b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-25T01:21:25+00:00</i>

</div>

<pre class='commit-message'>up maximum number of records to track</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/928fdff4f6dbc92f33988f09b93794d4e51c5dc8">928fdff4</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-25T15:02:25-05:00</i>

</div>

<pre class='commit-message'>bug fix for modules creating more than max records</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0459f8ffc08951d813fc320a581bdd5fe4b4da31">0459f8ff</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-25T15:34:13-05:00</i>

</div>

<pre class='commit-message'>add a partial flag to darshan's header

this indicates whether the log file is complete or only partially

written</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ca2023049f9caf1d5b3e57a6526c9f9f82f27597">ca202304</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-29T16:37:27-05:00</i>

</div>

<pre class='commit-message'>code cleanup + finalized dev-modular docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ac48a609de5402d84d461208453113bf7a631570">ac48a609</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-30T22:41:26-05:00</i>

</div>

<pre class='commit-message'>update darshan-util to recognize partial log files</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c53dd65a6b6b983c1d4cb65abb3d1fbc1001933e">c53dd65a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-09-30T22:54:45-05:00</i>

</div>

<pre class='commit-message'>add reduction to check for partial flag</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9274a0db254dad9177a80c1781259daab3b254cf">9274a0db</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-01T10:36:59-05:00</i>

</div>

<pre class='commit-message'>update ChangeLog for 3.0 pre-release</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f6fbcec40d2573e0c34f519c58460b0b4c611d47">f6fbcec4</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-01T11:19:41-05:00</i>

</div>

<pre class='commit-message'>Merge remote-tracking branch 'origin/master' into dev-modular

Conflicts:

        ChangeLog

        darshan-runtime/configure

        darshan-runtime/configure.in

        darshan-runtime/darshan.h

        darshan-runtime/lib/darshan-posix.c

        darshan-util/configure

        darshan-util/configure.in

        darshan-util/doc/darshan-util.txt</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/41575785a890ced023ae36e6781ab7bd63440274">41575785</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-01T11:46:02-05:00</i>

</div>

<pre class='commit-message'>small text changes in darshan-util docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1fae7e7c73e7d0da9d5b90317e4e7341d4cc1c6d">1fae7e7c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-06T16:09:09-05:00</i>

</div>

<pre class='commit-message'>small fixes for COPYRIGHT, ChangeLog, & comments</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4150a0f951a78a299cbfb777a2c4c561c1c4e655">4150a0f9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-12T12:03:16-05:00</i>

</div>

<pre class='commit-message'>update makefile to install uthash header</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/df02ace59f38349b7f49c67120b2fb7e2324dc36">df02ace5</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-12T16:44:25-05:00</i>

</div>

<pre class='commit-message'>make uthash header install in correct directory</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/320968568dddcd8f81c466c85c03046a08fbc710">32096856</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-13T09:50:02-05:00</i>

</div>

<pre class='commit-message'>make install updates to support external tools</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/48352d366d6530b22d3fc4a2821c8b7a7e3ea214">48352d36</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-10-29T16:13:28-05:00</i>

</div>

<pre class='commit-message'>make maximum runtime records configurable

use --with-max-records at configure time to change from default

of 2048 records</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/98c93e0f45ef35bfde0da5d2b143e0b0de9f6ce5">98c93e0f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-02T12:34:38-06:00</i>

</div>

<pre class='commit-message'>darshan-runtime commits for configurable mem

user can now give a config option or use a runtime environment

variable to control darshan's per module memory requirements.</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/40d1dc03b7e336f5127f70a68c3a8069fd92536c">40d1dc03</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-02T13:31:25-06:00</i>

</div>

<pre class='commit-message'>darshan-util updates to support partial logs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2950f613462a892c1847281e5ce9f4257c082d89">2950f613</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-02T16:45:56-06:00</i>

</div>

<pre class='commit-message'>fix random typos in comments</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/45eb36b1ff0dede4ad5d27d913a20dbfa3524119">45eb36b1</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-03T13:09:06-06:00</i>

</div>

<pre class='commit-message'>update darshan version in configure</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/8731ade9ebdf93664a369aee29a6af4e2bb3e02f">8731ade9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-03T13:09:40-06:00</i>

</div>

<pre class='commit-message'>update darshan modularization docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ba451e9ab313b395f05ba6f69b480ba1089b0008">ba451e9a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-24T22:40:51-06:00</i>

</div>

<pre class='commit-message'>rework header org. for module-specific versions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/66181457ee55eaac707fe6db89e11abbb9246828">66181457</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-25T10:18:12-06:00</i>

</div>

<pre class='commit-message'>modify runtime code to store each module's version</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/173d38b85c7fc065820f2f700c3c109b605e036d">173d38b8</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-11-25T12:41:33-06:00</i>

</div>

<pre class='commit-message'>update util interfaces to support mod-specific ver</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b07e6220550f2505612039c2e104a3048bbd7ef7">b07e6220</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-12-01T10:44:14-05:00</i>

</div>

<pre class='commit-message'>support MPICH 3.1.1 in profile conf test

fixes #174</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/26c4ff222cc88cc4fa526cc7a25d494454735975">26c4ff22</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-08T11:55:49-06:00</i>

</div>

<pre class='commit-message'>bug fix in bzip2 init in darshan logutils</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d097c5f10377b71caf2e45c955579d2360cc4535">d097c5f1</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-08T14:00:48-06:00</i>

</div>

<pre class='commit-message'>move compression buffer alloc to shutdown time</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c371cdea3257003237f12c63ff688c35934a3464">c371cdea</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-08T14:25:51-06:00</i>

</div>

<pre class='commit-message'>fix possible race cond in module locking</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1c1a9baad96fc76a1963de0b15006c7fe866d9a8">1c1a9baa</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-11T13:02:32-06:00</i>

</div>

<pre class='commit-message'>cleanup a bunch of code and stale comments</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0">31e0fb03</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-11T14:27:31-06:00</i>

</div>

<pre class='commit-message'>update runtime, util, and modularization docs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9b021c1bba1f7a7f7889333fa1f07cba5df159a4">9b021c1b</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-11T16:20:12-06:00</i>

</div>

<pre class='commit-message'>update modules to print counter descriptions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/cb6a25259eed40e2fbeb7b407dc33dded469d554">cb6a2525</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-11T16:30:55-06:00</i>

</div>

<pre class='commit-message'>updated changelog prior to pre3 release</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/34e447c0e7e192b24028ae91375af0151fed374c">34e447c0</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-11T16:34:42-06:00</i>

</div>

<pre class='commit-message'>update configure versions for pre3</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/be6d404d078744bbc73c5f83ddd4a42b898d6e4f">be6d404d</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-16T15:55:01-06:00</i>

</div>

<pre class='commit-message'>forgotten header install</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7d455c98345f27688d46b139907ffa8f53ec0f6d">7d455c98</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2015-12-17T17:07:03-06:00</i>

</div>

<pre class='commit-message'>back out commit for fixing races -- not needed</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/31079257caf824b5a78ac9daf4ba22d7d7ce6502">31079257</a></strong>

<div>

<span>by Phil Carns</span>

<i>at 2015-12-18T09:41:40-05:00</i>

</div>

<pre class='commit-message'>loosen PMPI symbol check

- prevents inadverdent disabling of Darshan on some MPICH builds

- fixes #94</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2feaff78045a9cc32af586c7358d353c25ff1fdf">2feaff78</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-04T09:19:52-06:00</i>

</div>

<pre class='commit-message'>update runtime docs to give info on upgrading</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/64978d11e35bbe4dc0cbf077115705a6179928ff">64978d11</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-04T09:50:41-06:00</i>

</div>

<pre class='commit-message'>bug fix for resolving mpi_gather in shared lib</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/86533a49dbde4a80f2d2c153d61580e03d6cd914">86533a49</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-04T12:46:33-06:00</i>

</div>

<pre class='commit-message'>fix typos in counter descriptions</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d4413b4e774ce98123c003419dec380c2b9bf629">d4413b4e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-04T13:36:29-06:00</i>

</div>

<pre class='commit-message'>better error handling for opening old log files</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/72937c9a125bcbb5e5c71f17e1fffb5e2f0d8b2b">72937c9a</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-04T15:05:02-06:00</i>

</div>

<pre class='commit-message'>more doc updates</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b5bba287d43d9e9bcad4afff4bd261a94fbcaa71">b5bba287</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-08T16:50:47-06:00</i>

</div>

<pre class='commit-message'>add the darshan-diff utility back</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4bbe8e640eeaea440a794de765a1b9fd54f8560d">4bbe8e64</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-08T17:07:01-06:00</i>

</div>

<pre class='commit-message'>add diff routines to remaining modules</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b770409e461781a9723c194b0c76e27620eefa79">b770409e</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-13T13:29:12-06:00</i>

</div>

<pre class='commit-message'>minor bug fix in darshan-diff tool</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/92ef4cdd30b0bee6c27a193821dbdf5881d35d31">92ef4cdd</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-20T11:08:32-08:00</i>

</div>

<pre class='commit-message'>fix some small configure bugs for cray systems</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d7c2e81c1bd8f5c84bc564d9b61a1da95fa4a8e1">d7c2e81c</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-21T16:15:41-06:00</i>

</div>

<pre class='commit-message'>small darshan-runtime documentation changes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2e9124099ae0471b9953f3c1ab27e30591823c5b">2e912409</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-27T16:15:30-06:00</i>

</div>

<pre class='commit-message'>add benchmarking hooks for generating logs</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f0278cc11ae5ba716f2c464f1cbe525b60520e97">f0278cc1</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-28T13:06:42-06:00</i>

</div>

<pre class='commit-message'>bug fix to unregister modules when shutting down</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b6ba418f51d89b5a016f727126e0bf5ea93a2e91">b6ba418f</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-28T11:14:09-08:00</i>

</div>

<pre class='commit-message'>bug fix for resolving mpi_barrier for shared exes</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e37c25dfadee839095feb7d6780f0beffcf27608">e37c25df</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-01-29T10:16:38-06:00</i>

</div>

<pre class='commit-message'>back out unregister of modules at shutdown</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/75bd30201d6b1cb374508fc4ccd7c87ce1a757e5">75bd3020</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-04T17:57:12-06:00</i>

</div>

<pre class='commit-message'>small reorg of darshan init code

this reorganization prevents darshan from being "enabled"

(i.e., accessible to modules) until the initialization

has completely finished.</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/48fd8632c311a5a57d92e50b000377f871928737">48fd8632</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-10T11:24:37-06:00</i>

</div>

<pre class='commit-message'>reorganize ld-opts and fix cray/fortran bug</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/18fa80f179c13e1d596b1d1a7352d73eeb66af88">18fa80f1</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-11T08:54:29-06:00</i>

</div>

<pre class='commit-message'>fix last commit to work on out of tree builds</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/919fe4a9a14845d55cc535bc7cc8a129a299a800">919fe4a9</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-15T16:47:36-06:00</i>

</div>

<pre class='commit-message'>cleanup darshan-test directory

moved most test programs and previous darshan output

to the 2.x directory</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/e4a2073026a7ff6b03d5bc07936e25befd1b8ca1">e4a20730</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-15T16:48:50-06:00</i>

</div>

<pre class='commit-message'>forgot to add 3 files to darshan-test/2.x</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/f6137524db0f85392411b7f4ab64c69f9f25ea6f">f6137524</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-19T09:57:37-08:00</i>

</div>

<pre class='commit-message'>bug fix in resetting of darshan mount info</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4a743c81b6c1cd7412a8114a6cb9632c2c01ba27">4a743c81</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-19T10:23:48-08:00</i>

</div>

<pre class='commit-message'>forgot to undefine hdf5/ncmpi symbols in ldopts</pre>

</li>

<li>

<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/0a6399f3acde4614a3adf9453505ed3d3e93b0ad">0a6399f3</a></strong>

<div>

<span>by Shane Snyder</span>

<i>at 2016-02-22T12:52:34-06:00</i>

</div>

<pre class='commit-message'>bug fix in darhsan-config linker options

Since we had to modify HDF5/PnetCDF linker flags to work on Cray

by undefining appropriate symbols, we need to update the linker

options used in darshan-config. darshan-config linker options need

to include the stubs libraries and use '--start-group' and

'--end-group' flags to resolve the symbols correctly</pre>

</li>

</ul>

<h4>188 changed files:</h4>

<ul>

<li class='file-stats'>

<a href='#diff-0'>

COPYRIGHT

</a>

</li>

<li class='file-stats'>

<a href='#diff-1'>

ChangeLog

</a>

</li>

<li class='file-stats'>

<a href='#diff-2'>

<span class='new-file'>

+

darshan-bgq-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-3'>

<span class='new-file'>

+

darshan-hdf5-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-4'>

darshan-log-format.h

</a>

</li>

<li class='file-stats'>

<a href='#diff-5'>

<span class='new-file'>

+

darshan-mpiio-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-6'>

<span class='new-file'>

+

darshan-null-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-7'>

<span class='new-file'>

+

darshan-pnetcdf-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-8'>

<span class='new-file'>

+

darshan-posix-log-format.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-9'>

darshan-runtime/Makefile.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-10'>

darshan-runtime/configure

</a>

</li>

<li class='file-stats'>

<a href='#diff-11'>

darshan-runtime/configure.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-12'>

<span class='new-file'>

+

darshan-runtime/darshan-common.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-13'>

darshan-runtime/darshan-config.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-14'>

<span class='new-file'>

+

darshan-runtime/darshan-core.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-15'>

darshan-runtime/darshan-dynamic.h

</a>

</li>

<li class='file-stats'>

<a href='#diff-16'>

darshan-runtime/darshan-gen-cc.pl.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-17'>

darshan-runtime/darshan-gen-cxx.pl.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-18'>

darshan-runtime/darshan-gen-fortran.pl.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-19'>

darshan-runtime/darshan-mk-log-dirs.pl.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-20'>

darshan-runtime/darshan-runtime-config.h.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-21'>

darshan-runtime/darshan.h

</a>

</li>

<li class='file-stats'>

<a href='#diff-22'>

darshan-runtime/doc/darshan-runtime.txt

</a>

</li>

<li class='file-stats'>

<a href='#diff-23'>

<span class='new-file'>

+

darshan-runtime/lib/darshan-bgq.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-24'>

<span class='new-file'>

+

darshan-runtime/lib/darshan-common.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-25'>

darshan-runtime/lib/darshan-mpi-init-finalize.c

→

darshan-runtime/lib/darshan-core-init-finalize.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-26'>

<span class='new-file'>

+

darshan-runtime/lib/darshan-core.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-27'>

darshan-runtime/lib/darshan-hdf5-stubs.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-28'>

darshan-runtime/lib/darshan-hdf5.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-29'>

<span class='deleted-file'>

−

darshan-runtime/lib/darshan-mpi-io.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-30'>

<span class='new-file'>

+

darshan-runtime/lib/darshan-mpiio.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-31'>

<span class='new-file'>

+

darshan-runtime/lib/darshan-null.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-32'>

darshan-runtime/lib/darshan-pnetcdf-stubs.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-33'>

darshan-runtime/lib/darshan-pnetcdf.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-34'>

darshan-runtime/lib/darshan-posix.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-35'>

darshan-runtime/lib/pkgconfig/darshan-runtime.pc.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-36'>

<span class='new-file'>

+

darshan-runtime/share/ld-opts/darshan-base-ld-opts.in

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-37'>

<span class='new-file'>

+

darshan-runtime/share/ld-opts/darshan-hdf5-ld-opts

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-38'>

<span class='new-file'>

+

darshan-runtime/share/ld-opts/darshan-pnetcdf-ld-opts

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-39'>

<span class='new-file'>

+

darshan-runtime/share/ld-opts/darshan-posix-ld-opts

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-40'>

<span class='new-file'>

+

darshan-runtime/uthash.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-41'>

<span class='new-file'>

+

darshan-runtime/utlist.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-42'>

darshan-test/analysis.sql

→

darshan-test/2.x/analysis.sql

</a>

</li>

<li class='file-stats'>

<a href='#diff-43'>

darshan-test/cp-bench.c

→

darshan-test/2.x/cp-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-44'>

darshan-test/cp-realpath-bench.c

→

darshan-test/2.x/cp-realpath-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-45'>

darshan-test/cp-search-bench.c

→

darshan-test/2.x/cp-search-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-46'>

darshan-test/cp-wtime-bench.c

→

darshan-test/2.x/cp-wtime-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-47'>

darshan-test/cp-wtimewrap-bench.c

→

darshan-test/2.x/cp-wtimewrap-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-48'>

darshan-test/darshan-anon-core-hours.pl

→

darshan-test/2.x/darshan-anon-core-hours.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-49'>

darshan-test/darshan-file-counter-hack.pl

→

darshan-test/2.x/darshan-file-counter-hack.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-50'>

darshan-test/darshan-gather-counts.pl

→

darshan-test/2.x/darshan-gather-counts.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-51'>

darshan-test/darshan-gather-intervals.pl

→

darshan-test/2.x/darshan-gather-intervals.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-52'>

darshan-test/darshan-gather-mpi-posix-usage.pl

→

darshan-test/2.x/darshan-gather-mpi-posix-usage.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-53'>

darshan-test/darshan-gather-stats.pl

→

darshan-test/2.x/darshan-gather-stats.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-54'>

darshan-test/darshan.mysql

→

darshan-test/2.x/darshan.mysql

</a>

</li>

<li class='file-stats'>

<a href='#diff-55'>

darshan-test/example-output/README.txt

→

darshan-test/2.x/example-output/README.txt

</a>

</li>

<li class='file-stats'>

<a href='#diff-56'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-57'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-58'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-59'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-60'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-61'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-62'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-63'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-64'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-65'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-66'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-67'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-68'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-69'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-70'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-71'>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-72'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-73'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-74'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-75'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-76'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-77'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-78'>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-79'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-80'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-81'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-82'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-83'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-84'>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-85'>

darshan-test/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-86'>

darshan-test/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-87'>

darshan-test/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-88'>

darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-89'>

darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-90'>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-91'>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-92'>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-93'>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-94'>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz

→

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-95'>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz

→

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-96'>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz

→

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-97'>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz

→

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-98'>

darshan-test/fsstats-merge.pl

→

darshan-test/2.x/fsstats-merge.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-99'>

darshan-test/fsstats-runner.bash

→

darshan-test/2.x/fsstats-runner.bash

</a>

</li>

<li class='file-stats'>

<a href='#diff-100'>

darshan-test/gz-bench.c

→

darshan-test/2.x/gz-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-101'>

darshan-test/gztest.c

→

darshan-test/2.x/gztest.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-102'>

darshan-test/output.txt

→

darshan-test/2.x/output.txt

</a>

</li>

<li class='file-stats'>

<a href='#diff-103'>

darshan-test/parallel-fsstats.pl

→

darshan-test/2.x/parallel-fsstats.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-104'>

darshan-test/patches/99_cu_futimens.patch

→

darshan-test/2.x/patches/99_cu_futimens.patch

</a>

</li>

<li class='file-stats'>

<a href='#diff-105'>

darshan-test/patches/README

→

darshan-test/2.x/patches/README

</a>

</li>

<li class='file-stats'>

<a href='#diff-106'>

darshan-test/patches/coreutils-cp-mpi.patch

→

darshan-test/2.x/patches/coreutils-cp-mpi.patch

</a>

</li>

<li class='file-stats'>

<a href='#diff-107'>

darshan-test/patches/darshan-coll-write-single.patch

→

darshan-test/2.x/patches/darshan-coll-write-single.patch

</a>

</li>

<li class='file-stats'>

<a href='#diff-108'>

darshan-test/patches/darshan-coll-write-strategy-test.patch

→

darshan-test/2.x/patches/darshan-coll-write-strategy-test.patch

</a>

</li>

<li class='file-stats'>

<a href='#diff-109'>

darshan-test/pcarns_cp_id25476_1-16-15155_2.darshan.gz

→

darshan-test/2.x/pcarns_cp_id25476_1-16-15155_2.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-110'>

darshan-test/perf/350665.output

→

darshan-test/2.x/perf/350665.output

</a>

</li>

<li class='file-stats'>

<a href='#diff-111'>

darshan-test/perf/350690.output

→

darshan-test/2.x/perf/350690.output

</a>

</li>

<li class='file-stats'>

<a href='#diff-112'>

darshan-test/perf/350892.output

→

darshan-test/2.x/perf/350892.output

</a>

</li>

<li class='file-stats'>

<a href='#diff-113'>

darshan-test/perf/354969.output

→

darshan-test/2.x/perf/354969.output

</a>

</li>

<li class='file-stats'>

<a href='#diff-114'>

darshan-test/perf/README

→

darshan-test/2.x/perf/README

</a>

</li>

<li class='file-stats'>

<a href='#diff-115'>

darshan-test/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz

→

darshan-test/2.x/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-116'>

darshan-test/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz

→

darshan-test/2.x/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-117'>

darshan-test/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz

→

darshan-test/2.x/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-118'>

darshan-test/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz

→

darshan-test/2.x/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz

</a>

</li>

<li class='file-stats'>

<a href='#diff-119'>

darshan-test/perf/partshared.c

→

darshan-test/2.x/perf/partshared.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-120'>

darshan-test/perf/single_and_multi_file.ior

→

darshan-test/2.x/perf/single_and_multi_file.ior

</a>

</li>

<li class='file-stats'>

<a href='#diff-121'>

darshan-test/seek-test.c

→

darshan-test/2.x/seek-test.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-122'>

darshan-test/stat-perf.c

→

darshan-test/2.x/stat-perf.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-123'>

darshan-test/test-darshan-compiles-alcf2.sh

→

darshan-test/2.x/test-darshan-compiles-alcf2.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-124'>

darshan-test/test-darshan-compiles.sh

→

darshan-test/2.x/test-darshan-compiles.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-125'>

darshan-test/test-darshan-log-formats.sh

→

darshan-test/2.x/test-darshan-log-formats.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-126'>

<span class='deleted-file'>

−

darshan-test/ADDING-COUNTERS.txt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-127'>

<span class='deleted-file'>

−

darshan-test/cuserid-test.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-128'>

darshan-test/darshan-shutdown-bench.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-129'>

darshan-test/regression/README.txt

</a>

</li>

<li class='file-stats'>

<a href='#diff-130'>

darshan-test/regression/test-cases/cxxpi.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-131'>

darshan-test/regression/test-cases/fperf-f77.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-132'>

darshan-test/regression/test-cases/fperf-f90.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-133'>

darshan-test/regression/test-cases/mpi-io-test.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-134'>

darshan-test/regression/workstation-profile-conf/env.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-135'>

darshan-util/Makefile.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-136'>

darshan-util/configure

</a>

</li>

<li class='file-stats'>

<a href='#diff-137'>

darshan-util/configure.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-138'>

darshan-util/darshan-analyzer.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-139'>

<span class='new-file'>

+

darshan-util/darshan-bgq-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-140'>

<span class='new-file'>

+

darshan-util/darshan-bgq-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-141'>

darshan-util/darshan-convert-logs.pl

</a>

</li>

<li class='file-stats'>

<a href='#diff-142'>

darshan-util/darshan-convert.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-143'>

darshan-util/darshan-diff.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-144'>

<span class='new-file'>

+

darshan-util/darshan-hdf5-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-145'>

<span class='new-file'>

+

darshan-util/darshan-hdf5-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-146'>

darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-147'>

darshan-util/darshan-job-summary/share/hist-eps.gplt

→

darshan-util/darshan-job-summary/share/access-hist-eps.gplt

</a>

</li>

<li class='file-stats'>

<a href='#diff-148'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/align-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-149'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/counts-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-150'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/counts-svg.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-151'>

<span class='new-file'>

+

darshan-util/darshan-job-summary/share/file-access-eps.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-152'>

<span class='new-file'>

+

darshan-util/darshan-job-summary/share/file-access-table.tex

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-153'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/hist-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-154'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/hist-svg.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-155'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/iodist-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-156'>

<span class='new-file'>

+

darshan-util/darshan-job-summary/share/job-table.tex

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-157'>

darshan-util/darshan-job-summary/share/counts-eps.gplt

→

darshan-util/darshan-job-summary/share/op-counts-eps.gplt

</a>

</li>

<li class='file-stats'>

<a href='#diff-158'>

darshan-util/darshan-job-summary/share/pattern-eps.gplt

</a>

</li>

<li class='file-stats'>

<a href='#diff-159'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/pattern-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-160'>

darshan-util/darshan-job-summary/share/summary.tex

</a>

</li>

<li class='file-stats'>

<a href='#diff-161'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/time-summary-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-162'>

<span class='new-file'>

+

darshan-util/darshan-job-summary/share/title.tex

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-163'>

<span class='deleted-file'>

−

darshan-util/darshan-job-summary/share/types-pdf.gplt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-164'>

<span class='deleted-file'>

−

darshan-util/darshan-load-mysql.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-165'>

<span class='deleted-file'>

−

darshan-util/darshan-log-params.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-166'>

darshan-util/darshan-logutils.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-167'>

darshan-util/darshan-logutils.h

</a>

</li>

<li class='file-stats'>

<a href='#diff-168'>

<span class='new-file'>

+

darshan-util/darshan-mpiio-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-169'>

<span class='new-file'>

+

darshan-util/darshan-mpiio-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-170'>

<span class='new-file'>

+

darshan-util/darshan-null-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-171'>

<span class='new-file'>

+

darshan-util/darshan-null-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-172'>

darshan-util/darshan-parser.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-173'>

<span class='new-file'>

+

darshan-util/darshan-pnetcdf-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-174'>

<span class='new-file'>

+

darshan-util/darshan-pnetcdf-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-175'>

<span class='new-file'>

+

darshan-util/darshan-posix-logutils.c

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-176'>

<span class='new-file'>

+

darshan-util/darshan-posix-logutils.h

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-177'>

darshan-util/darshan-summary-per-file.sh

</a>

</li>

<li class='file-stats'>

<a href='#diff-178'>

darshan-util/doc/darshan-util.txt

</a>

</li>

<li class='file-stats'>

<a href='#diff-179'>

darshan-util/darshan-job-summary/bin/jenkins.c

→

darshan-util/jenkins-hash-gen.c

</a>

</li>

<li class='file-stats'>

<a href='#diff-180'>

darshan-util/maint/darshan-util.pc.in

</a>

</li>

<li class='file-stats'>

<a href='#diff-181'>

<span class='new-file'>

+

doc/Makefile

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-182'>

<span class='new-file'>

+

doc/darshan-dev-modular-runtime.png

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-183'>

<span class='new-file'>

+

doc/darshan-modularization-design-notes.txt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-184'>

<span class='new-file'>

+

doc/darshan-modularization-whiteboard.pdf

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-185'>

<span class='new-file'>

+

doc/darshan-modularization.txt

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-186'>

<span class='new-file'>

+

doc/docbook-xsl.css

</span>

</a>

</li>

<li class='file-stats'>

<a href='#diff-187'>

maint/config/check_zlib.m4

</a>

</li>

</ul>

<h4>Changes:</h4>

<li id='diff-0'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-0'>

<strong>

COPYRIGHT

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/COPYRIGHT

</span><span style="color: #000000;background-color: #ddffdd">+++ b/COPYRIGHT

</span><span style="color: #aaaaaa">@@ -6,7 +6,7 @@ which must be included in the prologue of the code and in all source listings

of the code.

 Copyright Notice

<span style="color: #000000;background-color: #ffdddd">- + 2009 University of Chicago

</span><span style="color: #000000;background-color: #ddffdd">+ + 2015 University of Chicago

</span> 

 Permission is hereby granted to use, reproduce, prepare derivative works, and

 to redistribute to others.  This software was authored by:

</code></pre>

<br>

</li>

<li id='diff-1'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-1'>

<strong>

ChangeLog

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/ChangeLog

</span><span style="color: #000000;background-color: #ddffdd">+++ b/ChangeLog

</span><span style="color: #aaaaaa">@@ -2,6 +2,56 @@

</span> Darshan Release Change Log

 --------------------------

<span style="color: #000000;background-color: #ddffdd">+Darshan-3.0.0-pre3

+=============

+* add module-specific version fields to header to allow utilities

+  to handle different versions of a module's I/O data for backwards

+  compatibility -- NOTE: this breaks the log file parsing for logs

+  obtained using Darshan-3.0.0-pre2 & Darshan-3.0.0-pre1 

+* bug fix in regression test scripts for setting proper environment

+  variables to use MPI profiling configuration for Fortran apps

+* bug fix in bzip2 log writing implementation in darshan-logutils

+* possible race conditions resolved in each module's shutdown code

+* general code, comment, and documentation cleanup

+* addition of module-specific counter descriptions printed prior

+  to parsing a modules I/O data in darshan-parser

+

+Darshan-3.0.0-pre2

+=============

+* add fix to install appropriate headers for linking external

+  applications with darshan-util (reported by Matthieu Dorier)

+* add darshan-util Ruby bindings for the new modularized version

+  of Darshan (3.0) (Matthieu Dorier)

+* add enhancement to darshan-runtime to allow per-module instrumentation

+  memory to be user configurable using a configure option or a runtime

+  environment variable

+

+Darshan-3.0.0-pre1

+=============

+* new version of Darshan with the following features/improvements:

+    - hooks for developers to add their own instrumentation module

+      implementations to capture new I/O characterization data

+        - these instrumentation modules can be used to instrument new

+          I/O interfaces or gather system-specific parameters, for instance

+    - modularized log format allows new module-specific utilities to

+      access their I/O characterization data independently

+        - this new format also allows new counters to be added to existing

+          instrumentation modules without breaking existing utilities

+    - Darshan logs now contain a mapping of Darshan's unique record

+      identifiers to full file names, instead of fix-sized file name

+      suffixes

+    - a new instrumentation module for capturing BG/Q-specific parameters

+      (BG/Q environment is automatically detected at configure time)

+      (implemented by Kevin Harms)

+    - new darshan-parser and darshan-job-summary output to utilize the

+      new modularized log format

+* updated documentation outlining changes in this release, as well as

+  steps for adding new instrumentation modules is given in the top-level

+  'doc' directory.

+    - documentation for configuring and using the darshan-runtime and

+      darshan-util components are mostly the same and still located in

+      their respective directories ('darshan-runtime/doc' and 'darshan-util/doc')

+

</span> darshan-2.3.2-pre1

 =============

 * Fix gnuplot version number check to allow darshan-job-summary.pl to work

</code></pre>

<br>

</li>

<li id='diff-2'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-2'>

<strong>

darshan-bgq-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-bgq-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,77 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_BGQ_LOG_FORMAT_H

+#define __DARSHAN_BGQ_LOG_FORMAT_H

+

+/* current BGQ log format version */

+#define DARSHAN_BGQ_VER 1

+

+#define BGQ_COUNTERS \

+    /* control system jobid*/\

+    X(BGQ_CSJOBID) \

+    /* number of BGQ compute nodes */\

+    X(BGQ_NNODES) \

+    /* number of MPI ranks per node */\

+    X(BGQ_RANKSPERNODE) \

+    /* size in MB of DDR3 per node */\

+    X(BGQ_DDRPERNODE) \

+    /* number of i/o nodes */\

+    X(BGQ_INODES) \

+    /* dimension of A torus */\

+    X(BGQ_ANODES) \

+    /* dimension of B torus */\

+    X(BGQ_BNODES) \

+    /* dimension of C torus */\

+    X(BGQ_CNODES) \

+    /* dimension of D torus */\

+    X(BGQ_DNODES) \

+    /* dimension of E torus */\

+    X(BGQ_ENODES) \

+    /* which dimensions are torus */\

+    X(BGQ_TORUSENABLED) \

+    /* end of counters */\

+    X(BGQ_NUM_INDICES)

+

+#define BGQ_F_COUNTERS \

+    /* timestamp when data was collected */\

+    X(BGQ_F_TIMESTAMP) \

+    /* end of counters */\

+    X(BGQ_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer counters for the "BGQ" example module */

+enum darshan_bgq_indices

+{

+    BGQ_COUNTERS

+};

+

+/* floating point counters for the "BGQ" example module */

+enum darshan_bgq_f_indices

+{

+    BGQ_F_COUNTERS

+};

+#undef X

+

+/* the darshan_bgq_record structure encompasses the high-level data/counters

+ * which would actually be logged to file by Darshan for the "BGQ" example

+ * module. This example implementation logs the following data for each

+ * record:

+ *      - a corresponding Darshan record identifier

+ *      - the rank of the process responsible for the record

+ *      - integer I/O counters (operation counts, I/O sizes, etc.)

+ *      - floating point I/O counters (timestamps, cumulative timers, etc.)

+ */

+struct darshan_bgq_record

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int alignment;

+    int64_t counters[BGQ_NUM_INDICES];

+    double fcounters[BGQ_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_BGQ_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-3'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-3'>

<strong>

darshan-hdf5-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-hdf5-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,57 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_HDF5_LOG_FORMAT_H

+#define __DARSHAN_HDF5_LOG_FORMAT_H

+

+/* current HDF5 log format version */

+#define DARSHAN_HDF5_VER 1

+

+#define HDF5_COUNTERS \

+    /* count of HDF5 opens */\

+    X(HDF5_OPENS) \

+    /* end of counters */\

+    X(HDF5_NUM_INDICES)

+

+#define HDF5_F_COUNTERS \

+    /* timestamp of first open */\

+    X(HDF5_F_OPEN_TIMESTAMP) \

+    /* timestamp of last close */\

+    X(HDF5_F_CLOSE_TIMESTAMP) \

+    /* end of counters*/\

+    X(HDF5_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer statistics for HDF5 file records */

+enum darshan_hdf5_indices

+{

+    HDF5_COUNTERS

+};

+

+/* floating point statistics for HDF5 file records */

+enum darshan_hdf5_f_indices

+{

+    HDF5_F_COUNTERS

+};

+#undef X

+

+/* file record structure for HDF5 files. a record is created and stored for

+ * every HDF5 file opened by the original application. For the HDF5 module,

+ * the record includes:

+ *      - a corresponding record identifier (created by hashing the file path)

+ *      - the rank of the process which opened the file (-1 for shared files)

+ *      - integer file I/O statistics (open, read/write counts, etc)

+ *      - floating point I/O statistics (timestamps, cumulative timers, etc.)

+ */

+struct darshan_hdf5_file

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int64_t counters[HDF5_NUM_INDICES];

+    double fcounters[HDF5_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_HDF5_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-4'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-4'>

<strong>

darshan-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-log-format.h

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-log-format.h

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #ifndef __DARSHAN_LOG_FORMAT_H

<span style="color: #aaaaaa">@@ -19,232 +20,63 @@

</span> #endif

 /* update this on file format changes */

<span style="color: #000000;background-color: #ffdddd">-#define CP_VERSION "2.06"

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_LOG_VERSION "3.00"

</span> 

 /* magic number for validating output files and checking byte order */

<span style="color: #000000;background-color: #ffdddd">-#define CP_MAGIC_NR 6567223

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_MAGIC_NR 6567223

</span> 

 /* size (in bytes) of job record */

<span style="color: #000000;background-color: #ffdddd">-#define CP_JOB_RECORD_SIZE 4096

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_JOB_RECORD_SIZE 4096

</span> 

 /* max length of exe string within job record (not counting '\0') */

<span style="color: #000000;background-color: #ffdddd">-#define CP_EXE_LEN (CP_JOB_RECORD_SIZE - sizeof(struct darshan_job) - 1)

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_EXE_LEN (DARSHAN_JOB_RECORD_SIZE - sizeof(struct darshan_job) - 1)

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* size (in bytes) of each file record */

-#define CP_FILE_RECORD_SIZE (sizeof(struct darshan_file))

</span><span style="color: #000000;background-color: #ddffdd">+/* max number of modules that can be used in a darshan log */

+#define DARSHAN_MAX_MODS 16

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* max length of name suffix string within file record (not counting '\0') */

-#define CP_NAME_SUFFIX_LEN 15

</span><span style="color: #000000;background-color: #ddffdd">+/* simple macros for accessing module flag bitfields */

+#define DARSHAN_MOD_FLAG_SET(flags, id) flags = (flags | (1 << id))

+#define DARSHAN_MOD_FLAG_UNSET(flags, id) flags = (flags & ~(1 << id))

+#define DARSHAN_MOD_FLAG_ISSET(flags, id) (flags & (1 << id))

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* per file statistics */

-enum darshan_indices

</span><span style="color: #000000;background-color: #ddffdd">+/* compression method used on darshan log file */

+enum darshan_comp_type

</span> {

<span style="color: #000000;background-color: #ffdddd">-    CP_INDEP_OPENS = 0,          /* count of MPI independent opens */

-    CP_COLL_OPENS,               /* count of MPI collective opens */

-    CP_INDEP_READS,              /* count of independent MPI reads */

-    CP_INDEP_WRITES,             /* count of independent MPI writes */

-    CP_COLL_READS,               /* count of collective MPI reads */

-    CP_COLL_WRITES,              /* count of collective MPI writes */

-    CP_SPLIT_READS,              /* count of split collective MPI reads */

-    CP_SPLIT_WRITES,             /* count of split collective MPI writes */

-    CP_NB_READS,                 /* count of nonblocking MPI reads */

-    CP_NB_WRITES,                /* count of nonblocking MPI writes */

-    CP_SYNCS,

-    CP_POSIX_READS,              /* count of posix reads */

-    CP_POSIX_WRITES,             /* count of posix writes */

-    CP_POSIX_OPENS,              /* count of posix opens */

-    CP_POSIX_SEEKS,              /* count of posix seeks */

-    CP_POSIX_STATS,              /* count of posix stat/lstat/fstats */

-    CP_POSIX_MMAPS,              /* count of posix mmaps */

-    CP_POSIX_FREADS,

-    CP_POSIX_FWRITES,

-    CP_POSIX_FOPENS,

-    CP_POSIX_FSEEKS,

-    CP_POSIX_FSYNCS,

-    CP_POSIX_FDSYNCS,

-    /* pnetcdf stuff */

-    CP_INDEP_NC_OPENS,

-    CP_COLL_NC_OPENS,

-    /* HDF5 stuff */

-    CP_HDF5_OPENS,

-    /* type categories */

-    CP_COMBINER_NAMED,           /* count of each MPI datatype category */

-    CP_COMBINER_DUP,

-    CP_COMBINER_CONTIGUOUS,

-    CP_COMBINER_VECTOR,

-    CP_COMBINER_HVECTOR_INTEGER,

-    CP_COMBINER_HVECTOR,

-    CP_COMBINER_INDEXED,

-    CP_COMBINER_HINDEXED_INTEGER,

-    CP_COMBINER_HINDEXED,

-    CP_COMBINER_INDEXED_BLOCK,

-    CP_COMBINER_STRUCT_INTEGER,

-    CP_COMBINER_STRUCT,

-    CP_COMBINER_SUBARRAY,

-    CP_COMBINER_DARRAY,

-    CP_COMBINER_F90_REAL,

-    CP_COMBINER_F90_COMPLEX,

-    CP_COMBINER_F90_INTEGER,

-    CP_COMBINER_RESIZED,

-    CP_HINTS,                     /* count of MPI hints used */

-    CP_VIEWS,                     /* count of MPI set view calls */

-    CP_MODE,                      /* mode of file */

-    CP_BYTES_READ,                /* total bytes read */

-    CP_BYTES_WRITTEN,             /* total bytes written */

-    CP_MAX_BYTE_READ,             /* highest offset byte read */

-    CP_MAX_BYTE_WRITTEN,          /* highest offset byte written */

-    CP_CONSEC_READS,              /* count of consecutive reads */

-    CP_CONSEC_WRITES,             /* count of consecutive writes */

-    CP_SEQ_READS,                 /* count of sequential reads */

-    CP_SEQ_WRITES,                /* count of sequential writes */

-    CP_RW_SWITCHES,               /* number of times switched between read and write */

-    CP_MEM_NOT_ALIGNED,           /* count of accesses not mem aligned */

-    CP_MEM_ALIGNMENT,             /* mem alignment in bytes */

-    CP_FILE_NOT_ALIGNED,          /* count of accesses not file aligned */

-    CP_FILE_ALIGNMENT,            /* file alignment in bytes */

-    CP_MAX_READ_TIME_SIZE,

-    CP_MAX_WRITE_TIME_SIZE,

-    /* buckets */

-    CP_SIZE_READ_0_100,           /* count of posix read size ranges */

-    CP_SIZE_READ_100_1K,

-    CP_SIZE_READ_1K_10K,

-    CP_SIZE_READ_10K_100K,

-    CP_SIZE_READ_100K_1M,

-    CP_SIZE_READ_1M_4M,

-    CP_SIZE_READ_4M_10M,

-    CP_SIZE_READ_10M_100M,

-    CP_SIZE_READ_100M_1G,

-    CP_SIZE_READ_1G_PLUS,

-    /* buckets */

-    CP_SIZE_WRITE_0_100,          /* count of posix write size ranges */

-    CP_SIZE_WRITE_100_1K,

-    CP_SIZE_WRITE_1K_10K,

-    CP_SIZE_WRITE_10K_100K,

-    CP_SIZE_WRITE_100K_1M,

-    CP_SIZE_WRITE_1M_4M,

-    CP_SIZE_WRITE_4M_10M,

-    CP_SIZE_WRITE_10M_100M,

-    CP_SIZE_WRITE_100M_1G,

-    CP_SIZE_WRITE_1G_PLUS,

-    /* buckets */

-    CP_SIZE_READ_AGG_0_100,       /* count of MPI read size ranges */

-    CP_SIZE_READ_AGG_100_1K,

-    CP_SIZE_READ_AGG_1K_10K,

-    CP_SIZE_READ_AGG_10K_100K,

-    CP_SIZE_READ_AGG_100K_1M,

-    CP_SIZE_READ_AGG_1M_4M,

-    CP_SIZE_READ_AGG_4M_10M,

-    CP_SIZE_READ_AGG_10M_100M,

-    CP_SIZE_READ_AGG_100M_1G,

-    CP_SIZE_READ_AGG_1G_PLUS,

-    /* buckets */

-    CP_SIZE_WRITE_AGG_0_100,      /* count of MPI write size ranges */

-    CP_SIZE_WRITE_AGG_100_1K,

-    CP_SIZE_WRITE_AGG_1K_10K,

-    CP_SIZE_WRITE_AGG_10K_100K,

-    CP_SIZE_WRITE_AGG_100K_1M,

-    CP_SIZE_WRITE_AGG_1M_4M,

-    CP_SIZE_WRITE_AGG_4M_10M,

-    CP_SIZE_WRITE_AGG_10M_100M,

-    CP_SIZE_WRITE_AGG_100M_1G,

-    CP_SIZE_WRITE_AGG_1G_PLUS,

-    /* buckets */

-    CP_EXTENT_READ_0_100,          /* count of MPI read extent ranges */

-    CP_EXTENT_READ_100_1K,

-    CP_EXTENT_READ_1K_10K,

-    CP_EXTENT_READ_10K_100K,

-    CP_EXTENT_READ_100K_1M,

-    CP_EXTENT_READ_1M_4M,

-    CP_EXTENT_READ_4M_10M,

-    CP_EXTENT_READ_10M_100M,

-    CP_EXTENT_READ_100M_1G,

-    CP_EXTENT_READ_1G_PLUS,

-    /* buckets */

-    CP_EXTENT_WRITE_0_100,         /* count of MPI write extent ranges */

-    CP_EXTENT_WRITE_100_1K,

-    CP_EXTENT_WRITE_1K_10K,

-    CP_EXTENT_WRITE_10K_100K,

-    CP_EXTENT_WRITE_100K_1M,

-    CP_EXTENT_WRITE_1M_4M,

-    CP_EXTENT_WRITE_4M_10M,

-    CP_EXTENT_WRITE_10M_100M,

-    CP_EXTENT_WRITE_100M_1G,

-    CP_EXTENT_WRITE_1G_PLUS,

-    /* counters */

-    CP_STRIDE1_STRIDE,             /* the four most frequently appearing strides */

-    CP_STRIDE2_STRIDE,

-    CP_STRIDE3_STRIDE,

-    CP_STRIDE4_STRIDE,

-    CP_STRIDE1_COUNT,              /* count of each of the most frequent strides */

-    CP_STRIDE2_COUNT,

-    CP_STRIDE3_COUNT,

-    CP_STRIDE4_COUNT,

-    CP_ACCESS1_ACCESS,             /* the four most frequently appearing access sizes */

-    CP_ACCESS2_ACCESS,

-    CP_ACCESS3_ACCESS,

-    CP_ACCESS4_ACCESS,

-    CP_ACCESS1_COUNT,              /* count of each of the most frequent access sizes */

-    CP_ACCESS2_COUNT,

-    CP_ACCESS3_COUNT,

-    CP_ACCESS4_COUNT,

-    CP_DEVICE,                     /* device id reported by stat */

-    CP_SIZE_AT_OPEN,

-    CP_FASTEST_RANK,

-    CP_FASTEST_RANK_BYTES,

-    CP_SLOWEST_RANK,

-    CP_SLOWEST_RANK_BYTES,

-

-    CP_NUM_INDICES,

</span><span style="color: #000000;background-color: #ddffdd">+    DARSHAN_ZLIB_COMP,

+    DARSHAN_BZIP2_COMP,

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* floating point statistics */

-enum f_darshan_indices

-{

-    /* NOTE: adjust cp_normalize_timestamps() function if any TIMESTAMPS are

-     * added or modified in this list

-     */

-    CP_F_OPEN_TIMESTAMP = 0,    /* timestamp of first open */

-    CP_F_READ_START_TIMESTAMP,  /* timestamp of first read */

-    CP_F_WRITE_START_TIMESTAMP, /* timestamp of first write */

-    CP_F_CLOSE_TIMESTAMP,       /* timestamp of last close */

-    CP_F_READ_END_TIMESTAMP,    /* timestamp of last read */

-    CP_F_WRITE_END_TIMESTAMP,   /* timestamp of last write */

-    CP_F_POSIX_READ_TIME,       /* cumulative posix read time */

-    CP_F_POSIX_WRITE_TIME,      /* cumulative posix write time */

-    CP_F_POSIX_META_TIME,       /* cumulative posix meta time */

-    CP_F_MPI_META_TIME,         /* cumulative mpi-io meta time */

-    CP_F_MPI_READ_TIME,         /* cumulative mpi-io read time */

-    CP_F_MPI_WRITE_TIME,        /* cumulative mpi-io write time */

-    CP_F_MAX_READ_TIME,

-    CP_F_MAX_WRITE_TIME,

-    /* Total I/O and meta time consumed by fastest and slowest ranks, 

-     * reported in either MPI or POSIX time depending on how the file 

-     * was accessed.

-     */

-    CP_F_FASTEST_RANK_TIME,     

-    CP_F_SLOWEST_RANK_TIME,

-    CP_F_VARIANCE_RANK_TIME,

-    CP_F_VARIANCE_RANK_BYTES,

</span><span style="color: #000000;background-color: #ddffdd">+typedef uint64_t darshan_record_id;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_F_NUM_INDICES,

</span><span style="color: #000000;background-color: #ddffdd">+/* the darshan_log_map structure is used to indicate the location of

+ * specific module data in a Darshan log. Note that 'off' and 'len' are

+ * the respective offset and length of the data in the file, in

+ * *compressed* terms

+ */

+struct darshan_log_map

+{

+    uint64_t off;

+    uint64_t len;

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* statistics for any kind of file */

-struct darshan_file

</span><span style="color: #000000;background-color: #ddffdd">+/* the darshan header stores critical metadata needed for correctly

+ * reading the contents of the corresponding Darshan log

+ */

+struct darshan_header

</span> {

<span style="color: #000000;background-color: #ffdddd">-    uint64_t hash;

-    int64_t rank;

-    char name_suffix[CP_NAME_SUFFIX_LEN+1];

-    int64_t counters[CP_NUM_INDICES];

-    double fcounters[CP_F_NUM_INDICES];

</span><span style="color: #000000;background-color: #ddffdd">+    char version_string[8];

+    int64_t magic_nr;

+    unsigned char comp_type;

+    uint32_t partial_flag;

+    struct darshan_log_map rec_map;

+    struct darshan_log_map mod_map[DARSHAN_MAX_MODS];

+    uint32_t mod_ver[DARSHAN_MAX_MODS];

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* statistics for the job as a whole */

</span><span style="color: #000000;background-color: #ddffdd">+/* job-level metadata stored for this application */

</span> #define DARSHAN_JOB_METADATA_LEN 1024 /* including null terminator! */

 struct darshan_job

 {

<span style="color: #000000;background-color: #ffdddd">-    char version_string[8];

-    int64_t magic_nr;

</span>     int64_t uid;

     int64_t start_time;

     int64_t end_time;

<span style="color: #aaaaaa">@@ -253,11 +85,68 @@ struct darshan_job

</span>     char metadata[DARSHAN_JOB_METADATA_LEN];

 };

<span style="color: #000000;background-color: #ffdddd">-/* This macro can be used to identify files that have been opened using

- * pnetcdf, hdf5, or mpi-io, but were never opened at the posix level.  As a

- * result the record will not necessarily have all of the expected fields

- * populated.

</span><span style="color: #000000;background-color: #ddffdd">+/* minimal record stored for each file/object accessed by Darshan */

+struct darshan_record

+{

+    char* name;

+    darshan_record_id id;

+};

+

+

+/************************************************

+ *** module-specific includes and definitions ***

+ ************************************************/

+

+#include "darshan-null-log-format.h"

+#include "darshan-posix-log-format.h"

+#include "darshan-mpiio-log-format.h"

+#include "darshan-hdf5-log-format.h"

+#include "darshan-pnetcdf-log-format.h"

+#include "darshan-bgq-log-format.h"

+

+/* X-macro for keeping module ordering consistent */

+/* NOTE: first val used to define module enum values, 

+ * second val used to define module name strings,

+ * third val is the log format version for the module,

+ * and fourth val is used to provide the name of a 

+ * corresponding logutils structure for parsing module

+ * data out of the log file (only used in darshan-util

+ * component -- NULL can be passed if there are no

+ * logutil definitions)

+ */

+#define DARSHAN_MODULE_IDS \

+    X(DARSHAN_NULL_MOD,     "NULL",     DARSHAN_NULL_VER,       NULL) \

+    X(DARSHAN_POSIX_MOD,    "POSIX",    DARSHAN_POSIX_VER,      &posix_logutils) \

+    X(DARSHAN_MPIIO_MOD,    "MPI-IO",   DARSHAN_MPIIO_VER,      &mpiio_logutils) \

+    X(DARSHAN_HDF5_MOD,     "HDF5",     DARSHAN_HDF5_VER,       &hdf5_logutils) \

+    X(DARSHAN_PNETCDF_MOD,  "PNETCDF",  DARSHAN_PNETCDF_VER,    &pnetcdf_logutils) \

+    X(DARSHAN_BGQ_MOD,      "BG/Q",     DARSHAN_BGQ_VER,        &bgq_logutils)

+

+/* unique identifiers to distinguish between available darshan modules */

+/* NOTES: - valid ids range from [0...DARSHAN_MAX_MODS-1]

+ *        - order of ids control module shutdown order (and consequently, order in log file)

</span>  */

<span style="color: #000000;background-color: #ffdddd">-#define CP_FILE_PARTIAL(__file)((((__file)->counters[CP_POSIX_OPENS] || (__file)->counters[CP_POSIX_FOPENS] || (__file)->counters[CP_POSIX_STATS]) ? 0 : 1))

</span><span style="color: #000000;background-color: #ddffdd">+#define X(a, b, c, d) a,

+typedef enum

+{

+    DARSHAN_MODULE_IDS

+} darshan_module_id;

+#undef X

+

+/* module name strings */

+#define X(a, b, c, d) b,

+static char * const darshan_module_names[] =

+{

+    DARSHAN_MODULE_IDS

+};

+#undef X

+

+/* module version numbers */

+#define X(a, b, c, d) c,

+static const int darshan_module_versions[] =

+{

+    DARSHAN_MODULE_IDS

+};

+#undef X

</span> 

 #endif /* __DARSHAN_LOG_FORMAT_H */

</code></pre>

<br>

</li>

<li id='diff-5'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-5'>

<strong>

darshan-mpiio-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-mpiio-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,156 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_MPIIO_LOG_FORMAT_H

+#define __DARSHAN_MPIIO_LOG_FORMAT_H

+

+/* current MPI-IO log format version */

+#define DARSHAN_MPIIO_VER 1

+

+/* TODO: maybe use a counter to track cases in which a derived datatype is used? */

+

+#define MPIIO_COUNTERS \

+    /* count of MPI independent opens */\

+    X(MPIIO_INDEP_OPENS) \

+    /* count of MPI collective opens */\

+    X(MPIIO_COLL_OPENS) \

+    /* count of MPI independent reads */\

+    X(MPIIO_INDEP_READS) \

+    /* count of MPI independent writes */\

+    X(MPIIO_INDEP_WRITES) \

+    /* count of MPI collective reads */\

+    X(MPIIO_COLL_READS) \

+    /* count of MPI collective writes */\

+    X(MPIIO_COLL_WRITES) \

+    /* count of MPI split collective reads */\

+    X(MPIIO_SPLIT_READS) \

+    /* count of MPI split collective writes */\

+    X(MPIIO_SPLIT_WRITES) \

+    /* count of MPI nonblocking reads */\

+    X(MPIIO_NB_READS) \

+    /* count of MPI nonblocking writes */\

+    X(MPIIO_NB_WRITES) \

+    /* count of MPI file syncs */\

+    X(MPIIO_SYNCS) \

+    /* count of MPI hints used */\

+    X(MPIIO_HINTS) \

+    /* count of MPI set view calls */\

+    X(MPIIO_VIEWS) \

+    /* MPI-IO access mode of the file */\

+    X(MPIIO_MODE) \

+    /* total bytes read at MPI-IO layer */\

+    X(MPIIO_BYTES_READ) \

+    /* total bytes written at MPI-IO layer */\

+    X(MPIIO_BYTES_WRITTEN) \

+    /* number of times switching between MPI read and write */\

+    X(MPIIO_RW_SWITCHES) \

+    /* sizes of the maximum read/write operations */\

+    X(MPIIO_MAX_READ_TIME_SIZE) \

+    X(MPIIO_MAX_WRITE_TIME_SIZE) \

+    /* buckets for MPI read size ranges */\

+    X(MPIIO_SIZE_READ_AGG_0_100) \

+    X(MPIIO_SIZE_READ_AGG_100_1K) \

+    X(MPIIO_SIZE_READ_AGG_1K_10K) \

+    X(MPIIO_SIZE_READ_AGG_10K_100K) \

+    X(MPIIO_SIZE_READ_AGG_100K_1M) \

+    X(MPIIO_SIZE_READ_AGG_1M_4M) \

+    X(MPIIO_SIZE_READ_AGG_4M_10M) \

+    X(MPIIO_SIZE_READ_AGG_10M_100M) \

+    X(MPIIO_SIZE_READ_AGG_100M_1G) \

+    X(MPIIO_SIZE_READ_AGG_1G_PLUS) \

+    /* buckets for MPI write size ranges */\

+    X(MPIIO_SIZE_WRITE_AGG_0_100) \

+    X(MPIIO_SIZE_WRITE_AGG_100_1K) \

+    X(MPIIO_SIZE_WRITE_AGG_1K_10K) \

+    X(MPIIO_SIZE_WRITE_AGG_10K_100K) \

+    X(MPIIO_SIZE_WRITE_AGG_100K_1M) \

+    X(MPIIO_SIZE_WRITE_AGG_1M_4M) \

+    X(MPIIO_SIZE_WRITE_AGG_4M_10M) \

+    X(MPIIO_SIZE_WRITE_AGG_10M_100M) \

+    X(MPIIO_SIZE_WRITE_AGG_100M_1G) \

+    X(MPIIO_SIZE_WRITE_AGG_1G_PLUS) \

+    /* the four most frequently appearing MPI access sizes */\

+    X(MPIIO_ACCESS1_ACCESS) \

+    X(MPIIO_ACCESS2_ACCESS) \

+    X(MPIIO_ACCESS3_ACCESS) \

+    X(MPIIO_ACCESS4_ACCESS) \

+    /* count of each of the most frequent MPI access sizes */\

+    X(MPIIO_ACCESS1_COUNT) \

+    X(MPIIO_ACCESS2_COUNT) \

+    X(MPIIO_ACCESS3_COUNT) \

+    X(MPIIO_ACCESS4_COUNT) \

+    /* rank and number of bytes moved for fastest/slowest ranks */\

+    X(MPIIO_FASTEST_RANK) \

+    X(MPIIO_FASTEST_RANK_BYTES) \

+    X(MPIIO_SLOWEST_RANK) \

+    X(MPIIO_SLOWEST_RANK_BYTES) \

+    /* end of counters */\

+    X(MPIIO_NUM_INDICES)

+

+#define MPIIO_F_COUNTERS \

+    /* timestamp of first open */\

+    X(MPIIO_F_OPEN_TIMESTAMP) \

+    /* timestamp of first read */\

+    X(MPIIO_F_READ_START_TIMESTAMP) \

+    /* timestamp of first write */\

+    X(MPIIO_F_WRITE_START_TIMESTAMP) \

+    /* timestamp of last read */\

+    X(MPIIO_F_READ_END_TIMESTAMP) \

+    /* timestamp of last write */\

+    X(MPIIO_F_WRITE_END_TIMESTAMP) \

+    /* timestamp of last close */\

+    X(MPIIO_F_CLOSE_TIMESTAMP) \

+    /* cumulative MPI-IO read time */\

+    X(MPIIO_F_READ_TIME) \

+    /* cumulative MPI-IO write time */\

+    X(MPIIO_F_WRITE_TIME) \

+    /* cumulative MPI-IO meta time */\

+    X(MPIIO_F_META_TIME) \

+    /* maximum MPI-IO read duration */\

+    X(MPIIO_F_MAX_READ_TIME) \

+    /* maximum MPI-IO write duration */\

+    X(MPIIO_F_MAX_WRITE_TIME) \

+    /* total i/o and meta time for fastest/slowest ranks */\

+    X(MPIIO_F_FASTEST_RANK_TIME) \

+    X(MPIIO_F_SLOWEST_RANK_TIME) \

+    /* variance of total i/o time and bytes moved across all ranks */\

+    /* NOTE: for shared records only */\

+    X(MPIIO_F_VARIANCE_RANK_TIME) \

+    X(MPIIO_F_VARIANCE_RANK_BYTES) \

+    /* end of counters*/\

+    X(MPIIO_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer statistics for MPI-IO file records */

+enum darshan_mpiio_indices

+{

+    MPIIO_COUNTERS

+};

+

+/* floating point statistics for MPI-IO file records */

+enum darshan_mpiio_f_indices

+{

+    MPIIO_F_COUNTERS

+};

+#undef X

+

+/* file record structure for MPI-IO files. a record is created and stored for

+ * every MPI-IO file opened by the original application. For the MPI-IO module,

+ * the record includes:

+ *      - a corresponding record identifier (created by hashing the file path)

+ *      - the rank of the process which opened the file (-1 for shared files)

+ *      - integer file I/O statistics (open, read/write counts, etc)

+ *      - floating point I/O statistics (timestamps, cumulative timers, etc.)

+ */

+struct darshan_mpiio_file

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int64_t counters[MPIIO_NUM_INDICES];

+    double fcounters[MPIIO_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_MPIIO_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-6'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-6'>

<strong>

darshan-null-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-null-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,60 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_NULL_LOG_FORMAT_H

+#define __DARSHAN_NULL_LOG_FORMAT_H

+

+/* current log format version, to support backwards compatibility */

+#define DARSHAN_NULL_VER 1

+

+#define NULL_COUNTERS \

+    /* count of number of 'bar' function calls */\

+    X(NULL_BARS) \

+    /* arbitrary data value set by last call to 'bar' */\

+    X(NULL_BAR_DAT) \

+    /* end of counters */\

+    X(NULL_NUM_INDICES)

+

+#define NULL_F_COUNTERS \

+    /* timestamp of the first call to function 'bar' */\

+    X(NULL_F_BAR_TIMESTAMP) \

+    /* timer indicating duration of last call to 'bar' */\

+    X(NULL_F_BAR_DURATION) \

+    /* end of counters */\

+    X(NULL_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer counters for the "NULL" example module */

+enum darshan_null_indices

+{

+    NULL_COUNTERS

+};

+

+/* floating point counters for the "NULL" example module */

+enum darshan_null_f_indices

+{

+    NULL_F_COUNTERS

+};

+#undef X

+

+/* the darshan_null_record structure encompasses the high-level data/counters

+ * which would actually be logged to file by Darshan for the "NULL" example

+ * module. This example implementation logs the following data for each

+ * record:

+ *      - a corresponding Darshan record identifier

+ *      - the rank of the process responsible for the record

+ *      - integer I/O counters (operation counts, I/O sizes, etc.)

+ *      - floating point I/O counters (timestamps, cumulative timers, etc.)

+ */

+struct darshan_null_record

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int64_t counters[NULL_NUM_INDICES];

+    double fcounters[NULL_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_NULL_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-7'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-7'>

<strong>

darshan-pnetcdf-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-pnetcdf-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,59 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_PNETCDF_LOG_FORMAT_H

+#define __DARSHAN_PNETCDF_LOG_FORMAT_H

+

+/* current PNETCDF log format version */

+#define DARSHAN_PNETCDF_VER 1

+

+#define PNETCDF_COUNTERS \

+    /* count of PNETCDF independent opens */\

+    X(PNETCDF_INDEP_OPENS) \

+    /* count of PNETCDF collective opens */\

+    X(PNETCDF_COLL_OPENS) \

+    /* end of counters */\

+    X(PNETCDF_NUM_INDICES)

+

+#define PNETCDF_F_COUNTERS \

+    /* timestamp of first open */\

+    X(PNETCDF_F_OPEN_TIMESTAMP) \

+    /* timestamp of last close */\

+    X(PNETCDF_F_CLOSE_TIMESTAMP) \

+    /* end of counters*/\

+    X(PNETCDF_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer statistics for PNETCDF file records */

+enum darshan_pnetcdf_indices

+{

+    PNETCDF_COUNTERS

+};

+

+/* floating point statistics for PNETCDF file records */

+enum darshan_pnetcdf_f_indices

+{

+    PNETCDF_F_COUNTERS

+};

+#undef X

+

+/* file record structure for PNETCDF files. a record is created and stored for

+ * every PNETCDF file opened by the original application. For the PNETCDF module,

+ * the record includes:

+ *      - a corresponding record identifier (created by hashing the file path)

+ *      - the rank of the process which opened the file (-1 for shared files)

+ *      - integer file I/O statistics (open, read/write counts, etc)

+ *      - floating point I/O statistics (timestamps, cumulative timers, etc.)

+ */

+struct darshan_pnetcdf_file

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int64_t counters[PNETCDF_NUM_INDICES];

+    double fcounters[PNETCDF_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_PNETCDF_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-8'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-8'>

<strong>

darshan-posix-log-format.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-posix-log-format.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,180 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ *  (C) 2009 by Argonne National Laboratory.

+ *      See COPYRIGHT in top-level directory.

+ */

+

+#ifndef __DARSHAN_POSIX_LOG_FORMAT_H

+#define __DARSHAN_POSIX_LOG_FORMAT_H

+

+/* current POSIX log format version */

+#define DARSHAN_POSIX_VER 1

+

+#define POSIX_COUNTERS \

+    /* count of posix opens */\

+    X(POSIX_OPENS) \

+    /* count of posix reads */\

+    X(POSIX_READS) \

+    /* count of posix writes */\

+    X(POSIX_WRITES) \

+    /* count of posix seeks */\

+    X(POSIX_SEEKS) \

+    /* count of posix stat/lstat/fstats */\

+    X(POSIX_STATS) \

+    /* count of posix mmaps */\

+    X(POSIX_MMAPS) \

+    /* count of posix fopens */\

+    X(POSIX_FOPENS) \

+    /* count of posix freads */\

+    X(POSIX_FREADS) \

+    /* count of posix fwrites */\

+    X(POSIX_FWRITES) \

+    /* count of posix fseeks */\

+    X(POSIX_FSEEKS) \

+    /* count of posix fsyncs */\

+    X(POSIX_FSYNCS) \

+    /* count of posix fdatasyncs */\

+    X(POSIX_FDSYNCS) \

+    /* mode of file */\

+    X(POSIX_MODE) \

+    /* total bytes read */\

+    X(POSIX_BYTES_READ) \

+    /* total bytes written */\

+    X(POSIX_BYTES_WRITTEN) \

+    /* highest offset byte read */\

+    X(POSIX_MAX_BYTE_READ) \

+    /* highest offset byte written */\

+    X(POSIX_MAX_BYTE_WRITTEN) \

+    /* count of consecutive reads */\

+    X(POSIX_CONSEC_READS) \

+    /* count of consecutive writes */\

+    X(POSIX_CONSEC_WRITES) \

+    /* count of sequential reads */\

+    X(POSIX_SEQ_READS) \

+    /* count of sequential writes */\

+    X(POSIX_SEQ_WRITES) \

+    /* number of times switched between read and write */\

+    X(POSIX_RW_SWITCHES) \

+    /* count of accesses not mem aligned */\

+    X(POSIX_MEM_NOT_ALIGNED) \

+    /* mem alignment in bytes */\

+    X(POSIX_MEM_ALIGNMENT) \

+    /* count of accesses not file aligned */\

+    X(POSIX_FILE_NOT_ALIGNED) \

+    /* file alignment in bytes */\

+    X(POSIX_FILE_ALIGNMENT) \

+    X(POSIX_MAX_READ_TIME_SIZE) \

+    X(POSIX_MAX_WRITE_TIME_SIZE) \

+    /* buckets for POSIX read size ranges */\

+    X(POSIX_SIZE_READ_0_100) \

+    X(POSIX_SIZE_READ_100_1K) \

+    X(POSIX_SIZE_READ_1K_10K) \

+    X(POSIX_SIZE_READ_10K_100K) \

+    X(POSIX_SIZE_READ_100K_1M) \

+    X(POSIX_SIZE_READ_1M_4M) \

+    X(POSIX_SIZE_READ_4M_10M) \

+    X(POSIX_SIZE_READ_10M_100M) \

+    X(POSIX_SIZE_READ_100M_1G) \

+    X(POSIX_SIZE_READ_1G_PLUS) \

+    /* buckets for POSIX write size ranges */\

+    X(POSIX_SIZE_WRITE_0_100) \

+    X(POSIX_SIZE_WRITE_100_1K) \

+    X(POSIX_SIZE_WRITE_1K_10K) \

+    X(POSIX_SIZE_WRITE_10K_100K) \

+    X(POSIX_SIZE_WRITE_100K_1M) \

+    X(POSIX_SIZE_WRITE_1M_4M) \

+    X(POSIX_SIZE_WRITE_4M_10M) \

+    X(POSIX_SIZE_WRITE_10M_100M) \

+    X(POSIX_SIZE_WRITE_100M_1G) \

+    X(POSIX_SIZE_WRITE_1G_PLUS) \

+    /* the four most frequently appearing strides */\

+    X(POSIX_STRIDE1_STRIDE) \

+    X(POSIX_STRIDE2_STRIDE) \

+    X(POSIX_STRIDE3_STRIDE) \

+    X(POSIX_STRIDE4_STRIDE) \

+    /* count of each of the most frequent strides */\

+    X(POSIX_STRIDE1_COUNT) \

+    X(POSIX_STRIDE2_COUNT) \

+    X(POSIX_STRIDE3_COUNT) \

+    X(POSIX_STRIDE4_COUNT) \

+    /* the four most frequently appearing access sizes */\

+    X(POSIX_ACCESS1_ACCESS) \

+    X(POSIX_ACCESS2_ACCESS) \

+    X(POSIX_ACCESS3_ACCESS) \

+    X(POSIX_ACCESS4_ACCESS) \

+    /* count of each of the most frequent access sizes */\

+    X(POSIX_ACCESS1_COUNT) \

+    X(POSIX_ACCESS2_COUNT) \

+    X(POSIX_ACCESS3_COUNT) \

+    X(POSIX_ACCESS4_COUNT) \

+    /* rank and number of bytes moved for fastest/slowest ranks */\

+    X(POSIX_FASTEST_RANK) \

+    X(POSIX_FASTEST_RANK_BYTES) \

+    X(POSIX_SLOWEST_RANK) \

+    X(POSIX_SLOWEST_RANK_BYTES) \

+    /* end of counters */\

+    X(POSIX_NUM_INDICES)

+

+#define POSIX_F_COUNTERS \

+    /* timestamp of first open */\

+    X(POSIX_F_OPEN_TIMESTAMP) \

+    /* timestamp of first read */\

+    X(POSIX_F_READ_START_TIMESTAMP) \

+    /* timestamp of first write */\

+    X(POSIX_F_WRITE_START_TIMESTAMP) \

+    /* timestamp of last read */\

+    X(POSIX_F_READ_END_TIMESTAMP) \

+    /* timestamp of last write */\

+    X(POSIX_F_WRITE_END_TIMESTAMP) \

+    /* timestamp of last close */\

+    X(POSIX_F_CLOSE_TIMESTAMP) \

+    /* cumulative posix read time */\

+    X(POSIX_F_READ_TIME) \

+    /* cumulative posix write time */\

+    X(POSIX_F_WRITE_TIME) \

+    /* cumulative posix meta time */\

+    X(POSIX_F_META_TIME) \

+    /* maximum posix read duration */\

+    X(POSIX_F_MAX_READ_TIME) \

+    /* maximum posix write duration */\

+    X(POSIX_F_MAX_WRITE_TIME) \

+    /* total i/o and meta time consumed for fastest/slowest ranks */\

+    X(POSIX_F_FASTEST_RANK_TIME) \

+    X(POSIX_F_SLOWEST_RANK_TIME) \

+    /* variance of total i/o time and bytes moved across all ranks */\

+    /* NOTE: for shared records only */\

+    X(POSIX_F_VARIANCE_RANK_TIME) \

+    X(POSIX_F_VARIANCE_RANK_BYTES) \

+    /* end of counters */\

+    X(POSIX_F_NUM_INDICES)

+

+#define X(a) a,

+/* integer statistics for POSIX file records */

+enum darshan_posix_indices

+{

+    POSIX_COUNTERS

+};

+

+/* floating point statistics for POSIX file records */

+enum darshan_posix_f_indices

+{

+    POSIX_F_COUNTERS

+};

+#undef X

+

+/* file record structure for POSIX files. a record is created and stored for

+ * every POSIX file opened by the original application. For the POSIX module,

+ * the record includes:

+ *      - a corresponding record identifier (created by hashing the file path)

+ *      - the rank of the process which opened the file (-1 for shared files)

+ *      - integer file I/O statistics (open, read/write counts, etc)

+ *      - floating point I/O statistics (timestamps, cumulative timers, etc.)

+ */

+struct darshan_posix_file

+{

+    darshan_record_id f_id;

+    int64_t rank;

+    int64_t counters[POSIX_NUM_INDICES];

+    double fcounters[POSIX_F_NUM_INDICES];

+};

+

+#endif /* __DARSHAN_POSIX_LOG_FORMAT_H */

</span></code></pre>

<br>

</li>

<li id='diff-9'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-9'>

<strong>

darshan-runtime/Makefile.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/Makefile.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/Makefile.in

</span><span style="color: #aaaaaa">@@ -1,4 +1,6 @@

</span><span style="color: #000000;background-color: #ffdddd">-all: lib/libdarshan-posix.a lib/libdarshan-mpi-io.a lib/libdarshan-stubs.a

</span><span style="color: #000000;background-color: #ddffdd">+all: lib/libdarshan.a lib/libdarshan-stubs.a lib/darshan-null.o

+

+#TODO: each module provides own makefile with module-specific objects, build options, etc.

</span> 

 DESTDIR =

 srcdir = @srcdir@

<span style="color: #aaaaaa">@@ -15,60 +17,95 @@ CC = @CC@

</span> LD = @LD@

 DISABLE_LDPRELOAD = @DISABLE_LDPRELOAD@

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_USE_BGQ = @DARSHAN_USE_BGQ@

</span> DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h

 DARSHAN_VERSION = @DARSHAN_VERSION@

 ifndef DISABLE_LDPRELOAD

<span style="color: #000000;background-color: #ffdddd">-all: lib/libdarshan.so 

</span><span style="color: #000000;background-color: #ddffdd">+all: lib/libdarshan.so lib/darshan-null.po

</span> endif

 VPATH = $(srcdir)

<span style="color: #000000;background-color: #ffdddd">-CFLAGS = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I ../ -I $(srcdir) -I$(srcdir)/../ @CFLAGS@ @CPPFLAGS@ -D_LARGEFILE64_SOURCE 

</span><span style="color: #000000;background-color: #ddffdd">+CFLAGS = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I ../ -I $(srcdir) -I$(srcdir)/../ @CFLAGS@ @CPPFLAGS@ -D_LARGEFILE64_SOURCE

</span> 

 CFLAGS_SHARED = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I$(srcdir) -I$(srcdir)/../ @CFLAGS@ @CPPFLAGS@ -D_LARGEFILE64_SOURCE -shared -fpic -DPIC -DDARSHAN_PRELOAD

 LIBS = -lz @LIBBZ2@

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o

+DARSHAN_DYNAMIC_MOD_OBJS = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po

+

+ifdef DARSHAN_USE_BGQ

+DARSHAN_STATIC_MOD_OBJS += lib/darshan-bgq.o

+DARSHAN_DYNAMIC_MOD_OBJS += lib/darshan-bgq.po

+CFLAGS += -DDARSHAN_BGQ

+CFLAGS_SHARED += -DDARSHAN_BGQ

+endif

+

</span> lib::

        @mkdir -p $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-mpi-io.o: lib/darshan-mpi-io.c darshan.h darshan-dynamic.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-core-init-finalize.o: lib/darshan-core-init-finalize.c darshan.h darshan-core.h $(DARSHAN_LOG_FORMAT) | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-mpi-io.po: lib/darshan-mpi-io.c darshan.h darshan-dynamic.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-core-init-finalize.po: lib/darshan-core-init-finalize.c darshan.h darshan-core.h $(DARSHAN_LOG_FORMAT) | lib

</span>   $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-mpi-init-finalize.o: lib/darshan-mpi-init-finalize.c darshan.h darshan-dynamic.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-core.o: lib/darshan-core.c darshan.h darshan-core.h $(DARSHAN_LOG_FORMAT) | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-mpi-init-finalize.po: lib/darshan-mpi-init-finalize.c darshan.h darshan-dynamic.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-core.po: lib/darshan-core.c darshan.h darshan-core.h $(DARSHAN_LOG_FORMAT) | lib

</span>   $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-pnetcdf-stubs.o: lib/darshan-pnetcdf-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib 

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-common.o: lib/darshan-common.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-pnetcdf.o: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) | lib 

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-common.po: lib/darshan-common.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

+       $(CC) $(CFLAGS_SHARED) -c $< -o $@

+

+lib/darshan-null.o: lib/darshan-null.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-null-log-format.h | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-pnetcdf.po: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) | lib 

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-null.po: lib/darshan-null.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-null-log-format.h | lib

</span>   $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-hdf5.o: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-posix.o: lib/darshan-posix.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-posix-log-format.h | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-hdf5-stubs.o: lib/darshan-hdf5-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-posix.po: lib/darshan-posix.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-posix-log-format.h | lib

+       $(CC) $(CFLAGS_SHARED) -c $< -o $@

+

+lib/darshan-mpiio.o: lib/darshan-mpiio.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | lib

+       $(CC) $(CFLAGS) -c $< -o $@

+

+lib/darshan-mpiio.po: lib/darshan-mpiio.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | lib

+       $(CC) $(CFLAGS_SHARED) -c $< -o $@

+

+lib/darshan-bgq.o: lib/darshan-bgq.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-hdf5.po: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-bgq.po: lib/darshan-bgq.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | lib

</span>   $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-posix.o: lib/darshan-posix.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-hdf5.o: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | lib

</span>   $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-lib/darshan-posix.po: lib/darshan-posix.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

</span><span style="color: #000000;background-color: #ddffdd">+lib/darshan-hdf5.po: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | lib

</span>   $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ddffdd">+lib/darshan-hdf5-stubs.o: lib/darshan-hdf5-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

+       $(CC) $(CFLAGS) -c $< -o $@

+

+lib/darshan-pnetcdf.o: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | lib

+       $(CC) $(CFLAGS) -c $< -o $@

+

+lib/darshan-pnetcdf.po: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | lib

+       $(CC) $(CFLAGS_SHARED) -c $< -o $@

+

+lib/darshan-pnetcdf-stubs.o: lib/darshan-pnetcdf-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib

+       $(CC) $(CFLAGS) -c $< -o $@

+

</span> lib/lookup3.o: lib/lookup3.c

        $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #aaaaaa">@@ -81,25 +118,19 @@ lib/lookup8.o: lib/lookup8.c

</span> lib/lookup8.po: lib/lookup8.c

        $(CC) $(CFLAGS_SHARED) -c $< -o $@

<span style="color: #000000;background-color: #ffdddd">-%.i: %.c

-       $(CC) -E $(CFLAGS) -c $< -o $@

-

-lib/libdarshan-mpi-io.a: lib/darshan-mpi-io.o lib/darshan-mpi-init-finalize.o lib/darshan-pnetcdf.o lib/darshan-hdf5.o

</span><span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o $(DARSHAN_STATIC_MOD_OBJS) lib/lookup3.o lib/lookup8.o

</span>   ar rcs $@ $^

<span style="color: #000000;background-color: #ffdddd">-lib/libdarshan-stubs.a: lib/darshan-pnetcdf-stubs.o lib/darshan-hdf5-stubs.o

-       ar rcs $@ $^

</span><span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po $(DARSHAN_DYNAMIC_MOD_OBJS) lib/lookup3.po lib/lookup8.po

+       $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ -lpthread -lrt -lz -ldl

</span> 

<span style="color: #000000;background-color: #ffdddd">-lib/libdarshan-posix.a: lib/darshan-posix.o lib/lookup3.o lib/lookup8.o 

</span><span style="color: #000000;background-color: #ddffdd">+lib/libdarshan-stubs.a: lib/darshan-hdf5-stubs.o lib/darshan-pnetcdf-stubs.o

</span>   ar rcs $@ $^

<span style="color: #000000;background-color: #ffdddd">-lib/libdarshan.so: lib/darshan-mpi-io.po lib/darshan-mpi-init-finalize.po lib/darshan-pnetcdf.po lib/darshan-hdf5.po lib/darshan-posix.po lib/lookup3.po lib/lookup8.po

-       $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ -lpthread -lrt -lz -ldl

</span> 

 install:: all

        install -d $(libdir)

<span style="color: #000000;background-color: #ffdddd">-        install -m 755 lib/libdarshan-posix.a $(libdir)

-       install -m 755 lib/libdarshan-mpi-io.a $(libdir)

</span><span style="color: #000000;background-color: #ddffdd">+   install -m 755 lib/libdarshan.a $(libdir)

</span>   install -m 755 lib/libdarshan-stubs.a $(libdir)

 ifndef DISABLE_LDPRELOAD

        install -m 755 lib/libdarshan.so $(libdir)

<span style="color: #aaaaaa">@@ -121,6 +152,11 @@ endif

</span>   install -m 755 share/mpi-profile/darshan-bg-cc.conf $(datarootdir)/mpi-profile/darshan-bg-cc.conf

        install -m 755 share/mpi-profile/darshan-bg-cxx.conf $(datarootdir)/mpi-profile/darshan-bg-cxx.conf

        install -m 755 share/mpi-profile/darshan-bg-f.conf $(datarootdir)/mpi-profile/darshan-bg-f.conf

<span style="color: #000000;background-color: #ddffdd">+        install -d $(datarootdir)/ld-opts

+       install -m 644 share/ld-opts/darshan-base-ld-opts $(datarootdir)/ld-opts/darshan-base-ld-opts

+       install -m 644 $(srcdir)/share/ld-opts/darshan-posix-ld-opts $(datarootdir)/ld-opts/darshan-posix-ld-opts

+       install -m 644 $(srcdir)/share/ld-opts/darshan-hdf5-ld-opts $(datarootdir)/ld-opts/darshan-hdf5-ld-opts

+       install -m 644 $(srcdir)/share/ld-opts/darshan-pnetcdf-ld-opts $(datarootdir)/ld-opts/darshan-pnetcdf-ld-opts

</span>   install -d $(libdir)/pkgconfig

        install -m 644 lib/pkgconfig/darshan-runtime.pc $(libdir)/pkgconfig/darshan-runtime.pc

<span style="color: #aaaaaa">@@ -128,5 +164,5 @@ clean::

</span>   rm -f *.o *.a lib/*.o lib/*.po lib/*.a lib/*.so 

 distclean:: clean

<span style="color: #000000;background-color: #ffdddd">-        rm -f darshan-runtime-config.h darshan-gen-cxx.pl darshan-gen-fortran.pl darshan-gen-cc.pl darshan-mk-log-dirs.pl aclocal.m4 autom4te.cache/* config.status config.log Makefile 

</span><span style="color: #000000;background-color: #ddffdd">+   rm -f darshan-runtime-config.h darshan-gen-cxx.pl darshan-gen-fortran.pl darshan-gen-cc.pl darshan-mk-log-dirs.pl darshan-config darshan-base-ld-opts aclocal.m4 autom4te.cache/* config.status config.log Makefile 

</span>   rm -rf autom4te.cache

</code></pre>

<br>

</li>

<li id='diff-10'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-10'>

<strong>

darshan-runtime/configure

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/configure

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/configure

</span><span style="color: #aaaaaa">@@ -1,6 +1,6 @@

</span> #! /bin/sh

 # Guess values for system-dependent variables and create Makefiles.

<span style="color: #000000;background-color: #ffdddd">-# Generated by GNU Autoconf 2.69 for darshan-runtime 2.3.2-pre1.

<span style="color: #000000;background-color: #ddffdd">+# Generated by GNU Autoconf 2.69 for darshan-runtime 3.0.0-pre3.

</span> #

 #

 # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.

<span style="color: #aaaaaa">@@ -577,8 +577,8 @@ MAKEFLAGS=

# Identity of this package.

 PACKAGE_NAME='darshan-runtime'

 PACKAGE_TARNAME='darshan-runtime'

<span style="color: #000000;background-color: #ffdddd">-PACKAGE_VERSION='2.3.2-pre1'

-PACKAGE_STRING='darshan-runtime 2.3.2-pre1'

</span><span style="color: #000000;background-color: #ddffdd">+PACKAGE_VERSION='3.0.0-pre3'

+PACKAGE_STRING='darshan-runtime 3.0.0-pre3'

</span> PACKAGE_BUGREPORT=''

 PACKAGE_URL=''

<span style="color: #aaaaaa">@@ -621,12 +621,12 @@ ac_includes_default="\

</span> 

 ac_subst_vars='LTLIBOBJS

 LIBOBJS

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_USE_BGQ

</span> MPICH_LIB_OLD

 DARSHAN_VERSION

 DISABLE_LDPRELOAD

<span style="color: #000000;background-color: #ffdddd">-CP_WRAPPERS

-__CP_LOG_PATH

-darshan_bin_path

</span><span style="color: #000000;background-color: #ddffdd">+__DARSHAN_LOG_PATH

+darshan_share_path

</span> darshan_lib_path

 PRI_MACROS_BROKEN

 EGREP

<span style="color: #aaaaaa">@@ -683,17 +683,17 @@ SHELL'

</span> ac_subst_files=''

 ac_user_opts='

 enable_option_checking

<span style="color: #000000;background-color: #ffdddd">-with_zlib_for_mpi

</span> with_zlib

 enable_cuserid

 enable_ld_preload

 enable_group_readable_logs

<span style="color: #000000;background-color: #ffdddd">-enable_stat_at_open

</span> with_mem_align

 with_log_path_by_env

 with_log_hints

 with_log_path

 with_jobid_env

<span style="color: #000000;background-color: #ddffdd">+enable_bgq_mod

+with_mod_mem

</span> '

       ac_precious_vars='build_alias

 host_alias

<span style="color: #aaaaaa">@@ -1244,7 +1244,7 @@ if test "$ac_init_help" = "long"; then

# Omit some internal or obsolete options to make the list less imposing.

   # This message is too long to be a string in the A/UX 3.1 sh.

   cat <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-\`configure' configures darshan-runtime 2.3.2-pre1 to adapt to many kinds of systems.

<span style="color: #000000;background-color: #ddffdd">+\`configure' configures darshan-runtime 3.0.0-pre3 to adapt to many kinds of systems.

</span> 

 Usage: $0 [OPTION]... [VAR=VALUE]...

<span style="color: #aaaaaa">@@ -1305,7 +1305,7 @@ fi

</span> 

 if test -n "$ac_init_help"; then

   case $ac_init_help in

<span style="color: #000000;background-color: #ffdddd">-     short | recursive ) echo "Configuration of darshan-runtime 2.3.2-pre1:";;

</span><span style="color: #000000;background-color: #ddffdd">+     short | recursive ) echo "Configuration of darshan-runtime 3.0.0-pre3:";;

</span>    esac

   cat <<\_ACEOF

<span style="color: #aaaaaa">@@ -1316,22 +1316,22 @@ Optional Features:

</span>   --disable-cuserid       Disables attempted use of cuserid() at run time

   --disable-ld-preload    Disables support for LD_PRELOAD library

   --enable-group-readable-logs Set log files to be group readable

<span style="color: #000000;background-color: #ffdddd">-  --enable-stat-at-open   Perform stat() call at open time to gather extra statistics

</span><span style="color: #000000;background-color: #ddffdd">+  --disable-bgq-mod       Disables compilation and use of BG/Q module (for BG/Q systems)

</span> 

 Optional Packages:

   --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]

   --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)

<span style="color: #000000;background-color: #ffdddd">-

</span>   --with-zlib=DIR root directory path of zlib installation defaults to

                     /usr/local or /usr if not found in /usr/local

   --without-zlib to disable zlib usage completely

   --with-mem-align=<num>  Memory alignment in bytes

   --with-log-path-by-env=<env var list> Comma separated list of environment variables to check for log path location before --with-log-path path

<span style="color: #000000;background-color: #ffdddd">-  --with-log-hints=<hint1=x,hint2=y>  Semicolon-separated list of MPI-IO hints for log file write

</span><span style="color: #000000;background-color: #ddffdd">+  --with-log-hints=<hint1=x;hint2=y>  Semicolon-separated list of MPI-IO hints for log file write

</span>   --with-log-path=<path>  Location to store log files at run time

   --with-jobid-env=<name> Name of environment variable that stores the jobid

     (specify "NONE" if no appropriate environment variable is available:

     Darshan will use rank 0's pid instead)

<span style="color: #000000;background-color: #ddffdd">+  --with-mod-mem=<num>  Maximum amount of memory (in MiB) for each Darshan module

</span> 

 Some influential environment variables:

   CC          C compiler command

<span style="color: #aaaaaa">@@ -1409,7 +1409,7 @@ fi

</span> test -n "$ac_init_help" && exit $ac_status

 if $ac_init_version; then

   cat <<\_ACEOF

<span style="color: #000000;background-color: #ffdddd">-darshan-runtime configure 2.3.2-pre1

</span><span style="color: #000000;background-color: #ddffdd">+darshan-runtime configure 3.0.0-pre3

</span> generated by GNU Autoconf 2.69

 Copyright (C) 2012 Free Software Foundation, Inc.

<span style="color: #aaaaaa">@@ -1761,7 +1761,7 @@ cat >config.log <<_ACEOF

</span> This file contains any messages produced by compilers while

 running configure, to aid debugging if configure makes a mistake.

<span style="color: #000000;background-color: #ffdddd">-It was created by darshan-runtime $as_me 2.3.2-pre1, which was

</span><span style="color: #000000;background-color: #ddffdd">+It was created by darshan-runtime $as_me 3.0.0-pre3, which was

</span> generated by GNU Autoconf 2.69.  Invocation command line was

   $ $0 $@

<span style="color: #aaaaaa">@@ -3050,15 +3050,6 @@ $as_echo "no" >&6; }

</span> fi

 rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext

<span style="color: #000000;background-color: #ffdddd">-

-# Check whether --with-zlib-for-mpi was given.

-if test "${with_zlib_for_mpi+set}" = set; then :

-  withval=$with_zlib_for_mpi; as_fn_error $? "with-zlib-for-mpi argument is deprecated.  Please see Darshan documentation." "$LINENO" 5

-

-fi

-

-

-

</span> ac_ext=c

 ac_cpp='$CPP $CPPFLAGS'

 ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'

<span style="color: #aaaaaa">@@ -3467,8 +3458,8 @@ if test "${with_zlib+set}" = set; then :

</span>     ZLIB_HOME="$withval"

     LDFLAGS="$LDFLAGS -L${ZLIB_HOME}/lib"

     CPPFLAGS="$CPPFLAGS -I${ZLIB_HOME}/include"

<span style="color: #000000;background-color: #ffdddd">-    __CP_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

-    __CP_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span><span style="color: #000000;background-color: #ddffdd">+    __DARSHAN_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

+    __DARSHAN_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span>   else

     { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Sorry, $withval does not exist, checking usual places" >&5

 $as_echo "$as_me: WARNING: Sorry, $withval does not exist, checking usual places" >&2;}

<span style="color: #aaaaaa">@@ -3542,7 +3533,7 @@ if test "${enable_cuserid+set}" = set; then :

</span>   enableval=$enable_cuserid; if test "x$enableval" = "xno" ; then

 $as_echo "#define DARSHAN_DISABLE_CUSERID 1" >>confdefs.h

<span style="color: #000000;background-color: #ffdddd">-,

</span><span style="color: #000000;background-color: #ddffdd">+

</span> fi

 fi

<span style="color: #aaaaaa">@@ -3561,18 +3552,7 @@ fi

</span> if test "${enable_group_readable_logs+set}" = set; then :

   enableval=$enable_group_readable_logs; if test "x$enableval" = "xyes" ; then

<span style="color: #000000;background-color: #ffdddd">-$as_echo "#define __CP_GROUP_READABLE_LOGS 1" >>confdefs.h

-

-fi

-

-fi

-

-

-# Check whether --enable-stat-at-open was given.

-if test "${enable_stat_at_open+set}" = set; then :

-  enableval=$enable_stat_at_open; if test "x$enableval" = "xyes" ; then

-

-$as_echo "#define __CP_STAT_AT_OPEN 1" >>confdefs.h

</span><span style="color: #000000;background-color: #ddffdd">+$as_echo "#define __DARSHAN_GROUP_READABLE_LOGS 1" >>confdefs.h

</span> 

 fi

<span style="color: #aaaaaa">@@ -3587,7 +3567,7 @@ if test "${with_mem_align+set}" = set; then :

</span>     else

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define __CP_MEM_ALIGNMENT ${withval}

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_MEM_ALIGNMENT ${withval}

</span> _ACEOF

         GOT_ALIGNMENT=1

<span style="color: #aaaaaa">@@ -3604,10 +3584,10 @@ if test "${with_log_path_by_env+set}" = set; then :

</span>     else

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define __CP_LOG_ENV "${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_LOG_ENV "${withval}"

</span> _ACEOF

<span style="color: #000000;background-color: #ffdddd">-        __CP_LOG_ENV="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_LOG_ENV="${withval}"

</span>         GOT_LOG_PATH=1

     fi

<span style="color: #aaaaaa">@@ -3622,10 +3602,10 @@ if test "${with_log_hints+set}" = set; then :

</span>     else

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define __CP_LOG_HINTS "${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_LOG_HINTS "${withval}"

</span> _ACEOF

<span style="color: #000000;background-color: #ffdddd">-        __CP_LOG_HINTS="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_LOG_HINTS="${withval}"

</span>         GOT_LOG_HINTS=1

     fi

<span style="color: #aaaaaa">@@ -3640,10 +3620,10 @@ if test "${with_log_path+set}" = set; then :

</span>     else

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define __CP_LOG_PATH "${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_LOG_PATH "${withval}"

</span> _ACEOF

<span style="color: #000000;background-color: #ffdddd">-        __CP_LOG_PATH="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_LOG_PATH="${withval}"

</span>         GOT_LOG_PATH=1

     fi

<span style="color: #aaaaaa">@@ -3658,10 +3638,10 @@ if test "${with_jobid_env+set}" = set; then :

</span>     else

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define CP_JOBID "${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_JOBID "${withval}"

</span> _ACEOF

<span style="color: #000000;background-color: #ffdddd">-        CP_JOBID="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_JOBID="${withval}"

</span>         GOT_JOBID=1

     fi

<span style="color: #aaaaaa">@@ -3677,11 +3657,11 @@ fi

</span> if test x$GOT_JOBID != x1; then

     as_fn_error $? "must provide --with-jobid-env=<name> argument to configure." "$LINENO" 5

 fi

<span style="color: #000000;background-color: #ffdddd">-__CP_LOG_HINTS_DEFAULT="romio_no_indep_rw=true;cb_nodes=4"

</span><span style="color: #000000;background-color: #ddffdd">+__DARSHAN_LOG_HINTS_DEFAULT="romio_no_indep_rw=true;cb_nodes=4"

</span> if test x$GOT_LOG_HINTS != x1; then

 cat >>confdefs.h <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-#define __CP_LOG_HINTS "$__CP_LOG_HINTS_DEFAULT"

</span><span style="color: #000000;background-color: #ddffdd">+#define __DARSHAN_LOG_HINTS "$__DARSHAN_LOG_HINTS_DEFAULT"

</span> _ACEOF

 fi

<span style="color: #aaaaaa">@@ -4079,11 +4059,6 @@ fi

</span> done

<span style="color: #000000;background-color: #ffdddd">-# libc functions wrapped by darshan

-CP_WRAPPERS="-Wl,-u,MPI_Init,-u,MPI_Wtime,-wrap,write,-wrap,open,-wrap,creat,-wrap,creat64,-wrap,open64,-wrap,close,-wrap,read,-wrap,lseek,-wrap,lseek64,-wrap,pread,-wrap,pwrite,-wrap,readv,-wrap,writev,-wrap,__xstat,-wrap,__lxstat,-wrap,__fxstat,-wrap,__xstat64,-wrap,__lxstat64,-wrap,__fxstat64,-wrap,mmap,-wrap,mmap64,-wrap,fopen,-wrap,fclose,-wrap,fread,-wrap,fwrite,-wrap,fseek,-wrap,fopen64,-wrap,pread64,-wrap,pwrite64,-wrap,fsync,-wrap,fdatasync,-wrap,ncmpi_create,-wrap,ncmpi_open,-wrap,ncmpi_close,-wrap,H5Fcreate,-wrap,H5Fopen,-wrap,H5Fclose,-wrap,aio_write,-wrap,aio_write64,-wrap,aio_read,-wrap,aio_read64,-wrap,lio_listio,-wrap,lio_listio64,-wrap,aio_return,-wrap,aio_return64,-wrap,mkstemp,-wrap,mkostemp,-wrap,mkstemps,-wrap,mkostemps"

-

-# ,-u,__wrap___fxstat64,-u,pthread_mutex_lock,-u,pthread_mutex_unlock

-

</span> # We need to know the value of the $libdir and $bindir variables so that

 # we can reference the correct path in the darshan compiler wrappers.

 # Unfortunately, those two variables are not normally evaluated by autoconf.

<span style="color: #aaaaaa">@@ -4099,7 +4074,7 @@ saveExecprefix=$exec_prefix

</span> test "x$prefix" = xNONE && prefix=$ac_default_prefix

 test "x$exec_prefix" = xNONE && exec_prefix=$prefix

 eval darshan_lib_path=$libdir

<span style="color: #000000;background-color: #ffdddd">-eval darshan_bin_path=$bindir

</span><span style="color: #000000;background-color: #ddffdd">+eval darshan_share_path=$datarootdir

</span> prefix=$savePrefix

 exec_prefix=$saveExecprefix

<span style="color: #aaaaaa">@@ -4251,21 +4226,81 @@ $as_echo "#define __D_MPI_REQUEST MPI_Request" >>confdefs.h

</span> fi

<span style="color: #000000;background-color: #ffdddd">-# attempt to detect librarly naming convention in mpi compiler script

-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for old (pre-3.1.1) style MPICH library naming convention" >&5

</span><span style="color: #000000;background-color: #ddffdd">+# attempt to detect library naming convention in mpi compiler script

+if $CC -show foo.c -o foo >& /dev/null; then :

+  { $as_echo "$as_me:${as_lineno-$LINENO}: checking for old (pre-3.1.1) style MPICH library naming convention" >&5

</span> $as_echo_n "checking for old (pre-3.1.1) style MPICH library naming convention... " >&6; }

<span style="color: #000000;background-color: #ffdddd">-if $CC -show foo.c -o foo |grep lmpich >& /dev/null; then :

</span><span style="color: #000000;background-color: #ddffdd">+   if $CC -show foo.c -o foo |grep lmpich >& /dev/null; then :

+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5

+$as_echo "yes" >&6; }

+           MPICH_LIB_OLD=1

+else

+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5

+$as_echo "no" >&6; }

+           MPICH_LIB_OLD=0

+fi

+else

+  MPICH_LIB_OLD=0

+fi

+

+# check to see whether the bgq instrumentation module should be built

+# Check whether --enable-bgq_mod was given.

+if test "${enable_bgq_mod+set}" = set; then :

+  enableval=$enable_bgq_mod;

+else

+  enable_bgq_mod=check

+

+fi

+

+if test x$enable_bgq_mod != xno; then

+    { $as_echo "$as_me:${as_lineno-$LINENO}: checking for BG/Q environment" >&5

+$as_echo_n "checking for BG/Q environment... " >&6; }

+    cat confdefs.h - <<_ACEOF >conftest.$ac_ext

+/* end confdefs.h.  */

+

+        #ifndef __bgq__

+        #error __bgq__ not set

+        #endif

+

+int

+main ()

+{

+

+  ;

+  return 0;

+}

+_ACEOF

+if ac_fn_c_try_compile "$LINENO"; then :

</span>   { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5

 $as_echo "yes" >&6; }

<span style="color: #000000;background-color: #ffdddd">-    MPICH_LIB_OLD=1

</span><span style="color: #000000;background-color: #ddffdd">+        DARSHAN_USE_BGQ=1

</span> else

   { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5

 $as_echo "no" >&6; }

<span style="color: #000000;background-color: #ffdddd">-    MPICH_LIB_OLD=0

</span><span style="color: #000000;background-color: #ddffdd">+        if test "x$enable_bgq_mod" = xyes; then :

+  as_fn_error $? "BG/Q module enabled in non-BG/Q environment" "$LINENO" 5

+fi

+fi

+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext

+fi

+

+

+# Check whether --with-mod-mem was given.

+if test "${with_mod_mem+set}" = set; then :

+  withval=$with_mod_mem; if test x$withval = xyes; then

+        as_fn_error $? "--with-mod-mem must be given a number" "$LINENO" 5

+    else

+

+cat >>confdefs.h <<_ACEOF

+#define __DARSHAN_MOD_MEM_MAX ${withval}

+_ACEOF

+

+    fi

+

</span> fi

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_VERSION="2.3.2-pre1"

</span> 

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_VERSION="3.0.0-pre3"

</span> 

<span style="color: #aaaaaa">@@ -4274,7 +4309,7 @@ DARSHAN_VERSION="2.3.2-pre1"

</span> 

<span style="color: #000000;background-color: #ffdddd">-ac_config_files="$ac_config_files Makefile darshan-mk-log-dirs.pl darshan-gen-cc.pl darshan-gen-cxx.pl darshan-gen-fortran.pl darshan-config share/craype-1.x/darshan-module share/craype-2.x/darshan-module lib/pkgconfig/darshan-runtime.pc share/mpi-profile/darshan-cc.conf share/mpi-profile/darshan-cxx.conf share/mpi-profile/darshan-f.conf share/mpi-profile/darshan-bg-cc.conf share/mpi-profile/darshan-bg-cxx.conf share/mpi-profile/darshan-bg-f.conf"

</span><span style="color: #000000;background-color: #ddffdd">+ac_config_files="$ac_config_files Makefile darshan-mk-log-dirs.pl darshan-gen-cc.pl darshan-gen-cxx.pl darshan-gen-fortran.pl darshan-config share/craype-1.x/darshan-module share/craype-2.x/darshan-module lib/pkgconfig/darshan-runtime.pc share/mpi-profile/darshan-cc.conf share/mpi-profile/darshan-cxx.conf share/mpi-profile/darshan-f.conf share/mpi-profile/darshan-bg-cc.conf share/mpi-profile/darshan-bg-cxx.conf share/mpi-profile/darshan-bg-f.conf share/ld-opts/darshan-base-ld-opts"

</span> 

 cat >confcache <<\_ACEOF

 # This file is a shell script that caches the results of configure

<span style="color: #aaaaaa">@@ -4783,7 +4818,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1

</span> # report actual input values of CONFIG_FILES etc. instead of their

 # values after options handling.

 ac_log="

<span style="color: #000000;background-color: #ffdddd">-This file was extended by darshan-runtime $as_me 2.3.2-pre1, which was

</span><span style="color: #000000;background-color: #ddffdd">+This file was extended by darshan-runtime $as_me 3.0.0-pre3, which was

</span> generated by GNU Autoconf 2.69.  Invocation command line was

   CONFIG_FILES    = $CONFIG_FILES

<span style="color: #aaaaaa">@@ -4845,7 +4880,7 @@ _ACEOF

</span> cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1

 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"

 ac_cs_version="\\

<span style="color: #000000;background-color: #ffdddd">-darshan-runtime config.status 2.3.2-pre1

</span><span style="color: #000000;background-color: #ddffdd">+darshan-runtime config.status 3.0.0-pre3

</span> configured by $0, generated by GNU Autoconf 2.69,

   with options \\"\$ac_cs_config\\"

<span style="color: #aaaaaa">@@ -4983,6 +5018,7 @@ do

</span>     "share/mpi-profile/darshan-bg-cc.conf") CONFIG_FILES="$CONFIG_FILES share/mpi-profile/darshan-bg-cc.conf" ;;

     "share/mpi-profile/darshan-bg-cxx.conf") CONFIG_FILES="$CONFIG_FILES share/mpi-profile/darshan-bg-cxx.conf" ;;

     "share/mpi-profile/darshan-bg-f.conf") CONFIG_FILES="$CONFIG_FILES share/mpi-profile/darshan-bg-f.conf" ;;

<span style="color: #000000;background-color: #ddffdd">+    "share/ld-opts/darshan-base-ld-opts") CONFIG_FILES="$CONFIG_FILES share/ld-opts/darshan-base-ld-opts" ;;

</span> 

   *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;;

   esac

</code></pre>

<br>

</li>

<li id='diff-11'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-11'>

<strong>

darshan-runtime/configure.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/configure.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/configure.in

<span style="color: #aaaaaa">@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.

</span> dnl You may need to use autoheader as well if changing any DEFINEs

 dnl sanity checks, output header, location of scripts used here

<span style="color: #000000;background-color: #ffdddd">-AC_INIT([darshan-runtime], [2.3.2-pre1])

</span><span style="color: #000000;background-color: #ddffdd">+AC_INIT([darshan-runtime], [3.0.0-pre3])

</span> AC_CONFIG_SRCDIR([darshan.h])

 AC_CONFIG_AUX_DIR(../maint/config)

 AC_CONFIG_HEADER(darshan-runtime-config.h)

<span style="color: #aaaaaa">@@ -21,18 +21,13 @@ AC_TRY_COMPILE([#include <mpi.h>], [int ret = MPI_Init(0, (void*)0)],

</span>     AC_MSG_ERROR(CC doesn't appear to be a valid MPI compiler.  See INSTALL document or try adding CC=mpicc to your configure command line.)

 )

<span style="color: #000000;background-color: #ffdddd">-AC_ARG_WITH(zlib-for-mpi,,

-    AC_MSG_ERROR(with-zlib-for-mpi argument is deprecated.  Please see Darshan documentation.)

-)

-

-

</span> dnl runtime libraries require zlib

 CHECK_ZLIB

 AC_ARG_ENABLE(cuserid, 

 [  --disable-cuserid       Disables attempted use of cuserid() at run time], 

 [if test "x$enableval" = "xno" ; then

<span style="color: #000000;background-color: #ffdddd">-    AC_DEFINE(DARSHAN_DISABLE_CUSERID, 1, Define if cuserid() should be disabled),

</span><span style="color: #000000;background-color: #ddffdd">+    AC_DEFINE(DARSHAN_DISABLE_CUSERID, 1, Define if cuserid() should be disabled)

</span> fi]

 ,)

<span style="color: #aaaaaa">@@ -46,14 +41,7 @@ fi]

</span> AC_ARG_ENABLE(group-readable-logs, 

 [  --enable-group-readable-logs Set log files to be group readable], 

 [if test "x$enableval" = "xyes" ; then

<span style="color: #000000;background-color: #ffdddd">-    AC_DEFINE(__CP_GROUP_READABLE_LOGS, 1, Define if Darshan should set log files to be group readable)

-fi]

-,)

-

-AC_ARG_ENABLE(stat-at-open, 

-[  --enable-stat-at-open   Perform stat() call at open time to gather extra statistics], 

-[if test "x$enableval" = "xyes" ; then

-    AC_DEFINE(__CP_STAT_AT_OPEN, 1, Define if Darshan should stat files at open time to collect extra statistics)

</span><span style="color: #000000;background-color: #ddffdd">+    AC_DEFINE(__DARSHAN_GROUP_READABLE_LOGS, 1, Define if Darshan should set log files to be group readable)

</span> fi]

 ,)

<span style="color: #aaaaaa">@@ -62,7 +50,7 @@ AC_ARG_WITH(mem-align,

</span>     if test x$withval = xyes; then

         AC_MSG_ERROR(--with-mem-align must be given a number in bytes)

     else

<span style="color: #000000;background-color: #ffdddd">-        AC_DEFINE_UNQUOTED(__CP_MEM_ALIGNMENT, ${withval}, Memory alignment in bytes)

</span><span style="color: #000000;background-color: #ddffdd">+        AC_DEFINE_UNQUOTED(__DARSHAN_MEM_ALIGNMENT, ${withval}, Memory alignment in bytes)

</span>         GOT_ALIGNMENT=1

     fi

 )

<span style="color: #aaaaaa">@@ -72,19 +60,19 @@ AC_ARG_WITH(log-path-by-env,

</span>     if test x$withval = xyes; then

         AC_MSG_ERROR(--with-log-path-by-env must be given at least one variable name)

     else

<span style="color: #000000;background-color: #ffdddd">-        AC_DEFINE_UNQUOTED(__CP_LOG_ENV, "${withval}", Comma separated list of env. variables to use for log path)

-        __CP_LOG_ENV="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        AC_DEFINE_UNQUOTED(__DARSHAN_LOG_ENV, "${withval}", Comma separated list of env. variables to use for log path)

+        __DARSHAN_LOG_ENV="${withval}"

</span>         GOT_LOG_PATH=1

     fi

 )

 AC_ARG_WITH(log-hints,

<span style="color: #000000;background-color: #ffdddd">-[  --with-log-hints=<hint1=x,hint2=y>  Semicolon-separated list of MPI-IO hints for log file write],

</span><span style="color: #000000;background-color: #ddffdd">+[  --with-log-hints=<hint1=x;hint2=y>  Semicolon-separated list of MPI-IO hints for log file write],

</span>     if test x$withval = xyes; then

         AC_MSG_ERROR(--with-log-hints must be given an argument)

     else

<span style="color: #000000;background-color: #ffdddd">-        AC_DEFINE_UNQUOTED(__CP_LOG_HINTS, "${withval}", Semicolon-separated list of MPI-IO hints for log file write)

-        __CP_LOG_HINTS="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        AC_DEFINE_UNQUOTED(__DARSHAN_LOG_HINTS, "${withval}", Semicolon-separated list of MPI-IO hints for log file write)

+        __DARSHAN_LOG_HINTS="${withval}"

</span>         GOT_LOG_HINTS=1

     fi

 )

<span style="color: #aaaaaa">@@ -94,8 +82,8 @@ AC_ARG_WITH(log-path,

</span>     if test x$withval = xyes; then

         AC_MSG_ERROR(--with-log-path must be given a pathname)

     else

<span style="color: #000000;background-color: #ffdddd">-        AC_DEFINE_UNQUOTED(__CP_LOG_PATH, "${withval}", Location to store log files at run time)

-        __CP_LOG_PATH="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        AC_DEFINE_UNQUOTED(__DARSHAN_LOG_PATH, "${withval}", Location to store log files at run time)

+        __DARSHAN_LOG_PATH="${withval}"

</span>         GOT_LOG_PATH=1

     fi

 )

<span style="color: #aaaaaa">@@ -107,8 +95,8 @@ AC_ARG_WITH(jobid-env,

</span>     if test x$withval = xyes; then

         AC_MSG_ERROR(--with-jobid-env must be given a name)

     else

<span style="color: #000000;background-color: #ffdddd">-        AC_DEFINE_UNQUOTED(CP_JOBID, "${withval}", Name of the environment variable that stores the jobid)

-        CP_JOBID="${withval}"

</span><span style="color: #000000;background-color: #ddffdd">+        AC_DEFINE_UNQUOTED(__DARSHAN_JOBID, "${withval}", Name of the environment variable that stores the jobid)

+        __DARSHAN_JOBID="${withval}"

</span>         GOT_JOBID=1

     fi

 )

<span style="color: #aaaaaa">@@ -122,10 +110,10 @@ fi

</span> if test x$GOT_JOBID != x1; then

     AC_MSG_ERROR(must provide --with-jobid-env=<name> argument to configure.)

 fi

<span style="color: #000000;background-color: #ffdddd">-__CP_LOG_HINTS_DEFAULT="romio_no_indep_rw=true;cb_nodes=4"

</span><span style="color: #000000;background-color: #ddffdd">+__DARSHAN_LOG_HINTS_DEFAULT="romio_no_indep_rw=true;cb_nodes=4"

</span> if test x$GOT_LOG_HINTS != x1; then

 dnl use default hints

<span style="color: #000000;background-color: #ffdddd">-    AC_DEFINE_UNQUOTED(__CP_LOG_HINTS, "$__CP_LOG_HINTS_DEFAULT", Comma-separated list of MPI-IO hints for log file write)

</span><span style="color: #000000;background-color: #ddffdd">+    AC_DEFINE_UNQUOTED(__DARSHAN_LOG_HINTS, "$__DARSHAN_LOG_HINTS_DEFAULT", Comma-separated list of MPI-IO hints for log file write)

</span> fi

 # checks to see how we can print 64 bit values on this architecture

<span style="color: #aaaaaa">@@ -187,11 +175,6 @@ CFLAGS="$old_cflags"

</span> 

 AC_CHECK_HEADERS(mntent.h sys/mount.h)

<span style="color: #000000;background-color: #ffdddd">-# libc functions wrapped by darshan

-CP_WRAPPERS="-Wl,-u,MPI_Init,-u,MPI_Wtime,-wrap,write,-wrap,open,-wrap,creat,-wrap,creat64,-wrap,open64,-wrap,close,-wrap,read,-wrap,lseek,-wrap,lseek64,-wrap,pread,-wrap,pwrite,-wrap,readv,-wrap,writev,-wrap,__xstat,-wrap,__lxstat,-wrap,__fxstat,-wrap,__xstat64,-wrap,__lxstat64,-wrap,__fxstat64,-wrap,mmap,-wrap,mmap64,-wrap,fopen,-wrap,fclose,-wrap,fread,-wrap,fwrite,-wrap,fseek,-wrap,fopen64,-wrap,pread64,-wrap,pwrite64,-wrap,fsync,-wrap,fdatasync,-wrap,ncmpi_create,-wrap,ncmpi_open,-wrap,ncmpi_close,-wrap,H5Fcreate,-wrap,H5Fopen,-wrap,H5Fclose,-wrap,aio_write,-wrap,aio_write64,-wrap,aio_read,-wrap,aio_read64,-wrap,lio_listio,-wrap,lio_listio64,-wrap,aio_return,-wrap,aio_return64,-wrap,mkstemp,-wrap,mkostemp,-wrap,mkstemps,-wrap,mkostemps"

-

-# ,-u,__wrap___fxstat64,-u,pthread_mutex_lock,-u,pthread_mutex_unlock

-

</span> # We need to know the value of the $libdir and $bindir variables so that 

 # we can reference the correct path in the darshan compiler wrappers.  

 # Unfortunately, those two variables are not normally evaluated by autoconf.  

<span style="color: #aaaaaa">@@ -207,7 +190,7 @@ saveExecprefix=$exec_prefix

</span> test "x$prefix" = xNONE && prefix=$ac_default_prefix

 test "x$exec_prefix" = xNONE && exec_prefix=$prefix

 eval darshan_lib_path=$libdir

<span style="color: #000000;background-color: #ffdddd">-eval darshan_bin_path=$bindir

</span><span style="color: #000000;background-color: #ddffdd">+eval darshan_share_path=$datarootdir

</span> prefix=$savePrefix

 exec_prefix=$saveExecprefix

<span style="color: #aaaaaa">@@ -287,24 +270,56 @@ AC_CHECK_TYPE([MPIO_Request],

</span>           AC_DEFINE(__D_MPI_REQUEST, MPI_Request, Generalized request type for MPI-IO)

        ,[#include <mpi.h>])

<span style="color: #000000;background-color: #ffdddd">-# attempt to detect librarly naming convention in mpi compiler script

-AC_MSG_CHECKING(for old (pre-3.1.1) style MPICH library naming convention)

-AS_IF([$CC -show foo.c -o foo |grep lmpich >& /dev/null], 

-    AC_MSG_RESULT(yes)

-    MPICH_LIB_OLD=1,

-    AC_MSG_RESULT(no)

-    MPICH_LIB_OLD=0)

</span><span style="color: #000000;background-color: #ddffdd">+# attempt to detect library naming convention in mpi compiler script

+AS_IF([$CC -show foo.c -o foo >& /dev/null],

+       AC_MSG_CHECKING(for old (pre-3.1.1) style MPICH library naming convention)

+       AS_IF([$CC -show foo.c -o foo |grep lmpich >& /dev/null], 

+           AC_MSG_RESULT(yes)

+           MPICH_LIB_OLD=1,

+           AC_MSG_RESULT(no)

+           MPICH_LIB_OLD=0),

+       MPICH_LIB_OLD=0)

+

+# check to see whether the bgq instrumentation module should be built

+AC_ARG_ENABLE(

+    [bgq_mod],

+    [  --disable-bgq-mod       Disables compilation and use of BG/Q module (for BG/Q systems)],

+    [],

+    [enable_bgq_mod=check]

+)

+if test x$enable_bgq_mod != xno; then

+    AC_MSG_CHECKING(for BG/Q environment)

+    AC_TRY_COMPILE([

+        #ifndef __bgq__

+        #error __bgq__ not set

+        #endif

+        ], [],

+        AC_MSG_RESULT(yes)

+        DARSHAN_USE_BGQ=1,

+        AC_MSG_RESULT(no)

+        AS_IF([test "x$enable_bgq_mod" = xyes],

+            AC_MSG_ERROR(BG/Q module enabled in non-BG/Q environment),

+            []))

+fi

</span> 

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_VERSION="AC_PACKAGE_VERSION"

</span><span style="color: #000000;background-color: #ddffdd">+AC_ARG_WITH(mod-mem,

+[  --with-mod-mem=<num>  Maximum amount of memory (in MiB) for each Darshan module],

+    if test x$withval = xyes; then

+        AC_MSG_ERROR(--with-mod-mem must be given a number)

+    else

+        AC_DEFINE_UNQUOTED(__DARSHAN_MOD_MEM_MAX, ${withval}, Maximum memory (in MiB) for each Darshan module)

+    fi

+)

</span> 

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_VERSION="AC_PACKAGE_VERSION"

</span> AC_SUBST(darshan_lib_path)

<span style="color: #000000;background-color: #ffdddd">-AC_SUBST(darshan_bin_path)

</span><span style="color: #000000;background-color: #ddffdd">+AC_SUBST(darshan_share_path)

</span> AC_SUBST(LDFLAGS)

<span style="color: #000000;background-color: #ffdddd">-AC_SUBST(__CP_LOG_PATH)

-AC_SUBST(CP_WRAPPERS)

</span><span style="color: #000000;background-color: #ddffdd">+AC_SUBST(__DARSHAN_LOG_PATH)

</span> AC_SUBST(DISABLE_LDPRELOAD)

 AC_SUBST(DARSHAN_VERSION)

 AC_SUBST(MPICH_LIB_OLD)

<span style="color: #000000;background-color: #ddffdd">+AC_SUBST(DARSHAN_USE_BGQ)

</span> AC_OUTPUT(Makefile

 darshan-mk-log-dirs.pl

 darshan-gen-cc.pl

<span style="color: #aaaaaa">@@ -320,4 +335,5 @@ share/mpi-profile/darshan-f.conf

</span> share/mpi-profile/darshan-bg-cc.conf

 share/mpi-profile/darshan-bg-cxx.conf

 share/mpi-profile/darshan-bg-f.conf

<span style="color: #000000;background-color: #ddffdd">+share/ld-opts/darshan-base-ld-opts

</span> )

</code></pre>

<br>

</li>

<li id='diff-12'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-12'>

<strong>

darshan-runtime/darshan-common.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-common.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,192 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_COMMON_H

+#define __DARSHAN_COMMON_H

+

+/* increment a timer counter, making sure not to account for overlap

+ * with previous operations

+ *

+ * NOTE: __timer is the corresponding timer counter variable, __tm1 is

+ * the start timestamp of the operation, __tm2 is the end timestamp of

+ * the operation, and __last is the timestamp of the end of the previous

+ * I/O operation (which we don't want to overlap with).

+ */

+#define DARSHAN_TIMER_INC_NO_OVERLAP(__timer, __tm1, __tm2, __last) do{ \

+    if(__tm1 > __last) \

+        __timer += (__tm2 - __tm1); \

+    else \

+        __timer += (__tm2 - __last); \

+    if(__tm2 > __last) \

+        __last = __tm2; \

+} while(0)

+

+/* increment histogram bucket depending on the given __value

+ *

+ * NOTE: This macro can be used to build a histogram of access

+ * sizes, offsets, etc. It assumes a 10-bucket histogram, with

+ * __bucket_base_p pointing to the first counter in the sequence

+ * of buckets (i.e., the smallest bucket). The size ranges of each

+ * bucket are:

+ *      * 0 - 100 bytes

+ *      * 100 - 1 KiB

+ *      * 1 KiB - 10 KiB

+ *      * 10 KiB - 100 KiB

+ *      * 100 KiB - 1 MiB

+ *      * 1 MiB - 4 MiB

+ *      * 4 MiB - 10 MiB

+ *      * 10 MiB - 100 MiB

+ *      * 100 MiB - 1 GiB

+ *      * 1 GiB+

+ */

+#define DARSHAN_BUCKET_INC(__bucket_base_p, __value) do {\

+    if(__value < 101) \

+        *(__bucket_base_p) += 1; \

+    else if(__value < 1025) \

+        *(__bucket_base_p + 1) += 1; \

+    else if(__value < 10241) \

+        *(__bucket_base_p + 2) += 1; \

+    else if(__value < 102401) \

+        *(__bucket_base_p + 3) += 1; \

+    else if(__value < 1048577) \

+        *(__bucket_base_p + 4) += 1; \

+    else if(__value < 4194305) \

+        *(__bucket_base_p + 5) += 1; \

+    else if(__value < 10485761) \

+        *(__bucket_base_p + 6) += 1; \

+    else if(__value < 104857601) \

+        *(__bucket_base_p + 7) += 1; \

+    else if(__value < 1073741825) \

+        *(__bucket_base_p + 8) += 1; \

+    else \

+        *(__bucket_base_p + 9) += 1; \

+} while(0)

+

+/* potentially set or increment a common value counter, depending on the __count

+ * for the given __value

+ *

+ * NOTE: This macro is hardcoded to expect that Darshan will only track the 4

+ * most common (i.e., frequently occuring) values. __val_p is a pointer to the

+ * base of the value counters (i.e., the first of 4 contiguous common value

+ * counters) and __cnt_p is a pointer to the base of the count counters (i.e.

+ * the first of 4 contiguous common count counters). It is assumed your counters

+ * are stored as int64_t types.

+ */

+#define DARSHAN_COMMON_VAL_COUNTER_INC(__val_p, __cnt_p, __value, __count) do {\

+    int i; \

+    int set = 0; \

+    int64_t min = *(__cnt_p); \

+    int min_index = 0; \

+    if(__value == 0) break; \

+    for(i=0; i<4; i++) { \

+        /* increment bucket if already exists */ \

+        if(*(__val_p + i) == __value) { \

+            *(__cnt_p + i) += __count; \

+            set = 1; \

+            break; \

+        } \

+        /* otherwise find the least frequently used bucket */ \

+        else if(*(__cnt_p + i) < min) { \

+            min = *(__cnt_p + i); \

+            min_index = i; \

+        } \

+    } \

+    if(!set && (__count > min)) { \

+        *(__cnt_p + min_index) = __count; \

+        *(__val_p + min_index) = __value; \

+    } \

+} while(0)

+

+/* maximum number of common values that darshan will track per file at

+ * runtime; at shutdown time these will be reduced to the 4 most

+ * frequently occuring ones

+ */

+#define DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT 32

+struct darshan_common_val_counter

+{

+    int64_t val;

+    int freq;

+};

+

+/* i/o type (read or write) */

+enum darshan_io_type

+{

+    DARSHAN_IO_READ = 1,

+    DARSHAN_IO_WRITE = 2,

+};

+

+/* struct used for calculating variances */

+struct darshan_variance_dt

+{

+    double n;

+    double T;

+    double S;

+};

+

+/***********************************************

+* darshan-common functions for darshan modules *

+***********************************************/

+

+/* darshan_clean_file_path()

+ *

+ * Allocate a new string that contains a new cleaned-up version of

+ * the file path given in 'path' argument. Converts relative paths

+ * to absolute paths and filters out some potential noise in the

+ * path string.

+ */

+char* darshan_clean_file_path(

+    const char* path);

+

+/* darshan_common_val_counter()

+ *

+ * Potentially increment an existing common value counter or allocate

+ * a new one to keep track of commonly occuring values. Example use

+ * cases would be to track the most frequent access sizes or strides

+ * used by a specific module, for instance. 'common_val_root' is the

+ * root pointer for the tree which stores common value info, 

+ * 'common_val_count' is a pointer to the number of nodes in the 

+ * tree (i.e., the number of allocated common value counters), and

+ * 'val' is the new value to attempt to add.

+ */

+void darshan_common_val_counter(

+    void** common_val_root,

+    int* common_val_count,

+    int64_t val);

+

+/* darshan_walk_common_vals()

+ *

+ * Walks the tree of common value counters and determines the 4 most

+ * frequently occuring values, storing the common values in the

+ * appropriate counter fields of the given record. 'common_val_root'

+ * is the root of the tree which stores the common value info, 'val_p'

+ * is a pointer to the base counter (i.e., the first) of the common

+ * values (which are assumed to be 4 total and contiguous in memory),

+ * and 'cnt_p' is a pointer to the base counter of the common counts

+ * (which are again expected to be contiguous in memory).

+ */

+void darshan_walk_common_vals(

+    void* common_val_root,

+    int64_t* val_p,

+    int64_t* cnt_p);

+

+/* darshan_variance_reduce()

+ *

+ * MPI reduction operation to calculate variances on counters in

+ * data records which are shared across all processes. This could

+ * be used, for instance, to find the variance in I/O time or total

+ * bytes moved for a given data record. This function needs to be

+ * passed to MPI_Op_create to obtain a corresponding MPI operation

+ * which can be used to complete the reduction.  For more details,

+ * consult the documentation for MPI_Op_create. Example use cases

+ * can be found in the POSIX and MPIIO modules.

+ */

+void darshan_variance_reduce(

+    void *invec,

+    void *inoutvec,

+    int *len,

+    MPI_Datatype *dt);

+

+#endif /* __DARSHAN_COMMON_H */

</span></code></pre>

<br>

</li>

<li id='diff-13'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-13'>

<strong>

darshan-runtime/darshan-config.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-config.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-config.in

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/bin/sh

<span style="color: #000000;background-color: #ffdddd">-CP_WRAPPERS="@CP_WRAPPERS@"

</span> DARSHAN_LIB_PATH="@darshan_lib_path@"

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_SHARE_PATH="@darshan_share_path@"

</span> DARSHAN_LD_FLAGS="@LDFLAGS@"

 # NOTE:

<span style="color: #aaaaaa">@@ -10,9 +10,9 @@ DARSHAN_LD_FLAGS="@LDFLAGS@"

</span> #   final one is necessary to give the linker a change to resolve indirect

 #   dependencies on PnetCDF and HDF5 symbols (if the app used a library which 

 #   in turn used one of those HLLs).

<span style="color: #000000;background-color: #ffdddd">-PRE_LD_FLAGS="-L$DARSHAN_LIB_PATH $DARSHAN_LD_FLAGS -ldarshan-mpi-io -lz $CP_WRAPPERS"

-POST_LD_FLAGS="-L$DARSHAN_LIB_PATH -ldarshan-posix -ldarshan-mpi-io -lz -lrt -lpthread"

</span> 

<span style="color: #000000;background-color: #ddffdd">+PRE_LD_FLAGS="-L$DARSHAN_LIB_PATH $DARSHAN_LD_FLAGS -ldarshan -lz -Wl,@$DARSHAN_SHARE_PATH/ld-opts/darshan-base-ld-opts"

+POST_LD_FLAGS="-L$DARSHAN_LIB_PATH -Wl,--start-group -ldarshan -ldarshan-stubs -Wl,--end-group -lz -lrt -lpthread"

</span> 

 usage="\

 Usage: darshan-config [--pre-ld-flags] [--post-ld-flags]"

</code></pre>

<br>

</li>

<li id='diff-14'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-14'>

<strong>

darshan-runtime/darshan-core.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-core.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,76 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_CORE_H

+#define __DARSHAN_CORE_H

+

+#include <unistd.h>

+#include <sys/types.h>

+#include <stdint.h>

+

+#include "uthash.h"

+#include "darshan-log-format.h"

+

+/* Environment variable to override __DARSHAN_JOBID */

+#define DARSHAN_JOBID_OVERRIDE "DARSHAN_JOBID"

+

+/* Environment variable to override __DARSHAN_LOG_PATH */

+#define DARSHAN_LOG_PATH_OVERRIDE "DARSHAN_LOGPATH"

+

+/* Environment variable to override __DARSHAN_LOG_HINTS */

+#define DARSHAN_LOG_HINTS_OVERRIDE "DARSHAN_LOGHINTS"

+

+/* Environment variable to override __DARSHAN_MEM_ALIGNMENT */

+#define DARSHAN_MEM_ALIGNMENT_OVERRIDE "DARSHAN_MEMALIGN"

+

+/* Environment variable to override memory per module */

+#define DARSHAN_MOD_MEM_OVERRIDE "DARSHAN_MODMEM"

+

+/* Maximum amount of memory per instrumentation module in MiB */

+#ifdef __DARSHAN_MOD_MEM_MAX

+#define DARSHAN_MOD_MEM_MAX (__DARSHAN_MOD_MEM_MAX * 1024 * 1024)

+#else

+#define DARSHAN_MOD_MEM_MAX (2 * 1024 * 1024) /* 2 MiB default */

+#endif

+

+/* Default runtime compression buffer size */

+#define DARSHAN_COMP_BUF_SIZE DARSHAN_MOD_MEM_MAX

+

+/* in memory structure to keep up with job level data */

+struct darshan_core_runtime

+{

+    struct darshan_header log_header;

+    struct darshan_job log_job;

+    char exe[DARSHAN_EXE_LEN+1];

+    struct darshan_core_record_ref *rec_hash;

+    int rec_count;

+    struct darshan_core_module* mod_array[DARSHAN_MAX_MODS];

+    char *comp_buf;

+    double wtime_offset;

+    char *trailing_data;

+};

+

+struct darshan_core_module

+{

+    darshan_module_id id;

+    struct darshan_module_funcs mod_funcs;

+};

+

+struct darshan_core_record_ref

+{

+    struct darshan_record rec;

+    uint64_t mod_flags;

+    uint64_t global_mod_flags;

+    UT_hash_handle hlink;

+};

+

+void darshan_core_initialize(int argc, char **argv);

+void darshan_core_shutdown(void);

+

+uint32_t darshan_hashlittle(const void *key, size_t length, uint32_t initval);

+uint64_t darshan_hash(const register unsigned char *k, register uint64_t length, register uint64_t level);

+

+#endif /* __DARSHAN_CORE_H */

</span></code></pre>

<br>

</li>

<li id='diff-15'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-15'>

<strong>

darshan-runtime/darshan-dynamic.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-dynamic.h

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-dynamic.h

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2012 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #ifndef __DARSHAN_DYNAMIC_H

<span style="color: #aaaaaa">@@ -8,16 +9,10 @@

</span> 

 #ifdef DARSHAN_PRELOAD

<span style="color: #000000;background-color: #ffdddd">-#include <dlfcn.h>

-#include <stdlib.h>

-

-#define DARSHAN_MPI_CALL(func) __real_ ## func

-

</span> #define DARSHAN_EXTERN_DECL(name,ret,args) \

   extern ret (*__real_ ## name)args;

 DARSHAN_EXTERN_DECL(PMPI_File_close, int, (MPI_File *fh));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_EXTERN_DECL(PMPI_File_set_size, int, (MPI_File fh, MPI_Offset size));

</span> DARSHAN_EXTERN_DECL(PMPI_File_iread_at, int, (MPI_File fh, MPI_Offset offset, void *buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST *request));

 DARSHAN_EXTERN_DECL(PMPI_File_iread, int, (MPI_File fh, void  *buf, int  count, MPI_Datatype  datatype, __D_MPI_REQUEST  *request));

 DARSHAN_EXTERN_DECL(PMPI_File_iread_shared, int, (MPI_File fh, void *buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST *request));

<span style="color: #aaaaaa">@@ -104,7 +99,6 @@ DARSHAN_EXTERN_DECL(PMPI_File_write_shared, int, (MPI_File fh, void *buf, int co

</span> DARSHAN_EXTERN_DECL(PMPI_Finalize, int, ());

 DARSHAN_EXTERN_DECL(PMPI_Init, int, (int *argc, char ***argv));

 DARSHAN_EXTERN_DECL(PMPI_Init_thread, int, (int *argc, char ***argv, int required, int *provided));

<span style="color: #000000;background-color: #ffdddd">-

</span> DARSHAN_EXTERN_DECL(PMPI_Wtime, double, ());

 DARSHAN_EXTERN_DECL(PMPI_Allreduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm));

 DARSHAN_EXTERN_DECL(PMPI_Bcast, int, (void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm));

<span style="color: #aaaaaa">@@ -116,16 +110,28 @@ DARSHAN_EXTERN_DECL(PMPI_Type_contiguous, int, (int count, MPI_Datatype oldtype,

</span> DARSHAN_EXTERN_DECL(PMPI_Type_extent, int, (MPI_Datatype datatype, MPI_Aint *extent));

 DARSHAN_EXTERN_DECL(PMPI_Type_free, int, (MPI_Datatype *datatype));

 DARSHAN_EXTERN_DECL(PMPI_Type_hindexed, int, (int count, int *array_of_blocklengths, MPI_Aint *array_of_displacements, MPI_Datatype oldtype, MPI_Datatype *newtype));

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_EXTERN_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *num_integers, int *num_addresses, int *num_datatypes, int *combiner));

+DARSHAN_EXTERN_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));

</span> DARSHAN_EXTERN_DECL(PMPI_Op_create, int, (MPI_User_function *function, int commute, MPI_Op *op));

 DARSHAN_EXTERN_DECL(PMPI_Op_free, int, (MPI_Op *op));

<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST

+DARSHAN_EXTERN_DECL(PMPI_Reduce, int, (const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));

+#else

</span> DARSHAN_EXTERN_DECL(PMPI_Reduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_EXTERN_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *num_integers, int *num_addresses, int *num_datatypes, int *combiner));

-DARSHAN_EXTERN_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));

-

</span><span style="color: #000000;background-color: #ddffdd">+#endif

+#ifdef HAVE_MPIIO_CONST

+DARSHAN_EXTERN_DECL(PMPI_Send, int, (const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));

</span> #else

<span style="color: #000000;background-color: #ffdddd">-

-#define DARSHAN_MPI_CALL(func) func

-

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_EXTERN_DECL(PMPI_Send, int, (void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));

</span> #endif

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_EXTERN_DECL(PMPI_Recv, int, (void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status));

+#ifdef HAVE_MPIIO_CONST

+DARSHAN_EXTERN_DECL(PMPI_Gather, int, (const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));

+#else

+DARSHAN_EXTERN_DECL(PMPI_Gather, int, (void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));

+#endif

+DARSHAN_EXTERN_DECL(PMPI_Barrier, int, (MPI_Comm comm));

</span> 

 #endif

<span style="color: #000000;background-color: #ddffdd">+

+#endif /* __DARSHAN_DYNAMIC_H */

</span></code></pre>

<br>

</li>

<li id='diff-16'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-16'>

<strong>

darshan-runtime/darshan-gen-cc.pl.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-cc.pl.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-cc.pl.in

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/perl -w

 #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2009 by Argonne National Laboratory.

-#      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

 # takes a standard mpicc script as an argument and tried to generate a

<span style="color: #aaaaaa">@@ -207,7 +207,7 @@ print OUTPUT<<"EOF";

</span>     # if the user is using another PMPI based tool

     # if allargs includes any libraries, then we need to get 

<span style="color: #000000;background-color: #ffdddd">-    # -ldarshan-mpi-io in there first

</span><span style="color: #000000;background-color: #ddffdd">+    # -ldarshan in there first

</span>     argno=0;

     once=0;

     compiler_check=0;

<span style="color: #aaaaaa">@@ -215,7 +215,7 @@ print OUTPUT<<"EOF";

</span>     for arg in "\$\{newallargs\[\@\]\}"; do

         res=`expr match "\$arg" "-l"`;

         if [ \$res -gt 0 -a \$once -lt 1 ]; then

<span style="color: #000000;background-color: #ffdddd">-            newallargs[\$argno]=-ldarshan-mpi-io

</span><span style="color: #000000;background-color: #ddffdd">+            newallargs[\$argno]=-ldarshan

</span>             argno=`expr \$argno + 1`

             newallargs[\$argno]=\$arg;

             once=1;

<span style="color: #aaaaaa">@@ -261,8 +261,8 @@ print OUTPUT<<"EOF";

# is MPI in there?

     grep MPI \$tmpfile >& /dev/null

     rc_mpi=\$?

<span style="color: #000000;background-color: #ffdddd">-    # is PMPI being used for any init, finalize, or mpi-io calls?

-    grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

<span style="color: #000000;background-color: #ddffdd">+    # is PMPI being used for PMPI_File_open?

+    grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

rc_pmpi=\$?

     rm \$tmpfile >& /dev/null

</code></pre>

<br>

</li>

<li id='diff-17'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-17'>

<strong>

darshan-runtime/darshan-gen-cxx.pl.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-cxx.pl.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-cxx.pl.in

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/perl -w

 #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2009 by Argonne National Laboratory.

-#      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

 # takes a standard mpicxx script as an argument and tried to generate a

<span style="color: #aaaaaa">@@ -207,7 +207,7 @@ print OUTPUT<<"EOF";

</span>     # if the user is using another PMPI based tool

     # if allargs includes any libraries, then we need to get 

<span style="color: #000000;background-color: #ffdddd">-    # -ldarshan-mpi-io in there first

</span><span style="color: #000000;background-color: #ddffdd">+    # -ldarshan in there first

</span>     argno=0;

     once=0;

     compiler_check=0;

<span style="color: #aaaaaa">@@ -215,7 +215,7 @@ print OUTPUT<<"EOF";

</span>     for arg in "\$\{newallargs\[\@\]\}"; do

         res=`expr match "\$arg" "-l"`;

         if [ \$res -gt 0 -a \$once -lt 1 ]; then

<span style="color: #000000;background-color: #ffdddd">-            newallargs[\$argno]=-ldarshan-mpi-io

</span><span style="color: #000000;background-color: #ddffdd">+            newallargs[\$argno]=-ldarshan

</span>             argno=`expr \$argno + 1`

             newallargs[\$argno]=\$arg;

             once=1;

<span style="color: #aaaaaa">@@ -263,8 +263,8 @@ print OUTPUT<<"EOF";

</span>     grep MPI \$tmpfile >& /dev/null

     rc_mpi=\$?

<span style="color: #000000;background-color: #ffdddd">-    # is PMPI being used for any init, finalize, or mpi-io calls?

-    grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

<span style="color: #000000;background-color: #ddffdd">+    # is PMPI being used for PMPI_File_open?

+    grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

rc_pmpi=\$?

     CXXMPICH=-lmpichcxx

</code></pre>

<br>

</li>

<li id='diff-18'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-18'>

<strong>

darshan-runtime/darshan-gen-fortran.pl.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-fortran.pl.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-fortran.pl.in

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/perl -w

 #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2009 by Argonne National Laboratory.

-#      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

 # takes a standard mpif* script as an argument and tried to generate a

<span style="color: #aaaaaa">@@ -215,7 +215,7 @@ print OUTPUT<<"EOF";

</span>     # if the user is using another PMPI based tool

     # if allargs includes any libraries, then we need to get 

<span style="color: #000000;background-color: #ffdddd">-    # -ldarshan-mpi-io in there first

</span><span style="color: #000000;background-color: #ddffdd">+    # -ldarshan in there first

</span>     argno=0;

     once=0;

     compiler_check=0;

<span style="color: #aaaaaa">@@ -223,7 +223,7 @@ print OUTPUT<<"EOF";

</span>     for arg in "\$\{newallargs\[\@\]\}"; do

         res=`expr match "\$arg" "-l"`;

         if [ \$res -gt 0 -a \$once -lt 1 ]; then

<span style="color: #000000;background-color: #ffdddd">-            newallargs[\$argno]=-ldarshan-mpi-io

</span><span style="color: #000000;background-color: #ddffdd">+            newallargs[\$argno]=-ldarshan

</span>             argno=`expr \$argno + 1`

             newallargs[\$argno]=\$arg;

             once=1;

<span style="color: #aaaaaa">@@ -271,8 +271,8 @@ print OUTPUT<<"EOF";

</span>     grep -i MPI \$tmpfile >& /dev/null

     rc_mpi=\$?

<span style="color: #000000;background-color: #ffdddd">-    # is PMPI being used for any init, finalize, or mpi-io calls?

-    grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

<span style="color: #000000;background-color: #ddffdd">+    # is PMPI being used for PMPI_File_open?

+    grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null

rc_pmpi=\$?

</code></pre>

<br>

</li>

<li id='diff-19'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-19'>

<strong>

darshan-runtime/darshan-mk-log-dirs.pl.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-mk-log-dirs.pl.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-mk-log-dirs.pl.in

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/perl -w

 #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2009 by Argonne National Laboratory.

-#      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

 use File::Basename;

<span style="color: #aaaaaa">@@ -10,7 +10,7 @@ use File::Basename;

</span> # LOGDIR/<year>/<month>/<day>/

 # use log dir specified at configure time

<span style="color: #000000;background-color: #ffdddd">-$LOGDIR = "@__CP_LOG_PATH@";

</span><span style="color: #000000;background-color: #ddffdd">+$LOGDIR = "@__DARSHAN_LOG_PATH@";

</span> 

 my $year = (localtime)[5] + 1900;

</code></pre>

<br>

</li>

<li id='diff-20'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-20'>

<strong>

darshan-runtime/darshan-runtime-config.h.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-runtime-config.h.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-runtime-config.h.in

</span><span style="color: #aaaaaa">@@ -3,9 +3,6 @@

</span> /* Define if building universal (internal helper macro) */

 #undef AC_APPLE_UNIVERSAL_BUILD

<span style="color: #000000;background-color: #ffdddd">-/* Name of the environment variable that stores the jobid */

-#undef CP_JOBID

-

</span> /* Define if cuserid() should be disabled */

 #undef DARSHAN_DISABLE_CUSERID

<span style="color: #aaaaaa">@@ -91,23 +88,25 @@

</span> #endif

 /* Define if Darshan should set log files to be group readable */

<span style="color: #000000;background-color: #ffdddd">-#undef __CP_GROUP_READABLE_LOGS

</span><span style="color: #000000;background-color: #ddffdd">+#undef __DARSHAN_GROUP_READABLE_LOGS

+

+/* Name of the environment variable that stores the jobid */

+#undef __DARSHAN_JOBID

</span> 

 /* Comma separated list of env. variables to use for log path */

<span style="color: #000000;background-color: #ffdddd">-#undef __CP_LOG_ENV

</span><span style="color: #000000;background-color: #ddffdd">+#undef __DARSHAN_LOG_ENV

</span> 

 /* Comma-separated list of MPI-IO hints for log file write */

<span style="color: #000000;background-color: #ffdddd">-#undef __CP_LOG_HINTS

</span><span style="color: #000000;background-color: #ddffdd">+#undef __DARSHAN_LOG_HINTS

</span> 

 /* Location to store log files at run time */

<span style="color: #000000;background-color: #ffdddd">-#undef __CP_LOG_PATH

</span><span style="color: #000000;background-color: #ddffdd">+#undef __DARSHAN_LOG_PATH

</span> 

 /* Memory alignment in bytes */

<span style="color: #000000;background-color: #ffdddd">-#undef __CP_MEM_ALIGNMENT

</span><span style="color: #000000;background-color: #ddffdd">+#undef __DARSHAN_MEM_ALIGNMENT

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* Define if Darshan should stat files at open time to collect extra

-   statistics */

-#undef __CP_STAT_AT_OPEN

</span><span style="color: #000000;background-color: #ddffdd">+/* Maximum memory (in MiB) for each Darshan module */

+#undef __DARSHAN_MOD_MEM_MAX

</span> 

 /* Generalized request type for MPI-IO */

 #undef __D_MPI_REQUEST

</code></pre>

<br>

</li>

<li id='diff-21'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-21'>

<strong>

darshan-runtime/darshan.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan.h

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan.h

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #ifndef __DARSHAN_H

<span style="color: #aaaaaa">@@ -10,252 +11,147 @@

</span> #include <sys/types.h>

 #include <stdint.h>

 #include <mpi.h>

<span style="color: #000000;background-color: #ffdddd">-#include "darshan-log-format.h"

-

-/* Environment variable to override CP_JOBID */

-#define CP_JOBID_OVERRIDE "DARSHAN_JOBID"

-

-/* Environment variable to override __CP_LOG_PATH */

-#define CP_LOG_PATH_OVERRIDE "DARSHAN_LOGPATH"

-

-/* Environment variable to override __CP_LOG_PATH */

-#define CP_LOG_HINTS_OVERRIDE "DARSHAN_LOGHINTS"

-

-/* Environment variable to override __CP_MEM_ALIGNMENT */

-#define CP_MEM_ALIGNMENT_OVERRIDE "DARSHAN_MEMALIGN"

-

-/* maximum number of files per process we will track */

-#define CP_MAX_FILES 1024

-

-/* TODO: figure out how to pick good value here */

-/* hash table size parameters */

-#define CP_HASH_BITS 8

-#define CP_HASH_SIZE (1 << CP_HASH_BITS)

-#define CP_HASH_MASK (CP_HASH_SIZE - 1)

-

-/* maximum number of access sizes and stride sizes that darshan will track

- * per file at runtime; at log time they will be reduced into the 4 most

- * frequently occurring ones 

- */

-#define CP_MAX_ACCESS_COUNT_RUNTIME 32

-

-/* flags to indicate properties of file records */

-#define CP_FLAG_CONDENSED 1<<0

-#define CP_FLAG_NOTIMING 1<<1

-

-/* calculation of compression buffer size (defaults to 50% of the maximum

- * memory that Darshan is allowed to consume on a process) 

- */

-#define CP_COMP_BUF_SIZE ((CP_MAX_FILES * sizeof(struct darshan_file))/2)

-

-enum cp_io_type

-{

-    CP_READ = 1,

-    CP_WRITE = 2,

-};

-

-struct cp_access_counter

-{

-    int64_t size;

-    int freq;

-};

-

-struct darshan_io_tracker;

-

-/* in memory structure to keep up with file level data */

-struct darshan_file_runtime

-{

-    struct darshan_file* log_file;

-    struct darshan_file_runtime* name_next;

-    struct darshan_file_runtime* name_prev;

-    void* access_root;

-    int access_count;

-    void* stride_root;

-    int stride_count;

-    int64_t last_byte_read;

-    int64_t last_byte_written;

-    int64_t offset;

-    enum cp_io_type last_io_type;

-    double last_posix_write_end;

-    double last_mpi_write_end;

-    double last_posix_read_end;

-    double last_mpi_read_end;

-    double last_posix_meta_end;

-    double last_mpi_meta_end;

-    struct darshan_aio_tracker* aio_list_head;

-    struct darshan_aio_tracker* aio_list_tail;

-};

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* handles used by various APIs to refer to files */

-enum darshan_handle_type 

-{

-    DARSHAN_FD = 1,

-    DARSHAN_FH,

-    DARSHAN_NCID,

-    DARSHAN_HID

-};

</span><span style="color: #000000;background-color: #ddffdd">+#include "darshan-log-format.h"

+#include "darshan-common.h"

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define DARSHAN_FILE_HANDLE_MAX (sizeof(MPI_File))

-/* This struct is used to track a reference to a file by file 

- * descriptor, MPI file handle, ncdf id, etc.

</span><span style="color: #000000;background-color: #ddffdd">+/* macros for declaring wrapper functions and calling MPI routines

+ * consistently regardless of whether static or dynamic linking is used

</span>  */

<span style="color: #000000;background-color: #ffdddd">-struct darshan_file_ref

-{

-    struct darshan_file_runtime* file;

-    char handle[DARSHAN_FILE_HANDLE_MAX];

-    int handle_sz;

-    enum darshan_handle_type handle_type;

-    struct darshan_file_ref* next;

-    struct darshan_file_ref* prev;

-};

</span><span style="color: #000000;background-color: #ddffdd">+#ifdef DARSHAN_PRELOAD

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* in memory structure to keep up with job level data */

-struct darshan_job_runtime

-{

-    struct darshan_job log_job;

-    char exe[CP_EXE_LEN+1];

-    struct darshan_file file_array[CP_MAX_FILES];

-    struct darshan_file_runtime file_runtime_array[CP_MAX_FILES];

-    char comp_buf[CP_COMP_BUF_SIZE];

-    int flags;

-    int file_count;

-    struct darshan_file_runtime* name_table[CP_HASH_SIZE];

-    struct darshan_file_ref* handle_table[CP_HASH_SIZE];

-    double wtime_offset;

-    char* trailing_data;

-};

</span><span style="color: #000000;background-color: #ddffdd">+#include <dlfcn.h>

+#include <stdlib.h>

</span> 

<span style="color: #000000;background-color: #ffdddd">-extern pthread_mutex_t cp_mutex;

-#define CP_LOCK() pthread_mutex_lock(&cp_mutex)

-#define CP_UNLOCK() pthread_mutex_unlock(&cp_mutex)

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_FORWARD_DECL(__func,__ret,__args) \

+  __ret (*__real_ ## __func)__args = NULL

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_SET(__file, __counter, __value) do {\

-    (__file)->log_file->counters[__counter] = __value; \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_DECL(__func) __func

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_F_SET(__file, __counter, __value) do {\

-    (__file)->log_file->fcounters[__counter] = __value; \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_MPI_CALL(__func) __real_ ## __func

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_INC(__file, __counter, __value) do {\

-    (__file)->log_file->counters[__counter] += __value; \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#define MAP_OR_FAIL(__func) \

+    if (!(__real_ ## __func)) \

+    { \

+        __real_ ## __func = dlsym(RTLD_NEXT, #__func); \

+        if(!(__real_ ## __func)) { \

+           fprintf(stderr, "Darshan failed to map symbol: %s\n", #__func); \

+           exit(1); \

+       } \

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_F_INC(__file, __counter, __value) do {\

-    (__file)->log_file->fcounters[__counter] += __value; \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#else

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_F_INC_NO_OVERLAP(__file, __tm1, __tm2, __last, __counter) do { \

-    if(__tm1 > __last) \

-        CP_F_INC(__file, __counter, (__tm2-__tm1)); \

-    else \

-        CP_F_INC(__file, __counter, (__tm2 - __last)); \

-    if(__tm2 > __last) \

-        __last = __tm2; \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_FORWARD_DECL(__name,__ret,__args) \

+  extern __ret __real_ ## __name __args;

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_VALUE(__file, __counter) \

-    ((__file)->log_file->counters[__counter])

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_DECL(__name) __wrap_ ## __name

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_F_VALUE(__file, __counter) \

-    ((__file)->log_file->fcounters[__counter])

</span><span style="color: #000000;background-color: #ddffdd">+#define DARSHAN_MPI_CALL(__func) __func

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_MAX(__file, __counter, __value) do {\

-    if((__file)->log_file->counters[__counter] < __value) \

-    { \

-        (__file)->log_file->counters[__counter] = __value; \

-    } \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#define MAP_OR_FAIL(__func)

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_COUNTER_INC(__file, __value, __count, __maxflag, __validx, __cntidx) do {\

-    int i; \

-    int set = 0; \

-    int64_t min = CP_VALUE(__file, __cntidx); \

-    int min_index = 0; \

-    if(__value == 0) break; \

-    for(i=0; i<4; i++) { \

-        /* increment bucket if already exists */ \

-        if(CP_VALUE(__file, __validx + i) == __value) { \

-            CP_INC(__file, __cntidx + i, (__count)); \

-            set = 1; \

-            break; \

-        } \

-        /* otherwise find the least frequently used bucket */ \

-        else if(CP_VALUE(__file, __cntidx + i) < min) { \

-            min = CP_VALUE(__file, __cntidx + i); \

-            min_index = i; \

-        } \

-    } \

-    if((!set && !__maxflag) || (!set && __maxflag && (__count) > min)) { \

-        CP_SET(__file, __cntidx+min_index, __count); \

-        CP_SET(__file, __validx+min_index, __value); \

-    } \

-} while(0)

</span><span style="color: #000000;background-color: #ddffdd">+#endif

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_BUCKET_INC(__file, __counter_base, __value) do {\

-    if(__value < 101) \

-        (__file)->log_file->counters[__counter_base] += 1; \

-    else if(__value < 1025) \

-        (__file)->log_file->counters[__counter_base+1] += 1; \

-    else if(__value < 10241) \

-        (__file)->log_file->counters[__counter_base+2] += 1; \

-    else if(__value < 102401) \

-        (__file)->log_file->counters[__counter_base+3] += 1; \

-    else if(__value < 1048577) \

-        (__file)->log_file->counters[__counter_base+4] += 1; \

-    else if(__value < 4194305) \

-        (__file)->log_file->counters[__counter_base+5] += 1; \

-    else if(__value < 10485761) \

-        (__file)->log_file->counters[__counter_base+6] += 1; \

-    else if(__value < 104857601) \

-        (__file)->log_file->counters[__counter_base+7] += 1; \

-    else if(__value < 1073741825) \

-        (__file)->log_file->counters[__counter_base+8] += 1; \

-    else \

-        (__file)->log_file->counters[__counter_base+9] += 1; \

-} while(0)

-

-enum cp_counter_type

</span><span style="color: #000000;background-color: #ddffdd">+/* module developers provide the following functions to darshan-core */

+struct darshan_module_funcs

</span> {

<span style="color: #000000;background-color: #ffdddd">-    CP_COUNTER_ACCESS,

-    CP_COUNTER_STRIDE

</span><span style="color: #000000;background-color: #ddffdd">+    /* perform any necessary pre-shutdown steps

+     *

+     * NOTE: this typically includes disabling wrapper functions so

+     * darshan-core can shutdown in a consistent state.

+     */

+    void (*begin_shutdown)(void);

+    /* retrieve module data to write to log file

+     *

+     * NOTE: module developers can use this function to run collective

+     * MPI operations at shutdown time. Typically this functionality

+     * has been used to reduce records shared globablly (given in the

+     * 'shared_recs' array) into a single data record.

+     */

+    void (*get_output_data)(

+        MPI_Comm mod_comm,  /* MPI communicator to run collectives with */

+        darshan_record_id *shared_recs, /* list of shared data record ids */

+        int shared_rec_count, /* count of shared data records */

+        void** mod_buf, /* output parameter to save module buffer address */

+        int* mod_buf_sz /* output parameter to save module buffer size */

+    );

+    /* shutdown module data structures */

+    void (*shutdown)(void);

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* checking alignment according to this document:

- * http://publib.boulder.ibm.com/infocenter/compbgpl/v9v111/index.jsp?topic=/com.ibm.bg9111.doc/bgusing/data_alignment.htm

</span><span style="color: #000000;background-color: #ddffdd">+/* paths that darshan will not trace */

+extern char* darshan_path_exclusions[]; /* defined in lib/darshan-core.c */

+

+/*****************************************************

+* darshan-core functions exported to darshan modules *

+*****************************************************/

+

+/* darshan_core_register_module()

+ *

+ * Register module identifier 'mod_id' with the darshan-core runtime

+ * environment, allowing the module to store I/O characterization data.

+ * 'funcs' is a pointer to a structure containing each of the function

+ * pointers required by darshan-core to shut down the module. The function

+ * returns the following integers passed in as pointers: 'rank' is the

+ * MPI rank of the calling process, 'mod_mem_limit' is the maximum amount

+ * of memory the module may use, and 'sys_mem_alignment' is the configured

+ * memory alignment value Darshan was configured with.

</span>  */

<span style="color: #000000;background-color: #ffdddd">-

-extern struct darshan_job_runtime* darshan_global_job;

-

-void darshan_initialize(int argc, char** argv, int nprocs, int rank);

-void darshan_finalize(struct darshan_job_runtime* job);

-void darshan_condense(void);

-void darshan_shutdown(int timing_flag);

-void darshan_walk_file_accesses(struct darshan_job_runtime* final_job);

-double darshan_wtime(void);

-void darshan_mnt_id_from_path(const char* path, int64_t* device_id, int64_t* block_size);

-char* darshan_get_exe_and_mounts(struct darshan_job_runtime* final_job);

-void darshan_mpi_initialize(int *argc, char ***argv);

-

-uint32_t darshan_hashlittle(const void *key, size_t length, uint32_t initval);

-uint64_t darshan_hash(const register unsigned char *k, register uint64_t length, register uint64_t level);

-

-struct darshan_file_runtime* darshan_file_by_name(const char* name);

-

-struct darshan_file_runtime* darshan_file_by_name_sethandle(

-    const char* name,

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type);

-

-struct darshan_file_runtime* darshan_file_by_handle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type);

-

-void darshan_file_closehandle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type);

</span><span style="color: #000000;background-color: #ddffdd">+void darshan_core_register_module(

+    darshan_module_id mod_id,

+    struct darshan_module_funcs *funcs,

+    int *rank,

+    int *mod_mem_limit,

+    int *sys_mem_alignment);

+

+/* darshan_core_unregister_module()

+ * 

+ * Unregisters module identifier 'mod_id' with the darshan-core runtime,

+ * removing the given module from the resulting I/O characterization log.

+ */

+void darshan_core_unregister_module(

+    darshan_module_id mod_id);

+

+/* darshan_core_register_record()

+ *

+ * Register a record with the darshan-core runtime, allowing it to be

+ * properly tracked and (potentially) correlated with records from other

+ * modules. 'name' is the the name of the Darshan record (e.g., the full

+ * file path) and 'len' is the size of the name pointer (string length

+ * for string names). 'mod_id' is the identifier of the calling module,

+ * 'printable_flag' indicates whether the name is a string, and 

+ * 'mod_limit_flag' is set if the calling module is out of memory (to

+ * prevent darshan-core from creating new records and to just search

+ * through existing records). 'rec_id' is an output pointer storing the

+ * correspoing Darshan record identifier and 'file_alignment' is an output

+ * pointer storing the file system alignment value for the given record.

+ */

+void darshan_core_register_record(

+    void *name,

+    int len,

+    darshan_module_id mod_id,

+    int printable_flag,

+    int mod_limit_flag,

+    darshan_record_id *rec_id,

+    int *file_alignment);

+

+/* darshan_core_unregister_record()

+ *

+ * Unregister record identifier 'rec_id' in the darshan-core runtime.

+ * This unregister is only in the context of module identifier 'mod_id',

+ * meaning that if the file record has other module's associated with

+ * it, then the record won't be completely removed.

+ */

+void darshan_core_unregister_record(

+    darshan_record_id rec_id,

+    darshan_module_id mod_id);

+

+/* darshan_core_wtime()

+ *

+ * Returns the elapsed time relative to (roughly) the start of

+ * the application.

+ */

+double darshan_core_wtime(void);

</span> 

 #endif /* __DARSHAN_H */

</code></pre>

<br>

</li>

<li id='diff-22'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-22'>

<strong>

darshan-runtime/doc/darshan-runtime.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/doc/darshan-runtime.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/doc/darshan-runtime.txt

<span style="color: #aaaaaa">@@ -16,7 +16,10 @@ used by the application.

</span> The darshan-runtime instrumentation only instruments MPI applications (the

 application must at least call `MPI_Init()` and `MPI_Finalize()`).  However,

 it captures both MPI-IO and POSIX file access.  It also captures limited

<span style="color: #000000;background-color: #ffdddd">-information about HDF5 and PnetCDF access.

</span><span style="color: #000000;background-color: #ddffdd">+information about HDF5 and PnetCDF access. Darshan also exposes an API that

+can be used to develop and add new instrumentation modules (for other I/O library

+interfaces or to gather system-specific data, for instance), as detailed in

+http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-modularization.html[this document].

</span> 

 This document provides generic installation instructions, but "recipes" for

 several common HPC systems are provided at the end of the document as well.

<span style="color: #aaaaaa">@@ -40,40 +43,34 @@ make

</span> make install

 ----

<span style="color: #000000;background-color: #ffdddd">-.Detecting file size and alignment

-[NOTE]

-====

-You can also add --enable-stat-at-open option to cause the Darshan library

-to issue an additional stat() system call on each file the first time that

-it is opened on each process.  This allows Darshan to detect the file

-alignment (and subsequent unaligned accesses).  It also allows Darshan to

-detect the size of files at open time before any I/O is performed.

-Unfortunately, this option can cause significant overhead at scale on file

-systems such as PVFS or Lustre that must contact every server for a given

-file in order to satisfy a stat request.  We therefore disable this

-feature by default.

-====

-

</span> .Explanation of configure arguments:

<span style="color: #000000;background-color: #ffdddd">-* `--with-mem-align` (mandatory): This value is system-dependent and will be

</span><span style="color: #000000;background-color: #ddffdd">+* `--with-mem-align=` (mandatory): This value is system-dependent and will be

</span> used by Darshan to determine if the buffer for a read or write operation is

 aligned in memory.

<span style="color: #000000;background-color: #ffdddd">-* `--with-log-path` (this, or `--with-log-path-by-env`, is mandatory): This

-specifies the parent directory for the directory tree where darshan logs

-will be placed

-* `--with-jobid-env` (mandatory): this specifies the environment variable that

</span><span style="color: #000000;background-color: #ddffdd">+* `--with-jobid-env=` (mandatory): this specifies the environment variable that

</span> Darshan should check to determine the jobid of a job.  Common values are

 `PBS_JOBID` or `COBALT_JOBID`.  If you are not using a scheduler (or your

 scheduler does not advertise the job ID) then you can specify `NONE` here.

 Darshan will fall back to using the pid of the rank 0 process if the

 specified environment variable is not set.

<span style="color: #000000;background-color: #ffdddd">-* `CC=`: specifies the MPI C compiler to use for compilation

-* `--with-log-path-by-env`: specifies an environment variable to use to

</span><span style="color: #000000;background-color: #ddffdd">+* `--with-log-path=` (this, or `--with-log-path-by-env`, is mandatory): This

+specifies the parent directory for the directory tree where darshan logs

+will be placed.

+* `--with-log-path-by-env=`: specifies an environment variable to use to

determine the log path at run time.

 * `--with-log-hints=`: specifies hints to use when writing the Darshan log

 file.  See `./configure --help` for details.

<span style="color: #000000;background-color: #ddffdd">+* `--with-mod-mem=`: specifies the maximum amount of memory (in MiB) that

+each Darshan module can consume.

</span> * `--with-zlib=`: specifies an alternate location for the zlib development

 header and library.

<span style="color: #000000;background-color: #ddffdd">+* `CC=`: specifies the MPI C compiler to use for compilation.

+* `--disable-cuserid`: disables use of cuserid() at runtime.

+* `--disable-ld-preload`: disables building of the Darshan LD_PRELOAD library

+* `--disable-bgq-mod`: disables building of the BG/Q module (default checks

+and only builds if BG/Q environment detected).

+* `--enable-group-readable-logs`: sets darshan log file permissions to allow

+group read access.

</span> 

 === Cross compilation

<span style="color: #aaaaaa">@@ -317,7 +314,7 @@ Please set your environment to use the GNU programming environment before

</span> configuring or compiling Darshan.  Although Darshan can be built with a

 variety of compilers, the GNU compilers are recommended because it will

 produce a Darshan library that is interoperable with the widest range

<span style="color: #000000;background-color: #ffdddd">-of compmilers and linkers.  On most Cray systems you can enable the GNU

</span><span style="color: #000000;background-color: #ddffdd">+of compilers and linkers.  On most Cray systems you can enable the GNU

</span> programming environment with a command similar to "module swap PrgEnv-pgi

 PrgEnv-gnu".  Please see your site documentation for information about

 how to switch programming environments.

<span style="color: #aaaaaa">@@ -469,6 +466,37 @@ older versions of Open MPI, please refer to the following mailing list discussio

</span> 

 http://www.open-mpi.org/community/lists/devel/2013/01/11907.php

<span style="color: #000000;background-color: #ddffdd">+== Upgrading to Darshan 3.x from 2.x

+

+Beginning with Darshan 3.0.0, Darshan has been rewritten to modularize its runtime environment

+and log file format to simplify the addition of new I/O characterization data. The process of

+compiling and installing the Darshan 3.x source code should essentially be identical to this

+process on Darshan 2.x. Therefore, the installation recipes given in the previous section

+should work irrespective of the Darshan version being used. Similarly, the manner in which

+Darshan is used should be the same across versions -- the sections in this document regarding

+Darshan link:darshan-runtime.html#_environment_preparation[environment preparation],

+instrumenting link:darshan-runtime.html#_instrumenting_statically_linked_applications[statically

+linked applications] and link:darshan-runtime.html#_instrumenting_dynamically_linked_applications[

+dynamically linked applications], and using link:darshan-runtime.html#_runtime_environment_variables[

+runtime environment variables] are equally applicable to both versions.

+

+However, we do provide some suggestions and expectations for system administrators to keep in

+mind when upgrading to Darshan 3.x:

+

+* Darshan 2.x and Darshan 3.x produce incompatible log file formats

+    ** log files named *.darshan.gz or *.darshan.bz2: Darshan 2.x format

+    ** log files named *.darshan: Darshan 3.x format

+        *** a field in the log file header indicates underlying compression method in 3.x logs

+    ** There is currently no tool for converting 2.x logs into the 3.x log format.

+    ** The `darshan-logutils` library will provide error messages to indicate whether a given

+log file is incompatible with the correspnonding library version. 

+

+* We encourage administrators to use the same log file directory for version 3.x as had been

+used for version 2.x.

+    ** Within this directory, the determination on which set of log utilities (version 2.x

+or version 3.x) to use can be based on the file extension for a given log (as explained

+above).

+

</span> == Runtime environment variables

 The Darshan library honors the following environment variables to modify

<span style="color: #aaaaaa">@@ -477,12 +505,9 @@ behavior at runtime:

</span> * DARSHAN_DISABLE: disables Darshan instrumentation

 * DARSHAN_INTERNAL_TIMING: enables internal instrumentation that will print the time required to startup and shutdown Darshan to stderr at run time.

 * DARSHAN_LOGHINTS: specifies the MPI-IO hints to use when storing the Darshan output file.  The format is a semicolon-delimited list of key=value pairs, for example: hint1=value1;hint2=value2

<span style="color: #000000;background-color: #ffdddd">-* DARSHAN_DISABLE_TIMING: disables the subset of Darshan instrumentation that gathers timing information

-* DARSHAN_MEMALIGN: specifies a value for memory alignment (CP_MEM_ALIGNMENT)

</span><span style="color: #000000;background-color: #ddffdd">+* DARSHAN_MEMALIGN: specifies a value for system memory alignment

</span> * DARSHAN_JOBID: specifies the name of the environment variable to use for the job identifier, such as PBS_JOBID

<span style="color: #000000;background-color: #ffdddd">-* DARSHAN_DISABLE_SHARED_REDUCTION: disables the step in Darshan aggregation

-in which files that were accessed by all ranks are collapsed into a single

-cumulative file record at rank 0.  This option retains more per-process

-information at the expense of creating larger log files.

<span style="color: #000000;background-color: #ddffdd">+* DARSHAN_DISABLE_SHARED_REDUCTION: disables the step in Darshan aggregation in which files that were accessed by all ranks are collapsed into a single cumulative file record at rank 0.  This option retains more per-process information at the expense of creating larger log files. Note that it is up to individual instrumentation module implementations whether this environment variable is actually honored.

* DARSHAN_LOGPATH: specifies the path to write Darshan log files to. Note that this directory needs to be formatted using the darshan-mk-log-dirs script.

<span style="color: #000000;background-color: #ffdddd">-* DARSHAN_LOGFILE: specifies the path (directory + Darshan log file name) to write the output Darshan log to. This overrides the default Darshan behavior of automatically generating a log file name and adding to a log file directory formatted using darshan-mk-log-dirs script.

<span style="color: #000000;background-color: #ddffdd">+* DARSHAN_LOGFILE: specifies the path (directory + Darshan log file name) to write the output Darshan log to. This overrides the default Darshan behavior of automatically generating a log file name and adding it to a log file directory formatted using darshan-mk-log-dirs script.

+* DARSHAN_MODMEM: specifies the maximum amount of memory (in MiB) a Darshan instrumentation module can consume at runtime.

</span></code></pre>

<br>

</li>

<li id='diff-23'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-23'>

<strong>

darshan-runtime/lib/darshan-bgq.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-bgq.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,330 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _XOPEN_SOURCE 500

+#define _GNU_SOURCE

+

+#include "darshan-runtime-config.h"

+#include <stdio.h>

+#include <unistd.h>

+#include <stdlib.h>

+#include <pthread.h>

+#include <assert.h>

+

+#include "uthash.h"

+#include "darshan.h"

+#include "darshan-dynamic.h"

+

+#include <mpix.h>

+#include <spi/include/kernel/location.h>

+#include <spi/include/kernel/process.h>

+#include <firmware/include/personality.h>

+

+/*

+ * Simple module which captures BG/Q hardware specific information about 

+ * the job.

+ * 

+ * This module does not intercept any system calls. It just pulls data

+ * from the personality struct at initialization.

+ */

+

+

+/*

+ * Global runtime struct for tracking data needed at runtime

+ */

+struct bgq_runtime

+{

+    struct darshan_bgq_record record;

+};

+

+static struct bgq_runtime *bgq_runtime = NULL;

+static pthread_mutex_t bgq_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+

+/* the instrumentation_disabled flag is used to toggle functions on/off */

+static int instrumentation_disabled = 0;

+

+/* my_rank indicates the MPI rank of this process */

+static int my_rank = -1;

+static int darshan_mem_alignment = 1;

+

+/* internal helper functions for the BGQ module */

+void bgq_runtime_initialize(void);

+

+/* forward declaration for module functions needed to interface with darshan-core */

+static void bgq_begin_shutdown(void);

+static void bgq_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs, int shared_rec_count, void **buffer, int *size);

+static void bgq_shutdown(void);

+

+/* macros for obtaining/releasing the BGQ module lock */

+#define BGQ_LOCK() pthread_mutex_lock(&bgq_runtime_mutex)

+#define BGQ_UNLOCK() pthread_mutex_unlock(&bgq_runtime_mutex)

+

+/*

+ * Function which updates all the counter data

+ */

+static void capture(struct darshan_bgq_record *rec)

+{

+    Personality_t person;

+    int r;

+

+    rec->counters[BGQ_CSJOBID] = Kernel_GetJobID();

+    rec->counters[BGQ_RANKSPERNODE] = Kernel_ProcessCount();

+    rec->counters[BGQ_INODES] = MPIX_IO_node_id();

+

+    r = Kernel_GetPersonality(&person, sizeof(person));

+    if (r == 0)

+    {

+        rec->counters[BGQ_NNODES] = ND_TORUS_SIZE(person.Network_Config);

+        rec->counters[BGQ_ANODES] = person.Network_Config.Anodes;

+        rec->counters[BGQ_BNODES] = person.Network_Config.Bnodes;

+        rec->counters[BGQ_CNODES] = person.Network_Config.Cnodes;

+        rec->counters[BGQ_DNODES] = person.Network_Config.Dnodes;

+        rec->counters[BGQ_ENODES] = person.Network_Config.Enodes;

+        rec->counters[BGQ_TORUSENABLED] =

+            (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_A) == ND_ENABLE_TORUS_DIM_A) << 0) |

+            (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_B) == ND_ENABLE_TORUS_DIM_B) << 1) |

+            (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_C) == ND_ENABLE_TORUS_DIM_C) << 2) |

+            (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_D) == ND_ENABLE_TORUS_DIM_D) << 3) |

+            (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_E) == ND_ENABLE_TORUS_DIM_E) << 4);

+

+        rec->counters[BGQ_DDRPERNODE] = person.DDR_Config.DDRSizeMB;

+    }

+

+    rec->rank = my_rank;

+    rec->fcounters[BGQ_F_TIMESTAMP] = darshan_core_wtime();

+

+    return;

+}

+

+/**********************************************************

+ * Internal functions for manipulating BGQ module state *

+ **********************************************************/

+

+void bgq_runtime_initialize()

+{

+    /* struct of function pointers for interfacing with darshan-core */

+    struct darshan_module_funcs bgq_mod_fns =

+    {

+        .begin_shutdown = bgq_begin_shutdown,

+        .get_output_data = bgq_get_output_data,

+        .shutdown = bgq_shutdown

+    };

+    int mem_limit;

+    char *recname = "darshan-internal-bgq";

+

+    BGQ_LOCK();

+

+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(bgq_runtime || instrumentation_disabled)

+        return;

+

+    /* register the BG/Q module with the darshan-core component */

+    darshan_core_register_module(

+        DARSHAN_BGQ_MOD,

+        &bgq_mod_fns,

+        &my_rank,

+        &mem_limit,

+        &darshan_mem_alignment);

+

+    /* return if no memory assigned by darshan-core */

+    if(mem_limit == 0)

+    {

+        instrumentation_disabled = 1;

+        BGQ_UNLOCK();

+        return;

+    }

+

+    /* not enough memory to fit bgq module */

+    if (mem_limit < sizeof(*bgq_runtime))

+    {

+        instrumentation_disabled = 1;

+        BGQ_UNLOCK();

+        return;

+    }

+

+    /* initialize module's global state */

+    bgq_runtime = malloc(sizeof(*bgq_runtime));

+    if(!bgq_runtime)

+    {

+        instrumentation_disabled = 1;

+        BGQ_UNLOCK();

+        return;

+    }

+    memset(bgq_runtime, 0, sizeof(*bgq_runtime));

+

+    darshan_core_register_record(

+        recname,

+        strlen(recname),

+        DARSHAN_BGQ_MOD,

+        1,

+        0,

+        &bgq_runtime->record.f_id,

+        &bgq_runtime->record.alignment);

+

+    /* if record is set to 0, darshan-core is out of space and will not

+     * track this record, so we should avoid tracking it, too

+     */

+    if(bgq_runtime->record.f_id == 0)

+    {

+        instrumentation_disabled = 1;

+        free(bgq_runtime);

+        bgq_runtime = NULL;

+        BGQ_UNLOCK();

+        return;

+    }

+

+    capture(&bgq_runtime->record);

+

+    BGQ_UNLOCK();

+

+    return;

+}

+

+/* Perform any necessary steps prior to shutting down for the BGQ module. */

+static void bgq_begin_shutdown()

+{

+    BGQ_LOCK();

+

+    /* In general, we want to disable all wrappers while Darshan shuts down. 

+     * This is to avoid race conditions and ensure data consistency, as

+     * executing wrappers could potentially modify module state while Darshan

+     * is in the process of shutting down. 

+     */

+    instrumentation_disabled = 1;

+

+    BGQ_UNLOCK();

+

+    return;

+}

+

+static int cmpr(const void *p1, const void *p2)

+{

+    const uint64_t *a = (uint64_t*) p1;

+    const uint64_t *b = (uint64_t*) p2;

+    return ((*a == *b) ?  0 : ((*a < *b) ? -1 : 1));

+}

+

+/* Pass output data for the BGQ module back to darshan-core to log to file. */

+static void bgq_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **buffer,

+    int *size)

+{

+    /* Just set the output buffer to point at the array of the BGQ module's

+     * I/O records, and set the output size according to the number of records

+     * currently being tracked.

+     */

+    int nprocs;

+    int result;

+    uint64_t *ion_ids;

+

+    if (my_rank == 0)

+    {

+        DARSHAN_MPI_CALL(PMPI_Comm_size)(mod_comm, &nprocs);

+        ion_ids = malloc(sizeof(*ion_ids)*nprocs);

+        result = (ion_ids != NULL); 

+    }

+    DARSHAN_MPI_CALL(PMPI_Bcast)(&result, 1, MPI_INT, 0, mod_comm);

+

+    if (bgq_runtime && result)

+    {

+        int i, found;

+        uint64_t val;

+

+        DARSHAN_MPI_CALL(PMPI_Gather)(&bgq_runtime->record.counters[BGQ_INODES],

+                                      1,

+                                      MPI_LONG_LONG_INT,

+                                      ion_ids,

+                                      1,

+                                      MPI_LONG_LONG_INT,

+                                      0,

+                                      mod_comm);

+        if (my_rank == 0)

+        {

+            qsort(ion_ids, nprocs, sizeof(*ion_ids), cmpr);

+            for (i = 1, val = ion_ids[0], found = 1; i < nprocs; i++)

+            {

+                if (val != ion_ids[i])

+                {

+                    val = ion_ids[i];

+                    found += 1;

+                }

+            }

+            bgq_runtime->record.counters[BGQ_INODES] = found;

+        }

+    }

+

+    if ((bgq_runtime) && (my_rank == 0))

+    {

+        *buffer = &bgq_runtime->record;

+        *size = sizeof(struct darshan_bgq_record);

+    }

+    else

+    {

+        *buffer = NULL;

+        *size   = 0;

+    }

+

+    return;

+}

+

+/* Shutdown the BGQ module by freeing up all data structures. */

+static void bgq_shutdown()

+{

+    if (bgq_runtime)

+    {

+        free(bgq_runtime);

+        bgq_runtime = NULL;

+    }

+

+    return;

+}

+

+#if 0

+static void bgq_record_reduction_op(

+    void* infile_v,

+    void* inoutfile_v,

+    int* len,

+    MPI_Datatype *datatype)

+{

+    int i;

+    int j;

+    struct darshan_bgq_record *infile = infile_v;

+    struct darshan_bgq_record *inoutfile = inoutfile_v;

+

+    for (i = 0; i<*len; i++)

+    {

+        for (j = 0; j < BGQ_NUM_INDICES; j++)

+        {

+            if (infile->counters[j] != inoutfile->counters[j])

+            {

+                // unexpected

+                fprintf(stderr,

+                        "%lu counter mismatch: %d [%lu] [%lu]\n",

+                        infile->f_id,

+                        j,

+                        infile->counters[j],

+                        inoutfile->counters[j]);

+            }

+        }

+        infile++;

+        inoutfile++;

+    }

+

+    return;

+}

+#endif

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-24'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-24'>

<strong>

darshan-runtime/lib/darshan-common.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-common.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,204 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+#define _XOPEN_SOURCE 500

+#define _GNU_SOURCE

+

+#include "darshan-runtime-config.h"

+

+#include <stdio.h>

+#include <stdlib.h>

+#include <string.h>

+#include <limits.h>

+#include <search.h>

+#include <assert.h>

+

+#include "darshan.h"

+

+static int darshan_common_val_compare(const void* a_p, const void* b_p);

+static void darshan_common_val_walker(const void* nodep, const VISIT which,

+    const int depth);

+

+char* darshan_clean_file_path(const char* path)

+{

+    char* newpath = NULL;

+    char* cwd = NULL;

+    char* filter = NULL;

+

+    if(!path || strlen(path) < 1)

+        return(NULL);

+

+    if(path[0] == '/')

+    {

+        /* it is already an absolute path */

+        newpath = malloc(strlen(path)+1);

+        if(newpath)

+        {

+            strcpy(newpath, path);

+        }

+    }

+    else

+    {

+        /* handle relative path */

+        cwd = malloc(PATH_MAX);

+        if(cwd)

+        {

+            if(getcwd(cwd, PATH_MAX))

+            {

+                newpath = malloc(strlen(path) + strlen(cwd) + 2);

+                if(newpath)

+                {

+                    sprintf(newpath, "%s/%s", cwd, path);

+                }

+            }

+            free(cwd);

+        }

+    }

+

+    if(!newpath)

+        return(NULL);

+

+    /* filter out any double slashes */

+    while((filter = strstr(newpath, "//")))

+    {

+        /* shift down one character */

+        memmove(filter, &filter[1], (strlen(&filter[1]) + 1));

+    }

+

+    /* filter out any /./ instances */

+    while((filter = strstr(newpath, "/./")))

+    {

+        /* shift down two characters */

+        memmove(filter, &filter[2], (strlen(&filter[2]) + 1));

+    }

+

+    /* return result */

+    return(newpath);

+}

+

+/* HACK: global variables for determining 4 most common values */

+static int64_t* walker_val_p = NULL;

+static int64_t* walker_cnt_p = NULL;

+

+void darshan_common_val_counter(void **common_val_root, int *common_val_count,

+    int64_t val)

+{

+    struct darshan_common_val_counter* counter;

+    struct darshan_common_val_counter* found;

+    struct darshan_common_val_counter tmp_counter;

+    void* tmp;

+

+    /* don't count any values of 0 */

+    if(val == 0)

+        return;

+

+    /* check to see if this val is already recorded */

+    tmp_counter.val = val;

+    tmp_counter.freq = 1;

+    tmp = tfind(&tmp_counter, common_val_root, darshan_common_val_compare);

+    if(tmp)

+    {

+        found = *(struct darshan_common_val_counter**)tmp;

+        found->freq++;

+        return;

+    }

+

+    /* we can add a new one as long as we haven't hit the limit */

+    if(*common_val_count < DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT)

+    {

+        counter = malloc(sizeof(*counter));

+        if(!counter)

+        {

+            return;

+        }

+

+        counter->val = val;

+        counter->freq = 1;

+

+        tmp = tsearch(counter, common_val_root, darshan_common_val_compare);

+        found = *(struct darshan_common_val_counter**)tmp;

+        /* if we get a new answer out here we are in trouble; this was

+         * already checked with the tfind()

+         */

+        assert(found == counter);

+

+        (*common_val_count)++;

+    }

+

+    return;

+}

+

+void darshan_walk_common_vals(void *common_val_root, int64_t* val_p,

+    int64_t* cnt_p)

+{

+    walker_val_p = val_p;

+    walker_cnt_p = cnt_p;

+

+    twalk(common_val_root, darshan_common_val_walker);

+    tdestroy(common_val_root, free);

+

+    return;

+}

+

+static void darshan_common_val_walker(const void* nodep, const VISIT which,

+    const int depth)

+{

+    struct darshan_common_val_counter* counter;

+

+    switch (which)

+    {

+        case postorder:

+        case leaf:

+            counter = *(struct darshan_common_val_counter**)nodep;

+            DARSHAN_COMMON_VAL_COUNTER_INC(walker_val_p, walker_cnt_p,

+                counter->val, counter->freq);

+        default:

+            break;

+    }

+

+    return;

+}

+

+static int darshan_common_val_compare(const void* a_p, const void* b_p)

+{

+    const struct darshan_common_val_counter* a = a_p;

+    const struct darshan_common_val_counter* b = b_p;

+

+    if(a->val < b->val)

+        return(-1);

+    if(a->val > b->val)

+        return(1);

+    return(0);

+}

+

+void darshan_variance_reduce(void *invec, void *inoutvec, int *len,

+    MPI_Datatype *dt)

+{

+    int i;

+    struct darshan_variance_dt *X = invec;

+    struct darshan_variance_dt *Y = inoutvec;

+    struct darshan_variance_dt  Z;

+

+    for (i=0; i<*len; i++,X++,Y++)

+    {

+        Z.n = X->n + Y->n;

+        Z.T = X->T + Y->T;

+        Z.S = X->S + Y->S + (X->n/(Y->n*Z.n)) *

+           ((Y->n/X->n)*X->T - Y->T) * ((Y->n/X->n)*X->T - Y->T);

+

+        *Y = Z;

+    }

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-25'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-25'>

<strong>

darshan-runtime/lib/darshan-mpi-init-finalize.c

</strong>

→

<strong>

darshan-runtime/lib/darshan-core-init-finalize.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-mpi-init-finalize.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-core-init-finalize.c

</span><span style="color: #aaaaaa">@@ -1,35 +1,24 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2012 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #define _XOPEN_SOURCE 500

<span style="color: #000000;background-color: #ffdddd">-#define _GNU_SOURCE /* for RTLD_NEXT */

</span><span style="color: #000000;background-color: #ddffdd">+#define _GNU_SOURCE

</span> 

 #include "darshan-runtime-config.h"

<span style="color: #000000;background-color: #ffdddd">-#include <stdlib.h>

</span> #include <stdio.h>

<span style="color: #000000;background-color: #ddffdd">+#include <stdlib.h>

+#include <mpi.h>

</span> 

<span style="color: #000000;background-color: #ffdddd">-#include "mpi.h"

</span> #include "darshan.h"

<span style="color: #000000;background-color: #ffdddd">-#include "darshan-dynamic.h"

</span><span style="color: #000000;background-color: #ddffdd">+#include "darshan-core.h"

</span> 

 #ifdef DARSHAN_PRELOAD

<span style="color: #000000;background-color: #ffdddd">-#include <dlfcn.h>

-

-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  ret (*__real_ ## name)args = NULL;

-

-#define MAP_OR_FAIL(func) \

-    __real_ ## func = dlsym(RTLD_NEXT, #func); \

-    if (!(__real_ ## func)) { \

-        fprintf(stderr, "Darshan failed to map symbol: %s\n", #func); \

-    }

-

</span> DARSHAN_FORWARD_DECL(PMPI_File_close, int, (MPI_File *fh));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(PMPI_File_set_size, int, (MPI_File fh, MPI_Offset size));

</span> DARSHAN_FORWARD_DECL(PMPI_File_iread_at, int, (MPI_File fh, MPI_Offset offset, void *buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST *request));

 DARSHAN_FORWARD_DECL(PMPI_File_iread, int, (MPI_File fh, void  *buf, int  count, MPI_Datatype  datatype, __D_MPI_REQUEST  *request));

 DARSHAN_FORWARD_DECL(PMPI_File_iread_shared, int, (MPI_File fh, void *buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST *request));

<span style="color: #aaaaaa">@@ -116,7 +105,6 @@ DARSHAN_FORWARD_DECL(PMPI_File_write_shared, int, (MPI_File fh, void *buf, int c

</span> DARSHAN_FORWARD_DECL(PMPI_Finalize, int, ());

 DARSHAN_FORWARD_DECL(PMPI_Init, int, (int *argc, char ***argv));

 DARSHAN_FORWARD_DECL(PMPI_Init_thread, int, (int *argc, char ***argv, int required, int *provided));

<span style="color: #000000;background-color: #ffdddd">-

</span> DARSHAN_FORWARD_DECL(PMPI_Wtime, double, ());

 DARSHAN_FORWARD_DECL(PMPI_Allreduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, MPI_Comm comm));

 DARSHAN_FORWARD_DECL(PMPI_Bcast, int, (void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm));

<span style="color: #aaaaaa">@@ -128,12 +116,26 @@ DARSHAN_FORWARD_DECL(PMPI_Type_contiguous, int, (int count, MPI_Datatype oldtype

</span> DARSHAN_FORWARD_DECL(PMPI_Type_extent, int, (MPI_Datatype datatype, MPI_Aint *extent));

 DARSHAN_FORWARD_DECL(PMPI_Type_free, int, (MPI_Datatype *datatype));

 DARSHAN_FORWARD_DECL(PMPI_Type_hindexed, int, (int count, int *array_of_blocklengths, MPI_Aint *array_of_displacements, MPI_Datatype oldtype, MPI_Datatype *newtype));

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *num_integers, int *num_addresses, int *num_datatypes, int *combiner));

+DARSHAN_FORWARD_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));

</span> DARSHAN_FORWARD_DECL(PMPI_Op_create, int, (MPI_User_function *function, int commute, MPI_Op *op));

 DARSHAN_FORWARD_DECL(PMPI_Op_free, int, (MPI_Op *op));

<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST

+DARSHAN_FORWARD_DECL(PMPI_Reduce, int, (const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));

+#else

</span> DARSHAN_FORWARD_DECL(PMPI_Reduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *num_integers, int *num_addresses, int *num_datatypes, int *combiner));

-DARSHAN_FORWARD_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));

-

</span><span style="color: #000000;background-color: #ddffdd">+#endif

+#ifdef HAVE_MPIIO_CONST

+DARSHAN_FORWARD_DECL(PMPI_Send, int, (const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));

+#else

+DARSHAN_FORWARD_DECL(PMPI_Send, int, (void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));

+#endif

+DARSHAN_FORWARD_DECL(PMPI_Recv, int, (void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status));

+#ifdef HAVE_MPIIO_CONST

+DARSHAN_FORWARD_DECL(PMPI_Gather, int, (const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));

+#else

+DARSHAN_FORWARD_DECL(PMPI_Gather, int, (void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));

+#endif

</span> 

 void resolve_mpi_symbols (void)

 {

<span style="color: #aaaaaa">@@ -141,7 +143,6 @@ void resolve_mpi_symbols (void)

</span>      * Overloaded functions

      */

     MAP_OR_FAIL(PMPI_File_close);

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(PMPI_File_set_size);

</span>     MAP_OR_FAIL(PMPI_File_iread_at);

     MAP_OR_FAIL(PMPI_File_iread);

     MAP_OR_FAIL(PMPI_File_iread_shared);

<span style="color: #aaaaaa">@@ -172,7 +173,6 @@ void resolve_mpi_symbols (void)

</span>     MAP_OR_FAIL(PMPI_Finalize);

     MAP_OR_FAIL(PMPI_Init);

     MAP_OR_FAIL(PMPI_Init_thread);

<span style="color: #000000;background-color: #ffdddd">-

</span>     /*

      * These function are not intercepted but are used

      * by darshan itself.

<span style="color: #aaaaaa">@@ -187,12 +187,15 @@ void resolve_mpi_symbols (void)

</span>     MAP_OR_FAIL(PMPI_Type_contiguous);

     MAP_OR_FAIL(PMPI_Type_extent);

     MAP_OR_FAIL(PMPI_Type_free);

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(PMPI_Type_size);

</span>     MAP_OR_FAIL(PMPI_Type_hindexed);

<span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(PMPI_Type_get_envelope);

+    MAP_OR_FAIL(PMPI_Type_size);

</span>     MAP_OR_FAIL(PMPI_Op_create);

     MAP_OR_FAIL(PMPI_Op_free);

     MAP_OR_FAIL(PMPI_Reduce);

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(PMPI_Type_get_envelope);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(PMPI_Send);

+    MAP_OR_FAIL(PMPI_Recv);

+    MAP_OR_FAIL(PMPI_Gather);

</span> 

     return;

 }

<span style="color: #aaaaaa">@@ -213,12 +216,20 @@ int MPI_Init(int *argc, char ***argv)

</span>         return(ret);

     }

<span style="color: #000000;background-color: #ffdddd">-    darshan_mpi_initialize(argc, argv);

</span><span style="color: #000000;background-color: #ddffdd">+    if(argc && argv)

+    {

+        darshan_core_initialize(*argc, *argv);

+    }

+    else

+    {

+        /* we don't see argc and argv here in fortran */

+        darshan_core_initialize(0, NULL);

+    }

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int MPI_Init_thread (int *argc, char ***argv, int required, int *provided)

</span><span style="color: #000000;background-color: #ddffdd">+int MPI_Init_thread(int *argc, char ***argv, int required, int *provided)

</span> {

     int ret;

<span style="color: #aaaaaa">@@ -227,12 +238,20 @@ int MPI_Init_thread (int *argc, char ***argv, int required, int *provided)

</span> #endif

     ret = DARSHAN_MPI_CALL(PMPI_Init_thread)(argc, argv, required, provided);

<span style="color: #000000;background-color: #ffdddd">-    if (ret != MPI_SUCCESS)

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret != MPI_SUCCESS)

</span>     {

         return(ret);

     }

<span style="color: #000000;background-color: #ffdddd">-    darshan_mpi_initialize(argc, argv);

</span><span style="color: #000000;background-color: #ddffdd">+    if(argc && argv)

+    {

+        darshan_core_initialize(*argc, *argv);

+    }

+    else

+    {

+        /* we don't see argc and argv here in fortran */

+        darshan_core_initialize(0, NULL);

+    }

</span> 

     return(ret);

 }

<span style="color: #aaaaaa">@@ -241,12 +260,17 @@ int MPI_Finalize(void)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    if(getenv("DARSHAN_INTERNAL_TIMING"))

-        darshan_shutdown(1);

-    else

-        darshan_shutdown(0);

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_core_shutdown();

</span> 

     ret = DARSHAN_MPI_CALL(PMPI_Finalize)();

     return(ret);

 }

<span style="color: #000000;background-color: #ddffdd">+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-26'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-26'>

<strong>

darshan-runtime/lib/darshan-core.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-core.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,1796 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _XOPEN_SOURCE 500

+#define _GNU_SOURCE

+

+#include "darshan-runtime-config.h"

+

+#include <stdio.h>

+#ifdef HAVE_MNTENT_H

+#include <mntent.h>

+#endif

+#include <stdlib.h>

+#include <string.h>

+#include <time.h>

+#include <limits.h>

+#include <pthread.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <sys/vfs.h>

+#include <zlib.h>

+#include <mpi.h>

+#include <assert.h>

+

+#include "uthash.h"

+#include "darshan.h"

+#include "darshan-core.h"

+#include "darshan-dynamic.h"

+

+extern char* __progname;

+extern char* __progname_full;

+

+/* internal variable delcarations */

+static struct darshan_core_runtime *darshan_core = NULL;

+static pthread_mutex_t darshan_core_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+static int my_rank = -1;

+static int nprocs = -1;

+static int darshan_mem_alignment = 1;

+

+/* paths prefixed with the following directories are not traced by darshan */

+char* darshan_path_exclusions[] = {

+"/etc/",

+"/dev/",

+"/usr/",

+"/bin/",

+"/boot/",

+"/lib/",

+"/opt/",

+"/sbin/",

+"/sys/",

+"/proc/",

+NULL

+};

+

+#ifdef DARSHAN_BGQ

+extern void bgq_runtime_initialize();

+#endif

+

+/* array of init functions for modules which need to be statically

+ * initialized by darshan at startup time

+ */

+void (*mod_static_init_fns[])(void) =

+{

+#ifdef DARSHAN_BGQ

+    &bgq_runtime_initialize,

+#endif

+    NULL

+};

+

+#define DARSHAN_CORE_LOCK() pthread_mutex_lock(&darshan_core_mutex)

+#define DARSHAN_CORE_UNLOCK() pthread_mutex_unlock(&darshan_core_mutex)

+

+/* FS mount information */

+#define DARSHAN_MAX_MNTS 64

+#define DARSHAN_MAX_MNT_PATH 256

+#define DARSHAN_MAX_MNT_TYPE 32

+struct mnt_data

+{

+    int block_size;

+    char path[DARSHAN_MAX_MNT_PATH];

+    char type[DARSHAN_MAX_MNT_TYPE];

+};

+static struct mnt_data mnt_data_array[DARSHAN_MAX_MNTS];

+static int mnt_data_count = 0;

+

+/* prototypes for internal helper functions */

+static void darshan_get_logfile_name(

+    char* logfile_name, int jobid, struct tm* start_tm);

+static void darshan_log_record_hints_and_ver(

+    struct darshan_core_runtime* core);

+static void darshan_get_exe_and_mounts_root(

+    struct darshan_core_runtime *core, char* trailing_data,

+    int space_left);

+static char* darshan_get_exe_and_mounts(

+    struct darshan_core_runtime *core);

+static void darshan_block_size_from_path(

+    const char *path, int *block_size);

+static void darshan_get_shared_records(

+    struct darshan_core_runtime *core, darshan_record_id **shared_recs,

+    int *shared_rec_cnt);

+static int darshan_log_open_all(

+    char *logfile_name, MPI_File *log_fh);

+static int darshan_deflate_buffer(

+    void **pointers, int *lengths, int count, char *comp_buf,

+    int *comp_buf_length);

+static int darshan_log_write_record_hash(

+    MPI_File log_fh, struct darshan_core_runtime *core,

+    uint64_t *inout_off);

+static int darshan_log_append_all(

+    MPI_File log_fh, struct darshan_core_runtime *core, void *buf,

+    int count, uint64_t *inout_off);

+static void darshan_core_cleanup(

+    struct darshan_core_runtime* core);

+

+/* *********************************** */

+

+void darshan_core_initialize(int argc, char **argv)

+{

+    struct darshan_core_runtime *init_core = NULL;

+    int i;

+    int internal_timing_flag = 0;

+    double init_start, init_time, init_max;

+    char *envstr;

+    char* truncate_string = "<TRUNCATED>";

+    int truncate_offset;

+    int chars_left = 0;

+    int ret;

+    int tmpval;

+

+    DARSHAN_MPI_CALL(PMPI_Comm_size)(MPI_COMM_WORLD, &nprocs);

+    DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &my_rank);

+

+    if(getenv("DARSHAN_INTERNAL_TIMING"))

+        internal_timing_flag = 1;

+

+    if(internal_timing_flag)

+        init_start = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    /* setup darshan runtime if darshan is enabled and hasn't been initialized already */

+    if(!getenv("DARSHAN_DISABLE") && !darshan_core)

+    {

+        #if (__DARSHAN_MEM_ALIGNMENT < 1)

+            #error Darshan must be configured with a positive value for --with-mem-align

+        #endif

+        envstr = getenv(DARSHAN_MEM_ALIGNMENT_OVERRIDE);

+        if(envstr)

+        {

+            ret = sscanf(envstr, "%d", &tmpval);

+            /* silently ignore if the env variable is set poorly */

+            if(ret == 1 && tmpval > 0)

+            {

+                darshan_mem_alignment = tmpval;

+            }

+        }

+        else

+        {

+            darshan_mem_alignment = __DARSHAN_MEM_ALIGNMENT;

+        }

+

+        /* avoid floating point errors on faulty input */

+        if (darshan_mem_alignment < 1)

+        {

+            darshan_mem_alignment = 1;

+        }

+

+        /* allocate structure to track darshan_core_runtime information */

+        init_core = malloc(sizeof(*init_core));

+        if(init_core)

+        {

+            memset(init_core, 0, sizeof(*init_core));

+

+            init_core->log_job.uid = getuid();

+            init_core->log_job.start_time = time(NULL);

+            init_core->log_job.nprocs = nprocs;

+            init_core->wtime_offset = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+            /* record exe and arguments */

+            for(i=0; i<argc; i++)

+            {

+                chars_left = DARSHAN_EXE_LEN-strlen(init_core->exe);

+                strncat(init_core->exe, argv[i], chars_left);

+                if(i < (argc-1))

+                {

+                    chars_left = DARSHAN_EXE_LEN-strlen(init_core->exe);

+                    strncat(init_core->exe, " ", chars_left);

+                }

+            }

+

+            /* if we don't see any arguments, then use glibc symbol to get

+             * program name at least (this happens in fortran)

+             */

+            if(argc == 0)

+            {

+                chars_left = DARSHAN_EXE_LEN-strlen(init_core->exe);

+                strncat(init_core->exe, __progname_full, chars_left);

+                chars_left = DARSHAN_EXE_LEN-strlen(init_core->exe);

+                strncat(init_core->exe, " <unknown args>", chars_left);

+            }

+

+            if(chars_left == 0)

+            {

+                /* we ran out of room; mark that string was truncated */

+                truncate_offset = DARSHAN_EXE_LEN - strlen(truncate_string);

+                sprintf(&init_core->exe[truncate_offset], "%s",

+                    truncate_string);

+            }

+

+            /* collect information about command line and mounted file systems */

+            init_core->trailing_data = darshan_get_exe_and_mounts(init_core);

+

+            /* bootstrap any modules with static initialization routines */

+            i = 0;

+            while(mod_static_init_fns[i])

+            {

+                (*mod_static_init_fns[i])();

+                i++;

+            }

+        }

+    }

+

+    if(internal_timing_flag)

+    {

+        init_time = DARSHAN_MPI_CALL(PMPI_Wtime)() - init_start;

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&init_time, &init_max, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        if(my_rank == 0)

+        {

+            fprintf(stderr, "#darshan:<op>\t<nprocs>\t<time>\n");

+            fprintf(stderr, "darshan:init\t%d\t%f\n", nprocs, init_max);

+        }

+    }

+

+    /* if darshan was successfully initialized, set the global pointer */

+    if(init_core)

+        darshan_core = init_core;

+

+    return;

+}

+

+void darshan_core_shutdown()

+{

+    int i;

+    char *logfile_name;

+    struct darshan_core_runtime *final_core;

+    int internal_timing_flag = 0;

+    char *envjobid;

+    char *jobid_str;

+    int jobid;

+    struct tm *start_tm;

+    time_t start_time_tmp;

+    int ret = 0;

+    int all_ret = 0;

+    int64_t first_start_time;

+    int64_t last_end_time;

+    int local_mod_use[DARSHAN_MAX_MODS] = {0};

+    int global_mod_use_count[DARSHAN_MAX_MODS] = {0};

+    darshan_record_id *shared_recs;

+    darshan_record_id *mod_shared_recs;

+    int shared_rec_cnt = 0;

+    double start_log_time;

+    double open1, open2;

+    double job1, job2;

+    double rec1, rec2;

+    double mod1[DARSHAN_MAX_MODS] = {0};

+    double mod2[DARSHAN_MAX_MODS] = {0};

+    double header1, header2;

+    double tm_end;

+    uint64_t gz_fp = 0;

+    MPI_File log_fh;

+    MPI_Status status;

+

+    if(getenv("DARSHAN_INTERNAL_TIMING"))

+        internal_timing_flag = 1;

+

+    start_log_time = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    /* disable darhan-core while we shutdown */

+    DARSHAN_CORE_LOCK();

+    if(!darshan_core)

+    {

+        DARSHAN_CORE_UNLOCK();

+        return;

+    }

+    final_core = darshan_core;

+    darshan_core = NULL;

+

+    final_core->comp_buf = malloc(DARSHAN_COMP_BUF_SIZE);

+    if(!(final_core->comp_buf))

+    {

+        darshan_core_cleanup(final_core);

+        return;

+    }

+

+    /* we also need to set which modules were registered on this process and

+     * call into those modules and give them a chance to perform any necessary

+     * pre-shutdown steps.

+     */

+    for(i = 0; i < DARSHAN_MAX_MODS; i++)

+    {

+        if(final_core->mod_array[i])

+        {

+            local_mod_use[i] = 1;

+            final_core->mod_array[i]->mod_funcs.begin_shutdown();

+        }

+    }

+    DARSHAN_CORE_UNLOCK();

+

+    logfile_name = malloc(PATH_MAX);

+    if(!logfile_name)

+    {

+        darshan_core_cleanup(final_core);

+        return;

+    }

+

+    /* set darshan job id/metadata and constuct log file name on rank 0 */

+    if(my_rank == 0)

+    {

+        /* Use DARSHAN_JOBID_OVERRIDE for the env var for __DARSHAN_JOBID */

+        envjobid = getenv(DARSHAN_JOBID_OVERRIDE);

+        if(!envjobid)

+        {

+            envjobid = __DARSHAN_JOBID;

+        }

+

+        /* find a job id */

+        jobid_str = getenv(envjobid);

+        if(jobid_str)

+        {

+            /* in cobalt we can find it in env var */

+            ret = sscanf(jobid_str, "%d", &jobid);

+        }

+        if(!jobid_str || ret != 1)

+        {

+            /* use pid as fall back */

+            jobid = getpid();

+        }

+

+        final_core->log_job.jobid = (int64_t)jobid;

+

+        /* if we are using any hints to write the log file, then record those

+         * hints with the darshan job information

+         */

+        darshan_log_record_hints_and_ver(final_core);

+

+        /* use human readable start time format in log filename */

+        start_time_tmp = final_core->log_job.start_time;

+        start_tm = localtime(&start_time_tmp);

+

+        /* construct log file name */

+        darshan_get_logfile_name(logfile_name, jobid, start_tm);

+    }

+

+    /* broadcast log file name */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(logfile_name, PATH_MAX, MPI_CHAR, 0,

+        MPI_COMM_WORLD);

+

+    if(strlen(logfile_name) == 0)

+    {

+        /* failed to generate log file name */

+        free(logfile_name);

+        darshan_core_cleanup(final_core);

+        return;

+    }

+

+    final_core->log_job.end_time = time(NULL);

+

+    /* reduce to report first start time and last end time across all ranks

+     * at rank 0

+     */

+    DARSHAN_MPI_CALL(PMPI_Reduce)(&final_core->log_job.start_time, &first_start_time, 1, MPI_LONG_LONG, MPI_MIN, 0, MPI_COMM_WORLD);

+    DARSHAN_MPI_CALL(PMPI_Reduce)(&final_core->log_job.end_time, &last_end_time, 1, MPI_LONG_LONG, MPI_MAX, 0, MPI_COMM_WORLD);

+    if(my_rank == 0)

+    {

+        final_core->log_job.start_time = first_start_time;

+        final_core->log_job.end_time = last_end_time;

+    }

+

+    /* reduce the number of times a module was opened globally and bcast to everyone */   

+    DARSHAN_MPI_CALL(PMPI_Allreduce)(local_mod_use, global_mod_use_count, DARSHAN_MAX_MODS, MPI_INT, MPI_SUM, MPI_COMM_WORLD);

+

+    /* get a list of records which are shared across all processes */

+    darshan_get_shared_records(final_core, &shared_recs, &shared_rec_cnt);

+

+    if(internal_timing_flag)

+        open1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+    /* collectively open the darshan log file */

+    ret = darshan_log_open_all(logfile_name, &log_fh);

+    if(internal_timing_flag)

+        open2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    /* error out if unable to open log file */

+    DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,

+        MPI_LOR, MPI_COMM_WORLD);

+    if(all_ret != 0)

+    {

+        if(my_rank == 0)

+        {

+            fprintf(stderr, "darshan library warning: unable to open log file %s\n",

+                logfile_name);

+        }

+        free(logfile_name);

+        darshan_core_cleanup(final_core);

+        return;

+    }

+

+    if(internal_timing_flag)

+        job1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+    /* rank 0 is responsible for writing the compressed darshan job information */

+    if(my_rank == 0)

+    {

+        void *pointers[2] = {&final_core->log_job, final_core->trailing_data};

+        int lengths[2] = {sizeof(struct darshan_job), strlen(final_core->trailing_data)};

+        int comp_buf_sz = 0;

+

+        /* compress the job info and the trailing mount/exe data */

+        all_ret = darshan_deflate_buffer(pointers, lengths, 2,

+            final_core->comp_buf, &comp_buf_sz);

+        if(all_ret)

+        {

+            fprintf(stderr, "darshan library warning: unable to compress job data\n");

+            unlink(logfile_name);

+        }

+        else

+        {

+            /* write the job information, preallocing space for the log header */

+            gz_fp += sizeof(struct darshan_header);

+            all_ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(log_fh, gz_fp,

+                final_core->comp_buf, comp_buf_sz, MPI_BYTE, &status);

+            if(all_ret != MPI_SUCCESS)

+            {

+                fprintf(stderr, "darshan library warning: unable to write job data to log file %s\n",

+                        logfile_name);

+                unlink(logfile_name);

+                

+            }

+            gz_fp += comp_buf_sz;

+        }

+    }

+

+    /* error out if unable to write job information */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(&all_ret, 1, MPI_INT, 0, MPI_COMM_WORLD);

+    if(all_ret != 0)

+    {

+        free(logfile_name);

+        darshan_core_cleanup(final_core);

+        return;

+    }

+    if(internal_timing_flag)

+        job2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    if(internal_timing_flag)

+        rec1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+    /* write the record name->id hash to the log file */

+    final_core->log_header.rec_map.off = gz_fp;

+    ret = darshan_log_write_record_hash(log_fh, final_core, &gz_fp);

+    final_core->log_header.rec_map.len = gz_fp - final_core->log_header.rec_map.off;

+

+    /* error out if unable to write record hash */

+    DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,

+        MPI_LOR, MPI_COMM_WORLD);

+    if(all_ret != 0)

+    {

+        if(my_rank == 0)

+        {

+            fprintf(stderr, "darshan library warning: unable to write record hash to log file %s\n",

+                logfile_name);

+            unlink(logfile_name);

+        }

+        free(logfile_name);

+        darshan_core_cleanup(final_core);

+        return;

+    }

+    if(internal_timing_flag)

+        rec2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    mod_shared_recs = malloc(shared_rec_cnt * sizeof(darshan_record_id));

+    assert(mod_shared_recs);

+

+    /* loop over globally used darshan modules and:

+     *      - perform shared file reductions, if possible

+     *      - get final output buffer

+     *      - compress (zlib) provided output buffer

+     *      - append compressed buffer to log file

+     *      - add module index info (file offset/length) to log header

+     *      - shutdown the module

+     */

+    for(i = 0; i < DARSHAN_MAX_MODS; i++)

+    {

+        struct darshan_core_module* this_mod = final_core->mod_array[i];

+        struct darshan_core_record_ref *ref = NULL;

+        int mod_shared_rec_cnt = 0;

+        void* mod_buf = NULL;

+        int mod_buf_sz = 0;

+        int j;

+

+        if(global_mod_use_count[i] == 0)

+        {

+            if(my_rank == 0)

+            {

+                final_core->log_header.mod_map[i].off = 0;

+                final_core->log_header.mod_map[i].len = 0;

+            }

+            continue;

+        }

+ 

+        if(internal_timing_flag)

+            mod1[i] = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+        /* set the shared file list for this module */

+        memset(mod_shared_recs, 0, shared_rec_cnt * sizeof(darshan_record_id));

+        for(j = 0; j < shared_rec_cnt; j++)

+        {

+            HASH_FIND(hlink, final_core->rec_hash, &shared_recs[j],

+                sizeof(darshan_record_id), ref);

+            assert(ref);

+            if(DARSHAN_MOD_FLAG_ISSET(ref->global_mod_flags, i))

+            {

+                mod_shared_recs[mod_shared_rec_cnt++] = shared_recs[j];

+            }

+        }

+

+        /* if module is registered locally, get the corresponding output buffer

+         * 

+         * NOTE: this function can be used to run collective operations across

+         * modules, if there are file records shared globally.

+         */

+        if(this_mod)

+        {

+            this_mod->mod_funcs.get_output_data(MPI_COMM_WORLD, mod_shared_recs,

+                mod_shared_rec_cnt, &mod_buf, &mod_buf_sz);

+        }

+

+        /* append this module's data to the darshan log */

+        final_core->log_header.mod_map[i].off = gz_fp;

+        ret = darshan_log_append_all(log_fh, final_core, mod_buf, mod_buf_sz, &gz_fp);

+        final_core->log_header.mod_map[i].len =

+            gz_fp - final_core->log_header.mod_map[i].off;

+

+        /* error out if the log append failed */

+        DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,

+            MPI_LOR, MPI_COMM_WORLD);

+        if(all_ret != 0)

+        {

+            if(my_rank == 0)

+            {

+                fprintf(stderr,

+                    "darshan library warning: unable to write %s module data to log file %s\n",

+                    darshan_module_names[i], logfile_name);

+                unlink(logfile_name);

+            }

+            free(logfile_name);

+            darshan_core_cleanup(final_core);

+            return;

+        }

+

+        /* shutdown module if registered locally */

+        if(this_mod)

+        {

+            this_mod->mod_funcs.shutdown();

+        }

+        if(internal_timing_flag)

+            mod2[i] = DARSHAN_MPI_CALL(PMPI_Wtime)();

+    }

+

+    if(internal_timing_flag)

+        header1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+    /* write out log header, after running 2 reduction on header variables:

+     *  1) reduce 'partial_flag' variable to determine which modules ran out

+     *     of memory for storing I/O data

+     *  2) reduce 'mod_ver' array to determine which log format version each

+     *     module used for this output log

+     */

+    if(my_rank == 0)

+    {

+        DARSHAN_MPI_CALL(PMPI_Reduce)(MPI_IN_PLACE,

+            &(final_core->log_header.partial_flag), 1, MPI_UINT32_T,

+            MPI_BOR, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(MPI_IN_PLACE,

+            final_core->log_header.mod_ver, DARSHAN_MAX_MODS, MPI_UINT32_T,

+            MPI_MAX, 0, MPI_COMM_WORLD);

+

+        /* rank 0 is responsible for writing the log header */

+        /* initialize the remaining header fields */

+        strcpy(final_core->log_header.version_string, DARSHAN_LOG_VERSION);

+        final_core->log_header.magic_nr = DARSHAN_MAGIC_NR;

+        final_core->log_header.comp_type = DARSHAN_ZLIB_COMP;

+

+        all_ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(log_fh, 0, &(final_core->log_header),

+            sizeof(struct darshan_header), MPI_BYTE, &status);

+        if(all_ret != MPI_SUCCESS)

+        {

+            fprintf(stderr, "darshan library warning: unable to write header to log file %s\n",

+                    logfile_name);

+            unlink(logfile_name);

+        }

+    }

+    else

+    {

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&(final_core->log_header.partial_flag),

+            &(final_core->log_header.partial_flag), 1, MPI_UINT32_T,

+            MPI_BOR, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(final_core->log_header.mod_ver,

+            final_core->log_header.mod_ver, DARSHAN_MAX_MODS, MPI_UINT32_T,

+            MPI_MAX, 0, MPI_COMM_WORLD);

+    }

+

+    /* error out if unable to write log header */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(&all_ret, 1, MPI_INT, 0, MPI_COMM_WORLD);

+    if(all_ret != 0)

+    {

+        free(logfile_name);

+        darshan_core_cleanup(final_core);

+        return;

+    }

+    if(internal_timing_flag)

+        header2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+    DARSHAN_MPI_CALL(PMPI_File_close)(&log_fh);

+

+    /* if we got this far, there are no errors, so rename from *.darshan_partial

+     * to *-<logwritetime>.darshan, which indicates that this log file is

+     * complete and ready for analysis

+     */

+    if(my_rank == 0)

+    {

+        if(getenv("DARSHAN_LOGFILE"))

+        {

+#ifdef __DARSHAN_GROUP_READABLE_LOGS

+            chmod(logfile_name, (S_IRUSR|S_IRGRP));

+#else

+            chmod(logfile_name, (S_IRUSR));

+#endif

+        }

+        else

+        {

+            char* tmp_index;

+            double end_log_time;

+            char* new_logfile_name;

+

+            new_logfile_name = malloc(PATH_MAX);

+            if(new_logfile_name)

+            {

+                new_logfile_name[0] = '\0';

+                end_log_time = DARSHAN_MPI_CALL(PMPI_Wtime)();

+                strcat(new_logfile_name, logfile_name);

+                tmp_index = strstr(new_logfile_name, ".darshan_partial");

+                sprintf(tmp_index, "_%d.darshan", (int)(end_log_time-start_log_time+1));

+                rename(logfile_name, new_logfile_name);

+                /* set permissions on log file */

+#ifdef __DARSHAN_GROUP_READABLE_LOGS

+                chmod(new_logfile_name, (S_IRUSR|S_IRGRP));

+#else

+                chmod(new_logfile_name, (S_IRUSR));

+#endif

+                free(new_logfile_name);

+            }

+        }

+    }

+

+    free(logfile_name);

+    darshan_core_cleanup(final_core);

+

+    if(internal_timing_flag)

+    {

+        double open_tm, open_slowest;

+        double header_tm, header_slowest;

+        double job_tm, job_slowest;

+        double rec_tm, rec_slowest;

+        double mod_tm[DARSHAN_MAX_MODS], mod_slowest[DARSHAN_MAX_MODS];

+        double all_tm, all_slowest;

+

+        tm_end = DARSHAN_MPI_CALL(PMPI_Wtime)();

+

+        open_tm = open2 - open1;

+        header_tm = header2 - header1;

+        job_tm = job2 - job1;

+        rec_tm = rec2 - rec1;

+        all_tm = tm_end - start_log_time;

+        for(i = 0;i < DARSHAN_MAX_MODS; i++)

+        {

+            mod_tm[i] = mod2[i] - mod1[i];

+        }

+

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&open_tm, &open_slowest, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&header_tm, &header_slowest, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&job_tm, &job_slowest, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&rec_tm, &rec_slowest, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(&all_tm, &all_slowest, 1,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+        DARSHAN_MPI_CALL(PMPI_Reduce)(mod_tm, mod_slowest, DARSHAN_MAX_MODS,

+            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

+

+        if(my_rank == 0)

+        {

+            fprintf(stderr, "#darshan:<op>\t<nprocs>\t<time>\n");

+            fprintf(stderr, "darshan:log_open\t%d\t%f\n", nprocs, open_slowest);

+            fprintf(stderr, "darshan:job_write\t%d\t%f\n", nprocs, job_slowest);

+            fprintf(stderr, "darshan:hash_write\t%d\t%f\n", nprocs, rec_slowest);

+            fprintf(stderr, "darshan:header_write\t%d\t%f\n", nprocs, header_slowest);

+            for(i = 0; i < DARSHAN_MAX_MODS; i++)

+            {

+                if(global_mod_use_count[i])

+                    fprintf(stderr, "darshan:%s_shutdown\t%d\t%f\n", darshan_module_names[i],

+                        nprocs, mod_slowest[i]);

+            }

+            fprintf(stderr, "darshan:core_shutdown\t%d\t%f\n", nprocs, all_slowest);

+        }

+    }

+    

+    return;

+}

+

+/* *********************************** */

+

+/* construct the darshan log file name */

+static void darshan_get_logfile_name(char* logfile_name, int jobid, struct tm* start_tm)

+{

+    char* user_logfile_name;

+    char* logpath;

+    char* logname_string;

+    char* logpath_override = NULL;

+#ifdef __DARSHAN_LOG_ENV

+    char env_check[256];

+    char* env_tok;

+#endif

+    uint64_t hlevel;

+    char hname[HOST_NAME_MAX];

+    uint64_t logmod;

+    char cuser[L_cuserid] = {0};

+    int ret;

+

+    /* first, check if user specifies a complete logpath to use */

+    user_logfile_name = getenv("DARSHAN_LOGFILE");

+    if(user_logfile_name)

+    {

+        if(strlen(user_logfile_name) >= (PATH_MAX-1))

+        {

+            fprintf(stderr, "darshan library warning: user log file name too long.\n");

+            logfile_name[0] = '\0';

+        }

+        else

+        {

+            strcpy(logfile_name, user_logfile_name);

+        }

+    }

+    else

+    {

+        /* otherwise, generate the log path automatically */

+

+        /* Use DARSHAN_LOG_PATH_OVERRIDE for the value or __DARSHAN_LOG_PATH */

+        logpath = getenv(DARSHAN_LOG_PATH_OVERRIDE);

+        if(!logpath)

+        {

+#ifdef __DARSHAN_LOG_PATH

+            logpath = __DARSHAN_LOG_PATH;

+#endif

+        }

+

+        /* get the username for this job.  In order we will try each of the

+         * following until one of them succeeds:

+         *

+         * - cuserid()

+         * - getenv("LOGNAME")

+         * - snprintf(..., geteuid());

+         *

+         * Note that we do not use getpwuid() because it generally will not

+         * work in statically compiled binaries.

+         */

+

+#ifndef DARSHAN_DISABLE_CUSERID

+        cuserid(cuser);

+#endif

+

+        /* if cuserid() didn't work, then check the environment */

+        if(strcmp(cuser, "") == 0)

+        {

+            logname_string = getenv("LOGNAME");

+            if(logname_string)

+            {

+                strncpy(cuser, logname_string, (L_cuserid-1));

+            }

+        }

+

+        /* if cuserid() and environment both fail, then fall back to uid */

+        if(strcmp(cuser, "") == 0)

+        {

+            uid_t uid = geteuid();

+            snprintf(cuser, sizeof(cuser), "%u", uid);

+        }

+

+        /* generate a random number to help differentiate the log */

+        hlevel=DARSHAN_MPI_CALL(PMPI_Wtime)() * 1000000;

+        (void)gethostname(hname, sizeof(hname));

+        logmod = darshan_hash((void*)hname,strlen(hname),hlevel);

+

+        /* see if darshan was configured using the --with-logpath-by-env

+         * argument, which allows the user to specify an absolute path to

+         * place logs via an env variable.

+         */

+#ifdef __DARSHAN_LOG_ENV

+        /* just silently skip if the environment variable list is too big */

+        if(strlen(__DARSHAN_LOG_ENV) < 256)

+        {

+            /* copy env variable list to a temporary buffer */

+            strcpy(env_check, __DARSHAN_LOG_ENV);

+            /* tokenize the comma-separated list */

+            env_tok = strtok(env_check, ",");

+            if(env_tok)

+            {

+                do

+                {

+                    /* check each env variable in order */

+                    logpath_override = getenv(env_tok);

+                    if(logpath_override)

+                    {

+                        /* stop as soon as we find a match */

+                        break;

+                    }

+                }while((env_tok = strtok(NULL, ",")));

+            }

+        }

+#endif

+

+        if(logpath_override)

+        {

+            ret = snprintf(logfile_name, PATH_MAX,

+                "%s/%s_%s_id%d_%d-%d-%d-%" PRIu64 ".darshan_partial",

+                logpath_override,

+                cuser, __progname, jobid,

+                (start_tm->tm_mon+1),

+                start_tm->tm_mday,

+                (start_tm->tm_hour*60*60 + start_tm->tm_min*60 + start_tm->tm_sec),

+                logmod);

+            if(ret == (PATH_MAX-1))

+            {

+                /* file name was too big; squish it down */

+                snprintf(logfile_name, PATH_MAX,

+                    "%s/id%d.darshan_partial",

+                    logpath_override, jobid);

+            }

+        }

+        else if(logpath)

+        {

+            ret = snprintf(logfile_name, PATH_MAX,

+                "%s/%d/%d/%d/%s_%s_id%d_%d-%d-%d-%" PRIu64 ".darshan_partial",

+                logpath, (start_tm->tm_year+1900),

+                (start_tm->tm_mon+1), start_tm->tm_mday,

+                cuser, __progname, jobid,

+                (start_tm->tm_mon+1),

+                start_tm->tm_mday,

+                (start_tm->tm_hour*60*60 + start_tm->tm_min*60 + start_tm->tm_sec),

+                logmod);

+            if(ret == (PATH_MAX-1))

+            {

+                /* file name was too big; squish it down */

+                snprintf(logfile_name, PATH_MAX,

+                    "%s/id%d.darshan_partial",

+                    logpath, jobid);

+            }

+        }

+        else

+        {

+            logfile_name[0] = '\0';

+        }

+    }

+

+    return;

+}

+

+/* record any hints used to write the darshan log in the log header */

+static void darshan_log_record_hints_and_ver(struct darshan_core_runtime* core)

+{

+    char* hints;

+    char* header_hints;

+    int meta_remain = 0;

+    char* m;

+

+    /* check environment variable to see if the default MPI file hints have

+     * been overridden

+     */

+    hints = getenv(DARSHAN_LOG_HINTS_OVERRIDE);

+    if(!hints)

+    {

+        hints = __DARSHAN_LOG_HINTS;

+    }

+

+    if(!hints || strlen(hints) < 1)

+        return;

+

+    header_hints = strdup(hints);

+    if(!header_hints)

+        return;

+

+    meta_remain = DARSHAN_JOB_METADATA_LEN -

+        strlen(core->log_job.metadata) - 1;

+    if(meta_remain >= (strlen(PACKAGE_VERSION) + 9))

+    {

+        sprintf(core->log_job.metadata, "lib_ver=%s\n", PACKAGE_VERSION);

+        meta_remain -= (strlen(PACKAGE_VERSION) + 9);

+    }

+    if(meta_remain >= (3 + strlen(header_hints)))

+    {

+        m = core->log_job.metadata + strlen(core->log_job.metadata);

+        /* We have room to store the hints in the metadata portion of

+         * the job header.  We just prepend an h= to the hints list.  The

+         * metadata parser will ignore = characters that appear in the value

+         * portion of the metadata key/value pair.

+         */

+        sprintf(m, "h=%s\n", header_hints);

+    }

+    free(header_hints);

+

+    return;

+}

+

+static int mnt_data_cmp(const void* a, const void* b)

+{

+    const struct mnt_data *d_a = (const struct mnt_data*)a;

+    const struct mnt_data *d_b = (const struct mnt_data*)b;

+

+    if(strlen(d_a->path) > strlen(d_b->path))

+        return(-1);

+    else if(strlen(d_a->path) < strlen(d_b->path))

+        return(1);

+    else

+        return(0);

+}

+

+/* adds an entry to table of mounted file systems */

+static void add_entry(char* trailing_data, int* space_left, struct mntent *entry)

+{

+    int ret;

+    char tmp_mnt[256];

+    struct statfs statfsbuf;

+

+    strncpy(mnt_data_array[mnt_data_count].path, entry->mnt_dir,

+        DARSHAN_MAX_MNT_PATH-1);

+    strncpy(mnt_data_array[mnt_data_count].type, entry->mnt_type,

+        DARSHAN_MAX_MNT_TYPE-1);

+    /* NOTE: we now try to detect the preferred block size for each file 

+     * system using fstatfs().  On Lustre we assume a size of 1 MiB 

+     * because fstatfs() reports 4 KiB. 

+     */

+#ifndef LL_SUPER_MAGIC

+#define LL_SUPER_MAGIC 0x0BD00BD0

+#endif

+    ret = statfs(entry->mnt_dir, &statfsbuf);

+    if(ret == 0 && statfsbuf.f_type != LL_SUPER_MAGIC)

+        mnt_data_array[mnt_data_count].block_size = statfsbuf.f_bsize;

+    else if(ret == 0 && statfsbuf.f_type == LL_SUPER_MAGIC)

+        mnt_data_array[mnt_data_count].block_size = 1024*1024;

+    else

+        mnt_data_array[mnt_data_count].block_size = 4096;

+

+    /* store mount information for use in header of darshan log */

+    ret = snprintf(tmp_mnt, 256, "\n%s\t%s",

+        entry->mnt_type, entry->mnt_dir);

+    if(ret < 256 && strlen(tmp_mnt) <= (*space_left))

+    {

+        strcat(trailing_data, tmp_mnt);

+        (*space_left) -= strlen(tmp_mnt);

+    }

+

+    mnt_data_count++;

+    return;

+}

+

+/* darshan_get_exe_and_mounts_root()

+ *

+ * collects command line and list of mounted file systems into a string that

+ * will be stored with the job header

+ */

+static void darshan_get_exe_and_mounts_root(struct darshan_core_runtime *core,

+    char* trailing_data, int space_left)

+{

+    FILE* tab;

+    struct mntent *entry;

+    char* exclude;

+    int tmp_index = 0;

+    int skip = 0;

+

+    /* skip these fs types */

+    static char* fs_exclusions[] = {

+        "tmpfs",

+        "proc",

+        "sysfs",

+        "devpts",

+        "binfmt_misc",

+        "fusectl",

+        "debugfs",

+        "securityfs",

+        "nfsd",

+        "none",

+        "rpc_pipefs",

+        "hugetlbfs",

+        "cgroup",

+        NULL

+    };

+

+    /* length of exe has already been safety checked in darshan initialization */

+    strcat(trailing_data, core->exe);

+    space_left = DARSHAN_EXE_LEN - strlen(trailing_data);

+

+    /* we make two passes through mounted file systems; in the first pass we

+     * grab any non-nfs mount points, then on the second pass we grab nfs

+     * mount points

+     */

+    mnt_data_count = 0;

+

+    tab = setmntent("/etc/mtab", "r");

+    if(!tab)

+        return;

+    /* loop through list of mounted file systems */

+    while(mnt_data_count<DARSHAN_MAX_MNTS && (entry = getmntent(tab)) != NULL)

+    {

+        /* filter out excluded fs types */

+        tmp_index = 0;

+        skip = 0;

+        while((exclude = fs_exclusions[tmp_index]))

+        {

+            if(!(strcmp(exclude, entry->mnt_type)))

+            {

+                skip =1;

+                break;

+            }

+            tmp_index++;

+        }

+

+        if(skip || (strcmp(entry->mnt_type, "nfs") == 0))

+            continue;

+

+        add_entry(trailing_data, &space_left, entry);

+    }

+    endmntent(tab);

+

+    tab = setmntent("/etc/mtab", "r");

+    if(!tab)

+        return;

+    /* loop through list of mounted file systems */

+    while(mnt_data_count<DARSHAN_MAX_MNTS && (entry = getmntent(tab)) != NULL)

+    {

+        if(strcmp(entry->mnt_type, "nfs") != 0)

+            continue;

+

+        add_entry(trailing_data, &space_left, entry);

+    }

+    endmntent(tab);

+

+    /* Sort mount points in order of longest path to shortest path.  This is

+     * necessary so that if we try to match file paths to mount points later

+     * we don't match on "/" every time.

+     */

+    qsort(mnt_data_array, mnt_data_count, sizeof(mnt_data_array[0]), mnt_data_cmp);

+    return;

+}

+

+/* darshan_get_exe_and_mounts()

+ *

+ * collects command line and list of mounted file systems into a string that

+ * will be stored with the job header

+ */

+static char* darshan_get_exe_and_mounts(struct darshan_core_runtime *core)

+{

+    char* trailing_data;

+    int space_left;

+

+    space_left = DARSHAN_EXE_LEN + 1;

+    trailing_data = malloc(space_left);

+    if(!trailing_data)

+    {

+        return(NULL);

+    }

+    memset(trailing_data, 0, space_left);

+

+    if(my_rank == 0)

+    {

+        darshan_get_exe_and_mounts_root(core, trailing_data, space_left);

+    }

+

+    /* broadcast trailing data to all nodes */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(trailing_data, space_left, MPI_CHAR, 0,

+        MPI_COMM_WORLD);

+    /* broadcast mount count to all nodes */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(&mnt_data_count, 1, MPI_INT, 0,

+        MPI_COMM_WORLD);

+    /* broadcast mount data to all nodes */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(mnt_data_array,

+        mnt_data_count*sizeof(mnt_data_array[0]), MPI_BYTE, 0, MPI_COMM_WORLD);

+

+    return(trailing_data);

+}

+

+static void darshan_block_size_from_path(const char *path, int *block_size)

+{

+    int i;

+    *block_size = -1;

+

+    for(i=0; i<mnt_data_count; i++)

+    {

+        if(!(strncmp(mnt_data_array[i].path, path, strlen(mnt_data_array[i].path))))

+        {

+            *block_size = mnt_data_array[i].block_size;

+            return;

+        }

+    }

+

+    return;

+}

+

+static void darshan_get_shared_records(struct darshan_core_runtime *core,

+    darshan_record_id **shared_recs, int *shared_rec_cnt)

+{

+    int i, j;

+    int tmp_cnt = core->rec_count;

+    struct darshan_core_record_ref *tmp, *ref;

+    darshan_record_id *id_array;

+    uint64_t *mod_flags;

+    uint64_t *global_mod_flags;

+

+    /* broadcast root's number of records to all other processes */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(&tmp_cnt, 1, MPI_INT, 0, MPI_COMM_WORLD);

+

+    /* use root record count to allocate data structures */

+    id_array = malloc(tmp_cnt * sizeof(darshan_record_id));

+    mod_flags = malloc(tmp_cnt * sizeof(uint64_t));

+    global_mod_flags = malloc(tmp_cnt * sizeof(uint64_t));

+    *shared_recs = malloc(tmp_cnt * sizeof(darshan_record_id));

+    assert(id_array && mod_flags && global_mod_flags && *shared_recs);

+

+    memset(mod_flags, 0, tmp_cnt * sizeof(uint64_t));

+    memset(global_mod_flags, 0, tmp_cnt * sizeof(uint64_t));

+    memset(*shared_recs, 0, tmp_cnt * sizeof(darshan_record_id));

+

+    /* first, determine list of records root process has opened */

+    if(my_rank == 0)

+    {

+        i = 0;

+        HASH_ITER(hlink, core->rec_hash, ref, tmp)

+        {

+            id_array[i++] = ref->rec.id;           

+        }

+    }

+

+    /* broadcast root's list of records to all other processes */

+    DARSHAN_MPI_CALL(PMPI_Bcast)(id_array, (tmp_cnt * sizeof(darshan_record_id)),

+        MPI_BYTE, 0, MPI_COMM_WORLD);

+

+    /* everyone looks to see if they opened the same records as root */

+    for(i=0; i<tmp_cnt; i++)

+    {

+        HASH_FIND(hlink, core->rec_hash, &id_array[i], sizeof(darshan_record_id), ref);

+        if(ref)

+        {

+            /* we opened that record too, save the mod_flags */

+            mod_flags[i] = ref->mod_flags;

+        }

+    }

+

+    /* now allreduce so everyone agrees which files are shared and

+     * which modules accessed them collectively

+     */

+    DARSHAN_MPI_CALL(PMPI_Allreduce)(mod_flags, global_mod_flags, tmp_cnt,

+        MPI_UINT64_T, MPI_BAND, MPI_COMM_WORLD);

+

+    j = 0;

+    for(i=0; i<tmp_cnt; i++)

+    {

+        if(global_mod_flags[i] != 0)

+        {

+            (*shared_recs)[j++] = id_array[i];

+

+            /* set global_mod_flags so we know which modules collectively

+             * accessed this module. we need this info to support shared

+             * file reductions

+             */

+            HASH_FIND(hlink, core->rec_hash, &id_array[i], sizeof(darshan_record_id), ref);

+            assert(ref);

+            ref->global_mod_flags = global_mod_flags[i];

+        }

+    }

+    *shared_rec_cnt = j;

+

+    return;

+}

+

+static int darshan_log_open_all(char *logfile_name, MPI_File *log_fh)

+{

+    char *hints;

+    char *tok_str;

+    char *orig_tok_str;

+    char *key;

+    char *value;

+    char *saveptr = NULL;

+    int ret;

+    MPI_Info info;

+

+    /* check environment variable to see if the default MPI file hints have

+     * been overridden

+     */

+    MPI_Info_create(&info);

+

+    hints = getenv(DARSHAN_LOG_HINTS_OVERRIDE);

+    if(!hints)

+    {

+        hints = __DARSHAN_LOG_HINTS;

+    }

+

+    if(hints && strlen(hints) > 0)

+    {

+        tok_str = strdup(hints);

+        if(tok_str)

+        {

+            orig_tok_str = tok_str;

+            do

+            {

+                /* split string on semicolon */

+                key = strtok_r(tok_str, ";", &saveptr);

+                if(key)

+                {

+                    tok_str = NULL;

+                    /* look for = sign splitting key/value pairs */

+                    value = index(key, '=');

+                    if(value)

+                    {

+                        /* break key and value into separate null terminated strings */

+                        value[0] = '\0';

+                        value++;

+                        if(strlen(key) > 0)

+                            MPI_Info_set(info, key, value);

+                    }

+                }

+            }while(key != NULL);

+            free(orig_tok_str);

+        }

+    }

+

+    /* open the darshan log file for writing */

+    ret = DARSHAN_MPI_CALL(PMPI_File_open)(MPI_COMM_WORLD, logfile_name,

+        MPI_MODE_CREATE | MPI_MODE_WRONLY | MPI_MODE_EXCL, info, log_fh);

+    if(ret != MPI_SUCCESS)

+        return(-1);

+

+    MPI_Info_free(&info);

+    return(0);

+}

+

+static int darshan_deflate_buffer(void **pointers, int *lengths, int count,

+    char *comp_buf, int *comp_buf_length)

+{

+    int ret = 0;

+    int i;

+    int total_target = 0;

+    z_stream tmp_stream;

+

+    /* just return if there is no data */

+    for(i = 0; i < count; i++)

+    {

+        total_target += lengths[i];

+    }

+    if(total_target)

+    {

+        total_target = 0;

+    }

+    else

+    {

+        *comp_buf_length = 0;

+        return(0);

+    }

+

+    memset(&tmp_stream, 0, sizeof(tmp_stream));

+    tmp_stream.zalloc = Z_NULL;

+    tmp_stream.zfree = Z_NULL;

+    tmp_stream.opaque = Z_NULL;

+

+    /* initialize the zlib compression parameters */

+    /* TODO: check these parameters? */

+//    ret = deflateInit2(&tmp_stream, Z_DEFAULT_COMPRESSION, Z_DEFLATED,

+//        15 + 16, 8, Z_DEFAULT_STRATEGY);

+    ret = deflateInit(&tmp_stream, Z_DEFAULT_COMPRESSION);

+    if(ret != Z_OK)

+    {

+        return(-1);

+    }

+

+    tmp_stream.next_out = (unsigned char *)comp_buf;

+    tmp_stream.avail_out = DARSHAN_COMP_BUF_SIZE;

+

+    /* loop over the input pointers */

+    for(i = 0; i < count; i++)

+    {

+        total_target += lengths[i];

+        tmp_stream.next_in = pointers[i];

+        tmp_stream.avail_in = lengths[i];

+        /* while we have not finished consuming all of the data available to

+         * this point in the loop

+         */

+        while(tmp_stream.total_in < total_target)

+        {

+            if(tmp_stream.avail_out == 0)

+            {

+                /* We ran out of buffer space for compression.  In theory,

+                 * we could start using some of the file_array buffer space

+                 * without having to malloc again.  In practice, this case 

+                 * is going to be practically impossible to hit.

+                 */

+                deflateEnd(&tmp_stream);

+                return(-1);

+            }

+

+            /* compress data */

+            ret = deflate(&tmp_stream, Z_NO_FLUSH);

+            if(ret != Z_OK)

+            {

+                deflateEnd(&tmp_stream);

+                return(-1);

+            }

+        }

+    }

+

+    /* flush compression and end */

+    ret = deflate(&tmp_stream, Z_FINISH);

+    if(ret != Z_STREAM_END)

+    {

+        deflateEnd(&tmp_stream);

+        return(-1);

+    }

+    deflateEnd(&tmp_stream);

+

+    *comp_buf_length = tmp_stream.total_out;

+    return(0);

+}

+

+/* NOTE: the map written to file may contain duplicate id->name entries if a

+ *       record is opened by multiple ranks, but not all ranks

+ */

+static int darshan_log_write_record_hash(MPI_File log_fh, struct darshan_core_runtime *core,

+    uint64_t *inout_off)

+{

+    int ret;

+    struct darshan_core_record_ref *ref, *tmp;

+    uint32_t name_len;

+    size_t record_sz;

+    size_t hash_buf_sz = 0;

+    char *hash_buf;

+    char *hash_buf_off;

+

+    /* allocate a buffer to store at most 64 bytes for each registered record */

+    /* NOTE: this buffer may be reallocated if estimate is too small */

+    hash_buf_sz = core->rec_count * 64;

+    hash_buf = malloc(hash_buf_sz);

+    if(!hash_buf)

+    {

+        return(-1);

+    }

+

+    /* serialize the record hash into a buffer for writing */

+    hash_buf_off = hash_buf;

+    HASH_ITER(hlink, core->rec_hash, ref, tmp)

+    {

+        /* to avoid duplicate records, only rank 0 will write shared records */

+        if(my_rank > 0 && ref->global_mod_flags)

+            continue;

+

+        name_len = strlen(ref->rec.name);

+        record_sz = sizeof(darshan_record_id) + sizeof(uint32_t) + name_len;

+        /* make sure there is room in the buffer for this record */

+        if((hash_buf_off + record_sz) > (hash_buf + hash_buf_sz))

+        {

+            char *tmp_buf;

+            size_t old_buf_sz;

+

+            /* if no room, reallocate the hash buffer at twice the current size */

+            old_buf_sz = hash_buf_off - hash_buf;

+            hash_buf_sz *= 2;

+            tmp_buf = malloc(hash_buf_sz);

+            if(!tmp_buf)

+            {

+                free(hash_buf);

+                return(-1);

+            }

+

+            memcpy(tmp_buf, hash_buf, old_buf_sz);

+            free(hash_buf);

+            hash_buf = tmp_buf;

+            hash_buf_off = hash_buf + old_buf_sz;

+        }

+

+        /* now serialize the record into the hash buffer.

+         * NOTE: darshan record hash serialization method: 

+         *          ... darshan_record_id | (uint32_t) path_len | path ...

+         */

+        *((darshan_record_id *)hash_buf_off) = ref->rec.id;

+        hash_buf_off += sizeof(darshan_record_id);

+        *((uint32_t *)hash_buf_off) = name_len;

+        hash_buf_off += sizeof(uint32_t);

+        memcpy(hash_buf_off, ref->rec.name, name_len);

+        hash_buf_off += name_len;

+    }

+    hash_buf_sz = hash_buf_off - hash_buf;

+

+    /* collectively write out the record hash to the darshan log */

+    ret = darshan_log_append_all(log_fh, core, hash_buf, hash_buf_sz, inout_off);

+

+    free(hash_buf);

+

+    return(ret);

+}

+

+/* NOTE: inout_off contains the starting offset of this append at the beginning

+ *       of the call, and contains the ending offset at the end of the call.

+ *       This variable is only valid on the root rank (rank 0).

+ */

+static int darshan_log_append_all(MPI_File log_fh, struct darshan_core_runtime *core,

+    void *buf, int count, uint64_t *inout_off)

+{

+    MPI_Offset send_off, my_off;

+    MPI_Status status;

+    int comp_buf_sz = 0;

+    int ret;

+

+    /* compress the input buffer */

+    ret = darshan_deflate_buffer((void **)&buf, &count, 1,

+        core->comp_buf, &comp_buf_sz);

+    if(ret < 0)

+        comp_buf_sz = 0;

+

+    /* figure out where everyone is writing using scan */

+    send_off = comp_buf_sz;

+    if(my_rank == 0)

+    {

+        send_off += *inout_off; /* rank 0 knows the beginning offset */

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Scan)(&send_off, &my_off, 1, MPI_OFFSET,

+        MPI_SUM, MPI_COMM_WORLD);

+    /* scan in inclusive; subtract local size back out */

+    my_off -= comp_buf_sz;

+

+    if(ret == 0)

+    {

+        /* no compression errors, proceed with the collective write */

+        ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(log_fh, my_off,

+            core->comp_buf, comp_buf_sz, MPI_BYTE, &status);

+    }

+    else

+    {

+        /* error during compression. preserve and return error to caller,

+         * but participate in collective write to avoid deadlock.

+         */

+        (void)DARSHAN_MPI_CALL(PMPI_File_write_at_all)(log_fh, my_off,

+            core->comp_buf, comp_buf_sz, MPI_BYTE, &status);

+    }

+

+    if(nprocs > 1)

+    {

+        /* send the ending offset from rank (n-1) to rank 0 */

+        if(my_rank == (nprocs-1))

+        {

+            my_off += comp_buf_sz;

+            DARSHAN_MPI_CALL(PMPI_Send)(&my_off, 1, MPI_OFFSET, 0, 0,

+                MPI_COMM_WORLD);

+        }

+        if(my_rank == 0)

+        {

+            DARSHAN_MPI_CALL(PMPI_Recv)(&my_off, 1, MPI_OFFSET, (nprocs-1), 0,

+                MPI_COMM_WORLD, &status);

+

+            *inout_off = my_off;

+        }

+    }

+    else

+    {

+        *inout_off = my_off + comp_buf_sz;

+    }

+

+    if(ret != 0)

+        return(-1);

+    return(0);

+}

+

+/* free darshan core data structures to shutdown */

+static void darshan_core_cleanup(struct darshan_core_runtime* core)

+{

+    struct darshan_core_record_ref *tmp, *ref;

+    int i;

+

+    HASH_ITER(hlink, core->rec_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, core->rec_hash, ref);

+        free(ref->rec.name);

+        free(ref);

+    }

+

+    for(i = 0; i < DARSHAN_MAX_MODS; i++)

+    {

+        if(core->mod_array[i])

+        {        

+            free(core->mod_array[i]);

+            core->mod_array[i] = NULL;

+        }

+    }

+

+    free(core->trailing_data);

+    free(core);

+

+    return;

+}

+

+/* crude benchmarking hook into darshan-core to benchmark Darshan

+ * shutdown overhead using a variety of application I/O workloads

+ */

+extern void darshan_posix_shutdown_bench_setup();

+extern void darshan_mpiio_shutdown_bench_setup();

+void darshan_shutdown_bench(int argc, char **argv)

+{

+    /* clear out existing core runtime structure */

+    if(darshan_core)

+    {

+        darshan_core_cleanup(darshan_core);

+        darshan_core = NULL;

+    }

+

+    /***********************************************************/

+    /* restart darshan */

+    darshan_core_initialize(argc, argv);

+

+    darshan_posix_shutdown_bench_setup(1);

+    darshan_mpiio_shutdown_bench_setup(1);

+

+    if(my_rank == 0)

+        printf("# 1 unique file per proc\n");

+    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

+    darshan_core_shutdown();

+    darshan_core = NULL;

+

+    sleep(1);

+

+    /***********************************************************/

+    /* restart darshan */

+    darshan_core_initialize(argc, argv);

+

+    darshan_posix_shutdown_bench_setup(2);

+    darshan_mpiio_shutdown_bench_setup(2);

+

+    if(my_rank == 0)

+        printf("# 1 shared file per proc\n");

+    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

+    darshan_core_shutdown();

+    darshan_core = NULL;

+

+    sleep(1);

+

+    /***********************************************************/

+    /* restart darshan */

+    darshan_core_initialize(argc, argv);

+

+    darshan_posix_shutdown_bench_setup(3);

+    darshan_mpiio_shutdown_bench_setup(3);

+

+    if(my_rank == 0)

+        printf("# 1024 unique files per proc\n");

+    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

+    darshan_core_shutdown();

+    darshan_core = NULL;

+

+    sleep(1);

+

+    /***********************************************************/

+    /* restart darshan */

+    darshan_core_initialize(argc, argv);

+

+    darshan_posix_shutdown_bench_setup(4);

+    darshan_mpiio_shutdown_bench_setup(4);

+

+    if(my_rank == 0)

+        printf("# 1024 shared files per proc\n");

+    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

+    darshan_core_shutdown();

+    darshan_core = NULL;

+

+    sleep(1);

+

+    /***********************************************************/

+

+    return;

+}

+

+/* ********************************************************* */

+

+void darshan_core_register_module(

+    darshan_module_id mod_id,

+    struct darshan_module_funcs *funcs,

+    int *rank,

+    int *mod_mem_limit,

+    int *sys_mem_alignment)

+{

+    int ret;

+    int tmpval;

+    struct darshan_core_module* mod;

+    char *mod_mem_str = NULL;

+    *mod_mem_limit = 0;

+

+    if(!darshan_core || (mod_id >= DARSHAN_MAX_MODS))

+        return;

+

+    if(sys_mem_alignment)

+        *sys_mem_alignment = darshan_mem_alignment;

+

+    /* see if this module is already registered */

+    DARSHAN_CORE_LOCK();

+    if(darshan_core->mod_array[mod_id])

+    {

+        /* if module is already registered just return */

+        /* NOTE: we do not recalculate memory limit here, just set to 0 */

+        DARSHAN_CORE_UNLOCK();

+        return;

+    }

+

+    /* this module has not been registered yet, allocate and initialize it */

+    mod = malloc(sizeof(*mod));

+    if(!mod)

+    {

+        DARSHAN_CORE_UNLOCK();

+        return;

+    }

+    memset(mod, 0, sizeof(*mod));

+    mod->id = mod_id;

+    mod->mod_funcs = *funcs;

+

+    /* register module with darshan */

+    darshan_core->mod_array[mod_id] = mod;

+    darshan_core->log_header.mod_ver[mod_id] = darshan_module_versions[mod_id];

+

+    /* get the calling process's rank */

+    if(rank)

+        *rank = my_rank;

+

+    /* set the maximum amount of memory this module can use */

+    mod_mem_str = getenv(DARSHAN_MOD_MEM_OVERRIDE);

+    if(mod_mem_str)

+    {

+        ret = sscanf(mod_mem_str, "%d", &tmpval);

+        /* silently ignore if the env variable is set poorly */

+        if(ret == 1 && tmpval > 0)

+            *mod_mem_limit = (tmpval * 1024 * 1024); /* convert to MiB */

+        else

+            *mod_mem_limit = DARSHAN_MOD_MEM_MAX;

+    }

+    else

+    {

+        *mod_mem_limit = DARSHAN_MOD_MEM_MAX;

+    }

+    DARSHAN_CORE_UNLOCK();

+

+    return;

+}

+

+void darshan_core_unregister_module(

+    darshan_module_id mod_id)

+{

+    struct darshan_core_record_ref *ref, *tmp;

+

+    if(!darshan_core)

+        return;

+

+    DARSHAN_CORE_LOCK();

+

+    if(darshan_core->mod_array[mod_id])

+    {

+        /* iterate all records and disassociate this module from them */

+        HASH_ITER(hlink, darshan_core->rec_hash, ref, tmp)

+        {

+            darshan_core_unregister_record(ref->rec.id, mod_id);

+        }

+

+        free(darshan_core->mod_array[mod_id]);

+        darshan_core->mod_array[mod_id] = NULL;

+    }

+    DARSHAN_CORE_UNLOCK();

+

+    return;

+}

+

+void darshan_core_register_record(

+    void *name,

+    int len,

+    darshan_module_id mod_id,

+    int printable_flag,

+    int mod_limit_flag,

+    darshan_record_id *rec_id,

+    int *file_alignment)

+{

+    darshan_record_id tmp_rec_id;

+    struct darshan_core_record_ref *ref;

+

+    *rec_id = 0;

+

+    if(!darshan_core)

+        return;

+

+    /* TODO: what do you do with printable flag? */

+

+    /* hash the input name to get a unique id for this record */

+    tmp_rec_id = darshan_hash(name, len, 0);

+

+    /* check to see if we've already stored the id->name mapping for this record */

+    DARSHAN_CORE_LOCK();

+    HASH_FIND(hlink, darshan_core->rec_hash, &tmp_rec_id, sizeof(darshan_record_id), ref);

+    if(!ref)

+    {

+        /* record not found -- add it to the hash if this module has not already used

+         * all of its memory

+         */

+  

+        if(mod_limit_flag)

+        {

+            /* if this module is OOM, set a flag in the header to indicate this */

+            DARSHAN_MOD_FLAG_SET(darshan_core->log_header.partial_flag, mod_id);

+            DARSHAN_CORE_UNLOCK();

+            return;

+        }

+

+        ref = malloc(sizeof(struct darshan_core_record_ref));

+        if(ref)

+        {

+            ref->mod_flags = ref->global_mod_flags = 0;

+            ref->rec.id = tmp_rec_id;

+            ref->rec.name = malloc(strlen(name) + 1);

+            if(ref->rec.name)

+                strcpy(ref->rec.name, name);

+

+            HASH_ADD(hlink, darshan_core->rec_hash, rec.id, sizeof(darshan_record_id), ref);

+            darshan_core->rec_count++;

+        }

+    }

+    DARSHAN_MOD_FLAG_SET(ref->mod_flags, mod_id);

+    DARSHAN_CORE_UNLOCK();

+

+    if(file_alignment)

+        darshan_block_size_from_path(name, file_alignment);

+

+    *rec_id = tmp_rec_id;

+    return;

+}

+

+/* TODO: test */

+void darshan_core_unregister_record(

+    darshan_record_id rec_id,

+    darshan_module_id mod_id)

+{

+    struct darshan_core_record_ref *ref;

+

+    if(!darshan_core)

+        return;

+

+    DARSHAN_CORE_LOCK();

+    HASH_FIND(hlink, darshan_core->rec_hash, &rec_id, sizeof(darshan_record_id), ref);

+    assert(ref); 

+

+    /* disassociate this module from the given record id */

+    DARSHAN_MOD_FLAG_UNSET(ref->mod_flags, mod_id);

+    if(!(ref->mod_flags))

+    {

+        /* if no other modules are associated with this rec, delete it */

+        HASH_DELETE(hlink, darshan_core->rec_hash, ref);

+    }

+    DARSHAN_CORE_UNLOCK();

+

+    return;

+}

+

+double darshan_core_wtime()

+{

+    if(!darshan_core)

+    {

+        return(0);

+    }

+

+    return(DARSHAN_MPI_CALL(PMPI_Wtime)() - darshan_core->wtime_offset);

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-27'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-27'>

<strong>

darshan-runtime/lib/darshan-hdf5-stubs.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-hdf5-stubs.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-hdf5-stubs.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 /* This file contains stubs for the H5F functions intercepted by Darshan.

</code></pre>

<br>

</li>

<li id='diff-28'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-28'>

<strong>

darshan-runtime/lib/darshan-hdf5.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-hdf5.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-hdf5.c

</span><span style="color: #aaaaaa">@@ -1,74 +1,105 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #include "darshan-runtime-config.h"

 #include <stdio.h>

<span style="color: #000000;background-color: #ffdddd">-#include <pthread.h>

</span><span style="color: #000000;background-color: #ddffdd">+#include <unistd.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <stdarg.h>

</span> #include <string.h>

<span style="color: #000000;background-color: #ffdddd">-#include "mpi.h"

-#include "darshan.h"

-

-/* hope this doesn't change any time soon */

-typedef int hid_t; 

-typedef int herr_t;

-

-#ifdef DARSHAN_PRELOAD

-

-#define __USE_GNU

-#include <dlfcn.h>

</span><span style="color: #000000;background-color: #ddffdd">+#include <time.h>

</span> #include <stdlib.h>

<span style="color: #000000;background-color: #ddffdd">+#include <errno.h>

+#include <search.h>

+#include <assert.h>

+#define __USE_GNU

+#include <pthread.h>

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  ret (*__real_ ## name)args = NULL;

-

-#define DARSHAN_DECL(__name) __name

-

-#define MAP_OR_FAIL(func) \

-    if (!(__real_ ## func)) \

-    { \

-        __real_ ## func = dlsym(RTLD_NEXT, #func); \

-        if(!(__real_ ## func)) { \

-            fprintf(stderr, "Darshan failed to map symbol: %s\n", #func); \

-            exit(1); \

-        } \

-    }

-

-#else

-

-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  extern ret __real_ ## name args;

-

-#define DARSHAN_DECL(__name) __wrap_ ## __name

</span><span style="color: #000000;background-color: #ddffdd">+#include "uthash.h"

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define MAP_OR_FAIL(func) 

</span><span style="color: #000000;background-color: #ddffdd">+#include "darshan.h"

+#include "darshan-dynamic.h"

</span> 

<span style="color: #000000;background-color: #ffdddd">-#endif

</span><span style="color: #000000;background-color: #ddffdd">+/* hope this doesn't change any time soon */

+typedef int hid_t;

+typedef int herr_t;

</span> 

 DARSHAN_FORWARD_DECL(H5Fcreate, hid_t, (const char *filename, unsigned flags, hid_t create_plist, hid_t access_plist));

 DARSHAN_FORWARD_DECL(H5Fopen, hid_t, (const char *filename, unsigned flags, hid_t access_plist));

 DARSHAN_FORWARD_DECL(H5Fclose, herr_t, (hid_t file_id));

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_hid(int hid);

-static void darshan_file_close_hid(int hid);

-static struct darshan_file_runtime* darshan_file_by_name_sethid(const char* name, int hid);

</span><span style="color: #000000;background-color: #ddffdd">+/* structure to track i/o stats for a given hdf5 file at runtime */

+struct hdf5_file_runtime

+{

+    struct darshan_hdf5_file* file_record;

+    UT_hash_handle hlink;

+};

+

+/* structure to associate a HDF5 hid with an existing file runtime structure */

+struct hdf5_file_runtime_ref

+{

+    struct hdf5_file_runtime* file;

+    hid_t hid;

+    UT_hash_handle hlink;

+};

+

+/* necessary state for storing HDF5 file records and coordinating with

+ * darshan-core at shutdown time

+ */

+struct hdf5_runtime

+{

+    struct hdf5_file_runtime* file_runtime_array;

+    struct darshan_hdf5_file* file_record_array;

+    int file_array_size;

+    int file_array_ndx;

+    struct hdf5_file_runtime *file_hash;

+    struct hdf5_file_runtime_ref* hid_hash;

+};

+

+static struct hdf5_runtime *hdf5_runtime = NULL;

+static pthread_mutex_t hdf5_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+static int instrumentation_disabled = 0;

+static int my_rank = -1;

+

+static void hdf5_runtime_initialize(void);

+static struct hdf5_file_runtime* hdf5_file_by_name(const char *name);

+static struct hdf5_file_runtime* hdf5_file_by_name_sethid(const char* name, hid_t hid);

+static struct hdf5_file_runtime* hdf5_file_by_hid(hid_t hid);

+static void hdf5_file_close_hid(hid_t hid);

+static int hdf5_record_compare(const void* a, const void* b);

+static void hdf5_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype);

+

+static void hdf5_begin_shutdown(void);

+static void hdf5_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,

+    int shared_rec_count, void **hdf5_buf, int *hdf5_buf_sz);

+static void hdf5_shutdown(void);

+

+#define HDF5_LOCK() pthread_mutex_lock(&hdf5_runtime_mutex)

+#define HDF5_UNLOCK() pthread_mutex_unlock(&hdf5_runtime_mutex)

+

+/*********************************************************

+ *        Wrappers for HDF5 functions of interest        * 

+ *********************************************************/

</span> 

 hid_t DARSHAN_DECL(H5Fcreate)(const char *filename, unsigned flags,

     hid_t create_plist, hid_t access_plist)

 {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct hdf5_file_runtime* file;

</span>     char* tmp;

     double tm1;

     MAP_OR_FAIL(H5Fcreate);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_H5Fcreate(filename, flags, create_plist, access_plist);

     if(ret >= 0)

<span style="color: #000000;background-color: #ffdddd">-    {  

-        CP_LOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    {

</span>         /* use ROMIO approach to strip prefix if present */

         /* strip off prefix if there is one, but only skip prefixes

          * if they are greater than length one to allow for windows

<span style="color: #aaaaaa">@@ -79,15 +110,16 @@ hid_t DARSHAN_DECL(H5Fcreate)(const char *filename, unsigned flags,

</span>             filename = tmp + 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        file = darshan_file_by_name_sethid(filename, ret);

</span><span style="color: #000000;background-color: #ddffdd">+        HDF5_LOCK();

+        hdf5_runtime_initialize();

+        file = hdf5_file_by_name_sethid(filename, ret);

</span>         if(file)

         {

<span style="color: #000000;background-color: #ffdddd">-            if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0)

-                CP_F_SET(file, CP_F_OPEN_TIMESTAMP,

-                tm1);

-            CP_INC(file, CP_HDF5_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+            if(file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] == 0)

+                file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] = tm1;

+            file->file_record->counters[HDF5_OPENS] += 1;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        HDF5_UNLOCK();

</span>     }

     return(ret);

<span style="color: #aaaaaa">@@ -97,17 +129,16 @@ hid_t DARSHAN_DECL(H5Fopen)(const char *filename, unsigned flags,

</span>     hid_t access_plist)

 {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct hdf5_file_runtime* file;

</span>     char* tmp;

     double tm1;

     MAP_OR_FAIL(H5Fopen);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_H5Fopen(filename, flags, access_plist);

     if(ret >= 0)

<span style="color: #000000;background-color: #ffdddd">-    {  

-        CP_LOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    {

</span>         /* use ROMIO approach to strip prefix if present */

         /* strip off prefix if there is one, but only skip prefixes

          * if they are greater than length one to allow for windows

<span style="color: #aaaaaa">@@ -118,16 +149,16 @@ hid_t DARSHAN_DECL(H5Fopen)(const char *filename, unsigned flags,

</span>             filename = tmp + 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        file = darshan_file_by_name_sethid(filename, ret);

</span><span style="color: #000000;background-color: #ddffdd">+        HDF5_LOCK();

+        hdf5_runtime_initialize();

+        file = hdf5_file_by_name_sethid(filename, ret);

</span>         if(file)

         {

<span style="color: #000000;background-color: #ffdddd">-            if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0)

-                CP_F_SET(file, CP_F_OPEN_TIMESTAMP,

-                tm1);

-            CP_INC(file, CP_HDF5_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+            if(file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] == 0)

+                file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] = tm1;

+            file->file_record->counters[HDF5_OPENS] += 1;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-

-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        HDF5_UNLOCK();

</span>     }

     return(ret);

<span style="color: #aaaaaa">@@ -136,49 +167,418 @@ hid_t DARSHAN_DECL(H5Fopen)(const char *filename, unsigned flags,

</span> 

 herr_t DARSHAN_DECL(H5Fclose)(hid_t file_id)

 {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct hdf5_file_runtime* file;

</span>     int ret;

     MAP_OR_FAIL(H5Fclose);

     ret = __real_H5Fclose(file_id);

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_hid(file_id);

</span><span style="color: #000000;background-color: #ddffdd">+    HDF5_LOCK();

+    hdf5_runtime_initialize();

+    file = hdf5_file_by_hid(file_id);

</span>     if(file)

     {

<span style="color: #000000;background-color: #ffdddd">-        CP_F_SET(file, CP_F_CLOSE_TIMESTAMP, PMPI_Wtime());

-        darshan_file_close_hid(file_id);

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->fcounters[HDF5_F_CLOSE_TIMESTAMP] =

+            darshan_core_wtime();

+        hdf5_file_close_hid(file_id);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    HDF5_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_name_sethid(const char* name, int hid)

</span><span style="color: #000000;background-color: #ddffdd">+/*********************************************************

+ * Internal functions for manipulating HDF5 module state *

+ *********************************************************/

+

+/* initialize internal HDF5 module data strucutres and register with darshan-core */

+static void hdf5_runtime_initialize()

+{

+    int mem_limit;

+    struct darshan_module_funcs hdf5_mod_fns =

+    {

+        .begin_shutdown = &hdf5_begin_shutdown,

+        .get_output_data = &hdf5_get_output_data,

+        .shutdown = &hdf5_shutdown

+    };

+

+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(hdf5_runtime || instrumentation_disabled)

+        return;

+

+    /* register hdf5 module with darshan-core */

+    darshan_core_register_module(

+        DARSHAN_HDF5_MOD,

+        &hdf5_mod_fns,

+        &my_rank,

+        &mem_limit,

+        NULL);

+

+    /* return if no memory assigned by darshan-core */

+    if(mem_limit == 0)

+        return;

+

+    hdf5_runtime = malloc(sizeof(*hdf5_runtime));

+    if(!hdf5_runtime)

+        return;

+    memset(hdf5_runtime, 0, sizeof(*hdf5_runtime));

+

+    /* set maximum number of file records according to max memory limit */

+    /* NOTE: maximum number of records is based on the size of a hdf5 file record */

+    /* TODO: should we base memory usage off file record or total runtime structure sizes? */

+    hdf5_runtime->file_array_size = mem_limit / sizeof(struct darshan_hdf5_file);

+    hdf5_runtime->file_array_ndx = 0;

+

+    /* allocate array of runtime file records */

+    hdf5_runtime->file_runtime_array = malloc(hdf5_runtime->file_array_size *

+                                              sizeof(struct hdf5_file_runtime));

+    hdf5_runtime->file_record_array = malloc(hdf5_runtime->file_array_size *

+                                             sizeof(struct darshan_hdf5_file));

+    if(!hdf5_runtime->file_runtime_array || !hdf5_runtime->file_record_array)

+    {

+        hdf5_runtime->file_array_size = 0;

+        return;

+    }

+    memset(hdf5_runtime->file_runtime_array, 0, hdf5_runtime->file_array_size *

+           sizeof(struct hdf5_file_runtime));

+    memset(hdf5_runtime->file_record_array, 0, hdf5_runtime->file_array_size *

+           sizeof(struct darshan_hdf5_file));

+

+    return;

+}

+

+/* get a HDF5 file record for the given file path */

+static struct hdf5_file_runtime* hdf5_file_by_name(const char *name)

+{

+    struct hdf5_file_runtime *file = NULL;

+    char *newname = NULL;

+    darshan_record_id file_id;

+    int limit_flag;

+

+    if(!hdf5_runtime || instrumentation_disabled)

+        return(NULL);

+

+    newname = darshan_clean_file_path(name);

+    if(!newname)

+        newname = (char*)name;

+

+    limit_flag = (hdf5_runtime->file_array_ndx >= hdf5_runtime->file_array_size);

+

+    /* get a unique id for this file from darshan core */

+    darshan_core_register_record(

+        (void*)newname,

+        strlen(newname),

+        DARSHAN_HDF5_MOD,

+        1,

+        limit_flag,

+        &file_id,

+        NULL);

+

+    /* if record is set to 0, darshan-core is out of space and will not

+     * track this record, so we should avoid tracking it, too

+     */

+    if(file_id == 0)

+    {

+        if(newname != name)

+            free(newname);

+        return(NULL);

+    }

+

+    /* search the hash table for this file record, and return if found */

+    HASH_FIND(hlink, hdf5_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);

+    if(file)

+    {

+        if(newname != name)

+            free(newname);

+        return(file);

+    }

+

+    /* no existing record, assign a new file record from the global array */

+    file = &(hdf5_runtime->file_runtime_array[hdf5_runtime->file_array_ndx]);

+    file->file_record = &(hdf5_runtime->file_record_array[hdf5_runtime->file_array_ndx]);

+    file->file_record->f_id = file_id;

+    file->file_record->rank = my_rank;

+

+    /* add new record to file hash table */

+    HASH_ADD(hlink, hdf5_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);

+    hdf5_runtime->file_array_ndx++;

+

+    if(newname != name)

+        free(newname);

+    return(file);

+}

+

+/* get a HDF5 file record for the given file path, and also create a

+ * reference structure using the returned hid

+ */

+static struct hdf5_file_runtime* hdf5_file_by_name_sethid(const char* name, hid_t hid)

+{

+    struct hdf5_file_runtime* file;

+    struct hdf5_file_runtime_ref* ref;

+

+    if(!hdf5_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* find file record by name first */

+    file = hdf5_file_by_name(name);

+

+    if(!file)

+        return(NULL);

+

+    /* search hash table for existing file ref for this fd */

+    HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);

+    if(ref)

+    {

+        /* we have a reference.  Make sure it points to the correct file

+         * and return it

+         */

+        ref->file = file;

+        return(file);

+    }

+

+    /* if we hit this point, then we don't have a reference for this fd

+     * in the table yet.  Add it.

+     */

+    ref = malloc(sizeof(*ref));

+    if(!ref)

+        return(NULL);

+    memset(ref, 0, sizeof(*ref));

+

+    ref->file = file;

+    ref->hid = hid;

+    HASH_ADD(hlink, hdf5_runtime->hid_hash, hid, sizeof(hid_t), ref);

+

+    return(file);

+}

+

+/* get a HDF5 file record for the given hid */

+static struct hdf5_file_runtime* hdf5_file_by_hid(hid_t hid)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct hdf5_file_runtime_ref* ref;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_name_sethandle(name, &hid, sizeof(hid), DARSHAN_HID);

-    return(tmp_file);

</span><span style="color: #000000;background-color: #ddffdd">+    if(!hdf5_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* search hash table for existing file ref for this hid */

+    HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);

+    if(ref)

+        return(ref->file);

+

+    return(NULL);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static void darshan_file_close_hid(int hid)

</span><span style="color: #000000;background-color: #ddffdd">+/* free up HDF5 reference data structures for the given hid */

+static void hdf5_file_close_hid(hid_t hid)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    darshan_file_closehandle(&hid, sizeof(hid), DARSHAN_HID);

</span><span style="color: #000000;background-color: #ddffdd">+    struct hdf5_file_runtime_ref* ref;

+

+    if(!hdf5_runtime || instrumentation_disabled)

+        return;

+

+    /* search hash table for this hid */

+    HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);

+    if(ref)

+    {

+        /* we have a reference, delete it */

+        HASH_DELETE(hlink, hdf5_runtime->hid_hash, ref);

+        free(ref);

+    }

+

+    return;

+}

+

+/* compare function for sorting file records by descending rank */

+static int hdf5_record_compare(const void* a_p, const void* b_p)

+{

+    const struct darshan_hdf5_file* a = a_p;

+    const struct darshan_hdf5_file* b = b_p;

+

+    if(a->rank < b->rank)

+        return 1;

+    if(a->rank > b->rank)

+        return -1;

+

+    return 0;

+}

+

+static void hdf5_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype)

+{

+    struct darshan_hdf5_file tmp_file;

+    struct darshan_hdf5_file *infile = infile_v;

+    struct darshan_hdf5_file *inoutfile = inoutfile_v;

+    int i, j;

+

+    assert(hdf5_runtime);

+

+    for(i=0; i<*len; i++)

+    {

+        memset(&tmp_file, 0, sizeof(struct darshan_hdf5_file));

+        tmp_file.f_id = infile->f_id;

+        tmp_file.rank = -1;

+

+        /* sum */

+        for(j=HDF5_OPENS; j<=HDF5_OPENS; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        /* min non-zero (if available) value */

+        for(j=HDF5_F_OPEN_TIMESTAMP; j<=HDF5_F_OPEN_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = infile->fcounters[j];

+        }

+

+        /* max */

+        for(j=HDF5_F_CLOSE_TIMESTAMP; j<=HDF5_F_CLOSE_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j])

+                tmp_file.fcounters[j] = infile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+        }

+

+        /* update pointers */

+        *inoutfile = tmp_file;

+        inoutfile++;

+        infile++;

+    }

+

</span>     return;

 }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_hid(int hid)

</span><span style="color: #000000;background-color: #ddffdd">+/************************************************************************

+ * Functions exported by HDF5 module for coordinating with darshan-core *

+ ************************************************************************/

+

+static void hdf5_begin_shutdown()

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    assert(hdf5_runtime);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_handle(&hid, sizeof(hid), DARSHAN_HID);

-    

-    return(tmp_file);

</span><span style="color: #000000;background-color: #ddffdd">+    HDF5_LOCK();

+    /* disable further instrumentation while Darshan shuts down */

+    instrumentation_disabled = 1;

+    HDF5_UNLOCK();

+

+    return;

</span> }

<span style="color: #000000;background-color: #ddffdd">+static void hdf5_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **hdf5_buf,

+    int *hdf5_buf_sz)

+{

+    struct hdf5_file_runtime *file;

+    int i;

+    struct darshan_hdf5_file *red_send_buf = NULL;

+    struct darshan_hdf5_file *red_recv_buf = NULL;

+    MPI_Datatype red_type;

+    MPI_Op red_op;

+

+    assert(hdf5_runtime);

+

+    /* if there are globally shared files, do a shared file reduction */

+    /* NOTE: the shared file reduction is also skipped if the 

+     * DARSHAN_DISABLE_SHARED_REDUCTION environment variable is set.

+     */

+    if(shared_rec_count && !getenv("DARSHAN_DISABLE_SHARED_REDUCTION"))

+    {

+        /* necessary initialization of shared records */

+        for(i = 0; i < shared_rec_count; i++)

+        {

+            HASH_FIND(hlink, hdf5_runtime->file_hash, &shared_recs[i],

+                sizeof(darshan_record_id), file);

+            assert(file);

+

+            file->file_record->rank = -1;

+        }

+

+        /* sort the array of files descending by rank so that we get all of the 

+         * shared files (marked by rank -1) in a contiguous portion at end 

+         * of the array

+         */

+        qsort(hdf5_runtime->file_record_array, hdf5_runtime->file_array_ndx,

+            sizeof(struct darshan_hdf5_file), hdf5_record_compare);

+

+        /* make *send_buf point to the shared files at the end of sorted array */

+        red_send_buf =

+            &(hdf5_runtime->file_record_array[hdf5_runtime->file_array_ndx-shared_rec_count]);

+

+        /* allocate memory for the reduction output on rank 0 */

+        if(my_rank == 0)

+        {

+            red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_hdf5_file));

+            if(!red_recv_buf)

+            {

+                return;

+            }

+        }

+

+        /* construct a datatype for a HDF5 file record.  This is serving no purpose

+         * except to make sure we can do a reduction on proper boundaries

+         */

+        DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_hdf5_file),

+            MPI_BYTE, &red_type);

+        DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);

+

+        /* register a HDF5 file record reduction operator */

+        DARSHAN_MPI_CALL(PMPI_Op_create)(hdf5_record_reduction_op, 1, &red_op);

+

+        /* reduce shared HDF5 file records */

+        DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,

+            shared_rec_count, red_type, red_op, 0, mod_comm);

+

+        /* clean up reduction state */

+        if(my_rank == 0)

+        {

+            int tmp_ndx = hdf5_runtime->file_array_ndx - shared_rec_count;

+            memcpy(&(hdf5_runtime->file_record_array[tmp_ndx]), red_recv_buf,

+                shared_rec_count * sizeof(struct darshan_hdf5_file));

+            free(red_recv_buf);

+        }

+        else

+        {

+            hdf5_runtime->file_array_ndx -= shared_rec_count;

+        }

+

+        DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);

+        DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);

+    }

+

+    *hdf5_buf = (void *)(hdf5_runtime->file_record_array);

+    *hdf5_buf_sz = hdf5_runtime->file_array_ndx * sizeof(struct darshan_hdf5_file);

+

+    return;

+}

+

+static void hdf5_shutdown()

+{

+    struct hdf5_file_runtime_ref *ref, *tmp;

+

+    assert(hdf5_runtime);

+

+    HASH_ITER(hlink, hdf5_runtime->hid_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, hdf5_runtime->hid_hash, ref);

+        free(ref);

+    }

+

+    HASH_CLEAR(hlink, hdf5_runtime->file_hash); /* these entries are freed all at once below */

+

+    free(hdf5_runtime->file_runtime_array);

+    free(hdf5_runtime->file_record_array);

+    free(hdf5_runtime);

+    hdf5_runtime = NULL;

+

+    return;

+}

</span> 

 /*

  * Local variables:

</code></pre>

<br>

</li>

<li id='diff-29'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-29'>

<strong>

darshan-runtime/lib/darshan-mpi-io.c

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-mpi-io.c

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,2606 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-/*

- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

- */

-

-#define _XOPEN_SOURCE 500

-#define _GNU_SOURCE /* for tdestroy() */

-

-#include "darshan-runtime-config.h"

-

-#include <stdio.h>

-#ifdef HAVE_MNTENT_H

-#include <mntent.h>

-#endif

-#include <stdlib.h>

-#include <string.h>

-#include <time.h>

-#include <limits.h>

-#include <unistd.h>

-#include <pthread.h>

-#include <sys/types.h>

-#include <sys/stat.h>

-#include <sys/vfs.h>

-#include <zlib.h>

-#include <assert.h>

-#include <search.h>

-

-#include "mpi.h"

-#include "darshan.h"

-#include "darshan-dynamic.h"

-

-extern char* __progname;

-

-/* maximum number of memory segments each process will write to the log */

-#define CP_MAX_MEM_SEGMENTS 8

-

-/* Some old versions of MPI don't provide all of these COMBINER definitions.  

- * If any are missing then we define them to an arbitrary value just to 

- * prevent compile errors in DATATYPE_INC().

- */

-#ifndef MPI_COMBINER_NAMED

-    #define MPI_COMBINER_NAMED CP_COMBINER_NAMED

-#endif

-#ifndef MPI_COMBINER_DUP

-    #define MPI_COMBINER_DUP CP_COMBINER_DUP

-#endif

-#ifndef MPI_COMBINER_CONTIGUOUS

-    #define MPI_COMBINER_CONTIGUOUS CP_COMBINER_CONTIGUOUS

-#endif

-#ifndef MPI_COMBINER_VECTOR

-    #define MPI_COMBINER_VECTOR CP_COMBINER_VECTOR

-#endif

-#ifndef MPI_COMBINER_HVECTOR_INTEGER

-    #define MPI_COMBINER_HVECTOR_INTEGER CP_COMBINER_HVECTOR_INTEGER

-#endif

-#ifndef MPI_COMBINER_HVECTOR

-    #define MPI_COMBINER_HVECTOR CP_COMBINER_HVECTOR

-#endif

-#ifndef MPI_COMBINER_INDEXED

-    #define MPI_COMBINER_INDEXED CP_COMBINER_INDEXED

-#endif

-#ifndef MPI_COMBINER_HINDEXED_INTEGER

-    #define MPI_COMBINER_HINDEXED_INTEGER CP_COMBINER_HINDEXED_INTEGER

-#endif

-#ifndef MPI_COMBINER_HINDEXED

-    #define MPI_COMBINER_HINDEXED CP_COMBINER_HINDEXED

-#endif

-#ifndef MPI_COMBINER_INDEXED_BLOCK

-    #define MPI_COMBINER_INDEXED_BLOCK CP_COMBINER_INDEXED_BLOCK

-#endif

-#ifndef MPI_COMBINER_STRUCT_INTEGER

-    #define MPI_COMBINER_STRUCT_INTEGER CP_COMBINER_STRUCT_INTEGER

-#endif

-#ifndef MPI_COMBINER_STRUCT

-    #define MPI_COMBINER_STRUCT CP_COMBINER_STRUCT

-#endif

-#ifndef MPI_COMBINER_SUBARRAY

-    #define MPI_COMBINER_SUBARRAY CP_COMBINER_SUBARRAY

-#endif

-#ifndef MPI_COMBINER_DARRAY

-    #define MPI_COMBINER_DARRAY CP_COMBINER_DARRAY

-#endif

-#ifndef MPI_COMBINER_F90_REAL

-    #define MPI_COMBINER_F90_REAL CP_COMBINER_F90_REAL

-#endif

-#ifndef MPI_COMBINER_F90_COMPLEX

-    #define MPI_COMBINER_F90_COMPLEX CP_COMBINER_F90_COMPLEX

-#endif

-#ifndef MPI_COMBINER_F90_INTEGER

-    #define MPI_COMBINER_F90_INTEGER CP_COMBINER_F90_INTEGER

-#endif

-#ifndef MPI_COMBINER_RESIZED

-    #define MPI_COMBINER_RESIZED CP_COMBINER_RESIZED

-#endif

-

-#define CP_DATATYPE_INC(__file, __datatype) do {\

-    int num_integers, num_addresses, num_datatypes, combiner, ret; \

-    ret = DARSHAN_MPI_CALL(PMPI_Type_get_envelope)(__datatype, &num_integers, \

-        &num_addresses, &num_datatypes, &combiner); \

-    if(ret == MPI_SUCCESS) { \

-        switch(combiner) { \

-            case MPI_COMBINER_NAMED:\

-                CP_INC(__file,CP_COMBINER_NAMED,1); break; \

-            case MPI_COMBINER_DUP:\

-                CP_INC(__file,CP_COMBINER_DUP,1); break; \

-            case MPI_COMBINER_CONTIGUOUS:\

-                CP_INC(__file,CP_COMBINER_CONTIGUOUS,1); break; \

-            case MPI_COMBINER_VECTOR:\

-                CP_INC(__file,CP_COMBINER_VECTOR,1); break; \

-            case MPI_COMBINER_HVECTOR_INTEGER:\

-                CP_INC(__file,CP_COMBINER_HVECTOR_INTEGER,1); break; \

-            case MPI_COMBINER_HVECTOR:\

-                CP_INC(__file,CP_COMBINER_HVECTOR,1); break; \

-            case MPI_COMBINER_INDEXED:\

-                CP_INC(__file,CP_COMBINER_INDEXED,1); break; \

-            case MPI_COMBINER_HINDEXED_INTEGER:\

-                CP_INC(__file,CP_COMBINER_HINDEXED_INTEGER,1); break; \

-            case MPI_COMBINER_HINDEXED:\

-                CP_INC(__file,CP_COMBINER_HINDEXED,1); break; \

-            case MPI_COMBINER_INDEXED_BLOCK:\

-                CP_INC(__file,CP_COMBINER_INDEXED_BLOCK,1); break; \

-            case MPI_COMBINER_STRUCT_INTEGER:\

-                CP_INC(__file,CP_COMBINER_STRUCT_INTEGER,1); break; \

-            case MPI_COMBINER_STRUCT:\

-                CP_INC(__file,CP_COMBINER_STRUCT,1); break; \

-            case MPI_COMBINER_SUBARRAY:\

-                CP_INC(__file,CP_COMBINER_SUBARRAY,1); break; \

-            case MPI_COMBINER_DARRAY:\

-                CP_INC(__file,CP_COMBINER_DARRAY,1); break; \

-            case MPI_COMBINER_F90_REAL:\

-                CP_INC(__file,CP_COMBINER_F90_REAL,1); break; \

-            case MPI_COMBINER_F90_COMPLEX:\

-                CP_INC(__file,CP_COMBINER_F90_COMPLEX,1); break; \

-            case MPI_COMBINER_F90_INTEGER:\

-                CP_INC(__file,CP_COMBINER_F90_INTEGER,1); break; \

-            case MPI_COMBINER_RESIZED:\

-                CP_INC(__file,CP_COMBINER_RESIZED,1); break; \

-        } \

-    } \

-} while(0)

-

-#define CP_RECORD_MPI_WRITE(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \

-    struct darshan_file_runtime* file; \

-    int size = 0; \

-    MPI_Aint extent = 0; \

-    if(__ret != MPI_SUCCESS) break; \

-    file = darshan_file_by_fh(__fh); \

-    if(!file) break; \

-    DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size);  \

-    size = size * __count; \

-    DARSHAN_MPI_CALL(PMPI_Type_extent)(__datatype, &extent); \

-    CP_BUCKET_INC(file, CP_SIZE_WRITE_AGG_0_100, size); \

-    CP_BUCKET_INC(file, CP_EXTENT_WRITE_0_100, extent); \

-    CP_INC(file, __counter, 1); \

-    CP_DATATYPE_INC(file, __datatype); \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_mpi_write_end, CP_F_MPI_WRITE_TIME); \

-    if(CP_F_VALUE(file, CP_F_WRITE_START_TIMESTAMP) == 0) \

-        CP_F_SET(file, CP_F_WRITE_START_TIMESTAMP, __tm1); \

-    CP_F_SET(file, CP_F_WRITE_END_TIMESTAMP, __tm2); \

-} while(0)

-

-#define CP_RECORD_MPI_READ(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \

-    struct darshan_file_runtime* file; \

-    int size = 0; \

-    MPI_Aint extent = 0; \

-    if(__ret != MPI_SUCCESS) break; \

-    file = darshan_file_by_fh(__fh); \

-    if(!file) break; \

-    DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size);  \

-    size = size * __count; \

-    DARSHAN_MPI_CALL(PMPI_Type_extent)(__datatype, &extent); \

-    CP_BUCKET_INC(file, CP_SIZE_READ_AGG_0_100, size); \

-    CP_BUCKET_INC(file, CP_EXTENT_READ_0_100, extent); \

-    CP_INC(file, __counter, 1); \

-    CP_DATATYPE_INC(file, __datatype); \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_mpi_read_end, CP_F_MPI_READ_TIME); \

-    if(CP_F_VALUE(file, CP_F_READ_START_TIMESTAMP) == 0) \

-        CP_F_SET(file, CP_F_READ_START_TIMESTAMP, __tm1); \

-    CP_F_SET(file, CP_F_READ_END_TIMESTAMP, __tm2); \

-} while(0)

-

-static void cp_normalize_timestamps(struct darshan_job_runtime* final_job);

-static void cp_log_construct_indices(struct darshan_job_runtime* final_job,

-    int rank, int* inout_count, int* lengths, void** pointers, char*

-    trailing_data);

-static int cp_log_write(struct darshan_job_runtime* final_job, int rank, 

-    char* logfile_name, int count, int* lengths, void** pointers, double start_log_time);

-static void cp_log_record_hints_and_ver(struct darshan_job_runtime* final_job, int rank);

-static int cp_log_reduction(struct darshan_job_runtime* final_job, int rank, 

-    char* logfile_name, MPI_Offset* next_offset);

-static void darshan_file_reduce(void* infile_v, 

-    void* inoutfile_v, int *len, 

-    MPI_Datatype *datatype);

-static int cp_log_compress(struct darshan_job_runtime* final_job,

-    int rank, int* inout_count, int* lengths, void** pointers);

-static int file_compare(const void* a, const void* b);

-static int darshan_file_variance(

-    struct darshan_file *infile_array,

-    struct darshan_file *outfile_array,

-    int count, int rank);

-static void pairwise_variance_reduce (

-    void *invec, void *inoutvec, int *len, MPI_Datatype *dt);

-#if 0

-static void debug_mounts(const char* mtab_file, const char* out_file);

-#endif

-

-static struct darshan_file_runtime* darshan_file_by_fh(MPI_File fh);

-static void darshan_file_close_fh(MPI_File fh);

-static struct darshan_file_runtime* darshan_file_by_name_setfh(const char* name, MPI_File fh);

-

-#define CP_MAX_MNTS 64

-#define CP_MAX_MNT_PATH 256

-#define CP_MAX_MNT_TYPE 32

-struct mnt_data

-{

-    int64_t hash;

-    int64_t block_size;

-    char path[CP_MAX_MNT_PATH];

-    char type[CP_MAX_MNT_TYPE];

-};

-static struct mnt_data mnt_data_array[CP_MAX_MNTS];

-static int mnt_data_count = 0;

-

-struct variance_dt

-{

-    double n;

-    double T;

-    double S;

-};

-

-void darshan_mpi_initialize(int *argc, char ***argv)

-{

-    int nprocs;

-    int rank;

-    int timing_flag = 0;

-    double init_start, init_time, init_max;

-

-    DARSHAN_MPI_CALL(PMPI_Comm_size)(MPI_COMM_WORLD, &nprocs);

-    DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &rank);

-    

-    if(getenv("DARSHAN_INTERNAL_TIMING"))

-        timing_flag = 1;

-

-    if(timing_flag)

-        init_start = DARSHAN_MPI_CALL(PMPI_Wtime)();

-

-    if(argc && argv)

-    {

-        darshan_initialize(*argc, *argv, nprocs, rank);

-    }

-    else

-    {

-        /* we don't see argc and argv here in fortran */

-        darshan_initialize(0, NULL, nprocs, rank);

-    }

-    

-    if(timing_flag)

-    {

-        init_time = DARSHAN_MPI_CALL(PMPI_Wtime)() - init_start;

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&init_time, &init_max, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-        if(rank == 0)

-        {

-            printf("#darshan:<op>\t<nprocs>\t<time>\n");

-            printf("darshan:init\t%d\t%f\n", nprocs, init_max);

-        }

-    }

-

-    return;

-}

-

-void darshan_shutdown(int timing_flag)

-{

-    int rank;

-    char* logfile_name;

-    struct darshan_job_runtime* final_job;

-    double start_log_time = 0;

-    int all_ret = 0;

-    int local_ret = 0;

-    MPI_Offset next_offset = 0;

-    char* user_logfile_name;

-    char* jobid_str;

-    char* envjobid;

-    char* logpath;

-    int jobid;

-    int index_count = 0;

-    int lengths[CP_MAX_MEM_SEGMENTS];

-    void* pointers[CP_MAX_MEM_SEGMENTS];

-    int ret;

-    double red1=0, red2=0, gz1=0, gz2=0, write1=0, write2=0, tm_end=0;

-    double bcst=0;

-    int nprocs;

-    time_t start_time_tmp = 0;

-    uint64_t logmod;

-    char hname[HOST_NAME_MAX];

-    char* logpath_override = NULL;

-#ifdef __CP_LOG_ENV

-    char env_check[256];

-    char* env_tok;

-#endif

-    uint64_t hlevel;

-    int64_t first_start_time;

-    int64_t last_end_time;

-

-    CP_LOCK();

-    if(!darshan_global_job)

-    {

-        CP_UNLOCK();

-        return;

-    }

-    /* disable further tracing while hanging onto the data so that we can

-     * write it out

-     */

-    final_job = darshan_global_job;

-    darshan_global_job = NULL;

-    CP_UNLOCK();

-

-    start_log_time = DARSHAN_MPI_CALL(PMPI_Wtime)();

-

-    /* figure out which access sizes to log */

-    darshan_walk_file_accesses(final_job);

-

-    /* if the records have been condensed, then zero out fields that are no

-     * longer valid for safety 

-     */

-    if(final_job->flags & CP_FLAG_CONDENSED && final_job->file_count)

-    {

-        CP_SET(&final_job->file_runtime_array[0], CP_MODE, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_CONSEC_READS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_CONSEC_WRITES, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_SEQ_READS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_SEQ_WRITES, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE1_STRIDE, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE2_STRIDE, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE3_STRIDE, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE4_STRIDE, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE1_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE2_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE3_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_STRIDE4_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS1_ACCESS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS2_ACCESS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS3_ACCESS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS4_ACCESS, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS1_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS2_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS3_COUNT, 0);

-        CP_SET(&final_job->file_runtime_array[0], CP_ACCESS4_COUNT, 0);

-        

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_OPEN_TIMESTAMP, 0);

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_CLOSE_TIMESTAMP, 0);

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_READ_START_TIMESTAMP, 0);

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_READ_END_TIMESTAMP, 0);

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_WRITE_START_TIMESTAMP, 0);

-        CP_F_SET(&final_job->file_runtime_array[0], CP_F_WRITE_END_TIMESTAMP, 0);

-    }

-

-    logfile_name = malloc(PATH_MAX);

-    if(!logfile_name)

-    {

-        darshan_finalize(final_job);

-        return;

-    }

-

-    DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &rank);

-

-    /* construct log file name */

-    if(rank == 0)

-    {

-        /* Use CP_JOBID_OVERRIDE for the env var or CP_JOBID */

-        envjobid = getenv(CP_JOBID_OVERRIDE);

-        if (!envjobid)

-        {

-            envjobid = CP_JOBID;

-        }

-

-        /* find a job id */

-        jobid_str = getenv(envjobid);

-        if(jobid_str)

-        {

-            /* in cobalt we can find it in env var */

-            ret = sscanf(jobid_str, "%d", &jobid);

-        }

-        if(!jobid_str || ret != 1)

-        {

-            /* use pid as fall back */

-            jobid = getpid();

-        }

-

-        /* add jobid to darshan runtime info */

-        final_job->log_job.jobid = (int64_t)jobid;

-

-        /* if user specifies a logfile name (and path), use that. otherwise automatically generate */

-        user_logfile_name = getenv("DARSHAN_LOGFILE");

-        if(user_logfile_name)

-        {

-            if(strlen(user_logfile_name) >= PATH_MAX)

-            {

-                logfile_name[0] = '\0';

-                fprintf(stderr, "darshan library warning: user given log file path too long\n");

-            }

-            else

-            {

-                strncpy(logfile_name, user_logfile_name, PATH_MAX);

-            }

-        }

-        else

-        {

-            char cuser[L_cuserid] = {0};

-            struct tm* my_tm;

-

-            /* Use CP_LOG_PATH_OVERRIDE for the value or __CP_LOG_PATH */

-            logpath = getenv(CP_LOG_PATH_OVERRIDE);

-            if (!logpath)

-            {

-#ifdef __CP_LOG_PATH

-                logpath = __CP_LOG_PATH;

-#endif

-            }

-

-            /* break out time into something human readable */

-            start_time_tmp += final_job->log_job.start_time;

-            my_tm = localtime(&start_time_tmp);

-

-            /* get the username for this job.  In order we will try each of the

-             * following until one of them succeeds:

-             *

-             * - cuserid()

-             * - getenv("LOGNAME")

-             * - snprintf(..., geteuid());

-             *

-             * Note that we do not use getpwuid() because it generally will not

-             * work in statically compiled binaries.

-             */

-

-#ifndef DARSHAN_DISABLE_CUSERID

-            cuserid(cuser);

-#endif

-

-            /* if cuserid() didn't work, then check the environment */

-            if (strcmp(cuser, "") == 0)

-            {

-                char* logname_string;

-                logname_string = getenv("LOGNAME");

-                if(logname_string)

-                {

-                    strncpy(cuser, logname_string, (L_cuserid-1));

-                }

-

-            }

-

-            /* if cuserid() and environment both fail, then fall back to uid */

-            if (strcmp(cuser, "") == 0)

-            {

-                uid_t uid = geteuid();

-                snprintf(cuser, sizeof(cuser), "%u", uid);

-            }

-

-            /* generate a random number to help differentiate the log */

-            hlevel=DARSHAN_MPI_CALL(PMPI_Wtime)() * 1000000;

-            (void) gethostname(hname, sizeof(hname));

-            logmod = darshan_hash((void*)hname,strlen(hname),hlevel);

-

-            /* see if darshan was configured using the --with-logpath-by-env

-             * argument, which allows the user to specify an absolute path to

-             * place logs via an env variable.

-             */

-#ifdef __CP_LOG_ENV

-            /* just silently skip if the environment variable list is too big */

-            if(strlen(__CP_LOG_ENV) < 256)

-            {

-                /* copy env variable list to a temporary buffer */

-                strcpy(env_check, __CP_LOG_ENV);

-                /* tokenize the comma-separated list */

-                env_tok = strtok(env_check, ",");

-                if(env_tok)

-                {

-                    do

-                    {

-                        /* check each env variable in order */

-                        logpath_override = getenv(env_tok); 

-                        if(logpath_override)

-                        {

-                            /* stop as soon as we find a match */

-                            break;

-                        }

-                    }while((env_tok = strtok(NULL, ",")));

-                }

-            }

-#endif

-           

-            if(logpath_override)

-            {

-                ret = snprintf(logfile_name, PATH_MAX, 

-                    "%s/%s_%s_id%d_%d-%d-%d-%" PRIu64 ".darshan_partial",

-                    logpath_override, 

-                    cuser, __progname, jobid,

-                    (my_tm->tm_mon+1), 

-                my_tm->tm_mday, 

-                (my_tm->tm_hour*60*60 + my_tm->tm_min*60 + my_tm->tm_sec),

-                logmod);

-                if(ret == (PATH_MAX-1))

-                {

-                    /* file name was too big; squish it down */

-                    snprintf(logfile_name, PATH_MAX,

-                        "%s/id%d.darshan_partial",

-                        logpath_override, jobid);

-                }

-            }

-            else if(logpath)

-            {

-                ret = snprintf(logfile_name, PATH_MAX, 

-                    "%s/%d/%d/%d/%s_%s_id%d_%d-%d-%d-%" PRIu64 ".darshan_partial",

-                    logpath, (my_tm->tm_year+1900), 

-                    (my_tm->tm_mon+1), my_tm->tm_mday, 

-                    cuser, __progname, jobid,

-                    (my_tm->tm_mon+1), 

-                    my_tm->tm_mday, 

-                    (my_tm->tm_hour*60*60 + my_tm->tm_min*60 + my_tm->tm_sec),

-                    logmod);

-                if(ret == (PATH_MAX-1))

-                {

-                    /* file name was too big; squish it down */

-                    snprintf(logfile_name, PATH_MAX,

-                        "%s/id%d.darshan_partial",

-                        logpath, jobid);

-                }

-            }

-            else

-            {

-                logfile_name[0] = '\0';

-            }

-        }

-    }

-

-    /* broadcast log file name */

-    bcst=DARSHAN_MPI_CALL(PMPI_Wtime)();

-    DARSHAN_MPI_CALL(PMPI_Bcast)(logfile_name, PATH_MAX, MPI_CHAR, 0,

-        MPI_COMM_WORLD);

-

-    if(strlen(logfile_name) == 0)

-    {

-        /* failed to generate log file name */

-        darshan_finalize(final_job);

-        return;

-    }

-

-    final_job->log_job.end_time = time(NULL);

-

-    /* reduce to report first start time and last end time across all ranks

-     * at rank 0

-     */

-    DARSHAN_MPI_CALL(PMPI_Reduce)(&final_job->log_job.start_time, &first_start_time, 1, MPI_LONG_LONG, MPI_MIN, 0, MPI_COMM_WORLD); 

-    DARSHAN_MPI_CALL(PMPI_Reduce)(&final_job->log_job.end_time, &last_end_time, 1, MPI_LONG_LONG, MPI_MAX, 0, MPI_COMM_WORLD); 

-    if(rank == 0)

-    {

-        final_job->log_job.start_time = first_start_time;

-        final_job->log_job.end_time = last_end_time;

-    }

-

-    /* reduce records for shared files */

-    if(timing_flag)

-        red1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-    if(getenv("DARSHAN_DISABLE_SHARED_REDUCTION"))

-    {

-        local_ret = 0;

-    }

-    else

-    {

-        local_ret = cp_log_reduction(final_job, rank, logfile_name, 

-            &next_offset);

-    }

-    if(timing_flag)

-        red2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-    DARSHAN_MPI_CALL(PMPI_Allreduce)(&local_ret, &all_ret, 1, MPI_INT, MPI_LOR, 

-        MPI_COMM_WORLD);

-

-    /* adjust timestamps in any remaining records */

-    cp_normalize_timestamps(final_job);

-

-    /* if we are using any hints to write the log file, then record those

-     * hints in the log file header

-     */

-    cp_log_record_hints_and_ver(final_job, rank);

-

-    if(all_ret == 0)

-    {

-        /* collect data to write from local process */

-        cp_log_construct_indices(final_job, rank, &index_count, lengths, 

-            pointers, final_job->trailing_data);

-    }

-

-    if(all_ret == 0)

-    {

-        /* compress data */

-        if(timing_flag)

-            gz1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-        local_ret = cp_log_compress(final_job, rank, &index_count, 

-            lengths, pointers);

-        if(timing_flag)

-            gz2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-        DARSHAN_MPI_CALL(PMPI_Allreduce)(&local_ret, &all_ret, 1,

-            MPI_INT, MPI_LOR, MPI_COMM_WORLD);

-    }

-

-    if(all_ret == 0)

-    {

-        /* actually write out log file */

-        if(timing_flag)

-            write1 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-        local_ret = cp_log_write(final_job, rank, logfile_name, 

-            index_count, lengths, pointers, start_log_time);

-        if(timing_flag)

-            write2 = DARSHAN_MPI_CALL(PMPI_Wtime)();

-        DARSHAN_MPI_CALL(PMPI_Allreduce)(&local_ret, &all_ret, 1,

-            MPI_INT, MPI_LOR, MPI_COMM_WORLD);

-    }

-

-    if(rank == 0)

-    {

-        if(all_ret != 0)

-        {

-            fprintf(stderr, "darshan library warning: unable to write log file %s\n", logfile_name);

-            /* if any process failed to write log, then delete the whole 

-             * file so we don't leave corrupted results

-             */

-            unlink(logfile_name);

-        }

-        else if (user_logfile_name)

-        {

-            /* we do not need to rename file, just change the permissions */

-#ifdef __CP_GROUP_READABLE_LOGS

-            chmod(user_logfile_name, (S_IRUSR|S_IRGRP));

-#else

-            chmod(user_logfile_name, (S_IRUSR));

-#endif

-        }

-        else

-        {

-            /* rename from *.darshan_partial to *-<logwritetime>.darshan.gz,

-             * which indicates that this log file is complete and ready for

-             * analysis

-             */ 

-            char* mod_index;

-            double end_log_time;

-            char* new_logfile_name;

-

-            new_logfile_name = malloc(PATH_MAX);

-            if(new_logfile_name)

-            {

-                new_logfile_name[0] = '\0';

-                end_log_time = DARSHAN_MPI_CALL(PMPI_Wtime)();

-                strcat(new_logfile_name, logfile_name);

-                mod_index = strstr(new_logfile_name, ".darshan_partial");

-                sprintf(mod_index, "_%d.darshan.gz", (int)(end_log_time-start_log_time+1));

-                rename(logfile_name, new_logfile_name);

-                /* set permissions on log file */

-#ifdef __CP_GROUP_READABLE_LOGS

-                chmod(new_logfile_name, (S_IRUSR|S_IRGRP)); 

-#else

-                chmod(new_logfile_name, (S_IRUSR)); 

-#endif

-                free(new_logfile_name);

-            }

-        }

-    }

-

-    if(final_job->trailing_data)

-        free(final_job->trailing_data);

-    mnt_data_count = 0;

-    free(logfile_name);

-    darshan_finalize(final_job);

-    

-    if(timing_flag)

-    {

-        double red_tm, red_slowest;

-        double gz_tm, gz_slowest;

-        double write_tm, write_slowest;

-        double all_tm, all_slowest;

-        double bcst_tm, bcst_slowest;

-        

-        tm_end = DARSHAN_MPI_CALL(PMPI_Wtime)();

-

-        bcst_tm= red1-bcst;

-        red_tm = red2-red1;

-        gz_tm = gz2-gz1;

-        write_tm = write2-write1;

-        all_tm = tm_end-start_log_time;

-

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&red_tm, &red_slowest, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&gz_tm, &gz_slowest, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&write_tm, &write_slowest, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&all_tm, &all_slowest, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-        DARSHAN_MPI_CALL(PMPI_Reduce)(&bcst_tm, &bcst_slowest, 1,

-            MPI_DOUBLE, MPI_MAX, 0, MPI_COMM_WORLD);

-

-        if(rank == 0)

-        {

-            DARSHAN_MPI_CALL(PMPI_Comm_size)(MPI_COMM_WORLD, &nprocs);

-            printf("#darshan:<op>\t<nprocs>\t<time>\n");

-            printf("darshan:bcst\t%d\t%f\n", nprocs, bcst_slowest);

-            printf("darshan:reduce\t%d\t%f\n", nprocs, red_slowest);

-            printf("darshan:gzip\t%d\t%f\n", nprocs, gz_slowest);

-            printf("darshan:write\t%d\t%f\n", nprocs, write_slowest);

-            printf("darshan:bcast+reduce+gzip+write\t%d\t%f\n", nprocs, all_slowest);

-        }

-    }

-

-    return;

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_open(MPI_Comm comm, const char *filename, int amode, MPI_Info info, MPI_File *fh) 

-#else

-int MPI_File_open(MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_File *fh) 

-#endif

-{

-    int ret;

-    struct darshan_file_runtime* file;

-    char* tmp;

-    int comm_size;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_open)(comm, filename, amode, info, fh);

-    tm2 = darshan_wtime();

-

-    if(ret == MPI_SUCCESS)

-    {

-        CP_LOCK();

-

-        /* use ROMIO approach to strip prefix if present */

-        /* strip off prefix if there is one, but only skip prefixes

-         * if they are greater than length one to allow for windows

-         * drive specifications (e.g. c:\...) 

-         */

-        tmp = strchr(filename, ':');

-        if (tmp > filename + 1) {

-            filename = tmp + 1;

-        }

-

-        file = darshan_file_by_name_setfh(filename, (*fh));

-        if(file)

-        {

-            CP_SET(file, CP_MODE, amode);

-            CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_mpi_meta_end, CP_F_MPI_META_TIME);

-            if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0)

-                CP_F_SET(file, CP_F_OPEN_TIMESTAMP,

-                tm1);

-            DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);

-            if(comm_size == 1)

-            {

-                CP_INC(file, CP_INDEP_OPENS, 1);

-            }

-            else

-            {

-                CP_INC(file, CP_COLL_OPENS, 1);

-            }

-            if(info != MPI_INFO_NULL)

-            {

-                CP_INC(file, CP_HINTS, 1);

-            }

-        }

-

-        CP_UNLOCK();

-    }

-

-    return(ret);

-}

-

-int MPI_File_close(MPI_File *fh) 

-{

-    struct darshan_file_runtime* file;

-    MPI_File tmp_fh = *fh;

-    double tm1, tm2;

-    int ret;

-    

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_close)(fh);

-    tm2 = darshan_wtime();

-

-    CP_LOCK();

-    file = darshan_file_by_fh(tmp_fh);

-    if(file)

-    {

-        CP_F_SET(file, CP_F_CLOSE_TIMESTAMP, DARSHAN_MPI_CALL(PMPI_Wtime)());

-        CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_mpi_meta_end, CP_F_MPI_META_TIME);

-        darshan_file_close_fh(tmp_fh);

-    }

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int MPI_File_sync(MPI_File fh)

-{

-    int ret;

-    struct darshan_file_runtime* file;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_sync)(fh);

-    tm2 = darshan_wtime();

-    if(ret == MPI_SUCCESS)

-    {

-        CP_LOCK();

-        file = darshan_file_by_fh(fh);

-        if(file)

-        {

-            CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_mpi_write_end, CP_F_MPI_WRITE_TIME);

-            CP_INC(file, CP_SYNCS, 1);

-        }

-        CP_UNLOCK();

-    }

-

-    return(ret);

-}

-

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype, 

-    MPI_Datatype filetype, const char *datarep, MPI_Info info)

-#else

-int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype, 

-    MPI_Datatype filetype, char *datarep, MPI_Info info)

-#endif

-{

-    int ret;

-    struct darshan_file_runtime* file;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_set_view)(fh, disp, etype,

-        filetype, datarep, info);

-    tm2 = darshan_wtime();

-    if(ret == MPI_SUCCESS)

-    {

-        CP_LOCK();

-        file = darshan_file_by_fh(fh);

-        if(file)

-        {

-            CP_INC(file, CP_VIEWS, 1);

-            if(info != MPI_INFO_NULL)

-            {

-                CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_mpi_meta_end, CP_F_MPI_META_TIME);

-                CP_INC(file, CP_HINTS, 1);

-            }

-            CP_DATATYPE_INC(file, filetype);

-        }

-        CP_UNLOCK();

-    }

-

-    return(ret);

-}

-

-int MPI_File_read(MPI_File fh, void *buf, int count, 

-    MPI_Datatype datatype, MPI_Status *status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read)(fh, buf, count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_INDEP_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_at(MPI_File fh, MPI_Offset offset, void *buf,

-    int count, MPI_Datatype datatype, MPI_Status *status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_at)(fh, offset, buf,

-        count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_INDEP_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_at_all(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype, MPI_Status * status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all)(fh, offset, buf,

-        count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_COLL_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_all)(fh, buf, count,

-        datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_COLL_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_shared)(fh, buf, count,

-        datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_INDEP_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_ordered(MPI_File fh, void * buf, int count, 

-    MPI_Datatype datatype, MPI_Status * status)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered)(fh, buf, count,

-        datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_COLL_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all_begin)(fh, offset, buf,

-        count, datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_SPLIT_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_all_begin)(fh, buf, count, datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_SPLIT_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_read_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered_begin)(fh, buf, count,

-        datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_SPLIT_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_iread_at(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iread_at)(fh, offset, buf, count,

-        datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_NB_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_iread(MPI_File fh, void * buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST * request)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iread)(fh, buf, count, datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_NB_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-int MPI_File_iread_shared(MPI_File fh, void * buf, int count,

-    MPI_Datatype datatype, __D_MPI_REQUEST * request)

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iread_shared)(fh, buf, count,

-        datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_READ(ret, fh, count, datatype, CP_NB_READS, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write(MPI_File fh, const void *buf, int count, 

-    MPI_Datatype datatype, MPI_Status *status)

-#else

-int MPI_File_write(MPI_File fh, void *buf, int count, 

-    MPI_Datatype datatype, MPI_Status *status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write)(fh, buf, count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_INDEP_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_at(MPI_File fh, MPI_Offset offset, const void *buf,

-    int count, MPI_Datatype datatype, MPI_Status *status)

-#else

-int MPI_File_write_at(MPI_File fh, MPI_Offset offset, void *buf,

-    int count, MPI_Datatype datatype, MPI_Status *status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(fh, offset, buf,

-        count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_INDEP_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, const void * buf,

-    int count, MPI_Datatype datatype, MPI_Status * status)

-#else

-int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype, MPI_Status * status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(fh, offset, buf,

-        count, datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_COLL_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_all(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-#else

-int MPI_File_write_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_all)(fh, buf, count,

-        datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_COLL_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_shared(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-#else

-int MPI_File_write_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_shared)(fh, buf, count,

-        datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_INDEP_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_ordered(MPI_File fh, const void * buf, int count, 

-    MPI_Datatype datatype, MPI_Status * status)

-#else

-int MPI_File_write_ordered(MPI_File fh, void * buf, int count, 

-    MPI_Datatype datatype, MPI_Status * status)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered)(fh, buf, count,

-         datatype, status);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_COLL_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, const void * buf,

-    int count, MPI_Datatype datatype)

-#else

-int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all_begin)(fh, offset,

-        buf, count, datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_SPLIT_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_all_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)

-#else

-int MPI_File_write_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_all_begin)(fh, buf, count, datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_SPLIT_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_write_ordered_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)

-#else

-int MPI_File_write_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered_begin)(fh, buf, count,

-        datatype);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_SPLIT_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, const void * buf,

-    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

-#else

-int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, void * buf,

-    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_at)(fh, offset, buf,

-        count, datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_NB_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_iwrite(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST * request)

-#else

-int MPI_File_iwrite(MPI_File fh, void * buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST * request)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite)(fh, buf, count, datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_NB_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-#ifdef HAVE_MPIIO_CONST

-int MPI_File_iwrite_shared(MPI_File fh, const void * buf, int count,

-    MPI_Datatype datatype, __D_MPI_REQUEST * request)

-#else

-int MPI_File_iwrite_shared(MPI_File fh, void * buf, int count,

-    MPI_Datatype datatype, __D_MPI_REQUEST * request)

-#endif

-{

-    int ret;

-    double tm1, tm2;

-

-    tm1 = darshan_wtime();

-    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_shared)(fh, buf, count,

-        datatype, request);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_MPI_WRITE(ret, fh, count, datatype, CP_NB_WRITES, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-/* cp_log_reduction()

- *

- * Identify shared files and reduce them to one log entry

- *

- * returns 0 on success, -1 on failure

- */

-static int cp_log_reduction(struct darshan_job_runtime* final_job, int rank, 

-    char* logfile_name, MPI_Offset* next_offset)

-{

-    /* TODO: these need to be allocated differently now, too big */

-    uint64_t hash_array[CP_MAX_FILES] = {0};

-    int mask_array[CP_MAX_FILES] = {0};

-    int all_mask_array[CP_MAX_FILES] = {0};

-    int ret;

-    int i;

-    int j;

-    MPI_Op reduce_op;

-    MPI_Datatype rtype;

-    struct darshan_file* tmp_array = NULL;

-    int shared_count = 0;

-    double mpi_time, posix_time;

-

-    /* register a reduction operation */

-    ret = DARSHAN_MPI_CALL(PMPI_Op_create)(darshan_file_reduce, 1, &reduce_op); 

-    if(ret != 0)

-    {

-        return(-1);

-    }

-

-    /* construct a datatype for a file record.  This is serving no purpose

-     * except to make sure we can do a reduction on proper boundaries

-     */

-    DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_file),

-        MPI_BYTE, &rtype); 

-    DARSHAN_MPI_CALL(PMPI_Type_commit)(&rtype); 

-

-    /* gather list of files that root process has opened */

-    if(rank == 0)

-    {

-        for(i=0; i<final_job->file_count; i++)

-        {

-            hash_array[i] = final_job->file_array[i].hash;

-        }

-    }

-

-    /* broadcast list of files to all other processes */

-    ret = DARSHAN_MPI_CALL(PMPI_Bcast)(hash_array,

-        (CP_MAX_FILES * sizeof(uint64_t)), 

-        MPI_BYTE, 0, MPI_COMM_WORLD);

-    if(ret != 0)

-    {

-        DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-        DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-        return(-1);

-    }

-

-    /* everyone looks to see if they have also opened that same file */

-    for(i=0; (i<CP_MAX_FILES && hash_array[i] != 0); i++)

-    {

-        for(j=0; j<final_job->file_count; j++)

-        {

-            if(hash_array[i] && final_job->file_array[j].hash == hash_array[i])

-            {

-                /* we opened that file too */

-                mask_array[i] = 1;

-                break;

-            }

-        }

-    }

-

-    /* now allreduce so that everyone agrees on which files are shared */

-    ret = DARSHAN_MPI_CALL(PMPI_Allreduce)(mask_array, all_mask_array,

-        CP_MAX_FILES, MPI_INT, MPI_LAND, MPI_COMM_WORLD);

-    if(ret != 0)

-    {

-        DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-        DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-        return(-1);

-    }

-

-    /* walk through mask array counting entries and marking corresponding

-     * files with a rank of -1

-     */

-    for(i=0; i<CP_MAX_FILES; i++)

-    {

-        if(all_mask_array[i])

-        {

-            shared_count++;

-            for(j=0; j<final_job->file_count; j++)

-            {

-                if(final_job->file_array[j].hash == hash_array[i])

-                {

-                    posix_time = 

-                      final_job->file_array[j].fcounters[CP_F_POSIX_META_TIME] +

-                      final_job->file_array[j].fcounters[CP_F_POSIX_READ_TIME] +

-                      final_job->file_array[j].fcounters[CP_F_POSIX_WRITE_TIME];

-                    mpi_time = 

-                      final_job->file_array[j].fcounters[CP_F_MPI_META_TIME] +

-                      final_job->file_array[j].fcounters[CP_F_MPI_READ_TIME] +

-                      final_job->file_array[j].fcounters[CP_F_MPI_WRITE_TIME];

-

-                    /*

-                     * Initialize fastest/slowest info prior

-                     * to the reduction.

-                     */

-                    final_job->file_array[j].counters[CP_FASTEST_RANK] =

-                      final_job->file_array[j].rank;

-                    final_job->file_array[j].counters[CP_FASTEST_RANK_BYTES] =

-                      final_job->file_array[j].counters[CP_BYTES_READ] +

-                      final_job->file_array[j].counters[CP_BYTES_WRITTEN];

-                    /* use MPI timing if this file was accessed with MPI */

-                    if(mpi_time > 0)

-                    {

-                        final_job->file_array[j].fcounters[CP_F_FASTEST_RANK_TIME] =

-                        mpi_time;

-                    }

-                    else

-                    {

-                        final_job->file_array[j].fcounters[CP_F_FASTEST_RANK_TIME] =

-                        posix_time;

-                    }

-

-                    /* Until reduction occurs, we assume that this rank is

-                     * both the fastest and slowest.  It is up to the

-                     * reduction operator to find the true min and max if it

-                     * is a shared file.

-                     */

-                    final_job->file_array[j].counters[CP_SLOWEST_RANK] =

-                        final_job->file_array[j].counters[CP_FASTEST_RANK];

-                    final_job->file_array[j].counters[CP_SLOWEST_RANK_BYTES] =

-                        final_job->file_array[j].counters[CP_FASTEST_RANK_BYTES];

-                    final_job->file_array[j].fcounters[CP_F_SLOWEST_RANK_TIME] =

-                        final_job->file_array[j].fcounters[CP_F_FASTEST_RANK_TIME];

-

-                    final_job->file_array[j].rank = -1;

-                    break;

-                }

-            }

-        }

-    }

-

-    if(shared_count)

-    {

-        if(rank == 0)

-        {

-            /* root proc needs to allocate memory to store reduction */

-            tmp_array = malloc(shared_count*sizeof(struct darshan_file));

-            if(!tmp_array)

-            {

-                /* TODO: think more about how to handle errors like this */

-                DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-                DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-                return(-1);

-            }

-        }

-

-        /* sort the array of files descending by rank so that we get all of the 

-         * shared files (marked by rank -1) in a contiguous portion at end 

-         * of the array

-         */

-        qsort(final_job->file_array, final_job->file_count, 

-            sizeof(struct darshan_file), file_compare);

-

-        ret = DARSHAN_MPI_CALL(PMPI_Reduce)(

-            &final_job->file_array[final_job->file_count-shared_count], 

-            tmp_array, shared_count, rtype, reduce_op, 0, MPI_COMM_WORLD);

-        if(ret != 0)

-        {

-            DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-            DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-            return(-1);

-        }

-

-        ret = darshan_file_variance(

-            &final_job->file_array[final_job->file_count-shared_count],

-            tmp_array, shared_count, rank);

-        if (ret)

-        {

-            DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-            DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-            return(-1);

-        }

-

-        if(rank == 0)

-        {

-            /* root replaces local files with shared ones */

-            memcpy(&final_job->file_array[final_job->file_count-shared_count],

-                tmp_array, shared_count*sizeof(struct darshan_file));

-            free(tmp_array);

-            tmp_array = NULL;

-        }

-        else

-        {

-            /* everyone else simply discards those file records */

-            final_job->file_count -= shared_count;

-        }

-    }

-    

-    DARSHAN_MPI_CALL(PMPI_Op_free)(&reduce_op);

-    DARSHAN_MPI_CALL(PMPI_Type_free)(&rtype);

-

-    return(0);

-}

-

-/* TODO: should we use more of the CP macros here? */

-static void darshan_file_reduce(void* infile_v, 

-    void* inoutfile_v, int *len, 

-    MPI_Datatype *datatype)

-{

-    struct darshan_file tmp_file;

-    struct darshan_file* infile = infile_v;

-    struct darshan_file* inoutfile = inoutfile_v;

-    struct darshan_file_runtime tmp_runtime;

-    int i;

-    int j;

-    int k;

-

-    for(i=0; i<*len; i++)

-    {

-        memset(&tmp_file, 0, sizeof(tmp_file));

-

-        tmp_file.hash = infile->hash;

-        tmp_file.rank = -1; /* indicates shared across all procs */

-

-        /* sum */

-        for(j=CP_INDEP_OPENS; j<=CP_VIEWS; j++)

-        {

-            tmp_file.counters[j] = infile->counters[j] + 

-                inoutfile->counters[j];

-        }

-

-        /* pick one, favoring complete records if available */

-        if(CP_FILE_PARTIAL(infile))

-            tmp_file.counters[CP_MODE] = inoutfile->counters[CP_MODE];

-        else

-            tmp_file.counters[CP_MODE] = infile->counters[CP_MODE];

-

-

-        /* sum */

-        for(j=CP_BYTES_READ; j<=CP_BYTES_WRITTEN; j++)

-        {

-            tmp_file.counters[j] = infile->counters[j] + 

-                inoutfile->counters[j];

-        }

-

-        /* max */

-        for(j=CP_MAX_BYTE_READ; j<=CP_MAX_BYTE_WRITTEN; j++)

-        {

-            tmp_file.counters[j] = (

-                (infile->counters[j] > inoutfile->counters[j]) ? 

-                infile->counters[j] :

-                inoutfile->counters[j]);

-        }

-

-        /* sum */

-        for(j=CP_CONSEC_READS; j<=CP_MEM_NOT_ALIGNED; j++)

-        {

-            tmp_file.counters[j] = infile->counters[j] + 

-                inoutfile->counters[j];

-        }

-

-        /* pick one, favoring complete records if available */

-        if(CP_FILE_PARTIAL(infile))

-            tmp_file.counters[CP_MEM_ALIGNMENT] = inoutfile->counters[CP_MEM_ALIGNMENT];

-        else

-            tmp_file.counters[CP_MEM_ALIGNMENT] = infile->counters[CP_MEM_ALIGNMENT];

-

-        /* sum */

-        for(j=CP_FILE_NOT_ALIGNED; j<=CP_FILE_NOT_ALIGNED; j++)

-        {

-            tmp_file.counters[j] = infile->counters[j] + 

-                inoutfile->counters[j];

-        }

-

-        /* pick one, favoring complete records if available */

-        if(CP_FILE_PARTIAL(infile))

-            tmp_file.counters[CP_FILE_ALIGNMENT] = inoutfile->counters[CP_FILE_ALIGNMENT];

-        else

-            tmp_file.counters[CP_FILE_ALIGNMENT] = infile->counters[CP_FILE_ALIGNMENT];

-        

-        /* skip CP_MAX_*_TIME_SIZE; handled in floating point section */

-

-        /* sum */

-        for(j=CP_SIZE_READ_0_100; j<=CP_EXTENT_WRITE_1G_PLUS; j++)

-        {

-            tmp_file.counters[j] = infile->counters[j] + 

-                inoutfile->counters[j];

-        }

-

-        /* pick the 4 most common strides out of the 8 we have to chose from */

-

-        /* first collapse any duplicates */

-        for(j=CP_STRIDE1_STRIDE; j<=CP_STRIDE4_STRIDE; j++)

-        {

-            for(k=CP_STRIDE1_STRIDE; k<=CP_STRIDE4_STRIDE; k++)

-            {

-                if(infile->counters[j] == inoutfile->counters[k])

-                {

-                    infile->counters[j+4] += inoutfile->counters[k+4];

-                    inoutfile->counters[k] = 0;

-                    inoutfile->counters[k+4] = 0;

-                }

-            }

-        }

-

-        /* placeholder so we can re-use macros */

-        tmp_runtime.log_file = &tmp_file;

-        /* first set */

-        for(j=CP_STRIDE1_STRIDE; j<=CP_STRIDE4_STRIDE; j++)

-        {

-            CP_COUNTER_INC(&tmp_runtime, infile->counters[j],

-                infile->counters[j+4], 1, CP_STRIDE1_STRIDE, CP_STRIDE1_COUNT);

-        }

-        /* second set */

-        for(j=CP_STRIDE1_STRIDE; j<=CP_STRIDE4_STRIDE; j++)

-        {

-            CP_COUNTER_INC(&tmp_runtime, inoutfile->counters[j],

-                inoutfile->counters[j+4], 1, CP_STRIDE1_STRIDE, CP_STRIDE1_COUNT);

-        }

-

-        /* TODO: subroutine so we don't duplicate so much */

-        /* same for access counts */

-

-        /* first collapse any duplicates */

-        for(j=CP_ACCESS1_ACCESS; j<=CP_ACCESS4_ACCESS; j++)

-        {

-            for(k=CP_ACCESS1_ACCESS; k<=CP_ACCESS4_ACCESS; k++)

-            {

-                if(infile->counters[j] == inoutfile->counters[k])

-                {

-                    infile->counters[j+4] += inoutfile->counters[k+4];

-                    inoutfile->counters[k] = 0;

-                    inoutfile->counters[k+4] = 0;

-                }

-            }

-        }

-

-        /* placeholder so we can re-use macros */

-        tmp_runtime.log_file = &tmp_file;

-        /* first set */

-        for(j=CP_ACCESS1_ACCESS; j<=CP_ACCESS4_ACCESS; j++)

-        {

-            CP_COUNTER_INC(&tmp_runtime, infile->counters[j],

-                infile->counters[j+4], 1, CP_ACCESS1_ACCESS, CP_ACCESS1_COUNT);

-        }

-        /* second set */

-        for(j=CP_ACCESS1_ACCESS; j<=CP_ACCESS4_ACCESS; j++)

-        {

-            CP_COUNTER_INC(&tmp_runtime, inoutfile->counters[j],

-                inoutfile->counters[j+4], 1, CP_ACCESS1_ACCESS, CP_ACCESS1_COUNT);

-        }

-

-        /* min non-zero (if available) value */

-        for(j=CP_F_OPEN_TIMESTAMP; j<=CP_F_WRITE_START_TIMESTAMP; j++)

-        {

-            if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)

-                tmp_file.fcounters[j] = inoutfile->fcounters[j];

-            else

-                tmp_file.fcounters[j] = infile->fcounters[j];

-        }

-

-        /* max */

-        for(j=CP_F_CLOSE_TIMESTAMP; j<=CP_F_WRITE_END_TIMESTAMP; j++)

-        {

-            if(infile->fcounters[j] > inoutfile->fcounters[j])

-                tmp_file.fcounters[j] = infile->fcounters[j];

-            else

-                tmp_file.fcounters[j] = inoutfile->fcounters[j];

-        }

-

-        /* sum */

-        for(j=CP_F_POSIX_READ_TIME; j<=CP_F_MPI_WRITE_TIME; j++)

-        {

-            tmp_file.fcounters[j] = infile->fcounters[j] + 

-                inoutfile->fcounters[j];

-        }

-

-        /* max (special case) */

-        if(infile->fcounters[CP_F_MAX_WRITE_TIME] > 

-            inoutfile->fcounters[CP_F_MAX_WRITE_TIME])

-        {

-            tmp_file.fcounters[CP_F_MAX_WRITE_TIME] = 

-                infile->fcounters[CP_F_MAX_WRITE_TIME];

-            tmp_file.counters[CP_MAX_WRITE_TIME_SIZE] = 

-                infile->counters[CP_MAX_WRITE_TIME_SIZE];

-        }

-        else

-        {

-            tmp_file.fcounters[CP_F_MAX_WRITE_TIME] = 

-                inoutfile->fcounters[CP_F_MAX_WRITE_TIME];

-            tmp_file.counters[CP_MAX_WRITE_TIME_SIZE] = 

-                inoutfile->counters[CP_MAX_WRITE_TIME_SIZE];

-        }

-

-        if(infile->fcounters[CP_F_MAX_READ_TIME] > 

-            inoutfile->fcounters[CP_F_MAX_READ_TIME])

-        {

-            tmp_file.fcounters[CP_F_MAX_READ_TIME] = 

-                infile->fcounters[CP_F_MAX_READ_TIME];

-            tmp_file.counters[CP_MAX_READ_TIME_SIZE] = 

-                infile->counters[CP_MAX_READ_TIME_SIZE];

-        }

-        else

-        {

-            tmp_file.fcounters[CP_F_MAX_READ_TIME] = 

-                inoutfile->fcounters[CP_F_MAX_READ_TIME];

-            tmp_file.counters[CP_MAX_READ_TIME_SIZE] = 

-                inoutfile->counters[CP_MAX_READ_TIME_SIZE];

-        }

-

-        /* min (zeroes are ok here; some procs don't do I/O) */

-        if(infile->fcounters[CP_F_FASTEST_RANK_TIME] <

-           inoutfile->fcounters[CP_F_FASTEST_RANK_TIME])

-        {

-            tmp_file.counters[CP_FASTEST_RANK] =

-                infile->counters[CP_FASTEST_RANK];

-            tmp_file.counters[CP_FASTEST_RANK_BYTES] = 

-                infile->counters[CP_FASTEST_RANK_BYTES];

-            tmp_file.fcounters[CP_F_FASTEST_RANK_TIME] =

-                infile->fcounters[CP_F_FASTEST_RANK_TIME];

-        }

-        else

-        {

-            tmp_file.counters[CP_FASTEST_RANK] =

-                inoutfile->counters[CP_FASTEST_RANK];

-            tmp_file.counters[CP_FASTEST_RANK_BYTES] =

-                inoutfile->counters[CP_FASTEST_RANK_BYTES];

-            tmp_file.fcounters[CP_F_FASTEST_RANK_TIME] = 

-                inoutfile->fcounters[CP_F_FASTEST_RANK_TIME];

-        }

-

-        /* max */

-        if(infile->fcounters[CP_F_SLOWEST_RANK_TIME] >

-           inoutfile->fcounters[CP_F_SLOWEST_RANK_TIME])

-        {

-            tmp_file.counters[CP_SLOWEST_RANK] =

-                infile->counters[CP_SLOWEST_RANK];

-            tmp_file.counters[CP_SLOWEST_RANK_BYTES] =

-                infile->counters[CP_SLOWEST_RANK_BYTES];

-            tmp_file.fcounters[CP_F_SLOWEST_RANK_TIME] = 

-                infile->fcounters[CP_F_SLOWEST_RANK_TIME];

-        }

-        else

-        {

-            tmp_file.counters[CP_SLOWEST_RANK] = 

-                inoutfile->counters[CP_SLOWEST_RANK];

-            tmp_file.counters[CP_SLOWEST_RANK_BYTES] = 

-                inoutfile->counters[CP_SLOWEST_RANK_BYTES];

-            tmp_file.fcounters[CP_F_SLOWEST_RANK_TIME] = 

-                inoutfile->fcounters[CP_F_SLOWEST_RANK_TIME];

-        }

-

-        /* pick one device id and file size, favoring complete records if

-         * available

-         */

-        if(CP_FILE_PARTIAL(infile))

-        {

-            tmp_file.counters[CP_DEVICE] = inoutfile->counters[CP_DEVICE];

-            tmp_file.counters[CP_SIZE_AT_OPEN] = inoutfile->counters[CP_SIZE_AT_OPEN];

-        }

-        else

-        {

-            tmp_file.counters[CP_DEVICE] = infile->counters[CP_DEVICE];

-            tmp_file.counters[CP_SIZE_AT_OPEN] = infile->counters[CP_SIZE_AT_OPEN];

-        }

-

-        /* pick one name suffix (every file record should have this, whether

-         * it is a partial record or not

-         */

-        strcpy(tmp_file.name_suffix, infile->name_suffix);

-

-        *inoutfile = tmp_file;

-        inoutfile++;

-        infile++;

-    }

-    

-    return;

-}

-/* cp_log_construct_indices()

- *

- * create memory datatypes to describe the log data to write out

- */

-static void cp_log_construct_indices(struct darshan_job_runtime* final_job, 

-    int rank, int* inout_count, int* lengths, void** pointers, char*

-    trailing_data)

-{

-    *inout_count = 0;

-

-    if(rank == 0)

-    {

-        /* root process is responsible for writing header */

-        lengths[*inout_count] = sizeof(final_job->log_job);

-        pointers[*inout_count] = &final_job->log_job;

-        (*inout_count)++;

-

-        /* also string containing exe command line */

-        lengths[*inout_count] = CP_EXE_LEN + 1; 

-        pointers[*inout_count] = trailing_data;

-        (*inout_count)++;

-    }

-

-    /* everyone adds their own file records, if present */

-    if(final_job->file_count > 0)

-    {

-        lengths[*inout_count] = final_job->file_count*CP_FILE_RECORD_SIZE;

-        pointers[*inout_count] = final_job->file_array;

-        (*inout_count)++;

-    }

-    

-    return;

-}

-

-/* cp_log_write()

- *

- * actually write log information to disk

- */

-static int cp_log_write(struct darshan_job_runtime* final_job, int rank, 

-    char* logfile_name, int count, int* lengths, void** pointers, double start_log_time)

-{

-    int ret;

-    MPI_File fh;

-    MPI_Status status;

-    MPI_Datatype mtype;

-    int my_total = 0;

-    long my_total_long;

-    long offset;

-    int i;

-    MPI_Aint displacements[CP_MAX_MEM_SEGMENTS];

-    void* buf;

-    int failed_write = 0;

-    char* hints;

-    char* key;

-    char* value;

-    char* tok_str;

-    char* orig_tok_str;

-    char* saveptr = NULL;

-    MPI_Info info;

-

-    /* skip building a datatype if we don't have anything to write */

-    if(count > 0)

-    {

-        /* construct data type to describe everything we are writing */

-        /* NOTE: there may be a bug in MPI-IO when using MPI_BOTTOM with an

-         * hindexed data type.  We will instead use the first pointer as a base

-         * and adjust the displacements relative to it.

-         */

-        buf = pointers[0];

-        for(i=0; i<count; i++)

-        {

-            /* use this transform to be compiler safe */

-            uintptr_t ptr  = (uintptr_t) pointers[i];

-            uintptr_t base = (uintptr_t) buf;

-            displacements[i] = (MPI_Aint)(ptr - base);

-        }

-        DARSHAN_MPI_CALL(PMPI_Type_hindexed)(count, lengths, displacements,

-            MPI_BYTE, &mtype);

-        DARSHAN_MPI_CALL(PMPI_Type_commit)(&mtype); 

-    }

-    

-    MPI_Info_create(&info);

-

-    /* check environment variable to see if the default MPI file hints have

-     * been overridden

-     */

-    hints = getenv(CP_LOG_HINTS_OVERRIDE);

-    if(!hints)

-    {

-        hints = __CP_LOG_HINTS;

-    }

-

-    if(hints && strlen(hints) > 0)

-    {

-        tok_str = strdup(hints);

-        if(tok_str)

-        {

-            orig_tok_str = tok_str;

-            do

-            {

-                /* split string on semicolon */

-                key = strtok_r(tok_str, ";", &saveptr);

-                if(key)

-                {

-                    tok_str = NULL;

-                    /* look for = sign splitting key/value pairs */

-                    value = index(key, '=');

-                    if(value)

-                    {

-                        /* break key and value into separate null terminated strings */

-                        value[0] = '\0';

-                        value++;

-                        if(strlen(key) > 0)

-                            MPI_Info_set(info, key, value);

-                    }

-                }

-            }while(key != NULL);

-            free(orig_tok_str);

-        }

-    }

-    

-    ret = DARSHAN_MPI_CALL(PMPI_File_open)(MPI_COMM_WORLD, logfile_name,

-        MPI_MODE_CREATE | MPI_MODE_WRONLY | MPI_MODE_EXCL, info, &fh);

-    MPI_Info_free(&info);

-    if(ret != MPI_SUCCESS)

-    {

-        if(rank == 0)

-        {

-            int msg_len;

-            char msg[MPI_MAX_ERROR_STRING] = {0};

-            

-            MPI_Error_string(ret, msg, &msg_len);

-            fprintf(stderr, "darshan library warning: unable to open log file %s: %s\n", logfile_name, msg);

-        }

-        if(count > 0)

-            DARSHAN_MPI_CALL(PMPI_Type_free)(&mtype);

-        return(-1);

-    }

-  

-    /* figure out where everyone is writing */

-    if(count > 0)

-        DARSHAN_MPI_CALL(PMPI_Type_size)(mtype, &my_total);

-    else

-        my_total = 0;

-    my_total_long = my_total;

-    DARSHAN_MPI_CALL(PMPI_Scan)(&my_total_long, &offset, 1,

-        MPI_LONG, MPI_SUM, MPI_COMM_WORLD); 

-    /* scan is inclusive; subtract local size back out */

-    offset -= my_total_long;

-

-    if(count > 0)

-    {

-        /* collectively write out file records from all processes */

-        ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(fh, offset, buf, 

-            1, mtype, &status);

-        if(ret != MPI_SUCCESS)

-        {

-            failed_write = 1;

-        }

-    }

-    else

-    {

-        /* nothing to write, but we need to participate in the 

-         * collectivee anyway 

-         */

-        ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(fh, offset, NULL, 

-            0, MPI_BYTE, &status);

-        if(ret != MPI_SUCCESS)

-        {

-            failed_write = 1;

-        }

-    }

-

-    DARSHAN_MPI_CALL(PMPI_File_close)(&fh);

-

-    if(count > 0)

-        DARSHAN_MPI_CALL(PMPI_Type_free)(&mtype);

-

-    if(failed_write)

-    {

-        return(-1);

-    }

-    return(0);

-}

-

-/* cp_log_compress()

- *

- * gzip memory buffers to write to log file.  Modifies the count, lengths,

- * and pointers to reference new buffer (or buffers)

- *

- * returns 0 on success, -1 on error

- */

-/* TODO: pick settings for compression (memory, level, etc.) */

-static int cp_log_compress(struct darshan_job_runtime* final_job,

-    int rank, int* inout_count, int* lengths, void** pointers)

-{

-    int ret = 0;

-    z_stream tmp_stream;

-    int total_target = 0;

-    int i;

-    int no_data_flag = 1;

-

-    /* do we actually have anything to write? */

-    for(i=0; i<*inout_count; i++)

-    {

-        if(lengths[i])

-        {

-            no_data_flag = 0;

-            break;

-        }

-    }

-

-    if(no_data_flag)

-    {

-        /* nothing to compress */

-        *inout_count = 0;

-        return(0);

-    }

-

-    memset(&tmp_stream, 0, sizeof(tmp_stream));

-    tmp_stream.zalloc = Z_NULL;

-    tmp_stream.zfree = Z_NULL;

-    tmp_stream.opaque = Z_NULL;

-

-    ret = deflateInit2(&tmp_stream, Z_DEFAULT_COMPRESSION, Z_DEFLATED,

-        31, 8, Z_DEFAULT_STRATEGY);

-    if(ret != Z_OK)

-    {

-        return(-1);

-    }

-

-    tmp_stream.next_out = (void*)final_job->comp_buf;

-    tmp_stream.avail_out = CP_COMP_BUF_SIZE;

-

-    /* loop through all pointers to be compressed */

-    for(i=0; i<*inout_count; i++)

-    {

-        total_target += lengths[i];

-        tmp_stream.next_in = pointers[i];

-        tmp_stream.avail_in = lengths[i];

-        /* while we have not finished consuming all of the data available to

-         * this point in the loop

-         */

-        while(tmp_stream.total_in < total_target)

-        {

-            if(tmp_stream.avail_out == 0)

-            {

-                /* We ran out of buffer space for compression.  In theory,

-                 * we could start using some of the file_array buffer space

-                 * without having to malloc again.  In practice, this case 

-                 * is going to be practically impossible to hit.

-                 */

-                deflateEnd(&tmp_stream);

-                return(-1);

-            }

-

-            /* compress data */

-            ret = deflate(&tmp_stream, Z_NO_FLUSH);

-            if(ret != Z_OK)

-            {

-                deflateEnd(&tmp_stream);

-                return(-1);

-            }

-        }

-    }

-    

-    /* flush compression and end */

-    ret = deflate(&tmp_stream, Z_FINISH);

-    if(ret != Z_STREAM_END)

-    {

-        deflateEnd(&tmp_stream);

-        return(-1);

-    }

-    deflateEnd(&tmp_stream);

-

-    /* substitute our new buffer */

-    pointers[0] = final_job->comp_buf;

-    lengths[0] = tmp_stream.total_out;

-    *inout_count = 1;

-

-    return(0);

-}

-

-static struct darshan_file_runtime* walker_file = NULL;

-static int walker_validx;

-static int walker_cntidx;

-

-static void cp_access_walker(const void* nodep, const VISIT which, const int depth)

-{

-    struct cp_access_counter* counter;

-

-    switch (which)

-    {

-        case postorder:

-        case leaf:

-            counter = *(struct cp_access_counter**)nodep;

-#if 0

-            printf("   type %d size: %" PRId64 ", freq: %d\n", walker_validx, counter->size, counter->freq);

-#endif

-            CP_COUNTER_INC(walker_file, counter->size, counter->freq, 1, walker_validx, walker_cntidx);

-        default:

-            break;

-    }

-

-    return;

-};

-

-/* darshan_walk_file_accesses()

- *

- * goes through runtime collections of accesses sizes and chooses the 4 most

- * common for logging

- */

-void darshan_walk_file_accesses(struct darshan_job_runtime* final_job)

-{

-    int i;

-

-    for(i=0; i<final_job->file_count; i++)

-    {

-        #if 0

-        printf("file: %d\n", i);

-        #endif

-        

-        /* walk trees for both access sizes and stride sizes to pick 4 most

-         * common of each

-         */

-

-        /* NOTE: setting global variables here for cp_access_walker() */

-        walker_file = &final_job->file_runtime_array[i];

-        walker_validx = CP_ACCESS1_ACCESS;

-        walker_cntidx = CP_ACCESS1_COUNT;

-        twalk(walker_file->access_root,

-            cp_access_walker);

-        tdestroy(walker_file->access_root, free);

-

-        walker_validx = CP_STRIDE1_STRIDE;

-        walker_cntidx = CP_STRIDE1_COUNT;

-        twalk(walker_file->stride_root,

-            cp_access_walker);

-        tdestroy(walker_file->stride_root, free);

-    }

-

-    return;

-}

-

-static int file_compare(const void* a, const void* b)

-{

-    const struct darshan_file* f_a = a;

-    const struct darshan_file* f_b = b;

-    

-    if(f_a->rank < f_b->rank)

-        return 1;

-    if(f_a->rank > f_b->rank)

-        return -1;

-    

-    return 0;

-}

-

-static int mnt_data_cmp(const void* a, const void* b)

-{

-    const struct mnt_data *d_a = (const struct mnt_data*)a;

-    const struct mnt_data *d_b = (const struct mnt_data*)b;

-

-    if(strlen(d_a->path) > strlen(d_b->path))

-        return(-1);

-    else if(strlen(d_a->path) < strlen(d_b->path))

-        return(1);

-    else

-        return(0);

-}

-

-/* adds an entry to table of mounted file systems */

-static void add_entry(char* trailing_data, int* space_left, struct mntent *entry)

-{

-    int ret;

-    char tmp_mnt[256];

-    struct statfs statfsbuf;

-    

-    strncpy(mnt_data_array[mnt_data_count].path, entry->mnt_dir, 

-        CP_MAX_MNT_PATH-1);

-    strncpy(mnt_data_array[mnt_data_count].type, entry->mnt_type, 

-        CP_MAX_MNT_TYPE-1);

-    mnt_data_array[mnt_data_count].hash = 

-        darshan_hash((void*)mnt_data_array[mnt_data_count].path, 

-        strlen(mnt_data_array[mnt_data_count].path), 0);

-    /* NOTE: we now try to detect the preferred block size for each file 

-     * system using fstatfs().  On Lustre we assume a size of 1 MiB 

-     * because fstatfs() reports 4 KiB. 

-     */

-#ifndef LL_SUPER_MAGIC

-#define LL_SUPER_MAGIC 0x0BD00BD0

-#endif

-    ret = statfs(entry->mnt_dir, &statfsbuf);

-    if(ret == 0 && statfsbuf.f_type != LL_SUPER_MAGIC)

-        mnt_data_array[mnt_data_count].block_size = statfsbuf.f_bsize;

-    else if(ret == 0 && statfsbuf.f_type == LL_SUPER_MAGIC)

-        mnt_data_array[mnt_data_count].block_size = 1024*1024;

-    else

-        mnt_data_array[mnt_data_count].block_size = 4096;

-

-    /* store mount information for use in header of darshan log */

-    ret = snprintf(tmp_mnt, 256, "\n%" PRId64 "\t%s\t%s", 

-        mnt_data_array[mnt_data_count].hash,

-        entry->mnt_type, entry->mnt_dir);

-    if(ret < 256 && strlen(tmp_mnt) <= (*space_left))

-    {

-        strcat(trailing_data, tmp_mnt);

-        (*space_left) -= strlen(tmp_mnt);

-    }

-    

-    mnt_data_count++;

-    return;

-}

-

-/* darshan_get_exe_and_mounts_root()

- *

- * collects command line and list of mounted file systems into a string that

- * will be stored with the job header

- */

-static void darshan_get_exe_and_mounts_root(struct darshan_job_runtime* final_job, char* trailing_data, int space_left)

-{

-    FILE* tab;

-    struct mntent *entry;

-    char* exclude;

-    int tmp_index = 0;

-    int skip = 0;

-

-    /* skip these fs types */

-    static char* fs_exclusions[] = {

-        "tmpfs",

-        "proc",

-        "sysfs",

-        "devpts",

-        "binfmt_misc",

-        "fusectl",

-        "debugfs",

-        "securityfs",

-        "nfsd",

-        "none",

-        "rpc_pipefs",

-       "hugetlbfs",

-       "cgroup",

-        NULL

-    };

-

-    /* length of exe has already been safety checked in darshan-posix.c */

-    strcat(trailing_data, final_job->exe);

-    space_left = CP_EXE_LEN - strlen(trailing_data);

-

-    /* we make two passes through mounted file systems; in the first pass we

-     * grab any non-nfs mount points, then on the second pass we grab nfs

-     * mount points

-     */

-

-    tab = setmntent("/etc/mtab", "r");

-    if(!tab)

-        return;

-    /* loop through list of mounted file systems */

-    while(mnt_data_count<CP_MAX_MNTS && (entry = getmntent(tab)) != NULL)

-    {

-        /* filter out excluded fs types */

-        tmp_index = 0;

-        skip = 0;

-        while((exclude = fs_exclusions[tmp_index]))

-        {

-            if(!(strcmp(exclude, entry->mnt_type)))

-            {

-                skip =1;

-                break; 

-            }

-            tmp_index++;

-        }

-

-        if(skip || (strcmp(entry->mnt_type, "nfs") == 0))

-            continue;

-        

-        add_entry(trailing_data, &space_left, entry);

-    }

-    endmntent(tab);

-

-    tab = setmntent("/etc/mtab", "r");

-    if(!tab)

-        return;

-    /* loop through list of mounted file systems */

-    while(mnt_data_count<CP_MAX_MNTS && (entry = getmntent(tab)) != NULL)

-    {

-        if(strcmp(entry->mnt_type, "nfs") != 0)

-            continue;

-        

-        add_entry(trailing_data, &space_left, entry);

-    }

-    endmntent(tab);

-

-    /* Sort mount points in order of longest path to shortest path.  This is

-     * necessary so that if we try to match file paths to mount points later

-     * we don't match on "/" every time.

-     */

-    qsort(mnt_data_array, mnt_data_count, sizeof(mnt_data_array[0]), mnt_data_cmp);

-    return;

-}

-

-/* darshan_get_exe_and_mounts()

- *

- * collects command line and list of mounted file systems into a string that

- * will be stored with the job header

- */

-char* darshan_get_exe_and_mounts(struct darshan_job_runtime* final_job)

-{

-    char* trailing_data;

-    int space_left;

-    int rank;

-

-    space_left = CP_EXE_LEN + 1;

-    trailing_data = malloc(space_left);

-    if(!trailing_data)

-    {

-        return(NULL);

-    }

-    memset(trailing_data, 0, space_left);

-

-    DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &rank);

-    if(rank == 0)

-    {

-        darshan_get_exe_and_mounts_root(final_job, trailing_data, space_left);

-    }

-

-    /* broadcast trailing data to all nodes */

-    DARSHAN_MPI_CALL(PMPI_Bcast)(trailing_data, space_left, MPI_CHAR, 0,

-        MPI_COMM_WORLD);

-    /* broadcast mount count to all nodes */

-    DARSHAN_MPI_CALL(PMPI_Bcast)(&mnt_data_count, 1, MPI_INT, 0,

-        MPI_COMM_WORLD);

-    /* broadcast mount data to all nodes */

-    DARSHAN_MPI_CALL(PMPI_Bcast)(mnt_data_array, 

-        mnt_data_count*sizeof(mnt_data_array[0]), MPI_BYTE, 0, MPI_COMM_WORLD);

-

-    return(trailing_data);

-}

-

-/*

- * Computes population variance of bytes moved and total time

- * for each rank on a shared file.

- */

-static int darshan_file_variance(

-    struct darshan_file *infile_array,

-    struct darshan_file *outfile_array,

-    int count, int rank)

-{

-    MPI_Op pw_var_op = MPI_OP_NULL;

-    MPI_Datatype var_dt = MPI_BYTE;

-    int ret;

-    int i;

-    struct variance_dt* var_array = NULL;

-    struct variance_dt* varres_array = NULL;

-

-    ret = DARSHAN_MPI_CALL(PMPI_Op_create)(pairwise_variance_reduce, 1,

-        &pw_var_op);

-    if (ret != MPI_SUCCESS)

-    {

-        goto error_handler;

-    }

-

-    ret = DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct variance_dt),

-        MPI_BYTE, &var_dt);

-    if (ret != MPI_SUCCESS)

-    {

-        goto error_handler;

-    }

-

-    ret = DARSHAN_MPI_CALL(PMPI_Type_commit)(&var_dt);

-    if (ret != MPI_SUCCESS)

-    {

-        goto error_handler;

-    }

-

-    var_array = malloc(count*sizeof(struct variance_dt));

-    if(!var_array)

-    {

-        goto error_handler;

-    }       

-

-    if (rank == 0)

-    {

-        varres_array = malloc(count*sizeof(struct variance_dt));

-        if(!varres_array)

-        {

-            goto error_handler;

-        }

-    }

- 

-    /*

-     * total time

-     */

-

-    for(i=0; i<count; i++)

-    {

-        var_array[i].n = 1;

-        var_array[i].S = 0;

-        var_array[i].T = infile_array[i].fcounters[CP_F_POSIX_META_TIME] +

-                         infile_array[i].fcounters[CP_F_POSIX_READ_TIME] +

-                         infile_array[i].fcounters[CP_F_POSIX_WRITE_TIME];

-    } 

-

-    ret = DARSHAN_MPI_CALL(PMPI_Reduce)(

-             var_array, varres_array, count, var_dt, pw_var_op,

-             0, MPI_COMM_WORLD);

-    if(ret != MPI_SUCCESS)

-    {

-        goto error_handler;

-    }

-

-    if (rank == 0)

-    {

-        for(i=0; i<count; i++)

-        {

-            outfile_array[i].fcounters[CP_F_VARIANCE_RANK_TIME] =

-                (varres_array[i].S / varres_array[i].n);

-        }

-    }

-

-    /*

-     * total bytes

-     */

-    for(i=0; i<count; i++)

-    {

-        var_array[i].n = 1;

-        var_array[i].S = 0;

-        var_array[i].T = (double)

-                         infile_array[i].counters[CP_BYTES_READ] +

-                         infile_array[i].counters[CP_BYTES_WRITTEN];

-    } 

-

-    ret = DARSHAN_MPI_CALL(PMPI_Reduce)(

-             var_array, varres_array, count, var_dt, pw_var_op,

-             0, MPI_COMM_WORLD);

-    if(ret != MPI_SUCCESS)

-    {

-        goto error_handler;

-    }

-

-    if (rank == 0)

-    {

-        for(i=0; i<count; i++)

-        {

-            outfile_array[i].fcounters[CP_F_VARIANCE_RANK_BYTES] =

-                (varres_array[i].S / varres_array[i].n);

-        }

-    }

-

-    ret = 0;

-

-error_handler:

-    if (var_dt != MPI_BYTE) DARSHAN_MPI_CALL(PMPI_Type_free)(&var_dt);

-    if (pw_var_op != MPI_OP_NULL) DARSHAN_MPI_CALL(PMPI_Op_free)(&pw_var_op);

-    if (var_array) free(var_array);

-    if (varres_array) free(varres_array);

-

-    return ret;

-}

-

-static void pairwise_variance_reduce (

-    void *invec, void *inoutvec, int *len, MPI_Datatype *dt)

-{

-    int i;

-    struct variance_dt *X = invec;

-    struct variance_dt *Y = inoutvec;

-    struct variance_dt  Z;

-

-    for (i=0; i<*len; i++,X++,Y++)

-    {

-        Z.n = X->n + Y->n;

-        Z.T = X->T + Y->T;

-        Z.S = X->S + Y->S + (X->n/(Y->n*Z.n)) *

-           ((Y->n/X->n)*X->T - Y->T) * ((Y->n/X->n)*X->T - Y->T);

-

-        *Y = Z;

-    }

-

-    return;

-}

-

-/* record any hints used to write the darshan log in the log header */

-static void cp_log_record_hints_and_ver(struct darshan_job_runtime* final_job, int rank)

-{

-    char* hints;

-    char* header_hints;

-    int meta_remain = 0;

-    char* m;

-

-    /* only need to do this on first process */

-    if(rank > 0)

-        return;

-

-    /* check environment variable to see if the default MPI file hints have

-     * been overridden

-     */

-    hints = getenv(CP_LOG_HINTS_OVERRIDE);

-    if(!hints)

-    {

-        hints = __CP_LOG_HINTS;

-    }

-

-    if(!hints || strlen(hints) < 1)

-        return;

-

-    header_hints = strdup(hints);

-    if(!header_hints)

-        return;

-

-    meta_remain = DARSHAN_JOB_METADATA_LEN - 

-        strlen(final_job->log_job.metadata) - 1;

-    if(meta_remain >= (strlen(PACKAGE_VERSION) + 9))

-    {

-        sprintf(final_job->log_job.metadata, "lib_ver=%s\n", PACKAGE_VERSION);

-        meta_remain -= (strlen(PACKAGE_VERSION) + 9);

-    }

-    if(meta_remain >= (3 + strlen(header_hints)))

-    {

-        m = final_job->log_job.metadata + strlen(final_job->log_job.metadata);

-        /* We have room to store the hints in the metadata portion of

-         * the job header.  We just prepend an h= to the hints list.  The

-         * metadata parser will ignore = characters that appear in the value

-         * portion of the metadata key/value pair.

-         */

-        sprintf(m, "h=%s\n", header_hints);

-    }

-    free(header_hints);

-

-    return;

-}

-

-#if 0

-static void debug_mounts(const char* mtab_file, const char* out_file)

-{

-    FILE* tab;

-    struct mntent *entry;

-    int ret;

-    struct stat statbuf;

-    FILE* out;

-

-    out = fopen(out_file, "w");

-    if(!out)

-    {

-        perror("darshan: fopen");

-        return;

-    }

-

-    tab = setmntent(mtab_file, "r");

-    if(!tab)

-    {

-        perror("darshan: setmnt");

-        return;

-    }

-

-    while((entry = getmntent(tab)) != NULL)

-    {

-        ret = stat(entry->mnt_dir, &statbuf);

-        if(ret == 0)

-        {

-            int64_t tmp_st_dev = statbuf.st_dev;

-

-            fprintf(out, "%" PRId64 "\t%s\t%s\n", tmp_st_dev, 

-                entry->mnt_type, entry->mnt_dir);

-        }

-        else

-        {

-            perror("darshan: stat");

-        }

-    }

-    return;

-}

-#endif

-

-static struct darshan_file_runtime* darshan_file_by_name_setfh(const char* name, MPI_File fh)

-{

-    struct darshan_file_runtime* tmp_file;

-

-    tmp_file = darshan_file_by_name_sethandle(name, &fh, sizeof(fh), DARSHAN_FH);

-    return(tmp_file);

-}

-

-static void darshan_file_close_fh(MPI_File fh)

-{

-    darshan_file_closehandle(&fh, sizeof(fh), DARSHAN_FH);

-    return;

-}

-

-static struct darshan_file_runtime* darshan_file_by_fh(MPI_File fh)

-{

-    struct darshan_file_runtime* tmp_file;

-

-    tmp_file = darshan_file_by_handle(&fh, sizeof(fh), DARSHAN_FH);

-    

-    return(tmp_file);

-}

-

-/* find the device id and block size for the specified file, based on 

- * data from the mount entries.

- */

-void darshan_mnt_id_from_path(const char* path, int64_t* device_id, int64_t* block_size)

-{

-    int i;

-    *device_id = -1;

-    *block_size = -1;

-

-    for(i=0; i<mnt_data_count; i++)

-    {

-        if(!(strncmp(mnt_data_array[i].path, path, strlen(mnt_data_array[i].path))))

-        {

-            *device_id = mnt_data_array[i].hash;

-            *block_size = mnt_data_array[i].block_size;

-            return;

-        }

-    }

-

-    return;

-}

-

-/* iterates through counters and adjusts timestamps to be relative to

- * MPI_Init()

- */

-static void cp_normalize_timestamps(struct darshan_job_runtime* final_job)

-{

-    int i;

-    int j;

-

-    for(i=0; i<final_job->file_count; i++)

-    {

-        for(j=CP_F_OPEN_TIMESTAMP; j<=CP_F_WRITE_END_TIMESTAMP; j++)

-        {

-            if(final_job->file_array[i].fcounters[j] > final_job->wtime_offset)

-                final_job->file_array[i].fcounters[j] -= final_job->wtime_offset;

-        }

-    }

-

-    return;

-}

-

-/*

- * Local variables:

- *  c-indent-level: 4

- *  c-basic-offset: 4

- * End:

- *

- * vim: ts=8 sts=4 sw=4 expandtab

- */

</span></code></pre>

<br>

</li>

<li id='diff-30'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-30'>

<strong>

darshan-runtime/lib/darshan-mpiio.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-mpiio.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,1541 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#include "darshan-runtime-config.h"

+#include <stdio.h>

+#include <unistd.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <stdarg.h>

+#include <string.h>

+#include <time.h>

+#include <stdlib.h>

+#include <errno.h>

+#include <search.h>

+#include <assert.h>

+#define __USE_GNU

+#include <pthread.h>

+

+#include "uthash.h"

+

+#include "darshan.h"

+#include "darshan-dynamic.h"

+

+/* The mpiio_file_runtime structure maintains necessary runtime metadata

+ * for the MPIIO file record (darshan_mpiio_file structure, defined in

+ * darshan-mpiio-log-format.h) pointed to by 'file_record'. This metadata

+ * assists with the instrumenting of specific statistics in the file record.

+ * 'hlink' is a hash table link structure used to add/remove this record

+ * from the hash table of MPIIO file records for this process. 

+ *

+ * RATIONALE: the MPIIO module needs to track some stateful, volatile 

+ * information about each open file (like the current file offset, most recent 

+ * access time, etc.) to aid in instrumentation, but this information can't be

+ * stored in the darshan_mpiio_file struct because we don't want it to appear in

+ * the final darshan log file.  We therefore associate a mpiio_file_runtime

+ * struct with each darshan_mpiio_file struct in order to track this information.

+  *

+ * NOTE: There is a one-to-one mapping of mpiio_file_runtime structs to

+ * darshan_mpiio_file structs.

+ *

+ * NOTE: The mpiio_file_runtime struct contains a pointer to a darshan_mpiio_file

+ * struct (see the *file_record member) rather than simply embedding an entire

+ * darshan_mpiio_file struct.  This is done so that all of the darshan_mpiio_file

+ * structs can be kept contiguous in memory as a single array to simplify

+ * reduction, compression, and storage.

+ */

+struct mpiio_file_runtime

+{

+    struct darshan_mpiio_file* file_record;

+    enum darshan_io_type last_io_type;

+    double last_meta_end;

+    double last_read_end;

+    double last_write_end;

+    void *access_root;

+    int access_count;

+    UT_hash_handle hlink;

+};

+

+/* The mpiio_file_runtime_ref structure is used to associate a MPIIO

+ * file handle with an already existing MPIIO file record. This is

+ * necessary as many MPIIO I/O functions take only a file handle as input,

+ * but MPIIO file records are indexed by their full file paths (i.e., darshan

+ * record identifiers for MPIIO files are created by hashing the file path).

+ * In other words, this structure is necessary as it allows us to look up a

+ * file record either by a pathname (mpiio_file_runtime) or by MPIIO file

+ * descriptor (mpiio_file_runtime_ref), depending on which parameters are

+ * available. This structure includes another hash table link, since separate

+ * hashes are maintained for mpiio_file_runtime structures and mpiio_file_runtime_ref

+ * structures.

+ *

+ * RATIONALE: In theory the file handle information could be included in the

+ * mpiio_file_runtime struct rather than in a separate structure here.  The

+ * reason we don't do that is to handle the potential for an MPI implementation

+ * to produce a new file handle instance each time MPI_File_open() is called on a

+ * file.  Thus there might be multiple file handles referring to the same

+ * underlying record.

+ *

+ * NOTE: there are potentially multiple mpiio_file_runtime_ref structures

+ * referring to a single mpiio_file_runtime structure.  Most of the time there is

+ * only one, however.

+ */

+struct mpiio_file_runtime_ref

+{

+    struct mpiio_file_runtime* file;

+    MPI_File fh;

+    UT_hash_handle hlink;

+};

+

+/* The mpiio_runtime structure maintains necessary state for storing

+ * MPI-IO file records and for coordinating with darshan-core at 

+ * shutdown time.

+ */

+struct mpiio_runtime

+{

+    struct mpiio_file_runtime* file_runtime_array;

+    struct darshan_mpiio_file* file_record_array;

+    int file_array_size;

+    int file_array_ndx;

+    struct mpiio_file_runtime* file_hash;

+    struct mpiio_file_runtime_ref* fh_hash;

+};

+

+static struct mpiio_runtime *mpiio_runtime = NULL;

+static pthread_mutex_t mpiio_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+static int instrumentation_disabled = 0;

+static int my_rank = -1;

+

+static void mpiio_runtime_initialize(void);

+static struct mpiio_file_runtime* mpiio_file_by_name(const char *name);

+static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File fh);

+static struct mpiio_file_runtime* mpiio_file_by_fh(MPI_File fh);

+static void mpiio_file_close_fh(MPI_File fh);

+static int mpiio_record_compare(const void* a, const void* b);

+static void mpiio_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype);

+static void mpiio_shared_record_variance(MPI_Comm mod_comm,

+    struct darshan_mpiio_file *inrec_array, struct darshan_mpiio_file *outrec_array,

+    int shared_rec_count);

+

+static void mpiio_begin_shutdown(void);

+static void mpiio_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,

+    int shared_rec_count, void **mpiio_buf, int *mpiio_buf_sz);

+static void mpiio_shutdown(void);

+

+#define MPIIO_LOCK() pthread_mutex_lock(&mpiio_runtime_mutex)

+#define MPIIO_UNLOCK() pthread_mutex_unlock(&mpiio_runtime_mutex)

+

+#define MPIIO_RECORD_OPEN(__ret, __path, __fh, __comm, __mode, __info, __tm1, __tm2) do { \

+    struct mpiio_file_runtime* file; \

+    char *exclude; \

+    int tmp_index = 0; \

+    int comm_size; \

+    if(__ret != MPI_SUCCESS) break; \

+    while((exclude=darshan_path_exclusions[tmp_index])) { \

+        if(!(strncmp(exclude, __path, strlen(exclude)))) \

+            break; \

+        tmp_index++; \

+    } \

+    if(exclude) break; \

+    file = mpiio_file_by_name_setfh(__path, __fh); \

+    if(!file) break; \

+    file->file_record->counters[MPIIO_MODE] = __mode; \

+    DARSHAN_MPI_CALL(PMPI_Comm_size)(__comm, &comm_size); \

+    if(comm_size == 1) \

+        file->file_record->counters[MPIIO_INDEP_OPENS] += 1; \

+    else \

+        file->file_record->counters[MPIIO_COLL_OPENS] += 1; \

+    if(__info != MPI_INFO_NULL) \

+        file->file_record->counters[MPIIO_HINTS] += 1; \

+    if(file->file_record->fcounters[MPIIO_F_OPEN_TIMESTAMP] == 0) \

+        file->file_record->fcounters[MPIIO_F_OPEN_TIMESTAMP] = __tm1; \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[MPIIO_F_META_TIME], __tm1, __tm2, file->last_meta_end); \

+} while(0)

+

+#define MPIIO_RECORD_READ(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \

+    struct mpiio_file_runtime* file; \

+    int size = 0; \

+    double __elapsed = __tm2-__tm1; \

+    if(__ret != MPI_SUCCESS) break; \

+    file = mpiio_file_by_fh(__fh); \

+    if(!file) break; \

+    DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size);  \

+    size = size * __count; \

+    DARSHAN_BUCKET_INC(&(file->file_record->counters[MPIIO_SIZE_READ_AGG_0_100]), size); \

+    darshan_common_val_counter(&file->access_root, &file->access_count, size); \

+    file->file_record->counters[MPIIO_BYTES_READ] += size; \

+    file->file_record->counters[__counter] += 1; \

+    if(file->last_io_type == DARSHAN_IO_WRITE) \

+        file->file_record->counters[MPIIO_RW_SWITCHES] += 1; \

+    file->last_io_type = DARSHAN_IO_READ; \

+    if(file->file_record->fcounters[MPIIO_F_READ_START_TIMESTAMP] == 0) \

+        file->file_record->fcounters[MPIIO_F_READ_START_TIMESTAMP] = __tm1; \

+    file->file_record->fcounters[MPIIO_F_READ_END_TIMESTAMP] = __tm2; \

+    if(file->file_record->fcounters[MPIIO_F_MAX_READ_TIME] < __elapsed) { \

+        file->file_record->fcounters[MPIIO_F_MAX_READ_TIME] = __elapsed; \

+        file->file_record->counters[MPIIO_MAX_READ_TIME_SIZE] = size; } \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[MPIIO_F_READ_TIME], __tm1, __tm2, file->last_read_end); \

+} while(0)

+

+#define MPIIO_RECORD_WRITE(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \

+    struct mpiio_file_runtime* file; \

+    int size = 0; \

+    double __elapsed = __tm2-__tm1; \

+    if(__ret != MPI_SUCCESS) break; \

+    file = mpiio_file_by_fh(__fh); \

+    if(!file) break; \

+    DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size);  \

+    size = size * __count; \

+    DARSHAN_BUCKET_INC(&(file->file_record->counters[MPIIO_SIZE_WRITE_AGG_0_100]), size); \

+    darshan_common_val_counter(&file->access_root, &file->access_count, size); \

+    file->file_record->counters[MPIIO_BYTES_WRITTEN] += size; \

+    file->file_record->counters[__counter] += 1; \

+    if(file->last_io_type == DARSHAN_IO_READ) \

+        file->file_record->counters[MPIIO_RW_SWITCHES] += 1; \

+    file->last_io_type = DARSHAN_IO_WRITE; \

+    if(file->file_record->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] == 0) \

+        file->file_record->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] = __tm1; \

+    file->file_record->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] = __tm2; \

+    if(file->file_record->fcounters[MPIIO_F_MAX_WRITE_TIME] < __elapsed) { \

+        file->file_record->fcounters[MPIIO_F_MAX_WRITE_TIME] = __elapsed; \

+        file->file_record->counters[MPIIO_MAX_WRITE_TIME_SIZE] = size; } \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[MPIIO_F_WRITE_TIME], __tm1, __tm2, file->last_write_end); \

+} while(0)

+

+/**********************************************************

+ *        Wrappers for MPI-IO functions of interest       * 

+ **********************************************************/

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_open(MPI_Comm comm, const char *filename, int amode, MPI_Info info, MPI_File *fh) 

+#else

+int MPI_File_open(MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_File *fh) 

+#endif

+{

+    int ret;

+    char* tmp;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_open)(comm, filename, amode, info, fh);

+    tm2 = darshan_core_wtime();

+

+    /* use ROMIO approach to strip prefix if present */

+    /* strip off prefix if there is one, but only skip prefixes

+     * if they are greater than length one to allow for windows

+     * drive specifications (e.g. c:\...) 

+     */

+    tmp = strchr(filename, ':');

+    if (tmp > filename + 1) {

+        filename = tmp + 1;

+    }

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_OPEN(ret, filename, (*fh), comm, amode, info, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read(MPI_File fh, void *buf, int count,

+    MPI_Datatype datatype, MPI_Status *status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read)(fh, buf, count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write(MPI_File fh, const void *buf, int count,

+    MPI_Datatype datatype, MPI_Status *status)

+#else

+int MPI_File_write(MPI_File fh, void *buf, int count,

+    MPI_Datatype datatype, MPI_Status *status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write)(fh, buf, count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_at(MPI_File fh, MPI_Offset offset, void *buf,

+    int count, MPI_Datatype datatype, MPI_Status *status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_at)(fh, offset, buf,

+        count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_at(MPI_File fh, MPI_Offset offset, const void *buf,

+    int count, MPI_Datatype datatype, MPI_Status *status)

+#else

+int MPI_File_write_at(MPI_File fh, MPI_Offset offset, void *buf,

+    int count, MPI_Datatype datatype, MPI_Status *status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(fh, offset, buf,

+        count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_all)(fh, buf, count,

+        datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_all(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+#else

+int MPI_File_write_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_all)(fh, buf, count,

+        datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_at_all(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype, MPI_Status * status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all)(fh, offset, buf,

+        count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, const void * buf,

+    int count, MPI_Datatype datatype, MPI_Status * status)

+#else

+int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype, MPI_Status * status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(fh, offset, buf,

+        count, datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_shared)(fh, buf, count,

+        datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_shared(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+#else

+int MPI_File_write_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_shared)(fh, buf, count,

+        datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_ordered(MPI_File fh, void * buf, int count,

+    MPI_Datatype datatype, MPI_Status * status)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered)(fh, buf, count,

+        datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_ordered(MPI_File fh, const void * buf, int count,

+    MPI_Datatype datatype, MPI_Status * status)

+#else

+int MPI_File_write_ordered(MPI_File fh, void * buf, int count,

+    MPI_Datatype datatype, MPI_Status * status)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered)(fh, buf, count,

+         datatype, status);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_all_begin)(fh, buf, count, datatype);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_all_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)

+#else

+int MPI_File_write_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_all_begin)(fh, buf, count, datatype);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all_begin)(fh, offset, buf,

+        count, datatype);

+    tm2 = darshan_core_wtime();

+    

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, const void * buf,

+    int count, MPI_Datatype datatype)

+#else

+int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all_begin)(fh, offset,

+        buf, count, datatype);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_read_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered_begin)(fh, buf, count,

+        datatype);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_write_ordered_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)

+#else

+int MPI_File_write_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered_begin)(fh, buf, count,

+        datatype);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_iread(MPI_File fh, void * buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST * request)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iread)(fh, buf, count, datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_iwrite(MPI_File fh, const void * buf, int count,

+    MPI_Datatype datatype, __D_MPI_REQUEST * request)

+#else

+int MPI_File_iwrite(MPI_File fh, void * buf, int count,

+    MPI_Datatype datatype, __D_MPI_REQUEST * request)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite)(fh, buf, count, datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_iread_at(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iread_at)(fh, offset, buf, count,

+        datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, const void * buf,

+    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

+#else

+int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, void * buf,

+    int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_at)(fh, offset, buf,

+        count, datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_iread_shared(MPI_File fh, void * buf, int count,

+    MPI_Datatype datatype, __D_MPI_REQUEST * request)

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iread_shared)(fh, buf, count,

+        datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_iwrite_shared(MPI_File fh, const void * buf, int count,

+    MPI_Datatype datatype, __D_MPI_REQUEST * request)

+#else

+int MPI_File_iwrite_shared(MPI_File fh, void * buf, int count,

+    MPI_Datatype datatype, __D_MPI_REQUEST * request)

+#endif

+{

+    int ret;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_shared)(fh, buf, count,

+        datatype, request);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);

+    MPIIO_UNLOCK();

+    return(ret);

+}

+

+int MPI_File_sync(MPI_File fh)

+{

+    int ret;

+    struct mpiio_file_runtime* file;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_sync)(fh);

+    tm2 = darshan_core_wtime();

+

+    if(ret == MPI_SUCCESS)

+    {

+        MPIIO_LOCK();

+        mpiio_runtime_initialize();

+        file = mpiio_file_by_fh(fh);

+        if(file)

+        {

+            file->file_record->counters[MPIIO_SYNCS] += 1;

+            DARSHAN_TIMER_INC_NO_OVERLAP(

+                file->file_record->fcounters[MPIIO_F_WRITE_TIME],

+                tm1, tm2, file->last_write_end);

+        }

+        MPIIO_UNLOCK();

+    }

+

+    return(ret);

+}

+

+#ifdef HAVE_MPIIO_CONST

+int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype,

+    MPI_Datatype filetype, const char *datarep, MPI_Info info)

+#else

+int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype,

+    MPI_Datatype filetype, char *datarep, MPI_Info info)

+#endif

+{

+    int ret;

+    struct mpiio_file_runtime* file;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_set_view)(fh, disp, etype, filetype,

+        datarep, info);

+    tm2 = darshan_core_wtime();

+

+    if(ret == MPI_SUCCESS)

+    {

+        MPIIO_LOCK();

+        mpiio_runtime_initialize();

+        file = mpiio_file_by_fh(fh);

+        if(file)

+        {

+            file->file_record->counters[MPIIO_VIEWS] += 1;

+            if(info != MPI_INFO_NULL)

+            {

+                file->file_record->counters[MPIIO_HINTS] += 1;

+                DARSHAN_TIMER_INC_NO_OVERLAP(

+                    file->file_record->fcounters[MPIIO_F_META_TIME],

+                    tm1, tm2, file->last_meta_end);

+           }

+        }

+        MPIIO_UNLOCK();

+    }

+

+    return(ret);

+}

+

+int MPI_File_close(MPI_File *fh)

+{

+    int ret;

+    struct mpiio_file_runtime* file;

+    MPI_File tmp_fh = *fh;

+    double tm1, tm2;

+

+    tm1 = darshan_core_wtime();

+    ret = DARSHAN_MPI_CALL(PMPI_File_close)(fh);

+    tm2 = darshan_core_wtime();

+

+    MPIIO_LOCK();

+    mpiio_runtime_initialize();

+    file = mpiio_file_by_fh(tmp_fh);

+    if(file)

+    {

+        file->file_record->fcounters[MPIIO_F_CLOSE_TIMESTAMP] =

+            darshan_core_wtime();

+        DARSHAN_TIMER_INC_NO_OVERLAP(

+            file->file_record->fcounters[MPIIO_F_META_TIME],

+            tm1, tm2, file->last_meta_end);

+        mpiio_file_close_fh(tmp_fh);

+    }

+    MPIIO_UNLOCK();

+

+    return(ret);

+}

+

+/***********************************************************

+ * Internal functions for manipulating MPI-IO module state *

+ ***********************************************************/

+

+/* initialize data structures and register with darshan-core component */

+static void mpiio_runtime_initialize()

+{

+    int mem_limit;

+    struct darshan_module_funcs mpiio_mod_fns =

+    {

+        .begin_shutdown = &mpiio_begin_shutdown,

+        .get_output_data = &mpiio_get_output_data,

+        .shutdown = &mpiio_shutdown

+    };

+

+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(mpiio_runtime || instrumentation_disabled)

+        return;

+

+    /* register the mpiio module with darshan core */

+    darshan_core_register_module(

+        DARSHAN_MPIIO_MOD,

+        &mpiio_mod_fns,

+        &my_rank,

+        &mem_limit,

+        NULL);

+

+    /* return if no memory assigned by darshan core */

+    if(mem_limit == 0)

+        return;

+

+    mpiio_runtime = malloc(sizeof(*mpiio_runtime));

+    if(!mpiio_runtime)

+        return;

+    memset(mpiio_runtime, 0, sizeof(*mpiio_runtime));

+

+    /* set maximum number of file records according to max memory limit */

+    /* NOTE: maximum number of records is based on the size of a mpiio file record */

+    mpiio_runtime->file_array_size = mem_limit / sizeof(struct darshan_mpiio_file);

+    mpiio_runtime->file_array_ndx = 0;

+

+    /* allocate array of runtime file records */

+    mpiio_runtime->file_runtime_array = malloc(mpiio_runtime->file_array_size *

+                                               sizeof(struct mpiio_file_runtime));

+    mpiio_runtime->file_record_array = malloc(mpiio_runtime->file_array_size *

+                                              sizeof(struct darshan_mpiio_file));

+    if(!mpiio_runtime->file_runtime_array || !mpiio_runtime->file_record_array)

+    {

+        mpiio_runtime->file_array_size = 0;

+        return;

+    }

+    memset(mpiio_runtime->file_runtime_array, 0, mpiio_runtime->file_array_size *

+           sizeof(struct mpiio_file_runtime));

+    memset(mpiio_runtime->file_record_array, 0, mpiio_runtime->file_array_size *

+           sizeof(struct darshan_mpiio_file));

+

+    return;

+}

+

+/* get a MPIIO file record for the given file path */

+static struct mpiio_file_runtime* mpiio_file_by_name(const char *name)

+{

+    struct mpiio_file_runtime *file = NULL;

+    char *newname = NULL;

+    darshan_record_id file_id;

+    int limit_flag;

+

+    if(!mpiio_runtime || instrumentation_disabled)

+        return(NULL);

+

+    newname = darshan_clean_file_path(name);

+    if(!newname)

+        newname = (char*)name;

+

+    limit_flag = (mpiio_runtime->file_array_ndx >= mpiio_runtime->file_array_size);

+

+    /* get a unique id for this file from darshan core */

+    darshan_core_register_record(

+        (void*)newname,

+        strlen(newname),

+        DARSHAN_MPIIO_MOD,

+        1,

+        limit_flag,

+        &file_id,

+        NULL);

+

+    /* the file record id is set to 0 if no memory is available for tracking

+     * new records -- just fall through and ignore this record

+     */

+    if(file_id == 0)

+    {

+        if(newname != name)

+            free(newname);

+        return(NULL);

+    }

+

+    /* search the hash table for this file record, and return if found */

+    HASH_FIND(hlink, mpiio_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);

+    if(file)

+    {

+        if(newname != name)

+            free(newname);

+        return(file);

+    }

+

+    /* no existing record, assign a new file record from the global array */

+    file = &(mpiio_runtime->file_runtime_array[mpiio_runtime->file_array_ndx]);

+    file->file_record = &(mpiio_runtime->file_record_array[mpiio_runtime->file_array_ndx]);

+    file->file_record->f_id = file_id;

+    file->file_record->rank = my_rank;

+

+    /* add new record to file hash table */

+    HASH_ADD(hlink, mpiio_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);

+    mpiio_runtime->file_array_ndx++;

+

+    if(newname != name)

+        free(newname);

+    return(file);

+}

+

+/* get an MPIIO file record for the given file path, and also create a

+ * reference structure using the corresponding file handle

+ */

+static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File fh)

+{

+    struct mpiio_file_runtime* file;

+    struct mpiio_file_runtime_ref* ref;

+

+    if(!mpiio_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* find file record by name first */

+    file = mpiio_file_by_name(name);

+

+    if(!file)

+        return(NULL);

+

+    /* search hash table for existing file ref for this fh */

+    HASH_FIND(hlink, mpiio_runtime->fh_hash, &fh, sizeof(fh), ref);

+    if(ref)

+    {

+        /* we have a reference.  Make sure it points to the correct file

+         * and return it

+         */

+        ref->file = file;

+        return(file);

+    }

+

+    /* if we hit this point, then we don't have a reference for this fh

+     * in the table yet.  Add it.

+     */

+    ref = malloc(sizeof(*ref));

+    if(!ref)

+        return(NULL);

+    memset(ref, 0, sizeof(*ref));

+

+    ref->file = file;

+    ref->fh = fh;    

+    HASH_ADD(hlink, mpiio_runtime->fh_hash, fh, sizeof(fh), ref);

+

+    return(file);

+}

+

+/* get an MPIIO file record for the given file handle */

+static struct mpiio_file_runtime* mpiio_file_by_fh(MPI_File fh)

+{

+    struct mpiio_file_runtime_ref* ref;

+

+    if(!mpiio_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* search hash table for existing file ref for this file handle */

+    HASH_FIND(hlink, mpiio_runtime->fh_hash, &fh, sizeof(fh), ref);

+    if(ref)

+        return(ref->file);

+

+    return(NULL);

+}

+

+/* free up reference data structures for the given file handle */

+static void mpiio_file_close_fh(MPI_File fh)

+{

+    struct mpiio_file_runtime_ref* ref;

+

+    if(!mpiio_runtime || instrumentation_disabled)

+        return;

+

+    /* search hash table for this fd */

+    HASH_FIND(hlink, mpiio_runtime->fh_hash, &fh, sizeof(fh), ref);

+    if(ref)

+    {

+        /* we have a reference, delete it */

+        HASH_DELETE(hlink, mpiio_runtime->fh_hash, ref);

+        free(ref);

+    }

+

+    return;

+}

+

+/* compare function for sorting file records by descending rank */

+static int mpiio_record_compare(const void* a_p, const void* b_p)

+{

+    const struct darshan_mpiio_file* a = a_p;

+    const struct darshan_mpiio_file* b = b_p;

+

+    if(a->rank < b->rank)

+        return 1;

+    if(a->rank > b->rank)

+        return -1;

+

+    return 0;

+}

+

+static void mpiio_record_reduction_op(

+    void* infile_v,

+    void* inoutfile_v,

+    int *len,

+    MPI_Datatype *datatype)

+{

+    struct darshan_mpiio_file tmp_file;

+    struct darshan_mpiio_file *infile = infile_v;

+    struct darshan_mpiio_file *inoutfile = inoutfile_v;

+    int i, j, k;

+

+    assert(mpiio_runtime);

+

+    for(i=0; i<*len; i++)

+    {

+        memset(&tmp_file, 0, sizeof(struct darshan_mpiio_file));

+

+        tmp_file.f_id = infile->f_id;

+        tmp_file.rank = -1;

+

+        /* sum */

+        for(j=MPIIO_INDEP_OPENS; j<=MPIIO_VIEWS; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        tmp_file.counters[MPIIO_MODE] = infile->counters[MPIIO_MODE];

+

+        /* sum */

+        for(j=MPIIO_BYTES_READ; j<=MPIIO_RW_SWITCHES; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        /* skip MPIIO_MAX_*_TIME_SIZE; handled in floating point section */

+

+        for(j=MPIIO_SIZE_READ_AGG_0_100; j<=MPIIO_SIZE_WRITE_AGG_1G_PLUS; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        /* first collapse any duplicates */

+        for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)

+        {

+            for(k=MPIIO_ACCESS1_ACCESS; k<=MPIIO_ACCESS4_ACCESS; k++)

+            {

+                if(infile->counters[j] == inoutfile->counters[k])

+                {

+                    infile->counters[j+4] += inoutfile->counters[k+4];

+                    inoutfile->counters[k] = 0;

+                    inoutfile->counters[k+4] = 0;

+                }

+            }

+        }

+

+        /* first set */

+        for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[MPIIO_ACCESS1_ACCESS]),

+                &(tmp_file.counters[MPIIO_ACCESS1_COUNT]), infile->counters[j],

+                infile->counters[j+4]);

+        }

+

+        /* second set */

+        for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[MPIIO_ACCESS1_ACCESS]),

+                &(tmp_file.counters[MPIIO_ACCESS1_COUNT]), inoutfile->counters[j],

+                inoutfile->counters[j+4]);

+        }

+

+        /* min non-zero (if available) value */

+        for(j=MPIIO_F_OPEN_TIMESTAMP; j<=MPIIO_F_WRITE_START_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = infile->fcounters[j];

+        }

+

+        /* max */

+        for(j=MPIIO_F_READ_END_TIMESTAMP; j<= MPIIO_F_CLOSE_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j])

+                tmp_file.fcounters[j] = infile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+        }

+

+        /* sum */

+        for(j=MPIIO_F_READ_TIME; j<=MPIIO_F_META_TIME; j++)

+        {

+            tmp_file.fcounters[j] = infile->fcounters[j] + inoutfile->fcounters[j];

+        }

+

+        /* max (special case) */

+        if(infile->fcounters[MPIIO_F_MAX_READ_TIME] >

+            inoutfile->fcounters[MPIIO_F_MAX_READ_TIME])

+        {

+            tmp_file.fcounters[MPIIO_F_MAX_READ_TIME] =

+                infile->fcounters[MPIIO_F_MAX_READ_TIME];

+            tmp_file.counters[MPIIO_MAX_READ_TIME_SIZE] =

+                infile->counters[MPIIO_MAX_READ_TIME_SIZE];

+        }

+        else

+        {

+            tmp_file.fcounters[MPIIO_F_MAX_READ_TIME] =

+                inoutfile->fcounters[MPIIO_F_MAX_READ_TIME];

+            tmp_file.counters[MPIIO_MAX_READ_TIME_SIZE] =

+                inoutfile->counters[MPIIO_MAX_READ_TIME_SIZE];

+        }

+

+        if(infile->fcounters[MPIIO_F_MAX_WRITE_TIME] >

+            inoutfile->fcounters[MPIIO_F_MAX_WRITE_TIME])

+        {

+            tmp_file.fcounters[MPIIO_F_MAX_WRITE_TIME] =

+                infile->fcounters[MPIIO_F_MAX_WRITE_TIME];

+            tmp_file.counters[MPIIO_MAX_WRITE_TIME_SIZE] =

+                infile->counters[MPIIO_MAX_WRITE_TIME_SIZE];

+        }

+        else

+        {

+            tmp_file.fcounters[MPIIO_F_MAX_WRITE_TIME] =

+                inoutfile->fcounters[MPIIO_F_MAX_WRITE_TIME];

+            tmp_file.counters[MPIIO_MAX_WRITE_TIME_SIZE] =

+                inoutfile->counters[MPIIO_MAX_WRITE_TIME_SIZE];

+        }

+

+        /* min (zeroes are ok here; some procs don't do I/O) */

+        if(infile->fcounters[MPIIO_F_FASTEST_RANK_TIME] <

+            inoutfile->fcounters[MPIIO_F_FASTEST_RANK_TIME])

+        {

+            tmp_file.counters[MPIIO_FASTEST_RANK] =

+                infile->counters[MPIIO_FASTEST_RANK];

+            tmp_file.counters[MPIIO_FASTEST_RANK_BYTES] =

+                infile->counters[MPIIO_FASTEST_RANK_BYTES];

+            tmp_file.fcounters[MPIIO_F_FASTEST_RANK_TIME] =

+                infile->fcounters[MPIIO_F_FASTEST_RANK_TIME];

+        }

+        else

+        {

+            tmp_file.counters[MPIIO_FASTEST_RANK] =

+                inoutfile->counters[MPIIO_FASTEST_RANK];

+            tmp_file.counters[MPIIO_FASTEST_RANK_BYTES] =

+                inoutfile->counters[MPIIO_FASTEST_RANK_BYTES];

+            tmp_file.fcounters[MPIIO_F_FASTEST_RANK_TIME] =

+                inoutfile->fcounters[MPIIO_F_FASTEST_RANK_TIME];

+        }

+

+        /* max */

+        if(infile->fcounters[MPIIO_F_SLOWEST_RANK_TIME] >

+           inoutfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME])

+        {

+            tmp_file.counters[MPIIO_SLOWEST_RANK] =

+                infile->counters[MPIIO_SLOWEST_RANK];

+            tmp_file.counters[MPIIO_SLOWEST_RANK_BYTES] =

+                infile->counters[MPIIO_SLOWEST_RANK_BYTES];

+            tmp_file.fcounters[MPIIO_F_SLOWEST_RANK_TIME] =

+                infile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];

+        }

+        else

+        {

+            tmp_file.counters[MPIIO_SLOWEST_RANK] =

+                inoutfile->counters[MPIIO_SLOWEST_RANK];

+            tmp_file.counters[MPIIO_SLOWEST_RANK_BYTES] =

+                inoutfile->counters[MPIIO_SLOWEST_RANK_BYTES];

+            tmp_file.fcounters[MPIIO_F_SLOWEST_RANK_TIME] =

+                inoutfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];

+        }

+

+        /* update pointers */

+        *inoutfile = tmp_file;

+        inoutfile++;

+        infile++;

+    }

+

+    return;

+}

+

+static void mpiio_shared_record_variance(MPI_Comm mod_comm,

+    struct darshan_mpiio_file *inrec_array, struct darshan_mpiio_file *outrec_array,

+    int shared_rec_count)

+{

+    MPI_Datatype var_dt;

+    MPI_Op var_op;

+    int i;

+    struct darshan_variance_dt *var_send_buf = NULL;

+    struct darshan_variance_dt *var_recv_buf = NULL;

+

+    DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_variance_dt),

+        MPI_BYTE, &var_dt);

+    DARSHAN_MPI_CALL(PMPI_Type_commit)(&var_dt);

+

+    DARSHAN_MPI_CALL(PMPI_Op_create)(darshan_variance_reduce, 1, &var_op);

+

+    var_send_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));

+    if(!var_send_buf)

+        return;

+

+    if(my_rank == 0)

+    {

+        var_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));

+

+        if(!var_recv_buf)

+            return;

+    }

+

+    /* get total i/o time variances for shared records */

+

+    for(i=0; i<shared_rec_count; i++)

+    {

+        var_send_buf[i].n = 1;

+        var_send_buf[i].S = 0;

+        var_send_buf[i].T = inrec_array[i].fcounters[MPIIO_F_READ_TIME] +

+                            inrec_array[i].fcounters[MPIIO_F_WRITE_TIME] +

+                            inrec_array[i].fcounters[MPIIO_F_META_TIME];

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,

+        var_dt, var_op, 0, mod_comm);

+

+    if(my_rank == 0)

+    {

+        for(i=0; i<shared_rec_count; i++)

+        {

+            outrec_array[i].fcounters[MPIIO_F_VARIANCE_RANK_TIME] =

+                (var_recv_buf[i].S / var_recv_buf[i].n);

+        }

+    }

+

+    /* get total bytes moved variances for shared records */

+

+    for(i=0; i<shared_rec_count; i++)

+    {

+        var_send_buf[i].n = 1;

+        var_send_buf[i].S = 0;

+        var_send_buf[i].T = (double)

+                            inrec_array[i].counters[MPIIO_BYTES_READ] +

+                            inrec_array[i].counters[MPIIO_BYTES_WRITTEN];

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,

+        var_dt, var_op, 0, mod_comm);

+

+    if(my_rank == 0)

+    {

+        for(i=0; i<shared_rec_count; i++)

+        {

+            outrec_array[i].fcounters[MPIIO_F_VARIANCE_RANK_BYTES] =

+                (var_recv_buf[i].S / var_recv_buf[i].n);

+        }

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Type_free)(&var_dt);

+    DARSHAN_MPI_CALL(PMPI_Op_free)(&var_op);

+    free(var_send_buf);

+    free(var_recv_buf);

+

+    return;

+}

+

+/* mpiio module shutdown benchmark routine */

+void darshan_mpiio_shutdown_bench_setup(int test_case)

+{

+    char filepath[256];

+    MPI_File *fh_array;

+    int64_t *size_array;

+    int i;

+    intptr_t j;

+

+    if(mpiio_runtime)

+        mpiio_shutdown();

+

+    mpiio_runtime_initialize();

+

+    srand(my_rank);

+    fh_array = malloc(1024 * sizeof(MPI_File));

+    size_array = malloc(DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT * sizeof(int64_t));

+    assert(fh_array && size_array);

+

+    for(j = 0; j < 1024; j++)

+        fh_array[j] = (MPI_File)j;

+    for(i = 0; i < DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT; i++)

+        size_array[i] = rand();

+

+    switch(test_case)

+    {

+        case 1: /* single file-per-process */

+            snprintf(filepath, 256, "fpp-0_rank-%d", my_rank);

+

+            MPIIO_RECORD_OPEN(MPI_SUCCESS, filepath, fh_array[0], MPI_COMM_SELF,

+                2, MPI_INFO_NULL, 0, 1);

+            MPIIO_RECORD_WRITE(MPI_SUCCESS, fh_array[0], size_array[0], MPI_BYTE,

+                MPIIO_INDEP_WRITES, 1, 2);

+

+            break;

+        case 2: /* single shared file */

+            snprintf(filepath, 256, "shared-0");

+

+            MPIIO_RECORD_OPEN(MPI_SUCCESS, filepath, fh_array[0], MPI_COMM_WORLD,

+                2, MPI_INFO_NULL, 0, 1);

+            MPIIO_RECORD_WRITE(MPI_SUCCESS, fh_array[0], size_array[0], MPI_BYTE,

+                MPIIO_COLL_WRITES, 1, 2);

+

+            break;

+        case 3: /* 1024 unique files per proc */

+            for(i = 0; i < 1024; i++)

+            {

+                snprintf(filepath, 256, "fpp-%d_rank-%d", i , my_rank);

+

+                MPIIO_RECORD_OPEN(MPI_SUCCESS, filepath, fh_array[i], MPI_COMM_SELF,

+                    2, MPI_INFO_NULL, 0, 1);

+                MPIIO_RECORD_WRITE(MPI_SUCCESS, fh_array[i],

+                    size_array[i % DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT],

+                    MPI_BYTE, MPIIO_INDEP_WRITES, 1, 2);

+            }

+

+            break;

+        case 4: /* 1024 shared files per proc */

+            for(i = 0; i < 1024; i++)

+            {

+                snprintf(filepath, 256, "shared-%d", i);

+

+                MPIIO_RECORD_OPEN(MPI_SUCCESS, filepath, fh_array[i], MPI_COMM_WORLD,

+                    2, MPI_INFO_NULL, 0, 1);

+                MPIIO_RECORD_WRITE(MPI_SUCCESS, fh_array[i],

+                    size_array[i % DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT],

+                    MPI_BYTE, MPIIO_COLL_WRITES, 1, 2);

+            }

+            break;

+        default:

+            fprintf(stderr, "Error: invalid Darshan benchmark test case.\n");

+            return;

+    }

+

+    return;

+}

+

+/**************************************************************************

+ * Functions exported by MPI-IO module for coordinating with darshan-core *

+ **************************************************************************/

+

+static void mpiio_begin_shutdown()

+{

+    assert(mpiio_runtime);

+

+    MPIIO_LOCK();

+    /* disable further instrumentation while Darshan shuts down */

+    instrumentation_disabled = 1;

+    MPIIO_UNLOCK();

+

+    return;

+}

+

+static void mpiio_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **mpiio_buf,

+    int *mpiio_buf_sz)

+{

+    struct mpiio_file_runtime *file;

+    struct mpiio_file_runtime* tmp;

+    int i;

+    double mpiio_time;

+    void *red_send_buf = NULL;

+    void *red_recv_buf = NULL;

+    MPI_Datatype red_type;

+    MPI_Op red_op;

+

+    assert(mpiio_runtime);

+

+    /* go through and set the 4 most common access sizes for MPI-IO */

+    for(i = 0; i < mpiio_runtime->file_array_ndx; i++)

+    {

+        tmp = &(mpiio_runtime->file_runtime_array[i]);

+

+        /* common access sizes */

+        darshan_walk_common_vals(tmp->access_root,

+            &(tmp->file_record->counters[MPIIO_ACCESS1_ACCESS]),

+            &(tmp->file_record->counters[MPIIO_ACCESS1_COUNT]));

+    }

+

+    /* if there are globally shared files, do a shared file reduction */

+    /* NOTE: the shared file reduction is also skipped if the 

+     * DARSHAN_DISABLE_SHARED_REDUCTION environment variable is set.

+     */

+    if(shared_rec_count && !getenv("DARSHAN_DISABLE_SHARED_REDUCTION"))

+    {

+        /* necessary initialization of shared records */

+        for(i = 0; i < shared_rec_count; i++)

+        {

+            HASH_FIND(hlink, mpiio_runtime->file_hash, &shared_recs[i],

+                sizeof(darshan_record_id), file);

+            assert(file);

+

+            mpiio_time =

+                file->file_record->fcounters[MPIIO_F_READ_TIME] +

+                file->file_record->fcounters[MPIIO_F_WRITE_TIME] +

+                file->file_record->fcounters[MPIIO_F_META_TIME];

+

+            /* initialize fastest/slowest info prior to the reduction */

+            file->file_record->counters[MPIIO_FASTEST_RANK] =

+                file->file_record->rank;

+            file->file_record->counters[MPIIO_FASTEST_RANK_BYTES] =

+                file->file_record->counters[MPIIO_BYTES_READ] +

+                file->file_record->counters[MPIIO_BYTES_WRITTEN];

+            file->file_record->fcounters[MPIIO_F_FASTEST_RANK_TIME] =

+                mpiio_time;

+

+            /* until reduction occurs, we assume that this rank is both

+             * the fastest and slowest. It is up to the reduction operator

+             * to find the true min and max.

+             */

+            file->file_record->counters[MPIIO_SLOWEST_RANK] =

+                file->file_record->counters[MPIIO_FASTEST_RANK];

+            file->file_record->counters[MPIIO_SLOWEST_RANK_BYTES] =

+                file->file_record->counters[MPIIO_FASTEST_RANK_BYTES];

+            file->file_record->fcounters[MPIIO_F_SLOWEST_RANK_TIME] =

+                file->file_record->fcounters[MPIIO_F_FASTEST_RANK_TIME];

+

+            file->file_record->rank = -1;

+        }

+

+        /* sort the array of files descending by rank so that we get all of the 

+         * shared files (marked by rank -1) in a contiguous portion at end 

+         * of the array

+         */

+        qsort(mpiio_runtime->file_record_array, mpiio_runtime->file_array_ndx,

+            sizeof(struct darshan_mpiio_file), mpiio_record_compare);

+

+        /* make *send_buf point to the shared files at the end of sorted array */

+        red_send_buf =

+            &(mpiio_runtime->file_record_array[mpiio_runtime->file_array_ndx-shared_rec_count]);

+

+        /* allocate memory for the reduction output on rank 0 */

+        if(my_rank == 0)

+        {

+            red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_mpiio_file));

+            if(!red_recv_buf)

+            {

+                return;

+            }

+        }

+

+        /* construct a datatype for a MPIIO file record.  This is serving no purpose

+         * except to make sure we can do a reduction on proper boundaries

+         */

+        DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_mpiio_file),

+            MPI_BYTE, &red_type);

+        DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);

+

+        /* register a MPIIO file record reduction operator */

+        DARSHAN_MPI_CALL(PMPI_Op_create)(mpiio_record_reduction_op, 1, &red_op);

+

+        /* reduce shared MPIIO file records */

+        DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,

+            shared_rec_count, red_type, red_op, 0, mod_comm);

+

+        /* get the time and byte variances for shared files */

+        mpiio_shared_record_variance(mod_comm, red_send_buf, red_recv_buf,

+            shared_rec_count);

+

+        /* clean up reduction state */

+        if(my_rank == 0)

+        {

+            int tmp_ndx = mpiio_runtime->file_array_ndx - shared_rec_count;

+            memcpy(&(mpiio_runtime->file_record_array[tmp_ndx]), red_recv_buf,

+                shared_rec_count * sizeof(struct darshan_mpiio_file));

+            free(red_recv_buf);

+        }

+        else

+        {

+            mpiio_runtime->file_array_ndx -= shared_rec_count;

+        }

+

+        DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);

+        DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);

+    }

+

+    *mpiio_buf = (void *)(mpiio_runtime->file_record_array);

+    *mpiio_buf_sz = mpiio_runtime->file_array_ndx * sizeof(struct darshan_mpiio_file);

+

+    return;

+}

+

+static void mpiio_shutdown()

+{

+    struct mpiio_file_runtime_ref *ref, *tmp;

+

+    assert(mpiio_runtime);

+

+    HASH_ITER(hlink, mpiio_runtime->fh_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, mpiio_runtime->fh_hash, ref);

+        free(ref);

+    }

+

+    HASH_CLEAR(hlink, mpiio_runtime->file_hash); /* these entries are freed all at once below */

+

+    free(mpiio_runtime->file_runtime_array);

+    free(mpiio_runtime->file_record_array);

+    free(mpiio_runtime);

+    mpiio_runtime = NULL;

+    instrumentation_disabled = 0;

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-31'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-31'>

<strong>

darshan-runtime/lib/darshan-null.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-null.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,394 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _XOPEN_SOURCE 500

+#define _GNU_SOURCE

+

+#include "darshan-runtime-config.h"

+#include <stdio.h>

+#include <unistd.h>

+#include <stdlib.h>

+#include <pthread.h>

+#include <assert.h>

+

+#include "uthash.h"

+#include "darshan.h"

+

+/* The "NULL" module is an example instrumentation module implementation provided

+ * with Darshan, primarily to indicate how arbitrary modules may be integrated

+ * into Darshan. In particular, this module demonstrates how to develop wrapper

+ * functions for intercepting functions of interest, how to best manage necessary

+ * runtime data structures, and how to coordinate with the darshan-core component,

+ * among other things. This module is not linked with the darshan-runtime library; 

+ * it is intended mainly to serve as a basic stubbed out module implementation

+ * that may be reused and expanded on by developers adding new instrumentation modules.

+ */

+

+/* The DARSHAN_FORWARD_DECL macro (defined in darshan.h) is used to provide forward

+ * declarations for wrapped funcions, regardless if Darshan is used with statically

+ * or dynamically linked executables.

+ */

+DARSHAN_FORWARD_DECL(foo, int, (const char *name, int arg1, int arg2));

+

+/* The null_record_runtime structure maintains necessary runtime metadata

+ * for a "NULL" module data record (darshan_null_record structure, defined

+ * in darshan-null-log-format.h). This metadata assists with the instrumenting

+ * of specific statistics in the file record.

+ *

+ * RATIONALE: In general, a module may need to track some stateful, volatile 

+ * information regarding specific I/O statistics to aid in the instrumentation

+ * process. However, this information should not be stored in the darshan_null_record

+ * struct because we don't want it to appear in the final darshan log file.

+ * We therefore associate a null_record_runtime structure with each darshan_null_record

+ * structure in order to track this information.

+ *

+ * NOTE: The null_record_runtime struct contains a pointer to a darshan_null_record

+ * struct (see the *record_p member) rather than simply embedding an entire

+ * darshan_null_record struct.  This is done so that all of the darshan_null_record

+ * structs can be kept contiguous in memory as a single array to simplify

+ * reduction, compression, and storage.

+ */

+struct null_record_runtime

+{

+    /* Darshan record for the "NULL" example module */

+    struct darshan_null_record* record_p;

+

+    /* ... other runtime data ... */

+

+    /* hash table link for this record */

+    /* NOTE: it is entirely up to the module developer how to persist module

+     * records in memory as the instrumented application runs. These records

+     * could just as easily be stored in an array or linked list. That said,

+     * the data structure selection should be mindful of the resulting memory

+     * footprint and search time complexity to attempt minimize Darshan overheads.

+     * hash table and linked list implementations are available in uthash.h and

+     * utlist.h, respectively.

+     */

+    UT_hash_handle hlink;

+};

+

+/* The null_runtime structure simply encapsulates global data structures needed

+ * by the module for instrumenting functions of interest and providing the output

+ * I/O data for this module to the darshan-core component at shutdown time.

+ */

+struct null_runtime

+{

+    /* runtime_record_array is the array of runtime records for the "NULL" module. */

+    struct null_record_runtime* runtime_record_array;

+    /* record_array is the array of high-level Darshan records for the "NULL" module,

+     * each corresponding to the the runtime record structure stored at the same array

+     * index in runtime_record_array.

+     */

+    struct darshan_null_record* record_array;

+    /* file_array_size is the maximum amount of records that can be stored in 

+     * record_array (and consequentially, runtime_record_array).

+     */

+    int rec_array_size;

+    /* file_array_ndx is the current index into both runtime_record_array and

+     * record_array.

+     */

+    int rec_array_ndx;

+    /* record_hash is a pointer to a hash table of null_record_runtime structures

+     * currently maintained by the "NULL" module.

+     */

+    struct null_record_runtime* record_hash;

+};

+

+/* null_runtime is the global data structure encapsulating "NULL" module state */

+static struct null_runtime *null_runtime = NULL;

+/* The null_runtime_mutex is a lock used when updating the null_runtime global

+ * structure (or any other global data structures). This is necessary to avoid race

+ * conditions as multiple threads execute function wrappers and update module state.

+ * NOTE: Recursive mutexes are used in case functions wrapped by this module call

+ * other wrapped functions that would result in deadlock, otherwise. This mechanism

+ * may not be necessary for all instrumentation modules.

+ */

+static pthread_mutex_t null_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+/* the instrumentation_disabled flag is used to toggle wrapper functions on/off */

+static int instrumentation_disabled = 0;

+/* my_rank indicates the MPI rank of this process */

+static int my_rank = -1;

+

+/* internal helper functions for the "NULL" module */

+static void null_runtime_initialize(void);

+static struct null_record_runtime* null_record_by_name(const char *name);

+

+/* forward declaration for module functions needed to interface with darshan-core */

+static void null_begin_shutdown(void);

+static void null_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,

+    int shared_rec_count, void **null_buf, int *null_buf_sz);

+static void null_shutdown(void);

+

+/* macros for obtaining/releasing the "NULL" module lock */

+#define NULL_LOCK() pthread_mutex_lock(&null_runtime_mutex)

+#define NULL_UNLOCK() pthread_mutex_unlock(&null_runtime_mutex)

+

+/* macro for instrumenting the "NULL" module's foo function */

+/* NOTE: this macro makes use of the DARSHAN_COUNTER_* macros defined

+ * and documented in darshan.h.

+ */

+#define NULL_RECORD_FOO(__ret, __name, __dat, __tm1, __tm2) do{ \

+    struct null_record_runtime* rec; \

+    double elapsed = __tm2 - __tm1; \

+    /* if foo returns error (return code < 0), don't instrument anything */ \

+    if(__ret < 0) break; \

+    /* use '__name' to lookup a corresponding "NULL" record */ \

+    rec = null_record_by_name(__name); \

+    if(!rec) break; \

+    /* increment counter indicating number of calls to 'bar' */ \

+    rec->record_p->counters[NULL_BARS] += 1; \

+    /* store data value for most recent call to 'bar' */ \

+    rec->record_p->counters[NULL_BAR_DAT] = __dat; \

+    /* store timestamp of most recent call to 'bar' */ \

+    rec->record_p->fcounters[NULL_F_BAR_TIMESTAMP] = __tm1; \

+    /* store duration of most recent call to 'bar' */ \

+    rec->record_p->fcounters[NULL_F_BAR_DURATION] = elapsed; \

+} while(0)

+

+/**********************************************************

+ *    Wrappers for "NULL" module functions of interest    * 

+ **********************************************************/

+

+/* The DARSHAN_DECL macro provides the appropriate wrapper function names,

+ * depending on whether the Darshan library is statically or dynamically linked.

+ */

+int DARSHAN_DECL(foo)(const char* name, int arg1, int arg2)

+{

+    ssize_t ret;

+    double tm1, tm2;

+

+    /* The MAP_OR_FAIL macro attempts to obtain the address of the actual

+     * underlying foo function call (__real_foo), in the case of LD_PRELOADing

+     * the Darshan library. For statically linked executables, this macro is

+     * just a NOP. 

+     */

+    MAP_OR_FAIL(foo);

+

+    /* In general, Darshan wrappers begin by calling the real version of the

+     * given wrapper function. Timers are used to record the duration of this

+     * operation. */

+    tm1 = darshan_core_wtime();

+    ret = __real_foo(name, arg1, arg2);

+    tm2 = darshan_core_wtime();

+

+    NULL_LOCK();

+

+    /* Before attempting to instrument I/O statistics for function foo, make

+     * sure the "NULL" module runtime environment has been initialized. 

+     * NOTE: this runtime environment is initialized only once -- if the

+     * appropriate structures have already been initialized, this function simply

+     * returns.

+     */

+    null_runtime_initialize();

+

+    /* Call macro for instrumenting data for foo function calls. */

+    NULL_RECORD_FOO(ret, name, arg1+arg2, tm1, tm2);

+

+    NULL_UNLOCK();

+

+    return(ret);

+}

+

+/**********************************************************

+ * Internal functions for manipulating POSIX module state *

+ **********************************************************/

+

+/* Initialize internal POSIX module data structures and register with darshan-core. */

+static void null_runtime_initialize()

+{

+    /* struct of function pointers for interfacing with darshan-core */

+    struct darshan_module_funcs null_mod_fns =

+    {

+        .begin_shutdown = &null_begin_shutdown,

+        .get_output_data = &null_get_output_data,

+        .shutdown = &null_shutdown

+    };

+    int mem_limit; /* max. memory this module can consume, dictated by darshan-core */

+

+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(null_runtime || instrumentation_disabled)

+        return;

+

+    /* register the "NULL" module with the darshan-core component */

+    darshan_core_register_module(

+        DARSHAN_NULL_MOD,   /* Darshan module identifier, defined in darshan-log-format.h */

+        &null_mod_fns,

+        &my_rank,

+        &mem_limit,

+        NULL);

+

+    /* return if no memory assigned by darshan-core */

+    if(mem_limit == 0)

+        return;

+

+    /* initialize module's global state */

+    null_runtime = malloc(sizeof(*null_runtime));

+    if(!null_runtime)

+        return;

+    memset(null_runtime, 0, sizeof(*null_runtime));

+

+    /* Set the maximum number of data records this module may track, as indicated

+     * by mem_limit (set by darshan-core).

+     * NOTE: We interpret the maximum memory limit to be related to the maximum

+     * amount of data which may be written to log by a single process for a given

+     * module. We therefore use this maximum memory limit to determine how many

+     * darshan_null_record structures we can track per process.

+     */

+    null_runtime->rec_array_size = mem_limit / sizeof(struct darshan_null_record);

+    null_runtime->rec_array_ndx = 0;

+

+    /* allocate both record arrays (runtime and high-level records) */

+    null_runtime->runtime_record_array = malloc(null_runtime->rec_array_size *

+                                                sizeof(struct null_record_runtime));

+    null_runtime->record_array = malloc(null_runtime->rec_array_size *

+                                        sizeof(struct darshan_null_record));

+    if(!null_runtime->runtime_record_array || !null_runtime->record_array)

+    {

+        null_runtime->rec_array_size = 0;

+        return;

+    }

+    memset(null_runtime->runtime_record_array, 0, null_runtime->rec_array_size *

+           sizeof(struct null_record_runtime));

+    memset(null_runtime->record_array, 0, null_runtime->rec_array_size *

+           sizeof(struct darshan_null_record));

+

+    return;

+}

+

+/* Search for and return a "NULL" module record corresponding to name parameter. */

+static struct null_record_runtime* null_record_by_name(const char *name)

+{

+    struct null_record_runtime *rec = NULL;

+    darshan_record_id rec_id;

+    int limit_flag;

+

+    /* Don't search for a record if the "NULL" module is not initialized or

+     * if instrumentation has been toggled off.

+     */

+    if(!null_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* stop tracking new records if we are tracking our maximum count */

+    limit_flag = (null_runtime->rec_array_ndx >= null_runtime->rec_array_size);

+

+    /* get a unique record identifier for this record from darshan-core */

+    darshan_core_register_record(

+        (void*)name,

+        strlen(name),

+        DARSHAN_NULL_MOD,

+        1,

+        limit_flag,

+        &rec_id,

+        NULL);

+

+    /* the file record id is set to 0 if no memory is available for tracking

+     * new records -- just fall through and ignore this record

+     */

+    if(rec_id == 0)

+    {

+        return(NULL);

+    }

+

+    /* search the hash table for this file record, and return if found */

+    HASH_FIND(hlink, null_runtime->record_hash, &rec_id, sizeof(darshan_record_id), rec);

+    if(rec)

+    {

+        return(rec);

+    }

+

+    /* no existing record, assign a new one from the global array */

+    rec = &(null_runtime->runtime_record_array[null_runtime->rec_array_ndx]);

+    rec->record_p = &(null_runtime->record_array[null_runtime->rec_array_ndx]);

+

+    /* set the darshan record id and corresponding process rank for this record */

+    rec->record_p->f_id = rec_id;

+    rec->record_p->rank = my_rank;

+

+    /* add new record to file hash table */

+    HASH_ADD(hlink, null_runtime->record_hash, record_p->f_id, sizeof(darshan_record_id), rec);

+    null_runtime->rec_array_ndx++;

+

+    return(rec);

+}

+

+/******************************************************************************

+ * Functions exported by the "NULL" module for coordinating with darshan-core *

+ ******************************************************************************/

+

+/* Perform any necessary steps prior to shutting down for the "NULL" module. */

+static void null_begin_shutdown()

+{

+    assert(null_runtime);

+

+    NULL_LOCK();

+

+    /* In general, we want to disable all wrappers while Darshan shuts down. 

+     * This is to avoid race conditions and ensure data consistency, as

+     * executing wrappers could potentially modify module state while Darshan

+     * is in the process of shutting down. 

+     */

+    instrumentation_disabled = 1;

+

+    /* ... any other code which needs to be executed before beginning shutdown process ... */

+

+    NULL_UNLOCK();

+

+    return;

+}

+

+/* Pass output data for the "NULL" module back to darshan-core to log to file. */

+static void null_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **null_buf,

+    int *null_buf_sz)

+{

+    assert(null_runtime);

+

+    /* NOTE: this function can be used to run collective operations prior to

+     * shutting down the module, as implied by the MPI communicator passed in

+     * as the first agrument. Typically, module developers will want to run a

+     * reduction on shared data records (passed in in the 'shared_recs' array),

+     * but other collective routines can be run here as well. For a detailed

+     * example illustrating how to run shared file reductions, consider the

+     * POSIX or MPIIO instrumentation modules, as they both implement this

+     * functionality.

+     */

+

+    /* Just set the output buffer to point at the array of the "NULL" module's

+     * I/O records, and set the output size according to the number of records

+     * currently being tracked.

+     */

+    *null_buf = (void *)(null_runtime->record_array);

+    *null_buf_sz = null_runtime->rec_array_ndx * sizeof(struct darshan_null_record);

+

+    return;

+}

+

+/* Shutdown the "NULL" module by freeing up all data structures. */

+static void null_shutdown()

+{

+    assert(null_runtime);

+

+    HASH_CLEAR(hlink, null_runtime->record_hash); /* these hash entries are freed all at once below */

+

+    free(null_runtime->runtime_record_array);

+    free(null_runtime->record_array);

+    free(null_runtime);

+    null_runtime = NULL;

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-32'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-32'>

<strong>

darshan-runtime/lib/darshan-pnetcdf-stubs.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-pnetcdf-stubs.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-pnetcdf-stubs.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 /* This file contains stubs for the ncmpi functions intercepted by Darshan.

</code></pre>

<br>

</li>

<li id='diff-33'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-33'>

<strong>

darshan-runtime/lib/darshan-pnetcdf.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-pnetcdf.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-pnetcdf.c

</span><span style="color: #aaaaaa">@@ -1,70 +1,102 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #include "darshan-runtime-config.h"

 #include <stdio.h>

<span style="color: #000000;background-color: #ffdddd">-#include <pthread.h>

</span><span style="color: #000000;background-color: #ddffdd">+#include <unistd.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <stdarg.h>

</span> #include <string.h>

<span style="color: #000000;background-color: #ffdddd">-#include "mpi.h"

-#include "darshan.h"

-

-#ifdef DARSHAN_PRELOAD

-#define __USE_GNU

-#include <dlfcn.h>

</span><span style="color: #000000;background-color: #ddffdd">+#include <time.h>

</span> #include <stdlib.h>

<span style="color: #000000;background-color: #ddffdd">+#include <errno.h>

+#include <search.h>

+#include <assert.h>

+#define __USE_GNU

+#include <pthread.h>

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  ret (*__real_ ## name)args = NULL;

-         

-#define DARSHAN_DECL(__name) __name

-

-#define MAP_OR_FAIL(func) \

-    if (!(__real_ ## func)) \

-    { \

-        __real_ ## func = dlsym(RTLD_NEXT, #func); \

-        if(!(__real_ ## func)) { \

-            fprintf(stderr, "Darshan failed to map symbol: %s\n", #func); \

-            exit(1); \

-        } \

-    }

- 

-#else   

-    

-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  extern ret __real_ ## name args;

-

-#define DARSHAN_DECL(__name) __wrap_ ## __name

-

-#define MAP_OR_FAIL(func)

</span><span style="color: #000000;background-color: #ddffdd">+#include "uthash.h"

</span> 

<span style="color: #000000;background-color: #ffdddd">-#endif

</span><span style="color: #000000;background-color: #ddffdd">+#include "darshan.h"

+#include "darshan-dynamic.h"

</span> 

 DARSHAN_FORWARD_DECL(ncmpi_create, int, (MPI_Comm comm, const char *path, int cmode, MPI_Info info, int *ncidp));

 DARSHAN_FORWARD_DECL(ncmpi_open, int, (MPI_Comm comm, const char *path, int omode, MPI_Info info, int *ncidp));

 DARSHAN_FORWARD_DECL(ncmpi_close, int, (int ncid));

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_ncid(int ncid);

-static void darshan_file_close_ncid(int ncid);

-static struct darshan_file_runtime* darshan_file_by_name_setncid(const char* name, int ncid);

</span><span style="color: #000000;background-color: #ddffdd">+/* structure to track i/o stats for a given PNETCDF file at runtime */

+struct pnetcdf_file_runtime

+{

+    struct darshan_pnetcdf_file* file_record;

+    UT_hash_handle hlink;

+};

+

+/* structure to associate a PNETCDF ncid with an existing file runtime structure */

+struct pnetcdf_file_runtime_ref

+{

+    struct pnetcdf_file_runtime* file;

+    int ncid;

+    UT_hash_handle hlink;

+};

</span> 

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(ncmpi_create)(MPI_Comm comm, const char *path, 

</span><span style="color: #000000;background-color: #ddffdd">+/* necessary state for storing PNETCDF file records and coordinating with

+ * darshan-core at shutdown time

+ */

+struct pnetcdf_runtime

+{

+    struct pnetcdf_file_runtime* file_runtime_array;

+    struct darshan_pnetcdf_file* file_record_array;

+    int file_array_size;

+    int file_array_ndx;

+    struct pnetcdf_file_runtime *file_hash;

+    struct pnetcdf_file_runtime_ref* ncid_hash;

+};

+

+static struct pnetcdf_runtime *pnetcdf_runtime = NULL;

+static pthread_mutex_t pnetcdf_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+static int instrumentation_disabled = 0;

+static int my_rank = -1;

+

+static void pnetcdf_runtime_initialize(void);

+static struct pnetcdf_file_runtime* pnetcdf_file_by_name(const char *name);

+static struct pnetcdf_file_runtime* pnetcdf_file_by_name_setncid(const char* name, int ncid);

+static struct pnetcdf_file_runtime* pnetcdf_file_by_ncid(int ncid);

+static void pnetcdf_file_close_ncid(int ncid);

+static int pnetcdf_record_compare(const void* a, const void* b);

+static void pnetcdf_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype);

+

+static void pnetcdf_begin_shutdown(void);

+static void pnetcdf_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,

+    int shared_rec_count, void **pnetcdf_buf, int *pnetcdf_buf_sz);

+static void pnetcdf_shutdown(void);

+

+#define PNETCDF_LOCK() pthread_mutex_lock(&pnetcdf_runtime_mutex)

+#define PNETCDF_UNLOCK() pthread_mutex_unlock(&pnetcdf_runtime_mutex)

+

+/*********************************************************

+ *      Wrappers for PNETCDF functions of interest       * 

+ *********************************************************/

+

+int DARSHAN_DECL(ncmpi_create)(MPI_Comm comm, const char *path,

</span>     int cmode, MPI_Info info, int *ncidp)

 {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct pnetcdf_file_runtime* file;

</span>     char* tmp;

     int comm_size;

     double tm1;

     MAP_OR_FAIL(ncmpi_create);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_ncmpi_create(comm, path, cmode, info, ncidp);

     if(ret == 0)

<span style="color: #000000;background-color: #ffdddd">-    {  

-        CP_LOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    {

</span>         /* use ROMIO approach to strip prefix if present */

         /* strip off prefix if there is one, but only skip prefixes

          * if they are greater than length one to allow for windows

<span style="color: #aaaaaa">@@ -75,44 +107,44 @@ int DARSHAN_DECL(ncmpi_create)(MPI_Comm comm, const char *path,

</span>             path = tmp + 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        file = darshan_file_by_name_setncid(path, (*ncidp));

</span><span style="color: #000000;background-color: #ddffdd">+        PNETCDF_LOCK();

+        pnetcdf_runtime_initialize();

+        file = pnetcdf_file_by_name_setncid(path, (*ncidp));

</span>         if(file)

         {

<span style="color: #000000;background-color: #ffdddd">-            if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0)

-                CP_F_SET(file, CP_F_OPEN_TIMESTAMP,

-                tm1);

-            PMPI_Comm_size(comm, &comm_size);

</span><span style="color: #000000;background-color: #ddffdd">+            if(file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] == 0)

+                file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] = tm1;

+            DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);

</span>             if(comm_size == 1)

             {

<span style="color: #000000;background-color: #ffdddd">-                CP_INC(file, CP_INDEP_NC_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+                file->file_record->counters[PNETCDF_INDEP_OPENS] += 1;

</span>             }

             else

             {

<span style="color: #000000;background-color: #ffdddd">-                CP_INC(file, CP_COLL_NC_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+                file->file_record->counters[PNETCDF_COLL_OPENS] += 1;

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        PNETCDF_UNLOCK();

</span>     }

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(ncmpi_open)(MPI_Comm comm, const char *path, 

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(ncmpi_open)(MPI_Comm comm, const char *path,

</span>     int omode, MPI_Info info, int *ncidp)

 {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct pnetcdf_file_runtime* file;

</span>     char* tmp;

     int comm_size;

     double tm1;

     MAP_OR_FAIL(ncmpi_open);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_ncmpi_open(comm, path, omode, info, ncidp);

     if(ret == 0)

<span style="color: #000000;background-color: #ffdddd">-    {  

-        CP_LOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    {

</span>         /* use ROMIO approach to strip prefix if present */

         /* strip off prefix if there is one, but only skip prefixes

          * if they are greater than length one to allow for windows

<span style="color: #aaaaaa">@@ -123,74 +155,442 @@ int DARSHAN_DECL(ncmpi_open)(MPI_Comm comm, const char *path,

</span>             path = tmp + 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        file = darshan_file_by_name_setncid(path, (*ncidp));

</span><span style="color: #000000;background-color: #ddffdd">+        PNETCDF_LOCK();

+        pnetcdf_runtime_initialize();

+        file = pnetcdf_file_by_name_setncid(path, (*ncidp));

</span>         if(file)

         {

<span style="color: #000000;background-color: #ffdddd">-            if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0)

-                CP_F_SET(file, CP_F_OPEN_TIMESTAMP,

-                tm1);

-            PMPI_Comm_size(comm, &comm_size);

</span><span style="color: #000000;background-color: #ddffdd">+            if(file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] == 0)

+                file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] = tm1;

+            DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);

</span>             if(comm_size == 1)

             {

<span style="color: #000000;background-color: #ffdddd">-                CP_INC(file, CP_INDEP_NC_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+                file->file_record->counters[PNETCDF_INDEP_OPENS] += 1;

</span>             }

             else

             {

<span style="color: #000000;background-color: #ffdddd">-                CP_INC(file, CP_COLL_NC_OPENS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+                file->file_record->counters[PNETCDF_COLL_OPENS] += 1;

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        PNETCDF_UNLOCK();

</span>     }

     return(ret);

<span style="color: #000000;background-color: #ffdddd">-

</span> }

 int DARSHAN_DECL(ncmpi_close)(int ncid)

 {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct pnetcdf_file_runtime* file;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(ncmpi_close); 

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(ncmpi_close);

</span> 

     ret = __real_ncmpi_close(ncid);

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_ncid(ncid);

</span><span style="color: #000000;background-color: #ddffdd">+    PNETCDF_LOCK();

+    pnetcdf_runtime_initialize();

+    file = pnetcdf_file_by_ncid(ncid);

</span>     if(file)

     {

<span style="color: #000000;background-color: #ffdddd">-        CP_F_SET(file, CP_F_CLOSE_TIMESTAMP, PMPI_Wtime());

-        darshan_file_close_ncid(ncid);

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->fcounters[PNETCDF_F_CLOSE_TIMESTAMP] =

+            darshan_core_wtime();

+        pnetcdf_file_close_ncid(ncid);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    PNETCDF_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_name_setncid(const char* name, int ncid)

</span><span style="color: #000000;background-color: #ddffdd">+/************************************************************

+ * Internal functions for manipulating PNETCDF module state *

+ ************************************************************/

+

+/* initialize internal PNETCDF module data strucutres and register with darshan-core */

+static void pnetcdf_runtime_initialize()

+{

+    int mem_limit;

+    struct darshan_module_funcs pnetcdf_mod_fns =

+    {

+        .begin_shutdown = &pnetcdf_begin_shutdown,

+        .get_output_data = &pnetcdf_get_output_data,

+        .shutdown = &pnetcdf_shutdown

+    };

+

+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(pnetcdf_runtime || instrumentation_disabled)

+        return;

+

+    /* register pnetcdf module with darshan-core */

+    darshan_core_register_module(

+        DARSHAN_PNETCDF_MOD,

+        &pnetcdf_mod_fns,

+        &my_rank,

+        &mem_limit,

+        NULL);

+

+    /* return if no memory assigned by darshan-core */

+    if(mem_limit == 0)

+        return;

+

+    pnetcdf_runtime = malloc(sizeof(*pnetcdf_runtime));

+    if(!pnetcdf_runtime)

+        return;

+    memset(pnetcdf_runtime, 0, sizeof(*pnetcdf_runtime));

+

+    /* set maximum number of file records according to max memory limit */

+    /* NOTE: maximum number of records is based on the size of a pnetcdf file record */

+    /* TODO: should we base memory usage off file record or total runtime structure sizes? */

+    pnetcdf_runtime->file_array_size = mem_limit / sizeof(struct darshan_pnetcdf_file);

+    pnetcdf_runtime->file_array_ndx = 0;

+

+    /* allocate array of runtime file records */

+    pnetcdf_runtime->file_runtime_array = malloc(pnetcdf_runtime->file_array_size *

+                                                 sizeof(struct pnetcdf_file_runtime));

+    pnetcdf_runtime->file_record_array = malloc(pnetcdf_runtime->file_array_size *

+                                                sizeof(struct darshan_pnetcdf_file));

+    if(!pnetcdf_runtime->file_runtime_array || !pnetcdf_runtime->file_record_array)

+    {

+        pnetcdf_runtime->file_array_size = 0;

+        return;

+    }

+    memset(pnetcdf_runtime->file_runtime_array, 0, pnetcdf_runtime->file_array_size *

+           sizeof(struct pnetcdf_file_runtime));

+    memset(pnetcdf_runtime->file_record_array, 0, pnetcdf_runtime->file_array_size *

+           sizeof(struct darshan_pnetcdf_file));

+

+    return;

+}

+

+/* get a PNETCDF file record for the given file path */

+static struct pnetcdf_file_runtime* pnetcdf_file_by_name(const char *name)

+{

+    struct pnetcdf_file_runtime *file = NULL;

+    char *newname = NULL;

+    darshan_record_id file_id;

+    int limit_flag;

+

+    if(!pnetcdf_runtime || instrumentation_disabled)

+        return(NULL);

+

+    newname = darshan_clean_file_path(name);

+    if(!newname)

+        newname = (char*)name;

+

+    limit_flag = (pnetcdf_runtime->file_array_ndx >= pnetcdf_runtime->file_array_size);

+

+    /* get a unique id for this file from darshan core */

+    darshan_core_register_record(

+        (void*)newname,

+        strlen(newname),

+        DARSHAN_PNETCDF_MOD,

+        1,

+        limit_flag,

+        &file_id,

+        NULL);

+

+    /* the file record id is set to 0 if no memory is available for tracking

+     * new records -- just fall through and ignore this record

+     */

+    if(file_id == 0)

+    {

+        if(newname != name)

+            free(newname);

+        return(NULL);

+    }

+

+    /* search the hash table for this file record, and return if found */

+    HASH_FIND(hlink, pnetcdf_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);

+    if(file)

+    {

+        if(newname != name)

+            free(newname);

+        return(file);

+    }

+

+    /* no existing record, assign a new file record from the global array */

+    file = &(pnetcdf_runtime->file_runtime_array[pnetcdf_runtime->file_array_ndx]);

+    file->file_record = &(pnetcdf_runtime->file_record_array[pnetcdf_runtime->file_array_ndx]);

+    file->file_record->f_id = file_id;

+    file->file_record->rank = my_rank;

+

+    /* add new record to file hash table */

+    HASH_ADD(hlink, pnetcdf_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);

+    pnetcdf_runtime->file_array_ndx++;

+

+    if(newname != name)

+        free(newname);

+    return(file);

+}

+

+/* get a PNETCDF file record for the given file path, and also create a

+ * reference structure using the returned ncid

+ */

+static struct pnetcdf_file_runtime* pnetcdf_file_by_name_setncid(const char* name, int ncid)

+{

+    struct pnetcdf_file_runtime* file;

+    struct pnetcdf_file_runtime_ref* ref;

+

+    if(!pnetcdf_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* find file record by name first */

+    file = pnetcdf_file_by_name(name);

+

+    if(!file)

+        return(NULL);

+

+    /* search hash table for existing file ref for this ncid */

+    HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);

+    if(ref)

+    {

+        /* we have a reference.  Make sure it points to the correct file

+         * and return it

+         */

+        ref->file = file;

+        return(file);

+    }

+

+    /* if we hit this point, then we don't have a reference for this ncid

+     * in the table yet.  Add it.

+     */

+    ref = malloc(sizeof(*ref));

+    if(!ref)

+        return(NULL);

+    memset(ref, 0, sizeof(*ref));

+

+    ref->file = file;

+    ref->ncid = ncid;

+    HASH_ADD(hlink, pnetcdf_runtime->ncid_hash, ncid, sizeof(int), ref);

+

+    return(file);

+}

+

+/* get a PNETCDF file record for the given ncid */

+static struct pnetcdf_file_runtime* pnetcdf_file_by_ncid(int ncid)

+{

+    struct pnetcdf_file_runtime_ref* ref;

+

+    if(!pnetcdf_runtime || instrumentation_disabled)

+        return(NULL);

+

+    /* search hash table for existing file ref for this ncid */

+    HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);

+    if(ref)

+        return(ref->file);

+

+    return(NULL);

+}

+

+/* free up PNETCDF reference data structures for the given ncid */

+static void pnetcdf_file_close_ncid(int ncid)

+{

+    struct pnetcdf_file_runtime_ref* ref;

+

+    if(!pnetcdf_runtime || instrumentation_disabled)

+        return;

+

+    /* search hash table for this ncid */

+    HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);

+    if(ref)

+    {

+        /* we have a reference, delete it */

+        HASH_DELETE(hlink, pnetcdf_runtime->ncid_hash, ref);

+        free(ref);

+    }

+

+    return;

+}

+

+/* compare function for sorting file records by descending rank */

+static int pnetcdf_record_compare(const void* a_p, const void* b_p)

+{

+    const struct darshan_pnetcdf_file* a = a_p;

+    const struct darshan_pnetcdf_file* b = b_p;

+

+    if(a->rank < b->rank)

+        return 1;

+    if(a->rank > b->rank)

+        return -1;

+

+    return 0;

+}

+

+static void pnetcdf_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_pnetcdf_file tmp_file;

+    struct darshan_pnetcdf_file *infile = infile_v;

+    struct darshan_pnetcdf_file *inoutfile = inoutfile_v;

+    int i, j;

+

+    assert(pnetcdf_runtime);

+

+    for(i=0; i<*len; i++)

+    {

+        memset(&tmp_file, 0, sizeof(struct darshan_pnetcdf_file));

+        tmp_file.f_id = infile->f_id;

+        tmp_file.rank = -1;

+

+        /* sum */

+        for(j=PNETCDF_INDEP_OPENS; j<=PNETCDF_COLL_OPENS; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_name_sethandle(name, &ncid, sizeof(ncid), DARSHAN_NCID);

-    return(tmp_file);

</span><span style="color: #000000;background-color: #ddffdd">+        /* min non-zero (if available) value */

+        for(j=PNETCDF_F_OPEN_TIMESTAMP; j<=PNETCDF_F_OPEN_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = infile->fcounters[j];

+        }

+

+        /* max */

+        for(j=PNETCDF_F_CLOSE_TIMESTAMP; j<=PNETCDF_F_CLOSE_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j])

+                tmp_file.fcounters[j] = infile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+        }

+

+        /* update pointers */

+        *inoutfile = tmp_file;

+        inoutfile++;

+        infile++;

+    }

+

+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-static void darshan_file_close_ncid(int ncid)

</span><span style="color: #000000;background-color: #ddffdd">+/***************************************************************************

+ * Functions exported by PNETCDF module for coordinating with darshan-core *

+ ***************************************************************************/

+

+static void pnetcdf_begin_shutdown()

</span> {

<span style="color: #000000;background-color: #ffdddd">-    darshan_file_closehandle(&ncid, sizeof(ncid), DARSHAN_NCID);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(pnetcdf_runtime);

+

+    PNETCDF_LOCK();

+    /* disable further instrumentation while Darshan shuts down */

+    instrumentation_disabled = 1;

+    PNETCDF_UNLOCK();

+

</span>     return;

 }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_ncid(int ncid)

</span><span style="color: #000000;background-color: #ddffdd">+static void pnetcdf_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **pnetcdf_buf,

+    int *pnetcdf_buf_sz)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct pnetcdf_file_runtime *file;

+    int i;

+    struct darshan_pnetcdf_file *red_send_buf = NULL;

+    struct darshan_pnetcdf_file *red_recv_buf = NULL;

+    MPI_Datatype red_type;

+    MPI_Op red_op;

+

+    assert(pnetcdf_runtime);

+

+    /* if there are globally shared files, do a shared file reduction */

+    /* NOTE: the shared file reduction is also skipped if the 

+     * DARSHAN_DISABLE_SHARED_REDUCTION environment variable is set.

+     */

+    if(shared_rec_count && !getenv("DARSHAN_DISABLE_SHARED_REDUCTION"))

+    {

+        /* necessary initialization of shared records */

+        for(i = 0; i < shared_rec_count; i++)

+        {

+            HASH_FIND(hlink, pnetcdf_runtime->file_hash, &shared_recs[i],

+                sizeof(darshan_record_id), file);

+            assert(file);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_handle(&ncid, sizeof(ncid), DARSHAN_NCID);

-    

-    return(tmp_file);

</span><span style="color: #000000;background-color: #ddffdd">+            file->file_record->rank = -1;

+        }

+

+        /* sort the array of files descending by rank so that we get all of the 

+         * shared files (marked by rank -1) in a contiguous portion at end 

+         * of the array

+         */

+        qsort(pnetcdf_runtime->file_record_array, pnetcdf_runtime->file_array_ndx,

+            sizeof(struct darshan_pnetcdf_file), pnetcdf_record_compare);

+

+        /* make *send_buf point to the shared files at the end of sorted array */

+        red_send_buf =

+            &(pnetcdf_runtime->file_record_array[pnetcdf_runtime->file_array_ndx-shared_rec_count]);

+

+        /* allocate memory for the reduction output on rank 0 */

+        if(my_rank == 0)

+        {

+            red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_pnetcdf_file));

+            if(!red_recv_buf)

+            {

+                return;

+            }

+        }

+

+        /* construct a datatype for a PNETCDF file record.  This is serving no purpose

+         * except to make sure we can do a reduction on proper boundaries

+         */

+        DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_pnetcdf_file),

+            MPI_BYTE, &red_type);

+        DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);

+

+        /* register a PNETCDF file record reduction operator */

+        DARSHAN_MPI_CALL(PMPI_Op_create)(pnetcdf_record_reduction_op, 1, &red_op);

+

+        /* reduce shared PNETCDF file records */

+        DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,

+            shared_rec_count, red_type, red_op, 0, mod_comm);

+

+        /* clean up reduction state */

+        if(my_rank == 0)

+        {

+            int tmp_ndx = pnetcdf_runtime->file_array_ndx - shared_rec_count;

+            memcpy(&(pnetcdf_runtime->file_record_array[tmp_ndx]), red_recv_buf,

+                shared_rec_count * sizeof(struct darshan_pnetcdf_file));

+            free(red_recv_buf);

+        }

+        else

+        {

+            pnetcdf_runtime->file_array_ndx -= shared_rec_count;

+        }

+

+        DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);

+        DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);

+    }

+

+    *pnetcdf_buf = (void *)(pnetcdf_runtime->file_record_array);

+    *pnetcdf_buf_sz = pnetcdf_runtime->file_array_ndx * sizeof(struct darshan_pnetcdf_file);

+

+    return;

</span> }

<span style="color: #000000;background-color: #ddffdd">+static void pnetcdf_shutdown()

+{

+    struct pnetcdf_file_runtime_ref *ref, *tmp;

+

+    assert(pnetcdf_runtime);

+

+    HASH_ITER(hlink, pnetcdf_runtime->ncid_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, pnetcdf_runtime->ncid_hash, ref);

+        free(ref);

+    }

+

+    HASH_CLEAR(hlink, pnetcdf_runtime->file_hash); /* these entries are freed all at once below */

+

+    free(pnetcdf_runtime->file_runtime_array);

+    free(pnetcdf_runtime->file_record_array);

+    free(pnetcdf_runtime);

+    pnetcdf_runtime = NULL;

</span> 

<span style="color: #000000;background-color: #ddffdd">+    return;

+}

</span> 

 /*

  * Local variables:

</code></pre>

<br>

</li>

<li id='diff-34'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-34'>

<strong>

darshan-runtime/lib/darshan-posix.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-posix.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-posix.c

</span><span style="color: #aaaaaa">@@ -1,8 +1,12 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

<span style="color: #000000;background-color: #ddffdd">+#define _XOPEN_SOURCE 500

+#define _GNU_SOURCE

+

</span> #include "darshan-runtime-config.h"

 #include <stdio.h>

 #include <unistd.h>

<span style="color: #aaaaaa">@@ -19,12 +23,14 @@

</span> #include <search.h>

 #include <assert.h>

 #include <libgen.h>

<span style="color: #000000;background-color: #ffdddd">-#include <limits.h>

</span> #include <aio.h>

<span style="color: #000000;background-color: #ffdddd">-#define __USE_GNU

</span> #include <pthread.h>

<span style="color: #000000;background-color: #ddffdd">+#include "uthash.h"

+#include "utlist.h"

+

</span> #include "darshan.h"

<span style="color: #000000;background-color: #ddffdd">+#include "darshan-dynamic.h"

</span> 

 #ifndef HAVE_OFF64_T

 typedef int64_t off64_t;

<span style="color: #aaaaaa">@@ -33,478 +39,478 @@ typedef int64_t off64_t;

</span> #define aiocb64 aiocb

 #endif

<span style="color: #000000;background-color: #ffdddd">-extern char* __progname_full;

-

-#ifdef DARSHAN_PRELOAD

-#define __USE_GNU

-#include <dlfcn.h>

-#include <stdlib.h>

-

-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  ret (*__real_ ## name)args = NULL;

-

-#define DARSHAN_DECL(__name) __name

-

-#define DARSHAN_MPI_CALL(func) __real_ ## func

-

-#define MAP_OR_FAIL(func) \

-    if (!(__real_ ## func)) \

-    { \

-        __real_ ## func = dlsym(RTLD_NEXT, #func); \

-        if(!(__real_ ## func)) { \

-           fprintf(stderr, "Darshan failed to map symbol: %s\n", #func); \

-           exit(1); \

-       } \

-    }

-

-

-extern double (*__real_PMPI_Wtime)(void);

-

-#else

-

-#define DARSHAN_FORWARD_DECL(name,ret,args) \

-  extern ret __real_ ## name args;

-

-#define DARSHAN_DECL(__name) __wrap_ ## __name

-

-#define MAP_OR_FAIL(func)

-

-#define DARSHAN_MPI_CALL(func) func

-

-#endif

-

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(open, int, (const char *path, int flags, ...));

+DARSHAN_FORWARD_DECL(open64, int, (const char *path, int flags, ...));

+DARSHAN_FORWARD_DECL(creat, int, (const char* path, mode_t mode));

+DARSHAN_FORWARD_DECL(creat64, int, (const char* path, mode_t mode));

+DARSHAN_FORWARD_DECL(fopen, FILE*, (const char *path, const char *mode));

+DARSHAN_FORWARD_DECL(fopen64, FILE*, (const char *path, const char *mode));

</span> DARSHAN_FORWARD_DECL(mkstemp, int, (char *template));

 DARSHAN_FORWARD_DECL(mkostemp, int, (char *template, int flags));

 DARSHAN_FORWARD_DECL(mkstemps, int, (char *template, int suffixlen));

 DARSHAN_FORWARD_DECL(mkostemps, int, (char *template, int suffixlen, int flags));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(creat, int, (const char* path, mode_t mode));

-DARSHAN_FORWARD_DECL(creat64, int, (const char* path, mode_t mode));

-DARSHAN_FORWARD_DECL(open, int, (const char *path, int flags, ...));

-DARSHAN_FORWARD_DECL(open64, int, (const char *path, int flags, ...));

-DARSHAN_FORWARD_DECL(close, int, (int fd));

-DARSHAN_FORWARD_DECL(write, ssize_t, (int fd, const void *buf, size_t count));

</span> DARSHAN_FORWARD_DECL(read, ssize_t, (int fd, void *buf, size_t count));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(lseek, off_t, (int fd, off_t offset, int whence));

-DARSHAN_FORWARD_DECL(lseek64, off64_t, (int fd, off64_t offset, int whence));

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(write, ssize_t, (int fd, const void *buf, size_t count));

</span> DARSHAN_FORWARD_DECL(pread, ssize_t, (int fd, void *buf, size_t count, off_t offset));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(pread64, ssize_t, (int fd, void *buf, size_t count, off64_t offset));

</span> DARSHAN_FORWARD_DECL(pwrite, ssize_t, (int fd, const void *buf, size_t count, off_t offset));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(pwrite64, ssize_t, (int fd, const void *buf, size_t count, off64_t offset

-));

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(pread64, ssize_t, (int fd, void *buf, size_t count, off64_t offset));

+DARSHAN_FORWARD_DECL(pwrite64, ssize_t, (int fd, const void *buf, size_t count, off64_t offset));

</span> DARSHAN_FORWARD_DECL(readv, ssize_t, (int fd, const struct iovec *iov, int iovcnt));

 DARSHAN_FORWARD_DECL(writev, ssize_t, (int fd, const struct iovec *iov, int iovcnt));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(__fxstat, int, (int vers, int fd, struct stat *buf));

-DARSHAN_FORWARD_DECL(__fxstat64, int, (int vers, int fd, struct stat64 *buf));

-DARSHAN_FORWARD_DECL(__lxstat, int, (int vers, const char* path, struct stat *buf));

-DARSHAN_FORWARD_DECL(__lxstat64, int, (int vers, const char* path, struct stat64 *buf));

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(fread, size_t, (void *ptr, size_t size, size_t nmemb, FILE *stream));

+DARSHAN_FORWARD_DECL(fwrite, size_t, (const void *ptr, size_t size, size_t nmemb, FILE *stream));

+DARSHAN_FORWARD_DECL(lseek, off_t, (int fd, off_t offset, int whence));

+DARSHAN_FORWARD_DECL(lseek64, off64_t, (int fd, off64_t offset, int whence));

+DARSHAN_FORWARD_DECL(fseek, int, (FILE *stream, long offset, int whence));

</span> DARSHAN_FORWARD_DECL(__xstat, int, (int vers, const char* path, struct stat *buf));

 DARSHAN_FORWARD_DECL(__xstat64, int, (int vers, const char* path, struct stat64 *buf));

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(__lxstat, int, (int vers, const char* path, struct stat *buf));

+DARSHAN_FORWARD_DECL(__lxstat64, int, (int vers, const char* path, struct stat64 *buf));

+DARSHAN_FORWARD_DECL(__fxstat, int, (int vers, int fd, struct stat *buf));

+DARSHAN_FORWARD_DECL(__fxstat64, int, (int vers, int fd, struct stat64 *buf));

</span> DARSHAN_FORWARD_DECL(mmap, void*, (void *addr, size_t length, int prot, int flags, int fd, off_t offset));

 DARSHAN_FORWARD_DECL(mmap64, void*, (void *addr, size_t length, int prot, int flags, int fd, off64_t offset));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(fopen, FILE*, (const char *path, const char *mode));

-DARSHAN_FORWARD_DECL(fopen64, FILE*, (const char *path, const char *mode));

-DARSHAN_FORWARD_DECL(fclose, int, (FILE *fp));

-DARSHAN_FORWARD_DECL(fread, size_t, (void *ptr, size_t size, size_t nmemb, FILE *stream));

-DARSHAN_FORWARD_DECL(fwrite, size_t, (const void *ptr, size_t size, size_t nmemb, FILE *stream));

-DARSHAN_FORWARD_DECL(fseek, int, (FILE *stream, long offset, int whence));

</span> DARSHAN_FORWARD_DECL(fsync, int, (int fd));

 DARSHAN_FORWARD_DECL(fdatasync, int, (int fd));

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(close, int, (int fd));

+DARSHAN_FORWARD_DECL(fclose, int, (FILE *fp));

</span> DARSHAN_FORWARD_DECL(aio_read, int, (struct aiocb *aiocbp));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(aio_read64, int, (struct aiocb64 *aiocbp));

</span> DARSHAN_FORWARD_DECL(aio_write, int, (struct aiocb *aiocbp));

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(aio_read64, int, (struct aiocb64 *aiocbp));

</span> DARSHAN_FORWARD_DECL(aio_write64, int, (struct aiocb64 *aiocbp));

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_FORWARD_DECL(lio_listio, int, (int mode, struct aiocb *const aiocb_list[], int nitems, struct sigevent *sevp));

-DARSHAN_FORWARD_DECL(lio_listio64, int, (int mode, struct aiocb64 *const aiocb_list[], int nitems, struct sigevent *sevp));

</span> DARSHAN_FORWARD_DECL(aio_return, ssize_t, (struct aiocb *aiocbp));

 DARSHAN_FORWARD_DECL(aio_return64, ssize_t, (struct aiocb64 *aiocbp));

<span style="color: #000000;background-color: #ffdddd">-

-pthread_mutex_t cp_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

-struct darshan_job_runtime* darshan_global_job = NULL;

-static int my_rank = -1;

-static struct stat64 cp_stat_buf;

-static int darshan_mem_alignment = 1;

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_FORWARD_DECL(lio_listio, int, (int mode, struct aiocb *const aiocb_list[], int nitems, struct sigevent *sevp));

+DARSHAN_FORWARD_DECL(lio_listio64, int, (int mode, struct aiocb64 *const aiocb_list[], int nitems, struct sigevent *sevp));

</span> 

 /* struct to track information about aio operations in flight */

<span style="color: #000000;background-color: #ffdddd">-struct darshan_aio_tracker

</span><span style="color: #000000;background-color: #ddffdd">+struct posix_aio_tracker

</span> {

     double tm1;

     void *aiocbp;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_aio_tracker* next;

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_aio_tracker* next;

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* these are paths that we will not trace */

-static char* exclusions[] = {

-"/etc/",

-"/dev/",

-"/usr/",

-"/bin/",

-"/boot/",

-"/lib/",

-"/opt/",

-"/sbin/",

-"/sys/",

-"/proc/",

-NULL

</span><span style="color: #000000;background-color: #ddffdd">+/* The posix_file_runtime structure maintains necessary runtime metadata

+ * for the POSIX file record (darshan_posix_file structure, defined in

+ * darshan-posix-log-format.h) pointed to by 'file_record'. This metadata

+ * assists with the instrumenting of specific statistics in the file record.

+ * 'hlink' is a hash table link structure used to add/remove this record

+ * from the hash table of POSIX file records for this process. 

+ *

+ * RATIONALE: the POSIX module needs to track some stateful, volatile 

+ * information about each open file (like the current file offset, most recent 

+ * access time, etc.) to aid in instrumentation, but this information can't be

+ * stored in the darshan_posix_file struct because we don't want it to appear in

+ * the final darshan log file.  We therefore associate a posix_file_runtime

+ * struct with each darshan_posix_file struct in order to track this information.

+  *

+ * NOTE: There is a one-to-one mapping of posix_file_runtime structs to

+ * darshan_posix_file structs.

+ *

+ * NOTE: The posix_file_runtime struct contains a pointer to a darshan_posix_file

+ * struct (see the *file_record member) rather than simply embedding an entire

+ * darshan_posix_file struct.  This is done so that all of the darshan_posix_file

+ * structs can be kept contiguous in memory as a single array to simplify

+ * reduction, compression, and storage.

+ */

+struct posix_file_runtime

+{

+    struct darshan_posix_file* file_record;

+    int64_t offset;

+    int64_t last_byte_read;

+    int64_t last_byte_written;

+    enum darshan_io_type last_io_type;

+    double last_meta_end;

+    double last_read_end;

+    double last_write_end;

+    void* access_root;

+    int access_count;

+    void* stride_root;

+    int stride_count;

+    struct posix_aio_tracker* aio_list;

+    UT_hash_handle hlink;

</span> };

<span style="color: #000000;background-color: #ffdddd">-static double posix_wtime(void);

</span><span style="color: #000000;background-color: #ddffdd">+/* The posix_file_runtime_ref structure is used to associate a POSIX

+ * file descriptor with an already existing POSIX file record. This is

+ * necessary as many POSIX I/O functions take only an input file descriptor,

+ * but POSIX file records are indexed by their full file paths (i.e., darshan

+ * record identifiers for POSIX files are created by hashing the file path).

+ * In other words, this structure is necessary as it allows us to look up a

+ * file record either by a pathname (posix_file_runtime) or by POSIX file

+ * descriptor (posix_file_runtime_ref), depending on which parameters are

+ * available. This structure includes another hash table link, since separate

+ * hashes are maintained for posix_file_runtime structures and posix_file_runtime_ref

+ * structures.

+ *

+ * RATIONALE: In theory the fd information could be included in the

+ * posix_file_runtime struct rather than in a separate structure here.  The

+ * reason we don't do that is because the same file could be opened multiple

+ * times by a given process with different file descriptors and thus

+ * simulataneously referenced using different file descriptors.  This practice is

+ * not common, but we must support it.

+ *

+ * NOTE: there are potentially multiple posix_file_runtime_ref structures

+ * referring to a single posix_file_runtime structure.  Most of the time there is

+ * only one, however.

+ */

+struct posix_file_runtime_ref

+{

+    struct posix_file_runtime* file;

+    int fd;

+    UT_hash_handle hlink;

+};

</span> 

<span style="color: #000000;background-color: #ffdddd">-static void cp_access_counter(struct darshan_file_runtime* file, ssize_t size,     enum cp_counter_type type);

</span><span style="color: #000000;background-color: #ddffdd">+/* The posix_runtime structure maintains necessary state for storing

+ * POSIX file records and for coordinating with darshan-core at 

+ * shutdown time.

+ */

+struct posix_runtime

+{

+    struct posix_file_runtime* file_runtime_array;

+    struct darshan_posix_file* file_record_array;

+    int file_array_size;

+    int file_array_ndx;

+    struct posix_file_runtime* file_hash;

+    struct posix_file_runtime_ref* fd_hash;

+};

</span> 

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_ref* ref_by_handle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type);

</span><span style="color: #000000;background-color: #ddffdd">+static struct posix_runtime *posix_runtime = NULL;

+static pthread_mutex_t posix_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;

+static int instrumentation_disabled = 0;

+static int my_rank = -1;

+static int darshan_mem_alignment = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_fd(int fd);

-static void darshan_file_close_fd(int fd);

-static struct darshan_file_runtime* darshan_file_by_name_setfd(const char* name, int fd);

-static char* clean_path(const char* path);

-static void darshan_aio_tracker_add(int fd, void *aiocbp);

-static struct darshan_aio_tracker* darshan_aio_tracker_del(int fd, void *aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+static void posix_runtime_initialize(void);

+static struct posix_file_runtime* posix_file_by_name(const char *name);

+static struct posix_file_runtime* posix_file_by_name_setfd(const char* name, int fd);

+static struct posix_file_runtime* posix_file_by_fd(int fd);

+static void posix_file_close_fd(int fd);

+static void posix_aio_tracker_add(int fd, void *aiocbp);

+static struct posix_aio_tracker* posix_aio_tracker_del(int fd, void *aiocbp);

+static int posix_record_compare(const void* a, const void* b);

+static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype);

+static void posix_shared_record_variance(MPI_Comm mod_comm,

+    struct darshan_posix_file *inrec_array, struct darshan_posix_file *outrec_array,

+    int shared_rec_count);

+

+static void posix_begin_shutdown(void);

+static void posix_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,

+    int shared_rec_count, void **posix_buf, int *posix_buf_sz);

+static void posix_shutdown(void);

+

+#define POSIX_LOCK() pthread_mutex_lock(&posix_runtime_mutex)

+#define POSIX_UNLOCK() pthread_mutex_unlock(&posix_runtime_mutex)

+

+#define POSIX_RECORD_OPEN(__ret, __path, __mode, __stream_flag, __tm1, __tm2) do { \

+    struct posix_file_runtime* file; \

+    char* exclude; \

+    int tmp_index = 0; \

+    if(__ret < 0) break; \

+    while((exclude = darshan_path_exclusions[tmp_index])) { \

+        if(!(strncmp(exclude, __path, strlen(exclude)))) \

+            break; \

+        tmp_index++; \

+    } \

+    if(exclude) break; \

+    file = posix_file_by_name_setfd(__path, __ret); \

+    if(!file) break; \

+    if(__mode) \

+        file->file_record->counters[POSIX_MODE] = __mode; \

+    file->offset = 0; \

+    file->last_byte_written = 0; \

+    file->last_byte_read = 0; \

+    if(__stream_flag)\

+        file->file_record->counters[POSIX_FOPENS] += 1; \

+    else \

+        file->file_record->counters[POSIX_OPENS] += 1; \

+    if(file->file_record->fcounters[POSIX_F_OPEN_TIMESTAMP] == 0) \

+        file->file_record->fcounters[POSIX_F_OPEN_TIMESTAMP] = __tm1; \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_META_TIME], __tm1, __tm2, file->last_meta_end); \

+} while(0)

</span> 

<span style="color: #000000;background-color: #ffdddd">-#define CP_RECORD_WRITE(__ret, __fd, __count, __pwrite_flag, __pwrite_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \

</span><span style="color: #000000;background-color: #ddffdd">+#define POSIX_RECORD_READ(__ret, __fd, __pread_flag, __pread_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \

</span>     size_t stride; \

     int64_t this_offset; \

<span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file; \

</span>     int64_t file_alignment; \

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file; \

</span>     double __elapsed = __tm2-__tm1; \

     if(__ret < 0) break; \

<span style="color: #000000;background-color: #ffdddd">-    file = darshan_file_by_fd(__fd); \

</span><span style="color: #000000;background-color: #ddffdd">+    file = posix_file_by_fd(__fd); \

</span>     if(!file) break; \

<span style="color: #000000;background-color: #ffdddd">-    if(__pwrite_flag) \

-        this_offset = __pwrite_offset; \

</span><span style="color: #000000;background-color: #ddffdd">+    if(__pread_flag) \

+        this_offset = __pread_offset; \

</span>     else \

         this_offset = file->offset; \

<span style="color: #000000;background-color: #ffdddd">-    file_alignment = CP_VALUE(file, CP_FILE_ALIGNMENT); \

-    if(this_offset > file->last_byte_written) \

-        CP_INC(file, CP_SEQ_WRITES, 1); \

-    if(this_offset == (file->last_byte_written + 1)) \

-        CP_INC(file, CP_CONSEC_WRITES, 1); \

-    if(this_offset > 0 && this_offset > file->last_byte_written \

-        && file->last_byte_written != 0) \

-        stride = this_offset - file->last_byte_written - 1; \

</span><span style="color: #000000;background-color: #ddffdd">+    if(this_offset > file->last_byte_read) \

+        file->file_record->counters[POSIX_SEQ_READS] += 1;  \

+    if(this_offset == (file->last_byte_read + 1)) \

+        file->file_record->counters[POSIX_CONSEC_READS] += 1;  \

+    if(this_offset > 0 && this_offset > file->last_byte_read \

+        && file->last_byte_read != 0) \

+        stride = this_offset - file->last_byte_read - 1; \

</span>     else \

         stride = 0; \

<span style="color: #000000;background-color: #ffdddd">-    file->last_byte_written = this_offset + __ret - 1; \

</span><span style="color: #000000;background-color: #ddffdd">+    file->last_byte_read = this_offset + __ret - 1; \

</span>     file->offset = this_offset + __ret; \

<span style="color: #000000;background-color: #ffdddd">-    CP_MAX(file, CP_MAX_BYTE_WRITTEN, (this_offset + __ret -1)); \

-    CP_INC(file, CP_BYTES_WRITTEN, __ret); \

</span><span style="color: #000000;background-color: #ddffdd">+    if(file->file_record->counters[POSIX_MAX_BYTE_READ] < (this_offset + __ret - 1)) \

+        file->file_record->counters[POSIX_MAX_BYTE_READ] = (this_offset + __ret - 1); \

+    file->file_record->counters[POSIX_BYTES_READ] += __ret; \

</span>     if(__stream_flag) \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_POSIX_FWRITES, 1); \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_FREADS] += 1; \

</span>     else \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_POSIX_WRITES, 1); \

-    CP_BUCKET_INC(file, CP_SIZE_WRITE_0_100, __ret); \

-    cp_access_counter(file, stride, CP_COUNTER_STRIDE); \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_READS] += 1; \

+    DARSHAN_BUCKET_INC(&(file->file_record->counters[POSIX_SIZE_READ_0_100]), __ret); \

+    darshan_common_val_counter(&file->access_root, &file->access_count, __ret); \

+    darshan_common_val_counter(&file->stride_root, &file->stride_count, stride); \

</span>     if(!__aligned) \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_MEM_NOT_ALIGNED, 1); \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_MEM_NOT_ALIGNED] += 1; \

+    file_alignment = file->file_record->counters[POSIX_FILE_ALIGNMENT]; \

</span>     if(file_alignment > 0 && (this_offset % file_alignment) != 0) \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_FILE_NOT_ALIGNED, 1); \

-    cp_access_counter(file, __ret, CP_COUNTER_ACCESS); \

-    if(file->last_io_type == CP_READ) \

-        CP_INC(file, CP_RW_SWITCHES, 1); \

-    file->last_io_type = CP_WRITE; \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_posix_write_end, CP_F_POSIX_WRITE_TIME); \

-    if(CP_F_VALUE(file, CP_F_WRITE_START_TIMESTAMP) == 0) \

-        CP_F_SET(file, CP_F_WRITE_START_TIMESTAMP, __tm1); \

-    CP_F_SET(file, CP_F_WRITE_END_TIMESTAMP, __tm2); \

-    if(CP_F_VALUE(file, CP_F_MAX_WRITE_TIME) < __elapsed){ \

-        CP_F_SET(file, CP_F_MAX_WRITE_TIME, __elapsed); \

-        CP_SET(file, CP_MAX_WRITE_TIME_SIZE, __ret); } \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_FILE_NOT_ALIGNED] += 1; \

+    if(file->last_io_type == DARSHAN_IO_WRITE) \

+        file->file_record->counters[POSIX_RW_SWITCHES] += 1; \

+    file->last_io_type = DARSHAN_IO_READ; \

+    if(file->file_record->fcounters[POSIX_F_READ_START_TIMESTAMP] == 0) \

+        file->file_record->fcounters[POSIX_F_READ_START_TIMESTAMP] = __tm1; \

+    file->file_record->fcounters[POSIX_F_READ_END_TIMESTAMP] = __tm2; \

+    if(file->file_record->fcounters[POSIX_F_MAX_READ_TIME] < __elapsed) { \

+        file->file_record->fcounters[POSIX_F_MAX_READ_TIME] = __elapsed; \

+        file->file_record->counters[POSIX_MAX_READ_TIME_SIZE] = __ret; } \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_READ_TIME], __tm1, __tm2, file->last_read_end); \

</span> } while(0)

<span style="color: #000000;background-color: #ffdddd">-#define CP_RECORD_READ(__ret, __fd, __count, __pread_flag, __pread_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \

</span><span style="color: #000000;background-color: #ddffdd">+#define POSIX_RECORD_WRITE(__ret, __fd, __pwrite_flag, __pwrite_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \

</span>     size_t stride; \

     int64_t this_offset; \

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file; \

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file; \

</span>     int64_t file_alignment; \

     double __elapsed = __tm2-__tm1; \

     if(__ret < 0) break; \

<span style="color: #000000;background-color: #ffdddd">-    file = darshan_file_by_fd(__fd); \

</span><span style="color: #000000;background-color: #ddffdd">+    file = posix_file_by_fd(__fd); \

</span>     if(!file) break; \

<span style="color: #000000;background-color: #ffdddd">-    if(__pread_flag)\

-        this_offset = __pread_offset; \

</span><span style="color: #000000;background-color: #ddffdd">+    if(__pwrite_flag) \

+        this_offset = __pwrite_offset; \

</span>     else \

         this_offset = file->offset; \

<span style="color: #000000;background-color: #ffdddd">-    file_alignment = CP_VALUE(file, CP_FILE_ALIGNMENT); \

-    if(this_offset > file->last_byte_read) \

-        CP_INC(file, CP_SEQ_READS, 1); \

-    if(this_offset == (file->last_byte_read + 1)) \

-        CP_INC(file, CP_CONSEC_READS, 1); \

-    if(this_offset > 0 && this_offset > file->last_byte_read \

-        && file->last_byte_read != 0) \

-        stride = this_offset - file->last_byte_read - 1; \

</span><span style="color: #000000;background-color: #ddffdd">+    if(this_offset > file->last_byte_written) \

+        file->file_record->counters[POSIX_SEQ_WRITES] += 1; \

+    if(this_offset == (file->last_byte_written + 1)) \

+        file->file_record->counters[POSIX_CONSEC_WRITES] += 1; \

+    if(this_offset > 0 && this_offset > file->last_byte_written \

+        && file->last_byte_written != 0) \

+        stride = this_offset - file->last_byte_written - 1; \

</span>     else \

         stride = 0; \

<span style="color: #000000;background-color: #ffdddd">-    file->last_byte_read = this_offset + __ret - 1; \

-    CP_MAX(file, CP_MAX_BYTE_READ, (this_offset + __ret -1)); \

</span><span style="color: #000000;background-color: #ddffdd">+    file->last_byte_written = this_offset + __ret - 1; \

</span>     file->offset = this_offset + __ret; \

<span style="color: #000000;background-color: #ffdddd">-    CP_INC(file, CP_BYTES_READ, __ret); \

-    if(__stream_flag)\

-        CP_INC(file, CP_POSIX_FREADS, 1); \

-    else\

-        CP_INC(file, CP_POSIX_READS, 1); \

-    CP_BUCKET_INC(file, CP_SIZE_READ_0_100, __ret); \

-    cp_access_counter(file, stride, CP_COUNTER_STRIDE); \

</span><span style="color: #000000;background-color: #ddffdd">+    if(file->file_record->counters[POSIX_MAX_BYTE_WRITTEN] < (this_offset + __ret - 1)) \

+        file->file_record->counters[POSIX_MAX_BYTE_WRITTEN] = (this_offset + __ret - 1); \

+    file->file_record->counters[POSIX_BYTES_WRITTEN] += __ret; \

+    if(__stream_flag) \

+        file->file_record->counters[POSIX_FWRITES] += 1; \

+    else \

+        file->file_record->counters[POSIX_WRITES] += 1; \

+    DARSHAN_BUCKET_INC(&(file->file_record->counters[POSIX_SIZE_WRITE_0_100]), __ret); \

+    darshan_common_val_counter(&file->access_root, &file->access_count, __ret); \

+    darshan_common_val_counter(&file->stride_root, &file->stride_count, stride); \

</span>     if(!__aligned) \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_MEM_NOT_ALIGNED, 1); \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_MEM_NOT_ALIGNED] += 1; \

+    file_alignment = file->file_record->counters[POSIX_FILE_ALIGNMENT]; \

</span>     if(file_alignment > 0 && (this_offset % file_alignment) != 0) \

<span style="color: #000000;background-color: #ffdddd">-        CP_INC(file, CP_FILE_NOT_ALIGNED, 1); \

-    cp_access_counter(file, __ret, CP_COUNTER_ACCESS); \

-    if(file->last_io_type == CP_WRITE) \

-        CP_INC(file, CP_RW_SWITCHES, 1); \

-    file->last_io_type = CP_READ; \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_posix_read_end, CP_F_POSIX_READ_TIME); \

-    if(CP_F_VALUE(file, CP_F_READ_START_TIMESTAMP) == 0) \

-        CP_F_SET(file, CP_F_READ_START_TIMESTAMP, __tm1); \

-    CP_F_SET(file, CP_F_READ_END_TIMESTAMP, __tm2); \

-    if(CP_F_VALUE(file, CP_F_MAX_READ_TIME) < __elapsed){ \

-        CP_F_SET(file, CP_F_MAX_READ_TIME, __elapsed); \

-        CP_SET(file, CP_MAX_READ_TIME_SIZE, __ret); } \

</span><span style="color: #000000;background-color: #ddffdd">+        file->file_record->counters[POSIX_FILE_NOT_ALIGNED] += 1; \

+    if(file->last_io_type == DARSHAN_IO_READ) \

+        file->file_record->counters[POSIX_RW_SWITCHES] += 1; \

+    file->last_io_type = DARSHAN_IO_WRITE; \

+    if(file->file_record->fcounters[POSIX_F_WRITE_START_TIMESTAMP] == 0) \

+        file->file_record->fcounters[POSIX_F_WRITE_START_TIMESTAMP] = __tm1; \

+    file->file_record->fcounters[POSIX_F_WRITE_END_TIMESTAMP] = __tm2; \

+    if(file->file_record->fcounters[POSIX_F_MAX_WRITE_TIME] < __elapsed) { \

+        file->file_record->fcounters[POSIX_F_MAX_WRITE_TIME] = __elapsed; \

+        file->file_record->counters[POSIX_MAX_WRITE_TIME_SIZE] = __ret; } \

+    DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_WRITE_TIME], __tm1, __tm2, file->last_write_end); \

</span> } while(0)

<span style="color: #000000;background-color: #ffdddd">-#define CP_LOOKUP_RECORD_STAT(__path, __statbuf, __tm1, __tm2) do { \

</span><span style="color: #000000;background-color: #ddffdd">+#define POSIX_LOOKUP_RECORD_STAT(__path, __statbuf, __tm1, __tm2) do { \

</span>     char* exclude; \

     int tmp_index = 0; \

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file; \

-    while((exclude = exclusions[tmp_index])) { \

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file; \

+    while((exclude = darshan_path_exclusions[tmp_index])) { \

</span>         if(!(strncmp(exclude, __path, strlen(exclude)))) \

             break; \

         tmp_index++; \

     } \

     if(exclude) break; \

<span style="color: #000000;background-color: #ffdddd">-    file = darshan_file_by_name(__path); \

-    if (file) \

</span><span style="color: #000000;background-color: #ddffdd">+    file = posix_file_by_name(__path); \

+    if(file) \

</span>     { \

<span style="color: #000000;background-color: #ffdddd">-        CP_RECORD_STAT(file, __statbuf, __tm1, __tm2); \

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_RECORD_STAT(file, __statbuf, __tm1, __tm2); \

</span>     } \

 } while(0)

<span style="color: #000000;background-color: #ffdddd">-    

-#define CP_RECORD_STAT(__file, __statbuf, __tm1, __tm2) do { \

-    if(!CP_VALUE((__file), CP_POSIX_STATS) && !CP_VALUE((__file), CP_POSIX_OPENS)){ \

-        CP_SET((__file), CP_FILE_ALIGNMENT, (__statbuf)->st_blksize); \

-        CP_SET((__file), CP_SIZE_AT_OPEN, (__statbuf)->st_size); \

-    }\

-    (__file)->log_file->rank = my_rank; \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME); \

-    CP_INC(__file, CP_POSIX_STATS, 1); \

</span><span style="color: #000000;background-color: #ddffdd">+#define POSIX_RECORD_STAT(__file, __statbuf, __tm1, __tm2) do { \

+    DARSHAN_TIMER_INC_NO_OVERLAP((__file)->file_record->fcounters[POSIX_F_META_TIME], __tm1, __tm2, (__file)->last_meta_end); \

+    (__file)->file_record->counters[POSIX_STATS] += 1; \

</span> } while(0)

<span style="color: #000000;background-color: #ffdddd">-#ifdef __CP_STAT_AT_OPEN

-#define CP_STAT_FILE(_f, _p, _r) do { \

-    if(!CP_VALUE((_f), CP_POSIX_STATS) && !CP_VALUE((_f), CP_POSIX_OPENS)){ \

-        if(fstat64(_r, &cp_stat_buf) == 0) { \

-            CP_SET(_f, CP_FILE_ALIGNMENT, cp_stat_buf.st_blksize); \

-            CP_SET(_f, CP_SIZE_AT_OPEN, cp_stat_buf.st_size); \

-        }\

-    }\

-}while(0)

-#else

-#define CP_STAT_FILE(_f, _p, _r) do { }while(0)

-#endif

-

-#define CP_RECORD_OPEN(__ret, __path, __mode, __stream_flag, __tm1, __tm2) do { \

-    struct darshan_file_runtime* file; \

-    char* exclude; \

-    int tmp_index = 0; \

-    if(__ret < 0) break; \

-    while((exclude = exclusions[tmp_index])) { \

-        if(!(strncmp(exclude, __path, strlen(exclude)))) \

-            break; \

-        tmp_index++; \

-    } \

-    if(exclude) break; \

-    file = darshan_file_by_name_setfd(__path, __ret); \

-    if(!file) break; \

-    CP_STAT_FILE(file, __path, __ret); \

-    file->log_file->rank = my_rank; \

-    if(__mode) \

-        CP_SET(file, CP_MODE, __mode); \

-    file->offset = 0; \

-    file->last_byte_written = 0; \

-    file->last_byte_read = 0; \

-    if(__stream_flag)\

-        CP_INC(file, CP_POSIX_FOPENS, 1); \

-    else \

-        CP_INC(file, CP_POSIX_OPENS, 1); \

-    if(CP_F_VALUE(file, CP_F_OPEN_TIMESTAMP) == 0) \

-        CP_F_SET(file, CP_F_OPEN_TIMESTAMP, __tm1); \

-    CP_F_INC_NO_OVERLAP(file, __tm1, __tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME); \

-} while (0)

</span><span style="color: #000000;background-color: #ddffdd">+/**********************************************************

+ *      Wrappers for POSIX I/O functions of interest      * 

+ **********************************************************/

</span> 

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(close)(int fd)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(open)(const char *path, int flags, ...)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

-    int tmp_fd = fd;

-    double tm1, tm2;

</span><span style="color: #000000;background-color: #ddffdd">+    int mode = 0;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(close);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(open);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_close(fd);

-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    if(flags & O_CREAT) 

+    {

+        va_list arg;

+        va_start(arg, flags);

+        mode = va_arg(arg, int);

+        va_end(arg);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(tmp_fd);

-    if(file)

</span><span style="color: #000000;background-color: #ddffdd">+        tm1 = darshan_core_wtime();

+        ret = __real_open(path, flags, mode);

+        tm2 = darshan_core_wtime();

+    }

+    else

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        file->last_byte_written = 0;

-        file->last_byte_read = 0;

-        CP_F_SET(file, CP_F_CLOSE_TIMESTAMP, posix_wtime());

-        CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME);

-        darshan_file_close_fd(tmp_fd);

</span><span style="color: #000000;background-color: #ddffdd">+        tm1 = darshan_core_wtime();

+        ret = __real_open(path, flags);

+        tm2 = darshan_core_wtime();

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(fclose)(FILE *fp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(open64)(const char *path, int flags, ...)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

-    int tmp_fd = fileno(fp);

-    double tm1, tm2;

</span><span style="color: #000000;background-color: #ddffdd">+    int mode = 0;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(fclose);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(open64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_fclose(fp);

-    tm2 = darshan_wtime();

-    

-    CP_LOCK();

-    file = darshan_file_by_fd(tmp_fd);

-    if(file)

</span><span style="color: #000000;background-color: #ddffdd">+    if(flags & O_CREAT)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        file->last_byte_written = 0;

-        file->last_byte_read = 0;

-        CP_F_SET(file, CP_F_CLOSE_TIMESTAMP, posix_wtime());

-        CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME);

-        darshan_file_close_fd(tmp_fd);

</span><span style="color: #000000;background-color: #ddffdd">+        va_list arg;

+        va_start(arg, flags);

+        mode = va_arg(arg, int);

+        va_end(arg);

+

+        tm1 = darshan_core_wtime();

+        ret = __real_open64(path, flags, mode);

+        tm2 = darshan_core_wtime();

+    }

+    else

+    {

+        tm1 = darshan_core_wtime();

+        ret = __real_open64(path, flags);

+        tm2 = darshan_core_wtime();

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

-int DARSHAN_DECL(fsync)(int fd)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(creat)(const char* path, mode_t mode)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(fsync);

-

-    tm1 = darshan_wtime();

-    ret = __real_fsync(fd);

-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(creat);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_creat(path, mode);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_write_end, CP_F_POSIX_WRITE_TIME); \

-        CP_INC(file, CP_POSIX_FSYNCS, 1);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(fdatasync)(int fd)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(creat64)(const char* path, mode_t mode)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(fdatasync);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(creat64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_fdatasync(fd);

-    tm2 = darshan_wtime();

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_creat64(path, mode);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_write_end, CP_F_POSIX_WRITE_TIME); \

-        CP_INC(file, CP_POSIX_FDSYNCS, 1);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

-void* DARSHAN_DECL(mmap64)(void *addr, size_t length, int prot, int flags,

-    int fd, off64_t offset)

</span><span style="color: #000000;background-color: #ddffdd">+FILE* DARSHAN_DECL(fopen)(const char *path, const char *mode)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    void* ret;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    FILE* ret;

+    int fd;

+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(mmap64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fopen);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_mmap64(addr, length, prot, flags, fd, offset);

-    if(ret == MAP_FAILED)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_fopen(path, mode);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_INC(file, CP_POSIX_MMAPS, 1);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret == NULL)

+        fd = -1;

+    else

+        fd = fileno(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(fd, path, 0, 1, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

-void* DARSHAN_DECL(mmap)(void *addr, size_t length, int prot, int flags,

-    int fd, off_t offset)

</span><span style="color: #000000;background-color: #ddffdd">+FILE* DARSHAN_DECL(fopen64)(const char *path, const char *mode)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    void* ret;

-    struct darshan_file_runtime* file;

-

-    MAP_OR_FAIL(mmap);

</span><span style="color: #000000;background-color: #ddffdd">+    FILE* ret;

+    int fd;

+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_mmap(addr, length, prot, flags, fd, offset);

-    if(ret == MAP_FAILED)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fopen64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_INC(file, CP_POSIX_MMAPS, 1);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_fopen64(path, mode);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(ret);

-}

-

-int DARSHAN_DECL(creat)(const char* path, mode_t mode)

-{

-    int ret;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(creat);

-

-    tm1 = darshan_wtime();

-    ret = __real_creat(path, mode);

-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret == NULL)

+        fd = -1;

+    else

+        fd = fileno(ret);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(fd, path, 0, 1, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #aaaaaa">@@ -516,13 +522,14 @@ int DARSHAN_DECL(mkstemp)(char* template)

</span> 

     MAP_OR_FAIL(mkstemp);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_mkstemp(template);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #aaaaaa">@@ -534,18 +541,18 @@ int DARSHAN_DECL(mkostemp)(char* template, int flags)

</span> 

     MAP_OR_FAIL(mkostemp);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_mkostemp(template, flags);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

</span> int DARSHAN_DECL(mkstemps)(char* template, int suffixlen)

 {

     int ret;

<span style="color: #aaaaaa">@@ -553,18 +560,18 @@ int DARSHAN_DECL(mkstemps)(char* template, int suffixlen)

</span> 

     MAP_OR_FAIL(mkstemps);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_mkstemps(template, suffixlen);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

</span> int DARSHAN_DECL(mkostemps)(char* template, int suffixlen, int flags)

 {

     int ret;

<span style="color: #aaaaaa">@@ -572,282 +579,102 @@ int DARSHAN_DECL(mkostemps)(char* template, int suffixlen, int flags)

</span> 

     MAP_OR_FAIL(mkostemps);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_mkostemps(template, suffixlen, flags);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-

-    CP_LOCK();

-    CP_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(creat64)(const char* path, mode_t mode)

-{

-    int ret;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(creat64);

-

-    tm1 = darshan_wtime();

-    ret = __real_creat64(path, mode);

-    tm2 = darshan_wtime();

-

-    CP_LOCK();

-    CP_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(open64)(const char* path, int flags, ...)

-{

-    int mode = 0;

-    int ret;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(open64);

-

-    if (flags & O_CREAT) 

-    {

-        va_list arg;

-        va_start(arg, flags);

-        mode = va_arg(arg, int);

-        va_end(arg);

-

-        tm1 = darshan_wtime();

-        ret = __real_open64(path, flags, mode);

-        tm2 = darshan_wtime();

-    }

-    else

-    {

-        tm1 = darshan_wtime();

-        ret = __real_open64(path, flags);

-        tm2 = darshan_wtime();

-    }

-

-    CP_LOCK();

-    CP_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(open)(const char *path, int flags, ...)

-{

-    int mode = 0;

-    int ret;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(open);

-

-    if (flags & O_CREAT) 

-    {

-        va_list arg;

-        va_start(arg, flags);

-        mode = va_arg(arg, int);

-        va_end(arg);

-

-        tm1 = darshan_wtime();

-        ret = __real_open(path, flags, mode);

-        tm2 = darshan_wtime();

-    }

-    else

-    {

-        tm1 = darshan_wtime();

-        ret = __real_open(path, flags);

-        tm2 = darshan_wtime();

-    }

-

-    CP_LOCK();

-    CP_RECORD_OPEN(ret, path, mode, 0, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-FILE* DARSHAN_DECL(fopen64)(const char *path, const char *mode)

-{

-    FILE* ret;

-    int fd;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(fopen64);

-

-    tm1 = darshan_wtime();

-    ret = __real_fopen64(path, mode);

-    tm2 = darshan_wtime();

-    if(ret == 0)

-        fd = -1;

-    else

-        fd = fileno(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_RECORD_OPEN(fd, path, 0, 1, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_OPEN(ret, template, 0, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-FILE* DARSHAN_DECL(fopen)(const char *path, const char *mode)

-{

-    FILE* ret;

-    int fd;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(fopen);

-

-    tm1 = darshan_wtime();

-    ret = __real_fopen(path, mode);

-    tm2 = darshan_wtime();

-    if(ret == 0)

-        fd = -1;

-    else

-        fd = fileno(ret);

-

-    CP_LOCK();

-    CP_RECORD_OPEN(fd, path, 0, 1, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(__xstat64)(int vers, const char *path, struct stat64 *buf)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(read)(int fd, void *buf, size_t count)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    ssize_t ret;

+    int aligned_flag = 0;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__xstat64);

-

-    tm1 = darshan_wtime();

-    ret = __real___xstat64(vers, path, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

-

-    CP_LOCK();

-    CP_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(__lxstat64)(int vers, const char *path, struct stat64 *buf)

-{

-    int ret;

-    double tm1, tm2;

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(read);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__lxstat64);

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real___lxstat64(vers, path, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_read(fd, buf, count);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_READ(ret, fd, 0, 0, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(__fxstat64)(int vers, int fd, struct stat64 *buf)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(write)(int fd, const void *buf, size_t count)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    ssize_t ret;

+    int aligned_flag = 0;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__fxstat64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(write);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real___fxstat64(vers, fd, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* skip logging if this was triggered internally */

-    if(buf == &cp_stat_buf)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_write(fd, buf, count);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_RECORD_STAT(file, buf, tm1, tm2);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_WRITE(ret, fd, 0, 0, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

-int DARSHAN_DECL(__xstat)(int vers, const char *path, struct stat *buf)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(pread)(int fd, void *buf, size_t count, off_t offset)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    ssize_t ret;

+    int aligned_flag = 0;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__xstat);

-

-    tm1 = darshan_wtime();

-    ret = __real___xstat(vers, path, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

-

-    CP_LOCK();

-    CP_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

-    CP_UNLOCK();

-

-    return(ret);

-}

-

-int DARSHAN_DECL(__lxstat)(int vers, const char *path, struct stat *buf)

-{

-    int ret;

-    double tm1, tm2;

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(pread);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__lxstat);

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real___lxstat(vers, path, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_pread(fd, buf, count, offset);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    CP_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_READ(ret, fd, 1, offset, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(__fxstat)(int vers, int fd, struct stat *buf)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(pwrite)(int fd, const void *buf, size_t count, off_t offset)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    ssize_t ret;

+    int aligned_flag = 0;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(__fxstat);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(pwrite);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real___fxstat(vers, fd, buf);

-    tm2 = darshan_wtime();

-    if(ret < 0 || !S_ISREG(buf->st_mode))

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* skip logging if this was triggered internally */

-    if((void*)buf == (void*)&cp_stat_buf)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_pwrite(fd, buf, count, offset);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

-    {

-        CP_RECORD_STAT(file, buf, tm1, tm2);

-    }

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_WRITE(ret, fd, 1, offset, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #aaaaaa">@@ -860,56 +687,17 @@ ssize_t DARSHAN_DECL(pread64)(int fd, void *buf, size_t count, off64_t offset)

</span> 

     MAP_OR_FAIL(pread64);

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_pread64(fd, buf, count, offset);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_READ(ret, fd, count, 1, offset, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-ssize_t DARSHAN_DECL(pread)(int fd, void *buf, size_t count, off_t offset)

-{

-    ssize_t ret;

-    int aligned_flag = 0;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(pread);

-

-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

-

-    tm1 = darshan_wtime();

-    ret = __real_pread(fd, buf, count, offset);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_READ(ret, fd, count, 1, offset, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-

-ssize_t DARSHAN_DECL(pwrite)(int fd, const void *buf, size_t count, off_t offset)

-{

-    ssize_t ret;

-    int aligned_flag = 0;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(pwrite);

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_READ(ret, fd, 1, offset, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_pwrite(fd, buf, count, offset);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_WRITE(ret, fd, count, 1, offset, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

</span>     return(ret);

 }

<span style="color: #aaaaaa">@@ -921,15 +709,17 @@ ssize_t DARSHAN_DECL(pwrite64)(int fd, const void *buf, size_t count, off64_t of

</span> 

     MAP_OR_FAIL(pwrite64);

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)buf % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_pwrite64(fd, buf, count, offset);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_WRITE(ret, fd, count, 1, offset, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_WRITE(ret, fd, 1, offset, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

+

</span>     return(ret);

 }

<span style="color: #aaaaaa">@@ -948,12 +738,15 @@ ssize_t DARSHAN_DECL(readv)(int fd, const struct iovec *iov, int iovcnt)

</span>             aligned_flag = 0;

     }

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_readv(fd, iov, iovcnt);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_READ(ret, fd, count, 0, 0, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_READ(ret, fd, 0, 0, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

+

</span>     return(ret);

 }

<span style="color: #aaaaaa">@@ -968,16 +761,19 @@ ssize_t DARSHAN_DECL(writev)(int fd, const struct iovec *iov, int iovcnt)

</span> 

     for(i=0; i<iovcnt; i++)

     {

<span style="color: #000000;background-color: #ffdddd">-        if(!((unsigned long)iov[i].iov_base % darshan_mem_alignment == 0))

</span><span style="color: #000000;background-color: #ddffdd">+        if(((unsigned long)iov[i].iov_base % darshan_mem_alignment) != 0)

</span>             aligned_flag = 0;

     }

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_writev(fd, iov, iovcnt);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_WRITE(ret, fd, count, 0, 0, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_RECORD_WRITE(ret, fd, 0, 0, aligned_flag, 0, tm1, tm2);

+    POSIX_UNLOCK();

+

</span>     return(ret);

 }

<span style="color: #aaaaaa">@@ -989,1309 +785,1485 @@ size_t DARSHAN_DECL(fread)(void *ptr, size_t size, size_t nmemb, FILE *stream)

</span> 

     MAP_OR_FAIL(fread);

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)ptr % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)ptr % darshan_mem_alignment == 0) aligned_flag = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_fread(ptr, size, nmemb, stream);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

-    CP_LOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

</span>     if(ret > 0)

<span style="color: #000000;background-color: #ffdddd">-        CP_RECORD_READ(size*ret, fileno(stream), (size*nmemb), 0, 0, aligned_flag, 1, tm1, tm2);

</span><span style="color: #000000;background-color: #ddffdd">+    {

+        POSIX_RECORD_READ(size*ret, fileno(stream), 0, 0,

+            aligned_flag, 1, tm1, tm2);

+    }

</span>     else

<span style="color: #000000;background-color: #ffdddd">-        CP_RECORD_READ(ret, fileno(stream), (size*nmemb), 0, 0, aligned_flag, 1, tm1, tm2);

-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    {

+        POSIX_RECORD_READ(ret, fileno(stream), 0, 0,

+            aligned_flag, 1, tm1, tm2);

+    }

+    POSIX_UNLOCK();

+

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-ssize_t DARSHAN_DECL(read)(int fd, void *buf, size_t count)

</span><span style="color: #000000;background-color: #ddffdd">+size_t DARSHAN_DECL(fwrite)(const void *ptr, size_t size, size_t nmemb, FILE *stream)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    ssize_t ret;

</span><span style="color: #000000;background-color: #ddffdd">+    size_t ret;

</span>     int aligned_flag = 0;

     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(read);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fwrite);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    if((unsigned long)ptr % darshan_mem_alignment == 0) aligned_flag = 1;

+

+    tm1 = darshan_core_wtime();

+    ret = __real_fwrite(ptr, size, nmemb, stream);

+    tm2 = darshan_core_wtime();

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    if(ret > 0)

+    {

+        POSIX_RECORD_WRITE(size*ret, fileno(stream), 0, 0,

+            aligned_flag, 1, tm1, tm2);

+    }

+    else

+    {

+        POSIX_RECORD_WRITE(ret, fileno(stream), 0, 0,

+            aligned_flag, 1, tm1, tm2);

+    }

+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_read(fd, buf, count);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_READ(ret, fd, count, 0, 0, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-ssize_t DARSHAN_DECL(write)(int fd, const void *buf, size_t count)

</span><span style="color: #000000;background-color: #ddffdd">+off_t DARSHAN_DECL(lseek)(int fd, off_t offset, int whence)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    ssize_t ret;

-    int aligned_flag = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    off_t ret;

+    struct posix_file_runtime* file;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(write);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(lseek);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)buf % darshan_mem_alignment == 0)

-        aligned_flag = 1;

-

-    tm1 = darshan_wtime();

-    ret = __real_write(fd, buf, count);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    CP_RECORD_WRITE(ret, fd, count, 0, 0, aligned_flag, 0, tm1, tm2);

-    CP_UNLOCK();

-    return(ret);

-}

-

-size_t DARSHAN_DECL(fwrite)(const void *ptr, size_t size, size_t nmemb, FILE *stream)

-{

-    size_t ret;

-    int aligned_flag = 0;

-    double tm1, tm2;

-

-    MAP_OR_FAIL(fwrite);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_lseek(fd, offset, whence);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if((unsigned long)ptr % darshan_mem_alignment == 0)

-        aligned_flag = 1;

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret >= 0)

+    {

+        POSIX_LOCK();

+        posix_runtime_initialize();

+        file = posix_file_by_fd(fd);

+        if(file)

+        {

+            file->offset = ret;

+            DARSHAN_TIMER_INC_NO_OVERLAP(

+                file->file_record->fcounters[POSIX_F_META_TIME],

+                tm1, tm2, file->last_meta_end);

+            file->file_record->counters[POSIX_SEEKS] += 1;

+        }

+        POSIX_UNLOCK();

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_fwrite(ptr, size, nmemb, stream);

-    tm2 = darshan_wtime();

-    CP_LOCK();

-    if(ret > 0)

-        CP_RECORD_WRITE(size*ret, fileno(stream), (size*nmemb), 0, 0, aligned_flag, 1, tm1, tm2);

-    else

-        CP_RECORD_WRITE(ret, fileno(stream), 0, 0, 0, aligned_flag, 1, tm1, tm2);

-    CP_UNLOCK();

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-off64_t DARSHAN_DECL(lseek64)(int fd, off64_t offset, int whence)

</span><span style="color: #000000;background-color: #ddffdd">+off_t DARSHAN_DECL(lseek64)(int fd, off_t offset, int whence)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    off64_t ret;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    off_t ret;

+    struct posix_file_runtime* file;

</span>     double tm1, tm2;

     MAP_OR_FAIL(lseek64);

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

</span>     ret = __real_lseek64(fd, offset, whence);

<span style="color: #000000;background-color: #ffdddd">-    tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    tm2 = darshan_core_wtime();

+

</span>     if(ret >= 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        CP_LOCK();

-        file = darshan_file_by_fd(fd);

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        file = posix_file_by_fd(fd);

</span>         if(file)

         {

             file->offset = ret;

<span style="color: #000000;background-color: #ffdddd">-            CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME);

-            CP_INC(file, CP_POSIX_SEEKS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+            DARSHAN_TIMER_INC_NO_OVERLAP(

+                file->file_record->fcounters[POSIX_F_META_TIME],

+                tm1, tm2, file->last_meta_end);

+            file->file_record->counters[POSIX_SEEKS] += 1;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_UNLOCK();

</span>     }

<span style="color: #000000;background-color: #ddffdd">+

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-off_t DARSHAN_DECL(lseek)(int fd, off_t offset, int whence)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(fseek)(FILE *stream, long offset, int whence)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    off_t ret;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

+    struct posix_file_runtime* file;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(lseek);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fseek);

+

+    tm1 = darshan_core_wtime();

+    ret = __real_fseek(stream, offset, whence);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_lseek(fd, offset, whence);

-    tm2 = darshan_wtime();

</span>     if(ret >= 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        CP_LOCK();

-        file = darshan_file_by_fd(fd);

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        file = posix_file_by_fd(fileno(stream));

</span>         if(file)

         {

<span style="color: #000000;background-color: #ffdddd">-            file->offset = ret;

-            CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME);

-            CP_INC(file, CP_POSIX_SEEKS, 1);

</span><span style="color: #000000;background-color: #ddffdd">+            file->offset = ftell(stream);

+            DARSHAN_TIMER_INC_NO_OVERLAP(

+                file->file_record->fcounters[POSIX_F_META_TIME],

+                tm1, tm2, file->last_meta_end);

+            file->file_record->counters[POSIX_FSEEKS] += 1;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_UNLOCK();

</span>     }

<span style="color: #000000;background-color: #ddffdd">+

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-ssize_t DARSHAN_DECL(aio_return64)(struct aiocb64 *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__xstat)(int vers, const char *path, struct stat *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    double tm2;

-    struct darshan_aio_tracker *tmp;

-    int aligned_flag = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_return64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__xstat);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_return64(aiocbp);

-    tm2 = darshan_wtime();

-    tmp = darshan_aio_tracker_del(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___xstat(vers, path, buf);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(tmp)

-    {

-        if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)

-            aligned_flag = 1;

-        CP_LOCK();

-        if(aiocbp->aio_lio_opcode == LIO_WRITE)

-        {

-            CP_RECORD_WRITE(ret, aiocbp->aio_fildes, aiocbp->aio_nbytes,

-                1, aiocbp->aio_offset, aligned_flag, 0, tmp->tm1, tm2);

-        }

-        if(aiocbp->aio_lio_opcode == LIO_READ)

-        {

-            CP_RECORD_READ(ret, aiocbp->aio_fildes, aiocbp->aio_nbytes,

-                1, aiocbp->aio_offset, aligned_flag, 0, tmp->tm1, tm2);

-        }

-        CP_UNLOCK();

-        free(tmp);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-ssize_t DARSHAN_DECL(aio_return)(struct aiocb *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__xstat64)(int vers, const char *path, struct stat64 *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    double tm2;

-    struct darshan_aio_tracker *tmp;

-    int aligned_flag = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_return);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__xstat64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_return(aiocbp);

-    tm2 = darshan_wtime();

-    tmp = darshan_aio_tracker_del(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___xstat64(vers, path, buf);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(tmp)

-    {

-        if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)

-            aligned_flag = 1;

-        CP_LOCK();

-        if(aiocbp->aio_lio_opcode == LIO_WRITE)

-        {

-            CP_RECORD_WRITE(ret, aiocbp->aio_fildes, aiocbp->aio_nbytes,

-                1, aiocbp->aio_offset, aligned_flag, 0, tmp->tm1, tm2);

-        }

-        if(aiocbp->aio_lio_opcode == LIO_READ)

-        {

-            CP_RECORD_READ(ret, aiocbp->aio_fildes, aiocbp->aio_nbytes,

-                1, aiocbp->aio_offset, aligned_flag, 0, tmp->tm1, tm2);

-        }

-        CP_UNLOCK();

-        free(tmp);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(lio_listio)(int mode, struct aiocb *const aiocb_list[],

-    int nitems, struct sigevent *sevp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__lxstat)(int vers, const char *path, struct stat *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    int i;

</span><span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(lio_listio);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__lxstat);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_lio_listio(mode, aiocb_list, nitems, sevp);

-    if(ret == 0)

-    {

-        for(i=0; i<nitems; i++)

-        {

-            darshan_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);        

-        }

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___lxstat(vers, path, buf);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(lio_listio64)(int mode, struct aiocb64 *const aiocb_list[],

-    int nitems, struct sigevent *sevp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__lxstat64)(int vers, const char *path, struct stat64 *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    int i;

</span><span style="color: #000000;background-color: #ddffdd">+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(lio_listio64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__lxstat64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_lio_listio64(mode, aiocb_list, nitems, sevp);

-    if(ret == 0)

-    {

-        for(i=0; i<nitems; i++)

-        {

-            darshan_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);        

-        }

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___lxstat64(vers, path, buf);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    POSIX_LOOKUP_RECORD_STAT(path, buf, tm1, tm2);

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(aio_write64)(struct aiocb64 *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__fxstat)(int vers, int fd, struct stat *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file;

+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_write64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__fxstat);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_write64(aiocbp);

-    if(ret == 0)

-        darshan_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___fxstat(vers, fd, buf);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

+    {

+        POSIX_RECORD_STAT(file, buf, tm1, tm2);

+    }

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(aio_write)(struct aiocb *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(__fxstat64)(int vers, int fd, struct stat64 *buf)

</span> {

     int ret;

<span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file;

+    double tm1, tm2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_write);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(__fxstat64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_write(aiocbp);

-    if(ret == 0)

-        darshan_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real___fxstat64(vers, fd, buf);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0 || !S_ISREG(buf->st_mode))

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

+    {

+        POSIX_RECORD_STAT(file, buf, tm1, tm2);

+    }

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(aio_read64)(struct aiocb64 *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+void* DARSHAN_DECL(mmap)(void *addr, size_t length, int prot, int flags,

+    int fd, off_t offset)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    void* ret;

+    struct posix_file_runtime* file;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_read64);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(mmap);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_read64(aiocbp);

-    if(ret == 0)

-        darshan_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_mmap(addr, length, prot, flags, fd, offset);

+    if(ret == MAP_FAILED)

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

+    {

+        file->file_record->counters[POSIX_MMAPS] += 1;

+    }

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(aio_read)(struct aiocb *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+void* DARSHAN_DECL(mmap64)(void *addr, size_t length, int prot, int flags,

+    int fd, off64_t offset)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    void* ret;

+    struct posix_file_runtime* file;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(aio_read);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(mmap64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = __real_aio_read(aiocbp);

-    if(ret == 0)

-        darshan_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_mmap64(addr, length, prot, flags, fd, offset);

+    if(ret == MAP_FAILED)

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

+    {

+        file->file_record->counters[POSIX_MMAPS] += 1;

+    }

+    POSIX_UNLOCK();

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-int DARSHAN_DECL(fseek)(FILE *stream, long offset, int whence)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(fsync)(int fd)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file;

</span>     double tm1, tm2;

<span style="color: #000000;background-color: #ffdddd">-    MAP_OR_FAIL(fseek);

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fsync);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tm1 = darshan_wtime();

-    ret = __real_fseek(stream, offset, whence);

-    tm2 = darshan_wtime();

-    if(ret >= 0)

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_fsync(fd);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0)

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        CP_LOCK();

-        file = darshan_file_by_fd(fileno(stream));

-        if(file)

-        {

-            file->offset = ret;

-            CP_F_INC_NO_OVERLAP(file, tm1, tm2, file->last_posix_meta_end, CP_F_POSIX_META_TIME);

-            CP_INC(file, CP_POSIX_FSEEKS, 1);

-        }

-        CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+        DARSHAN_TIMER_INC_NO_OVERLAP(

+            file->file_record->fcounters[POSIX_F_WRITE_TIME],

+            tm1, tm2, file->last_write_end);

+        file->file_record->counters[POSIX_FSYNCS] += 1;

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();

+

</span>     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-void darshan_finalize(struct darshan_job_runtime* job)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(fdatasync)(int fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    if(!job)

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

+    struct posix_file_runtime* file;

+    double tm1, tm2;

+

+    MAP_OR_FAIL(fdatasync);

+

+    tm1 = darshan_core_wtime();

+    ret = __real_fdatasync(fd);

+    tm2 = darshan_core_wtime();

+

+    if(ret < 0)

+        return(ret);

+

+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        DARSHAN_TIMER_INC_NO_OVERLAP(

+            file->file_record->fcounters[POSIX_F_WRITE_TIME],

+            tm1, tm2, file->last_write_end);

+        file->file_record->counters[POSIX_FDSYNCS] += 1;

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    free(job);

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-void darshan_initialize(int argc, char** argv,  int nprocs, int rank)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(close)(int fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int i;

-    char* disable;

-    char* disable_timing;

-    char* envstr;

-    char* truncate_string = "<TRUNCATED>";

-    int truncate_offset;

-    int chars_left = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file;

+    double tm1, tm2;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    int tmpval;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    disable = getenv("DARSHAN_DISABLE");

-    if(disable)

-    {

-        /* turn off tracing */

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(close);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    disable_timing = getenv("DARSHAN_DISABLE_TIMING");

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_close(fd);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(darshan_global_job != NULL)

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        file->last_byte_written = 0;

+        file->last_byte_read = 0;

+        file->file_record->fcounters[POSIX_F_CLOSE_TIMESTAMP] =

+            darshan_core_wtime();

+        DARSHAN_TIMER_INC_NO_OVERLAP(

+            file->file_record->fcounters[POSIX_F_META_TIME],

+            tm1, tm2, file->last_meta_end);

+        posix_file_close_fd(fd);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();    

</span> 

<span style="color: #000000;background-color: #ffdddd">-    #if (__CP_MEM_ALIGNMENT < 1)

-        #error Darshan must be configured with a positive value for --with-mem-align

-    #endif

-    envstr = getenv("DARSHAN_MEMALIGN");

-    if (envstr)

-    {

-        ret = sscanf(envstr, "%d", &tmpval);

-        /* silently ignore if the env variable is set poorly */

-        if(ret == 1 && tmpval > 0)

-        {

-            darshan_mem_alignment = tmpval;

-        }

-    }

-    else

-    {

-        darshan_mem_alignment = __CP_MEM_ALIGNMENT;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* avoid floating point errors on faulty input */

-    if (darshan_mem_alignment < 1)

-    {

-        darshan_mem_alignment = 1;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(fclose)(FILE *fp)

+{

+    struct posix_file_runtime* file;

+    int fd = fileno(fp);

+    double tm1, tm2;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* allocate structure to track darshan_global_job information */

-    darshan_global_job = malloc(sizeof(*darshan_global_job));

-    if(!darshan_global_job)

-    {

-        return;

-    }

-    memset(darshan_global_job, 0, sizeof(*darshan_global_job));

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(fclose);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(disable_timing)

-    {

-        darshan_global_job->flags |= CP_FLAG_NOTIMING;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    tm1 = darshan_core_wtime();

+    ret = __real_fclose(fp);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* set up file records */

-    for(i=0; i<CP_MAX_FILES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    file = posix_file_by_fd(fd);

+    if(file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        darshan_global_job->file_runtime_array[i].log_file = 

-            &darshan_global_job->file_array[i];

</span><span style="color: #000000;background-color: #ddffdd">+        file->last_byte_written = 0;

+        file->last_byte_read = 0;

+        file->file_record->fcounters[POSIX_F_CLOSE_TIMESTAMP] =

+            darshan_core_wtime();

+        DARSHAN_TIMER_INC_NO_OVERLAP(

+            file->file_record->fcounters[POSIX_F_META_TIME],

+            tm1, tm2, file->last_meta_end);

+        posix_file_close_fd(fd);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    strcpy(darshan_global_job->log_job.version_string, CP_VERSION);

-    darshan_global_job->log_job.magic_nr = CP_MAGIC_NR;

-    darshan_global_job->log_job.uid = getuid();

-    darshan_global_job->log_job.start_time = time(NULL);

-    darshan_global_job->log_job.nprocs = nprocs;

-    darshan_global_job->wtime_offset = posix_wtime();

-    my_rank = rank;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* record exe and arguments */

-    for(i=0; i<argc; i++)

-    {

-        chars_left = CP_EXE_LEN-strlen(darshan_global_job->exe);

-        strncat(darshan_global_job->exe, argv[i], chars_left);

-        if(i < (argc-1))

-        {

-            chars_left = CP_EXE_LEN-strlen(darshan_global_job->exe);

-            strncat(darshan_global_job->exe, " ", chars_left);

-        }

-    }

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(aio_read)(struct aiocb *aiocbp)

+{

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* if we don't see any arguments, then use glibc symbol to get

-     * program name at least (this happens in fortran)

-     */

-    if(argc == 0)

-    {

-        chars_left = CP_EXE_LEN-strlen(darshan_global_job->exe);

-        strncat(darshan_global_job->exe, __progname_full, chars_left);

-        chars_left = CP_EXE_LEN-strlen(darshan_global_job->exe);

-        strncat(darshan_global_job->exe, " <unknown args>", chars_left);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_read);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(chars_left == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_aio_read(aiocbp);

+    if(ret == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* we ran out of room; mark that string was truncated */

-        truncate_offset = CP_EXE_LEN - strlen(truncate_string);

-        sprintf(&darshan_global_job->exe[truncate_offset], "%s", 

-            truncate_string);

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

+        POSIX_UNLOCK();

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* collect information about command line and 

-     * mounted file systems 

-     */

-    darshan_global_job->trailing_data = 

-        darshan_get_exe_and_mounts(darshan_global_job);

-

-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_condense()

- *

- * collapses all file statistics into a single unified set of counters; used

- * when we have opened too many files to track independently

- */

-void darshan_condense(void)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(aio_write)(struct aiocb *aiocbp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* base_file;

-    struct darshan_file_runtime* iter_file;

-    int i;

-    int j;

-

-    if(!darshan_global_job)

-        return;

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    base_file = &darshan_global_job->file_runtime_array[0];

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_write);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* iterate through files */

-    for(j=1; j<darshan_global_job->file_count; j++)

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_aio_write(aiocbp);

+    if(ret == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        iter_file = &darshan_global_job->file_runtime_array[j];

-

-        /* iterate through records */

-        for(i=0; i<CP_NUM_INDICES; i++)

-        {

-            switch(i)

-            {

-                /* NOTE: several fields cease to make sense if the records

-                 * have been condensed.  Just let them get summed anyway.

-                 */

-                /* TODO: double check this */

-

-                /* keep up with global maxes in case they are helpful */

-                case CP_MAX_BYTE_READ:

-                case CP_MAX_BYTE_WRITTEN:

-                    CP_MAX(base_file, i, CP_VALUE(iter_file, i));

-                    break;

-

-                /* do nothing with these; they are handled in the floating

-                 * point loop 

-                 */

-                case CP_MAX_WRITE_TIME_SIZE:

-                case CP_MAX_READ_TIME_SIZE:

-                    break;

-

-                /* pick one */

-                case CP_DEVICE:

-                case CP_SIZE_AT_OPEN:

-                    CP_SET(base_file, i, CP_VALUE(iter_file, i));

-                    break;

-

-                /* most records can simply be added */

-                default:

-                    CP_INC(base_file, i, CP_VALUE(iter_file, i));

-                    break;

-            }

-        }

-        for(i=0; i<CP_F_NUM_INDICES; i++)

-        {

-            switch(i)

-            {

-                case CP_F_MAX_WRITE_TIME:

-                    if(CP_F_VALUE(iter_file, i) > CP_F_VALUE(base_file, i))

-                    {

-                        CP_F_SET(base_file, i, CP_F_VALUE(iter_file, i));

-                        CP_SET(base_file, CP_MAX_WRITE_TIME_SIZE, 

-                            CP_VALUE(iter_file, CP_MAX_WRITE_TIME_SIZE));

-                    }

-                    break;

-                case CP_F_MAX_READ_TIME:

-                    if(CP_F_VALUE(iter_file, i) > CP_F_VALUE(base_file, i))

-                    {

-                        CP_F_SET(base_file, i, CP_F_VALUE(iter_file, i));

-                        CP_SET(base_file, CP_MAX_READ_TIME_SIZE, 

-                            CP_VALUE(iter_file, CP_MAX_READ_TIME_SIZE));

-                    }

-                    break;

-                default:

-                    CP_F_SET(base_file, i, CP_F_VALUE(iter_file, i) + CP_F_VALUE(base_file, i));

-                    break;

-            }

-        }

-

-        if(base_file->aio_list_tail)

-        {

-            /* base has an aio list already; add on to it */

-            assert(base_file->aio_list_head);

-            base_file->aio_list_tail->next = iter_file->aio_list_head;

-            if(iter_file->aio_list_tail)

-                base_file->aio_list_tail = iter_file->aio_list_tail;

-        }

-        else

-        {

-            /* take on list from iter */

-            base_file->aio_list_head = iter_file->aio_list_head;

-            base_file->aio_list_tail = iter_file->aio_list_tail;

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

+        POSIX_UNLOCK();

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    

-    base_file->log_file->hash = 0;

-    

-    darshan_global_job->flags |= CP_FLAG_CONDENSED;

-    darshan_global_job->file_count = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* clear hash tables for safety */

-    memset(darshan_global_job->name_table, 0, CP_HASH_SIZE*sizeof(struct darshan_file_runtime*));

-    memset(darshan_global_job->handle_table, 0, CP_HASH_SIZE*sizeof(*darshan_global_job->handle_table));

-    

-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_name_setfd(const char* name, int fd)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(aio_read64)(struct aiocb64 *aiocbp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_name_sethandle(name, &fd, sizeof(fd), DARSHAN_FD);

-    return(tmp_file);

-}

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_read64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static void darshan_file_close_fd(int fd)

-{

-    darshan_file_closehandle(&fd, sizeof(fd), DARSHAN_FD);

-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_aio_read64(aiocbp);

+    if(ret == 0)

+    {

+        POSIX_LOCK();

+        posix_runtime_initialize();

+        posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

+        POSIX_UNLOCK();

+    }

+

+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_runtime* darshan_file_by_fd(int fd)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(aio_write64)(struct aiocb64 *aiocbp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_file = darshan_file_by_handle(&fd, sizeof(fd), DARSHAN_FD);

-    

-    return(tmp_file);

-}

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_write64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static int access_comparison(const void* a_p, const void* b_p)

-{

-    const struct cp_access_counter* a = a_p;

-    const struct cp_access_counter* b = b_p;

-

-    if(a->size < b->size)

-        return(-1);

-    if(a->size > b->size)

-        return(1);

-    return(0);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_aio_write64(aiocbp);

+    if(ret == 0)

+    {

+        POSIX_LOCK();

+        posix_runtime_initialize();

+        posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);

+        POSIX_UNLOCK();

+    }

+

+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* cp_access_counter()

- *

- * records the occurance of a particular access size for a file,

- * current implementation uses glibc red black tree

- */

-static void cp_access_counter(struct darshan_file_runtime* file, ssize_t size, enum cp_counter_type type)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(aio_return)(struct aiocb *aiocbp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct cp_access_counter* counter;

-    struct cp_access_counter* found;

-    void* tmp;

-    void** root;

-    int* count;

-    struct cp_access_counter tmp_counter;

-

-    /* don't count sizes or strides of 0 */

-    if(size == 0)

-        return;

-    

-    switch(type)

-    {

-        case CP_COUNTER_ACCESS:

-            root = &file->access_root;

-            count = &file->access_count;

-            break;

-        case CP_COUNTER_STRIDE:

-            root = &file->stride_root;

-            count = &file->stride_count;

-            break;

-        default:

-            return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

+    double tm2;

+    struct posix_aio_tracker *tmp;

+    int aligned_flag = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* check to see if this size is already recorded */

-    tmp_counter.size = size;

-    tmp_counter.freq = 1;

-    tmp = tfind(&tmp_counter, root, access_comparison);

-    if(tmp)

-    {

-        found = *(struct cp_access_counter**)tmp;

-        found->freq++;

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_return);

+

+    ret = __real_aio_return(aiocbp);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* we can add a new one as long as we haven't hit the limit */

-    if(*count < CP_MAX_ACCESS_COUNT_RUNTIME)

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    tmp = posix_aio_tracker_del(aiocbp->aio_fildes, aiocbp);

+    if (tmp)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        counter = malloc(sizeof(*counter));

-        if(!counter)

</span><span style="color: #000000;background-color: #ddffdd">+        if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)

+            aligned_flag = 1;

+        if(aiocbp->aio_lio_opcode == LIO_WRITE)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            return;

</span><span style="color: #000000;background-color: #ddffdd">+            POSIX_RECORD_WRITE(ret, aiocbp->aio_fildes,

+                1, aiocbp->aio_offset, aligned_flag, 0,

+                tmp->tm1, tm2);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-

-        counter->size = size;

-        counter->freq = 1;

-

-        tmp = tsearch(counter, root, access_comparison);

-        found = *(struct cp_access_counter**)tmp;

-        /* if we get a new answer out here we are in trouble; this was

-         * already checked with the tfind()

-         */

-        assert(found == counter);

-

-        (*count)++;

</span><span style="color: #000000;background-color: #ddffdd">+        else if(aiocbp->aio_lio_opcode == LIO_READ)

+        {

+            POSIX_RECORD_READ(ret, aiocbp->aio_fildes,

+                1, aiocbp->aio_offset, aligned_flag, 0,

+                tmp->tm1, tm2);

+        }

+        free(tmp);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* NOTE: we disable internal benchmarking routines when building shared

- * libraries so that when Darshan is loaded with LD_PRELOAD it does not

- * depend on MPI routines.

- */

-#ifndef DARSHAN_PRELOAD

-void darshan_shutdown_bench(int argc, char** argv, int rank, int nprocs)

</span><span style="color: #000000;background-color: #ddffdd">+ssize_t DARSHAN_DECL(aio_return64)(struct aiocb64 *aiocbp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int* fd_array;

-    int64_t* size_array;

-    int i;

-    int nfiles;

-    char path[256];

-    int iters;

-    

-    /* combinations to build:

-     * - 1 unique file per proc

-     * - 1 shared file per proc

-     * - 1024 unique file per proc

-     * - 1024 shared per proc

-     */

-

-    srand(rank);

-    fd_array = malloc(sizeof(int)*CP_MAX_FILES);

-    size_array = malloc(sizeof(int64_t)*CP_MAX_ACCESS_COUNT_RUNTIME);

-

-    assert(fd_array&&size_array);

-

-    for(i=0; i<CP_MAX_FILES; i++)

-        fd_array[i] = i;

-    for(i=0; i<CP_MAX_ACCESS_COUNT_RUNTIME; i++)

-        size_array[i] = rand();

-

-    /* clear out existing stuff */

-    darshan_walk_file_accesses(darshan_global_job);

-    darshan_finalize(darshan_global_job);

-    darshan_global_job = NULL;

-

-    /***********************************************************/

-    /* reset darshan to start clean */

-    darshan_initialize(argc, argv, nprocs, rank);

-

-    /* populate one unique file per proc */

-    nfiles = 1;

-    iters = 1;

-    for(i=0; i<nfiles; i++)

-    {

-        sprintf(path, "%d-%d", i, rank);

-        CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

+    double tm2;

+    struct posix_aio_tracker *tmp;

+    int aligned_flag = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for(i=0; i<iters; i++)

-    {

-        CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(aio_return64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(rank == 0)

-        printf("# 1 unique file per proc\n");

-    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

-    darshan_shutdown(1);

-    darshan_global_job = NULL;

-

-    /***********************************************************/

-    /* reset darshan to start clean */

-    sleep(1);

-    darshan_initialize(argc, argv, nprocs, rank);

-

-    /* populate one shared file per proc */

-    nfiles = 1;

-    iters = 1;

-    for(i=0; i<nfiles; i++)

-    {

-        sprintf(path, "%d", i);

-        CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_aio_return64(aiocbp);

+    tm2 = darshan_core_wtime();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for(i=0; i<iters; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    posix_runtime_initialize();

+    tmp = posix_aio_tracker_del(aiocbp->aio_fildes, aiocbp);

+    if (tmp)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

</span><span style="color: #000000;background-color: #ddffdd">+        if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)

+            aligned_flag = 1;

+        if(aiocbp->aio_lio_opcode == LIO_WRITE)

+        {

+            POSIX_RECORD_WRITE(ret, aiocbp->aio_fildes,

+                1, aiocbp->aio_offset, aligned_flag, 0,

+                tmp->tm1, tm2);

+        }

+        else if(aiocbp->aio_lio_opcode == LIO_READ)

+        {

+            POSIX_RECORD_READ(ret, aiocbp->aio_fildes,

+                1, aiocbp->aio_offset, aligned_flag, 0,

+                tmp->tm1, tm2);

+        }

+        free(tmp);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    POSIX_UNLOCK();

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(rank == 0)

-        printf("# 1 shared file across procs\n");

-    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

-    darshan_shutdown(1);

-    darshan_global_job = NULL;

-

-    /***********************************************************/

-    /* reset darshan to start clean */

-    sleep(1);

-    darshan_initialize(argc, argv, nprocs, rank);

-

-    /* populate 1024 unique file per proc */

-    nfiles = 1024;

-    iters = 1024;

-    for(i=0; i<nfiles; i++)

-    {

-        sprintf(path, "%d-%d", i, rank);

-        CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for(i=0; i<iters; i++)

-    {

-        CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(lio_listio)(int mode, struct aiocb *const aiocb_list[],

+    int nitems, struct sigevent *sevp)

+{

+    int ret;

+    int i;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(rank == 0)

-        printf("# 1024 unique files per proc\n");

-    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

-    darshan_shutdown(1);

-    darshan_global_job = NULL;

-

-    /***********************************************************/

-    /* reset darshan to start clean */

-    sleep(1);

-    darshan_initialize(argc, argv, nprocs, rank);

-

-    /* populate 1024 shared file per proc */

-    nfiles = 1024;

-    iters = 1024;

-    for(i=0; i<nfiles; i++)

-    {

-        sprintf(path, "%d", i);

-        CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(lio_listio);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for(i=0; i<iters; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_lio_listio(mode, aiocb_list, nitems, sevp);

+    if(ret == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        for(i = 0; i < nitems; i++)

+        {

+            posix_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);

+        }

+        POSIX_UNLOCK();

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(rank == 0)

-        printf("# 1024 shared files across procs\n");

-    DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);

-    darshan_shutdown(1);

-    darshan_global_job = NULL;

-

-    darshan_initialize(argc, argv, nprocs, rank);

-

-    free(fd_array);

-    free(size_array);

-

-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-#endif

</span> 

<span style="color: #000000;background-color: #ffdddd">-void darshan_search_bench(int argc, char** argv, int iters)

</span><span style="color: #000000;background-color: #ddffdd">+int DARSHAN_DECL(lio_listio64)(int mode, struct aiocb64 *const aiocb_list[],

+    int nitems, struct sigevent *sevp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int* fd_array;

-    int64_t* size_array;

-    int i,j;

-    int skip = 32;

-    int nfiles;

-    char path[256];

-    double tm1, tm2;

-    

-    fd_array = malloc(sizeof(int)*CP_MAX_FILES);

-    size_array = malloc(sizeof(int64_t)*CP_MAX_ACCESS_COUNT_RUNTIME);

-

-    assert(fd_array&&size_array);

-

-    for(i=0; i<CP_MAX_FILES; i++)

-        fd_array[i] = i;

-    for(i=0; i<CP_MAX_ACCESS_COUNT_RUNTIME; i++)

-        size_array[i] = rand();

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

+    int i;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    printf("#<iters>\t<numfiles>\t<numsizes>\t<total time>\t<per iter>\n");

</span><span style="color: #000000;background-color: #ddffdd">+    MAP_OR_FAIL(lio_listio64);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for(j=0; j<2; j++)

</span><span style="color: #000000;background-color: #ddffdd">+    ret = __real_lio_listio64(mode, aiocb_list, nitems, sevp);

+    if(ret == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* warm up */

-        /* reset darshan to start clean */

-        darshan_walk_file_accesses(darshan_global_job);

-        darshan_finalize(darshan_global_job);

-        darshan_global_job = NULL;

-        darshan_initialize(argc, argv, 1, 0);

-

-        nfiles = 1;

-        /* populate entries for each file */

-        for(i=0; i<nfiles; i++)

</span><span style="color: #000000;background-color: #ddffdd">+        POSIX_LOCK();

+        posix_runtime_initialize();

+        for(i = 0; i < nitems; i++)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            sprintf(path, "%d", i);

-            CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

</span><span style="color: #000000;background-color: #ddffdd">+            posix_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);

</span>         }

<span style="color: #000000;background-color: #ddffdd">+        POSIX_UNLOCK();

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-        for(i=0; i<iters; i++)

-        {

-            if(j==0)

-            {

-                CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

-            }

-            else

-            {

-                CP_RECORD_WRITE(size_array[0], fd_array[i%nfiles], size_array[0], 0, 0, 1, 0, 1, 2);

-            }

-        }

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-        /* real timing */

-        for(nfiles=0; nfiles<=CP_MAX_FILES; nfiles += skip)

-        {

-            if(nfiles == 0)

-                nfiles = 1;

</span><span style="color: #000000;background-color: #ddffdd">+/**********************************************************

+ * Internal functions for manipulating POSIX module state *

+ **********************************************************/

</span> 

<span style="color: #000000;background-color: #ffdddd">-            /* reset darshan to start clean */

-            darshan_walk_file_accesses(darshan_global_job);

-            darshan_finalize(darshan_global_job);

-            darshan_global_job = NULL;

-            darshan_initialize(argc, argv, 1, 0);

</span><span style="color: #000000;background-color: #ddffdd">+/* initialize internal POSIX module data structures and register with darshan-core */

+static void posix_runtime_initialize()

+{

+    int mem_limit;

+    struct darshan_module_funcs posix_mod_fns =

+    {

+        .begin_shutdown = &posix_begin_shutdown,

+        .get_output_data = &posix_get_output_data,

+        .shutdown = &posix_shutdown

+    };

</span> 

<span style="color: #000000;background-color: #ffdddd">-            /* populate entries for each file */

-            for(i=0; i<nfiles; i++)

-            {

-                sprintf(path, "%d", i);

-                CP_RECORD_OPEN(fd_array[i], path, 777, 0, 0, 0);

-            }

</span><span style="color: #000000;background-color: #ddffdd">+    /* don't do anything if already initialized or instrumenation is disabled */

+    if(posix_runtime || instrumentation_disabled)

+        return;

</span> 

<span style="color: #000000;background-color: #ffdddd">-            tm1 = darshan_wtime();

-            for(i=0; i<iters; i++)

-            {

-                if(j==0)

-                {

-                    CP_RECORD_WRITE(size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], fd_array[i%nfiles], size_array[(i/nfiles)%CP_MAX_ACCESS_COUNT_RUNTIME], 0, 0, 1, 0, 1, 2);

-                }

-                else

-                {

-                    CP_RECORD_WRITE(size_array[0], fd_array[i%nfiles], size_array[0], 0, 0, 1, 0, 1, 2);

-                }

-            }

-            tm2 = darshan_wtime();

</span><span style="color: #000000;background-color: #ddffdd">+    /* register the posix module with darshan core */

+    darshan_core_register_module(

+        DARSHAN_POSIX_MOD,

+        &posix_mod_fns,

+        &my_rank,

+        &mem_limit,

+        &darshan_mem_alignment);

</span> 

<span style="color: #000000;background-color: #ffdddd">-            /* printf("#<iters>\t<numfiles>\t<numsizes>\t<total time>\t<per iter>\n"); */

-            printf("%d\t%d\t%d\t%f\t%.12f\n", iters, nfiles, (j==0?CP_MAX_ACCESS_COUNT_RUNTIME:1), tm2-tm1, (tm2-tm1)/iters);

</span><span style="color: #000000;background-color: #ddffdd">+    /* return if no memory assigned by darshan core */

+    if(mem_limit == 0)

+        return;

</span> 

<span style="color: #000000;background-color: #ffdddd">-            if(nfiles == 1)

-                nfiles = 0;

-        }

</span><span style="color: #000000;background-color: #ddffdd">+    posix_runtime = malloc(sizeof(*posix_runtime));

+    if(!posix_runtime)

+        return;

+    memset(posix_runtime, 0, sizeof(*posix_runtime));

+

+    /* set maximum number of file records according to max memory limit */

+    /* NOTE: maximum number of records is based on the size of a posix file record */

+    /* TODO: should we base memory usage off file record or total runtime structure sizes? */

+    posix_runtime->file_array_size = mem_limit / sizeof(struct darshan_posix_file);

+    posix_runtime->file_array_ndx = 0;

+

+    /* allocate array of runtime file records */

+    posix_runtime->file_runtime_array = malloc(posix_runtime->file_array_size *

+                                               sizeof(struct posix_file_runtime));

+    posix_runtime->file_record_array = malloc(posix_runtime->file_array_size *

+                                              sizeof(struct darshan_posix_file));

+    if(!posix_runtime->file_runtime_array || !posix_runtime->file_record_array)

+    {

+        posix_runtime->file_array_size = 0;

+        return;

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    memset(posix_runtime->file_runtime_array, 0, posix_runtime->file_array_size *

+           sizeof(struct posix_file_runtime));

+    memset(posix_runtime->file_record_array, 0, posix_runtime->file_array_size *

+           sizeof(struct darshan_posix_file));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    free(fd_array);

-    free(size_array);

-}

-

-static double posix_wtime(void)

-{

-    return DARSHAN_MPI_CALL(PMPI_Wtime)();

</span><span style="color: #000000;background-color: #ddffdd">+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-double darshan_wtime(void)

</span><span style="color: #000000;background-color: #ddffdd">+/* get a POSIX file record for the given file path */

+static struct posix_file_runtime* posix_file_by_name(const char *name)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    if(!darshan_global_job || darshan_global_job->flags & CP_FLAG_NOTIMING)

-    {

-        return(0);

-    }

-    

-    return(posix_wtime());

-}

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime *file = NULL;

+    char *newname = NULL;

+    darshan_record_id file_id;

+    int file_alignment;

+    int limit_flag;

</span> 

<span style="color: #000000;background-color: #ffdddd">-struct darshan_file_runtime* darshan_file_by_name(const char* name)

-{

-    struct darshan_file_runtime* tmp_file;

-    uint64_t tmp_hash = 0;

-    char* suffix_pointer;

-    int hash_index;

-    char* newname = NULL;

-    int64_t device_id;

-    int64_t block_size;

-

-    if(!darshan_global_job)

</span><span style="color: #000000;background-color: #ddffdd">+    if(!posix_runtime || instrumentation_disabled)

</span>         return(NULL);

<span style="color: #000000;background-color: #ffdddd">-    /* if we have already condensed the data, then just hand the first file

-     * back

-     */

-    if(darshan_global_job->flags & CP_FLAG_CONDENSED)

-    {

-        return(&darshan_global_job->file_runtime_array[0]);

-    }

-

-    /* try to construct absolute path */

-    newname = clean_path(name);

</span><span style="color: #000000;background-color: #ddffdd">+    newname = darshan_clean_file_path(name);

</span>     if(!newname)

         newname = (char*)name;

<span style="color: #000000;background-color: #ffdddd">-    tmp_hash = darshan_hash((void*)newname, strlen(newname), 0);

</span><span style="color: #000000;background-color: #ddffdd">+    limit_flag = (posix_runtime->file_array_ndx >= posix_runtime->file_array_size);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* search hash table */

-    hash_index = tmp_hash & CP_HASH_MASK;

-    tmp_file = darshan_global_job->name_table[hash_index];

-    while(tmp_file)

-    {

-        if(tmp_file->log_file->hash == tmp_hash)

-        {

-            if(newname != name)

-                free(newname);

-            return(tmp_file);

-        }

-        tmp_file = tmp_file->name_next;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    /* get a unique id for this file from darshan core */

+    darshan_core_register_record(

+        (void*)newname,

+        strlen(newname),

+        DARSHAN_POSIX_MOD,

+        1,

+        limit_flag,

+        &file_id,

+        &file_alignment);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* see if we need to condense */

-    if(darshan_global_job->file_count >= CP_MAX_FILES)

</span><span style="color: #000000;background-color: #ddffdd">+    /* the file record id is set to 0 if no memory is available for tracking

+     * new records -- just fall through and ignore this record

+     */

+    if(file_id == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        darshan_condense();

</span>         if(newname != name)

             free(newname);

<span style="color: #000000;background-color: #ffdddd">-        return(&darshan_global_job->file_runtime_array[0]);

</span><span style="color: #000000;background-color: #ddffdd">+        return(NULL);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* new, unique file */

-    tmp_file = &darshan_global_job->file_runtime_array[darshan_global_job->file_count];

-

-    darshan_mnt_id_from_path(newname, &device_id, &block_size);

-    CP_SET(tmp_file, CP_DEVICE, device_id);

-    CP_SET(tmp_file, CP_FILE_ALIGNMENT, block_size);

-    CP_SET(tmp_file, CP_MEM_ALIGNMENT, darshan_mem_alignment);

-    tmp_file->log_file->hash = tmp_hash;

-

-    /* record last N characters of file name too */

-    suffix_pointer = (char*)newname;

-    if(strlen(newname) > CP_NAME_SUFFIX_LEN)

</span><span style="color: #000000;background-color: #ddffdd">+    /* search the hash table for this file record, and return if found */

+    HASH_FIND(hlink, posix_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);

+    if(file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        suffix_pointer += (strlen(newname) - CP_NAME_SUFFIX_LEN);

</span><span style="color: #000000;background-color: #ddffdd">+        if(newname != name)

+            free(newname);

+        return(file);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    strcpy(tmp_file->log_file->name_suffix, suffix_pointer); 

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* if the "stat at open" functionality is disabled, then go ahead and

-     * mark certain counters with invalid values to make sure that they are

-     * not mis-interpretted.

-     */

-#ifndef __CP_STAT_AT_OPEN

-    CP_SET(tmp_file, CP_SIZE_AT_OPEN, -1);

-    if(CP_VALUE(tmp_file, CP_FILE_ALIGNMENT) == -1)

-        CP_SET(tmp_file, CP_FILE_NOT_ALIGNED, -1);

-#endif

-

-    darshan_global_job->file_count++;

</span><span style="color: #000000;background-color: #ddffdd">+    /* no existing record, assign a new file record from the global array */

+    file = &(posix_runtime->file_runtime_array[posix_runtime->file_array_ndx]);

+    file->file_record = &(posix_runtime->file_record_array[posix_runtime->file_array_ndx]);

+    file->file_record->f_id = file_id;

+    file->file_record->rank = my_rank;

+    file->file_record->counters[POSIX_MEM_ALIGNMENT] = darshan_mem_alignment;

+    file->file_record->counters[POSIX_FILE_ALIGNMENT] = file_alignment;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* put into hash table, head of list at that index */

-    tmp_file->name_prev = NULL;

-    tmp_file->name_next = darshan_global_job->name_table[hash_index];

-    if(tmp_file->name_next)

-        tmp_file->name_next->name_prev = tmp_file;

-    darshan_global_job->name_table[hash_index] = tmp_file;

</span><span style="color: #000000;background-color: #ddffdd">+    /* add new record to file hash table */

+    HASH_ADD(hlink, posix_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);

+    posix_runtime->file_array_ndx++;

</span> 

     if(newname != name)

         free(newname);

<span style="color: #000000;background-color: #ffdddd">-    return(tmp_file);

</span><span style="color: #000000;background-color: #ddffdd">+    return(file);

</span> }

<span style="color: #000000;background-color: #ffdddd">-

-struct darshan_file_runtime* darshan_file_by_name_sethandle(

-    const char* name,

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type)

</span><span style="color: #000000;background-color: #ddffdd">+/* get a POSIX file record for the given file path, and also create a

+ * reference structure using the returned file descriptor

+ */

+static struct posix_file_runtime* posix_file_by_name_setfd(const char* name, int fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_runtime* file;

-    uint64_t tmp_hash;

-    int hash_index;

-    struct darshan_file_ref* tmp_ref;

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime* file;

+    struct posix_file_runtime_ref* ref;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(!darshan_global_job)

-    {

</span><span style="color: #000000;background-color: #ddffdd">+    if(!posix_runtime || instrumentation_disabled)

</span>         return(NULL);

<span style="color: #000000;background-color: #ffdddd">-    }

</span> 

     /* find file record by name first */

<span style="color: #000000;background-color: #ffdddd">-    file = darshan_file_by_name(name);

</span><span style="color: #000000;background-color: #ddffdd">+    file = posix_file_by_name(name);

</span> 

     if(!file)

         return(NULL);

<span style="color: #000000;background-color: #ffdddd">-    /* search hash table */

-    tmp_ref = ref_by_handle(handle, handle_sz, handle_type);

-    if(tmp_ref)

</span><span style="color: #000000;background-color: #ddffdd">+    /* search hash table for existing file ref for this fd */

+    HASH_FIND(hlink, posix_runtime->fd_hash, &fd, sizeof(int), ref);

+    if(ref)

</span>     {

         /* we have a reference.  Make sure it points to the correct file

          * and return it

          */

<span style="color: #000000;background-color: #ffdddd">-        tmp_ref->file = file;

</span><span style="color: #000000;background-color: #ddffdd">+        ref->file = file;

</span>         return(file);

     }

<span style="color: #000000;background-color: #ffdddd">-    /* if we hit this point, then we don't have a reference for this handle

</span><span style="color: #000000;background-color: #ddffdd">+    /* if we hit this point, then we don't have a reference for this fd

* in the table yet.  Add it.

      */

<span style="color: #000000;background-color: #ffdddd">-    tmp_hash = darshan_hash(handle, handle_sz, 0);

-    hash_index = tmp_hash & CP_HASH_MASK;

-    tmp_ref = malloc(sizeof(*tmp_ref));

-    if(!tmp_ref)

</span><span style="color: #000000;background-color: #ddffdd">+    ref = malloc(sizeof(*ref));

+    if(!ref)

</span>         return(NULL);

<span style="color: #000000;background-color: #ddffdd">+    memset(ref, 0, sizeof(*ref));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memset(tmp_ref, 0, sizeof(*tmp_ref));

-    tmp_ref->file = file;

-    memcpy(tmp_ref->handle, handle, handle_sz);

-    tmp_ref->handle_sz = handle_sz;

-    tmp_ref->handle_type = handle_type;

-    tmp_ref->prev = NULL;

-    tmp_ref->next = darshan_global_job->handle_table[hash_index];

-    if(tmp_ref->next)

-        tmp_ref->next->prev = tmp_ref;

-    darshan_global_job->handle_table[hash_index] = tmp_ref;

</span><span style="color: #000000;background-color: #ddffdd">+    ref->file = file;

+    ref->fd = fd;    

+    HASH_ADD(hlink, posix_runtime->fd_hash, fd, sizeof(int), ref);

</span> 

     return(file);

 }

<span style="color: #000000;background-color: #ffdddd">-struct darshan_file_runtime* darshan_file_by_handle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type)

-{   

-    struct darshan_file_ref* tmp_ref;

</span><span style="color: #000000;background-color: #ddffdd">+/* get a POSIX file record for the given file descriptor */

+static struct posix_file_runtime* posix_file_by_fd(int fd)

+{

+    struct posix_file_runtime_ref* ref;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(!darshan_global_job)

-    {

</span><span style="color: #000000;background-color: #ddffdd">+    if(!posix_runtime || instrumentation_disabled)

</span>         return(NULL);

<span style="color: #000000;background-color: #ffdddd">-    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_ref = ref_by_handle(handle, handle_sz, handle_type);

-    if(tmp_ref)

-        return(tmp_ref->file);

-    else

-        return(NULL);

</span><span style="color: #000000;background-color: #ddffdd">+    /* search hash table for existing file ref for this fd */

+    HASH_FIND(hlink, posix_runtime->fd_hash, &fd, sizeof(int), ref);

+    if(ref)

+        return(ref->file);

</span> 

     return(NULL);

 }

<span style="color: #000000;background-color: #ffdddd">-void darshan_file_closehandle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type)

</span><span style="color: #000000;background-color: #ddffdd">+/* free up reference data structures for the given file descriptor */

+static void posix_file_close_fd(int fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file_ref* tmp_ref;

-    uint64_t tmp_hash;

-    int hash_index;

-    

-    if(!darshan_global_job)

-    {

</span><span style="color: #000000;background-color: #ddffdd">+    struct posix_file_runtime_ref* ref;

+

+    if(!posix_runtime || instrumentation_disabled)

</span>         return;

<span style="color: #000000;background-color: #ddffdd">+

+    /* search hash table for this fd */

+    HASH_FIND(hlink, posix_runtime->fd_hash, &fd, sizeof(int), ref);

+    if(ref)

+    {

+        /* we have a reference, delete it */

+        HASH_DELETE(hlink, posix_runtime->fd_hash, ref);

+        free(ref);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* search hash table */

-    tmp_hash = darshan_hash(handle, handle_sz, 0);

-    hash_index = tmp_hash & CP_HASH_MASK;

-    tmp_ref = darshan_global_job->handle_table[hash_index];

-    while(tmp_ref)

</span><span style="color: #000000;background-color: #ddffdd">+    return;

+}

+

+/* compare function for sorting file records by descending rank */

+static int posix_record_compare(const void* a_p, const void* b_p)

+{

+    const struct darshan_posix_file* a = a_p;

+    const struct darshan_posix_file* b = b_p;

+

+    if(a->rank < b->rank)

+        return 1;

+    if(a->rank > b->rank)

+        return -1;

+

+    return 0;

+}

+

+/* finds the tracker structure for a given aio operation, removes it from

+ * the linked list for the darshan_file structure, and returns a pointer.  

+ *

+ * returns NULL if aio operation not found

+ */

+static struct posix_aio_tracker* posix_aio_tracker_del(int fd, void *aiocbp)

+{

+    struct posix_aio_tracker *tracker = NULL, *iter, *tmp;

+    struct posix_file_runtime* file;

+

+    file = posix_file_by_fd(fd);

+    if (file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        if(tmp_ref->handle_sz == handle_sz &&

-            tmp_ref->handle_type == handle_type &&

-            memcmp(tmp_ref->handle, handle, handle_sz) == 0)

</span><span style="color: #000000;background-color: #ddffdd">+        LL_FOREACH_SAFE(file->aio_list, iter, tmp)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            /* we have a reference. */ 

-            if(!tmp_ref->prev)

</span><span style="color: #000000;background-color: #ddffdd">+            if (iter->aiocbp == aiocbp)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                /* head of list */

-                darshan_global_job->handle_table[hash_index] = tmp_ref->next;

-                if(tmp_ref->next)

-                    tmp_ref->next->prev = NULL;

-            }

-            else

-            {

-                /* not head of list */

-                if(tmp_ref->prev)

-                    tmp_ref->prev->next = tmp_ref->next;

-                if(tmp_ref->next)

-                    tmp_ref->next->prev = tmp_ref->prev;

</span><span style="color: #000000;background-color: #ddffdd">+                LL_DELETE(file->aio_list, iter);

+                tracker = iter;

+                break;

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            free(tmp_ref);

-            return;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        tmp_ref = tmp_ref->next;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    return(tracker);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static struct darshan_file_ref* ref_by_handle(

-    const void* handle,

-    int handle_sz,

-    enum darshan_handle_type handle_type)

-{   

-    uint64_t tmp_hash;

-    int hash_index;

-    struct darshan_file_ref* tmp_ref;

-

-    if(!darshan_global_job)

-    {

-        return(NULL);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+/* adds a tracker for the given aio operation */

+static void posix_aio_tracker_add(int fd, void *aiocbp)

+{

+    struct posix_aio_tracker* tracker;

+    struct posix_file_runtime* file;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* search hash table */

-    tmp_hash = darshan_hash(handle, handle_sz, 0);

-    hash_index = tmp_hash & CP_HASH_MASK;

-    tmp_ref = darshan_global_job->handle_table[hash_index];

-    while(tmp_ref)

</span><span style="color: #000000;background-color: #ddffdd">+    file = posix_file_by_fd(fd);

+    if (file)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        if(tmp_ref->handle_sz == handle_sz &&

-            tmp_ref->handle_type == handle_type &&

-            memcmp(tmp_ref->handle, handle, handle_sz) == 0)

</span><span style="color: #000000;background-color: #ddffdd">+        tracker = malloc(sizeof(*tracker));

+        if (tracker)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            /* we have a reference. */ 

-            return(tmp_ref);

</span><span style="color: #000000;background-color: #ddffdd">+            tracker->tm1 = darshan_core_wtime();

+            tracker->aiocbp = aiocbp;

+            LL_PREPEND(file->aio_list, tracker);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        tmp_ref = tmp_ref->next;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return(NULL);

</span><span style="color: #000000;background-color: #ddffdd">+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* Allocate a new string that contains a cleaned-up version of the path

- * passed in as an argument.  Converts relative paths to absolute paths and

- * filters out some potential noise in the path string.

- */

-static char* clean_path(const char* path)

</span><span style="color: #000000;background-color: #ddffdd">+static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,

+    int *len, MPI_Datatype *datatype)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    char* newpath = NULL;

-    char* cwd = NULL;

-    char* filter = NULL;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_posix_file tmp_file;

+    struct darshan_posix_file *infile = infile_v;

+    struct darshan_posix_file *inoutfile = inoutfile_v;

+    int i, j, k;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(!path || strlen(path) < 1)

-        return(NULL);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(posix_runtime);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(path[0] == '/')

</span><span style="color: #000000;background-color: #ddffdd">+    for(i=0; i<*len; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* it is already an absolute path */

-        newpath = malloc(strlen(path)+1);

-        if(newpath)

</span><span style="color: #000000;background-color: #ddffdd">+        memset(&tmp_file, 0, sizeof(struct darshan_posix_file));

+        tmp_file.f_id = infile->f_id;

+        tmp_file.rank = -1;

+

+        /* sum */

+        for(j=POSIX_OPENS; j<=POSIX_FDSYNCS; j++)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            strcpy(newpath, path);

</span><span style="color: #000000;background-color: #ddffdd">+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-    }

-    else

-    {

-        /* handle relative path */

-        cwd = malloc(PATH_MAX);

-        if(cwd)

</span><span style="color: #000000;background-color: #ddffdd">+

+        tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];

+

+        /* sum */

+        for(j=POSIX_BYTES_READ; j<=POSIX_BYTES_WRITTEN; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        /* max */

+        for(j=POSIX_MAX_BYTE_READ; j<=POSIX_MAX_BYTE_WRITTEN; j++)

+        {

+            tmp_file.counters[j] = (

+                (infile->counters[j] > inoutfile->counters[j]) ?

+                infile->counters[j] :

+                inoutfile->counters[j]);

+        }

+

+        /* sum */

+        for(j=POSIX_CONSEC_READS; j<=POSIX_MEM_NOT_ALIGNED; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];

+

+        /* sum */

+        for(j=POSIX_FILE_NOT_ALIGNED; j<=POSIX_FILE_NOT_ALIGNED; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];

+

+        /* skip POSIX_MAX_*_TIME_SIZE; handled in floating point section */

+

+        for(j=POSIX_SIZE_READ_0_100; j<=POSIX_SIZE_WRITE_1G_PLUS; j++)

+        {

+            tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];

+        }

+

+        /* first collapse any duplicates */

+        for(j=POSIX_STRIDE1_STRIDE; j<=POSIX_STRIDE4_STRIDE; j++)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            if(getcwd(cwd, PATH_MAX))

</span><span style="color: #000000;background-color: #ddffdd">+            for(k=POSIX_STRIDE1_STRIDE; k<=POSIX_STRIDE4_STRIDE; k++)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                newpath = malloc(strlen(path) + strlen(cwd) + 2);

-                if(newpath)

</span><span style="color: #000000;background-color: #ddffdd">+                if(infile->counters[j] == inoutfile->counters[k])

</span>                 {

<span style="color: #000000;background-color: #ffdddd">-                    sprintf(newpath, "%s/%s", cwd, path);

</span><span style="color: #000000;background-color: #ddffdd">+                    infile->counters[j+4] += inoutfile->counters[k+4];

+                    inoutfile->counters[k] = 0;

+                    inoutfile->counters[k+4] = 0;

</span>                 }

             }

<span style="color: #000000;background-color: #ffdddd">-            free(cwd);

</span>         }

<span style="color: #000000;background-color: #ddffdd">+

+        /* first set */

+        for(j=POSIX_STRIDE1_STRIDE; j<=POSIX_STRIDE4_STRIDE; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_STRIDE1_STRIDE]),

+                &(tmp_file.counters[POSIX_STRIDE1_COUNT]), infile->counters[j],

+                infile->counters[j+4]);

+        }

+        /* second set */

+        for(j=POSIX_STRIDE1_STRIDE; j<=POSIX_STRIDE4_STRIDE; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_STRIDE1_STRIDE]),

+                &(tmp_file.counters[POSIX_STRIDE1_COUNT]), inoutfile->counters[j],

+                inoutfile->counters[j+4]);

+        }

+

+        /* same for access counts */

+

+        /* first collapse any duplicates */

+        for(j=POSIX_ACCESS1_ACCESS; j<=POSIX_ACCESS4_ACCESS; j++)

+        {

+            for(k=POSIX_ACCESS1_ACCESS; k<=POSIX_ACCESS4_ACCESS; k++)

+            {

+                if(infile->counters[j] == inoutfile->counters[k])

+                {

+                    infile->counters[j+4] += inoutfile->counters[k+4];

+                    inoutfile->counters[k] = 0;

+                    inoutfile->counters[k+4] = 0;

+                }

+            }

+        }

+

+        /* first set */

+        for(j=POSIX_ACCESS1_ACCESS; j<=POSIX_ACCESS4_ACCESS; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_ACCESS1_ACCESS]),

+                &(tmp_file.counters[POSIX_ACCESS1_COUNT]), infile->counters[j],

+                infile->counters[j+4]);

+        }

+        /* second set */

+        for(j=POSIX_ACCESS1_ACCESS; j<=POSIX_ACCESS4_ACCESS; j++)

+        {

+            DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_ACCESS1_ACCESS]),

+                &(tmp_file.counters[POSIX_ACCESS1_COUNT]), inoutfile->counters[j],

+                inoutfile->counters[j+4]);

+        }

+

+        /* min non-zero (if available) value */

+        for(j=POSIX_F_OPEN_TIMESTAMP; j<=POSIX_F_WRITE_START_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j] &&

+               inoutfile->fcounters[j] > 0)

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = infile->fcounters[j];

+        }

+

+        /* max */

+        for(j=POSIX_F_READ_END_TIMESTAMP; j<=POSIX_F_CLOSE_TIMESTAMP; j++)

+        {

+            if(infile->fcounters[j] > inoutfile->fcounters[j])

+                tmp_file.fcounters[j] = infile->fcounters[j];

+            else

+                tmp_file.fcounters[j] = inoutfile->fcounters[j];

+        }

+

+        /* sum */

+        for(j=POSIX_F_READ_TIME; j<=POSIX_F_META_TIME; j++)

+        {

+            tmp_file.fcounters[j] = infile->fcounters[j] + inoutfile->fcounters[j];

+        }

+

+        /* max (special case) */

+        if(infile->fcounters[POSIX_F_MAX_READ_TIME] >

+            inoutfile->fcounters[POSIX_F_MAX_READ_TIME])

+        {

+            tmp_file.fcounters[POSIX_F_MAX_READ_TIME] =

+                infile->fcounters[POSIX_F_MAX_READ_TIME];

+            tmp_file.counters[POSIX_MAX_READ_TIME_SIZE] =

+                infile->counters[POSIX_MAX_READ_TIME_SIZE];

+        }

+        else

+        {

+            tmp_file.fcounters[POSIX_F_MAX_READ_TIME] =

+                inoutfile->fcounters[POSIX_F_MAX_READ_TIME];

+            tmp_file.counters[POSIX_MAX_READ_TIME_SIZE] =

+                inoutfile->counters[POSIX_MAX_READ_TIME_SIZE];

+        }

+

+        if(infile->fcounters[POSIX_F_MAX_WRITE_TIME] >

+            inoutfile->fcounters[POSIX_F_MAX_WRITE_TIME])

+        {

+            tmp_file.fcounters[POSIX_F_MAX_WRITE_TIME] =

+                infile->fcounters[POSIX_F_MAX_WRITE_TIME];

+            tmp_file.counters[POSIX_MAX_WRITE_TIME_SIZE] =

+                infile->counters[POSIX_MAX_WRITE_TIME_SIZE];

+        }

+        else

+        {

+            tmp_file.fcounters[POSIX_F_MAX_WRITE_TIME] =

+                inoutfile->fcounters[POSIX_F_MAX_WRITE_TIME];

+            tmp_file.counters[POSIX_MAX_WRITE_TIME_SIZE] =

+                inoutfile->counters[POSIX_MAX_WRITE_TIME_SIZE];

+        }

+

+        /* min (zeroes are ok here; some procs don't do I/O) */

+        if(infile->fcounters[POSIX_F_FASTEST_RANK_TIME] <

+           inoutfile->fcounters[POSIX_F_FASTEST_RANK_TIME])

+        {

+            tmp_file.counters[POSIX_FASTEST_RANK] =

+                infile->counters[POSIX_FASTEST_RANK];

+            tmp_file.counters[POSIX_FASTEST_RANK_BYTES] =

+                infile->counters[POSIX_FASTEST_RANK_BYTES];

+            tmp_file.fcounters[POSIX_F_FASTEST_RANK_TIME] =

+                infile->fcounters[POSIX_F_FASTEST_RANK_TIME];

+        }

+        else

+        {

+            tmp_file.counters[POSIX_FASTEST_RANK] =

+                inoutfile->counters[POSIX_FASTEST_RANK];

+            tmp_file.counters[POSIX_FASTEST_RANK_BYTES] =

+                inoutfile->counters[POSIX_FASTEST_RANK_BYTES];

+            tmp_file.fcounters[POSIX_F_FASTEST_RANK_TIME] =

+                inoutfile->fcounters[POSIX_F_FASTEST_RANK_TIME];

+        }

+

+        /* max */

+        if(infile->fcounters[POSIX_F_SLOWEST_RANK_TIME] >

+           inoutfile->fcounters[POSIX_F_SLOWEST_RANK_TIME])

+        {

+            tmp_file.counters[POSIX_SLOWEST_RANK] =

+                infile->counters[POSIX_SLOWEST_RANK];

+            tmp_file.counters[POSIX_SLOWEST_RANK_BYTES] =

+                infile->counters[POSIX_SLOWEST_RANK_BYTES];

+            tmp_file.fcounters[POSIX_F_SLOWEST_RANK_TIME] =

+                infile->fcounters[POSIX_F_SLOWEST_RANK_TIME];

+        }

+        else

+        {

+            tmp_file.counters[POSIX_SLOWEST_RANK] =

+                inoutfile->counters[POSIX_SLOWEST_RANK];

+            tmp_file.counters[POSIX_SLOWEST_RANK_BYTES] =

+                inoutfile->counters[POSIX_SLOWEST_RANK_BYTES];

+            tmp_file.fcounters[POSIX_F_SLOWEST_RANK_TIME] =

+                inoutfile->fcounters[POSIX_F_SLOWEST_RANK_TIME];

+        }

+

+        /* update pointers */

+        *inoutfile = tmp_file;

+        inoutfile++;

+        infile++;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(!newpath)

-        return(NULL);

</span><span style="color: #000000;background-color: #ddffdd">+    return;

+}

+

+static void posix_shared_record_variance(MPI_Comm mod_comm,

+    struct darshan_posix_file *inrec_array, struct darshan_posix_file *outrec_array,

+    int shared_rec_count)

+{

+    MPI_Datatype var_dt;

+    MPI_Op var_op;

+    int i;

+    struct darshan_variance_dt *var_send_buf = NULL;

+    struct darshan_variance_dt *var_recv_buf = NULL;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* filter out any double slashes */

-    while((filter = strstr(newpath, "//")))

</span><span style="color: #000000;background-color: #ddffdd">+    DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_variance_dt),

+        MPI_BYTE, &var_dt);

+    DARSHAN_MPI_CALL(PMPI_Type_commit)(&var_dt);

+

+    DARSHAN_MPI_CALL(PMPI_Op_create)(darshan_variance_reduce, 1, &var_op);

+

+    var_send_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));

+    if(!var_send_buf)

+        return;

+

+    if(my_rank == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* shift down one character */

-        memmove(filter, &filter[1], (strlen(&filter[1]) + 1));

</span><span style="color: #000000;background-color: #ddffdd">+        var_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));

+

+        if(!var_recv_buf)

+            return;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* filter out any /./ instances */

-    while((filter = strstr(newpath, "/./")))

</span><span style="color: #000000;background-color: #ddffdd">+    /* get total i/o time variances for shared records */

+

+    for(i=0; i<shared_rec_count; i++)

+    {

+        var_send_buf[i].n = 1;

+        var_send_buf[i].S = 0;

+        var_send_buf[i].T = inrec_array[i].fcounters[POSIX_F_READ_TIME] +

+                            inrec_array[i].fcounters[POSIX_F_WRITE_TIME] +

+                            inrec_array[i].fcounters[POSIX_F_META_TIME];

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,

+        var_dt, var_op, 0, mod_comm);

+

+    if(my_rank == 0)

+    {

+        for(i=0; i<shared_rec_count; i++)

+        {

+            outrec_array[i].fcounters[POSIX_F_VARIANCE_RANK_TIME] =

+                (var_recv_buf[i].S / var_recv_buf[i].n);

+        }

+    }

+

+    /* get total bytes moved variances for shared records */

+

+    for(i=0; i<shared_rec_count; i++)

+    {

+        var_send_buf[i].n = 1;

+        var_send_buf[i].S = 0;

+        var_send_buf[i].T = (double)

+                            inrec_array[i].counters[POSIX_BYTES_READ] +

+                            inrec_array[i].counters[POSIX_BYTES_WRITTEN];

+    }

+

+    DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,

+        var_dt, var_op, 0, mod_comm);

+

+    if(my_rank == 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* shift down two characters */

-        memmove(filter, &filter[2], (strlen(&filter[2]) + 1));

</span><span style="color: #000000;background-color: #ddffdd">+        for(i=0; i<shared_rec_count; i++)

+        {

+            outrec_array[i].fcounters[POSIX_F_VARIANCE_RANK_BYTES] =

+                (var_recv_buf[i].S / var_recv_buf[i].n);

+        }

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* return result */

-    return(newpath);

</span><span style="color: #000000;background-color: #ddffdd">+    DARSHAN_MPI_CALL(PMPI_Type_free)(&var_dt);

+    DARSHAN_MPI_CALL(PMPI_Op_free)(&var_op);

+    free(var_send_buf);

+    free(var_recv_buf);

+

+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* adds a tracker for the given aio operation */

-static void darshan_aio_tracker_add(int fd, void *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+/* posix module shutdown benchmark routine */

+void darshan_posix_shutdown_bench_setup(int test_case)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_aio_tracker* tracker;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    char filepath[256];

+    int *fd_array;

+    int64_t *size_array;

+    int i;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

</span><span style="color: #000000;background-color: #ddffdd">+    if(posix_runtime)

+        posix_shutdown();

+

+    posix_runtime_initialize();

+

+    srand(my_rank);

+    fd_array = malloc(1024 * sizeof(int));

+    size_array = malloc(DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT * sizeof(int64_t));

+    assert(fd_array && size_array);

+

+    for(i = 0; i < 1024; i++)

+        fd_array[i] = i;

+    for(i = 0; i < DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT; i++)

+        size_array[i] = rand();

+

+    switch(test_case)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        tracker = malloc(sizeof(*tracker));

-        if(tracker)

-        {

-            tracker->tm1 = darshan_wtime();

-            tracker->aiocbp = aiocbp;

-            tracker->next = NULL;

-            if(file->aio_list_tail)

</span><span style="color: #000000;background-color: #ddffdd">+        case 1: /* single file-per-process */

+            snprintf(filepath, 256, "fpp-0_rank-%d", my_rank);

+            

+            POSIX_RECORD_OPEN(fd_array[0], filepath, 777, 0, 0, 1);

+            POSIX_RECORD_WRITE(size_array[0], fd_array[0], 0, 0, 1, 0, 1, 2);

+

+            break;

+        case 2: /* single shared file */

+            snprintf(filepath, 256, "shared-0");

+

+            POSIX_RECORD_OPEN(fd_array[0], filepath, 777, 0, 0, 1);

+            POSIX_RECORD_WRITE(size_array[0], fd_array[0], 0, 0, 1, 0, 1, 2);

+

+            break;

+        case 3: /* 1024 unique files per proc */

+            for(i = 0; i < 1024; i++)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                file->aio_list_tail->next = tracker;

-                file->aio_list_tail = tracker;

</span><span style="color: #000000;background-color: #ddffdd">+                snprintf(filepath, 256, "fpp-%d_rank-%d", i , my_rank);

+

+                POSIX_RECORD_OPEN(fd_array[i], filepath, 777, 0, 0, 1);

+                POSIX_RECORD_WRITE(size_array[i % DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT],

+                    fd_array[i], 0, 0, 1, 0, 1, 2);

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            else

</span><span style="color: #000000;background-color: #ddffdd">+

+            break;

+        case 4: /* 1024 shared files per proc */

+            for(i = 0; i < 1024; i++)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                file->aio_list_head = file->aio_list_tail = tracker;

</span><span style="color: #000000;background-color: #ddffdd">+                snprintf(filepath, 256, "shared-%d", i);

+

+                POSIX_RECORD_OPEN(fd_array[i], filepath, 777, 0, 0, 1);

+                POSIX_RECORD_WRITE(size_array[i % DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT],

+                    fd_array[i], 0, 0, 1, 0, 1, 2);

</span>             }

<span style="color: #000000;background-color: #ffdddd">-        }

</span><span style="color: #000000;background-color: #ddffdd">+

+            break;

+        default:

+            fprintf(stderr, "Error: invalid Darshan benchmark test case.\n");

+            return;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+

+    free(fd_array);

+    free(size_array);

</span> 

     return;

 }

<span style="color: #000000;background-color: #ffdddd">-/* finds the tracker structure for a given aio operation, removes it from

- * the linked list for the darshan_file structure, and returns a pointer.  

- *

- * returns NULL if aio operation not found

- */

-static struct darshan_aio_tracker* darshan_aio_tracker_del(int fd, void *aiocbp)

</span><span style="color: #000000;background-color: #ddffdd">+/************************************************************************

+ * Functions exported by this module for coordinating with darshan-core *

+ ************************************************************************/

+

+static void posix_begin_shutdown()

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_aio_tracker *tmp=NULL, *prev;

-    struct darshan_file_runtime* file;

</span><span style="color: #000000;background-color: #ddffdd">+    assert(posix_runtime);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    CP_LOCK();

-    file = darshan_file_by_fd(fd);

-    if(file)

</span><span style="color: #000000;background-color: #ddffdd">+    POSIX_LOCK();

+    /* disable further instrumentation while Darshan shuts down */

+    instrumentation_disabled = 1;

+    POSIX_UNLOCK();

+

+    return;

+}

+

+static void posix_get_output_data(

+    MPI_Comm mod_comm,

+    darshan_record_id *shared_recs,

+    int shared_rec_count,

+    void **posix_buf,

+    int *posix_buf_sz)

+{

+    struct posix_file_runtime *file;

+    struct posix_file_runtime *tmp;

+    int i;

+    double posix_time;

+    struct darshan_posix_file *red_send_buf = NULL;

+    struct darshan_posix_file *red_recv_buf = NULL;

+    MPI_Datatype red_type;

+    MPI_Op red_op;

+

+    assert(posix_runtime);

+

+    /* go through file access data for each record and set the 4 most common

+     * stride/access size counters.

+     */

+    for(i = 0; i < posix_runtime->file_array_ndx; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* is there a tracker struct for this operation? */

-        tmp = file->aio_list_head; 

-        prev = NULL;

-        while(tmp)

-        {

-            if(tmp->aiocbp == aiocbp)

-            {

-                if(prev)

-                    prev->next = tmp->next;

-                else

-                    file->aio_list_head = tmp->next;

</span><span style="color: #000000;background-color: #ddffdd">+        tmp = &(posix_runtime->file_runtime_array[i]);

</span> 

<span style="color: #000000;background-color: #ffdddd">-                if(tmp == file->aio_list_tail)

-                    file->aio_list_tail = prev;

</span><span style="color: #000000;background-color: #ddffdd">+        /* common accesses */

+        darshan_walk_common_vals(tmp->access_root,

+            &(tmp->file_record->counters[POSIX_ACCESS1_ACCESS]),

+            &(tmp->file_record->counters[POSIX_ACCESS1_COUNT]));

+        /* common strides */

+        darshan_walk_common_vals(tmp->stride_root,

+            &(tmp->file_record->counters[POSIX_STRIDE1_STRIDE]),

+            &(tmp->file_record->counters[POSIX_STRIDE1_COUNT]));

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-                break;

-            }

-            else

</span><span style="color: #000000;background-color: #ddffdd">+    /* if there are globally shared files, do a shared file reduction */

+    /* NOTE: the shared file reduction is also skipped if the 

+     * DARSHAN_DISABLE_SHARED_REDUCTION environment variable is set.

+     */

+    if(shared_rec_count && !getenv("DARSHAN_DISABLE_SHARED_REDUCTION"))

+    {

+        /* necessary initialization of shared records */

+        for(i = 0; i < shared_rec_count; i++)

+        {

+            HASH_FIND(hlink, posix_runtime->file_hash, &shared_recs[i],

+                sizeof(darshan_record_id), file);

+            assert(file);

+

+            posix_time =

+                file->file_record->fcounters[POSIX_F_READ_TIME] +

+                file->file_record->fcounters[POSIX_F_WRITE_TIME] +

+                file->file_record->fcounters[POSIX_F_META_TIME];

+

+            /* initialize fastest/slowest info prior to the reduction */

+            file->file_record->counters[POSIX_FASTEST_RANK] =

+                file->file_record->rank;

+            file->file_record->counters[POSIX_FASTEST_RANK_BYTES] =

+                file->file_record->counters[POSIX_BYTES_READ] +

+                file->file_record->counters[POSIX_BYTES_WRITTEN];

+            file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME] =

+                posix_time;

+

+            /* until reduction occurs, we assume that this rank is both

+             * the fastest and slowest. It is up to the reduction operator

+             * to find the true min and max.

+             */

+            file->file_record->counters[POSIX_SLOWEST_RANK] =

+                file->file_record->counters[POSIX_FASTEST_RANK];

+            file->file_record->counters[POSIX_SLOWEST_RANK_BYTES] =

+                file->file_record->counters[POSIX_FASTEST_RANK_BYTES];

+            file->file_record->fcounters[POSIX_F_SLOWEST_RANK_TIME] =

+                file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME];

+

+            file->file_record->rank = -1;

+        }

+

+        /* sort the array of files descending by rank so that we get all of the 

+         * shared files (marked by rank -1) in a contiguous portion at end 

+         * of the array

+         */

+        qsort(posix_runtime->file_record_array, posix_runtime->file_array_ndx,

+            sizeof(struct darshan_posix_file), posix_record_compare);

+

+        /* make *send_buf point to the shared files at the end of sorted array */

+        red_send_buf =

+            &(posix_runtime->file_record_array[posix_runtime->file_array_ndx-shared_rec_count]);

+        

+        /* allocate memory for the reduction output on rank 0 */

+        if(my_rank == 0)

+        {

+            red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_posix_file));

+            if(!red_recv_buf)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                prev = tmp;

-                tmp = tmp->next;

</span><span style="color: #000000;background-color: #ddffdd">+                return;

</span>             }

         }

<span style="color: #000000;background-color: #ddffdd">+

+        /* construct a datatype for a POSIX file record.  This is serving no purpose

+         * except to make sure we can do a reduction on proper boundaries

+         */

+        DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_posix_file),

+            MPI_BYTE, &red_type);

+        DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);

+

+        /* register a POSIX file record reduction operator */

+        DARSHAN_MPI_CALL(PMPI_Op_create)(posix_record_reduction_op, 1, &red_op);

+

+        /* reduce shared POSIX file records */

+        DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,

+            shared_rec_count, red_type, red_op, 0, mod_comm);

+

+        /* get the time and byte variances for shared files */

+        posix_shared_record_variance(mod_comm, red_send_buf, red_recv_buf,

+            shared_rec_count);

+

+        /* clean up reduction state */

+        if(my_rank == 0)

+        {

+            int tmp_ndx = posix_runtime->file_array_ndx - shared_rec_count;

+            memcpy(&(posix_runtime->file_record_array[tmp_ndx]), red_recv_buf,

+                shared_rec_count * sizeof(struct darshan_posix_file));

+            free(red_recv_buf);

+        }

+        else

+        {

+            posix_runtime->file_array_ndx -= shared_rec_count;

+        }

+

+        DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);

+        DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);

+    }

+

+    *posix_buf = (void *)(posix_runtime->file_record_array);

+    *posix_buf_sz = posix_runtime->file_array_ndx * sizeof(struct darshan_posix_file);

+

+    return;

+}

+

+static void posix_shutdown()

+{

+    struct posix_file_runtime_ref *ref, *tmp;

+

+    assert(posix_runtime);

+

+    HASH_ITER(hlink, posix_runtime->fd_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, posix_runtime->fd_hash, ref);

+        free(ref);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    CP_UNLOCK();

</span><span style="color: #000000;background-color: #ddffdd">+    HASH_CLEAR(hlink, posix_runtime->file_hash); /* these entries are freed all at once below */

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(tmp);

</span><span style="color: #000000;background-color: #ddffdd">+    free(posix_runtime->file_runtime_array);

+    free(posix_runtime->file_record_array);

+    free(posix_runtime);

+    posix_runtime = NULL;

+    instrumentation_disabled = 0;

+    

+    return;

</span> }

 /*

</code></pre>

<br>

</li>

<li id='diff-35'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-35'>

<strong>

darshan-runtime/lib/pkgconfig/darshan-runtime.pc.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/pkgconfig/darshan-runtime.pc.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/pkgconfig/darshan-runtime.pc.in

</span><span style="color: #aaaaaa">@@ -9,10 +9,11 @@ Requires: zlib mpich

</span> Requires.private:

 darshan_prefix=@prefix@

<span style="color: #000000;background-color: #ddffdd">+darshan_share=@darshan_share_path@

</span> darshan_includedir=

 darshan_libdir= -L${darshan_prefix}/lib

<span style="color: #000000;background-color: #ffdddd">-darshan_linkopts="-Wl,-u,MPI_Init,-u,MPI_Wtime,-u,__wrap_H5Fcreate,-u,__wrap_ncmpi_create,-wrap,write,-wrap,open,-wrap,creat,-wrap,creat64,-wrap,open64,-wrap,close,-wrap,read,-wrap,lseek,-wrap,lseek64,-wrap,pread,-wrap,pwrite,-wrap,readv,-wrap,writev,-wrap,__xstat,-wrap,__lxstat,-wrap,__fxstat,-wrap,__xstat64,-wrap,__lxstat64,-wrap,__fxstat64,-wrap,mmap,-wrap,mmap64,-wrap,fopen,-wrap,fclose,-wrap,fread,-wrap,fwrite,-wrap,fseek,-wrap,fopen64,-wrap,pread64,-wrap,pwrite64,-wrap,fsync,-wrap,fdatasync,-wrap,ncmpi_create,-wrap,ncmpi_open,-wrap,ncmpi_close,-wrap,H5Fcreate,-wrap,H5Fopen,-wrap,H5Fclose,-wrap,aio_write,-wrap,aio_write64,-wrap,aio_read,-wrap,aio_read64,-wrap,lio_listio,-wrap,lio_listio64,-wrap,aio_return,-wrap,aio_return64,-wrap,mkstemp,-wrap,mkostemp,-wrap,mkstemps,-wrap,mkostemps"

</span><span style="color: #000000;background-color: #ddffdd">+darshan_linkopts="-Wl,@${darshan_share}/ld-opts/darshan-base-ld-opts"

</span> 

 Cflags:

 Libs:

<span style="color: #000000;background-color: #ffdddd">-Libs.private: ${darshan_linkopts} ${darshan_libdir} -lfmpich -lmpichcxx -Wl,--start-group -ldarshan-mpi-io -ldarshan-posix -ldarshan-stubs -Wl,--end-group

</span><span style="color: #000000;background-color: #ddffdd">+Libs.private: ${darshan_linkopts} ${darshan_libdir} -lfmpich -lmpichcxx -Wl,--start-group -ldarshan -ldarshan-stubs -Wl,--end-group

</span></code></pre>

<br>

</li>

<li id='diff-36'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-36'>

<strong>

darshan-runtime/share/ld-opts/darshan-base-ld-opts.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/share/ld-opts/darshan-base-ld-opts.in

</span><span style="color: #aaaaaa">@@ -0,0 +1,5 @@

</span><span style="color: #000000;background-color: #ddffdd">+--undefined=MPI_Init

+--undefined=MPI_Wtime

+@@darshan_share_path@/ld-opts/darshan-posix-ld-opts

+@@darshan_share_path@/ld-opts/darshan-hdf5-ld-opts

+@@darshan_share_path@/ld-opts/darshan-pnetcdf-ld-opts

</span></code></pre>

<br>

</li>

<li id='diff-37'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-37'>

<strong>

darshan-runtime/share/ld-opts/darshan-hdf5-ld-opts

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/share/ld-opts/darshan-hdf5-ld-opts

</span><span style="color: #aaaaaa">@@ -0,0 +1,4 @@

</span><span style="color: #000000;background-color: #ddffdd">+--undefined=__wrap_H5Fcreate

+--wrap=H5Fcreate

+--wrap=H5Fopen

+--wrap=H5Fclose

</span></code></pre>

<br>

</li>

<li id='diff-38'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-38'>

<strong>

darshan-runtime/share/ld-opts/darshan-pnetcdf-ld-opts

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/share/ld-opts/darshan-pnetcdf-ld-opts

</span><span style="color: #aaaaaa">@@ -0,0 +1,4 @@

</span><span style="color: #000000;background-color: #ddffdd">+--undefined=__wrap_ncmpi_create

+--wrap=ncmpi_create

+--wrap=ncmpi_open

+--wrap=ncmpi_close

</span></code></pre>

<br>

</li>

<li id='diff-39'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-39'>

<strong>

darshan-runtime/share/ld-opts/darshan-posix-ld-opts

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/share/ld-opts/darshan-posix-ld-opts

</span><span style="color: #aaaaaa">@@ -0,0 +1,43 @@

</span><span style="color: #000000;background-color: #ddffdd">+--wrap=open

+--wrap=open64

+--wrap=creat

+--wrap=creat64

+--wrap=fopen

+--wrap=fopen64

+--wrap=mkstemp

+--wrap=mkostemp

+--wrap=mkstemps

+--wrap=mkostemps

+--wrap=read

+--wrap=write

+--wrap=pread

+--wrap=pwrite

+--wrap=pread64

+--wrap=pwrite64

+--wrap=readv

+--wrap=writev

+--wrap=fread

+--wrap=fwrite

+--wrap=lseek

+--wrap=lseek64

+--wrap=fseek

+--wrap=__xstat

+--wrap=__xstat64

+--wrap=__lxstat

+--wrap=__lxstat64

+--wrap=__fxstat

+--wrap=__fxstat64

+--wrap=mmap

+--wrap=mmap64

+--wrap=fsync

+--wrap=fdatasync

+--wrap=close

+--wrap=fclose

+--wrap=aio_read

+--wrap=aio_write

+--wrap=aio_read64

+--wrap=aio_write64

+--wrap=aio_return

+--wrap=aio_return64

+--wrap=lio_listio

+--wrap=lio_listio64

</span></code></pre>

<br>

</li>

<li id='diff-40'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-40'>

<strong>

darshan-runtime/uthash.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/uthash.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,966 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+Copyright (c) 2003-2010, Troy D. Hanson     http://uthash.sourceforge.net

+All rights reserved.

+

+Redistribution and use in source and binary forms, with or without

+modification, are permitted provided that the following conditions are met:

+

+    * Redistributions of source code must retain the above copyright

+      notice, this list of conditions and the following disclaimer.

+

+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS

+IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED

+TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A

+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER

+OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,

+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,

+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR

+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF

+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING

+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+*/

+

+#ifndef UTHASH_H

+#define UTHASH_H 

+

+#include <string.h>   /* memcmp,strlen */

+#include <stddef.h>   /* ptrdiff_t */

+

+/* These macros use decltype or the earlier __typeof GNU extension.

+   As decltype is only available in newer compilers (VS2010 or gcc 4.3+

+   when compiling c++ source) this code uses whatever method is needed

+   or, for VS2008 where neither is available, uses casting workarounds. */

+#ifdef _MSC_VER         /* MS compiler */

+#if _MSC_VER >= 1600 && __cplusplus  /* VS2010 or newer in C++ mode */

+#define DECLTYPE(x) (decltype(x))

+#else                   /* VS2008 or older (or VS2010 in C mode) */

+#define NO_DECLTYPE

+#define DECLTYPE(x)

+#endif

+#else                   /* GNU, Sun and other compilers */

+#define DECLTYPE(x) (__typeof(x))

+#endif

+

+#ifdef NO_DECLTYPE

+#define DECLTYPE_ASSIGN(dst,src)                                                 \

+do {                                                                             \

+  char **_da_dst = (char**)(&(dst));                                             \

+  *_da_dst = (char*)(src);                                                       \

+} while(0)

+#else 

+#define DECLTYPE_ASSIGN(dst,src)                                                 \

+do {                                                                             \

+  (dst) = DECLTYPE(dst)(src);                                                    \

+} while(0)

+#endif

+

+/* a number of the hash function use uint32_t which isn't defined on win32 */

+#ifdef _MSC_VER

+typedef unsigned int uint32_t;

+#else

+#include <inttypes.h>   /* uint32_t */

+#endif

+

+#define UTHASH_VERSION 1.9.1

+

+#define uthash_fatal(msg) exit(-1)        /* fatal error (out of memory,etc) */

+#define uthash_malloc(sz) malloc(sz)      /* malloc fcn                      */

+#define uthash_free(ptr,sz) free(ptr)     /* free fcn                        */

+

+#define uthash_noexpand_fyi(tbl)          /* can be defined to log noexpand  */

+#define uthash_expand_fyi(tbl)            /* can be defined to log expands   */

+

+/* initial number of buckets */

+#define HASH_INITIAL_NUM_BUCKETS 32      /* initial number of buckets        */

+#define HASH_INITIAL_NUM_BUCKETS_LOG2 5  /* lg2 of initial number of buckets */

+#define HASH_BKT_CAPACITY_THRESH 10      /* expand when bucket count reaches */

+

+/* calculate the element whose hash handle address is hhe */

+#define ELMT_FROM_HH(tbl,hhp) ((void*)(((char*)(hhp)) - ((tbl)->hho)))

+

+#define HASH_FIND(hh,head,keyptr,keylen,out)                                     \

+do {                                                                             \

+  unsigned _hf_bkt,_hf_hashv;                                                    \

+  out=NULL;                                                                      \

+  if (head) {                                                                    \

+     HASH_FCN(keyptr,keylen, (head)->hh.tbl->num_buckets, _hf_hashv, _hf_bkt);   \

+     if (HASH_BLOOM_TEST((head)->hh.tbl, _hf_hashv)) {                           \

+       HASH_FIND_IN_BKT((head)->hh.tbl, hh, (head)->hh.tbl->buckets[ _hf_bkt ],  \

+                        keyptr,keylen,out);                                      \

+     }                                                                           \

+  }                                                                              \

+} while (0)

+

+#ifdef HASH_BLOOM

+#define HASH_BLOOM_BITLEN (1ULL << HASH_BLOOM)

+#define HASH_BLOOM_BYTELEN (HASH_BLOOM_BITLEN/8) + ((HASH_BLOOM_BITLEN%8) ? 1:0)

+#define HASH_BLOOM_MAKE(tbl)                                                     \

+do {                                                                             \

+  (tbl)->bloom_nbits = HASH_BLOOM;                                               \

+  (tbl)->bloom_bv = (uint8_t*)uthash_malloc(HASH_BLOOM_BYTELEN);                 \

+  if (!((tbl)->bloom_bv))  { uthash_fatal( "out of memory"); }                   \

+  memset((tbl)->bloom_bv, 0, HASH_BLOOM_BYTELEN);                                \

+  (tbl)->bloom_sig = HASH_BLOOM_SIGNATURE;                                       \

+} while (0);

+

+#define HASH_BLOOM_FREE(tbl)                                                     \

+do {                                                                             \

+  uthash_free((tbl)->bloom_bv, HASH_BLOOM_BYTELEN);                             \

+} while (0);

+

+#define HASH_BLOOM_BITSET(bv,idx) (bv[(idx)/8] |= (1U << ((idx)%8)))

+#define HASH_BLOOM_BITTEST(bv,idx) (bv[(idx)/8] & (1U << ((idx)%8)))

+

+#define HASH_BLOOM_ADD(tbl,hashv)                                                \

+  HASH_BLOOM_BITSET((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1)))

+

+#define HASH_BLOOM_TEST(tbl,hashv)                                               \

+  HASH_BLOOM_BITTEST((tbl)->bloom_bv, (hashv & (uint32_t)((1ULL << (tbl)->bloom_nbits) - 1)))

+

+#else

+#define HASH_BLOOM_MAKE(tbl) 

+#define HASH_BLOOM_FREE(tbl) 

+#define HASH_BLOOM_ADD(tbl,hashv) 

+#define HASH_BLOOM_TEST(tbl,hashv) (1)

+#endif

+

+#define HASH_MAKE_TABLE(hh,head)                                                 \

+do {                                                                             \

+  (head)->hh.tbl = (UT_hash_table*)uthash_malloc(                                \

+                  sizeof(UT_hash_table));                                        \

+  if (!((head)->hh.tbl))  { uthash_fatal( "out of memory"); }                    \

+  memset((head)->hh.tbl, 0, sizeof(UT_hash_table));                              \

+  (head)->hh.tbl->tail = &((head)->hh);                                          \

+  (head)->hh.tbl->num_buckets = HASH_INITIAL_NUM_BUCKETS;                        \

+  (head)->hh.tbl->log2_num_buckets = HASH_INITIAL_NUM_BUCKETS_LOG2;              \

+  (head)->hh.tbl->hho = (char*)(&(head)->hh) - (char*)(head);                    \

+  (head)->hh.tbl->buckets = (UT_hash_bucket*)uthash_malloc(                      \

+          HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket));               \

+  if (! (head)->hh.tbl->buckets) { uthash_fatal( "out of memory"); }             \

+  memset((head)->hh.tbl->buckets, 0,                                             \

+          HASH_INITIAL_NUM_BUCKETS*sizeof(struct UT_hash_bucket));               \

+  HASH_BLOOM_MAKE((head)->hh.tbl);                                               \

+  (head)->hh.tbl->signature = HASH_SIGNATURE;                                    \

+} while(0)

+

+#define HASH_ADD(hh,head,fieldname,keylen_in,add)                                \

+        HASH_ADD_KEYPTR(hh,head,&add->fieldname,keylen_in,add)

+ 

+#define HASH_ADD_KEYPTR(hh,head,keyptr,keylen_in,add)                            \

+do {                                                                             \

+ unsigned _ha_bkt;                                                               \

+ (add)->hh.next = NULL;                                                          \

+ (add)->hh.key = (char*)keyptr;                                                  \

+ (add)->hh.keylen = keylen_in;                                                   \

+ if (!(head)) {                                                                  \

+    head = (add);                                                                \

+    (head)->hh.prev = NULL;                                                      \

+    HASH_MAKE_TABLE(hh,head);                                                    \

+ } else {                                                                        \

+    (head)->hh.tbl->tail->next = (add);                                          \

+    (add)->hh.prev = ELMT_FROM_HH((head)->hh.tbl, (head)->hh.tbl->tail);         \

+    (head)->hh.tbl->tail = &((add)->hh);                                         \

+ }                                                                               \

+ (head)->hh.tbl->num_items++;                                                    \

+ (add)->hh.tbl = (head)->hh.tbl;                                                 \

+ HASH_FCN(keyptr,keylen_in, (head)->hh.tbl->num_buckets,                         \

+         (add)->hh.hashv, _ha_bkt);                                              \

+ HASH_ADD_TO_BKT((head)->hh.tbl->buckets[_ha_bkt],&(add)->hh);                   \

+ HASH_BLOOM_ADD((head)->hh.tbl,(add)->hh.hashv);                                 \

+ HASH_EMIT_KEY(hh,head,keyptr,keylen_in);                                        \

+ HASH_FSCK(hh,head);                                                             \

+} while(0)

+

+#define HASH_TO_BKT( hashv, num_bkts, bkt )                                      \

+do {                                                                             \

+  bkt = ((hashv) & ((num_bkts) - 1));                                            \

+} while(0)

+

+/* delete "delptr" from the hash table.

+ * "the usual" patch-up process for the app-order doubly-linked-list.

+ * The use of _hd_hh_del below deserves special explanation.

+ * These used to be expressed using (delptr) but that led to a bug

+ * if someone used the same symbol for the head and deletee, like

+ *  HASH_DELETE(hh,users,users);

+ * We want that to work, but by changing the head (users) below

+ * we were forfeiting our ability to further refer to the deletee (users)

+ * in the patch-up process. Solution: use scratch space to

+ * copy the deletee pointer, then the latter references are via that

+ * scratch pointer rather than through the repointed (users) symbol.

+ */

+#define HASH_DELETE(hh,head,delptr)                                              \

+do {                                                                             \

+    unsigned _hd_bkt;                                                            \

+    struct UT_hash_handle *_hd_hh_del;                                           \

+    if ( ((delptr)->hh.prev == NULL) && ((delptr)->hh.next == NULL) )  {         \

+        uthash_free((head)->hh.tbl->buckets,                                    \

+                    (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket) );\

+        HASH_BLOOM_FREE((head)->hh.tbl);                                         \

+        uthash_free((head)->hh.tbl, sizeof(UT_hash_table));                     \

+        head = NULL;                                                             \

+    } else {                                                                     \

+        _hd_hh_del = &((delptr)->hh);                                            \

+        if ((delptr) == ELMT_FROM_HH((head)->hh.tbl,(head)->hh.tbl->tail)) {     \

+            (head)->hh.tbl->tail =                                               \

+                (UT_hash_handle*)((char*)((delptr)->hh.prev) +                   \

+                (head)->hh.tbl->hho);                                            \

+        }                                                                        \

+        if ((delptr)->hh.prev) {                                                 \

+            ((UT_hash_handle*)((char*)((delptr)->hh.prev) +                      \

+                    (head)->hh.tbl->hho))->next = (delptr)->hh.next;             \

+        } else {                                                                 \

+            DECLTYPE_ASSIGN(head,(delptr)->hh.next);                             \

+        }                                                                        \

+        if (_hd_hh_del->next) {                                                  \

+            ((UT_hash_handle*)((char*)_hd_hh_del->next +                         \

+                    (head)->hh.tbl->hho))->prev =                                \

+                    _hd_hh_del->prev;                                            \

+        }                                                                        \

+        HASH_TO_BKT( _hd_hh_del->hashv, (head)->hh.tbl->num_buckets, _hd_bkt);   \

+        HASH_DEL_IN_BKT(hh,(head)->hh.tbl->buckets[_hd_bkt], _hd_hh_del);        \

+        (head)->hh.tbl->num_items--;                                             \

+    }                                                                            \

+    HASH_FSCK(hh,head);                                                          \

+} while (0)

+

+

+/* convenience forms of HASH_FIND/HASH_ADD/HASH_DEL */

+#define HASH_FIND_STR(head,findstr,out)                                          \

+    HASH_FIND(hh,head,findstr,strlen(findstr),out)

+#define HASH_ADD_STR(head,strfield,add)                                          \

+    HASH_ADD(hh,head,strfield,strlen(add->strfield),add)

+#define HASH_FIND_INT(head,findint,out)                                          \

+    HASH_FIND(hh,head,findint,sizeof(int),out)

+#define HASH_ADD_INT(head,intfield,add)                                          \

+    HASH_ADD(hh,head,intfield,sizeof(int),add)

+#define HASH_FIND_PTR(head,findptr,out)                                          \

+    HASH_FIND(hh,head,findptr,sizeof(void *),out)

+#define HASH_ADD_PTR(head,ptrfield,add)                                          \

+    HASH_ADD(hh,head,ptrfield,sizeof(void *),add)

+#define HASH_DEL(head,delptr)                                                    \

+    HASH_DELETE(hh,head,delptr)

+

+/* HASH_FSCK checks hash integrity on every add/delete when HASH_DEBUG is defined.

+ * This is for uthash developer only; it compiles away if HASH_DEBUG isn't defined.

+ */

+#ifdef HASH_DEBUG

+#define HASH_OOPS(...) do { fprintf(stderr,__VA_ARGS__); exit(-1); } while (0)

+#define HASH_FSCK(hh,head)                                                       \

+do {                                                                             \

+    unsigned _bkt_i;                                                             \

+    unsigned _count, _bkt_count;                                                 \

+    char *_prev;                                                                 \

+    struct UT_hash_handle *_thh;                                                 \

+    if (head) {                                                                  \

+        _count = 0;                                                              \

+        for( _bkt_i = 0; _bkt_i < (head)->hh.tbl->num_buckets; _bkt_i++) {       \

+            _bkt_count = 0;                                                      \

+            _thh = (head)->hh.tbl->buckets[_bkt_i].hh_head;                      \

+            _prev = NULL;                                                        \

+            while (_thh) {                                                       \

+               if (_prev != (char*)(_thh->hh_prev)) {                            \

+                   HASH_OOPS("invalid hh_prev %p, actual %p\n",                  \

+                    _thh->hh_prev, _prev );                                      \

+               }                                                                 \

+               _bkt_count++;                                                     \

+               _prev = (char*)(_thh);                                            \

+               _thh = _thh->hh_next;                                             \

+            }                                                                    \

+            _count += _bkt_count;                                                \

+            if ((head)->hh.tbl->buckets[_bkt_i].count !=  _bkt_count) {          \

+               HASH_OOPS("invalid bucket count %d, actual %d\n",                 \

+                (head)->hh.tbl->buckets[_bkt_i].count, _bkt_count);              \

+            }                                                                    \

+        }                                                                        \

+        if (_count != (head)->hh.tbl->num_items) {                               \

+            HASH_OOPS("invalid hh item count %d, actual %d\n",                   \

+                (head)->hh.tbl->num_items, _count );                             \

+        }                                                                        \

+        /* traverse hh in app order; check next/prev integrity, count */         \

+        _count = 0;                                                              \

+        _prev = NULL;                                                            \

+        _thh =  &(head)->hh;                                                     \

+        while (_thh) {                                                           \

+           _count++;                                                             \

+           if (_prev !=(char*)(_thh->prev)) {                                    \

+              HASH_OOPS("invalid prev %p, actual %p\n",                          \

+                    _thh->prev, _prev );                                         \

+           }                                                                     \

+           _prev = (char*)ELMT_FROM_HH((head)->hh.tbl, _thh);                    \

+           _thh = ( _thh->next ?  (UT_hash_handle*)((char*)(_thh->next) +        \

+                                  (head)->hh.tbl->hho) : NULL );                 \

+        }                                                                        \

+        if (_count != (head)->hh.tbl->num_items) {                               \

+            HASH_OOPS("invalid app item count %d, actual %d\n",                  \

+                (head)->hh.tbl->num_items, _count );                             \

+        }                                                                        \

+    }                                                                            \

+} while (0)

+#else

+#define HASH_FSCK(hh,head) 

+#endif

+

+/* When compiled with -DHASH_EMIT_KEYS, length-prefixed keys are emitted to 

+ * the descriptor to which this macro is defined for tuning the hash function.

+ * The app can #include <unistd.h> to get the prototype for write(2). */

+#ifdef HASH_EMIT_KEYS

+#define HASH_EMIT_KEY(hh,head,keyptr,fieldlen)                                   \

+do {                                                                             \

+    unsigned _klen = fieldlen;                                                   \

+    write(HASH_EMIT_KEYS, &_klen, sizeof(_klen));                                \

+    write(HASH_EMIT_KEYS, keyptr, fieldlen);                                     \

+} while (0)

+#else 

+#define HASH_EMIT_KEY(hh,head,keyptr,fieldlen)                    

+#endif

+

+/* default to Jenkin's hash unless overridden e.g. DHASH_FUNCTION=HASH_SAX */

+#ifdef HASH_FUNCTION 

+#define HASH_FCN HASH_FUNCTION

+#else

+#define HASH_FCN HASH_JEN

+#endif

+

+/* The Bernstein hash function, used in Perl prior to v5.6 */

+#define HASH_BER(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  unsigned _hb_keylen=keylen;                                                    \

+  char *_hb_key=(char*)(key);                                                    \

+  (hashv) = 0;                                                                   \

+  while (_hb_keylen--)  { (hashv) = ((hashv) * 33) + *_hb_key++; }               \

+  bkt = (hashv) & (num_bkts-1);                                                  \

+} while (0)

+

+

+/* SAX/FNV/OAT/JEN hash functions are macro variants of those listed at 

+ * http://eternallyconfuzzled.com/tuts/algorithms/jsw_tut_hashing.aspx */

+#define HASH_SAX(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  unsigned _sx_i;                                                                \

+  char *_hs_key=(char*)(key);                                                    \

+  hashv = 0;                                                                     \

+  for(_sx_i=0; _sx_i < keylen; _sx_i++)                                          \

+      hashv ^= (hashv << 5) + (hashv >> 2) + _hs_key[_sx_i];                     \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while (0)

+

+#define HASH_FNV(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  unsigned _fn_i;                                                                \

+  char *_hf_key=(char*)(key);                                                    \

+  hashv = 2166136261UL;                                                          \

+  for(_fn_i=0; _fn_i < keylen; _fn_i++)                                          \

+      hashv = (hashv * 16777619) ^ _hf_key[_fn_i];                               \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while(0);

+ 

+#define HASH_OAT(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  unsigned _ho_i;                                                                \

+  char *_ho_key=(char*)(key);                                                    \

+  hashv = 0;                                                                     \

+  for(_ho_i=0; _ho_i < keylen; _ho_i++) {                                        \

+      hashv += _ho_key[_ho_i];                                                   \

+      hashv += (hashv << 10);                                                    \

+      hashv ^= (hashv >> 6);                                                     \

+  }                                                                              \

+  hashv += (hashv << 3);                                                         \

+  hashv ^= (hashv >> 11);                                                        \

+  hashv += (hashv << 15);                                                        \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while(0)

+

+#define HASH_JEN_MIX(a,b,c)                                                      \

+do {                                                                             \

+  a -= b; a -= c; a ^= ( c >> 13 );                                              \

+  b -= c; b -= a; b ^= ( a << 8 );                                               \

+  c -= a; c -= b; c ^= ( b >> 13 );                                              \

+  a -= b; a -= c; a ^= ( c >> 12 );                                              \

+  b -= c; b -= a; b ^= ( a << 16 );                                              \

+  c -= a; c -= b; c ^= ( b >> 5 );                                               \

+  a -= b; a -= c; a ^= ( c >> 3 );                                               \

+  b -= c; b -= a; b ^= ( a << 10 );                                              \

+  c -= a; c -= b; c ^= ( b >> 15 );                                              \

+} while (0)

+

+#define HASH_JEN(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  unsigned _hj_i,_hj_j,_hj_k;                                                    \

+  char *_hj_key=(char*)(key);                                                    \

+  hashv = 0xfeedbeef;                                                            \

+  _hj_i = _hj_j = 0x9e3779b9;                                                    \

+  _hj_k = keylen;                                                                \

+  while (_hj_k >= 12) {                                                          \

+    _hj_i +=    (_hj_key[0] + ( (unsigned)_hj_key[1] << 8 )                      \

+        + ( (unsigned)_hj_key[2] << 16 )                                         \

+        + ( (unsigned)_hj_key[3] << 24 ) );                                      \

+    _hj_j +=    (_hj_key[4] + ( (unsigned)_hj_key[5] << 8 )                      \

+        + ( (unsigned)_hj_key[6] << 16 )                                         \

+        + ( (unsigned)_hj_key[7] << 24 ) );                                      \

+    hashv += (_hj_key[8] + ( (unsigned)_hj_key[9] << 8 )                         \

+        + ( (unsigned)_hj_key[10] << 16 )                                        \

+        + ( (unsigned)_hj_key[11] << 24 ) );                                     \

+                                                                                 \

+     HASH_JEN_MIX(_hj_i, _hj_j, hashv);                                          \

+                                                                                 \

+     _hj_key += 12;                                                              \

+     _hj_k -= 12;                                                                \

+  }                                                                              \

+  hashv += keylen;                                                               \

+  switch ( _hj_k ) {                                                             \

+     case 11: hashv += ( (unsigned)_hj_key[10] << 24 );                          \

+     case 10: hashv += ( (unsigned)_hj_key[9] << 16 );                           \

+     case 9:  hashv += ( (unsigned)_hj_key[8] << 8 );                            \

+     case 8:  _hj_j += ( (unsigned)_hj_key[7] << 24 );                           \

+     case 7:  _hj_j += ( (unsigned)_hj_key[6] << 16 );                           \

+     case 6:  _hj_j += ( (unsigned)_hj_key[5] << 8 );                            \

+     case 5:  _hj_j += _hj_key[4];                                               \

+     case 4:  _hj_i += ( (unsigned)_hj_key[3] << 24 );                           \

+     case 3:  _hj_i += ( (unsigned)_hj_key[2] << 16 );                           \

+     case 2:  _hj_i += ( (unsigned)_hj_key[1] << 8 );                            \

+     case 1:  _hj_i += _hj_key[0];                                               \

+  }                                                                              \

+  HASH_JEN_MIX(_hj_i, _hj_j, hashv);                                             \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while(0)

+

+/* The Paul Hsieh hash function */

+#undef get16bits

+#if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__)             \

+  || defined(_MSC_VER) || defined (__BORLANDC__) || defined (__TURBOC__)

+#define get16bits(d) (*((const uint16_t *) (d)))

+#endif

+

+#if !defined (get16bits)

+#define get16bits(d) ((((uint32_t)(((const uint8_t *)(d))[1])) << 8)             \

+                       +(uint32_t)(((const uint8_t *)(d))[0]) )

+#endif

+#define HASH_SFH(key,keylen,num_bkts,hashv,bkt)                                  \

+do {                                                                             \

+  char *_sfh_key=(char*)(key);                                                   \

+  uint32_t _sfh_tmp, _sfh_len = keylen;                                          \

+                                                                                 \

+  int _sfh_rem = _sfh_len & 3;                                                   \

+  _sfh_len >>= 2;                                                                \

+  hashv = 0xcafebabe;                                                            \

+                                                                                 \

+  /* Main loop */                                                                \

+  for (;_sfh_len > 0; _sfh_len--) {                                              \

+    hashv    += get16bits (_sfh_key);                                            \

+    _sfh_tmp       = (get16bits (_sfh_key+2) << 11) ^ hashv;                     \

+    hashv     = (hashv << 16) ^ _sfh_tmp;                                        \

+    _sfh_key += 2*sizeof (uint16_t);                                             \

+    hashv    += hashv >> 11;                                                     \

+  }                                                                              \

+                                                                                 \

+  /* Handle end cases */                                                         \

+  switch (_sfh_rem) {                                                            \

+    case 3: hashv += get16bits (_sfh_key);                                       \

+            hashv ^= hashv << 16;                                                \

+            hashv ^= _sfh_key[sizeof (uint16_t)] << 18;                          \

+            hashv += hashv >> 11;                                                \

+            break;                                                               \

+    case 2: hashv += get16bits (_sfh_key);                                       \

+            hashv ^= hashv << 11;                                                \

+            hashv += hashv >> 17;                                                \

+            break;                                                               \

+    case 1: hashv += *_sfh_key;                                                  \

+            hashv ^= hashv << 10;                                                \

+            hashv += hashv >> 1;                                                 \

+  }                                                                              \

+                                                                                 \

+    /* Force "avalanching" of final 127 bits */                                  \

+    hashv ^= hashv << 3;                                                         \

+    hashv += hashv >> 5;                                                         \

+    hashv ^= hashv << 4;                                                         \

+    hashv += hashv >> 17;                                                        \

+    hashv ^= hashv << 25;                                                        \

+    hashv += hashv >> 6;                                                         \

+    bkt = hashv & (num_bkts-1);                                                  \

+} while(0);

+

+#ifdef HASH_USING_NO_STRICT_ALIASING

+/* The MurmurHash exploits some CPU's (e.g. x86) tolerance for unaligned reads.

+ * For other types of CPU's (e.g. Sparc) an unaligned read causes a bus error.

+ * So MurmurHash comes in two versions, the faster unaligned one and the slower

+ * aligned one. We only use the faster one on CPU's where we know it's safe. 

+ *

+ * Note the preprocessor built-in defines can be emitted using:

+ *

+ *   gcc -m64 -dM -E - < /dev/null                  (on gcc)

+ *   cc -## a.c (where a.c is a simple test file)   (Sun Studio)

+ */

+#if (defined(__i386__) || defined(__x86_64__)) 

+#define HASH_MUR HASH_MUR_UNALIGNED

+#else

+#define HASH_MUR HASH_MUR_ALIGNED

+#endif

+

+/* Appleby's MurmurHash fast version for unaligned-tolerant archs like i386 */

+#define HASH_MUR_UNALIGNED(key,keylen,num_bkts,hashv,bkt)                        \

+do {                                                                             \

+  const unsigned int _mur_m = 0x5bd1e995;                                        \

+  const int _mur_r = 24;                                                         \

+  hashv = 0xcafebabe ^ keylen;                                                   \

+  char *_mur_key = (char *)(key);                                                \

+  uint32_t _mur_tmp, _mur_len = keylen;                                          \

+                                                                                 \

+  for (;_mur_len >= 4; _mur_len-=4) {                                            \

+    _mur_tmp = *(uint32_t *)_mur_key;                                            \

+    _mur_tmp *= _mur_m;                                                          \

+    _mur_tmp ^= _mur_tmp >> _mur_r;                                              \

+    _mur_tmp *= _mur_m;                                                          \

+    hashv *= _mur_m;                                                             \

+    hashv ^= _mur_tmp;                                                           \

+    _mur_key += 4;                                                               \

+  }                                                                              \

+                                                                                 \

+  switch(_mur_len)                                                               \

+  {                                                                              \

+    case 3: hashv ^= _mur_key[2] << 16;                                          \

+    case 2: hashv ^= _mur_key[1] << 8;                                           \

+    case 1: hashv ^= _mur_key[0];                                                \

+            hashv *= _mur_m;                                                     \

+  };                                                                             \

+                                                                                 \

+  hashv ^= hashv >> 13;                                                          \

+  hashv *= _mur_m;                                                               \

+  hashv ^= hashv >> 15;                                                          \

+                                                                                 \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while(0)

+

+/* Appleby's MurmurHash version for alignment-sensitive archs like Sparc */

+#define HASH_MUR_ALIGNED(key,keylen,num_bkts,hashv,bkt)                          \

+do {                                                                             \

+  const unsigned int _mur_m = 0x5bd1e995;                                        \

+  const int _mur_r = 24;                                                         \

+  hashv = 0xcafebabe ^ (keylen);                                                 \

+  char *_mur_key = (char *)(key);                                                \

+  uint32_t _mur_len = keylen;                                                    \

+  int _mur_align = (int)_mur_key & 3;                                            \

+                                                                                 \

+  if (_mur_align && (_mur_len >= 4)) {                                           \

+    unsigned _mur_t = 0, _mur_d = 0;                                             \

+    switch(_mur_align) {                                                         \

+      case 1: _mur_t |= _mur_key[2] << 16;                                       \

+      case 2: _mur_t |= _mur_key[1] << 8;                                        \

+      case 3: _mur_t |= _mur_key[0];                                             \

+    }                                                                            \

+    _mur_t <<= (8 * _mur_align);                                                 \

+    _mur_key += 4-_mur_align;                                                    \

+    _mur_len -= 4-_mur_align;                                                    \

+    int _mur_sl = 8 * (4-_mur_align);                                            \

+    int _mur_sr = 8 * _mur_align;                                                \

+                                                                                 \

+    for (;_mur_len >= 4; _mur_len-=4) {                                          \

+      _mur_d = *(unsigned *)_mur_key;                                            \

+      _mur_t = (_mur_t >> _mur_sr) | (_mur_d << _mur_sl);                        \

+      unsigned _mur_k = _mur_t;                                                  \

+      _mur_k *= _mur_m;                                                          \

+      _mur_k ^= _mur_k >> _mur_r;                                                \

+      _mur_k *= _mur_m;                                                          \

+      hashv *= _mur_m;                                                           \

+      hashv ^= _mur_k;                                                           \

+      _mur_t = _mur_d;                                                           \

+      _mur_key += 4;                                                             \

+    }                                                                            \

+    _mur_d = 0;                                                                  \

+    if(_mur_len >= _mur_align) {                                                 \

+      switch(_mur_align) {                                                       \

+        case 3: _mur_d |= _mur_key[2] << 16;                                     \

+        case 2: _mur_d |= _mur_key[1] << 8;                                      \

+        case 1: _mur_d |= _mur_key[0];                                           \

+      }                                                                          \

+      unsigned _mur_k = (_mur_t >> _mur_sr) | (_mur_d << _mur_sl);               \

+      _mur_k *= _mur_m;                                                          \

+      _mur_k ^= _mur_k >> _mur_r;                                                \

+      _mur_k *= _mur_m;                                                          \

+      hashv *= _mur_m;                                                           \

+      hashv ^= _mur_k;                                                           \

+      _mur_k += _mur_align;                                                      \

+      _mur_len -= _mur_align;                                                    \

+                                                                                 \

+      switch(_mur_len)                                                           \

+      {                                                                          \

+        case 3: hashv ^= _mur_key[2] << 16;                                      \

+        case 2: hashv ^= _mur_key[1] << 8;                                       \

+        case 1: hashv ^= _mur_key[0];                                            \

+                hashv *= _mur_m;                                                 \

+      }                                                                          \

+    } else {                                                                     \

+      switch(_mur_len)                                                           \

+      {                                                                          \

+        case 3: _mur_d ^= _mur_key[2] << 16;                                     \

+        case 2: _mur_d ^= _mur_key[1] << 8;                                      \

+        case 1: _mur_d ^= _mur_key[0];                                           \

+        case 0: hashv ^= (_mur_t >> _mur_sr) | (_mur_d << _mur_sl);              \

+        hashv *= _mur_m;                                                         \

+      }                                                                          \

+    }                                                                            \

+                                                                                 \

+    hashv ^= hashv >> 13;                                                        \

+    hashv *= _mur_m;                                                             \

+    hashv ^= hashv >> 15;                                                        \

+  } else {                                                                       \

+    for (;_mur_len >= 4; _mur_len-=4) {                                          \

+      unsigned _mur_k = *(unsigned*)_mur_key;                                    \

+      _mur_k *= _mur_m;                                                          \

+      _mur_k ^= _mur_k >> _mur_r;                                                \

+      _mur_k *= _mur_m;                                                          \

+      hashv *= _mur_m;                                                           \

+      hashv ^= _mur_k;                                                           \

+      _mur_key += 4;                                                             \

+    }                                                                            \

+    switch(_mur_len)                                                             \

+    {                                                                            \

+      case 3: hashv ^= _mur_key[2] << 16;                                        \

+      case 2: hashv ^= _mur_key[1] << 8;                                         \

+      case 1: hashv ^= _mur_key[0];                                              \

+      hashv *= _mur_m;                                                           \

+    }                                                                            \

+                                                                                 \

+    hashv ^= hashv >> 13;                                                        \

+    hashv *= _mur_m;                                                             \

+    hashv ^= hashv >> 15;                                                        \

+  }                                                                              \

+  bkt = hashv & (num_bkts-1);                                                    \

+} while(0)

+#endif  /* HASH_USING_NO_STRICT_ALIASING */

+

+/* key comparison function; return 0 if keys equal */

+#define HASH_KEYCMP(a,b,len) memcmp(a,b,len) 

+

+/* iterate over items in a known bucket to find desired item */

+#define HASH_FIND_IN_BKT(tbl,hh,head,keyptr,keylen_in,out)                       \

+do {                                                                             \

+ if (head.hh_head) DECLTYPE_ASSIGN(out,ELMT_FROM_HH(tbl,head.hh_head));          \

+ else out=NULL;                                                                  \

+ while (out) {                                                                   \

+    if (out->hh.keylen == keylen_in) {                                           \

+        if ((HASH_KEYCMP(out->hh.key,keyptr,keylen_in)) == 0) break;             \

+    }                                                                            \

+    if (out->hh.hh_next) DECLTYPE_ASSIGN(out,ELMT_FROM_HH(tbl,out->hh.hh_next)); \

+    else out = NULL;                                                             \

+ }                                                                               \

+} while(0)

+

+/* add an item to a bucket  */

+#define HASH_ADD_TO_BKT(head,addhh)                                              \

+do {                                                                             \

+ head.count++;                                                                   \

+ (addhh)->hh_next = head.hh_head;                                                \

+ (addhh)->hh_prev = NULL;                                                        \

+ if (head.hh_head) { (head).hh_head->hh_prev = (addhh); }                        \

+ (head).hh_head=addhh;                                                           \

+ if (head.count >= ((head.expand_mult+1) * HASH_BKT_CAPACITY_THRESH)             \

+     && (addhh)->tbl->noexpand != 1) {                                           \

+       HASH_EXPAND_BUCKETS((addhh)->tbl);                                        \

+ }                                                                               \

+} while(0)

+

+/* remove an item from a given bucket */

+#define HASH_DEL_IN_BKT(hh,head,hh_del)                                          \

+    (head).count--;                                                              \

+    if ((head).hh_head == hh_del) {                                              \

+      (head).hh_head = hh_del->hh_next;                                          \

+    }                                                                            \

+    if (hh_del->hh_prev) {                                                       \

+        hh_del->hh_prev->hh_next = hh_del->hh_next;                              \

+    }                                                                            \

+    if (hh_del->hh_next) {                                                       \

+        hh_del->hh_next->hh_prev = hh_del->hh_prev;                              \

+    }                                                                

+

+/* Bucket expansion has the effect of doubling the number of buckets

+ * and redistributing the items into the new buckets. Ideally the

+ * items will distribute more or less evenly into the new buckets

+ * (the extent to which this is true is a measure of the quality of

+ * the hash function as it applies to the key domain). 

+ * 

+ * With the items distributed into more buckets, the chain length

+ * (item count) in each bucket is reduced. Thus by expanding buckets

+ * the hash keeps a bound on the chain length. This bounded chain 

+ * length is the essence of how a hash provides constant time lookup.

+ * 

+ * The calculation of tbl->ideal_chain_maxlen below deserves some

+ * explanation. First, keep in mind that we're calculating the ideal

+ * maximum chain length based on the *new* (doubled) bucket count.

+ * In fractions this is just n/b (n=number of items,b=new num buckets).

+ * Since the ideal chain length is an integer, we want to calculate 

+ * ceil(n/b). We don't depend on floating point arithmetic in this

+ * hash, so to calculate ceil(n/b) with integers we could write

+ * 

+ *      ceil(n/b) = (n/b) + ((n%b)?1:0)

+ * 

+ * and in fact a previous version of this hash did just that.

+ * But now we have improved things a bit by recognizing that b is

+ * always a power of two. We keep its base 2 log handy (call it lb),

+ * so now we can write this with a bit shift and logical AND:

+ * 

+ *      ceil(n/b) = (n>>lb) + ( (n & (b-1)) ? 1:0)

+ * 

+ */

+#define HASH_EXPAND_BUCKETS(tbl)                                                 \

+do {                                                                             \

+    unsigned _he_bkt;                                                            \

+    unsigned _he_bkt_i;                                                          \

+    struct UT_hash_handle *_he_thh, *_he_hh_nxt;                                 \

+    UT_hash_bucket *_he_new_buckets, *_he_newbkt;                                \

+    _he_new_buckets = (UT_hash_bucket*)uthash_malloc(                            \

+             2 * tbl->num_buckets * sizeof(struct UT_hash_bucket));              \

+    if (!_he_new_buckets) { uthash_fatal( "out of memory"); }                    \

+    memset(_he_new_buckets, 0,                                                   \

+            2 * tbl->num_buckets * sizeof(struct UT_hash_bucket));               \

+    tbl->ideal_chain_maxlen =                                                    \

+       (tbl->num_items >> (tbl->log2_num_buckets+1)) +                           \

+       ((tbl->num_items & ((tbl->num_buckets*2)-1)) ? 1 : 0);                    \

+    tbl->nonideal_items = 0;                                                     \

+    for(_he_bkt_i = 0; _he_bkt_i < tbl->num_buckets; _he_bkt_i++)                \

+    {                                                                            \

+        _he_thh = tbl->buckets[ _he_bkt_i ].hh_head;                             \

+        while (_he_thh) {                                                        \

+           _he_hh_nxt = _he_thh->hh_next;                                        \

+           HASH_TO_BKT( _he_thh->hashv, tbl->num_buckets*2, _he_bkt);            \

+           _he_newbkt = &(_he_new_buckets[ _he_bkt ]);                           \

+           if (++(_he_newbkt->count) > tbl->ideal_chain_maxlen) {                \

+             tbl->nonideal_items++;                                              \

+             _he_newbkt->expand_mult = _he_newbkt->count /                       \

+                                        tbl->ideal_chain_maxlen;                 \

+           }                                                                     \

+           _he_thh->hh_prev = NULL;                                              \

+           _he_thh->hh_next = _he_newbkt->hh_head;                               \

+           if (_he_newbkt->hh_head) _he_newbkt->hh_head->hh_prev =               \

+                _he_thh;                                                         \

+           _he_newbkt->hh_head = _he_thh;                                        \

+           _he_thh = _he_hh_nxt;                                                 \

+        }                                                                        \

+    }                                                                            \

+    uthash_free( tbl->buckets, tbl->num_buckets*sizeof(struct UT_hash_bucket) );\

+    tbl->num_buckets *= 2;                                                       \

+    tbl->log2_num_buckets++;                                                     \

+    tbl->buckets = _he_new_buckets;                                              \

+    tbl->ineff_expands = (tbl->nonideal_items > (tbl->num_items >> 1)) ?         \

+        (tbl->ineff_expands+1) : 0;                                              \

+    if (tbl->ineff_expands > 1) {                                                \

+        tbl->noexpand=1;                                                         \

+        uthash_noexpand_fyi(tbl);                                                \

+    }                                                                            \

+    uthash_expand_fyi(tbl);                                                      \

+} while(0)

+

+

+/* This is an adaptation of Simon Tatham's O(n log(n)) mergesort */

+/* Note that HASH_SORT assumes the hash handle name to be hh. 

+ * HASH_SRT was added to allow the hash handle name to be passed in. */

+#define HASH_SORT(head,cmpfcn) HASH_SRT(hh,head,cmpfcn)

+#define HASH_SRT(hh,head,cmpfcn)                                                 \

+do {                                                                             \

+  unsigned _hs_i;                                                                \

+  unsigned _hs_looping,_hs_nmerges,_hs_insize,_hs_psize,_hs_qsize;               \

+  struct UT_hash_handle *_hs_p, *_hs_q, *_hs_e, *_hs_list, *_hs_tail;            \

+  if (head) {                                                                    \

+      _hs_insize = 1;                                                            \

+      _hs_looping = 1;                                                           \

+      _hs_list = &((head)->hh);                                                  \

+      while (_hs_looping) {                                                      \

+          _hs_p = _hs_list;                                                      \

+          _hs_list = NULL;                                                       \

+          _hs_tail = NULL;                                                       \

+          _hs_nmerges = 0;                                                       \

+          while (_hs_p) {                                                        \

+              _hs_nmerges++;                                                     \

+              _hs_q = _hs_p;                                                     \

+              _hs_psize = 0;                                                     \

+              for ( _hs_i = 0; _hs_i  < _hs_insize; _hs_i++ ) {                  \

+                  _hs_psize++;                                                   \

+                  _hs_q = (UT_hash_handle*)((_hs_q->next) ?                      \

+                          ((void*)((char*)(_hs_q->next) +                        \

+                          (head)->hh.tbl->hho)) : NULL);                         \

+                  if (! (_hs_q) ) break;                                         \

+              }                                                                  \

+              _hs_qsize = _hs_insize;                                            \

+              while ((_hs_psize > 0) || ((_hs_qsize > 0) && _hs_q )) {           \

+                  if (_hs_psize == 0) {                                          \

+                      _hs_e = _hs_q;                                             \

+                      _hs_q = (UT_hash_handle*)((_hs_q->next) ?                  \

+                              ((void*)((char*)(_hs_q->next) +                    \

+                              (head)->hh.tbl->hho)) : NULL);                     \

+                      _hs_qsize--;                                               \

+                  } else if ( (_hs_qsize == 0) || !(_hs_q) ) {                   \

+                      _hs_e = _hs_p;                                             \

+                      _hs_p = (UT_hash_handle*)((_hs_p->next) ?                  \

+                              ((void*)((char*)(_hs_p->next) +                    \

+                              (head)->hh.tbl->hho)) : NULL);                     \

+                      _hs_psize--;                                               \

+                  } else if ((                                                   \

+                      cmpfcn(DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_p)), \

+                             DECLTYPE(head)(ELMT_FROM_HH((head)->hh.tbl,_hs_q))) \

+                             ) <= 0) {                                           \

+                      _hs_e = _hs_p;                                             \

+                      _hs_p = (UT_hash_handle*)((_hs_p->next) ?                  \

+                              ((void*)((char*)(_hs_p->next) +                    \

+                              (head)->hh.tbl->hho)) : NULL);                     \

+                      _hs_psize--;                                               \

+                  } else {                                                       \

+                      _hs_e = _hs_q;                                             \

+                      _hs_q = (UT_hash_handle*)((_hs_q->next) ?                  \

+                              ((void*)((char*)(_hs_q->next) +                    \

+                              (head)->hh.tbl->hho)) : NULL);                     \

+                      _hs_qsize--;                                               \

+                  }                                                              \

+                  if ( _hs_tail ) {                                              \

+                      _hs_tail->next = ((_hs_e) ?                                \

+                            ELMT_FROM_HH((head)->hh.tbl,_hs_e) : NULL);          \

+                  } else {                                                       \

+                      _hs_list = _hs_e;                                          \

+                  }                                                              \

+                  _hs_e->prev = ((_hs_tail) ?                                    \

+                     ELMT_FROM_HH((head)->hh.tbl,_hs_tail) : NULL);              \

+                  _hs_tail = _hs_e;                                              \

+              }                                                                  \

+              _hs_p = _hs_q;                                                     \

+          }                                                                      \

+          _hs_tail->next = NULL;                                                 \

+          if ( _hs_nmerges <= 1 ) {                                              \

+              _hs_looping=0;                                                     \

+              (head)->hh.tbl->tail = _hs_tail;                                   \

+              DECLTYPE_ASSIGN(head,ELMT_FROM_HH((head)->hh.tbl, _hs_list));      \

+          }                                                                      \

+          _hs_insize *= 2;                                                       \

+      }                                                                          \

+      HASH_FSCK(hh,head);                                                        \

+ }                                                                               \

+} while (0)

+

+/* This function selects items from one hash into another hash. 

+ * The end result is that the selected items have dual presence 

+ * in both hashes. There is no copy of the items made; rather 

+ * they are added into the new hash through a secondary hash 

+ * hash handle that must be present in the structure. */

+#define HASH_SELECT(hh_dst, dst, hh_src, src, cond)                              \

+do {                                                                             \

+  unsigned _src_bkt, _dst_bkt;                                                   \

+  void *_last_elt=NULL, *_elt;                                                   \

+  UT_hash_handle *_src_hh, *_dst_hh, *_last_elt_hh=NULL;                         \

+  ptrdiff_t _dst_hho = ((char*)(&(dst)->hh_dst) - (char*)(dst));                 \

+  if (src) {                                                                     \

+    for(_src_bkt=0; _src_bkt < (src)->hh_src.tbl->num_buckets; _src_bkt++) {     \

+      for(_src_hh = (src)->hh_src.tbl->buckets[_src_bkt].hh_head;                \

+          _src_hh;                                                               \

+          _src_hh = _src_hh->hh_next) {                                          \

+          _elt = ELMT_FROM_HH((src)->hh_src.tbl, _src_hh);                       \

+          if (cond(_elt)) {                                                      \

+            _dst_hh = (UT_hash_handle*)(((char*)_elt) + _dst_hho);               \

+            _dst_hh->key = _src_hh->key;                                         \

+            _dst_hh->keylen = _src_hh->keylen;                                   \

+            _dst_hh->hashv = _src_hh->hashv;                                     \

+            _dst_hh->prev = _last_elt;                                           \

+            _dst_hh->next = NULL;                                                \

+            if (_last_elt_hh) { _last_elt_hh->next = _elt; }                     \

+            if (!dst) {                                                          \

+              DECLTYPE_ASSIGN(dst,_elt);                                         \

+              HASH_MAKE_TABLE(hh_dst,dst);                                       \

+            } else {                                                             \

+              _dst_hh->tbl = (dst)->hh_dst.tbl;                                  \

+            }                                                                    \

+            HASH_TO_BKT(_dst_hh->hashv, _dst_hh->tbl->num_buckets, _dst_bkt);    \

+            HASH_ADD_TO_BKT(_dst_hh->tbl->buckets[_dst_bkt],_dst_hh);            \

+            (dst)->hh_dst.tbl->num_items++;                                      \

+            _last_elt = _elt;                                                    \

+            _last_elt_hh = _dst_hh;                                              \

+          }                                                                      \

+      }                                                                          \

+    }                                                                            \

+  }                                                                              \

+  HASH_FSCK(hh_dst,dst);                                                         \

+} while (0)

+

+#define HASH_CLEAR(hh,head)                                                      \

+do {                                                                             \

+  if (head) {                                                                    \

+    uthash_free((head)->hh.tbl->buckets,                                        \

+                (head)->hh.tbl->num_buckets*sizeof(struct UT_hash_bucket));     \

+    uthash_free((head)->hh.tbl, sizeof(UT_hash_table));                         \

+    (head)=NULL;                                                                 \

+  }                                                                              \

+} while(0)

+

+#define HASH_ITER(hh,head,el,tmp)                                               \

+for((el)=(head),(tmp)=(head)?(head)->hh.next:NULL;                              \

+  el; (el)=(tmp),(tmp)=(tmp)?(tmp)->hh.next:NULL) 

+

+/* obtain a count of items in the hash */

+#define HASH_COUNT(head) HASH_CNT(hh,head) 

+#define HASH_CNT(hh,head) ((head)?((head)->hh.tbl->num_items):0)

+

+typedef struct UT_hash_bucket {

+   struct UT_hash_handle *hh_head;

+   unsigned count;

+

+   /* expand_mult is normally set to 0. In this situation, the max chain length

+    * threshold is enforced at its default value, HASH_BKT_CAPACITY_THRESH. (If

+    * the bucket's chain exceeds this length, bucket expansion is triggered). 

+    * However, setting expand_mult to a non-zero value delays bucket expansion

+    * (that would be triggered by additions to this particular bucket)

+    * until its chain length reaches a *multiple* of HASH_BKT_CAPACITY_THRESH.

+    * (The multiplier is simply expand_mult+1). The whole idea of this

+    * multiplier is to reduce bucket expansions, since they are expensive, in

+    * situations where we know that a particular bucket tends to be overused.

+    * It is better to let its chain length grow to a longer yet-still-bounded

+    * value, than to do an O(n) bucket expansion too often. 

+    */

+   unsigned expand_mult;

+

+} UT_hash_bucket;

+

+/* random signature used only to find hash tables in external analysis */

+#define HASH_SIGNATURE 0xa0111fe1

+#define HASH_BLOOM_SIGNATURE 0xb12220f2

+

+typedef struct UT_hash_table {

+   UT_hash_bucket *buckets;

+   unsigned num_buckets, log2_num_buckets;

+   unsigned num_items;

+   struct UT_hash_handle *tail; /* tail hh in app order, for fast append    */

+   ptrdiff_t hho; /* hash handle offset (byte pos of hash handle in element */

+

+   /* in an ideal situation (all buckets used equally), no bucket would have

+    * more than ceil(#items/#buckets) items. that's the ideal chain length. */

+   unsigned ideal_chain_maxlen;

+

+   /* nonideal_items is the number of items in the hash whose chain position

+    * exceeds the ideal chain maxlen. these items pay the penalty for an uneven

+    * hash distribution; reaching them in a chain traversal takes >ideal steps */

+   unsigned nonideal_items;

+

+   /* ineffective expands occur when a bucket doubling was performed, but 

+    * afterward, more than half the items in the hash had nonideal chain

+    * positions. If this happens on two consecutive expansions we inhibit any

+    * further expansion, as it's not helping; this happens when the hash

+    * function isn't a good fit for the key domain. When expansion is inhibited

+    * the hash will still work, albeit no longer in constant time. */

+   unsigned ineff_expands, noexpand;

+

+   uint32_t signature; /* used only to find hash tables in external analysis */

+#ifdef HASH_BLOOM

+   uint32_t bloom_sig; /* used only to test bloom exists in external analysis */

+   uint8_t *bloom_bv;

+   char bloom_nbits;

+#endif

+

+} UT_hash_table;

+

+typedef struct UT_hash_handle {

+   struct UT_hash_table *tbl;

+   void *prev;                       /* prev element in app order      */

+   void *next;                       /* next element in app order      */

+   struct UT_hash_handle *hh_prev;   /* previous hh in bucket order    */

+   struct UT_hash_handle *hh_next;   /* next hh in bucket order        */

+   void *key;                        /* ptr to enclosing struct's key  */

+   unsigned keylen;                  /* enclosing struct's key len     */

+   unsigned hashv;                   /* result of hash-fcn(key)        */

+} UT_hash_handle;

+

+#endif /* UTHASH_H */

</span></code></pre>

<br>

</li>

<li id='diff-41'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-41'>

<strong>

darshan-runtime/utlist.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/utlist.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,490 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+Copyright (c) 2007-2010, Troy D. Hanson   http://uthash.sourceforge.net

+All rights reserved.

+

+Redistribution and use in source and binary forms, with or without

+modification, are permitted provided that the following conditions are met:

+

+    * Redistributions of source code must retain the above copyright

+      notice, this list of conditions and the following disclaimer.

+

+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS

+IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED

+TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A

+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER

+OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,

+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,

+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR

+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF

+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING

+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS

+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+*/

+

+#ifndef UTLIST_H

+#define UTLIST_H

+

+#define UTLIST_VERSION 1.9.1

+

+/* 

+ * This file contains macros to manipulate singly and doubly-linked lists.

+ *

+ * 1. LL_ macros:  singly-linked lists.

+ * 2. DL_ macros:  doubly-linked lists.

+ * 3. CDL_ macros: circular doubly-linked lists.

+ *

+ * To use singly-linked lists, your structure must have a "next" pointer.

+ * To use doubly-linked lists, your structure must "prev" and "next" pointers.

+ * Either way, the pointer to the head of the list must be initialized to NULL.

+ * 

+ * ----------------.EXAMPLE -------------------------

+ * struct item {

+ *      int id;

+ *      struct item *prev, *next;

+ * }

+ *

+ * struct item *list = NULL:

+ *

+ * int main() {

+ *      struct item *item;

+ *      ... allocate and populate item ...

+ *      DL_APPEND(list, item);

+ * }

+ * --------------------------------------------------

+ *

+ * For doubly-linked lists, the append and delete macros are O(1)

+ * For singly-linked lists, append and delete are O(n) but prepend is O(1)

+ * The sort macro is O(n log(n)) for all types of single/double/circular lists.

+ */

+

+/* These macros use decltype or the earlier __typeof GNU extension.

+   As decltype is only available in newer compilers (VS2010 or gcc 4.3+

+   when compiling c++ code), this code uses whatever method is needed

+   or, for VS2008 where neither is available, uses casting workarounds. */

+#ifdef _MSC_VER            /* MS compiler */

+#if _MSC_VER >= 1600 && __cplusplus  /* VS2010 and newer in C++ mode */

+#define LDECLTYPE(x) decltype(x)

+#else                     /* VS2008 or older (or VS2010 in C mode) */

+#define NO_DECLTYPE

+#define LDECLTYPE(x) char*

+#endif

+#else                      /* GNU, Sun and other compilers */

+#define LDECLTYPE(x) __typeof(x)

+#endif

+

+/* for VS2008 we use some workarounds to get around the lack of decltype,

+ * namely, we always reassign our tmp variable to the list head if we need

+ * to dereference its prev/next pointers, and save/restore the real head.*/

+#ifdef NO_DECLTYPE

+#define _SV(elt,list) _tmp = (char*)(list); {char **_alias = (char**)&(list); *_alias = (elt); }

+#define _NEXT(elt,list) ((char*)((list)->next))

+#define _NEXTASGN(elt,list,to) { char **_alias = (char**)&((list)->next); *_alias=(char*)(to); }

+#define _PREV(elt,list) ((char*)((list)->prev))

+#define _PREVASGN(elt,list,to) { char **_alias = (char**)&((list)->prev); *_alias=(char*)(to); }

+#define _RS(list) { char **_alias = (char**)&(list); *_alias=_tmp; }

+#define _CASTASGN(a,b) { char **_alias = (char**)&(a); *_alias=(char*)(b); }

+#else 

+#define _SV(elt,list)

+#define _NEXT(elt,list) ((elt)->next)

+#define _NEXTASGN(elt,list,to) ((elt)->next)=(to)

+#define _PREV(elt,list) ((elt)->prev)

+#define _PREVASGN(elt,list,to) ((elt)->prev)=(to)

+#define _RS(list)

+#define _CASTASGN(a,b) (a)=(b)

+#endif

+

+/******************************************************************************

+ * The sort macro is an adaptation of Simon Tatham's O(n log(n)) mergesort    *

+ * Unwieldy variable names used here to avoid shadowing passed-in variables.  *

+ *****************************************************************************/

+#define LL_SORT(list, cmp)                                                                     \

+do {                                                                                           \

+  LDECLTYPE(list) _ls_p;                                                                       \

+  LDECLTYPE(list) _ls_q;                                                                       \

+  LDECLTYPE(list) _ls_e;                                                                       \

+  LDECLTYPE(list) _ls_tail;                                                                    \

+  LDECLTYPE(list) _ls_oldhead;                                                                 \

+  LDECLTYPE(list) _tmp;                                                                        \

+  int _ls_insize, _ls_nmerges, _ls_psize, _ls_qsize, _ls_i, _ls_looping;                       \

+  if (list) {                                                                                  \

+    _ls_insize = 1;                                                                            \

+    _ls_looping = 1;                                                                           \

+    while (_ls_looping) {                                                                      \

+      _CASTASGN(_ls_p,list);                                                                   \

+      _CASTASGN(_ls_oldhead,list);                                                             \

+      list = NULL;                                                                             \

+      _ls_tail = NULL;                                                                         \

+      _ls_nmerges = 0;                                                                         \

+      while (_ls_p) {                                                                          \

+        _ls_nmerges++;                                                                         \

+        _ls_q = _ls_p;                                                                         \

+        _ls_psize = 0;                                                                         \

+        for (_ls_i = 0; _ls_i < _ls_insize; _ls_i++) {                                         \

+          _ls_psize++;                                                                         \

+          _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list);                               \

+          if (!_ls_q) break;                                                                   \

+        }                                                                                      \

+        _ls_qsize = _ls_insize;                                                                \

+        while (_ls_psize > 0 || (_ls_qsize > 0 && _ls_q)) {                                    \

+          if (_ls_psize == 0) {                                                                \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+          } else if (_ls_qsize == 0 || !_ls_q) {                                               \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+          } else if (cmp(_ls_p,_ls_q) <= 0) {                                                  \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+          } else {                                                                             \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+          }                                                                                    \

+          if (_ls_tail) {                                                                      \

+            _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,_ls_e); _RS(list);                     \

+          } else {                                                                             \

+            _CASTASGN(list,_ls_e);                                                             \

+          }                                                                                    \

+          _ls_tail = _ls_e;                                                                    \

+        }                                                                                      \

+        _ls_p = _ls_q;                                                                         \

+      }                                                                                        \

+      _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,NULL); _RS(list);                            \

+      if (_ls_nmerges <= 1) {                                                                  \

+        _ls_looping=0;                                                                         \

+      }                                                                                        \

+      _ls_insize *= 2;                                                                         \

+    }                                                                                          \

+  } else _tmp=NULL; /* quiet gcc unused variable warning */                                    \

+} while (0)

+

+#define DL_SORT(list, cmp)                                                                     \

+do {                                                                                           \

+  LDECLTYPE(list) _ls_p;                                                                       \

+  LDECLTYPE(list) _ls_q;                                                                       \

+  LDECLTYPE(list) _ls_e;                                                                       \

+  LDECLTYPE(list) _ls_tail;                                                                    \

+  LDECLTYPE(list) _ls_oldhead;                                                                 \

+  LDECLTYPE(list) _tmp;                                                                        \

+  int _ls_insize, _ls_nmerges, _ls_psize, _ls_qsize, _ls_i, _ls_looping;                       \

+  if (list) {                                                                                  \

+    _ls_insize = 1;                                                                            \

+    _ls_looping = 1;                                                                           \

+    while (_ls_looping) {                                                                      \

+      _CASTASGN(_ls_p,list);                                                                   \

+      _CASTASGN(_ls_oldhead,list);                                                             \

+      list = NULL;                                                                             \

+      _ls_tail = NULL;                                                                         \

+      _ls_nmerges = 0;                                                                         \

+      while (_ls_p) {                                                                          \

+        _ls_nmerges++;                                                                         \

+        _ls_q = _ls_p;                                                                         \

+        _ls_psize = 0;                                                                         \

+        for (_ls_i = 0; _ls_i < _ls_insize; _ls_i++) {                                         \

+          _ls_psize++;                                                                         \

+          _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list);                               \

+          if (!_ls_q) break;                                                                   \

+        }                                                                                      \

+        _ls_qsize = _ls_insize;                                                                \

+        while (_ls_psize > 0 || (_ls_qsize > 0 && _ls_q)) {                                    \

+          if (_ls_psize == 0) {                                                                \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+          } else if (_ls_qsize == 0 || !_ls_q) {                                               \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+          } else if (cmp(_ls_p,_ls_q) <= 0) {                                                  \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+          } else {                                                                             \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+          }                                                                                    \

+          if (_ls_tail) {                                                                      \

+            _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,_ls_e); _RS(list);                     \

+          } else {                                                                             \

+            _CASTASGN(list,_ls_e);                                                             \

+          }                                                                                    \

+          _SV(_ls_e,list); _PREVASGN(_ls_e,list,_ls_tail); _RS(list);                          \

+          _ls_tail = _ls_e;                                                                    \

+        }                                                                                      \

+        _ls_p = _ls_q;                                                                         \

+      }                                                                                        \

+      _CASTASGN(list->prev, _ls_tail);                                                         \

+      _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,NULL); _RS(list);                            \

+      if (_ls_nmerges <= 1) {                                                                  \

+        _ls_looping=0;                                                                         \

+      }                                                                                        \

+      _ls_insize *= 2;                                                                         \

+    }                                                                                          \

+  } else _tmp=NULL; /* quiet gcc unused variable warning */                                    \

+} while (0)

+

+#define CDL_SORT(list, cmp)                                                                    \

+do {                                                                                           \

+  LDECLTYPE(list) _ls_p;                                                                       \

+  LDECLTYPE(list) _ls_q;                                                                       \

+  LDECLTYPE(list) _ls_e;                                                                       \

+  LDECLTYPE(list) _ls_tail;                                                                    \

+  LDECLTYPE(list) _ls_oldhead;                                                                 \

+  LDECLTYPE(list) _tmp;                                                                        \

+  LDECLTYPE(list) _tmp2;                                                                       \

+  int _ls_insize, _ls_nmerges, _ls_psize, _ls_qsize, _ls_i, _ls_looping;                       \

+  if (list) {                                                                                  \

+    _ls_insize = 1;                                                                            \

+    _ls_looping = 1;                                                                           \

+    while (_ls_looping) {                                                                      \

+      _CASTASGN(_ls_p,list);                                                                   \

+      _CASTASGN(_ls_oldhead,list);                                                             \

+      list = NULL;                                                                             \

+      _ls_tail = NULL;                                                                         \

+      _ls_nmerges = 0;                                                                         \

+      while (_ls_p) {                                                                          \

+        _ls_nmerges++;                                                                         \

+        _ls_q = _ls_p;                                                                         \

+        _ls_psize = 0;                                                                         \

+        for (_ls_i = 0; _ls_i < _ls_insize; _ls_i++) {                                         \

+          _ls_psize++;                                                                         \

+          _SV(_ls_q,list);                                                                     \

+          if (_NEXT(_ls_q,list) == _ls_oldhead) {                                              \

+            _ls_q = NULL;                                                                      \

+          } else {                                                                             \

+            _ls_q = _NEXT(_ls_q,list);                                                         \

+          }                                                                                    \

+          _RS(list);                                                                           \

+          if (!_ls_q) break;                                                                   \

+        }                                                                                      \

+        _ls_qsize = _ls_insize;                                                                \

+        while (_ls_psize > 0 || (_ls_qsize > 0 && _ls_q)) {                                    \

+          if (_ls_psize == 0) {                                                                \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+            if (_ls_q == _ls_oldhead) { _ls_q = NULL; }                                        \

+          } else if (_ls_qsize == 0 || !_ls_q) {                                               \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+            if (_ls_p == _ls_oldhead) { _ls_p = NULL; }                                        \

+          } else if (cmp(_ls_p,_ls_q) <= 0) {                                                  \

+            _ls_e = _ls_p; _SV(_ls_p,list); _ls_p = _NEXT(_ls_p,list); _RS(list); _ls_psize--; \

+            if (_ls_p == _ls_oldhead) { _ls_p = NULL; }                                        \

+          } else {                                                                             \

+            _ls_e = _ls_q; _SV(_ls_q,list); _ls_q = _NEXT(_ls_q,list); _RS(list); _ls_qsize--; \

+            if (_ls_q == _ls_oldhead) { _ls_q = NULL; }                                        \

+          }                                                                                    \

+          if (_ls_tail) {                                                                      \

+            _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,_ls_e); _RS(list);                     \

+          } else {                                                                             \

+            _CASTASGN(list,_ls_e);                                                             \

+          }                                                                                    \

+          _SV(_ls_e,list); _PREVASGN(_ls_e,list,_ls_tail); _RS(list);                          \

+          _ls_tail = _ls_e;                                                                    \

+        }                                                                                      \

+        _ls_p = _ls_q;                                                                         \

+      }                                                                                        \

+      _CASTASGN(list->prev,_ls_tail);                                                          \

+      _CASTASGN(_tmp2,list);                                                                   \

+      _SV(_ls_tail,list); _NEXTASGN(_ls_tail,list,_tmp2); _RS(list);                           \

+      if (_ls_nmerges <= 1) {                                                                  \

+        _ls_looping=0;                                                                         \

+      }                                                                                        \

+      _ls_insize *= 2;                                                                         \

+    }                                                                                          \

+  } else _tmp=NULL; /* quiet gcc unused variable warning */                                    \

+} while (0)

+

+/******************************************************************************

+ * singly linked list macros (non-circular)                                   *

+ *****************************************************************************/

+#define LL_PREPEND(head,add)                                                                   \

+do {                                                                                           \

+  (add)->next = head;                                                                          \

+  head = add;                                                                                  \

+} while (0)

+

+#define LL_APPEND(head,add)                                                                    \

+do {                                                                                           \

+  LDECLTYPE(head) _tmp;                                                                        \

+  (add)->next=NULL;                                                                            \

+  if (head) {                                                                                  \

+    _tmp = head;                                                                               \

+    while (_tmp->next) { _tmp = _tmp->next; }                                                  \

+    _tmp->next=(add);                                                                          \

+  } else {                                                                                     \

+    (head)=(add);                                                                              \

+  }                                                                                            \

+} while (0)

+

+#define LL_DELETE(head,del)                                                                    \

+do {                                                                                           \

+  LDECLTYPE(head) _tmp;                                                                        \

+  if ((head) == (del)) {                                                                       \

+    (head)=(head)->next;                                                                       \

+  } else {                                                                                     \

+    _tmp = head;                                                                               \

+    while (_tmp->next && (_tmp->next != (del))) {                                              \

+      _tmp = _tmp->next;                                                                       \

+    }                                                                                          \

+    if (_tmp->next) {                                                                          \

+      _tmp->next = ((del)->next);                                                              \

+    }                                                                                          \

+  }                                                                                            \

+} while (0)

+

+/* Here are VS2008 replacements for LL_APPEND and LL_DELETE */

+#define LL_APPEND_VS2008(head,add)                                                             \

+do {                                                                                           \

+  if (head) {                                                                                  \

+    (add)->next = head;     /* use add->next as a temp variable */                             \

+    while ((add)->next->next) { (add)->next = (add)->next->next; }                             \

+    (add)->next->next=(add);                                                                   \

+  } else {                                                                                     \

+    (head)=(add);                                                                              \

+  }                                                                                            \

+  (add)->next=NULL;                                                                            \

+} while (0)

+

+#define LL_DELETE_VS2008(head,del)                                                             \

+do {                                                                                           \

+  if ((head) == (del)) {                                                                       \

+    (head)=(head)->next;                                                                       \

+  } else {                                                                                     \

+    char *_tmp = (char*)(head);                                                                \

+    while (head->next && (head->next != (del))) {                                              \

+      head = head->next;                                                                       \

+    }                                                                                          \

+    if (head->next) {                                                                          \

+      head->next = ((del)->next);                                                              \

+    }                                                                                          \

+    {                                                                                          \

+      char **_head_alias = (char**)&(head);                                                    \

+      *_head_alias = _tmp;                                                                     \

+    }                                                                                          \

+  }                                                                                            \

+} while (0)

+#ifdef NO_DECLTYPE

+#undef LL_APPEND

+#define LL_APPEND LL_APPEND_VS2008

+#undef LL_DELETE

+#define LL_DELETE LL_DELETE_VS2008

+#endif

+/* end VS2008 replacements */

+

+#define LL_FOREACH(head,el)                                                                    \

+    for(el=head;el;el=el->next)

+

+#define LL_FOREACH_SAFE(head,el,tmp)                                                           \

+  for((el)=(head);(el) && (tmp = (el)->next, 1); (el) = tmp)

+

+#define LL_SEARCH_SCALAR(head,out,field,val)                                                   \

+do {                                                                                           \

+    LL_FOREACH(head,out) {                                                                     \

+      if ((out)->field == (val)) break;                                                        \

+    }                                                                                          \

+} while(0) 

+

+#define LL_SEARCH(head,out,elt,cmp)                                                            \

+do {                                                                                           \

+    LL_FOREACH(head,out) {                                                                     \

+      if ((cmp(out,elt))==0) break;                                                            \

+    }                                                                                          \

+} while(0) 

+

+/******************************************************************************

+ * doubly linked list macros (non-circular)                                   *

+ *****************************************************************************/

+#define DL_PREPEND(head,add)                                                                   \

+do {                                                                                           \

+ (add)->next = head;                                                                           \

+ if (head) {                                                                                   \

+   (add)->prev = (head)->prev;                                                                 \

+   (head)->prev = (add);                                                                       \

+ } else {                                                                                      \

+   (add)->prev = (add);                                                                        \

+ }                                                                                             \

+ (head) = (add);                                                                               \

+} while (0)

+

+#define DL_APPEND(head,add)                                                                    \

+do {                                                                                           \

+  if (head) {                                                                                  \

+      (add)->prev = (head)->prev;                                                              \

+      (head)->prev->next = (add);                                                              \

+      (head)->prev = (add);                                                                    \

+      (add)->next = NULL;                                                                      \

+  } else {                                                                                     \

+      (head)=(add);                                                                            \

+      (head)->prev = (head);                                                                   \

+      (head)->next = NULL;                                                                     \

+  }                                                                                            \

+} while (0);

+

+#define DL_DELETE(head,del)                                                                    \

+do {                                                                                           \

+  if ((del)->prev == (del)) {                                                                  \

+      (head)=NULL;                                                                             \

+  } else if ((del)==(head)) {                                                                  \

+      (del)->next->prev = (del)->prev;                                                         \

+      (head) = (del)->next;                                                                    \

+  } else {                                                                                     \

+      (del)->prev->next = (del)->next;                                                         \

+      if ((del)->next) {                                                                       \

+          (del)->next->prev = (del)->prev;                                                     \

+      } else {                                                                                 \

+          (head)->prev = (del)->prev;                                                          \

+      }                                                                                        \

+  }                                                                                            \

+} while (0);

+

+

+#define DL_FOREACH(head,el)                                                                    \

+    for(el=head;el;el=el->next)

+

+/* this version is safe for deleting the elements during iteration */

+#define DL_FOREACH_SAFE(head,el,tmp)                                                           \

+  for((el)=(head);(el) && (tmp = (el)->next, 1); (el) = tmp)

+

+/* these are identical to their singly-linked list counterparts */

+#define DL_SEARCH_SCALAR LL_SEARCH_SCALAR

+#define DL_SEARCH LL_SEARCH

+

+/******************************************************************************

+ * circular doubly linked list macros                                         *

+ *****************************************************************************/

+#define CDL_PREPEND(head,add)                                                                  \

+do {                                                                                           \

+ if (head) {                                                                                   \

+   (add)->prev = (head)->prev;                                                                 \

+   (add)->next = (head);                                                                       \

+   (head)->prev = (add);                                                                       \

+   (add)->prev->next = (add);                                                                  \

+ } else {                                                                                      \

+   (add)->prev = (add);                                                                        \

+   (add)->next = (add);                                                                        \

+ }                                                                                             \

+(head)=(add);                                                                                  \

+} while (0)

+

+#define CDL_DELETE(head,del)                                                                   \

+do {                                                                                           \

+  if ( ((head)==(del)) && ((head)->next == (head))) {                                          \

+      (head) = 0L;                                                                             \

+  } else {                                                                                     \

+     (del)->next->prev = (del)->prev;                                                          \

+     (del)->prev->next = (del)->next;                                                          \

+     if ((del) == (head)) (head)=(del)->next;                                                  \

+  }                                                                                            \

+} while (0);

+

+#define CDL_FOREACH(head,el)                                                                   \

+    for(el=head;el;el=(el->next==head ? 0L : el->next)) 

+

+#define CDL_FOREACH_SAFE(head,el,tmp1,tmp2)                                                    \

+  for((el)=(head), ((tmp1)=(head)?((head)->prev):NULL);                                        \

+      (el) && ((tmp2)=(el)->next, 1);                                                          \

+      ((el) = (((el)==(tmp1)) ? 0L : (tmp2))))

+

+#define CDL_SEARCH_SCALAR(head,out,field,val)                                                  \

+do {                                                                                           \

+    CDL_FOREACH(head,out) {                                                                    \

+      if ((out)->field == (val)) break;                                                        \

+    }                                                                                          \

+} while(0) 

+

+#define CDL_SEARCH(head,out,elt,cmp)                                                           \

+do {                                                                                           \

+    CDL_FOREACH(head,out) {                                                                    \

+      if ((cmp(out,elt))==0) break;                                                            \

+    }                                                                                          \

+} while(0) 

+

+#endif /* UTLIST_H */

+

</span></code></pre>

<br>

</li>

<li id='diff-42'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-42'>

<strong>

darshan-test/analysis.sql

</strong>

→

<strong>

darshan-test/2.x/analysis.sql

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/analysis.sql

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/analysis.sql

</span></code></pre>

<br>

</li>

<li id='diff-43'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-43'>

<strong>

darshan-test/cp-bench.c

</strong>

→

<strong>

darshan-test/2.x/cp-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cp-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/cp-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-44'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-44'>

<strong>

darshan-test/cp-realpath-bench.c

</strong>

→

<strong>

darshan-test/2.x/cp-realpath-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cp-realpath-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/cp-realpath-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-45'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-45'>

<strong>

darshan-test/cp-search-bench.c

</strong>

→

<strong>

darshan-test/2.x/cp-search-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cp-search-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/cp-search-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-46'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-46'>

<strong>

darshan-test/cp-wtime-bench.c

</strong>

→

<strong>

darshan-test/2.x/cp-wtime-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cp-wtime-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/cp-wtime-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-47'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-47'>

<strong>

darshan-test/cp-wtimewrap-bench.c

</strong>

→

<strong>

darshan-test/2.x/cp-wtimewrap-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cp-wtimewrap-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/cp-wtimewrap-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-48'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-48'>

<strong>

darshan-test/darshan-anon-core-hours.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-anon-core-hours.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-anon-core-hours.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-anon-core-hours.pl

</span></code></pre>

<br>

</li>

<li id='diff-49'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-49'>

<strong>

darshan-test/darshan-file-counter-hack.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-file-counter-hack.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-file-counter-hack.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-file-counter-hack.pl

</span></code></pre>

<br>

</li>

<li id='diff-50'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-50'>

<strong>

darshan-test/darshan-gather-counts.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-gather-counts.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-gather-counts.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-gather-counts.pl

</span></code></pre>

<br>

</li>

<li id='diff-51'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-51'>

<strong>

darshan-test/darshan-gather-intervals.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-gather-intervals.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-gather-intervals.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-gather-intervals.pl

</span></code></pre>

<br>

</li>

<li id='diff-52'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-52'>

<strong>

darshan-test/darshan-gather-mpi-posix-usage.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-gather-mpi-posix-usage.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-gather-mpi-posix-usage.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-gather-mpi-posix-usage.pl

</span></code></pre>

<br>

</li>

<li id='diff-53'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-53'>

<strong>

darshan-test/darshan-gather-stats.pl

</strong>

→

<strong>

darshan-test/2.x/darshan-gather-stats.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-gather-stats.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan-gather-stats.pl

</span></code></pre>

<br>

</li>

<li id='diff-54'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-54'>

<strong>

darshan-test/darshan.mysql

</strong>

→

<strong>

darshan-test/2.x/darshan.mysql

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan.mysql

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/darshan.mysql

</span></code></pre>

<br>

</li>

<li id='diff-55'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-55'>

<strong>

darshan-test/example-output/README.txt

</strong>

→

<strong>

darshan-test/2.x/example-output/README.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/example-output/README.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/example-output/README.txt

</span></code></pre>

<br>

</li>

<li id='diff-56'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-56'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.0_id178062_12-31-64877_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-57'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-57'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.10_id178062_12-31-64874_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-58'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-58'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.11_id178062_12-31-64875_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-59'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-59'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.12_id178062_12-31-64874_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-60'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-60'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.13_id178062_12-31-64878_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-61'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-61'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.14_id178062_12-31-64878_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-62'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-62'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.15_id178062_12-31-64878_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-63'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-63'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.1_id178062_12-31-64874_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-64'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-64'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.2_id178062_12-31-64879_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-65'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-65'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.3_id178062_12-31-64875_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-66'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-66'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.4_id178062_12-31-64876_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-67'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-67'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.5_id178062_12-31-64872_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-68'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-68'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.6_id178062_12-31-64878_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-69'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-69'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.7_id178062_12-31-64875_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-70'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-70'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.8_id178062_12-31-64878_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-71'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-71'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-1.1.9_id178062_12-31-64880_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-72'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-72'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.0_id180936_12-31-64879_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-73'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-73'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.0.2_id205555_12-8-36850-17729250439914587931_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-74'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-74'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.1.1_id240158_6-23-63667-13564362473375795675_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-75'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-75'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.0_id100_2-29-58016-11764280592576253912_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-76'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-76'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.4_id100_12-13-62209-215568122286385789_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-77'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-77'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.5_id100_1-1-168-10859250841018226043_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-78'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-78'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc32-2.2.6_id100_1-1-169-3365979776611785968_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-79'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-79'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.4_id47560_12-13-67385-3671058397549123273_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-80'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-80'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.5_id60773_2-26-55789-7981255518662743653_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-81'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-81'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.6_id80230_4-23-63443-13434414552960059406_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-82'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-82'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.8_id121829_8-28-59756-8698475815754165825_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-83'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-83'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.2.9_id259616_5-6-66953-5018725671498589377_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-84'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-84'>

<strong>

darshan-test/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-ppc64-2.3.1_id426989_3-13-61875-4723177368895607730_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-85'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-85'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.0_id3251_7-16-33980_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-86'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-86'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.0.2_id18650_12-7-37315-16484262108403302634_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-87'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-87'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86-2.1.1_id19454_6-23-43783-16484262108403302634_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-88'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-88'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.4_id20631_12-13-43744-16977121959925885678_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-89'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-89'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86-64-2.2.6_id23755_4-23-38771-2124448682003854839_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-90'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-90'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.0_id16795_7-16-38059_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-91'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-91'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.0.2_id15172_12-8-37621-274509493643487663_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-92'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-92'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.1.1_id20947_6-23-63417-274509493643487663_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-93'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-93'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.0_id3850_2-29-36600-13333843235489639491_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-94'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-94'>

<strong>

darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz and b/darshan-test/2.x/example-output/carns_mpi-io-test-x86_64-2.2.5_id5824_2-27-41403-11500169623986228326_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-95'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-95'>

<strong>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz and b/darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.8_id12451_8-28-46056-5041743303181360857_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-96'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-96'>

<strong>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz and b/darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.2.9_id8751_5-6-43885-16506732777848558488_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-97'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-97'>

<strong>

darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz and b/darshan-test/2.x/example-output/pcarns_mpi-io-test-x86_64-2.3.1_id13154_3-13-47125-7023224190677163982_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-98'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-98'>

<strong>

darshan-test/fsstats-merge.pl

</strong>

→

<strong>

darshan-test/2.x/fsstats-merge.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/fsstats-merge.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/fsstats-merge.pl

</span></code></pre>

<br>

</li>

<li id='diff-99'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-99'>

<strong>

darshan-test/fsstats-runner.bash

</strong>

→

<strong>

darshan-test/2.x/fsstats-runner.bash

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/fsstats-runner.bash

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/fsstats-runner.bash

</span></code></pre>

<br>

</li>

<li id='diff-100'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-100'>

<strong>

darshan-test/gz-bench.c

</strong>

→

<strong>

darshan-test/2.x/gz-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/gz-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/gz-bench.c

</span></code></pre>

<br>

</li>

<li id='diff-101'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-101'>

<strong>

darshan-test/gztest.c

</strong>

→

<strong>

darshan-test/2.x/gztest.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/gztest.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/gztest.c

</span></code></pre>

<br>

</li>

<li id='diff-102'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-102'>

<strong>

darshan-test/output.txt

</strong>

→

<strong>

darshan-test/2.x/output.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/output.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/output.txt

</span></code></pre>

<br>

</li>

<li id='diff-103'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-103'>

<strong>

darshan-test/parallel-fsstats.pl

</strong>

→

<strong>

darshan-test/2.x/parallel-fsstats.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/parallel-fsstats.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/parallel-fsstats.pl

</span></code></pre>

<br>

</li>

<li id='diff-104'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-104'>

<strong>

darshan-test/patches/99_cu_futimens.patch

</strong>

→

<strong>

darshan-test/2.x/patches/99_cu_futimens.patch

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/patches/99_cu_futimens.patch

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/patches/99_cu_futimens.patch

</span></code></pre>

<br>

</li>

<li id='diff-105'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-105'>

<strong>

darshan-test/patches/README

</strong>

→

<strong>

darshan-test/2.x/patches/README

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/patches/README

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/patches/README

</span></code></pre>

<br>

</li>

<li id='diff-106'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-106'>

<strong>

darshan-test/patches/coreutils-cp-mpi.patch

</strong>

→

<strong>

darshan-test/2.x/patches/coreutils-cp-mpi.patch

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/patches/coreutils-cp-mpi.patch

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/patches/coreutils-cp-mpi.patch

</span></code></pre>

<br>

</li>

<li id='diff-107'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-107'>

<strong>

darshan-test/patches/darshan-coll-write-single.patch

</strong>

→

<strong>

darshan-test/2.x/patches/darshan-coll-write-single.patch

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/patches/darshan-coll-write-single.patch

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/patches/darshan-coll-write-single.patch

</span></code></pre>

<br>

</li>

<li id='diff-108'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-108'>

<strong>

darshan-test/patches/darshan-coll-write-strategy-test.patch

</strong>

→

<strong>

darshan-test/2.x/patches/darshan-coll-write-strategy-test.patch

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/patches/darshan-coll-write-strategy-test.patch

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/patches/darshan-coll-write-strategy-test.patch

</span></code></pre>

<br>

</li>

<li id='diff-109'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-109'>

<strong>

darshan-test/pcarns_cp_id25476_1-16-15155_2.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/pcarns_cp_id25476_1-16-15155_2.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/pcarns_cp_id25476_1-16-15155_2.darshan.gz and b/darshan-test/2.x/pcarns_cp_id25476_1-16-15155_2.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-110'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-110'>

<strong>

darshan-test/perf/350665.output

</strong>

→

<strong>

darshan-test/2.x/perf/350665.output

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/350665.output

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/350665.output

</span></code></pre>

<br>

</li>

<li id='diff-111'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-111'>

<strong>

darshan-test/perf/350690.output

</strong>

→

<strong>

darshan-test/2.x/perf/350690.output

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/350690.output

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/350690.output

</span></code></pre>

<br>

</li>

<li id='diff-112'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-112'>

<strong>

darshan-test/perf/350892.output

</strong>

→

<strong>

darshan-test/2.x/perf/350892.output

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/350892.output

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/350892.output

</span></code></pre>

<br>

</li>

<li id='diff-113'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-113'>

<strong>

darshan-test/perf/354969.output

</strong>

→

<strong>

darshan-test/2.x/perf/354969.output

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/354969.output

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/354969.output

</span></code></pre>

<br>

</li>

<li id='diff-114'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-114'>

<strong>

darshan-test/perf/README

</strong>

→

<strong>

darshan-test/2.x/perf/README

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/README

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/README

</span></code></pre>

<br>

</li>

<li id='diff-115'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-115'>

<strong>

darshan-test/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz and b/darshan-test/2.x/perf/harms_IOR_id350665_2-8-78651_12.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-116'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-116'>

<strong>

darshan-test/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz and b/darshan-test/2.x/perf/harms_IOR_id350690_2-8-83458_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-117'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-117'>

<strong>

darshan-test/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz and b/darshan-test/2.x/perf/harms_IOR_id350892_2-9-74258_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-118'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-118'>

<strong>

darshan-test/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz

</strong>

→

<strong>

darshan-test/2.x/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files a/darshan-test/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz and b/darshan-test/2.x/perf/harms_partshared_id354969_2-25-56979_1.darshan.gz differ

</code></pre>

<br>

</li>

<li id='diff-119'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-119'>

<strong>

darshan-test/perf/partshared.c

</strong>

→

<strong>

darshan-test/2.x/perf/partshared.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/partshared.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/partshared.c

</span></code></pre>

<br>

</li>

<li id='diff-120'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-120'>

<strong>

darshan-test/perf/single_and_multi_file.ior

</strong>

→

<strong>

darshan-test/2.x/perf/single_and_multi_file.ior

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/perf/single_and_multi_file.ior

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/perf/single_and_multi_file.ior

</span></code></pre>

<br>

</li>

<li id='diff-121'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-121'>

<strong>

darshan-test/seek-test.c

</strong>

→

<strong>

darshan-test/2.x/seek-test.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/seek-test.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/seek-test.c

</span></code></pre>

<br>

</li>

<li id='diff-122'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-122'>

<strong>

darshan-test/stat-perf.c

</strong>

→

<strong>

darshan-test/2.x/stat-perf.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/stat-perf.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/stat-perf.c

</span></code></pre>

<br>

</li>

<li id='diff-123'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-123'>

<strong>

darshan-test/test-darshan-compiles-alcf2.sh

</strong>

→

<strong>

darshan-test/2.x/test-darshan-compiles-alcf2.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/test-darshan-compiles-alcf2.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/test-darshan-compiles-alcf2.sh

</span></code></pre>

<br>

</li>

<li id='diff-124'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-124'>

<strong>

darshan-test/test-darshan-compiles.sh

</strong>

→

<strong>

darshan-test/2.x/test-darshan-compiles.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/test-darshan-compiles.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/test-darshan-compiles.sh

</span></code></pre>

<br>

</li>

<li id='diff-125'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-125'>

<strong>

darshan-test/test-darshan-log-formats.sh

</strong>

→

<strong>

darshan-test/2.x/test-darshan-log-formats.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/test-darshan-log-formats.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/2.x/test-darshan-log-formats.sh

</span></code></pre>

<br>

</li>

<li id='diff-126'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-126'>

<strong>

darshan-test/ADDING-COUNTERS.txt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/ADDING-COUNTERS.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,17 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-Notes from Florin Isaila re: adding new counters as of 2014/03/06

----------------

-

-1. Added the counters in darshan-log-format.h

-2. Added the names of the counters in darshan-util/darshan-logutils.c

-3. Added the new profiled function declarations (e.g., PMPI_Sent) in

-darshan-runtime/darshan-dynamic.h

- - DARSHAN_EXTERN_DECL(...)

-4. Added the new profiled function declarations (e.g., PMPI_Sent) in

-darshan-runtime/lib/darshan-mpi-init-finalize.c 

- - DARSHAN_FORWARD_DECL(...)

- - MAP_OR_FAIL(...)

-5. Updated the reducer in darshan-runtime/lib/darshan-mpi-io.c in the

-function: 

- static void darshan_file_reduce(void* infile_v, 

-   void* inoutfile_v, int *len, 

-   MPI_Datatype *datatype);

</span></code></pre>

<br>

</li>

<li id='diff-127'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-127'>

<strong>

darshan-test/cuserid-test.c

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/cuserid-test.c

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,23 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#define _XOPEN_SOURCE

-

-#include <stdio.h>

-#include <unistd.h>

-

-#include <mpi.h>

-

-int main(int argc, char **argv)

-{

-    int rank;

-    char user_string[L_cuserid];

-

-    MPI_Init(&argc,&argv);

-    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

-

-    cuserid(user_string);

-

-    printf("rank %d: user: %s\n", rank, user_string);

-

-    MPI_Finalize();

-    return(0);

-}

-

</span></code></pre>

<br>

</li>

<li id='diff-128'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-128'>

<strong>

darshan-test/darshan-shutdown-bench.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/darshan-shutdown-bench.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/darshan-shutdown-bench.c

</span><span style="color: #aaaaaa">@@ -1,5 +1,5 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

<span style="color: #000000;background-color: #ddffdd">+ *  (C) 2015 by Argonne National Laboratory.

*      See COPYRIGHT in top-level directory.

  */

<span style="color: #aaaaaa">@@ -20,16 +20,11 @@

</span>  * benchmarking hooks for us.  This should only be used by special-purpose

  * benchmarking tools.

  */

<span style="color: #000000;background-color: #ffdddd">-void darshan_shutdown_bench(int argc, char** argv, int rank, int nprocs);

</span><span style="color: #000000;background-color: #ddffdd">+void darshan_shutdown_bench(int argc, char** argv);

</span> 

 int main(int argc, char **argv) 

 {

<span style="color: #000000;background-color: #ffdddd">-    int nprocs;

-    int mynod;

-

</span>     MPI_Init(&argc, &argv);

<span style="color: #000000;background-color: #ffdddd">-    MPI_Comm_rank(MPI_COMM_WORLD, &mynod);

-    MPI_Comm_size(MPI_COMM_WORLD, &nprocs);

</span> 

     if(argc != 1)

     {

<span style="color: #aaaaaa">@@ -38,7 +33,7 @@ int main(int argc, char **argv)

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    darshan_shutdown_bench(argc, argv, mynod, nprocs);

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_shutdown_bench(argc, argv);

</span> 

     MPI_Finalize();

     return(0);

</code></pre>

<br>

</li>

<li id='diff-129'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-129'>

<strong>

darshan-test/regression/README.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/README.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/README.txt

</span><span style="color: #aaaaaa">@@ -8,7 +8,12 @@ The master script must be executed with three arguments:

</span> 2) path to temporary directory (for building executables, collecting logs, 

    etc. during test)

 3) platform type; options include:

<span style="color: #000000;background-color: #ffdddd">-   - ws (for a standard workstation)

</span><span style="color: #000000;background-color: #ddffdd">+   - workstation-static (for static instrumentation on a standard workstation)

+   - workstation-dynamic (for dynamic instrumentation on a standard workstation)

+   - workstation-profile-conf (for static instrumentation using MPI profiling

+     configuration hooks on a standard workstation)

+   - bg-profile-conf (for static instrumentation using MPI profiling configuration

+     hooks on BGQ platform)

</span> 

 The platform type should map to a subdirectory containing scripts

 that describe how to perform platform-specific tasks (like loading or

</code></pre>

<br>

</li>

<li id='diff-130'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-130'>

<strong>

darshan-test/regression/test-cases/cxxpi.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/test-cases/cxxpi.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/test-cases/cxxpi.sh

</span><span style="color: #aaaaaa">@@ -29,7 +29,7 @@ fi

</span> 

 # check results

 # in this case we want to confirm that the open counts are zero; cxxpi does not do any IO

<span style="color: #000000;background-color: #ffdddd">-POSIX_OPENS=`grep CP_POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

</span><span style="color: #000000;background-color: #ddffdd">+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

</span> if [ "$POSIX_OPENS"x != ""x ]; then

     echo "Error: Found unexpected POSIX open count of $POSIX_OPENS" 1>&2

     exit 1

</code></pre>

<br>

</li>

<li id='diff-131'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-131'>

<strong>

darshan-test/regression/test-cases/fperf-f77.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/test-cases/fperf-f77.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/test-cases/fperf-f77.sh

</span><span style="color: #aaaaaa">@@ -29,14 +29,14 @@ fi

</span> 

 # check results

 # in this case we want to confirm that both the MPI and POSIX open counters were triggered

<span style="color: #000000;background-color: #ffdddd">-MPI_OPENS=`grep CP_COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $MPI_OPENS -gt 0 ]; then

-    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$POSIX_OPENS" -gt 0 ]; then

+    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

<span style="color: #000000;background-color: #ffdddd">-POSIX_OPENS=`grep CP_POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $POSIX_OPENS -gt 0 ]; then

-    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$MPI_OPENS" -gt 0 ]; then

+    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

</code></pre>

<br>

</li>

<li id='diff-132'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-132'>

<strong>

darshan-test/regression/test-cases/fperf-f90.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/test-cases/fperf-f90.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/test-cases/fperf-f90.sh

</span><span style="color: #aaaaaa">@@ -29,14 +29,14 @@ fi

</span> 

 # check results

 # in this case we want to confirm that both the MPI and POSIX open counters were triggered

<span style="color: #000000;background-color: #ffdddd">-MPI_OPENS=`grep CP_COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $MPI_OPENS -gt 0 ]; then

-    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$POSIX_OPENS" -gt 0 ]; then

+    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

<span style="color: #000000;background-color: #ffdddd">-POSIX_OPENS=`grep CP_POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $POSIX_OPENS -gt 0 ]; then

-    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$MPI_OPENS" -gt 0 ]; then

+    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

</code></pre>

<br>

</li>

<li id='diff-133'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-133'>

<strong>

darshan-test/regression/test-cases/mpi-io-test.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/test-cases/mpi-io-test.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/test-cases/mpi-io-test.sh

</span><span style="color: #aaaaaa">@@ -29,16 +29,15 @@ fi

</span> 

 # check results

 # in this case we want to confirm that both the MPI and POSIX open counters were triggered

<span style="color: #000000;background-color: #ffdddd">-MPI_OPENS=`grep CP_INDEP_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $MPI_OPENS -gt 0 ]; then

-    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$POSIX_OPENS" -gt 0 ]; then

+    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

<span style="color: #000000;background-color: #ffdddd">-POSIX_OPENS=`grep CP_POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`

-if [ ! $POSIX_OPENS -gt 0 ]; then

-    echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2

</span><span style="color: #000000;background-color: #ddffdd">+MPI_OPENS=`grep INDEP_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`

+if [ ! "$MPI_OPENS" -gt 0 ]; then

+    echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2

</span>     exit 1

 fi

<span style="color: #000000;background-color: #ffdddd">-

</span> exit 0

</code></pre>

<br>

</li>

<li id='diff-134'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-134'>

<strong>

darshan-test/regression/workstation-profile-conf/env.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/workstation-profile-conf/env.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/workstation-profile-conf/env.sh

</span><span style="color: #aaaaaa">@@ -34,5 +34,8 @@ export MPICC_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cc

</span> export MPICXX_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cxx

 export MPIF90_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f

 export MPIF77_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f

<span style="color: #000000;background-color: #ddffdd">+# MPICH 3.1.1 and newer use MPIFORT rather than MPIF90 and MPIF77 in env var

+# name

+export MPIFORT_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f

</span> 

 export DARSHAN_RUNJOB="mpiexec -n $DARSHAN_DEFAULT_NPROCS"

</code></pre>

<br>

</li>

<li id='diff-135'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-135'>

<strong>

darshan-util/Makefile.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/Makefile.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/Makefile.in

</span><span style="color: #aaaaaa">@@ -1,4 +1,4 @@

</span><span style="color: #000000;background-color: #ffdddd">-all: darshan-parser darshan-convert darshan-diff darshan-analyzer darshan-log-params darshan-util-lib

</span><span style="color: #000000;background-color: #ddffdd">+all: libdarshan-util.a darshan-null-logutils.o darshan-analyzer darshan-convert darshan-diff darshan-parser jenkins-hash-gen

</span> 

 DESTDIR =

 srcdir = @srcdir@

<span style="color: #aaaaaa">@@ -13,15 +13,20 @@ libdir = $(DESTDIR)@libdir@

</span> pkgconfigdir = $(DESTDIR)$(libdir)/pkgconfig

 DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h

<span style="color: #000000;background-color: #ddffdd">+DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h

+DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h

+DARSHAN_STATIC_MOD_OBJS = darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o

+DARSHAN_DYNAMIC_MOD_OBJS = darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po

+

</span> DARSHAN_ENABLE_SHARED=@DARSHAN_ENABLE_SHARED@

 VPATH = $(srcdir)

 ifeq ($(DARSHAN_ENABLE_SHARED),1)

<span style="color: #000000;background-color: #ffdddd">-all: libdarshan-util.so

</span><span style="color: #000000;background-color: #ddffdd">+all: libdarshan-util.so darshan-null-logutils.po

</span> endif

<span style="color: #000000;background-color: #ffdddd">-cp_zlib_link_flags = @__CP_ZLIB_LINK_FLAGS@

-cp_zlib_include_flags = @__CP_ZLIB_INCLUDE_FLAGS@

</span><span style="color: #000000;background-color: #ddffdd">+cp_zlib_link_flags = @__DARSHAN_ZLIB_LINK_FLAGS@

+cp_zlib_include_flags = @__DARSHAN_ZLIB_INCLUDE_FLAGS@

</span> 

 # deliberately avoid large file support for host side utilities to avoid

 # potentially buggy libz 64 bit offset support

<span style="color: #aaaaaa">@@ -41,60 +46,100 @@ mktestdir::

</span> uthash-1.9.2:

        tar xjvf $(srcdir)/extern/uthash-1.9.2.tar.bz2

<span style="color: #000000;background-color: #ffdddd">-darshan-parser: darshan-parser.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o | uthash-1.9.2

-       $(CC) $(CFLAGS) $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS) 

</span><span style="color: #000000;background-color: #ddffdd">+darshan-logutils.o: darshan-logutils.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) | uthash-1.9.2

+       $(CC) $(CFLAGS) -c  $< -o $@

+darshan-logutils.po: darshan-logutils.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

</span> 

<span style="color: #000000;background-color: #ffdddd">-darshan-convert: darshan-convert.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o lookup3.o

-       $(CC) $(CFLAGS)  $(LDFLAGS) $< darshan-logutils.o lookup3.o -o $@ $(LIBS)

</span><span style="color: #000000;background-color: #ddffdd">+darshan-null-logutils.o: darshan-null-logutils.c darshan-logutils.h darshan-null-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-null-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS) -c  $< -o $@

+darshan-null-logutils.po: darshan-null-logutils.c darshan-logutils.h darshan-null-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-null-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

</span> 

<span style="color: #000000;background-color: #ffdddd">-darshan-analyzer: darshan-analyzer.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o

-       $(CC) $(CFLAGS)  $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS)

</span><span style="color: #000000;background-color: #ddffdd">+darshan-posix-logutils.o: darshan-posix-logutils.c darshan-logutils.h darshan-posix-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-posix-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS) -c  $< -o $@

+darshan-posix-logutils.po: darshan-posix-logutils.c darshan-logutils.h darshan-posix-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-posix-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

</span> 

<span style="color: #000000;background-color: #ffdddd">-darshan-log-params: darshan-log-params.c $(DARSHAN_LOG_FORMAT)

-       $(CC) $(CFLAGS)  $(LDFLAGS) $< -o $@ $(LIBS)

</span><span style="color: #000000;background-color: #ddffdd">+darshan-mpiio-logutils.o: darshan-mpiio-logutils.c darshan-logutils.h darshan-mpiio-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS) -c  $< -o $@

+darshan-mpiio-logutils.po: darshan-mpiio-logutils.c darshan-logutils.h darshan-mpiio-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

</span> 

<span style="color: #000000;background-color: #ffdddd">-jenkins: util/bin/jenkins.o lookup3.o

-       $(CC) $(CFLAGS)  $(LDFLAGS) $< -o $@ lookup3.o $(LIBS)

</span><span style="color: #000000;background-color: #ddffdd">+darshan-hdf5-logutils.o: darshan-hdf5-logutils.c darshan-logutils.h darshan-hdf5-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS) -c  $< -o $@

+darshan-hdf5-logutils.po: darshan-hdf5-logutils.c darshan-logutils.h darshan-hdf5-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

</span> 

<span style="color: #000000;background-color: #ffdddd">-darshan-diff: darshan-diff.o $(DARSHAN_LOG_FORMAT) darshan-logutils.o darshan-logutils.h

-       $(CC) $(CFLAGS)  $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS)

-darshan-diff.o: darshan-diff.c

</span><span style="color: #000000;background-color: #ddffdd">+darshan-pnetcdf-logutils.o: darshan-pnetcdf-logutils.c darshan-logutils.h darshan-pnetcdf-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | uthash-1.9.2

</span>   $(CC) $(CFLAGS) -c  $< -o $@

<span style="color: #000000;background-color: #ffdddd">-darshan-logutils.o: darshan-logutils.c

</span><span style="color: #000000;background-color: #ddffdd">+darshan-pnetcdf-logutils.po: darshan-pnetcdf-logutils.c darshan-logutils.h darshan-pnetcdf-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | uthash-1.9.2

+       $(CC) $(CFLAGS_SHARED) -c  $< -o $@

+

+darshan-bgq-logutils.o: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2

</span>   $(CC) $(CFLAGS) -c  $< -o $@

<span style="color: #000000;background-color: #ffdddd">-darshan-logutils.po: darshan-logutils.c

</span><span style="color: #000000;background-color: #ddffdd">+darshan-bgq-logutils.po: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2

</span>   $(CC) $(CFLAGS_SHARED) -c  $< -o $@

<span style="color: #000000;background-color: #ffdddd">-libdarshan-util.so: darshan-logutils.po

</span><span style="color: #000000;background-color: #ddffdd">+libdarshan-util.a: darshan-logutils.o $(DARSHAN_STATIC_MOD_OBJS)

+       ar rcs libdarshan-util.a $^

+

+libdarshan-util.so: darshan-logutils.po $(DARSHAN_DYNAMIC_MOD_OBJS)

</span>   $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ $(LIBS)

<span style="color: #000000;background-color: #ffdddd">-darshan-util-lib: darshan-logutils.o

-       ar rcs libdarshan-util.a $<

-

-test/gztest: test/gztest.c mktestdir

-       $(CC) $(CFLAGS)  $(LDFLAGS) -lz $< -o $@

-

-test/gz-bench: test/gz-bench.c mktestdir

-       $(CC) $(CFLAGS)  $(LDFLAGS) -lz $< -o $@

</span><span style="color: #000000;background-color: #ddffdd">+jenkins-hash-gen: jenkins-hash-gen.c lookup3.o

+       $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)

</span> 

 lookup3.o: lookup3.c

        $(CC) $(CFLAGS) -c $< -o $@

<span style="color: #000000;background-color: #ddffdd">+darshan-analyzer: darshan-analyzer.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2

+       $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)

+

+darshan-convert: darshan-convert.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) libdarshan-util.a lookup3.o | uthash-1.9.2

+       $(CC) $(CFLAGS) $(LDFLAGS) $< lookup3.o libdarshan-util.a -o $@ $(LIBS)

+

+darshan-diff: darshan-diff.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2

+       $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS) 

+

+darshan-parser: darshan-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2

+       $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS) 

+

+#test/gztest: test/gztest.c mktestdir

+#      $(CC) $(CFLAGS)  $(LDFLAGS) -lz $< -o $@

+

+#test/gz-bench: test/gz-bench.c mktestdir

+#      $(CC) $(CFLAGS)  $(LDFLAGS) -lz $< -o $@

+

</span> install:: all

        install -d $(bindir)

        install -d $(libdir)

        install -d $(includedir)

        install -d $(pkgconfigdir)

<span style="color: #000000;background-color: #ffdddd">-        install -m 755 darshan-parser $(bindir)

</span><span style="color: #000000;background-color: #ddffdd">+   install -m 755 darshan-analyzer $(bindir)

</span>   install -m 755 darshan-convert $(bindir)

        install -m 755 darshan-diff $(bindir)

<span style="color: #000000;background-color: #ffdddd">-        install -m 755 darshan-analyzer $(bindir)

</span><span style="color: #000000;background-color: #ddffdd">+   install -m 755 darshan-parser $(bindir)

</span>   install -m 755 $(srcdir)/darshan-summary-per-file.sh $(bindir)

        install -m 755 libdarshan-util.a $(libdir)

 ifeq ($(DARSHAN_ENABLE_SHARED),1)

        install -m 755 libdarshan-util.so $(libdir)

 endif

        install -m 644 $(srcdir)/darshan-logutils.h $(includedir)

<span style="color: #000000;background-color: #ddffdd">+        install -m 644 $(srcdir)/darshan-posix-logutils.h $(includedir)

+       install -m 644 $(srcdir)/darshan-mpiio-logutils.h $(includedir)

+       install -m 644 $(srcdir)/darshan-hdf5-logutils.h $(includedir)

+       install -m 644 $(srcdir)/darshan-pnetcdf-logutils.h $(includedir)

+       install -m 644 $(srcdir)/darshan-bgq-logutils.h $(includedir)

+       install -m 644 $(srcdir)/darshan-null-logutils.h $(includedir)

+       install -m 644 $(srcdir)/../darshan-posix-log-format.h $(includedir)

+       install -m 644 $(srcdir)/../darshan-mpiio-log-format.h $(includedir)

+       install -m 644 $(srcdir)/../darshan-hdf5-log-format.h $(includedir)

+       install -m 644 $(srcdir)/../darshan-pnetcdf-log-format.h $(includedir)

+       install -m 644 $(srcdir)/../darshan-bgq-log-format.h $(includedir)

+       install -d $(includedir)/uthash-1.9.2

+       install -d $(includedir)/uthash-1.9.2/src

+       install -m 644 uthash-1.9.2/src/uthash.h $(includedir)/uthash-1.9.2/src/

</span>   install -m 644 $(DARSHAN_LOG_FORMAT) $(includedir)

        install -m 755 darshan-job-summary/bin/darshan-job-summary.pl $(bindir)

        install -d $(libdir)/TeX

<span style="color: #aaaaaa">@@ -108,7 +153,7 @@ endif

</span> 

 clean::

<span style="color: #000000;background-color: #ffdddd">-        rm -f *.o *.a darshan-parser darshan-convert darshan-diff darshan-analyzer darshan-log-params

</span><span style="color: #000000;background-color: #ddffdd">+   rm -f *.o *.po *.a darshan-analyzer darshan-convert darshan-parser jenkins-hash-gen

</span> 

 distclean:: clean

        rm -f darshan-runtime-config.h aclocal.m4 autom4te.cache/* config.status config.log Makefile util/bin/darshan-job-summary.pl

</code></pre>

<br>

</li>

<li id='diff-136'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-136'>

<strong>

darshan-util/configure

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/configure

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/configure

</span><span style="color: #aaaaaa">@@ -1,6 +1,6 @@

</span> #! /bin/sh

 # Guess values for system-dependent variables and create Makefiles.

<span style="color: #000000;background-color: #ffdddd">-# Generated by GNU Autoconf 2.69 for darshan-util 2.3.2-pre1.

<span style="color: #000000;background-color: #ddffdd">+# Generated by GNU Autoconf 2.69 for darshan-util 3.0.0-pre3.

</span> #

 #

 # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.

<span style="color: #aaaaaa">@@ -577,8 +577,8 @@ MAKEFLAGS=

# Identity of this package.

 PACKAGE_NAME='darshan-util'

 PACKAGE_TARNAME='darshan-util'

<span style="color: #000000;background-color: #ffdddd">-PACKAGE_VERSION='2.3.2-pre1'

-PACKAGE_STRING='darshan-util 2.3.2-pre1'

</span><span style="color: #000000;background-color: #ddffdd">+PACKAGE_VERSION='3.0.0-pre3'

+PACKAGE_STRING='darshan-util 3.0.0-pre3'

</span> PACKAGE_BUGREPORT=''

 PACKAGE_URL=''

<span style="color: #aaaaaa">@@ -623,9 +623,9 @@ ac_subst_vars='LTLIBOBJS

</span> LIBOBJS

 DARSHAN_UTIL_VERSION

 DARSHAN_ENABLE_SHARED

<span style="color: #000000;background-color: #ffdddd">-__CP_PDFLATEX_HALT_ON_ERROR

-__CP_ZLIB_INCLUDE_FLAGS

-__CP_ZLIB_LINK_FLAGS

</span><span style="color: #000000;background-color: #ddffdd">+__DARSHAN_PDFLATEX_HALT_ON_ERROR

+__DARSHAN_ZLIB_INCLUDE_FLAGS

+__DARSHAN_ZLIB_LINK_FLAGS

</span> HAVE_PDFLATEX

 PRI_MACROS_BROKEN

 LIBBZ2

<span style="color: #aaaaaa">@@ -1236,7 +1236,7 @@ if test "$ac_init_help" = "long"; then

# Omit some internal or obsolete options to make the list less imposing.

   # This message is too long to be a string in the A/UX 3.1 sh.

   cat <<_ACEOF

<span style="color: #000000;background-color: #ffdddd">-\`configure' configures darshan-util 2.3.2-pre1 to adapt to many kinds of systems.

<span style="color: #000000;background-color: #ddffdd">+\`configure' configures darshan-util 3.0.0-pre3 to adapt to many kinds of systems.

</span> 

 Usage: $0 [OPTION]... [VAR=VALUE]...

<span style="color: #aaaaaa">@@ -1297,7 +1297,7 @@ fi

</span> 

 if test -n "$ac_init_help"; then

   case $ac_init_help in

<span style="color: #000000;background-color: #ffdddd">-     short | recursive ) echo "Configuration of darshan-util 2.3.2-pre1:";;

</span><span style="color: #000000;background-color: #ddffdd">+     short | recursive ) echo "Configuration of darshan-util 3.0.0-pre3:";;

</span>    esac

   cat <<\_ACEOF

<span style="color: #aaaaaa">@@ -1393,7 +1393,7 @@ fi

</span> test -n "$ac_init_help" && exit $ac_status

 if $ac_init_version; then

   cat <<\_ACEOF

<span style="color: #000000;background-color: #ffdddd">-darshan-util configure 2.3.2-pre1

</span><span style="color: #000000;background-color: #ddffdd">+darshan-util configure 3.0.0-pre3

</span> generated by GNU Autoconf 2.69

 Copyright (C) 2012 Free Software Foundation, Inc.

<span style="color: #aaaaaa">@@ -1758,7 +1758,7 @@ cat >config.log <<_ACEOF

</span> This file contains any messages produced by compilers while

 running configure, to aid debugging if configure makes a mistake.

<span style="color: #000000;background-color: #ffdddd">-It was created by darshan-util $as_me 2.3.2-pre1, which was

</span><span style="color: #000000;background-color: #ddffdd">+It was created by darshan-util $as_me 3.0.0-pre3, which was

</span> generated by GNU Autoconf 2.69.  Invocation command line was

   $ $0 $@

<span style="color: #aaaaaa">@@ -3431,8 +3431,8 @@ if test "${with_zlib+set}" = set; then :

</span>     ZLIB_HOME="$withval"

     LDFLAGS="$LDFLAGS -L${ZLIB_HOME}/lib"

     CPPFLAGS="$CPPFLAGS -I${ZLIB_HOME}/include"

<span style="color: #000000;background-color: #ffdddd">-    __CP_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

-    __CP_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span><span style="color: #000000;background-color: #ddffdd">+    __DARSHAN_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

+    __DARSHAN_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span>   else

     { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Sorry, $withval does not exist, checking usual places" >&5

 $as_echo "$as_me: WARNING: Sorry, $withval does not exist, checking usual places" >&2;}

<span style="color: #aaaaaa">@@ -3805,9 +3805,9 @@ $as_echo_n "checking for -halt-on-error argument to pdflatex... " >&6; }

</span>     if test "x$PDFLATEX_GREP" != "x"; then

         { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5

 $as_echo "yes" >&6; }

<span style="color: #000000;background-color: #ffdddd">-        __CP_PDFLATEX_HALT_ON_ERROR="-halt-on-error"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_PDFLATEX_HALT_ON_ERROR="-halt-on-error"

</span>     else

<span style="color: #000000;background-color: #ffdddd">-        __CP_PDFLATEX_HALT_ON_ERROR=""

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_PDFLATEX_HALT_ON_ERROR=""

</span>         { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5

 $as_echo "no" >&6; }

     fi

<span style="color: #aaaaaa">@@ -4105,7 +4105,7 @@ fi

</span> done

<span style="color: #000000;background-color: #ffdddd">-DARSHAN_UTIL_VERSION="2.3.2-pre1"

</span><span style="color: #000000;background-color: #ddffdd">+DARSHAN_UTIL_VERSION="3.0.0-pre3"

</span> 

<span style="color: #aaaaaa">@@ -4621,7 +4621,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1

</span> # report actual input values of CONFIG_FILES etc. instead of their

 # values after options handling.

 ac_log="

<span style="color: #000000;background-color: #ffdddd">-This file was extended by darshan-util $as_me 2.3.2-pre1, which was

</span><span style="color: #000000;background-color: #ddffdd">+This file was extended by darshan-util $as_me 3.0.0-pre3, which was

</span> generated by GNU Autoconf 2.69.  Invocation command line was

   CONFIG_FILES    = $CONFIG_FILES

<span style="color: #aaaaaa">@@ -4683,7 +4683,7 @@ _ACEOF

</span> cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1

 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"

 ac_cs_version="\\

<span style="color: #000000;background-color: #ffdddd">-darshan-util config.status 2.3.2-pre1

</span><span style="color: #000000;background-color: #ddffdd">+darshan-util config.status 3.0.0-pre3

</span> configured by $0, generated by GNU Autoconf 2.69,

   with options \\"\$ac_cs_config\\"

</code></pre>

<br>

</li>

<li id='diff-137'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-137'>

<strong>

darshan-util/configure.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/configure.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/configure.in

<span style="color: #aaaaaa">@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.

</span> dnl You may need to use autoheader as well if changing any DEFINEs

 dnl sanity checks, output header, location of scripts used here

<span style="color: #000000;background-color: #ffdddd">-AC_INIT([darshan-util], [2.3.2-pre1])

</span><span style="color: #000000;background-color: #ddffdd">+AC_INIT([darshan-util], [3.0.0-pre3])

</span> AC_CONFIG_SRCDIR([darshan-logutils.h])

 AC_CONFIG_AUX_DIR(../maint/config)

 AC_CONFIG_HEADER(darshan-util-config.h)

<span style="color: #aaaaaa">@@ -30,9 +30,9 @@ if test x$HAVE_PDFLATEX == xyes; then

</span>     PDFLATEX_GREP=`pdflatex --help |grep halt-on-error`

     if test "x$PDFLATEX_GREP" != "x"; then

         AC_MSG_RESULT(yes)

<span style="color: #000000;background-color: #ffdddd">-        __CP_PDFLATEX_HALT_ON_ERROR="-halt-on-error"

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_PDFLATEX_HALT_ON_ERROR="-halt-on-error"

</span>     else

<span style="color: #000000;background-color: #ffdddd">-        __CP_PDFLATEX_HALT_ON_ERROR=""

</span><span style="color: #000000;background-color: #ddffdd">+        __DARSHAN_PDFLATEX_HALT_ON_ERROR=""

</span>         AC_MSG_RESULT(no)

     fi

 fi

<span style="color: #aaaaaa">@@ -82,9 +82,9 @@ AC_CHECK_FUNCS([strndup])

</span> 

 DARSHAN_UTIL_VERSION="AC_PACKAGE_VERSION"

<span style="color: #000000;background-color: #ffdddd">-AC_SUBST(__CP_ZLIB_LINK_FLAGS)

-AC_SUBST(__CP_ZLIB_INCLUDE_FLAGS)

-AC_SUBST(__CP_PDFLATEX_HALT_ON_ERROR)

</span><span style="color: #000000;background-color: #ddffdd">+AC_SUBST(__DARSHAN_ZLIB_LINK_FLAGS)

+AC_SUBST(__DARSHAN_ZLIB_INCLUDE_FLAGS)

+AC_SUBST(__DARSHAN_PDFLATEX_HALT_ON_ERROR)

</span> AC_SUBST(DARSHAN_ENABLE_SHARED)

 AC_SUBST(DARSHAN_UTIL_VERSION)

 AC_OUTPUT(Makefile

</code></pre>

<br>

</li>

<li id='diff-138'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-138'>

<strong>

darshan-util/darshan-analyzer.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-analyzer.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-analyzer.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,13 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

</span> #include <stdlib.h>

 #include <stdio.h>

 #include <string.h>

<span style="color: #000000;background-color: #ddffdd">+#include <assert.h>

</span> #include <ftw.h>

 #include <zlib.h>

<span style="color: #aaaaaa">@@ -11,10 +18,8 @@

</span> #define BUCKET3 0.60

 #define BUCKET4 0.80

<span style="color: #000000;background-color: #ffdddd">-char * base = NULL;

-

-int total_single = 0;

-int total_multi  = 0;

</span><span style="color: #000000;background-color: #ddffdd">+int total_shared = 0;

+int total_fpp    = 0;

</span> int total_mpio   = 0;

 int total_pnet   = 0;

 int total_hdf5   = 0;

<span style="color: #aaaaaa">@@ -25,66 +30,68 @@ int bucket2 = 0;

</span> int bucket3 = 0;

 int bucket4 = 0;

 int bucket5 = 0;

<span style="color: #000000;background-color: #ffdddd">-int fail    = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_pnet, int *used_hdf5, int *used_multi, int *used_single)

</span><span style="color: #000000;background-color: #ddffdd">+int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_pnet, int *used_hdf5, int *used_shared, int *used_fpp)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_job job;

-    struct darshan_file cp_file;

-    char tmp_string[4096];

-    darshan_fd zfile;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    darshan_fd file;

+    struct darshan_job job;

+    struct darshan_mod_logutil_funcs *psx_mod = mod_logutils[DARSHAN_POSIX_MOD];

+    struct darshan_posix_file psx_rec;

+    darshan_record_id rec_id;

</span>     int f_count;

     double total_io_time;

     double total_job_time;

<span style="color: #000000;background-color: #ffdddd">-    zfile = darshan_log_open(fname, "r");

-    if (zfile == NULL)

-    {

-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", fname);

-        return -1;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    assert(psx_mod);

+    memset(&psx_rec, 0, sizeof(struct darshan_posix_file));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_getjob(zfile, &job);

-    if (ret < 0)

</span><span style="color: #000000;background-color: #ddffdd">+    file = darshan_log_open(fname);

+    if (file == NULL)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        perror("darshan_log_getjob");

-        fprintf(stderr, "%s\n", fname);

-        darshan_log_close(zfile);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "darshan_log_open() failed to open %s.\n", fname);

</span>         return -1;

     }

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_getexe(zfile, tmp_string);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_getjob(file, &job);

</span>     if (ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        perror("darshan_log_getexe");

-        fprintf(stderr, "%s\n", fname);

-        darshan_log_close(zfile);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "darshan_log_getjob() failed on file %s.\n", fname);

+        darshan_log_close(file);

</span>         return -1;

     }

     f_count = 0;

     total_io_time = 0.0;

<span style="color: #000000;background-color: #ffdddd">-    while ((ret = darshan_log_getfile(zfile, &job, &cp_file)) == 1)

</span><span style="color: #000000;background-color: #ddffdd">+    while((ret = psx_mod->log_get_record(file, &psx_rec, &rec_id)) == 1)

</span>     {

         f_count   += 1;

<span style="color: #000000;background-color: #ffdddd">-        if (cp_file.rank == -1)

-            *used_single = 1;

</span><span style="color: #000000;background-color: #ddffdd">+        if (psx_rec.rank == -1)

+            *used_shared = 1;

</span>         else

<span style="color: #000000;background-color: #ffdddd">-            *used_multi = 1;

</span><span style="color: #000000;background-color: #ddffdd">+            *used_fpp = 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        *used_mpio += cp_file.counters[CP_INDEP_OPENS];

-        *used_mpio += cp_file.counters[CP_COLL_OPENS];

-        *used_pnet += cp_file.counters[CP_INDEP_NC_OPENS];

-        *used_pnet += cp_file.counters[CP_COLL_NC_OPENS];

-        *used_hdf5 += cp_file.counters[CP_HDF5_OPENS];

</span><span style="color: #000000;background-color: #ddffdd">+        total_io_time += (psx_rec.fcounters[POSIX_F_READ_TIME] +

+                         psx_rec.fcounters[POSIX_F_WRITE_TIME] +

+                         psx_rec.fcounters[POSIX_F_META_TIME]);

</span> 

<span style="color: #000000;background-color: #ffdddd">-        total_io_time += cp_file.fcounters[CP_F_POSIX_READ_TIME];

-        total_io_time += cp_file.fcounters[CP_F_POSIX_WRITE_TIME];

-        total_io_time += cp_file.fcounters[CP_F_POSIX_META_TIME];

</span><span style="color: #000000;background-color: #ddffdd">+        memset(&psx_rec, 0, sizeof(struct darshan_posix_file));

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    if (ret < 0)

+    {

+        fprintf(stderr, "Error: unable to read posix file record in log file %s.\n", fname);

+        darshan_log_close(file);

+        return -1;

+    }

+

+    if (file->mod_map[DARSHAN_MPIIO_MOD].len > 0)

+        *used_mpio += 1;

+    if (file->mod_map[DARSHAN_HDF5_MOD].len > 0)

+        *used_hdf5 += 1;

+    if (file->mod_map[DARSHAN_PNETCDF_MOD].len > 0)

+        *used_pnet += 1;

</span> 

     total_job_time = (double)job.end_time - (double)job.start_time;

     if (total_job_time < 1.0)

<span style="color: #aaaaaa">@@ -101,7 +108,7 @@ int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_p

</span>         *io_ratio = 0.0;

     }

<span style="color: #000000;background-color: #ffdddd">-    darshan_log_close(zfile);

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_log_close(file);

</span> 

     return 0;

 }

<span style="color: #aaaaaa">@@ -112,20 +119,20 @@ int tree_walk (const char *fpath, const struct stat *sb, int typeflag)

</span>     int used_mpio = 0;

     int used_pnet = 0;

     int used_hdf5 = 0;

<span style="color: #000000;background-color: #ffdddd">-    int used_multi = 0;

-    int used_single = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    int used_shared = 0;

+    int used_fpp = 0;

</span> 

     if (typeflag != FTW_F) return 0;

<span style="color: #000000;background-color: #ffdddd">-    process_log(fpath,&io_ratio,&used_mpio,&used_pnet,&used_hdf5,&used_multi,&used_single);

</span><span style="color: #000000;background-color: #ddffdd">+    process_log(fpath,&io_ratio,&used_mpio,&used_pnet,&used_hdf5,&used_shared,&used_fpp);

</span> 

     total_count++;

     if (used_mpio > 0) total_mpio++;

     if (used_pnet > 0) total_pnet++;

     if (used_hdf5 > 0) total_hdf5++;

<span style="color: #000000;background-color: #ffdddd">-    if (used_single > 0) total_single++;

-    if (used_multi  > 0) total_multi++;

</span><span style="color: #000000;background-color: #ddffdd">+    if (used_shared > 0) total_shared++;

+    if (used_fpp > 0) total_fpp++;

</span> 

     if (io_ratio <= BUCKET1)

         bucket1++;

<span style="color: #aaaaaa">@@ -137,22 +144,18 @@ int tree_walk (const char *fpath, const struct stat *sb, int typeflag)

</span>         bucket4++;

     else if (io_ratio > BUCKET4)

         bucket5++;

<span style="color: #000000;background-color: #ffdddd">-    else

-    {

-        printf("iorat: %lf\n", io_ratio);

-        fail++;

-    }

</span> 

     return 0;

 }

 int main(int argc, char **argv)

 {

<span style="color: #000000;background-color: #ddffdd">+    char * base = NULL;

</span>     int ret = 0;

     if(argc != 2)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: bad arguments.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: directory of Darshan logs required as argument.\n");

</span>         return(-1);

     }

<span style="color: #aaaaaa">@@ -165,13 +168,13 @@ int main(int argc, char **argv)

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    printf ("   log: %s\n", base);

-    printf (" total: %d\n", total_count);

-    printf ("single: %lf [%d]\n", (double)total_single/(double)total_count, total_single);

-    printf (" multi: %lf [%d]\n", (double)total_multi/(double)total_count, total_multi);

-    printf ("  mpio: %lf [%d]\n", (double)total_mpio/(double)total_count, total_mpio);

-    printf ("  pnet: %lf [%d]\n", (double)total_pnet/(double)total_count, total_pnet);

-    printf ("  hdf5: %lf [%d]\n", (double)total_hdf5/(double)total_count, total_hdf5);

</span><span style="color: #000000;background-color: #ddffdd">+    printf ("log dir: %s\n", base);

+    printf ("  total: %d\n", total_count);

+    printf (" shared: %lf [%d]\n", (double)total_shared/(double)total_count, total_shared);

+    printf ("    fpp: %lf [%d]\n", (double)total_fpp/(double)total_count, total_fpp);

+    printf ("   mpio: %lf [%d]\n", (double)total_mpio/(double)total_count, total_mpio);

+    printf ("   pnet: %lf [%d]\n", (double)total_pnet/(double)total_count, total_pnet);

+    printf ("   hdf5: %lf [%d]\n", (double)total_hdf5/(double)total_count, total_hdf5);

</span>     printf ("%.2lf-%.2lf: %d\n", (double)0.0,     (double)BUCKET1, bucket1);

     printf ("%.2lf-%.2lf: %d\n", (double)BUCKET1, (double)BUCKET2, bucket2);

     printf ("%.2lf-%.2lf: %d\n", (double)BUCKET2, (double)BUCKET3, bucket3);

<span style="color: #aaaaaa">@@ -179,3 +182,12 @@ int main(int argc, char **argv)

</span>     printf ("%.2lf-%.2lf: %d\n", (double)BUCKET4, (double)100.0,   bucket5);

     return 0;

 }

<span style="color: #000000;background-color: #ddffdd">+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-139'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-139'>

<strong>

darshan-util/darshan-bgq-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-bgq-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,217 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* counter name strings for the BGQ module */

+#define X(a) #a,

+char *bgq_counter_names[] = {

+    BGQ_COUNTERS

+};

+

+char *bgq_f_counter_names[] = {

+    BGQ_F_COUNTERS

+};

+#undef X

+

+static int darshan_log_get_bgq_rec(darshan_fd fd, void* bgq_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf, int ver);

+static void darshan_log_print_bgq_rec(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_bgq_description(void);

+static void darshan_log_print_bgq_rec_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+struct darshan_mod_logutil_funcs bgq_logutils =

+{

+    .log_get_record = &darshan_log_get_bgq_rec,

+    .log_put_record = &darshan_log_put_bgq_rec,

+    .log_print_record = &darshan_log_print_bgq_rec,

+    .log_print_description = &darshan_log_print_bgq_description,

+    .log_print_diff = &darshan_log_print_bgq_rec_diff

+};

+

+static int darshan_log_get_bgq_rec(darshan_fd fd, void* bgq_buf,

+    darshan_record_id* rec_id)

+{

+    struct darshan_bgq_record *rec;

+    int i;

+    int ret;

+

+    ret = darshan_log_getmod(fd, DARSHAN_BGQ_MOD, bgq_buf,

+        sizeof(struct darshan_bgq_record));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_bgq_record))

+        return(0);

+    else

+    {

+        rec = (struct darshan_bgq_record *)bgq_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&rec->f_id);

+            DARSHAN_BSWAP64(&rec->rank);

+            for(i=0; i<BGQ_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&rec->counters[i]);

+            for(i=0; i<BGQ_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&rec->fcounters[i]);

+        }

+

+        *rec_id = rec->f_id;

+        return(1);

+    }

+}

+

+static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf, int ver)

+{

+    struct darshan_bgq_record *rec = (struct darshan_bgq_record *)bgq_buf;

+    int ret;

+

+    ret = darshan_log_putmod(fd, DARSHAN_BGQ_MOD, rec,

+        sizeof(struct darshan_bgq_record), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+static void darshan_log_print_bgq_rec(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_bgq_record *bgq_file_rec =

+        (struct darshan_bgq_record *)file_rec;

+

+    for(i=0; i<BGQ_NUM_INDICES; i++)

+    {

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+            bgq_file_rec->rank, bgq_file_rec->f_id, bgq_counter_names[i],

+            bgq_file_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<BGQ_F_NUM_INDICES; i++)

+    {

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+            bgq_file_rec->rank, bgq_file_rec->f_id, bgq_f_counter_names[i],

+            bgq_file_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+static void darshan_log_print_bgq_description()

+{

+    printf("\n# description of BGQ counters:\n");

+    printf("#   BGQ_CSJOBID: BGQ control system job ID.\n");

+    printf("#   BGQ_NNODES: number of BGQ compute nodes for this job.\n");

+    printf("#   BGQ_RANKSPERNODE: number of MPI ranks per compute node.\n");

+    printf("#   BGQ_DDRPERNODE: size in MB of DDR3 per compute node.\n");

+    printf("#   BGQ_INODES: number of BGQ I/O nodes for this job.\n");

+    printf("#   BGQ_*NODES: dimension of A, B, C, D, & E dimensions of torus.\n");

+    printf("#   BGQ_TORUSENABLED: which dimensions of the torus are enabled.\n");

+    printf("#   BGQ_F_TIMESTAMP: timestamp when the BGQ data was collected.\n");

+

+    DARSHAN_PRINT_HEADER();

+

+    return;

+}

+

+static void darshan_log_print_bgq_rec_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_bgq_record *file1 = (struct darshan_bgq_record *)file_rec1;

+    struct darshan_bgq_record *file2 = (struct darshan_bgq_record *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<BGQ_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file1->rank, file1->f_id, bgq_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file2->rank, file2->f_id, bgq_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file1->rank, file1->f_id, bgq_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file2->rank, file2->f_id, bgq_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<BGQ_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file1->rank, file1->f_id, bgq_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file2->rank, file2->f_id, bgq_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file1->rank, file1->f_id, bgq_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],

+                file2->rank, file2->f_id, bgq_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-140'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-140'>

<strong>

darshan-util/darshan-bgq-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-bgq-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,15 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_BGQ_LOG_UTILS_H

+#define __DARSHAN_BGQ_LOG_UTILS_H

+

+extern char *bgq_counter_names[];

+extern char *bgq_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs bgq_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-141'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-141'>

<strong>

darshan-util/darshan-convert-logs.pl

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-convert-logs.pl

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-convert-logs.pl

</span><span style="color: #aaaaaa">@@ -1,8 +1,10 @@

</span> #!/usr/bin/perl -w

<span style="color: #000000;background-color: #ddffdd">+

</span> #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2011 by Argonne National Laboratory.

-#      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

<span style="color: #000000;background-color: #ddffdd">+

# Convert a list of logs and add metadata from another file.

 #

 # List of Logs: logfile path, one per line.  This can be generated with a 

<span style="color: #aaaaaa">@@ -18,7 +20,7 @@

</span> #

 my $darshan_convert = "./darshan-convert";

<span style="color: #000000;background-color: #ffdddd">-my $jenkins = "./jenkins";

</span><span style="color: #000000;background-color: #ddffdd">+my $jenkins_hash_gen = "./jenkins-hash-gen";

</span> 

 sub load_annotations($$)

 {

<span style="color: #aaaaaa">@@ -78,16 +80,17 @@ sub main()

</span>                 $day = $3;

                 $logname = $4;

             }

<span style="color: #000000;background-color: #ffdddd">-            $hashed_fname = `$jenkins --64 --key $hash_key $logname`;

</span><span style="color: #000000;background-color: #ddffdd">+            $hashed_fname = `$jenkins_hash_gen --64 --key $hash_key $logname`;

</span>             chomp($hashed_fname);

             @args = ("$darshan_convert",

<span style="color: #000000;background-color: #ddffdd">+                     "--bzip2",

</span>                      "--obfuscate",

                      "--key=$hash_key",

                      "--annotate=$annotation",

                      "--reset-md",

                      "$logfile",

<span style="color: #000000;background-color: #ffdddd">-                     "$output_path/$year/$month/$day/$hashed_fname.bz2");

</span><span style="color: #000000;background-color: #ddffdd">+                     "$output_path/$year/$month/$day/$hashed_fname.darshan");

</span>             $rc = system(@args);

             if ($rc) {

                 print("$hashed_fname\t$logfile:failed:$rc\n");

</code></pre>

<br>

</li>

<li id='diff-142'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-142'>

<strong>

darshan-util/darshan-convert.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-convert.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-convert.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2011 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #include <stdio.h>

<span style="color: #aaaaaa">@@ -18,6 +19,8 @@

</span> 

 #include "darshan-logutils.h"

<span style="color: #000000;background-color: #ddffdd">+#define DEF_MOD_BUF_SIZE 1024 /* 1 KiB is enough for all current mod records ... */

+

</span> extern uint32_t darshan_hashlittle(const void *key, size_t length, uint32_t initval);

 int usage (char *exename)

<span style="color: #aaaaaa">@@ -25,6 +28,7 @@ int usage (char *exename)

</span>     fprintf(stderr, "Usage: %s [options] <infile> <outfile>\n", exename);

     fprintf(stderr, "       Converts darshan log from infile to outfile.\n");

     fprintf(stderr, "       rewrites the log file into the newest format.\n");

<span style="color: #000000;background-color: #ddffdd">+    fprintf(stderr, "       --bzip2 Use bzip2 compression instead of zlib.\n");

</span>     fprintf(stderr, "       --obfuscate Obfuscate items in the log.\n");

     fprintf(stderr, "       --key <key> Key to use when obfuscating.\n");

     fprintf(stderr, "       --annotate <string> Additional metadata to add.\n");

<span style="color: #aaaaaa">@@ -35,15 +39,15 @@ int usage (char *exename)

</span> }

 void parse_args (int argc, char **argv, char **infile, char **outfile,

<span style="color: #000000;background-color: #ffdddd">-                 int *obfuscate, int *reset_md, int *key, char **annotate, uint64_t* hash)

</span><span style="color: #000000;background-color: #ddffdd">+                 int *bzip2, int *obfuscate, int *reset_md, int *key,

+                 char **annotate, uint64_t* hash)

</span> {

     int index;

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    *reset_md = 0;

-

</span>     static struct option long_opts[] =

     {

<span style="color: #000000;background-color: #ddffdd">+        {"bzip2", 0, NULL, 'b'},

</span>         {"annotate", 1, NULL, 'a'},

         {"obfuscate", 0, NULL, 'o'},

         {"reset-md", 0, NULL, 'r'},

<span style="color: #aaaaaa">@@ -53,6 +57,10 @@ void parse_args (int argc, char **argv, char **infile, char **outfile,

</span>         { 0, 0, 0, 0 }

     };

<span style="color: #000000;background-color: #ddffdd">+    *bzip2 = 0;

+    *obfuscate = 0;

+    *reset_md = 0;

+    *key = 0;

</span>     *hash = 0;

     while(1)

<span style="color: #aaaaaa">@@ -63,6 +71,9 @@ void parse_args (int argc, char **argv, char **infile, char **outfile,

</span> 

         switch(c)

         {

<span style="color: #000000;background-color: #ddffdd">+            case 'b':

+                *bzip2 = 1;

+                break;

</span>             case 'a':

                 *annotate = optarg;

                 break;

<span style="color: #aaaaaa">@@ -129,13 +140,22 @@ void obfuscate_exe(int key, char *exe)

</span>     return;

 }

<span style="color: #000000;background-color: #ffdddd">-void obfuscate_file(int key, struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+void obfuscate_filenames(int key, struct darshan_record_ref *rec_hash)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_record_ref *ref, *tmp;

</span>     uint32_t hashed;

<span style="color: #000000;background-color: #ddffdd">+    char tmp_string[128] = {0};

</span> 

<span style="color: #000000;background-color: #ffdddd">-    hashed = darshan_hashlittle(file->name_suffix, sizeof(file->name_suffix), key);

-    memset(file->name_suffix, 0, sizeof(file->name_suffix));

-    sprintf(file->name_suffix, "%u", hashed);

</span><span style="color: #000000;background-color: #ddffdd">+    HASH_ITER(hlink, rec_hash, ref, tmp)

+    {

+        hashed = darshan_hashlittle(ref->rec.name, strlen(ref->rec.name), key);

+        sprintf(tmp_string, "%u", hashed);

+        free(ref->rec.name);

+        ref->rec.name = malloc(strlen(tmp_string) + 1);

+        assert(ref->rec.name);

+        memcpy(ref->rec.name, tmp_string, strlen(tmp_string));

+        ref->rec.name[strlen(tmp_string)] = '\0';

+    }

</span> 

     return;

 }

<span style="color: #aaaaaa">@@ -175,7 +195,24 @@ void add_annotation (char *annotation,

</span>         {

             fprintf(stderr,

                     "not enough space left in metadata for: current=%s token=%s (remain=%d:need=%d)\n",

<span style="color: #000000;background-color: #ffdddd">-                    job->metadata, token, remaining-1, strlen(token)+1);

</span><span style="color: #000000;background-color: #ddffdd">+                    job->metadata, token, remaining-1, (int)strlen(token)+1);

+        }

+    }

+

+    return;

+}

+

+static void remove_hash_recs(struct darshan_record_ref **rec_hash, darshan_record_id hash)

+{

+    struct darshan_record_ref *ref, *tmp;

+

+    HASH_ITER(hlink, *rec_hash, ref, tmp)

+    {

+        if(ref->rec.id != hash)

+        {

+            HASH_DELETE(hlink, *rec_hash, ref);

+            free(ref->rec.name);

+            free(ref);

</span>         }

     }

<span style="color: #aaaaaa">@@ -188,52 +225,46 @@ int main(int argc, char **argv)

</span>     char *infile_name;

     char *outfile_name;

     struct darshan_job job;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file cp_file;

-    char tmp_string[4096];

</span><span style="color: #000000;background-color: #ddffdd">+    char tmp_string[4096] = {0};

</span>     darshan_fd infile;

     darshan_fd outfile;

     int i;

     int mount_count;

<span style="color: #000000;background-color: #ffdddd">-    int64_t* devs;

</span>     char** mnt_pts;

     char** fs_types;

<span style="color: #000000;background-color: #ffdddd">-    int last_rank = 0;

-    int obfuscate = 0;

-    int key = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_record_ref *rec_hash = NULL;

+    struct darshan_record_ref *ref, *tmp;

+    char mod_buf[DEF_MOD_BUF_SIZE];

+    enum darshan_comp_type comp_type;

+    int bzip2;

+    int obfuscate;

+    int key;

</span>     char *annotation = NULL;

<span style="color: #000000;background-color: #ffdddd">-    uint64_t hash;

-    int reset_md = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_record_id hash;

+    int reset_md;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    parse_args(argc, argv, &infile_name, &outfile_name, &obfuscate, &reset_md, &key, &annotation, &hash);

</span><span style="color: #000000;background-color: #ddffdd">+    parse_args(argc, argv, &infile_name, &outfile_name, &bzip2, &obfuscate,

+               &reset_md, &key, &annotation, &hash);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    infile = darshan_log_open(infile_name, "r");

</span><span style="color: #000000;background-color: #ddffdd">+    infile = darshan_log_open(infile_name);

</span>     if(!infile)

<span style="color: #000000;background-color: #ffdddd">-    {

-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", infile_name);

</span>         return(-1);

<span style="color: #000000;background-color: #ffdddd">-    }

</span>  

<span style="color: #000000;background-color: #ffdddd">-    /* TODO: safety check that outfile_name doesn't exist; we don't want to

-     * overwrite something by accident.

-     */

-    outfile = darshan_log_open(outfile_name, "w");

</span><span style="color: #000000;background-color: #ddffdd">+    comp_type = bzip2 ? comp_type = DARSHAN_BZIP2_COMP : DARSHAN_ZLIB_COMP;

+    outfile = darshan_log_create(outfile_name, comp_type, infile->partial_flag);

</span>     if(!outfile)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", outfile_name);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    /* TODO: for now this tool is just reading the input file and throwing

-     * away the data.  Need to write the log_put*() functions and use this

-     * program as a test harness

-     */

-  

</span>     /* read job info */

     ret = darshan_log_getjob(infile, &job);

     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to read job information from log file.\n");

</span>         darshan_log_close(infile);

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(outfile);

+        unlink(outfile_name);

</span>         return(-1);

     }

<span style="color: #aaaaaa">@@ -244,7 +275,7 @@ int main(int argc, char **argv)

</span>     ret = darshan_log_putjob(outfile, &job);

     if (ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to write job information to log file.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

</span>         darshan_log_close(outfile);

         return(-1);

     }

<span style="color: #aaaaaa">@@ -252,8 +283,9 @@ int main(int argc, char **argv)

</span>     ret = darshan_log_getexe(infile, tmp_string);

     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to read trailing job information.\n");

</span>         darshan_log_close(infile);

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(outfile);

+        unlink(outfile_name);

</span>         return(-1);

     }

<span style="color: #aaaaaa">@@ -262,69 +294,101 @@ int main(int argc, char **argv)

</span>     ret = darshan_log_putexe(outfile, tmp_string);

     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to write trailing job information.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

</span>         darshan_log_close(outfile);

         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-   

-    ret = darshan_log_getmounts(infile, &devs, &mnt_pts, &fs_types, &mount_count);

</span><span style="color: #000000;background-color: #ddffdd">+

+    ret = darshan_log_getmounts(infile, &mnt_pts, &fs_types, &mount_count);

</span>     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to read trailing job information.\n");

</span>         darshan_log_close(infile);

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(outfile);

+        unlink(outfile_name);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_putmounts(outfile, devs, mnt_pts, fs_types, mount_count);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_putmounts(outfile, mnt_pts, fs_types, mount_count);

</span>     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to write mount information.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

</span>         darshan_log_close(outfile);

         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_getfile(infile, &job, &cp_file);

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_gethash(infile, &rec_hash);

</span>     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: failed to process log file.\n");

-        fflush(stderr);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

+        darshan_log_close(outfile);

+        unlink(outfile_name);

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(ret == 0)

</span><span style="color: #000000;background-color: #ddffdd">+

+    /* NOTE: obfuscating filepaths breaks the ability to map files

+     * to the corresponding FS & mount info maintained by darshan

+     */

+    if(obfuscate) obfuscate_filenames(key, rec_hash);

+    if(hash) remove_hash_recs(&rec_hash, hash);

+

+    ret = darshan_log_puthash(outfile, rec_hash);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        goto done;

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(infile);

+        darshan_log_close(outfile);

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    do

</span><span style="color: #000000;background-color: #ddffdd">+    /* loop over each module and convert it's data to the new format */

+    for(i=0; i<DARSHAN_MAX_MODS; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        if(cp_file.rank != -1 && cp_file.rank < last_rank)

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_record_id rec_id;

+

+        /* check each module for any data */

+        if(infile->mod_map[i].len == 0)

+            continue;

+        else if(!mod_logutils[i])

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fprintf(stderr, "Error: log file contains out of order rank data.\n");

-            fflush(stderr);

-            return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Warning: no log utility handlers defined "

+                "for module %s, SKIPPING.\n", darshan_module_names[i]);

+            continue;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if(cp_file.rank != -1)

-            last_rank = cp_file.rank;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if(!hash || hash == cp_file.hash)

</span><span style="color: #000000;background-color: #ddffdd">+        /* we have module data to convert */

+        memset(mod_buf, 0, DEF_MOD_BUF_SIZE);

+

+        ret = mod_logutils[i]->log_get_record(infile, mod_buf, &rec_id);

+        if(ret != 1)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            if (obfuscate) obfuscate_file(key, &cp_file);

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Error: failed to parse the first %s module record.\n",

+                darshan_module_names[i]);

+            darshan_log_close(infile);

+            darshan_log_close(outfile);

+            unlink(outfile_name);

+            return(-1);

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-            ret = darshan_log_putfile(outfile, &job, &cp_file);

-            if (ret < 0)

</span><span style="color: #000000;background-color: #ddffdd">+        /* loop over each of the module's records and convert */

+        do

+        {

+            if(!hash || hash == rec_id)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                fprintf(stderr, "Error: failed to write file record.\n");

-                break;

</span><span style="color: #000000;background-color: #ddffdd">+                ret = mod_logutils[i]->log_put_record(outfile, mod_buf, infile->mod_ver[i]);

+                if(ret < 0)

+                {

+                    darshan_log_close(infile);

+                    darshan_log_close(outfile);

+                    return(-1);

+                }

+

+                memset(mod_buf, 0, DEF_MOD_BUF_SIZE);

</span>             }

<span style="color: #000000;background-color: #ffdddd">-        }

-    } while((ret = darshan_log_getfile(infile, &job, &cp_file)) == 1);

-

-    if(ret < 0)

-    {

-        fprintf(stderr, "Error: failed to process log file.\n");

-        fflush(stderr);

</span><span style="color: #000000;background-color: #ddffdd">+        } while((ret = mod_logutils[i]->log_get_record(infile, mod_buf, &rec_id)) == 1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-done:

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_log_close(infile);

+    darshan_log_close(outfile);

+

</span>     for(i=0; i<mount_count; i++)

     {

         free(mnt_pts[i]);

<span style="color: #aaaaaa">@@ -332,15 +396,25 @@ done:

</span>     }

     if(mount_count > 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        free(devs);

</span>         free(mnt_pts);

         free(fs_types);

     }

<span style="color: #000000;background-color: #ffdddd">- 

-    darshan_log_close(infile);

-    darshan_log_close(outfile);

</span><span style="color: #000000;background-color: #ddffdd">+

+    HASH_ITER(hlink, rec_hash, ref, tmp)

+    {

+        HASH_DELETE(hlink, rec_hash, ref);

+        free(ref->rec.name);

+        free(ref);

+    }

</span> 

     return(ret);

 }

<span style="color: #000000;background-color: #ffdddd">-

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-143'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-143'>

<strong>

darshan-util/darshan-diff.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-diff.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-diff.c

</span><span style="color: #aaaaaa">@@ -1,149 +1,477 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #include <stdio.h>

<span style="color: #000000;background-color: #ffdddd">-#include <zlib.h>

-#include <string.h>

-#include "darshan-log-format.h"

</span><span style="color: #000000;background-color: #ddffdd">+#include <stdlib.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <assert.h>

+

</span> #include "darshan-logutils.h"

<span style="color: #000000;background-color: #ddffdd">+#include "uthash-1.9.2/src/uthash.h"

+

+#define DEF_MOD_BUF_SIZE 1024 /* 1 KiB is enough for all current mod records ... */

+

+/* XXX: this structure is a temporary hack to get at the rank for each module's record */

+struct darshan_base_rec

+{

+    darshan_record_id f_id;

+    int64_t rank;

+};

+

+struct darshan_mod_record_ref

+{

+    int rank;

+    char mod_dat[DEF_MOD_BUF_SIZE];

+    struct darshan_mod_record_ref *prev;

+    struct darshan_mod_record_ref *next;

+};

</span> 

<span style="color: #000000;background-color: #ddffdd">+struct darshan_file_record_ref

+{

+    darshan_record_id rec_id;

+    struct darshan_mod_record_ref *mod_recs[DARSHAN_MAX_MODS];

+    UT_hash_handle hlink;

+};

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* utility functions just for darshan-diff */

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_build_global_record_hash(

+    darshan_fd fd, struct darshan_file_record_ref **rec_hash);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static void cd_print_str(char * prefix, char * arg1, char *arg2)

</span><span style="color: #000000;background-color: #ddffdd">+static void print_str_diff(char *prefix, char *arg1, char *arg2)

</span> {

     printf("- %s %s\n", prefix, arg1);

     printf("+ %s %s\n", prefix, arg2);

<span style="color: #000000;background-color: #ddffdd">+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-static void cd_print_int(char * prefix, int arg1, int arg2)

-{

-    printf("- %s %d\n", prefix, arg1);

-    printf("+ %s %d\n", prefix, arg2);

-}

-static void cd_print_int64(char * prefix, int64_t arg1, int64_t arg2)

</span><span style="color: #000000;background-color: #ddffdd">+

+static void print_int64_diff(char *prefix, int64_t arg1, int64_t arg2)

</span> {

     printf("- %s %" PRId64 "\n", prefix, arg1);

     printf("+ %s %" PRId64 "\n", prefix, arg2);

<span style="color: #000000;background-color: #ddffdd">+    return;

</span> }

<span style="color: #000000;background-color: #ffdddd">-

-int main(int argc, char ** argv)

</span><span style="color: #000000;background-color: #ddffdd">+int main(int argc, char *argv[])

</span> {

<span style="color: #000000;background-color: #ddffdd">+    char *logfile1, *logfile2;

</span>     darshan_fd file1, file2;

     struct darshan_job job1, job2;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file cp_file1, cp_file2;

</span>     char exe1[4096], exe2[4096];

<span style="color: #000000;background-color: #ffdddd">-    int i, ret1,ret2;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_record_ref *name_hash1 = NULL, *name_hash2 = NULL;

+    struct darshan_record_ref *name_ref1, *name_ref2;

+    struct darshan_file_record_ref *rec_hash1 = NULL, *rec_hash2 = NULL;

+    struct darshan_file_record_ref *rec_ref1, *rec_ref2, *rec_tmp;

+    struct darshan_mod_record_ref *mod_rec1, *mod_rec2;

+    void *mod_buf1, *mod_buf2;

+    struct darshan_base_rec *base_rec1, *base_rec2;

+    char *file_name1, *file_name2;

+    int i;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if (argc != 3)

</span><span style="color: #000000;background-color: #ddffdd">+    if(argc != 3)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Usage: %s <file1> <file2>\n", argv[0]);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Usage: darshan-diff <logfile1> <logfile2>\n");

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    file1 = darshan_log_open(argv[1], "r");

-    if(!file1) {

-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", argv[1]);

</span><span style="color: #000000;background-color: #ddffdd">+    logfile1 = argv[1];

+    logfile2 = argv[2];

+

+    file1 = darshan_log_open(logfile1);

+    if(!file1)

+    {

+        fprintf(stderr, "Error: unable to open darshan log file %s.\n", logfile1);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    file2 = darshan_log_open(argv[2], "r");

-    if(!file2) {

-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", argv[2]);

</span><span style="color: #000000;background-color: #ddffdd">+

+    file2 = darshan_log_open(logfile2);

+    if(!file2)

+    {

+        darshan_log_close(file1);

+        fprintf(stderr, "Error: unable to open darshan log file %s.\n", logfile2);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if (darshan_log_getjob(file1, &job1))

</span><span style="color: #000000;background-color: #ddffdd">+    /* get job data for each log file */

+    ret = darshan_log_getjob(file1, &job1);

+    if(ret < 0)

</span>     {

         darshan_log_close(file1);

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(file2);

+        fprintf(stderr, "Error: unable to read job info for darshan log file %s.\n", logfile1);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if (darshan_log_getjob(file2, &job2))

</span><span style="color: #000000;background-color: #ddffdd">+

+    ret = darshan_log_getjob(file2, &job2);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(file1);

</span>         darshan_log_close(file2);

<span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: unable to read job info for darshan log file %s.\n", logfile2);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if (darshan_log_getexe(file1, exe1))

</span><span style="color: #000000;background-color: #ddffdd">+    /* get exe string for each log file */

+    ret = darshan_log_getexe(file1, exe1);

+    if(ret < 0)

</span>     {

         darshan_log_close(file1);

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(file2);

+        fprintf(stderr, "Error: unable to read exe for darshan log file %s.\n", logfile1);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if (darshan_log_getexe(file2, exe2))

</span><span style="color: #000000;background-color: #ddffdd">+

+    ret = darshan_log_getexe(file2, exe2);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(file1);

</span>         darshan_log_close(file2);

<span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: unable to read exe for darshan log file %s.\n", logfile2);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if (strcmp(exe1, exe2)) 

-        cd_print_str("# exe: ", exe1, exe2);

</span><span style="color: #000000;background-color: #ddffdd">+    /* print diff of exe and job data */

+    if (strcmp(exe1, exe2))

+        print_str_diff("# exe: ", exe1, exe2);

</span> 

     if (job1.uid != job2.uid)

<span style="color: #000000;background-color: #ffdddd">-        cd_print_int("# uid:", job1.uid, job2.uid);

</span><span style="color: #000000;background-color: #ddffdd">+        print_int64_diff("# uid:", job1.uid, job2.uid);

</span>     if (job1.start_time != job2.start_time)

<span style="color: #000000;background-color: #ffdddd">-        cd_print_int64("# start_time:", 

-                (int64_t)job1.start_time, (int64_t)job2.start_time);

-    if (job1.end_time!= job2.end_time)

-        cd_print_int64("# end_time:", 

-                (int64_t)job1.end_time,(int64_t)job2.end_time);

-    if (job1.nprocs!= job2.nprocs)

-        cd_print_int("# nprocs:", job1.nprocs, job2.nprocs);

</span><span style="color: #000000;background-color: #ddffdd">+        print_int64_diff("# start_time:", job1.start_time, job2.start_time);

+    if (job1.end_time != job2.end_time)

+        print_int64_diff("# end_time:", job1.end_time, job2.end_time);

+    if (job1.nprocs != job2.nprocs)

+        print_int64_diff("# nprocs:", job1.nprocs, job2.nprocs);

</span>     if ((job1.end_time-job1.start_time) != (job2.end_time - job2.start_time))

<span style="color: #000000;background-color: #ffdddd">-        cd_print_int64("# run time:", 

-                (int64_t)(job1.end_time - job1.start_time +1),

</span><span style="color: #000000;background-color: #ddffdd">+        print_int64_diff("# run time:",

+                (int64_t)(job1.end_time - job1.start_time + 1),

</span>                 (int64_t)(job2.end_time - job2.start_time + 1));

<span style="color: #000000;background-color: #ffdddd">-    /* if for some reason no files were accessed, then we'll have to fix-up the

-     * buffers in the while loop */

-

-    do {

-        ret1 = darshan_log_getfile(file1, &job1, &cp_file1);

-       if (ret1 < 0) 

-       {

-               perror("darshan_log_getfile");

-               darshan_log_close(file1);

-               return(-1);

-       }

-        ret2 = darshan_log_getfile(file2, &job2, &cp_file2);

-       if (ret2 < 0) 

-       {

-               perror("darshan_log_getfile");

-               darshan_log_close(file2);

-               return(-1);

-       }

-

-        for(i=0; i<CP_NUM_INDICES; i++) {

-            if (cp_file1.counters[i] != cp_file2.counters[i]) {

-               printf("- ");

-               printf("%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t...%s\n",

-                       cp_file1.rank, cp_file1.hash, darshan_names[i], 

-                       cp_file1.counters[i], cp_file1.name_suffix);

-               printf("+ ");

-               printf("%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t...%s\n",

-                       cp_file2.rank, cp_file2.hash, darshan_names[i], 

-                       cp_file2.counters[i], cp_file2.name_suffix);

</span><span style="color: #000000;background-color: #ddffdd">+    /* get hash of record ids to file names for each log */

+    ret = darshan_log_gethash(file1, &name_hash1);

+    if(ret < 0)

+    {

+        darshan_log_close(file1);

+        darshan_log_close(file2);

+        fprintf(stderr, "Error: unable to read record hash for darshan log file %s.\n", logfile1);

+        return(-1);

+    }

+

+    ret = darshan_log_gethash(file2, &name_hash2);

+    if(ret < 0)

+    {

+        darshan_log_close(file1);

+        darshan_log_close(file2);

+        fprintf(stderr, "Error: unable to read record hash for darshan log file %s.\n", logfile2);

+        return(-1);

+    }

+

+    /* build hash tables of all records opened by all modules for each darshan log file */

+    ret = darshan_build_global_record_hash(file1, &rec_hash1);

+    if(ret < 0)

+    {

+        fprintf(stderr, "Error: unable to build record hash for darshan log file %s.\n", logfile1);

+        darshan_log_close(file1);

+        darshan_log_close(file2);

+        return(-1);

+    }

+

+    ret = darshan_build_global_record_hash(file2, &rec_hash2);

+    if(ret < 0)

+    {

+        fprintf(stderr, "Error: unable to build record hash for darshan log file %s.\n", logfile2);

+        darshan_log_close(file1);

+        darshan_log_close(file2);

+        return(-1);

+    }

+

+    /* iterate records for the first log file and correlate/diff with records from

+     * the second log file

+     */

+    HASH_ITER(hlink, rec_hash1, rec_ref1, rec_tmp)

+    {

+        printf("\n");

+

+        /* search hash2 for this record */

+        HASH_FIND(hlink, rec_hash2, &(rec_ref1->rec_id), sizeof(darshan_record_id), rec_ref2);

+

+        for(i = 0; i < DARSHAN_MAX_MODS; i++)

+        {

+            /* TODO: skip modules that don't have the same format version, for now */

+            if(rec_ref1->mod_recs[i] && rec_ref2 && rec_ref2->mod_recs[i] &&

+                (file1->mod_ver[i] != file2->mod_ver[i]))

+            {

+                fprintf(stderr, "Warning: skipping %s module data due to incompatible"

+                    "version numbers (file1=%d, file2=%d).\n",

+                    darshan_module_names[i], file1->mod_ver[i], file2->mod_ver[i]);

+                continue;

</span>             }

<span style="color: #000000;background-color: #ffdddd">-        }

-        for(i=0; i<CP_F_NUM_INDICES; i++) {

-            if (cp_file1.fcounters[i] != cp_file2.fcounters[i]) {

-               printf("- ");

-               printf("%" PRId64 "\t%" PRIu64 "\t%s\t%f\t...%s\n",

-                       cp_file1.rank, cp_file1.hash, darshan_f_names[i], 

-                       cp_file1.fcounters[i], cp_file1.name_suffix);

-               printf("+ ");

-               printf("%" PRId64 "\t%" PRIu64 "\t%s\t%f\t...%s\n",

-                       cp_file2.rank, cp_file2.hash, darshan_f_names[i], 

-                       cp_file2.fcounters[i], cp_file2.name_suffix);

</span><span style="color: #000000;background-color: #ddffdd">+

+            while(1)

+            {

+                mod_rec1 = rec_ref1->mod_recs[i];

+                if(rec_ref2)

+                    mod_rec2 = rec_ref2->mod_recs[i];

+                else

+                    mod_rec2 = NULL;

+

+                if(mod_rec1 == NULL)

+                    mod_buf1 = NULL;

+                else

+                    mod_buf1 = mod_rec1->mod_dat;

+                if(mod_rec2 == NULL)

+                    mod_buf2 = NULL;

+                else

+                    mod_buf2 = mod_rec2->mod_dat;

+

+                base_rec1 = (struct darshan_base_rec *)mod_buf1;

+                base_rec2 = (struct darshan_base_rec *)mod_buf2;

+

+                if(!base_rec1 && !base_rec2)

+                {

+                    /* break out if there are no more records for this module */

+                    break;

+                }

+                else if(base_rec1 && base_rec2)

+                {

+                    /* make sure if we have module records for this darshan record

+                     * from both log files, that we are performing the diff rank-by-rank

+                     * (i.e., we diff rank 1 records together, rank 2 records together,

+                     * and so on)

+                     */

+                    if(base_rec1->rank < base_rec2->rank)

+                        mod_buf2 = NULL;

+                    else if(base_rec1->rank > base_rec2->rank)

+                        mod_buf1 = NULL;

+                }

+

+                /* get corresponding file name for each record */

+                if(mod_buf1)

+                {

+                    HASH_FIND(hlink, name_hash1, &(base_rec1->f_id),

+                        sizeof(darshan_record_id), name_ref1);

+                    assert(name_ref1);

+                    file_name1 = name_ref1->rec.name;

+                }

+                if(mod_buf2)

+                {

+                    HASH_FIND(hlink, name_hash2, &(base_rec2->f_id),

+                        sizeof(darshan_record_id), name_ref2);

+                    assert(name_ref2);

+                    file_name2 = name_ref2->rec.name;

+                }

+

+                mod_logutils[i]->log_print_diff(mod_buf1, file_name1, mod_buf2, file_name2);

+

+                /* remove records which we have diffed already */

+                if(mod_buf1)

+                {

+                    if(mod_rec1->next == mod_rec1)

+                    {

+                        rec_ref1->mod_recs[i] = NULL;

+                    }

+                    else

+                    {

+                        mod_rec1->prev->next = mod_rec1->next;

+                        mod_rec1->next->prev = mod_rec1->prev;

+                        rec_ref1->mod_recs[i] = mod_rec1->next;

+                    }

+                    free(mod_rec1);

+                }

+                if(mod_buf2)

+                {

+                    if(mod_rec2->next == mod_rec2)

+                    {

+                        rec_ref2->mod_recs[i] = NULL;

+                    }

+                    else

+                    {

+                        mod_rec2->prev->next = mod_rec2->next;

+                        mod_rec2->next->prev = mod_rec2->prev;

+                        rec_ref2->mod_recs[i] = mod_rec2->next;

+                    }

+                    free(mod_rec2);

+                }

</span>             }

         }

<span style="color: #000000;background-color: #ddffdd">+        HASH_DELETE(hlink, rec_hash1, rec_ref1);

+        free(rec_ref1);

+        if(rec_ref2)

+        {

+            HASH_DELETE(hlink, rec_hash2, rec_ref2);

+            free(rec_ref2);

+        }

+    }

+

+    /* iterate any remaning records from the 2nd log file and print the diff output --

+     * NOTE: that these records do not have corresponding records in the first log file

+     */

+    HASH_ITER(hlink, rec_hash2, rec_ref2, rec_tmp)

+    {

+        printf("\n");

</span> 

<span style="color: #000000;background-color: #ffdddd">-    } while (ret1 == 1 || ret2 == 1);

</span><span style="color: #000000;background-color: #ddffdd">+        for(i = 0; i < DARSHAN_MAX_MODS; i++)

+        {

+            while(rec_ref2->mod_recs[i])

+            {

+                mod_rec2 = rec_ref2->mod_recs[i];

+                base_rec2 = (struct darshan_base_rec *)mod_rec2->mod_dat;

</span> 

<span style="color: #000000;background-color: #ddffdd">+                HASH_FIND(hlink, name_hash2, &(base_rec2->f_id),

+                    sizeof(darshan_record_id), name_ref2);

+                assert(name_ref2);

+                file_name2 = name_ref2->rec.name;

+

+                mod_logutils[i]->log_print_diff(NULL, NULL, mod_rec2->mod_dat, file_name2);

+                

+                /* remove the record we just diffed */

+                if(mod_rec2->next == mod_rec2)

+                {

+                    rec_ref2->mod_recs[i] = NULL;

+                }

+                else

+                {

+                    mod_rec2->prev->next = mod_rec2->next;

+                    mod_rec2->next->prev = mod_rec2->prev;

+                    rec_ref2->mod_recs[i] = mod_rec2->next;

+                }

+                free(mod_rec2);

+            }

+        }

+

+        HASH_DELETE(hlink, rec_hash2, rec_ref2);

+        free(rec_ref2);

+    }

+

+    HASH_ITER(hlink, name_hash1, name_ref1, name_ref2)

+    {

+        HASH_DELETE(hlink, name_hash1, name_ref1);

+        free(name_ref1->rec.name);

+        free(name_ref1);

+    }

+    HASH_ITER(hlink, name_hash2, name_ref2, name_ref1)

+    {

+        HASH_DELETE(hlink, name_hash2, name_ref2);

+        free(name_ref2->rec.name);

+        free(name_ref2);

+    }

</span> 

     darshan_log_close(file1);

     darshan_log_close(file2);

<span style="color: #000000;background-color: #ddffdd">+

+    return(0);

+}

+

+static int darshan_build_global_record_hash(

+    darshan_fd fd, struct darshan_file_record_ref **rec_hash)

+{

+    struct darshan_mod_record_ref *mod_rec;

+    struct darshan_file_record_ref *file_rec;

+    darshan_record_id tmp_rec_id;

+    struct darshan_base_rec *base_rec;

+    int i;

+    int ret;

+

+    /* iterate over all modules in each log file, adding records to the

+     * appropriate hash table

+     */

+    for(i = 0; i < DARSHAN_MAX_MODS; i++)

+    {

+        while(1)

+        {

+            if(!mod_logutils[i]) break;

+

+            mod_rec = malloc(sizeof(struct darshan_mod_record_ref));

+            assert(mod_rec);

+            memset(mod_rec, 0, sizeof(struct darshan_mod_record_ref));

+

+            ret = mod_logutils[i]->log_get_record(fd, mod_rec->mod_dat, &tmp_rec_id);

+            if(ret < 0)

+            {

+                fprintf(stderr, "Error: unable to read module %s data from log file.\n",

+                    darshan_module_names[i]);

+                free(mod_rec);

+                return(-1);

+            }

+            else if(ret == 0)

+            {

+                free(mod_rec);

+                break;

+            }

+            else

+            {

+                base_rec = (struct darshan_base_rec *)mod_rec->mod_dat;

+                mod_rec->rank = base_rec->rank;

+

+                HASH_FIND(hlink, *rec_hash, &tmp_rec_id, sizeof(darshan_record_id), file_rec);

+                if(!file_rec)

+                {

+                    /* there is no entry in the global hash table of darshan records

+                     * for this log file, so create one and add it.

+                     */

+                    file_rec = malloc(sizeof(struct darshan_file_record_ref));

+                    assert(file_rec);

+

+                    memset(file_rec, 0, sizeof(struct darshan_file_record_ref));

+                    file_rec->rec_id = tmp_rec_id;

+                    HASH_ADD(hlink, *rec_hash, rec_id, sizeof(darshan_record_id), file_rec);

+

+                }

+

+                /* add new record into the linked list of this module's records */

+                if(file_rec->mod_recs[i])

+                {

+                    /* there is already an initialized linked list for this module */

+

+                    /* we start at the end of the list and work backwards to insert this

+                     * record (the list is sorted according to increasing ranks, and in

+                     * general, darshan log records are sorted according to increasing

+                     * ranks, as well)

+                     */

+                    struct darshan_mod_record_ref *tmp_mod_rec = file_rec->mod_recs[i]->prev;

+                    while(1)

+                    {

+                        if(mod_rec->rank > tmp_mod_rec->rank)

+                        {

+                            /* insert new module record after this record */

+                            mod_rec->prev = tmp_mod_rec;

+                            mod_rec->next = tmp_mod_rec->next;

+                            tmp_mod_rec->next->prev = mod_rec;

+                            tmp_mod_rec->next = mod_rec;

+                            break;

+                        }

+                        else if(mod_rec->rank < tmp_mod_rec->rank)

+                        {

+                            /* insert new module record before this record */

+                            mod_rec->prev = tmp_mod_rec->prev;

+                            mod_rec->next = tmp_mod_rec;

+                            tmp_mod_rec->prev->next = mod_rec;

+                            tmp_mod_rec->prev = mod_rec;

+                            if(file_rec->mod_recs[i] == mod_rec->next)

+                                file_rec->mod_recs[i] = mod_rec;

+                            break;

+                        }

+

+                        tmp_mod_rec = tmp_mod_rec->prev;

+                        assert(tmp_mod_rec != file_rec->mod_recs[i]);

+                    }

+                }

+                else

+                {

+                    /* there are currently no records for this module, so just

+                     * initialize a new linked list

+                     */

+                    mod_rec->prev = mod_rec->next = mod_rec;

+                    file_rec->mod_recs[i] = mod_rec;

+                }

+            }

+        }

+    }

+

</span>     return(0);

 }

</code></pre>

<br>

</li>

<li id='diff-144'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-144'>

<strong>

darshan-util/darshan-hdf5-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-hdf5-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,212 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* counter name strings for the HDF5 module */

+#define X(a) #a,

+char *hdf5_counter_names[] = {

+    HDF5_COUNTERS

+};

+

+char *hdf5_f_counter_names[] = {

+    HDF5_F_COUNTERS

+};

+#undef X

+

+static int darshan_log_get_hdf5_file(darshan_fd fd, void* hdf5_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf, int ver);

+static void darshan_log_print_hdf5_file(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_hdf5_description(void);

+static void darshan_log_print_hdf5_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+struct darshan_mod_logutil_funcs hdf5_logutils =

+{

+    .log_get_record = &darshan_log_get_hdf5_file,

+    .log_put_record = &darshan_log_put_hdf5_file,

+    .log_print_record = &darshan_log_print_hdf5_file,

+    .log_print_description = &darshan_log_print_hdf5_description,

+    .log_print_diff = &darshan_log_print_hdf5_file_diff

+};

+

+static int darshan_log_get_hdf5_file(darshan_fd fd, void* hdf5_buf,

+    darshan_record_id* rec_id)

+{

+    struct darshan_hdf5_file *file;

+    int i;

+    int ret;

+

+    ret = darshan_log_getmod(fd, DARSHAN_HDF5_MOD, hdf5_buf,

+        sizeof(struct darshan_hdf5_file));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_hdf5_file))

+        return(0);

+    else

+    {

+        file = (struct darshan_hdf5_file *)hdf5_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&file->f_id);

+            DARSHAN_BSWAP64(&file->rank);

+            for(i=0; i<HDF5_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->counters[i]);

+            for(i=0; i<HDF5_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->fcounters[i]);

+        }

+

+        *rec_id = file->f_id;

+        return(1);

+    }

+}

+

+static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf, int ver)

+{

+    struct darshan_hdf5_file *file = (struct darshan_hdf5_file *)hdf5_buf;

+    int ret;

+

+    ret = darshan_log_putmod(fd, DARSHAN_HDF5_MOD, file,

+        sizeof(struct darshan_hdf5_file), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+static void darshan_log_print_hdf5_file(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_hdf5_file *hdf5_file_rec =

+        (struct darshan_hdf5_file *)file_rec;

+

+    for(i=0; i<HDF5_NUM_INDICES; i++)

+    {

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+            hdf5_file_rec->rank, hdf5_file_rec->f_id, hdf5_counter_names[i],

+            hdf5_file_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<HDF5_F_NUM_INDICES; i++)

+    {

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+            hdf5_file_rec->rank, hdf5_file_rec->f_id, hdf5_f_counter_names[i],

+            hdf5_file_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+static void darshan_log_print_hdf5_description()

+{

+    printf("\n# description of HDF5 counters:\n");

+    printf("#   HDF5_OPENS: HDF5 file open operation counts.\n");

+    printf("#   HDF5_F_OPEN_TIMESTAMP: timestamp of first HDF5 file open.\n");

+    printf("#   HDF5_F_CLOSE_TIMESTAMP: timestamp of last HDF5 file close.\n");

+

+    DARSHAN_PRINT_HEADER();

+

+    return;

+}

+

+static void darshan_log_print_hdf5_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_hdf5_file *file1 = (struct darshan_hdf5_file *)file_rec1;

+    struct darshan_hdf5_file *file2 = (struct darshan_hdf5_file *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<HDF5_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file1->rank, file1->f_id, hdf5_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file2->rank, file2->f_id, hdf5_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file1->rank, file1->f_id, hdf5_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file2->rank, file2->f_id, hdf5_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<HDF5_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file1->rank, file1->f_id, hdf5_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file2->rank, file2->f_id, hdf5_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file1->rank, file1->f_id, hdf5_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],

+                file2->rank, file2->f_id, hdf5_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-145'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-145'>

<strong>

darshan-util/darshan-hdf5-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-hdf5-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,15 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_HDF5_LOG_UTILS_H

+#define __DARSHAN_HDF5_LOG_UTILS_H

+

+extern char *hdf5_counter_names[];

+extern char *hdf5_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs hdf5_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-146'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-146'>

<strong>

darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in

</span><span style="color: #aaaaaa">@@ -1,6 +1,6 @@

</span> #!/usr/bin/perl -w

 #

<span style="color: #000000;background-color: #ffdddd">-#  (C) 2009 by Argonne National Laboratory.

<span style="color: #000000;background-color: #ddffdd">+#  (C) 2015 by Argonne National Laboratory.

#      See COPYRIGHT in top-level directory.

 #

<span style="color: #aaaaaa">@@ -26,27 +26,17 @@ my $pdflatex       = "pdflatex";

</span> my $epstopdf       = "epstopdf";

 my $cp             = "cp";

 my $mv             = "mv";

<span style="color: #000000;background-color: #ffdddd">-my $gnuplot;

-# Prefer gnuplot installed with darshan, if present.

-if(-x "$PREFIX/bin/gnuplot")

-{

-    $gnuplot       = "$PREFIX/bin/gnuplot";

-}

-else

-{

-    $gnuplot       = "gnuplot";

-}

</span><span style="color: #000000;background-color: #ddffdd">+my $gnuplot        = "gnuplot";

</span> 

 my $orig_dir = getcwd;

 my $output_file = "summary.pdf";

 my $verbose_flag = 0;

 my $input_file = "";

<span style="color: #000000;background-color: #ffdddd">-my %access_hash = ();

</span><span style="color: #000000;background-color: #ddffdd">+my %posix_access_hash = ();

+my %mpiio_access_hash = ();

</span> my @access_size = ();

 my %hash_files = ();

<span style="color: #000000;background-color: #ffdddd">-my $size_est_flag = 0;

-my $read_interval_overflow_flag = 0;

-my $write_interval_overflow_flag = 0;

</span><span style="color: #000000;background-color: #ddffdd">+my $partial_flag = 0;

</span> 

 # data structures for calculating performance

 my %hash_unique_file_time = ();

<span style="color: #aaaaaa">@@ -63,8 +53,7 @@ if ($verbose_flag)

</span>     print "verbose: $tmp_dir\n";

 }

<span style="color: #000000;background-color: #ffdddd">-

-open(TRACE, "$darshan_parser $input_file |") || die("Can't execute \"$darshan_parser $input_file\": $!\n");

</span><span style="color: #000000;background-color: #ddffdd">+open(PARSE_OUT, "$darshan_parser $input_file |") || die("Can't execute \"$darshan_parser $input_file\": $!\n");

</span> 

 open(FA_READ, ">$tmp_dir/file-access-read.dat") || die("error opening output file: $!\n");

 open(FA_WRITE, ">$tmp_dir/file-access-write.dat") || die("error opening output file: $!\n");

<span style="color: #aaaaaa">@@ -90,210 +79,242 @@ my $cumul_meta_shared = 0;

</span> my $cumul_meta_indep = 0;

 my $first_data_line = 1;

<span style="color: #000000;background-color: #ffdddd">-my $current_rank = 0;

-my $current_hash = 0;

</span> my %file_record_hash = ();

<span style="color: #000000;background-color: #ffdddd">-

</span> my %fs_data = ();

<span style="color: #000000;background-color: #ffdddd">-while ($line = <TRACE>) {

</span><span style="color: #000000;background-color: #ddffdd">+while($line = <PARSE_OUT>)

+{

</span>     chomp($line);

<span style="color: #000000;background-color: #ffdddd">-    

-    if ($line =~ /^\s*$/) {

</span><span style="color: #000000;background-color: #ddffdd">+

+    if ($line =~ /^\s*$/)

+    {

</span>         # ignore blank lines

     }

<span style="color: #000000;background-color: #ffdddd">-    elsif ($line =~ /^#/) {

-       if ($line =~ /^# exe: /) {

-           ($junk, $cmdline) = split(':', $line, 2);

</span><span style="color: #000000;background-color: #ddffdd">+    elsif ($line =~ /^#/)

+    {

+        if ($line =~ /^# exe: /)

+        {

+            $f_save = "";

+            ($junk, $cmdline) = split(':', $line, 2);

+

</span>             # add escape characters if needed for special characters in

             # command line

<span style="color: #000000;background-color: #ffdddd">-            $cmdline = encode('latex', $cmdline);

-       }

-       if ($line =~ /^# nprocs: /) {

-           ($junk, $nprocs) = split(':', $line, 2);

-           $procreads[$nprocs] = 0;

-       }

-       if ($line =~ /^# run time: /) {

-           ($junk, $runtime) = split(':', $line, 2);

-       }

-       if ($line =~ /^# start_time: /) {

-           ($junk, $starttime) = split(':', $line, 2);

-       }

-       if ($line =~ /^# uid: /) {

-           ($junk, $uid) = split(':', $line, 2);

-       }

-        if ($line =~ /^# jobid: /) {

-           ($junk, $jobid) = split(':', $line, 2);

</span><span style="color: #000000;background-color: #ddffdd">+            if ($cmdline =~ /<unknown args>/)

+            {

+                # fortran "<unknown args> seems to throw things off,

+                # so we don't encode that if it's present

+                $f_save = substr($cmdline, -14);

+                $cmdline = substr($cmdline, 0, -14); 

+            }

+            $cmdline = encode('latex', $cmdline) . $f_save;

+        }

+        elsif ($line =~ /^# nprocs: /)

+        {

+            ($junk, $nprocs) = split(':', $line, 2);

+        }

+        elsif ($line =~ /^# run time: /)

+        {

+            ($junk, $runtime) = split(':', $line, 2);

+        }

+        elsif ($line =~ /^# start_time: /)

+        {

+            ($junk, $starttime) = split(':', $line, 2);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($line =~ /^# darshan log version: /) {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($line =~ /^# uid: /)

+        {        

+            ($junk, $uid) = split(':', $line, 2);

+        }

+        elsif ($line =~ /^# jobid: /)

+        {

+            ($junk, $jobid) = split(':', $line, 2);

+        }

+        elsif ($line =~ /^# darshan log version: /)

+        {

</span>             ($junk, $version) = split(':', $line, 2);

             $version =~ s/^\s+//;

<span style="color: #000000;background-color: #ffdddd">-            ($major, $minor) = split(/\./, $version, 2);

</span><span style="color: #000000;background-color: #ddffdd">+        }

+        elsif ($line =~ /^# \*WARNING\*: .* contains incomplete data!/)

+        {

+            $partial_flag = 1;

</span>         }

     }

<span style="color: #000000;background-color: #ffdddd">-    else {

</span><span style="color: #000000;background-color: #ddffdd">+    else

+    {

</span>         # parse line

<span style="color: #000000;background-color: #ffdddd">-        @fields = split(/[\t ]+/, $line);

</span><span style="color: #000000;background-color: #ddffdd">+        @fields = split(/[\t ]+/, $line);

</span> 

         # encode the file system name to protect against special characters

<span style="color: #000000;background-color: #ffdddd">-        $fields[5] = encode('latex', $fields[5]);

-        

</span><span style="color: #000000;background-color: #ddffdd">+        $fields[6] = encode('latex', $fields[6]);

+

# is this our first piece of data?

         if($first_data_line)

         {

<span style="color: #000000;background-color: #ffdddd">-            $current_rank = $fields[0];

-            $current_hash = $fields[1];

</span>             $first_data_line = 0;

         }

         # is this a new file record?

<span style="color: #000000;background-color: #ffdddd">-        if($fields[0] != $current_rank || $fields[1] != $current_hash)

</span><span style="color: #000000;background-color: #ddffdd">+        if (!defined $file_record_hash{$fields[2]})

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            # process previous record

-            process_file_record($current_rank, $current_hash, \%file_record_hash);

-

-            # reset variables for next record 

-            $current_rank = $fields[0];

-            $current_hash = $fields[1];

-            %file_record_hash = ();

-            $file_record_hash{CP_NAME_SUFFIX} = $fields[4];

</span><span style="color: #000000;background-color: #ddffdd">+            $file_record_hash{$fields[2]} = ();

+            $file_record_hash{$fields[2]}{FILE_NAME} = $fields[5];

+            $file_record_hash{$fields[2]}{RANK} = $fields[1];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        $file_record_hash{$fields[2]} = $fields[3];

-

-       $summary{$fields[2]} += $fields[3];

-

-       # record per-process POSIX read count

-       if ($fields[2] eq "CP_POSIX_READS" || $fields[2] eq "CP_POSIX_FREADS") {

-           if ($fields[0] == -1) {

-               $procreads[$nprocs] += $fields[3];

-           }

-           else {

-               $procreads[$fields[0]] += $fields[3];

-           }

-       }

-

-       # record per-proces POSIX write count

-       if ($fields[2] eq "CP_POSIX_WRITES" || $fields[2] eq "CP_POSIX_FWRITES") {

-           if ($fields[0] == -1) {

-               $procwrites[$nprocs] += $fields[3];

-           }

-           else {

-               $procwrites[$fields[0]] += $fields[3];

-           }

-       }

-

-        # seperate accumulators for independent and shared reads and writes

-        if ($fields[2] eq "CP_F_POSIX_READ_TIME" && $fields[0] == -1){

-            $cumul_read_shared += $fields[3];

-        }

-        if ($fields[2] eq "CP_F_POSIX_READ_TIME" && $fields[0] != -1){

-            $cumul_read_indep += $fields[3];

-        }

-        if ($fields[2] eq "CP_F_POSIX_WRITE_TIME" && $fields[0] == -1){

-            $cumul_write_shared += $fields[3];

-        }

-        if ($fields[2] eq "CP_F_POSIX_WRITE_TIME" && $fields[0] != -1){

-            $cumul_write_indep += $fields[3];

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        $file_record_hash{$fields[2]}{$fields[3]} = $fields[4];

+        $summary{$fields[3]} += $fields[4];

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_F_POSIX_META_TIME" && $fields[0] == -1){

-            $cumul_meta_shared += $fields[3];

-        }

-        if ($fields[2] eq "CP_F_POSIX_META_TIME" && $fields[0] != -1){

-            $cumul_meta_indep += $fields[3];

-        }

-

-        if ((($fields[2] eq "CP_BYTES_READ") or

-             ($fields[2] eq "CP_BYTES_WRITTEN")) and

-            not defined($fs_data{$fields[5]}))

</span><span style="color: #000000;background-color: #ddffdd">+        # accumulate independent and shared data as well as fs data

+        if ($fields[3] eq "POSIX_F_READ_TIME")

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            $fs_data{$fields[5]} = [0,0];

</span><span style="color: #000000;background-color: #ddffdd">+            if ($fields[1] == -1)

+            {

+                $cumul_read_shared += $fields[4];

+            }

+            else

+            {

+                $cumul_read_indep += $fields[4];

+            }

</span>         }

<span style="color: #000000;background-color: #ffdddd">-

-        if ($fields[2] eq "CP_BYTES_READ" && $fields[0] == -1){

-            $cumul_read_bytes_shared += $fields[3];

-            $fs_data{$fields[5]}->[0] += $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_WRITE_TIME")

+        {

+            if ($fields[1] == -1)

+            {

+                $cumul_write_shared += $fields[4];

+            }

+            else

+            {

+                $cumul_write_indep += $fields[4];

+            }

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_BYTES_READ" && $fields[0] != -1){

-            $cumul_read_bytes_indep += $fields[3];

-            $fs_data{$fields[5]}->[0] += $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_META_TIME")

+        {

+            if ($fields[1] == -1)

+            {

+                $cumul_meta_shared += $fields[4];

+            }

+            else

+            {

+                $cumul_meta_indep += $fields[4];

+            }

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_BYTES_WRITTEN" && $fields[0] == -1){

-            $cumul_write_bytes_shared += $fields[3];

-            $fs_data{$fields[5]}->[1] += $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_BYTES_READ")

+        {

+            if ($fields[1] == -1)

+            {

+                $cumul_read_bytes_shared += $fields[4];

+            }

+            else

+            {

+                $cumul_read_bytes_indep += $fields[4];

+            }

+            if (not defined $fs_data{$fields[6]})

+            {

+                $fs_data{$fields[6]} = [0,0];

+            }

+            $fs_data{$fields[6]}->[0] += $fields[4];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_BYTES_WRITTEN" && $fields[0] != -1){

-            $cumul_write_bytes_indep += $fields[3];

-            $fs_data{$fields[5]}->[1] += $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_BYTES_WRITTEN")

+        {

+            if ($fields[1] == -1)

+            {

+                $cumul_write_bytes_shared += $fields[4];

+            }

+            else

+            {

+                $cumul_write_bytes_indep += $fields[4];

+            }

+            if (not defined $fs_data{$fields[6]})

+            {

+                $fs_data{$fields[6]} = [0,0];

+            }

+            $fs_data{$fields[6]}->[1] += $fields[4];

</span>         }

         # record start and end of reads and writes

<span style="color: #000000;background-color: #ffdddd">-

-        if ($fields[2] eq "CP_F_READ_START_TIMESTAMP") {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_READ_START_TIMESTAMP")

+        {

</span>             # store until we find the end

<span style="color: #000000;background-color: #ffdddd">-            # adjust for systems that give absolute time stamps

-            $last_read_start = $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+            $last_read_start = $fields[4];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_F_READ_END_TIMESTAMP" && $fields[3] != 0) {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_READ_END_TIMESTAMP" && $fields[4] != 0)

+        {

</span>             # assume we got the read start already 

<span style="color: #000000;background-color: #ffdddd">-            my $xdelta = $fields[3] - $last_read_start;

-            # adjust for systems that have absolute time stamps 

-            if($last_read_start > $starttime) {

-                $last_read_start -= $starttime;

-            }

-            if($fields[3] > $runtime && !$read_interval_overflow_flag)

</span><span style="color: #000000;background-color: #ddffdd">+            my $xdelta = $fields[4] - $last_read_start;

+            if($fields[1] == -1)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                $read_interval_overflow_flag = 1;

-                print "Warning: detected read access at time $fields[3] but runtime is only $runtime seconds.\n";

-            }

-            if($fields[0] == -1){

</span>                 print FA_READ_SH "$last_read_start\t0\t$xdelta\t0\n";

             }

<span style="color: #000000;background-color: #ffdddd">-            else{

-                print FA_READ "$last_read_start\t$fields[0]\t$xdelta\t0\n";

</span><span style="color: #000000;background-color: #ddffdd">+            else

+            {

+                print FA_READ "$last_read_start\t$fields[1]\t$xdelta\t0\n";

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_F_WRITE_START_TIMESTAMP") {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_WRITE_START_TIMESTAMP")

+        {

</span>             # store until we find the end

<span style="color: #000000;background-color: #ffdddd">-            $last_write_start = $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+            $last_write_start = $fields[4];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] eq "CP_F_WRITE_END_TIMESTAMP" && $fields[3] != 0) {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] eq "POSIX_F_WRITE_END_TIMESTAMP" && $fields[4] != 0)

+        {

</span>             # assume we got the write start already 

<span style="color: #000000;background-color: #ffdddd">-            my $xdelta = $fields[3] - $last_write_start;

-            # adjust for systems that have absolute time stamps 

-            if($last_write_start > $starttime) {

-                $last_write_start -= $starttime;

-            }

-            if($fields[3] > $runtime && !$write_interval_overflow_flag)

</span><span style="color: #000000;background-color: #ddffdd">+            my $xdelta = $fields[4] - $last_write_start;

+            if($fields[1] == -1)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                $write_interval_overflow_flag = 1;

-                print "Warning: detected write access at time $fields[3] but runtime is only $runtime seconds.\n";

-            }

-            if($fields[0] == -1){

</span>                 print FA_WRITE_SH "$last_write_start\t0\t$xdelta\t0\n";

             }

<span style="color: #000000;background-color: #ffdddd">-            else{

-                print FA_WRITE "$last_write_start\t$fields[0]\t$xdelta\t0\n";

</span><span style="color: #000000;background-color: #ddffdd">+            else

+            {

+                print FA_WRITE "$last_write_start\t$fields[1]\t$xdelta\t0\n";

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] =~ /^CP_ACCESS(.)_ACCESS/) {

-            $access_size[$1] = $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+        # record common access counter info

+        elsif ($fields[3] =~ /^POSIX_ACCESS(.)_ACCESS/)

+        {

+            $access_size[$1] = $fields[4];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ($fields[2] =~ /^CP_ACCESS(.)_COUNT/) {

</span><span style="color: #000000;background-color: #ddffdd">+        elsif ($fields[3] =~ /^MPIIO_ACCESS(.)_ACCESS/)

+        {

+            $access_size[$1] = $fields[4];

+        }

+        elsif ($fields[3] =~ /^POSIX_ACCESS(.)_COUNT/)

+        {

</span>             my $tmp_access_size = $access_size[$1];

<span style="color: #000000;background-color: #ffdddd">-            if(defined $access_hash{$tmp_access_size}){

-                $access_hash{$tmp_access_size} += $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+            if(defined $posix_access_hash{$tmp_access_size})

+            {

+                $posix_access_hash{$tmp_access_size} += $fields[4];

+            }

+            else

+            {

+                $posix_access_hash{$tmp_access_size} = $fields[4];

+            }

+        }

+        elsif ($fields[3] =~ /^MPIIO_ACCESS(.)_COUNT/)

+        {

+            my $tmp_access_size = $access_size[$1];

+            if(defined $mpiio_access_hash{$tmp_access_size})

+            {

+                $mpiio_access_hash{$tmp_access_size} += $fields[4];

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            else{

-                $access_hash{$tmp_access_size} = $fields[3];

</span><span style="color: #000000;background-color: #ddffdd">+            else

+            {

+                $mpiio_access_hash{$tmp_access_size} = $fields[4];

</span>             }

         }

     }

 }

<span style="color: #000000;background-color: #ddffdd">+close(PARSE_OUT) || die "darshan-parser failure: $! $?";

</span> 

<span style="color: #000000;background-color: #ffdddd">-close(TRACE) || die "darshan-parser failure: $! $?";

<span style="color: #000000;background-color: #ddffdd">+# Fudge one point at the end to make xrange match in read and write plots.

+# For some reason I can't get the xrange command to work.  -Phil

+print FA_READ "$runtime\t-1\t0\t0\n";

+print FA_WRITE "$runtime\t-1\t0\t0\n";

+print FA_READ_SH "$runtime\t0\t0\t0\n";

+print FA_WRITE_SH "$runtime\t0\t0\t0\n";

+close(FA_READ);

+close(FA_READ_SH);

+close(FA_WRITE);

+close(FA_WRITE_SH);

</span> 

 #

 # Exit out if there are no actual file accesses

<span style="color: #aaaaaa">@@ -303,237 +324,270 @@ if ($first_data_line)

</span>     $strtm = strftime("%a %b %e %H:%M:%S %Y", localtime($starttime));

     print "This darshan log has no file records. No summary was produced.\n";

<span style="color: #000000;background-color: #ffdddd">-    print "    jobid:$jobid\n";

-    print "      uid:$uid\n";

</span><span style="color: #000000;background-color: #ddffdd">+    print "    jobid: $jobid\n";

+    print "      uid: $uid\n";

</span>     print "starttime: $strtm ($starttime )\n";

<span style="color: #000000;background-color: #ffdddd">-    print "  runtime:$runtime (seconds)\n";

-    print "   nprocs:$nprocs\n";

</span><span style="color: #000000;background-color: #ddffdd">+    print "  runtime: $runtime (seconds)\n";

+    print "   nprocs: $nprocs\n";

</span>     print "  version: $version\n";

     exit(1);

 }

<span style="color: #000000;background-color: #ffdddd">-# process last file record

-$file_record_hash{CP_NAME_SUFFIX} = $fields[4];

-process_file_record($current_rank, $current_hash, \%file_record_hash);

-

-# Fudge one point at the end to make xrange match in read and write plots.

-# For some reason I can't get the xrange command to work.  -Phil

-print FA_READ "$runtime\t-1\t0\t0\n";

-print FA_WRITE "$runtime\t-1\t0\t0\n";

-print FA_READ_SH "$runtime\t0\t0\t0\n";

-print FA_WRITE_SH "$runtime\t0\t0\t0\n";

-close(FA_READ);

-close(FA_WRITE);

-close(FA_READ_SH);

-close(FA_WRITE_SH);

-

-# counts of operations

-open(COUNTS, ">$tmp_dir/counts.dat") || die("error opening output file: $!\n");

-print COUNTS "# P=POSIX, MI=MPI-IO indep., MC=MPI-IO coll., R=read, W=write\n";

-print COUNTS "# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat\n";

-my $total_posix_opens = $summary{CP_POSIX_OPENS} + $summary{CP_POSIX_FOPENS};

-my $total_syncs = $summary{CP_POSIX_FSYNCS} + $summary{CP_POSIX_FDSYNCS};

-print COUNTS "Read, ", $summary{CP_POSIX_READS} + $summary{CP_POSIX_FREADS}, ", ",

-    $summary{CP_INDEP_READS}, ", ", $summary{CP_COLL_READS}, "\n",

-    "Write, ", $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FWRITES}, ", ", 

-    $summary{CP_INDEP_WRITES}, ", ", $summary{CP_COLL_WRITES}, "\n",

-    "Open, ", $total_posix_opens, ", ", $summary{CP_INDEP_OPENS},", ",

-    $summary{CP_COLL_OPENS}, "\n",

-    "Stat, ", $summary{CP_POSIX_STATS}, ", 0, 0\n",

-    "Seek, ", $summary{CP_POSIX_SEEKS}, ", 0, 0\n",

-    "Mmap, ", $summary{CP_POSIX_MMAPS}, ", 0, 0\n",

-    "Fsync, ", $total_syncs, ", 0, 0\n";

-close COUNTS;

-

-# histograms of reads and writes

-open (HIST, ">$tmp_dir/hist.dat") || die("error opening output file: $!\n");

-print HIST "# size_range read write\n";

-print HIST "0-100, ", $summary{CP_SIZE_READ_0_100}, ", ",

-                 $summary{CP_SIZE_WRITE_0_100}, "\n";

-print HIST "101-1K, ", $summary{CP_SIZE_READ_100_1K}, ", ",

-                 $summary{CP_SIZE_WRITE_100_1K}, "\n";

-print HIST "1K-10K, ", $summary{CP_SIZE_READ_1K_10K}, ", ",

-                 $summary{CP_SIZE_WRITE_1K_10K}, "\n";

-print HIST "10K-100K, ", $summary{CP_SIZE_READ_10K_100K}, ", ",

-                 $summary{CP_SIZE_WRITE_10K_100K}, "\n";

-print HIST "100K-1M, ", $summary{CP_SIZE_READ_100K_1M}, ", ",

-                 $summary{CP_SIZE_WRITE_100K_1M}, "\n";

-print HIST "1M-4M, ", $summary{CP_SIZE_READ_1M_4M}, ", ",

-                 $summary{CP_SIZE_WRITE_1M_4M}, "\n";

-print HIST "4M-10M, ", $summary{CP_SIZE_READ_4M_10M}, ", ",

-                 $summary{CP_SIZE_WRITE_4M_10M}, "\n";

-print HIST "10M-100M, ", $summary{CP_SIZE_READ_10M_100M}, ", ",

-                 $summary{CP_SIZE_WRITE_10M_100M}, "\n";

-print HIST "100M-1G, ", $summary{CP_SIZE_READ_100M_1G}, ", ",

-                 $summary{CP_SIZE_WRITE_100M_1G}, "\n";

-print HIST "1G+, ", $summary{CP_SIZE_READ_1G_PLUS}, ", ",

-                 $summary{CP_SIZE_WRITE_1G_PLUS}, "\n";

-close HIST;

-

-# sequential and consecutive accesses

-open (PATTERN, ">$tmp_dir/pattern.dat") || die("error opening output file: $!\n");

-print PATTERN "# op total sequential consecutive\n";

-print PATTERN "Read, ", $summary{CP_POSIX_READS} + $summary{CP_POSIX_FREADS}, ", ",

-    $summary{CP_SEQ_READS}, ", ", $summary{CP_CONSEC_READS}, "\n";

-print PATTERN "Write, ", $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FWRITES}, ", ",

-    $summary{CP_SEQ_WRITES}, ", ", $summary{CP_CONSEC_WRITES}, "\n";

-close PATTERN;

-

-# aligned I/O

-open (ALIGN, ">$tmp_dir/align.dat") || die("error opening output file: $!\n");

-print ALIGN "# total unaligned_mem unaligned_file align_mem align_file\n";

-print ALIGN $summary{CP_POSIX_READS} + $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FREADS} + $summary{CP_POSIX_FWRITES}

-, ", ",

-    $summary{CP_MEM_NOT_ALIGNED}, ", ", $summary{CP_FILE_NOT_ALIGNED}, "\n";

-close ALIGN;

-

-# MPI types

-open (TYPES, ">$tmp_dir/types.dat") || die("error opening output file: $!\n");

-print TYPES "# type use_count\n";

-print TYPES "Named, ", $summary{CP_COMBINER_NAMED}, "\n";

-print TYPES "Dup, ", $summary{CP_COMBINER_DUP}, "\n";

-print TYPES "Contig, ", $summary{CP_COMBINER_CONTIGUOUS}, "\n";

-print TYPES "Vector, ", $summary{CP_COMBINER_VECTOR}, "\n";

-print TYPES "HvecInt, ", $summary{CP_COMBINER_HVECTOR_INTEGER}, "\n";

-print TYPES "Hvector, ", $summary{CP_COMBINER_HVECTOR}, "\n";

-print TYPES "Indexed, ", $summary{CP_COMBINER_INDEXED}, "\n";

-print TYPES "HindInt, ", $summary{CP_COMBINER_HINDEXED_INTEGER}, "\n";

-print TYPES "Hindexed, ", $summary{CP_COMBINER_HINDEXED}, "\n";

-print TYPES "IndBlk, ", $summary{CP_COMBINER_INDEXED_BLOCK}, "\n";

-print TYPES "StructInt, ", $summary{CP_COMBINER_STRUCT_INTEGER}, "\n";

-print TYPES "Struct, ", $summary{CP_COMBINER_STRUCT}, "\n";

-print TYPES "Subarray, ", $summary{CP_COMBINER_SUBARRAY}, "\n";

-print TYPES "Darray, ", $summary{CP_COMBINER_DARRAY}, "\n";

-print TYPES "F90Real, ", $summary{CP_COMBINER_F90_REAL}, "\n";

-print TYPES "F90Complex, ", $summary{CP_COMBINER_F90_COMPLEX}, "\n";

-print TYPES "F90Int, ", $summary{CP_COMBINER_F90_INTEGER}, "\n";

-print TYPES "Resized, ", $summary{CP_COMBINER_RESIZED}, "\n";

-close TYPES;

-

-# generate histogram of process I/O counts

-#

-# NOTE: NEED TO FILL IN ACTUAL WRITE DATA!!!

-#

-$minprocread = (defined $procreads[0]) ? $procreads[0] : 0;

-$maxprocread = (defined $procreads[0]) ? $procreads[0] : 0;

-for ($i=1; $i < $nprocs; $i++) {

-    $rdi = (defined $procreads[$i]) ? $procreads[$i] : 0;

-    $minprocread = ($rdi > $minprocread) ? $minprocread : $rdi;

-    $maxprocread = ($rdi < $maxprocread) ? $maxprocread : $rdi;

-}

-$minprocread += $procreads[$nprocs];

-$maxprocread += $procreads[$nprocs];

-# print "$minprocread $maxprocread\n";

-

-@bucket = ( 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 );

-

-for ($i=0; $i < $nprocs; $i++) {

-    $mysize = ((defined $procreads[$i]) ? $procreads[$i] : 0) +

-       $procreads[$nprocs];

-    $mysize -= $minprocread;

-    $mybucket = ($mysize > 0) ?

-       (($mysize * 10) / ($maxprocread - $minprocread)) : 0;

-    $bucket[$mybucket]++;

</span><span style="color: #000000;background-color: #ddffdd">+foreach $key (keys %file_record_hash)

+{

+    process_file_record($key, \%{$file_record_hash{$key}});

</span> }

<span style="color: #000000;background-color: #ffdddd">-open(IODIST, ">$tmp_dir/iodist.dat") || die("error opening output file: $!\n");

-print IODIST "# bucket n_procs_rd n_procs_wr\n";

-print IODIST "# NOTE: WRITES ARE A COPY OF READS FOR NOW!!!\n";

-

-$bucketsize = $maxprocread - $minprocread / 10;

-# TODO: do writes also, is dropping a 0 in for now

-for ($i=0; $i < 10; $i++) {

-    print IODIST $bucketsize * $i + $minprocread, "-",

-    $bucketsize * ($i+1) + $minprocread, ", ", $bucket[$i], ", 0\n";

-}

-close IODIST;

</span><span style="color: #000000;background-color: #ddffdd">+# copy template files to tmp tmp_dir

+system "$cp $PREFIX/share/*.gplt $tmp_dir/";

+system "$cp $PREFIX/share/*.tex $tmp_dir/";

</span> 

<span style="color: #000000;background-color: #ffdddd">-# generate title for summary

-($executable, $junk) = split(' ', $cmdline, 2);

-@parts = split('/', $executable);

-$cmd = $parts[$#parts];

</span> 

<span style="color: #000000;background-color: #ffdddd">-@timearray = localtime($starttime);

-$year = $timearray[5] + 1900;

-$mon = $timearray[4] + 1;

-$mday = $timearray[3];

</span><span style="color: #000000;background-color: #ddffdd">+# summary of time spent in POSIX & MPI-IO functions

+open(TIME_SUMMARY, ">$tmp_dir/time-summary.dat") || die("error opening output file:$!\n");

+print TIME_SUMMARY "# <type>, <app time>, <read>, <write>, <meta>\n";

+print TIME_SUMMARY "POSIX, ", ((($runtime * $nprocs - $summary{POSIX_F_READ_TIME} -

+    $summary{POSIX_F_WRITE_TIME} -

+    $summary{POSIX_F_META_TIME})/($runtime * $nprocs)) * 100);

+print TIME_SUMMARY ", ", (($summary{POSIX_F_READ_TIME}/($runtime * $nprocs))*100);

+print TIME_SUMMARY ", ", (($summary{POSIX_F_WRITE_TIME}/($runtime * $nprocs))*100);

+print TIME_SUMMARY ", ", (($summary{POSIX_F_META_TIME}/($runtime * $nprocs))*100), "\n";

+if (defined $summary{MPIIO_INDEP_OPENS})

+{

+    print TIME_SUMMARY "MPI-IO, ", ((($runtime * $nprocs - $summary{MPIIO_F_READ_TIME} -

+        $summary{MPIIO_F_WRITE_TIME} -

+        $summary{MPIIO_F_META_TIME})/($runtime * $nprocs)) * 100);

+    print TIME_SUMMARY ", ", (($summary{MPIIO_F_READ_TIME}/($runtime * $nprocs))*100);

+    print TIME_SUMMARY ", ", (($summary{MPIIO_F_WRITE_TIME}/($runtime * $nprocs))*100);

+    print TIME_SUMMARY ", ", (($summary{MPIIO_F_META_TIME}/($runtime * $nprocs))*100), "\n";

+}

+close TIME_SUMMARY;

</span> 

<span style="color: #000000;background-color: #ffdddd">-open(TITLE, ">$tmp_dir/title.tex") || die("error opening output file:$!\n");

-print TITLE "

-\\rhead{\\thepage\\ of \\pageref{LastPage}}

-\\chead[

-\\large $cmd ($mon/$mday/$year)

-]

</span><span style="color: #000000;background-color: #ddffdd">+# counts of operations

+open(PSX_OP_COUNTS, ">$tmp_dir/posix-op-counts.dat") || die("error opening output file: $!\n");

+print PSX_OP_COUNTS "# <operation>, <POSIX count>\n";

+print PSX_OP_COUNTS

+    "Read, ", $summary{POSIX_READS} + $summary{POSIX_FREADS}, "\n",

+    "Write, ", $summary{POSIX_WRITES} + $summary{POSIX_FWRITES}, "\n",

+    "Open, ", $summary{POSIX_OPENS} + $summary{POSIX_FOPENS}, "\n",

+    "Stat, ", $summary{POSIX_STATS}, "\n",

+    "Seek, ", $summary{POSIX_SEEKS}, "\n",

+    "Mmap, ", $summary{POSIX_MMAPS}, "\n",

+    "Fsync, ", $summary{POSIX_FSYNCS} + $summary{POSIX_FDSYNCS}, "\n";

+close PSX_OP_COUNTS;

+

+if (defined $summary{MPIIO_INDEP_OPENS})

</span> {

<span style="color: #000000;background-color: #ffdddd">-\\large $cmd ($mon/$mday/$year)

</span><span style="color: #000000;background-color: #ddffdd">+    # TODO: do we want to look at MPI split or non-blocking i/o here? 

+    open(MPI_OP_COUNTS, ">$tmp_dir/mpiio-op-counts.dat") || die("error opening output file: $!\n");

+    print MPI_OP_COUNTS "# <operation>, <MPI Ind. count>, <MPI Coll. count>\n";

+    print MPI_OP_COUNTS

+        "Read, ", $summary{MPIIO_INDEP_READS}, ", ", $summary{MPIIO_COLL_READS}, "\n",

+        "Write, ", $summary{MPIIO_INDEP_WRITES}, ", ", $summary{MPIIO_COLL_WRITES}, "\n",

+        "Open, ", $summary{MPIIO_INDEP_OPENS},", ", $summary{MPIIO_COLL_OPENS}, "\n",

+        "Stat, ", "0, 0\n",

+        "Seek, ", "0, 0\n",

+        "Mmap, ", "0, 0\n",

+        "Fsync, ", "0, ", $summary{MPIIO_SYNCS}, "\n";

+    close MPI_OP_COUNTS;

</span> }

<span style="color: #000000;background-color: #ffdddd">-\\cfoot[

-\\scriptsize{$cmdline}

-]

</span><span style="color: #000000;background-color: #ddffdd">+

+# histograms of reads and writes (for POSIX and MPI-IO modules)

+open (IO_HIST, ">$tmp_dir/posix-access-hist.dat") || die("error opening output file: $!\n");

+print IO_HIST "# <size_range>, <POSIX_reads>, <POSIX_writes>\n";

+print IO_HIST "0-100, ",

+              $summary{POSIX_SIZE_READ_0_100}, ", ",

+              $summary{POSIX_SIZE_WRITE_0_100}, "\n";

+print IO_HIST "101-1K, ",

+              $summary{POSIX_SIZE_READ_100_1K}, ", ",

+              $summary{POSIX_SIZE_WRITE_100_1K}, "\n";

+print IO_HIST "1K-10K, ",

+              $summary{POSIX_SIZE_READ_1K_10K}, ", ",

+              $summary{POSIX_SIZE_WRITE_1K_10K}, "\n";

+print IO_HIST "10K-100K, ",

+              $summary{POSIX_SIZE_READ_10K_100K}, ", ",

+              $summary{POSIX_SIZE_WRITE_10K_100K}, "\n";

+print IO_HIST "100K-1M, ",

+              $summary{POSIX_SIZE_READ_100K_1M}, ", ",

+              $summary{POSIX_SIZE_WRITE_100K_1M}, "\n";

+print IO_HIST "1M-4M, ",

+              $summary{POSIX_SIZE_READ_1M_4M}, ", ",

+              $summary{POSIX_SIZE_WRITE_1M_4M}, "\n";

+print IO_HIST "4M-10M, ",

+              $summary{POSIX_SIZE_READ_4M_10M}, ", ",

+              $summary{POSIX_SIZE_WRITE_4M_10M}, "\n";

+print IO_HIST "10M-100M, ",

+              $summary{POSIX_SIZE_READ_10M_100M}, ", ",

+              $summary{POSIX_SIZE_WRITE_10M_100M}, "\n";

+print IO_HIST "100M-1G, ",

+              $summary{POSIX_SIZE_READ_100M_1G}, ", ",

+              $summary{POSIX_SIZE_WRITE_100M_1G}, "\n";

+print IO_HIST "1G+, ",

+              $summary{POSIX_SIZE_READ_1G_PLUS}, ", ",

+              $summary{POSIX_SIZE_WRITE_1G_PLUS}, "\n";

+close IO_HIST;

+

+if (defined $summary{MPIIO_INDEP_OPENS})

</span> {

<span style="color: #000000;background-color: #ffdddd">-\\scriptsize{$cmdline}

</span><span style="color: #000000;background-color: #ddffdd">+    open (IO_HIST, ">$tmp_dir/mpiio-access-hist.dat") || die("error opening output file: $!\n");

+    print IO_HIST "# <size_range>, <MPIIO_reads>, <MPIIO_writes>\n";

+    print IO_HIST "0-100, ",

+                  $summary{MPIIO_SIZE_READ_AGG_0_100}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_0_100}, "\n";

+    print IO_HIST "101-1K, ",

+                  $summary{MPIIO_SIZE_READ_AGG_100_1K}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_100_1K}, "\n";

+    print IO_HIST "1K-10K, ",

+                  $summary{MPIIO_SIZE_READ_AGG_1K_10K}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_1K_10K}, "\n";

+    print IO_HIST "10K-100K, ",

+                  $summary{MPIIO_SIZE_READ_AGG_10K_100K}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_10K_100K}, "\n";

+    print IO_HIST "100K-1M, ",

+                  $summary{MPIIO_SIZE_READ_AGG_100K_1M}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_100K_1M}, "\n";

+    print IO_HIST "1M-4M, ",

+                  $summary{MPIIO_SIZE_READ_AGG_1M_4M}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_1M_4M}, "\n";

+    print IO_HIST "4M-10M, ",

+                  $summary{MPIIO_SIZE_READ_AGG_4M_10M}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_4M_10M}, "\n";

+    print IO_HIST "10M-100M, ",

+                  $summary{MPIIO_SIZE_READ_AGG_10M_100M}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_10M_100M}, "\n";

+    print IO_HIST "100M-1G, ",

+                  $summary{MPIIO_SIZE_READ_AGG_100M_1G}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_100M_1G}, "\n";

+    print IO_HIST "1G+, ",

+                  $summary{MPIIO_SIZE_READ_AGG_1G_PLUS}, ", ",

+                  $summary{MPIIO_SIZE_WRITE_AGG_1G_PLUS}, "\n";

+    close IO_HIST;

</span> }

<span style="color: #000000;background-color: #ffdddd">-";

-close TITLE;

</span> 

<span style="color: #000000;background-color: #ffdddd">-open(TABLES, ">$tmp_dir/job-table.tex") || die("error opening output file:$!\n");

-print TABLES "

-\\begin{tabular}{|p{.47\\columnwidth}|p{.35\\columnwidth}|p{.47\\columnwidth}|p{.6\\columnwidth}|}

-\\hline

-jobid: $jobid \& uid: $uid \& nprocs: $nprocs \& runtime: $runtime seconds\\\\

-\\hline

-\\end{tabular}

-";

-close TABLES;

</span><span style="color: #000000;background-color: #ddffdd">+# sequential and consecutive access patterns

+open (PATTERN, ">$tmp_dir/pattern.dat") || die("error opening output file: $!\n");

+print PATTERN "# op total sequential consecutive\n";

+print PATTERN "Read, ", $summary{POSIX_READS} + $summary{POSIX_FREADS}, ", ",

+    $summary{POSIX_SEQ_READS}, ", ", $summary{POSIX_CONSEC_READS}, "\n";

+print PATTERN "Write, ", $summary{POSIX_WRITES} + $summary{POSIX_FWRITES}, ", ",

+    $summary{POSIX_SEQ_WRITES}, ", ", $summary{POSIX_CONSEC_WRITES}, "\n";

+close PATTERN;

</span> 

<span style="color: #000000;background-color: #ffdddd">-open(TABLES, ">$tmp_dir/access-table.tex") || die("error opening output file:$!\n");

-print TABLES "

-\\begin{tabular}{r|r}

-\\multicolumn{2}{c}{ } \\\\

-\\multicolumn{2}{c}{Most Common Access Sizes} \\\\

</span><span style="color: #000000;background-color: #ddffdd">+# table of common access sizes

+open(ACCESS_TABLE, ">$tmp_dir/access-table.tex") || die("error opening output file:$!\n");

+print ACCESS_TABLE "

+\\begin{threeparttable}

+\\begin{tabular}{r|r|r}

+\\multicolumn{3}{c}{ } \\\\

+\\multicolumn{3}{c}{Most Common Access Sizes} \\\\

</span> \\hline

<span style="color: #000000;background-color: #ffdddd">-access size \& count \\\\

</span><span style="color: #000000;background-color: #ddffdd">+\& access size \& count \\\\

</span> \\hline

 \\hline

 ";

<span style="color: #000000;background-color: #ffdddd">-# sort access sizes (descending)

</span><span style="color: #000000;background-color: #ddffdd">+# sort POSIX & MPI-IO access sizes (descending)

</span> my $i = 0;

<span style="color: #000000;background-color: #ffdddd">-foreach $value (sort {$access_hash{$b} <=> $access_hash{$a} } keys %access_hash) {

-    if($i == 4) {

-        last;

</span><span style="color: #000000;background-color: #ddffdd">+my $tmp_access_count = 0;

+foreach $value (keys %posix_access_hash) {

+    if ($posix_access_hash{$value} > 0) {

+        $tmp_access_count++;

+        if ($tmp_access_count == 4) {

+            last;

+        }

+    }

+}

+if ($tmp_access_count > 0)

+{

+    foreach $value (sort {$posix_access_hash{$b} <=> $posix_access_hash{$a} } keys %posix_access_hash)

+    {

+        if ($i == 4) {

+            last;

+        }

+        if ($posix_access_hash{$value} == 0) {

+            last;

+        }

+

+        if ($i == 0) {

+            print ACCESS_TABLE "

+            \\multirow{$tmp_access_count}{*}{POSIX} \& $value \& $posix_access_hash{$value} \\\\\n

+            ";

+        }

+        else {

+            print ACCESS_TABLE "

+            \& $value \& $posix_access_hash{$value} \\\\\n

+            ";

+        }

+        $i++;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if($access_hash{$value} == 0) {

-        last;

</span><span style="color: #000000;background-color: #ddffdd">+}

+

+$i = 0;

+$tmp_access_count = 0;

+foreach $value (keys %mpiio_access_hash) {

+    if ($mpiio_access_hash{$value} > 0) {

+        $tmp_access_count++;

+        if ($tmp_access_count == 4) {

+            last;

+        }

+    }

+}

+if ($tmp_access_count > 0)

+{

+    foreach $value (sort {$mpiio_access_hash{$b} <=> $mpiio_access_hash{$a} } keys %mpiio_access_hash)

+    {

+        if ($i == 4) {

+            last;

+        }

+        if ($mpiio_access_hash{$value} == 0) {

+            last;

+        }

+

+        if ($i == 0) {

+            print ACCESS_TABLE "

+            \\hline

+            \\multirow{$tmp_access_count}{*}{MPI-IO \\textbf{\\ddag}} \& $value \& $mpiio_access_hash{$value} \\\\\n

+            ";

+        }

+        else {

+            print ACCESS_TABLE "

+            \& $value \& $mpiio_access_hash{$value} \\\\\n

+            ";

+        }

+        $i++;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    print TABLES "$value \& $access_hash{$value} \\\\\n";

-    $i++;

</span> }

<span style="color: #000000;background-color: #ffdddd">-print TABLES "

</span><span style="color: #000000;background-color: #ddffdd">+print ACCESS_TABLE "

</span> \\hline

 \\end{tabular}

 ";

<span style="color: #000000;background-color: #ffdddd">-close TABLES;

</span><span style="color: #000000;background-color: #ddffdd">+if ($tmp_access_count > 0)

+{

+    print ACCESS_TABLE "

+    \\begin{tablenotes}

+    \\item[\\normalsize \\textbf{\\ddag}] \\scriptsize NOTE: MPI-IO accesses are given in terms of aggregate datatype size.

+    \\end{tablenotes}

+    ";

+}

+print ACCESS_TABLE "

+\\end{threeparttable}

+";

+close ACCESS_TABLE;

</span> 

<span style="color: #000000;background-color: #ffdddd">-open(TABLES, ">$tmp_dir/file-count-table.tex") || die("error opening output file:$!\n");

-print TABLES "

</span><span style="color: #000000;background-color: #ddffdd">+# file count table

+open(FILE_CNT_TABLE, ">$tmp_dir/file-count-table.tex") || die("error opening output file:$!\n");

+print FILE_CNT_TABLE "

</span> \\begin{tabular}{r|r|r|r}

 \\multicolumn{4}{c}{ } \\\\

 \\multicolumn{4}{c}{File Count Summary} \\\\

<span style="color: #000000;background-color: #ffdddd">-";

-if($size_est_flag == 1)

-{

-print TABLES "

-\\multicolumn{4}{c}{(estimated by I/O access offsets)} \\\\

-";

-}

-print TABLES "

</span><span style="color: #000000;background-color: #ddffdd">+\\multicolumn{4}{c}{(estimated by POSIX I/O access offsets)} \\\\

</span> \\hline

 type \& number of files \& avg. size \& max size \\\\

 \\hline

 \\hline

 ";

<span style="color: #000000;background-color: #ddffdd">+

</span> my $counter;

 my $sum;

 my $max;

<span style="color: #aaaaaa">@@ -567,7 +621,7 @@ if($counter > 0) { $avg = $sum / $counter; }

</span> else { $avg = 0; }

 $avg = format_bytes($avg);

 $max = format_bytes($max);

<span style="color: #000000;background-color: #ffdddd">-print TABLES "total opened \& $counter \& $avg \& $max \\\\\n";

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "total opened \& $counter \& $avg \& $max \\\\\n";

</span> 

 $counter = 0;

 $sum = 0;

<span style="color: #aaaaaa">@@ -599,7 +653,7 @@ if($counter > 0) { $avg = $sum / $counter; }

</span> else { $avg = 0; }

 $avg = format_bytes($avg);

 $max = format_bytes($max);

<span style="color: #000000;background-color: #ffdddd">-print TABLES "read-only files \& $counter \& $avg \& $max \\\\\n";

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "read-only files \& $counter \& $avg \& $max \\\\\n";

</span> 

 $counter = 0;

 $sum = 0;

<span style="color: #aaaaaa">@@ -631,7 +685,7 @@ if($counter > 0) { $avg = $sum / $counter; }

</span> else { $avg = 0; }

 $avg = format_bytes($avg);

 $max = format_bytes($max);

<span style="color: #000000;background-color: #ffdddd">-print TABLES "write-only files \& $counter \& $avg \& $max \\\\\n";

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "write-only files \& $counter \& $avg \& $max \\\\\n";

</span> 

 $counter = 0;

 $sum = 0;

<span style="color: #aaaaaa">@@ -663,7 +717,7 @@ if($counter > 0) { $avg = $sum / $counter; }

</span> else { $avg = 0; }

 $avg = format_bytes($avg);

 $max = format_bytes($max);

<span style="color: #000000;background-color: #ffdddd">-print TABLES "read/write files \& $counter \& $avg \& $max \\\\\n";

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "read/write files \& $counter \& $avg \& $max \\\\\n";

</span> 

 $counter = 0;

 $sum = 0;

<span style="color: #aaaaaa">@@ -697,244 +751,69 @@ if($counter > 0) { $avg = $sum / $counter; }

</span> else { $avg = 0; }

 $avg = format_bytes($avg);

 $max = format_bytes($max);

<span style="color: #000000;background-color: #ffdddd">-print TABLES "created files \& $counter \& $avg \& $max \\\\\n";

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "created files \& $counter \& $avg \& $max \\\\\n";

</span> 

<span style="color: #000000;background-color: #ffdddd">-print TABLES "

</span><span style="color: #000000;background-color: #ddffdd">+print FILE_CNT_TABLE "

</span> \\hline

 \\end{tabular}

 ";

<span style="color: #000000;background-color: #ffdddd">-close(TABLES);

-

-

-#

-# Generate Per Filesystem Data

-#

-open(TABLES, ">$tmp_dir/fs-data-table.tex") || die("error opening output files:$!\n");

-if (($major > 1) or ($minor > 23))

-{

-    print TABLES "

-    \\begin{tabular}{c|r|r|r|r}

-    \\multicolumn{5}{c}{ } \\\\

-    \\multicolumn{5}{c}{Data Transfer Per Filesystem} \\\\

-    \\hline

-    \\multirow{2}{*}{File System} \& \\multicolumn{2}{c}{Write} \\vline \& \\multicolumn{2}{c}{Read} \\\\

-    \\cline{2-5}

-    \& MiB \& Ratio \& MiB \& Ratio \\\\\

-    \\hline

-    \\hline

-    ";

-    foreach $key (keys %fs_data)

-    {

-        my $wr_total_mb = ($fs_data{$key}->[1] / (1024*1024));

-        my $rd_total_mb = ($fs_data{$key}->[0] / (1024*1024));

-        my $wr_total_rt;

-

-        if ($cumul_write_bytes_shared+$cumul_write_bytes_indep)

-        {

-            $wr_total_rt = ($fs_data{$key}->[1] / ($cumul_write_bytes_shared + $cumul_write_bytes_indep));

-        }

-        else

-        {

-            $wr_total_rt = 0;

-        }

-

-        my $rd_total_rt;

-        if ($cumul_read_bytes_shared+$cumul_read_bytes_indep)

-        {

-            $rd_total_rt = ($fs_data{$key}->[0] / ($cumul_read_bytes_shared + $cumul_read_bytes_indep));

-        }

-        else

-        {

-            $rd_total_rt = 0;

-        }

-

-        printf TABLES "%s \& %.5f \& %.5f \& %.5f \& %.5f \\\\\n",

-            $key, $wr_total_mb, $wr_total_rt, $rd_total_mb, $rd_total_rt;

-}

-print TABLES "

</span><span style="color: #000000;background-color: #ddffdd">+close(FILE_CNT_TABLE);

+

+# generate per filesystem data

+open(FS_TABLE, ">$tmp_dir/fs-data-table.tex") || die("error opening output files:$!\n");

+print FS_TABLE "

+\\begin{tabular}{c|r|r|r|r}

+\\multicolumn{5}{c}{ } \\\\

+\\multicolumn{5}{c}{Data Transfer Per Filesystem (POSIX)} \\\\

+\\hline

+\\multirow{2}{*}{File System} \& \\multicolumn{2}{c}{Write} \\vline \& \\multicolumn{2}{c}{Read} \\\\

+\\cline{2-5}

+\& MiB \& Ratio \& MiB \& Ratio \\\\\

+\\hline

</span> \\hline

<span style="color: #000000;background-color: #ffdddd">-\\end{tabular}

-";

-}

-else

-{

-    print TABLES "

-\\rule{0in}{1in}

-\\parbox{5in}{Log versions prior to 1.24 do not support per-filesystem data.}

-";

-}

-close(TABLES);

-

-

-open(TIME, ">$tmp_dir/time-summary.dat") || die("error opening output file:$!\n");

-print TIME "# <type>, <app time>, <read>, <write>, <meta>\n";

-print TIME "POSIX, ", ((($runtime * $nprocs - $summary{CP_F_POSIX_READ_TIME} -

-    $summary{CP_F_POSIX_WRITE_TIME} -

-    $summary{CP_F_POSIX_META_TIME})/($runtime * $nprocs)) * 100);

-print TIME ", ", (($summary{CP_F_POSIX_READ_TIME}/($runtime * $nprocs))*100);

-print TIME ", ", (($summary{CP_F_POSIX_WRITE_TIME}/($runtime * $nprocs))*100);

-print TIME ", ", (($summary{CP_F_POSIX_META_TIME}/($runtime * $nprocs))*100), "\n";

-print TIME "MPI-IO, ", ((($runtime * $nprocs - $summary{CP_F_MPI_READ_TIME} -

-    $summary{CP_F_MPI_WRITE_TIME} -

-    $summary{CP_F_MPI_META_TIME})/($runtime * $nprocs)) * 100);

-print TIME ", ", (($summary{CP_F_MPI_READ_TIME}/($runtime * $nprocs))*100);

-print TIME ", ", (($summary{CP_F_MPI_WRITE_TIME}/($runtime * $nprocs))*100);

-print TIME ", ", (($summary{CP_F_MPI_META_TIME}/($runtime * $nprocs))*100), "\n";

-close TIME;

-

-# copy template files to tmp tmp_dir

-system "$cp $PREFIX/share/*.gplt $tmp_dir/";

-system "$cp $PREFIX/share/*.tex $tmp_dir/";

-

-# generate template for file access plot (we have to set range)

-my $ymax = $nprocs;

-my $yinc = int($nprocs / 8);

-if($yinc == 0) {$yinc=1;}

-my $ymaxtic = $nprocs-1;

-open(FILEACC, ">$tmp_dir/file-access-read-eps.gplt") || die("error opening output file:$!\n");

-print FILEACC "#!/usr/bin/gnuplot -persist

-

-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in

-set output \"file-access-read.eps\"

-set ylabel \"MPI rank\"

-set xlabel \"hours:minutes:seconds\"

-set xdata time

-set timefmt \"%s\"

-set format x \"%H:%M:%S\"

-set yrange [-1:$ymax]

-set title \"Timespan from first to last read access on independent files\"

-set xrange [\"0\":\"$runtime\"]

-set ytics 0,$yinc,$ymaxtic

-#set ytics -1,1

-set lmargin 4

-

-# color blindness work around

-set style line 2 lc 3

-set style line 3 lc 4

-set style line 4 lc 5

-set style line 5 lc 2

-set style increment user

-

-# lw 3 to make lines thicker...

-# note that writes are slightly offset for better visibility

-plot \"file-access-read.dat\" using 1:2:3:4 with vectors nohead filled notitle

-";

-close FILEACC;

-

-open(FILEACC, ">$tmp_dir/file-access-write-eps.gplt") || die("error opening output file:$!\n");

-print FILEACC "#!/usr/bin/gnuplot -persist

-

-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in

-set output \"file-access-write.eps\"

-set ylabel \"MPI rank\"

-set xlabel \"hours:minutes:seconds\"

-set xdata time

-set timefmt \"%s\"

-set format x \"%H:%M:%S\"

-set title \"Timespan from first to last write access on independent files\"

-set yrange [-1:$ymax]

-set xrange [\"0\":\"$runtime\"]

-#set ytics -1,1

-set ytics 0,$yinc,$ymaxtic

-set lmargin 4

-

-# color blindness work around

-set style line 2 lc 3

-set style line 3 lc 4

-set style line 4 lc 5

-set style line 5 lc 2

-set style increment user

-

-# lw 3 to make lines thicker...

-plot \"file-access-write.dat\" using 1:2:3:4 with vectors nohead filled lt 2 notitle

-";

-close FILEACC;

-

-open(FILEACC, ">$tmp_dir/file-access-shared-eps.gplt") || die("error opening output file:$!\n");

-print FILEACC "#!/usr/bin/gnuplot -persist

-

-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in

-set output \"file-access-shared.eps\"

-set xlabel \"hours:minutes:seconds\"

-set xdata time

-set timefmt \"%s\"

-set format x \"%H:%M:%S\"

-unset ytics

-set ylabel \"All processes\"

-set xrange [\"0\":\"$runtime\"]

-set yrange [-1:1]

-set title \"Timespan from first to last access on files shared by all processes\"

-set lmargin 4

-

-# color blindness work around

-set style line 2 lc 3

-set style line 3 lc 4

-set style line 4 lc 5

-set style line 5 lc 2

-set style increment user

-

-plot \"file-access-read-sh.dat\" using 1:2:3:4 with vectors nohead filled lw 10 title \"read\", \\

-\"file-access-write-sh.dat\" using 1:((\$2)-.2):3:4 with vectors nohead filled lw 10 title \"write\"

</span> ";

<span style="color: #000000;background-color: #ffdddd">-close FILEACC;

-

-$cumul_read_indep /= $nprocs;

-$cumul_read_bytes_indep /= $nprocs;

-$cumul_read_bytes_indep /= 1048576.0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-$cumul_write_indep /= $nprocs;

-$cumul_write_bytes_indep /= $nprocs;

-$cumul_write_bytes_indep /= 1048576.0;

-

-$cumul_read_shared /= $nprocs;

-$cumul_read_bytes_shared /= $nprocs;

-$cumul_read_bytes_shared /= 1048576.0;

</span><span style="color: #000000;background-color: #ddffdd">+foreach $key (keys %fs_data)

+{

+    my $wr_total_mb = ($fs_data{$key}->[1] / (1024*1024));

+    my $rd_total_mb = ($fs_data{$key}->[0] / (1024*1024));

</span> 

<span style="color: #000000;background-color: #ffdddd">-$cumul_write_shared /= $nprocs;

-$cumul_write_bytes_shared /= $nprocs;

-$cumul_write_bytes_shared /= 1048576.0;

</span><span style="color: #000000;background-color: #ddffdd">+    my $wr_total_rt;

+    if ($cumul_write_bytes_shared+$cumul_write_bytes_indep)

+    {

+        $wr_total_rt = ($fs_data{$key}->[1] / ($cumul_write_bytes_shared + $cumul_write_bytes_indep));

+    }

+    else

+    {

+        $wr_total_rt = 0;

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-$cumul_meta_shared /= $nprocs;

-$cumul_meta_indep /= $nprocs;

</span><span style="color: #000000;background-color: #ddffdd">+    my $rd_total_rt;

+    if ($cumul_read_bytes_shared+$cumul_read_bytes_indep)

+    {

+        $rd_total_rt = ($fs_data{$key}->[0] / ($cumul_read_bytes_shared + $cumul_read_bytes_indep));

+    }

+    else

+    {

+        $rd_total_rt = 0;

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-open(FILEACC, ">$tmp_dir/file-access-table.tex") || die("error opening output file:$!\n");

-print FILEACC "

-\\begin{tabular}{l|p{1.7in}r}

-\\multicolumn{3}{c}{Average I/O per process} \\\\

-\\hline

- \& Cumulative time spent in I/O functions (seconds) \& Amount of I/O (MB) \\\\

-\\hline

-\\hline

-";

</span><span style="color: #000000;background-color: #ddffdd">+    printf FS_TABLE "%s \& %.5f \& %.5f \& %.5f \& %.5f \\\\\n",

+        $key, $wr_total_mb, $wr_total_rt, $rd_total_mb, $rd_total_rt;

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-# printf to get consistent precision in output

-printf(FILEACC "Independent reads \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\", 

-    $cumul_read_indep, $cumul_read_bytes_indep);

-printf(FILEACC "Independent writes \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\", 

-    $cumul_write_indep, $cumul_write_bytes_indep);

-printf(FILEACC "Independent metadata \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{N/A} \\\\", 

-    $cumul_meta_indep);

-printf(FILEACC "Shared reads \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\", 

-    $cumul_read_shared, $cumul_read_bytes_shared);

-printf(FILEACC "Shared writes \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\", 

-    $cumul_write_shared, $cumul_write_bytes_shared);

-printf(FILEACC "Shared metadata \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{N/A} \\\\", 

-    $cumul_meta_shared);

-

-print FILEACC "

</span><span style="color: #000000;background-color: #ddffdd">+print FS_TABLE "

</span> \\hline

 \\end{tabular}

 ";

<span style="color: #000000;background-color: #ffdddd">-close(FILEACC);

</span><span style="color: #000000;background-color: #ddffdd">+close FS_TABLE;

</span> 

<span style="color: #000000;background-color: #ffdddd">-#

-# Variance Data

-#

-open(VARP, ">$tmp_dir/variance-table.tex") || die("error opening output file:$!\n");

-print VARP "

</span><span style="color: #000000;background-color: #ddffdd">+# variance data

+open(VAR_TABLE, ">$tmp_dir/variance-table.tex") || die("error opening output file:$!\n");

+print VAR_TABLE "

</span> \\begin{tabular}{c|r|r|r|r|r|r|r|r|r}

 \\multicolumn{10}{c}{} \\\\

<span style="color: #000000;background-color: #ffdddd">-\\multicolumn{10}{c}{Variance in Shared Files} \\\\

</span><span style="color: #000000;background-color: #ddffdd">+\\multicolumn{10}{c}{Variance in Shared Files (POSIX)} \\\\

</span> \\hline

 File \& Processes \& \\multicolumn{3}{c}{Fastest} \\vline \&

 \\multicolumn{3}{c}{Slowest} \\vline \& \\multicolumn{2}{c}{\$\\sigma\$} \\\\

<span style="color: #aaaaaa">@@ -955,16 +834,16 @@ foreach $key (sort { $hash_files{$b}{'slowest_time'} <=> $hash_files{$a}{'slowes

</span>         my $vb = sprintf("%.3g", sqrt($hash_files{$key}{'variance_bytes'}));

         my $fast_bytes = format_bytes($hash_files{$key}{'fastest_bytes'});

         my $slow_bytes = format_bytes($hash_files{$key}{'slowest_bytes'});

<span style="color: #000000;background-color: #ffdddd">-        my $name = encode('latex', $hash_files{$key}{'name'});

</span><span style="color: #000000;background-color: #ddffdd">+        my $name = encode('latex', "..." . substr($hash_files{$key}{'name'}, -12));

</span> 

<span style="color: #000000;background-color: #ffdddd">-        print VARP "

</span><span style="color: #000000;background-color: #ddffdd">+        print VAR_TABLE "

</span>                $name \&

                $hash_files{$key}{'procs'} \&

                $hash_files{$key}{'fastest_rank'} \&

                $hash_files{$key}{'fastest_time'} \&

                $fast_bytes \&

                $hash_files{$key}{'slowest_rank'} \&

<span style="color: #000000;background-color: #ffdddd">-               $hash_files{$key}{'slowest_time'} \& 

</span><span style="color: #000000;background-color: #ddffdd">+               $hash_files{$key}{'slowest_time'} \&

</span>                $slow_bytes \&

                $vt \&

                $vb \\\\

<span style="color: #aaaaaa">@@ -973,11 +852,11 @@ foreach $key (sort { $hash_files{$b}{'slowest_time'} <=> $hash_files{$a}{'slowes

</span>     }

 }

<span style="color: #000000;background-color: #ffdddd">-print VARP "

</span><span style="color: #000000;background-color: #ddffdd">+print VAR_TABLE "

</span> \\hline

 \\end{tabular}

 ";

<span style="color: #000000;background-color: #ffdddd">-close(VARP);

</span><span style="color: #000000;background-color: #ddffdd">+close VAR_TABLE;

</span> 

 # calculate performance

 ##########################################################################

<span style="color: #aaaaaa">@@ -993,34 +872,114 @@ print("Slowest shared file time: $shared_file_time\n");

</span> print("Total bytes read and written by app (may be incorrect): $total_job_bytes\n");

 my $tmp_total_time = $slowest_uniq_time+$shared_file_time;

 print("Total absolute I/O time: $tmp_total_time\n");

<span style="color: #000000;background-color: #ddffdd">+print("**NOTE: above shared and unique file times calculated using MPI-IO timers if MPI-IO interface used on a given file, POSIX timers otherwise.\n");

</span> 

 # move to tmp_dir

 chdir $tmp_dir;

<span style="color: #000000;background-color: #ffdddd">-# execute gnuplot scripts

-system "$gnuplot counts-eps.gplt";

-system "$epstopdf counts.eps";

-system "$gnuplot hist-eps.gplt";

-system "$epstopdf hist.eps";

-system "$gnuplot pattern-eps.gplt";

-system "$epstopdf pattern.eps";

</span><span style="color: #000000;background-color: #ddffdd">+# gather data to be used for document title (headers/footers)

+($executable, $junk) = split(' ', $cmdline, 2);

+@parts = split('/', $executable);

+$cmd = $parts[$#parts];

+@timearray = localtime($starttime);

+$year = $timearray[5] + 1900;

+$mon = $timearray[4] + 1;

+$mday = $timearray[3];

+

+# detect gnuplot ranges for file access graphs

+my $ymax = $nprocs;

+my $yinc = int($nprocs / 8);

+if($yinc == 0) {$yinc=1;}

+my $ymaxtic = $nprocs-1;

+

+# reformat cumulative i/o data for file access table

+my $cri = $cumul_read_indep / $nprocs;

+my $crbi = $cumul_read_bytes_indep / ($nprocs * 1048576.0);

+

+my $cwi = $cumul_write_indep / $nprocs;

+my $cwbi = $cumul_write_bytes_indep / ($nprocs * 1048576.0);

+

+my $crs = $cumul_read_shared / $nprocs;

+my $crbs = $cumul_read_bytes_shared / ($nprocs * 1048576.0);

+

+my $cws = $cumul_write_shared / $nprocs;

+my $cwbs = $cumul_write_bytes_shared / ($nprocs * 1048576.0);

+

+my $cmi = $cumul_meta_indep / $nprocs;

+my $cms = $cumul_meta_shared / $nprocs;

+

+# do any extra work needed for plotting mpi-io graphs

+if (defined $summary{MPIIO_INDEP_OPENS})

+{

+    system "$gnuplot -e \"data_file='mpiio-access-hist.dat'; graph_title='MPI-IO Access Sizes {/Times-Bold=32 \263}'; \\

+    output_file='mpiio-access-hist.eps'\" access-hist-eps.gplt";

+    system "$epstopdf mpiio-access-hist.eps";

+

+    open(OP_COUNTS_PLT, ">>$tmp_dir/op-counts-eps.gplt") || die("error opening output file: $!\n");

+    my $tmp_sz = -s "$tmp_dir/op-counts-eps.gplt";

+    # overwrite existing newline

+    truncate(OP_COUNTS_PLT, $tmp_sz-1);

+    print OP_COUNTS_PLT ", \\

+    \"mpiio-op-counts.dat\" using 2:xtic(1) title \"MPI-IO Indep.\", \\

+    \"\" using 3 title \"MPI-IO Coll.\"\n";

+    close OP_COUNTS_PLT;

+}

+

+# execute base gnuplot scripts

</span> system "$gnuplot time-summary-eps.gplt";

 system "$epstopdf time-summary.eps";

<span style="color: #000000;background-color: #ffdddd">-system "$gnuplot file-access-read-eps.gplt";

</span><span style="color: #000000;background-color: #ddffdd">+system "$gnuplot op-counts-eps.gplt";

+system "$epstopdf op-counts.eps";

+system "$gnuplot -e \"data_file='posix-access-hist.dat'; graph_title='POSIX Access Sizes'; \\

+output_file='posix-access-hist.eps'\" access-hist-eps.gplt";

+system "$epstopdf posix-access-hist.eps";

+system "$gnuplot -e \"ymax=$ymax; yinc=$yinc; ymaxtic=$ymaxtic; runtime='$runtime'\" file-access-eps.gplt";

</span> system "$epstopdf file-access-read.eps";

<span style="color: #000000;background-color: #ffdddd">-system "$gnuplot file-access-write-eps.gplt";

</span> system "$epstopdf file-access-write.eps";

<span style="color: #000000;background-color: #ffdddd">-system "$gnuplot file-access-shared-eps.gplt";

</span> system "$epstopdf file-access-shared.eps";

<span style="color: #000000;background-color: #ffdddd">-

-#system "gnuplot align-pdf.gplt";

-#system "gnuplot iodist-pdf.gplt";

-#system "gnuplot types-pdf.gplt";

</span><span style="color: #000000;background-color: #ddffdd">+system "$gnuplot pattern-eps.gplt";

+system "$epstopdf pattern.eps";

</span> 

 # generate summary PDF

<span style="color: #000000;background-color: #ddffdd">+# NOTE: we pass arguments to the latex template using '\def' commands

</span> # NOTE: an autoconf test determines if -halt-on-error is available and sets

<span style="color: #000000;background-color: #ffdddd">-# __CP_PDFLATEX_HALT_ON_ERROR accordingly

-$system_rc = system "$pdflatex @__CP_PDFLATEX_HALT_ON_ERROR@ summary.tex > latex.output";

</span><span style="color: #000000;background-color: #ddffdd">+# __DARSHAN_PDFLATEX_HALT_ON_ERROR accordingly

+my $latex_cmd_line = "\"\\def\\titlecmd{$cmd} \\

+    \\def\\titlemon{$mon} \\

+    \\def\\titlemday{$mday} \\

+    \\def\\titleyear{$year} \\

+    \\def\\titlecmdline{$cmdline} \\

+    \\def\\jobid{$jobid} \\

+    \\def\\jobuid{$uid} \\

+    \\def\\jobnprocs{$nprocs} \\

+    \\def\\jobruntime{$runtime} \\

+    \\def\\filecri{$cri} \\

+    \\def\\filecrbi{$crbi} \\

+    \\def\\filecwi{$cwi} \\

+    \\def\\filecwbi{$cwbi} \\

+    \\def\\filecrs{$crs} \\

+    \\def\\filecrbs{$crbs} \\

+    \\def\\filecws{$cws} \\

+    \\def\\filecwbs{$cwbs} \\

+    \\def\\filecmi{$cmi} \\

+    \\def\\filecms{$cms} \\

+    \\input{summary.tex}\" \\

+    @__DARSHAN_PDFLATEX_HALT_ON_ERROR@";

+

+if ($partial_flag == 1)

+{

+    my $partial_log_flags = "\\def\\incompletelog{1} \\";

+    $latex_cmd_line = substr($latex_cmd_line, 0, 1) . $partial_log_flags . substr($latex_cmd_line, 1);

+}

+

+if (defined $summary{MPIIO_INDEP_OPENS})

+{

+    my $mpiio_latex_flags = "\\def\\inclmpiio{1} \\";

+    $latex_cmd_line = substr($latex_cmd_line, 0, 1) . $mpiio_latex_flags . substr($latex_cmd_line, 1);

+}

+

+

+$system_rc = system "$pdflatex $latex_cmd_line > latex.output";

</span> if($system_rc)

 {

     print("LaTeX generation (phase1) failed [$system_rc], aborting summary creation.\n");

<span style="color: #aaaaaa">@@ -1028,7 +987,7 @@ if($system_rc)

</span>     system("tail latex.output");

     exit(1);

 }

<span style="color: #000000;background-color: #ffdddd">-$system_rc = system "$pdflatex @__CP_PDFLATEX_HALT_ON_ERROR@ summary.tex > latex.output2";

</span><span style="color: #000000;background-color: #ddffdd">+$system_rc = system "$pdflatex $latex_cmd_line > latex.output2";

</span> if($system_rc)

 {

     print("LaTeX generation (phase2) failed [$system_rc], aborting summary creation.\n");

<span style="color: #aaaaaa">@@ -1044,53 +1003,43 @@ system "$mv $tmp_dir/summary.pdf $output_file";

</span> 

 sub process_file_record

 {

<span style="color: #000000;background-color: #ffdddd">-    my $rank = $_[0];

-    my $hash = $_[1];

-    my(%file_record) = %{$_[2]};

-

-    if($file_record{'CP_INDEP_OPENS'} == 0 &&

-        $file_record{'CP_COLL_OPENS'} == 0 &&

-        $file_record{'CP_POSIX_OPENS'} == 0 &&

-        $file_record{'CP_POSIX_FOPENS'} == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    my $hash = $_[0];

+    my(%file_record) = %{$_[1]};

+    my $rank = $file_record{'RANK'};

+

+    if(!defined $file_record{'POSIX_OPENS'})

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        # file wasn't really opened, just stat probably

</span><span style="color: #000000;background-color: #ddffdd">+        # ignore data records that don't have POSIX & MPI data

</span>         return;

     }

<span style="color: #000000;background-color: #ffdddd">-    # record smallest open time size reported by any rank

-    if(!defined($hash_files{$hash}{'min_open_size'}) ||

-        $hash_files{$hash}{'min_open_size'} > 

-        $file_record{'CP_SIZE_AT_OPEN'})

</span><span style="color: #000000;background-color: #ddffdd">+    if($file_record{'POSIX_OPENS'} == 0 &&

+        $file_record{'POSIX_FOPENS'} == 0 &&

+        (!defined $file_record{'MPIIO_INDEP_OPENS'} ||

+        ($file_record{'MPIIO_INDEP_OPENS'} == 0 && $file_record{'MPIIO_COLL_OPENS'} == 0)))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        # size at open will be set to -1 if the darshan library was not

-        # configured to stat files at open time

-        if($file_record{'CP_SIZE_AT_OPEN'} < 0)

-        {

-            $hash_files{$hash}{'min_open_size'} = 0;

-            # set flag indicating that file sizes are estimated 

-            $size_est_flag = 1;

-        }

-        else

-        {

-            $hash_files{$hash}{'min_open_size'} = 

-                $file_record{'CP_SIZE_AT_OPEN'};

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        # file wasn't really opened, just stat probably

+        return;

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    # record smallest open time size reported by any rank

+    # XXX this isn't doable since dropping SIZE_AT_OPEN counter

+    $hash_files{$hash}{'min_open_size'} = 0;

+

</span>     # record largest size that the file reached at any rank

     if(!defined($hash_files{$hash}{'max_size'}) ||

         $hash_files{$hash}{'max_size'} <  

<span style="color: #000000;background-color: #ffdddd">-        ($file_record{'CP_MAX_BYTE_READ'} + 1))

</span><span style="color: #000000;background-color: #ddffdd">+        ($file_record{'POSIX_MAX_BYTE_READ'} + 1))

</span>     {

         $hash_files{$hash}{'max_size'} = 

<span style="color: #000000;background-color: #ffdddd">-            $file_record{'CP_MAX_BYTE_READ'} + 1;

</span><span style="color: #000000;background-color: #ddffdd">+            $file_record{'POSIX_MAX_BYTE_READ'} + 1;

</span>     }

     if(!defined($hash_files{$hash}{'max_size'}) ||

         $hash_files{$hash}{'max_size'} <  

<span style="color: #000000;background-color: #ffdddd">-        ($file_record{'CP_MAX_BYTE_WRITTEN'} + 1))

</span><span style="color: #000000;background-color: #ddffdd">+        ($file_record{'POSIX_MAX_BYTE_WRITTEN'} + 1))

</span>     {

         $hash_files{$hash}{'max_size'} = 

<span style="color: #000000;background-color: #ffdddd">-            $file_record{'CP_MAX_BYTE_WRITTEN'} + 1;

</span><span style="color: #000000;background-color: #ddffdd">+            $file_record{'POSIX_MAX_BYTE_WRITTEN'} + 1;

</span>     }

     # make sure there is an initial value for read and write flags

<span style="color: #aaaaaa">@@ -1103,22 +1052,23 @@ sub process_file_record

</span>         $hash_files{$hash}{'was_written'} = 0;

     }

<span style="color: #000000;background-color: #ffdddd">-    if($file_record{'CP_INDEP_OPENS'} > 0 ||

-        $file_record{'CP_COLL_OPENS'} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+    if(defined $file_record{'MPIIO_INDEP_OPENS'} &&

+        ($file_record{'MPIIO_INDEP_OPENS'} > 0 ||

+        $file_record{'MPIIO_COLL_OPENS'} > 0))

</span>     {

         # mpi file

<span style="color: #000000;background-color: #ffdddd">-        if($file_record{'CP_INDEP_READS'} > 0 ||

-            $file_record{'CP_COLL_READS'} > 0 ||

-            $file_record{'CP_SPLIT_READS'} > 0 ||

-            $file_record{'CP_NB_READS'} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+        if($file_record{'MPIIO_INDEP_READS'} > 0 ||

+            $file_record{'MPIIO_COLL_READS'} > 0 ||

+            $file_record{'MPIIO_SPLIT_READS'} > 0 ||

+            $file_record{'MPIIO_NB_READS'} > 0)

</span>         {

             # data was read from the file

             $hash_files{$hash}{'was_read'} = 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        if($file_record{'CP_INDEP_WRITES'} > 0 ||

-            $file_record{'CP_COLL_WRITES'} > 0 ||

-            $file_record{'CP_SPLIT_WRITES'} > 0 ||

-            $file_record{'CP_NB_WRITES'} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+        if($file_record{'MPIIO_INDEP_WRITES'} > 0 ||

+            $file_record{'MPIIO_COLL_WRITES'} > 0 ||

+            $file_record{'MPIIO_SPLIT_WRITES'} > 0 ||

+            $file_record{'MPIIO_NB_WRITES'} > 0)

</span>         {

             # data was written to the file

             $hash_files{$hash}{'was_written'} = 1;

<span style="color: #aaaaaa">@@ -1127,42 +1077,42 @@ sub process_file_record

</span>     else

     {

         # posix file

<span style="color: #000000;background-color: #ffdddd">-        if($file_record{'CP_POSIX_READS'} > 0 ||

-            $file_record{'CP_POSIX_FREADS'} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+        if($file_record{'POSIX_READS'} > 0 ||

+            $file_record{'POSIX_FREADS'} > 0)

</span>         {

             # data was read from the file

             $hash_files{$hash}{'was_read'} = 1;

         }

<span style="color: #000000;background-color: #ffdddd">-        if($file_record{'CP_POSIX_WRITES'} > 0 ||

-            $file_record{'CP_POSIX_FWRITES'} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+        if($file_record{'POSIX_WRITES'} > 0 ||

+            $file_record{'POSIX_FWRITES'} > 0)

</span>         {

             # data was written to the file 

             $hash_files{$hash}{'was_written'} = 1;

         }

     }

<span style="color: #000000;background-color: #ffdddd">-    $hash_files{$hash}{'name'} = $file_record{CP_NAME_SUFFIX};

</span><span style="color: #000000;background-color: #ddffdd">+    $hash_files{$hash}{'name'} = $file_record{FILE_NAME};

</span> 

     if ($rank == -1)

     {

         $hash_files{$hash}{'procs'}          = $nprocs;

<span style="color: #000000;background-color: #ffdddd">-        $hash_files{$hash}{'slowest_rank'}   = $file_record{'CP_SLOWEST_RANK'};

-        $hash_files{$hash}{'slowest_time'}   = $file_record{'CP_F_SLOWEST_RANK_TIME'};

-        $hash_files{$hash}{'slowest_bytes'}  = $file_record{'CP_SLOWEST_RANK_BYTES'};

-        $hash_files{$hash}{'fastest_rank'}   = $file_record{'CP_FASTEST_RANK'};

-        $hash_files{$hash}{'fastest_time'}   = $file_record{'CP_F_FASTEST_RANK_TIME'};

-        $hash_files{$hash}{'fastest_bytes'}  = $file_record{'CP_FASTEST_RANK_BYTES'};

-        $hash_files{$hash}{'variance_time'}  = $file_record{'CP_F_VARIANCE_RANK_TIME'};

-        $hash_files{$hash}{'variance_bytes'} = $file_record{'CP_F_VARIANCE_RANK_BYTES'};

</span><span style="color: #000000;background-color: #ddffdd">+        $hash_files{$hash}{'slowest_rank'}   = $file_record{'POSIX_SLOWEST_RANK'};

+        $hash_files{$hash}{'slowest_time'}   = $file_record{'POSIX_F_SLOWEST_RANK_TIME'};

+        $hash_files{$hash}{'slowest_bytes'}  = $file_record{'POSIX_SLOWEST_RANK_BYTES'};

+        $hash_files{$hash}{'fastest_rank'}   = $file_record{'POSIX_FASTEST_RANK'};

+        $hash_files{$hash}{'fastest_time'}   = $file_record{'POSIX_F_FASTEST_RANK_TIME'};

+        $hash_files{$hash}{'fastest_bytes'}  = $file_record{'POSIX_FASTEST_RANK_BYTES'};

+        $hash_files{$hash}{'variance_time'}  = $file_record{'POSIX_F_VARIANCE_RANK_TIME'};

+        $hash_files{$hash}{'variance_bytes'} = $file_record{'POSIX_F_VARIANCE_RANK_BYTES'};

</span>     }

     else

     {

<span style="color: #000000;background-color: #ffdddd">-        my $total_time = $file_record{'CP_F_POSIX_META_TIME'} +

-                         $file_record{'CP_F_POSIX_READ_TIME'} +

-                         $file_record{'CP_F_POSIX_WRITE_TIME'};

</span><span style="color: #000000;background-color: #ddffdd">+        my $total_time = $file_record{'POSIX_F_META_TIME'} +

+                         $file_record{'POSIX_F_READ_TIME'} +

+                         $file_record{'POSIX_F_WRITE_TIME'};

</span> 

<span style="color: #000000;background-color: #ffdddd">-        my $total_bytes = $file_record{'CP_BYTES_READ'} +

-                          $file_record{'CP_BYTES_WRITTEN'};

</span><span style="color: #000000;background-color: #ddffdd">+        my $total_bytes = $file_record{'POSIX_BYTES_READ'} +

+                          $file_record{'POSIX_BYTES_WRITTEN'};

</span> 

         if(!defined($hash_files{$hash}{'slowest_time'}) ||

            $hash_files{$hash}{'slowest_time'} < $total_time)

<span style="color: #aaaaaa">@@ -1218,26 +1168,28 @@ sub process_file_record

</span> 

     # if this is a non-shared file, then add the time spent here to the

     # total for that particular rank

<span style="color: #000000;background-color: #ddffdd">+    # XXX mpiio or posix? should we do both or just pick mpiio over posix?

</span>     if ($rank != -1)

     {

         # is it mpi-io or posix?

<span style="color: #000000;background-color: #ffdddd">-        if($file_record{CP_INDEP_OPENS} > 0 ||

-            $file_record{CP_COLL_OPENS} > 0)

</span><span style="color: #000000;background-color: #ddffdd">+        if(defined $file_record{MPIIO_INDEP_OPENS} &&

+            ($file_record{MPIIO_INDEP_OPENS} > 0 ||

+            $file_record{MPIIO_COLL_OPENS} > 0))

</span>         {

             # add up mpi times

             if(defined($hash_unique_file_time{$rank}))

             {

                 $hash_unique_file_time{$rank} +=

<span style="color: #000000;background-color: #ffdddd">-                    $file_record{CP_F_MPI_META_TIME} + 

-                    $file_record{CP_F_MPI_READ_TIME} + 

-                    $file_record{CP_F_MPI_WRITE_TIME};

</span><span style="color: #000000;background-color: #ddffdd">+                    $file_record{MPIIO_F_META_TIME} + 

+                    $file_record{MPIIO_F_READ_TIME} + 

+                    $file_record{MPIIO_F_WRITE_TIME};

</span>             }

             else

             {

                 $hash_unique_file_time{$rank} =

<span style="color: #000000;background-color: #ffdddd">-                    $file_record{CP_F_MPI_META_TIME} + 

-                    $file_record{CP_F_MPI_READ_TIME} + 

-                    $file_record{CP_F_MPI_WRITE_TIME};

</span><span style="color: #000000;background-color: #ddffdd">+                    $file_record{MPIIO_F_META_TIME} + 

+                    $file_record{MPIIO_F_READ_TIME} + 

+                    $file_record{MPIIO_F_WRITE_TIME};

</span>             }

         }

         else

<span style="color: #aaaaaa">@@ -1246,86 +1198,60 @@ sub process_file_record

</span>             if(defined($hash_unique_file_time{$rank}))

             {

                 $hash_unique_file_time{$rank} +=

<span style="color: #000000;background-color: #ffdddd">-                    $file_record{CP_F_POSIX_META_TIME} + 

-                    $file_record{CP_F_POSIX_READ_TIME} + 

-                    $file_record{CP_F_POSIX_WRITE_TIME};

</span><span style="color: #000000;background-color: #ddffdd">+                    $file_record{POSIX_F_META_TIME} + 

+                    $file_record{POSIX_F_READ_TIME} + 

+                    $file_record{POSIX_F_WRITE_TIME};

</span>             }

             else

             {

                 $hash_unique_file_time{$rank} =

<span style="color: #000000;background-color: #ffdddd">-                    $file_record{CP_F_POSIX_META_TIME} + 

-                    $file_record{CP_F_POSIX_READ_TIME} + 

-                    $file_record{CP_F_POSIX_WRITE_TIME};

</span><span style="color: #000000;background-color: #ddffdd">+                    $file_record{POSIX_F_META_TIME} + 

+                    $file_record{POSIX_F_READ_TIME} + 

+                    $file_record{POSIX_F_WRITE_TIME};

</span>             }

         }

     }

     else

     {

<span style="color: #000000;background-color: #ffdddd">-

</span>         # cumulative time spent on shared files by slowest proc

<span style="color: #000000;background-color: #ffdddd">-        if($major > 1)

<span style="color: #000000;background-color: #ddffdd">+        # is it mpi-io or posix?

+        if(defined $file_record{MPIIO_INDEP_OPENS} &&

+            ($file_record{MPIIO_INDEP_OPENS} > 0 ||

+            $file_record{MPIIO_COLL_OPENS} > 0))

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            # new file format

-            $shared_file_time += $file_record{'CP_F_SLOWEST_RANK_TIME'};

</span><span style="color: #000000;background-color: #ddffdd">+            $shared_file_time += $file_record{'MPIIO_F_SLOWEST_RANK_TIME'};

</span>         }

         else

         {

<span style="color: #000000;background-color: #ffdddd">-            # old file format.  Guess time spent as duration between first open

-            # and last io

-            if($file_record{'CP_F_READ_END_TIMESTAMP'} >

-                $file_record{'CP_F_WRITE_END_TIMESTAMP'})

-            {

-                # be careful of files that were opened but not read or

-                # written

-                if($file_record{'CP_F_READ_END_TIMESTAMP'} > $file_record{'CP_F_OPEN_TIMESTAMP'}) {

-                    $shared_file_time += $file_record{'CP_F_READ_END_TIMESTAMP'} -

-                        $file_record{'CP_F_OPEN_TIMESTAMP'};

-                }

-            }

-            else

-            {

-                if($file_record{'CP_F_WRITE_END_TIMESTAMP'} > $file_record{'CP_F_OPEN_TIMESTAMP'}) {

-                    $shared_file_time += $file_record{'CP_F_WRITE_END_TIMESTAMP'} -

-                        $file_record{'CP_F_OPEN_TIMESTAMP'};

-                }

-            }

</span><span style="color: #000000;background-color: #ddffdd">+            $shared_file_time += $file_record{'POSIX_F_SLOWEST_RANK_TIME'};

</span>         }

     }

<span style="color: #000000;background-color: #ffdddd">-    my $mpi_did_read = 

-        $file_record{'CP_INDEP_READS'} + 

-        $file_record{'CP_COLL_READS'} + 

-        $file_record{'CP_NB_READS'} + 

-        $file_record{'CP_SPLIT_READS'};

</span><span style="color: #000000;background-color: #ddffdd">+    my $mpi_did_read = 0;

+    if (defined $file_record{MPIIO_INDEP_OPENS})

+    {

+        $mpi_did_read =

+            $file_record{'MPIIO_INDEP_READS'} + 

+            $file_record{'MPIIO_COLL_READS'} + 

+            $file_record{'MPIIO_NB_READS'} + 

+            $file_record{'MPIIO_SPLIT_READS'};

+    }

</span> 

     # add up how many bytes were transferred

<span style="color: #000000;background-color: #ffdddd">-    if(($file_record{CP_INDEP_OPENS} > 0 ||

-        $file_record{CP_COLL_OPENS} > 0) && (!($mpi_did_read)))

</span><span style="color: #000000;background-color: #ddffdd">+    if(defined $file_record{MPIIO_INDEP_OPENS} &&

+        ($file_record{MPIIO_INDEP_OPENS} > 0 ||

+        $file_record{MPIIO_COLL_OPENS} > 0) && (!($mpi_did_read)))

</span>     {

         # mpi file that was only written; disregard any read accesses that

         # may have been performed for sieving at the posix level

<span style="color: #000000;background-color: #ffdddd">-        $total_job_bytes += $file_record{'CP_BYTES_WRITTEN'}; 

</span><span style="color: #000000;background-color: #ddffdd">+        $total_job_bytes += $file_record{'POSIX_BYTES_WRITTEN'}; 

</span>     }

     else

     {

         # normal case

<span style="color: #000000;background-color: #ffdddd">-        $total_job_bytes += $file_record{'CP_BYTES_WRITTEN'} +

-            $file_record{'CP_BYTES_READ'};

</span><span style="color: #000000;background-color: #ddffdd">+        $total_job_bytes += $file_record{'POSIX_BYTES_WRITTEN'} +

+            $file_record{'POSIX_BYTES_READ'};

</span>     }

<span style="color: #000000;background-color: #ffdddd">-

-    # TODO 

-    # (detect mpi or posix and):

-    # - sum meta time per rank for uniq files

-    # - sum io time per rank for uniq files

-    # - sum time from first open to last io for shared files

-    # - sum meta time/nprocs for shared files

-    # - sum io time/nprocs for shared files

-    

-    # TODO: ideas

-    # graph time spent performing I/O per rank

-    # for rank that spent the most time performing I/O:

-    # - meta on ro files, meta on wo files, read time, write time

-    # table with nfiles accessed, ro, wo, rw, created

</span> }

 sub process_args

<span style="color: #aaaaaa">@@ -1364,19 +1290,11 @@ sub process_args

</span>     $input_file = $ARGV[0];

     # give default output file a similar name to the input file.

<span style="color: #000000;background-color: #ffdddd">-    #   log.darshan.gz => log.pdf

</span>     #   log_name => log_name.pdf

     if (not $opt_output)

     {

         $output_file = basename($input_file);

<span style="color: #000000;background-color: #ffdddd">-        if ($output_file =~ /\.darshan\.gz$/)

-        {

-            $output_file =~ s/\.darshan\.gz$/\.pdf/;

-        }

-        else

-        {

-            $output_file .= ".pdf";

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        $output_file .= ".pdf";

</span>     }

     return;

<span style="color: #aaaaaa">@@ -1401,14 +1319,14 @@ sub check_prereqs

</span>         }

     }

<span style="color: #000000;background-color: #ffdddd">-    # check  gnuplot version

</span><span style="color: #000000;background-color: #ddffdd">+    # check gnuplot version

</span>     $output = `$gnuplot --version`;

     if($? != 0)

     {

         print("error: failed to execute $gnuplot.\n");

         exit(1);

     }

<span style="color: #000000;background-color: #ffdddd">-    

</span><span style="color: #000000;background-color: #ddffdd">+

</span>     $output =~ /gnuplot (\d+)\.(\d+)/;

     if($1 < 4 || ($1 < 5 && $2 < 2))

     {

</code></pre>

<br>

</li>

<li id='diff-147'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-147'>

<strong>

darshan-util/darshan-job-summary/share/hist-eps.gplt

</strong>

→

<strong>

darshan-util/darshan-job-summary/share/access-hist-eps.gplt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/hist-eps.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/access-hist-eps.gplt

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/gnuplot -persist

 set terminal postscript eps color enhanced font "Helvetica" 18

<span style="color: #000000;background-color: #ffdddd">-set output "hist.eps"

</span><span style="color: #000000;background-color: #ddffdd">+set output output_file

</span> set size 0.8,1.0

 set style data histogram

<span style="color: #aaaaaa">@@ -11,7 +11,7 @@ set boxwidth 0.9

</span> set bmargin 5

 set border 3 front linetype -1 linewidth 1.000

<span style="color: #000000;background-color: #ffdddd">-set key out bottom horiz

</span><span style="color: #000000;background-color: #ddffdd">+set key out bottom center horiz

</span> set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

 set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

<span style="color: #aaaaaa">@@ -22,7 +22,6 @@ set ylabel "Count (Total, All Procs)"

</span> set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

 set bmargin 7

<span style="color: #000000;background-color: #ffdddd">-set title "I/O Sizes"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "hist.dat" using 2:xtic(1) title "Read", \

-      "" using 3 title "Write"

</span><span style="color: #000000;background-color: #ddffdd">+set title graph_title

+plot data_file using 2:xtic(1) title "Read", \

+     "" using 3 title "Write"

</span></code></pre>

<br>

</li>

<li id='diff-148'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-148'>

<strong>

darshan-util/darshan-job-summary/share/align-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/align-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,29 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "align.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-unset xtics

-# set xtics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set datafile separator ","

-

-set ylabel "Ops (Total, All Procs)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Alignment"

-plot  "align.dat" using 1 title "Total", \

-      "" using 3 title "Unaligned in File", \

-      "" using 2 title "Unaligned in Memory"

-

-set key title ""

</span></code></pre>

<br>

</li>

<li id='diff-149'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-149'>

<strong>

darshan-util/darshan-job-summary/share/counts-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/counts-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,28 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "counts.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Ops (Total, All Procs)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Operation Counts"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "counts.dat" using 2:xtic(1) title "POSIX", \

-      "" using 3 title "MPI-IO Ind.", \

-      "" using 4 title "MPI-IO Coll."

</span></code></pre>

<br>

</li>

<li id='diff-150'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-150'>

<strong>

darshan-util/darshan-job-summary/share/counts-svg.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/counts-svg.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,27 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal svg enhanced size 800 600 fname "Gill Sans" fsize 15

-set output "counts.svg"

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Operations (Total, All Processes)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Operation Counts"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "counts.dat" using 2:xtic(1) title "POSIX", \

-      "" using 3 title "MPI-IO Indep.", \

-      "" using 4 title "MPI-IO Coll."

</span></code></pre>

<br>

</li>

<li id='diff-151'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-151'>

<strong>

darshan-util/darshan-job-summary/share/file-access-eps.gplt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/file-access-eps.gplt

</span><span style="color: #aaaaaa">@@ -0,0 +1,40 @@

</span><span style="color: #000000;background-color: #ddffdd">+#!/usr/bin/gnuplot -persist

+

+set terminal postscript eps color solid font "Helvetica" 18 size 10in,2.5in

+set output "file-access-read.eps"

+set ylabel "MPI rank"

+set xlabel "hours:minutes:seconds"

+set xdata time

+set timefmt "%s"

+set format x "%H:%M:%S"

+set yrange [-1:ymax]

+set title "Timespan from first to last read access on independent files (POSIX)"

+set xrange ["0":runtime]

+set ytics 0,yinc,ymaxtic

+set lmargin 6

+

+# color blindness work around

+set style line 2 lc 3

+set style line 3 lc 4

+set style line 4 lc 5

+set style line 5 lc 2

+set style increment user

+

+# lw 3 to make lines thicker...

+# note that writes are slightly offset for better visibility

+plot "file-access-read.dat" using 1:2:3:4 with vectors nohead filled notitle

+

+set output "file-access-write.eps"

+set title "Timespan from first to last write access on independent files (POSIX)"

+

+# lw 3 to make lines thicker...

+plot "file-access-write.dat" using 1:2:3:4 with vectors nohead filled lt 2 notitle

+

+set output "file-access-shared.eps"

+unset ytics

+set ylabel "All processes"

+set yrange [-1:1]

+set title "Timespan from first to last access on files shared by all processes (POSIX)"

+

+plot "file-access-read-sh.dat" using 1:2:3:4 with vectors nohead filled lw 10 title "read", \

+"file-access-write-sh.dat" using 1:(($2)-.2):3:4 with vectors nohead filled lw 10 title "write"

</span></code></pre>

<br>

</li>

<li id='diff-152'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-152'>

<strong>

darshan-util/darshan-job-summary/share/file-access-table.tex

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/file-access-table.tex

</span><span style="color: #aaaaaa">@@ -0,0 +1,14 @@

</span><span style="color: #000000;background-color: #ddffdd">+\begin{tabular}{l|p{1.7in}r}

+\multicolumn{3}{c}{Average I/O per process (POSIX)} \\

+\hline

+ & Cumulative time spent in I/O functions (seconds) & Amount of I/O (MB) \\

+\hline

+\hline

+Independent reads & \multicolumn{1}{r}{\filecri} & \multicolumn{1}{r}{\filecrbi} \\

+Independent writes & \multicolumn{1}{r}{\filecwi} & \multicolumn{1}{r}{\filecwbi} \\

+Independent metadata & \multicolumn{1}{r}{\filecmi} & \multicolumn{1}{r}{N/A} \\

+Shared reads & \multicolumn{1}{r}{\filecrs} & \multicolumn{1}{r}{\filecrbs} \\

+Shared writes & \multicolumn{1}{r}{\filecws} & \multicolumn{1}{r}{\filecwbs} \\

+Shared metadata & \multicolumn{1}{r}{\filecms} & \multicolumn{1}{r}{N/A} \\

+\hline

+\end{tabular}

</span></code></pre>

<br>

</li>

<li id='diff-153'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-153'>

<strong>

darshan-util/darshan-job-summary/share/hist-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/hist-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,27 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "hist.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Count (Total, All Procs)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Sizes"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "hist.dat" using 2:xtic(1) title "Read", \

-      "" using 3 title "Write"

</span></code></pre>

<br>

</li>

<li id='diff-154'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-154'>

<strong>

darshan-util/darshan-job-summary/share/hist-svg.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/hist-svg.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,26 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal svg enhanced size 800 600 fname "Gill Sans" fsize 15

-set output "hist.svg"

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Count (Total, All Processes)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Sizes"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "hist.dat" using 2:xtic(1) title "Read", \

-      "" using 3 title "Write"

</span></code></pre>

<br>

</li>

<li id='diff-155'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-155'>

<strong>

darshan-util/darshan-job-summary/share/iodist-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/iodist-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,29 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "iodist.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-# set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set xlabel "Number of Files Accessed"

-set xlabel offset character 0,-1,0 font "" textcolor lt -1 norotate

-

-set ylabel "Number of Processes"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Operations per Client"

-plot  "iodist.dat" using 2:xtic(1) title "Read", \

-      "" using 3 title "Write (NOT WORKING YET)"

</span></code></pre>

<br>

</li>

<li id='diff-156'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-156'>

<strong>

darshan-util/darshan-job-summary/share/job-table.tex

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/job-table.tex

</span><span style="color: #aaaaaa">@@ -0,0 +1,5 @@

</span><span style="color: #000000;background-color: #ddffdd">+\begin{tabular}{|p{.47\columnwidth}|p{.35\columnwidth}|p{.47\columnwidth}|p{.6\columnwidth}|}

+\hline

+jobid: \jobid & uid: \jobuid & nprocs: \jobnprocs & runtime: \jobruntime \space seconds\\

+\hline

+\end{tabular}

</span></code></pre>

<br>

</li>

<li id='diff-157'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-157'>

<strong>

darshan-util/darshan-job-summary/share/counts-eps.gplt

</strong>

→

<strong>

darshan-util/darshan-job-summary/share/op-counts-eps.gplt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/counts-eps.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/op-counts-eps.gplt

</span><span style="color: #aaaaaa">@@ -1,7 +1,7 @@

</span> #!/usr/bin/gnuplot -persist

 set terminal postscript eps color enhanced "Helvetica" 18

<span style="color: #000000;background-color: #ffdddd">-set output "counts.eps"

</span><span style="color: #000000;background-color: #ddffdd">+set output "op-counts.eps"

</span> set size 0.8,1.0

 set style data histogram

<span style="color: #aaaaaa">@@ -11,7 +11,7 @@ set boxwidth 0.9

</span> set bmargin 5

 set border 3 front linetype -1 linewidth 1.000

<span style="color: #000000;background-color: #ffdddd">-set key out bottom horiz

</span><span style="color: #000000;background-color: #ddffdd">+set key out bottom center horiz

</span> set xtics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

 set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

<span style="color: #aaaaaa">@@ -22,7 +22,4 @@ set ylabel "Ops (Total, All Processes)"

</span> set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

 set title "I/O Operation Counts"

<span style="color: #000000;background-color: #ffdddd">-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "counts.dat" using 2:xtic(1) title "POSIX", \

-      "" using 3 title "MPI-IO Indep.", \

-      "" using 4 title "MPI-IO Coll."

</span><span style="color: #000000;background-color: #ddffdd">+plot "posix-op-counts.dat" using 2:xtic(1) title "POSIX"

</span></code></pre>

<br>

</li>

<li id='diff-158'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-158'>

<strong>

darshan-util/darshan-job-summary/share/pattern-eps.gplt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/pattern-eps.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/pattern-eps.gplt

</span><span style="color: #aaaaaa">@@ -22,7 +22,7 @@ set datafile separator ","

</span> set ylabel "Ops (Total, All Procs)"

 set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

<span style="color: #000000;background-color: #ffdddd">-set title "I/O Pattern"

</span><span style="color: #000000;background-color: #ddffdd">+set title "POSIX I/O Pattern"

</span> plot  "pattern.dat" using 2:xtic(1) title "Total", \

       "" using 3 title "Sequential", \

       "" using 4 title "Consecutive"

</code></pre>

<br>

</li>

<li id='diff-159'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-159'>

<strong>

darshan-util/darshan-job-summary/share/pattern-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/pattern-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,28 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "pattern.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key out bottom horiz

-set key title ""

-set datafile separator ","

-

-set ylabel "Ops (Total, All Procs)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "I/O Pattern"

-plot  "pattern.dat" using 2:xtic(1) title "Total", \

-      "" using 3 title "Sequential", \

-      "" using 4 title "Consecutive"

</span></code></pre>

<br>

</li>

<li id='diff-160'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-160'>

<strong>

darshan-util/darshan-job-summary/share/summary.tex

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/summary.tex

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/summary.tex

</span><span style="color: #aaaaaa">@@ -1,11 +1,13 @@

</span> \documentclass[11pt,letterpaper,twocolumn]{article}

<span style="color: #000000;background-color: #ffdddd">-% \usepackage{html}

</span> \usepackage{charter}

 \usepackage{graphicx}

 \usepackage{fancyhdr}

 \usepackage{lastpage}

 \usepackage{subfigure}

 \usepackage{multirow}

<span style="color: #000000;background-color: #ddffdd">+\usepackage{threeparttable}

+\usepackage{color}

+

</span> %

 % GET THE MARGINS RIGHT, THE UGLY WAY

 %

<span style="color: #aaaaaa">@@ -23,7 +25,7 @@

</span> % BEGINNING OF DOCUMENT

 %

<span style="color: #000000;background-color: #ffdddd">-\input{title}

</span><span style="color: #000000;background-color: #ddffdd">+\input{title.tex}

</span> 

 \begin{document}

 \fontfamily{cmss}

<span style="color: #aaaaaa">@@ -31,35 +33,56 @@

</span> 

 \pagestyle{fancy}

<span style="color: #000000;background-color: #ddffdd">+\ifdefined\incompletelog

+\twocolumn[

+\vspace{3.5in}

+\center

+{\bf \textcolor{red}{WARNING}}: This Darshan log contains incomplete data

+which may skew results in this document.

+\endcenter

+]

+\newpage

+\fi

+

</span> \begin{figure*}[!h]

 \centering

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{job-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \input{job-table.tex}

</span> }\\

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \includegraphics[scale=0.8]{time-summary.pdf}

</span><span style="color: #000000;background-color: #ddffdd">+    \includegraphics[scale=0.75]{time-summary.pdf}

</span> }

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \includegraphics[scale=0.8]{counts.pdf}

</span><span style="color: #000000;background-color: #ddffdd">+    \includegraphics[scale=0.75]{op-counts.pdf}

</span> }

<span style="color: #000000;background-color: #ddffdd">+\end{figure*}

+

+\begin{figure*}[!h]

+\centering

</span> \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \includegraphics[scale=0.8]{hist.pdf}

</span><span style="color: #000000;background-color: #ddffdd">+    \includegraphics[scale=0.75]{posix-access-hist.pdf}

</span> }

<span style="color: #000000;background-color: #ddffdd">+\ifdefined\inclmpiio

</span> \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \includegraphics[scale=0.8]{pattern.pdf}

</span><span style="color: #000000;background-color: #ddffdd">+    \includegraphics[scale=0.75]{mpiio-access-hist.pdf}

</span> }

<span style="color: #000000;background-color: #ddffdd">+\fi

+\end{figure*}

+

+\begin{figure*}[!h]

+\centering

</span> \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{access-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \input{access-table.tex}

</span> }

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{file-count-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \input{file-count-table.tex}

</span> }

 \end{figure*}

<span style="color: #aaaaaa">@@ -79,11 +102,11 @@

</span> }

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{file-access-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \input{file-access-table.tex}

</span> }

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{fs-data-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \input{fs-data-table.tex}

</span> }

 \end{figure*}

<span style="color: #aaaaaa">@@ -91,29 +114,16 @@

</span> \centering

 \subfigure

 {

<span style="color: #000000;background-color: #ffdddd">-    \input{variance-table}

</span><span style="color: #000000;background-color: #ddffdd">+    \includegraphics[scale=0.65]{pattern.pdf}

+}

+\scriptsize

+\\ \textit{sequential}: An I/O op issued at an offset greater than where the previous I/O op ended.

+\\ \textit{consecutive}: An I/O op issued at the offset immediately following the end of the previous I/O op. \\

+\normalsize

+\subfigure

+{

+    \input{variance-table.tex}

</span> }

 \end{figure*}

<span style="color: #000000;background-color: #ffdddd">-

-

-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

-

-% at this point, things fall onto page 2

-%\begin{figure}[!h]

-%\includegraphics[scale=0.7]{types.pdf}

-%\end{figure}

-%\begin{figure}[!h]

-%\includegraphics[scale=0.7]{align.pdf}

-%\end{figure}

-%\begin{figure}[!h]

-%\includegraphics[scale=0.7]{iodist.pdf}

-%\end{figure}

-%

-%\begin{figure}[!h]

-%\centering

-%\input{stride-table}

-%\end{figure}

-

-

</span> \end{document}

</code></pre>

<br>

</li>

<li id='diff-161'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-161'>

<strong>

darshan-util/darshan-job-summary/share/time-summary-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/time-summary-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,30 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "time-summary.pdf"

-set size 0.8,1.0

-

-set style data histograms

-set style histogram rowstacked

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set key out bottom horiz

-set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Percentage of run time"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-set yrange [0:100]

-

-set title "Average I/O cost per process"

-

-set bmargin 7

-

-plot \

-    newhistogram "", "time-summary.dat" using 3:xtic(1) title "Read", "" using 4 title "Write", "" using 5 title "Metadata", "" using 2 title "Other (including application compute)"

</span></code></pre>

<br>

</li>

<li id='diff-162'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-162'>

<strong>

darshan-util/darshan-job-summary/share/title.tex

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-job-summary/share/title.tex

</span><span style="color: #aaaaaa">@@ -0,0 +1,16 @@

</span><span style="color: #000000;background-color: #ddffdd">+\renewcommand{\footrulewidth}{0.4pt}

+

+\rhead{\thepage\ of \pageref{LastPage}}

+\chead[

+\large{\titlecmd \space (\titlemon/\titlemday/\titleyear)}

+]

+{

+\large{\titlecmd \space (\titlemon/\titlemday/\titleyear)}

+}

+\cfoot[

+\scriptsize{\titlecmdline}

+]

+{

+\scriptsize{\titlecmdline}

+}

+

</span></code></pre>

<br>

</li>

<li id='diff-163'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-163'>

<strong>

darshan-util/darshan-job-summary/share/types-pdf.gplt

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/share/types-pdf.gplt

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,26 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#!/usr/bin/gnuplot -persist

-

-set terminal pdf enhanced fname "Helvetica" fsize 10

-set output "types.pdf"

-set size 0.8,1.0

-

-set style data histogram

-set style histogram cluster gap 1

-set style fill solid border -1

-set boxwidth 0.9

-set bmargin 5

-set border 3 front linetype -1 linewidth 1.000

-

-set xtics border in scale 1,0.5 nomirror rotate by -45 offset character 0, 0, 0

-set ytics border in scale 1,0.5 nomirror norotate  offset character 0, 0, 0

-set yrange [0:]

-

-set key title ""

-set datafile separator ","

-

-set ylabel "Count (Total, All Procs)"

-set ylabel offset character 2,0,0 font "" textcolor lt -1 rotate by 90

-

-set title "MPI Datatype Use"

-# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat

-plot  "types.dat" using 2:xtic(1) notitle;

</span></code></pre>

<br>

</li>

<li id='diff-164'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-164'>

<strong>

darshan-util/darshan-load-mysql.c

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-load-mysql.c

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,509 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#include <stdlib.h>

-#include <stdio.h>

-#include <string.h>

-#include <ftw.h>

-#include <libgen.h>

-#include <mysql.h>

-#include <regex.h>

-#include <sys/types.h>

-#include <getopt.h>

-#include <readline/readline.h>

-

-#include "darshan-logutils.h"

-

-#define MAXSQL (1024*1024)

-#define STOPWALK (1)

-#define CONTWALK (0)

-

-#define OPT_HOST (1)

-#define OPT_USER (2)

-#define OPT_PASS (3)

-#define OPT_DB   (4)

-#define OPT_PATH (5)

-

-const char *insert_job_fmt  = "insert into %s values('%d','%s','%s','%s',\

-'%d','%ld','%ld','%d')";

-const char *insert_mnt_fmt  = "insert into %s values('%d','%d','%" PRId64 "','%s','%s')";

-const char *insert_file_fmt = "insert into %s values('%d','%ld','%" PRId64 "','%d',\

-'%s',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\

-'%.16lf','%.16lf','%.16lf','%.16lf','%.16lf',\

-'%.16lf','%.16lf','%.16lf','%.16lf','%.16lf',\

-'%.16lf','%.16lf','%.16lf','%.16lf')";

-

-MYSQL *mysql = NULL;

-int debug = 0;

-

-int tree_walk (const char *fpath, const struct stat *sb, int typeflag)

-{

-    struct darshan_file file;

-    struct darshan_job  job;

-    darshan_fd          dfile = NULL;

-    int                 ret;

-    int                 rc;

-    int                 nofiles;

-    char                exe[4096];

-    char               *base;

-    char               *dash;

-    char               *username;

-    char               *jobid;

-    char               *sqlstmt = NULL;

-    int                 count;

-    int                 i;

-    int64_t            *devs;

-    char              **mnts;

-    char              **fstypes;

-    regex_t             regex;

-    regmatch_t          match[1];

-    char               *filepath = NULL;

-    int                 prev_rank;

-

-    rc      = CONTWALK;

-    count   = 0;

-

-    /* Only Process Files */

-    if (typeflag != FTW_F) return CONTWALK;

-

-    sqlstmt = malloc(MAXSQL);

-    if (!sqlstmt)

-    {

-        perror("malloc");

-        rc = STOPWALK;

-        goto exit;

-    }

-

-    filepath = strdup(fpath);

-    if (!filepath)

-    {

-        perror("strdup");

-        rc = STOPWALK;

-        goto exit;

-    }

-

-    /* Process Log Files */

-    dfile = darshan_log_open(fpath, "r");

-    if (dfile == NULL)

-    {

-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", fpath);

-        rc = CONTWALK;

-        goto exit;

-    }

-

-    ret = darshan_log_getjob(dfile, &job);

-    if (ret < 0)

-    {

-        perror("darshan_log_getjob");

-        fprintf(stderr, "%s\n", fpath);

-        rc = CONTWALK;

-        goto exit;

-    }

-

-    memset(exe, 0, sizeof(exe));

-

-    ret = darshan_log_getexe(dfile, exe, &nofiles);

-    if (ret < 0)

-    {

-        perror("darshan_log_getexe");

-        fprintf(stderr, "%s\n", fpath);

-        rc = CONTWALK;

-        goto exit;

-    }

-

-    base     = basename(filepath);

-    username = base;

-    dash     = index(base, '_');

-    *dash    = '\0';

-    jobid    = dash+1;

-

-    /*

-     * Find jobid for log file name

-     */

-    ret = regcomp(&regex, "_id[[:digit:]]+_", REG_EXTENDED);

-    if (ret != 0)

-    {

-        char buf[256];

-        regerror(ret, &regex, buf, sizeof(buf));

-        fprintf(stderr, "regcomp: %s\n", buf);

-        rc = STOPWALK;

-        goto exit;

-    }

-

-    ret = regexec(&regex, jobid, 1, match, 0);

-    if (ret != 0)

-    {

-        char buf[256];

-        regerror(ret, &regex, buf, sizeof(buf));

-        fprintf(stderr, "regexec: %s\n", buf);

-        rc = STOPWALK;

-        goto exit;

-    }

-

-    regfree(&regex);

-

-    dash   = jobid;

-    jobid += (match[0].rm_so + 3);

-    dash  += (match[0].rm_eo - 1);

-    *dash  = 0;

-

-    /*

-     * Insert Job Record

-     */

-    snprintf(sqlstmt, MAXSQL, insert_job_fmt, "darshan_job_intrepid",

-        atoi(jobid), username, job.version_string, exe, job.uid,

-        job.start_time, job.end_time, job.nprocs);

-

-    if (debug) printf("sql: %s\n", sqlstmt);

-    ret = mysql_query(mysql, sqlstmt);

-    if (ret)

-    {

-        fprintf(stderr, "log not processed: %s [mysql: %d (%s)] : \

-jobid=%d start_time=%ld\n",

-            fpath, mysql_errno(mysql), mysql_error(mysql),

-            atoi(jobid), job.start_time);

-        rc = CONTWALK;

-        goto exit;

-    }

-

-    /*

-     * Insert MountPoint Record (if present)

-     */

-    ret = darshan_log_getmounts(dfile,&devs,&mnts,&fstypes,&count,&nofiles);

-    if (ret < 0)

-    {

-        perror("darshan_log_getmounts");

-        fprintf(stderr, "%s\n", fpath);

-        rc = STOPWALK;

-        goto exit;

-    }

-

-    for (i=0; (i<count); i++)

-    {

-        snprintf(sqlstmt,MAXSQL,insert_mnt_fmt, "darshan_mountpoints_intrepid",

-            atoi(jobid), job.start_time, devs[i], mnts[i], fstypes[i]);

-

-        if (debug) printf("sql: %s\n", sqlstmt);

-        ret = mysql_query(mysql, sqlstmt);

-        if (ret)

-        {

-            fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),

-                mysql_error(mysql));

-            rc = STOPWALK;

-            goto exit;

-        }

-    }

-

-    /*

-     * Insert File Records (if present)

-     */

-    if (!nofiles)

-    {

-        while ((ret = darshan_log_getfile(dfile, &job, &file)) == 1)

-        {

-            /*

-             * Work around issue where bogus file data was in older logs.

-             * Bogus data was files taht were 'stat'd but not opened.

-             */

-            if (file.rank != -1 && file.rank < prev_rank)

-            {

-                continue;

-            }

-            if (file.rank != -1)

-                prev_rank = file.rank;

-

-            snprintf(sqlstmt, MAXSQL, insert_file_fmt, "darshan_file_intrepid",

-                atoi(jobid), job.start_time, file.hash, file.rank, file.name_suffix,

-                file.counters[CP_INDEP_OPENS],

-                file.counters[CP_COLL_OPENS],

-                file.counters[CP_INDEP_READS],

-                file.counters[CP_INDEP_WRITES],

-                file.counters[CP_COLL_READS],

-                file.counters[CP_COLL_WRITES],

-                file.counters[CP_SPLIT_READS],

-                file.counters[CP_SPLIT_WRITES],

-                file.counters[CP_NB_READS],

-                file.counters[CP_NB_WRITES],

-                file.counters[CP_SYNCS],

-                file.counters[CP_POSIX_READS],

-                file.counters[CP_POSIX_WRITES],

-                file.counters[CP_POSIX_OPENS],

-                file.counters[CP_POSIX_SEEKS],

-                file.counters[CP_POSIX_STATS],

-                file.counters[CP_POSIX_MMAPS],

-                file.counters[CP_POSIX_FREADS],

-                file.counters[CP_POSIX_FWRITES],

-                file.counters[CP_POSIX_FOPENS],

-                file.counters[CP_POSIX_FSEEKS],

-                file.counters[CP_POSIX_FSYNCS],

-                file.counters[CP_POSIX_FDSYNCS],

-                file.counters[CP_INDEP_NC_OPENS],

-                file.counters[CP_COLL_NC_OPENS],

-                file.counters[CP_HDF5_OPENS],

-                file.counters[CP_COMBINER_NAMED],

-                file.counters[CP_COMBINER_DUP],

-                file.counters[CP_COMBINER_CONTIGUOUS],

-                file.counters[CP_COMBINER_VECTOR],

-                file.counters[CP_COMBINER_HVECTOR_INTEGER],

-                file.counters[CP_COMBINER_HVECTOR],

-                file.counters[CP_COMBINER_INDEXED],

-                file.counters[CP_COMBINER_HINDEXED_INTEGER],

-                file.counters[CP_COMBINER_HINDEXED],

-                file.counters[CP_COMBINER_INDEXED_BLOCK],

-                file.counters[CP_COMBINER_STRUCT_INTEGER],

-                file.counters[CP_COMBINER_STRUCT],

-                file.counters[CP_COMBINER_SUBARRAY],

-                file.counters[CP_COMBINER_DARRAY],

-                file.counters[CP_COMBINER_F90_REAL],

-                file.counters[CP_COMBINER_F90_COMPLEX],

-                file.counters[CP_COMBINER_F90_INTEGER],

-                file.counters[CP_COMBINER_RESIZED],

-                file.counters[CP_HINTS],

-                file.counters[CP_VIEWS],

-                file.counters[CP_MODE],

-                file.counters[CP_BYTES_READ],

-                file.counters[CP_BYTES_WRITTEN],

-                file.counters[CP_MAX_BYTE_READ],

-                file.counters[CP_MAX_BYTE_WRITTEN],

-                file.counters[CP_CONSEC_READS],

-                file.counters[CP_CONSEC_WRITES],

-                file.counters[CP_SEQ_READS],

-                file.counters[CP_SEQ_WRITES],

-                file.counters[CP_RW_SWITCHES],

-                file.counters[CP_MEM_NOT_ALIGNED],

-                file.counters[CP_MEM_ALIGNMENT],

-                file.counters[CP_FILE_NOT_ALIGNED],

-                file.counters[CP_FILE_ALIGNMENT],

-                file.counters[CP_MAX_READ_TIME_SIZE],

-                file.counters[CP_MAX_WRITE_TIME_SIZE],

-                file.counters[CP_SIZE_READ_0_100],

-                file.counters[CP_SIZE_READ_100_1K],

-                file.counters[CP_SIZE_READ_1K_10K],

-                file.counters[CP_SIZE_READ_10K_100K],

-                file.counters[CP_SIZE_READ_100K_1M],

-                file.counters[CP_SIZE_READ_1M_4M],

-                file.counters[CP_SIZE_READ_4M_10M],

-                file.counters[CP_SIZE_READ_10M_100M],

-                file.counters[CP_SIZE_READ_100M_1G],

-                file.counters[CP_SIZE_READ_1G_PLUS],

-                file.counters[CP_SIZE_WRITE_0_100],

-                file.counters[CP_SIZE_WRITE_100_1K],

-                file.counters[CP_SIZE_WRITE_1K_10K],

-                file.counters[CP_SIZE_WRITE_10K_100K],

-                file.counters[CP_SIZE_WRITE_100K_1M],

-                file.counters[CP_SIZE_WRITE_1M_4M],

-                file.counters[CP_SIZE_WRITE_4M_10M],

-                file.counters[CP_SIZE_WRITE_10M_100M],

-                file.counters[CP_SIZE_WRITE_100M_1G],

-                file.counters[CP_SIZE_WRITE_1G_PLUS],

-                file.counters[CP_SIZE_READ_AGG_0_100],

-                file.counters[CP_SIZE_READ_AGG_100_1K],

-                file.counters[CP_SIZE_READ_AGG_1K_10K],

-                file.counters[CP_SIZE_READ_AGG_10K_100K],

-                file.counters[CP_SIZE_READ_AGG_100K_1M],

-                file.counters[CP_SIZE_READ_AGG_1M_4M],

-                file.counters[CP_SIZE_READ_AGG_4M_10M],

-                file.counters[CP_SIZE_READ_AGG_10M_100M],

-                file.counters[CP_SIZE_READ_AGG_100M_1G],

-                file.counters[CP_SIZE_READ_AGG_1G_PLUS],

-                file.counters[CP_SIZE_WRITE_AGG_0_100],

-                file.counters[CP_SIZE_WRITE_AGG_100_1K],

-                file.counters[CP_SIZE_WRITE_AGG_1K_10K],

-                file.counters[CP_SIZE_WRITE_AGG_10K_100K],

-                file.counters[CP_SIZE_WRITE_AGG_100K_1M],

-                file.counters[CP_SIZE_WRITE_AGG_1M_4M],

-                file.counters[CP_SIZE_WRITE_AGG_4M_10M],

-                file.counters[CP_SIZE_WRITE_AGG_10M_100M],

-                file.counters[CP_SIZE_WRITE_AGG_100M_1G],

-                file.counters[CP_SIZE_WRITE_AGG_1G_PLUS],

-                file.counters[CP_EXTENT_READ_0_100],

-                file.counters[CP_EXTENT_READ_100_1K],

-                file.counters[CP_EXTENT_READ_1K_10K], 

-                file.counters[CP_EXTENT_READ_10K_100K],

-                file.counters[CP_EXTENT_READ_100K_1M],

-                file.counters[CP_EXTENT_READ_1M_4M],

-                file.counters[CP_EXTENT_READ_4M_10M],

-                file.counters[CP_EXTENT_READ_10M_100M],

-                file.counters[CP_EXTENT_READ_100M_1G],

-                file.counters[CP_EXTENT_READ_1G_PLUS],

-                file.counters[CP_EXTENT_WRITE_0_100],

-                file.counters[CP_EXTENT_WRITE_100_1K],

-                file.counters[CP_EXTENT_WRITE_1K_10K],

-                file.counters[CP_EXTENT_WRITE_10K_100K],

-                file.counters[CP_EXTENT_WRITE_100K_1M],

-                file.counters[CP_EXTENT_WRITE_1M_4M],

-                file.counters[CP_EXTENT_WRITE_4M_10M],

-                file.counters[CP_EXTENT_WRITE_10M_100M],

-                file.counters[CP_EXTENT_WRITE_100M_1G],

-                file.counters[CP_EXTENT_WRITE_1G_PLUS],

-                file.counters[CP_STRIDE1_STRIDE],

-                file.counters[CP_STRIDE2_STRIDE],

-                file.counters[CP_STRIDE3_STRIDE],

-                file.counters[CP_STRIDE4_STRIDE],

-                file.counters[CP_STRIDE1_COUNT],

-                file.counters[CP_STRIDE2_COUNT],

-                file.counters[CP_STRIDE3_COUNT],

-                file.counters[CP_STRIDE4_COUNT],

-                file.counters[CP_ACCESS1_ACCESS],

-                file.counters[CP_ACCESS2_ACCESS],

-                file.counters[CP_ACCESS3_ACCESS],

-                file.counters[CP_ACCESS4_ACCESS],

-                file.counters[CP_ACCESS1_COUNT],

-                file.counters[CP_ACCESS2_COUNT],

-                file.counters[CP_ACCESS3_COUNT],

-                file.counters[CP_ACCESS4_COUNT],

-                file.counters[CP_DEVICE],

-                file.counters[CP_SIZE_AT_OPEN],

-                file.fcounters[CP_F_OPEN_TIMESTAMP],

-                file.fcounters[CP_F_READ_START_TIMESTAMP],

-                file.fcounters[CP_F_WRITE_START_TIMESTAMP],

-                file.fcounters[CP_F_CLOSE_TIMESTAMP],

-                file.fcounters[CP_F_READ_END_TIMESTAMP],

-                file.fcounters[CP_F_WRITE_END_TIMESTAMP],

-                file.fcounters[CP_F_POSIX_READ_TIME],

-                file.fcounters[CP_F_POSIX_WRITE_TIME],

-                file.fcounters[CP_F_POSIX_META_TIME],

-                file.fcounters[CP_F_MPI_META_TIME],

-                file.fcounters[CP_F_MPI_READ_TIME],

-                file.fcounters[CP_F_MPI_WRITE_TIME],

-                file.fcounters[CP_F_MAX_READ_TIME],

-                file.fcounters[CP_F_MAX_WRITE_TIME]);

-

-            if (debug) printf("sql: %s\n", sqlstmt);

-            ret = mysql_query(mysql, sqlstmt);

-            if (ret)

-            {

-                fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),

-                    mysql_error(mysql));

-                rc = STOPWALK;

-                goto exit;

-            }

-        }

-    }

-

-exit:

-    if (dfile) darshan_log_close(dfile);

-

-    if (count > 0)

-    {

-        for(i=0; i<count; i++)

-        {

-            if (mnts[i]) free(mnts[i]);

-            if (fstypes[i]) free(fstypes[i]);

-        }

-        if (devs) free(devs);

-        if (mnts) free(mnts);

-        if (fstypes) free(fstypes);

-    }

-

-    if (sqlstmt)

-    {

-        free(sqlstmt);

-    }

-

-    if (filepath)

-    {

-        free(filepath);

-    }

-    

-    return rc;

-}

-

-int main (int argc, char **argv)

-{

-    char base[256] = "";

-    char host[256] = "";

-    char user[256] = "";

-    char pass[256] = "";

-    char db[256]   = "";

-    int  ret = 0;

-

-    while(1)

-    {

-        static struct option options[] = {

-            {"host", 1, NULL, OPT_HOST},

-            {"user", 1, NULL, OPT_USER},

-            {"pass", 1, NULL, OPT_PASS},

-            {"db",   1, NULL, OPT_DB},

-            {"path", 1, NULL, OPT_PATH},

-            {NULL,   0, NULL, 0}

-        };

-        int o = getopt_long(argc, argv, "", options, NULL);

-

-        if (o == -1) break;

-

-        switch(o)

-        {

-        case OPT_HOST:

-            strncpy(host, optarg, sizeof(host));

-            break;

-        case OPT_USER:

-            strncpy(user, optarg, sizeof(user));

-            break;

-        case OPT_PASS:

-            strncpy(pass, optarg, sizeof(pass));

-            break;

-        case OPT_DB:

-            strncpy(db, optarg, sizeof(db));

-            break;

-        case OPT_PATH:

-            strncpy(base, optarg, sizeof(base));

-            break;

-        }

-    }

-

-    if (strcmp(pass, "") == 0)

-    {

-        char *line = readline(NULL);

-        if (line)

-        {

-            strncpy(pass, line, sizeof(pass));

-        }

-    }

-

-    mysql = mysql_init(NULL);

-    if (mysql == NULL)

-    {

-        fprintf(stderr, "mysql_init failed");

-        exit(-1);

-    }

-

-    mysql = mysql_real_connect(mysql, host, user, pass, db, 0, NULL, 0);

-    if (mysql == NULL)

-    {

-        fprintf(stderr, "mysql_real_connect");

-        exit(-1);

-    }

-

-    /* Turn off auto commits, hopefuly for performance sake */

-    (void) mysql_autocommit(mysql, 0);

-

-    ret = ftw(base, tree_walk, 512);

-    if(ret != 0)

-    {

-        fprintf(stderr, "Error: failed to walk path: %s\n", base);

-        return(-1);

-    }

-

-    ret = mysql_commit(mysql);

-    if (ret)

-    {

-        fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),

-            mysql_error(mysql));

-    }

-

-    mysql_close(mysql);

-

-    return 0;

-}

</span></code></pre>

<br>

</li>

<li id='diff-165'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-165'>

<strong>

darshan-util/darshan-log-params.c

</strong>

deleted

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-log-params.c

</span><span style="color: #000000;background-color: #ddffdd">+++ /dev/null

</span><span style="color: #aaaaaa">@@ -1,14 +0,0 @@

</span><span style="color: #000000;background-color: #ffdddd">-#include <stdio.h>

-#include "darshan-log-format.h"

-

-int main(int argc, char **argv) 

-{

-    printf("version: %s\n", CP_VERSION);

-    printf("CP_JOB_RECORD_SIZE: %d\n", CP_JOB_RECORD_SIZE);

-    printf("CP_EXE_LEN: %zu\n", CP_EXE_LEN);

-    printf("CP_FILE_RECORD_SIZE: %zu\n", CP_FILE_RECORD_SIZE);

-    printf("CP_NAME_SUFFIX_LEN: %d\n", CP_NAME_SUFFIX_LEN);

-    printf("CP_NUM_INDICES: %d\n", CP_NUM_INDICES);

-    printf("CP_F_NUM_INDICES: %d\n", CP_F_NUM_INDICES);

-    return(0);

-}

</span></code></pre>

<br>

</li>

<li id='diff-166'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-166'>

<strong>

darshan-util/darshan-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #define _GNU_SOURCE

<span style="color: #aaaaaa">@@ -14,513 +15,278 @@

</span> #include <sys/types.h>

 #include <sys/stat.h>

 #include <fcntl.h>

<span style="color: #000000;background-color: #ffdddd">-

-#include <zlib.h>

-#ifdef HAVE_LIBBZ2

-#include <bzlib.h>

-#endif

</span><span style="color: #000000;background-color: #ddffdd">+#include <errno.h>

</span> 

 #include "darshan-logutils.h"

<span style="color: #000000;background-color: #ffdddd">-struct darshan_fd_s

</span><span style="color: #000000;background-color: #ddffdd">+/* default input buffer size for decompression algorithm */

+#define DARSHAN_DEF_COMP_BUF_SZ (1024*1024) /* 1 MiB */

+

+/* special identifers for referring to header, job, and

+ * record map regions of the darshan log file

+ */

+#define DARSHAN_HEADER_REGION_ID    (-3)

+#define DARSHAN_JOB_REGION_ID       (-2)

+#define DARSHAN_REC_MAP_REGION_ID   (-1)

+

+struct darshan_dz_state

</span> {

<span style="color: #000000;background-color: #ffdddd">-    gzFile gzf;

-#ifdef HAVE_LIBBZ2

-    BZFILE* bzf;

-#endif

-    int64_t pos;

-    char mode[2];

-    int swap_flag;

-    char version[10];

-    int job_struct_size;

-    char* name;

-    int COMPAT_CP_EXE_LEN;

</span><span style="color: #000000;background-color: #ddffdd">+    /* (libz/bzip2) stream data structure for managing

+     * compression and decompression state */

+    void *strm;

+    /* buffer for staging compressed data to/from log file */

+    unsigned char *buf;

+    /* size of staging buffer */

+    int size;

+    /* for reading logs, flag indicating end of log file region */

+    int eor;

+    /* the region we last tried reading/writing */

+    int prev_reg_id;

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* isn't there a clever c way to avoid this? */

-char *darshan_names[] = {

-    "CP_INDEP_OPENS",

-    "CP_COLL_OPENS",               /* count of MPI collective opens */

-    "CP_INDEP_READS",              /* count of independent MPI reads */

-    "CP_INDEP_WRITES",             /* count of independent MPI writes */

-    "CP_COLL_READS",               /* count of collective MPI reads */

-    "CP_COLL_WRITES",              /* count of collective MPI writes */

-    "CP_SPLIT_READS",              /* count of split collective MPI reads */

-    "CP_SPLIT_WRITES",             /* count of split collective MPI writes */

-    "CP_NB_READS",                 /* count of nonblocking MPI reads */

-    "CP_NB_WRITES",                /* count of nonblocking MPI writes */

-    "CP_SYNCS",                    /* count of MPI_File_sync */

-    "CP_POSIX_READS",              /* count of posix reads */

-    "CP_POSIX_WRITES",             /* count of posix writes */

-    "CP_POSIX_OPENS",              /* count of posix opens */

-    "CP_POSIX_SEEKS",              /* count of posix seeks */

-    "CP_POSIX_STATS",              /* count of posix stat/lstat/fstats */

-    "CP_POSIX_MMAPS",              /* count of posix mmaps */

-    "CP_POSIX_FREADS",

-    "CP_POSIX_FWRITES",

-    "CP_POSIX_FOPENS",

-    "CP_POSIX_FSEEKS",

-    "CP_POSIX_FSYNCS",

-    "CP_POSIX_FDSYNCS",

-    "CP_INDEP_NC_OPENS",

-    "CP_COLL_NC_OPENS",

-    "CP_HDF5_OPENS",

-    "CP_COMBINER_NAMED",           /* count of each MPI datatype category */

-    "CP_COMBINER_DUP",

-    "CP_COMBINER_CONTIGUOUS",

-    "CP_COMBINER_VECTOR",

-    "CP_COMBINER_HVECTOR_INTEGER",

-    "CP_COMBINER_HVECTOR",

-    "CP_COMBINER_INDEXED",

-    "CP_COMBINER_HINDEXED_INTEGER",

-    "CP_COMBINER_HINDEXED",

-    "CP_COMBINER_INDEXED_BLOCK",

-    "CP_COMBINER_STRUCT_INTEGER",

-    "CP_COMBINER_STRUCT",

-    "CP_COMBINER_SUBARRAY",

-    "CP_COMBINER_DARRAY",

-    "CP_COMBINER_F90_REAL",

-    "CP_COMBINER_F90_COMPLEX",

-    "CP_COMBINER_F90_INTEGER",

-    "CP_COMBINER_RESIZED",

-    "CP_HINTS",                     /* count of MPI hints used */

-    "CP_VIEWS",                     /* count of MPI set view calls */

-    "CP_MODE",                      /* mode of file */

-    "CP_BYTES_READ",                /* total bytes read */

-    "CP_BYTES_WRITTEN",             /* total bytes written */

-    "CP_MAX_BYTE_READ",             /* highest offset byte read */

-    "CP_MAX_BYTE_WRITTEN",          /* highest offset byte written */

-    "CP_CONSEC_READS",              /* count of consecutive reads */

-    "CP_CONSEC_WRITES",             /* count of consecutive writes */

-    "CP_SEQ_READS",                 /* count of sequential reads */

-    "CP_SEQ_WRITES",                /* count of sequential writes */

-    "CP_RW_SWITCHES",

-    "CP_MEM_NOT_ALIGNED",           /* count of accesses not mem aligned */

-    "CP_MEM_ALIGNMENT",             /* mem alignment in bytes */

-    "CP_FILE_NOT_ALIGNED",          /* count of accesses not file aligned */

-    "CP_FILE_ALIGNMENT",            /* file alignment in bytes */

-    "CP_MAX_READ_TIME_SIZE",

-    "CP_MAX_WRITE_TIME_SIZE",

-    "CP_SIZE_READ_0_100",           /* count of posix read size ranges */

-    "CP_SIZE_READ_100_1K",

-    "CP_SIZE_READ_1K_10K",

-    "CP_SIZE_READ_10K_100K",

-    "CP_SIZE_READ_100K_1M",

-    "CP_SIZE_READ_1M_4M",

-    "CP_SIZE_READ_4M_10M",

-    "CP_SIZE_READ_10M_100M",

-    "CP_SIZE_READ_100M_1G",

-    "CP_SIZE_READ_1G_PLUS",

-    "CP_SIZE_WRITE_0_100",          /* count of posix write size ranges */

-    "CP_SIZE_WRITE_100_1K",

-    "CP_SIZE_WRITE_1K_10K",

-    "CP_SIZE_WRITE_10K_100K",

-    "CP_SIZE_WRITE_100K_1M",

-    "CP_SIZE_WRITE_1M_4M",

-    "CP_SIZE_WRITE_4M_10M",

-    "CP_SIZE_WRITE_10M_100M",

-    "CP_SIZE_WRITE_100M_1G",

-    "CP_SIZE_WRITE_1G_PLUS",

-    "CP_SIZE_READ_AGG_0_100",       /* count of MPI read size ranges */

-    "CP_SIZE_READ_AGG_100_1K",

-    "CP_SIZE_READ_AGG_1K_10K",

-    "CP_SIZE_READ_AGG_10K_100K",

-    "CP_SIZE_READ_AGG_100K_1M",

-    "CP_SIZE_READ_AGG_1M_4M",

-    "CP_SIZE_READ_AGG_4M_10M",

-    "CP_SIZE_READ_AGG_10M_100M",

-    "CP_SIZE_READ_AGG_100M_1G",

-    "CP_SIZE_READ_AGG_1G_PLUS",

-    "CP_SIZE_WRITE_AGG_0_100",      /* count of MPI write size ranges */

-    "CP_SIZE_WRITE_AGG_100_1K",

-    "CP_SIZE_WRITE_AGG_1K_10K",

-    "CP_SIZE_WRITE_AGG_10K_100K",

-    "CP_SIZE_WRITE_AGG_100K_1M",

-    "CP_SIZE_WRITE_AGG_1M_4M",

-    "CP_SIZE_WRITE_AGG_4M_10M",

-    "CP_SIZE_WRITE_AGG_10M_100M",

-    "CP_SIZE_WRITE_AGG_100M_1G",

-    "CP_SIZE_WRITE_AGG_1G_PLUS",

-    "CP_EXTENT_READ_0_100",          /* count of MPI read extent ranges */

-    "CP_EXTENT_READ_100_1K",

-    "CP_EXTENT_READ_1K_10K",

-    "CP_EXTENT_READ_10K_100K",

-    "CP_EXTENT_READ_100K_1M",

-    "CP_EXTENT_READ_1M_4M",

-    "CP_EXTENT_READ_4M_10M",

-    "CP_EXTENT_READ_10M_100M",

-    "CP_EXTENT_READ_100M_1G",

-    "CP_EXTENT_READ_1G_PLUS",

-    "CP_EXTENT_WRITE_0_100",         /* count of MPI write extent ranges */

-    "CP_EXTENT_WRITE_100_1K",

-    "CP_EXTENT_WRITE_1K_10K",

-    "CP_EXTENT_WRITE_10K_100K",

-    "CP_EXTENT_WRITE_100K_1M",

-    "CP_EXTENT_WRITE_1M_4M",

-    "CP_EXTENT_WRITE_4M_10M",

-    "CP_EXTENT_WRITE_10M_100M",

-    "CP_EXTENT_WRITE_100M_1G",

-    "CP_EXTENT_WRITE_1G_PLUS",

-    "CP_STRIDE1_STRIDE",             /* the four most frequently appearing strides */

-    "CP_STRIDE2_STRIDE",

-    "CP_STRIDE3_STRIDE",

-    "CP_STRIDE4_STRIDE",

-    "CP_STRIDE1_COUNT",              /* count of each of the most frequent strides */

-    "CP_STRIDE2_COUNT",

-    "CP_STRIDE3_COUNT",

-    "CP_STRIDE4_COUNT",

-    "CP_ACCESS1_ACCESS",

-    "CP_ACCESS2_ACCESS",

-    "CP_ACCESS3_ACCESS",

-    "CP_ACCESS4_ACCESS",

-    "CP_ACCESS1_COUNT",

-    "CP_ACCESS2_COUNT",

-    "CP_ACCESS3_COUNT",

-    "CP_ACCESS4_COUNT",

-    "CP_DEVICE",

-    "CP_SIZE_AT_OPEN",

-    "CP_FASTEST_RANK",

-    "CP_FASTEST_RANK_BYTES",

-    "CP_SLOWEST_RANK",

-    "CP_SLOWEST_RANK_BYTES",

-

-    "CP_NUM_INDICES"

</span><span style="color: #000000;background-color: #ddffdd">+/* internal fd data structure */

+struct darshan_fd_int_state

+{

+    /* posix file descriptor for the log file */

+    int fildes;

+    /* file pointer position */

+    int64_t pos;

+    /* flag indicating whether log file was created (and written) */

+    int creat_flag;

+    /* compression type used on log file (libz or bzip2) */

+    enum darshan_comp_type comp_type;

+    /* log file path name */

+    char logfile_path[PATH_MAX];

+    /* pointer to exe & mount data in darshan job data structure */

+    char *exe_mnt_data;

+    /* whether previous file operations have failed */

+    int err;

+

+    /* compression/decompression state */

+    struct darshan_dz_state dz;

</span> };

<span style="color: #000000;background-color: #ffdddd">-/* isn't there a clever c way to avoid this? */

-char *darshan_f_names[] = {

-    "CP_F_OPEN_TIMESTAMP",        /* timestamp of first open */

-    "CP_F_READ_START_TIMESTAMP",  /* timestamp of first read */

-    "CP_F_WRITE_START_TIMESTAMP", /* timestamp of first write */

-    "CP_F_CLOSE_TIMESTAMP",       /* timestamp of last close */

-    "CP_F_READ_END_TIMESTAMP",    /* timestamp of last read */

-    "CP_F_WRITE_END_TIMESTAMP",   /* timestamp of last write */

-    "CP_F_POSIX_READ_TIME",       /* cumulative posix read time */

-    "CP_F_POSIX_WRITE_TIME",      /* cumulative posix write time */

-    "CP_F_POSIX_META_TIME",       /* cumulative posix meta time */

-    "CP_F_MPI_META_TIME",         /* cumulative mpi-io metadata time */

-    "CP_F_MPI_READ_TIME",         /* cumulative mpi-io read time */

-    "CP_F_MPI_WRITE_TIME",        /* cumulative mpi-io write time */

-    "CP_F_MAX_READ_TIME",

-    "CP_F_MAX_WRITE_TIME",

-    "CP_F_FASTEST_RANK_TIME",

-    "CP_F_SLOWEST_RANK_TIME",

-    "CP_F_VARIANCE_RANK_TIME",

-    "CP_F_VARIANCE_RANK_BYTES",

-

-    "CP_F_NUM_INDICES"

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_getheader(darshan_fd fd);

+static int darshan_log_putheader(darshan_fd fd);

+static int darshan_log_seek(darshan_fd fd, off_t offset);

+static int darshan_log_read(darshan_fd fd, void *buf, int len);

+static int darshan_log_write(darshan_fd fd, void *buf, int len);

+static int darshan_log_dzinit(struct darshan_fd_int_state *state);

+static void darshan_log_dzdestroy(struct darshan_fd_int_state *state);

+static int darshan_log_dzread(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_dzwrite(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_libz_read(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_libz_write(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_libz_flush(darshan_fd fd, int region_id);

+#ifdef HAVE_LIBBZ2

+static int darshan_log_bzip2_read(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_bzip2_write(darshan_fd fd, int region_id, void *buf, int len);

+static int darshan_log_bzip2_flush(darshan_fd fd, int region_id);

+#endif

+static int darshan_log_dzload(darshan_fd fd, struct darshan_log_map map);

+static int darshan_log_dzunload(darshan_fd fd, struct darshan_log_map *map_p);

+

+/* each module's implementation of the darshan logutil functions */

+#define X(a, b, c, d) d,

+struct darshan_mod_logutil_funcs *mod_logutils[DARSHAN_MAX_MODS] =

+{

+    DARSHAN_MODULE_IDS

</span> };

<span style="color: #000000;background-color: #ddffdd">+#undef X

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* function pointers so that we can switch functions depending on what file

- * version is detected

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_open()

+ *

+ * open an existing darshan log file for reading only

+ *

+ * returns file descriptor on success, NULL on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int (*getjob_internal)(darshan_fd file, struct darshan_job *job);

-int (*getfile_internal)(darshan_fd fd, 

-    struct darshan_job *job, 

-    struct darshan_file *file);

-#define JOB_SIZE_124 28

-#define JOB_SIZE_200 56

-#define JOB_SIZE_201 120

-#define CP_JOB_RECORD_SIZE_200 1024

-#define CP_JOB_RECORD_SIZE_1x 1024

-

-/* internal routines for parsing different file versions */

-static int getjob_internal_204(darshan_fd file, struct darshan_job *job);

-static int getjob_internal_201(darshan_fd file, struct darshan_job *job);

-static int getjob_internal_200(darshan_fd file, struct darshan_job *job);

-static int getfile_internal_204(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file);

-static int getfile_internal_200(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file);

-static int getjob_internal_124(darshan_fd file, struct darshan_job *job);

-static int getfile_internal_124(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file);

-static int getfile_internal_122(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file);

-static int getfile_internal_121(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file);

-static int getfile_internal_1x(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file, int n_counters, int n_fcounters);

-static void shift_missing_1_24(struct darshan_file* file);

-static void shift_missing_1_22(struct darshan_file* file);

-static void shift_missing_1_21(struct darshan_file* file);

-

-static int darshan_log_seek(darshan_fd fd, int64_t offset);

-static int darshan_log_read(darshan_fd fd, void* buf, int len);

-static int darshan_log_write(darshan_fd fd, void* buf, int len);

-static const char* darshan_log_error(darshan_fd fd, int* errnum);

-

-/* a rather crude API for accessing raw binary darshan files */

-darshan_fd darshan_log_open(const char *name, const char* mode)

</span><span style="color: #000000;background-color: #ddffdd">+darshan_fd darshan_log_open(const char *name)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int test_fd;

-    uint8_t magic[2];

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_fd tmp_fd;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    int try_bz2 = 1;

-    int len = strlen(name);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* we only allows "w" or "r" modes, nothing fancy */

-    assert(strlen(mode) == 1);

-    assert(mode[0] == 'r' || mode[0] == 'w');

-

-    darshan_fd tmp_fd = malloc(sizeof(*tmp_fd));

</span><span style="color: #000000;background-color: #ddffdd">+    /* allocate a darshan file descriptor */

+    tmp_fd = malloc(sizeof(*tmp_fd));

</span>     if(!tmp_fd)

         return(NULL);

     memset(tmp_fd, 0, sizeof(*tmp_fd));

<span style="color: #000000;background-color: #ddffdd">+    tmp_fd->state = malloc(sizeof(struct darshan_fd_int_state));

+    if(!tmp_fd->state)

+    {

+        free(tmp_fd->state);

+        return(NULL);

+    }

+    memset(tmp_fd->state, 0, sizeof(struct darshan_fd_int_state));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_fd->mode[0] = mode[0];

-    tmp_fd->mode[1] = mode[1];

-    tmp_fd->name  = strdup(name);

-    if(!tmp_fd->name)

</span><span style="color: #000000;background-color: #ddffdd">+    /* open the log file in read mode */

+    tmp_fd->state->fildes = open(name, O_RDONLY);

+    if(tmp_fd->state->fildes < 0)

</span>     {

<span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to open darshan log file %s.\n", name);

+        free(tmp_fd->state);

</span>         free(tmp_fd);

         return(NULL);

     }

<span style="color: #000000;background-color: #ddffdd">+    strncpy(tmp_fd->state->logfile_path, name, PATH_MAX);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(mode, "r") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* read the header from the log file to init fd data structures */

+    ret = darshan_log_getheader(tmp_fd);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* Try to detect if existing file is a bzip2 file or not.  Both 

-         * libbz2 and libz will fall back to normal I/O (without compression) 

-         * automatically, so we need to do some detection manually up front 

-         * in order to get a chance to try both compression formats.

-         */

-        test_fd = open(name, O_RDONLY);

-        if(test_fd < 0)

-        {

-            perror("open");

-            free(tmp_fd->name);

-            free(tmp_fd);

-            return(NULL);

-        }

-        ret = read(test_fd, &magic, 2);

-        if(ret != 2)

-        {

-            fprintf(stderr, "Error: failed to read any data from %s.\n", 

-                name);

-            free(tmp_fd->name);

-            free(tmp_fd);

-            close(test_fd);

-            return(NULL);

-        }

-        /* header magic for bz2 */

-        if(magic[0] != 0x42 && magic[1] != 0x5A)

-        {

-            try_bz2 = 0;

-        }

-        close(test_fd);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to read darshan log file header.\n");

+        close(tmp_fd->state->fildes);

+        free(tmp_fd->state);

+        free(tmp_fd);

+        return(NULL);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(mode, "w") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* initialize compression data structures */

+    ret = darshan_log_dzinit(tmp_fd->state);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* TODO: is this the behavior that we want? */

-        /* if we are writing a new file, go by the file extension to tell

-         * whether to use bz2 or not?

-         */

-        if(len >= 3 && name[len-3] == 'b' && name[len-2] == 'z' && name[len-1] == '2')

-            try_bz2 = 1;

-        else

-            try_bz2 = 0;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to initialize decompression data structures.\n");

+        close(tmp_fd->state->fildes);

+        free(tmp_fd->state);

+        free(tmp_fd);

+        return(NULL);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(try_bz2)

</span><span style="color: #000000;background-color: #ddffdd">+    return(tmp_fd);

+}

+

+/* darshan_log_create()

+ *

+ * create a darshan log file for writing with the given compression method

+ *

+ * returns file descriptor on success, NULL on failure

+ */

+darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type,

+    int partial_flag)

+{

+    darshan_fd tmp_fd;

+    int ret;

+

+    /* allocate a darshan file descriptor */

+    tmp_fd = malloc(sizeof(*tmp_fd));

+    if(!tmp_fd)

+        return(NULL);

+    memset(tmp_fd, 0, sizeof(*tmp_fd));

+    tmp_fd->state = malloc(sizeof(struct darshan_fd_int_state));

+    if(!tmp_fd->state)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        tmp_fd->bzf = BZ2_bzopen(name, mode);

-        if(!tmp_fd->bzf)

-        {

-            free(tmp_fd->name);

-            free(tmp_fd);

-            return(NULL);

-        }

-        return(tmp_fd);

</span><span style="color: #000000;background-color: #ddffdd">+        free(tmp_fd);

+        return(NULL);

+    }

+    memset(tmp_fd->state, 0, sizeof(struct darshan_fd_int_state));

+

+    /* create the log for writing, making sure to not overwrite existing log */

+    tmp_fd->state->fildes = creat(name, 0400);

+    if(tmp_fd->state->fildes < 0)

+    {

+        fprintf(stderr, "Error: failed to open darshan log file %s.\n", name);

+        free(tmp_fd->state);

+        free(tmp_fd);

+        return(NULL);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-#else

-    if(try_bz2)

</span><span style="color: #000000;background-color: #ddffdd">+    tmp_fd->state->creat_flag = 1;

+    tmp_fd->state->comp_type = comp_type;

+    tmp_fd->partial_flag = partial_flag;

+    strncpy(tmp_fd->state->logfile_path, name, PATH_MAX);

+

+    /* position file pointer to prealloc space for the log file header

+     * NOTE: the header is written at close time, after all internal data

+     * structures have been properly set

+     */

+    ret = darshan_log_seek(tmp_fd, sizeof(struct darshan_header));

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: this Darshan build does not support bz2 files.\n");

-        fprintf(stderr, "Error: please install libbz2-dev and reconfigure.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: unable to seek in darshan log file.\n");

+        close(tmp_fd->state->fildes);

+        free(tmp_fd->state);

+        free(tmp_fd);

+        unlink(name);

</span>         return(NULL);

     }

<span style="color: #000000;background-color: #ffdddd">-#endif

</span> 

<span style="color: #000000;background-color: #ffdddd">-    tmp_fd->gzf = gzopen(name, mode);

-    if(!tmp_fd->gzf)

</span><span style="color: #000000;background-color: #ddffdd">+    /* initialize compression data structures */

+    ret = darshan_log_dzinit(tmp_fd->state);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        free(tmp_fd->name);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to initialize compression data structures.\n");

+        close(tmp_fd->state->fildes);

+        free(tmp_fd->state);

</span>         free(tmp_fd);

<span style="color: #000000;background-color: #ffdddd">-        tmp_fd = NULL;

</span><span style="color: #000000;background-color: #ddffdd">+        unlink(name);

+        return(NULL);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return tmp_fd;

</span><span style="color: #000000;background-color: #ddffdd">+

+    return(tmp_fd);

</span> }

 /* darshan_log_getjob()

  *

<span style="color: #000000;background-color: #ddffdd">+ * read job level metadata from the darshan log file

+ *

</span>  * returns 0 on success, -1 on failure

  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_getjob(darshan_fd file, struct darshan_job *job)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_getjob(darshan_fd fd, struct darshan_job *job)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    char job_buf[DARSHAN_JOB_RECORD_SIZE] = {0};

+    int job_buf_sz = DARSHAN_JOB_RECORD_SIZE;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    char buffer[DARSHAN_JOB_METADATA_LEN];

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(file, 0);

-    if(ret < 0)

-        return(ret);

-

-    /* read version number first so we know how to digest the rest of the

-     * file

-     */

-    ret = darshan_log_read(file, file->version, 10);

-    if(ret < 10)

-    {

-        fprintf(stderr, "Error: invalid log file (failed to read version).\n");

-        return(-1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

+    assert(fd->job_map.len > 0 && fd->job_map.off > 0);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(file->version, "2.06") == 0)

-    {

-        getjob_internal = getjob_internal_204;

-        getfile_internal = getfile_internal_204;

-        file->job_struct_size = sizeof(*job);

-        file->COMPAT_CP_EXE_LEN = CP_EXE_LEN;

-    }

-    else if(strcmp(file->version, "2.05") == 0)

-    {

-        getjob_internal = getjob_internal_204;

-        getfile_internal = getfile_internal_204;

-        file->job_struct_size = sizeof(*job);

-        file->COMPAT_CP_EXE_LEN = CP_EXE_LEN;

-    }

-    else if(strcmp(file->version, "2.04") == 0)

-    {

-        getjob_internal = getjob_internal_204;

-        getfile_internal = getfile_internal_204;

-        file->job_struct_size = sizeof(*job);

-        file->COMPAT_CP_EXE_LEN = CP_EXE_LEN;

-    }

-    else if(strcmp(file->version, "2.03") == 0)

-    {

-        getjob_internal = getjob_internal_201;

-        getfile_internal = getfile_internal_200;

-        file->job_struct_size = JOB_SIZE_201;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_200-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "2.02") == 0)

-    {

-        getjob_internal = getjob_internal_201;

-        getfile_internal = getfile_internal_200;

-        file->job_struct_size = JOB_SIZE_201;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_200-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "2.01") == 0)

-    {

-        getjob_internal = getjob_internal_201;

-        getfile_internal = getfile_internal_200;

-        file->job_struct_size = JOB_SIZE_201;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_200-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "2.00") == 0)

-    {

-        getjob_internal = getjob_internal_200;

-        getfile_internal = getfile_internal_200;

-        file->job_struct_size = JOB_SIZE_200;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_200-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "1.24") == 0)

-    {

-        getjob_internal = getjob_internal_124;

-        getfile_internal = getfile_internal_124;

-        file->job_struct_size = JOB_SIZE_124;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_1x-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "1.23") == 0)

-    {

-        /* same as 1.24, except that mnt points may be incorrect */

-        getjob_internal = getjob_internal_124;

-        getfile_internal = getfile_internal_124;

-        file->job_struct_size = JOB_SIZE_124;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_1x-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "1.22") == 0)

-    {

-        getjob_internal = getjob_internal_124;

-        getfile_internal = getfile_internal_122;

-        file->job_struct_size = JOB_SIZE_124;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_1x-file->job_struct_size-1;

-    }

-    else if(strcmp(file->version, "1.21") == 0)

-    {

-        getjob_internal = getjob_internal_124;

-        getfile_internal = getfile_internal_121;

-        file->job_struct_size = JOB_SIZE_124;

-        file->COMPAT_CP_EXE_LEN = CP_JOB_RECORD_SIZE_1x-file->job_struct_size-1;

-    }

-    else

</span><span style="color: #000000;background-color: #ddffdd">+    /* read the compressed job data from the log file */

+    ret = darshan_log_dzread(fd, DARSHAN_JOB_REGION_ID, job_buf, job_buf_sz);

+    if(ret <= sizeof(*job))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: incompatible darshan file.\n");

-        fprintf(stderr, "Error: expected version %s, but got %s\n", 

-                CP_VERSION, file->version);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to read darshan log file job data.\n");

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    ret = getjob_internal(file, job);

</span><span style="color: #000000;background-color: #ddffdd">+    memcpy(job, job_buf, sizeof(*job));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if (ret == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    if(fd->swap_flag)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_STRNDUP

-        char *metadata = strndup(job->metadata, sizeof(job->metadata));

-#else

-        char *metadata = strdup(job->metadata);

-#endif

-        char *kv;

-        char *key;

-        char *value;

-        char *save;

-

-        for(kv=strtok_r(metadata, "\n", &save);

-            kv != NULL;

-            kv=strtok_r(NULL, "\n", &save))

-        {

-            /* NOTE: we intentionally only split on the first = character.

-             * There may be additional = characters in the value portion

-             * (for example, when storing mpi-io hints).

-             */

-            strcpy(buffer, kv);

-            key = buffer;

-            value = index(buffer, '=');

-            if(!value)

-                continue;

-            /* convert = to a null terminator to split key and value */

-            value[0] = '\0';

-            value++;

-            if (strcmp(key, "prev_ver") == 0)

-            {

-                strncpy(job->version_string, value, sizeof(job->version_string));

-            }

-        }

-        free(metadata);

</span><span style="color: #000000;background-color: #ddffdd">+        /* swap bytes if necessary */

+        DARSHAN_BSWAP64(&job->uid);

+        DARSHAN_BSWAP64(&job->start_time);

+        DARSHAN_BSWAP64(&job->end_time);

+        DARSHAN_BSWAP64(&job->nprocs);

+        DARSHAN_BSWAP64(&job->jobid);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    /* save trailing exe & mount information, so it can be retrieved later */

+    if(!(state->exe_mnt_data))

+        state->exe_mnt_data = malloc(DARSHAN_EXE_LEN+1);

+    if(!(state->exe_mnt_data))

+        return(-1);

+    memcpy(state->exe_mnt_data, &job_buf[sizeof(*job)], DARSHAN_EXE_LEN+1);

+

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_putjob()

- * write job header in gzfile

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_putjob()

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * return 0 on success, -1 on failure.

</span><span style="color: #000000;background-color: #ddffdd">+ * write job level metadata to darshan log file

+ *

+ * returns 0 on success, -1 on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_putjob(darshan_fd file, struct darshan_job *job)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_putjob(darshan_fd fd, struct darshan_job *job)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     struct darshan_job job_copy;

<span style="color: #000000;background-color: #ffdddd">-    char    pv_str[64];

-    int     ret;

</span>     int len;

<span style="color: #000000;background-color: #ddffdd">+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(file, 0);

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

+

+    memset(&job_copy, 0, sizeof(*job));

+    memcpy(&job_copy, job, sizeof(*job));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memset(&job_copy, 0, sizeof(job_copy));

-    memcpy(&job_copy, job, sizeof(job_copy));

</span>     /* check for newline in existing metadata, add if needed */

     len = strlen(job_copy.metadata);

     if(len > 0 && len < DARSHAN_JOB_METADATA_LEN)

<span style="color: #aaaaaa">@@ -532,56 +298,74 @@ int darshan_log_putjob(darshan_fd file, struct darshan_job *job)

</span>         }

     }

<span style="color: #000000;background-color: #ffdddd">-    sprintf(pv_str, "prev_ver=%s\n", job->version_string);

-    sprintf(job_copy.version_string, "%s", CP_VERSION);

-    if(strlen(job_copy.metadata) + strlen(pv_str) < DARSHAN_JOB_METADATA_LEN)

-        strncat(job_copy.metadata, pv_str, strlen(pv_str));

-    else

-        sprintf(job_copy.metadata, "%s", pv_str);

-    job_copy.magic_nr = CP_MAGIC_NR;

-

-    ret = darshan_log_write(file, &job_copy, sizeof(job_copy));

-    if (ret != sizeof(job_copy))

</span><span style="color: #000000;background-color: #ddffdd">+    /* write the compressed job data to log file */

+    ret = darshan_log_dzwrite(fd, DARSHAN_JOB_REGION_ID, &job_copy, sizeof(*job));

+    if(ret != sizeof(*job))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: failed to write job header: %d\n", ret);

</span><span style="color: #000000;background-color: #ddffdd">+        state->err = -1;

+        fprintf(stderr, "Error: failed to write darshan log file job data.\n");

</span>         return(-1);

     }

     return(0);

 }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_log_getfile()

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_getexe()

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * return 1 if file record found, 0 on eof, and -1 on error

</span><span style="color: #000000;background-color: #ddffdd">+ * reads the application exe name from darshan log file

+ * 

+ * returns 0 on success, -1 on failure 

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_getfile(darshan_fd fd, struct darshan_job *job, struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_getexe(darshan_fd fd, char *buf)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    char *newline;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    ret = getfile_internal(fd, job, file);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    /* if the exe/mount data has not been saved yet, read in the job info */

+    if(!(state->exe_mnt_data))

+    {

+        struct darshan_job job;

+        ret = darshan_log_getjob(fd, &job);

+

+        if(ret < 0 || !(state->exe_mnt_data))

+            return(-1);

+    }

+

+    /* exe string is located before the first line break */

+    newline = strchr(state->exe_mnt_data, '\n');

+

+    /* copy over the exe string */

+    if(newline)

+        memcpy(buf, state->exe_mnt_data, (newline - state->exe_mnt_data));

+

+    return (0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_log_putfile()

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_putexe()

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * return 0 if file record written, -1 on error.

</span><span style="color: #000000;background-color: #ddffdd">+ * wrties the application exe name to darshan log file

+ * NOTE: this needs to be called immediately following put_job as it

+ * expects the file pointer to be positioned immediately following

+ * the darshan job information

+ *

+ * returns 0 on success, -1 on failure 

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_putfile(darshan_fd fd, struct darshan_job *job, struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_putexe(darshan_fd fd, char *buf)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int     ret;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    int len = strlen(buf);

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->pos < CP_JOB_RECORD_SIZE)

-    {

-        ret = darshan_log_seek(fd, CP_JOB_RECORD_SIZE);

-        if(ret < 0)

-            return(ret);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    assert(fd->state);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_write(fd, file, sizeof(*file));

-    if (ret != sizeof(*file))

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_dzwrite(fd, DARSHAN_JOB_REGION_ID, buf, len);

+    if(ret != len)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: writing file record failed: %d\n", ret);

</span><span style="color: #000000;background-color: #ddffdd">+        state->err = -1;

+        fprintf(stderr, "Error: failed to write exe string to darshan log file.\n");

</span>         return(-1);

     }

<span style="color: #aaaaaa">@@ -590,32 +374,35 @@ int darshan_log_putfile(darshan_fd fd, struct darshan_job *job, struct darshan_f

</span> 

 /* darshan_log_getmounts()

  * 

<span style="color: #000000;background-color: #ffdddd">- * retrieves mount table information from the log.  Note that devs, mnt_pts,

- * and fs_types are arrays that will be allocated by the function and must

- * be freed by the caller.  count will indicate the size of the arrays

</span><span style="color: #000000;background-color: #ddffdd">+ * retrieves mount table information from the log. Note that mnt_pts and

+ * fs_types are arrays that will be allocated by the function and must be

+ * freed by the caller. count will indicate the size of the arrays

+ *

+ * returns 0 on success, -1 on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_getmounts(darshan_fd fd, int64_t** devs, char*** mnt_pts, char***

-    fs_types, int* count)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,

+    char*** fs_types, int* count)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

-    char* pos;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    char *pos;

</span>     int array_index = 0;

<span style="color: #000000;background-color: #ffdddd">-    char buf[fd->COMPAT_CP_EXE_LEN+1];

</span><span style="color: #000000;background-color: #ddffdd">+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(fd, fd->job_struct_size);

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, buf, (fd->COMPAT_CP_EXE_LEN + 1));

-    if (ret < (fd->COMPAT_CP_EXE_LEN + 1))

</span><span style="color: #000000;background-color: #ddffdd">+    /* if the exe/mount data has not been saved yet, read in the job info */

+    if(!(state->exe_mnt_data))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        perror("darshan_log_read");

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        struct darshan_job job;

+        ret = darshan_log_getjob(fd, &job);

+

+        if(ret < 0 || !(state->exe_mnt_data))

+            return(-1);

</span>     }

     /* count entries */

     *count = 0;

<span style="color: #000000;background-color: #ffdddd">-    pos = buf;

</span><span style="color: #000000;background-color: #ddffdd">+    pos = state->exe_mnt_data;

</span>     while((pos = strchr(pos, '\n')) != NULL)

     {

         pos++;

<span style="color: #aaaaaa">@@ -629,30 +416,27 @@ int darshan_log_getmounts(darshan_fd fd, int64_t** devs, char*** mnt_pts, char**

</span>     }

     /* allocate output arrays */

<span style="color: #000000;background-color: #ffdddd">-    *devs = malloc((*count)*sizeof(int64_t));

-    assert(*devs);

</span>     *mnt_pts = malloc((*count)*sizeof(char*));

     assert(*mnt_pts);

     *fs_types = malloc((*count)*sizeof(char*));

     assert(*fs_types);

<span style="color: #000000;background-color: #ffdddd">-    

</span><span style="color: #000000;background-color: #ddffdd">+

</span>     /* work backwards through the table and parse each line (except for

      * first, which holds command line information)

      */

<span style="color: #000000;background-color: #ffdddd">-    while((pos = strrchr(buf, '\n')) != NULL)

</span><span style="color: #000000;background-color: #ddffdd">+    while((pos = strrchr(state->exe_mnt_data, '\n')) != NULL)

</span>     {

         /* overestimate string lengths */

<span style="color: #000000;background-color: #ffdddd">-        (*mnt_pts)[array_index] = malloc(fd->COMPAT_CP_EXE_LEN);

</span><span style="color: #000000;background-color: #ddffdd">+        (*mnt_pts)[array_index] = malloc(DARSHAN_EXE_LEN);

</span>         assert((*mnt_pts)[array_index]);

<span style="color: #000000;background-color: #ffdddd">-        (*fs_types)[array_index] = malloc(fd->COMPAT_CP_EXE_LEN);

</span><span style="color: #000000;background-color: #ddffdd">+        (*fs_types)[array_index] = malloc(DARSHAN_EXE_LEN);

</span>         assert((*fs_types)[array_index]);

<span style="color: #000000;background-color: #ffdddd">-        

-        ret = sscanf(++pos, "%" PRId64 "\t%s\t%s", &(*devs)[array_index],

-            (*fs_types)[array_index], (*mnt_pts)[array_index]);

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if(ret != 3)

</span><span style="color: #000000;background-color: #ddffdd">+        ret = sscanf(++pos, "%s\t%s", (*fs_types)[array_index],

+            (*mnt_pts)[array_index]);

+        if(ret != 2)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fprintf(stderr, "Error: poorly formatted mount table in log file.\n");

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Error: poorly formatted mount table in darshan log file.\n");

</span>             return(-1);

         }

         pos--;

<span style="color: #aaaaaa">@@ -660,1141 +444,1210 @@ int darshan_log_getmounts(darshan_fd fd, int64_t** devs, char*** mnt_pts, char**

</span>         array_index++;

     }

<span style="color: #000000;background-color: #ffdddd">-    return (0);

</span><span style="color: #000000;background-color: #ddffdd">+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_log_putmounts

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_putmounts()

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * encode mount information back into mtab format.

</span><span style="color: #000000;background-color: #ddffdd">+ * writes mount information to the darshan log file

+ * NOTE: this function call should follow immediately after the call

+ * to darshan_log_putexe(), as it assumes the darshan log file pointer

+ * is pointing to the offset immediately following the exe string

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * returns 0 on success, -1 on failure.

</span><span style="color: #000000;background-color: #ddffdd">+ * returns 0 on success, -1 on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_putmounts(darshan_fd fd, int64_t* devs, char** mnt_pts, char** fs_types, int count)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_putmounts(darshan_fd fd, char** mnt_pts, char** fs_types, int count)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int     ret;

-    char    line[1024];

-    int     i;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    int i;

+    char line[1024];

+    char mnt_dat[DARSHAN_EXE_LEN] = {0};

+    int mnt_dat_sz = 0;

+    char *tmp;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ddffdd">+    assert(state);

+

+    /* write each mount entry to file */

+    tmp = mnt_dat;

</span>     for(i=count-1; i>=0; i--)

     {

<span style="color: #000000;background-color: #ffdddd">-        sprintf(line, "\n%" PRId64 "\t%s\t%s",

-                devs[i], fs_types[i], mnt_pts[i]);

-        ret = darshan_log_write(fd, line, strlen(line));

-        if (ret != strlen(line))

-        {

-            fprintf(stderr, "Error: failed to write mount entry: %d\n", ret);

-            return(-1);

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        sprintf(line, "\n%s\t%s", fs_types[i], mnt_pts[i]);

+

+        memcpy(tmp, line, strlen(line));

+        tmp += strlen(line);

+        mnt_dat_sz += strlen(line);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* seek ahead to end of exe region, will be zero filled */

-    ret = darshan_log_seek(fd, CP_JOB_RECORD_SIZE);

-    if (ret)

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_dzwrite(fd, DARSHAN_JOB_REGION_ID, mnt_dat, mnt_dat_sz);

+    if (ret != mnt_dat_sz)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: forward seek failed: %d\n", CP_JOB_RECORD_SIZE);

</span><span style="color: #000000;background-color: #ddffdd">+        state->err = -1;

+        fprintf(stderr, "Error: failed to write darshan log mount data.\n");

+        return(-1);

</span>     }

     return(0);

 }

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_getexe(darshan_fd fd, char *buf)

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_gethash()

+ *

+ * read the hash of records from the darshan log file

+ *

+ * returns 0 on success, -1 on failure

+ */

+int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int ret;

-    char* newline;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    char *hash_buf;

+    int hash_buf_sz;

+    char *buf_ptr;

+    darshan_record_id *rec_id_ptr;

+    uint32_t *path_len_ptr, tmp_path_len;

+    char *path_ptr;

+    struct darshan_record_ref *ref;

+    int read;

+    int read_req_sz;

+    int buf_remaining = 0;

+

+    assert(state);

+

+    /* just return if there is no record mapping data */

+    if(fd->rec_map.len == 0)

+    {

+        *hash = NULL;

+        return(0);

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(fd, fd->job_struct_size);

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    /* default to hash buffer twice as big as default compression buf */

+    hash_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ * 2);

+    if(!hash_buf)

+        return(-1);

+    memset(hash_buf, 0, DARSHAN_DEF_COMP_BUF_SZ * 2);

+    hash_buf_sz = DARSHAN_DEF_COMP_BUF_SZ * 2;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, buf, (fd->COMPAT_CP_EXE_LEN + 1));

-    if (ret < (fd->COMPAT_CP_EXE_LEN + 1))

</span><span style="color: #000000;background-color: #ddffdd">+    do

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        perror("darshan_log_read");

-        return(-1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        /* read chunks of the darshan record id -> file name mapping from log file,

+         * constructing a hash table in the process

+         */

+        read_req_sz = hash_buf_sz - buf_remaining;

+        read = darshan_log_dzread(fd, DARSHAN_REC_MAP_REGION_ID,

+            hash_buf + buf_remaining, read_req_sz);

+        if(read < 0)

+        {

+            fprintf(stderr, "Error: failed to read record hash from darshan log file.\n");

+            free(hash_buf);

+            return(-1);

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* this call is only supposed to return the exe string, but starting in

-     * log format 1.23 there could be a table of mount entry information

-     * after the exe.  Look for newline character and truncate there.

-     */

-    newline = strchr(buf, '\n');

-    if(newline)

-        *newline = '\0';

</span><span style="color: #000000;background-color: #ddffdd">+        /* work through the hash buffer -- deserialize the mapping data and

+         * add to the output hash table

+         * NOTE: these mapping pairs are variable in length, so we have to be able

+         * to handle incomplete mappings temporarily here

+         */

+        buf_ptr = hash_buf;

+        buf_remaining += read;

+        while(buf_remaining > (sizeof(darshan_record_id) + sizeof(uint32_t)))

+        {

+            /* see if we have enough buf space to read in the next full record */

+            tmp_path_len = *(uint32_t *)(buf_ptr + sizeof(darshan_record_id));

+            if(fd->swap_flag)

+                DARSHAN_BSWAP32(&tmp_path_len);

+

+            /* we need to read more before we continue deserializing */

+            if(buf_remaining <

+                (sizeof(darshan_record_id) + sizeof(uint32_t) + tmp_path_len))

+                break;

+

+            /* get pointers for each field of this darshan record */

+            /* NOTE: darshan record hash serialization method: 

+             *          ... darshan_record_id | (uint32_t) path_len | path ...

+             */

+            rec_id_ptr = (darshan_record_id *)buf_ptr;

+            buf_ptr += sizeof(darshan_record_id);

+            path_len_ptr = (uint32_t *)buf_ptr;

+            buf_ptr += sizeof(uint32_t);

+            path_ptr = (char *)buf_ptr;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return (0);

-}

</span><span style="color: #000000;background-color: #ddffdd">+            if(fd->swap_flag)

+            {

+                /* we need to sort out endianness issues before deserializing */

+                DARSHAN_BSWAP64(rec_id_ptr);

+                DARSHAN_BSWAP32(path_len_ptr);

+            }

</span> 

<span style="color: #000000;background-color: #ffdddd">-/* darshan_log_putexe()

- *

- * Write the exe string to the log.

- *

- * return 0 on success, -1 on failure.

- */

-int darshan_log_putexe(darshan_fd fd, char *buf)

-{

-    int     ret;

-    int     len;

</span><span style="color: #000000;background-color: #ddffdd">+            HASH_FIND(hlink, *hash, rec_id_ptr, sizeof(darshan_record_id), ref);

+            if(!ref)

+            {

+                ref = malloc(sizeof(*ref));

+                if(!ref)

+                {

+                    free(hash_buf);

+                    return(-1);

+                }

+                ref->rec.name = malloc(*path_len_ptr + 1);

+                if(!ref->rec.name)

+                {

+                    free(ref);

+                    free(hash_buf);

+                    return(-1);

+                }

+

+                /* set the fields for this record */

+                ref->rec.id = *rec_id_ptr;

+                memcpy(ref->rec.name, path_ptr, *path_len_ptr);

+                ref->rec.name[*path_len_ptr] = '\0';

+

+                /* add this record to the hash */

+                HASH_ADD(hlink, *hash, rec.id, sizeof(darshan_record_id), ref);

+            }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(fd, sizeof(struct darshan_job));

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+            buf_ptr += *path_len_ptr;

+            buf_remaining -=

+                (sizeof(darshan_record_id) + sizeof(uint32_t) + *path_len_ptr);

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    len = strlen(buf);

</span><span style="color: #000000;background-color: #ddffdd">+        /* copy any leftover data to beginning of buffer to parse next */

+        memcpy(hash_buf, buf_ptr, buf_remaining);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_write(fd, buf, len);

-    if (ret != len)

-    {

-        fprintf(stderr, "Error: failed to write exe info: %d\n", ret);

-        ret = -1;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        /* we keep reading until we get a short read informing us we have

+         * read all of the record hash

+         */

+    } while(read == read_req_sz);

+    assert(buf_remaining == 0);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    free(hash_buf);

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-void darshan_log_close(darshan_fd file)

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_puthash()

+ *

+ * writes the hash table of records to the darshan log file

+ * NOTE: this function call should follow immediately after the call

+ * to darshan_log_putmounts(), as it assumes the darshan log file pointer

+ * is pointing to the offset immediately following the mount information

+ *

+ * returns 0 on success, -1 on failure

+ */

+int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash)

</span> {

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(file->bzf)

-        BZ2_bzclose(file->bzf);

-#endif

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    char *hash_buf;

+    int hash_buf_sz;

+    struct darshan_record_ref *ref, *tmp;

+    char *buf_ptr;

+    int path_len;

+    int wrote;

+

+    assert(state);

+

+    /* allocate memory for largest possible hash record */

+    hash_buf_sz = sizeof(darshan_record_id) + sizeof(uint32_t) + PATH_MAX;

+    hash_buf = malloc(hash_buf_sz);

+    if(!hash_buf)

+        return(-1);

+    memset(hash_buf, 0, hash_buf_sz);

+

+    /* individually serialize each hash record and write to log file */

+    HASH_ITER(hlink, hash, ref, tmp)

+    {

+        buf_ptr = hash_buf;

+        path_len = strlen(ref->rec.name);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(file->gzf)

-        gzclose(file->gzf);

</span><span style="color: #000000;background-color: #ddffdd">+        /* the hash buffer has space to serialize this record

+         * NOTE: darshan record hash serialization method: 

+         *          ... darshan_record_id | (uint32_t) path_len | path ...

+         */

+        *((darshan_record_id *)buf_ptr) = ref->rec.id;

+        buf_ptr += sizeof(darshan_record_id);

+        *((uint32_t *)buf_ptr) = path_len;

+        buf_ptr += sizeof(uint32_t);

+        memcpy(buf_ptr, ref->rec.name, path_len);

+        buf_ptr += path_len;

+

+        /* write this hash entry to log file */

+        wrote = darshan_log_dzwrite(fd, DARSHAN_REC_MAP_REGION_ID,

+            hash_buf, (buf_ptr - hash_buf));

+        if(wrote != (buf_ptr - hash_buf))

+        {

+            state->err = -1;

+            fprintf(stderr, "Error: failed to write record hash to darshan log file.\n");

+            free(hash_buf);

+            return(-1);

+        }

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    free(file->name);

-    free(file);

</span><span style="color: #000000;background-color: #ddffdd">+    free(hash_buf);

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* darshan_log_print_version_warnings()

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_getmod()

+ *

+ * get a chunk of module data from the darshan log file

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * Print summary of any problems with the detected log format

</span><span style="color: #000000;background-color: #ddffdd">+ * returns number of bytes read on success, -1 on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-void darshan_log_print_version_warnings(struct darshan_job *job)

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,

+    void *mod_buf, int mod_buf_sz)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.06") == 0)

-    {

-        /* current version */

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.05") == 0)

-    {

-        printf("# WARNING: version 2.05 log format has the following limitations:\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.04") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    if(mod_id < 0 || mod_id >= DARSHAN_MAX_MODS)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("# WARNING: version 2.04 log format has the following limitations:\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: invalid Darshan module id.\n");

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.03") == 0)

-    {

-        /* no meaningful change to interpretation of log file, 2.03 just

-         * increased the header space available for annotations.

-         */

-        printf("# WARNING: version 2.03 log format has the following limitations:\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    if(fd->mod_map[mod_id].len == 0)

+        return(0); /* no data corresponding to this mod_id */

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.02") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* read this module's data from the log file */

+    ret = darshan_log_dzread(fd, mod_id, mod_buf, mod_buf_sz);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("# WARNING: version 2.01 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr,

+            "Error: failed to read module %s data from darshan log file.\n",

+            darshan_module_names[mod_id]);

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.01") == 0)

-    {

-        printf("# WARNING: version 2.01 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "2.00") == 0)

-    {

-        printf("# WARNING: version 2.00 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

- 

-    if(strcmp(job->version_string, "1.24") == 0)

-    {

-        printf("# WARNING: version 1.24 log format does not support the following parameters:\n");

-        printf("#   CP_FASTEST_RANK\n");

-        printf("#   CP_FASTEST_RANK_BYTES\n");

-        printf("#   CP_SLOWEST_RANK\n");

-        printf("#   CP_SLOWEST_RANK_BYTES\n");

-        printf("#   CP_F_FASTEST_RANK_TIME\n");

-        printf("#   CP_F_SLOWEST_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_BYTES\n");

-        printf("# WARNING: version 1.24 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - does not store the job id in the file.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

-    

-    if(strcmp(job->version_string, "1.23") == 0)

-    {

-        printf("# WARNING: version 1.23 log format does not support the following parameters:\n");

-        printf("#   CP_FASTEST_RANK\n");

-        printf("#   CP_FASTEST_RANK_BYTES\n");

-        printf("#   CP_SLOWEST_RANK\n");

-        printf("#   CP_SLOWEST_RANK_BYTES\n");

-        printf("#   CP_F_FASTEST_RANK_TIME\n");

-        printf("#   CP_F_SLOWEST_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_BYTES\n");

-        printf("# WARNING: version 1.23 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - may have incorrect mount point mappings for files with rank > 0.\n");

-        printf("# - does not store the job id in the file.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_putmod()

+ *

+ * write a chunk of module data to the darshan log file

+ * NOTE: this function call should be called directly after the

+ * put_hash() function, as it expects the file pointer to be

+ * positioned directly past the record hash location. Also,

+ * for a set of modules with data to write to file, this function

+ * should be called in order of increasing module identifiers,

+ * as the darshan log file format expects this ordering.

+ *

+ * returns number of bytes written on success, -1 on failure

+ */

+int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,

+    void *mod_buf, int mod_buf_sz, int ver)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

+

+    assert(state);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "1.22") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    if(mod_id < 0 || mod_id >= DARSHAN_MAX_MODS)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("# WARNING: version 1.22 log format does not support the following parameters:\n");

-        printf("#   CP_DEVICE\n");

-        printf("#   CP_SIZE_AT_OPEN\n");

-        printf("#   CP_FASTEST_RANK\n");

-        printf("#   CP_FASTEST_RANK_BYTES\n");

-        printf("#   CP_SLOWEST_RANK\n");

-        printf("#   CP_SLOWEST_RANK_BYTES\n");

-        printf("#   CP_F_FASTEST_RANK_TIME\n");

-        printf("#   CP_F_SLOWEST_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_BYTES\n");

-        printf("# WARNING: version 1.22 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - does not record mounted file systems, mount points, or fs types.\n");

-        printf("# - attributes syncs to cumulative metadata time, rather than cumulative write time.\n");

-        printf("# - does not store the job id in the file.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        state->err = -1;

+        fprintf(stderr, "Error: invalid Darshan module id.\n");

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(strcmp(job->version_string, "1.21") == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* write the module chunk to the log file */

+    ret = darshan_log_dzwrite(fd, mod_id, mod_buf, mod_buf_sz);

+    if(ret != mod_buf_sz)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("# WARNING: version 1.21 log format does not support the following parameters:\n");

-        printf("#   CP_INDEP_NC_OPENS\n");

-        printf("#   CP_COLL_NC_OPENS\n");

-        printf("#   CP_HDF5_OPENS\n");

-        printf("#   CP_MAX_READ_TIME_SIZE\n");

-        printf("#   CP_MAX_WRITE_TIME_SIZE\n");

-        printf("#   CP_DEVICE\n");

-        printf("#   CP_SIZE_AT_OPEN\n");

-        printf("#   CP_F_MAX_READ_TIME\n");

-        printf("#   CP_F_MAX_WRITE_TIME\n");

-        printf("#   CP_FASTEST_RANK\n");

-        printf("#   CP_FASTEST_RANK_BYTES\n");

-        printf("#   CP_SLOWEST_RANK\n");

-        printf("#   CP_SLOWEST_RANK_BYTES\n");

-        printf("#   CP_F_FASTEST_RANK_TIME\n");

-        printf("#   CP_F_SLOWEST_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_TIME\n");

-        printf("#   CP_F_VARIANCE_RANK_BYTES\n");

-        printf("# WARNING: version 1.21 log format has the following limitations:\n");

-        printf("# - *_TIMESTAMP fields are not normalized relative to MPI_Init() time.\n");

-        printf("# - does not record mounted file systems, mount points, or fs types.\n");

-        printf("# - attributes syncs to cumulative metadata time, rather than cumulative write time.\n");

-        printf("# - does not store the job id in the file.\n");

-        printf("# - inaccurate statistics in some multi-threaded cases.\n");

-        printf("# - CP_F_SLOWEST_RANK_TIME and CP_F_FASTEST_RANK_TIME only report elapsed time at the POSIX level.\n");

-        printf("# - CP_F_OPEN_TIMESTAMP marks when the first open completed rather than when the first open started.\n");

-        return;

</span><span style="color: #000000;background-color: #ddffdd">+        state->err = -1;

+        fprintf(stderr,

+            "Error: failed to write module %s data to darshan log file.\n",

+            darshan_module_names[mod_id]);

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    fprintf(stderr, "Error: version %s not supported by parser.\n",

-        job->version_string);

-    return;

</span><span style="color: #000000;background-color: #ddffdd">+    /* set the version number for this module's data */

+    fd->mod_ver[mod_id] = ver;

+

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* shift_missing_1_21()

</span><span style="color: #000000;background-color: #ddffdd">+/* darshan_log_close()

+ *

+ * close an open darshan file descriptor, freeing any resources

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * translates indices to account for counters that weren't present in log

- * format 1.21

- */

-/*******************************

- * version 1.21 to 2.00 differences 

- * - added:

- *   - CP_INDEP_NC_OPENS

- *   - CP_COLL_NC_OPENS

- *   - CP_HDF5_OPENS

- *   - CP_MAX_READ_TIME_SIZE

- *   - CP_MAX_WRITE_TIME_SIZE

- *   - CP_DEVICE

- *   - CP_SIZE_AT_OPEN

- *   - CP_FASTEST_RANK

- *   - CP_FASTEST_RANK_BYTES

- *   - CP_SLOWEST_RANK

- *   - CP_SLOWEST_RANK_BYTES

- *   - CP_F_MAX_READ_TIME

- *   - CP_F_MAX_WRITE_TIME

- *   - CP_F_FASTEST_RANK_TIME

- *   - CP_F_SLOWEST_RANK_TIME

- *   - CP_F_VARIANCE_RANK_TIME

- *   - CP_F_VARIANCE_RANK_BYTES

- * - changed params:

- *   - CP_FILE_RECORD_SIZE: 1184 to 1328

- *   - CP_NUM_INDICES: 133 to 144

- *   - CP_F_NUM_INDICES: 12 to 18

</span>  */

<span style="color: #000000;background-color: #ffdddd">-static void shift_missing_1_21(struct darshan_file* file)

</span><span style="color: #000000;background-color: #ddffdd">+void darshan_log_close(darshan_fd fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int c_index = 0;

-    int missing_counters[] = {

-        CP_INDEP_NC_OPENS,

-        CP_COLL_NC_OPENS,

-        CP_HDF5_OPENS,

-        CP_MAX_READ_TIME_SIZE,

-        CP_MAX_WRITE_TIME_SIZE,

-        CP_DEVICE,

-        CP_SIZE_AT_OPEN,

-        CP_FASTEST_RANK,

-        CP_FASTEST_RANK_BYTES,

-        CP_SLOWEST_RANK,

-        CP_SLOWEST_RANK_BYTES,

-        -1};

-    int missing_f_counters[] = {

-        CP_F_MAX_READ_TIME,

-        CP_F_MAX_WRITE_TIME,

-        CP_F_FASTEST_RANK_TIME,

-        CP_F_SLOWEST_RANK_TIME,

-        CP_F_VARIANCE_RANK_TIME,

-        CP_F_VARIANCE_RANK_BYTES,

-        -1};

-

-    c_index = 0;

-    while(missing_counters[c_index] != -1)

-    {

-        int missing_counter = missing_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_NUM_INDICES - 1))

-        {

-            /* shift down */

-            memmove(&file->counters[missing_counter+1],

-                &file->counters[missing_counter],

-                (CP_NUM_INDICES-missing_counter-1)*sizeof(int64_t));

-        }

-        /* zero out missing counter */

-        file->counters[missing_counter] = 0;

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    c_index = 0;

-    while(missing_f_counters[c_index] != -1)

</span><span style="color: #000000;background-color: #ddffdd">+    assert(state);

+

+    /* if the file was created for writing */

+    if(state->creat_flag)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int missing_counter = missing_f_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_F_NUM_INDICES - 1))

</span><span style="color: #000000;background-color: #ddffdd">+        /* flush the last region of the log to file */

+        switch(state->comp_type)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            /* shift down */

-            memmove(&file->fcounters[missing_counter+1],

-                &file->fcounters[missing_counter],

-                (CP_F_NUM_INDICES-missing_counter-1)*sizeof(double));

</span><span style="color: #000000;background-color: #ddffdd">+            case DARSHAN_ZLIB_COMP:

+                ret = darshan_log_libz_flush(fd, state->dz.prev_reg_id);

+                if(ret == 0)

+                    break;

+#ifdef HAVE_LIBBZ2

+            case DARSHAN_BZIP2_COMP:

+                ret = darshan_log_bzip2_flush(fd, state->dz.prev_reg_id);

+                if(ret == 0)

+                    break;

+#endif 

+            default:

+                /* if flush fails, remove the output log file */

+                state->err = -1;

+                fprintf(stderr, "Error: final flush to log file failed.\n");

+                break;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        /* zero out missing counter */

-        file->fcounters[missing_counter] = 0;

-    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return;

-}

-

-/* shift_missing_1_22()

- *

- * translates indices to account for counters that weren't present in log

- * format 1.22

- */

-/*******************************

- * version 1.22 to 2.00 differences

- *

- * - added:

- *   - CP_DEVICE

- *   - CP_SIZE_AT_OPEN

- *   - CP_FASTEST_RANK

- *   - CP_FASTEST_RANK_BYTES

- *   - CP_SLOWEST_RANK

- *   - CP_SLOWEST_RANK_BYTES

- *   - CP_F_FASTEST_RANK_TIME

- *   - CP_F_SLOWEST_RANK_TIME

- *   - CP_F_VARIANCE_RANK_TIME

- *   - CP_F_VARIANCE_RANK_BYTES

- * - changed params:

- *   - CP_FILE_RECORD_SIZE: 1240 to 1328

- *   - CP_NUM_INDICES: 138 to 144

- *   - CP_F_NUM_INDICES: 14 to 18

- */

-static void shift_missing_1_22(struct darshan_file* file)

-{

-    int c_index = 0;

-    int missing_counters[] = {

-        CP_DEVICE,

-        CP_SIZE_AT_OPEN,

-        CP_FASTEST_RANK,

-        CP_FASTEST_RANK_BYTES,

-        CP_SLOWEST_RANK,

-        CP_SLOWEST_RANK_BYTES,

-        -1};

-    int missing_f_counters[] = {

-        CP_F_FASTEST_RANK_TIME,

-        CP_F_SLOWEST_RANK_TIME,

-        CP_F_VARIANCE_RANK_TIME,

-        CP_F_VARIANCE_RANK_BYTES,

-        -1};

-

-    c_index = 0;

-    while(missing_counters[c_index] != -1)

-    {

-        int missing_counter = missing_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_NUM_INDICES - 1))

</span><span style="color: #000000;background-color: #ddffdd">+        /* if no errors flushing, write the log header before closing */

+        if(state->err != -1)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            /* shift down */

-            memmove(&file->counters[missing_counter+1],

-                &file->counters[missing_counter],

-                (CP_NUM_INDICES-missing_counter-1)*sizeof(int64_t));

</span><span style="color: #000000;background-color: #ddffdd">+            ret = darshan_log_putheader(fd);

+            if(ret < 0)

+                state->err = -1;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        /* zero out missing counter */

-        file->counters[missing_counter] = 0;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    c_index = 0;

-    while(missing_f_counters[c_index] != -1)

</span><span style="color: #000000;background-color: #ddffdd">+    close(state->fildes);

+

+    /* remove output log file if error writing to it */

+    if((state->creat_flag) && (state->err == -1))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int missing_counter = missing_f_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_F_NUM_INDICES - 1))

-        {

-            /* shift down */

-            memmove(&file->fcounters[missing_counter+1],

-                &file->fcounters[missing_counter],

-                (CP_F_NUM_INDICES-missing_counter-1)*sizeof(double));

-        }

-        /* zero out missing counter */

-        file->fcounters[missing_counter] = 0;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Unlinking darshan log file %s ...\n",

+            state->logfile_path);

+        unlink(state->logfile_path);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    darshan_log_dzdestroy(state);

+    if(state->exe_mnt_data)

+        free(state->exe_mnt_data);

+    free(state);

+    free(fd);

+

</span>     return;

 }

<span style="color: #000000;background-color: #ffdddd">-/* shift_missing_1_24()

- *

- * translates indices to account for counters that weren't present in log

- * format 1.24

- */

-/*******************************

- * version 1.24 to 2.00 differences

</span><span style="color: #000000;background-color: #ddffdd">+/* **************************************************** */

+

+/* read the header of the darshan log and set internal fd data structures

+ * NOTE: this is the only portion of the darshan log that is uncompressed

</span>  *

<span style="color: #000000;background-color: #ffdddd">- * - added:

- *   - CP_FASTEST_RANK

- *   - CP_FASTEST_RANK_BYTES

- *   - CP_SLOWEST_RANK

- *   - CP_SLOWEST_RANK_BYTES

- *   - CP_F_FASTEST_RANK_TIME

- *   - CP_F_SLOWEST_RANK_TIME

- *   - CP_F_VARIANCE_RANK_TIME

- *   - CP_F_VARIANCE_RANK_BYTES

- * - changed params:

- *   - CP_FILE_RECORD_SIZE: ? to 1328

- *   - CP_NUM_INDICES: 140 to 144

- *   - CP_F_NUM_INDICES: 14 to 18

</span><span style="color: #000000;background-color: #ddffdd">+ * returns 0 on success, -1 on failure

</span>  */

<span style="color: #000000;background-color: #ffdddd">-static void shift_missing_1_24(struct darshan_file* file)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_getheader(darshan_fd fd)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int c_index = 0;

-    int missing_counters[] = {

-        CP_FASTEST_RANK,

-        CP_FASTEST_RANK_BYTES,

-        CP_SLOWEST_RANK,

-        CP_SLOWEST_RANK_BYTES,

-        -1};

-    int missing_f_counters[] = {

-        CP_F_FASTEST_RANK_TIME,

-        CP_F_SLOWEST_RANK_TIME,

-        CP_F_VARIANCE_RANK_TIME,

-        CP_F_VARIANCE_RANK_BYTES,

-        -1};

-

-    c_index = 0;

-    while(missing_counters[c_index] != -1)

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    struct darshan_header header;

+    int i;

+    int ret;

+

+    ret = darshan_log_seek(fd, 0);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int missing_counter = missing_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_NUM_INDICES - 1))

-        {

-            /* shift down */

-            memmove(&file->counters[missing_counter+1],

-                &file->counters[missing_counter],

-                (CP_NUM_INDICES-missing_counter-1)*sizeof(int64_t));

-        }

-        /* zero out missing counter */

-        file->counters[missing_counter] = 0;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: unable to seek in darshan log file.\n");

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    c_index = 0;

-    while(missing_f_counters[c_index] != -1)

</span><span style="color: #000000;background-color: #ddffdd">+    /* read the version number so we know how to process this log */

+    ret = darshan_log_read(fd, &fd->version, 8);

+    if(ret < 8)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int missing_counter = missing_f_counters[c_index];

-        c_index++;

-        if(missing_counter < (CP_F_NUM_INDICES - 1))

-        {

-            /* shift down */

-            memmove(&file->fcounters[missing_counter+1],

-                &file->fcounters[missing_counter],

-                (CP_F_NUM_INDICES-missing_counter-1)*sizeof(double));

-        }

-        /* zero out missing counter */

-        file->fcounters[missing_counter] = 0;

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: invalid log file (failed to read version).\n");

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return;

-}

-

-static int getjob_internal_204(darshan_fd file, struct darshan_job *job)

-{

-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    /* other log file versions can be detected and handled here */

+    if(strcmp(fd->version, "3.00"))

+    {

+        fprintf(stderr, "Error: incompatible darshan file.\n");

+        fprintf(stderr, "Error: expected version %s\n", DARSHAN_LOG_VERSION);

+        return(-1);

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(file, 0);

</span><span style="color: #000000;background-color: #ddffdd">+    /* seek back so we can read the entire header */

+    ret = darshan_log_seek(fd, 0);

</span>     if(ret < 0)

<span style="color: #000000;background-color: #ffdddd">-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+    {

+        fprintf(stderr, "Error: unable to seek in darshan log file.\n");

+        return(-1);

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(file, job, sizeof(*job));

-    if (ret < sizeof(*job))

</span><span style="color: #000000;background-color: #ddffdd">+    /* read uncompressed header from log file */

+    ret = darshan_log_read(fd, &header, sizeof(header));

+    if(ret != sizeof(header))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: invalid log file (too short).\n");

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to read darshan log file header.\n");

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    if(job->magic_nr == CP_MAGIC_NR)

</span><span style="color: #000000;background-color: #ddffdd">+    if(header.magic_nr == DARSHAN_MAGIC_NR)

</span>     {

         /* no byte swapping needed, this file is in host format already */

<span style="color: #000000;background-color: #ffdddd">-        file->swap_flag = 0;

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        fd->swap_flag = 0;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-

-    /* try byte swapping */

-    DARSHAN_BSWAP64(&job->magic_nr);

-    if(job->magic_nr == CP_MAGIC_NR)

</span><span style="color: #000000;background-color: #ddffdd">+    else

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        file->swap_flag = 1;

-        DARSHAN_BSWAP64(&job->uid);

-        DARSHAN_BSWAP64(&job->start_time);

-        DARSHAN_BSWAP64(&job->end_time);

-        DARSHAN_BSWAP64(&job->nprocs);

-        DARSHAN_BSWAP64(&job->jobid);

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        /* try byte swapping */

+        DARSHAN_BSWAP64(&(header.magic_nr));

+        if(header.magic_nr == DARSHAN_MAGIC_NR)

+        {

+            fd->swap_flag = 1;

+

+            /* swap the log map variables in the header */

+            DARSHAN_BSWAP64(&(header.rec_map.off));

+            DARSHAN_BSWAP64(&(header.rec_map.len));

+            for(i = 0; i < DARSHAN_MAX_MODS; i++)

+            {

+                DARSHAN_BSWAP64(&(header.mod_map[i].off));

+                DARSHAN_BSWAP64(&(header.mod_map[i].len));

+            }

+        }

+        else

+        {

+            /* otherwise this file is just broken */

+            fprintf(stderr, "Error: bad magic number in darshan log file.\n");

+            return(-1);

+        }

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* otherwise this file is just broken */

-    fprintf(stderr, "Error: bad magic number in darshan file.\n");

-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    /* set some fd fields based on what's stored in the header */

+    state->comp_type = header.comp_type;

+    fd->partial_flag = header.partial_flag;

+    memcpy(fd->mod_ver, header.mod_ver, DARSHAN_MAX_MODS * sizeof(uint32_t));

+

+    /* save the mapping of data within log file to this file descriptor */

+    fd->job_map.off = sizeof(struct darshan_header);

+    fd->job_map.len = header.rec_map.off - fd->job_map.off;

+    memcpy(&fd->rec_map, &(header.rec_map), sizeof(struct darshan_log_map));

+    memcpy(&fd->mod_map, &(header.mod_map), DARSHAN_MAX_MODS * sizeof(struct darshan_log_map));

+

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static int getjob_internal_201(darshan_fd file, struct darshan_job *job)

</span><span style="color: #000000;background-color: #ddffdd">+/* write a darshan header to log file

+ *

+ * returns 0 on success, -1 on failure

+ */

+static int darshan_log_putheader(darshan_fd fd)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    struct darshan_header header;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_job_201

-    {

-        char version_string[8];

-        int64_t magic_nr;

-        int64_t uid;

-        int64_t start_time;

-        int64_t end_time;

-        int64_t nprocs;

-        int64_t jobid;

-        char metadata[64];

-    } job_201;

-    memset(job, 0, sizeof(job_201));

-    memset(job, 0, sizeof(*job));

-

-    ret = darshan_log_seek(file, 0);

-    if(ret < 0)

-        return(ret);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(file, &job_201, sizeof(job_201));

-    if (ret < sizeof(job_201))

</span><span style="color: #000000;background-color: #ddffdd">+    ret = darshan_log_seek(fd, 0);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: invalid log file (too short).\n");

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: unable to seek in darshan log file.\n");

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    memcpy(job->version_string, job_201.version_string, 8);

-    job->magic_nr   = job_201.magic_nr;

-    job->uid        = job_201.uid;

-    job->start_time = job_201.start_time;

-    job->end_time   = job_201.end_time;

-    job->nprocs     = job_201.nprocs;

-    job->jobid      = job_201.jobid;

-    strncpy(job->metadata, job_201.metadata, 64);

</span><span style="color: #000000;background-color: #ddffdd">+    memset(&header, 0, sizeof(header));

+    strcpy(header.version_string, DARSHAN_LOG_VERSION);

+    header.magic_nr = DARSHAN_MAGIC_NR;

+    header.comp_type = state->comp_type;

+    header.partial_flag = fd->partial_flag;

+

+    /* copy the mapping information to the header */

+    memcpy(&header.rec_map, &fd->rec_map, sizeof(struct darshan_log_map));

+    memcpy(&header.mod_map, &fd->mod_map, DARSHAN_MAX_MODS * sizeof(struct darshan_log_map));

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(job->magic_nr == CP_MAGIC_NR)

</span><span style="color: #000000;background-color: #ddffdd">+    /* write header to file */

+    ret = darshan_log_write(fd, &header, sizeof(header));

+    if(ret != sizeof(header))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* no byte swapping needed, this file is in host format already */

-        file->swap_flag = 0;

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        fprintf(stderr, "Error: failed to write Darshan log file header.\n");

+        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* try byte swapping */

-    DARSHAN_BSWAP64(&job->magic_nr);

-    if(job->magic_nr == CP_MAGIC_NR)

</span><span style="color: #000000;background-color: #ddffdd">+    return(0);

+}

+

+/* return 0 on successful seek to offset, -1 on failure.

+ */

+static int darshan_log_seek(darshan_fd fd, off_t offset)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    off_t ret_off;

+

+    if(state->pos == offset)

+        return(0);

+

+    ret_off = lseek(state->fildes, offset, SEEK_SET);

+    if(ret_off == offset)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        file->swap_flag = 1;

-        DARSHAN_BSWAP64(&job->uid);

-        DARSHAN_BSWAP64(&job->start_time);

-        DARSHAN_BSWAP64(&job->end_time);

-        DARSHAN_BSWAP64(&job->nprocs);

-        DARSHAN_BSWAP64(&job->jobid);

</span><span style="color: #000000;background-color: #ddffdd">+        state->pos = offset;

</span>         return(0);

     }

<span style="color: #000000;background-color: #ffdddd">-    /* otherwise this file is just broken */

-    fprintf(stderr, "Error: bad magic number in darshan file.\n");

</span>     return(-1);

 }

<span style="color: #000000;background-color: #ffdddd">-

-static int getjob_internal_200(darshan_fd file, struct darshan_job *job)

<span style="color: #000000;background-color: #ddffdd">+/* return amount read on success, 0 on EOF, -1 on failure.

+ */

+static int darshan_log_read(darshan_fd fd, void* buf, int len)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_job_200

-    {

-        char version_string[8];

-        int64_t magic_nr;

-        int64_t uid;

-        int64_t start_time;

-        int64_t end_time;

-        int64_t nprocs;

-        int64_t jobid;

-    } job_200;

-

-    memset(job, 0, sizeof(job_200));

-    memset(job, 0, sizeof(*job));

-

-    ret = darshan_log_seek(file, 0);

-    if(ret < 0)

-        return(ret);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(file, &job_200, sizeof(job_200));

-    if (ret < sizeof(job_200))

-    {

-        fprintf(stderr, "Error: invalid log file (too short).\n");

-        return(-1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    /* read data from the log file using the given map */

+    ret = read(state->fildes, buf, len);

+    if(ret > 0)

+        state->pos += ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memcpy(job->version_string, job_200.version_string, 8);

-    job->magic_nr   = job_200.magic_nr;

-    job->uid        = job_200.uid;

-    job->start_time = job_200.start_time;

-    job->end_time   = job_200.end_time;

-    job->nprocs     = job_200.nprocs;

-    job->jobid      = job_200.jobid;

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(job->magic_nr == CP_MAGIC_NR)

-    {

-        /* no byte swapping needed, this file is in host format already */

-        file->swap_flag = 0;

-        return(0);

-    }

<span style="color: #000000;background-color: #ddffdd">+/* return amount written on success, -1 on failure.

+ */

+static int darshan_log_write(darshan_fd fd, void* buf, int len)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* try byte swapping */

-    DARSHAN_BSWAP64(&job->magic_nr);

-    if(job->magic_nr == CP_MAGIC_NR)

-    {

-        file->swap_flag = 1;

-        DARSHAN_BSWAP64(&job->uid);

-        DARSHAN_BSWAP64(&job->start_time);

-        DARSHAN_BSWAP64(&job->end_time);

-        DARSHAN_BSWAP64(&job->nprocs);

-        DARSHAN_BSWAP64(&job->jobid);

-        return(0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    ret = write(state->fildes, buf, len);

+    if(ret > 0)

+        state->pos += ret;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* otherwise this file is just broken */

-    fprintf(stderr, "Error: bad magic number in darshan file.\n");

-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_204(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_dzinit(struct darshan_fd_int_state *state)

</span> {

     int ret;

<span style="color: #000000;background-color: #ffdddd">-    const char* err_string;

-    int i;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->pos < CP_JOB_RECORD_SIZE)

-    {

-        ret = darshan_log_seek(fd, CP_JOB_RECORD_SIZE);

-        if(ret < 0)

-            return(ret);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    /* initialize buffers for staging compressed data to/from log file */

+    state->dz.buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);

+    if(state->dz.buf == NULL)

+        return(-1);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* reset file record, so that diff compares against a zero'd out record

-     * if file is missing

-     */

-    memset(file, 0, sizeof(&file));

</span><span style="color: #000000;background-color: #ddffdd">+    state->dz.prev_reg_id = DARSHAN_HEADER_REGION_ID;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, file, sizeof(*file));

-    if(ret == sizeof(*file))

</span><span style="color: #000000;background-color: #ddffdd">+    switch(state->comp_type)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* got exactly one, correct size record */

-        if(fd->swap_flag)

</span><span style="color: #000000;background-color: #ddffdd">+        case DARSHAN_ZLIB_COMP:

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            /* swap bytes if necessary */

-            DARSHAN_BSWAP64(&file->hash);

-            DARSHAN_BSWAP64(&file->rank);

-            for(i=0; i<CP_NUM_INDICES; i++)

-                DARSHAN_BSWAP64(&file->counters[i]);

-            for(i=0; i<CP_F_NUM_INDICES; i++)

-                DARSHAN_BSWAP64(&file->fcounters[i]);

</span><span style="color: #000000;background-color: #ddffdd">+            z_stream *tmp_zstrm = malloc(sizeof(*tmp_zstrm));

+            if(!tmp_zstrm)

+            {

+                free(state->dz.buf);

+                return(-1);

+            }

+            tmp_zstrm->zalloc = Z_NULL;

+            tmp_zstrm->zfree = Z_NULL;

+            tmp_zstrm->opaque = Z_NULL;

+            tmp_zstrm->avail_in = 0;

+            tmp_zstrm->next_in = Z_NULL;

+

+            /* TODO: worth using {inflate/deflate}Init2 ?? */

+            if(!(state->creat_flag))

+            {

+                /* read only file, init inflate algorithm */

+                ret = inflateInit(tmp_zstrm);

+            }

+            else

+            {

+                /* write only file, init deflate algorithm */

+                ret = deflateInit(tmp_zstrm, Z_DEFAULT_COMPRESSION);

+                tmp_zstrm->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+                tmp_zstrm->next_out = state->dz.buf;

+            }

+            if(ret != Z_OK)

+            {

+                free(tmp_zstrm);

+                free(state->dz.buf);

+                return(-1);

+            }

+            state->dz.strm = tmp_zstrm;

+            break;

+        }

+#ifdef HAVE_LIBBZ2

+        case DARSHAN_BZIP2_COMP:

+        {

+            bz_stream *tmp_bzstrm = malloc(sizeof(*tmp_bzstrm));

+            if(!tmp_bzstrm)

+            {

+                free(state->dz.buf);

+                return(-1);

+            }

+            tmp_bzstrm->bzalloc = NULL;

+            tmp_bzstrm->bzfree = NULL;

+            tmp_bzstrm->opaque = NULL;

+            tmp_bzstrm->avail_in = 0;

+            tmp_bzstrm->next_in = Z_NULL;

+

+            if(!(state->creat_flag))

+            {

+                /* read only file, init decompress algorithm */

+                ret = BZ2_bzDecompressInit(tmp_bzstrm, 1, 0);

+            }

+            else

+            {

+                /* write only file, init compress algorithm */

+                ret = BZ2_bzCompressInit(tmp_bzstrm, 9, 1, 30);

+                tmp_bzstrm->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+                tmp_bzstrm->next_out = (char *)state->dz.buf;

+            }

+            if(ret != BZ_OK)

+            {

+                free(tmp_bzstrm);

+                free(state->dz.buf);

+                return(-1);

+            }

+            state->dz.strm = tmp_bzstrm;

+            break;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        return(1);

</span><span style="color: #000000;background-color: #ddffdd">+#endif

+        default:

+            fprintf(stderr, "Error: invalid compression type.\n");

+            return(-1);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    return(0);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(ret > 0)

</span><span style="color: #000000;background-color: #ddffdd">+static void darshan_log_dzdestroy(struct darshan_fd_int_state *state)

+{

+    switch(state->comp_type)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* got a short read */

-        fprintf(stderr, "Error: invalid file record (too small)\n");

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        case DARSHAN_ZLIB_COMP:

+            if(!(state->creat_flag))

+                inflateEnd(state->dz.strm);

+            else

+                deflateEnd(state->dz.strm);

+            free(state->dz.strm);

+            break;

+#ifdef HAVE_LIBBZ2

+        case DARSHAN_BZIP2_COMP:

+            if(!(state->creat_flag))

+                BZ2_bzDecompressEnd(state->dz.strm);

+            else

+                BZ2_bzCompressEnd(state->dz.strm);

+            free(state->dz.strm);

+            break;

+#endif

+        default:

+            fprintf(stderr, "Error: invalid compression type.\n");

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(ret == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    free(state->dz.buf);

+    return;

+}

+

+static int darshan_log_dzread(darshan_fd fd, int region_id, void *buf, int len)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

+

+    switch(state->comp_type)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* hit end of file */

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        case DARSHAN_ZLIB_COMP:

+            ret = darshan_log_libz_read(fd, region_id, buf, len);

+            break;

+#ifdef HAVE_LIBBZ2

+        case DARSHAN_BZIP2_COMP:

+            ret = darshan_log_bzip2_read(fd, region_id, buf, len);

+            break;

+#endif

+        default:

+            fprintf(stderr, "Error: invalid compression type.\n");

+            return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* all other errors */

-    err_string = darshan_log_error(fd, &ret);

-    fprintf(stderr, "Error: %s\n", err_string);

-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_200(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_dzwrite(darshan_fd fd, int region_id, void *buf, int len)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    const char* err_string;

-    int i;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->pos < CP_JOB_RECORD_SIZE_200)

</span><span style="color: #000000;background-color: #ddffdd">+    switch(state->comp_type)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        ret = darshan_log_seek(fd, CP_JOB_RECORD_SIZE_200);

-        if(ret < 0)

-            return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+        case DARSHAN_ZLIB_COMP:

+            ret = darshan_log_libz_write(fd, region_id, buf, len);

+            break;

+#ifdef HAVE_LIBBZ2

+        case DARSHAN_BZIP2_COMP:

+            ret = darshan_log_bzip2_write(fd, region_id, buf, len);

+            break;

+#endif

+        default:

+            fprintf(stderr, "Error: invalid compression type.\n");

+            return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* reset file record, so that diff compares against a zero'd out record

-     * if file is missing

-     */

-    memset(file, 0, sizeof(&file));

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, file, sizeof(*file));

-    if(ret == sizeof(*file))

-    {

-        /* got exactly one, correct size record */

-        if(fd->swap_flag)

-        {

-            /* swap bytes if necessary */

-            DARSHAN_BSWAP64(&file->hash);

-            DARSHAN_BSWAP64(&file->rank);

-            for(i=0; i<CP_NUM_INDICES; i++)

-                DARSHAN_BSWAP64(&file->counters[i]);

-            for(i=0; i<CP_F_NUM_INDICES; i++)

-                DARSHAN_BSWAP64(&file->fcounters[i]);

-        }

-        return(1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_libz_read(darshan_fd fd, int region_id, void *buf, int len)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

+    int total_bytes = 0;

+    int tmp_out_bytes;

+    struct darshan_log_map map;

+    z_stream *z_strmp = (z_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ddffdd">+    assert(z_strmp);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(ret > 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* if new log region, we reload buffers and clear eor flag */

+    if(region_id != state->dz.prev_reg_id)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* got a short read */

-        fprintf(stderr, "Error: invalid file record (too small)\n");

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        z_strmp->avail_in = 0;

+        state->dz.eor = 0;

+        state->dz.prev_reg_id = region_id;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(ret == 0)

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map = fd->job_map;

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map = fd->rec_map;

+    else

+        map = fd->mod_map[region_id];

+

+    z_strmp->avail_out = len;

+    z_strmp->next_out = buf;

+

+    /* we just decompress until the output buffer is full, assuming there

+     * is enough compressed data in file to satisfy the request size.

+     */

+    while(z_strmp->avail_out)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* hit end of file */

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        /* check if we need more compressed data */

+        if(z_strmp->avail_in == 0)

+        {

+            /* if the eor flag is set, clear it and return -- future

+             * reads of this log region will restart at the beginning

+             */

+            if(state->dz.eor)

+            {

+                state->dz.eor = 0;

+                break;

+            }

+

+            /* read more data from input file */

+            ret = darshan_log_dzload(fd, map);

+            if(ret < 0)

+                return(-1);

+            assert(state->dz.size > 0);

+

+            z_strmp->avail_in = state->dz.size;

+            z_strmp->next_in = state->dz.buf;

+        }

+

+        tmp_out_bytes = z_strmp->total_out;

+        ret = inflate(z_strmp, Z_NO_FLUSH);

+        if(ret != Z_OK && ret != Z_STREAM_END)

+        {

+            fprintf(stderr, "Error: unable to inflate darshan log data.\n");

+            return(-1);

+        }

+        total_bytes += (z_strmp->total_out - tmp_out_bytes);

+

+        /* reset the decompression if we encountered end of stream */

+        if(ret == Z_STREAM_END)

+            inflateReset(z_strmp);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* all other errors */

-    err_string = darshan_log_error(fd, &ret);

-    fprintf(stderr, "Error: %s\n", err_string);

-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(total_bytes);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* If we see version 1.24, assume that it is stored in big endian 32 bit

- * format.  Convert up to current format.

- */

-static int getjob_internal_124(darshan_fd fd, struct darshan_job *job)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_libz_write(darshan_fd fd, int region_id, void *buf, int len)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    char* buffer;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    uint32_t uid;

-    int32_t start_time;

-    int32_t end_time;

-    int32_t nprocs;

-

-#ifdef WORDS_BIGENDIAN

-    fd->swap_flag = 0;

-#else

-    fd->swap_flag = 1;

-#endif

</span><span style="color: #000000;background-color: #ddffdd">+    int total_bytes = 0;

+    int tmp_in_bytes;

+    int tmp_out_bytes;

+    struct darshan_log_map *map_p;

+    z_stream *z_strmp = (z_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memset(job, 0, sizeof(*job));

</span><span style="color: #000000;background-color: #ddffdd">+    assert(z_strmp);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    buffer = (char*)malloc(JOB_SIZE_124);

-    if(!buffer)

</span><span style="color: #000000;background-color: #ddffdd">+    /* if new log region, finish prev region's zstream and flush to log file */

+    if(region_id != state->dz.prev_reg_id)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        return(-1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        /* error out if the region we are writing to precedes the previous

+         * region we wrote -- we shouldn't be moving backwards in the log

+         */

+        if(region_id < state->dz.prev_reg_id)

+            return(-1);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_seek(fd, 0);

-    if(ret < 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+        if(state->dz.prev_reg_id != DARSHAN_HEADER_REGION_ID)

+        {

+            ret = darshan_log_libz_flush(fd, state->dz.prev_reg_id);

+            if(ret < 0)

+                return(-1);

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, buffer, JOB_SIZE_124);

-    if (ret < JOB_SIZE_124)

-    {

-        fprintf(stderr, "Error: invalid log file (could not read file record).\n");

-        free(buffer);

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        state->dz.prev_reg_id = region_id;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    /* pull job header information out of specific bytes in case struct

-     * padding is off

-     */

-    strncpy(job->version_string, buffer, 8);

-    uid = *((uint32_t*)&buffer[12]);

-    start_time = *((int32_t*)&buffer[16]);

-    end_time = *((int32_t*)&buffer[20]);

-    nprocs = *((int32_t*)&buffer[24]);

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map_p = &(fd->job_map);

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map_p = &(fd->rec_map);

+    else

+        map_p = &(fd->mod_map[region_id]);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    free(buffer);

</span><span style="color: #000000;background-color: #ddffdd">+    z_strmp->avail_in = len;

+    z_strmp->next_in = buf;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->swap_flag)

</span><span style="color: #000000;background-color: #ddffdd">+    /* compress input data until none left */

+    while(z_strmp->avail_in)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* byte swap */

-        DARSHAN_BSWAP32(&uid);

-        DARSHAN_BSWAP32(&start_time);

-        DARSHAN_BSWAP32(&end_time);

-        DARSHAN_BSWAP32(&nprocs);

-    }

-

-    job->uid += uid;

-    job->start_time += start_time;

-    job->end_time += end_time;

-    job->nprocs += nprocs;

-    job->jobid = 0; /* old log versions did not have this field */

-    

-    /* set magic number */

-    job->magic_nr = CP_MAGIC_NR;

-

-    return(0);

-}

</span><span style="color: #000000;background-color: #ddffdd">+        /* if we are out of output, flush to log file */

+        if(z_strmp->avail_out == 0)

+        {

+            assert(state->dz.size == DARSHAN_DEF_COMP_BUF_SZ);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_124(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file)

-{

-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+            ret = darshan_log_dzunload(fd, map_p);

+            if(ret < 0)

+                return(-1);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = getfile_internal_1x(fd, job, file, 140, 14);

-    if(ret <= 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+            z_strmp->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+            z_strmp->next_out = state->dz.buf;

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    shift_missing_1_24(file);

</span><span style="color: #000000;background-color: #ddffdd">+        tmp_in_bytes = z_strmp->total_in;

+        tmp_out_bytes = z_strmp->total_out;

+        ret = deflate(z_strmp, Z_NO_FLUSH);

+        if(ret != Z_OK)

+        {

+            fprintf(stderr, "Error: unable to deflate darshan log data.\n");

+            return(-1);

+        }

+        total_bytes += (z_strmp->total_in - tmp_in_bytes);

+        state->dz.size += (z_strmp->total_out - tmp_out_bytes);

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(total_bytes);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_122(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_libz_flush(darshan_fd fd, int region_id)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    int tmp_out_bytes;

+    struct darshan_log_map *map_p;

+    z_stream *z_strmp = (z_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = getfile_internal_1x(fd, job, file, 138, 14);

-    if(ret <= 0)

-        return(ret);

-

-    shift_missing_1_22(file);

</span><span style="color: #000000;background-color: #ddffdd">+    assert(z_strmp);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(1);

-}

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map_p = &(fd->job_map);

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map_p = &(fd->rec_map);

+    else

+        map_p = &(fd->mod_map[region_id]);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_121(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file)

-{

-    int ret;

</span><span style="color: #000000;background-color: #ddffdd">+    /* make sure deflate finishes this stream */

+    z_strmp->avail_in = 0;

+    z_strmp->next_in = NULL;

+    do

+    {

+        tmp_out_bytes = z_strmp->total_out;

+        ret = deflate(z_strmp, Z_FINISH);

+        if(ret < 0)

+        {

+            fprintf(stderr, "Error: unable to deflate darshan log data.\n");

+            return(-1);

+        }

+        state->dz.size += (z_strmp->total_out - tmp_out_bytes);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = getfile_internal_1x(fd, job, file, 133, 12);

-    if(ret <= 0)

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+        if(state->dz.size)

+        {

+            /* flush to file */

+            if(darshan_log_dzunload(fd, map_p) < 0)

+                return(-1);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    shift_missing_1_21(file);

</span><span style="color: #000000;background-color: #ddffdd">+            z_strmp->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+            z_strmp->next_out = state->dz.buf;

+        }

+    } while (ret != Z_STREAM_END);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(1);

</span><span style="color: #000000;background-color: #ddffdd">+    deflateReset(z_strmp);

+    return(0);

</span> }

<span style="color: #000000;background-color: #ffdddd">-static int getfile_internal_1x(darshan_fd fd, struct darshan_job *job, 

-    struct darshan_file *file, int n_counters, int n_fcounters)

</span><span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_LIBBZ2

+

+static int darshan_log_bzip2_read(darshan_fd fd, int region_id, void *buf, int len)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    char* buffer;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ffdddd">-    const char* err_string;

-    int i;

-    uint64_t hash;

-    int32_t rank;

-    int64_t* counters;

-    double* fcounters;

-    char* name_suffix;

-    int FILE_SIZE_1x = (32 + n_counters*8 + n_fcounters*8);

</span><span style="color: #000000;background-color: #ddffdd">+    int total_bytes = 0;

+    int tmp_out_bytes;

+    struct darshan_log_map map;

+    bz_stream *bz_strmp = (bz_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memset(file, 0, sizeof(*file));

</span><span style="color: #000000;background-color: #ddffdd">+    assert(bz_strmp);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* set file pointer if this is the first file record; otherwise pick up

-     * where we left off last time

-     */

-    if(fd->pos < CP_JOB_RECORD_SIZE_1x)

-    {

-        ret = darshan_log_seek(fd, CP_JOB_RECORD_SIZE_1x);

-        if(ret < 0)

-            return(ret);

-    }

-    

-    /* space for file struct, int64 array, and double array */

-    buffer = (char*)malloc(FILE_SIZE_1x);

-    if(!buffer)

</span><span style="color: #000000;background-color: #ddffdd">+    /* if new log region, we reload buffers and clear eor flag */

+    if(region_id != state->dz.prev_reg_id)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        bz_strmp->avail_in = 0;

+        state->dz.eor = 0;

+        state->dz.prev_reg_id = region_id;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_read(fd, buffer, FILE_SIZE_1x);

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map = fd->job_map;

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map = fd->rec_map;

+    else

+        map = fd->mod_map[region_id];

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(ret > 0 && ret < FILE_SIZE_1x)

-    {

-        /* got a short read */

-        fprintf(stderr, "Error: invalid file record (too small)\n");

-        free(buffer);

-        return(-1);

-    }

-    else if(ret == 0)

-    {

-        /* hit end of file */

-        free(buffer);

-        return(0);

-    }

-    else if(ret <= 0)

</span><span style="color: #000000;background-color: #ddffdd">+    bz_strmp->avail_out = len;

+    bz_strmp->next_out = buf;

+

+    /* we just decompress until the output buffer is full, assuming there

+     * is enough compressed data in file to satisfy the request size.

+     */

+    while(bz_strmp->avail_out)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* all other errors */

-        err_string = darshan_log_error(fd, &ret);

-        fprintf(stderr, "Error: %s\n", err_string);

-        free(buffer);

-        return(-1);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        /* check if we need more compressed data */

+        if(bz_strmp->avail_in == 0)

+        {

+            /* if the eor flag is set, clear it and return -- future

+             * reads of this log region will restart at the beginning

+             */

+            if(state->dz.eor)

+            {

+                state->dz.eor = 0;

+                break;

+            }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* got exactly one, correct size record */

-    hash = *((int64_t*)&buffer[0]);

-    rank = *((int32_t*)&buffer[8]);

-    counters = ((int64_t*)&buffer[16]);

-    fcounters = ((double*)&buffer[16 + n_counters*8]);

-    name_suffix = &buffer[16 + n_counters*8 + n_fcounters*8];

</span><span style="color: #000000;background-color: #ddffdd">+            /* read more data from input file */

+            ret = darshan_log_dzload(fd, map);

+            if(ret < 0)

+                return(-1);

+            assert(state->dz.size > 0);

</span> 

<span style="color: #000000;background-color: #ddffdd">+            bz_strmp->avail_in = state->dz.size;

+            bz_strmp->next_in = (char *)state->dz.buf;

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->swap_flag)

-    {

-        /* swap bytes if necessary */

-        DARSHAN_BSWAP64(&hash);

-        DARSHAN_BSWAP32(&rank);

-        for(i=0; i<n_counters; i++)

-            DARSHAN_BSWAP64(&counters[i]);

-        for(i=0; i<n_fcounters; i++)

-            DARSHAN_BSWAP64(&fcounters[i]);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        tmp_out_bytes = bz_strmp->total_out_lo32;

+        ret = BZ2_bzDecompress(bz_strmp);

+        if(ret != BZ_OK && ret != BZ_STREAM_END)

+        {

+            fprintf(stderr, "Error: unable to decompress darshan log data.\n");

+            return(-1);

+        }

+        total_bytes += (bz_strmp->total_out_lo32 - tmp_out_bytes);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* assign into new format */

-    file->hash = hash;

-    file->rank += rank;

-    memcpy(file->counters, counters, n_counters*8);

-    memcpy(file->fcounters, fcounters, n_fcounters*8);

-    memcpy(file->name_suffix, name_suffix, 12);

</span><span style="color: #000000;background-color: #ddffdd">+        /* reset the decompression if we encountered end of stream */

+        if(ret == BZ_STREAM_END)

+        {

+            BZ2_bzDecompressEnd(bz_strmp);

+            BZ2_bzDecompressInit(bz_strmp, 1, 0);

+        }

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    free(buffer);

-    return(1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(total_bytes);

</span> }

<span style="color: #000000;background-color: #ffdddd">-/* return amount written on success, -1 on failure.

- */

-static int darshan_log_write(darshan_fd fd, void* buf, int len)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_bzip2_write(darshan_fd fd, int region_id, void *buf, int len)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    int total_bytes = 0;

+    int tmp_in_bytes;

+    int tmp_out_bytes;

+    struct darshan_log_map *map_p;

+    bz_stream *bz_strmp = (bz_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->gzf)

</span><span style="color: #000000;background-color: #ddffdd">+    assert(bz_strmp);

+

+    /* if new log region, finish prev region's zstream and flush to log file */

+    if(region_id != state->dz.prev_reg_id)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        ret = gzwrite(fd->gzf, buf, len);

-        if(ret > 0)

-            fd->pos += ret;

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+        /* error out if the region we are writing to precedes the previous

+         * region we wrote -- we shouldn't be moving backwards in the log

+         */

+        if(region_id < state->dz.prev_reg_id)

+            return(-1);

+

+        if(state->dz.prev_reg_id != DARSHAN_HEADER_REGION_ID)

+        {

+            ret = darshan_log_bzip2_flush(fd, state->dz.prev_reg_id);

+            if(ret < 0)

+                return(-1);

+        }

+

+        state->dz.prev_reg_id = region_id;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(fd->bzf)

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map_p = &(fd->job_map);

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map_p = &(fd->rec_map);

+    else

+        map_p = &(fd->mod_map[region_id]);

+

+    bz_strmp->avail_in = len;

+    bz_strmp->next_in = buf;

+

+    /* compress input data until none left */

+    while(bz_strmp->avail_in)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        ret = BZ2_bzwrite(fd->bzf, buf, len);

-        if(ret > 0)

-            fd->pos += ret;

-        return(ret);

</span><span style="color: #000000;background-color: #ddffdd">+        /* if we are out of output, flush to log file */

+        if(bz_strmp->avail_out == 0)

+        {

+            assert(state->dz.size == DARSHAN_DEF_COMP_BUF_SZ);

+

+            ret = darshan_log_dzunload(fd, map_p);

+            if(ret < 0)

+                return(-1);

+

+            bz_strmp->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+            bz_strmp->next_out = (char *)state->dz.buf;

+        }

+

+        tmp_in_bytes = bz_strmp->total_in_lo32;

+        tmp_out_bytes = bz_strmp->total_out_lo32;

+        ret = BZ2_bzCompress(bz_strmp, BZ_RUN);

+        if(ret != BZ_RUN_OK)

+        {

+            fprintf(stderr, "Error: unable to compress darshan log data.\n");

+            return(-1);

+        }

+        total_bytes += (bz_strmp->total_in_lo32 - tmp_in_bytes);

+        state->dz.size += (bz_strmp->total_out_lo32 - tmp_out_bytes);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-#endif

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    return(total_bytes);

</span> }

<span style="color: #000000;background-color: #ffdddd">-

-/* return amount read on success, 0 on EOF, -1 on failure.

- */

-static int darshan_log_read(darshan_fd fd, void* buf, int len)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_bzip2_flush(darshan_fd fd, int region_id)

</span> {

<span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

</span>     int ret;

<span style="color: #000000;background-color: #ddffdd">+    int tmp_out_bytes;

+    struct darshan_log_map *map_p;

+    bz_stream *bz_strmp = (bz_stream *)state->dz.strm;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->gzf)

-    {

-        ret = gzread(fd->gzf, buf, len);

-        if(ret > 0)

-            fd->pos += ret;

-        return(ret);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    assert(bz_strmp);

</span> 

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(fd->bzf)

</span><span style="color: #000000;background-color: #ddffdd">+    if(region_id == DARSHAN_JOB_REGION_ID)

+        map_p = &(fd->job_map);

+    else if(region_id == DARSHAN_REC_MAP_REGION_ID)

+        map_p = &(fd->rec_map);

+    else

+        map_p = &(fd->mod_map[region_id]);

+

+    /* make sure deflate finishes this stream */

+    bz_strmp->avail_in = 0;

+    bz_strmp->next_in = NULL;

+    do

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        ret = BZ2_bzread(fd->bzf, buf, len);

-        if(ret > 0)

-            fd->pos += ret;

-        return(ret);

-    }

-#endif

</span><span style="color: #000000;background-color: #ddffdd">+        tmp_out_bytes = bz_strmp->total_out_lo32;

+        ret = BZ2_bzCompress(bz_strmp, BZ_FINISH);

+        if(ret < 0)

+        {

+            fprintf(stderr, "Error: unable to compress darshan log data.\n");

+            return(-1);

+        }

+        state->dz.size += (bz_strmp->total_out_lo32 - tmp_out_bytes);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(-1);

-}

</span><span style="color: #000000;background-color: #ddffdd">+        if(state->dz.size)

+        {

+            /* flush to file */

+            if(darshan_log_dzunload(fd, map_p) < 0)

+                return(-1);

</span> 

<span style="color: #000000;background-color: #ddffdd">+            bz_strmp->avail_out = DARSHAN_DEF_COMP_BUF_SZ;

+            bz_strmp->next_out = (char *)state->dz.buf;

+        }

+    } while (ret != BZ_STREAM_END);

</span> 

<span style="color: #000000;background-color: #ffdddd">-static const char* darshan_log_error(darshan_fd fd, int* errnum)

-{

-    if(fd->gzf)

-    {

-        return(gzerror(fd->gzf, errnum));

-    }

</span><span style="color: #000000;background-color: #ddffdd">+    

+    BZ2_bzCompressEnd(bz_strmp);

+    BZ2_bzCompressInit(bz_strmp, 9, 1, 30);

+    return(0);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(fd->bzf)

-    {

-        return(BZ2_bzerror(fd->bzf, errnum));

-    }

</span> #endif

<span style="color: #000000;background-color: #ffdddd">-    *errnum = 0;

-    return(NULL);

-}

-

-/* return 0 on successful seek to offset, -1 on failure.

- */

-static int darshan_log_seek(darshan_fd fd, int64_t offset)

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_dzload(darshan_fd fd, struct darshan_log_map map)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    z_off_t zoff = 0;

-    z_off_t zoff_ret = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

+    unsigned int remaining;

+    unsigned int read_size;

+    unsigned int read_so_far = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->pos == offset)

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+    state->dz.size = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(fd->gzf)

</span><span style="color: #000000;background-color: #ddffdd">+    /* seek to the appropriate portion of the log file, if out of range */

+    if((state->pos < map.off) || (state->pos >= (map.off + map.len)))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        zoff += offset;

-        zoff_ret = gzseek(fd->gzf, zoff, SEEK_SET);

-        if(zoff_ret == zoff)

</span><span style="color: #000000;background-color: #ddffdd">+        ret = darshan_log_seek(fd, map.off);

+        if(ret < 0)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fd->pos = offset;

-            return(0);

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Error: unable to seek in darshan log file.\n");

+            return(-1);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        return(-1);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-#ifdef HAVE_LIBBZ2

-    if(fd->bzf)

</span><span style="color: #000000;background-color: #ddffdd">+    /* read more compressed data from file to staging buffer */

+    remaining = (map.off + map.len) - state->pos;

+    read_size = (remaining > DARSHAN_DEF_COMP_BUF_SZ) ?

+        DARSHAN_DEF_COMP_BUF_SZ : remaining;

+    do

+    {

+        ret = darshan_log_read(fd, state->dz.buf + read_so_far,

+            read_size - read_so_far);

+        if(ret <= 0)

+            break;

+        read_so_far += ret;

+    } while(read_so_far < read_size);

+    if(ret < 0)

+    {

+        fprintf(stderr, "Error: unable to read compressed data from file.\n");

+        return(-1);

+    }

+    if((read_size == remaining) || (ret == 0))

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int64_t counter;

-        char dummy = '\0';

-        int ret;

</span><span style="color: #000000;background-color: #ddffdd">+        state->dz.eor = 1;

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-        /* There is no seek in bzip2.  Just close, reopen, and throw away 

-         * data until the correct offset.  Very slow, but we don't expect to

-         * do this often.

-         */

-        if(fd->mode[0] == 'r' && offset < fd->pos)

-        {

-            /* to seek backwards in read-only mode we just close and re-open

-             * the file

-             */

-            BZ2_bzclose(fd->bzf);

-            fd->bzf = BZ2_bzopen(fd->name, fd->mode);

-            if(!fd->bzf)

-                return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    state->dz.size = read_size;

+    return(0);

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-            fd->pos = 0;

-        }

-        else if(fd->mode[0] == 'w' && offset < fd->pos)

-        {

-            /* there isn't any convenient way to seek backwards in a

-             * write-only bzip2 file, but we shouldn't need that

-             * functionality in darshan anyway.

-             */

-            fprintf(stderr, "Error: seeking backwards in a bzip2 compressed darshan output file is not supported.\n");

-            return(-1);

-        }

</span><span style="color: #000000;background-color: #ddffdd">+static int darshan_log_dzunload(darshan_fd fd, struct darshan_log_map *map_p)

+{

+    struct darshan_fd_int_state *state = fd->state;

+    int ret;

+    unsigned int write_so_far = 0;

+

+    /* initialize map structure for this log region */

+    if(map_p->off == 0)

+        map_p->off = state->pos;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        for(counter=0; counter<(offset-fd->pos); counter++)

</span><span style="color: #000000;background-color: #ddffdd">+    /* write more compressed data from staging buffer to file */

+    do

+    {

+        ret = darshan_log_write(fd, state->dz.buf + write_so_far,

+            state->dz.size - write_so_far);

+        if(ret <= 0)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            if(fd->mode[0] == 'r')

-            {

-                ret = BZ2_bzread(fd->bzf, &dummy, 1);

-            }

-            else

-            {

-                ret = BZ2_bzwrite(fd->bzf, &dummy, 1);

-            }

-            if(ret != 1)

-                return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Error: unable to write compressed data to file.\n");

+            return(-1);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        fd->pos += counter;

-        return(0);

-    }

-#endif

</span><span style="color: #000000;background-color: #ddffdd">+        write_so_far += ret;

+    } while(write_so_far < state->dz.size);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+    map_p->len += state->dz.size;

+    state->dz.size = 0;

+    return (0);

</span> }

<span style="color: #000000;background-color: #ddffdd">+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-167'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-167'>

<strong>

darshan-util/darshan-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.h

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.h

</span><span style="color: #aaaaaa">@@ -1,54 +1,151 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #ifndef __DARSHAN_LOG_UTILS_H

 #define __DARSHAN_LOG_UTILS_H

<span style="color: #000000;background-color: #ffdddd">-#include <darshan-log-format.h>

</span> 

<span style="color: #000000;background-color: #ddffdd">+#include <linux/limits.h>

+#include <zlib.h>

+#ifdef HAVE_LIBBZ2

+#include <bzlib.h>

+#endif

+

+#include "uthash-1.9.2/src/uthash.h"

+

+#include "darshan-log-format.h"

+

+struct darshan_fd_int_state;

+

+/* darshan file descriptor definition */

+struct darshan_fd_s

+{

+    /* log file version */

+    char version[8];

+    /* flag indicating whether byte swapping needs to be

+     * performed on log file data */

+    int swap_flag;

+    /* flag indicating whether a log file contains partial data */

+    int partial_flag;

+    /* log file offset/length maps for each log file region */

+    struct darshan_log_map job_map;

+    struct darshan_log_map rec_map;

+    struct darshan_log_map mod_map[DARSHAN_MAX_MODS];

+    /* module-specific log-format versions contained in log */

+    uint32_t mod_ver[DARSHAN_MAX_MODS];

+

+    /* KEEP OUT -- remaining state hidden in logutils source */

+    struct darshan_fd_int_state *state;

+};

</span> typedef struct darshan_fd_s* darshan_fd;

<span style="color: #000000;background-color: #ffdddd">-extern char *darshan_names[];

-extern char *darshan_f_names[];

-

-darshan_fd darshan_log_open(const char *name, const char* mode);

-int darshan_log_getjob(darshan_fd file, struct darshan_job *job);

-int darshan_log_putjob(darshan_fd file, struct darshan_job *job);

-int darshan_log_getfile(darshan_fd fd, 

-    struct darshan_job* job, 

-    struct darshan_file *file);

-int darshan_log_putfile(darshan_fd fd, 

-    struct darshan_job* job, 

-    struct darshan_file *file);

</span><span style="color: #000000;background-color: #ddffdd">+struct darshan_record_ref

+{

+    struct darshan_record rec;

+    UT_hash_handle hlink;

+};

+

+/* functions to be implemented by each module for integration with

+ * darshan log file utilities (e.g., parser & convert tools)

+ */

+struct darshan_mod_logutil_funcs

+{

+    /* retrieve a single module record from the log file. 

+     * return 1 on successful read of record, 0 on no more

+     * module data, -1 on error

+     *      - 'fd' is the file descriptor to get record from

+     *      - 'buf' is the buffer to store the record in

+     *      - 'rec_id' is the corresponding darshan record id

+     */

+    int (*log_get_record)(

+        darshan_fd fd,

+        void* buf,

+        darshan_record_id* rec_id

+    );

+    /* put a single module record into the log file.

+     * return 0 on success, -1 on error

+     *      - 'fd' is the file descriptor to put record into

+     *      - 'buf' is the buffer containing the record data

+     *      - 'rec_id' is the corresponding darshan record id

+     */

+    int (*log_put_record)(

+        darshan_fd fd,

+        void *buf,

+        int ver

+    );

+    /* print the counters for a given log record

+     *      - 'file_rec' is the record's data buffer

+     *      - 'file_name' is the file path string for the record

+     *      - 'mnt-pt' is the file path mount point string

+     *      - 'fs_type' is the file system type string

+     *      - 'ver' is the version of the record

+     */

+    void (*log_print_record)(

+        void *file_rec,

+        char *file_name,

+        char *mnt_pt,

+        char *fs_type,

+        int ver

+    );

+    /* print module-specific description of I/O characterization data */

+    void (*log_print_description)(void);

+    /* print a text diff of 2 module I/O records */

+    void (*log_print_diff)(

+        void *rec1,

+        char *name1,

+        void *rec2,

+        char *name2

+    );

+};

+

+extern struct darshan_mod_logutil_funcs *mod_logutils[];

+

+#include "darshan-posix-logutils.h"

+#include "darshan-mpiio-logutils.h"

+#include "darshan-hdf5-logutils.h"

+#include "darshan-pnetcdf-logutils.h"

+#include "darshan-bgq-logutils.h"

+

+darshan_fd darshan_log_open(const char *name);

+darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type,

+    int partial_flag);

+int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);

+int darshan_log_putjob(darshan_fd fd, struct darshan_job *job);

</span> int darshan_log_getexe(darshan_fd fd, char *buf);

 int darshan_log_putexe(darshan_fd fd, char *buf);

<span style="color: #000000;background-color: #ffdddd">-int darshan_log_getmounts(darshan_fd fd,

-    int64_t** devs,

-    char*** mnt_pts,

-    char*** fs_types,

-    int* count);

-int darshan_log_putmounts(darshan_fd fd,

-    int64_t* devs,

-    char** mnt_pts,

-    char** fs_types,

-    int count);

</span><span style="color: #000000;background-color: #ddffdd">+int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,

+    char*** fs_types, int* count);

+int darshan_log_putmounts(darshan_fd fd, char** mnt_pts,

+    char** fs_types, int count);

+int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);

+int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash);

+int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,

+    void *mod_buf, int mod_buf_sz);

+int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,

+    void *mod_buf, int mod_buf_sz, int ver);

</span> void darshan_log_close(darshan_fd file);

<span style="color: #000000;background-color: #ffdddd">-void darshan_log_print_version_warnings(struct darshan_job *job);

-

-/* convenience macros for printing out counters */

-#define CP_PRINT_HEADER() printf("#<rank>\t<file>\t<counter>\t<value>\t<name suffix>\t<mount pt>\t<fs type>\n")

-#define CP_PRINT(__job, __file, __counter, __mnt_pt, __fs_type) do {\

-        printf("%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t...%s\t%s\t%s\n", \

-            (__file)->rank, (__file)->hash, darshan_names[__counter], \

-            (__file)->counters[__counter], (__file)->name_suffix, \

-            __mnt_pt, __fs_type); \

</span><span style="color: #000000;background-color: #ddffdd">+

+/* convenience macros for printing Darshan counters */

+#define DARSHAN_PRINT_HEADER() \

+    printf("\n#<module>\t<rank>\t<record id>\t<counter>\t<value>" \

+           "\t<file name>\t<mount pt>\t<fs type>\n")

+

+#define DARSHAN_COUNTER_PRINT(__mod_name, __rank, __file_id, \

+                              __counter, __counter_val, __file_name, \

+                              __mnt_pt, __fs_type) do { \

+    printf("%s\t%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t%s\t%s\t%s\n", \

+        __mod_name, __rank, __file_id, __counter, __counter_val, \

+        __file_name, __mnt_pt, __fs_type); \

</span> } while(0)

<span style="color: #000000;background-color: #ffdddd">-#define CP_F_PRINT(__job, __file, __counter, __mnt_pt, __fs_type) do {\

-        printf("%" PRId64 "\t%" PRIu64 "\t%s\t%f\t...%s\t%s\t%s\n", \

-            (__file)->rank, (__file)->hash, darshan_f_names[__counter], \

-            (__file)->fcounters[__counter], (__file)->name_suffix, \

-            __mnt_pt, __fs_type); \

</span><span style="color: #000000;background-color: #ddffdd">+

+#define DARSHAN_F_COUNTER_PRINT(__mod_name, __rank, __file_id, \

+                                __counter, __counter_val, __file_name, \

+                                __mnt_pt, __fs_type) do { \

+    printf("%s\t%" PRId64 "\t%" PRIu64 "\t%s\t%f\t%s\t%s\t%s\n", \

+        __mod_name, __rank, __file_id, __counter, __counter_val, \

+        __file_name, __mnt_pt, __fs_type); \

</span> } while(0)

 /* naive byte swap implementation */

</code></pre>

<br>

</li>

<li id='diff-168'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-168'>

<strong>

darshan-util/darshan-mpiio-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-mpiio-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,234 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* counter name strings for the MPI-IO module */

+#define X(a) #a,

+char *mpiio_counter_names[] = {

+    MPIIO_COUNTERS

+};

+

+char *mpiio_f_counter_names[] = {

+    MPIIO_F_COUNTERS

+};

+#undef X

+

+static int darshan_log_get_mpiio_file(darshan_fd fd, void* mpiio_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf, int ver);

+static void darshan_log_print_mpiio_file(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_mpiio_description(void);

+static void darshan_log_print_mpiio_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+struct darshan_mod_logutil_funcs mpiio_logutils =

+{

+    .log_get_record = &darshan_log_get_mpiio_file,

+    .log_put_record = &darshan_log_put_mpiio_file,

+    .log_print_record = &darshan_log_print_mpiio_file,

+    .log_print_description = &darshan_log_print_mpiio_description,

+    .log_print_diff = &darshan_log_print_mpiio_file_diff

+};

+

+static int darshan_log_get_mpiio_file(darshan_fd fd, void* mpiio_buf,

+    darshan_record_id* rec_id)

+{

+    struct darshan_mpiio_file *file;

+    int i;

+    int ret;

+

+    ret = darshan_log_getmod(fd, DARSHAN_MPIIO_MOD, mpiio_buf,

+        sizeof(struct darshan_mpiio_file));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_mpiio_file))

+        return(0);

+    else

+    {

+        file = (struct darshan_mpiio_file *)mpiio_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&file->f_id);

+            DARSHAN_BSWAP64(&file->rank);

+            for(i=0; i<MPIIO_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->counters[i]);

+            for(i=0; i<MPIIO_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->fcounters[i]);

+        }

+

+        *rec_id = file->f_id;

+        return(1);

+    }

+}

+

+static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf, int ver)

+{

+    struct darshan_mpiio_file *file = (struct darshan_mpiio_file *)mpiio_buf;

+    int ret;

+

+    ret = darshan_log_putmod(fd, DARSHAN_MPIIO_MOD, file,

+        sizeof(struct darshan_mpiio_file), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+static void darshan_log_print_mpiio_file(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_mpiio_file *mpiio_file_rec =

+        (struct darshan_mpiio_file *)file_rec;

+

+    for(i=0; i<MPIIO_NUM_INDICES; i++)

+    {

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+            mpiio_file_rec->rank, mpiio_file_rec->f_id, mpiio_counter_names[i],

+            mpiio_file_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<MPIIO_F_NUM_INDICES; i++)

+    {

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+            mpiio_file_rec->rank, mpiio_file_rec->f_id, mpiio_f_counter_names[i],

+            mpiio_file_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+static void darshan_log_print_mpiio_description()

+{

+    printf("\n# description of MPIIO counters:\n");

+    printf("#   MPIIO_INDEP_*: MPI independent operation counts.\n");

+    printf("#   MPIIO_COLL_*: MPI collective operation counts.\n");

+    printf("#   MPIIO_SPLIT_*: MPI split collective operation counts.\n");

+    printf("#   MPIIO_NB_*: MPI non blocking operation counts.\n");

+    printf("#   READS,WRITES,and OPENS are types of operations.\n");

+    printf("#   MPIIO_SYNCS: MPI file sync operation counts.\n");

+    printf("#   MPIIO_HINTS: number of times MPI hints were used.\n");

+    printf("#   MPIIO_VIEWS: number of times MPI file views were used.\n");

+    printf("#   MPIIO_MODE: MPI-IO access mode that file was opened with.\n");

+    printf("#   MPIIO_BYTES_*: total bytes read and written at MPI-IO layer.\n");

+    printf("#   MPIIO_RW_SWITCHES: number of times access alternated between read and write.\n");

+    printf("#   MPIIO_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");

+    printf("#   MPIIO_SIZE_*_AGG_*: histogram of MPI datatype total sizes for read and write operations.\n");

+    printf("#   MPIIO_ACCESS*_ACCESS: the four most common total access sizes.\n");

+    printf("#   MPIIO_ACCESS*_COUNT: count of the four most common total access sizes.\n");

+    printf("#   MPIIO_*_RANK: rank of the processes that were the fastest and slowest at I/O (for shared files).\n");

+    printf("#   MPIIO_*_RANK_BYTES: total bytes transferred at MPI-IO layer by the fastest and slowest ranks (for shared files).\n");

+    printf("#   MPIIO_F_OPEN_TIMESTAMP: timestamp of first open.\n");

+    printf("#   MPIIO_F_*_START_TIMESTAMP: timestamp of first MPI-IO read/write.\n");

+    printf("#   MPIIO_F_*_END_TIMESTAMP: timestamp of last MPI-IO read/write.\n");

+    printf("#   MPIIO_F_CLOSE_TIMESTAMP: timestamp of last close.\n");

+    printf("#   MPIIO_F_READ/WRITE/META_TIME: cumulative time spent in MPI-IO read, write, or metadata operations.\n");

+    printf("#   MPIIO_F_MAX_*_TIME: duration of the slowest MPI-IO read and write operations.\n");

+    printf("#   MPIIO_F_*_RANK_TIME: fastest and slowest I/O time for a single rank (for shared files).\n");

+    printf("#   MPIIO_F_VARIANCE_RANK_*: variance of total I/O time and bytes moved for all ranks (for shared files).\n");

+

+    DARSHAN_PRINT_HEADER();

+

+    return;

+}

+

+static void darshan_log_print_mpiio_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_mpiio_file *file1 = (struct darshan_mpiio_file *)file_rec1;

+    struct darshan_mpiio_file *file2 = (struct darshan_mpiio_file *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<MPIIO_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file1->rank, file1->f_id, mpiio_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file2->rank, file2->f_id, mpiio_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file1->rank, file1->f_id, mpiio_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file2->rank, file2->f_id, mpiio_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<MPIIO_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file1->rank, file1->f_id, mpiio_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file2->rank, file2->f_id, mpiio_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file1->rank, file1->f_id, mpiio_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],

+                file2->rank, file2->f_id, mpiio_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-169'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-169'>

<strong>

darshan-util/darshan-mpiio-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-mpiio-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,15 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_MPIIO_LOG_UTILS_H

+#define __DARSHAN_MPIIO_LOG_UTILS_H

+

+extern char *mpiio_counter_names[];

+extern char *mpiio_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs mpiio_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-170'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-170'>

<strong>

darshan-util/darshan-null-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-null-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,234 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* integer counter name strings for the NULL module */

+#define X(a) #a,

+char *null_counter_names[] = {

+    NULL_COUNTERS

+};

+

+/* floating point counter name strings for the NULL module */

+char *null_f_counter_names[] = {

+    NULL_F_COUNTERS

+};

+#undef X

+

+/* prototypes for each of the NULL module's logutil functions */

+static int darshan_log_get_null_record(darshan_fd fd, void* null_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_null_record(darshan_fd fd, void* null_buf, int ver);

+static void darshan_log_print_null_record(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_null_description(void);

+static void darshan_log_print_null_record_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+/* structure storing each function needed for implementing the darshan

+ * logutil interface. these functions are used for reading, writing, and

+ * printing module data in a consistent manner.

+ */

+struct darshan_mod_logutil_funcs null_logutils =

+{

+    .log_get_record = &darshan_log_get_null_record,

+    .log_put_record = &darshan_log_put_null_record,

+    .log_print_record = &darshan_log_print_null_record,

+    .log_print_description = &darshan_log_print_null_description,

+    .log_print_diff = &darshan_log_print_null_record_diff

+};

+

+/* retrieve a NULL record from log file descriptor 'fd', storing the

+ * buffer in 'null_buf' and the corresponding Darshan record id in

+ * 'rec_id'. Return 1 on successful record read, 0 on no more data,

+ * and -1 on error.

+ */

+static int darshan_log_get_null_record(darshan_fd fd, void* null_buf, 

+    darshan_record_id* rec_id)

+{

+    struct darshan_null_record *rec;

+    int i;

+    int ret;

+

+    /* read a NULL module record from the darshan log file */

+    ret = darshan_log_getmod(fd, DARSHAN_NULL_MOD, null_buf,

+        sizeof(struct darshan_null_record));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_null_record))

+        return(0);

+    else

+    {

+        /* if the read was successful, do any necessary byte-swapping */

+        rec = (struct darshan_null_record *)null_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&rec->f_id);

+            DARSHAN_BSWAP64(&rec->rank);

+            for(i=0; i<NULL_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&rec->counters[i]);

+            for(i=0; i<NULL_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&rec->fcounters[i]);

+        }

+

+        /* set the output record id */

+        *rec_id = rec->f_id;

+        return(1);

+    }

+}

+

+/* write the NULL record stored in 'null_buf' to log file descriptor 'fd'.

+ * Return 0 on success, -1 on failure

+ */

+static int darshan_log_put_null_record(darshan_fd fd, void* null_buf, int ver)

+{

+    struct darshan_null_record *rec = (struct darshan_null_record *)null_buf;

+    int ret;

+

+    /* append NULL record to darshan log file */

+    ret = darshan_log_putmod(fd, DARSHAN_NULL_MOD, rec,

+        sizeof(struct darshan_null_record), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+/* print all I/O data record statistics for the given NULL record */

+static void darshan_log_print_null_record(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_null_record *null_rec =

+        (struct darshan_null_record *)file_rec;

+

+    /* print each of the integer and floating point counters for the NULL module */

+    for(i=0; i<NULL_NUM_INDICES; i++)

+    {

+        /* macro defined in darshan-logutils.h */

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+            null_rec->rank, null_rec->f_id, null_counter_names[i],

+            null_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<NULL_F_NUM_INDICES; i++)

+    {

+        /* macro defined in darshan-logutils.h */

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+            null_rec->rank, null_rec->f_id, null_f_counter_names[i],

+            null_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+/* print out a description of the NULL module record fields */

+static void darshan_log_print_null_description()

+{

+    printf("\n# description of NULL counters:\n");

+    printf("#   NULL_BARS: number of 'bar' function calls.\n");

+    printf("#   NULL_BAR_DAT: value set by last call to function 'bar'.\n");

+    printf("#   NULL_F_BAR_TIMESTAMP: timestamp of the first call to function 'bar'.\n");

+    printf("#   NULL_F_BAR_DURATION: duration of the last call to function 'bar'.\n");

+

+    return;

+}

+

+static void darshan_log_print_null_record_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_null_record *file1 = (struct darshan_null_record *)file_rec1;

+    struct darshan_null_record *file2 = (struct darshan_null_record *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<NULL_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file1->rank, file1->f_id, null_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file2->rank, file2->f_id, null_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file1->rank, file1->f_id, null_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file2->rank, file2->f_id, null_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<NULL_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file1->rank, file1->f_id, null_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file2->rank, file2->f_id, null_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file1->rank, file1->f_id, null_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],

+                file2->rank, file2->f_id, null_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-171'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-171'>

<strong>

darshan-util/darshan-null-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-null-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,18 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_NULL_LOG_UTILS_H

+#define __DARSHAN_NULL_LOG_UTILS_H

+

+/* declare NULL module counter name strings and logutil definition as

+ * extern variables so they can be used in other utilities

+ */

+extern char *null_counter_names[];

+extern char *null_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs null_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-172'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-172'>

<strong>

darshan-util/darshan-parser.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-parser.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-parser.c

</span><span style="color: #aaaaaa">@@ -1,6 +1,7 @@

</span> /*

<span style="color: #000000;background-color: #ffdddd">- *  (C) 2009 by Argonne National Laboratory.

- *      See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

</span>  */

 #include <stdio.h>

<span style="color: #aaaaaa">@@ -15,9 +16,11 @@

</span> #include <getopt.h>

 #include <assert.h>

<span style="color: #000000;background-color: #ddffdd">+#include "uthash-1.9.2/src/uthash.h"

+

</span> #include "darshan-logutils.h"

<span style="color: #000000;background-color: #ffdddd">-#include "uthash-1.9.2/src/uthash.h"

</span><span style="color: #000000;background-color: #ddffdd">+#define DEF_MOD_BUF_SIZE 1024 /* 1 KiB is enough for all current mod records ... */

</span> 

 /*

  * Options

<span style="color: #aaaaaa">@@ -41,7 +44,6 @@

</span> #define FILETYPE_PARTSHARED (1 << 2)

 #define max(a,b) (((a) > (b)) ? (a) : (b))

<span style="color: #000000;background-color: #ffdddd">-#define max3(a,b,c) (((a) > (b)) ? (((a) > (c)) ? (a) : (c)) : (((b) > (c)) ? (b) : (c)))

</span> 

 /*

  * Datatypes

<span style="color: #aaaaaa">@@ -49,36 +51,14 @@

</span> typedef struct hash_entry_s

 {

     UT_hash_handle hlink;

<span style="color: #000000;background-color: #ffdddd">-    uint64_t hash;

</span><span style="color: #000000;background-color: #ddffdd">+    darshan_record_id rec_id;

</span>     int64_t type;

     int64_t procs;

<span style="color: #000000;background-color: #ffdddd">-    int64_t counters[CP_NUM_INDICES];

-    double  fcounters[CP_F_NUM_INDICES];

</span><span style="color: #000000;background-color: #ddffdd">+    void *rec_dat;

</span>     double cumul_time;

<span style="color: #000000;background-color: #ffdddd">-    double meta_time;

</span>     double slowest_time;

<span style="color: #000000;background-color: #ffdddd">-    char name_suffix[CP_NAME_SUFFIX_LEN+1];

</span> } hash_entry_t;

<span style="color: #000000;background-color: #ffdddd">-typedef struct perf_data_s

-{

-    int64_t total_bytes;

-    double slowest_rank_time;

-    double slowest_rank_meta_time;

-    int slowest_rank_rank;

-    double shared_time_by_cumul;

-    double shared_time_by_open;

-    double shared_time_by_open_lastio;

-    double shared_time_by_slowest;

-    double shared_meta_time;

-    double agg_perf_by_cumul;

-    double agg_perf_by_open;

-    double agg_perf_by_open_lastio;

-    double agg_perf_by_slowest;

-    double *rank_cumul_io_time;

-    double *rank_cumul_md_time;

-} perf_data_t;

-

</span> typedef struct file_data_s

 {

     int64_t total;

<span style="color: #aaaaaa">@@ -101,15 +81,41 @@ typedef struct file_data_s

</span>     int64_t shared_max;

 } file_data_t;

<span style="color: #000000;background-color: #ddffdd">+typedef struct perf_data_s

+{

+    int64_t total_bytes;

+    double slowest_rank_time;

+    double slowest_rank_meta_time;

+    int slowest_rank_rank;

+    double shared_time_by_cumul;

+    double shared_time_by_open;

+    double shared_time_by_open_lastio;

+    double shared_time_by_slowest;

+    double shared_meta_time;

+    double agg_perf_by_cumul;

+    double agg_perf_by_open;

+    double agg_perf_by_open_lastio;

+    double agg_perf_by_slowest;

+    double *rank_cumul_io_time;

+    double *rank_cumul_md_time;

+} perf_data_t;

+

</span> /*

  * Prototypes

  */

<span style="color: #000000;background-color: #ffdddd">-void accum_perf(struct darshan_file *, hash_entry_t *, perf_data_t *);

-void calc_perf(struct darshan_job *, hash_entry_t *, perf_data_t *);

</span><span style="color: #000000;background-color: #ddffdd">+void posix_accum_file(struct darshan_posix_file *pfile, hash_entry_t *hfile, int64_t nprocs);

+void posix_accum_perf(struct darshan_posix_file *pfile, perf_data_t *pdata);

+void posix_calc_file(hash_entry_t *file_hash, file_data_t *fdata);

+void posix_print_total_file(struct darshan_posix_file *pfile);

+void posix_file_list(hash_entry_t *file_hash, struct darshan_record_ref *rec_hash, int detail_flag);

+

+void mpiio_accum_file(struct darshan_mpiio_file *mfile, hash_entry_t *hfile, int64_t nprocs);

+void mpiio_accum_perf(struct darshan_mpiio_file *mfile, perf_data_t *pdata);

+void mpiio_calc_file(hash_entry_t *file_hash, file_data_t *fdata);

+void mpiio_print_total_file(struct darshan_mpiio_file *mfile);

+void mpiio_file_list(hash_entry_t *file_hash, struct darshan_record_ref *rec_hash, int detail_flag);

</span> 

<span style="color: #000000;background-color: #ffdddd">-void accum_file(struct darshan_job *, struct darshan_file *, hash_entry_t *, file_data_t *);

-void calc_file(struct darshan_job *, hash_entry_t *, file_data_t *);

-void file_list(struct darshan_job *, hash_entry_t *, int);

</span><span style="color: #000000;background-color: #ddffdd">+void calc_perf(perf_data_t *pdata, int64_t nprocs);

</span> 

 int usage (char *exename)

 {

<span style="color: #aaaaaa">@@ -191,65 +197,74 @@ int main(int argc, char **argv)

</span> {

     int ret;

     int mask;

<span style="color: #000000;background-color: #ddffdd">+    int i, j;

</span>     char *filename;

<span style="color: #000000;background-color: #ddffdd">+    char tmp_string[4096] = {0};

+    darshan_fd fd;

</span>     struct darshan_job job;

<span style="color: #000000;background-color: #ffdddd">-    struct darshan_file cp_file;

-    char tmp_string[4096];

-    time_t tmp_time = 0;

-    darshan_fd file;

-    int i;

</span><span style="color: #000000;background-color: #ddffdd">+    struct darshan_record_ref *rec_hash = NULL;

+    struct darshan_record_ref *ref, *tmp_ref;

</span>     int mount_count;

<span style="color: #000000;background-color: #ffdddd">-    int64_t* devs;

</span>     char** mnt_pts;

     char** fs_types;

<span style="color: #000000;background-color: #ffdddd">-    int last_rank = 0;

</span><span style="color: #000000;background-color: #ddffdd">+    time_t tmp_time = 0;

</span>     char *token;

     char *save;

     char buffer[DARSHAN_JOB_METADATA_LEN];

<span style="color: #000000;background-color: #ddffdd">+    int empty_mods = 0;

+    char mod_buf[DEF_MOD_BUF_SIZE];

</span> 

     hash_entry_t *file_hash = NULL;

     hash_entry_t *curr = NULL;

<span style="color: #000000;background-color: #ffdddd">-    hash_entry_t *tmp = NULL;

</span><span style="color: #000000;background-color: #ddffdd">+    hash_entry_t *tmp_file = NULL;

</span>     hash_entry_t total;

<span style="color: #000000;background-color: #ffdddd">-    perf_data_t pdata;

</span>     file_data_t fdata;

<span style="color: #000000;background-color: #ddffdd">+    perf_data_t pdata;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    memset(&pdata, 0, sizeof(pdata));

</span>     memset(&total, 0, sizeof(total));

<span style="color: #000000;background-color: #ddffdd">+    memset(&fdata, 0, sizeof(fdata));

+    memset(&pdata, 0, sizeof(pdata));

</span> 

     mask = parse_args(argc, argv, &filename);

<span style="color: #000000;background-color: #ffdddd">-    file = darshan_log_open(filename, "r");

-    if(!file)

</span><span style="color: #000000;background-color: #ddffdd">+    fd = darshan_log_open(filename);

+    if(!fd)

+        return(-1);

+

+    /* read darshan job info */

+    ret = darshan_log_getjob(fd, &job);

+    if(ret < 0)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "darshan_log_open() failed to open %s\n.", filename);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(fd);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-   

-    /* read job info */

-    ret = darshan_log_getjob(file, &job);

</span><span style="color: #000000;background-color: #ddffdd">+

+    /* get the original command line for this job */

+    ret = darshan_log_getexe(fd, tmp_string);

</span>     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to read job information from log file.\n");

-        darshan_log_close(file);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(fd);

</span>         return(-1);

     }

<span style="color: #000000;background-color: #ffdddd">-    /* warn user about any missing information in this log format */

-    darshan_log_print_version_warnings(&job);

</span><span style="color: #000000;background-color: #ddffdd">+    /* get the mount information for this log */

+    ret = darshan_log_getmounts(fd, &mnt_pts, &fs_types, &mount_count);

+    if(ret < 0)

+    {

+        darshan_log_close(fd);

+        return(-1);

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    ret = darshan_log_getexe(file, tmp_string);

</span><span style="color: #000000;background-color: #ddffdd">+    /* read hash of darshan records */

+    ret = darshan_log_gethash(fd, &rec_hash);

</span>     if(ret < 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: unable to read trailing job information.\n");

-        darshan_log_close(file);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(fd);

</span>         return(-1);

     }

     /* print job summary */

<span style="color: #000000;background-color: #ffdddd">-    printf("# darshan log version: %s\n", job.version_string);

-    printf("# size of file statistics: %zu bytes\n", sizeof(cp_file));

-    printf("# size of job statistics: %zu bytes\n", sizeof(job));

</span><span style="color: #000000;background-color: #ddffdd">+    printf("# darshan log version: %s\n", fd->version);

</span>     printf("# exe: %s\n", tmp_string);

     printf("# uid: %" PRId64 "\n", job.uid);

     printf("# jobid: %" PRId64 "\n", job.jobid);

<span style="color: #aaaaaa">@@ -282,98 +297,51 @@ int main(int argc, char **argv)

</span>         value++;

         printf("# metadata: %s = %s\n", key, value);

     }

<span style="color: #000000;background-color: #ffdddd">- 

-    /* print table of mounted file systems */

-    ret = darshan_log_getmounts(file, &devs, &mnt_pts, &fs_types, &mount_count);

-    printf("\n# mounted file systems (device, mount point, and fs type)\n");

</span><span style="color: #000000;background-color: #ddffdd">+

+    /* print breakdown of each log file region's contribution to file size */

+    printf("\n# log file regions\n");

</span>     printf("# -------------------------------------------------------\n");

<span style="color: #000000;background-color: #ffdddd">-    for(i=0; i<mount_count; i++)

-    {

-        printf("# mount entry: %" PRId64 "\t%s\t%s\n", devs[i], mnt_pts[i], fs_types[i]);

-    }

-  

-    /* try to retrieve first record (may not exist) */

-    ret = darshan_log_getfile(file, &job, &cp_file);

-    if(ret < 0)

</span><span style="color: #000000;background-color: #ddffdd">+    printf("# header: %zu bytes (uncompressed)\n", sizeof(struct darshan_header));

+    printf("# job data: %zu bytes (compressed)\n", fd->job_map.len);

+    printf("# record table: %zu bytes (compressed)\n", fd->rec_map.len);

+    for(i=0; i<DARSHAN_MAX_MODS; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: failed to parse log file.\n");

-        fflush(stderr);

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        if(fd->mod_map[i].len)

+        {

+            printf("# %s module: %zu bytes (compressed), ver=%d\n",

+                darshan_module_names[i], fd->mod_map[i].len, fd->mod_ver[i]);

+        }

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if(ret == 0)

</span><span style="color: #000000;background-color: #ddffdd">+

+    /* print table of mounted file systems */

+    printf("\n# mounted file systems (mount point and fs type)\n");

+    printf("# -------------------------------------------------------\n");

+    for(i=0; i<mount_count; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* it looks like the app didn't open any files */

-        printf("# no files opened.\n");

-        darshan_log_close(file);

-        return(0);

</span><span style="color: #000000;background-color: #ddffdd">+        printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if ((mask & OPTION_BASE))

</span><span style="color: #000000;background-color: #ddffdd">+    if(mask & OPTION_BASE)

</span>     {

         printf("\n# description of columns:\n");

<span style="color: #000000;background-color: #ddffdd">+        printf("#   <module>: module responsible for this I/O record.\n");

</span>         printf("#   <rank>: MPI rank.  -1 indicates that the file is shared\n");

         printf("#      across all processes and statistics are aggregated.\n");

<span style="color: #000000;background-color: #ffdddd">-        printf("#   <file>: hash of file path.  0 indicates that statistics\n");

-        printf("#      are condensed to refer to all files opened at the given\n");

-        printf("#      process.\n");

-        printf("#   <counter> and <value>: statistical counters.\n");

</span><span style="color: #000000;background-color: #ddffdd">+        printf("#   <record id>: hash of the record's file path\n");

+        printf("#   <counter name> and <counter value>: statistical counters.\n");

</span>         printf("#      A value of -1 indicates that Darshan could not monitor\n");

         printf("#      that counter, and its value should be ignored.\n");

<span style="color: #000000;background-color: #ffdddd">-        printf("#   <name suffix>: last %d characters of file name.\n", CP_NAME_SUFFIX_LEN);

</span><span style="color: #000000;background-color: #ddffdd">+        printf("#   <file name>: full file path for the record.\n");

</span>         printf("#   <mount pt>: mount point that the file resides on.\n");

         printf("#   <fs type>: type of file system that the file resides on.\n");

<span style="color: #000000;background-color: #ffdddd">-        printf("\n# description of counters:\n");

-        printf("#   CP_POSIX_*: posix operation counts.\n");

-        printf("#   CP_COLL_*: MPI collective operation counts.\n");

-        printf("#   CP_INDEP_*: MPI independent operation counts.\n");

-        printf("#   CP_SPIT_*: MPI split collective operation counts.\n");

-        printf("#   CP_NB_*: MPI non blocking operation counts.\n");

-        printf("#   READS,WRITES,OPENS,SEEKS,STATS, and MMAPS are types of operations.\n");

-        printf("#   CP_*_NC_OPENS: number of indep. and collective pnetcdf opens.\n");

-        printf("#   CP_HDF5_OPENS: number of hdf5 opens.\n");

-        printf("#   CP_COMBINER_*: combiner counts for MPI mem and file datatypes.\n");

-        printf("#   CP_HINTS: number of times MPI hints were used.\n");

-        printf("#   CP_VIEWS: number of times MPI file views were used.\n");

-        printf("#   CP_MODE: mode that file was opened in.\n");

-        printf("#   CP_BYTES_*: total bytes read and written.\n");

-        printf("#   CP_MAX_BYTE_*: highest offset byte read and written.\n");

-        printf("#   CP_CONSEC_*: number of exactly adjacent reads and writes.\n");

-        printf("#   CP_SEQ_*: number of reads and writes from increasing offsets.\n");

-        printf("#   CP_RW_SWITCHES: number of times access alternated between read and write.\n");

-        printf("#   CP_*_ALIGNMENT: memory and file alignment.\n");

-        printf("#   CP_*_NOT_ALIGNED: number of reads and writes that were not aligned.\n");

-        printf("#   CP_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");

-        printf("#   CP_SIZE_READ_*: histogram of read access sizes.\n");

-        printf("#   CP_SIZE_READ_AGG_*: histogram of MPI datatype total sizes.\n");

-        printf("#   CP_EXTENT_READ_*: histogram of MPI datatype extents.\n");

-        printf("#   CP_STRIDE*_STRIDE: the four most common strides detected.\n");

-        printf("#   CP_STRIDE*_COUNT: count of the four most common strides.\n");

-        printf("#   CP_ACCESS*_ACCESS: the four most common access sizes.\n");

-        printf("#   CP_ACCESS*_COUNT: count of the four most common access sizes.\n");

-        printf("#   CP_DEVICE: File system identifier.\n");

-        printf("#   CP_SIZE_AT_OPEN: size of file when first opened.\n");

-        printf("#   CP_*_RANK_BYTES: fastest, slowest and variance of bytes transfer.\n");

-        printf("#   CP_F_OPEN_TIMESTAMP: timestamp of first open (mpi or posix).\n");

-        printf("#   CP_F_*_START_TIMESTAMP: timestamp of first read/write (mpi or posix).\n");

-        printf("#   CP_F_*_END_TIMESTAMP: timestamp of last read/write (mpi or posix).\n");

-        printf("#   CP_F_CLOSE_TIMESTAMP: timestamp of last close (mpi or posix).\n");

-        printf("#   CP_F_POSIX_READ/WRITE_TIME: cumulative time spent in posix reads or writes.\n");

-        printf("#   CP_F_MPI_READ/WRITE_TIME: cumulative time spent in mpi-io reads or writes.\n");

-        printf("#   CP_F_POSIX_META_TIME: cumulative time spent in posix open, close, fsync, stat and seek, .\n");

-        printf("#   CP_F_MPI_META_TIME: cumulative time spent in mpi-io open, close, set_view, and sync.\n");

-        printf("#   CP_MAX_*_TIME: duration of the slowest read and write operations.\n");

-        printf("#   CP_*_RANK_TIME: fastest, slowest variance of transfer time. Note that these counters show MPI-IO time for files accessed using MPI-IO, and POSIX time otherwise.\n");

-

-        printf("\n");

-        CP_PRINT_HEADER();

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    /* warn user if this log file is incomplete */

</span>     pdata.rank_cumul_io_time = malloc(sizeof(double)*job.nprocs);

     pdata.rank_cumul_md_time = malloc(sizeof(double)*job.nprocs);

     if (!pdata.rank_cumul_io_time || !pdata.rank_cumul_md_time)

     {

<span style="color: #000000;background-color: #ffdddd">-        perror("malloc failed");

-        darshan_log_close(file);

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_log_close(fd);

</span>         return(-1);

     }

     else

<span style="color: #aaaaaa">@@ -382,169 +350,289 @@ int main(int argc, char **argv)

</span>         memset(pdata.rank_cumul_md_time, 0, sizeof(double)*job.nprocs);

     }

<span style="color: #000000;background-color: #ffdddd">-    do

</span><span style="color: #000000;background-color: #ddffdd">+    for(i=0; i<DARSHAN_MAX_MODS; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        char* mnt_pt = NULL;

-        char* fs_type = NULL;

-        hash_entry_t *hfile = NULL;

</span><span style="color: #000000;background-color: #ddffdd">+        darshan_record_id rec_id;

+        void *save_io, *save_md;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if(cp_file.rank != -1 && cp_file.rank < last_rank)

</span><span style="color: #000000;background-color: #ddffdd">+        /* check each module for any data */

+        if(fd->mod_map[i].len == 0)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fprintf(stderr, "Error: log file contains out of order rank data.\n");

-            fflush(stderr);

-            return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+            empty_mods++;

+            continue;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if(cp_file.rank != -1)

-            last_rank = cp_file.rank;

-        

-        for(i=0; i<mount_count; i++)

</span><span style="color: #000000;background-color: #ddffdd">+        /* skip modules with no logutil definitions */

+        else if(!mod_logutils[i])

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            if(cp_file.counters[CP_DEVICE] == devs[i])

-            {

-                mnt_pt = mnt_pts[i];

-                fs_type = fs_types[i];

-                break;

-            }

</span><span style="color: #000000;background-color: #ddffdd">+            fprintf(stderr, "Warning: no log utility handlers defined "

+                "for module %s, SKIPPING.\n", darshan_module_names[i]);

+            continue;

+        }

+        /* currently we only do base parsing for non MPI & POSIX modules */

+        else if((i != DARSHAN_POSIX_MOD) && (i != DARSHAN_MPIIO_MOD) &&

+                !(mask & OPTION_BASE))

+            continue;

+

+        /* this module has data to be parsed and printed */

+        memset(mod_buf, 0, DEF_MOD_BUF_SIZE);

+

+        printf("\n# *******************************************************\n");

+        printf("# %s module data\n", darshan_module_names[i]);

+        printf("# *******************************************************\n");

+

+        /* print warning if this module only stored partial data */

+        if(DARSHAN_MOD_FLAG_ISSET(fd->partial_flag, i))

+            printf("\n# *WARNING*: The %s module contains incomplete data!\n"

+                   "#            This happens when a module runs out of\n"

+                   "#            memory to store new record data.\n",

+                   darshan_module_names[i]);

+

+        if(mask & OPTION_BASE)

+        {

+            /* print a header describing the module's I/O characterization data */

+            if(mod_logutils[i]->log_print_description)

+                mod_logutils[i]->log_print_description();

+        }

+

+        ret = mod_logutils[i]->log_get_record(fd, mod_buf, &rec_id);

+        if(ret != 1)

+        {

+            fprintf(stderr, "Error: failed to parse the first %s module record.\n",

+                darshan_module_names[i]);

+            ret = -1;

+            goto cleanup;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if(!mnt_pt)

-            mnt_pt = "UNKNOWN";

-        if(!fs_type)

-            fs_type = "UNKNOWN";

</span> 

<span style="color: #000000;background-color: #ffdddd">-        HASH_FIND(hlink,file_hash,&cp_file.hash,sizeof(int64_t),hfile);

-        if (!hfile)

</span><span style="color: #000000;background-color: #ddffdd">+        /* loop over each of this module's records and print them */

+        do

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            hfile = (hash_entry_t*) malloc(sizeof(*hfile));

-            if (!hfile)

</span><span style="color: #000000;background-color: #ddffdd">+            char *mnt_pt = NULL;

+            char *fs_type = NULL;

+            hash_entry_t *hfile = NULL;

+

+            /* get the pathname for this record */

+            HASH_FIND(hlink, rec_hash, &rec_id, sizeof(darshan_record_id), ref);

+            assert(ref);

+

+            /* get mount point and fs type associated with this record */

+            for(j=0; j<mount_count; j++)

+            {

+                if(strncmp(mnt_pts[j], ref->rec.name, strlen(mnt_pts[j])) == 0)

+                {

+                    mnt_pt = mnt_pts[j];

+                    fs_type = fs_types[j];

+                    break;

+                }

+            }

+            if(!mnt_pt)

+                mnt_pt = "UNKNOWN";

+            if(!fs_type)

+                fs_type = "UNKNOWN";

+

+            if(mask & OPTION_BASE)

+            {

+                /* print the corresponding module data for this record */

+                mod_logutils[i]->log_print_record(mod_buf, ref->rec.name,

+                    mnt_pt, fs_type, fd->mod_ver[i]);

+            }

+

+            /* we calculate more detailed stats for POSIX and MPI-IO modules, 

+             * if the parser is executed with more than the base option

+             */

+            if(i != DARSHAN_POSIX_MOD && i != DARSHAN_MPIIO_MOD)

+                continue;

+

+            HASH_FIND(hlink, file_hash, &rec_id, sizeof(darshan_record_id), hfile);

+            if(!hfile)

+            {

+                hfile = malloc(sizeof(*hfile));

+                if(!hfile)

+                {

+                    ret = -1;

+                    goto cleanup;

+                }

+

+                /* init */

+                memset(hfile, 0, sizeof(*hfile));

+                hfile->rec_id = rec_id;

+                hfile->type = 0;

+                hfile->procs = 0;

+                hfile->rec_dat = NULL;

+                hfile->cumul_time = 0.0;

+                hfile->slowest_time = 0.0;

+

+                HASH_ADD(hlink, file_hash, rec_id, sizeof(darshan_record_id), hfile);

+            }

+

+            if(i == DARSHAN_POSIX_MOD)

+            {

+                posix_accum_file((struct darshan_posix_file*)mod_buf, &total, job.nprocs);

+                posix_accum_file((struct darshan_posix_file*)mod_buf, hfile, job.nprocs);

+                posix_accum_perf((struct darshan_posix_file*)mod_buf, &pdata);

+            }

+            else if(i == DARSHAN_MPIIO_MOD)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                fprintf(stderr,"malloc failure");

-                exit(1);

</span><span style="color: #000000;background-color: #ddffdd">+                mpiio_accum_file((struct darshan_mpiio_file*)mod_buf, &total, job.nprocs);

+                mpiio_accum_file((struct darshan_mpiio_file*)mod_buf, hfile, job.nprocs);

+                mpiio_accum_perf((struct darshan_mpiio_file*)mod_buf, &pdata);

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            /* init */

-            memset(hfile, 0, sizeof(*hfile));

-            hfile->hash          = cp_file.hash;

-            memcpy(hfile->name_suffix, cp_file.name_suffix, CP_NAME_SUFFIX_LEN+1);

-            hfile->type          = 0;

-            hfile->procs         = 0;

-            hfile->cumul_time    = 0.0;

-            hfile->meta_time     = 0.0;

-            hfile->slowest_time  = 0.0;

-

-            HASH_ADD(hlink,file_hash,hash,sizeof(int64_t),hfile);

</span><span style="color: #000000;background-color: #ddffdd">+            memset(mod_buf, 0, DEF_MOD_BUF_SIZE);

+

+        } while((ret = mod_logutils[i]->log_get_record(fd, mod_buf, &rec_id)) == 1);

+        if (ret < 0)

+        {

+            ret = -1;

+            goto cleanup;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        accum_file(&job, &cp_file, &total, NULL);

-        accum_file(&job, &cp_file, hfile, &fdata);

-        accum_perf(&cp_file, hfile, &pdata);

</span><span style="color: #000000;background-color: #ddffdd">+        /* we calculate more detailed stats for POSIX and MPI-IO modules, 

+         * if the parser is executed with more than the base option

+         */

+        if(i != DARSHAN_POSIX_MOD && i != DARSHAN_MPIIO_MOD)

+            continue;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if ((mask & OPTION_BASE))

</span><span style="color: #000000;background-color: #ddffdd">+        /* Total Calc */

+        if(mask & OPTION_TOTAL)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            for(i=0; i<CP_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+            if(i == DARSHAN_POSIX_MOD)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                CP_PRINT(&job, &cp_file, i, mnt_pt, fs_type);

</span><span style="color: #000000;background-color: #ddffdd">+                posix_print_total_file((struct darshan_posix_file*)total.rec_dat);

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            for(i=0; i<CP_F_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+            else if(i == DARSHAN_MPIIO_MOD)

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                CP_F_PRINT(&job, &cp_file, i, mnt_pt, fs_type);

</span><span style="color: #000000;background-color: #ddffdd">+                mpiio_print_total_file((struct darshan_mpiio_file*)total.rec_dat);

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-    }while((ret = darshan_log_getfile(file, &job, &cp_file)) == 1);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* Total Calc */

-    if ((mask & OPTION_TOTAL))

-    {

-        for(i=0; i<CP_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+        /* File Calc */

+        if(mask & OPTION_FILE)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            printf("total_%s: %" PRId64 "\n",

-                   darshan_names[i], total.counters[i]);

</span><span style="color: #000000;background-color: #ddffdd">+            if(i == DARSHAN_POSIX_MOD)

+            {

+                posix_calc_file(file_hash, &fdata);

+            }

+            else if(i == DARSHAN_MPIIO_MOD)

+            {

+                mpiio_calc_file(file_hash, &fdata);

+            }

+

+            printf("\n# files\n");

+            printf("# -----\n");

+            printf("# total: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.total,

+                   fdata.total_size,

+                   fdata.total_max);

+            printf("# read_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.read_only,

+                   fdata.read_only_size,

+                   fdata.read_only_max);

+            printf("# write_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.write_only,

+                   fdata.write_only_size,

+                   fdata.write_only_max);

+            printf("# read_write: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.read_write,

+                   fdata.read_write_size,

+                   fdata.read_write_max);

+            printf("# unique: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.unique,

+                   fdata.unique_size,

+                   fdata.unique_max);

+            printf("# shared: %" PRId64 " %" PRId64 " %" PRId64 "\n",

+                   fdata.shared,

+                   fdata.shared_size,

+                   fdata.shared_max);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        for(i=0; i<CP_F_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+

+        /* Perf Calc */

+        if(mask & OPTION_PERF)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            printf("total_%s: %lf\n",

-                   darshan_f_names[i], total.fcounters[i]);

</span><span style="color: #000000;background-color: #ddffdd">+            calc_perf(&pdata, job.nprocs);

+

+            printf("\n# performance\n");

+            printf("# -----------\n");

+            printf("# total_bytes: %" PRId64 "\n", pdata.total_bytes);

+            printf("#\n");

+            printf("# I/O timing for unique files (seconds):\n");

+            printf("# ...........................\n");

+            printf("# unique files: slowest_rank_io_time: %lf\n", pdata.slowest_rank_time);

+            printf("# unique files: slowest_rank_meta_time: %lf\n", pdata.slowest_rank_meta_time);

+            printf("# unique files: slowest rank: %d\n", pdata.slowest_rank_rank);

+            printf("#\n");

+            printf("# I/O timing for shared files (seconds):\n");

+            printf("# (multiple estimates shown; time_by_slowest is generally the most accurate)\n");

+            printf("# ...........................\n");

+            printf("# shared files: time_by_cumul_io_only: %lf\n", pdata.shared_time_by_cumul);

+            printf("# shared files: time_by_cumul_meta_only: %lf\n", pdata.shared_meta_time);

+            printf("# shared files: time_by_open: %lf\n", pdata.shared_time_by_open);

+            printf("# shared files: time_by_open_lastio: %lf\n", pdata.shared_time_by_open_lastio);

+            printf("# shared files: time_by_slowest: %lf\n", pdata.shared_time_by_slowest);

+            printf("#\n");

+            printf("# Aggregate performance, including both shared and unique files (MiB/s):\n");

+            printf("# (multiple estimates shown; agg_perf_by_slowest is generally the most accurate)\n");

+            printf("# ...........................\n");

+            printf("# agg_perf_by_cumul: %lf\n", pdata.agg_perf_by_cumul);

+            printf("# agg_perf_by_open: %lf\n", pdata.agg_perf_by_open);

+            printf("# agg_perf_by_open_lastio: %lf\n", pdata.agg_perf_by_open_lastio);

+            printf("# agg_perf_by_slowest: %lf\n", pdata.agg_perf_by_slowest);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* Perf Calc */

-    calc_perf(&job, file_hash, &pdata);

-    if ((mask & OPTION_PERF))

-    {

-        printf("\n# performance\n");

-        printf("# -----------\n");

-        printf("# total_bytes: %" PRId64 "\n", pdata.total_bytes);

-        printf("#\n");

-        printf("# I/O timing for unique files (seconds):\n");

-        printf("# ...........................\n");

-        printf("# unique files: slowest_rank_io_time: %lf\n", pdata.slowest_rank_time);

-        printf("# unique files: slowest_rank_meta_time: %lf\n", pdata.slowest_rank_meta_time);

-        printf("# unique files: slowest rank: %d\n", pdata.slowest_rank_rank);

-        printf("#\n");

-        printf("# I/O timing for shared files (seconds):\n");

-        printf("# (multiple estimates shown; time_by_slowest is generally the most accurate)\n");

-        printf("# ...........................\n");

-        printf("# shared files: time_by_cumul_io_only: %lf\n", pdata.shared_time_by_cumul);

-        printf("# shared files: time_by_cumul_meta_only: %lf\n", pdata.shared_meta_time);

-        printf("# shared files: time_by_open: %lf\n", pdata.shared_time_by_open);

-        printf("# shared files: time_by_open_lastio: %lf\n", pdata.shared_time_by_open_lastio);

-        printf("# shared files: time_by_slowest: %lf\n", pdata.shared_time_by_slowest);

-        printf("#\n");

-        printf("# Aggregate performance, including both shared and unique files (MiB/s):\n");

-        printf("# (multiple estimates shown; agg_perf_by_slowest is generally the most accurate)\n");

-        printf("# ...........................\n");

-        printf("# agg_perf_by_cumul: %lf\n", pdata.agg_perf_by_cumul);

-        printf("# agg_perf_by_open: %lf\n", pdata.agg_perf_by_open);

-        printf("# agg_perf_by_open_lastio: %lf\n", pdata.agg_perf_by_open_lastio);

-        printf("# agg_perf_by_slowest: %lf\n", pdata.agg_perf_by_slowest);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+        if((mask & OPTION_FILE_LIST) || (mask & OPTION_FILE_LIST_DETAILED))

+        {

+            if(i == DARSHAN_POSIX_MOD)

+            {

+                if(mask & OPTION_FILE_LIST_DETAILED)

+                    posix_file_list(file_hash, rec_hash, 1);

+                else

+                    posix_file_list(file_hash, rec_hash, 0);

+            }

+            else if(i == DARSHAN_MPIIO_MOD)

+            {

+                if(mask & OPTION_FILE_LIST_DETAILED)

+                    mpiio_file_list(file_hash, rec_hash, 1);

+                else

+                    mpiio_file_list(file_hash, rec_hash, 0);

+            }

+        }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* File Calc */

-    calc_file(&job, file_hash, &fdata);

-    if ((mask & OPTION_FILE))

-    {

-        printf("\n# files\n");

-        printf("# -----\n");

-        printf("# total: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.total,

-               fdata.total_size,

-               fdata.total_max);

-        printf("# read_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.read_only,

-               fdata.read_only_size,

-               fdata.read_only_max);

-        printf("# write_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.write_only,

-               fdata.write_only_size,

-               fdata.write_only_max);

-        printf("# read_write: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.read_write,

-               fdata.read_write_size,

-               fdata.read_write_max);

-        printf("# unique: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.unique,

-               fdata.unique_size,

-               fdata.unique_max);

-        printf("# shared: %" PRId64 " %" PRId64 " %" PRId64 "\n",

-               fdata.shared,

-               fdata.shared_size,

-               fdata.shared_max);

</span><span style="color: #000000;background-color: #ddffdd">+        /* reset data structures for next module */

+        if(total.rec_dat) free(total.rec_dat);

+        memset(&total, 0, sizeof(total));

+        memset(&fdata, 0, sizeof(fdata));

+        save_io = pdata.rank_cumul_io_time;

+        save_md = pdata.rank_cumul_md_time;

+        memset(&pdata, 0, sizeof(pdata));

+        memset(save_io, 0, sizeof(double)*job.nprocs);

+        memset(save_md, 0, sizeof(double)*job.nprocs);

+        pdata.rank_cumul_io_time = save_io;

+        pdata.rank_cumul_md_time = save_md;

+

+        HASH_ITER(hlink, file_hash, curr, tmp_file)

+        {

+            HASH_DELETE(hlink, file_hash, curr);

+            if(curr->rec_dat) free(curr->rec_dat);

+            free(curr);

+        }

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    if(empty_mods == DARSHAN_MAX_MODS)

+        printf("\n# no module data available.\n");

+    ret = 0;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if ((mask & OPTION_FILE_LIST) || mask & OPTION_FILE_LIST_DETAILED)

-    {

-        if(mask & OPTION_FILE_LIST_DETAILED)

-            file_list(&job, file_hash, 1);

-        else

-            file_list(&job, file_hash, 0);

-    }

</span><span style="color: #000000;background-color: #ddffdd">+cleanup:

+    darshan_log_close(fd);

+    free(pdata.rank_cumul_io_time);

+    free(pdata.rank_cumul_md_time);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(ret < 0)

</span><span style="color: #000000;background-color: #ddffdd">+    /* free record hash data */

+    HASH_ITER(hlink, rec_hash, ref, tmp_ref)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        fprintf(stderr, "Error: failed to parse log file.\n");

-        fflush(stderr);

-        return(-1);

</span><span style="color: #000000;background-color: #ddffdd">+        HASH_DELETE(hlink, rec_hash, ref);

+        free(ref->rec.name);

+        free(ref);

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    /* free mount info */

</span>     for(i=0; i<mount_count; i++)

     {

         free(mnt_pts[i]);

<span style="color: #aaaaaa">@@ -552,75 +640,44 @@ int main(int argc, char **argv)

</span>     }

     if(mount_count > 0)

     {

<span style="color: #000000;background-color: #ffdddd">-        free(devs);

</span>         free(mnt_pts);

         free(fs_types);

     }

<span style="color: #000000;background-color: #ffdddd">- 

-    darshan_log_close(file);

</span> 

<span style="color: #000000;background-color: #ffdddd">-    HASH_ITER(hlink, file_hash, curr, tmp)

-    {

-        HASH_DELETE(hlink, file_hash, curr);

-        free(curr);

-    }

-

-    return(0);

</span><span style="color: #000000;background-color: #ddffdd">+    return(ret);

</span> }

<span style="color: #000000;background-color: #ffdddd">-void accum_file(struct darshan_job *job,

-                struct darshan_file *dfile,

-                hash_entry_t *hfile, 

-                file_data_t *fdata)

</span><span style="color: #000000;background-color: #ddffdd">+void posix_accum_file(struct darshan_posix_file *pfile,

+                      hash_entry_t *hfile,

+                      int64_t nprocs)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int i;

</span><span style="color: #000000;background-color: #ddffdd">+    int i, j;

+    int set;

+    int min_ndx;

+    int64_t min;

+    struct darshan_posix_file* tmp;

</span> 

     hfile->procs += 1;

<span style="color: #000000;background-color: #ffdddd">-    if (dfile->rank == -1)

</span><span style="color: #000000;background-color: #ddffdd">+    if(pfile->rank == -1)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        if(job->version_string[0] == '1')

-        {

-            hfile->slowest_time = 

-                max((dfile->fcounters[CP_F_READ_END_TIMESTAMP] 

-                    - dfile->fcounters[CP_F_OPEN_TIMESTAMP]),

-                    (dfile->fcounters[CP_F_WRITE_END_TIMESTAMP] 

-                    - dfile->fcounters[CP_F_OPEN_TIMESTAMP]));

-            if(hfile->slowest_time < 0)

-                hfile->slowest_time = 0;

-        }

-        else

-        {

-            hfile->slowest_time = dfile->fcounters[CP_F_SLOWEST_RANK_TIME];

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->slowest_time = pfile->fcounters[POSIX_F_SLOWEST_RANK_TIME];

</span>     }

     else

     {

<span style="color: #000000;background-color: #ffdddd">-        if(dfile->counters[CP_INDEP_OPENS] || dfile->counters[CP_COLL_OPENS])

-        {

-            /* MPI file */

-            hfile->slowest_time = max(hfile->slowest_time, 

-                (dfile->fcounters[CP_F_MPI_META_TIME] +

-                dfile->fcounters[CP_F_MPI_READ_TIME] +

-                dfile->fcounters[CP_F_MPI_WRITE_TIME]));

-        }

-        else

-        {

-            /* POSIX file */

-            hfile->slowest_time = max(hfile->slowest_time, 

-                (dfile->fcounters[CP_F_POSIX_META_TIME] +

-                dfile->fcounters[CP_F_POSIX_READ_TIME] +

-                dfile->fcounters[CP_F_POSIX_WRITE_TIME]));

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->slowest_time = max(hfile->slowest_time, 

+            (pfile->fcounters[POSIX_F_META_TIME] +

+            pfile->fcounters[POSIX_F_READ_TIME] +

+            pfile->fcounters[POSIX_F_WRITE_TIME]));

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    if (dfile->rank == -1)

</span><span style="color: #000000;background-color: #ddffdd">+    if(pfile->rank == -1)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        hfile->procs = job->nprocs;

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->procs = nprocs;

</span>         hfile->type |= FILETYPE_SHARED;

     }

<span style="color: #000000;background-color: #ffdddd">-    else if (hfile->procs > 1)

</span><span style="color: #000000;background-color: #ddffdd">+    else if(hfile->procs > 1)

</span>     {

         hfile->type &= (~FILETYPE_UNIQUE);

         hfile->type |= FILETYPE_PARTSHARED;

<span style="color: #aaaaaa">@@ -630,137 +687,162 @@ void accum_file(struct darshan_job *job,

</span>         hfile->type |= FILETYPE_UNIQUE;

     }

<span style="color: #000000;background-color: #ffdddd">-    if(dfile->counters[CP_INDEP_OPENS] || dfile->counters[CP_COLL_OPENS])

-    {

-        hfile->cumul_time += dfile->fcounters[CP_F_MPI_META_TIME] +

-                             dfile->fcounters[CP_F_MPI_READ_TIME] +

-                             dfile->fcounters[CP_F_MPI_WRITE_TIME];

-    }

-    else

</span><span style="color: #000000;background-color: #ddffdd">+    hfile->cumul_time += pfile->fcounters[POSIX_F_META_TIME] +

+                         pfile->fcounters[POSIX_F_READ_TIME] +

+                         pfile->fcounters[POSIX_F_WRITE_TIME];

+

+    if(hfile->rec_dat == NULL)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        hfile->cumul_time += dfile->fcounters[CP_F_POSIX_META_TIME] +

-                             dfile->fcounters[CP_F_POSIX_READ_TIME] +

-                             dfile->fcounters[CP_F_POSIX_WRITE_TIME];

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->rec_dat = malloc(sizeof(struct darshan_posix_file));

+        assert(hfile->rec_dat);

+        memset(hfile->rec_dat, 0, sizeof(struct darshan_posix_file));

</span>     }

<span style="color: #000000;background-color: #ddffdd">+    tmp = (struct darshan_posix_file*)hfile->rec_dat;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for (i = 0; i < CP_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    for(i = 0; i < POSIX_NUM_INDICES; i++)

</span>     {

         switch(i)

         {

<span style="color: #000000;background-color: #ffdddd">-        case CP_DEVICE:

-        case CP_MODE:

-        case CP_MEM_ALIGNMENT:

-        case CP_FILE_ALIGNMENT:

-            if(CP_FILE_PARTIAL(hfile))

-                hfile->counters[i] = dfile->counters[i];

-            break;

-        case CP_SIZE_AT_OPEN:

-            if (hfile->counters[i] == -1)

-            {

-                hfile->counters[i] = dfile->counters[i];

-            }

-            if (hfile->counters[i] > dfile->counters[i] && !CP_FILE_PARTIAL(dfile))

-            {

-                hfile->counters[i] = dfile->counters[i];

-            }

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_MODE:

+        case POSIX_MEM_ALIGNMENT:

+        case POSIX_FILE_ALIGNMENT:

+            tmp->counters[i] = pfile->counters[i];

</span>             break;

<span style="color: #000000;background-color: #ffdddd">-        case CP_MAX_BYTE_READ:

-        case CP_MAX_BYTE_WRITTEN:

-            if (hfile->counters[i] < dfile->counters[i])

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_MAX_BYTE_READ:

+        case POSIX_MAX_BYTE_WRITTEN:

+            if (tmp->counters[i] < pfile->counters[i])

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                hfile->counters[i] = dfile->counters[i];

</span><span style="color: #000000;background-color: #ddffdd">+                tmp->counters[i] = pfile->counters[i];

</span>             }

             break;

<span style="color: #000000;background-color: #ffdddd">-

-        case CP_STRIDE1_STRIDE:

-        case CP_STRIDE2_STRIDE:

-        case CP_STRIDE3_STRIDE:

-        case CP_STRIDE4_STRIDE:

-        case CP_ACCESS1_ACCESS:

-        case CP_ACCESS2_ACCESS:

-        case CP_ACCESS3_ACCESS:

-        case CP_ACCESS4_ACCESS:

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_STRIDE1_STRIDE:

+        case POSIX_STRIDE2_STRIDE:

+        case POSIX_STRIDE3_STRIDE:

+        case POSIX_STRIDE4_STRIDE:

+        case POSIX_ACCESS1_ACCESS:

+        case POSIX_ACCESS2_ACCESS:

+        case POSIX_ACCESS3_ACCESS:

+        case POSIX_ACCESS4_ACCESS:

</span>            /*

             * do nothing here because these will be stored

             * when the _COUNT is accessed.

             */

            break;

<span style="color: #000000;background-color: #ffdddd">- 

-        case CP_STRIDE1_COUNT:

-        case CP_STRIDE2_COUNT:

-        case CP_STRIDE3_COUNT:

-        case CP_STRIDE4_COUNT:

-        case CP_ACCESS1_COUNT:

-        case CP_ACCESS2_COUNT:

-        case CP_ACCESS3_COUNT:

-        case CP_ACCESS4_COUNT:

-            if (hfile->counters[i] < dfile->counters[i])

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_STRIDE1_COUNT:

+        case POSIX_STRIDE2_COUNT:

+        case POSIX_STRIDE3_COUNT:

+        case POSIX_STRIDE4_COUNT:

+            set = 0;

+            min_ndx = POSIX_STRIDE1_COUNT;

+            min = tmp->counters[min_ndx];

+            for(j = POSIX_STRIDE1_COUNT; j <= POSIX_STRIDE4_COUNT; j++)

+            {

+                if(tmp->counters[j-4] == pfile->counters[i-4])

+                {

+                    tmp->counters[j] += pfile->counters[i];

+                    set = 1;

+                    break;

+                }

+                if(tmp->counters[j] < min)

+                {

+                    min_ndx = j;

+                    min = tmp->counters[j];

+                }

+            }

+            if(!set && (pfile->counters[i] > min))

+            {

+                tmp->counters[min_ndx] = pfile->counters[i];

+                tmp->counters[min_ndx-4] = pfile->counters[i-4];

+            }

+            break;

+        case POSIX_ACCESS1_COUNT:

+        case POSIX_ACCESS2_COUNT:

+        case POSIX_ACCESS3_COUNT:

+        case POSIX_ACCESS4_COUNT:

+            set = 0;

+            min_ndx = POSIX_ACCESS1_COUNT;

+            min = tmp->counters[min_ndx];

+            for(j = POSIX_ACCESS1_COUNT; j <= POSIX_ACCESS4_COUNT; j++)

+            {

+                if(tmp->counters[j-4] == pfile->counters[i-4])

+                {

+                    tmp->counters[j] += pfile->counters[i];

+                    set = 1;

+                    break;

+                }

+                if(tmp->counters[j] < min)

+                {

+                    min_ndx = j;

+                    min = tmp->counters[j];

+                }

+            }

+            if(!set && (pfile->counters[i] > min))

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                hfile->counters[i]   = dfile->counters[i];

-                hfile->counters[i-4] = dfile->counters[i-4];

</span><span style="color: #000000;background-color: #ddffdd">+                tmp->counters[i] = pfile->counters[i];

+                tmp->counters[i-4] = pfile->counters[i-4];

</span>             }

             break;

<span style="color: #000000;background-color: #ffdddd">-        case CP_FASTEST_RANK:

-        case CP_SLOWEST_RANK:

-        case CP_FASTEST_RANK_BYTES:

-        case CP_SLOWEST_RANK_BYTES:

-            hfile->counters[i] = 0;

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_FASTEST_RANK:

+        case POSIX_SLOWEST_RANK:

+        case POSIX_FASTEST_RANK_BYTES:

+        case POSIX_SLOWEST_RANK_BYTES:

+            tmp->counters[i] = 0;

</span>             break;

<span style="color: #000000;background-color: #ffdddd">-        case CP_MAX_READ_TIME_SIZE:

-        case CP_MAX_WRITE_TIME_SIZE:

</span><span style="color: #000000;background-color: #ddffdd">+        case POSIX_MAX_READ_TIME_SIZE:

+        case POSIX_MAX_WRITE_TIME_SIZE:

</span>             break;

         default:

<span style="color: #000000;background-color: #ffdddd">-            hfile->counters[i] += dfile->counters[i];

</span><span style="color: #000000;background-color: #ddffdd">+            tmp->counters[i] += pfile->counters[i];

</span>             break;

         }

     }

<span style="color: #000000;background-color: #ffdddd">-    for (i = 0; i < CP_F_NUM_INDICES; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    for(i = 0; i < POSIX_F_NUM_INDICES; i++)

</span>     {

         switch(i)

         {

<span style="color: #000000;background-color: #ffdddd">-            case CP_F_OPEN_TIMESTAMP:

-            case CP_F_READ_START_TIMESTAMP:

-            case CP_F_WRITE_START_TIMESTAMP:

-                if(hfile->fcounters[i] == 0 || 

-                    hfile->fcounters[i] > dfile->fcounters[i])

</span><span style="color: #000000;background-color: #ddffdd">+            case POSIX_F_OPEN_TIMESTAMP:

+            case POSIX_F_READ_START_TIMESTAMP:

+            case POSIX_F_WRITE_START_TIMESTAMP:

+                if(tmp->fcounters[i] == 0 || 

+                    tmp->fcounters[i] > pfile->fcounters[i])

</span>                 {

<span style="color: #000000;background-color: #ffdddd">-                    hfile->fcounters[i] = dfile->fcounters[i];

</span><span style="color: #000000;background-color: #ddffdd">+                    tmp->fcounters[i] = pfile->fcounters[i];

</span>                 }

                 break;

<span style="color: #000000;background-color: #ffdddd">-            case CP_F_CLOSE_TIMESTAMP:

-            case CP_F_READ_END_TIMESTAMP:

-            case CP_F_WRITE_END_TIMESTAMP:

-                if(hfile->fcounters[i] == 0 || 

-                    hfile->fcounters[i] < dfile->fcounters[i])

</span><span style="color: #000000;background-color: #ddffdd">+            case POSIX_F_READ_END_TIMESTAMP:

+            case POSIX_F_WRITE_END_TIMESTAMP:

+            case POSIX_F_CLOSE_TIMESTAMP:

+                if(tmp->fcounters[i] == 0 || 

+                    tmp->fcounters[i] < pfile->fcounters[i])

</span>                 {

<span style="color: #000000;background-color: #ffdddd">-                    hfile->fcounters[i] = dfile->fcounters[i];

</span><span style="color: #000000;background-color: #ddffdd">+                    tmp->fcounters[i] = pfile->fcounters[i];

</span>                 }

                 break;

<span style="color: #000000;background-color: #ffdddd">-            case CP_F_FASTEST_RANK_TIME:

-            case CP_F_SLOWEST_RANK_TIME:

-            case CP_F_VARIANCE_RANK_TIME:

-            case CP_F_VARIANCE_RANK_BYTES:

-                hfile->fcounters[i] = 0;

</span><span style="color: #000000;background-color: #ddffdd">+            case POSIX_F_FASTEST_RANK_TIME:

+            case POSIX_F_SLOWEST_RANK_TIME:

+            case POSIX_F_VARIANCE_RANK_TIME:

+            case POSIX_F_VARIANCE_RANK_BYTES:

+                tmp->fcounters[i] = 0;

</span>                 break;

<span style="color: #000000;background-color: #ffdddd">-            case CP_F_MAX_READ_TIME:

-                if (hfile->fcounters[i] > dfile->fcounters[i])

</span><span style="color: #000000;background-color: #ddffdd">+            case POSIX_F_MAX_READ_TIME:

+                if(tmp->fcounters[i] < pfile->fcounters[i])

</span>                 {

<span style="color: #000000;background-color: #ffdddd">-                    hfile->fcounters[i] = dfile->fcounters[i];

-                    hfile->counters[CP_MAX_READ_TIME_SIZE] =

-                        dfile->counters[CP_MAX_READ_TIME_SIZE];

</span><span style="color: #000000;background-color: #ddffdd">+                    tmp->fcounters[i] = pfile->fcounters[i];

+                    tmp->counters[POSIX_MAX_READ_TIME_SIZE] =

+                        pfile->counters[POSIX_MAX_READ_TIME_SIZE];

</span>                 }

                 break;

<span style="color: #000000;background-color: #ffdddd">-            case CP_F_MAX_WRITE_TIME:

-                if (hfile->fcounters[i] > dfile->fcounters[i])

</span><span style="color: #000000;background-color: #ddffdd">+            case POSIX_F_MAX_WRITE_TIME:

+                if(tmp->fcounters[i] < pfile->fcounters[i])

</span>                 {

<span style="color: #000000;background-color: #ffdddd">-                    hfile->fcounters[i] = dfile->fcounters[i];

-                    hfile->counters[CP_MAX_WRITE_TIME_SIZE] =

-                        dfile->counters[CP_MAX_WRITE_TIME_SIZE];

</span><span style="color: #000000;background-color: #ddffdd">+                    tmp->fcounters[i] = pfile->fcounters[i];

+                    tmp->counters[POSIX_MAX_WRITE_TIME_SIZE] =

+                        pfile->counters[POSIX_MAX_WRITE_TIME_SIZE];

</span>                 }

                 break;

             default:

<span style="color: #000000;background-color: #ffdddd">-                hfile->fcounters[i] += dfile->fcounters[i];

</span><span style="color: #000000;background-color: #ddffdd">+                tmp->fcounters[i] += pfile->fcounters[i];

</span>                 break;

         }

     }

<span style="color: #aaaaaa">@@ -768,187 +850,255 @@ void accum_file(struct darshan_job *job,

</span>     return;

 }

<span style="color: #000000;background-color: #ffdddd">-void file_list(struct darshan_job *djob, hash_entry_t *file_hash, int detail_flag)

</span><span style="color: #000000;background-color: #ddffdd">+void mpiio_accum_file(struct darshan_mpiio_file *mfile,

+                      hash_entry_t *hfile,

+                      int64_t nprocs)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    hash_entry_t *curr = NULL;

-    hash_entry_t *tmp = NULL;

-    char* type;

-    int i;

</span><span style="color: #000000;background-color: #ddffdd">+    int i, j;

+    int set;

+    int min_ndx;

+    int64_t min;

+    struct darshan_mpiio_file* tmp;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    /* TODO: list of columns:

-     *

-     * normal mode?

-     * - hash

-     * - suffix

-     * - MPI or POSIX

-     * - nprocs

-     * - slowest I/O time

-     * - average cumulative I/O time

-     *

-     * detailed mode?

-     * - first open

-     * - first read

-     * - first write

-     * - last close

-     * - last read

-     * - last write

-     * - MPI indep opens

-     * - MPI coll opens

-     * - POSIX opens

-     * - r histogram (POSIX)

-     * - w histogram (POSIX)

-     */

</span><span style="color: #000000;background-color: #ddffdd">+    hfile->procs += 1;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    if(detail_flag)

-        printf("\n# Per-file summary of I/O activity (detailed).\n");

</span><span style="color: #000000;background-color: #ddffdd">+    if(mfile->rank == -1)

+    {

+        hfile->slowest_time = mfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];

+    }

</span>     else

<span style="color: #000000;background-color: #ffdddd">-        printf("\n# Per-file summary of I/O activity.\n");

</span><span style="color: #000000;background-color: #ddffdd">+    {

+        hfile->slowest_time = max(hfile->slowest_time, 

+            (mfile->fcounters[MPIIO_F_META_TIME] +

+            mfile->fcounters[MPIIO_F_READ_TIME] +

+            mfile->fcounters[MPIIO_F_WRITE_TIME]));

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-    printf("# <hash>: hash of file name\n");

-    printf("# <suffix>: last %d characters of file name\n", CP_NAME_SUFFIX_LEN);

-    printf("# <type>: MPI or POSIX\n");

-    printf("# <nprocs>: number of processes that opened the file\n");

-    printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");

-    printf("# <avg>: average time in seconds consumed in IO per process\n");

-    if(detail_flag)

</span><span style="color: #000000;background-color: #ddffdd">+    if(mfile->rank == -1)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");

-        printf("# <end_{open/read/write}>: end timestamp of last open, read, or write\n");

-        printf("# <mpi_indep_opens>: independent MPI_File_open calls\n");

-        printf("# <mpi_coll_opens>: collective MPI_File_open calls\n");

-        printf("# <posix_opens>: POSIX open calls\n");

-        printf("# <CP_SIZE_READ_*>: POSIX read size histogram\n");

-        printf("# <CP_SIZE_WRITE_*>: POSIX write size histogram\n");

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->procs = nprocs;

+        hfile->type |= FILETYPE_SHARED;

+

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    

-    printf("\n# <hash>\t<suffix>\t<type>\t<nprocs>\t<slowest>\t<avg>");

-    if(detail_flag)

</span><span style="color: #000000;background-color: #ddffdd">+    else if(hfile->procs > 1)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        printf("\t<start_open>\t<start_read>\t<start_write>");

-        printf("\t<end_open>\t<end_read>\t<end_write>");

-        printf("\t<mpi_indep_opens>\t<mpi_coll_opens>\t<posix_opens>");

-        for(i=CP_SIZE_READ_0_100; i<= CP_SIZE_WRITE_1G_PLUS; i++)

-            printf("\t%s", darshan_names[i]);

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->type &= (~FILETYPE_UNIQUE);

+        hfile->type |= FILETYPE_PARTSHARED;

+    }

+    else

+    {

+        hfile->type |= FILETYPE_UNIQUE;

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    printf("\n");

</span> 

<span style="color: #000000;background-color: #ffdddd">-    HASH_ITER(hlink, file_hash, curr, tmp)

</span><span style="color: #000000;background-color: #ddffdd">+    hfile->cumul_time += mfile->fcounters[MPIIO_F_META_TIME] +

+                         mfile->fcounters[MPIIO_F_READ_TIME] +

+                         mfile->fcounters[MPIIO_F_WRITE_TIME];

+

+    if(hfile->rec_dat == NULL)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        if(curr->counters[CP_INDEP_OPENS] || curr->counters[CP_COLL_OPENS])

-            type = "MPI";

-        else

-            type = "POSIX";

</span><span style="color: #000000;background-color: #ddffdd">+        hfile->rec_dat = malloc(sizeof(struct darshan_mpiio_file));

+        assert(hfile->rec_dat);

+        memset(hfile->rec_dat, 0, sizeof(struct darshan_mpiio_file));

+    }

+    tmp = (struct darshan_mpiio_file*)hfile->rec_dat;

</span> 

<span style="color: #000000;background-color: #ffdddd">-        printf("%" PRIu64 "\t%s\t%s\t%" PRId64 "\t%f\t%f",

-            curr->hash,

-            curr->name_suffix,

-            type,

-            curr->procs,

-            curr->slowest_time,

-            curr->cumul_time/(double)curr->procs);

-        if(detail_flag)

</span><span style="color: #000000;background-color: #ddffdd">+    for(i = 0; i < MPIIO_NUM_INDICES; i++)

+    {

+        switch(i)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            for(i=CP_F_OPEN_TIMESTAMP; i<=CP_F_WRITE_END_TIMESTAMP; i++)

</span><span style="color: #000000;background-color: #ddffdd">+        case MPIIO_MODE:

+            tmp->counters[i] = mfile->counters[i];

+            break;

+        case MPIIO_ACCESS1_ACCESS:

+        case MPIIO_ACCESS2_ACCESS:

+        case MPIIO_ACCESS3_ACCESS:

+        case MPIIO_ACCESS4_ACCESS:

+            /*

+             * do nothing here because these will be stored

+             * when the _COUNT is accessed.

+             */

+            break;

+        case MPIIO_ACCESS1_COUNT:

+        case MPIIO_ACCESS2_COUNT:

+        case MPIIO_ACCESS3_COUNT:

+        case MPIIO_ACCESS4_COUNT:

+            set = 0;

+            min_ndx = MPIIO_ACCESS1_COUNT;

+            min = tmp->counters[min_ndx];

+            for(j = MPIIO_ACCESS1_COUNT; j <= MPIIO_ACCESS4_COUNT; j++)

+            {

+                if(tmp->counters[j-4] == mfile->counters[i-4])

+                {

+                    tmp->counters[j] += mfile->counters[i];

+                    set = 1;

+                    break;

+                }

+                if(tmp->counters[j] < min)

+                {

+                    min_ndx = j;

+                    min = tmp->counters[j];

+                }

+            }

+            if(!set && (mfile->counters[i] > min))

</span>             {

<span style="color: #000000;background-color: #ffdddd">-                printf("\t%f", curr->fcounters[i]);

</span><span style="color: #000000;background-color: #ddffdd">+                tmp->counters[i] = mfile->counters[i];

+                tmp->counters[i-4] = mfile->counters[i-4];

</span>             }

<span style="color: #000000;background-color: #ffdddd">-            printf("\t%" PRId64 "\t%" PRId64 "\t%" PRId64, curr->counters[CP_INDEP_OPENS], curr->counters[CP_COLL_OPENS], curr->counters[CP_POSIX_OPENS]);

-            for(i=CP_SIZE_READ_0_100; i<= CP_SIZE_WRITE_1G_PLUS; i++)

-                printf("\t%" PRId64, curr->counters[i]);

</span><span style="color: #000000;background-color: #ddffdd">+            break;

+        case MPIIO_FASTEST_RANK:

+        case MPIIO_SLOWEST_RANK:

+        case MPIIO_FASTEST_RANK_BYTES:

+        case MPIIO_SLOWEST_RANK_BYTES:

+            tmp->counters[i] = 0;

+            break;

+        case MPIIO_MAX_READ_TIME_SIZE:

+        case MPIIO_MAX_WRITE_TIME_SIZE:

+            break;

+        default:

+            tmp->counters[i] += mfile->counters[i];

+            break;

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        printf("\n");

</span>     }

<span style="color: #000000;background-color: #ffdddd">-    return;

-}

-

-void calc_file(struct darshan_job *djob,

-               hash_entry_t *file_hash, 

-               file_data_t *fdata)

-{

-    hash_entry_t *curr = NULL;

-    hash_entry_t *tmp = NULL;

-

-    memset(fdata, 0, sizeof(*fdata));

-

-    HASH_ITER(hlink, file_hash, curr, tmp)

</span><span style="color: #000000;background-color: #ddffdd">+    for(i = 0; i < MPIIO_F_NUM_INDICES; i++)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        int64_t max;

-        int64_t r;

-        int64_t w;

-

-        max = max3(curr->counters[CP_SIZE_AT_OPEN],

-                   curr->counters[CP_MAX_BYTE_READ],

-                   curr->counters[CP_MAX_BYTE_WRITTEN]);

-

-        r = (curr->counters[CP_POSIX_READS]+

-             curr->counters[CP_POSIX_FREADS]+

-             curr->counters[CP_INDEP_READS]+

-             curr->counters[CP_COLL_READS]+

-             curr->counters[CP_SPLIT_READS]+

-             curr->counters[CP_NB_READS]);

</span><span style="color: #000000;background-color: #ddffdd">+        switch(i)

+        {

+            case MPIIO_F_OPEN_TIMESTAMP:

+            case MPIIO_F_READ_START_TIMESTAMP:

+            case MPIIO_F_WRITE_START_TIMESTAMP:

+                if(tmp->fcounters[i] == 0 || 

+                    tmp->fcounters[i] > mfile->fcounters[i])

+                {

+                    tmp->fcounters[i] = mfile->fcounters[i];

+                }

+                break;

+            case MPIIO_F_READ_END_TIMESTAMP:

+            case MPIIO_F_WRITE_END_TIMESTAMP:

+            case MPIIO_F_CLOSE_TIMESTAMP:

+                if(tmp->fcounters[i] == 0 || 

+                    tmp->fcounters[i] < mfile->fcounters[i])

+                {

+                    tmp->fcounters[i] = mfile->fcounters[i];

+                }

+                break;

+            case MPIIO_F_FASTEST_RANK_TIME:

+            case MPIIO_F_SLOWEST_RANK_TIME:

+            case MPIIO_F_VARIANCE_RANK_TIME:

+            case MPIIO_F_VARIANCE_RANK_BYTES:

+                tmp->fcounters[i] = 0;

+                break;

+            case MPIIO_F_MAX_READ_TIME:

+                if(tmp->fcounters[i] < mfile->fcounters[i])

+                {

+                    tmp->fcounters[i] = mfile->fcounters[i];

+                    tmp->counters[MPIIO_MAX_READ_TIME_SIZE] =

+                        mfile->counters[MPIIO_MAX_READ_TIME_SIZE];

+                }

+                break;

+            case MPIIO_F_MAX_WRITE_TIME:

+                if(tmp->fcounters[i] < mfile->fcounters[i])

+                {

+                    tmp->fcounters[i] = mfile->fcounters[i];

+                    tmp->counters[MPIIO_MAX_WRITE_TIME_SIZE] =

+                        mfile->counters[MPIIO_MAX_WRITE_TIME_SIZE];

+                }

+                break;

+            default:

+                tmp->fcounters[i] += mfile->fcounters[i];

+                break;

+        }

+    }

</span> 

<span style="color: #000000;background-color: #ffdddd">-        w = (curr->counters[CP_POSIX_WRITES]+

-             curr->counters[CP_POSIX_FWRITES]+

-             curr->counters[CP_INDEP_WRITES]+

-             curr->counters[CP_COLL_WRITES]+

-             curr->counters[CP_SPLIT_WRITES]+

-             curr->counters[CP_NB_WRITES]);

</span><span style="color: #000000;background-color: #ddffdd">+    return;

+}

</span> 

<span style="color: #000000;background-color: #ffdddd">-        fdata->total += 1;

-        fdata->total_size += max;

-        fdata->total_max = max(fdata->total_max, max);

</span><span style="color: #000000;background-color: #ddffdd">+void posix_accum_perf(struct darshan_posix_file *pfile,

+                      perf_data_t *pdata)

+{

+    pdata->total_bytes += pfile->counters[POSIX_BYTES_READ] +

+                          pfile->counters[POSIX_BYTES_WRITTEN];

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if (r && !w)

</span><span style="color: #000000;background-color: #ddffdd">+    /*

+     * Calculation of Shared File Time

+     *   Four Methods!!!!

+     *     by_cumul: sum time counters and divide by nprocs

+     *               (inaccurate if lots of variance between procs)

+     *     by_open: difference between timestamp of open and close

+     *              (inaccurate if file is left open without i/o happening)

+     *     by_open_lastio: difference between timestamp of open and the

+     *                     timestamp of last i/o

+     *                     (similar to above but fixes case where file is left

+     *                      open after io is complete)

+     *     by_slowest: use slowest rank time from log data

+     *                 (most accurate but requires newer log version)

+     */

+    if(pfile->rank == -1)

+    {

+        /* by_open */

+        if(pfile->fcounters[POSIX_F_CLOSE_TIMESTAMP] >

+            pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fdata->read_only += 1;

-            fdata->read_only_size += max;

-            fdata->read_only_max = max(fdata->read_only_max, max);

</span><span style="color: #000000;background-color: #ddffdd">+            pdata->shared_time_by_open +=

+                pfile->fcounters[POSIX_F_CLOSE_TIMESTAMP] -

+                pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if (!r && w)

</span><span style="color: #000000;background-color: #ddffdd">+        /* by_open_lastio */

+        if(pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] >

+            pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP])

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fdata->write_only += 1;

-            fdata->write_only_size += max;

-            fdata->write_only_max = max(fdata->write_only_max, max);

</span><span style="color: #000000;background-color: #ddffdd">+            /* be careful: file may have been opened but not read or written */

+            if(pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] > pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])

+            {

+                pdata->shared_time_by_open_lastio += 

+                    pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] - 

+                    pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];

+            }

</span>         }

<span style="color: #000000;background-color: #ffdddd">-

-        if (r && w)

</span><span style="color: #000000;background-color: #ddffdd">+        else

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            fdata->read_write += 1;

-            fdata->read_write_size += max;

-            fdata->read_write_max = max(fdata->read_write_max, max);

</span><span style="color: #000000;background-color: #ddffdd">+            /* be careful: file may have been opened but not read or written */

+            if(pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP] > pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])

+            {

+                pdata->shared_time_by_open_lastio += 

+                    pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP] - 

+                    pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];

+            }

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))

-        {

-            fdata->shared += 1;

-            fdata->shared_size += max;

-            fdata->shared_max = max(fdata->shared_max, max);

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        pdata->shared_time_by_cumul +=

+            pfile->fcounters[POSIX_F_META_TIME] +

+            pfile->fcounters[POSIX_F_READ_TIME] +

+            pfile->fcounters[POSIX_F_WRITE_TIME];

+        pdata->shared_meta_time += pfile->fcounters[POSIX_F_META_TIME];

</span> 

<span style="color: #000000;background-color: #ffdddd">-        if ((curr->type & (FILETYPE_UNIQUE)))

-        {

-            fdata->unique += 1;

-            fdata->unique_size += max;

-            fdata->unique_max = max(fdata->unique_max, max);

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        /* by_slowest */

+        pdata->shared_time_by_slowest +=

+            pfile->fcounters[POSIX_F_SLOWEST_RANK_TIME];

+    }

+

+    /*

+     * Calculation of Unique File Time

+     *   record the data for each file and sum it 

+     */

+    else

+    {

+        pdata->rank_cumul_io_time[pfile->rank] +=

+            (pfile->fcounters[POSIX_F_META_TIME] +

+            pfile->fcounters[POSIX_F_READ_TIME] +

+            pfile->fcounters[POSIX_F_WRITE_TIME]);

+        pdata->rank_cumul_md_time[pfile->rank] += pfile->fcounters[POSIX_F_META_TIME];

</span>     }

     return;

 }

<span style="color: #000000;background-color: #ffdddd">-void accum_perf(struct darshan_file *dfile,

-                hash_entry_t *hfile,

-                perf_data_t *pdata)

</span><span style="color: #000000;background-color: #ddffdd">+void mpiio_accum_perf(struct darshan_mpiio_file *mfile,

+                      perf_data_t *pdata)

</span> {

<span style="color: #000000;background-color: #ffdddd">-    int64_t mpi_file;

-

-    pdata->total_bytes += dfile->counters[CP_BYTES_READ] +

-                          dfile->counters[CP_BYTES_WRITTEN];

-

-    mpi_file = dfile->counters[CP_INDEP_OPENS] +

-               dfile->counters[CP_COLL_OPENS];

</span><span style="color: #000000;background-color: #ddffdd">+    pdata->total_bytes += mfile->counters[MPIIO_BYTES_READ] +

+                          mfile->counters[MPIIO_BYTES_WRITTEN];

</span> 

     /*

      * Calculation of Shared File Time

<span style="color: #aaaaaa">@@ -964,61 +1114,49 @@ void accum_perf(struct darshan_file *dfile,

</span>      *     by_slowest: use slowest rank time from log data

      *                 (most accurate but requires newer log version)

      */

<span style="color: #000000;background-color: #ffdddd">-    if (dfile->rank == -1)

</span><span style="color: #000000;background-color: #ddffdd">+    if(mfile->rank == -1)

</span>     {

<span style="color: #000000;background-color: #ffdddd">-        /* by_open (same for MPI or POSIX) */

-        if (dfile->fcounters[CP_F_CLOSE_TIMESTAMP] >

-            dfile->fcounters[CP_F_OPEN_TIMESTAMP])

</span><span style="color: #000000;background-color: #ddffdd">+        /* by_open */

+        if(mfile->fcounters[MPIIO_F_CLOSE_TIMESTAMP] >

+            mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])

</span>         {

             pdata->shared_time_by_open +=

<span style="color: #000000;background-color: #ffdddd">-                dfile->fcounters[CP_F_CLOSE_TIMESTAMP] -

-                dfile->fcounters[CP_F_OPEN_TIMESTAMP];

</span><span style="color: #000000;background-color: #ddffdd">+                mfile->fcounters[MPIIO_F_CLOSE_TIMESTAMP] -

+                mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        /* by_open_lastio (same for MPI or POSIX) */

-        if (dfile->fcounters[CP_F_READ_END_TIMESTAMP] >

-            dfile->fcounters[CP_F_WRITE_END_TIMESTAMP])

</span><span style="color: #000000;background-color: #ddffdd">+        /* by_open_lastio */

+        if(mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] >

+            mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP])

</span>         {

             /* be careful: file may have been opened but not read or written */

<span style="color: #000000;background-color: #ffdddd">-            if(dfile->fcounters[CP_F_READ_END_TIMESTAMP] > dfile->fcounters[CP_F_OPEN_TIMESTAMP])

</span><span style="color: #000000;background-color: #ddffdd">+            if(mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] > mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])

</span>             {

                 pdata->shared_time_by_open_lastio += 

<span style="color: #000000;background-color: #ffdddd">-                    dfile->fcounters[CP_F_READ_END_TIMESTAMP] - 

-                    dfile->fcounters[CP_F_OPEN_TIMESTAMP];

</span><span style="color: #000000;background-color: #ddffdd">+                    mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] - 

+                    mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];

</span>             }

         }

         else

         {

             /* be careful: file may have been opened but not read or written */

<span style="color: #000000;background-color: #ffdddd">-            if(dfile->fcounters[CP_F_WRITE_END_TIMESTAMP] > dfile->fcounters[CP_F_OPEN_TIMESTAMP])

</span><span style="color: #000000;background-color: #ddffdd">+            if(mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] > mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])

</span>             {

                 pdata->shared_time_by_open_lastio += 

<span style="color: #000000;background-color: #ffdddd">-                    dfile->fcounters[CP_F_WRITE_END_TIMESTAMP] - 

-                    dfile->fcounters[CP_F_OPEN_TIMESTAMP];

</span><span style="color: #000000;background-color: #ddffdd">+                    mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] - 

+                    mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];

</span>             }

         }

<span style="color: #000000;background-color: #ffdddd">-        /* by_cumul */

-        if (mpi_file)

-        {

-            pdata->shared_time_by_cumul +=

-                dfile->fcounters[CP_F_MPI_META_TIME] +

-                dfile->fcounters[CP_F_MPI_READ_TIME] +

-                dfile->fcounters[CP_F_MPI_WRITE_TIME];

-            pdata->shared_meta_time += dfile->fcounters[CP_F_MPI_META_TIME];

-        }

-        else

-        {

-            pdata->shared_time_by_cumul +=

-                dfile->fcounters[CP_F_POSIX_META_TIME] +

-                dfile->fcounters[CP_F_POSIX_READ_TIME] +

-                dfile->fcounters[CP_F_POSIX_WRITE_TIME];

-            pdata->shared_meta_time += dfile->fcounters[CP_F_POSIX_META_TIME];

-        }

</span><span style="color: #000000;background-color: #ddffdd">+        pdata->shared_time_by_cumul +=

+            mfile->fcounters[MPIIO_F_META_TIME] +

+            mfile->fcounters[MPIIO_F_READ_TIME] +

+            mfile->fcounters[MPIIO_F_WRITE_TIME];

+        pdata->shared_meta_time += mfile->fcounters[MPIIO_F_META_TIME];

</span> 

<span style="color: #000000;background-color: #ffdddd">-        /* by_slowest (same for MPI or POSIX) */

</span><span style="color: #000000;background-color: #ddffdd">+        /* by_slowest */

</span>         pdata->shared_time_by_slowest +=

<span style="color: #000000;background-color: #ffdddd">-            dfile->fcounters[CP_F_SLOWEST_RANK_TIME];

</span><span style="color: #000000;background-color: #ddffdd">+            mfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];

</span>     }

     /*

<span style="color: #aaaaaa">@@ -1027,38 +1165,169 @@ void accum_perf(struct darshan_file *dfile,

</span>      */

     else

     {

<span style="color: #000000;background-color: #ffdddd">-        if (mpi_file)

</span><span style="color: #000000;background-color: #ddffdd">+        pdata->rank_cumul_io_time[mfile->rank] +=

+            (mfile->fcounters[MPIIO_F_META_TIME] +

+            mfile->fcounters[MPIIO_F_READ_TIME] +

+            mfile->fcounters[MPIIO_F_WRITE_TIME]);

+        pdata->rank_cumul_md_time[mfile->rank] += mfile->fcounters[MPIIO_F_META_TIME];

+    }

+

+    return;

+}

+

+void posix_calc_file(hash_entry_t *file_hash, 

+                     file_data_t *fdata)

+{

+    hash_entry_t *curr = NULL;

+    hash_entry_t *tmp = NULL;

+    struct darshan_posix_file *file_rec;

+

+    memset(fdata, 0, sizeof(*fdata));

+    HASH_ITER(hlink, file_hash, curr, tmp)

+    {

+        int64_t bytes;

+        int64_t r;

+        int64_t w;

+

+        file_rec = (struct darshan_posix_file*)curr->rec_dat;

+        assert(file_rec);

+

+        bytes = file_rec->counters[POSIX_BYTES_READ] +

+                file_rec->counters[POSIX_BYTES_WRITTEN];

+

+        r = (file_rec->counters[POSIX_READS]+

+             file_rec->counters[POSIX_FREADS]);

+

+        w = (file_rec->counters[POSIX_WRITES]+

+             file_rec->counters[POSIX_FWRITES]);

+

+        fdata->total += 1;

+        fdata->total_size += bytes;

+        fdata->total_max = max(fdata->total_max, bytes);

+

+        if (r && !w)

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            pdata->rank_cumul_io_time[dfile->rank] += dfile->fcounters[CP_F_MPI_META_TIME] +

-                                dfile->fcounters[CP_F_MPI_READ_TIME] +

-                                dfile->fcounters[CP_F_MPI_WRITE_TIME];

-            pdata->rank_cumul_md_time[dfile->rank] += dfile->fcounters[CP_F_MPI_META_TIME];

</span><span style="color: #000000;background-color: #ddffdd">+            fdata->read_only += 1;

+            fdata->read_only_size += bytes;

+            fdata->read_only_max = max(fdata->read_only_max, bytes);

</span>         }

<span style="color: #000000;background-color: #ffdddd">-        else

</span><span style="color: #000000;background-color: #ddffdd">+

+        if (!r && w)

+        {

+            fdata->write_only += 1;

+            fdata->write_only_size += bytes;

+            fdata->write_only_max = max(fdata->write_only_max, bytes);

+        }

+

+        if (r && w)

+        {

+            fdata->read_write += 1;

+            fdata->read_write_size += bytes;

+            fdata->read_write_max = max(fdata->read_write_max, bytes);

+        }

+

+        if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))

+        {

+            fdata->shared += 1;

+            fdata->shared_size += bytes;

+            fdata->shared_max = max(fdata->shared_max, bytes);

+        }

+

+        if ((curr->type & (FILETYPE_UNIQUE)))

+        {

+            fdata->unique += 1;

+            fdata->unique_size += bytes;

+            fdata->unique_max = max(fdata->unique_max, bytes);

+        }

+    }

+

+    return;

+}

+

+void mpiio_calc_file(hash_entry_t *file_hash, 

+                     file_data_t *fdata)

+{

+    hash_entry_t *curr = NULL;

+    hash_entry_t *tmp = NULL;

+    struct darshan_mpiio_file *file_rec;

+

+    memset(fdata, 0, sizeof(*fdata));

+    HASH_ITER(hlink, file_hash, curr, tmp)

+    {

+        int64_t bytes;

+        int64_t r;

+        int64_t w;

+

+        file_rec = (struct darshan_mpiio_file*)curr->rec_dat;

+        assert(file_rec);

+

+        bytes = file_rec->counters[MPIIO_BYTES_READ] +

+                file_rec->counters[MPIIO_BYTES_WRITTEN];

+

+        r = (file_rec->counters[MPIIO_INDEP_READS]+

+             file_rec->counters[MPIIO_COLL_READS] +

+             file_rec->counters[MPIIO_SPLIT_READS] +

+             file_rec->counters[MPIIO_NB_READS]);

+

+        w = (file_rec->counters[MPIIO_INDEP_WRITES]+

+             file_rec->counters[MPIIO_COLL_WRITES] +

+             file_rec->counters[MPIIO_SPLIT_WRITES] +

+             file_rec->counters[MPIIO_NB_WRITES]);

+

+        fdata->total += 1;

+        fdata->total_size += bytes;

+        fdata->total_max = max(fdata->total_max, bytes);

+

+        if (r && !w)

+        {

+            fdata->read_only += 1;

+            fdata->read_only_size += bytes;

+            fdata->read_only_max = max(fdata->read_only_max, bytes);

+        }

+

+        if (!r && w)

+        {

+            fdata->write_only += 1;

+            fdata->write_only_size += bytes;

+            fdata->write_only_max = max(fdata->write_only_max, bytes);

+        }

+

+        if (r && w)

+        {

+            fdata->read_write += 1;

+            fdata->read_write_size += bytes;

+            fdata->read_write_max = max(fdata->read_write_max, bytes);

+        }

+

+        if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))

</span>         {

<span style="color: #000000;background-color: #ffdddd">-            pdata->rank_cumul_io_time[dfile->rank] += dfile->fcounters[CP_F_POSIX_META_TIME] +

-                                dfile->fcounters[CP_F_POSIX_READ_TIME] +

-                                dfile->fcounters[CP_F_POSIX_WRITE_TIME];

-            pdata->rank_cumul_md_time[dfile->rank] += dfile->fcounters[CP_F_POSIX_META_TIME];

</span><span style="color: #000000;background-color: #ddffdd">+            fdata->shared += 1;

+            fdata->shared_size += bytes;

+            fdata->shared_max = max(fdata->shared_max, bytes);

+        }

</span> 

<span style="color: #000000;background-color: #ddffdd">+        if ((curr->type & (FILETYPE_UNIQUE)))

+        {

+            fdata->unique += 1;

+            fdata->unique_size += bytes;

+            fdata->unique_max = max(fdata->unique_max, bytes);

</span>         }

     }

     return;

 }

<span style="color: #000000;background-color: #ffdddd">-void calc_perf(struct darshan_job *djob,

-               hash_entry_t *hash_rank_uniq,

-               perf_data_t *pdata)

</span><span style="color: #000000;background-color: #ddffdd">+void calc_perf(perf_data_t *pdata,

+               int64_t nprocs)

</span> {

     int64_t i;

     pdata->shared_time_by_cumul =

<span style="color: #000000;background-color: #ffdddd">-        pdata->shared_time_by_cumul / (double)djob->nprocs;

</span><span style="color: #000000;background-color: #ddffdd">+        pdata->shared_time_by_cumul / (double)nprocs;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    pdata->shared_meta_time = pdata->shared_meta_time / (double)djob->nprocs;

</span><span style="color: #000000;background-color: #ddffdd">+    pdata->shared_meta_time = pdata->shared_meta_time / (double)nprocs;

</span> 

<span style="color: #000000;background-color: #ffdddd">-    for (i=0; i<djob->nprocs; i++)

</span><span style="color: #000000;background-color: #ddffdd">+    for (i=0; i<nprocs; i++)

</span>     {

         if (pdata->rank_cumul_io_time[i] > pdata->slowest_rank_time)

         {

<span style="color: #aaaaaa">@@ -1091,3 +1360,232 @@ void calc_perf(struct darshan_job *djob,

</span> 

     return;

 }

<span style="color: #000000;background-color: #ddffdd">+

+void posix_print_total_file(struct darshan_posix_file *pfile)

+{

+    int i;

+    printf("\n");

+    for(i = 0; i < POSIX_NUM_INDICES; i++)

+    {

+        printf("total_%s: %"PRId64"\n",

+            posix_counter_names[i], pfile->counters[i]);

+    }

+    for(i = 0; i < POSIX_F_NUM_INDICES; i++)

+    {

+        printf("total_%s: %lf\n",

+            posix_f_counter_names[i], pfile->fcounters[i]);

+    }

+    return;

+}

+

+void mpiio_print_total_file(struct darshan_mpiio_file *mfile)

+{

+    int i;

+    printf("\n");

+    for(i = 0; i < MPIIO_NUM_INDICES; i++)

+    {

+        printf("total_%s: %"PRId64"\n",

+            mpiio_counter_names[i], mfile->counters[i]);

+    }

+    for(i = 0; i < MPIIO_F_NUM_INDICES; i++)

+    {

+        printf("total_%s: %lf\n",

+            mpiio_f_counter_names[i], mfile->fcounters[i]);

+    }

+    return;

+}

+

+void posix_file_list(hash_entry_t *file_hash,

+                     struct darshan_record_ref *rec_hash,

+                     int detail_flag)

+{

+    hash_entry_t *curr = NULL;

+    hash_entry_t *tmp = NULL;

+    struct darshan_posix_file *file_rec = NULL;

+    struct darshan_record_ref *ref = NULL;

+    int i;

+

+    /* list of columns:

+     *

+     * normal mode

+     * - file id

+     * - file name

+     * - nprocs

+     * - slowest I/O time

+     * - average cumulative I/O time

+     *

+     * detailed mode

+     * - first open

+     * - first read

+     * - first write

+     * - last read

+     * - last write

+     * - last close

+     * - POSIX opens

+     * - r histogram

+     * - w histogram

+     */

+

+    if(detail_flag)

+        printf("\n# Per-file summary of I/O activity (detailed).\n");

+    else

+        printf("\n# Per-file summary of I/O activity.\n");

+

+    printf("# <record_id>: darshan record id for this file\n");

+    printf("# <file_name>: full file name\n");

+    printf("# <nprocs>: number of processes that opened the file\n");

+    printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");

+    printf("# <avg>: average time in seconds consumed in IO per process\n");

+    if(detail_flag)

+    {

+        printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");

+        printf("# <end_{read/write/close}>: end timestamp of last read, write, or close\n");

+        printf("# <posix_opens>: POSIX open calls\n");

+        printf("# <POSIX_SIZE_READ_*>: POSIX read size histogram\n");

+        printf("# <POSIX_SIZE_WRITE_*>: POSIX write size histogram\n");

+    }

+    

+    printf("\n# <record_id>\t<file_name>\t<nprocs>\t<slowest>\t<avg>");

+    if(detail_flag)

+    {

+        printf("\t<start_open>\t<start_read>\t<start_write>");

+        printf("\t<end_read>\t<end_write>\t<end_close>\t<posix_opens>");

+        for(i=POSIX_SIZE_READ_0_100; i<= POSIX_SIZE_WRITE_1G_PLUS; i++)

+            printf("\t<%s>", posix_counter_names[i]);

+    }

+    printf("\n");

+

+    HASH_ITER(hlink, file_hash, curr, tmp)

+    {

+        file_rec = (struct darshan_posix_file*)curr->rec_dat;

+        assert(file_rec);

+

+        HASH_FIND(hlink, rec_hash, &(curr->rec_id), sizeof(darshan_record_id), ref);

+        assert(ref);

+

+        printf("%" PRIu64 "\t%s\t%" PRId64 "\t%f\t%f",

+            curr->rec_id,

+            ref->rec.name,

+            curr->procs,

+            curr->slowest_time,

+            curr->cumul_time/(double)curr->procs);

+

+        if(detail_flag)

+        {

+            for(i=POSIX_F_OPEN_TIMESTAMP; i<=POSIX_F_CLOSE_TIMESTAMP; i++)

+            {

+                printf("\t%f", file_rec->fcounters[i]);

+            }

+            printf("\t%" PRId64, file_rec->counters[POSIX_OPENS]);

+            for(i=POSIX_SIZE_READ_0_100; i<= POSIX_SIZE_WRITE_1G_PLUS; i++)

+                printf("\t%" PRId64, file_rec->counters[i]);

+        }

+        printf("\n");

+    }

+

+    return;

+}

+

+void mpiio_file_list(hash_entry_t *file_hash,

+                     struct darshan_record_ref *rec_hash,

+                     int detail_flag)

+{

+    hash_entry_t *curr = NULL;

+    hash_entry_t *tmp = NULL;

+    struct darshan_mpiio_file *file_rec = NULL;

+    struct darshan_record_ref *ref = NULL;

+    int i;

+

+    /* list of columns:

+     *

+     * normal mode

+     * - file id

+     * - file name

+     * - nprocs

+     * - slowest I/O time

+     * - average cumulative I/O time

+     *

+     * detailed mode

+     * - first open

+     * - first read

+     * - first write

+     * - last read

+     * - last write

+     * - last close

+     * - MPI indep opens

+     * - MPI coll opens

+     * - r histogram

+     * - w histogram

+     */

+

+    if(detail_flag)

+        printf("\n# Per-file summary of I/O activity (detailed).\n");

+    else

+        printf("\n# Per-file summary of I/O activity.\n");

+

+    printf("# <record_id>: darshan record id for this file\n");

+    printf("# <file_name>: full file name\n");

+    printf("# <nprocs>: number of processes that opened the file\n");

+    printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");

+    printf("# <avg>: average time in seconds consumed in IO per process\n");

+    if(detail_flag)

+    {

+        printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");

+        printf("# <end_{read/write/close}>: end timestamp of last read, write, or close\n");

+        printf("# <mpi_indep_opens>: independent MPI_File_open calls\n");

+        printf("# <mpi_coll_opens>: collective MPI_File_open calls\n");

+        printf("# <MPIIO_SIZE_READ_AGG_*>: MPI-IO aggregate read size histogram\n");

+        printf("# <MPIIO_SIZE_WRITE_AGG_*>: MPI-IO aggregate write size histogram\n");

+    }

+    

+    printf("\n# <record_id>\t<file_name>\t<nprocs>\t<slowest>\t<avg>");

+    if(detail_flag)

+    {

+        printf("\t<start_open>\t<start_read>\t<start_write>");

+        printf("\t<end_read>\t<end_write>\t<end_close>");

+        printf("\t<mpi_indep_opens>\t<mpi_coll_opens>");

+        for(i=MPIIO_SIZE_READ_AGG_0_100; i<= MPIIO_SIZE_WRITE_AGG_1G_PLUS; i++)

+            printf("\t<%s>", mpiio_counter_names[i]);

+    }

+    printf("\n");

+

+    HASH_ITER(hlink, file_hash, curr, tmp)

+    {

+        file_rec = (struct darshan_mpiio_file*)curr->rec_dat;

+        assert(file_rec);

+

+        HASH_FIND(hlink, rec_hash, &(curr->rec_id), sizeof(darshan_record_id), ref);

+        assert(ref);

+

+        printf("%" PRIu64 "\t%s\t%" PRId64 "\t%f\t%f",

+            curr->rec_id,

+            ref->rec.name,

+            curr->procs,

+            curr->slowest_time,

+            curr->cumul_time/(double)curr->procs);

+

+        if(detail_flag)

+        {

+            for(i=MPIIO_F_OPEN_TIMESTAMP; i<=MPIIO_F_CLOSE_TIMESTAMP; i++)

+            {

+                printf("\t%f", file_rec->fcounters[i]);

+            }

+            printf("\t%" PRId64 "\t%" PRId64, file_rec->counters[MPIIO_INDEP_OPENS],

+                file_rec->counters[MPIIO_COLL_OPENS]);

+            for(i=MPIIO_SIZE_READ_AGG_0_100; i<= MPIIO_SIZE_WRITE_AGG_1G_PLUS; i++)

+                printf("\t%" PRId64, file_rec->counters[i]);

+        }

+        printf("\n");

+    }

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-173'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-173'>

<strong>

darshan-util/darshan-pnetcdf-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-pnetcdf-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,212 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* counter name strings for the PNETCDF module */

+#define X(a) #a,

+char *pnetcdf_counter_names[] = {

+    PNETCDF_COUNTERS

+};

+

+char *pnetcdf_f_counter_names[] = {

+    PNETCDF_F_COUNTERS

+};

+#undef X

+

+static int darshan_log_get_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf, int ver);

+static void darshan_log_print_pnetcdf_file(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_pnetcdf_description(void);

+static void darshan_log_print_pnetcdf_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+struct darshan_mod_logutil_funcs pnetcdf_logutils =

+{

+    .log_get_record = &darshan_log_get_pnetcdf_file,

+    .log_put_record = &darshan_log_put_pnetcdf_file,

+    .log_print_record = &darshan_log_print_pnetcdf_file,

+    .log_print_description = &darshan_log_print_pnetcdf_description,

+    .log_print_diff = &darshan_log_print_pnetcdf_file_diff

+};

+

+static int darshan_log_get_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf,

+    darshan_record_id* rec_id)

+{

+    struct darshan_pnetcdf_file *file;

+    int i;

+    int ret;

+

+    ret = darshan_log_getmod(fd, DARSHAN_PNETCDF_MOD, pnetcdf_buf,

+        sizeof(struct darshan_pnetcdf_file));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_pnetcdf_file))

+        return(0);

+    else

+    {

+        file = (struct darshan_pnetcdf_file *)pnetcdf_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&file->f_id);

+            DARSHAN_BSWAP64(&file->rank);

+            for(i=0; i<PNETCDF_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->counters[i]);

+            for(i=0; i<PNETCDF_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->fcounters[i]);

+        }

+

+        *rec_id = file->f_id;

+        return(1);

+    }

+}

+

+static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf, int ver)

+{

+    struct darshan_pnetcdf_file *file = (struct darshan_pnetcdf_file *)pnetcdf_buf;

+    int ret;

+

+    ret = darshan_log_putmod(fd, DARSHAN_PNETCDF_MOD, file,

+        sizeof(struct darshan_pnetcdf_file), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+static void darshan_log_print_pnetcdf_file(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_pnetcdf_file *pnetcdf_file_rec =

+        (struct darshan_pnetcdf_file *)file_rec;

+

+    for(i=0; i<PNETCDF_NUM_INDICES; i++)

+    {

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+            pnetcdf_file_rec->rank, pnetcdf_file_rec->f_id, pnetcdf_counter_names[i],

+            pnetcdf_file_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<PNETCDF_F_NUM_INDICES; i++)

+    {

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+            pnetcdf_file_rec->rank, pnetcdf_file_rec->f_id, pnetcdf_f_counter_names[i],

+            pnetcdf_file_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+static void darshan_log_print_pnetcdf_description()

+{

+    printf("\n# description of PNETCDF counters:\n");

+    printf("#   PNETCDF_INDEP_OPENS: PNETCDF independent file open operation counts.\n");

+    printf("#   PNETCDF_COLL_OPENS: PNETCDF collective file open operation counts.\n");

+    printf("#   PNETCDF_F_OPEN_TIMESTAMP: timestamp of first PNETCDF file open.\n");

+    printf("#   PNETCDF_F_CLOSE_TIMESTAMP: timestamp of last PNETCDF file close.\n");

+

+    DARSHAN_PRINT_HEADER();

+

+    return;

+}

+

+static void darshan_log_print_pnetcdf_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_pnetcdf_file *file1 = (struct darshan_pnetcdf_file *)file_rec1;

+    struct darshan_pnetcdf_file *file2 = (struct darshan_pnetcdf_file *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<PNETCDF_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file1->rank, file1->f_id, pnetcdf_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file2->rank, file2->f_id, pnetcdf_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file1->rank, file1->f_id, pnetcdf_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file2->rank, file2->f_id, pnetcdf_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<PNETCDF_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file1->rank, file1->f_id, pnetcdf_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file2->rank, file2->f_id, pnetcdf_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file1->rank, file1->f_id, pnetcdf_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],

+                file2->rank, file2->f_id, pnetcdf_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-174'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-174'>

<strong>

darshan-util/darshan-pnetcdf-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-pnetcdf-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,15 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_PNETCDF_LOG_UTILS_H

+#define __DARSHAN_PNETCDF_LOG_UTILS_H

+

+extern char *pnetcdf_counter_names[];

+extern char *pnetcdf_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs pnetcdf_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-175'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-175'>

<strong>

darshan-util/darshan-posix-logutils.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-posix-logutils.c

</span><span style="color: #aaaaaa">@@ -0,0 +1,234 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#define _GNU_SOURCE

+#include "darshan-util-config.h"

+#include <stdio.h>

+#include <string.h>

+#include <assert.h>

+#include <stdlib.h>

+#include <unistd.h>

+#include <inttypes.h>

+#include <sys/types.h>

+#include <sys/stat.h>

+#include <fcntl.h>

+#include <errno.h>

+

+#include "darshan-logutils.h"

+

+/* counter name strings for the POSIX module */

+#define X(a) #a,

+char *posix_counter_names[] = {

+    POSIX_COUNTERS

+};

+

+char *posix_f_counter_names[] = {

+    POSIX_F_COUNTERS

+};

+#undef X

+

+static int darshan_log_get_posix_file(darshan_fd fd, void* posix_buf,

+    darshan_record_id* rec_id);

+static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf, int ver);

+static void darshan_log_print_posix_file(void *file_rec,

+    char *file_name, char *mnt_pt, char *fs_type, int ver);

+static void darshan_log_print_posix_description(void);

+static void darshan_log_print_posix_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2);

+

+struct darshan_mod_logutil_funcs posix_logutils =

+{

+    .log_get_record = &darshan_log_get_posix_file,

+    .log_put_record = &darshan_log_put_posix_file,

+    .log_print_record = &darshan_log_print_posix_file,

+    .log_print_description = &darshan_log_print_posix_description,

+    .log_print_diff = &darshan_log_print_posix_file_diff

+};

+

+static int darshan_log_get_posix_file(darshan_fd fd, void* posix_buf, 

+    darshan_record_id* rec_id)

+{

+    struct darshan_posix_file *file;

+    int i;

+    int ret;

+

+    ret = darshan_log_getmod(fd, DARSHAN_POSIX_MOD, posix_buf,

+        sizeof(struct darshan_posix_file));

+    if(ret < 0)

+        return(-1);

+    else if(ret < sizeof(struct darshan_posix_file))

+        return(0);

+    else

+    {

+        file = (struct darshan_posix_file *)posix_buf;

+        if(fd->swap_flag)

+        {

+            /* swap bytes if necessary */

+            DARSHAN_BSWAP64(&file->f_id);

+            DARSHAN_BSWAP64(&file->rank);

+            for(i=0; i<POSIX_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->counters[i]);

+            for(i=0; i<POSIX_F_NUM_INDICES; i++)

+                DARSHAN_BSWAP64(&file->fcounters[i]);

+        }

+

+        *rec_id = file->f_id;

+        return(1);

+    }

+}

+

+static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf, int ver)

+{

+    struct darshan_posix_file *file = (struct darshan_posix_file *)posix_buf;

+    int ret;

+

+    ret = darshan_log_putmod(fd, DARSHAN_POSIX_MOD, file,

+        sizeof(struct darshan_posix_file), ver);

+    if(ret < 0)

+        return(-1);

+

+    return(0);

+}

+

+static void darshan_log_print_posix_file(void *file_rec, char *file_name,

+    char *mnt_pt, char *fs_type, int ver)

+{

+    int i;

+    struct darshan_posix_file *posix_file_rec =

+        (struct darshan_posix_file *)file_rec;

+

+    for(i=0; i<POSIX_NUM_INDICES; i++)

+    {

+        DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+            posix_file_rec->rank, posix_file_rec->f_id, posix_counter_names[i],

+            posix_file_rec->counters[i], file_name, mnt_pt, fs_type);

+    }

+

+    for(i=0; i<POSIX_F_NUM_INDICES; i++)

+    {

+        DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+            posix_file_rec->rank, posix_file_rec->f_id, posix_f_counter_names[i],

+            posix_file_rec->fcounters[i], file_name, mnt_pt, fs_type);

+    }

+

+    return;

+}

+

+static void darshan_log_print_posix_description()

+{

+    printf("\n# description of POSIX counters:\n");

+    printf("#   POSIX_*: posix operation counts.\n");

+    printf("#   READS,WRITES,OPENS,SEEKS,STATS, and MMAPS are types of operations.\n");

+    printf("#   POSIX_MODE: mode that file was opened in.\n");

+    printf("#   POSIX_BYTES_*: total bytes read and written.\n");

+    printf("#   POSIX_MAX_BYTE_*: highest offset byte read and written.\n");

+    printf("#   POSIX_CONSEC_*: number of exactly adjacent reads and writes.\n");

+    printf("#   POSIX_SEQ_*: number of reads and writes from increasing offsets.\n");

+    printf("#   POSIX_RW_SWITCHES: number of times access alternated between read and write.\n");

+    printf("#   POSIX_*_ALIGNMENT: memory and file alignment.\n");

+    printf("#   POSIX_*_NOT_ALIGNED: number of reads and writes that were not aligned.\n");

+    printf("#   POSIX_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");

+    printf("#   POSIX_SIZE_*_*: histogram of read and write access sizes.\n");

+    printf("#   POSIX_STRIDE*_STRIDE: the four most common strides detected.\n");

+    printf("#   POSIX_STRIDE*_COUNT: count of the four most common strides.\n");

+    printf("#   POSIX_ACCESS*_ACCESS: the four most common access sizes.\n");

+    printf("#   POSIX_ACCESS*_COUNT: count of the four most common access sizes.\n");

+    printf("#   POSIX_*_RANK: rank of the processes that were the fastest and slowest at I/O (for shared files).\n");

+    printf("#   POSIX_*_RANK_BYTES: bytes transferred by the fastest and slowest ranks (for shared files).\n");

+    printf("#   POSIX_F_OPEN_TIMESTAMP: timestamp of first open.\n");

+    printf("#   POSIX_F_*_START_TIMESTAMP: timestamp of first read/write.\n");

+    printf("#   POSIX_F_*_END_TIMESTAMP: timestamp of last read/write.\n");

+    printf("#   POSIX_F_CLOSE_TIMESTAMP: timestamp of last close.\n");

+    printf("#   POSIX_F_READ/WRITE/META_TIME: cumulative time spent in read, write, or metadata operations.\n");

+    printf("#   POSIX_F_MAX_*_TIME: duration of the slowest read and write operations.\n");

+    printf("#   POSIX_F_*_RANK_TIME: fastest and slowest I/O time for a single rank (for shared files).\n");

+    printf("#   POSIX_F_VARIANCE_RANK_*: variance of total I/O time and bytes moved for all ranks (for shared files).\n");

+

+    DARSHAN_PRINT_HEADER();

+

+    return;

+}

+

+static void darshan_log_print_posix_file_diff(void *file_rec1, char *file_name1,

+    void *file_rec2, char *file_name2)

+{

+    struct darshan_posix_file *file1 = (struct darshan_posix_file *)file_rec1;

+    struct darshan_posix_file *file2 = (struct darshan_posix_file *)file_rec2;

+    int i;

+

+    /* NOTE: we assume that both input records are the same module format version */

+

+    for(i=0; i<POSIX_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file1->rank, file1->f_id, posix_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file2->rank, file2->f_id, posix_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+        else if(file1->counters[i] != file2->counters[i])

+        {

+            printf("- ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file1->rank, file1->f_id, posix_counter_names[i],

+                file1->counters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file2->rank, file2->f_id, posix_counter_names[i],

+                file2->counters[i], file_name2, "", "");

+        }

+    }

+

+    for(i=0; i<POSIX_F_NUM_INDICES; i++)

+    {

+        if(!file2)

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file1->rank, file1->f_id, posix_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            

+        }

+        else if(!file1)

+        {

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file2->rank, file2->f_id, posix_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+        else if(file1->fcounters[i] != file2->fcounters[i])

+        {

+            printf("- ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file1->rank, file1->f_id, posix_f_counter_names[i],

+                file1->fcounters[i], file_name1, "", "");

+            printf("+ ");

+            DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],

+                file2->rank, file2->f_id, posix_f_counter_names[i],

+                file2->fcounters[i], file_name2, "", "");

+        }

+    }

+

+    return;

+}

+

+/*

+ * Local variables:

+ *  c-indent-level: 4

+ *  c-basic-offset: 4

+ * End:

+ *

+ * vim: ts=8 sts=4 sw=4 expandtab

+ */

</span></code></pre>

<br>

</li>

<li id='diff-176'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-176'>

<strong>

darshan-util/darshan-posix-logutils.h

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-posix-logutils.h

</span><span style="color: #aaaaaa">@@ -0,0 +1,15 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+ * Copyright (C) 2015 University of Chicago.

+ * See COPYRIGHT notice in top-level directory.

+ *

+ */

+

+#ifndef __DARSHAN_POSIX_LOG_UTILS_H

+#define __DARSHAN_POSIX_LOG_UTILS_H

+

+extern char *posix_counter_names[];

+extern char *posix_f_counter_names[];

+

+extern struct darshan_mod_logutil_funcs posix_logutils;

+

+#endif

</span></code></pre>

<br>

</li>

<li id='diff-177'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-177'>

<strong>

darshan-util/darshan-summary-per-file.sh

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-summary-per-file.sh

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-summary-per-file.sh

</span><span style="color: #aaaaaa">@@ -1,8 +1,8 @@

</span> #!/bin/bash

 #

<span style="color: #000000;background-color: #ffdddd">-# (C) 2013 by Argonne National Laboratory.

-#     See COPYRIGHT in top-level directory.

<span style="color: #000000;background-color: #ddffdd">+# Copyright (C) 2015 University of Chicago.

+# See COPYRIGHT notice in top-level directory.

</span> #

 # change behavior of shell error code following failure of a piped command

<span style="color: #aaaaaa">@@ -14,7 +14,7 @@ if [ $# -ne 2 ]; then

</span> fi

 # count number of files present in log

<span style="color: #000000;background-color: #ffdddd">-filecount=`darshan-parser --file-list $1| egrep -v '^(#|$)' | wc -l`

</span><span style="color: #000000;background-color: #ddffdd">+filecount=`darshan-parser --file-list $1| egrep -v '^(#|$)' | cut -f 1-2 | uniq | wc -l`

</span> rc=$?

 if [ $rc -ne 0 ]; then

    exit $rc

<span style="color: #aaaaaa">@@ -29,7 +29,7 @@ fi

</span> 

 # loop through all files in log

 counter=0

<span style="color: #000000;background-color: #ffdddd">-darshan-parser --file-list $1| egrep -v '^(#|$)' | 

</span><span style="color: #000000;background-color: #ddffdd">+darshan-parser --file-list $1| egrep -v '^(#|$)' | cut -f 1-2 | uniq |

</span> while read -r hash suffix stuff ; do

         counter=$((counter+1))

        file=$(basename $suffix)

</code></pre>

<br>

</li>

<li id='diff-178'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-178'>

<strong>

darshan-util/doc/darshan-util.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/doc/darshan-util.txt

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/doc/darshan-util.txt

</span><span style="color: #aaaaaa">@@ -39,7 +39,6 @@ make

</span> make install

 ----

<span style="color: #000000;background-color: #ffdddd">-CC variable

</span> [NOTE]

 The darshan-util package is intended to be used on a login node or

 workstation.  For most use cases this means that you should

<span style="color: #aaaaaa">@@ -67,12 +66,16 @@ application will likely be found in a centralized directory, with the path

</span> and log file name in the following format:

 ----

<span style="color: #000000;background-color: #ffdddd">-<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan.gz

</span><span style="color: #000000;background-color: #ddffdd">+<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan

</span> ----

 This is a binary format file that summarizes I/O activity. As of version

 2.0.0 of Darshan, this file is portable and does not have to be analyzed on

<span style="color: #000000;background-color: #ffdddd">-the same system that executed the job. 

</span><span style="color: #000000;background-color: #ddffdd">+the same system that executed the job. Also, note that Darshan logs generated

+with Darshan versions preceding version 3.0 will have the extension `darshan.gz`

+(or `darshan.bz2` if compressed using bzip2 format). These logs are not compatible

+with Darshan 3.0 utilities, and thus must be analyzed using an appropriate version

+(2.x) of the darshan-util package.

</span> 

 === darshan-job-summary.pl

<span style="color: #aaaaaa">@@ -91,9 +94,8 @@ produce a `carns_my-app_id114525_7-27-58921_19.pdf` output file).

</span> You can also manually specify the name of the output file using the

 `--output` argument.

<span style="color: #000000;background-color: #ffdddd">-An example of the output produced by darshan-job-summary.pl can be found at

-http://www.mcs.anl.gov/research/projects/darshan/files/2012/06/pcarns_mpi-io-test_id3406_6-7-47644-13333843235489639491_1.pdf

-.

</span><span style="color: #000000;background-color: #ddffdd">+An example of the output produced by darshan-job-summary.pl can be found

+link:http://www.mcs.anl.gov/research/projects/darshan/docs/ssnyder_ior-hdf5_id3655016_9-23-29011-12333993518351519212_1.darshan.pdf[HERE].

</span> 

 === darshan-summary-per-file.sh

<span style="color: #aaaaaa">@@ -160,8 +162,6 @@ of each line in the default header section of the output:

</span> |====

 |output line | description

 | "# darshan log version" | internal version number of the Darshan log file

<span style="color: #000000;background-color: #ffdddd">-| "# size of file statistics" | uncompressed size of each file record in the binary log file

-| "# size of job statistics" |  uncompressed size of the overall job statistics in the binary log file

</span> | "# exe" | name of the executable that generated the log file

 | "# uid" | user id that the job ran as

 | "# jobid" | job id from the scheduler

<span style="color: #aaaaaa">@@ -173,6 +173,19 @@ of each line in the default header section of the output:

</span> | "# run time" | run time of the job in seconds

 |====

<span style="color: #000000;background-color: #ddffdd">+==== Log file region sizes

+

+The next portion of the parser output displays the size of each region

+contained within the given log file. Each log file will contain the

+following regions:

+

+* header - constant-sized uncompressed header providing data on how to properly access the log

+* job data - job-level metadata (e.g., start/end time and exe name) for the log

+* record table - a table mapping Darshan record identifiers to full file name paths

+* module data - each module (e.g., POSIX, MPI-IO, etc.) stores their I/O characterization data in distinct regions of the log

+

+All regions of the log file are compressed (in libz or bzip2 format), except the header.

+

</span> ==== Table of mounted file systems

 The next portion of the output shows a table of all general purpose file

<span style="color: #aaaaaa">@@ -180,140 +193,194 @@ systems that were mounted while the job was running. Each line uses the

</span> following format:

 ----

<span style="color: #000000;background-color: #ffdddd">-<device> <mount point> <fs type>

</span><span style="color: #000000;background-color: #ddffdd">+<mount point> <fs type>

</span> ----

<span style="color: #000000;background-color: #ffdddd">-The device field is the device ID as reported by the stat() system call.

-Note that this device ID may change if the node is rebooted or the file

-system is remounted.

-

</span> ==== Format of I/O characterization fields

 The remainder of the output will show characteristics for each file that was

 opened by the application. Each line uses the following format:

 ----

<span style="color: #000000;background-color: #ffdddd">-<rank> <file name hash> <counter name> <counter value> <file name suffix> <mount point> <fs type>

</span><span style="color: #000000;background-color: #ddffdd">+<module> <rank> <record id> <counter name> <counter value> <file name> <mount point> <fs type>

</span> ----

<span style="color: #000000;background-color: #ffdddd">-The `<rank>` column indicates the rank of the process that opened the file. A

-rank value of -1 indicates that all processes opened the same file. In that

-case, the value of the counter represents an aggregate across all processes. The

-`<file name hash>` is a 64 bit hash of the file path/name that was opened. It

-is used as a way to uniquely differentiate each file. The `<counter name>` is

-the name of the statistic that the line is reporting, while the `<counter

-value>` is the value of that statistic. A value of -1 indicates that Darshan

-was unable to collect statistics for that particular counter, and the value

-should be ignored.  The `<file name suffix>` shows the last

-11 characters of the file name. The `<mount point>` is the mount point of the

-file system that this file belongs to. The `<fs type>` is the type of file

-system.

</span><span style="color: #000000;background-color: #ddffdd">+The `<module>` column specifies the module responsible for recording this piece

+of I/O characterization data. The `<rank>` column indicates the rank of the process

+that opened the file. A rank value of -1 indicates that all processes opened the

+same file. In that case, the value of the counter represents an aggregate across all

+processes. The `<record id>` is a 64 bit hash of the file path/name that was opened.

+It is used as a way to uniquely differentiate each file. The `<counter name>` is

+the name of the statistic that the line is reporting, while the `<counter value>` is

+the value of that statistic. A value of -1 indicates that Darshan was unable to

+collect statistics for that particular counter, and the value should be ignored.

+The `<file name>` field shows the complete file name the record corresponds to. The

+`<mount point>` is the mount point of the file system that this file belongs to and

+`<fs type>` is the type of that file system.

</span> 

 ==== I/O characterization fields

<span style="color: #000000;background-color: #ffdddd">-The following table shows a list of integer statistics that are available

-for each file, along with a description of each.

-Unless otherwise noted, counters include all variants of the call in

-question, such a `read()`, `pread()`, and `readv()` for CP_POSIX_READS.

-

</span><span style="color: #000000;background-color: #ddffdd">+The following tables show a list of integer statistics that are available for each of

+Darshan's current instrumentation modules, along with a description of each. Unless

+otherwise noted, counters include all variants of the call in question, such as

+`read()`, `pread()`, and `readv()` for POSIX_READS.

</span> 

<span style="color: #000000;background-color: #ddffdd">+.POSIX module

</span> [cols="40%,60%",options="header"]

 |====

<span style="color: #000000;background-color: #ffdddd">-| output line | description

-| CP_POSIX_READS |  Count of POSIX read operations

-| CP_POSIX_WRITES | Count of POSIX write operations

-| CP_POSIX_OPENS |  Count of how many times the file was opened

-| CP_POSIX_SEEKS |  Count of POSIX seek operations

-| CP_POSIX_STATS |  Count of POSIX stat operations

-| CP_POSIX_MMAPS |  Count of POSIX mmap operations

-| CP_POSIX_FREADS | Count of stream read operations

-| CP_POSIX_FWRITES |    Count of stream write operations

-| CP_POSIX_FOPENS | Count of stream open operations

-| CP_POSIX_FSEEKS | Count of stream seek operations

-| CP_POSIX_FSYNCS | Count of fsync operations

-| CP_POSIX_FDSYNCS |    Count of fdatasync operations

-| CP_INDEP_OPENS |  Count of non-collective MPI opens

-| CP_COLL_OPENS |   Count of collective MPI opens

-| CP_INDEP_READS |  Count of non-collective MPI reads

-| CP_INDEP_WRITES | Count of non-collective MPI writes

-| CP_COLL_READS |   Count of collective MPI reads

-| CP_COLL_WRITES |  Count of collective MPI writes

-| CP_SPLIT_READS |  Count of MPI split collective reads

-| CP_SPLIT_WRITES | Count of MPI split collective writes

-| CP_NB_READS | Count of MPI non-blocking reads

-| CP_NB_WRITES |    Count of MPI non-blocking writes

-| CP_SYNCS |    Count of MPI file syncs

-| CP_INDEP_NC_OPENS |   Count of independent Parallel NetCDF opens

-| CP_COLL_NC_OPENS |    Count of collective Parallel NetCDF opens

-| CP_HDF5_OPENS |   Count of HDF5 opens

-| CP_COMBINER_* |   Count of each type of MPI datatype (both in memory and in file)

-| CP_HINTS |    Count of MPI file hints used

-| CP_VIEWS |    Count of MPI file views used

-| CP_MODE | Mode that the file was last opened in

-| CP_BYTES_READ |   Total number of bytes that were read from the file

-| CP_BYTES_WRITTEN |    Total number of bytes written to the file

-| CP_MAX_BYTE_READ |    Highest offset in the file that was read

-| CP_MAX_BYTE_WRITTEN | Highest offset in the file that was written

-| CP_CONSEC_READS | Number of consecutive reads (that were immediately adjacent to the previous access)

-| CP_CONSEC_WRITES |    Number of consecutive writes (that were immediately adjacent to the previous access)

-| CP_SEQ_READS |    Number of sequential reads (at a higher offset than where the previous access left off)

-| CP_SEQ_WRITES |   Number of sequential writes (at a higher offset than where the previous access left off)

-| CP_RW_SWITCHES |  Number of times that access toggled between read and write in consecutive operations

-| CP_MEM_NOT_ALIGNED |  Number of times that a read or write was not aligned in memory

-| CP_MEM_ALIGNMENT |    Memory alignment value (chosen at compile time)

-| CP_FILE_NOT_ALIGNED | Number of times that a read or write was not aligned in file

-| CP_FILE_ALIGNMENT |   File alignment value.  This value is detected at

</span><span style="color: #000000;background-color: #ddffdd">+| counter name | description

+| POSIX_OPENS | Count of how many times the file was opened

+| POSIX_READS | Count of POSIX read operations

+| POSIX_WRITES | Count of POSIX write operations

+| POSIX_SEEKS | Count of POSIX seek operations

+| POSIX_STATS | Count of POSIX stat operations

+| POSIX_MMAPS | Count of POSIX mmap operations

+| POSIX_FOPENS | Count of POSIX stream open operations

+| POSIX_FREADS | Count of POSIX stream read operations

+| POSIX_FWRITES | Count of POSIX stream write operations

+| POSIX_FSEEKS | Count of POSIX stream seek operations

+| POSIX_FSYNCS | Count of POSIX fsync operations

+| POSIX_FDSYNCS | Count of POSIX fdatasync operations

+| POSIX_MODE | Mode that the file was last opened in

+| POSIX_BYTES_READ | Total number of bytes that were read from the file

+| POSIX_BYTES_WRITTEN | Total number of bytes written to the file

+| POSIX_MAX_BYTE_READ | Highest offset in the file that was read

+| POSIX_MAX_BYTE_WRITTEN | Highest offset in the file that was written

+| POSIX_CONSEC_READS | Number of consecutive reads (that were immediately adjacent to the previous access)

+| POSIX_CONSEC_WRITES | Number of consecutive writes (that were immediately adjacent to the previous access)

+| POSIX_SEQ_READS | Number of sequential reads (at a higher offset than where the previous access left off)

+| POSIX_SEQ_WRITES | Number of sequential writes (at a higher offset than where the previous access left off)

+| POSIX_RW_SWITCHES | Number of times that access toggled between read and write in consecutive operations

+| POSIX_MEM_NOT_ALIGNED | Number of times that a read or write was not aligned in memory

+| POSIX_MEM_ALIGNMENT | Memory alignment value (chosen at compile time)

+| POSIX_FILE_NOT_ALIGNED | Number of times that a read or write was not aligned in file

+| POSIX_FILE_ALIGNMENT | File alignment value.  This value is detected at

</span> runtime on most file systems.  On Lustre, however, Darshan assumes a default

 value of 1 MiB for optimal file alignment.

<span style="color: #000000;background-color: #ffdddd">-| CP_MAX_READ_TIME_SIZE |   Size of the slowest POSIX read operation

-| CP_MAX_WRITE_TIME_SIZE |  Size of the slowest POSIX write operation

-| CP_SIZE_READ_* |  Histogram of read access sizes at POSIX level

-| CP_SIZE_READ_AGG_* |  Histogram of total size of read accesses at MPI level, even if access is noncontiguous

-| CP_EXTENT_READ_* |    Histogram of read extents

-| CP_SIZE_WRITE_* | Histogram of write access sizes at POSIX level

-| CP_SIZE_WRITE_AGG_* | Histogram of total size of write accesses at MPI level, even if access is noncontiguous

-| CP_EXTENT_WRITE_* |   Histogram of write extents

-| CP_STRIDE[1-4]_STRIDE |   Size of 4 most common stride patterns

-| CP_STRIDE[1-4]_COUNT |    Count of 4 most common stride patterns

-| CP_ACCESS[1-4]_ACCESS |   4 most common access sizes

-| CP_ACCESS[1-4]_COUNT |    Count of 4 most common access sizes

-| CP_DEVICE |   File system identifier; correlates with mount table shown earlier.  In Darshan 2.2.5 and earlier, this is the device ID reported by stat(), in Darshan 2.2.6 and later, this is an opaque identifier generated by Darshan.

-| CP_SIZE_AT_OPEN | Size of file at first open time

-| CP_FASTEST_RANK | The MPI rank of the rank with smallest time spent in I/O

-| CP_FASTEST_RANK_BYTES |   The number of bytes transferred by the rank with smallest time spent in I/O

-| CP_SLOWEST_RANK | The MPI rank of the rank with largest time spent in I/O

-| CP_SLOWEST_RANK_BYTES |   The number of bytes transferred by the rank with the largest time spent in I/O

</span><span style="color: #000000;background-color: #ddffdd">+| POSIX_MAX_READ_TIME_SIZE | Size of the slowest POSIX read operation

+| POSIX_MAX_WRITE_TIME_SIZE | Size of the slowest POSIX write operation

+| POSIX_SIZE_READ_* | Histogram of read access sizes at POSIX level

+| POSIX_SIZE_WRITE_* | Histogram of write access sizes at POSIX level

+| POSIX_STRIDE[1-4]_STRIDE | Size of 4 most common stride patterns

+| POSIX_STRIDE[1-4]_COUNT | Count of 4 most common stride patterns

+| POSIX_ACCESS[1-4]_ACCESS | 4 most common POSIX access sizes

+| POSIX_ACCESS[1-4]_COUNT | Count of 4 most common POSIX access sizes

+| POSIX_FASTEST_RANK | The MPI rank of the rank with smallest time spent in POSIX I/O

+| POSIX_FASTEST_RANK_BYTES | The number of bytes transferred by the rank with smallest time spent in POSIX I/O

+| POSIX_SLOWEST_RANK | The MPI rank of the rank with largest time spent in POSIX I/O

+| POSIX_SLOWEST_RANK_BYTES | The number of bytes transferred by the rank with the largest time spent in POSIX I/O

+| POSIX_F_OPEN_TIMESTAMP | Timestamp of first time that the file was opened

+| POSIX_F_READ_START_TIMESTAMP | Timestamp that the first POSIX read operation began

+| POSIX_F_WRITE_START_TIMESTAMP | Timestamp that the first POSIX write operation began

+| POSIX_F_READ_END_TIMESTAMP | Timestamp that the last POSIX read operation ended

+| POSIX_F_WRITE_END_TIMESTAMP | Timestamp that the last POSIX write operation ended

+| POSIX_F_CLOSE_TIMESTAMP | Timestamp of the last time that the file was closed

+| POSIX_F_READ_TIME | Cumulative time spent reading at the POSIX level

+| POSIX_F_WRITE_TIME | Cumulative time spent in write, fsync, and fdatasync at the POSIX level

+| POSIX_F_META_TIME | Cumulative time spent in open, close, stat, and seek at the POSIX level

+| POSIX_F_MAX_READ_TIME | Duration of the slowest individual POSIX read operation

+| POSIX_F_MAX_WRITE_TIME | Duration of the slowest individual POSIX write operation

+| POSIX_F_FASTEST_RANK_TIME | The time of the rank which had the smallest amount of time spent in POSIX I/O (cumulative read, write, and meta times)

+| POSIX_F_SLOWEST_RANK_TIME | The time of the rank which had the largest amount of time spent in POSIX I/O

+| POSIX_F_VARIANCE_RANK_TIME | The population variance for POSIX I/O time of all the ranks

+| POSIX_F_VARIANCE_RANK_BYTES | The population variance for bytes transferred of all the ranks

+|====

+

+.MPI-IO module

+[cols="40%,60%",options="header"]

+|====

+| counter name | description

+| MPIIO_INDEP_OPENS | Count of non-collective MPI opens

+| MPIIO_COLL_OPENS | Count of collective MPI opens

+| MPIIO_INDEP_READS | Count of non-collective MPI reads

+| MPIIO_INDEP_WRITES | Count of non-collective MPI writes

+| MPIIO_COLL_READS | Count of collective MPI reads

+| MPIIO_COLL_WRITES | Count of collective MPI writes

+| MPIIO_SPLIT_READS | Count of MPI split collective reads

+| MPIIO_SPLIT_WRITES | Count of MPI split collective writes

+| MPIIO_NB_READS | Count of MPI non-blocking reads

+| MPIIO_NB_WRITES | Count of MPI non-blocking writes

+| MPIIO_SYNCS | Count of MPI file syncs

+| MPIIO_HINTS | Count of MPI file hints used

+| MPIIO_VIEWS | Count of MPI file views used

+| MPIIO_MODE | MPI mode that the file was last opened in

+| MPIIO_BYTES_READ | Total number of bytes that were read from the file at MPI level

+| MPIIO_BYTES_WRITTEN | Total number of bytes written to the file at MPI level

+| MPIIO_RW_SWITCHES | Number of times that access toggled between read and write in consecutive MPI operations

+| MPIIO_MAX_READ_TIME_SIZE | Size of the slowest MPI read operation

+| MPIIO_MAX_WRITE_TIME_SIZE | Size of the slowest MPI write operation

+| MPIIO_SIZE_READ_AGG_* | Histogram of total size of read accesses at MPI level, even if access is noncontiguous

+| MPIIO_SIZE_WRITE_AGG_* | Histogram of total size of write accesses at MPI level, even if access is noncontiguous

+| MPIIO_ACCESS[1-4]_ACCESS | 4 most common MPI aggregate access sizes

+| MPIIO_ACCESS[1-4]_COUNT | Count of 4 most common MPI aggregate access sizes

+| MPIIO_FASTEST_RANK | The MPI rank of the rank with smallest time spent in MPI I/O

+| MPIIO_FASTEST_RANK_BYTES | The number of bytes transferred by the rank with smallest time spent in MPI I/O

+| MPIIO_SLOWEST_RANK | The MPI rank of the rank with largest time spent in MPI I/O

+| MPIIO_SLOWEST_RANK_BYTES | The number of bytes transferred by the rank with the largest time spent in MPI I/O

+| MPIIO_F_OPEN_TIMESTAMP | Timestamp of first time that the file was opened at MPI level

+| MPIIO_F_READ_START_TIMESTAMP | Timestamp that the first MPI read operation began

+| MPIIO_F_WRITE_START_TIMESTAMP | Timestamp that the first MPI write operation begin

+| MPIIO_F_READ_END_TIMESTAMP | Timestamp that the last MPI read operation ended

+| MPIIO_F_WRITE_END_TIMESTAMP | Timestamp that the last MPI write operation ended

+| MPIIO_F_CLOSE_TIMESTAMP | Timestamp of the last time that the file was closed at MPI level

+| MPIIO_READ_TIME | Cumulative time spent reading at MPI level

+| MPIIO_WRITE_TIME | Cumulative time spent write and sync at MPI level

+| MPIIO_META_TIME | Cumulative time spent in open and close at MPI level

+| MPIIO_F_MAX_READ_TIME | Duration of the slowest individual MPI read operation

+| MPIIO_F_MAX_WRITE_TIME | Duration of the slowest individual MPI write operation

+| CP_F_FASTEST_RANK_TIME | The time of the rank which had the smallest amount of time spent in MPI I/O (cumulative read, write, and meta times)

+| CP_F_SLOWEST_RANK_TIME | The time of the rank which had the largest amount of time spent in MPI I/O

+| CP_F_VARIANCE_RANK_TIME | The population variance for MPI I/O time of all the ranks

+| CP_F_VARIANCE_RANK_BYTES | The population variance for bytes transferred of all the ranks at MPI level

</span> |====

<span style="color: #000000;background-color: #ffdddd">-The following is a list of floating point statistics that are available for

-each file:

</span><span style="color: #000000;background-color: #ddffdd">+.HDF5 module

+[cols="40%,60%",options="header"]

+|====

+| counter name | description

+| HDF5_OPENS | Count of HDF5 opens

+| HDF5_F_OPEN_TIMESTAMP | Timestamp of first time that the file was opened at HDF5 level

+| HDF5_F_CLOSE_TIMESTAMP | Timestamp of the last time that the file was closed at HDF5 level

+|====

</span> 

<span style="color: #000000;background-color: #ddffdd">+.PnetCDF module

</span> [cols="40%,60%",options="header"]

 |====

<span style="color: #000000;background-color: #ffdddd">-| output line |  description

-| CP_F_OPEN_TIMESTAMP | Timestamp of first time that the file was opened

-| CP_F_CLOSE_TIMESTAMP |    Timestamp of the last time that the file was closed

-| CP_F_READ_START_TIMESTAMP |   Timestamp that the first read operation began

-| CP_F_READ_END_TIMESTAMP | Timestamp that the last read operation ended

-| CP_F_WRITE_START_TIMESTAMP |  Timestamp that the first write operation begin

-| CP_F_WRITE_END_TIMESTAMP |    Timestamp that the last write operation ended

-| CP_F_POSIX_READ_TIME |    Cumulative time spent reading at the POSIX level

-| CP_F_POSIX_WRITE_TIME |   Cumulative time spent in write, fsync, and fdatasync at the POSIX level

-| CP_F_POSIX_META_TIME |    Cumulative time spent in open, close, stat, and seek at the POSIX level

-| CP_F_MPI_META_TIME |  Cumulative time spent in open and close at the MPI-IO level

-| CP_F_MPI_READ_TIME |  Cumulative time spent reading at the MPI-IO level

-| CP_F_MPI_WRITE_TIME | Cumulative time spent write and sync at the MPI-IO level

-| CP_F_MAX_READ_TIME |  Duration of the slowest individual POSIX read operation

-| CP_F_MAX_WRITE_TIME | Duration of the slowest individual POSIX write operation

-| CP_F_FASTEST_RANK_TIME |  The time of the rank which had the smallest amount of time spent in I/O.  If the file was accessed usign MPI-IO it combines the MPI meta, read, and write time.  If the file was not accessed with MPI-IO then it combines the posix meta, read, and write time.

-| CP_F_SLOWEST_RANK_TIME |  The time of the rank which had the largest amount of time spent in I/O

-| CP_F_VARIANCE_RANK_TIME | The population variance for I/O time of all the ranks

-| CP_F_VARIANCE_RANK_BYTES |    The population variance for bytes transferred of all the ranks

</span><span style="color: #000000;background-color: #ddffdd">+| counter name | description

+| PNETCDF_INDEP_OPENS | Count of PnetCDF independent opens

+| PNETCDF_COLL_OPENS | Count of PnetCDF collective opens

+| PNETCDF_F_OPEN_TIMESTAMP | Timestamp of first time that the file was opened at PnetCDF level

+| PNETCDF_F_CLOSE_TIMESTAMP | Timestamp of the last time that the file was closed at PnetCDF level

+|====

+

+===== Additional modules 

+

+.BG/Q module (if enabled on BG/Q systems)

+[cols="40%,60%",options="header"]

+|====

+| counter name | description

+| BGQ_CSJOBID | Control system job ID

+| BGQ_NNODES | Total number of BG/Q compute nodes

+| BGQ_RANKSPERNODE | Number of MPI ranks per compute node

+| BGQ_DDRPERNODE | Size of compute node DDR in MiB

+| BGQ_INODES | Total number of BG/Q I/O nodes

+| BGQ_ANODES | Dimension of A torus

+| BGQ_BNODES | Dimension of B torus

+| BGQ_CNODES | Dimension of C torus

+| BGQ_DNODES | Dimension of D torus

+| BGQ_ENODES | Dimension of E torus

+| BGQ_TORUSENABLED | Bitfield indicating enabled torus dimensions

+| BGQ_F_TIMESTAMP | Timestamp of when BG/Q data was collected

</span> |====

 ==== Additional summary output

 [[addsummary]]

<span style="color: #000000;background-color: #ddffdd">+The following sections describe addtitional parser options that provide

+summary I/O characterization data for the given log.

+

+*NOTE*: These options are currently only supported by the POSIX and MPI-IO modules.

+

</span> ===== Performance

 Job performance information can be generated using the `--perf` command-line option.

<span style="color: #aaaaaa">@@ -399,9 +466,9 @@ Byte and for the aggregate performance is MiB/s (1024*1024 Bytes/s).

</span> 

 ===== Files

 Use the `--file` option to get totals based on file usage.

<span style="color: #000000;background-color: #ffdddd">-The first column is the count of files for that type, the second column is

-number of bytes for that type and the third column is the maximum offset

-accessed.

</span><span style="color: #000000;background-color: #ddffdd">+Each line has 3 columns. The first column is the count of files for that

+type of file, the second column is number of bytes for that type, and the third

+column is the maximum offset accessed.

</span> 

 * total: All files

 * read_only: Files that were only read from

<span style="color: #aaaaaa">@@ -410,9 +477,6 @@ accessed.

</span> * unique: Files that were opened on only one rank

 * shared: File that were opened by more than one rank

<span style="color: #000000;background-color: #ffdddd">-Each line has 3 columns. The first column is the count of files for that

-type of file, the second column is number of bytes for that type, and the third

-column is the maximum offset accessed.

</span> 

 .Example output

 ----

<span style="color: #aaaaaa">@@ -426,33 +490,49 @@ column is the maximum offset accessed.

</span> # shared: 1540 236561051820 154157611

 ----

<span style="color: #000000;background-color: #ddffdd">+===== Totals

+

+Use the `--total` option to get all statistics as an aggregate total rather

+than broken down per file.  Each field is either summed across files and

+process (for values such as number of opens), set to global minimums and

+maximums (for values such as open time and close time), or zeroed out (for

+statistics that are nonsensical in aggregate).

+

+.Example output

+----

+total_POSIX_OPENS: 1024

+total_POSIX_READS: 0

+total_POSIX_WRITES: 16384

+total_POSIX_SEEKS: 16384

+total_POSIX_STATS: 1024

+total_POSIX_MMAPS: 0

+total_POSIX_FOPENS: 0

+total_POSIX_FREADS: 0

+total_POSIX_FWRITES: 0

+total_POSIX_BYTES_READ: 0

+total_POSIX_BYTES_WRITTEN: 68719476736

+total_POSIX_MAX_BYTE_READ: 0

+total_POSIX_MAX_BYTE_WRITTEN: 67108863

+...

+----

+

</span> ===== File list

 Use the `--file-list` option to produce a list of files opened by the

 application along with estimates of the amount of time spent accessing each

<span style="color: #000000;background-color: #ffdddd">-file.  Each file is represented with one line with six columns:

-

-* <hash>: hash of file name

-* <suffix>: last 15 characters of file name

-* <type>: MPI or POSIX. A file is considered of MPI type if it is opened

-usijng an MPI function (directly or indirectly by a higher library such as

-HDF or NetCDF). 

-* <nprocs>: number of processes that opened the file

-* <slowest>: (estimated) time in seconds consumed in IO by slowest process

-* <avg>: average time in seconds consumed in IO per process

<span style="color: #000000;background-color: #ddffdd">+file.

</span> 

 .Example output

 ----

 # Per-file summary of I/O activity.

<span style="color: #000000;background-color: #ffdddd">-# <hash>: hash of file name

-# <suffix>: last 15 characters of file name

-# <type>: MPI or POSIX

</span><span style="color: #000000;background-color: #ddffdd">+# <record_id>: darshan record id for this file

+# <file_name>: full file name

</span> # <nprocs>: number of processes that opened the file

 # <slowest>: (estimated) time in seconds consumed in IO by slowest process

 # <avg>: average time in seconds consumed in IO per process

<span style="color: #000000;background-color: #ffdddd">-# <hash>    <suffix>    <type>  <nprocs>    <slowest>   <avg>

-17028232952633024488    amples/boom.dat MPI 2   0.000363    0.012262

</span><span style="color: #000000;background-color: #ddffdd">+# <record_id>   <file_name> <nprocs>    <slowest>   <avg>

+5041708885572677970 /projects/SSSPPg/snyder/ior/ior.dat 1024    16.342061   1.705930

</span> ----

 This data could be post-processed to compute more in-depth statistics, such as

<span style="color: #aaaaaa">@@ -466,64 +546,18 @@ files according to how much time was spent performing I/O in each file.

</span> 

 The `--file-list-detailed` is the same as --file-list except that it

 produces many columns of output containing statistics broken down by file.

<span style="color: #000000;background-color: #ffdddd">-This option is mainly useful for automated analysis.  Each file opened by

-the job is represented using one output line with the following colums:

-

-* <hash>: hash of file name

-* <suffix>: last 15 characters of file name

-* <type>: MPI or POSIX. A file is considered of MPI type if it is opened

-using an MPI function (directly or indirectly by a higher library such as HDF,

-NetCDF). 

-* <nprocs>: number of processes that opened the file

-* <slowest>: (estimated) time in seconds consumed in IO by slowest process

-* <avg>: average time in seconds consumed in IO per process

-* <start_{open/read/write}>: start timestamp of first open, read, or write

-* <end_{open/read/write}>: end timestamp of last open, read, or write

-* <mpi_indep_opens>: independent MPI_File_open calls

-* <mpi_coll_opens>: collective MPI_File_open calls

-* <posix_opens>: POSIX open calls

-* <CP_SIZE_READ_*>: POSIX read size histogram

-* <CP_SIZE_WRITE_*>: POSIX write size histogram

-

-===== Totals

-

-Use the `--total` option to get all statistics as an aggregate total rather

-than broken down per file.  Each field is either summed across files and

-process (for values such as number of opens), set to global minimums and

-maximums (for values such as open time and close time), or zeroed out (for

-statistics that are nonsensical in aggregate).

-

-.Example output

-----

-total_CP_INDEP_OPENS: 0

-total_CP_COLL_OPENS: 196608

-total_CP_INDEP_READS: 0

-total_CP_INDEP_WRITES: 0

-total_CP_COLL_READS: 0

-total_CP_COLL_WRITES: 0

-total_CP_SPLIT_READS: 0

-total_CP_SPLIT_WRITES: 1179648

-total_CP_NB_READS: 0

-total_CP_NB_WRITES: 0

-total_CP_SYNCS: 0

-total_CP_POSIX_READS: 983045

-total_CP_POSIX_WRITES: 33795

-total_CP_POSIX_OPENS: 230918

-...

-----

<span style="color: #000000;background-color: #ddffdd">+This option is mainly useful for more detailed automated analysis.

</span> 

<span style="color: #000000;background-color: #ffdddd">-=== Other command line utilities

</span><span style="color: #000000;background-color: #ddffdd">+=== Other darshan-util utilities

</span> 

 The darshan-util package includes a number of other utilies that can be

 summarized briefly as follows:

 * darshan-convert: converts an existing log file to the newest log format.

<span style="color: #000000;background-color: #ffdddd">-If the output file has a .bz2 extension, then it will be re-compressed in

-bz2 format rather than gz format.  It also has command line options for

</span><span style="color: #000000;background-color: #ddffdd">+If the `--bzip2` flag is given, then the output file will be re-compressed in

+bzip2 format rather than libz format.  It also has command line options for

</span> anonymizing personal data, adding metadata annotation to the log header, and

 restricting the output to a specific instrumented file.

<span style="color: #000000;background-color: #ffdddd">-* darshan-diff: compares two darshan log files and shows counters that

-differ.

</span> * darshan-analyzer: walks an entire directory tree of Darshan log files and

 produces a summary of the types of access methods used in those log files.

 * darshan-logutils*: this is a library rather than an executable, but it

</code></pre>

<br>

</li>

<li id='diff-179'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-179'>

<strong>

darshan-util/darshan-job-summary/bin/jenkins.c

</strong>

→

<strong>

darshan-util/jenkins-hash-gen.c

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-job-summary/bin/jenkins.c

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/jenkins-hash-gen.c

</span></code></pre>

<br>

</li>

<li id='diff-180'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-180'>

<strong>

darshan-util/maint/darshan-util.pc.in

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/maint/darshan-util.pc.in

</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/maint/darshan-util.pc.in

</span><span style="color: #aaaaaa">@@ -3,8 +3,8 @@ exec_prefix = @exec_prefix@

</span> libdir = @libdir@

 includedir = @includedir@

 abs_top_builddir = @abs_top_builddir@

<span style="color: #000000;background-color: #ffdddd">-cp_zlib_include_flags = @__CP_ZLIB_INCLUDE_FLAGS@

-cp_zlib_link_flags = @__CP_ZLIB_LINK_FLAGS@

</span><span style="color: #000000;background-color: #ddffdd">+darshan_zlib_include_flags = @__DARSHAN_ZLIB_INCLUDE_FLAGS@

+darshan_zlib_link_flags = @__DARSHAN_ZLIB_LINK_FLAGS@

</span> LIBBZ2 = @LIBBZ2@

 Name: darshan-util

<span style="color: #aaaaaa">@@ -13,5 +13,5 @@ Version: @DARSHAN_UTIL_VERSION@

</span> URL: http://trac.mcs.anl.gov/projects/darshan/

 Requires:

 Libs: -L${libdir} -ldarshan-util 

<span style="color: #000000;background-color: #ffdddd">-Libs.private: ${cp_zlib_link_flags} -lz ${LIBBZ2}

-Cflags: -I${includedir} ${cp_zlib_include_flags}

</span><span style="color: #000000;background-color: #ddffdd">+Libs.private: ${darshan_zlib_link_flags} -lz ${LIBBZ2}

+Cflags: -I${includedir} ${darshan_zlib_include_flags}

</span></code></pre>

<br>

</li>

<li id='diff-181'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-181'>

<strong>

doc/Makefile

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/Makefile

</span><span style="color: #aaaaaa">@@ -0,0 +1,12 @@

</span><span style="color: #000000;background-color: #ddffdd">+OUTPUT=darshan-modularization.pdf darshan-modularization.html

+

+all:: $(OUTPUT)

+

+%.pdf: %.txt

+       a2x -f pdf $<

+

+%.html: %.txt

+       a2x -f xhtml $<

+

+clean::

+       rm -f $(OUTPUT)

</span></code></pre>

<br>

</li>

<li id='diff-182'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-182'>

<strong>

doc/darshan-dev-modular-runtime.png

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files /dev/null and b/doc/darshan-dev-modular-runtime.png differ

</code></pre>

<br>

</li>

<li id='diff-183'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-183'>

<strong>

doc/darshan-modularization-design-notes.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/darshan-modularization-design-notes.txt

</span><span style="color: #aaaaaa">@@ -0,0 +1,112 @@

</span><span style="color: #000000;background-color: #ddffdd">+Rough design notes on modularizing Darshan

+2014-09-24

+------------------------

+

+- Darshan is split into two parts (subdirs in the same repo):

+  - runtime: runtime instrumentation for MPI programs

+  - util: post-processing of logs

+

+Runtime design

+----------------

+

+- current code has the following responsibilities:

+  - init:

+    - set up data structures

+  - during runtime:

+    - track file names and handles

+    - memory allocation

+    - intercepting function calls

+    - updating counters

+  - shutdown:

+    - identify shared files

+    - aggregation/reduction

+    - compression

+    - write log

+

+- propose division of code in modular runtime library:

+  (these aren't literally separate libraries, they are probably all

+  combined):

+  - core lib: 

+    - central component that modules register with, coordinates shutdown

+  - modules:

+    - posix, mpi-io, pnetcdf, hdf5, asg, etc.

+    - register with the core lib and track statistics for a single API

+  - common/utility lib:

+    - contains utility functions

+    - not mandatory for a module to use this, but may make things easier

+

+- responsibilities of core library:

+  - track file names and map them to generic IDs

+    (keep full path names)

+  - tell modules how much memory they can consume

+  - kick off shutdown procedure

+  - perform generic (zlib) compression

+

+- at shutdown time, the core library will:

+  - create output file

+  - write header and index information

+  - write out filename->ID mapping

+  - perform its own aggregation step to identify files shared across ranks

+

+API:

+- core API (presented by core library, used by modules):

+  - register(const char* name, int* runtime_mem_limit, struct mod_fns *mfns)

+    - lets module register with the core library, provide its name and table

+      of function pointers, and get back a limit on how much RAM it can

+      consume

+  - lookup_id(void* name, int len, int64* ID, int printable_flag);

+    - used by module to convert a file name to a generic ID.  printable_flag

+      tells Darshan that the "name" is not a string (as in ASG use case)

+

+- module API (will be function pointers in struct mod_fns above, this is the

+  API that each module must present to the core library)

+  - prep_for_shutdown()

+    - tells the module that it should stop instrumenting and perform any

+      module-specific aggregation or custom compression that it wants to do

+      before Darshan stores its results

+  - get_output(void **buffer, int size)

+    - called by core library to get a pointer to the data that should be

+      written into the log file.  Darshan will zlib compress it and put it

+      in the right position in the output file.

+

+- how will the asg module fit in?

+  - it doesn't have file names

+  - will pass in object IDs instead that will still get mapped to generic

+    Darshan IDs just like a file name would have

+    - set flag telling Darshan that the "name" won't be printable

+

+- compiler script:

+  - how much do we want to modularize here?

+  - don't need to do this initially, but we could have the compiler script

+    call out to a predefined directory to look for scripts or files that let

+    each module describe the linker arguments to add

+    - avoid extremely large ld arguments

+

+- utility library:

+  - this is the part run to process existing logs

+  - file format:

+

+    - header (endianness, version number, etc.)

+    - job information (cmd line, start time, end time, etc.)

+    - indices 

+      - location/size of name->id mapping table

+      - location/size of each module's opaque data (with name)

+    - table of name->id mapping

+      - needs to handle variable length names (some of which won't be

+        printable)

+      - format it however makes sense for parsing

+      - compress this part since it will often contain mostly text

+    - opaque blobs containing data for each module

+      - modules will refer to files using ID from name->id table, won't

+        store full paths here

+

+  - each module can define its own parser, grapher, etc. as needed

+  - for convenience we may integrate posix and mpi-io support into the default

+    darshan tools

+

+- development notes

+  - do development in git branch

+  - ignore compatibility (we'll work that out later)

+  - strip down to basic example

+    - just do one or two posix counters to start, but exercise all of the

+      API and code organization stuff

</span></code></pre>

<br>

</li>

<li id='diff-184'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-184'>

<strong>

doc/darshan-modularization-whiteboard.pdf

</strong>

</a>

<hr>

<pre class="highlight"><code>Binary files /dev/null and b/doc/darshan-modularization-whiteboard.pdf differ

</code></pre>

<br>

</li>

<li id='diff-185'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-185'>

<strong>

doc/darshan-modularization.txt

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/darshan-modularization.txt

</span><span style="color: #aaaaaa">@@ -0,0 +1,554 @@

</span><span style="color: #000000;background-color: #ddffdd">+:data-uri:

+

+Darshan modularization branch development notes

+===============================================

+

+== Introduction

+

+Darshan is a lightweight toolkit for characterizing the I/O performance of instrumented

+HPC applications.

+

+Starting with version 3.0.0, the Darshan runtime environment and log file format have

+been redesigned such that new "instrumentation modules" can be added without breaking

+existing tools. Developers are given a framework to implement arbitrary instrumentation

+modules, which are responsible for gathering I/O data from a specific system component

+(which could be from an I/O library, platform-specific data, etc.). Darshan can then

+manage these modules at runtime and create a valid Darshan log regardless of how many

+or what types of modules are used.

+

+== Checking out and building the modularization branch

+

+The Darshan source code is available at the following GitLab project page:

+https://xgitlab.cels.anl.gov/darshan/darshan. It is worth noting that this page

+also provides issue tracking to provide users the ability to browse known issues

+with the code or to report new issues.

+

+The following commands can be used to clone the Darshan source code and checkout

+the modularization branch:

+

+----

+git clone git@xgitlab.cels.anl.gov:darshan/darshan.git

+cd darshan

+git checkout dev-modular

+----

+

+For details on configuring, building, and using the Darshan runtime and utility

+repositories, consult the documentation from previous versions

+(http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html[darshan-runtime] and

+http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-util.html[darshan-util]) -- the

+necessary steps for building these repositories should not have changed in the new version of

+Darshan.

+

+== Darshan dev-modular overview

+

+The Darshan source tree is organized into two primary components:

+

+* *darshan-runtime*: Darshan runtime framework necessary for instrumenting MPI

+applications and generating I/O characterization logs.

+

+* *darshan-util*: Darshan utilities for analyzing the contents of a given Darshan

+I/O characterization log.

+

+The following subsections provide detailed overviews of each of these components to

+give a better understanding of the architecture of the modularized version of Darshan.

+In link:darshan-modularization.html#_adding_new_instrumentation_modules[Section 4], we

+actually outline the necessary steps for integrating new instrumentation modules into

+Darshan.

+

+=== Darshan-runtime

+

+The primary responsibilities of the darshan-runtime component are:

+

+* intercepting I/O functions of interest from a target application;

+

+* extracting statistics, timing information, and other data characterizing the application's I/O workload;

+

+* compressing I/O characterization data and corresponding metadata;

+

+* logging the compressed I/O characterization to file for future evaluation

+

+The first two responsibilities are the burden of module developers, while the last two are handled

+automatically by Darshan.

+

+In general, instrumentation modules are composed of:

+

+* wrapper functions for intercepting I/O functions;

+

+* internal functions for initializing and maintaining internal data structures and module-specific

+  I/O characterization data;

+

+* a set of functions for interfacing with the Darshan runtime environment

+

+A block diagram illustrating the interaction of an example POSIX instrumentation module and the

+Darshan runtime environment is given below in Figure 1.

+

+.Darshan runtime environment

+image::darshan-dev-modular-runtime.png[align="center"]

+

+As shown in Figure 1, the Darshan runtime environment is just a library (libdarshan) which

+intercepts and instruments functions of interest made by an application to existing system

+libraries. Two primary components of this library are `darshan-core` and `darshan-common`.

+`darshan-core` is the central component which manages the initialization/shutdown of Darshan,

+coordinates with active instrumentation modules, and writes I/O characterization logs to disk,

+among other things. `darshan-core` intercepts `MPI_Init()` to initialize key internal data

+stuctures and intercepts `MPI_Finalize()` to initiate Darshan's shutdown process. `darshan-common`

+simply provides module developers with functionality that is likely to be reused across modules

+to minimize development and maintenance costs. Instrumentation modules must utilize `darshan-core`

+to register themselves and corresponding I/O records with Darshan so they can be added to the

+output I/O characterization. While not shown in Figure 1, numerous modules can be registered

+with Darshan at any given time and Darshan is capable of correlating records between these

+modules.

+

+In the next three subsections, we describe instrumentation modules, the `darshan-core` component,

+and the `darshan-common` component in more detail.

+

+==== Instrumentation modules

+

+The new modularized version of Darshan allows for the generation of I/O characterizations

+composed from numerous instrumentation modules, where an instrumentation module is simply a

+Darshan component responsible for capturing I/O data from some arbitrary source. For example,

+distinct instrumentation modules may be defined for different I/O interfaces or to gather

+system-specific I/O parameters from a given computing system. Each instrumentation module

+interfaces with the `darshan-core` component to coordinate its initialization and shutdown

+and to provide output I/O characterization data to be written to log.

+

+In general, there are two different methods an instrumentation module can use to initialize

+itself: static initialization at Darshan startup time or dynamic initialization within

+intercepted function calls during application execution. The initialization process should

+initialize module-specific data structures and register the module with the `darshan-core`

+component so it is included in the output I/O characterization.

+

+The static initialization approach is useful for modules that do not have function calls

+that can be intercepted and instead can just grab all I/O characterization data at Darshan

+startup or shutdown time. A module can be statically initialized at Darshan startup time

+by adding its initializatin routine to the `mod_static_init_fns` list at the top of the

+`lib/darshan-core.c` source file.

+

+*NOTE*: Modules may wish to add a corresponding configure option to disable the module

+from attempting to gather I/O data. The ability to disable a module using a configure

+option is especially necessary for system-specific modules which can not be built or

+used on other systems.

+

+Most instrumentation modules can just bootstrap themselves within wrapper functions during

+normal application execution. Each of Darshan's current I/O library instrumentation modules

+(POSIX, MPI-IO, HDF5, PnetCDF) follow this approach. Each wrapper function should just include

+logic to initialize data structures and register with `darshan-core` if this initialization

+has not already occurred. Darshan intercepts function calls of interest by inserting these

+wrappers at compile time for statically linked executables (e.g., using the linkers

+`--wrap` mechanism) and at runtime for dynamically linked executables (using LD_PRELOAD).

+

+*NOTE*: Modules should not perform any I/O or communication within wrapper functions. Darshan records

+I/O data independently on each application process, then merges the data from all processes when the

+job is shutting down. This defers expensive I/O and communication operations to the shutdown process,

+minimizing Darshan's impact on application I/O performance.

+

+When the instrumented application terminates and Darshan begins its shutdown procedure, it requires

+a way to interface with any active modules that have data to contribute to the output I/O characterization.

+Darshan requires that module developers implement the following functions to allow the Darshan runtime

+environment to coordinate with modules while shutting down:

+

+[source,c]

+struct darshan_module_funcs

+{

+    void (*begin_shutdown)(void);

+    void (*get_output_data)(

+        MPI_Comm mod_comm,

+        darshan_record_id *shared_recs,

+        int shared_rec_count,

+        void** mod_buf,

+        int* mod_buf_sz

+    );

+    void (*shutdown)(void);

+};

+

+`begin_shutdown()`

+

+This function informs the module that Darshan is about to begin shutting down. It should disable

+all wrappers to prevent the module from making future updates to internal data structures, primarily

+to ensure data consistency and avoid other race conditions.

+

+`get_output_data()`

+

+This function is responsible for packing all module I/O data into a single buffer to be written

+to the output I/O characterization. This function can be used to run collective MPI operations on

+module data; for instance, Darshan typically tries to reduce file records which are shared across

+all application processes into a single data record (more details on the shared file reduction

+mechanism are given in link:darshan-modularization.html#_shared_record_reductions[Section 5]).

+

+* _mod_comm_ is the MPI communicator to use for collective communication

+

+* _shared_recs_ is a list of Darshan record identifiers that are shared across all application

+processes

+

+* _shared_rec_count_ is the size of the shared record list

+

+* _mod_buf_ is a pointer to the buffer of this module's I/O characterization data

+

+* _mod_buf_sz_ is the size of the module's output buffer

+

+`shutdown()`

+

+This function is a signal from Darshan that it is safe to shutdown. It should clean up and free

+all internal data structures.

+

+==== darshan-core

+

+Within darshan-runtime, the darshan-core component manages the initialization and shutdown of the

+Darshan environment, provides an interface for modules to register themselves and their data

+records with Darshan, and manages the compressing and the writing of the resultant I/O

+characterization. As illustrated in Figure 1, the darshan-core runtime environment intercepts

+`MPI_Init` and `MPI_Finalize` routines to initialize and shutdown the Darshan runtime environment,

+respectively.

+

+Each of the functions provided by `darshan-core` to interface with instrumentation modules are

+described in detail below.

+

+[source,c]

+void darshan_core_register_module(

+    darshan_module_id mod_id,

+    struct darshan_module_funcs *funcs,

+    int *my_rank,

+    int *mod_mem_limit,

+    int *sys_mem_alignment);

+

+The `darshan_core_register_module` function registers Darshan instrumentation modules with the

+`darshan-core` runtime environment. This function needs to be called once for any module that

+will contribute data to Darshan's final I/O characterization. 

+

+* _mod_id_ is a unique identifier for the given module, which is defined in the Darshan log

+format header file (`darshan-log-format.h`).

+

+* _funcs_ is the structure of function pointers (as described above in the previous section) that

+a module developer must provide to interface with the darshan-core runtime. 

+

+* _my_rank_ is a pointer to an integer to store the calling process's application MPI rank in

+

+* _mod_mem_limit_ is a pointer to an integer which will store the amount of memory Darshan

+allows this module to use at runtime. Darshan's default module memory limit is currently set to

+2 MiB, but the user can choose a different value at configure time (using the `--with-mod-mem`

+configure option) or at runtime (using the DARSHAN_MODMEM environment variable). Note that Darshan

+does not allocate any memory for modules; it just informs a module how much memory it can use.

+

+* _sys_mem_alignment_ is a pointer to an integer which will store the system memory alignment value

+Darshan was configured with. This parameter may be set to `NULL` if a module is not concerned with the

+memory alignment value.

+

+[source,c]

+void darshan_core_unregister_module(

+    darshan_module_id mod_id);

+

+The `darshan_core_unregister_module` function disassociates the given module from the

+`darshan-core` runtime. Consequentially, Darshan does not interface with the given module at

+shutdown time and will not log any I/O data from the module. This function should only be used

+if a module registers itself with darshan-core but later decides it does not want to contribute

+any I/O data.

+

+* _mod_id_ is the unique identifer for the module being unregistered.

+

+[source,c]

+void darshan_core_register_record(

+    void *name,

+    int len,

+    darshan_module_id mod_id,

+    int printable_flag,

+    int mod_limit_flag,

+    darshan_record_id *rec_id,

+    int *file_alignment);

+

+The `darshan_core_register_record` function registers some data record with the darshan-core

+runtime. This record could reference a POSIX file or perhaps an object identifier for an

+object storage system, for instance.  A unique identifier for the given record name is

+generated by Darshan, which should then be used by the module for referencing the corresponding

+record.  This allows multiple modules to refer to a specific data record in a consistent manner

+and also provides a mechanism for mapping these records back to important metadata stored by

+darshan-core. It is safe (and likely necessary) to call this function many times for the same

+record -- darshan-core will just set the corresponding record identifier if the record has

+been previously registered.

+

+* _name_ is just the name of the data record, which could be a file path, object ID, etc.

+

+* _len_ is the size of the input record name. For string record names, this would just be the

+string length, but for nonprintable record names (e.g., an integer object identifier), this

+is the size of the record name type.

+

+* _mod_id_ is the identifier for the module attempting to register this record.

+

+* _printable_flag_ indicates whether the input record name is a printable ASCII string.

+

+* _mod_limit_flag_ indicates whether the calling module is out of memory to instrument new

+records or not. If this flag is set, darshan-core will not create new records and instead just

+search existing records for one corresponding to input _name_. 

+

+* _rec_id_ is a pointer to a variable which will store the unique record identifier generated

+by Darshan.

+

+* _file_alignment_ is a pointer to an integer which will store the the file alignment (block size)

+of the underlying storage system. This parameter may be set to `NULL` if it is not applicable to a

+given module.

+

+[source,c]

+void darshan_core_unregister_record(

+    darshan_record_id rec_id,

+    darshan_module_id mod_id);

+

+The `darshan_core_unregister_record` function disassociates the given module identifier from the

+given record identifier. If no other modules are associated with the given record identifier, then

+Darshan removes all internal references to the record. This function should only be used if a

+module registers a record with darshan-core, but later decides not to store the record internally.

+

+* _rec_id_ is the record identifier we want to unregister.

+

+* _mod_id_ is the module identifier that is unregistering _rec_id_.

+

+[source,c]

+double darshan_core_wtime(void);

+

+The `darshan_core_wtime` function simply returns a floating point number of seconds since

+Darshan was initialized. This functionality can be used to time the duration of application

+I/O calls or to store timestamps of when functions of interest were called.

+

+==== darshan-common

+

+`darshan-common` is a utility component of darshan-runtime, providing module developers with

+general functions that are likely to be reused across multiple modules. These functions are

+distinct from darshan-core functions since they do not require access to internal Darshan

+state.

+

+[source,c]

+char* darshan_clean_file_path(

+    const char* path);

+

+The `darshan_clean_file_path` function just cleans up the input path string, converting

+relative paths to absolute paths and suppressing any potential noise within the string.

+The address of the new string is returned and should be freed by the user.

+

+* _path_ is the input path string to be cleaned up.

+

+`darshan-common` also currently includes functions for maintaining counters that store

+common I/O values (such as common I/O access sizes or strides used by an application),

+as well as functions for calculating the variance of a given counter across all processes.

+As more modules are contributed, it is likely that more functionality can be refactored out

+of module implementations and maintained in darshan-common, facilitating code reuse and

+simplifying maintenance.

+

+=== Darshan-util

+

+The darshan-util component is composed of a log parsing library (libdarshan-util) and a

+corresponding set of utility programs that can parse and analyze Darshan I/O characterization

+logs using this library. The log parsing library includes a generic interface (see

+`darshan-logutils.h`) for retrieving specific portions of a given log file. Specifically,

+this interface allows utilities to retrieve a log's header metadata, job details, record

+identifier mapping, and any module-specific data contained within the log.

+

+Module developers may wish to define additional interfaces for parsing module-specific data

+that can then be integrated into the log parsing library. This extended functionality can be

+implemented in terms of the generic functions offered by darshan-logutils and by module-specific

+formatting information.

+

+==== darshan-logutils

+

+Here we define each function in the `darshan-logutils` interface, which can be used to create

+new log utilities and to implement module-specific interfaces into log files.

+

+[source,c]

+darshan_fd darshan_log_open(const char *name);

+

+Opens Darshan log file stored at path `name`. The log file must already exist and is opened

+for reading only. As part of the open routine, the log file header is read to set internal

+file descriptor data structures. Returns a Darshan file descriptor on success or `NULL` on error.

+

+[source,c]

+darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type, int partial_flag);

+

+Creates a new darshan log file for writing only at path `name`. `comp_type` denotes the underlying

+compression type used on the log file (currently either libz or bzip2) and `partial_flag`

+denotes whether the log is storing partial data (that is, all possible application file records

+were not tracked by darshan). Returns a Darshan file descriptor on success or `NULL` on error.

+

+[source,c]

+int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);

+int darshan_log_putjob(darshan_fd fd, struct darshan_job *job);

+

+Reads/writes `job` structure from/to the log file referenced by descriptor `fd`. The `darshan_job`

+structure is defined in `darshan-log-format.h`. Returns `0` on success, `-1` on failure.

+

+[source,c]

+int darshan_log_getexe(darshan_fd fd, char *buf);

+int darshan_log_putexe(darshan_fd fd, char *buf);

+

+Reads/writes the corresponding executable string (exe name and command line arguments)

+from/to the Darshan log referenced by `fd`. Returns `0` on success, `-1` on failure.

+

+[source,c]

+int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts, char*** fs_types, int* count);

+int darshan_log_putmounts(darshan_fd fd, char** mnt_pts, char** fs_types, int count);

+

+Reads/writes mounted file system information for the Darshan log referenced by `fd`. `mnt_pnts` points

+to an array of strings storing mount points, `fs_types` points to an array of strings storing file

+system types (e.g., ext4, nfs, etc.), and `count` points to an integer storing the total number

+of mounted file systems recorded by Darshan. Returns `0` on success, `-1` on failure.

+

+[source,c]

+int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);

+int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash);

+

+Reads/writes the hash table of Darshan record identifiers to full names for all records

+contained in the Darshan log referenced by `fd`. `hash` is a pointer to the hash table (of type

+struct darshan_record_ref *, which should be initialized to `NULL` for reading). This hash table

+is defined by the `uthash` hash table implementation and includes corresponding macros for

+searching, iterating, and deleting records from the hash. For detailed documentation on using this

+hash table, consult `uthash` documentation in `darshan-util/uthash-1.9.2/doc/txt/userguide.txt`.

+The `darshan-parser` utility (for parsing module information out of a Darshan log) provides an

+example of how this hash table may be used. Returns `0` on success, `-1` on failure.

+

+[source,c]

+int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id, void *mod_buf, int mod_buf_sz);

+int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id, void *mod_buf, int mod_buf_sz);

+

+Reads/writes a chunk of (uncompressed) module data for the module identified by `mod_id` from/to

+the Darshan log referenced by `fd`. `mod_buf_sz` specifies the number of uncompressed bytes to

+read/write from/to the file and store in `mod_buf`. The `darshan_log_getmod` routine can be

+repeatedly called to retrieve chunks of uncompressed data from a specific module region of the

+log file given by `fd`. The `darshan_log_putmod` routine just continually appends data to a

+specific module region in the log file given by `fd`. This function returns the number of bytes

+read/written on success, `-1` on failure.

+

+*NOTE*: Darshan use a reader makes right conversion strategy to rectify endianness issues

+between the machine a log was generated on and a machine analyzing the log. Accordingly,

+module-specific log utility functions will need to check the `swap_flag` variable of the Darshan

+file descriptor to determine if byte swapping is necessary. 32-bit and 64-bit byte swapping

+macros (DARSHAN_BSWAP32/DARSHAN_BSWAP64) are provided in `darshan-logutils.h`.

+

+[source,c]

+void darshan_log_close(darshan_fd fd);

+

+Close Darshan file descriptor `fd`. This routine *must* be called for newly created log files,

+as it flushes pending writes and writes a corresponding log file header before closing.

+

+*NOTE*: For newly created Darshan log files, care must be taken to write log file data in the

+correct order, since the log file write routines basically are appending data to the log file.

+The correct order for writing all log file data to file is: (1) job data, (2) exe string, (3)

+mount data, (4) record id -> file name map, (5) each module's data, in increasing order of

+module identifiers.

+

+== Adding new instrumentation modules

+

+In this section we outline each step necessary for adding a module to Darshan. To assist module

+developers, we have provided the example "NULL" module as part of the Darshan source tree

+(`darshan-null-log-format.h`, `darshan-runtime/lib/darshan-null.c`, and

+`darshan-util/darshan-null-logutils.*`) This example can be used as a minimal stubbed out module

+implementation that is heavily annotated to further clarify how modules interact with Darshan

+and to provide best practices to future module developers. For full-fledged module implementation

+examples, developers are encouraged to examine the POSIX and MPI-IO modules.

+

+=== Log format headers

+

+The following modifications to Darshan log format headers are required for defining

+the module's record structure:

+

+* Add a module identifier to the `DARSHAN_MODULE_IDS` macro at the top of the `darshan-log-format.h`

+header. In this macro, the first field is a corresponding enum value that can be used to

+identify the module, the second field is a string name for the module, the third field is the

+current version number of the given module's log format, and the fourth field is a corresponding

+pointer to a Darshan log utility implementation for this module (which can be set to `NULL`

+until the module has its own log utility implementation). 

+

+* Add a top-level header that defines an I/O data record structure for the module. Consider

+the "NULL" module and POSIX module log format headers for examples (`darshan-null-log-format.h`

+and `darshan-posix-log-format.h`, respectively).

+

+These log format headers are defined at the top level of the Darshan source tree, since both the

+darshan-runtime and darshan-util repositories depend on their definitions.

+

+=== Darshan-runtime

+

+==== Build modifications

+

+The following modifications to the darshan-runtime build system are necessary to integrate

+new instrumentation modules:

+

+* Necessary linker flags for inserting this module's wrapper functions need to be added to a

+module-specific file which is used when linking applications with Darshan. For an example,

+consider `darshan-runtime/darshan-posix-ld-opts`, the required linker options for the POSIX

+module. The base linker options file for Darshan (`darshan-runtime/darshan-base-ld-opts.in`)

+must also be updated to point to the new module-specific linker options file.

+

+* Targets must be added to `Makefile.in` to build static and shared objects for the module's

+source files, which will be stored in the `darshan-runtime/lib/` directory. The prerequisites

+to building static and dynamic versions of `libdarshan` must be updated to include these objects,

+as well.

+    - If the module defines a linker options file, a rule must also be added to install this

+      file with libdarshan.

+

+==== Instrumentation module implementation

+

+In addtion to the development notes from above and the exemplar "NULL" and POSIX modules, we

+provide the following notes to assist module developers:

+

+* Modules only need to include the `darshan.h` header to interface with darshan-core.

+

+* The file record identifier given when registering a record with darshan-core can be used

+to store the record structure in a hash table or some other structure.

+    - The `darshan_core_register_record` function is really more like a lookup function. It

+    may be called multiple times for the same record -- if the record already exists, the function

+    simply returns its record ID.

+    - It may be necessary to maintain a separate hash table for other handles which the module

+    may use to refer to a given record. For instance, the POSIX module may need to look up a

+    file record based on a given file descriptor, rather than a path name.

+

+=== Darshan-util

+

+==== Build modifications

+

+The following modifications to the darshan-util build system are necessary to integrate new

+instrumentation modules:

+

+* Update `Makefile.in` with new targets necessary for building module-specific logutil source.

+    - Make sure to add the module's logutil implementation objects as a prerequisite for

+building `libdarshan-util`. 

+    - Make sure to update `all`, `clean`, and `install` rules to reference updates.

+

+==== Module-specific logutils and utilities

+

+For a straightforward reference implementation of module-specific log utility functions,

+consider the implementations for the NULL module (`darshan-util/darshan-null-logutils.*`)

+and the POSIX module (`darshan-util/darshan-posix-logutils.*`). These module-specific log

+utility implementations are built on top of the `darshan_log_getmod()` and `darshan_log_putmod()`

+functions, and are used to read/write complete module records from/to file.

+

+Also, consider the `darshan-parser` source code for an example of a utility which can leverage

+`libdarshan-util` for analyzing the contents of a Darshan I/O characterization log with data

+from arbitrary instrumentation modules.

+

+== Shared record reductions

+

+Since Darshan perfers to aggregate data records which are shared across all processes into a single

+data record, module developers should consider implementing this functionality eventually, though it

+is not strictly required. 

+

+Module developers should implement the shared record reduction mechanism within the module's

+`get_output_data()` function, as it provides an MPI communicator for the module to use for

+collective communication and a list of record identifiers which are shared globally by the

+module (as described in link:darshan-modularization.html#_darshan_runtime[Section 3.1]).

+

+In general, implementing a shared record reduction involves the following steps:

+

+* reorganizing shared records into a contiguous region in the buffer of module records

+

+* allocating a record buffer to store the reduction output on application rank 0

+

+* creating an MPI reduction operation using the `MPI_Op_create()` function (see more

+http://www.mpich.org/static/docs/v3.1/www3/MPI_Op_create.html[here])

+

+* reducing all shared records using the created MPI reduction operation and the send

+and receive buffers described above

+

+For a more in-depth example of how to use the shared record reduction mechanism, consider

+the implementations of this in the POSIX or MPI-IO modules.

+

+== Other resources

+

+* https://xgitlab.cels.anl.gov/darshan/darshan[Darshan GitLab page]

+* http://www.mcs.anl.gov/research/projects/darshan/[Darshan project website]

+* http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html[darshan-runtime documentation]

+* http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-util.html[darshan-util documentation]

</span></code></pre>

<br>

</li>

<li id='diff-186'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-186'>

<strong>

doc/docbook-xsl.css

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- /dev/null

</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/docbook-xsl.css

</span><span style="color: #aaaaaa">@@ -0,0 +1,314 @@

</span><span style="color: #000000;background-color: #ddffdd">+/*

+  CSS stylesheet for XHTML produced by DocBook XSL stylesheets.

+  Tested with XSL stylesheets 1.61.2, 1.67.2

+*/

+

+span.strong {

+  font-weight: bold;

+}

+

+body blockquote {

+  margin-top: .75em;

+  line-height: 1.5;

+  margin-bottom: .75em;

+}

+

+html body {

+  margin: 1em 5% 1em 5%;

+  line-height: 1.2;

+}

+

+body div {

+  margin: 0;

+}

+

+h1, h2, h3, h4, h5, h6

+{

+  color: #527bbd;

+  font-family: tahoma, verdana, sans-serif;

+}

+

+div.toc p:first-child,

+div.list-of-figures p:first-child,

+div.list-of-tables p:first-child,

+div.list-of-examples p:first-child,

+div.example p.title,

+div.sidebar p.title

+{

+  font-weight: bold;

+  color: #527bbd;

+  font-family: tahoma, verdana, sans-serif;

+  margin-bottom: 0.2em;

+}

+

+body h1 {

+  margin: .0em 0 0 -4%;

+  line-height: 1.3;

+  border-bottom: 2px solid silver;

+}

+

+body h2 {

+  margin: 0.5em 0 0 -4%;

+  line-height: 1.3;

+  border-bottom: 2px solid silver;

+}

+

+body h3 {

+  margin: .8em 0 0 -3%;

+  line-height: 1.3;

+}

+

+body h4 {

+  margin: .8em 0 0 -3%;

+  line-height: 1.3;

+}

+

+body h5 {

+  margin: .8em 0 0 -2%;

+  line-height: 1.3;

+}

+

+body h6 {

+  margin: .8em 0 0 -1%;

+  line-height: 1.3;

+}

+

+body hr {

+  border: none; /* Broken on IE6 */

+}

+div.footnotes hr {

+  border: 1px solid silver;

+}

+

+div.navheader th, div.navheader td, div.navfooter td {

+  font-family: sans-serif;

+  font-size: 0.9em;

+  font-weight: bold;

+  color: #527bbd;

+}

+div.navheader img, div.navfooter img {

+  border-style: none;

+}

+div.navheader a, div.navfooter a {

+  font-weight: normal;

+}

+div.navfooter hr {

+  border: 1px solid silver;

+}

+

+body td {

+  line-height: 0;

+}

+

+body th {

+  line-height: 1.2;

+}

+

+ol {

+  line-height: 1.2;

+}

+

+ul, body dir, body menu {

+  line-height: 1.2;

+}

+

+html {

+  margin: 0; 

+  padding: 0;

+}

+

+body h1, body h2, body h3, body h4, body h5, body h6 {

+  margin-left: 0

+} 

+

+body pre {

+  margin: 0.5em 10% 0.5em 1em;

+  line-height: 1.0;

+  color: navy;

+}

+

+tt.literal, code.literal {

+  color: navy;

+}

+

+.programlisting, .screen {

+  border: 1px solid silver;

+  background: #f4f4f4;

+  margin: 0.5em 10% 0.5em 0;

+  padding: 0.5em 1em;

+}

+

+div.sidebar {

+  background: #ffffee;

+  margin: 1.0em 10% 0.5em 0;

+  padding: 0.5em 1em;

+  border: 1px solid silver;

+}

+div.sidebar * { padding: 0; }

+div.sidebar div { margin: 0; }

+div.sidebar p.title {

+  margin-top: 0.5em;

+  margin-bottom: 0.2em;

+}

+

+div.bibliomixed {

+  margin: 0.5em 5% 0.5em 1em;

+}

+

+div.glossary dt {

+  font-weight: bold;

+}

+div.glossary dd p {

+  margin-top: 0.2em;

+}

+

+dl {

+  margin: .8em 0;

+  line-height: 1.2;

+}

+

+dt {

+  margin-top: 0.5em;

+}

+

+dt span.term {

+  font-style: normal;

+  color: navy;

+}

+

+div.variablelist dd p {

+  margin-top: 0;

+}

+

+div.itemizedlist li, div.orderedlist li {

+  margin-left: -0.8em;

+  margin-top: 0.5em;

+}

+

+ul, ol {

+    list-style-position: outside;

+}

+

+div.sidebar ul, div.sidebar ol {

+    margin-left: 2.8em;

+}

+

+div.itemizedlist p.title,

+div.orderedlist p.title,

+div.variablelist p.title

+{

+  margin-bottom: -0.8em;

+}

+

+div.revhistory table {

+  border-collapse: collapse;

+  border: none;

+}

+div.revhistory th {

+  border: none;

+  color: #527bbd;

+  font-family: tahoma, verdana, sans-serif;

+}

+div.revhistory td {

+  border: 1px solid silver;

+}

+

+/* Keep TOC and index lines close together. */

+div.toc dl, div.toc dt,

+div.list-of-figures dl, div.list-of-figures dt,

+div.list-of-tables dl, div.list-of-tables dt,

+div.indexdiv dl, div.indexdiv dt

+{

+  line-height: normal;

+  margin-top: 0;

+  margin-bottom: 0;

+}

+

+/*

+  Table styling does not work because of overriding attributes in

+  generated HTML.

+*/

+div.table table,

+div.informaltable table

+{

+    margin-left: 0;

+    margin-right: 5%;

+    margin-bottom: 0.8em;

+}

+div.informaltable table

+{

+    margin-top: 0.4em

+}

+div.table thead,

+div.table tfoot,

+div.table tbody,

+div.informaltable thead,

+div.informaltable tfoot,

+div.informaltable tbody

+{

+    /* No effect in IE6. */

+    border-top: 3px solid #527bbd;

+    border-bottom: 3px solid #527bbd;

+}

+div.table thead, div.table tfoot,

+div.informaltable thead, div.informaltable tfoot

+{

+    font-weight: bold;

+}

+

+div.mediaobject img {

+    margin-bottom: 0.8em;

+}

+div.figure p.title,

+div.table p.title

+{

+  margin-top: 1em;

+  margin-bottom: 0.4em;

+}

+

+div.calloutlist p

+{

+  margin-top: 0em;

+  margin-bottom: 0.4em;

+}

+

+@media print {

+  div.navheader, div.navfooter { display: none; }

+}

+

+span.aqua { color: aqua; }

+span.black { color: black; }

+span.blue { color: blue; }

+span.fuchsia { color: fuchsia; }

+span.gray { color: gray; }

+span.green { color: green; }

+span.lime { color: lime; }

+span.maroon { color: maroon; }

+span.navy { color: navy; }

+span.olive { color: olive; }

+span.purple { color: purple; }

+span.red { color: red; }

+span.silver { color: silver; }

+span.teal { color: teal; }

+span.white { color: white; }

+span.yellow { color: yellow; }

+

+span.aqua-background { background: aqua; }

+span.black-background { background: black; }

+span.blue-background { background: blue; }

+span.fuchsia-background { background: fuchsia; }

+span.gray-background { background: gray; }

+span.green-background { background: green; }

+span.lime-background { background: lime; }

+span.maroon-background { background: maroon; }

+span.navy-background { background: navy; }

+span.olive-background { background: olive; }

+span.purple-background { background: purple; }

+span.red-background { background: red; }

+span.silver-background { background: silver; }

+span.teal-background { background: teal; }

+span.white-background { background: white; }

+span.yellow-background { background: yellow; }

+

+span.big { font-size: 2em; }

+span.small { font-size: 0.6em; }

</span></code></pre>

<br>

</li>

<li id='diff-187'>

<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad#diff-187'>

<strong>

maint/config/check_zlib.m4

</strong>

</a>

<hr>

<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/maint/config/check_zlib.m4

</span><span style="color: #000000;background-color: #ddffdd">+++ b/maint/config/check_zlib.m4

</span><span style="color: #aaaaaa">@@ -11,8 +11,8 @@ AC_ARG_WITH(zlib,

</span>     ZLIB_HOME="$withval"

     LDFLAGS="$LDFLAGS -L${ZLIB_HOME}/lib"

     CPPFLAGS="$CPPFLAGS -I${ZLIB_HOME}/include"

<span style="color: #000000;background-color: #ffdddd">-    __CP_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

-    __CP_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span><span style="color: #000000;background-color: #ddffdd">+    __DARSHAN_ZLIB_LINK_FLAGS="-L${ZLIB_HOME}/lib"

+    __DARSHAN_ZLIB_INCLUDE_FLAGS="-I${ZLIB_HOME}/include"

</span>   else

     AC_MSG_WARN([Sorry, $withval does not exist, checking usual places])

   fi

</code></pre>

<br>

</li>

</div>

<div class='footer' style='margin-top: 10px;'>

<p>

—

<br>

<a href="https://xgitlab.cels.anl.gov/darshan/darshan/compare/83f12d5f90c214f2cc852451f1b6c06a019158d1...0a6399f3acde4614a3adf9453505ed3d3e93b0ad">View it on GitLab</a>.

<br>

You're receiving this email because of your account on xgitlab.cels.anl.gov.

If you'd like to receive fewer emails, you can

adjust your notification settings.

</p>

</div>

</body>

</html>