<html lang='en'>
<head>
<meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
<title>
GitLab
</title>
</meta>
</head>
<style>
img {
max-width: 100%;
height: auto;
}
p.details {
font-style:italic;
color:#777
}
.footer p {
font-size:small;
color:#777
}
pre.commit-message {
white-space: pre-wrap;
}
.file-stats a {
text-decoration: none;
}
.file-stats .new-file {
color: #090;
}
.file-stats .deleted-file {
color: #B00;
}
</style>
<body>
<div class='content'>
<h3>Shane Snyder pushed to branch mmap-dev at <a href="https://xgitlab.cels.anl.gov/darshan/darshan">darshan / darshan</a></h3>
<h4>
Commits:
</h4>
<ul>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/ba451e9ab313b395f05ba6f69b480ba1089b0008">ba451e9a</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-11-24T22:40:51Z</i>
</div>
<pre class='commit-message'>rework header org. for module-specific versions</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/66181457ee55eaac707fe6db89e11abbb9246828">66181457</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-11-25T10:18:12Z</i>
</div>
<pre class='commit-message'>modify runtime code to store each module's version</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/173d38b85c7fc065820f2f700c3c109b605e036d">173d38b8</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-11-25T12:41:33Z</i>
</div>
<pre class='commit-message'>update util interfaces to support mod-specific ver</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b07e6220550f2505612039c2e104a3048bbd7ef7">b07e6220</a></strong>
<div>
<span>by Phil Carns</span>
<i>at 2015-12-01T10:44:14Z</i>
</div>
<pre class='commit-message'>support MPICH 3.1.1 in profile conf test
fixes #174</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/26c4ff222cc88cc4fa526cc7a25d494454735975">26c4ff22</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-08T11:55:49Z</i>
</div>
<pre class='commit-message'>bug fix in bzip2 init in darshan logutils</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d097c5f10377b71caf2e45c955579d2360cc4535">d097c5f1</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-08T14:00:48Z</i>
</div>
<pre class='commit-message'>move compression buffer alloc to shutdown time</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c371cdea3257003237f12c63ff688c35934a3464">c371cdea</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-08T14:25:51Z</i>
</div>
<pre class='commit-message'>fix possible race cond in module locking</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1c1a9baad96fc76a1963de0b15006c7fe866d9a8">1c1a9baa</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T13:02:32Z</i>
</div>
<pre class='commit-message'>cleanup a bunch of code and stale comments</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0">31e0fb03</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T14:27:31Z</i>
</div>
<pre class='commit-message'>update runtime, util, and modularization docs</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/9b021c1bba1f7a7f7889333fa1f07cba5df159a4">9b021c1b</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T16:20:12Z</i>
</div>
<pre class='commit-message'>update modules to print counter descriptions</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/cb6a25259eed40e2fbeb7b407dc33dded469d554">cb6a2525</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T16:30:55Z</i>
</div>
<pre class='commit-message'>updated changelog prior to pre3 release</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/34e447c0e7e192b24028ae91375af0151fed374c">34e447c0</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T16:34:42Z</i>
</div>
<pre class='commit-message'>update configure versions for pre3</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/be6d404d078744bbc73c5f83ddd4a42b898d6e4f">be6d404d</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-16T15:55:01Z</i>
</div>
<pre class='commit-message'>forgotten header install</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/7d455c98345f27688d46b139907ffa8f53ec0f6d">7d455c98</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-17T17:07:03Z</i>
</div>
<pre class='commit-message'>back out commit for fixing races -- not needed</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/31079257caf824b5a78ac9daf4ba22d7d7ce6502">31079257</a></strong>
<div>
<span>by Phil Carns</span>
<i>at 2015-12-18T09:41:40Z</i>
</div>
<pre class='commit-message'>loosen PMPI symbol check
- prevents inadverdent disabling of Darshan on some MPICH builds
- fixes #94</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/2feaff78045a9cc32af586c7358d353c25ff1fdf">2feaff78</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-04T09:19:52Z</i>
</div>
<pre class='commit-message'>update runtime docs to give info on upgrading</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/64978d11e35bbe4dc0cbf077115705a6179928ff">64978d11</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-04T09:50:41Z</i>
</div>
<pre class='commit-message'>bug fix for resolving mpi_gather in shared lib</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/86533a49dbde4a80f2d2c153d61580e03d6cd914">86533a49</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-04T12:46:33Z</i>
</div>
<pre class='commit-message'>fix typos in counter descriptions</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/d4413b4e774ce98123c003419dec380c2b9bf629">d4413b4e</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-04T13:36:29Z</i>
</div>
<pre class='commit-message'>better error handling for opening old log files</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/72937c9a125bcbb5e5c71f17e1fffb5e2f0d8b2b">72937c9a</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-04T15:05:02Z</i>
</div>
<pre class='commit-message'>more doc updates</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b5bba287d43d9e9bcad4afff4bd261a94fbcaa71">b5bba287</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-08T16:50:47Z</i>
</div>
<pre class='commit-message'>add the darshan-diff utility back</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4bbe8e640eeaea440a794de765a1b9fd54f8560d">4bbe8e64</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-08T17:07:01Z</i>
</div>
<pre class='commit-message'>add diff routines to remaining modules</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/b770409e461781a9723c194b0c76e27620eefa79">b770409e</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-13T13:29:12Z</i>
</div>
<pre class='commit-message'>minor bug fix in darshan-diff tool</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/265bdaba58faa4fe164d22601c2ee4051ee45b2e">265bdaba</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-19T16:18:22Z</i>
</div>
<pre class='commit-message'>Merge branch 'dev-modular' into mmap-dev
Conflicts:
darshan-runtime/Makefile.in
darshan-runtime/darshan-core.h
darshan-runtime/lib/darshan-core.c
darshan-util/Makefile.in
darshan-util/darshan-bgq-logutils.c
darshan-util/darshan-diff.c
darshan-util/darshan-hdf5-logutils.c
darshan-util/darshan-logutils.c
darshan-util/darshan-mpiio-logutils.c
darshan-util/darshan-null-logutils.c
darshan-util/darshan-parser.c
darshan-util/darshan-pnetcdf-logutils.c
darshan-util/darshan-posix-logutils.c</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/96801d4a5894134dd47159b4f5cf9c31e797d4b5">96801d4a</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2016-01-19T17:44:11Z</i>
</div>
<pre class='commit-message'>fix post-merge build errors
dev-modular should be completely merged into mmap-dev branch</pre>
</li>
</ul>
<h4>50 changed files:</h4>
<ul>
<li class='file-stats'>
<a href='#diff-0'>
ChangeLog
</a>
</li>
<li class='file-stats'>
<a href='#diff-1'>
darshan-bgq-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-2'>
darshan-hdf5-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-3'>
darshan-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-4'>
darshan-mpiio-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-5'>
darshan-null-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-6'>
darshan-pnetcdf-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-7'>
darshan-posix-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-8'>
darshan-runtime/Makefile.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-9'>
darshan-runtime/configure
</a>
</li>
<li class='file-stats'>
<a href='#diff-10'>
darshan-runtime/configure.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-11'>
darshan-runtime/darshan-core.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-12'>
darshan-runtime/darshan-dynamic.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-13'>
darshan-runtime/darshan-gen-cc.pl.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-14'>
darshan-runtime/darshan-gen-cxx.pl.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-15'>
darshan-runtime/darshan-gen-fortran.pl.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-16'>
darshan-runtime/darshan.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-17'>
darshan-runtime/doc/darshan-runtime.txt
</a>
</li>
<li class='file-stats'>
<a href='#diff-18'>
darshan-runtime/lib/darshan-bgq.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-19'>
darshan-runtime/lib/darshan-core-init-finalize.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-20'>
darshan-runtime/lib/darshan-core.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-21'>
darshan-runtime/lib/darshan-hdf5.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-22'>
darshan-runtime/lib/darshan-mpiio.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-23'>
darshan-runtime/lib/darshan-null.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-24'>
darshan-runtime/lib/darshan-pnetcdf.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-25'>
darshan-runtime/lib/darshan-posix.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-26'>
darshan-test/regression/workstation-profile-conf/env.sh
</a>
</li>
<li class='file-stats'>
<a href='#diff-27'>
darshan-util/Makefile.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-28'>
darshan-util/configure
</a>
</li>
<li class='file-stats'>
<a href='#diff-29'>
darshan-util/configure.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-30'>
darshan-util/darshan-bgq-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-31'>
darshan-util/darshan-bgq-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-32'>
darshan-util/darshan-convert.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-33'>
darshan-util/darshan-diff.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-34'>
darshan-util/darshan-hdf5-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-35'>
darshan-util/darshan-hdf5-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-36'>
darshan-util/darshan-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-37'>
darshan-util/darshan-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-38'>
darshan-util/darshan-mpiio-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-39'>
darshan-util/darshan-mpiio-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-40'>
darshan-util/darshan-null-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-41'>
darshan-util/darshan-null-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-42'>
darshan-util/darshan-parser.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-43'>
darshan-util/darshan-pnetcdf-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-44'>
darshan-util/darshan-pnetcdf-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-45'>
darshan-util/darshan-posix-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-46'>
darshan-util/darshan-posix-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-47'>
darshan-util/darshan-stitch-logs.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-48'>
darshan-util/doc/darshan-util.txt
</a>
</li>
<li class='file-stats'>
<a href='#diff-49'>
doc/darshan-modularization.txt
</a>
</li>
</ul>
<h4>Changes:</h4>
<li id='diff-0'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-0'>
<strong>
ChangeLog
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/ChangeLog
</span><span style="color: #000000;background-color: #ddffdd">+++ b/ChangeLog
</span><span style="color: #aaaaaa">@@ -2,6 +2,20 @@
</span> Darshan Release Change Log
--------------------------
<span style="color: #000000;background-color: #ddffdd">+Darshan-3.0.0-pre3
+=============
+* add module-specific version fields to header to allow utilities
+ to handle different versions of a module's I/O data for backwards
+ compatibility -- NOTE: this breaks the log file parsing for logs
+ obtained using Darshan-3.0.0-pre2 & Darshan-3.0.0-pre1
+* bug fix in regression test scripts for setting proper environment
+ variables to use MPI profiling configuration for Fortran apps
+* bug fix in bzip2 log writing implementation in darshan-logutils
+* possible race conditions resolved in each module's shutdown code
+* general code, comment, and documentation cleanup
+* addition of module-specific counter descriptions printed prior
+ to parsing a modules I/O data in darshan-parser
+
</span> Darshan-3.0.0-pre2
=============
* add fix to install appropriate headers for linking external
</code></pre>
<br>
</li>
<li id='diff-1'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-1'>
<strong>
darshan-bgq-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-bgq-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-bgq-log-format.h
</span><span style="color: #aaaaaa">@@ -7,28 +7,42 @@
</span> #ifndef __DARSHAN_BGQ_LOG_FORMAT_H
#define __DARSHAN_BGQ_LOG_FORMAT_H
-#include "darshan-log-format.h"
-
<span style="color: #000000;background-color: #ddffdd">+/* current BGQ log format version */
+#define DARSHAN_BGQ_VER 1
</span>
#define BGQ_COUNTERS \
<span style="color: #000000;background-color: #ffdddd">- X(BGQ_CSJOBID, "control system jobid") \
- X(BGQ_NNODES, "number of BGQ compute nodes") \
- X(BGQ_RANKSPERNODE, "number of MPI ranks per node") \
- X(BGQ_DDRPERNODE, "size in MB of DDR3 per node") \
- X(BGQ_INODES, "number of i/o nodes") \
- X(BGQ_ANODES, "dimension of A torus") \
- X(BGQ_BNODES, "dimension of B torus") \
- X(BGQ_CNODES, "dimension of C torus") \
- X(BGQ_DNODES, "dimension of D torus") \
- X(BGQ_ENODES, "dimension of E torus") \
- X(BGQ_TORUSENABLED, "which dimensions are torus") \
- X(BGQ_NUM_INDICES, "end of counters")
</span><span style="color: #000000;background-color: #ddffdd">+ /* control system jobid*/\
+ X(BGQ_CSJOBID) \
+ /* number of BGQ compute nodes */\
+ X(BGQ_NNODES) \
+ /* number of MPI ranks per node */\
+ X(BGQ_RANKSPERNODE) \
+ /* size in MB of DDR3 per node */\
+ X(BGQ_DDRPERNODE) \
+ /* number of i/o nodes */\
+ X(BGQ_INODES) \
+ /* dimension of A torus */\
+ X(BGQ_ANODES) \
+ /* dimension of B torus */\
+ X(BGQ_BNODES) \
+ /* dimension of C torus */\
+ X(BGQ_CNODES) \
+ /* dimension of D torus */\
+ X(BGQ_DNODES) \
+ /* dimension of E torus */\
+ X(BGQ_ENODES) \
+ /* which dimensions are torus */\
+ X(BGQ_TORUSENABLED) \
+ /* end of counters */\
+ X(BGQ_NUM_INDICES)
</span>
#define BGQ_F_COUNTERS \
<span style="color: #000000;background-color: #ffdddd">- X(BGQ_F_TIMESTAMP, "timestamp when data was collected") \
- X(BGQ_F_NUM_INDICES, "end of counters")
</span><span style="color: #000000;background-color: #ddffdd">+ /* timestamp when data was collected */\
+ X(BGQ_F_TIMESTAMP) \
+ /* end of counters */\
+ X(BGQ_F_NUM_INDICES)
</span>
-#define X(a, b) a,
<span style="color: #000000;background-color: #ddffdd">+#define X(a) a,
</span> /* integer counters for the "BGQ" example module */
enum darshan_bgq_indices
{
</code></pre>
<br>
</li>
<li id='diff-2'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-2'>
<strong>
darshan-hdf5-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-hdf5-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-hdf5-log-format.h
</span><span style="color: #aaaaaa">@@ -7,7 +7,8 @@
</span> #ifndef __DARSHAN_HDF5_LOG_FORMAT_H
#define __DARSHAN_HDF5_LOG_FORMAT_H
-#include "darshan-log-format.h"
<span style="color: #000000;background-color: #ddffdd">+/* current HDF5 log format version */
+#define DARSHAN_HDF5_VER 1
</span>
#define HDF5_COUNTERS \
/* count of HDF5 opens */\
</code></pre>
<br>
</li>
<li id='diff-3'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-3'>
<strong>
darshan-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-log-format.h
</span><span style="color: #aaaaaa">@@ -31,44 +31,9 @@
</span> /* max length of exe string within job record (not counting '\0') */
#define DARSHAN_EXE_LEN (DARSHAN_JOB_RECORD_SIZE - sizeof(struct darshan_job) - 1)
<span style="color: #000000;background-color: #ddffdd">+/* max number of modules that can be used in a darshan log */
</span> #define DARSHAN_MAX_MODS 16
-/* X-macro for keeping module ordering consistent */
-/* NOTE: first val used to define module enum values,
<span style="color: #000000;background-color: #ffdddd">- * second val used to define module name strings, and
- * third val is used to provide the name of a
- * corresponding logutils structure for parsing module
- * data out of the log file (only used in darshan-util
- * component -- NULL can be passed if there are no
- * logutil definitions)
- */
</span>-#define DARSHAN_MODULE_IDS \
<span style="color: #000000;background-color: #ffdddd">- X(DARSHAN_NULL_MOD, "NULL", NULL) \
- X(DARSHAN_POSIX_MOD, "POSIX", &posix_logutils) \
- X(DARSHAN_MPIIO_MOD, "MPI-IO", &mpiio_logutils) \
- X(DARSHAN_HDF5_MOD, "HDF5", &hdf5_logutils) \
- X(DARSHAN_PNETCDF_MOD, "PNETCDF", &pnetcdf_logutils) \
- X(DARSHAN_BGQ_MOD, "BG/Q", &bgq_logutils)
</span>-
-/* unique identifiers to distinguish between available darshan modules */
-/* NOTES: - valid ids range from [0...DARSHAN_MAX_MODS-1]
<span style="color: #000000;background-color: #ffdddd">- * - order of ids control module shutdown order (and consequently, order in log file)
- */
</span>-#define X(a, b, c) a,
-typedef enum
-{
<span style="color: #000000;background-color: #ffdddd">- DARSHAN_MODULE_IDS
</span>-} darshan_module_id;
-#undef X
-
-/* module name strings */
-#define X(a, b, c) b,
-static char * const darshan_module_names[] =
-{
<span style="color: #000000;background-color: #ffdddd">- DARSHAN_MODULE_IDS
</span>-};
-#undef X
-
/* simple macros for accessing module flag bitfields */
#define DARSHAN_MOD_FLAG_SET(flags, id) flags = (flags | (1 << id))
#define DARSHAN_MOD_FLAG_UNSET(flags, id) flags = (flags & ~(1 << id))
<span style="color: #aaaaaa">@@ -106,6 +71,7 @@ struct darshan_header
</span> uint32_t partial_flag;
struct darshan_log_map rec_map;
struct darshan_log_map mod_map[DARSHAN_MAX_MODS];
<span style="color: #000000;background-color: #ddffdd">+ uint32_t mod_ver[DARSHAN_MAX_MODS];
</span> };
/* job-level metadata stored for this application */
<span style="color: #aaaaaa">@@ -126,4 +92,61 @@ struct darshan_base_record
</span> int64_t rank;
};
<span style="color: #000000;background-color: #ddffdd">+
+/************************************************
+ *** module-specific includes and definitions ***
+ ************************************************/
+
+#include "darshan-null-log-format.h"
+#include "darshan-posix-log-format.h"
+#include "darshan-mpiio-log-format.h"
+#include "darshan-hdf5-log-format.h"
+#include "darshan-pnetcdf-log-format.h"
+#include "darshan-bgq-log-format.h"
+
+/* X-macro for keeping module ordering consistent */
+/* NOTE: first val used to define module enum values,
+ * second val used to define module name strings,
+ * third val is the log format version for the module,
+ * and fourth val is used to provide the name of a
+ * corresponding logutils structure for parsing module
+ * data out of the log file (only used in darshan-util
+ * component -- NULL can be passed if there are no
+ * logutil definitions)
+ */
+#define DARSHAN_MODULE_IDS \
+ X(DARSHAN_NULL_MOD, "NULL", DARSHAN_NULL_VER, NULL) \
+ X(DARSHAN_POSIX_MOD, "POSIX", DARSHAN_POSIX_VER, &posix_logutils) \
+ X(DARSHAN_MPIIO_MOD, "MPI-IO", DARSHAN_MPIIO_VER, &mpiio_logutils) \
+ X(DARSHAN_HDF5_MOD, "HDF5", DARSHAN_HDF5_VER, &hdf5_logutils) \
+ X(DARSHAN_PNETCDF_MOD, "PNETCDF", DARSHAN_PNETCDF_VER, &pnetcdf_logutils) \
+ X(DARSHAN_BGQ_MOD, "BG/Q", DARSHAN_BGQ_VER, &bgq_logutils)
+
+/* unique identifiers to distinguish between available darshan modules */
+/* NOTES: - valid ids range from [0...DARSHAN_MAX_MODS-1]
+ * - order of ids control module shutdown order (and consequently, order in log file)
+ */
+#define X(a, b, c, d) a,
+typedef enum
+{
+ DARSHAN_MODULE_IDS
+} darshan_module_id;
+#undef X
+
+/* module name strings */
+#define X(a, b, c, d) b,
+static char * const darshan_module_names[] =
+{
+ DARSHAN_MODULE_IDS
+};
+#undef X
+
+/* module version numbers */
+#define X(a, b, c, d) c,
+static const int darshan_module_versions[] =
+{
+ DARSHAN_MODULE_IDS
+};
+#undef X
+
</span> #endif /* __DARSHAN_LOG_FORMAT_H */
</code></pre>
<br>
</li>
<li id='diff-4'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-4'>
<strong>
darshan-mpiio-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-mpiio-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-mpiio-log-format.h
</span><span style="color: #aaaaaa">@@ -7,7 +7,8 @@
</span> #ifndef __DARSHAN_MPIIO_LOG_FORMAT_H
#define __DARSHAN_MPIIO_LOG_FORMAT_H
-#include "darshan-log-format.h"
<span style="color: #000000;background-color: #ddffdd">+/* current MPI-IO log format version */
+#define DARSHAN_MPIIO_VER 1
</span>
/* TODO: maybe use a counter to track cases in which a derived datatype is used? */
</code></pre>
<br>
</li>
<li id='diff-5'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-5'>
<strong>
darshan-null-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-null-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-null-log-format.h
</span><span style="color: #aaaaaa">@@ -7,7 +7,8 @@
</span> #ifndef __DARSHAN_NULL_LOG_FORMAT_H
#define __DARSHAN_NULL_LOG_FORMAT_H
-#include "darshan-log-format.h"
<span style="color: #000000;background-color: #ddffdd">+/* current log format version, to support backwards compatibility */
+#define DARSHAN_NULL_VER 1
</span>
#define NULL_COUNTERS \
/* count of number of 'bar' function calls */\
</code></pre>
<br>
</li>
<li id='diff-6'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-6'>
<strong>
darshan-pnetcdf-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-pnetcdf-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-pnetcdf-log-format.h
</span><span style="color: #aaaaaa">@@ -7,7 +7,8 @@
</span> #ifndef __DARSHAN_PNETCDF_LOG_FORMAT_H
#define __DARSHAN_PNETCDF_LOG_FORMAT_H
-#include "darshan-log-format.h"
<span style="color: #000000;background-color: #ddffdd">+/* current PNETCDF log format version */
+#define DARSHAN_PNETCDF_VER 1
</span>
#define PNETCDF_COUNTERS \
/* count of PNETCDF independent opens */\
</code></pre>
<br>
</li>
<li id='diff-7'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-7'>
<strong>
darshan-posix-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-posix-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-posix-log-format.h
</span><span style="color: #aaaaaa">@@ -6,7 +6,8 @@
</span> #ifndef __DARSHAN_POSIX_LOG_FORMAT_H
#define __DARSHAN_POSIX_LOG_FORMAT_H
-#include "darshan-log-format.h"
<span style="color: #000000;background-color: #ddffdd">+/* current POSIX log format version */
+#define DARSHAN_POSIX_VER 1
</span>
#define POSIX_COUNTERS \
/* count of posix opens */\
<span style="color: #aaaaaa">@@ -175,11 +176,4 @@ struct darshan_posix_file
</span> double fcounters[POSIX_F_NUM_INDICES];
};
-/* This macro can be used to identify files that have been opened using
<span style="color: #000000;background-color: #ffdddd">- * pnetcdf, hdf5, or mpi-io, but were never opened at the posix level. As a
- * result the record will not necessarily have all of the expected fields
- * populated.
- */
</span>-#define POSIX_FILE_PARTIAL(__file)((((__file)->counters[POSIX_OPENS] || (__file)->counters[POSIX_FOPENS] || (__file)->counters[POSIX_STATS]) ? 0 : 1))
-
#endif /* __DARSHAN_POSIX_LOG_FORMAT_H */
</code></pre>
<br>
</li>
<li id='diff-8'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-8'>
<strong>
darshan-runtime/Makefile.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/Makefile.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/Makefile.in
</span><span style="color: #aaaaaa">@@ -35,8 +35,15 @@ CFLAGS_SHARED = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I$(srcdir)
</span>
LIBS = -lz @LIBBZ2@
-static-mod-objs = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o
-dynamic-mod-objs = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o
+DARSHAN_DYNAMIC_MOD_OBJS = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po
+
+ifdef DARSHAN_USE_BGQ
+DARSHAN_STATIC_MOD_OBJS += lib/darshan-bgq.o
+DARSHAN_DYNAMIC_MOD_OBJS += lib/darshan-bgq.po
+CFLAGS += -DDARSHAN_BGQ
+CFLAGS_SHARED += -DDARSHAN_BGQ
+endif
</span>
lib::
@mkdir -p $@
<span style="color: #aaaaaa">@@ -113,10 +120,10 @@ lib/lookup8.o: lib/lookup8.c
</span> lib/lookup8.po: lib/lookup8.c
$(CC) $(CFLAGS_SHARED) -c $< -o $@
-lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o $(static-mod-objs) lib/lookup3.o lib/lookup8.o
<span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o $(DARSHAN_STATIC_MOD_OBJS) lib/lookup3.o lib/lookup8.o
</span> ar rcs $@ $^
-lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po $(dynamic-mod-objs) lib/lookup3.po lib/lookup8.po
<span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po $(DARSHAN_DYNAMIC_MOD_OBJS) lib/lookup3.po lib/lookup8.po
</span> $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ -lpthread -lrt -lz -ldl
lib/libdarshan-stubs.a: lib/darshan-hdf5-stubs.o lib/darshan-pnetcdf-stubs.o
</code></pre>
<br>
</li>
<li id='diff-9'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-9'>
<strong>
darshan-runtime/configure
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/configure
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/configure
</span><span style="color: #aaaaaa">@@ -1,6 +1,6 @@
</span> #! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.69 for darshan-runtime 3.0.0-pre2.
<span style="color: #000000;background-color: #ddffdd">+# Generated by GNU Autoconf 2.69 for darshan-runtime 3.0.0-pre3.
</span> #
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
<span style="color: #aaaaaa">@@ -577,8 +577,8 @@ MAKEFLAGS=
</span> # Identity of this package.
PACKAGE_NAME='darshan-runtime'
PACKAGE_TARNAME='darshan-runtime'
-PACKAGE_VERSION='3.0.0-pre2'
-PACKAGE_STRING='darshan-runtime 3.0.0-pre2'
<span style="color: #000000;background-color: #ddffdd">+PACKAGE_VERSION='3.0.0-pre3'
+PACKAGE_STRING='darshan-runtime 3.0.0-pre3'
</span> PACKAGE_BUGREPORT=''
PACKAGE_URL=''
<span style="color: #aaaaaa">@@ -1244,7 +1244,7 @@ if test "$ac_init_help" = "long"; then
</span> # Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
-\`configure' configures darshan-runtime 3.0.0-pre2 to adapt to many kinds of systems.
<span style="color: #000000;background-color: #ddffdd">+\`configure' configures darshan-runtime 3.0.0-pre3 to adapt to many kinds of systems.
</span>
Usage: $0 [OPTION]... [VAR=VALUE]...
<span style="color: #aaaaaa">@@ -1305,7 +1305,7 @@ fi
</span>
if test -n "$ac_init_help"; then
case $ac_init_help in
<span style="color: #000000;background-color: #ffdddd">- short | recursive ) echo "Configuration of darshan-runtime 3.0.0-pre2:";;
</span><span style="color: #000000;background-color: #ddffdd">+ short | recursive ) echo "Configuration of darshan-runtime 3.0.0-pre3:";;
</span> esac
cat <<\_ACEOF
<span style="color: #aaaaaa">@@ -1409,7 +1409,7 @@ fi
</span> test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
-darshan-runtime configure 3.0.0-pre2
<span style="color: #000000;background-color: #ddffdd">+darshan-runtime configure 3.0.0-pre3
</span> generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
<span style="color: #aaaaaa">@@ -1761,7 +1761,7 @@ cat >config.log <<_ACEOF
</span> This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
-It was created by darshan-runtime $as_me 3.0.0-pre2, which was
<span style="color: #000000;background-color: #ddffdd">+It was created by darshan-runtime $as_me 3.0.0-pre3, which was
</span> generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
<span style="color: #aaaaaa">@@ -4296,7 +4296,7 @@ _ACEOF
</span> fi
-DARSHAN_VERSION="3.0.0-pre2"
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_VERSION="3.0.0-pre3"
</span>
<span style="color: #aaaaaa">@@ -4815,7 +4815,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
</span> # report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
-This file was extended by darshan-runtime $as_me 3.0.0-pre2, which was
<span style="color: #000000;background-color: #ddffdd">+This file was extended by darshan-runtime $as_me 3.0.0-pre3, which was
</span> generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
<span style="color: #aaaaaa">@@ -4877,7 +4877,7 @@ _ACEOF
</span> cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
-darshan-runtime config.status 3.0.0-pre2
<span style="color: #000000;background-color: #ddffdd">+darshan-runtime config.status 3.0.0-pre3
</span> configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"
</code></pre>
<br>
</li>
<li id='diff-10'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-10'>
<strong>
darshan-runtime/configure.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/configure.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/configure.in
</span><span style="color: #aaaaaa">@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.
</span> dnl You may need to use autoheader as well if changing any DEFINEs
dnl sanity checks, output header, location of scripts used here
-AC_INIT([darshan-runtime], [3.0.0-pre2])
<span style="color: #000000;background-color: #ddffdd">+AC_INIT([darshan-runtime], [3.0.0-pre3])
</span> AC_CONFIG_SRCDIR([darshan.h])
AC_CONFIG_AUX_DIR(../maint/config)
AC_CONFIG_HEADER(darshan-runtime-config.h)
</code></pre>
<br>
</li>
<li id='diff-11'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-11'>
<strong>
darshan-runtime/darshan-core.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-core.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-core.h
</span><span style="color: #aaaaaa">@@ -56,7 +56,7 @@ struct darshan_core_runtime
</span> int rec_hash_cnt;
struct darshan_core_module* mod_array[DARSHAN_MAX_MODS];
int mod_mem_used;
<span style="color: #000000;background-color: #ffdddd">- char comp_buf[DARSHAN_COMP_BUF_SIZE];
</span><span style="color: #000000;background-color: #ddffdd">+ char *comp_buf;
</span> double wtime_offset;
};
</code></pre>
<br>
</li>
<li id='diff-12'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-12'>
<strong>
darshan-runtime/darshan-dynamic.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-dynamic.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-dynamic.h
</span><span style="color: #aaaaaa">@@ -114,10 +114,22 @@ DARSHAN_EXTERN_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *nu
</span> DARSHAN_EXTERN_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));
DARSHAN_EXTERN_DECL(PMPI_Op_create, int, (MPI_User_function *function, int commute, MPI_Op *op));
DARSHAN_EXTERN_DECL(PMPI_Op_free, int, (MPI_Op *op));
<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST
+DARSHAN_EXTERN_DECL(PMPI_Reduce, int, (const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));
+#else
</span> DARSHAN_EXTERN_DECL(PMPI_Reduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));
<span style="color: #000000;background-color: #ddffdd">+#endif
+#ifdef HAVE_MPIIO_CONST
+DARSHAN_EXTERN_DECL(PMPI_Send, int, (const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));
+#else
</span> DARSHAN_EXTERN_DECL(PMPI_Send, int, (void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));
<span style="color: #000000;background-color: #ddffdd">+#endif
</span> DARSHAN_EXTERN_DECL(PMPI_Recv, int, (void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status));
<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST
</span> DARSHAN_EXTERN_DECL(PMPI_Gather, int, (const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));
<span style="color: #000000;background-color: #ddffdd">+#else
+DARSHAN_EXTERN_DECL(PMPI_Gather, int, (void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));
+#endif
</span>
#endif
</code></pre>
<br>
</li>
<li id='diff-13'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-13'>
<strong>
darshan-runtime/darshan-gen-cc.pl.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-cc.pl.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-cc.pl.in
</span><span style="color: #aaaaaa">@@ -261,8 +261,8 @@ print OUTPUT<<"EOF";
</span> # is MPI in there?
grep MPI \$tmpfile >& /dev/null
rc_mpi=\$?
<span style="color: #000000;background-color: #ffdddd">- # is PMPI being used for any init, finalize, or mpi-io calls?
- grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span><span style="color: #000000;background-color: #ddffdd">+ # is PMPI being used for PMPI_File_open?
+ grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span> rc_pmpi=\$?
rm \$tmpfile >& /dev/null
</code></pre>
<br>
</li>
<li id='diff-14'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-14'>
<strong>
darshan-runtime/darshan-gen-cxx.pl.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-cxx.pl.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-cxx.pl.in
</span><span style="color: #aaaaaa">@@ -263,8 +263,8 @@ print OUTPUT<<"EOF";
</span> grep MPI \$tmpfile >& /dev/null
rc_mpi=\$?
<span style="color: #000000;background-color: #ffdddd">- # is PMPI being used for any init, finalize, or mpi-io calls?
- grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span><span style="color: #000000;background-color: #ddffdd">+ # is PMPI being used for PMPI_File_open?
+ grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span> rc_pmpi=\$?
CXXMPICH=-lmpichcxx
</code></pre>
<br>
</li>
<li id='diff-15'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-15'>
<strong>
darshan-runtime/darshan-gen-fortran.pl.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan-gen-fortran.pl.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan-gen-fortran.pl.in
</span><span style="color: #aaaaaa">@@ -271,8 +271,8 @@ print OUTPUT<<"EOF";
</span> grep -i MPI \$tmpfile >& /dev/null
rc_mpi=\$?
<span style="color: #000000;background-color: #ffdddd">- # is PMPI being used for any init, finalize, or mpi-io calls?
- grep -E \\(PMPI_File_\\)\\|\\(PMPI_Init\\)\\|\\(PMPI_Finalize\\) \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span><span style="color: #000000;background-color: #ddffdd">+ # is PMPI being used for PMPI_File_open?
+ grep -E PMPI_File_open \$tmpfile | grep -v -E \\(mpich.*\\.a\\) |grep \\(PMPI >& /dev/null
</span> rc_pmpi=\$?
</code></pre>
<br>
</li>
<li id='diff-16'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-16'>
<strong>
darshan-runtime/darshan.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan.h
</span><span style="color: #aaaaaa">@@ -125,17 +125,17 @@ void darshan_core_lookup_record(
</span>
/* darshan_core_register_record()
*
<span style="color: #000000;background-color: #ffdddd">- * Register the Darshan record given by 'name' with the darshan-core
- * runtime, allowing it to be properly tracked and (potentially)
- * correlated with records from other modules. 'len' is the size of
- * the name pointer (string length for string names), and 'printable_flag'
- * indicates whether the name is a string. 'mod_limit_flag' is set if
- * the calling module is out of memory (to prevent darshan-core from
- * creating new records and to just search existing records) and 'mod_id'
- * is the identifier of the calling module. 'rec_id' is an output pointer
- * storing the correspoing Darshan record identifier and 'file_alignment'
- * is an output pointer storing the file system alignment value for the
- * given record.
</span><span style="color: #000000;background-color: #ddffdd">+ * Register a record with the darshan-core runtime, allowing it to be
+ * properly tracked and (potentially) correlated with records from other
+ * modules. 'name' is the the name of the Darshan record (e.g., the full
+ * file path) and 'len' is the size of the name pointer (string length
+ * for string names). 'mod_id' is the identifier of the calling module,
+ * 'printable_flag' indicates whether the name is a string, and
+ * 'mod_limit_flag' is set if the calling module is out of memory (to
+ * prevent darshan-core from creating new records and to just search
+ * through existing records). 'rec_id' is an output pointer storing the
+ * correspoing Darshan record identifier and 'file_alignment' is an output
+ * pointer storing the file system alignment value for the given record.
</span> */
int darshan_core_register_record(
darshan_record_id rec_id,
</code></pre>
<br>
</li>
<li id='diff-17'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-17'>
<strong>
darshan-runtime/doc/darshan-runtime.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/doc/darshan-runtime.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/doc/darshan-runtime.txt
</span><span style="color: #aaaaaa">@@ -16,7 +16,10 @@ used by the application.
</span> The darshan-runtime instrumentation only instruments MPI applications (the
application must at least call `MPI_Init()` and `MPI_Finalize()`). However,
it captures both MPI-IO and POSIX file access. It also captures limited
-information about HDF5 and PnetCDF access.
<span style="color: #000000;background-color: #ddffdd">+information about HDF5 and PnetCDF access. Darshan also exposes an API that
+can be used to develop and add new instrumentation modules (for other I/O library
+interfaces or to gather system-specific data, for instance), as detailed in
+http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-modularization.html[this document].
</span>
This document provides generic installation instructions, but "recipes" for
several common HPC systems are provided at the end of the document as well.
<span style="color: #aaaaaa">@@ -311,7 +314,7 @@ Please set your environment to use the GNU programming environment before
</span> configuring or compiling Darshan. Although Darshan can be built with a
variety of compilers, the GNU compilers are recommended because it will
produce a Darshan library that is interoperable with the widest range
-of compmilers and linkers. On most Cray systems you can enable the GNU
<span style="color: #000000;background-color: #ddffdd">+of compilers and linkers. On most Cray systems you can enable the GNU
</span> programming environment with a command similar to "module swap PrgEnv-pgi
PrgEnv-gnu". Please see your site documentation for information about
how to switch programming environments.
<span style="color: #aaaaaa">@@ -463,6 +466,40 @@ older versions of Open MPI, please refer to the following mailing list discussio
</span>
http://www.open-mpi.org/community/lists/devel/2013/01/11907.php
<span style="color: #000000;background-color: #ddffdd">+== Upgrading to Darshan 3.x from 2.x
+
+Beginning with Darshan 3.0.0, Darshan has been rewritten to modularize its runtime environment
+and log file format to simplify the addition of new I/O characterization data. The process of
+compiling and installing the Darshan 3.x source code should essentially be identical to this
+process on Darshan 2.x. Therefore, the installation recipes given in the previous section
+should work irrespective of the Darshan version being used. Similarly, the manner in which
+Darshan is used should be the same across versions -- the sections in this document regarding
+Darshan link:darshan-runtime.html#_environment_preparation[environment preparation],
+instrumenting link:darshan-runtime.html#_instrumenting_statically_linked_applications[statically
+linked applications] and link:darshan-runtime.html#_instrumenting_dynamically_linked_applications[
+dynamically linked applications], and using link:darshan-runtime.html#_runtime_environment_variables[
+runtime environment variables] are equally applicable to both versions.
+
+However, we do provide some suggestions and expectations for system administrators to keep in
+mind when upgrading to Darshan 3.x:
+
+* Log file compatibility was broken in the upgrade, and thus 3.x log utilities do not
+work directly with logs generated by 2.x versions of Darshan (and vice versa).
+ - There is currently no tool for converting 2.x logs into the 3.x log format.
+ - The `darshan-logutils` library will provide error messages to indicate whether a given
+log file is incompatible with the correspnonding library version.
+
+* The darshan log file extension has been changed from `.darshan.gz` (or `.darshan.bz2` for
+log files converted to use bzip2 compression) to `.darshan`.
+ - A field in the Darshan log header is used to indicate whether a log is compressed using
+libz or bzip2 compression.
+
+* We encourage administrators to use the same log file directory for version 3.x as had been
+used for version 2.x.
+ - Within this directory, the determination on which set of log utilities (version 2.x
+or version 3.x) to use can be based on the file extension for a given log (as explained
+above).
+
</span> == Runtime environment variables
The Darshan library honors the following environment variables to modify
</code></pre>
<br>
</li>
<li id='diff-18'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-18'>
<strong>
darshan-runtime/lib/darshan-bgq.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-bgq.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-bgq.c
</span><span style="color: #aaaaaa">@@ -16,7 +16,6 @@
</span>
#include "uthash.h"
#include "darshan.h"
-#include "darshan-bgq-log-format.h"
#include "darshan-dynamic.h"
#include <mpix.h>
<span style="color: #aaaaaa">@@ -54,7 +53,7 @@ static int instrumentation_disabled = 0;
</span> static int my_rank = -1;
static int darshan_mem_alignment = 1;
-/* internal helper functions for the "NULL" module */
<span style="color: #000000;background-color: #ddffdd">+/* internal helper functions for the BGQ module */
</span> void bgq_runtime_initialize(void);
/* forward declaration for module functions needed to interface with darshan-core */
<span style="color: #aaaaaa">@@ -62,7 +61,7 @@ static void bgq_begin_shutdown(void);
</span> static void bgq_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs, int shared_rec_count, void **buffer, int *size);
static void bgq_shutdown(void);
-/* macros for obtaining/releasing the "NULL" module lock */
<span style="color: #000000;background-color: #ddffdd">+/* macros for obtaining/releasing the BGQ module lock */
</span> #define BGQ_LOCK() pthread_mutex_lock(&bgq_runtime_mutex)
#define BGQ_UNLOCK() pthread_mutex_unlock(&bgq_runtime_mutex)
<span style="color: #aaaaaa">@@ -141,13 +140,15 @@ void bgq_runtime_initialize()
</span> if(mem_limit == 0)
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
<span style="color: #000000;background-color: #ffdddd">- /* no enough memory to fit bgq module */
</span><span style="color: #000000;background-color: #ddffdd">+ /* not enough memory to fit bgq module */
</span> if (mem_limit < sizeof(*bgq_runtime))
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
<span style="color: #aaaaaa">@@ -156,6 +157,7 @@ void bgq_runtime_initialize()
</span> if(!bgq_runtime)
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
memset(bgq_runtime, 0, sizeof(*bgq_runtime));
<span style="color: #aaaaaa">@@ -188,7 +190,7 @@ void bgq_runtime_initialize()
</span> return;
}
-/* Perform any necessary steps prior to shutting down for the "NULL" module. */
<span style="color: #000000;background-color: #ddffdd">+/* Perform any necessary steps prior to shutting down for the BGQ module. */
</span> static void bgq_begin_shutdown()
{
BGQ_LOCK();
<span style="color: #aaaaaa">@@ -212,7 +214,7 @@ static int cmpr(const void *p1, const void *p2)
</span> return ((*a == *b) ? 0 : ((*a < *b) ? -1 : 1));
}
-/* Pass output data for the "BGQ" module back to darshan-core to log to file. */
<span style="color: #000000;background-color: #ddffdd">+/* Pass output data for the BGQ module back to darshan-core to log to file. */
</span> static void bgq_get_output_data(
MPI_Comm mod_comm,
darshan_record_id *shared_recs,
<span style="color: #aaaaaa">@@ -220,8 +222,7 @@ static void bgq_get_output_data(
</span> void **buffer,
int *size)
{
-
<span style="color: #000000;background-color: #ffdddd">- /* Just set the output buffer to point at the array of the "BGQ" module's
</span><span style="color: #000000;background-color: #ddffdd">+ /* Just set the output buffer to point at the array of the BGQ module's
</span> * I/O records, and set the output size according to the number of records
* currently being tracked.
*/
<span style="color: #aaaaaa">@@ -279,7 +280,7 @@ static void bgq_get_output_data(
</span> return;
}
-/* Shutdown the "BGQ" module by freeing up all data structures. */
<span style="color: #000000;background-color: #ddffdd">+/* Shutdown the BGQ module by freeing up all data structures. */
</span> static void bgq_shutdown()
{
if (bgq_runtime)
</code></pre>
<br>
</li>
<li id='diff-19'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-19'>
<strong>
darshan-runtime/lib/darshan-core-init-finalize.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-core-init-finalize.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-core-init-finalize.c
</span><span style="color: #aaaaaa">@@ -120,9 +120,22 @@ DARSHAN_FORWARD_DECL(PMPI_Type_get_envelope, int, (MPI_Datatype datatype, int *n
</span> DARSHAN_FORWARD_DECL(PMPI_Type_size, int, (MPI_Datatype datatype, int *size));
DARSHAN_FORWARD_DECL(PMPI_Op_create, int, (MPI_User_function *function, int commute, MPI_Op *op));
DARSHAN_FORWARD_DECL(PMPI_Op_free, int, (MPI_Op *op));
<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST
+DARSHAN_FORWARD_DECL(PMPI_Reduce, int, (const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));
+#else
</span> DARSHAN_FORWARD_DECL(PMPI_Reduce, int, (void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, MPI_Op op, int root, MPI_Comm comm));
<span style="color: #000000;background-color: #ddffdd">+#endif
+#ifdef HAVE_MPIIO_CONST
+DARSHAN_FORWARD_DECL(PMPI_Send, int, (const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));
+#else
</span> DARSHAN_FORWARD_DECL(PMPI_Send, int, (void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm));
<span style="color: #000000;background-color: #ddffdd">+#endif
</span> DARSHAN_FORWARD_DECL(PMPI_Recv, int, (void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status));
<span style="color: #000000;background-color: #ddffdd">+#ifdef HAVE_MPIIO_CONST
+DARSHAN_FORWARD_DECL(PMPI_Gather, int, (const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));
+#else
+DARSHAN_FORWARD_DECL(PMPI_Gather, int, (void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, int recvcount, MPI_Datatype recvtype, int root, MPI_Comm comm));
+#endif
</span>
void resolve_mpi_symbols (void)
{
<span style="color: #aaaaaa">@@ -182,6 +195,7 @@ void resolve_mpi_symbols (void)
</span> MAP_OR_FAIL(PMPI_Reduce);
MAP_OR_FAIL(PMPI_Send);
MAP_OR_FAIL(PMPI_Recv);
<span style="color: #000000;background-color: #ddffdd">+ MAP_OR_FAIL(PMPI_Gather);
</span>
return;
}
</code></pre>
<br>
</li>
<li id='diff-20'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-20'>
<strong>
darshan-runtime/lib/darshan-core.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-core.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-core.c
</span><span style="color: #aaaaaa">@@ -313,6 +313,8 @@ void darshan_core_initialize(int argc, char **argv)
</span> void darshan_core_shutdown()
{
struct darshan_core_runtime *final_core;
<span style="color: #000000;background-color: #ddffdd">+ struct darshan_header out_header;
+ struct darshan_job out_job;
</span> int internal_timing_flag = 0;
struct tm *start_tm;
time_t start_time_tmp;
<span style="color: #aaaaaa">@@ -336,8 +338,6 @@ void darshan_core_shutdown()
</span> int all_ret = 0;
int i;
uint64_t gz_fp = 0;
<span style="color: #000000;background-color: #ffdddd">- struct darshan_header out_header;
- struct darshan_job out_job;
</span> MPI_File log_fh;
MPI_Status status;
<span style="color: #aaaaaa">@@ -358,14 +358,14 @@ void darshan_core_shutdown()
</span> darshan_core = NULL;
DARSHAN_CORE_UNLOCK();
<span style="color: #000000;background-color: #ffdddd">- memcpy(&out_job, final_core->log_job_p, sizeof(struct darshan_job));
</span>-
/* XXX just copy mmap files somewhere else to avoid corruption */
DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);
if(my_rank == 0)
system("cp /tmp/darshan* ~/Desktop");
DARSHAN_MPI_CALL(PMPI_Barrier)(MPI_COMM_WORLD);
<span style="color: #000000;background-color: #ddffdd">+ memcpy(&out_job, final_core->log_job_p, sizeof(struct darshan_job));
+
</span> /* indicate in the metadata field of the temporary darshan log file that
* the darshan shutdown process was invoked on the data in the log. since
* we have no way of knowing how far the shutdown process got, the data
<span style="color: #aaaaaa">@@ -376,6 +376,10 @@ void darshan_core_shutdown()
</span> int meta_remain = DARSHAN_JOB_METADATA_LEN - strlen(final_core->log_job_p->metadata) - 1;
snprintf(m, meta_remain, "darshan_shutdown=yes\n");
<span style="color: #000000;background-color: #ddffdd">+ /* we also need to set which modules were registered on this process and
+ * call into those modules and give them a chance to perform any necessary
+ * pre-shutdown steps.
+ */
</span> for(i = 0; i < DARSHAN_MAX_MODS; i++)
{
if(final_core->mod_array[i])
<span style="color: #aaaaaa">@@ -385,6 +389,13 @@ void darshan_core_shutdown()
</span> }
}
<span style="color: #000000;background-color: #ddffdd">+ final_core->comp_buf = malloc(DARSHAN_COMP_BUF_SIZE);
+ if(!(final_core->comp_buf))
+ {
+ darshan_core_cleanup(final_core);
+ return;
+ }
+
</span> logfile_name = malloc(PATH_MAX);
if(!logfile_name)
{
<span style="color: #aaaaaa">@@ -639,6 +650,9 @@ void darshan_core_shutdown()
</span> */
DARSHAN_MPI_CALL(PMPI_Reduce)(&(final_core->log_hdr_p->partial_flag),
&(out_header.partial_flag), 1, MPI_UINT32_T, MPI_BOR, 0, MPI_COMM_WORLD);
<span style="color: #000000;background-color: #ddffdd">+ DARSHAN_MPI_CALL(PMPI_Reduce)(&(final_core->log_hdr_p->mod_ver),
+ &(out_header.mod_ver), DARSHAN_MAX_MODS, MPI_UINT32_T, MPI_MAX,
+ 0, MPI_COMM_WORLD);
</span> if(my_rank == 0)
{
/* rank 0 is responsible for writing the log header */
<span style="color: #aaaaaa">@@ -1550,6 +1564,8 @@ static void darshan_core_cleanup(struct darshan_core_runtime* core)
</span> }
}
<span style="color: #000000;background-color: #ddffdd">+ if(core->comp_buf)
+ free(core->comp_buf);
</span> free(core);
return;
<span style="color: #aaaaaa">@@ -1609,6 +1625,7 @@ void darshan_core_register_module(
</span>
/* register module with darshan */
darshan_core->mod_array[mod_id] = mod;
<span style="color: #000000;background-color: #ddffdd">+ darshan_core->log_hdr_p->mod_ver[mod_id] = darshan_module_versions[mod_id];
</span> DARSHAN_CORE_UNLOCK();
/* set the memory alignment and calling process's rank, if desired */
</code></pre>
<br>
</li>
<li id='diff-21'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-21'>
<strong>
darshan-runtime/lib/darshan-hdf5.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-hdf5.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-hdf5.c
</span><span style="color: #aaaaaa">@@ -22,7 +22,6 @@
</span> #include "uthash.h"
#include "darshan.h"
-#include "darshan-hdf5-log-format.h"
#include "darshan-dynamic.h"
/* hope this doesn't change any time soon */
<span style="color: #aaaaaa">@@ -511,7 +510,9 @@ static void hdf5_get_output_data(
</span> {
red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_hdf5_file));
if(!red_recv_buf)
<span style="color: #000000;background-color: #ddffdd">+ {
</span> return;
<span style="color: #000000;background-color: #ddffdd">+ }
</span> }
/* construct a datatype for a HDF5 file record. This is serving no purpose
</code></pre>
<br>
</li>
<li id='diff-22'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-22'>
<strong>
darshan-runtime/lib/darshan-mpiio.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-mpiio.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-mpiio.c
</span><span style="color: #aaaaaa">@@ -23,7 +23,6 @@
</span> #include "uthash.h"
#include "darshan.h"
-#include "darshan-mpiio-log-format.h"
#include "darshan-dynamic.h"
/* The mpiio_file_runtime structure maintains necessary runtime metadata
<span style="color: #aaaaaa">@@ -1386,7 +1385,9 @@ static void mpiio_get_output_data(
</span> {
red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_mpiio_file));
if(!red_recv_buf)
<span style="color: #000000;background-color: #ddffdd">+ {
</span> return;
<span style="color: #000000;background-color: #ddffdd">+ }
</span> }
/* construct a datatype for a MPIIO file record. This is serving no purpose
</code></pre>
<br>
</li>
<li id='diff-23'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-23'>
<strong>
darshan-runtime/lib/darshan-null.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-null.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-null.c
</span><span style="color: #aaaaaa">@@ -16,7 +16,6 @@
</span>
#include "uthash.h"
#include "darshan.h"
-#include "darshan-null-log-format.h"
/* The "NULL" module is an example instrumentation module implementation provided
* with Darshan, primarily to indicate how arbitrary modules may be integrated
</code></pre>
<br>
</li>
<li id='diff-24'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-24'>
<strong>
darshan-runtime/lib/darshan-pnetcdf.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-pnetcdf.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-pnetcdf.c
</span><span style="color: #aaaaaa">@@ -22,7 +22,6 @@
</span> #include "uthash.h"
#include "darshan.h"
-#include "darshan-pnetcdf-log-format.h"
#include "darshan-dynamic.h"
DARSHAN_FORWARD_DECL(ncmpi_create, int, (MPI_Comm comm, const char *path, int cmode, MPI_Info info, int *ncidp));
<span style="color: #aaaaaa">@@ -526,7 +525,9 @@ static void pnetcdf_get_output_data(
</span> {
red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_pnetcdf_file));
if(!red_recv_buf)
<span style="color: #000000;background-color: #ddffdd">+ {
</span> return;
<span style="color: #000000;background-color: #ddffdd">+ }
</span> }
/* construct a datatype for a PNETCDF file record. This is serving no purpose
</code></pre>
<br>
</li>
<li id='diff-25'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-25'>
<strong>
darshan-runtime/lib/darshan-posix.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-posix.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-posix.c
</span><span style="color: #aaaaaa">@@ -30,7 +30,6 @@
</span> #include "utlist.h"
#include "darshan.h"
-#include "darshan-posix-log-format.h"
#include "darshan-dynamic.h"
#ifndef HAVE_OFF64_T
<span style="color: #aaaaaa">@@ -40,8 +39,6 @@ typedef int64_t off64_t;
</span> #define aiocb64 aiocb
#endif
-/* TODO: more libc, fgetc, etc etc etc. */
-
DARSHAN_FORWARD_DECL(open, int, (const char *path, int flags, ...));
DARSHAN_FORWARD_DECL(open64, int, (const char *path, int flags, ...));
DARSHAN_FORWARD_DECL(creat, int, (const char* path, mode_t mode));
<span style="color: #aaaaaa">@@ -1892,10 +1889,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_MODE] = inoutfile->counters[POSIX_MODE];
- else
- tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
</span>
/* sum */
for(j=POSIX_BYTES_READ; j<=POSIX_BYTES_WRITTEN; j++)
<span style="color: #aaaaaa">@@ -1918,10 +1912,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_MEM_ALIGNMENT] = inoutfile->counters[POSIX_MEM_ALIGNMENT];
- else
- tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
</span>
/* sum */
for(j=POSIX_FILE_NOT_ALIGNED; j<=POSIX_FILE_NOT_ALIGNED; j++)
<span style="color: #aaaaaa">@@ -1929,10 +1920,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_FILE_ALIGNMENT] = inoutfile->counters[POSIX_FILE_ALIGNMENT];
- else
- tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
</span>
/* skip POSIX_MAX_*_TIME_SIZE; handled in floating point section */
<span style="color: #aaaaaa">@@ -2302,7 +2290,9 @@ static void posix_get_output_data(
</span> {
red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_posix_file));
if(!red_recv_buf)
<span style="color: #000000;background-color: #ddffdd">+ {
</span> return;
<span style="color: #000000;background-color: #ddffdd">+ }
</span> }
/* construct a datatype for a POSIX file record. This is serving no purpose
<span style="color: #aaaaaa">@@ -2363,7 +2353,7 @@ static void posix_shutdown()
</span> free(posix_runtime->file_runtime_array);
free(posix_runtime);
posix_runtime = NULL;
-
<span style="color: #000000;background-color: #ddffdd">+
</span> return;
}
</code></pre>
<br>
</li>
<li id='diff-26'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-26'>
<strong>
darshan-test/regression/workstation-profile-conf/env.sh
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-test/regression/workstation-profile-conf/env.sh
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-test/regression/workstation-profile-conf/env.sh
</span><span style="color: #aaaaaa">@@ -34,5 +34,8 @@ export MPICC_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cc
</span> export MPICXX_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-cxx
export MPIF90_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f
export MPIF77_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f
<span style="color: #000000;background-color: #ddffdd">+# MPICH 3.1.1 and newer use MPIFORT rather than MPIF90 and MPIF77 in env var
+# name
+export MPIFORT_PROFILE=$DARSHAN_PATH/share/mpi-profile/darshan-f
</span>
export DARSHAN_RUNJOB="mpiexec -n $DARSHAN_DEFAULT_NPROCS"
</code></pre>
<br>
</li>
<li id='diff-27'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-27'>
<strong>
darshan-util/Makefile.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/Makefile.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/Makefile.in
</span><span style="color: #aaaaaa">@@ -15,6 +15,10 @@ pkgconfigdir = $(DESTDIR)$(libdir)/pkgconfig
</span> DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h
DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h
DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS = darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o
+DARSHAN_DYNAMIC_MOD_OBJS = darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po
+
+#XXX BGQ
</span>
DARSHAN_ENABLE_SHARED=@DARSHAN_ENABLE_SHARED@
<span style="color: #aaaaaa">@@ -79,13 +83,12 @@ darshan-bgq-logutils.o: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-lo
</span> darshan-bgq-logutils.po: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2
$(CC) $(CFLAGS_SHARED) -c $< -o $@
<span style="color: #000000;background-color: #ddffdd">+libdarshan-util.a: darshan-logutils.o $(DARSHAN_STATIC_MOD_OBJS)
+ ar rcs libdarshan-util.a $^
</span>
-libdarshan-util.so: darshan-logutils.po darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po
<span style="color: #000000;background-color: #ddffdd">+libdarshan-util.so: darshan-logutils.po $(DARSHAN_DYNAMIC_MOD_OBJS)
</span> $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ $(LIBS)
-libdarshan-util.a: darshan-logutils.o darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o
- ar rcs libdarshan-util.a $^
-
jenkins-hash-gen: jenkins-hash-gen.c lookup3.o
$(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
<span style="color: #aaaaaa">@@ -93,19 +96,19 @@ lookup3.o: lookup3.c
</span> $(CC) $(CFLAGS) -c $< -o $@
darshan-analyzer: darshan-analyzer.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
<span style="color: #000000;background-color: #ddffdd">+ $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
</span>
-darshan-convert: darshan-convert.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) libdarshan-util.a lookup3.o | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
<span style="color: #000000;background-color: #ddffdd">+darshan-convert: darshan-convert.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a lookup3.o | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $< lookup3.o libdarshan-util.a -o $@ $(LIBS)
</span>
darshan-diff: darshan-diff.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
$(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
darshan-parser: darshan-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
<span style="color: #000000;background-color: #ddffdd">+ $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
</span>
-darshan-stitch-logs: darshan-stitch-logs.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
<span style="color: #000000;background-color: #ddffdd">+darshan-stitch-logs: darshan-stitch-logs.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
</span>
#test/gztest: test/gztest.c mktestdir
# $(CC) $(CFLAGS) $(LDFLAGS) -lz $< -o $@
<span style="color: #aaaaaa">@@ -122,6 +125,7 @@ install:: all
</span> install -m 755 darshan-convert $(bindir)
install -m 755 darshan-diff $(bindir)
install -m 755 darshan-parser $(bindir)
<span style="color: #000000;background-color: #ddffdd">+ install -m 755 darshan-stitch-logs $(bindir)
</span> install -m 755 $(srcdir)/darshan-summary-per-file.sh $(bindir)
install -m 755 libdarshan-util.a $(libdir)
ifeq ($(DARSHAN_ENABLE_SHARED),1)
<span style="color: #aaaaaa">@@ -133,6 +137,7 @@ endif
</span> install -m 644 $(srcdir)/darshan-hdf5-logutils.h $(includedir)
install -m 644 $(srcdir)/darshan-pnetcdf-logutils.h $(includedir)
install -m 644 $(srcdir)/darshan-bgq-logutils.h $(includedir)
<span style="color: #000000;background-color: #ddffdd">+ install -m 644 $(srcdir)/darshan-null-logutils.h $(includedir)
</span> install -m 644 $(srcdir)/../darshan-posix-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-mpiio-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-hdf5-log-format.h $(includedir)
</code></pre>
<br>
</li>
<li id='diff-28'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-28'>
<strong>
darshan-util/configure
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/configure
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/configure
</span><span style="color: #aaaaaa">@@ -1,6 +1,6 @@
</span> #! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.69 for darshan-util 3.0.0-pre2.
<span style="color: #000000;background-color: #ddffdd">+# Generated by GNU Autoconf 2.69 for darshan-util 3.0.0-pre3.
</span> #
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
<span style="color: #aaaaaa">@@ -577,8 +577,8 @@ MAKEFLAGS=
</span> # Identity of this package.
PACKAGE_NAME='darshan-util'
PACKAGE_TARNAME='darshan-util'
-PACKAGE_VERSION='3.0.0-pre2'
-PACKAGE_STRING='darshan-util 3.0.0-pre2'
<span style="color: #000000;background-color: #ddffdd">+PACKAGE_VERSION='3.0.0-pre3'
+PACKAGE_STRING='darshan-util 3.0.0-pre3'
</span> PACKAGE_BUGREPORT=''
PACKAGE_URL=''
<span style="color: #aaaaaa">@@ -1236,7 +1236,7 @@ if test "$ac_init_help" = "long"; then
</span> # Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
-\`configure' configures darshan-util 3.0.0-pre2 to adapt to many kinds of systems.
<span style="color: #000000;background-color: #ddffdd">+\`configure' configures darshan-util 3.0.0-pre3 to adapt to many kinds of systems.
</span>
Usage: $0 [OPTION]... [VAR=VALUE]...
<span style="color: #aaaaaa">@@ -1297,7 +1297,7 @@ fi
</span>
if test -n "$ac_init_help"; then
case $ac_init_help in
<span style="color: #000000;background-color: #ffdddd">- short | recursive ) echo "Configuration of darshan-util 3.0.0-pre2:";;
</span><span style="color: #000000;background-color: #ddffdd">+ short | recursive ) echo "Configuration of darshan-util 3.0.0-pre3:";;
</span> esac
cat <<\_ACEOF
<span style="color: #aaaaaa">@@ -1393,7 +1393,7 @@ fi
</span> test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
-darshan-util configure 3.0.0-pre2
<span style="color: #000000;background-color: #ddffdd">+darshan-util configure 3.0.0-pre3
</span> generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
<span style="color: #aaaaaa">@@ -1758,7 +1758,7 @@ cat >config.log <<_ACEOF
</span> This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
-It was created by darshan-util $as_me 3.0.0-pre2, which was
<span style="color: #000000;background-color: #ddffdd">+It was created by darshan-util $as_me 3.0.0-pre3, which was
</span> generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
<span style="color: #aaaaaa">@@ -4105,7 +4105,7 @@ fi
</span> done
-DARSHAN_UTIL_VERSION="3.0.0-pre2"
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_UTIL_VERSION="3.0.0-pre3"
</span>
<span style="color: #aaaaaa">@@ -4621,7 +4621,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
</span> # report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
-This file was extended by darshan-util $as_me 3.0.0-pre2, which was
<span style="color: #000000;background-color: #ddffdd">+This file was extended by darshan-util $as_me 3.0.0-pre3, which was
</span> generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
<span style="color: #aaaaaa">@@ -4683,7 +4683,7 @@ _ACEOF
</span> cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
-darshan-util config.status 3.0.0-pre2
<span style="color: #000000;background-color: #ddffdd">+darshan-util config.status 3.0.0-pre3
</span> configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"
</code></pre>
<br>
</li>
<li id='diff-29'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-29'>
<strong>
darshan-util/configure.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/configure.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/configure.in
</span><span style="color: #aaaaaa">@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.
</span> dnl You may need to use autoheader as well if changing any DEFINEs
dnl sanity checks, output header, location of scripts used here
-AC_INIT([darshan-util], [3.0.0-pre2])
<span style="color: #000000;background-color: #ddffdd">+AC_INIT([darshan-util], [3.0.0-pre3])
</span> AC_CONFIG_SRCDIR([darshan-logutils.h])
AC_CONFIG_AUX_DIR(../maint/config)
AC_CONFIG_HEADER(darshan-util-config.h)
</code></pre>
<br>
</li>
<li id='diff-30'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-30'>
<strong>
darshan-util/darshan-bgq-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-bgq-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-bgq-logutils.c
</span><span style="color: #aaaaaa">@@ -17,10 +17,10 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-bgq-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
-/* counter name strings for the POSIX module */
-#define X(a, b) #a,
<span style="color: #000000;background-color: #ddffdd">+/* counter name strings for the BGQ module */
+#define X(a) #a,
</span> char *bgq_counter_names[] = {
BGQ_COUNTERS
};
<span style="color: #aaaaaa">@@ -31,16 +31,22 @@ char *bgq_f_counter_names[] = {
</span> #undef X
static int darshan_log_get_bgq_rec(darshan_fd fd, void* bgq_buf);
-static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf, int ver);
</span> static void darshan_log_print_bgq_rec(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_bgq_description(void);
+static void darshan_log_print_bgq_rec_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2);
+static void darshan_log_agg_bgq_recs(void *rec, void *agg_rec, int init_flag);
</span>
struct darshan_mod_logutil_funcs bgq_logutils =
{
.log_get_record = &darshan_log_get_bgq_rec,
.log_put_record = &darshan_log_put_bgq_rec,
.log_print_record = &darshan_log_print_bgq_rec,
<span style="color: #000000;background-color: #ffdddd">- .log_agg_records = NULL /* TODO: how would aggregation work for the BG/Q module ? */
</span><span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_bgq_description,
+ .log_print_diff = &darshan_log_print_bgq_rec_diff,
+ .log_agg_records = &darshan_log_agg_bgq_recs
</span> };
static int darshan_log_get_bgq_rec(darshan_fd fd, void* bgq_buf)
<span style="color: #aaaaaa">@@ -73,13 +79,13 @@ static int darshan_log_get_bgq_rec(darshan_fd fd, void* bgq_buf)
</span> }
}
-static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf, int ver)
</span> {
struct darshan_bgq_record *rec = (struct darshan_bgq_record *)bgq_buf;
int ret;
ret = darshan_log_putmod(fd, DARSHAN_BGQ_MOD, rec,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_bgq_record));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_bgq_record), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -87,7 +93,7 @@ static int darshan_log_put_bgq_rec(darshan_fd fd, void* bgq_buf)
</span> }
static void darshan_log_print_bgq_rec(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_bgq_record *bgq_file_rec =
<span style="color: #aaaaaa">@@ -112,6 +118,102 @@ static void darshan_log_print_bgq_rec(void *file_rec, char *file_name,
</span> return;
}
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_print_bgq_description()
+{
+ printf("\n# description of BGQ counters:\n");
+ printf("# BGQ_CSJOBID: BGQ control system job ID.\n");
+ printf("# BGQ_NNODES: number of BGQ compute nodes for this job.\n");
+ printf("# BGQ_RANKSPERNODE: number of MPI ranks per compute node.\n");
+ printf("# BGQ_DDRPERNODE: size in MB of DDR3 per compute node.\n");
+ printf("# BGQ_INODES: number of BGQ I/O nodes for this job.\n");
+ printf("# BGQ_*NODES: dimension of A, B, C, D, & E dimensions of torus.\n");
+ printf("# BGQ_TORUSENABLED: which dimensions of the torus are enabled.\n");
+ printf("# BGQ_F_TIMESTAMP: timestamp when the BGQ data was collected.\n");
+
+ DARSHAN_PRINT_HEADER();
+
+ return;
+}
+
+static void darshan_log_print_bgq_rec_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+ struct darshan_bgq_record *file1 = (struct darshan_bgq_record *)file_rec1;
+ struct darshan_bgq_record *file2 = (struct darshan_bgq_record *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
+
+ for(i=0; i<BGQ_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file1->base_rec.rank, file1->base_rec.id, bgq_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file2->base_rec.rank, file2->base_rec.id, bgq_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file1->base_rec.rank, file1->base_rec.id, bgq_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file2->base_rec.rank, file2->base_rec.id, bgq_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<BGQ_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file1->base_rec.rank, file1->base_rec.id, bgq_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file2->base_rec.rank, file2->base_rec.id, bgq_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file1->base_rec.rank, file1->base_rec.id, bgq_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ file2->base_rec.rank, file2->base_rec.id, bgq_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
+
+ return;
+}
+
+
+static void darshan_log_agg_bgq_recs(void *rec, void *agg_rec, int init_flag)
+{
+ /* TODO: how would aggregation work for the BG/Q module ? */
+ return;
+}
+
</span> /*
* Local variables:
* c-indent-level: 4
</code></pre>
<br>
</li>
<li id='diff-31'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-31'>
<strong>
darshan-util/darshan-bgq-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-bgq-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-bgq-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_BGQ_LOG_UTILS_H
#define __DARSHAN_BGQ_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-bgq-log-format.h"
-
extern char *bgq_counter_names[];
extern char *bgq_f_counter_names[];
</code></pre>
<br>
</li>
<li id='diff-32'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-32'>
<strong>
darshan-util/darshan-convert.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-convert.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-convert.c
</span><span style="color: #aaaaaa">@@ -375,7 +375,7 @@ int main(int argc, char **argv)
</span>
if(!hash || hash == base_rec->id)
{
<span style="color: #000000;background-color: #ffdddd">- ret = mod_logutils[i]->log_put_record(outfile, mod_buf);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = mod_logutils[i]->log_put_record(outfile, mod_buf, infile->mod_ver[i]);
</span> if(ret < 0)
{
darshan_log_close(infile);
</code></pre>
<br>
</li>
<li id='diff-33'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-33'>
<strong>
darshan-util/darshan-diff.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-diff.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-diff.c
</span><span style="color: #aaaaaa">@@ -193,7 +193,6 @@ int main(int argc, char *argv[])
</span>
for(i = 0; i < DARSHAN_MAX_MODS; i++)
{
-#if 0
/* TODO: skip modules that don't have the same format version, for now */
if(rec_ref1->mod_recs[i] && rec_ref2 && rec_ref2->mod_recs[i] &&
(file1->mod_ver[i] != file2->mod_ver[i]))
<span style="color: #aaaaaa">@@ -203,7 +202,6 @@ int main(int argc, char *argv[])
</span> darshan_module_names[i], file1->mod_ver[i], file2->mod_ver[i]);
continue;
}
-#endif
while(1)
{
<span style="color: #aaaaaa">@@ -224,7 +222,6 @@ int main(int argc, char *argv[])
</span>
base_rec1 = (struct darshan_base_record *)mod_buf1;
base_rec2 = (struct darshan_base_record *)mod_buf2;
-
if(!base_rec1 && !base_rec2)
{
/* break out if there are no more records for this module */
<span style="color: #aaaaaa">@@ -307,8 +304,6 @@ int main(int argc, char *argv[])
</span> */
HASH_ITER(hlink, rec_hash2, rec_ref2, rec_tmp)
{
<span style="color: #000000;background-color: #ffdddd">- printf("\n");
</span>-
for(i = 0; i < DARSHAN_MAX_MODS; i++)
{
while(rec_ref2->mod_recs[i])
</code></pre>
<br>
</li>
<li id='diff-34'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-34'>
<strong>
darshan-util/darshan-hdf5-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-hdf5-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-hdf5-logutils.c
</span><span style="color: #aaaaaa">@@ -17,7 +17,7 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-hdf5-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
/* counter name strings for the HDF5 module */
#define X(a) #a,
<span style="color: #aaaaaa">@@ -31,9 +31,12 @@ char *hdf5_f_counter_names[] = {
</span> #undef X
static int darshan_log_get_hdf5_file(darshan_fd fd, void* hdf5_buf);
-static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf, int ver);
</span> static void darshan_log_print_hdf5_file(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_hdf5_description(void);
+static void darshan_log_print_hdf5_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2);
</span> static void darshan_log_agg_hdf5_files(void *rec, void *agg_rec, int init_flag);
struct darshan_mod_logutil_funcs hdf5_logutils =
<span style="color: #aaaaaa">@@ -41,6 +44,8 @@ struct darshan_mod_logutil_funcs hdf5_logutils =
</span> .log_get_record = &darshan_log_get_hdf5_file,
.log_put_record = &darshan_log_put_hdf5_file,
.log_print_record = &darshan_log_print_hdf5_file,
<span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_hdf5_description,
+ .log_print_diff = &darshan_log_print_hdf5_file_diff,
</span> .log_agg_records = &darshan_log_agg_hdf5_files
};
<span style="color: #aaaaaa">@@ -74,13 +79,13 @@ static int darshan_log_get_hdf5_file(darshan_fd fd, void* hdf5_buf)
</span> }
}
-static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf, int ver)
</span> {
struct darshan_hdf5_file *file = (struct darshan_hdf5_file *)hdf5_buf;
int ret;
ret = darshan_log_putmod(fd, DARSHAN_HDF5_MOD, file,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_hdf5_file));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_hdf5_file), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -88,7 +93,7 @@ static int darshan_log_put_hdf5_file(darshan_fd fd, void* hdf5_buf)
</span> }
static void darshan_log_print_hdf5_file(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_hdf5_file *hdf5_file_rec =
<span style="color: #aaaaaa">@@ -113,9 +118,92 @@ static void darshan_log_print_hdf5_file(void *file_rec, char *file_name,
</span> return;
}
-static void darshan_log_agg_hdf5_files(void *rec, void *agg_rec, int init_flag)
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_print_hdf5_description()
+{
+ printf("\n# description of HDF5 counters:\n");
+ printf("# HDF5_OPENS: HDF5 file open operation counts.\n");
+ printf("# HDF5_F_OPEN_TIMESTAMP: timestamp of first HDF5 file open.\n");
+ printf("# HDF5_F_CLOSE_TIMESTAMP: timestamp of last HDF5 file close.\n");
+
+ DARSHAN_PRINT_HEADER();
+
+ return;
+}
+
+static void darshan_log_print_hdf5_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
</span> {
<span style="color: #000000;background-color: #ddffdd">+ struct darshan_hdf5_file *file1 = (struct darshan_hdf5_file *)file_rec1;
+ struct darshan_hdf5_file *file2 = (struct darshan_hdf5_file *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
</span>
<span style="color: #000000;background-color: #ddffdd">+ for(i=0; i<HDF5_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file1->base_rec.rank, file1->base_rec.id, hdf5_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file2->base_rec.rank, file2->base_rec.id, hdf5_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file1->base_rec.rank, file1->base_rec.id, hdf5_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file2->base_rec.rank, file2->base_rec.id, hdf5_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<HDF5_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file1->base_rec.rank, file1->base_rec.id, hdf5_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file2->base_rec.rank, file2->base_rec.id, hdf5_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file1->base_rec.rank, file1->base_rec.id, hdf5_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ file2->base_rec.rank, file2->base_rec.id, hdf5_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
+
+ return;
+}
+
+static void darshan_log_agg_hdf5_files(void *rec, void *agg_rec, int init_flag)
+{
</span> return;
}
</code></pre>
<br>
</li>
<li id='diff-35'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-35'>
<strong>
darshan-util/darshan-hdf5-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-hdf5-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-hdf5-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_HDF5_LOG_UTILS_H
#define __DARSHAN_HDF5_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-hdf5-log-format.h"
-
extern char *hdf5_counter_names[];
extern char *hdf5_f_counter_names[];
</code></pre>
<br>
</li>
<li id='diff-36'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-36'>
<strong>
darshan-util/darshan-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.c
</span><span style="color: #aaaaaa">@@ -93,7 +93,7 @@ static int darshan_log_noz_read(darshan_fd fd, struct darshan_log_map map,
</span> void *buf, int len, int reset_strm_flag);
/* each module's implementation of the darshan logutil functions */
-#define X(a, b, c) c,
<span style="color: #000000;background-color: #ddffdd">+#define X(a, b, c, d) d,
</span> struct darshan_mod_logutil_funcs *mod_logutils[DARSHAN_MAX_MODS] =
{
DARSHAN_MODULE_IDS
<span style="color: #aaaaaa">@@ -139,6 +139,7 @@ darshan_fd darshan_log_open(const char *name)
</span> ret = darshan_log_getheader(tmp_fd);
if(ret < 0)
{
<span style="color: #000000;background-color: #ddffdd">+ fprintf(stderr, "Error: failed to read darshan log file header.\n");
</span> close(tmp_fd->state->fildes);
free(tmp_fd->state);
free(tmp_fd);
<span style="color: #aaaaaa">@@ -728,7 +729,7 @@ int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,
</span> * returns number of bytes written on success, -1 on failure
*/
int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,
<span style="color: #000000;background-color: #ffdddd">- void *mod_buf, int mod_buf_sz)
</span><span style="color: #000000;background-color: #ddffdd">+ void *mod_buf, int mod_buf_sz, int ver)
</span> {
struct darshan_fd_int_state *state = fd->state;
int ret;
<span style="color: #aaaaaa">@@ -753,6 +754,9 @@ int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,
</span> return(-1);
}
<span style="color: #000000;background-color: #ddffdd">+ /* set the version number for this module's data */
+ fd->mod_ver[mod_id] = ver;
+
</span> return(0);
}
<span style="color: #aaaaaa">@@ -839,6 +843,30 @@ static int darshan_log_getheader(darshan_fd fd)
</span> return(-1);
}
<span style="color: #000000;background-color: #ddffdd">+ /* read the version number so we know how to process this log */
+ ret = darshan_log_read(fd, &fd->version, 8);
+ if(ret < 8)
+ {
+ fprintf(stderr, "Error: invalid log file (failed to read version).\n");
+ return(-1);
+ }
+
+ /* other log file versions can be detected and handled here */
+ if(strcmp(fd->version, "3.00"))
+ {
+ fprintf(stderr, "Error: incompatible darshan file.\n");
+ fprintf(stderr, "Error: expected version %s\n", DARSHAN_LOG_VERSION);
+ return(-1);
+ }
+
+ /* seek back so we can read the entire header */
+ ret = darshan_log_seek(fd, 0);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to seek in darshan log file.\n");
+ return(-1);
+ }
+
</span> /* read uncompressed header from log file */
ret = darshan_log_read(fd, &header, sizeof(header));
if(ret != (int)sizeof(header))
<span style="color: #aaaaaa">@@ -847,9 +875,6 @@ static int darshan_log_getheader(darshan_fd fd)
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- /* save the version string */
- strncpy(fd->version, header.version_string, 8);
</span>-
if(header.magic_nr == DARSHAN_MAGIC_NR)
{
/* no byte swapping needed, this file is in host format already */
<span style="color: #aaaaaa">@@ -880,8 +905,10 @@ static int darshan_log_getheader(darshan_fd fd)
</span> }
}
<span style="color: #000000;background-color: #ddffdd">+ /* set some fd fields based on what's stored in the header */
</span> fd->comp_type = header.comp_type;
fd->partial_flag = header.partial_flag;
<span style="color: #000000;background-color: #ddffdd">+ memcpy(fd->mod_ver, header.mod_ver, DARSHAN_MAX_MODS * sizeof(uint32_t));
</span>
/* save the mapping of data within log file to this file descriptor */
memcpy(&fd->rec_map, &(header.rec_map), sizeof(struct darshan_log_map));
</code></pre>
<br>
</li>
<li id='diff-37'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-37'>
<strong>
darshan-util/darshan-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.h
</span><span style="color: #aaaaaa">@@ -35,6 +35,8 @@ struct darshan_fd_s
</span> struct darshan_log_map job_map;
struct darshan_log_map rec_map;
struct darshan_log_map mod_map[DARSHAN_MAX_MODS];
<span style="color: #000000;background-color: #ddffdd">+ /* module-specific log-format versions contained in log */
+ uint32_t mod_ver[DARSHAN_MAX_MODS];
</span>
/* KEEP OUT -- remaining state hidden in logutils source */
struct darshan_fd_int_state *state;
<span style="color: #aaaaaa">@@ -56,6 +58,9 @@ struct darshan_mod_logutil_funcs
</span> /* retrieve a single module record from the log file.
* return 1 on successful read of record, 0 on no more
* module data, -1 on error
<span style="color: #000000;background-color: #ddffdd">+ * - 'fd' is the file descriptor to get record from
+ * - 'buf' is the buffer to store the record in
+ * - 'rec_id' is the corresponding darshan record id
</span> */
int (*log_get_record)(
darshan_fd fd,
<span style="color: #aaaaaa">@@ -63,31 +68,44 @@ struct darshan_mod_logutil_funcs
</span> );
/* put a single module record into the log file.
* return 0 on success, -1 on error
<span style="color: #000000;background-color: #ddffdd">+ * - 'fd' is the file descriptor to put record into
+ * - 'buf' is the buffer containing the record data
+ * - 'rec_id' is the corresponding darshan record id
</span> */
int (*log_put_record)(
darshan_fd fd,
<span style="color: #000000;background-color: #ffdddd">- void *buf
</span><span style="color: #000000;background-color: #ddffdd">+ void *buf,
+ int ver
</span> );
<span style="color: #000000;background-color: #ffdddd">- /* print the counters for a given log file record */
</span><span style="color: #000000;background-color: #ddffdd">+ /* print the counters for a given log record
+ * - 'file_rec' is the record's data buffer
+ * - 'file_name' is the file path string for the record
+ * - 'mnt-pt' is the file path mount point string
+ * - 'fs_type' is the file system type string
+ * - 'ver' is the version of the record
+ */
</span> void (*log_print_record)(
void *file_rec,
char *file_name,
char *mnt_pt,
<span style="color: #000000;background-color: #ffdddd">- char *fs_type
- );
- /* combine two records into a single aggregate record */
- void (*log_agg_records)(
- void *rec,
- void *agg_rec,
- int init_flag
</span><span style="color: #000000;background-color: #ddffdd">+ char *fs_type,
+ int ver
</span> );
<span style="color: #000000;background-color: #ffdddd">- /* print a text diff of 2 module I/O records */
</span><span style="color: #000000;background-color: #ddffdd">+ /* print module-specific description of I/O characterization data */
+ void (*log_print_description)(void);
+ /* print a text diff of 2 module I/O records */
</span> void (*log_print_diff)(
void *rec1,
char *name1,
void *rec2,
char *name2
);
<span style="color: #000000;background-color: #ddffdd">+ /* combine two records into a single aggregate record */
+ void (*log_agg_records)(
+ void *rec,
+ void *agg_rec,
+ int init_flag
+ );
</span> };
extern struct darshan_mod_logutil_funcs *mod_logutils[];
<span style="color: #aaaaaa">@@ -114,7 +132,7 @@ int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash);
</span> int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,
void *mod_buf, int mod_buf_sz);
int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,
<span style="color: #000000;background-color: #ffdddd">- void *mod_buf, int mod_buf_sz);
</span><span style="color: #000000;background-color: #ddffdd">+ void *mod_buf, int mod_buf_sz, int ver);
</span> void darshan_log_close(darshan_fd file);
/* convenience macros for printing Darshan counters */
</code></pre>
<br>
</li>
<li id='diff-38'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-38'>
<strong>
darshan-util/darshan-mpiio-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-mpiio-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-mpiio-logutils.c
</span><span style="color: #aaaaaa">@@ -17,7 +17,7 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-mpiio-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
/* counter name strings for the MPI-IO module */
#define X(a) #a,
<span style="color: #aaaaaa">@@ -31,9 +31,12 @@ char *mpiio_f_counter_names[] = {
</span> #undef X
static int darshan_log_get_mpiio_file(darshan_fd fd, void* mpiio_buf);
-static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf, int ver);
</span> static void darshan_log_print_mpiio_file(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_mpiio_description(void);
+static void darshan_log_print_mpiio_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2);
</span> static void darshan_log_agg_mpiio_files(void *rec, void *agg_rec, int init_flag);
struct darshan_mod_logutil_funcs mpiio_logutils =
<span style="color: #aaaaaa">@@ -41,6 +44,8 @@ struct darshan_mod_logutil_funcs mpiio_logutils =
</span> .log_get_record = &darshan_log_get_mpiio_file,
.log_put_record = &darshan_log_put_mpiio_file,
.log_print_record = &darshan_log_print_mpiio_file,
<span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_mpiio_description,
+ .log_print_diff = &darshan_log_print_mpiio_file_diff,
</span> .log_agg_records = &darshan_log_agg_mpiio_files
};
<span style="color: #aaaaaa">@@ -74,13 +79,13 @@ static int darshan_log_get_mpiio_file(darshan_fd fd, void* mpiio_buf)
</span> }
}
-static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf, int ver)
</span> {
struct darshan_mpiio_file *file = (struct darshan_mpiio_file *)mpiio_buf;
int ret;
ret = darshan_log_putmod(fd, DARSHAN_MPIIO_MOD, file,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_mpiio_file));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_mpiio_file), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -88,7 +93,7 @@ static int darshan_log_put_mpiio_file(darshan_fd fd, void* mpiio_buf)
</span> }
static void darshan_log_print_mpiio_file(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_mpiio_file *mpiio_file_rec =
<span style="color: #aaaaaa">@@ -113,12 +118,117 @@ static void darshan_log_print_mpiio_file(void *file_rec, char *file_name,
</span> return;
}
-static void darshan_log_agg_mpiio_files(void *rec, void *agg_rec, int init_flag)
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_print_mpiio_description()
</span> {
<span style="color: #000000;background-color: #ddffdd">+ printf("\n# description of MPIIO counters:\n");
+ printf("# MPIIO_INDEP_*: MPI independent operation counts.\n");
+ printf("# MPIIO_COLL_*: MPI collective operation counts.\n");
+ printf("# MPIIO_SPLIT_*: MPI split collective operation counts.\n");
+ printf("# MPIIO_NB_*: MPI non blocking operation counts.\n");
+ printf("# READS,WRITES,and OPENS are types of operations.\n");
+ printf("# MPIIO_SYNCS: MPI file sync operation counts.\n");
+ printf("# MPIIO_HINTS: number of times MPI hints were used.\n");
+ printf("# MPIIO_VIEWS: number of times MPI file views were used.\n");
+ printf("# MPIIO_MODE: MPI-IO access mode that file was opened with.\n");
+ printf("# MPIIO_BYTES_*: total bytes read and written at MPI-IO layer.\n");
+ printf("# MPIIO_RW_SWITCHES: number of times access alternated between read and write.\n");
+ printf("# MPIIO_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");
+ printf("# MPIIO_SIZE_*_AGG_*: histogram of MPI datatype total sizes for read and write operations.\n");
+ printf("# MPIIO_ACCESS*_ACCESS: the four most common total access sizes.\n");
+ printf("# MPIIO_ACCESS*_COUNT: count of the four most common total access sizes.\n");
+ printf("# MPIIO_*_RANK: rank of the processes that were the fastest and slowest at I/O (for shared files).\n");
+ printf("# MPIIO_*_RANK_BYTES: total bytes transferred at MPI-IO layer by the fastest and slowest ranks (for shared files).\n");
+ printf("# MPIIO_F_OPEN_TIMESTAMP: timestamp of first open.\n");
+ printf("# MPIIO_F_*_START_TIMESTAMP: timestamp of first MPI-IO read/write.\n");
+ printf("# MPIIO_F_*_END_TIMESTAMP: timestamp of last MPI-IO read/write.\n");
+ printf("# MPIIO_F_CLOSE_TIMESTAMP: timestamp of last close.\n");
+ printf("# MPIIO_F_READ/WRITE/META_TIME: cumulative time spent in MPI-IO read, write, or metadata operations.\n");
+ printf("# MPIIO_F_MAX_*_TIME: duration of the slowest MPI-IO read and write operations.\n");
+ printf("# MPIIO_F_*_RANK_TIME: fastest and slowest I/O time for a single rank (for shared files).\n");
+ printf("# MPIIO_F_VARIANCE_RANK_*: variance of total I/O time and bytes moved for all ranks (for shared files).\n");
+
+ DARSHAN_PRINT_HEADER();
+
+ return;
+}
+
+static void darshan_log_print_mpiio_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+ struct darshan_mpiio_file *file1 = (struct darshan_mpiio_file *)file_rec1;
+ struct darshan_mpiio_file *file2 = (struct darshan_mpiio_file *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
+
+ for(i=0; i<MPIIO_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file1->base_rec.rank, file1->base_rec.id, mpiio_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file2->base_rec.rank, file2->base_rec.id, mpiio_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file1->base_rec.rank, file1->base_rec.id, mpiio_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file2->base_rec.rank, file2->base_rec.id, mpiio_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<MPIIO_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file1->base_rec.rank, file1->base_rec.id, mpiio_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file2->base_rec.rank, file2->base_rec.id, mpiio_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file1->base_rec.rank, file1->base_rec.id, mpiio_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ file2->base_rec.rank, file2->base_rec.id, mpiio_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
</span>
return;
}
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_agg_mpiio_files(void *rec, void *agg_rec, int init_flag)
+{
+ return;
+}
+
</span> /*
* Local variables:
* c-indent-level: 4
</code></pre>
<br>
</li>
<li id='diff-39'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-39'>
<strong>
darshan-util/darshan-mpiio-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-mpiio-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-mpiio-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_MPIIO_LOG_UTILS_H
#define __DARSHAN_MPIIO_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-mpiio-log-format.h"
-
extern char *mpiio_counter_names[];
extern char *mpiio_f_counter_names[];
</code></pre>
<br>
</li>
<li id='diff-40'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-40'>
<strong>
darshan-util/darshan-null-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-null-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-null-logutils.c
</span><span style="color: #aaaaaa">@@ -17,7 +17,7 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-null-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
/* integer counter name strings for the NULL module */
#define X(a) #a,
<span style="color: #aaaaaa">@@ -33,9 +33,12 @@ char *null_f_counter_names[] = {
</span>
/* prototypes for each of the NULL module's logutil functions */
static int darshan_log_get_null_record(darshan_fd fd, void* null_buf);
-static int darshan_log_put_null_record(darshan_fd fd, void* null_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_null_record(darshan_fd fd, void* null_buf, int ver);
</span> static void darshan_log_print_null_record(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_null_description(void);
+static void darshan_log_print_null_record_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2);
</span> static void darshan_log_agg_null_records(void *rec, void *agg_rec, int init_flag);
/* structure storing each function needed for implementing the darshan
<span style="color: #aaaaaa">@@ -47,12 +50,15 @@ struct darshan_mod_logutil_funcs null_logutils =
</span> .log_get_record = &darshan_log_get_null_record,
.log_put_record = &darshan_log_put_null_record,
.log_print_record = &darshan_log_print_null_record,
<span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_null_description,
+ .log_print_diff = &darshan_log_print_null_record_diff,
</span> .log_agg_records = &darshan_log_agg_null_records
};
/* retrieve a NULL record from log file descriptor 'fd', storing the
* buffer in 'null_buf' and the corresponding Darshan record id in
<span style="color: #000000;background-color: #ffdddd">- * 'rec_id'. Return 1 on successful record read, .
</span><span style="color: #000000;background-color: #ddffdd">+ * 'rec_id'. Return 1 on successful record read, 0 on no more data,
+ * and -1 on error.
</span> */
static int darshan_log_get_null_record(darshan_fd fd, void* null_buf)
{
<span style="color: #aaaaaa">@@ -89,14 +95,14 @@ static int darshan_log_get_null_record(darshan_fd fd, void* null_buf)
</span> /* write the NULL record stored in 'null_buf' to log file descriptor 'fd'.
* Return 0 on success, -1 on failure
*/
-static int darshan_log_put_null_record(darshan_fd fd, void* null_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_null_record(darshan_fd fd, void* null_buf, int ver)
</span> {
struct darshan_null_record *rec = (struct darshan_null_record *)null_buf;
int ret;
/* append NULL record to darshan log file */
ret = darshan_log_putmod(fd, DARSHAN_NULL_MOD, rec,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_null_record));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_null_record), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -105,7 +111,7 @@ static int darshan_log_put_null_record(darshan_fd fd, void* null_buf)
</span>
/* print all I/O data record statistics for the given NULL record */
static void darshan_log_print_null_record(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_null_record *null_rec =
<span style="color: #aaaaaa">@@ -133,12 +139,95 @@ static void darshan_log_print_null_record(void *file_rec, char *file_name,
</span> return;
}
-static void darshan_log_agg_null_records(void *rec, void *agg_rec, int init_flag)
<span style="color: #000000;background-color: #ddffdd">+/* print out a description of the NULL module record fields */
+static void darshan_log_print_null_description()
</span> {
<span style="color: #000000;background-color: #ddffdd">+ printf("\n# description of NULL counters:\n");
+ printf("# NULL_BARS: number of 'bar' function calls.\n");
+ printf("# NULL_BAR_DAT: value set by last call to function 'bar'.\n");
+ printf("# NULL_F_BAR_TIMESTAMP: timestamp of the first call to function 'bar'.\n");
+ printf("# NULL_F_BAR_DURATION: duration of the last call to function 'bar'.\n");
+
+ return;
+}
+
+static void darshan_log_print_null_record_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+ struct darshan_null_record *file1 = (struct darshan_null_record *)file_rec1;
+ struct darshan_null_record *file2 = (struct darshan_null_record *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
+
+ for(i=0; i<NULL_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file1->base_rec.rank, file1->base_rec.id, null_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file2->base_rec.rank, file2->base_rec.id, null_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file1->base_rec.rank, file1->base_rec.id, null_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file2->base_rec.rank, file2->base_rec.id, null_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<NULL_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file1->base_rec.rank, file1->base_rec.id, null_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file2->base_rec.rank, file2->base_rec.id, null_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file1->base_rec.rank, file1->base_rec.id, null_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_NULL_MOD],
+ file2->base_rec.rank, file2->base_rec.id, null_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
</span>
return;
}
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_agg_null_records(void *rec, void *agg_rec, int init_flag)
+{
+ return;
+}
+
</span> /*
* Local variables:
* c-indent-level: 4
</code></pre>
<br>
</li>
<li id='diff-41'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-41'>
<strong>
darshan-util/darshan-null-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-null-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-null-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_NULL_LOG_UTILS_H
#define __DARSHAN_NULL_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-null-log-format.h"
-
/* declare NULL module counter name strings and logutil definition as
* extern variables so they can be used in other utilities
*/
</code></pre>
<br>
</li>
<li id='diff-42'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-42'>
<strong>
darshan-util/darshan-parser.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-parser.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-parser.c
</span><span style="color: #aaaaaa">@@ -313,17 +313,17 @@ int main(int argc, char **argv)
</span> }
/* print breakdown of each log file region's contribution to file size */
<span style="color: #000000;background-color: #ffdddd">- printf("\n# log file region sizes\n");
</span><span style="color: #000000;background-color: #ddffdd">+ printf("\n# log file regions\n");
</span> printf("# -------------------------------------------------------\n");
<span style="color: #000000;background-color: #ffdddd">- printf("# header: %zu bytes\n", sizeof(struct darshan_header));
- printf("# job data: %zu bytes\n", fd->job_map.len);
- printf("# record table: %zu bytes\n", fd->rec_map.len);
</span><span style="color: #000000;background-color: #ddffdd">+ printf("# header: %zu bytes (uncompressed)\n", sizeof(struct darshan_header));
+ printf("# job data: %zu bytes (compressed)\n", fd->job_map.len);
+ printf("# record table: %zu bytes (compressed)\n", fd->rec_map.len);
</span> for(i=0; i<DARSHAN_MAX_MODS; i++)
{
if(fd->mod_map[i].len)
{
<span style="color: #000000;background-color: #ffdddd">- printf("# %s module: %zu bytes\n", darshan_module_names[i],
- fd->mod_map[i].len);
</span><span style="color: #000000;background-color: #ddffdd">+ printf("# %s module: %zu bytes (compressed), ver=%d\n",
+ darshan_module_names[i], fd->mod_map[i].len, fd->mod_ver[i]);
</span> }
}
<span style="color: #aaaaaa">@@ -335,6 +335,21 @@ int main(int argc, char **argv)
</span> printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);
}
<span style="color: #000000;background-color: #ddffdd">+ if(mask & OPTION_BASE)
+ {
+ printf("\n# description of columns:\n");
+ printf("# <module>: module responsible for this I/O record.\n");
+ printf("# <rank>: MPI rank. -1 indicates that the file is shared\n");
+ printf("# across all processes and statistics are aggregated.\n");
+ printf("# <record id>: hash of the record's file path\n");
+ printf("# <counter name> and <counter value>: statistical counters.\n");
+ printf("# A value of -1 indicates that Darshan could not monitor\n");
+ printf("# that counter, and its value should be ignored.\n");
+ printf("# <file name>: full file path for the record.\n");
+ printf("# <mount pt>: mount point that the file resides on.\n");
+ printf("# <fs type>: type of file system that the file resides on.\n");
+ }
+
</span> /* warn user if this log file is incomplete */
pdata.rank_cumul_io_time = malloc(sizeof(double)*job.nprocs);
pdata.rank_cumul_md_time = malloc(sizeof(double)*job.nprocs);
<span style="color: #aaaaaa">@@ -388,8 +403,9 @@ int main(int argc, char **argv)
</span>
if(mask & OPTION_BASE)
{
<span style="color: #000000;background-color: #ffdddd">- /* TODO: does each module print header of what each counter means??? */
- DARSHAN_PRINT_HEADER();
</span><span style="color: #000000;background-color: #ddffdd">+ /* print a header describing the module's I/O characterization data */
+ if(mod_logutils[i]->log_print_description)
+ mod_logutils[i]->log_print_description();
</span> }
ret = mod_logutils[i]->log_get_record(fd, mod_buf);
<span style="color: #aaaaaa">@@ -432,7 +448,7 @@ int main(int argc, char **argv)
</span> {
/* print the corresponding module data for this record */
mod_logutils[i]->log_print_record(mod_buf, ref->name,
<span style="color: #000000;background-color: #ffdddd">- mnt_pt, fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ mnt_pt, fs_type, fd->mod_ver[i]);
</span> }
/* we calculate more detailed stats for POSIX and MPI-IO modules,
<span style="color: #aaaaaa">@@ -705,8 +721,7 @@ void posix_accum_file(struct darshan_posix_file *pfile,
</span> case POSIX_MODE:
case POSIX_MEM_ALIGNMENT:
case POSIX_FILE_ALIGNMENT:
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(tmp))
- tmp->counters[i] = pfile->counters[i];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp->counters[i] = pfile->counters[i];
</span> break;
case POSIX_MAX_BYTE_READ:
case POSIX_MAX_BYTE_WRITTEN:
</code></pre>
<br>
</li>
<li id='diff-43'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-43'>
<strong>
darshan-util/darshan-pnetcdf-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-pnetcdf-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-pnetcdf-logutils.c
</span><span style="color: #aaaaaa">@@ -17,7 +17,7 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-pnetcdf-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
/* counter name strings for the PNETCDF module */
#define X(a) #a,
<span style="color: #aaaaaa">@@ -31,9 +31,12 @@ char *pnetcdf_f_counter_names[] = {
</span> #undef X
static int darshan_log_get_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf);
-static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf, int ver);
</span> static void darshan_log_print_pnetcdf_file(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span><span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_pnetcdf_description(void);
+static void darshan_log_print_pnetcdf_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2);
</span> static void darshan_log_agg_pnetcdf_files(void *rec, void *agg_rec, int init_flag);
struct darshan_mod_logutil_funcs pnetcdf_logutils =
<span style="color: #aaaaaa">@@ -41,6 +44,8 @@ struct darshan_mod_logutil_funcs pnetcdf_logutils =
</span> .log_get_record = &darshan_log_get_pnetcdf_file,
.log_put_record = &darshan_log_put_pnetcdf_file,
.log_print_record = &darshan_log_print_pnetcdf_file,
<span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_pnetcdf_description,
+ .log_print_diff = &darshan_log_print_pnetcdf_file_diff,
</span> .log_agg_records = &darshan_log_agg_pnetcdf_files
};
<span style="color: #aaaaaa">@@ -74,13 +79,13 @@ static int darshan_log_get_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf)
</span> }
}
-static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf, int ver)
</span> {
struct darshan_pnetcdf_file *file = (struct darshan_pnetcdf_file *)pnetcdf_buf;
int ret;
ret = darshan_log_putmod(fd, DARSHAN_PNETCDF_MOD, file,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_pnetcdf_file));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_pnetcdf_file), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -88,7 +93,7 @@ static int darshan_log_put_pnetcdf_file(darshan_fd fd, void* pnetcdf_buf)
</span> }
static void darshan_log_print_pnetcdf_file(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_pnetcdf_file *pnetcdf_file_rec =
<span style="color: #aaaaaa">@@ -113,9 +118,93 @@ static void darshan_log_print_pnetcdf_file(void *file_rec, char *file_name,
</span> return;
}
-static void darshan_log_agg_pnetcdf_files(void *rec, void *agg_rec, int init_flag)
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_print_pnetcdf_description()
+{
+ printf("\n# description of PNETCDF counters:\n");
+ printf("# PNETCDF_INDEP_OPENS: PNETCDF independent file open operation counts.\n");
+ printf("# PNETCDF_COLL_OPENS: PNETCDF collective file open operation counts.\n");
+ printf("# PNETCDF_F_OPEN_TIMESTAMP: timestamp of first PNETCDF file open.\n");
+ printf("# PNETCDF_F_CLOSE_TIMESTAMP: timestamp of last PNETCDF file close.\n");
+
+ DARSHAN_PRINT_HEADER();
+
+ return;
+}
+
+static void darshan_log_print_pnetcdf_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
</span> {
<span style="color: #000000;background-color: #ddffdd">+ struct darshan_pnetcdf_file *file1 = (struct darshan_pnetcdf_file *)file_rec1;
+ struct darshan_pnetcdf_file *file2 = (struct darshan_pnetcdf_file *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
</span>
<span style="color: #000000;background-color: #ddffdd">+ for(i=0; i<PNETCDF_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file1->base_rec.rank, file1->base_rec.id, pnetcdf_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file2->base_rec.rank, file2->base_rec.id, pnetcdf_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file1->base_rec.rank, file1->base_rec.id, pnetcdf_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file2->base_rec.rank, file2->base_rec.id, pnetcdf_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<PNETCDF_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file1->base_rec.rank, file1->base_rec.id, pnetcdf_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file2->base_rec.rank, file2->base_rec.id, pnetcdf_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file1->base_rec.rank, file1->base_rec.id, pnetcdf_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ file2->base_rec.rank, file2->base_rec.id, pnetcdf_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
+
+ return;
+}
+
+static void darshan_log_agg_pnetcdf_files(void *rec, void *agg_rec, int init_flag)
+{
</span> return;
}
</code></pre>
<br>
</li>
<li id='diff-44'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-44'>
<strong>
darshan-util/darshan-pnetcdf-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-pnetcdf-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-pnetcdf-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_PNETCDF_LOG_UTILS_H
#define __DARSHAN_PNETCDF_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-pnetcdf-log-format.h"
-
extern char *pnetcdf_counter_names[];
extern char *pnetcdf_f_counter_names[];
</code></pre>
<br>
</li>
<li id='diff-45'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-45'>
<strong>
darshan-util/darshan-posix-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-posix-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-posix-logutils.c
</span><span style="color: #aaaaaa">@@ -17,7 +17,7 @@
</span> #include <fcntl.h>
#include <errno.h>
-#include "darshan-posix-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#include "darshan-logutils.h"
</span>
/* counter name strings for the POSIX module */
#define X(a) #a,
<span style="color: #aaaaaa">@@ -31,20 +31,22 @@ char *posix_f_counter_names[] = {
</span> #undef X
static int darshan_log_get_posix_file(darshan_fd fd, void* posix_buf);
-static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf);
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf, int ver);
</span> static void darshan_log_print_posix_file(void *file_rec,
<span style="color: #000000;background-color: #ffdddd">- char *file_name, char *mnt_pt, char *fs_type);
</span>-static void darshan_log_agg_posix_files(void *rec, void *agg_rec, int init_flag);
<span style="color: #000000;background-color: #ddffdd">+ char *file_name, char *mnt_pt, char *fs_type, int ver);
+static void darshan_log_print_posix_description(void);
</span> static void darshan_log_print_posix_file_diff(void *file_rec1, char *file_name1,
void *file_rec2, char *file_name2);
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_agg_posix_files(void *rec, void *agg_rec, int init_flag);
</span>
struct darshan_mod_logutil_funcs posix_logutils =
{
.log_get_record = &darshan_log_get_posix_file,
.log_put_record = &darshan_log_put_posix_file,
.log_print_record = &darshan_log_print_posix_file,
<span style="color: #000000;background-color: #ddffdd">+ .log_print_description = &darshan_log_print_posix_description,
+ .log_print_diff = &darshan_log_print_posix_file_diff,
</span> .log_agg_records = &darshan_log_agg_posix_files,
<span style="color: #000000;background-color: #ffdddd">- .log_print_diff = &darshan_log_print_posix_file_diff
</span> };
static int darshan_log_get_posix_file(darshan_fd fd, void* posix_buf)
<span style="color: #aaaaaa">@@ -77,13 +79,13 @@ static int darshan_log_get_posix_file(darshan_fd fd, void* posix_buf)
</span> }
}
-static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf)
<span style="color: #000000;background-color: #ddffdd">+static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf, int ver)
</span> {
struct darshan_posix_file *file = (struct darshan_posix_file *)posix_buf;
int ret;
ret = darshan_log_putmod(fd, DARSHAN_POSIX_MOD, file,
<span style="color: #000000;background-color: #ffdddd">- sizeof(struct darshan_posix_file));
</span><span style="color: #000000;background-color: #ddffdd">+ sizeof(struct darshan_posix_file), ver);
</span> if(ret < 0)
return(-1);
<span style="color: #aaaaaa">@@ -91,7 +93,7 @@ static int darshan_log_put_posix_file(darshan_fd fd, void* posix_buf)
</span> }
static void darshan_log_print_posix_file(void *file_rec, char *file_name,
<span style="color: #000000;background-color: #ffdddd">- char *mnt_pt, char *fs_type)
</span><span style="color: #000000;background-color: #ddffdd">+ char *mnt_pt, char *fs_type, int ver)
</span> {
int i;
struct darshan_posix_file *posix_file_rec =
<span style="color: #aaaaaa">@@ -116,6 +118,113 @@ static void darshan_log_print_posix_file(void *file_rec, char *file_name,
</span> return;
}
<span style="color: #000000;background-color: #ddffdd">+static void darshan_log_print_posix_description()
+{
+ printf("\n# description of POSIX counters:\n");
+ printf("# POSIX_*: posix operation counts.\n");
+ printf("# READS,WRITES,OPENS,SEEKS,STATS, and MMAPS are types of operations.\n");
+ printf("# POSIX_MODE: mode that file was opened in.\n");
+ printf("# POSIX_BYTES_*: total bytes read and written.\n");
+ printf("# POSIX_MAX_BYTE_*: highest offset byte read and written.\n");
+ printf("# POSIX_CONSEC_*: number of exactly adjacent reads and writes.\n");
+ printf("# POSIX_SEQ_*: number of reads and writes from increasing offsets.\n");
+ printf("# POSIX_RW_SWITCHES: number of times access alternated between read and write.\n");
+ printf("# POSIX_*_ALIGNMENT: memory and file alignment.\n");
+ printf("# POSIX_*_NOT_ALIGNED: number of reads and writes that were not aligned.\n");
+ printf("# POSIX_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");
+ printf("# POSIX_SIZE_*_*: histogram of read and write access sizes.\n");
+ printf("# POSIX_STRIDE*_STRIDE: the four most common strides detected.\n");
+ printf("# POSIX_STRIDE*_COUNT: count of the four most common strides.\n");
+ printf("# POSIX_ACCESS*_ACCESS: the four most common access sizes.\n");
+ printf("# POSIX_ACCESS*_COUNT: count of the four most common access sizes.\n");
+ printf("# POSIX_*_RANK: rank of the processes that were the fastest and slowest at I/O (for shared files).\n");
+ printf("# POSIX_*_RANK_BYTES: bytes transferred by the fastest and slowest ranks (for shared files).\n");
+ printf("# POSIX_F_OPEN_TIMESTAMP: timestamp of first open.\n");
+ printf("# POSIX_F_*_START_TIMESTAMP: timestamp of first read/write.\n");
+ printf("# POSIX_F_*_END_TIMESTAMP: timestamp of last read/write.\n");
+ printf("# POSIX_F_CLOSE_TIMESTAMP: timestamp of last close.\n");
+ printf("# POSIX_F_READ/WRITE/META_TIME: cumulative time spent in read, write, or metadata operations.\n");
+ printf("# POSIX_F_MAX_*_TIME: duration of the slowest read and write operations.\n");
+ printf("# POSIX_F_*_RANK_TIME: fastest and slowest I/O time for a single rank (for shared files).\n");
+ printf("# POSIX_F_VARIANCE_RANK_*: variance of total I/O time and bytes moved for all ranks (for shared files).\n");
+
+ DARSHAN_PRINT_HEADER();
+
+ return;
+}
+
+static void darshan_log_print_posix_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+ struct darshan_posix_file *file1 = (struct darshan_posix_file *)file_rec1;
+ struct darshan_posix_file *file2 = (struct darshan_posix_file *)file_rec2;
+ int i;
+
+ /* NOTE: we assume that both input records are the same module format version */
+
+ for(i=0; i<POSIX_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file1->base_rec.rank, file1->base_rec.id, posix_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file2->base_rec.rank, file2->base_rec.id, posix_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ else if(file1->counters[i] != file2->counters[i])
+ {
+ printf("- ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file1->base_rec.rank, file1->base_rec.id, posix_counter_names[i],
+ file1->counters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file2->base_rec.rank, file2->base_rec.id, posix_counter_names[i],
+ file2->counters[i], file_name2, "", "");
+ }
+ }
+
+ for(i=0; i<POSIX_F_NUM_INDICES; i++)
+ {
+ if(!file2)
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file1->base_rec.rank, file1->base_rec.id, posix_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+
+ }
+ else if(!file1)
+ {
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file2->base_rec.rank, file2->base_rec.id, posix_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ else if(file1->fcounters[i] != file2->fcounters[i])
+ {
+ printf("- ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file1->base_rec.rank, file1->base_rec.id, posix_f_counter_names[i],
+ file1->fcounters[i], file_name1, "", "");
+ printf("+ ");
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ file2->base_rec.rank, file2->base_rec.id, posix_f_counter_names[i],
+ file2->fcounters[i], file_name2, "", "");
+ }
+ }
+
+ return;
+}
+
</span> static void darshan_log_agg_posix_files(void *rec, void *agg_rec, int init_flag)
{
struct darshan_posix_file *psx_rec = (struct darshan_posix_file *)rec;
<span style="color: #aaaaaa">@@ -300,78 +409,6 @@ static void darshan_log_agg_posix_files(void *rec, void *agg_rec, int init_flag)
</span> return;
}
-static void darshan_log_print_posix_file_diff(void *file_rec1, char *file_name1,
<span style="color: #000000;background-color: #ffdddd">- void *file_rec2, char *file_name2)
</span>-{
<span style="color: #000000;background-color: #ffdddd">- struct darshan_posix_file *file1 = (struct darshan_posix_file *)file_rec1;
- struct darshan_posix_file *file2 = (struct darshan_posix_file *)file_rec2;
- int i;
</span>-
<span style="color: #000000;background-color: #ffdddd">- /* NOTE: we assume that both input records are the same module format version */
</span>-
<span style="color: #000000;background-color: #ffdddd">- for(i=0; i<POSIX_NUM_INDICES; i++)
- {
- if(!file2)
- {
- printf("- ");
- DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file1->base_rec.rank, file1->base_rec.id, posix_counter_names[i],
- file1->counters[i], file_name1, "", "");
</span>-
<span style="color: #000000;background-color: #ffdddd">- }
- else if(!file1)
- {
- printf("+ ");
- DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file2->base_rec.rank, file2->base_rec.id, posix_counter_names[i],
- file2->counters[i], file_name2, "", "");
- }
- else if(file1->counters[i] != file2->counters[i])
- {
- printf("- ");
- DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file1->base_rec.rank, file1->base_rec.id, posix_counter_names[i],
- file1->counters[i], file_name1, "", "");
- printf("+ ");
- DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file2->base_rec.rank, file2->base_rec.id, posix_counter_names[i],
- file2->counters[i], file_name2, "", "");
- }
- }
</span>-
<span style="color: #000000;background-color: #ffdddd">- for(i=0; i<POSIX_F_NUM_INDICES; i++)
- {
- if(!file2)
- {
- printf("- ");
- DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file1->base_rec.rank, file1->base_rec.id, posix_f_counter_names[i],
- file1->fcounters[i], file_name1, "", "");
</span>-
<span style="color: #000000;background-color: #ffdddd">- }
- else if(!file1)
- {
- printf("+ ");
- DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file2->base_rec.rank, file2->base_rec.id, posix_f_counter_names[i],
- file2->fcounters[i], file_name2, "", "");
- }
- else if(file1->fcounters[i] != file2->fcounters[i])
- {
- printf("- ");
- DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file1->base_rec.rank, file1->base_rec.id, posix_f_counter_names[i],
- file1->fcounters[i], file_name1, "", "");
- printf("+ ");
- DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
- file2->base_rec.rank, file2->base_rec.id, posix_f_counter_names[i],
- file2->fcounters[i], file_name2, "", "");
- }
- }
</span>-
<span style="color: #000000;background-color: #ffdddd">- return;
</span>-}
-
/*
* Local variables:
* c-indent-level: 4
</code></pre>
<br>
</li>
<li id='diff-46'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-46'>
<strong>
darshan-util/darshan-posix-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-posix-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-posix-logutils.h
</span><span style="color: #aaaaaa">@@ -7,9 +7,6 @@
</span> #ifndef __DARSHAN_POSIX_LOG_UTILS_H
#define __DARSHAN_POSIX_LOG_UTILS_H
-#include "darshan-logutils.h"
-#include "darshan-posix-log-format.h"
-
extern char *posix_counter_names[];
extern char *posix_f_counter_names[];
</code></pre>
<br>
</li>
<li id='diff-47'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-47'>
<strong>
darshan-util/darshan-stitch-logs.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-stitch-logs.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-stitch-logs.c
</span><span style="color: #aaaaaa">@@ -404,7 +404,7 @@ int main(int argc, char *argv[])
</span> /* iterate over active darshan modules and gather module data to write
* to the stitched together output log
*/
<span style="color: #000000;background-color: #ffdddd">- for(i = 0; i < DARSHAN_MPIIO_MOD; i++)
</span><span style="color: #000000;background-color: #ddffdd">+ for(i = 0; i < DARSHAN_MAX_MODS; i++)
</span> {
if(!mod_logutils[i]) continue;
<span style="color: #aaaaaa">@@ -424,21 +424,6 @@ int main(int argc, char *argv[])
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- /* write out the shared records first */
- HASH_ITER(hlink, shared_rec_hash, sref, stmp)
- {
- ret = mod_logutils[i]->log_put_record(stitch_fd, sref->agg_rec);
- if(ret < 0)
- {
- fprintf(stderr,
- "Error: unable to write %s module record to output darshan log.\n",
- darshan_module_names[i]);
- globfree(&globbuf);
- darshan_log_close(stitch_fd);
- unlink(stitch_logname);
- return(-1);
- }
- }
</span> }
for(j = 0; j < globbuf.gl_pathc; j++)
<span style="color: #aaaaaa">@@ -456,6 +441,25 @@ int main(int argc, char *argv[])
</span> return(-1);
}
<span style="color: #000000;background-color: #ddffdd">+ if(j == 0 && shared_rec_hash)
+ {
+ /* write out the shared records first */
+ HASH_ITER(hlink, shared_rec_hash, sref, stmp)
+ {
+ ret = mod_logutils[i]->log_put_record(stitch_fd, sref->agg_rec, in_fd->mod_ver[i]);
+ if(ret < 0)
+ {
+ fprintf(stderr,
+ "Error: unable to write %s module record to output darshan log.\n",
+ darshan_module_names[i]);
+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
+ return(-1);
+ }
+ }
+ }
+
</span> /* loop over module records and write them to output file */
while((ret = mod_logutils[i]->log_get_record(in_fd, mod_buf)) == 1)
{
<span style="color: #aaaaaa">@@ -465,7 +469,7 @@ int main(int argc, char *argv[])
</span> if(sref)
continue; /* skip shared records */
<span style="color: #000000;background-color: #ffdddd">- ret = mod_logutils[i]->log_put_record(stitch_fd, mod_buf);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = mod_logutils[i]->log_put_record(stitch_fd, mod_buf, in_fd->mod_ver[i]);
</span> if(ret < 0)
{
fprintf(stderr,
</code></pre>
<br>
</li>
<li id='diff-48'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-48'>
<strong>
darshan-util/doc/darshan-util.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/doc/darshan-util.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/doc/darshan-util.txt
</span><span style="color: #aaaaaa">@@ -66,12 +66,16 @@ application will likely be found in a centralized directory, with the path
</span> and log file name in the following format:
----
-<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan.gz
<span style="color: #000000;background-color: #ddffdd">+<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan
</span> ----
This is a binary format file that summarizes I/O activity. As of version
2.0.0 of Darshan, this file is portable and does not have to be analyzed on
-the same system that executed the job.
<span style="color: #000000;background-color: #ddffdd">+the same system that executed the job. Also, note that Darshan logs generated
+with Darshan versions preceding version 3.0 will have the extension `darshan.gz`
+(or `darshan.bz2` if compressed using bzip2 format). These logs are not compatible
+with Darshan 3.0 utilities, and thus must be analyzed using an appropriate version
+(2.x) of the darshan-util package.
</span>
=== darshan-job-summary.pl
<span style="color: #aaaaaa">@@ -462,9 +466,9 @@ Byte and for the aggregate performance is MiB/s (1024*1024 Bytes/s).
</span>
===== Files
Use the `--file` option to get totals based on file usage.
-The first column is the count of files for that type, the second column is
-number of bytes for that type and the third column is the maximum offset
-accessed.
<span style="color: #000000;background-color: #ddffdd">+Each line has 3 columns. The first column is the count of files for that
+type of file, the second column is number of bytes for that type, and the third
+column is the maximum offset accessed.
</span>
* total: All files
* read_only: Files that were only read from
<span style="color: #aaaaaa">@@ -473,9 +477,6 @@ accessed.
</span> * unique: Files that were opened on only one rank
* shared: File that were opened by more than one rank
-Each line has 3 columns. The first column is the count of files for that
-type of file, the second column is number of bytes for that type, and the third
-column is the maximum offset accessed.
.Example output
----
<span style="color: #aaaaaa">@@ -557,8 +558,6 @@ If the `--bzip2` flag is given, then the output file will be re-compressed in
</span> bzip2 format rather than libz format. It also has command line options for
anonymizing personal data, adding metadata annotation to the log header, and
restricting the output to a specific instrumented file.
-* darshan-diff: compares two darshan log files and shows counters that
-differ.
* darshan-analyzer: walks an entire directory tree of Darshan log files and
produces a summary of the types of access methods used in those log files.
* darshan-logutils*: this is a library rather than an executable, but it
</code></pre>
<br>
</li>
<li id='diff-49'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5#diff-49'>
<strong>
doc/darshan-modularization.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/doc/darshan-modularization.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/darshan-modularization.txt
</span><span style="color: #aaaaaa">@@ -32,8 +32,8 @@ cd darshan
</span> git checkout dev-modular
----
-For details on configuring and building the Darshan runtime and utility repositories,
-consult the documentation from previous versions
<span style="color: #000000;background-color: #ddffdd">+For details on configuring, building, and using the Darshan runtime and utility
+repositories, consult the documentation from previous versions
</span> (http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html[darshan-runtime] and
http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-util.html[darshan-util]) -- the
necessary steps for building these repositories should not have changed in the new version of
<span style="color: #aaaaaa">@@ -121,13 +121,13 @@ component so it is included in the output I/O characterization.
</span> The static initialization approach is useful for modules that do not have function calls
that can be intercepted and instead can just grab all I/O characterization data at Darshan
startup or shutdown time. A module can be statically initialized at Darshan startup time
-by adding it to the `mod_static_init_fns` list at the top of the `lib/darshan-core.c`
-source file.
<span style="color: #000000;background-color: #ddffdd">+by adding its initializatin routine to the `mod_static_init_fns` list at the top of the
+`lib/darshan-core.c` source file.
</span>
-*NOTE*: Modules which require static initialization should typically provide a corresponding
-configure option to prevent the module from being built and capturing I/O data. The ability
-to disable a module using a configure option is especially necessary for system-specific
-modules which can not be built or used on other systems.
<span style="color: #000000;background-color: #ddffdd">+*NOTE*: Modules may wish to add a corresponding configure option to disable the module
+from attempting to gather I/O data. The ability to disable a module using a configure
+option is especially necessary for system-specific modules which can not be built or
+used on other systems.
</span>
Most instrumentation modules can just bootstrap themselves within wrapper functions during
normal application execution. Each of Darshan's current I/O library instrumentation modules
<span style="color: #aaaaaa">@@ -197,7 +197,7 @@ Within darshan-runtime, the darshan-core component manages the initialization an
</span> Darshan environment, provides an interface for modules to register themselves and their data
records with Darshan, and manages the compressing and the writing of the resultant I/O
characterization. As illustrated in Figure 1, the darshan-core runtime environment intercepts
-`MPI_Init` and `MPI_Finalize` routines to initialize and shutdown the darshan runtime environment,
<span style="color: #000000;background-color: #ddffdd">+`MPI_Init` and `MPI_Finalize` routines to initialize and shutdown the Darshan runtime environment,
</span> respectively.
Each of the functions provided by `darshan-core` to interface with instrumentation modules are
<span style="color: #aaaaaa">@@ -212,8 +212,8 @@ void darshan_core_register_module(
</span> int *sys_mem_alignment);
The `darshan_core_register_module` function registers Darshan instrumentation modules with the
-`darshan-core` runtime environment. This function needs to be called at least once for any module
-that will contribute data to Darshan's final I/O characterization.
<span style="color: #000000;background-color: #ddffdd">+`darshan-core` runtime environment. This function needs to be called once for any module that
+will contribute data to Darshan's final I/O characterization.
</span>
* _mod_id_ is a unique identifier for the given module, which is defined in the Darshan log
format header file (`darshan-log-format.h`).
<span style="color: #aaaaaa">@@ -398,8 +398,8 @@ struct darshan_record_ref *, which should be initialized to `NULL` for reading).
</span> is defined by the `uthash` hash table implementation and includes corresponding macros for
searching, iterating, and deleting records from the hash. For detailed documentation on using this
hash table, consult `uthash` documentation in `darshan-util/uthash-1.9.2/doc/txt/userguide.txt`.
-The `darshan-posix-parser` utility (for parsing POSIX module information out of a Darshan log)
-provides an example of how this hash table may be used. Returns `0` on success, `-1` on failure.
<span style="color: #000000;background-color: #ddffdd">+The `darshan-parser` utility (for parsing module information out of a Darshan log) provides an
+example of how this hash table may be used. Returns `0` on success, `-1` on failure.
</span>
[source,c]
int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id, void *mod_buf, int mod_buf_sz);
<span style="color: #aaaaaa">@@ -448,9 +448,10 @@ the module's record structure:
</span>
* Add a module identifier to the `DARSHAN_MODULE_IDS` macro at the top of the `darshan-log-format.h`
header. In this macro, the first field is a corresponding enum value that can be used to
-identify the module, the second field is a string name for the module, and the third field
-is a corresponding pointer to a Darshan log utility implementation for this module (which
-can be set to `NULL` until the module has its own log utility implementation).
<span style="color: #000000;background-color: #ddffdd">+identify the module, the second field is a string name for the module, the third field is the
+current version number of the given module's log format, and the fourth field is a corresponding
+pointer to a Darshan log utility implementation for this module (which can be set to `NULL`
+until the module has its own log utility implementation).
</span>
* Add a top-level header that defines an I/O data record structure for the module. Consider
the "NULL" module and POSIX module log format headers for examples (`darshan-null-log-format.h`
</code></pre>
<br>
</li>
</div>
<div class='footer' style='margin-top: 10px;'>
<p>
—
<br>
<a href="https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5">View it on GitLab</a>
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","action":{"@type":"ViewAction","name":["merge_requests","issues","commit"],"url":"https://xgitlab.cels.anl.gov/darshan/darshan/compare/f78c456f736a007d5efc644293836f9bfc431691...96801d4a5894134dd47159b4f5cf9c31e797d4b5"}}</script>
</p>
</div>
</body>
</html>