<html lang='en'>
<head>
<meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
<title>
GitLab
</title>
</meta>
</head>
<style>
img {
max-width: 100%;
height: auto;
}
p.details {
font-style:italic;
color:#777
}
.footer p {
font-size:small;
color:#777
}
pre.commit-message {
white-space: pre-wrap;
}
.file-stats a {
text-decoration: none;
}
.file-stats .new-file {
color: #090;
}
.file-stats .deleted-file {
color: #B00;
}
</style>
<body>
<div class='content'>
<h3>Shane Snyder pushed to branch dev-modular at <a href="https://xgitlab.cels.anl.gov/darshan/darshan">darshan / darshan</a></h3>
<h4>
Commits:
</h4>
<ul>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/1c1a9baad96fc76a1963de0b15006c7fe866d9a8">1c1a9baa</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T13:02:32Z</i>
</div>
<pre class='commit-message'>cleanup a bunch of code and stale comments</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0">31e0fb03</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-11T14:27:31Z</i>
</div>
<pre class='commit-message'>update runtime, util, and modularization docs</pre>
</li>
</ul>
<h4>12 changed files:</h4>
<ul>
<li class='file-stats'>
<a href='#diff-0'>
darshan-posix-log-format.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-1'>
darshan-runtime/Makefile.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-2'>
darshan-runtime/darshan.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-3'>
darshan-runtime/doc/darshan-runtime.txt
</a>
</li>
<li class='file-stats'>
<a href='#diff-4'>
darshan-runtime/lib/darshan-bgq.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-5'>
darshan-runtime/lib/darshan-posix.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-6'>
darshan-util/Makefile.in
</a>
</li>
<li class='file-stats'>
<a href='#diff-7'>
darshan-util/darshan-logutils.h
</a>
</li>
<li class='file-stats'>
<a href='#diff-8'>
darshan-util/darshan-null-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-9'>
darshan-util/darshan-parser.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-10'>
darshan-util/doc/darshan-util.txt
</a>
</li>
<li class='file-stats'>
<a href='#diff-11'>
doc/darshan-modularization.txt
</a>
</li>
</ul>
<h4>Changes:</h4>
<li id='diff-0'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-0'>
<strong>
darshan-posix-log-format.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-posix-log-format.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-posix-log-format.h
</span><span style="color: #aaaaaa">@@ -177,11 +177,4 @@ struct darshan_posix_file
</span> double fcounters[POSIX_F_NUM_INDICES];
};
-/* This macro can be used to identify files that have been opened using
<span style="color: #000000;background-color: #ffdddd">- * pnetcdf, hdf5, or mpi-io, but were never opened at the posix level. As a
- * result the record will not necessarily have all of the expected fields
- * populated.
- */
</span>-#define POSIX_FILE_PARTIAL(__file)((((__file)->counters[POSIX_OPENS] || (__file)->counters[POSIX_FOPENS] || (__file)->counters[POSIX_STATS]) ? 0 : 1))
-
#endif /* __DARSHAN_POSIX_LOG_FORMAT_H */
</code></pre>
<br>
</li>
<li id='diff-1'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-1'>
<strong>
darshan-runtime/Makefile.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/Makefile.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/Makefile.in
</span><span style="color: #aaaaaa">@@ -33,12 +33,12 @@ CFLAGS_SHARED = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I$(srcdir)
</span>
LIBS = -lz @LIBBZ2@
-static-mod-objs = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o
-dynamic-mod-objs = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o
+DARSHAN_DYNAMIC_MOD_OBJS = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po
</span>
ifdef DARSHAN_USE_BGQ
-static-mod-objs += lib/darshan-bgq.o
-dynamic-mod-objs += lib/darshan-bgq.po
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS += lib/darshan-bgq.o
+DARSHAN_DYNAMIC_MOD_OBJS += lib/darshan-bgq.po
</span> CFLAGS += -DDARSHAN_BGQ
CFLAGS_SHARED += -DDARSHAN_BGQ
endif
<span style="color: #aaaaaa">@@ -118,10 +118,10 @@ lib/lookup8.o: lib/lookup8.c
</span> lib/lookup8.po: lib/lookup8.c
$(CC) $(CFLAGS_SHARED) -c $< -o $@
-lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o $(static-mod-objs) lib/lookup3.o lib/lookup8.o
<span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o $(DARSHAN_STATIC_MOD_OBJS) lib/lookup3.o lib/lookup8.o
</span> ar rcs $@ $^
-lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po $(dynamic-mod-objs) lib/lookup3.po lib/lookup8.po
<span style="color: #000000;background-color: #ddffdd">+lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po $(DARSHAN_DYNAMIC_MOD_OBJS) lib/lookup3.po lib/lookup8.po
</span> $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ -lpthread -lrt -lz -ldl
lib/libdarshan-stubs.a: lib/darshan-hdf5-stubs.o lib/darshan-pnetcdf-stubs.o
</code></pre>
<br>
</li>
<li id='diff-2'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-2'>
<strong>
darshan-runtime/darshan.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/darshan.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/darshan.h
</span><span style="color: #aaaaaa">@@ -115,17 +115,17 @@ void darshan_core_unregister_module(
</span>
/* darshan_core_register_record()
*
<span style="color: #000000;background-color: #ffdddd">- * Register the Darshan record given by 'name' with the darshan-core
- * runtime, allowing it to be properly tracked and (potentially)
- * correlated with records from other modules. 'len' is the size of
- * the name pointer (string length for string names), and 'printable_flag'
- * indicates whether the name is a string. 'mod_limit_flag' is set if
- * the calling module is out of memory (to prevent darshan-core from
- * creating new records and to just search existing records) and 'mod_id'
- * is the identifier of the calling module. 'rec_id' is an output pointer
- * storing the correspoing Darshan record identifier and 'file_alignment'
- * is an output pointer storing the file system alignment value for the
- * given record.
</span><span style="color: #000000;background-color: #ddffdd">+ * Register a record with the darshan-core runtime, allowing it to be
+ * properly tracked and (potentially) correlated with records from other
+ * modules. 'name' is the the name of the Darshan record (e.g., the full
+ * file path) and 'len' is the size of the name pointer (string length
+ * for string names). 'mod_id' is the identifier of the calling module,
+ * 'printable_flag' indicates whether the name is a string, and
+ * 'mod_limit_flag' is set if the calling module is out of memory (to
+ * prevent darshan-core from creating new records and to just search
+ * through existing records). 'rec_id' is an output pointer storing the
+ * correspoing Darshan record identifier and 'file_alignment' is an output
+ * pointer storing the file system alignment value for the given record.
</span> */
void darshan_core_register_record(
void *name,
</code></pre>
<br>
</li>
<li id='diff-3'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-3'>
<strong>
darshan-runtime/doc/darshan-runtime.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/doc/darshan-runtime.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/doc/darshan-runtime.txt
</span><span style="color: #aaaaaa">@@ -16,7 +16,10 @@ used by the application.
</span> The darshan-runtime instrumentation only instruments MPI applications (the
application must at least call `MPI_Init()` and `MPI_Finalize()`). However,
it captures both MPI-IO and POSIX file access. It also captures limited
-information about HDF5 and PnetCDF access.
<span style="color: #000000;background-color: #ddffdd">+information about HDF5 and PnetCDF access. Darshan also exposes an API that
+can be used to develop and add new instrumentation modules (for other I/O library
+interfaces or to gather system-specific data, for instance), as detailed in
+http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-modularization.html[this document].
</span>
This document provides generic installation instructions, but "recipes" for
several common HPC systems are provided at the end of the document as well.
<span style="color: #aaaaaa">@@ -311,7 +314,7 @@ Please set your environment to use the GNU programming environment before
</span> configuring or compiling Darshan. Although Darshan can be built with a
variety of compilers, the GNU compilers are recommended because it will
produce a Darshan library that is interoperable with the widest range
-of compmilers and linkers. On most Cray systems you can enable the GNU
<span style="color: #000000;background-color: #ddffdd">+of compilers and linkers. On most Cray systems you can enable the GNU
</span> programming environment with a command similar to "module swap PrgEnv-pgi
PrgEnv-gnu". Please see your site documentation for information about
how to switch programming environments.
</code></pre>
<br>
</li>
<li id='diff-4'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-4'>
<strong>
darshan-runtime/lib/darshan-bgq.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-bgq.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-bgq.c
</span><span style="color: #aaaaaa">@@ -50,7 +50,7 @@ static int instrumentation_disabled = 0;
</span> static int my_rank = -1;
static int darshan_mem_alignment = 1;
-/* internal helper functions for the "NULL" module */
<span style="color: #000000;background-color: #ddffdd">+/* internal helper functions for the BGQ module */
</span> void bgq_runtime_initialize(void);
/* forward declaration for module functions needed to interface with darshan-core */
<span style="color: #aaaaaa">@@ -58,7 +58,7 @@ static void bgq_begin_shutdown(void);
</span> static void bgq_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs, int shared_rec_count, void **buffer, int *size);
static void bgq_shutdown(void);
-/* macros for obtaining/releasing the "NULL" module lock */
<span style="color: #000000;background-color: #ddffdd">+/* macros for obtaining/releasing the BGQ module lock */
</span> #define BGQ_LOCK() pthread_mutex_lock(&bgq_runtime_mutex)
#define BGQ_UNLOCK() pthread_mutex_unlock(&bgq_runtime_mutex)
<span style="color: #aaaaaa">@@ -133,13 +133,15 @@ void bgq_runtime_initialize()
</span> if(mem_limit == 0)
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
<span style="color: #000000;background-color: #ffdddd">- /* no enough memory to fit bgq module */
</span><span style="color: #000000;background-color: #ddffdd">+ /* not enough memory to fit bgq module */
</span> if (mem_limit < sizeof(*bgq_runtime))
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
<span style="color: #aaaaaa">@@ -148,6 +150,7 @@ void bgq_runtime_initialize()
</span> if(!bgq_runtime)
{
instrumentation_disabled = 1;
<span style="color: #000000;background-color: #ddffdd">+ BGQ_UNLOCK();
</span> return;
}
memset(bgq_runtime, 0, sizeof(*bgq_runtime));
<span style="color: #aaaaaa">@@ -180,7 +183,7 @@ void bgq_runtime_initialize()
</span> return;
}
-/* Perform any necessary steps prior to shutting down for the "NULL" module. */
<span style="color: #000000;background-color: #ddffdd">+/* Perform any necessary steps prior to shutting down for the BGQ module. */
</span> static void bgq_begin_shutdown()
{
BGQ_LOCK();
<span style="color: #aaaaaa">@@ -204,7 +207,7 @@ static int cmpr(const void *p1, const void *p2)
</span> return ((*a == *b) ? 0 : ((*a < *b) ? -1 : 1));
}
-/* Pass output data for the "BGQ" module back to darshan-core to log to file. */
<span style="color: #000000;background-color: #ddffdd">+/* Pass output data for the BGQ module back to darshan-core to log to file. */
</span> static void bgq_get_output_data(
MPI_Comm mod_comm,
darshan_record_id *shared_recs,
<span style="color: #aaaaaa">@@ -212,7 +215,7 @@ static void bgq_get_output_data(
</span> void **buffer,
int *size)
{
<span style="color: #000000;background-color: #ffdddd">- /* Just set the output buffer to point at the array of the "BGQ" module's
</span><span style="color: #000000;background-color: #ddffdd">+ /* Just set the output buffer to point at the array of the BGQ module's
</span> * I/O records, and set the output size according to the number of records
* currently being tracked.
*/
<span style="color: #aaaaaa">@@ -273,7 +276,7 @@ static void bgq_get_output_data(
</span> return;
}
-/* Shutdown the "BGQ" module by freeing up all data structures. */
<span style="color: #000000;background-color: #ddffdd">+/* Shutdown the BGQ module by freeing up all data structures. */
</span> static void bgq_shutdown()
{
BGQ_LOCK();
</code></pre>
<br>
</li>
<li id='diff-5'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-5'>
<strong>
darshan-runtime/lib/darshan-posix.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-posix.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-posix.c
</span><span style="color: #aaaaaa">@@ -39,8 +39,6 @@ typedef int64_t off64_t;
</span> #define aiocb64 aiocb
#endif
-/* TODO: more libc, fgetc, etc etc etc. */
-
DARSHAN_FORWARD_DECL(open, int, (const char *path, int flags, ...));
DARSHAN_FORWARD_DECL(open64, int, (const char *path, int flags, ...));
DARSHAN_FORWARD_DECL(creat, int, (const char* path, mode_t mode));
<span style="color: #aaaaaa">@@ -1728,10 +1726,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_MODE] = inoutfile->counters[POSIX_MODE];
- else
- tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
</span>
/* sum */
for(j=POSIX_BYTES_READ; j<=POSIX_BYTES_WRITTEN; j++)
<span style="color: #aaaaaa">@@ -1754,10 +1749,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_MEM_ALIGNMENT] = inoutfile->counters[POSIX_MEM_ALIGNMENT];
- else
- tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
</span>
/* sum */
for(j=POSIX_FILE_NOT_ALIGNED; j<=POSIX_FILE_NOT_ALIGNED; j++)
<span style="color: #aaaaaa">@@ -1765,10 +1757,7 @@ static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
</span> tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(infile))
- tmp_file.counters[POSIX_FILE_ALIGNMENT] = inoutfile->counters[POSIX_FILE_ALIGNMENT];
- else
- tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
</span>
/* skip POSIX_MAX_*_TIME_SIZE; handled in floating point section */
</code></pre>
<br>
</li>
<li id='diff-6'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-6'>
<strong>
darshan-util/Makefile.in
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/Makefile.in
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/Makefile.in
</span><span style="color: #aaaaaa">@@ -15,6 +15,8 @@ pkgconfigdir = $(DESTDIR)$(libdir)/pkgconfig
</span> DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h
DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h
DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h
<span style="color: #000000;background-color: #ddffdd">+DARSHAN_STATIC_MOD_OBJS = darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o
+DARSHAN_DYNAMIC_MOD_OBJS = darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po
</span>
DARSHAN_ENABLE_SHARED=@DARSHAN_ENABLE_SHARED@
<span style="color: #aaaaaa">@@ -79,13 +81,12 @@ darshan-bgq-logutils.o: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-lo
</span> darshan-bgq-logutils.po: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2
$(CC) $(CFLAGS_SHARED) -c $< -o $@
<span style="color: #000000;background-color: #ddffdd">+libdarshan-util.a: darshan-logutils.o $(DARSHAN_STATIC_MOD_OBJS)
+ ar rcs libdarshan-util.a $^
</span>
-libdarshan-util.so: darshan-logutils.po darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po
<span style="color: #000000;background-color: #ddffdd">+libdarshan-util.so: darshan-logutils.po $(DARSHAN_DYNAMIC_MOD_OBJS)
</span> $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ $(LIBS)
-libdarshan-util.a: darshan-logutils.o darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o
- ar rcs libdarshan-util.a $^
-
jenkins-hash-gen: jenkins-hash-gen.c lookup3.o
$(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
</code></pre>
<br>
</li>
<li id='diff-7'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-7'>
<strong>
darshan-util/darshan-logutils.h
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.h
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.h
</span><span style="color: #aaaaaa">@@ -55,6 +55,9 @@ struct darshan_mod_logutil_funcs
</span> /* retrieve a single module record from the log file.
* return 1 on successful read of record, 0 on no more
* module data, -1 on error
<span style="color: #000000;background-color: #ddffdd">+ * - 'fd' is the file descriptor to get record from
+ * - 'buf' is the buffer to store the record in
+ * - 'rec_id' is the corresponding darshan record id
</span> */
int (*log_get_record)(
darshan_fd fd,
<span style="color: #aaaaaa">@@ -63,13 +66,22 @@ struct darshan_mod_logutil_funcs
</span> );
/* put a single module record into the log file.
* return 0 on success, -1 on error
<span style="color: #000000;background-color: #ddffdd">+ * - 'fd' is the file descriptor to put record into
+ * - 'buf' is the buffer containing the record data
+ * - 'rec_id' is the corresponding darshan record id
</span> */
int (*log_put_record)(
darshan_fd fd,
void *buf,
int ver
);
<span style="color: #000000;background-color: #ffdddd">- /* print the counters for a given log file record */
</span><span style="color: #000000;background-color: #ddffdd">+ /* print the counters for a given log record
+ * - 'file_rec' is the record's data buffer
+ * - 'file_name' is the file path string for the record
+ * - 'mnt-pt' is the file path mount point string
+ * - 'fs_type' is the file system type string
+ * - 'ver' is the version of the record
+ */
</span> void (*log_print_record)(
void *file_rec,
char *file_name,
</code></pre>
<br>
</li>
<li id='diff-8'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-8'>
<strong>
darshan-util/darshan-null-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-null-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-null-logutils.c
</span><span style="color: #aaaaaa">@@ -51,7 +51,8 @@ struct darshan_mod_logutil_funcs null_logutils =
</span>
/* retrieve a NULL record from log file descriptor 'fd', storing the
* buffer in 'null_buf' and the corresponding Darshan record id in
<span style="color: #000000;background-color: #ffdddd">- * 'rec_id'. Return 1 on successful record read, .
</span><span style="color: #000000;background-color: #ddffdd">+ * 'rec_id'. Return 1 on successful record read, 0 on no more data,
+ * and -1 on error.
</span> */
static int darshan_log_get_null_record(darshan_fd fd, void* null_buf,
darshan_record_id* rec_id)
</code></pre>
<br>
</li>
<li id='diff-9'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-9'>
<strong>
darshan-util/darshan-parser.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-parser.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-parser.c
</span><span style="color: #aaaaaa">@@ -690,8 +690,7 @@ void posix_accum_file(struct darshan_posix_file *pfile,
</span> case POSIX_MODE:
case POSIX_MEM_ALIGNMENT:
case POSIX_FILE_ALIGNMENT:
<span style="color: #000000;background-color: #ffdddd">- if(POSIX_FILE_PARTIAL(tmp))
- tmp->counters[i] = pfile->counters[i];
</span><span style="color: #000000;background-color: #ddffdd">+ tmp->counters[i] = pfile->counters[i];
</span> break;
case POSIX_MAX_BYTE_READ:
case POSIX_MAX_BYTE_WRITTEN:
</code></pre>
<br>
</li>
<li id='diff-10'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-10'>
<strong>
darshan-util/doc/darshan-util.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/doc/darshan-util.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/doc/darshan-util.txt
</span><span style="color: #aaaaaa">@@ -66,12 +66,16 @@ application will likely be found in a centralized directory, with the path
</span> and log file name in the following format:
----
-<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan.gz
<span style="color: #000000;background-color: #ddffdd">+<YEAR>/<MONTH>/<DAY>/<USERNAME>_<BINARY_NAME>_<JOB_ID>_<DATE>_<UNIQUE_ID>_<TIMING>.darshan
</span> ----
This is a binary format file that summarizes I/O activity. As of version
2.0.0 of Darshan, this file is portable and does not have to be analyzed on
-the same system that executed the job.
<span style="color: #000000;background-color: #ddffdd">+the same system that executed the job. Also, note that Darshan logs generated
+with Darshan versions preceding version 3.0 will have the extension `darshan.gz`
+(or `darshan.bz2` if compressed using bzip2 format). These logs are not compatible
+with Darshan 3.0 utilities, and thus must be analyzed using an appropriate version
+(2.x) of the darshan-util package.
</span>
=== darshan-job-summary.pl
<span style="color: #aaaaaa">@@ -462,9 +466,9 @@ Byte and for the aggregate performance is MiB/s (1024*1024 Bytes/s).
</span>
===== Files
Use the `--file` option to get totals based on file usage.
-The first column is the count of files for that type, the second column is
-number of bytes for that type and the third column is the maximum offset
-accessed.
<span style="color: #000000;background-color: #ddffdd">+Each line has 3 columns. The first column is the count of files for that
+type of file, the second column is number of bytes for that type, and the third
+column is the maximum offset accessed.
</span>
* total: All files
* read_only: Files that were only read from
<span style="color: #aaaaaa">@@ -473,9 +477,6 @@ accessed.
</span> * unique: Files that were opened on only one rank
* shared: File that were opened by more than one rank
-Each line has 3 columns. The first column is the count of files for that
-type of file, the second column is number of bytes for that type, and the third
-column is the maximum offset accessed.
.Example output
----
<span style="color: #aaaaaa">@@ -557,8 +558,6 @@ If the `--bzip2` flag is given, then the output file will be re-compressed in
</span> bzip2 format rather than libz format. It also has command line options for
anonymizing personal data, adding metadata annotation to the log header, and
restricting the output to a specific instrumented file.
-* darshan-diff: compares two darshan log files and shows counters that
-differ.
* darshan-analyzer: walks an entire directory tree of Darshan log files and
produces a summary of the types of access methods used in those log files.
* darshan-logutils*: this is a library rather than an executable, but it
</code></pre>
<br>
</li>
<li id='diff-11'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0#diff-11'>
<strong>
doc/darshan-modularization.txt
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/doc/darshan-modularization.txt
</span><span style="color: #000000;background-color: #ddffdd">+++ b/doc/darshan-modularization.txt
</span><span style="color: #aaaaaa">@@ -32,8 +32,8 @@ cd darshan
</span> git checkout dev-modular
----
-For details on configuring and building the Darshan runtime and utility repositories,
-consult the documentation from previous versions
<span style="color: #000000;background-color: #ddffdd">+For details on configuring, building, and using the Darshan runtime and utility
+repositories, consult the documentation from previous versions
</span> (http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html[darshan-runtime] and
http://www.mcs.anl.gov/research/projects/darshan/docs/darshan-util.html[darshan-util]) -- the
necessary steps for building these repositories should not have changed in the new version of
<span style="color: #aaaaaa">@@ -121,13 +121,13 @@ component so it is included in the output I/O characterization.
</span> The static initialization approach is useful for modules that do not have function calls
that can be intercepted and instead can just grab all I/O characterization data at Darshan
startup or shutdown time. A module can be statically initialized at Darshan startup time
-by adding it to the `mod_static_init_fns` list at the top of the `lib/darshan-core.c`
-source file.
<span style="color: #000000;background-color: #ddffdd">+by adding its initializatin routine to the `mod_static_init_fns` list at the top of the
+`lib/darshan-core.c` source file.
</span>
-*NOTE*: Modules which require static initialization should typically provide a corresponding
-configure option to prevent the module from being built and capturing I/O data. The ability
-to disable a module using a configure option is especially necessary for system-specific
-modules which can not be built or used on other systems.
<span style="color: #000000;background-color: #ddffdd">+*NOTE*: Modules may wish to add a corresponding configure option to disable the module
+from attempting to gather I/O data. The ability to disable a module using a configure
+option is especially necessary for system-specific modules which can not be built or
+used on other systems.
</span>
Most instrumentation modules can just bootstrap themselves within wrapper functions during
normal application execution. Each of Darshan's current I/O library instrumentation modules
<span style="color: #aaaaaa">@@ -197,7 +197,7 @@ Within darshan-runtime, the darshan-core component manages the initialization an
</span> Darshan environment, provides an interface for modules to register themselves and their data
records with Darshan, and manages the compressing and the writing of the resultant I/O
characterization. As illustrated in Figure 1, the darshan-core runtime environment intercepts
-`MPI_Init` and `MPI_Finalize` routines to initialize and shutdown the darshan runtime environment,
<span style="color: #000000;background-color: #ddffdd">+`MPI_Init` and `MPI_Finalize` routines to initialize and shutdown the Darshan runtime environment,
</span> respectively.
Each of the functions provided by `darshan-core` to interface with instrumentation modules are
<span style="color: #aaaaaa">@@ -212,8 +212,8 @@ void darshan_core_register_module(
</span> int *sys_mem_alignment);
The `darshan_core_register_module` function registers Darshan instrumentation modules with the
-`darshan-core` runtime environment. This function needs to be called at least once for any module
-that will contribute data to Darshan's final I/O characterization.
<span style="color: #000000;background-color: #ddffdd">+`darshan-core` runtime environment. This function needs to be called once for any module that
+will contribute data to Darshan's final I/O characterization.
</span>
* _mod_id_ is a unique identifier for the given module, which is defined in the Darshan log
format header file (`darshan-log-format.h`).
<span style="color: #aaaaaa">@@ -398,8 +398,8 @@ struct darshan_record_ref *, which should be initialized to `NULL` for reading).
</span> is defined by the `uthash` hash table implementation and includes corresponding macros for
searching, iterating, and deleting records from the hash. For detailed documentation on using this
hash table, consult `uthash` documentation in `darshan-util/uthash-1.9.2/doc/txt/userguide.txt`.
-The `darshan-posix-parser` utility (for parsing POSIX module information out of a Darshan log)
-provides an example of how this hash table may be used. Returns `0` on success, `-1` on failure.
<span style="color: #000000;background-color: #ddffdd">+The `darshan-parser` utility (for parsing module information out of a Darshan log) provides an
+example of how this hash table may be used. Returns `0` on success, `-1` on failure.
</span>
[source,c]
int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id, void *mod_buf, int mod_buf_sz);
<span style="color: #aaaaaa">@@ -448,9 +448,10 @@ the module's record structure:
</span>
* Add a module identifier to the `DARSHAN_MODULE_IDS` macro at the top of the `darshan-log-format.h`
header. In this macro, the first field is a corresponding enum value that can be used to
-identify the module, the second field is a string name for the module, and the third field
-is a corresponding pointer to a Darshan log utility implementation for this module (which
-can be set to `NULL` until the module has its own log utility implementation).
<span style="color: #000000;background-color: #ddffdd">+identify the module, the second field is a string name for the module, the third field is the
+current version number of the given module's log format, and the fourth field is a corresponding
+pointer to a Darshan log utility implementation for this module (which can be set to `NULL`
+until the module has its own log utility implementation).
</span>
* Add a top-level header that defines an I/O data record structure for the module. Consider
the "NULL" module and POSIX module log format headers for examples (`darshan-null-log-format.h`
</code></pre>
<br>
</li>
</div>
<div class='footer' style='margin-top: 10px;'>
<p>
—
<br>
<a href="https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0">View it on GitLab</a>
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","action":{"@type":"ViewAction","name":["merge_requests","issues","commit"],"url":"https://xgitlab.cels.anl.gov/darshan/darshan/compare/c371cdea3257003237f12c63ff688c35934a3464...31e0fb031c8a3e45f894f5fdeeb431cdf5c237c0"}}</script>
</p>
</div>
</body>
</html>