[Darshan-commits] [Darshan] branch, dev-modular, updated. darshan-2.3.1-52-gb4a0aed
Service Account
git at mcs.anl.gov
Wed Mar 18 12:54:27 CDT 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "".
The branch, dev-modular has been updated
via b4a0aed40c78cfa4904b3b0134defc8ed0c04181 (commit)
from 657e1b1a816abdb4ca7e2226a7c2573533ce0040 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit b4a0aed40c78cfa4904b3b0134defc8ed0c04181
Author: Shane Snyder <ssnyder at mcs.anl.gov>
Date: Wed Mar 18 12:53:31 2015 -0500
first cut at complete dev-modular documentation
-----------------------------------------------------------------------
Summary of changes:
darshan-util/Makefile.in | 1 -
darshan-util/darshan-logutils.h | 8 ++-
darshan-util/darshan-posix-parser.c | 1 +
doc/darshan-modularization.txt | 95 +++++++++++++++++++++++++++++++++--
4 files changed, 96 insertions(+), 9 deletions(-)
Diff of changes:
diff --git a/darshan-util/Makefile.in b/darshan-util/Makefile.in
index 60f3436..55a97dc 100644
--- a/darshan-util/Makefile.in
+++ b/darshan-util/Makefile.in
@@ -1,5 +1,4 @@
all: darshan-util-lib darshan-posix-parser
-#all: darshan-parser darshan-convert darshan-diff darshan-analyzer darshan-log-params darshan-util-lib
DESTDIR =
srcdir = @srcdir@
diff --git a/darshan-util/darshan-logutils.h b/darshan-util/darshan-logutils.h
index c0dd82e..6ca1787 100644
--- a/darshan-util/darshan-logutils.h
+++ b/darshan-util/darshan-logutils.h
@@ -17,6 +17,8 @@
/* default to a compression buffer size of 4 MiB */
#define DARSHAN_DEF_DECOMP_BUF_SZ (4*1024*1024)
+/* TODO: we need to refactor s.t. utilities don't know implementation
+ of this, but module-specific functions do */
struct darshan_fd_s
{
gzFile gzf;
@@ -35,12 +37,12 @@ struct darshan_record_ref
};
darshan_fd darshan_log_open(const char *name, const char* mode);
-int darshan_log_getheader(darshan_fd file, struct darshan_header *header);
-int darshan_log_getjob(darshan_fd file, struct darshan_job *job);
+int darshan_log_getheader(darshan_fd fd, struct darshan_header *header);
+int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);
int darshan_log_getexe(darshan_fd fd, char *buf);
int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
char*** fs_types, int* count);
-int darshan_log_gethash(darshan_fd file, struct darshan_record_ref **hash);
+int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);
int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id,
void *moddat_buf, int moddat_buf_sz);
void darshan_log_close(darshan_fd file);
diff --git a/darshan-util/darshan-posix-parser.c b/darshan-util/darshan-posix-parser.c
index 597c042..cf829d1 100644
--- a/darshan-util/darshan-posix-parser.c
+++ b/darshan-util/darshan-posix-parser.c
@@ -15,6 +15,7 @@
#include <getopt.h>
#include <assert.h>
+#include "darshan-logutils.h"
#include "darshan-posix-logutils.h"
#include "uthash-1.9.2/src/uthash.h"
diff --git a/doc/darshan-modularization.txt b/doc/darshan-modularization.txt
index 7c4af24..0d9eb5b 100644
--- a/doc/darshan-modularization.txt
+++ b/doc/darshan-modularization.txt
@@ -287,7 +287,80 @@ simplifying maintenance.
=== Darshan-util
-Text.
+The darshan-util component is composed of a log parsing library (libdarshan-util) and a
+corresponding set of utility programs that can parse and analyze Darshan I/O characterization
+logs using this library. The log parsing library includes a generic interface (see
+`darshan-logutils.h`) for retrieving specific portions of a given log file. Specifically,
+this interface allows utilities to retrieve a log's header metadata, job details, record
+identifier mapping, and any module-specific data contained within the log.
+
+Module developers may wish to define additional interfaces for parsing module-specific data
+that can then be integrated into the log parsing library. This extended functionality can be
+implemented in terms of the generic functions offered by darshan-logutils and by module-specific
+formatting information.
+
+==== darshan-logutils
+
+Here we define each function of the `darshan-logutils` interface, which can be used to implement
+new log analysis utilities and to define module-specific functionality.
+
+[source,c]
+darshan_fd darshan_log_open(const char *name, const char* mode);
+
+Opens Darshan log file stored at path `name`. `mode` can only be `r` for reading or `w` for
+writing. Returns a Darshan file descriptor on success, or `NULL` on error.
+
+[source,c]
+int darshan_log_getheader(darshan_fd fd, struct darshan_header *header);
+
+Fills in `header` structure from the log file referenced by descriptor `fd`. The `darshan_header`
+structure is defined in `darshan-log-format.h`. Returns `0` on success, `-1` on failure.
+
+[source,c]
+int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);
+
+Fills in `job` structure from the log file referenced by descriptor `fd`. The `darshan_job`
+structure is defined in `darshan-log-format.h`. Returns `0` on success, `-1` on failure.
+
+[source,c]
+int darshan_log_getexe(darshan_fd fd, char *buf);
+
+Fills in `buf` with the corresponding executable string (exe name and command line arguments)
+for the Darshan log referenced by `fd`. Returns `0` on success, `-1` on failure.
+
+[source,c]
+int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts, char*** fs_types, int* count);
+
+Returns mounted file system information for the Darshan log referenced by `fd`. `mnt_pnts` points
+to an array of strings storing mount points, `fs_types` points to an array of strings storing file
+system types (e.g., ext4, nfs, etc.), and `count` points to an integer storing the total number
+of mounted file systems recorded by Darshan. Returns `0` on success, `-1` on failure.
+
+[source,c]
+int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);
+
+Builds hash table of Darshan record identifiers to full names for all records contained
+in the Darshan log referenced by `fd`. `hash` is a pointer to the hash table (of type
+struct darshan_record_ref *, which should be initialized to `NULL`). This hash table is
+defined by the `uthash` hash table implementation and includes corresponding macros for
+searching, iterating, and deleting records from the hash. For detailed documentation on using this
+hash table, consult `uthash` documentation in `darshan-util/uthash-1.9.2/doc/txt/userguide.txt`.
+The `darshan-posix-parser` utility (for parsing POSIX module information out of a Darshan log)
+provides an example of how this hash table may be used. Returns `0` on success, `-1` on failure.
+
+[source,c]
+int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id, void *moddat_buf, int moddat_buf_sz);
+
+Retrieves a chunk of (uncompressed) data for the module identified by `mod_id` from the Darshan log
+referenced by `fd`. `moddat_buf_sz` specifies the number of uncompressed bytes to read from the file
+and store in `moddat_buf`. This function may be repeatedly called to retrieve sequential chunks of data
+from a given Darshan file descriptor. This function returns `1` if `moddat_buf_sz` bytes were read
+successfully, `0` if no more data is available for this module, and `-1` otherwise.
+
+[source,c]
+void darshan_log_close(darshan_fd fd);
+
+Close Darshan file descriptor `fd`. Returns `0` on success, `-1` on failure.
== Adding new instrumentation modules
@@ -318,9 +391,6 @@ of `CP_WRAPPERS` in `darshan-config.in`.
source files, which will be stored in the `lib/` directory. The prerequisites to building
static and dynamic versions of `lib-darshan` must be updated to include these objects, as well.
-It is necessary to rerun the `prepare` script and reconfigure darshan-runtime for these changes
-to take effect.
-
==== Instrumentation module implementation
An exemplar instrumentation module for POSIX I/O functions is given in `lib/darshan-posix.c` as
@@ -347,7 +417,22 @@ to store the record structure in a hash table or some other structure.
==== Build modifications
-Text.
+The following modifications to the darshan-util build system are necessary to integrate new
+instrumentation modules:
+
+* Update `Makefile.in` with new targets necessary for building new utilities and module-specific
+log parsing source.
+ - Make sure to update `all`, `clean`, and `install` targets to reference updates.
+ - If adding new log parsing functionality, make sure to add it as a prerequisite source for
+ building libdarshan-util.
+
+==== Module-specific logutils and utilities
+
+For a straightforward reference implementation of module-specific log parsing functionality for
+the POSIX module, consider files `darshan-posix-logutils.c` and `darshan-posix-logutils.h`.
+
+Also, the `darshan-posix-parser` source provides a simple example of a utility which can leverage
+libdarshan-util for analyzing the contents of a given Darshan I/O characterization log.
== Other resources
hooks/post-receive
--
More information about the Darshan-commits
mailing list