[Darshan-commits] [Git][darshan/darshan][master] 31 commits: Check in Darshan eXtended Lustre Tracing(DXLT) code
Shane Snyder
xgitlab at cels.anl.gov
Fri Dec 16 11:16:39 CST 2016
Shane Snyder pushed to branch master at darshan / darshan
Commits:
db79f903 by Cong Xu at 2016-10-04T17:35:03-07:00
Check in Darshan eXtended Lustre Tracing(DXLT) code
1. Use Lustre Module in Darshan
2. Integrate DXLT IO tracing code into one module
3. Add hooks in POSIX and MPI Modules
4. Add Intel copyright
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
2815aea0 by Cong Xu at 2016-10-10T12:31:36-07:00
Check in Darshan eXtended Lustre Tracing(DXLT) code
1. Use Lustre Module in Darshan
2. Integrate DXLT IO tracing code into one module
3. Add hooks in POSIX and MPI Modules
4. Add Intel copyright
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
ed28a4b2 by Cong Xu at 2016-10-10T12:34:00-07:00
Merge branch 'master' of xgitlab:intel-hpdd/darshan-dxlt
- - - - -
e5df8a50 by Cong Xu at 2016-10-13T20:04:17-07:00
Address stripe_size=0 and stripe_count=0 bugs
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
cc099d3e by Cong Xu at 2016-10-13T23:36:39-07:00
1. Remove redundant information in dxlt_file_record.
2. Refactor DXLT to be a generic IO tracing tool for files on any file system.
3. Add file type information in DXLT log parser.
Next steps:
1. Based on the file type information in DXLT log parser, add information for specific file types. (For instance, add OST information for files on Lustre)
2. In the process of parsing DXLT logs, figure out how to obtain Lustre striping information from Lustre module.
3. Rename DXLT to be "Darshan eXtended Tracing (DXT)" or "Darshan eXtended IO Tracing (DXIOT)".
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
9f1491bd by Cong Xu at 2016-10-14T21:35:35-07:00
Address comments from Shane:
1. At log parsing time, pass Lustre striping information to DXLT module
a. Obtain Lustre striping information in Lustre Module.
b. Cache file striping information and map it to file name.
c. Pass file striping information to DXLT module for each file.
2. In DXLT module, print Lustre striping and OST information if file system type is Lustre.
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
5db2a039 by Cong Xu at 2016-10-17T15:34:48-07:00
Change DXLT name to Darshan eXtended Tracing (DXT)
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
56a61bac by Shane Snyder at 2016-10-20T14:46:55-05:00
make DXT module request 0 memory from darshan-core
- - - - -
03e86bd8 by Shane Snyder at 2016-10-24T13:38:12-05:00
reorganize dxt & posix module interactions
- - - - -
9b40a778 by Shane Snyder at 2016-10-24T15:04:13-05:00
reorganize dxt & mpiio module interactions
- - - - -
3d2de3ab by Shane Snyder at 2016-10-24T16:57:10-05:00
modify dxt cleanup code to reuse rec_ref interface
- - - - -
7e0952c1 by Shane Snyder at 2016-10-25T13:32:16-05:00
cleanup dxt shutdown routines
- - - - -
2f428895 by Shane Snyder at 2016-10-26T12:42:14-05:00
add stubbed out stand-alone dxt parser
- - - - -
7458044e by Cong Xu at 2016-10-26T14:00:29-07:00
Move redundant write_traces and read_traces buffer pointers to dxlt_file_record_ref
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
fcfcfe8a by Cong Xu at 2016-10-28T10:49:59-07:00
Implement a separate dxt parser for parsing dxt logs
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
7a09fdee by Cong Xu at 2016-11-01T12:57:55-07:00
Avoid segment fault if user uses darshan-parser to parse DXT logs
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
faa23f57 by Cong Xu at 2016-11-08T12:44:38-08:00
Add hostname information to DXT
Signed-off-by: Cong Xu <cong.xu at intel.com>
- - - - -
d1fc064b by Cong Xu at 2016-11-17T13:00:22-08:00
Address DXT parser segment fault when Lustre module is disabled
Signed-off-by: Cong Xu <congxu at localhost.localdomain>
- - - - -
c599503b by Shane Snyder at 2016-11-30T16:29:02-06:00
cleanup some unneeded variables in dxt
no instrumenation_disabled flags are needed since this
functionality will be managed by the overlying MPI-IO and POSIX
modules (DXT calls are embedded in wrappers in these modules).
- - - - -
ce3d9ff3 by Shane Snyder at 2016-11-30T16:56:20-06:00
free dxt buffers at shutdown time
- - - - -
4bbcbd21 by Shane Snyder at 2016-11-30T17:49:43-06:00
dxt modules now indicate if they have partial data
- - - - -
ab22a28c by Shane Snyder at 2016-12-01T13:29:38-06:00
make sure to byte swap segment data
- - - - -
981c0f33 by Shane Snyder at 2016-12-01T15:09:30-06:00
cleanup dxt parser code
- - - - -
fef3f1fc by Shane Snyder at 2016-12-01T17:09:03-06:00
bug fix in lustre logutils
- - - - -
b800d571 by Shane Snyder at 2016-12-02T14:53:35-06:00
cleanup dxt logutils code
- - - - -
2f4d0dbc by Shane Snyder at 2016-12-08T17:59:10-06:00
update dxt parser to indicate if data is partial
- - - - -
6d33b9e9 by Cong Xu at 2016-12-08T22:46:53-08:00
Address darshan-dxt-parser segment fault
Signed-off-by: Cong Xu <congxu at localhost.localdomain>
- - - - -
6806ff0c by Cong Xu at 2016-12-09T16:15:18-08:00
Merge branch 'master' of https://xgitlab.cels.anl.gov/darshan/darshan
- - - - -
c65c0b74 by Shane Snyder at 2016-12-15T11:35:54-06:00
remove dead dxt code from lustre module
- - - - -
61272682 by Shane Snyder at 2016-12-16T11:13:49-06:00
don't disable MPI-IO hints when using DXT
- - - - -
7ec53306 by Shane Snyder at 2016-12-16T11:16:17-06:00
Merge branch 'master' into 'master'
Merge Darshan eXtended Tracing (DXT) to Darshan Main stream
See merge request !2
- - - - -
13 changed files:
- + darshan-dxt-log-format.h
- darshan-log-format.h
- darshan-runtime/Makefile.in
- darshan-runtime/lib/darshan-core.c
- + darshan-runtime/lib/darshan-dxt.c
- darshan-runtime/lib/darshan-mpiio.c
- darshan-runtime/lib/darshan-posix.c
- darshan-util/Makefile.in
- + darshan-util/darshan-dxt-logutils.c
- + darshan-util/darshan-dxt-logutils.h
- + darshan-util/darshan-dxt-parser.c
- darshan-util/darshan-logutils.h
- darshan-util/darshan-lustre-logutils.c
Changes:
=====================================
darshan-dxt-log-format.h
=====================================
--- /dev/null
+++ b/darshan-dxt-log-format.h
@@ -0,0 +1,45 @@
+/*
+ * Copyright (C) 2016 Intel Corporation.
+ * See COPYRIGHT in top-level directory.
+ */
+
+#ifndef __DARSHAN_DXT_LOG_FORMAT_H
+#define __DARSHAN_DXT_LOG_FORMAT_H
+
+/* current DXT log format version */
+#define DXT_POSIX_VER 1
+#define DXT_MPIIO_VER 1
+
+#define HOSTNAME_SIZE 64
+
+/*
+ * DXT, the segment_info structure maintains detailed Segment IO tracing
+ * information
+ */
+typedef struct segment_info {
+ int64_t offset;
+ int64_t length;
+ double start_time;
+ double end_time;
+} segment_info;
+
+#define X(a) a,
+#undef X
+
+/* file record structure for DXT files. a record is created and stored for
+ * every DXT file opened by the original application. For the DXT module,
+ * the record includes:
+ * - a darshan_base_record structure, which contains the record id & rank
+ * - integer file I/O statistics (open, read/write counts, etc)
+ * - floating point I/O statistics (timestamps, cumulative timers, etc.)
+ */
+struct dxt_file_record {
+ struct darshan_base_record base_rec;
+ int64_t shared_record; /* -1 means it is a shared file record */
+ char hostname[HOSTNAME_SIZE];
+
+ int64_t write_count;
+ int64_t read_count;
+};
+
+#endif /* __DARSHAN_DXT_LOG_FORMAT_H */
=====================================
darshan-log-format.h
=====================================
--- a/darshan-log-format.h
+++ b/darshan-log-format.h
@@ -117,6 +117,8 @@ struct darshan_base_record
#include "darshan-bgq-log-format.h"
#include "darshan-lustre-log-format.h"
#include "darshan-stdio-log-format.h"
+/* DXT */
+#include "darshan-dxt-log-format.h"
/* X-macro for keeping module ordering consistent */
/* NOTE: first val used to define module enum values,
@@ -136,7 +138,10 @@ struct darshan_base_record
X(DARSHAN_PNETCDF_MOD, "PNETCDF", DARSHAN_PNETCDF_VER, &pnetcdf_logutils) \
X(DARSHAN_BGQ_MOD, "BG/Q", DARSHAN_BGQ_VER, &bgq_logutils) \
X(DARSHAN_LUSTRE_MOD, "LUSTRE", DARSHAN_LUSTRE_VER, &lustre_logutils) \
- X(DARSHAN_STDIO_MOD, "STDIO", DARSHAN_STDIO_VER, &stdio_logutils)
+ X(DARSHAN_STDIO_MOD, "STDIO", DARSHAN_STDIO_VER, &stdio_logutils) \
+ /* DXT */ \
+ X(DXT_POSIX_MOD, "X_POSIX", DXT_POSIX_VER, &dxt_posix_logutils) \
+ X(DXT_MPIIO_MOD, "X_MPIIO", DXT_MPIIO_VER, &dxt_mpiio_logutils)
/* unique identifiers to distinguish between available darshan modules */
/* NOTES: - valid ids range from [0...DARSHAN_MAX_MODS-1]
=====================================
darshan-runtime/Makefile.in
=====================================
--- a/darshan-runtime/Makefile.in
+++ b/darshan-runtime/Makefile.in
@@ -35,8 +35,8 @@ CFLAGS_SHARED = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I$(srcdir)
LIBS = -lz @LIBBZ2@
-DARSHAN_STATIC_MOD_OBJS = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o lib/darshan-stdio.o
-DARSHAN_DYNAMIC_MOD_OBJS = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po lib/darshan-stdio.po
+DARSHAN_STATIC_MOD_OBJS = lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o lib/darshan-stdio.o lib/darshan-dxt.o
+DARSHAN_DYNAMIC_MOD_OBJS = lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-hdf5.po lib/darshan-pnetcdf.po lib/darshan-stdio.po lib/darshan-dxt.po
ifdef DARSHAN_USE_BGQ
DARSHAN_STATIC_MOD_OBJS += lib/darshan-bgq.o
@@ -127,6 +127,12 @@ lib/darshan-stdio.o: lib/darshan-stdio.c darshan.h darshan-common.h $(DARSHAN_LO
lib/darshan-stdio.po: lib/darshan-stdio.c darshan.h darshan-dynamic.h darshan-common.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-stdio-log-format.h | lib
$(CC) $(CFLAGS_SHARED) -c $< -o $@
+lib/darshan-dxt.o: lib/darshan-dxt.c darshan.h darshan-common.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-dxt-log-format.h | lib
+ $(CC) $(CFLAGS) -c $< -o $@
+
+lib/darshan-dxt.po: lib/darshan-dxt.c darshan.h darshan-dynamic.h darshan-common.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-dxt-log-format.h | lib
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
+
lib/lookup3.o: lib/lookup3.c
$(CC) $(CFLAGS) -c $< -o $@
=====================================
darshan-runtime/lib/darshan-core.c
=====================================
--- a/darshan-runtime/lib/darshan-core.c
+++ b/darshan-runtime/lib/darshan-core.c
@@ -607,6 +607,12 @@ void darshan_core_shutdown()
final_core->log_hdr_p->mod_map[i].len =
gz_fp - final_core->log_hdr_p->mod_map[i].off;
+ /* XXX: DXT manages its own module memory buffers, so we need to
+ * explicitly free them
+ */
+ if(i == DXT_POSIX_MOD || i == DXT_MPIIO_MOD)
+ free(mod_buf);
+
/* error out if the log append failed */
DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,
MPI_LOR, MPI_COMM_WORLD);
=====================================
darshan-runtime/lib/darshan-dxt.c
=====================================
--- /dev/null
+++ b/darshan-runtime/lib/darshan-dxt.c
@@ -0,0 +1,821 @@
+/*
+ * Copyright (C) 2016 Intel Corporation.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#define _XOPEN_SOURCE 500
+#define _GNU_SOURCE
+
+#include "darshan-runtime-config.h"
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdarg.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <sys/uio.h>
+#include <sys/mman.h>
+#include <search.h>
+#include <assert.h>
+#include <libgen.h>
+#include <pthread.h>
+
+#include "utlist.h"
+#include "uthash.h"
+#include "darshan.h"
+#include "darshan-dynamic.h"
+
+#ifndef HAVE_OFF64_T
+typedef int64_t off64_t;
+#endif
+
+/* maximum amount of memory to use for storing DXT records */
+#define DXT_IO_TRACE_MEM_MAX (4 * 1024 * 1024) /* 4 MiB */
+
+/* initial size of read/write trace buffer (in number of segments) */
+/* NOTE: when this size is exceeded, the buffer size is doubled */
+#define IO_TRACE_BUF_SIZE 64
+
+/* XXX: dirty hack -- If DXT runs out of memory to store trace data in,
+ * we should set a flag so that log parsers know that the log has
+ * incomplete data. This functionality is typically handled automatically
+ * when registering records with Darshan, but DXT modules don't
+ * register records and manage their own memory. Since DXT modules request
+ * 0 memory when registering themselves, any attempt to register a record
+ * will result in setting the partial flag for the module, which is
+ * exactly what we do here.
+ */
+#define SET_DXT_MOD_PARTIAL_FLAG(mod_id) \
+ darshan_core_register_record(0, NULL, mod_id, 1, NULL);
+
+/* The dxt_file_record_ref structure maintains necessary runtime metadata
+ * for the DXT file record (dxt_file_record structure, defined in
+ * darshan-dxt-log-format.h) pointed to by 'file_rec'. This metadata
+ * assists with the instrumenting of specific statistics in the file record.
+ *
+ * NOTE: we use the 'darshan_record_ref' interface (in darshan-common) to
+ * associate different types of handles with this dxt_file_record_ref struct.
+ * This allows us to index this struct (and the underlying file record) by using
+ * either the corresponding Darshan record identifier (derived from the filename)
+ * or by a generated file descriptor, for instance. Note that, while there should
+ * only be a single Darshan record identifier that indexes a dxt_file_record_ref,
+ * there could be multiple open file descriptors that index it.
+ */
+struct dxt_file_record_ref
+{
+ struct dxt_file_record *file_rec;
+
+ int64_t write_available_buf;
+ int64_t read_available_buf;
+
+ segment_info *write_traces;
+ segment_info *read_traces;
+};
+
+/* The dxt_runtime structure maintains necessary state for storing
+ * DXT file records and for coordinating with darshan-core at
+ * shutdown time.
+ */
+struct dxt_posix_runtime
+{
+ void *rec_id_hash;
+ int file_rec_count;
+ char *record_buf;
+ int record_buf_size;
+};
+
+struct dxt_mpiio_runtime
+{
+ void *rec_id_hash;
+ int file_rec_count;
+ char *record_buf;
+ int record_buf_size;
+};
+
+/* dxt read/write instrumentation wrappers for POSIX and MPI-IO */
+void dxt_posix_write(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time);
+void dxt_posix_read(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time);
+void dxt_mpiio_write(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time);
+void dxt_mpiio_read(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time);
+
+static void check_wr_trace_buf(
+ struct dxt_file_record_ref *rec_ref);
+static void check_rd_trace_buf(
+ struct dxt_file_record_ref *rec_ref);
+static void dxt_posix_runtime_initialize(
+ void);
+static void dxt_mpiio_runtime_initialize(
+ void);
+static struct dxt_file_record_ref *dxt_posix_track_new_file_record(
+ darshan_record_id rec_id);
+static struct dxt_file_record_ref *dxt_mpiio_track_new_file_record(
+ darshan_record_id rec_id);
+static void dxt_posix_cleanup_runtime(
+ void);
+static void dxt_mpiio_cleanup_runtime(
+ void);
+
+static void dxt_posix_shutdown(
+ MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **dxt_buf, int *dxt_buf_sz);
+static void dxt_mpiio_shutdown(
+ MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **dxt_buf, int *dxt_buf_sz);
+
+static struct dxt_posix_runtime *dxt_posix_runtime = NULL;
+static struct dxt_mpiio_runtime *dxt_mpiio_runtime = NULL;
+static pthread_mutex_t dxt_runtime_mutex =
+ PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
+
+static int dxt_my_rank = -1;
+static int dxt_mem_remaining = DXT_IO_TRACE_MEM_MAX;
+
+#define DXT_LOCK() pthread_mutex_lock(&dxt_runtime_mutex)
+#define DXT_UNLOCK() pthread_mutex_unlock(&dxt_runtime_mutex)
+
+
+/**********************************************************
+ * Wrappers for DXT I/O functions of interest *
+ **********************************************************/
+
+static void check_wr_trace_buf(struct dxt_file_record_ref *rec_ref)
+{
+ struct dxt_file_record *file_rec = rec_ref->file_rec;
+
+ int write_count = file_rec->write_count;
+ int write_available_buf = rec_ref->write_available_buf;
+
+ if (write_count >= write_available_buf) {
+ int write_count_inc;
+ if(write_available_buf == 0)
+ write_count_inc = IO_TRACE_BUF_SIZE;
+ else
+ write_count_inc = write_available_buf;
+
+ DXT_LOCK();
+ if((write_count_inc * sizeof(segment_info)) > dxt_mem_remaining)
+ write_count_inc = dxt_mem_remaining / sizeof(segment_info);
+
+ dxt_mem_remaining -= (write_count_inc * sizeof(segment_info));
+ DXT_UNLOCK();
+
+ if(write_count_inc > 0)
+ {
+ write_available_buf += write_count_inc;
+ rec_ref->write_traces =
+ (segment_info *)realloc(rec_ref->write_traces,
+ write_available_buf * sizeof(segment_info));
+
+ rec_ref->write_available_buf = write_available_buf;
+ }
+ }
+}
+
+static void check_rd_trace_buf(struct dxt_file_record_ref *rec_ref)
+{
+ struct dxt_file_record *file_rec = rec_ref->file_rec;
+
+ int read_count = file_rec->read_count;
+ int read_available_buf = rec_ref->read_available_buf;
+
+ if (read_count >= read_available_buf) {
+ int read_count_inc;
+ if(read_available_buf == 0)
+ read_count_inc = IO_TRACE_BUF_SIZE;
+ else
+ read_count_inc = read_available_buf;
+
+ DXT_LOCK();
+ if((read_count_inc * sizeof(segment_info)) > dxt_mem_remaining)
+ read_count_inc = dxt_mem_remaining / sizeof(segment_info);
+
+ dxt_mem_remaining -= (read_count_inc * sizeof(segment_info));
+ DXT_UNLOCK();
+
+ if(read_count_inc > 0)
+ {
+ read_available_buf += read_count_inc;
+ rec_ref->read_traces =
+ (segment_info *)realloc(rec_ref->read_traces,
+ read_available_buf * sizeof(segment_info));
+
+ rec_ref->read_available_buf = read_available_buf;
+ }
+ }
+}
+
+void dxt_posix_write(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time)
+{
+ struct dxt_file_record_ref* rec_ref = NULL;
+ struct dxt_file_record *file_rec;
+
+ /* make sure dxt posix runtime is initialized properly */
+ if(!dxt_posix_runtime)
+ {
+ dxt_posix_runtime_initialize();
+ if(!dxt_posix_runtime) return;
+ }
+
+ rec_ref = darshan_lookup_record_ref(dxt_posix_runtime->rec_id_hash,
+ &rec_id, sizeof(darshan_record_id));
+ if(!rec_ref)
+ {
+ /* track new dxt file record */
+ rec_ref = dxt_posix_track_new_file_record(rec_id);
+ if(!rec_ref) return;
+ }
+
+ file_rec = rec_ref->file_rec;
+ check_wr_trace_buf(rec_ref);
+ if(file_rec->write_count == rec_ref->write_available_buf)
+ {
+ /* no more memory for i/o segments ... back out */
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_POSIX_MOD);
+ return;
+ }
+
+ rec_ref->write_traces[file_rec->write_count].offset = offset;
+ rec_ref->write_traces[file_rec->write_count].length = length;
+ rec_ref->write_traces[file_rec->write_count].start_time = start_time;
+ rec_ref->write_traces[file_rec->write_count].end_time = end_time;
+ file_rec->write_count += 1;
+}
+
+void dxt_posix_read(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time)
+{
+ struct dxt_file_record_ref* rec_ref = NULL;
+ struct dxt_file_record *file_rec;
+
+ /* make sure dxt posix runtime is initialized properly */
+ if(!dxt_posix_runtime)
+ {
+ dxt_posix_runtime_initialize();
+ if(!dxt_posix_runtime) return;
+ }
+
+ rec_ref = darshan_lookup_record_ref(dxt_posix_runtime->rec_id_hash,
+ &rec_id, sizeof(darshan_record_id));
+ if (!rec_ref) {
+ /* track new dxt file record */
+ rec_ref = dxt_posix_track_new_file_record(rec_id);
+ if(!rec_ref) return;
+ }
+
+ file_rec = rec_ref->file_rec;
+ check_rd_trace_buf(rec_ref);
+ if(file_rec->read_count == rec_ref->read_available_buf)
+ {
+ /* no more memory for i/o segments ... back out */
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_POSIX_MOD);
+ return;
+ }
+
+ rec_ref->read_traces[file_rec->read_count].offset = offset;
+ rec_ref->read_traces[file_rec->read_count].length = length;
+ rec_ref->read_traces[file_rec->read_count].start_time = start_time;
+ rec_ref->read_traces[file_rec->read_count].end_time = end_time;
+ file_rec->read_count += 1;
+}
+
+void dxt_mpiio_write(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time)
+{
+ struct dxt_file_record_ref* rec_ref = NULL;
+ struct dxt_file_record *file_rec;
+
+ /* make sure dxt mpiio runtime is initialized properly */
+ if(!dxt_mpiio_runtime)
+ {
+ dxt_mpiio_runtime_initialize();
+ if(!dxt_mpiio_runtime) return;
+ }
+
+ rec_ref = darshan_lookup_record_ref(dxt_mpiio_runtime->rec_id_hash,
+ &rec_id, sizeof(darshan_record_id));
+ if(!rec_ref)
+ {
+ /* track new dxt file record */
+ rec_ref = dxt_mpiio_track_new_file_record(rec_id);
+ if(!rec_ref) return;
+ }
+
+ file_rec = rec_ref->file_rec;
+ check_wr_trace_buf(rec_ref);
+ if(file_rec->write_count == rec_ref->write_available_buf)
+ {
+ /* no more memory for i/o segments ... back out */
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_MPIIO_MOD);
+ return;
+ }
+
+ rec_ref->write_traces[file_rec->write_count].length = length;
+ rec_ref->write_traces[file_rec->write_count].start_time = start_time;
+ rec_ref->write_traces[file_rec->write_count].end_time = end_time;
+ file_rec->write_count += 1;
+}
+
+void dxt_mpiio_read(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time)
+{
+ struct dxt_file_record_ref* rec_ref = NULL;
+ struct dxt_file_record *file_rec;
+
+ /* make sure dxt mpiio runtime is initialized properly */
+ if(!dxt_mpiio_runtime)
+ {
+ dxt_mpiio_runtime_initialize();
+ if(!dxt_mpiio_runtime) return;
+ }
+
+ rec_ref = darshan_lookup_record_ref(dxt_mpiio_runtime->rec_id_hash,
+ &rec_id, sizeof(darshan_record_id));
+ if(!rec_ref)
+ {
+ /* track new dxt file record */
+ rec_ref = dxt_mpiio_track_new_file_record(rec_id);
+ if(!rec_ref) return;
+ }
+
+ file_rec = rec_ref->file_rec;
+ check_rd_trace_buf(rec_ref);
+ if(file_rec->read_count == rec_ref->read_available_buf)
+ {
+ /* no more memory for i/o segments ... back out */
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_MPIIO_MOD);
+ return;
+ }
+
+ rec_ref->read_traces[file_rec->read_count].length = length;
+ rec_ref->read_traces[file_rec->read_count].start_time = start_time;
+ rec_ref->read_traces[file_rec->read_count].end_time = end_time;
+ file_rec->read_count += 1;
+}
+
+
+/**********************************************************
+ * Internal functions for manipulating DXT module state *
+ **********************************************************/
+
+/* initialize internal DXT module data structures and register with darshan-core */
+static void dxt_posix_runtime_initialize()
+{
+ /* DXT modules request 0 memory -- buffers will be managed internally by DXT
+ * and passed back to darshan-core at shutdown time to allow DXT more control
+ * over realloc'ing module memory as needed.
+ */
+ int dxt_psx_buf_size = 0;
+
+ /* register the DXT module with darshan core */
+ darshan_core_register_module(
+ DXT_POSIX_MOD,
+ &dxt_posix_shutdown,
+ &dxt_psx_buf_size,
+ &dxt_my_rank,
+ NULL);
+
+ /* return if darshan-core allocates an unexpected amount of memory */
+ if(dxt_psx_buf_size != 0)
+ {
+ darshan_core_unregister_module(DXT_POSIX_MOD);
+ return;
+ }
+
+ dxt_posix_runtime = malloc(sizeof(*dxt_posix_runtime));
+ if(!dxt_posix_runtime)
+ {
+ darshan_core_unregister_module(DXT_POSIX_MOD);
+ return;
+ }
+ memset(dxt_posix_runtime, 0, sizeof(*dxt_posix_runtime));
+
+ return;
+}
+
+void dxt_mpiio_runtime_initialize()
+{
+ /* DXT modules request 0 memory -- buffers will be managed internally by DXT
+ * and passed back to darshan-core at shutdown time to allow DXT more control
+ * over realloc'ing module memory as needed.
+ */
+ int dxt_mpiio_buf_size = 0;
+
+ /* register the DXT module with darshan core */
+ darshan_core_register_module(
+ DXT_MPIIO_MOD,
+ &dxt_mpiio_shutdown,
+ &dxt_mpiio_buf_size,
+ &dxt_my_rank,
+ NULL);
+
+ /* return if darshan-core allocates an unexpected amount of memory */
+ if(dxt_mpiio_buf_size != 0)
+ {
+ darshan_core_unregister_module(DXT_MPIIO_MOD);
+ return;
+ }
+
+ dxt_mpiio_runtime = malloc(sizeof(*dxt_mpiio_runtime));
+ if(!dxt_mpiio_runtime)
+ {
+ darshan_core_unregister_module(DXT_MPIIO_MOD);
+ return;
+ }
+ memset(dxt_mpiio_runtime, 0, sizeof(*dxt_mpiio_runtime));
+
+ return;
+}
+
+static struct dxt_file_record_ref *dxt_posix_track_new_file_record(
+ darshan_record_id rec_id)
+{
+ struct dxt_file_record_ref *rec_ref = NULL;
+ struct dxt_file_record *file_rec = NULL;
+ int ret;
+
+ /* check if we have enough room for a new DXT record */
+ DXT_LOCK();
+ if(dxt_mem_remaining < sizeof(struct dxt_file_record))
+ {
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_POSIX_MOD);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+
+ rec_ref = malloc(sizeof(*rec_ref));
+ if(!rec_ref)
+ {
+ DXT_UNLOCK();
+ return(NULL);
+ }
+ memset(rec_ref, 0, sizeof(*rec_ref));
+
+ file_rec = malloc(sizeof(*file_rec));
+ if(!file_rec)
+ {
+ free(rec_ref);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+ memset(file_rec, 0, sizeof(*file_rec));
+
+ /* add a reference to this file record based on record id */
+ ret = darshan_add_record_ref(&(dxt_posix_runtime->rec_id_hash), &rec_id,
+ sizeof(darshan_record_id), rec_ref);
+ if(ret == 0)
+ {
+ free(file_rec);
+ free(rec_ref);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+
+ dxt_mem_remaining -= sizeof(struct dxt_file_record);
+ DXT_UNLOCK();
+
+ /* initialize record and record reference fields */
+ file_rec->base_rec.id = rec_id;
+ file_rec->base_rec.rank = dxt_my_rank;
+ gethostname(file_rec->hostname, HOSTNAME_SIZE);
+
+ rec_ref->file_rec = file_rec;
+ dxt_posix_runtime->file_rec_count++;
+
+ return(rec_ref);
+}
+
+static struct dxt_file_record_ref *dxt_mpiio_track_new_file_record(
+ darshan_record_id rec_id)
+{
+ struct dxt_file_record *file_rec = NULL;
+ struct dxt_file_record_ref *rec_ref = NULL;
+ int ret;
+
+ /* check if we have enough room for a new DXT record */
+ DXT_LOCK();
+ if(dxt_mem_remaining < sizeof(struct dxt_file_record))
+ {
+ SET_DXT_MOD_PARTIAL_FLAG(DXT_MPIIO_MOD);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+
+ rec_ref = malloc(sizeof(*rec_ref));
+ if(!rec_ref)
+ {
+ DXT_UNLOCK();
+ return(NULL);
+ }
+ memset(rec_ref, 0, sizeof(*rec_ref));
+
+ file_rec = malloc(sizeof(*file_rec));
+ if(!file_rec)
+ {
+ free(rec_ref);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+ memset(file_rec, 0, sizeof(*file_rec));
+
+ /* add a reference to this file record based on record id */
+ ret = darshan_add_record_ref(&(dxt_mpiio_runtime->rec_id_hash), &rec_id,
+ sizeof(darshan_record_id), rec_ref);
+ if(ret == 0)
+ {
+ free(file_rec);
+ free(rec_ref);
+ DXT_UNLOCK();
+ return(NULL);
+ }
+
+ dxt_mem_remaining -= sizeof(struct dxt_file_record);
+ DXT_UNLOCK();
+
+ /* initialize record and record reference fields */
+ file_rec->base_rec.id = rec_id;
+ file_rec->base_rec.rank = dxt_my_rank;
+ gethostname(file_rec->hostname, HOSTNAME_SIZE);
+
+ rec_ref->file_rec = file_rec;
+ dxt_mpiio_runtime->file_rec_count++;
+
+ return(rec_ref);
+}
+
+static void dxt_free_record_data(void *rec_ref_p)
+{
+ struct dxt_file_record_ref *dxt_rec_ref = (struct dxt_file_record_ref *)rec_ref_p;
+
+ free(dxt_rec_ref->write_traces);
+ free(dxt_rec_ref->read_traces);
+ free(dxt_rec_ref->file_rec);
+}
+
+static void dxt_posix_cleanup_runtime()
+{
+ darshan_iter_record_refs(dxt_posix_runtime->rec_id_hash, dxt_free_record_data);
+ darshan_clear_record_refs(&(dxt_posix_runtime->rec_id_hash), 1);
+
+ free(dxt_posix_runtime);
+ dxt_posix_runtime = NULL;
+
+ return;
+}
+
+static void dxt_mpiio_cleanup_runtime()
+{
+ darshan_iter_record_refs(dxt_mpiio_runtime->rec_id_hash, dxt_free_record_data);
+ darshan_clear_record_refs(&(dxt_mpiio_runtime->rec_id_hash), 1);
+
+ free(dxt_mpiio_runtime);
+ dxt_mpiio_runtime = NULL;
+
+ return;
+}
+
+
+/********************************************************************************
+ * shutdown function exported by this module for coordinating with darshan-core *
+ ********************************************************************************/
+
+static void dxt_serialize_posix_records(void *rec_ref_p)
+{
+ struct dxt_file_record_ref *rec_ref = (struct dxt_file_record_ref *)rec_ref_p;
+ struct dxt_file_record *file_rec;
+ int64_t record_size = 0;
+ int64_t record_write_count = 0;
+ int64_t record_read_count = 0;
+ void *tmp_buf_ptr;
+
+ assert(rec_ref);
+ file_rec = rec_ref->file_rec;
+ assert(file_rec);
+
+ record_write_count = file_rec->write_count;
+ record_read_count = file_rec->read_count;
+ if (record_write_count == 0 && record_read_count == 0)
+ return;
+
+ /*
+ * Buffer format:
+ * dxt_file_record + write_traces + read_traces
+ */
+ record_size = sizeof(struct dxt_file_record) +
+ (record_write_count + record_read_count) * sizeof(segment_info);
+
+ tmp_buf_ptr = (void *)(dxt_posix_runtime->record_buf +
+ dxt_posix_runtime->record_buf_size);
+
+ /*Copy struct dxt_file_record */
+ memcpy(tmp_buf_ptr, (void *)file_rec, sizeof(struct dxt_file_record));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr + sizeof(struct dxt_file_record));
+
+ /*Copy write record */
+ memcpy(tmp_buf_ptr, (void *)(rec_ref->write_traces),
+ record_write_count * sizeof(segment_info));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr +
+ record_write_count * sizeof(segment_info));
+
+ /*Copy read record */
+ memcpy(tmp_buf_ptr, (void *)(rec_ref->read_traces),
+ record_read_count * sizeof(segment_info));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr +
+ record_read_count * sizeof(segment_info));
+
+ dxt_posix_runtime->record_buf_size += record_size;
+
+#if 0
+ int i;
+ int64_t rank;
+ char *hostname;
+ int64_t offset;
+ int64_t length;
+ double start_time;
+ double end_time;
+
+ rank = file_rec->base_rec.rank;
+ hostname = file_rec->hostname;
+
+ printf("X_POSIX, record_id: %" PRIu64 "\n", rec_ref->file_rec->base_rec.id);
+ printf("X_POSIX, write_count is: %d read_count is: %d\n",
+ file_rec->write_count, file_rec->read_count);
+ printf("X_POSIX, rank: %d hostname: %s\n", rank, hostname);
+
+ for (i = 0; i < file_rec->write_count; i++) {
+ offset = rec_ref->write_traces[i].offset;
+ length = rec_ref->write_traces[i].length;
+ start_time = rec_ref->write_traces[i].start_time;
+ end_time = rec_ref->write_traces[i].end_time;
+
+ printf("X_POSIX, rank %d writes segment %lld [offset: %lld length: %lld start_time: %fs end_time: %fs]\n", rank, i, offset, length, start_time, end_time);
+ }
+
+ for (i = 0; i < file_rec->read_count; i++) {
+ offset = rec_ref->read_traces[i].offset;
+ length = rec_ref->read_traces[i].length;
+ start_time = rec_ref->read_traces[i].start_time;
+ end_time = rec_ref->read_traces[i].end_time;
+
+ printf("X_POSIX, rank %d reads segment %lld [offset: %lld length: %lld start_time: %fs end_time: %fs]\n", rank, i, offset, length, start_time, end_time);
+ }
+#endif
+}
+
+static void dxt_posix_shutdown(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **dxt_posix_buf,
+ int *dxt_posix_buf_sz)
+{
+ assert(dxt_posix_runtime);
+
+ *dxt_posix_buf_sz = 0;
+
+ dxt_posix_runtime->record_buf = malloc(DXT_IO_TRACE_MEM_MAX);
+ if(!(dxt_posix_runtime->record_buf))
+ return;
+ memset(dxt_posix_runtime->record_buf, 0, DXT_IO_TRACE_MEM_MAX);
+ dxt_posix_runtime->record_buf_size = 0;
+
+ /* iterate all dxt posix records and serialize them to the output buffer */
+ darshan_iter_record_refs(dxt_posix_runtime->rec_id_hash, dxt_serialize_posix_records);
+
+ /* set output */
+ *dxt_posix_buf = dxt_posix_runtime->record_buf;
+ *dxt_posix_buf_sz = dxt_posix_runtime->record_buf_size;
+
+ /* shutdown internal structures used for instrumenting */
+ dxt_posix_cleanup_runtime();
+
+ return;
+}
+
+static void dxt_serialize_mpiio_records(void *rec_ref_p)
+{
+ struct dxt_file_record_ref *rec_ref = (struct dxt_file_record_ref *)rec_ref_p;
+ struct dxt_file_record *file_rec;
+ int64_t record_size = 0;
+ int64_t record_write_count = 0;
+ int64_t record_read_count = 0;
+ void *tmp_buf_ptr;
+
+ assert(rec_ref);
+ file_rec = rec_ref->file_rec;
+ assert(file_rec);
+
+ record_write_count = file_rec->write_count;
+ record_read_count = file_rec->read_count;
+ if (record_write_count == 0 && record_read_count == 0)
+ return;
+
+ /*
+ * Buffer format:
+ * dxt_file_record + write_traces + read_traces
+ */
+ record_size = sizeof(struct dxt_file_record) +
+ (record_write_count + record_read_count) * sizeof(segment_info);
+
+ tmp_buf_ptr = (void *)(dxt_mpiio_runtime->record_buf +
+ dxt_mpiio_runtime->record_buf_size);
+
+ /*Copy struct dxt_file_record */
+ memcpy(tmp_buf_ptr, (void *)file_rec, sizeof(struct dxt_file_record));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr + sizeof(struct dxt_file_record));
+
+ /*Copy write record */
+ memcpy(tmp_buf_ptr, (void *)(rec_ref->write_traces),
+ record_write_count * sizeof(segment_info));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr +
+ record_write_count * sizeof(segment_info));
+
+ /*Copy read record */
+ memcpy(tmp_buf_ptr, (void *)(rec_ref->read_traces),
+ record_read_count * sizeof(segment_info));
+ tmp_buf_ptr = (void *)(tmp_buf_ptr +
+ record_read_count * sizeof(segment_info));
+
+ dxt_mpiio_runtime->record_buf_size += record_size;
+
+#if 0
+ int i;
+ int64_t rank;
+ char *hostname;
+ int64_t length;
+ double start_time;
+ double end_time;
+
+ rank = file_rec->base_rec.rank;
+ hostname = file_rec->hostname;
+
+ printf("X_MPIIO, record_id: %" PRIu64 "\n", rec_ref->file_rec->base_rec.id);
+ printf("X_MPIIO, write_count is: %d read_count is: %d\n",
+ file_rec->write_count, file_rec->read_count);
+ printf("X_MPIIO, rank: %d hostname: %s\n", rank, hostname);
+
+ for (i = 0; i < file_rec->write_count; i++) {
+ length = rec_ref->write_traces[i].length;
+ start_time = rec_ref->write_traces[i].start_time;
+ end_time = rec_ref->write_traces[i].end_time;
+
+ printf("X_MPIIO, rank %d writes segment %lld [length: %lld start_time: %fs end_time: %fs]\n", rank, i, length, start_time, end_time);
+ }
+
+ for (i = 0; i < file_rec->read_count; i++) {
+ length = rec_ref->read_traces[i].length;
+ start_time = rec_ref->read_traces[i].start_time;
+ end_time = rec_ref->read_traces[i].end_time;
+
+ printf("X_MPIIO, rank %d reads segment %lld [length: %lld start_time: %fs end_time: %fs]\n", rank, i, length, start_time, end_time);
+ }
+#endif
+}
+
+static void dxt_mpiio_shutdown(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **dxt_mpiio_buf,
+ int *dxt_mpiio_buf_sz)
+{
+ assert(dxt_mpiio_runtime);
+
+ *dxt_mpiio_buf_sz = 0;
+
+ dxt_mpiio_runtime->record_buf = malloc(DXT_IO_TRACE_MEM_MAX);
+ if(!(dxt_mpiio_runtime->record_buf))
+ return;
+ memset(dxt_mpiio_runtime->record_buf, 0, DXT_IO_TRACE_MEM_MAX);
+ dxt_mpiio_runtime->record_buf_size = 0;
+
+ /* iterate all dxt posix records and serialize them to the output buffer */
+ darshan_iter_record_refs(dxt_mpiio_runtime->rec_id_hash, dxt_serialize_mpiio_records);
+
+ /* set output */
+ *dxt_mpiio_buf = dxt_mpiio_runtime->record_buf;
+ *dxt_mpiio_buf_sz = dxt_mpiio_runtime->record_buf_size;
+
+ /* shutdown internal structures used for instrumenting */
+ dxt_mpiio_cleanup_runtime();
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
=====================================
darshan-runtime/lib/darshan-mpiio.c
=====================================
--- a/darshan-runtime/lib/darshan-mpiio.c
+++ b/darshan-runtime/lib/darshan-mpiio.c
@@ -87,10 +87,17 @@ static void mpiio_shutdown(
MPI_Comm mod_comm, darshan_record_id *shared_recs,
int shared_rec_count, void **mpiio_buf, int *mpiio_buf_sz);
+/* extern DXT function defs */
+extern void dxt_mpiio_write(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time);
+extern void dxt_mpiio_read(darshan_record_id rec_id, int64_t length,
+ double start_time, double end_time);
+
static struct mpiio_runtime *mpiio_runtime = NULL;
static pthread_mutex_t mpiio_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
static int instrumentation_disabled = 0;
static int my_rank = -1;
+static int enable_dxt_io_trace = 0;
#define MPIIO_LOCK() pthread_mutex_lock(&mpiio_runtime_mutex)
#define MPIIO_UNLOCK() pthread_mutex_unlock(&mpiio_runtime_mutex)
@@ -98,7 +105,9 @@ static int my_rank = -1;
#define MPIIO_PRE_RECORD() do { \
MPIIO_LOCK(); \
if(!instrumentation_disabled) { \
- if(!mpiio_runtime) mpiio_runtime_initialize(); \
+ if(!mpiio_runtime) { \
+ mpiio_runtime_initialize(); \
+ } \
if(mpiio_runtime) break; \
} \
MPIIO_UNLOCK(); \
@@ -154,6 +163,10 @@ static int my_rank = -1;
if(!rec_ref) break; \
DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size); \
size = size * __count; \
+ /* DXT to record detailed read tracing information */ \
+ if(enable_dxt_io_trace) { \
+ dxt_mpiio_read(rec_ref->file_rec->base_rec.id, size, __tm1, __tm2); \
+ } \
DARSHAN_BUCKET_INC(&(rec_ref->file_rec->counters[MPIIO_SIZE_READ_AGG_0_100]), size); \
darshan_common_val_counter(&rec_ref->access_root, &rec_ref->access_count, size, \
&(rec_ref->file_rec->counters[MPIIO_ACCESS1_ACCESS]), \
@@ -183,6 +196,10 @@ static int my_rank = -1;
if(!rec_ref) break; \
DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size); \
size = size * __count; \
+ /* DXT to record detailed write tracing information */ \
+ if(enable_dxt_io_trace) { \
+ dxt_mpiio_write(rec_ref->file_rec->base_rec.id, size, __tm1, __tm2); \
+ } \
DARSHAN_BUCKET_INC(&(rec_ref->file_rec->counters[MPIIO_SIZE_WRITE_AGG_0_100]), size); \
darshan_common_val_counter(&rec_ref->access_root, &rec_ref->access_count, size, \
&(rec_ref->file_rec->counters[MPIIO_ACCESS1_ACCESS]), \
@@ -844,6 +861,11 @@ static void mpiio_runtime_initialize()
}
memset(mpiio_runtime, 0, sizeof(*mpiio_runtime));
+ /* check if DXT (Darshan extended tracing) should be enabled */
+ if (getenv("ENABLE_DXT_IO_TRACE")) {
+ enable_dxt_io_trace = 1;
+ }
+
return;
}
=====================================
darshan-runtime/lib/darshan-posix.c
=====================================
--- a/darshan-runtime/lib/darshan-posix.c
+++ b/darshan-runtime/lib/darshan-posix.c
@@ -157,11 +157,18 @@ static void posix_shutdown(
MPI_Comm mod_comm, darshan_record_id *shared_recs,
int shared_rec_count, void **posix_buf, int *posix_buf_sz);
+/* extern DXT function defs */
+extern void dxt_posix_write(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time);
+extern void dxt_posix_read(darshan_record_id rec_id, int64_t offset,
+ int64_t length, double start_time, double end_time);
+
static struct posix_runtime *posix_runtime = NULL;
static pthread_mutex_t posix_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
static int instrumentation_disabled = 0;
static int my_rank = -1;
static int darshan_mem_alignment = 1;
+static int enable_dxt_io_trace = 0;
#define POSIX_LOCK() pthread_mutex_lock(&posix_runtime_mutex)
#define POSIX_UNLOCK() pthread_mutex_unlock(&posix_runtime_mutex)
@@ -169,7 +176,9 @@ static int darshan_mem_alignment = 1;
#define POSIX_PRE_RECORD() do { \
POSIX_LOCK(); \
if(!instrumentation_disabled) { \
- if(!posix_runtime) posix_runtime_initialize(); \
+ if(!posix_runtime) { \
+ posix_runtime_initialize(); \
+ } \
if(posix_runtime) break; \
} \
POSIX_UNLOCK(); \
@@ -228,6 +237,10 @@ static int darshan_mem_alignment = 1;
this_offset = __pread_offset; \
else \
this_offset = rec_ref->offset; \
+ /* DXT to record detailed read tracing information */ \
+ if(enable_dxt_io_trace) { \
+ dxt_posix_read(rec_ref->file_rec->base_rec.id, this_offset, __ret, __tm1, __tm2); \
+ } \
if(this_offset > rec_ref->last_byte_read) \
rec_ref->file_rec->counters[POSIX_SEQ_READS] += 1; \
if(this_offset == (rec_ref->last_byte_read + 1)) \
@@ -282,6 +295,10 @@ static int darshan_mem_alignment = 1;
this_offset = __pwrite_offset; \
else \
this_offset = rec_ref->offset; \
+ /* DXT to record detailed write tracing information */ \
+ if(enable_dxt_io_trace) { \
+ dxt_posix_write(rec_ref->file_rec->base_rec.id, this_offset, __ret, __tm1, __tm2); \
+ } \
if(this_offset > rec_ref->last_byte_written) \
rec_ref->file_rec->counters[POSIX_SEQ_WRITES] += 1; \
if(this_offset == (rec_ref->last_byte_written + 1)) \
@@ -1257,6 +1274,11 @@ static void posix_runtime_initialize()
}
memset(posix_runtime, 0, sizeof(*posix_runtime));
+ /* check if DXT (Darshan extended tracing) should be enabled */
+ if (getenv("ENABLE_DXT_IO_TRACE")) {
+ enable_dxt_io_trace = 1;
+ }
+
return;
}
=====================================
darshan-util/Makefile.in
=====================================
--- a/darshan-util/Makefile.in
+++ b/darshan-util/Makefile.in
@@ -1,4 +1,4 @@
-all: libdarshan-util.a darshan-null-logutils.o darshan-analyzer darshan-convert darshan-diff darshan-parser darshan-merge jenkins-hash-gen
+all: libdarshan-util.a darshan-null-logutils.o darshan-analyzer darshan-convert darshan-diff darshan-parser darshan-dxt-parser darshan-merge jenkins-hash-gen
DESTDIR =
srcdir = @srcdir@
@@ -13,10 +13,10 @@ libdir = $(DESTDIR)@libdir@
pkgconfigdir = $(DESTDIR)$(libdir)/pkgconfig
DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h
-DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h $(srcdir)/../darshan-bgq-log-format.h $(srcdir)/../darshan-lustre-log-format.h $(srcdir)/../darshan-stdio-log-format.h
-DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h darshan-bgq-logutils.h darshan-lustre-logutils.h darshan-stdio-logutils.h
-DARSHAN_STATIC_MOD_OBJS = darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o darshan-lustre-logutils.o darshan-stdio-logutils.o
-DARSHAN_DYNAMIC_MOD_OBJS = darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po darshan-lustre-logutils.po darshan-stdio-logutils.po
+DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h $(srcdir)/../darshan-bgq-log-format.h $(srcdir)/../darshan-lustre-log-format.h $(srcdir)/../darshan-stdio-log-format.h $(srcdir)/../darshan-dxt-log-format.h
+DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h darshan-bgq-logutils.h darshan-lustre-logutils.h darshan-stdio-logutils.h darshan-dxt-logutils.h
+DARSHAN_STATIC_MOD_OBJS = darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o darshan-lustre-logutils.o darshan-stdio-logutils.o darshan-dxt-logutils.o
+DARSHAN_DYNAMIC_MOD_OBJS = darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po darshan-lustre-logutils.po darshan-stdio-logutils.po darshan-dxt-logutils.po
DARSHAN_ENABLE_SHARED=@DARSHAN_ENABLE_SHARED@
@@ -91,6 +91,11 @@ darshan-stdio-logutils.o: darshan-stdio-logutils.c darshan-logutils.h darshan-st
darshan-stdio-logutils.po: darshan-stdio-logutils.c darshan-logutils.h darshan-stdio-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-stdio-log-format.h | uthash-1.9.2
$(CC) $(CFLAGS_SHARED) -c $< -o $@
+darshan-dxt-logutils.o: darshan-dxt-logutils.c darshan-logutils.h darshan-dxt-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-dxt-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS) -c $< -o $@
+darshan-dxt-logutils.po: darshan-dxt-logutils.c darshan-logutils.h darshan-dxt-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-dxt-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
+
libdarshan-util.a: darshan-logutils.o $(DARSHAN_STATIC_MOD_OBJS)
ar rcs libdarshan-util.a $^
@@ -115,6 +120,9 @@ darshan-diff: darshan-diff.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_
darshan-parser: darshan-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
$(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
+darshan-dxt-parser: darshan-dxt-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
+
darshan-merge: darshan-merge.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
$(CC) $(CFLAGS) $(LDFLAGS) $< libdarshan-util.a -o $@ $(LIBS)
@@ -133,6 +141,7 @@ install:: all
install -m 755 darshan-convert $(bindir)
install -m 755 darshan-diff $(bindir)
install -m 755 darshan-parser $(bindir)
+ install -m 755 darshan-dxt-parser $(bindir)
install -m 755 darshan-merge $(bindir)
install -m 755 $(srcdir)/darshan-summary-per-file.sh $(bindir)
install -m 755 libdarshan-util.a $(libdir)
@@ -148,6 +157,7 @@ endif
install -m 644 $(srcdir)/darshan-null-logutils.h $(includedir)
install -m 644 $(srcdir)/darshan-lustre-logutils.h $(includedir)
install -m 644 $(srcdir)/darshan-stdio-logutils.h $(includedir)
+ install -m 644 $(srcdir)/darshan-dxt-logutils.h $(includedir)
install -m 644 $(srcdir)/../darshan-null-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-posix-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-mpiio-log-format.h $(includedir)
@@ -156,6 +166,7 @@ endif
install -m 644 $(srcdir)/../darshan-bgq-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-lustre-log-format.h $(includedir)
install -m 644 $(srcdir)/../darshan-stdio-log-format.h $(includedir)
+ install -m 644 $(srcdir)/../darshan-dxt-log-format.h $(includedir)
install -d $(includedir)/uthash-1.9.2
install -d $(includedir)/uthash-1.9.2/src
install -m 644 uthash-1.9.2/src/uthash.h $(includedir)/uthash-1.9.2/src/
@@ -172,7 +183,7 @@ endif
clean::
- rm -f *.o *.po *.a *.so darshan-analyzer darshan-convert darshan-diff darshan-parser darshan-merge jenkins-hash-gen
+ rm -f *.o *.po *.a *.so darshan-analyzer darshan-convert darshan-diff darshan-parser darshan-dxt-parser darshan-merge jenkins-hash-gen
distclean:: clean
rm -f darshan-util-config.h aclocal.m4 autom4te.cache/* config.status config.log Makefile darshan-job-summary/bin/darshan-job-summary.pl maint/darshan-util.pc
=====================================
darshan-util/darshan-dxt-logutils.c
=====================================
--- /dev/null
+++ b/darshan-util/darshan-dxt-logutils.c
@@ -0,0 +1,448 @@
+/*
+ * Copyright (C) 2016 Intel Corporation.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#define _GNU_SOURCE
+#include "darshan-util-config.h"
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <errno.h>
+
+#include "darshan-logutils.h"
+
+static int dxt_log_get_posix_file(darshan_fd fd, void** dxt_posix_buf_p);
+static int dxt_log_put_posix_file(darshan_fd fd, void* dxt_posix_buf);
+
+static int dxt_log_get_mpiio_file(darshan_fd fd, void** dxt_mpiio_buf_p);
+static int dxt_log_put_mpiio_file(darshan_fd fd, void* dxt_mpiio_buf);
+
+static void dxt_swap_file_record(struct dxt_file_record *file_rec);
+static void dxt_swap_file_record(struct dxt_file_record *file_rec);
+
+struct darshan_mod_logutil_funcs dxt_posix_logutils =
+{
+ .log_get_record = &dxt_log_get_posix_file,
+ .log_put_record = &dxt_log_put_posix_file,
+ .log_print_record = NULL,
+ .log_print_description = NULL,
+ .log_print_diff = NULL,
+ .log_agg_records = NULL,
+};
+
+struct darshan_mod_logutil_funcs dxt_mpiio_logutils =
+{
+ .log_get_record = &dxt_log_get_mpiio_file,
+ .log_put_record = &dxt_log_put_mpiio_file,
+ .log_print_record = NULL,
+ .log_print_description = NULL,
+ .log_print_diff = NULL,
+ .log_agg_records = NULL,
+};
+
+static void dxt_swap_file_record(struct dxt_file_record *file_rec)
+{
+ DARSHAN_BSWAP64(&file_rec->base_rec.id);
+ DARSHAN_BSWAP64(&file_rec->base_rec.rank);
+ DARSHAN_BSWAP64(&file_rec->shared_record);
+ DARSHAN_BSWAP64(&file_rec->write_count);
+ DARSHAN_BSWAP64(&file_rec->read_count);
+}
+
+static void dxt_swap_segments(struct dxt_file_record *file_rec)
+{
+ int i;
+ segment_info *tmp_seg;
+
+ tmp_seg = (segment_info *)((void *)file_rec + sizeof(struct dxt_file_record));
+ for(i = 0; i < (file_rec->write_count + file_rec->read_count); i++)
+ {
+ DARSHAN_BSWAP64(&tmp_seg->offset);
+ DARSHAN_BSWAP64(&tmp_seg->length);
+ DARSHAN_BSWAP64(&tmp_seg->start_time);
+ DARSHAN_BSWAP64(&tmp_seg->end_time);
+ tmp_seg++;
+ }
+}
+
+static int dxt_log_get_posix_file(darshan_fd fd, void** dxt_posix_buf_p)
+{
+ struct dxt_file_record *rec = *((struct dxt_file_record **)dxt_posix_buf_p);
+ struct dxt_file_record tmp_rec;
+ int ret;
+ int64_t io_trace_size;
+
+ if(fd->mod_map[DXT_POSIX_MOD].len == 0)
+ return(0);
+
+ ret = darshan_log_get_mod(fd, DXT_POSIX_MOD, &tmp_rec,
+ sizeof(struct dxt_file_record));
+ if(ret < 0)
+ return (-1);
+ else if(ret < sizeof(struct dxt_file_record))
+ return (0);
+
+ if (fd->swap_flag)
+ {
+ /* swap bytes if necessary */
+ dxt_swap_file_record(&tmp_rec);
+ }
+
+ io_trace_size = (tmp_rec.write_count + tmp_rec.read_count) *
+ sizeof(segment_info);
+
+ if (*dxt_posix_buf_p == NULL)
+ {
+ rec = malloc(sizeof(struct dxt_file_record) + io_trace_size);
+ if (!rec)
+ return(-1);
+ }
+ memcpy(rec, &tmp_rec, sizeof(struct dxt_file_record));
+
+ if (io_trace_size > 0)
+ {
+ void *tmp_p = (void *)rec + sizeof(struct dxt_file_record);
+
+ ret = darshan_log_get_mod(fd, DXT_POSIX_MOD, tmp_p,
+ io_trace_size);
+ if (ret < io_trace_size)
+ ret = -1;
+ else
+ {
+ ret = 1;
+ if(fd->swap_flag)
+ {
+ /* byte swap trace data if necessary */
+ dxt_swap_segments(rec);
+ }
+ }
+ }
+ else
+ {
+ ret = 1;
+ }
+
+ if(*dxt_posix_buf_p == NULL)
+ {
+ if(ret == 1)
+ *dxt_posix_buf_p = rec;
+ else
+ free(rec);
+ }
+
+ return(ret);
+}
+
+static int dxt_log_get_mpiio_file(darshan_fd fd, void** dxt_mpiio_buf_p)
+{
+ struct dxt_file_record *rec = *((struct dxt_file_record **)dxt_mpiio_buf_p);
+ struct dxt_file_record tmp_rec;
+ int ret;
+ int64_t io_trace_size;
+
+ if(fd->mod_map[DXT_MPIIO_MOD].len == 0)
+ return(0);
+
+ ret = darshan_log_get_mod(fd, DXT_MPIIO_MOD, &tmp_rec,
+ sizeof(struct dxt_file_record));
+ if(ret < 0)
+ return (-1);
+ else if(ret < sizeof(struct dxt_file_record))
+ return (0);
+
+ if (fd->swap_flag)
+ {
+ /* swap bytes if necessary */
+ dxt_swap_file_record(&tmp_rec);
+ }
+
+ io_trace_size = (tmp_rec.write_count + tmp_rec.read_count) *
+ sizeof(segment_info);
+
+ if (*dxt_mpiio_buf_p == NULL)
+ {
+ rec = malloc(sizeof(struct dxt_file_record) + io_trace_size);
+ if (!rec)
+ return(-1);
+ }
+ memcpy(rec, &tmp_rec, sizeof(struct dxt_file_record));
+
+ if (io_trace_size > 0)
+ {
+ void *tmp_p = (void *)rec + sizeof(struct dxt_file_record);
+
+ ret = darshan_log_get_mod(fd, DXT_MPIIO_MOD, tmp_p,
+ io_trace_size);
+ if (ret < io_trace_size)
+ ret = -1;
+ else
+ {
+ ret = 1;
+ if(fd->swap_flag)
+ {
+ /* byte swap trace data if necessary */
+ dxt_swap_segments(rec);
+ }
+ }
+ }
+ else
+ {
+ ret = 1;
+ }
+
+ if(*dxt_mpiio_buf_p == NULL)
+ {
+ if(ret == 1)
+ *dxt_mpiio_buf_p = rec;
+ else
+ free(rec);
+ }
+
+ return(ret);
+}
+
+static int dxt_log_put_posix_file(darshan_fd fd, void* dxt_posix_buf)
+{
+ struct dxt_file_record *file_rec =
+ (struct dxt_file_record *)dxt_posix_buf;
+ int rec_size = sizeof(struct dxt_file_record) + (sizeof(segment_info) *
+ (file_rec->write_count + file_rec->read_count));
+ int ret;
+
+ ret = darshan_log_put_mod(fd, DXT_POSIX_MOD, file_rec,
+ rec_size, DXT_POSIX_VER);
+ if(ret < 0)
+ return(-1);
+
+ return(0);
+}
+
+static int dxt_log_put_mpiio_file(darshan_fd fd, void* dxt_mpiio_buf)
+{
+ struct dxt_file_record *file_rec =
+ (struct dxt_file_record *)dxt_mpiio_buf;
+ int rec_size = sizeof(struct dxt_file_record) + (sizeof(segment_info) *
+ (file_rec->write_count + file_rec->read_count));
+ int ret;
+
+ ret = darshan_log_put_mod(fd, DXT_MPIIO_MOD, file_rec,
+ rec_size, DXT_MPIIO_VER);
+ if(ret < 0)
+ return(-1);
+
+ return(0);
+}
+
+void dxt_log_print_posix_file(void *posix_file_rec, char *file_name,
+ char *mnt_pt, char *fs_type, struct lustre_record_ref *lustre_rec_ref)
+{
+ struct dxt_file_record *file_rec =
+ (struct dxt_file_record *)posix_file_rec;
+ int64_t offset;
+ int64_t length;
+ double start_time;
+ double end_time;
+ int i;
+
+ darshan_record_id f_id = file_rec->base_rec.id;
+ int64_t rank = file_rec->base_rec.rank;
+ char *hostname = file_rec->hostname;
+
+ int64_t write_count = file_rec->write_count;
+ int64_t read_count = file_rec->read_count;
+ segment_info *io_trace = (segment_info *)
+ ((void *)file_rec + sizeof(struct dxt_file_record));
+
+ /* Lustre File System */
+ struct darshan_lustre_record *rec;
+ int lustreFS = !strcmp(fs_type, "lustre");
+ int32_t stripe_size;
+ int32_t stripe_count;
+ int64_t cur_offset;
+ int print_count;
+ int ost_idx;
+
+ if (!lustre_rec_ref) {
+ lustreFS = 0;
+ }
+
+ printf("\n# DXT, file_id: %" PRIu64 ", file_name: %s\n", f_id, file_name);
+ printf("# DXT, rank: %d, hostname: %s\n", rank, hostname);
+ printf("# DXT, write_count: %d, read_count: %d\n",
+ write_count, read_count);
+
+ printf("# DXT, mnt_pt: %s, fs_type: %s\n", mnt_pt, fs_type);
+ if (lustreFS) {
+ rec = lustre_rec_ref->rec;
+ stripe_size = rec->counters[LUSTRE_STRIPE_SIZE];
+ stripe_count = rec->counters[LUSTRE_STRIPE_WIDTH];
+
+ printf("# DXT, Lustre stripe_size: %d, Lustre stripe_count: %d\n", stripe_size, stripe_count);
+ for (i = 0; i < stripe_count; i++) {
+ printf("# DXT, Lustre OSTs: %d\n", (rec->ost_ids)[i]);
+ }
+ }
+
+ /* Print header */
+ printf("# Module Rank Wt/Rd Segment Offset Length Start(s) End(s)");
+
+ if (lustreFS) {
+ printf(" [OST]");
+ }
+ printf("\n");
+
+ /* Print IO Traces information */
+ for (i = 0; i < write_count; i++) {
+ offset = io_trace[i].offset;
+ length = io_trace[i].length;
+ start_time = io_trace[i].start_time;
+ end_time = io_trace[i].end_time;
+
+ printf("%8s%8d%7s%9lld%16lld%13lld%12.4f%12.4f", "X_POSIX", rank, "write", i, offset, length, start_time, end_time);
+
+ if (lustreFS) {
+ cur_offset = offset;
+ ost_idx = (offset / stripe_size) % stripe_count;
+
+ print_count = 0;
+ while (cur_offset < offset + length) {
+ printf(" [%3d]", (rec->ost_ids)[ost_idx]);
+
+ cur_offset = (cur_offset / stripe_size + 1) * stripe_size;
+ ost_idx = (ost_idx == stripe_count - 1) ? 0 : ost_idx + 1;
+
+ print_count++;
+ if (print_count >= stripe_count)
+ break;
+ }
+ }
+
+ printf("\n");
+ }
+
+ for (i = write_count; i < write_count + read_count; i++) {
+ offset = io_trace[i].offset;
+ length = io_trace[i].length;
+ start_time = io_trace[i].start_time;
+ end_time = io_trace[i].end_time;
+
+ printf("%8s%8d%7s%9lld%16lld%13lld%12.4f%12.4f", "X_POSIX", rank, "read", i - write_count, offset, length, start_time, end_time);
+
+ if (lustreFS) {
+ cur_offset = offset;
+ ost_idx = (offset / stripe_size) % stripe_count;
+
+ print_count = 0;
+ while (cur_offset < offset + length) {
+ printf(" [%3d]", (rec->ost_ids)[ost_idx]);
+
+ cur_offset = (cur_offset / stripe_size + 1) * stripe_size;
+ ost_idx = (ost_idx == stripe_count - 1) ? 0 : ost_idx + 1;
+
+ print_count++;
+ if (print_count >= stripe_count)
+ break;
+ }
+ }
+
+ printf("\n");
+ }
+ return;
+}
+
+void dxt_log_print_mpiio_file(void *mpiio_file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ struct dxt_file_record *file_rec =
+ (struct dxt_file_record *)mpiio_file_rec;
+
+ int64_t offset;
+ int64_t length;
+ double start_time;
+ double end_time;
+ int i;
+
+ darshan_record_id f_id = file_rec->base_rec.id;
+ int64_t rank = file_rec->base_rec.rank;
+ char *hostname = file_rec->hostname;
+
+ int64_t write_count = file_rec->write_count;
+ int64_t read_count = file_rec->read_count;
+
+ segment_info *io_trace = (segment_info *)
+ ((void *)file_rec + sizeof(struct dxt_file_record));
+
+ printf("\n# DXT, file_id: %" PRIu64 ", file_name: %s\n", f_id, file_name);
+ printf("# DXT, rank: %d, hostname: %s\n", rank, hostname);
+ printf("# DXT, write_count: %d, read_count: %d\n",
+ write_count, read_count);
+
+ printf("# DXT, mnt_pt: %s, fs_type: %s\n", mnt_pt, fs_type);
+
+ /* Print header */
+ printf("# Module Rank Wt/Rd Segment Length Start(s) End(s)\n");
+
+ /* Print IO Traces information */
+ for (i = 0; i < write_count; i++) {
+ offset = io_trace[i].offset;
+ length = io_trace[i].length;
+ start_time = io_trace[i].start_time;
+ end_time = io_trace[i].end_time;
+
+ printf("%8s%8d%7s%9lld%13lld%12.4f%12.4f\n", "X_MPIIO", rank, "write", i, length, start_time, end_time);
+ }
+
+ for (i = write_count; i < write_count + read_count; i++) {
+ offset = io_trace[i].offset;
+ length = io_trace[i].length;
+ start_time = io_trace[i].start_time;
+ end_time = io_trace[i].end_time;
+
+ printf("%8s%8d%7s%9lld%13lld%12.4f%12.4f\n", "X_MPIIO", rank, "read", i - write_count, length, start_time, end_time);
+ }
+
+ return;
+}
+
+static void dxt_log_print_posix_description(int ver)
+{
+}
+
+static void dxt_log_print_mpiio_description(int ver)
+{
+}
+
+static void dxt_log_print_posix_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+}
+
+static void dxt_log_print_mpiio_file_diff(void *file_rec1, char *file_name1,
+ void *file_rec2, char *file_name2)
+{
+}
+
+static void dxt_log_agg_posix_files(void *rec, void *agg_rec, int init_flag)
+{
+}
+
+static void dxt_log_agg_mpiio_files(void *rec, void *agg_rec, int init_flag)
+{
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
=====================================
darshan-util/darshan-dxt-logutils.h
=====================================
--- /dev/null
+++ b/darshan-util/darshan-dxt-logutils.h
@@ -0,0 +1,19 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * Copyright (C) 2016 Intel Corporation.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_DXT_LOG_UTILS_H
+#define __DARSHAN_DXT_LOG_UTILS_H
+
+extern struct darshan_mod_logutil_funcs dxt_posix_logutils;
+extern struct darshan_mod_logutil_funcs dxt_mpiio_logutils;
+
+void dxt_log_print_posix_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type, struct lustre_record_ref *rec_ref);
+void dxt_log_print_mpiio_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+#endif
=====================================
darshan-util/darshan-dxt-parser.c
=====================================
--- /dev/null
+++ b/darshan-util/darshan-dxt-parser.c
@@ -0,0 +1,321 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <zlib.h>
+#include <time.h>
+#include <stdlib.h>
+#include <getopt.h>
+#include <assert.h>
+
+#include "uthash-1.9.2/src/uthash.h"
+
+#include "darshan-logutils.h"
+
+int usage (char *exename)
+{
+ fprintf(stderr, "Usage: %s <filename>\n", exename);
+
+ exit(1);
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+ int i, j;
+ char *filename;
+ char *comp_str;
+ char tmp_string[4096] = {0};
+ darshan_fd fd;
+ struct darshan_job job;
+ struct darshan_name_record_ref *name_hash = NULL;
+ struct darshan_name_record_ref *ref, *tmp_ref;
+ int mount_count;
+ struct darshan_mnt_info *mnt_data_array;
+ time_t tmp_time = 0;
+ int64_t run_time = 0;
+ char *token;
+ char *save;
+ char buffer[DARSHAN_JOB_METADATA_LEN];
+ struct lustre_record_ref *lustre_rec_ref, *tmp_lustre_rec_ref;
+ struct lustre_record_ref *lustre_rec_hash = NULL;
+ char *mod_buf = NULL;
+
+ if (argc != 2)
+ usage(argv[0]);
+
+ filename = argv[1];
+
+ fd = darshan_log_open(filename);
+ if (!fd)
+ return(-1);
+
+ /* read darshan job info */
+ ret = darshan_log_get_job(fd, &job);
+ if (ret < 0)
+ {
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* get the original command line for this job */
+ ret = darshan_log_get_exe(fd, tmp_string);
+ if (ret < 0)
+ {
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* get the mount information for this log */
+ ret = darshan_log_get_mounts(fd, &mnt_data_array, &mount_count);
+ if (ret < 0)
+ {
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* read hash of darshan records */
+ ret = darshan_log_get_namehash(fd, &name_hash);
+ if (ret < 0)
+ {
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* print any warnings related to this log file version */
+ darshan_log_print_version_warnings(fd->version);
+
+ if (fd->comp_type == DARSHAN_ZLIB_COMP)
+ comp_str = "ZLIB";
+ else if (fd->comp_type == DARSHAN_BZIP2_COMP)
+ comp_str = "BZIP2";
+ else if (fd->comp_type == DARSHAN_NO_COMP)
+ comp_str = "NONE";
+ else
+ comp_str = "UNKNOWN";
+
+ /* print job summary */
+ printf("# darshan log version: %s\n", fd->version);
+ printf("# compression method: %s\n", comp_str);
+ printf("# exe: %s\n", tmp_string);
+ printf("# uid: %" PRId64 "\n", job.uid);
+ printf("# jobid: %" PRId64 "\n", job.jobid);
+ printf("# start_time: %" PRId64 "\n", job.start_time);
+ tmp_time += job.start_time;
+ printf("# start_time_asci: %s", ctime(&tmp_time));
+ printf("# end_time: %" PRId64 "\n", job.end_time);
+ tmp_time = 0;
+ tmp_time += job.end_time;
+ printf("# end_time_asci: %s", ctime(&tmp_time));
+ printf("# nprocs: %" PRId64 "\n", job.nprocs);
+ if (job.end_time >= job.start_time)
+ run_time = job.end_time - job.start_time + 1;
+ printf("# run time: %" PRId64 "\n", run_time);
+ for (token = strtok_r(job.metadata, "\n", &save);
+ token != NULL;
+ token = strtok_r(NULL, "\n", &save))
+ {
+ char *key;
+ char *value;
+ /* NOTE: we intentionally only split on the first = character.
+ * There may be additional = characters in the value portion
+ * (for example, when storing mpi-io hints).
+ */
+ strcpy(buffer, token);
+ key = buffer;
+ value = index(buffer, '=');
+ if(!value)
+ continue;
+ /* convert = to a null terminator to split key and value */
+ value[0] = '\0';
+ value++;
+ printf("# metadata: %s = %s\n", key, value);
+ }
+
+ /* print breakdown of each log file region's contribution to file size */
+ printf("\n# log file regions\n");
+ printf("# -------------------------------------------------------\n");
+ printf("# header: %zu bytes (uncompressed)\n", sizeof(struct darshan_header));
+ printf("# job data: %zu bytes (compressed)\n", fd->job_map.len);
+ printf("# record table: %zu bytes (compressed)\n", fd->name_map.len);
+ for (i = 0; i < DARSHAN_MAX_MODS; i++)
+ {
+ if (fd->mod_map[i].len)
+ {
+ printf("# %s module: %zu bytes (compressed), ver=%d\n",
+ darshan_module_names[i], fd->mod_map[i].len, fd->mod_ver[i]);
+ }
+ }
+
+ /* print table of mounted file systems */
+ printf("\n# mounted file systems (mount point and fs type)\n");
+ printf("# -------------------------------------------------------\n");
+ for (i = 0; i < mount_count; i++)
+ {
+ printf("# mount entry:\t%s\t%s\n", mnt_data_array[i].mnt_path,
+ mnt_data_array[i].mnt_type);
+ }
+
+ /* just exit if there is no DXT data in this log file */
+ if(fd->mod_map[DXT_POSIX_MOD].len == 0 && fd->mod_map[DXT_MPIIO_MOD].len == 0)
+ {
+ printf("\n# no DXT module data available for this Darshan log.\n");
+ goto cleanup;
+ }
+
+ for (i = 0; i < DARSHAN_MAX_MODS; i++)
+ {
+ struct darshan_base_record *base_rec;
+
+ /* check each module for any data */
+ if (fd->mod_map[i].len == 0)
+ continue;
+ /* skip modules with no logutil definitions */
+ else if (!mod_logutils[i])
+ {
+ fprintf(stderr, "Warning: no log utility handlers defined "
+ "for module %s, SKIPPING.\n", darshan_module_names[i]);
+ continue;
+ }
+
+ if (i == DXT_POSIX_MOD || i == DXT_MPIIO_MOD) {
+ printf("\n# ***************************************************\n");
+ printf("# %s module data\n", darshan_module_names[i]);
+ printf("# ***************************************************\n");
+ }
+ else if (i != DARSHAN_LUSTRE_MOD)
+ continue;
+
+ /* print warning if this module only stored partial data */
+ if(DARSHAN_MOD_FLAG_ISSET(fd->partial_flag, i))
+ printf("\n# *WARNING*: The %s module contains incomplete data!\n"
+ "# This happens when a module runs out of\n"
+ "# memory to store new record data.\n",
+ darshan_module_names[i]);
+
+ /* loop over each of this module's records and print them */
+ while(1)
+ {
+ char *mnt_pt = NULL;
+ char *fs_type = NULL;
+ char *rec_name = NULL;
+
+ if (i == DARSHAN_LUSTRE_MOD) {
+ lustre_rec_ref = malloc(sizeof(*lustre_rec_ref));
+ assert(lustre_rec_ref);
+ memset(lustre_rec_ref, 0, sizeof(*lustre_rec_ref));
+
+ ret = mod_logutils[i]->log_get_record(fd,
+ (void **)&(lustre_rec_ref->rec));
+ } else {
+ ret = mod_logutils[i]->log_get_record(fd, (void **)&mod_buf);
+ }
+
+ if (ret < 1)
+ {
+ if (ret == -1)
+ {
+ fprintf(stderr, "Error: failed to parse %s module record.\n",
+ darshan_module_names[i]);
+ goto cleanup;
+ }
+ break;
+ }
+
+ if(i == DARSHAN_LUSTRE_MOD) {
+ HASH_ADD(hlink, lustre_rec_hash, rec->base_rec.id,
+ sizeof(darshan_record_id), lustre_rec_ref);
+ continue;
+ }
+
+ base_rec = (struct darshan_base_record *)mod_buf;
+
+ /* get the pathname for this record */
+ HASH_FIND(hlink, name_hash, &(base_rec->id),
+ sizeof(darshan_record_id), ref);
+
+ if (ref)
+ {
+ rec_name = ref->name_record->name;
+
+ /* get mount point and fs type associated with this record */
+ for (j = 0; j < mount_count; j++)
+ {
+ if(strncmp(mnt_data_array[j].mnt_path, rec_name,
+ strlen(mnt_data_array[j].mnt_path)) == 0)
+ {
+ mnt_pt = mnt_data_array[j].mnt_path;
+ fs_type = mnt_data_array[j].mnt_type;
+ break;
+ }
+ }
+ }
+
+ if (!mnt_pt)
+ mnt_pt = "UNKNOWN";
+ if (!fs_type)
+ fs_type = "UNKNOWN";
+
+ if (i == DXT_POSIX_MOD) {
+ /* look for corresponding lustre record and print DXT data */
+ HASH_FIND(hlink, lustre_rec_hash, &(base_rec->id),
+ sizeof(darshan_record_id), lustre_rec_ref);
+
+ dxt_log_print_posix_file(mod_buf, rec_name,
+ mnt_pt, fs_type, lustre_rec_ref);
+ } else if (i == DXT_MPIIO_MOD){
+ dxt_log_print_mpiio_file(mod_buf, rec_name,
+ mnt_pt, fs_type);
+ }
+
+ free(mod_buf);
+ mod_buf = NULL;
+ }
+ }
+
+ ret = 0;
+
+cleanup:
+ darshan_log_close(fd);
+
+ /* free record hash data */
+ HASH_ITER(hlink, name_hash, ref, tmp_ref)
+ {
+ HASH_DELETE(hlink, name_hash, ref);
+ free(ref->name_record);
+ free(ref);
+ }
+
+ /* free lustre record data */
+ HASH_ITER(hlink, lustre_rec_hash, lustre_rec_ref, tmp_lustre_rec_ref)
+ {
+ HASH_DELETE(hlink, lustre_rec_hash, lustre_rec_ref);
+ free(lustre_rec_ref->rec);
+ free(lustre_rec_ref);
+ }
+
+ /* free mount info */
+ if (mount_count > 0)
+ {
+ free(mnt_data_array);
+ }
+
+ return(ret);
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
=====================================
darshan-util/darshan-logutils.h
=====================================
--- a/darshan-util/darshan-logutils.h
+++ b/darshan-util/darshan-logutils.h
@@ -20,7 +20,7 @@
/* Maximum size of a record - Lustre OST lists can get huge, but 81920 is enough
* for 10K OSTs
*/
-#define DEF_MOD_BUF_SIZE 81920 /* 640 KiB */
+#define DEF_MOD_BUF_SIZE 81920
struct darshan_fd_int_state;
@@ -55,6 +55,13 @@ struct darshan_name_record_ref
UT_hash_handle hlink;
};
+/* DXT */
+struct lustre_record_ref
+{
+ struct darshan_lustre_record *rec;
+ UT_hash_handle hlink;
+};
+
struct darshan_mnt_info
{
char mnt_type[DARSHAN_EXE_LEN];
@@ -128,6 +135,9 @@ extern struct darshan_mod_logutil_funcs *mod_logutils[];
#include "darshan-lustre-logutils.h"
#include "darshan-stdio-logutils.h"
+/* DXT */
+#include "darshan-dxt-logutils.h"
+
darshan_fd darshan_log_open(const char *name);
darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type,
int partial_flag);
=====================================
darshan-util/darshan-lustre-logutils.c
=====================================
--- a/darshan-util/darshan-lustre-logutils.c
+++ b/darshan-util/darshan-lustre-logutils.c
@@ -61,6 +61,16 @@ static int darshan_log_get_lustre_record(darshan_fd fd, void** lustre_buf_p)
else if(ret < sizeof(struct darshan_lustre_record))
return(0);
+ /* swap bytes if necessary */
+ if(fd->swap_flag)
+ {
+ DARSHAN_BSWAP64(&tmp_rec.base_rec.id);
+ DARSHAN_BSWAP64(&tmp_rec.base_rec.rank);
+ for(i=0; i<LUSTRE_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&tmp_rec.counters[i]);
+ DARSHAN_BSWAP64(&tmp_rec.ost_ids[0]);
+ }
+
if(*lustre_buf_p == NULL)
{
rec = malloc(LUSTRE_RECORD_SIZE(tmp_rec.counters[LUSTRE_STRIPE_WIDTH]));
@@ -69,16 +79,6 @@ static int darshan_log_get_lustre_record(darshan_fd fd, void** lustre_buf_p)
}
memcpy(rec, &tmp_rec, sizeof(struct darshan_lustre_record));
- /* swap bytes if necessary */
- if(fd->swap_flag)
- {
- DARSHAN_BSWAP64(&rec->base_rec.id);
- DARSHAN_BSWAP64(&rec->base_rec.rank);
- for(i=0; i<LUSTRE_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&rec->counters[i]);
- DARSHAN_BSWAP64(&(rec->ost_ids[0]));
- }
-
/* now read the rest of the record */
if ( rec->counters[LUSTRE_STRIPE_WIDTH] > 1 ) {
ret = darshan_log_get_mod(
@@ -87,16 +87,16 @@ static int darshan_log_get_lustre_record(darshan_fd fd, void** lustre_buf_p)
(void*)(&(rec->ost_ids[1])),
(rec->counters[LUSTRE_STRIPE_WIDTH] - 1)*sizeof(OST_ID)
);
- if(ret < 0)
+ if(ret < (rec->counters[LUSTRE_STRIPE_WIDTH] - 1)*sizeof(OST_ID))
ret = -1;
- else if(ret < (rec->counters[LUSTRE_STRIPE_WIDTH] - 1)*sizeof(OST_ID))
- ret = 0;
else
+ {
ret = 1;
- /* swap bytes if necessary */
- if ( fd->swap_flag )
- for (i = 1; i < rec->counters[LUSTRE_STRIPE_WIDTH]; i++ )
- DARSHAN_BSWAP64(&(rec->ost_ids[i]));
+ /* swap bytes if necessary */
+ if ( fd->swap_flag )
+ for (i = 1; i < rec->counters[LUSTRE_STRIPE_WIDTH]; i++ )
+ DARSHAN_BSWAP64(&(rec->ost_ids[i]));
+ }
}
else
{
View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/45c766b95295c80913251f6450d8d6df3eff99b6...7ec53306b75486a0330275760a252d1024ed4ed9
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-commits/attachments/20161216/aa5eeca3/attachment-0001.html>
More information about the Darshan-commits
mailing list