[Darshan-commits] [Darshan] branch, dev-bgq-mod, updated. darshan-2.3.1-165-g7e0566b
Service Account
git at mcs.anl.gov
Wed Sep 2 19:07:04 CDT 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "".
The branch, dev-bgq-mod has been updated
via 7e0566b7f583000e7ae82a98b7a94a68c915dbd0 (commit)
via 835e28d98724d6685611fb5b485339aecd3417d0 (commit)
via 760e827e12c90f773ed0358a820e23cf45b432fe (commit)
via fab0224e40a04833418f345e4b391fa918156e44 (commit)
via e5172a88ac2646efa6049bc41e2ccafe51ea33ab (commit)
via 168fe00f798234d2da5bdd059939eb38f59f04c6 (commit)
via f42d933f17d04332649575a1206b2492c602d112 (commit)
via 7f0cf8630fc86db60aaef68928ef9023a156bb36 (commit)
via c39904ba0f3a690585b0d93b9cf902a425e3b5f6 (commit)
via 89a97a0f4de39f7ee149308af537390d5db0a8dc (commit)
via 8eb2f98d7ad53bfede836909ff9ba3956b2351a1 (commit)
via 88926470e0528fda0b182f247c0052c84276e3c0 (commit)
via 814b126671a39bcad4a3df69fc1273518f633168 (commit)
via e5bfa490dd6dd2d1318a352a20c34ca219951cf0 (commit)
via a94832b33200c1dfbcbbc05a8be71a7219e30446 (commit)
via a850ea7453abc28a3e46e554cd48ab49739ea156 (commit)
via dcc9c7feafae1d129b5ce5e4d6db38b10344259a (commit)
via eb0e5f5b95d4dda633ee97208b658e9d45a331df (commit)
via d4db229887c798c5b3cb3d2f9ce7e6f531f9a050 (commit)
via 093b9f22baae4320fe61e8903b721d29e7a8feed (commit)
via d8b6e0dc9610db8dcec6a4dce64b12d906526239 (commit)
via 53c9e15b4b2df3b67e99ee61ace8b456e86a931f (commit)
via baa3f7bca57ca182c15462528f6de0a4776f8906 (commit)
via 718c949783d8f39699000b2002e884a9f2d2ac35 (commit)
via 9e716a5770285d595ca6961ccf418242972c9a09 (commit)
via 9f202697a19e98ae8822037260f1ec342033d8f4 (commit)
via b2268e7b10a7494ea7df2cfaae622f628bdf701a (commit)
via 0d7fe6935fdccba8d604962228b5fdb0688f2656 (commit)
via 4f5d233935e74193e7298efe688bb03169693457 (commit)
via 30336460ceb8becd0dd219a2631f647b9f65ff6f (commit)
via 6959ca27817bb471896df14308f5605b732bc11f (commit)
via a2ef6c6965fc2df514bd89728f69b2f205ec615c (commit)
via f20198469abb49a27eef6a0d88e7fe8022266ac3 (commit)
via 3b0dacf7e5b8066fc7d911ed9e96f2f319da479e (commit)
via 3a362ed5a6ae8e08f3e2edbdb5c9360e42940e12 (commit)
via 659b579db8c0086e707570feef45c89211fc5d16 (commit)
via 328425b72ce7b211f940024719aa9aca7f4dbc38 (commit)
via 158337c98e18596458cf2b3102c3f4a63c2ebfae (commit)
via b6807fcc3b14ebb5e1323e219a72ddbfde2af64e (commit)
via 019fba2263b7ab66fecde7298f4c6dd84a4e1e2f (commit)
via dafe379082b24c552f1285c30a3fcf4edd8db8de (commit)
via f1468f4943f8bba74afbd2a3f84ce6fc0071a556 (commit)
via 8eea697a4c9e6eaca6cc6021b29521bf23e9c3d3 (commit)
via a0b3f9cc1820fa2e7828ad0919886e9739e03187 (commit)
via aa46bf39a7c294be76267303f98f8d34a5da3975 (commit)
via 4e6d0c89355ee38b733e83d594b77590f379ecde (commit)
via f4df41a355a7327706ed3a9fc8d65a0ab4496c0b (commit)
via a708b0ce36c2bc7fd25b44d1756624129bb870e0 (commit)
via 735415387319aa6e406800952327bf3aeef7e12c (commit)
via 6406c8726d1fe664f9de8147248da121c642ec6a (commit)
via e0eb27be2b7827593b442e104af8b45cb585e876 (commit)
via d2776592214b92e72cac8c3903823271ef541ba8 (commit)
via 53fa81e8bd7449c9caf48f78fb64e55829b9aa1d (commit)
via c572aa446c79a9192b8d2c30d33485da8c6f4ef2 (commit)
via f1ad34e8cf3f2aff0e12f83cd85e9f95726bd9e6 (commit)
via 628724ba033d37cad799ef52d95cbe74d2274776 (commit)
via 28f19402834f8b016441741b7dd87c0b4859e9f7 (commit)
via faa3101fd3c8aac2d7f3b51106b3a84f96a6b6f8 (commit)
via 347a0744d26f300de58cd784556ee5ff67da39a9 (commit)
via 3a01d053fa7a64b2a60c500e50987b52813f7658 (commit)
from 94efcc1729a753e7cdb2894f8219262c67ba3961 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 7e0566b7f583000e7ae82a98b7a94a68c915dbd0
Author: Kevin Harms <harms at alcf.anl.gov>
Date: Thu Sep 3 00:06:53 2015 +0000
Bug fixes after testing
commit 835e28d98724d6685611fb5b485339aecd3417d0
Author: Kevin Harms <harms at alcf.anl.gov>
Date: Wed Sep 2 01:44:21 2015 +0000
Updates to align with latest modular code base
commit 760e827e12c90f773ed0358a820e23cf45b432fe
Merge: 94efcc1729a753e7cdb2894f8219262c67ba3961 fab0224e40a04833418f345e4b391fa918156e44
Author: Kevin Harms <harms at alcf.anl.gov>
Date: Tue Sep 1 19:25:55 2015 +0000
merge from dev-modular
-----------------------------------------------------------------------
Summary of changes:
darshan-bgq-log-format.h | 38 +-
darshan-hdf5-log-format.h | 56 +
darshan-log-format.h | 50 +-
darshan-mpiio-log-format.h | 140 ++-
darshan-pnetcdf-log-format.h | 58 +
darshan-posix-log-format.h | 242 ++--
darshan-runtime/Makefile.in | 35 +-
darshan-runtime/darshan-base-ld-opts.in | 2 +
darshan-runtime/darshan-common.h | 192 +++
darshan-runtime/darshan-core.h | 13 +
darshan-runtime/darshan-hdf5-ld-opts | 3 +
darshan-runtime/darshan-pnetcdf-ld-opts | 3 +
darshan-runtime/darshan-posix-ld-opts | 8 +
darshan-runtime/darshan.h | 167 +--
darshan-runtime/lib/darshan-bgq.c | 104 +-
darshan-runtime/lib/darshan-common.c | 128 ++-
darshan-runtime/lib/darshan-core.c | 179 +--
darshan-runtime/lib/darshan-hdf5-stubs.c | 80 +
darshan-runtime/lib/darshan-hdf5.c | 576 +++++++
darshan-runtime/lib/darshan-mpiio.c | 1218 +++++++++++++--
darshan-runtime/lib/darshan-null.c | 44 +-
darshan-runtime/lib/darshan-pnetcdf-stubs.c | 77 +
darshan-runtime/lib/darshan-pnetcdf.c | 588 +++++++
darshan-runtime/lib/darshan-posix.c | 1016 +++++++-----
darshan-test/regression/run-all.sh | 2 +-
darshan-test/regression/test-cases/cxxpi.sh | 9 +-
darshan-test/regression/test-cases/fperf-f77.sh | 18 +-
darshan-test/regression/test-cases/fperf-f90.sh | 18 +-
darshan-test/regression/test-cases/mpi-io-test.sh | 16 +-
darshan-util/Makefile.in | 76 +-
darshan-util/darshan-analyzer.c | 142 +-
darshan-util/darshan-base-parser.c | 180 ---
darshan-util/darshan-bgq-logutils.c | 87 +-
darshan-util/darshan-bgq-logutils.h | 5 +-
darshan-util/darshan-convert-logs.pl | 7 +-
darshan-util/darshan-convert.c | 257 +++-
darshan-util/darshan-hdf5-logutils.c | 105 ++
darshan-util/darshan-hdf5-logutils.h | 18 +
.../bin/darshan-job-summary.pl.in | 1350 +----------------
darshan-util/darshan-load-mysql.c | 509 ------
darshan-util/darshan-log-params.c | 14 -
darshan-util/darshan-logutils.c | 946 ++++++++++--
darshan-util/darshan-logutils.h | 88 +-
darshan-util/darshan-mpiio-logutils.c | 87 +-
darshan-util/darshan-mpiio-logutils.h | 5 +-
darshan-util/darshan-mpiio-parser.c | 214 ---
darshan-util/darshan-parser.c | 1695 +++++++++++++-------
darshan-util/darshan-pnetcdf-logutils.c | 105 ++
darshan-util/darshan-pnetcdf-logutils.h | 18 +
darshan-util/darshan-posix-logutils.c | 87 +-
darshan-util/darshan-posix-logutils.h | 5 +-
darshan-util/darshan-posix-parser.c | 354 ----
.../bin/jenkins.c => jenkins-hash-gen.c} | 0
doc/darshan-dev-modular-runtime.png | Bin 30933 -> 35079 bytes
doc/darshan-modularization.txt | 72 +-
55 files changed, 6869 insertions(+), 4637 deletions(-)
create mode 100644 darshan-hdf5-log-format.h
create mode 100644 darshan-pnetcdf-log-format.h
create mode 100644 darshan-runtime/darshan-common.h
create mode 100644 darshan-runtime/darshan-hdf5-ld-opts
create mode 100644 darshan-runtime/darshan-pnetcdf-ld-opts
create mode 100644 darshan-runtime/lib/darshan-hdf5-stubs.c
create mode 100644 darshan-runtime/lib/darshan-hdf5.c
create mode 100644 darshan-runtime/lib/darshan-pnetcdf-stubs.c
create mode 100644 darshan-runtime/lib/darshan-pnetcdf.c
delete mode 100644 darshan-util/darshan-base-parser.c
create mode 100644 darshan-util/darshan-hdf5-logutils.c
create mode 100644 darshan-util/darshan-hdf5-logutils.h
delete mode 100644 darshan-util/darshan-load-mysql.c
delete mode 100644 darshan-util/darshan-log-params.c
delete mode 100644 darshan-util/darshan-mpiio-parser.c
create mode 100644 darshan-util/darshan-pnetcdf-logutils.c
create mode 100644 darshan-util/darshan-pnetcdf-logutils.h
delete mode 100644 darshan-util/darshan-posix-parser.c
rename darshan-util/{darshan-job-summary/bin/jenkins.c => jenkins-hash-gen.c} (100%)
Diff of changes:
diff --git a/darshan-bgq-log-format.h b/darshan-bgq-log-format.h
index 902b850..82353fd 100644
--- a/darshan-bgq-log-format.h
+++ b/darshan-bgq-log-format.h
@@ -9,30 +9,38 @@
#include "darshan-log-format.h"
+
+#define BGQ_COUNTERS \
+ X(BGQ_CSJOBID, "control system jobid") \
+ X(BGQ_NNODES, "number of BGQ compute nodes") \
+ X(BGQ_RANKSPERNODE, "number of MPI ranks per node") \
+ X(BGQ_DDRPERNODE, "size in MB of DDR3 per node") \
+ X(BGQ_INODES, "number of i/o nodes") \
+ X(BGQ_ANODES, "dimension of A torus") \
+ X(BGQ_BNODES, "dimension of B torus") \
+ X(BGQ_CNODES, "dimension of C torus") \
+ X(BGQ_DNODES, "dimension of D torus") \
+ X(BGQ_ENODES, "dimension of E torus") \
+ X(BGQ_TORUSENABLED, "which dimensions are torus") \
+ X(BGQ_NUM_INDICES, "end of counters")
+
+#define BGQ_F_COUNTERS \
+ X(BGQ_F_TIMESTAMP, "timestamp when data was collected") \
+ X(BGQ_F_NUM_INDICES, "end of counters")
+
+#define X(a, b) a,
/* integer counters for the "BGQ" example module */
enum darshan_bgq_indices
{
- BGQ_CSJOBID, // control system jobid
- BGQ_NNODES, // number of BGQ compute nodes
- BGQ_RANKSPERNODE, // number of MPI ranks per node
- BGQ_DDRPERNODE, // size in MB of DDR3 per node
- BGQ_INODES, // number of i/o nodes
- BGQ_ANODES, // dimension of A torus
- BGQ_BNODES, // dimension of B torus
- BGQ_CNODES, // dimension of C torus
- BGQ_DNODES, // dimension of D torus
- BGQ_ENODES, // dimension of E torus
- BGQ_TORUSENABLED, // which dimensions are torus
-
- BGQ_NUM_INDICES,
+ BGQ_COUNTERS
};
/* floating point counters for the "BGQ" example module */
enum darshan_bgq_f_indices
{
- BGQ_F_TIMESTAMP, // timestamp when data collected
- BGQ_F_NUM_INDICES,
+ BGQ_F_COUNTERS
};
+#undef X
/* the darshan_bgq_record structure encompasses the high-level data/counters
* which would actually be logged to file by Darshan for the "BGQ" example
diff --git a/darshan-hdf5-log-format.h b/darshan-hdf5-log-format.h
new file mode 100644
index 0000000..1a67092
--- /dev/null
+++ b/darshan-hdf5-log-format.h
@@ -0,0 +1,56 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_HDF5_LOG_FORMAT_H
+#define __DARSHAN_HDF5_LOG_FORMAT_H
+
+#include "darshan-log-format.h"
+
+#define HDF5_COUNTERS \
+ /* count of HDF5 opens */\
+ X(HDF5_OPENS) \
+ /* end of counters */\
+ X(HDF5_NUM_INDICES)
+
+#define HDF5_F_COUNTERS \
+ /* timestamp of first open */\
+ X(HDF5_F_OPEN_TIMESTAMP) \
+ /* timestamp of last close */\
+ X(HDF5_F_CLOSE_TIMESTAMP) \
+ /* end of counters*/\
+ X(HDF5_F_NUM_INDICES)
+
+#define X(a) a,
+/* integer statistics for HDF5 file records */
+enum darshan_hdf5_indices
+{
+ HDF5_COUNTERS
+};
+
+/* floating point statistics for HDF5 file records */
+enum darshan_hdf5_f_indices
+{
+ HDF5_F_COUNTERS
+};
+#undef X
+
+/* file record structure for HDF5 files. a record is created and stored for
+ * every HDF5 file opened by the original application. For the HDF5 module,
+ * the record includes:
+ * - a corresponding record identifier (created by hashing the file path)
+ * - the rank of the process which opened the file (-1 for shared files)
+ * - integer file I/O statistics (open, read/write counts, etc)
+ * - floating point I/O statistics (timestamps, cumulative timers, etc.)
+ */
+struct darshan_hdf5_file
+{
+ darshan_record_id f_id;
+ int64_t rank;
+ int64_t counters[HDF5_NUM_INDICES];
+ double fcounters[HDF5_F_NUM_INDICES];
+};
+
+#endif /* __DARSHAN_HDF5_LOG_FORMAT_H */
diff --git a/darshan-log-format.h b/darshan-log-format.h
index ada269f..a7cc4e3 100644
--- a/darshan-log-format.h
+++ b/darshan-log-format.h
@@ -31,32 +31,53 @@
/* max length of exe string within job record (not counting '\0') */
#define DARSHAN_EXE_LEN (DARSHAN_JOB_RECORD_SIZE - sizeof(struct darshan_job) - 1)
-typedef uint64_t darshan_record_id;
+#define DARSHAN_MAX_MODS 16
+
+/* TODO: do we want the logutil defs here ? */
+/* X-macro for keeping module ordering consistent */
+/* NOTE: first val used to define module enum values,
+ * second val used to define module name strings, and
+ * third val is used to provide the name of a
+ * corresponding logutils structure for parsing module
+ * data out of the log file (only used in darshan-util,
+ * just pass NULL (no quotes) if no log parsing
+ * functions are required).
+ */
+#define DARSHAN_MODULE_IDS \
+ X(DARSHAN_NULL_MOD, "NULL", NULL) \
+ X(DARSHAN_POSIX_MOD, "POSIX", posix_logutils) \
+ X(DARSHAN_MPIIO_MOD, "MPI-IO", mpiio_logutils) \
+ X(DARSHAN_HDF5_MOD, "HDF5", hdf5_logutils) \
+ X(DARSHAN_PNETCDF_MOD, "PNETCDF", pnetcdf_logutils) \
+ X(DARSHAN_BGQ_MOD, "BG/Q", bgq_logutils)
/* unique identifiers to distinguish between available darshan modules */
/* NOTES: - valid ids range from [0...DARSHAN_MAX_MODS-1]
* - order of ids control module shutdown order (and consequently, order in log file)
*/
-#define DARSHAN_MAX_MODS 16
+#define X(a, b, c) a,
typedef enum
{
- DARSHAN_NULL_MOD = 0,
- DARSHAN_POSIX_MOD,
- DARSHAN_MPIIO_MOD,
- DARSHAN_HDF5_MOD,
- DARSHAN_PNETCDF_MOD,
- DARSHAN_BGQ_MOD,
+ DARSHAN_MODULE_IDS
} darshan_module_id;
+#undef X
+/* module name strings */
+#define X(a, b, c) b,
static char * const darshan_module_names[] =
{
- "NULL",
- "POSIX",
- "MPI-IO",
- "HDF5",
- "PNETCDF",
- "BG/Q",
+ DARSHAN_MODULE_IDS
};
+#undef X
+
+/* compression method used on darshan log file */
+enum darshan_comp_type
+{
+ DARSHAN_ZLIB_COMP,
+ DARSHAN_BZIP2_COMP,
+};
+
+typedef uint64_t darshan_record_id;
/* the darshan_log_map structure is used to indicate the location of
* specific module data in a Darshan log. Note that 'off' and 'len' are
@@ -78,6 +99,7 @@ struct darshan_header
{
char version_string[8];
int64_t magic_nr;
+ unsigned char comp_type;
struct darshan_log_map rec_map;
struct darshan_log_map mod_map[DARSHAN_MAX_MODS];
};
diff --git a/darshan-mpiio-log-format.h b/darshan-mpiio-log-format.h
index 857133d..5aa28e2 100644
--- a/darshan-mpiio-log-format.h
+++ b/darshan-mpiio-log-format.h
@@ -9,29 +9,147 @@
#include "darshan-log-format.h"
+/* TODO: maybe use a counter to track cases in which a derived datatype is used? */
+
+#define MPIIO_COUNTERS \
+ /* count of MPI independent opens */\
+ X(MPIIO_INDEP_OPENS) \
+ /* count of MPI collective opens */\
+ X(MPIIO_COLL_OPENS) \
+ /* count of MPI independent reads */\
+ X(MPIIO_INDEP_READS) \
+ /* count of MPI independent writes */\
+ X(MPIIO_INDEP_WRITES) \
+ /* count of MPI collective reads */\
+ X(MPIIO_COLL_READS) \
+ /* count of MPI collective writes */\
+ X(MPIIO_COLL_WRITES) \
+ /* count of MPI split collective reads */\
+ X(MPIIO_SPLIT_READS) \
+ /* count of MPI split collective writes */\
+ X(MPIIO_SPLIT_WRITES) \
+ /* count of MPI nonblocking reads */\
+ X(MPIIO_NB_READS) \
+ /* count of MPI nonblocking writes */\
+ X(MPIIO_NB_WRITES) \
+ /* count of MPI file syncs */\
+ X(MPIIO_SYNCS) \
+ /* count of MPI hints used */\
+ X(MPIIO_HINTS) \
+ /* count of MPI set view calls */\
+ X(MPIIO_VIEWS) \
+ /* MPI-IO access mode of the file */\
+ X(MPIIO_MODE) \
+ /* total bytes read at MPI-IO layer */\
+ X(MPIIO_BYTES_READ) \
+ /* total bytes written at MPI-IO layer */\
+ X(MPIIO_BYTES_WRITTEN) \
+ /* number of times switching between MPI read and write */\
+ X(MPIIO_RW_SWITCHES) \
+ /* sizes of the maximum read/write operations */\
+ X(MPIIO_MAX_READ_TIME_SIZE) \
+ X(MPIIO_MAX_WRITE_TIME_SIZE) \
+ /* buckets for MPI read size ranges */\
+ X(MPIIO_SIZE_READ_AGG_0_100) \
+ X(MPIIO_SIZE_READ_AGG_100_1K) \
+ X(MPIIO_SIZE_READ_AGG_1K_10K) \
+ X(MPIIO_SIZE_READ_AGG_10K_100K) \
+ X(MPIIO_SIZE_READ_AGG_100K_1M) \
+ X(MPIIO_SIZE_READ_AGG_1M_4M) \
+ X(MPIIO_SIZE_READ_AGG_4M_10M) \
+ X(MPIIO_SIZE_READ_AGG_10M_100M) \
+ X(MPIIO_SIZE_READ_AGG_100M_1G) \
+ X(MPIIO_SIZE_READ_AGG_1G_PLUS) \
+ /* buckets for MPI write size ranges */\
+ X(MPIIO_SIZE_WRITE_AGG_0_100) \
+ X(MPIIO_SIZE_WRITE_AGG_100_1K) \
+ X(MPIIO_SIZE_WRITE_AGG_1K_10K) \
+ X(MPIIO_SIZE_WRITE_AGG_10K_100K) \
+ X(MPIIO_SIZE_WRITE_AGG_100K_1M) \
+ X(MPIIO_SIZE_WRITE_AGG_1M_4M) \
+ X(MPIIO_SIZE_WRITE_AGG_4M_10M) \
+ X(MPIIO_SIZE_WRITE_AGG_10M_100M) \
+ X(MPIIO_SIZE_WRITE_AGG_100M_1G) \
+ X(MPIIO_SIZE_WRITE_AGG_1G_PLUS) \
+ /* the four most frequently appearing MPI access sizes */\
+ X(MPIIO_ACCESS1_ACCESS) \
+ X(MPIIO_ACCESS2_ACCESS) \
+ X(MPIIO_ACCESS3_ACCESS) \
+ X(MPIIO_ACCESS4_ACCESS) \
+ /* count of each of the most frequent MPI access sizes */\
+ X(MPIIO_ACCESS1_COUNT) \
+ X(MPIIO_ACCESS2_COUNT) \
+ X(MPIIO_ACCESS3_COUNT) \
+ X(MPIIO_ACCESS4_COUNT) \
+ /* rank and number of bytes moved for fastest/slowest ranks */\
+ X(MPIIO_FASTEST_RANK) \
+ X(MPIIO_FASTEST_RANK_BYTES) \
+ X(MPIIO_SLOWEST_RANK) \
+ X(MPIIO_SLOWEST_RANK_BYTES) \
+ /* end of counters */\
+ X(MPIIO_NUM_INDICES)
+
+#define MPIIO_F_COUNTERS \
+ /* timestamp of first open */\
+ X(MPIIO_F_OPEN_TIMESTAMP) \
+ /* timestamp of first read */\
+ X(MPIIO_F_READ_START_TIMESTAMP) \
+ /* timestamp of first write */\
+ X(MPIIO_F_WRITE_START_TIMESTAMP) \
+ /* timestamp of last read */\
+ X(MPIIO_F_READ_END_TIMESTAMP) \
+ /* timestamp of last write */\
+ X(MPIIO_F_WRITE_END_TIMESTAMP) \
+ /* timestamp of last close */\
+ X(MPIIO_F_CLOSE_TIMESTAMP) \
+ /* cumulative MPI-IO read time */\
+ X(MPIIO_F_READ_TIME) \
+ /* cumulative MPI-IO write time */\
+ X(MPIIO_F_WRITE_TIME) \
+ /* cumulative MPI-IO meta time */\
+ X(MPIIO_F_META_TIME) \
+ /* maximum MPI-IO read duration */\
+ X(MPIIO_F_MAX_READ_TIME) \
+ /* maximum MPI-IO write duration */\
+ X(MPIIO_F_MAX_WRITE_TIME) \
+ /* total i/o and meta time for fastest/slowest ranks */\
+ X(MPIIO_F_FASTEST_RANK_TIME) \
+ X(MPIIO_F_SLOWEST_RANK_TIME) \
+ /* variance of total i/o time and bytes moved across all ranks */\
+ /* NOTE: for shared records only */\
+ X(MPIIO_F_VARIANCE_RANK_TIME) \
+ X(MPIIO_F_VARIANCE_RANK_BYTES) \
+ /* end of counters*/\
+ X(MPIIO_F_NUM_INDICES)
+
+#define X(a) a,
+/* integer statistics for MPI-IO file records */
enum darshan_mpiio_indices
{
- DARSHAN_MPIIO_INDEP_OPENS, /* independent opens */
- DARSHAN_MPIIO_COLL_OPENS, /* collective opens */
- DARSHAN_MPIIO_HINTS, /* how many times hints were used */
-
- DARSHAN_MPIIO_NUM_INDICES,
+ MPIIO_COUNTERS
};
+/* floating point statistics for MPI-IO file records */
enum darshan_mpiio_f_indices
{
- DARSHAN_MPIIO_F_META_TIME, /* cumulative metadata time */
- DARSHAN_MPIIO_F_OPEN_TIMESTAMP, /* first open timestamp */
-
- DARSHAN_MPIIO_F_NUM_INDICES,
+ MPIIO_F_COUNTERS
};
+#undef X
+/* file record structure for MPI-IO files. a record is created and stored for
+ * every MPI-IO file opened by the original application. For the MPI-IO module,
+ * the record includes:
+ * - a corresponding record identifier (created by hashing the file path)
+ * - the rank of the process which opened the file (-1 for shared files)
+ * - integer file I/O statistics (open, read/write counts, etc)
+ * - floating point I/O statistics (timestamps, cumulative timers, etc.)
+ */
struct darshan_mpiio_file
{
darshan_record_id f_id;
int64_t rank;
- int64_t counters[DARSHAN_MPIIO_NUM_INDICES];
- double fcounters[DARSHAN_MPIIO_F_NUM_INDICES];
+ int64_t counters[MPIIO_NUM_INDICES];
+ double fcounters[MPIIO_F_NUM_INDICES];
};
#endif /* __DARSHAN_MPIIO_LOG_FORMAT_H */
diff --git a/darshan-pnetcdf-log-format.h b/darshan-pnetcdf-log-format.h
new file mode 100644
index 0000000..8bb3565
--- /dev/null
+++ b/darshan-pnetcdf-log-format.h
@@ -0,0 +1,58 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_PNETCDF_LOG_FORMAT_H
+#define __DARSHAN_PNETCDF_LOG_FORMAT_H
+
+#include "darshan-log-format.h"
+
+#define PNETCDF_COUNTERS \
+ /* count of PNETCDF independent opens */\
+ X(PNETCDF_INDEP_OPENS) \
+ /* count of PNETCDF collective opens */\
+ X(PNETCDF_COLL_OPENS) \
+ /* end of counters */\
+ X(PNETCDF_NUM_INDICES)
+
+#define PNETCDF_F_COUNTERS \
+ /* timestamp of first open */\
+ X(PNETCDF_F_OPEN_TIMESTAMP) \
+ /* timestamp of last close */\
+ X(PNETCDF_F_CLOSE_TIMESTAMP) \
+ /* end of counters*/\
+ X(PNETCDF_F_NUM_INDICES)
+
+#define X(a) a,
+/* integer statistics for PNETCDF file records */
+enum darshan_pnetcdf_indices
+{
+ PNETCDF_COUNTERS
+};
+
+/* floating point statistics for PNETCDF file records */
+enum darshan_pnetcdf_f_indices
+{
+ PNETCDF_F_COUNTERS
+};
+#undef X
+
+/* file record structure for PNETCDF files. a record is created and stored for
+ * every PNETCDF file opened by the original application. For the PNETCDF module,
+ * the record includes:
+ * - a corresponding record identifier (created by hashing the file path)
+ * - the rank of the process which opened the file (-1 for shared files)
+ * - integer file I/O statistics (open, read/write counts, etc)
+ * - floating point I/O statistics (timestamps, cumulative timers, etc.)
+ */
+struct darshan_pnetcdf_file
+{
+ darshan_record_id f_id;
+ int64_t rank;
+ int64_t counters[PNETCDF_NUM_INDICES];
+ double fcounters[PNETCDF_F_NUM_INDICES];
+};
+
+#endif /* __DARSHAN_PNETCDF_LOG_FORMAT_H */
diff --git a/darshan-posix-log-format.h b/darshan-posix-log-format.h
index 23b5bf9..6ef9ac5 100644
--- a/darshan-posix-log-format.h
+++ b/darshan-posix-log-format.h
@@ -8,108 +8,157 @@
#include "darshan-log-format.h"
+#define POSIX_COUNTERS \
+ /* count of posix opens */\
+ X(POSIX_OPENS) \
+ /* count of posix reads */\
+ X(POSIX_READS) \
+ /* count of posix writes */\
+ X(POSIX_WRITES) \
+ /* count of posix seeks */\
+ X(POSIX_SEEKS) \
+ /* count of posix stat/lstat/fstats */\
+ X(POSIX_STATS) \
+ /* count of posix mmaps */\
+ X(POSIX_MMAPS) \
+ /* count of posix fopens */\
+ X(POSIX_FOPENS) \
+ /* count of posix freads */\
+ X(POSIX_FREADS) \
+ /* count of posix fwrites */\
+ X(POSIX_FWRITES) \
+ /* count of posix fseeks */\
+ X(POSIX_FSEEKS) \
+ /* count of posix fsyncs */\
+ X(POSIX_FSYNCS) \
+ /* count of posix fdatasyncs */\
+ X(POSIX_FDSYNCS) \
+ /* mode of file */\
+ X(POSIX_MODE) \
+ /* total bytes read */\
+ X(POSIX_BYTES_READ) \
+ /* total bytes written */\
+ X(POSIX_BYTES_WRITTEN) \
+ /* highest offset byte read */\
+ X(POSIX_MAX_BYTE_READ) \
+ /* highest offset byte written */\
+ X(POSIX_MAX_BYTE_WRITTEN) \
+ /* count of consecutive reads */\
+ X(POSIX_CONSEC_READS) \
+ /* count of consecutive writes */\
+ X(POSIX_CONSEC_WRITES) \
+ /* count of sequential reads */\
+ X(POSIX_SEQ_READS) \
+ /* count of sequential writes */\
+ X(POSIX_SEQ_WRITES) \
+ /* number of times switched between read and write */\
+ X(POSIX_RW_SWITCHES) \
+ /* count of accesses not mem aligned */\
+ X(POSIX_MEM_NOT_ALIGNED) \
+ /* mem alignment in bytes */\
+ X(POSIX_MEM_ALIGNMENT) \
+ /* count of accesses not file aligned */\
+ X(POSIX_FILE_NOT_ALIGNED) \
+ /* file alignment in bytes */\
+ X(POSIX_FILE_ALIGNMENT) \
+ X(POSIX_MAX_READ_TIME_SIZE) \
+ X(POSIX_MAX_WRITE_TIME_SIZE) \
+ /* buckets for POSIX read size ranges */\
+ X(POSIX_SIZE_READ_0_100) \
+ X(POSIX_SIZE_READ_100_1K) \
+ X(POSIX_SIZE_READ_1K_10K) \
+ X(POSIX_SIZE_READ_10K_100K) \
+ X(POSIX_SIZE_READ_100K_1M) \
+ X(POSIX_SIZE_READ_1M_4M) \
+ X(POSIX_SIZE_READ_4M_10M) \
+ X(POSIX_SIZE_READ_10M_100M) \
+ X(POSIX_SIZE_READ_100M_1G) \
+ X(POSIX_SIZE_READ_1G_PLUS) \
+ /* buckets for POSIX write size ranges */\
+ X(POSIX_SIZE_WRITE_0_100) \
+ X(POSIX_SIZE_WRITE_100_1K) \
+ X(POSIX_SIZE_WRITE_1K_10K) \
+ X(POSIX_SIZE_WRITE_10K_100K) \
+ X(POSIX_SIZE_WRITE_100K_1M) \
+ X(POSIX_SIZE_WRITE_1M_4M) \
+ X(POSIX_SIZE_WRITE_4M_10M) \
+ X(POSIX_SIZE_WRITE_10M_100M) \
+ X(POSIX_SIZE_WRITE_100M_1G) \
+ X(POSIX_SIZE_WRITE_1G_PLUS) \
+ /* the four most frequently appearing strides */\
+ X(POSIX_STRIDE1_STRIDE) \
+ X(POSIX_STRIDE2_STRIDE) \
+ X(POSIX_STRIDE3_STRIDE) \
+ X(POSIX_STRIDE4_STRIDE) \
+ /* count of each of the most frequent strides */\
+ X(POSIX_STRIDE1_COUNT) \
+ X(POSIX_STRIDE2_COUNT) \
+ X(POSIX_STRIDE3_COUNT) \
+ X(POSIX_STRIDE4_COUNT) \
+ /* the four most frequently appearing access sizes */\
+ X(POSIX_ACCESS1_ACCESS) \
+ X(POSIX_ACCESS2_ACCESS) \
+ X(POSIX_ACCESS3_ACCESS) \
+ X(POSIX_ACCESS4_ACCESS) \
+ /* count of each of the most frequent access sizes */\
+ X(POSIX_ACCESS1_COUNT) \
+ X(POSIX_ACCESS2_COUNT) \
+ X(POSIX_ACCESS3_COUNT) \
+ X(POSIX_ACCESS4_COUNT) \
+ /* rank and number of bytes moved for fastest/slowest ranks */\
+ X(POSIX_FASTEST_RANK) \
+ X(POSIX_FASTEST_RANK_BYTES) \
+ X(POSIX_SLOWEST_RANK) \
+ X(POSIX_SLOWEST_RANK_BYTES) \
+ /* end of counters */\
+ X(POSIX_NUM_INDICES)
+
+#define POSIX_F_COUNTERS \
+ /* timestamp of first open */\
+ X(POSIX_F_OPEN_TIMESTAMP) \
+ /* timestamp of first read */\
+ X(POSIX_F_READ_START_TIMESTAMP) \
+ /* timestamp of first write */\
+ X(POSIX_F_WRITE_START_TIMESTAMP) \
+ /* timestamp of last read */\
+ X(POSIX_F_READ_END_TIMESTAMP) \
+ /* timestamp of last write */\
+ X(POSIX_F_WRITE_END_TIMESTAMP) \
+ /* timestamp of last close */\
+ X(POSIX_F_CLOSE_TIMESTAMP) \
+ /* cumulative posix read time */\
+ X(POSIX_F_READ_TIME) \
+ /* cumulative posix write time */\
+ X(POSIX_F_WRITE_TIME) \
+ /* cumulative posix meta time */\
+ X(POSIX_F_META_TIME) \
+ /* maximum posix read duration */\
+ X(POSIX_F_MAX_READ_TIME) \
+ /* maximum posix write duration */\
+ X(POSIX_F_MAX_WRITE_TIME) \
+ /* total i/o and meta time consumed for fastest/slowest ranks */\
+ X(POSIX_F_FASTEST_RANK_TIME) \
+ X(POSIX_F_SLOWEST_RANK_TIME) \
+ /* variance of total i/o time and bytes moved across all ranks */\
+ /* NOTE: for shared records only */\
+ X(POSIX_F_VARIANCE_RANK_TIME) \
+ X(POSIX_F_VARIANCE_RANK_BYTES) \
+ /* end of counters */\
+ X(POSIX_F_NUM_INDICES)
+
+#define X(a) a,
/* integer statistics for POSIX file records */
enum darshan_posix_indices
{
- POSIX_OPENS, /* count of posix opens */
- POSIX_READS, /* count of posix reads */
- POSIX_WRITES, /* count of posix writes */
- POSIX_SEEKS, /* count of posix seeks */
- POSIX_STATS, /* count of posix stat/lstat/fstats */
- POSIX_MMAPS, /* count of posix mmaps */
- POSIX_FOPENS, /* count of posix fopens */
- POSIX_FREADS, /* count of posix freads */
- POSIX_FWRITES, /* count of posix fwrites */
- POSIX_FSEEKS, /* count of posix fseeks */
- POSIX_FSYNCS, /* count of posix fsyncs */
- POSIX_FDSYNCS, /* count of posix fdatasyncs */
- POSIX_MODE, /* mode of file */
- POSIX_BYTES_READ, /* total bytes read */
- POSIX_BYTES_WRITTEN, /* total bytes written */
- POSIX_MAX_BYTE_READ, /* highest offset byte read */
- POSIX_MAX_BYTE_WRITTEN, /* highest offset byte written */
- POSIX_CONSEC_READS, /* count of consecutive reads */
- POSIX_CONSEC_WRITES, /* count of consecutive writes */
- POSIX_SEQ_READS, /* count of sequential reads */
- POSIX_SEQ_WRITES, /* count of sequential writes */
- POSIX_RW_SWITCHES, /* number of times switched between read and write */
- POSIX_MEM_NOT_ALIGNED, /* count of accesses not mem aligned */
- POSIX_MEM_ALIGNMENT, /* mem alignment in bytes */
- POSIX_FILE_NOT_ALIGNED, /* count of accesses not file aligned */
- POSIX_FILE_ALIGNMENT, /* file alignment in bytes */
- POSIX_MAX_READ_TIME_SIZE,
- POSIX_MAX_WRITE_TIME_SIZE,
- /* buckets */
- POSIX_SIZE_READ_0_100, /* count of posix read size ranges */
- POSIX_SIZE_READ_100_1K,
- POSIX_SIZE_READ_1K_10K,
- POSIX_SIZE_READ_10K_100K,
- POSIX_SIZE_READ_100K_1M,
- POSIX_SIZE_READ_1M_4M,
- POSIX_SIZE_READ_4M_10M,
- POSIX_SIZE_READ_10M_100M,
- POSIX_SIZE_READ_100M_1G,
- POSIX_SIZE_READ_1G_PLUS,
- /* buckets */
- POSIX_SIZE_WRITE_0_100, /* count of posix write size ranges */
- POSIX_SIZE_WRITE_100_1K,
- POSIX_SIZE_WRITE_1K_10K,
- POSIX_SIZE_WRITE_10K_100K,
- POSIX_SIZE_WRITE_100K_1M,
- POSIX_SIZE_WRITE_1M_4M,
- POSIX_SIZE_WRITE_4M_10M,
- POSIX_SIZE_WRITE_10M_100M,
- POSIX_SIZE_WRITE_100M_1G,
- POSIX_SIZE_WRITE_1G_PLUS,
- /* stride/access counters */
- POSIX_STRIDE1_STRIDE, /* the four most frequently appearing strides */
- POSIX_STRIDE2_STRIDE,
- POSIX_STRIDE3_STRIDE,
- POSIX_STRIDE4_STRIDE,
- POSIX_STRIDE1_COUNT, /* count of each of the most frequent strides */
- POSIX_STRIDE2_COUNT,
- POSIX_STRIDE3_COUNT,
- POSIX_STRIDE4_COUNT,
- POSIX_ACCESS1_ACCESS, /* the four most frequently appearing access sizes */
- POSIX_ACCESS2_ACCESS,
- POSIX_ACCESS3_ACCESS,
- POSIX_ACCESS4_ACCESS,
- POSIX_ACCESS1_COUNT, /* count of each of the most frequent access sizes */
- POSIX_ACCESS2_COUNT,
- POSIX_ACCESS3_COUNT,
- POSIX_ACCESS4_COUNT,
- POSIX_FASTEST_RANK,
- POSIX_FASTEST_RANK_BYTES,
- POSIX_SLOWEST_RANK,
- POSIX_SLOWEST_RANK_BYTES,
-
- POSIX_NUM_INDICES,
+ POSIX_COUNTERS
};
/* floating point statistics for POSIX file records */
enum darshan_posix_f_indices
{
- POSIX_F_OPEN_TIMESTAMP = 0, /* timestamp of first open */
- POSIX_F_READ_START_TIMESTAMP, /* timestamp of first read */
- POSIX_F_WRITE_START_TIMESTAMP, /* timestamp of first write */
- POSIX_F_READ_END_TIMESTAMP, /* timestamp of last read */
- POSIX_F_WRITE_END_TIMESTAMP, /* timestamp of last write */
- POSIX_F_CLOSE_TIMESTAMP, /* timestamp of last close */
- POSIX_F_READ_TIME, /* cumulative posix read time */
- POSIX_F_WRITE_TIME, /* cumulative posix write time */
- POSIX_F_META_TIME, /* cumulative posix meta time */
- POSIX_F_MAX_READ_TIME,
- POSIX_F_MAX_WRITE_TIME,
- /* Total I/O and meta time consumed by fastest and slowest ranks */
- POSIX_F_FASTEST_RANK_TIME,
- POSIX_F_SLOWEST_RANK_TIME,
-#if 0
- F_VARIANCE_RANK_TIME,
- F_VARIANCE_RANK_BYTES,
-#endif
-
- POSIX_F_NUM_INDICES,
+ POSIX_F_COUNTERS
};
+#undef X
/* file record structure for POSIX files. a record is created and stored for
* every POSIX file opened by the original application. For the POSIX module,
@@ -127,4 +176,11 @@ struct darshan_posix_file
double fcounters[POSIX_F_NUM_INDICES];
};
+/* This macro can be used to identify files that have been opened using
+ * pnetcdf, hdf5, or mpi-io, but were never opened at the posix level. As a
+ * result the record will not necessarily have all of the expected fields
+ * populated.
+ */
+#define POSIX_FILE_PARTIAL(__file)((((__file)->counters[POSIX_OPENS] || (__file)->counters[POSIX_FOPENS] || (__file)->counters[POSIX_STATS]) ? 0 : 1))
+
#endif /* __DARSHAN_POSIX_LOG_FORMAT_H */
diff --git a/darshan-runtime/Makefile.in b/darshan-runtime/Makefile.in
index 8056d07..1acfa95 100644
--- a/darshan-runtime/Makefile.in
+++ b/darshan-runtime/Makefile.in
@@ -1,4 +1,4 @@
-all: lib/libdarshan.a lib/darshan-null.o
+all: lib/libdarshan.a lib/libdarshan-stubs.a lib/darshan-null.o
DESTDIR =
srcdir = @srcdir@
@@ -25,7 +25,7 @@ endif
VPATH = $(srcdir)
CFLAGS = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I ../ -I $(srcdir) -I$(srcdir)/../ @CFLAGS@ @CPPFLAGS@ -D_LARGEFILE64_SOURCE
-#
+
CFLAGS_SHARED = -DDARSHAN_CONFIG_H=\"darshan-runtime-config.h\" -I . -I$(srcdir) -I$(srcdir)/../ @CFLAGS@ @CPPFLAGS@ -D_LARGEFILE64_SOURCE -shared -fpic -DPIC -DDARSHAN_PRELOAD
LIBS = -lz @LIBBZ2@
@@ -78,6 +78,23 @@ lib/darshan-bgq.o: lib/darshan-bgq.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/.
lib/darshan-bgq.po: lib/darshan-bgq.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | lib
$(CC) $(CFLAGS_SHARED) -c $< -o $@
+lib/darshan-hdf5.o: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | lib
+ $(CC) $(CFLAGS) -c $< -o $@
+
+lib/darshan-hdf5.po: lib/darshan-hdf5.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | lib
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
+
+lib/darshan-hdf5-stubs.o: lib/darshan-hdf5-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib
+ $(CC) $(CFLAGS) -c $< -o $@
+
+lib/darshan-pnetcdf.o: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | lib
+ $(CC) $(CFLAGS) -c $< -o $@
+
+lib/darshan-pnetcdf.po: lib/darshan-pnetcdf.c darshan.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | lib
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
+
+lib/darshan-pnetcdf-stubs.o: lib/darshan-pnetcdf-stubs.c darshan.h $(DARSHAN_LOG_FORMAT) | lib
+ $(CC) $(CFLAGS) -c $< -o $@
lib/lookup3.o: lib/lookup3.c
$(CC) $(CFLAGS) -c $< -o $@
@@ -91,18 +108,20 @@ lib/lookup8.o: lib/lookup8.c
lib/lookup8.po: lib/lookup8.c
$(CC) $(CFLAGS_SHARED) -c $< -o $@
-#%.i: %.c
-# $(CC) -E $(CFLAGS) -c $< -o $@
-
-lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-bgq.o lib/lookup3.o lib/lookup8.o
+lib/libdarshan.a: lib/darshan-core-init-finalize.o lib/darshan-core.o lib/darshan-common.o lib/darshan-posix.o lib/darshan-mpiio.o lib/darshan-hdf5.o lib/darshan-pnetcdf.o lib/darshan-bgq.o lib/lookup3.o lib/lookup8.o
ar rcs $@ $^
-lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-bgq.po lib/lookup3.po lib/lookup8.po
+lib/libdarshan.so: lib/darshan-core-init-finalize.po lib/darshan-core.po lib/darshan-common.po lib/darshan-posix.po lib/darshan-mpiio.po lib/darshan-pnetcdf.po lib/darshan-bgq.po lib/lookup3.po lib/lookup8.po
$(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ -lpthread -lrt -lz -ldl
+lib/libdarshan-stubs.a: lib/darshan-hdf5-stubs.o lib/darshan-pnetcdf-stubs.o
+ ar rcs $@ $^
+
+
install:: all
install -d $(libdir)
install -m 755 lib/libdarshan.a $(libdir)
+ install -m 755 lib/libdarshan-stubs.a $(libdir)
ifndef DISABLE_LDPRELOAD
install -m 755 lib/libdarshan.so $(libdir)
endif
@@ -125,6 +144,8 @@ endif
install -m 755 share/mpi-profile/darshan-bg-f.conf $(datarootdir)/mpi-profile/darshan-bg-f.conf
install -m 644 darshan-base-ld-opts $(datarootdir)/darshan-base-ld-opts
install -m 644 $(srcdir)/darshan-posix-ld-opts $(datarootdir)/darshan-posix-ld-opts
+ install -m 644 $(srcdir)/darshan-hdf5-ld-opts $(datarootdir)/darshan-hdf5-ld-opts
+ install -m 644 $(srcdir)/darshan-pnetcdf-ld-opts $(datarootdir)/darshan-pnetcdf-ld-opts
install -d $(libdir)/pkgconfig
install -m 644 lib/pkgconfig/darshan-runtime.pc $(libdir)/pkgconfig/darshan-runtime.pc
diff --git a/darshan-runtime/darshan-base-ld-opts.in b/darshan-runtime/darshan-base-ld-opts.in
index 4f7fa6a..3f02f33 100644
--- a/darshan-runtime/darshan-base-ld-opts.in
+++ b/darshan-runtime/darshan-base-ld-opts.in
@@ -1 +1,3 @@
@@darshan_share_path@/darshan-posix-ld-opts
+@@darshan_share_path@/darshan-hdf5-ld-opts
+@@darshan_share_path@/darshan-pnetcdf-ld-opts
diff --git a/darshan-runtime/darshan-common.h b/darshan-runtime/darshan-common.h
new file mode 100644
index 0000000..4e42188
--- /dev/null
+++ b/darshan-runtime/darshan-common.h
@@ -0,0 +1,192 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_COMMON_H
+#define __DARSHAN_COMMON_H
+
+/* increment a timer counter, making sure not to account for overlap
+ * with previous operations
+ *
+ * NOTE: __timer is the corresponding timer counter variable, __tm1 is
+ * the start timestamp of the operation, __tm2 is the end timestamp of
+ * the operation, and __last is the timestamp of the end of the previous
+ * I/O operation (which we don't want to overlap with).
+ */
+#define DARSHAN_TIMER_INC_NO_OVERLAP(__timer, __tm1, __tm2, __last) do{ \
+ if(__tm1 > __last) \
+ __timer += (__tm2 - __tm1); \
+ else \
+ __timer += (__tm2 - __last); \
+ if(__tm2 > __last) \
+ __last = __tm2; \
+} while(0)
+
+/* increment histogram bucket depending on the given __value
+ *
+ * NOTE: This macro can be used to build a histogram of access
+ * sizes, offsets, etc. It assumes a 10-bucket histogram, with
+ * __bucket_base_p pointing to the first counter in the sequence
+ * of buckets (i.e., the smallest bucket). The size ranges of each
+ * bucket are:
+ * * 0 - 100 bytes
+ * * 100 - 1 KiB
+ * * 1 KiB - 10 KiB
+ * * 10 KiB - 100 KiB
+ * * 100 KiB - 1 MiB
+ * * 1 MiB - 4 MiB
+ * * 4 MiB - 10 MiB
+ * * 10 MiB - 100 MiB
+ * * 100 MiB - 1 GiB
+ * * 1 GiB+
+ */
+#define DARSHAN_BUCKET_INC(__bucket_base_p, __value) do {\
+ if(__value < 101) \
+ *(__bucket_base_p) += 1; \
+ else if(__value < 1025) \
+ *(__bucket_base_p + 1) += 1; \
+ else if(__value < 10241) \
+ *(__bucket_base_p + 2) += 1; \
+ else if(__value < 102401) \
+ *(__bucket_base_p + 3) += 1; \
+ else if(__value < 1048577) \
+ *(__bucket_base_p + 4) += 1; \
+ else if(__value < 4194305) \
+ *(__bucket_base_p + 5) += 1; \
+ else if(__value < 10485761) \
+ *(__bucket_base_p + 6) += 1; \
+ else if(__value < 104857601) \
+ *(__bucket_base_p + 7) += 1; \
+ else if(__value < 1073741825) \
+ *(__bucket_base_p + 8) += 1; \
+ else \
+ *(__bucket_base_p + 9) += 1; \
+} while(0)
+
+/* potentially set or increment a common value counter, depending on the __count
+ * for the given __value
+ *
+ * NOTE: This macro is hardcoded to expect that Darshan will only track the 4
+ * most common (i.e., frequently occuring) values. __val_p is a pointer to the
+ * base of the value counters (i.e., the first of 4 contiguous common value
+ * counters) and __cnt_p is a pointer to the base of the count counters (i.e.
+ * the first of 4 contiguous common count counters). It is assumed your counters
+ * are stored as int64_t types.
+ */
+#define DARSHAN_COMMON_VAL_COUNTER_INC(__val_p, __cnt_p, __value, __count) do {\
+ int i; \
+ int set = 0; \
+ int64_t min = *(__cnt_p); \
+ int min_index = 0; \
+ if(__value == 0) break; \
+ for(i=0; i<4; i++) { \
+ /* increment bucket if already exists */ \
+ if(*(__val_p + i) == __value) { \
+ *(__cnt_p + i) += __count; \
+ set = 1; \
+ break; \
+ } \
+ /* otherwise find the least frequently used bucket */ \
+ else if(*(__cnt_p + i) < min) { \
+ min = *(__cnt_p + i); \
+ min_index = i; \
+ } \
+ } \
+ if(!set && (__count > min)) { \
+ *(__cnt_p + min_index) = __count; \
+ *(__val_p + min_index) = __value; \
+ } \
+} while(0)
+
+/* maximum number of common values that darshan will track per file at
+ * runtime; at shutdown time these will be reduced to the 4 most
+ * frequently occuring ones
+ */
+#define DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT 32
+struct darshan_common_val_counter
+{
+ int64_t val;
+ int freq;
+};
+
+/* i/o type (read or write) */
+enum darshan_io_type
+{
+ DARSHAN_IO_READ = 1,
+ DARSHAN_IO_WRITE = 2,
+};
+
+/* struct used for calculating variances */
+struct darshan_variance_dt
+{
+ double n;
+ double T;
+ double S;
+};
+
+/***********************************************
+* darshan-common functions for darshan modules *
+***********************************************/
+
+/* darshan_clean_file_path()
+ *
+ * Allocate a new string that contains a new cleaned-up version of
+ * the file path given in 'path' argument. Converts relative paths
+ * to absolute paths and filters out some potential noise in the
+ * path string.
+ */
+char* darshan_clean_file_path(
+ const char* path);
+
+/* darshan_common_val_counter()
+ *
+ * Potentially increment an existing common value counter or allocate
+ * a new one to keep track of commonly occuring values. Example use
+ * cases would be to track the most frequent access sizes or strides
+ * used by a specific module, for instance. 'common_val_root' is the
+ * root pointer for the tree which stores common value info,
+ * 'common_val_count' is a pointer to the number of nodes in the
+ * tree (i.e., the number of allocated common value counters), and
+ * 'val' is the new value to attempt to add.
+ */
+void darshan_common_val_counter(
+ void** common_val_root,
+ int* common_val_count,
+ int64_t val);
+
+/* darshan_walk_common_vals()
+ *
+ * Walks the tree of common value counters and determines the 4 most
+ * frequently occuring values, storing the common values in the
+ * appropriate counter fields of the given record. 'common_val_root'
+ * is the root of the tree which stores the common value info, 'val_p'
+ * is a pointer to the base counter (i.e., the first) of the common
+ * values (which are assumed to be 4 total and contiguous in memory),
+ * and 'cnt_p' is a pointer to the base counter of the common counts
+ * (which are again expected to be contiguous in memory).
+ */
+void darshan_walk_common_vals(
+ void* common_val_root,
+ int64_t* val_p,
+ int64_t* cnt_p);
+
+/* darshan_variance_reduce()
+ *
+ * MPI reduction operation to calculate variances on counters in
+ * data records which are shared across all processes. This could
+ * be used, for instance, to find the variance in I/O time or total
+ * bytes moved for a given data record. This function needs to be
+ * passed to MPI_Op_create to obtain a corresponding MPI operation
+ * which can be used to complete the reduction. For more details,
+ * consult the documentation for MPI_Op_create. Example use cases
+ * can be found in the POSIX and MPIIO modules.
+ */
+void darshan_variance_reduce(
+ void *invec,
+ void *inoutvec,
+ int *len,
+ MPI_Datatype *dt);
+
+#endif /* __DARSHAN_COMMON_H */
diff --git a/darshan-runtime/darshan-core.h b/darshan-runtime/darshan-core.h
index 4f1cc83..7de647f 100644
--- a/darshan-runtime/darshan-core.h
+++ b/darshan-runtime/darshan-core.h
@@ -14,6 +14,19 @@
#include "uthash.h"
#include "darshan-log-format.h"
+/* TODO: drop CP_ and use DARSHAN_ */
+/* Environment variable to override CP_JOBID */
+#define CP_JOBID_OVERRIDE "DARSHAN_JOBID"
+
+/* Environment variable to override __CP_LOG_PATH */
+#define CP_LOG_PATH_OVERRIDE "DARSHAN_LOGPATH"
+
+/* Environment variable to override __CP_LOG_PATH */
+#define CP_LOG_HINTS_OVERRIDE "DARSHAN_LOGHINTS"
+
+/* Environment variable to override __CP_MEM_ALIGNMENT */
+#define CP_MEM_ALIGNMENT_OVERRIDE "DARSHAN_MEMALIGN"
+
#define DARSHAN_CORE_MAX_RECORDS 1024
/* TODO: revisit this default size if we change memory per module */
diff --git a/darshan-runtime/darshan-hdf5-ld-opts b/darshan-runtime/darshan-hdf5-ld-opts
new file mode 100644
index 0000000..0aafad8
--- /dev/null
+++ b/darshan-runtime/darshan-hdf5-ld-opts
@@ -0,0 +1,3 @@
+--wrap=H5Fcreate
+--wrap=H5Fopen
+--wrap=H5Fclose
diff --git a/darshan-runtime/darshan-pnetcdf-ld-opts b/darshan-runtime/darshan-pnetcdf-ld-opts
new file mode 100644
index 0000000..135f4a5
--- /dev/null
+++ b/darshan-runtime/darshan-pnetcdf-ld-opts
@@ -0,0 +1,3 @@
+--wrap=ncmpi_create
+--wrap=ncmpi_open
+--wrap=ncmpi_close
diff --git a/darshan-runtime/darshan-posix-ld-opts b/darshan-runtime/darshan-posix-ld-opts
index 4521894..1f969ad 100644
--- a/darshan-runtime/darshan-posix-ld-opts
+++ b/darshan-runtime/darshan-posix-ld-opts
@@ -33,3 +33,11 @@
--wrap=fdatasync
--wrap=close
--wrap=fclose
+--wrap=aio_read
+--wrap=aio_write
+--wrap=aio_read64
+--wrap=aio_write64
+--wrap=aio_return
+--wrap=aio_return64
+--wrap=lio_listio
+--wrap=lio_listio64
diff --git a/darshan-runtime/darshan.h b/darshan-runtime/darshan.h
index 6016d24..d6aacca 100644
--- a/darshan-runtime/darshan.h
+++ b/darshan-runtime/darshan.h
@@ -13,18 +13,7 @@
#include <mpi.h>
#include "darshan-log-format.h"
-
-/* Environment variable to override CP_JOBID */
-#define CP_JOBID_OVERRIDE "DARSHAN_JOBID"
-
-/* Environment variable to override __CP_LOG_PATH */
-#define CP_LOG_PATH_OVERRIDE "DARSHAN_LOGPATH"
-
-/* Environment variable to override __CP_LOG_PATH */
-#define CP_LOG_HINTS_OVERRIDE "DARSHAN_LOGHINTS"
-
-/* Environment variable to override __CP_MEM_ALIGNMENT */
-#define CP_MEM_ALIGNMENT_OVERRIDE "DARSHAN_MEMALIGN"
+#include "darshan-common.h"
/* macros for declaring wrapper functions and calling MPI routines
* consistently regardless of whether static or dynamic linking is used
@@ -64,101 +53,28 @@
#endif
-/* macros for manipulating a module's counter variables */
-/* NOTE: These macros assume a module's record stores integer
- * and floating point counters in arrays, named counters and
- * fcounters, respectively. __rec_p is the a pointer to the
- * data record, __counter is the counter in question, and
- * __value is the corresponding data value.
- */
-#define DARSHAN_COUNTER_SET(__rec_p, __counter, __value) do{ \
- (__rec_p)->counters[__counter] = __value; \
-} while(0)
-
-#define DARSHAN_COUNTER_F_SET(__rec_p, __counter, __value) do{ \
- (__rec_p)->fcounters[__counter] = __value; \
-} while(0)
-
-#define DARSHAN_COUNTER_INC(__rec_p, __counter, __value) do{ \
- (__rec_p)->counters[__counter] += __value; \
-} while(0)
-
-#define DARSHAN_COUNTER_F_INC(__rec_p, __counter, __value) do{ \
- (__rec_p)->fcounters[__counter] += __value; \
-} while(0)
-
-#define DARSHAN_COUNTER_F_INC_NO_OVERLAP(__rec_p, __tm1, __tm2, __last, __counter) do{ \
- if(__tm1 > __last) \
- DARSHAN_COUNTER_F_INC(__rec_p, __counter, (__tm2 - __tm1)); \
- else \
- DARSHAN_COUNTER_F_INC(__rec_p, __counter, (__tm2 - __last)); \
- if(__tm2 > __last) \
- __last = __tm2; \
-} while(0)
-
-#define DARSHAN_COUNTER_VALUE(__rec_p, __counter) \
- ((__rec_p)->counters[__counter])
-
-#define DARSHAN_COUNTER_F_VALUE(__rec_p, __counter) \
- ((__rec_p)->fcounters[__counter])
-
-#define DARSHAN_COUNTER_MAX(__rec_p, __counter, __value) do{ \
- if((__rec_p)->counters[__counter] < __value) \
- (__rec_p)->counters[__counter] = __value; \
-} while(0)
-
-/* NOTE: This macro is for storing histogram counters with 10
- * distinct buckets. For an example of how it is used, consult
- * the POSIX module implementation, which stores histograms of
- * access sizes for POSIX I/O functions.
- */
-#define DARSHAN_BUCKET_INC(__rec_p, __counter_base, __value) do {\
- if(__value < 101) \
- (__rec_p)->counters[__counter_base] += 1; \
- else if(__value < 1025) \
- (__rec_p)->counters[__counter_base+1] += 1; \
- else if(__value < 10241) \
- (__rec_p)->counters[__counter_base+2] += 1; \
- else if(__value < 102401) \
- (__rec_p)->counters[__counter_base+3] += 1; \
- else if(__value < 1048577) \
- (__rec_p)->counters[__counter_base+4] += 1; \
- else if(__value < 4194305) \
- (__rec_p)->counters[__counter_base+5] += 1; \
- else if(__value < 10485761) \
- (__rec_p)->counters[__counter_base+6] += 1; \
- else if(__value < 104857601) \
- (__rec_p)->counters[__counter_base+7] += 1; \
- else if(__value < 1073741825) \
- (__rec_p)->counters[__counter_base+8] += 1; \
- else \
- (__rec_p)->counters[__counter_base+9] += 1; \
-} while(0)
-
/* module developers provide the following functions to darshan-core */
struct darshan_module_funcs
{
- /* perform any necessary pre-shutdown steps */
+ /* perform any necessary pre-shutdown steps
+ *
+ * NOTE: this typically includes disabling wrapper functions so
+ * darshan-core can shutdown in a consistent state.
+ */
void (*begin_shutdown)(void);
- /* perform any necessary steps prior to reducing */
- void (*setup_reduction)(
- darshan_record_id *shared_recs, /* input list of shared records */
- int *shared_rec_count, /* in/out shared record count */
- void **send_buf, /* send buffer for shared file reduction */
- void **recv_buf, /* recv buffer for shared file reduction (root only) */
- int *rec_size /* size of records being stored for this module */
- );
- /* reduce records which are shared globally across this module */
- void (*record_reduction_op)(
- void* infile_v,
- void* inoutfile_v,
- int *len,
- MPI_Datatype *datatype
- );
- /* retrieve module data to write to log file */
+ /* retrieve module data to write to log file
+ *
+ * NOTE: module developers can use this function to run collective
+ * MPI operations at shutdown time. Typically this functionality
+ * has been used to reduce records shared globablly (given in the
+ * 'shared_recs' array) into a single data record.
+ */
void (*get_output_data)(
- void** buf, /* output parameter to save module buffer address */
- int* size /* output parameter to save module buffer size */
+ MPI_Comm mod_comm, /* MPI communicator to run collectives with */
+ darshan_record_id *shared_recs, /* list of shared data record ids */
+ int shared_rec_count, /* count of shared data records */
+ void** mod_buf, /* output parameter to save module buffer address */
+ int* mod_buf_sz /* output parameter to save module buffer size */
);
/* shutdown module data structures */
void (*shutdown)(void);
@@ -171,15 +87,44 @@ extern char* darshan_path_exclusions[]; /* defined in lib/darshan-core.c */
* darshan-core functions exported to darshan modules *
*****************************************************/
+/* darshan_core_register_module()
+ *
+ * Register module identifier 'mod_id' with the darshan-core runtime
+ * environment, allowing the module to store I/O characterization data.
+ * 'funcs' is a pointer to a structure containing each of the function
+ * pointers required by darshan-core to shut down the module. The function
+ * returns the following integers passed in as pointers: 'my_rank' is the
+ * MPI rank of the calling process, 'mod_mem_limit' is the maximum amount
+ * of memory the module may use, and 'sys_mem_alignment' is the configured
+ * memory alignment value Darshan was configured with.
+ */
void darshan_core_register_module(
darshan_module_id mod_id,
struct darshan_module_funcs *funcs,
+ int *my_rank,
int *mod_mem_limit,
int *sys_mem_alignment);
+/* darshan_core_unregister_module()
+ *
+ * Unregisters module identifier 'mod_id' with the darshan-core runtime,
+ * removing the given module from the resulting I/O characterization log.
+ */
void darshan_core_unregister_module(
darshan_module_id mod_id);
+/* darshan_core_register_record()
+ *
+ * Register the Darshan record given by 'name' with the darshan-core
+ * runtime, allowing it to be properly tracked and (potentially)
+ * correlated with records from other modules. 'len' is the size of
+ * the name pointer (string length for string names), 'printable_flag'
+ * indicates whether the name is a string, and 'mod_id' is the identifier
+ * of the calling module. 'rec_id' is an output pointer storing the
+ * correspoing Darshan record identifier and 'file_alignment' is an
+ * output pointer storing the file system alignment value for the given
+ * record.
+ */
void darshan_core_register_record(
void *name,
int len,
@@ -188,16 +133,22 @@ void darshan_core_register_record(
darshan_record_id *rec_id,
int *file_alignment);
+/* darshan_core_unregister_record()
+ *
+ * Unregister record identifier 'rec_id' in the darshan-core runtime.
+ * This unregister is only in the context of module identifier 'mod_id',
+ * meaning that if the file record has other module's associated with
+ * it, then the record won't be completely removed.
+ */
void darshan_core_unregister_record(
darshan_record_id rec_id,
darshan_module_id mod_id);
+/* darshan_core_wtime()
+ *
+ * Returns the elapsed time relative to (roughly) the start of
+ * the application.
+ */
double darshan_core_wtime(void);
-/***********************************************
-* darshan-common functions for darshan modules *
-***********************************************/
-
-char* darshan_clean_file_path(const char* path);
-
#endif /* __DARSHAN_H */
diff --git a/darshan-runtime/lib/darshan-bgq.c b/darshan-runtime/lib/darshan-bgq.c
index ea0a2cb..a7bca70 100644
--- a/darshan-runtime/lib/darshan-bgq.c
+++ b/darshan-runtime/lib/darshan-bgq.c
@@ -49,13 +49,14 @@ static int instrumentation_disabled = 0;
/* my_rank indicates the MPI rank of this process */
static int my_rank = -1;
+static int darshan_mem_alignment = 1;
/* internal helper functions for the "NULL" module */
void bgq_runtime_initialize(void);
/* forward declaration for module functions needed to interface with darshan-core */
static void bgq_begin_shutdown(void);
-static void bgq_get_output_data(void **buffer, int *size);
+static void bgq_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs, int shared_rec_count, void **buffer, int *size);
static void bgq_shutdown(void);
static void bgq_setup_reduction(darshan_record_id *shared_recs,int *shared_rec_count,void **send_buf,void **recv_buf,int *rec_size);
static void bgq_record_reduction_op(void* infile_v,void* inoutfile_v,int *len,MPI_Datatype *datatype);
@@ -75,8 +76,7 @@ static void capture(struct darshan_bgq_record *rec)
rec->counters[BGQ_CSJOBID] = Kernel_GetJobID();
rec->counters[BGQ_RANKSPERNODE] = Kernel_ProcessCount();
-
- rec->counters[BGQ_INODES] = MPIX_IO_node();
+ rec->counters[BGQ_INODES] = MPIX_IO_node_id();
r = Kernel_GetPersonality(&person, sizeof(person));
if (r == 0)
@@ -88,11 +88,11 @@ static void capture(struct darshan_bgq_record *rec)
rec->counters[BGQ_DNODES] = person.Network_Config.Dnodes;
rec->counters[BGQ_ENODES] = person.Network_Config.Enodes;
rec->counters[BGQ_TORUSENABLED] =
- ((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_A) << 0) |
- ((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_B) << 1) |
- ((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_C) << 2) |
- ((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_D) << 3) |
- ((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_E) << 4);
+ (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_A) == ND_ENABLE_TORUS_DIM_A) << 0) |
+ (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_B) == ND_ENABLE_TORUS_DIM_B) << 1) |
+ (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_C) == ND_ENABLE_TORUS_DIM_C) << 2) |
+ (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_D) == ND_ENABLE_TORUS_DIM_D) << 3) |
+ (((person.Network_Config.NetFlags & ND_ENABLE_TORUS_DIM_E) == ND_ENABLE_TORUS_DIM_E) << 4);
rec->counters[BGQ_DDRPERNODE] = person.DDR_Config.DDRSizeMB;
}
@@ -114,8 +114,6 @@ void bgq_runtime_initialize()
struct darshan_module_funcs bgq_mod_fns =
{
.begin_shutdown = bgq_begin_shutdown,
- .setup_reduction = bgq_setup_reduction,
- .record_reduction_op = bgq_record_reduction_op,
.get_output_data = bgq_get_output_data,
.shutdown = bgq_shutdown
};
@@ -132,8 +130,9 @@ void bgq_runtime_initialize()
darshan_core_register_module(
DARSHAN_BGQ_MOD,
&bgq_mod_fns,
+ &my_rank,
&mem_limit,
- NULL);
+ &darshan_mem_alignment);
/* return if no memory assigned by darshan-core */
if(mem_limit == 0)
@@ -162,7 +161,7 @@ void bgq_runtime_initialize()
recname,
strlen(recname),
1,
- DARSHAN_POSIX_MOD,
+ DARSHAN_BGQ_MOD,
&bgq_runtime->record.f_id,
&bgq_runtime->record.alignment);
@@ -192,8 +191,18 @@ static void bgq_begin_shutdown()
return;
}
+static int cmpr(const void *p1, const void *p2)
+{
+ const uint64_t *a = (uint64_t*) p1;
+ const uint64_t *b = (uint64_t*) p2;
+ return ((*a == *b) ? 0 : ((*a < *b) ? -1 : 1));
+}
+
/* Pass output data for the "BGQ" module back to darshan-core to log to file. */
static void bgq_get_output_data(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
void **buffer,
int *size)
{
@@ -202,6 +211,46 @@ static void bgq_get_output_data(
* I/O records, and set the output size according to the number of records
* currently being tracked.
*/
+ int nprocs;
+ int result;
+ uint64_t *ion_ids;
+
+ if (my_rank == 0)
+ {
+ DARSHAN_MPI_CALL(MPI_Comm_size)(mod_comm, &nprocs);
+ ion_ids = malloc(sizeof(*ion_ids)*nprocs);
+ result = (ion_ids != NULL);
+ }
+ DARSHAN_MPI_CALL(MPI_Bcast)(&result, 1, MPI_INT, 0, mod_comm);
+
+ if (bgq_runtime && result)
+ {
+ int i, found;
+ uint64_t val;
+
+ DARSHAN_MPI_CALL(MPI_Gather)(&bgq_runtime->record.counters[BGQ_INODES],
+ 1,
+ MPI_LONG_LONG_INT,
+ ion_ids,
+ 1,
+ MPI_LONG_LONG_INT,
+ 0,
+ mod_comm);
+ if (my_rank == 0)
+ {
+ qsort(ion_ids, nprocs, sizeof(*ion_ids), cmpr);
+ for (i = 1, val = ion_ids[0], found = 1; i < nprocs; i++)
+ {
+ if (val != ion_ids[i])
+ {
+ val = ion_ids[i];
+ found += 1;
+ }
+ }
+ bgq_runtime->record.counters[BGQ_INODES] = found;
+ }
+ }
+
if ((bgq_runtime) && (my_rank == 0))
{
*buffer = &bgq_runtime->record;
@@ -228,37 +277,6 @@ static void bgq_shutdown()
return;
}
-static void bgq_setup_reduction(
- darshan_record_id *shared_recs,
- int *shared_rec_count,
- void **send_buf,
- void **recv_buf,
- int *rec_size)
-{
- int i;
- int found;
-
- for (i = 0; i < *shared_rec_count; i++)
- {
- if (shared_recs[i] == bgq_runtime->record.f_id)
- {
- found = 1;
- break;
- }
- }
-
- if (found)
- {
- printf("found bgq shared record\n");
- *rec_size = sizeof(struct darshan_bgq_record);
- *shared_rec_count = 1;
- *send_buf = &bgq_runtime->record;
- *recv_buf = &bgq_runtime->record;
- }
-
- return;
-}
-
static void bgq_record_reduction_op(
void* infile_v,
void* inoutfile_v,
diff --git a/darshan-runtime/lib/darshan-common.c b/darshan-runtime/lib/darshan-common.c
index e9ce0d0..6f0468d 100644
--- a/darshan-runtime/lib/darshan-common.c
+++ b/darshan-runtime/lib/darshan-common.c
@@ -3,6 +3,8 @@
* See COPYRIGHT notice in top-level directory.
*
*/
+#define _XOPEN_SOURCE 500
+#define _GNU_SOURCE
#include "darshan-runtime-config.h"
@@ -10,13 +12,15 @@
#include <stdlib.h>
#include <string.h>
#include <limits.h>
+#include <search.h>
+#include <assert.h>
#include "darshan.h"
-/* Allocate a new string that contains a cleaned-up version of the path
- * passed in as an argument. Converts relative paths to absolute paths and
- * filters out some potential noise in the path string.
- */
+static int darshan_common_val_compare(const void* a_p, const void* b_p);
+static void darshan_common_val_walker(const void* nodep, const VISIT which,
+ const int depth);
+
char* darshan_clean_file_path(const char* path)
{
char* newpath = NULL;
@@ -74,6 +78,122 @@ char* darshan_clean_file_path(const char* path)
return(newpath);
}
+/* HACK: global variables for determining 4 most common values */
+static int64_t* walker_val_p = NULL;
+static int64_t* walker_cnt_p = NULL;
+
+void darshan_common_val_counter(void **common_val_root, int *common_val_count,
+ int64_t val)
+{
+ struct darshan_common_val_counter* counter;
+ struct darshan_common_val_counter* found;
+ struct darshan_common_val_counter tmp_counter;
+ void* tmp;
+
+ /* don't count any values of 0 */
+ if(val == 0)
+ return;
+
+ /* check to see if this val is already recorded */
+ tmp_counter.val = val;
+ tmp_counter.freq = 1;
+ tmp = tfind(&tmp_counter, common_val_root, darshan_common_val_compare);
+ if(tmp)
+ {
+ found = *(struct darshan_common_val_counter**)tmp;
+ found->freq++;
+ return;
+ }
+
+ /* we can add a new one as long as we haven't hit the limit */
+ if(*common_val_count < DARSHAN_COMMON_VAL_MAX_RUNTIME_COUNT)
+ {
+ counter = malloc(sizeof(*counter));
+ if(!counter)
+ {
+ return;
+ }
+
+ counter->val = val;
+ counter->freq = 1;
+
+ tmp = tsearch(counter, common_val_root, darshan_common_val_compare);
+ found = *(struct darshan_common_val_counter**)tmp;
+ /* if we get a new answer out here we are in trouble; this was
+ * already checked with the tfind()
+ */
+ assert(found == counter);
+
+ (*common_val_count)++;
+ }
+
+ return;
+}
+
+void darshan_walk_common_vals(void *common_val_root, int64_t* val_p,
+ int64_t* cnt_p)
+{
+ walker_val_p = val_p;
+ walker_cnt_p = cnt_p;
+
+ twalk(common_val_root, darshan_common_val_walker);
+ tdestroy(common_val_root, free);
+
+ return;
+}
+
+static void darshan_common_val_walker(const void* nodep, const VISIT which,
+ const int depth)
+{
+ struct darshan_common_val_counter* counter;
+
+ switch (which)
+ {
+ case postorder:
+ case leaf:
+ counter = *(struct darshan_common_val_counter**)nodep;
+ DARSHAN_COMMON_VAL_COUNTER_INC(walker_val_p, walker_cnt_p,
+ counter->val, counter->freq);
+ default:
+ break;
+ }
+
+ return;
+}
+
+static int darshan_common_val_compare(const void* a_p, const void* b_p)
+{
+ const struct darshan_common_val_counter* a = a_p;
+ const struct darshan_common_val_counter* b = b_p;
+
+ if(a->val < b->val)
+ return(-1);
+ if(a->val > b->val)
+ return(1);
+ return(0);
+}
+
+void darshan_variance_reduce(void *invec, void *inoutvec, int *len,
+ MPI_Datatype *dt)
+{
+ int i;
+ struct darshan_variance_dt *X = invec;
+ struct darshan_variance_dt *Y = inoutvec;
+ struct darshan_variance_dt Z;
+
+ for (i=0; i<*len; i++,X++,Y++)
+ {
+ Z.n = X->n + Y->n;
+ Z.T = X->T + Y->T;
+ Z.S = X->S + Y->S + (X->n/(Y->n*Z.n)) *
+ ((Y->n/X->n)*X->T - Y->T) * ((Y->n/X->n)*X->T - Y->T);
+
+ *Y = Z;
+ }
+
+ return;
+}
+
/*
* Local variables:
* c-indent-level: 4
diff --git a/darshan-runtime/lib/darshan-core.c b/darshan-runtime/lib/darshan-core.c
index 9cde110..07bb151 100644
--- a/darshan-runtime/lib/darshan-core.c
+++ b/darshan-runtime/lib/darshan-core.c
@@ -88,14 +88,14 @@ static void darshan_get_shared_records(
static int darshan_log_open_all(
char *logfile_name, MPI_File *log_fh);
static int darshan_deflate_buffer(
- void **pointers, int *lengths, int count, int nocomp_flag,
- char *comp_buf, int *comp_length);
+ void **pointers, int *lengths, int count, char *comp_buf,
+ int *comp_buf_length);
static int darshan_log_write_record_hash(
MPI_File log_fh, struct darshan_core_runtime *core,
uint64_t *inout_off);
static int darshan_log_append_all(
MPI_File log_fh, struct darshan_core_runtime *core, void *buf,
- int count, uint64_t *inout_off, uint64_t *agg_uncomp_sz);
+ int count, uint64_t *inout_off);
static void darshan_core_cleanup(
struct darshan_core_runtime* core);
@@ -243,7 +243,6 @@ void darshan_core_shutdown()
double header1, header2;
double tm_end;
uint64_t gz_fp = 0;
- uint64_t tmp_off = 0;
MPI_File log_fh;
MPI_Status status;
@@ -381,11 +380,11 @@ void darshan_core_shutdown()
if(my_rank == 0)
{
void *pointers[2] = {&final_core->log_job, final_core->trailing_data};
- int lengths[2] = {sizeof(struct darshan_job), DARSHAN_EXE_LEN+1};
+ int lengths[2] = {sizeof(struct darshan_job), strlen(final_core->trailing_data)};
int comp_buf_sz = 0;
/* compress the job info and the trailing mount/exe data */
- all_ret = darshan_deflate_buffer(pointers, lengths, 2, 0,
+ all_ret = darshan_deflate_buffer(pointers, lengths, 2,
final_core->comp_buf, &comp_buf_sz);
if(all_ret)
{
@@ -395,7 +394,7 @@ void darshan_core_shutdown()
else
{
/* write the job information, preallocing space for the log header */
- gz_fp += sizeof(struct darshan_header) + 23; /* gzip headers/trailers ... */
+ gz_fp += sizeof(struct darshan_header);
all_ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(log_fh, gz_fp,
final_core->comp_buf, comp_buf_sz, MPI_BYTE, &status);
if(all_ret != MPI_SUCCESS)
@@ -405,8 +404,6 @@ void darshan_core_shutdown()
unlink(logfile_name);
}
-
- /* set the beginning offset of record hash, which follows job info just written */
gz_fp += comp_buf_sz;
}
}
@@ -425,8 +422,9 @@ void darshan_core_shutdown()
if(internal_timing_flag)
rec1 = DARSHAN_MPI_CALL(PMPI_Wtime)();
/* write the record name->id hash to the log file */
+ final_core->log_header.rec_map.off = gz_fp;
ret = darshan_log_write_record_hash(log_fh, final_core, &gz_fp);
- tmp_off = final_core->log_header.rec_map.off + final_core->log_header.rec_map.len;
+ final_core->log_header.rec_map.len = gz_fp - final_core->log_header.rec_map.off;
/* error out if unable to write record hash */
DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,
@@ -457,8 +455,9 @@ void darshan_core_shutdown()
for(i = 0; i < DARSHAN_MAX_MODS; i++)
{
struct darshan_core_module* this_mod = final_core->mod_array[i];
- darshan_record_id mod_shared_recs[DARSHAN_CORE_MAX_RECORDS];
struct darshan_core_record_ref *ref = NULL;
+ darshan_record_id mod_shared_recs[DARSHAN_CORE_MAX_RECORDS];
+ int mod_shared_rec_cnt = 0;
void* mod_buf = NULL;
int mod_buf_sz = 0;
int j;
@@ -474,71 +473,37 @@ void darshan_core_shutdown()
}
if(internal_timing_flag)
- mod1[i] = DARSHAN_MPI_CALL(PMPI_Wtime)();
- /* if all processes used this module, prepare to do a shared file reduction */
- if(global_mod_use_count[i] == nprocs)
- {
- int shared_rec_count = 0;
- int rec_sz = 0;
- void *red_send_buf = NULL, *red_recv_buf = NULL;
- MPI_Datatype red_type;
- MPI_Op red_op;
-
- /* set the shared file list for this module */
- memset(mod_shared_recs, 0, DARSHAN_CORE_MAX_RECORDS * sizeof(darshan_record_id));
- for(j = 0; j < DARSHAN_CORE_MAX_RECORDS && shared_recs[j] != 0; j++)
- {
- HASH_FIND(hlink, final_core->rec_hash, &shared_recs[j],
- sizeof(darshan_record_id), ref);
- assert(ref);
- if(DARSHAN_CORE_MOD_ISSET(ref->global_mod_flags, i))
- {
- mod_shared_recs[shared_rec_count++] = shared_recs[j];
- }
- }
+ mod1[i] = DARSHAN_MPI_CALL(PMPI_Wtime)();
- /* if there are globally shared files, do a shared file reduction */
- if(shared_rec_count && this_mod->mod_funcs.setup_reduction &&
- this_mod->mod_funcs.record_reduction_op)
+ /* set the shared file list for this module */
+ memset(mod_shared_recs, 0, DARSHAN_CORE_MAX_RECORDS * sizeof(darshan_record_id));
+ for(j = 0; j < DARSHAN_CORE_MAX_RECORDS && shared_recs[j] != 0; j++)
+ {
+ HASH_FIND(hlink, final_core->rec_hash, &shared_recs[j],
+ sizeof(darshan_record_id), ref);
+ assert(ref);
+ if(DARSHAN_CORE_MOD_ISSET(ref->global_mod_flags, i))
{
- this_mod->mod_funcs.setup_reduction(mod_shared_recs, &shared_rec_count,
- &red_send_buf, &red_recv_buf, &rec_sz);
-
- if(shared_rec_count)
- {
- /* construct a datatype for a file record. This is serving no purpose
- * except to make sure we can do a reduction on proper boundaries
- */
- DARSHAN_MPI_CALL(PMPI_Type_contiguous)(rec_sz, MPI_BYTE, &red_type);
- DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);
-
- /* register a reduction operator for this module */
- DARSHAN_MPI_CALL(PMPI_Op_create)(this_mod->mod_funcs.record_reduction_op,
- 1, &red_op);
-
- /* reduce shared file records for this module */
- DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,
- shared_rec_count, red_type, red_op, 0, MPI_COMM_WORLD);
-
- DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);
- DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);
- }
+ mod_shared_recs[mod_shared_rec_cnt++] = shared_recs[j];
}
}
- /* if module is registered locally, get the corresponding output buffer */
+ /* if module is registered locally, get the corresponding output buffer
+ *
+ * NOTE: this function can be used to run collective operations across
+ * modules, if there are file records shared globally.
+ */
if(this_mod)
{
- /* get output buffer from module */
- this_mod->mod_funcs.get_output_data(&mod_buf, &mod_buf_sz);
+ this_mod->mod_funcs.get_output_data(MPI_COMM_WORLD, mod_shared_recs,
+ mod_shared_rec_cnt, &mod_buf, &mod_buf_sz);
}
- final_core->log_header.mod_map[i].off = tmp_off;
-
/* append this module's data to the darshan log */
- ret = darshan_log_append_all(log_fh, final_core, mod_buf, mod_buf_sz,
- &gz_fp, &(final_core->log_header.mod_map[i].len));
- tmp_off += final_core->log_header.mod_map[i].len;
+ final_core->log_header.mod_map[i].off = gz_fp;
+ ret = darshan_log_append_all(log_fh, final_core, mod_buf, mod_buf_sz, &gz_fp);
+ final_core->log_header.mod_map[i].len =
+ gz_fp - final_core->log_header.mod_map[i].off;
/* error out if the log append failed */
DARSHAN_MPI_CALL(PMPI_Allreduce)(&ret, &all_ret, 1, MPI_INT,
@@ -571,39 +536,19 @@ void darshan_core_shutdown()
/* rank 0 is responsible for writing the log header */
if(my_rank == 0)
{
- void *header_buf = &(final_core->log_header);
- int header_buf_sz = sizeof(struct darshan_header);
- int comp_buf_sz = 0;
-
/* initialize the remaining header fields */
strcpy(final_core->log_header.version_string, DARSHAN_LOG_VERSION);
final_core->log_header.magic_nr = DARSHAN_MAGIC_NR;
+ final_core->log_header.comp_type = DARSHAN_ZLIB_COMP;
- /* deflate the header */
- /* NOTE: the header is not actually compressed because space for it must
- * be preallocated before writing. i.e., the "compressed" header
- * must be constant sized, sizeof(struct darshan_header) + 23.
- * it is still necessary to deflate the header or the resulting log
- * file will not be a valid gzip file.
- */
- all_ret = darshan_deflate_buffer((void **)&header_buf, &header_buf_sz, 1, 1,
- final_core->comp_buf, &comp_buf_sz);
- if(all_ret)
+ all_ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(log_fh, 0, &(final_core->log_header),
+ sizeof(struct darshan_header), MPI_BYTE, &status);
+ if(all_ret != MPI_SUCCESS)
{
- fprintf(stderr, "darshan library warning: unable to compress header\n");
+ fprintf(stderr, "darshan library warning: unable to write header to log file %s\n",
+ logfile_name);
unlink(logfile_name);
}
- else
- {
- all_ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(log_fh, 0, final_core->comp_buf,
- comp_buf_sz, MPI_BYTE, &status);
- if(all_ret != MPI_SUCCESS)
- {
- fprintf(stderr, "darshan library warning: unable to write header to log file %s\n",
- logfile_name);
- unlink(logfile_name);
- }
- }
}
/* error out if unable to write log header */
@@ -620,7 +565,7 @@ void darshan_core_shutdown()
DARSHAN_MPI_CALL(PMPI_File_close)(&log_fh);
/* if we got this far, there are no errors, so rename from *.darshan_partial
- * to *-<logwritetime>.darshan.gz, which indicates that this log file is
+ * to *-<logwritetime>.darshan, which indicates that this log file is
* complete and ready for analysis
*/
if(my_rank == 0)
@@ -646,7 +591,7 @@ void darshan_core_shutdown()
end_log_time = DARSHAN_MPI_CALL(PMPI_Wtime)();
strcat(new_logfile_name, logfile_name);
tmp_index = strstr(new_logfile_name, ".darshan_partial");
- sprintf(tmp_index, "_%d.darshan.gz", (int)(end_log_time-start_log_time+1));
+ sprintf(tmp_index, "_%d.darshan", (int)(end_log_time-start_log_time+1));
rename(logfile_name, new_logfile_name);
/* set permissions on log file */
#ifdef __CP_GROUP_READABLE_LOGS
@@ -1145,7 +1090,6 @@ static void darshan_get_shared_records(struct darshan_core_runtime *core,
{
/* we opened that record too, save the mod_flags */
mod_flags[i] = ref->mod_flags;
- break;
}
}
@@ -1237,12 +1181,11 @@ static int darshan_log_open_all(char *logfile_name, MPI_File *log_fh)
}
static int darshan_deflate_buffer(void **pointers, int *lengths, int count,
- int nocomp_flag, char *comp_buf, int *comp_length)
+ char *comp_buf, int *comp_buf_length)
{
int ret = 0;
int i;
int total_target = 0;
- int z_comp_level;
z_stream tmp_stream;
/* just return if there is no data */
@@ -1256,7 +1199,7 @@ static int darshan_deflate_buffer(void **pointers, int *lengths, int count,
}
else
{
- *comp_length = 0;
+ *comp_buf_length = 0;
return(0);
}
@@ -1267,10 +1210,9 @@ static int darshan_deflate_buffer(void **pointers, int *lengths, int count,
/* initialize the zlib compression parameters */
/* TODO: check these parameters? */
- z_comp_level = nocomp_flag ? Z_NO_COMPRESSION : Z_DEFAULT_COMPRESSION;
- ret = deflateInit2(&tmp_stream, z_comp_level, Z_DEFLATED,
- 15 + 16, 8, Z_DEFAULT_STRATEGY);
-// ret = deflateInit(&tmp_stream, z_comp_level);
+// ret = deflateInit2(&tmp_stream, Z_DEFAULT_COMPRESSION, Z_DEFLATED,
+// 15 + 16, 8, Z_DEFAULT_STRATEGY);
+ ret = deflateInit(&tmp_stream, Z_DEFAULT_COMPRESSION);
if(ret != Z_OK)
{
return(-1);
@@ -1320,7 +1262,7 @@ static int darshan_deflate_buffer(void **pointers, int *lengths, int count,
}
deflateEnd(&tmp_stream);
- *comp_length = tmp_stream.total_out;
+ *comp_buf_length = tmp_stream.total_out;
return(0);
}
@@ -1390,15 +1332,10 @@ static int darshan_log_write_record_hash(MPI_File log_fh, struct darshan_core_ru
memcpy(hash_buf_off, ref->rec.name, name_len);
hash_buf_off += name_len;
}
-
- /* store uncompressed offset of the record hash */
- core->log_header.rec_map.off = sizeof(struct darshan_header) +
- DARSHAN_JOB_RECORD_SIZE;
+ hash_buf_sz = hash_buf_off - hash_buf;
/* collectively write out the record hash to the darshan log */
- hash_buf_sz = hash_buf_off - hash_buf;
- ret = darshan_log_append_all(log_fh, core, hash_buf, hash_buf_sz,
- inout_off, &(core->log_header.rec_map.len));
+ ret = darshan_log_append_all(log_fh, core, hash_buf, hash_buf_sz, inout_off);
free(hash_buf);
@@ -1407,22 +1344,18 @@ static int darshan_log_write_record_hash(MPI_File log_fh, struct darshan_core_ru
/* NOTE: inout_off contains the starting offset of this append at the beginning
* of the call, and contains the ending offset at the end of the call.
- * total_uncomp_sz will store the collective size of the uncompressed
- * buffer being written. This data is necessary to properly index data
- * when reading a darshan log. Also, these variables should only be valid
- * on the root rank (rank 0).
+ * This variable is only valid on the root rank (rank 0).
*/
static int darshan_log_append_all(MPI_File log_fh, struct darshan_core_runtime *core,
- void *buf, int count, uint64_t *inout_off, uint64_t *agg_uncomp_sz)
+ void *buf, int count, uint64_t *inout_off)
{
MPI_Offset send_off, my_off;
MPI_Status status;
- uint64_t uncomp_buf_sz = count;
int comp_buf_sz = 0;
int ret;
/* compress the input buffer */
- ret = darshan_deflate_buffer((void **)&buf, &count, 1, 0,
+ ret = darshan_deflate_buffer((void **)&buf, &count, 1,
core->comp_buf, &comp_buf_sz);
if(ret < 0)
comp_buf_sz = 0;
@@ -1454,16 +1387,16 @@ static int darshan_log_append_all(MPI_File log_fh, struct darshan_core_runtime *
core->comp_buf, comp_buf_sz, MPI_BYTE, &status);
}
- /* send the ending offset from rank (n-1) to rank 0 */
if(nprocs > 1)
{
+ /* send the ending offset from rank (n-1) to rank 0 */
if(my_rank == (nprocs-1))
{
my_off += comp_buf_sz;
DARSHAN_MPI_CALL(PMPI_Send)(&my_off, 1, MPI_OFFSET, 0, 0,
MPI_COMM_WORLD);
}
- else if(my_rank == 0)
+ if(my_rank == 0)
{
DARSHAN_MPI_CALL(PMPI_Recv)(&my_off, 1, MPI_OFFSET, (nprocs-1), 0,
MPI_COMM_WORLD, &status);
@@ -1476,10 +1409,6 @@ static int darshan_log_append_all(MPI_File log_fh, struct darshan_core_runtime *
*inout_off = my_off + comp_buf_sz;
}
- /* pass back the aggregate uncompressed size of this blob */
- DARSHAN_MPI_CALL(PMPI_Reduce)(&uncomp_buf_sz, agg_uncomp_sz, 1,
- MPI_UINT64_T, MPI_SUM, 0, MPI_COMM_WORLD);
-
if(ret != 0)
return(-1);
return(0);
@@ -1518,6 +1447,7 @@ static void darshan_core_cleanup(struct darshan_core_runtime* core)
void darshan_core_register_module(
darshan_module_id mod_id,
struct darshan_module_funcs *funcs,
+ int *my_rank,
int *mod_mem_limit,
int *sys_mem_alignment)
{
@@ -1554,6 +1484,9 @@ void darshan_core_register_module(
/* register module with darshan */
darshan_core->mod_array[mod_id] = mod;
+ /* get the calling process's rank */
+ DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, my_rank);
+
/* TODO: something smarter than just 2 MiB per module */
*mod_mem_limit = 2 * 1024 * 1024;
diff --git a/darshan-runtime/lib/darshan-hdf5-stubs.c b/darshan-runtime/lib/darshan-hdf5-stubs.c
new file mode 100644
index 0000000..42881c4
--- /dev/null
+++ b/darshan-runtime/lib/darshan-hdf5-stubs.c
@@ -0,0 +1,80 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+/* This file contains stubs for the H5F functions intercepted by Darshan.
+ * They are defined as weak symbols in order to satisfy dependencies of the
+ * hdf5 wrappers in cases where hdf5 is not being used.
+ */
+
+
+#include "darshan-runtime-config.h"
+#include <stdio.h>
+#include <pthread.h>
+#include <string.h>
+#include "mpi.h"
+#include "darshan.h"
+
+/* hope this doesn't change any time soon */
+typedef int hid_t;
+typedef int herr_t;
+
+hid_t H5Fcreate(const char *filename, unsigned flags,
+ hid_t create_plist, hid_t access_plist) __attribute__((weak));
+
+hid_t H5Fcreate(const char *filename, unsigned flags,
+ hid_t create_plist, hid_t access_plist)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan H5Fcreate() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+}
+
+hid_t H5Fopen(const char *filename, unsigned flags,
+ hid_t access_plist) __attribute__((weak));
+
+hid_t H5Fopen(const char *filename, unsigned flags,
+ hid_t access_plist)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan H5Fopen() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+}
+
+herr_t H5Fclose(hid_t file_id) __attribute__((weak));
+
+herr_t H5Fclose(hid_t file_id)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan H5Fclose() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-runtime/lib/darshan-hdf5.c b/darshan-runtime/lib/darshan-hdf5.c
new file mode 100644
index 0000000..b94010a
--- /dev/null
+++ b/darshan-runtime/lib/darshan-hdf5.c
@@ -0,0 +1,576 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#include "darshan-runtime-config.h"
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <stdarg.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <search.h>
+#include <assert.h>
+#define __USE_GNU
+#include <pthread.h>
+
+#include "uthash.h"
+
+#include "darshan.h"
+#include "darshan-hdf5-log-format.h"
+#include "darshan-dynamic.h"
+
+/* hope this doesn't change any time soon */
+typedef int hid_t;
+typedef int herr_t;
+
+DARSHAN_FORWARD_DECL(H5Fcreate, hid_t, (const char *filename, unsigned flags, hid_t create_plist, hid_t access_plist));
+DARSHAN_FORWARD_DECL(H5Fopen, hid_t, (const char *filename, unsigned flags, hid_t access_plist));
+DARSHAN_FORWARD_DECL(H5Fclose, herr_t, (hid_t file_id));
+
+/* structure to track i/o stats for a given hdf5 file at runtime */
+struct hdf5_file_runtime
+{
+ struct darshan_hdf5_file* file_record;
+ UT_hash_handle hlink;
+};
+
+/* structure to associate a HDF5 hid with an existing file runtime structure */
+struct hdf5_file_runtime_ref
+{
+ struct hdf5_file_runtime* file;
+ hid_t hid;
+ UT_hash_handle hlink;
+};
+
+/* necessary state for storing HDF5 file records and coordinating with
+ * darshan-core at shutdown time
+ */
+struct hdf5_runtime
+{
+ struct hdf5_file_runtime* file_runtime_array;
+ struct darshan_hdf5_file* file_record_array;
+ int file_array_size;
+ int file_array_ndx;
+ struct hdf5_file_runtime *file_hash;
+ struct hdf5_file_runtime_ref* hid_hash;
+};
+
+static struct hdf5_runtime *hdf5_runtime = NULL;
+static pthread_mutex_t hdf5_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
+static int instrumentation_disabled = 0;
+static int my_rank = -1;
+
+static void hdf5_runtime_initialize(void);
+static struct hdf5_file_runtime* hdf5_file_by_name(const char *name);
+static struct hdf5_file_runtime* hdf5_file_by_name_sethid(const char* name, hid_t hid);
+static struct hdf5_file_runtime* hdf5_file_by_hid(hid_t hid);
+static void hdf5_file_close_hid(hid_t hid);
+static int hdf5_record_compare(const void* a, const void* b);
+static void hdf5_record_reduction_op(void* infile_v, void* inoutfile_v,
+ int *len, MPI_Datatype *datatype);
+
+static void hdf5_begin_shutdown(void);
+static void hdf5_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **hdf5_buf, int *hdf5_buf_sz);
+static void hdf5_shutdown(void);
+
+#define HDF5_LOCK() pthread_mutex_lock(&hdf5_runtime_mutex)
+#define HDF5_UNLOCK() pthread_mutex_unlock(&hdf5_runtime_mutex)
+
+/*********************************************************
+ * Wrappers for HDF5 functions of interest *
+ *********************************************************/
+
+hid_t DARSHAN_DECL(H5Fcreate)(const char *filename, unsigned flags,
+ hid_t create_plist, hid_t access_plist)
+{
+ int ret;
+ struct hdf5_file_runtime* file;
+ char* tmp;
+ double tm1;
+
+ MAP_OR_FAIL(H5Fcreate);
+
+ tm1 = darshan_core_wtime();
+ ret = __real_H5Fcreate(filename, flags, create_plist, access_plist);
+ if(ret >= 0)
+ {
+ /* use ROMIO approach to strip prefix if present */
+ /* strip off prefix if there is one, but only skip prefixes
+ * if they are greater than length one to allow for windows
+ * drive specifications (e.g. c:\...)
+ */
+ tmp = strchr(filename, ':');
+ if (tmp > filename + 1) {
+ filename = tmp + 1;
+ }
+
+ HDF5_LOCK();
+ hdf5_runtime_initialize();
+ file = hdf5_file_by_name_sethid(filename, ret);
+ if(file)
+ {
+ if(file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] == 0)
+ file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] = tm1;
+ file->file_record->counters[HDF5_OPENS] += 1;
+ }
+ HDF5_UNLOCK();
+ }
+
+ return(ret);
+}
+
+hid_t DARSHAN_DECL(H5Fopen)(const char *filename, unsigned flags,
+ hid_t access_plist)
+{
+ int ret;
+ struct hdf5_file_runtime* file;
+ char* tmp;
+ double tm1;
+
+ MAP_OR_FAIL(H5Fopen);
+
+ tm1 = darshan_core_wtime();
+ ret = __real_H5Fopen(filename, flags, access_plist);
+ if(ret >= 0)
+ {
+ /* use ROMIO approach to strip prefix if present */
+ /* strip off prefix if there is one, but only skip prefixes
+ * if they are greater than length one to allow for windows
+ * drive specifications (e.g. c:\...)
+ */
+ tmp = strchr(filename, ':');
+ if (tmp > filename + 1) {
+ filename = tmp + 1;
+ }
+
+ HDF5_LOCK();
+ hdf5_runtime_initialize();
+ file = hdf5_file_by_name_sethid(filename, ret);
+ if(file)
+ {
+ if(file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] == 0)
+ file->file_record->fcounters[HDF5_F_OPEN_TIMESTAMP] = tm1;
+ file->file_record->counters[HDF5_OPENS] += 1;
+ }
+ HDF5_UNLOCK();
+ }
+
+ return(ret);
+
+}
+
+herr_t DARSHAN_DECL(H5Fclose)(hid_t file_id)
+{
+ struct hdf5_file_runtime* file;
+ int ret;
+
+ MAP_OR_FAIL(H5Fclose);
+
+ ret = __real_H5Fclose(file_id);
+
+ HDF5_LOCK();
+ hdf5_runtime_initialize();
+ file = hdf5_file_by_hid(file_id);
+ if(file)
+ {
+ file->file_record->fcounters[HDF5_F_CLOSE_TIMESTAMP] =
+ darshan_core_wtime();
+ hdf5_file_close_hid(file_id);
+ }
+ HDF5_UNLOCK();
+
+ return(ret);
+
+}
+
+/*********************************************************
+ * Internal functions for manipulating HDF5 module state *
+ *********************************************************/
+
+/* initialize internal HDF5 module data strucutres and register with darshan-core */
+static void hdf5_runtime_initialize()
+{
+ int mem_limit;
+ struct darshan_module_funcs hdf5_mod_fns =
+ {
+ .begin_shutdown = &hdf5_begin_shutdown,
+ .get_output_data = &hdf5_get_output_data,
+ .shutdown = &hdf5_shutdown
+ };
+
+ /* don't do anything if already initialized or instrumenation is disabled */
+ if(hdf5_runtime || instrumentation_disabled)
+ return;
+
+ /* register hdf5 module with darshan-core */
+ darshan_core_register_module(
+ DARSHAN_HDF5_MOD,
+ &hdf5_mod_fns,
+ &my_rank,
+ &mem_limit,
+ NULL);
+
+ /* return if no memory assigned by darshan-core */
+ if(mem_limit == 0)
+ return;
+
+ hdf5_runtime = malloc(sizeof(*hdf5_runtime));
+ if(!hdf5_runtime)
+ return;
+ memset(hdf5_runtime, 0, sizeof(*hdf5_runtime));
+
+ /* set maximum number of file records according to max memory limit */
+ /* NOTE: maximum number of records is based on the size of a hdf5 file record */
+ /* TODO: should we base memory usage off file record or total runtime structure sizes? */
+ hdf5_runtime->file_array_size = mem_limit / sizeof(struct darshan_hdf5_file);
+ hdf5_runtime->file_array_ndx = 0;
+
+ /* allocate array of runtime file records */
+ hdf5_runtime->file_runtime_array = malloc(hdf5_runtime->file_array_size *
+ sizeof(struct hdf5_file_runtime));
+ hdf5_runtime->file_record_array = malloc(hdf5_runtime->file_array_size *
+ sizeof(struct darshan_hdf5_file));
+ if(!hdf5_runtime->file_runtime_array || !hdf5_runtime->file_record_array)
+ {
+ hdf5_runtime->file_array_size = 0;
+ return;
+ }
+ memset(hdf5_runtime->file_runtime_array, 0, hdf5_runtime->file_array_size *
+ sizeof(struct hdf5_file_runtime));
+ memset(hdf5_runtime->file_record_array, 0, hdf5_runtime->file_array_size *
+ sizeof(struct darshan_hdf5_file));
+
+ return;
+}
+
+/* get a HDF5 file record for the given file path */
+static struct hdf5_file_runtime* hdf5_file_by_name(const char *name)
+{
+ struct hdf5_file_runtime *file = NULL;
+ char *newname = NULL;
+ darshan_record_id file_id;
+
+ if(!hdf5_runtime || instrumentation_disabled)
+ return(NULL);
+
+ newname = darshan_clean_file_path(name);
+ if(!newname)
+ newname = (char*)name;
+
+ /* get a unique id for this file from darshan core */
+ darshan_core_register_record(
+ (void*)newname,
+ strlen(newname),
+ 1,
+ DARSHAN_HDF5_MOD,
+ &file_id,
+ NULL);
+
+ /* search the hash table for this file record, and return if found */
+ HASH_FIND(hlink, hdf5_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);
+ if(file)
+ {
+ if(newname != name)
+ free(newname);
+ return(file);
+ }
+
+ if(hdf5_runtime->file_array_ndx < hdf5_runtime->file_array_size);
+ {
+ /* no existing record, assign a new file record from the global array */
+ file = &(hdf5_runtime->file_runtime_array[hdf5_runtime->file_array_ndx]);
+ file->file_record = &(hdf5_runtime->file_record_array[hdf5_runtime->file_array_ndx]);
+ file->file_record->f_id = file_id;
+ file->file_record->rank = my_rank;
+
+ /* add new record to file hash table */
+ HASH_ADD(hlink, hdf5_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);
+
+ hdf5_runtime->file_array_ndx++;
+ }
+
+ if(newname != name)
+ free(newname);
+ return(file);
+}
+
+/* get a HDF5 file record for the given file path, and also create a
+ * reference structure using the returned hid
+ */
+static struct hdf5_file_runtime* hdf5_file_by_name_sethid(const char* name, hid_t hid)
+{
+ struct hdf5_file_runtime* file;
+ struct hdf5_file_runtime_ref* ref;
+
+ if(!hdf5_runtime || instrumentation_disabled)
+ return(NULL);
+
+ /* find file record by name first */
+ file = hdf5_file_by_name(name);
+
+ if(!file)
+ return(NULL);
+
+ /* search hash table for existing file ref for this fd */
+ HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);
+ if(ref)
+ {
+ /* we have a reference. Make sure it points to the correct file
+ * and return it
+ */
+ ref->file = file;
+ return(file);
+ }
+
+ /* if we hit this point, then we don't have a reference for this fd
+ * in the table yet. Add it.
+ */
+ ref = malloc(sizeof(*ref));
+ if(!ref)
+ return(NULL);
+ memset(ref, 0, sizeof(*ref));
+
+ ref->file = file;
+ ref->hid = hid;
+ HASH_ADD(hlink, hdf5_runtime->hid_hash, hid, sizeof(hid_t), ref);
+
+ return(file);
+}
+
+/* get a HDF5 file record for the given hid */
+static struct hdf5_file_runtime* hdf5_file_by_hid(hid_t hid)
+{
+ struct hdf5_file_runtime_ref* ref;
+
+ if(!hdf5_runtime || instrumentation_disabled)
+ return(NULL);
+
+ /* search hash table for existing file ref for this hid */
+ HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);
+ if(ref)
+ return(ref->file);
+
+ return(NULL);
+}
+
+/* free up HDF5 reference data structures for the given hid */
+static void hdf5_file_close_hid(hid_t hid)
+{
+ struct hdf5_file_runtime_ref* ref;
+
+ if(!hdf5_runtime || instrumentation_disabled)
+ return;
+
+ /* search hash table for this hid */
+ HASH_FIND(hlink, hdf5_runtime->hid_hash, &hid, sizeof(hid_t), ref);
+ if(ref)
+ {
+ /* we have a reference, delete it */
+ HASH_DELETE(hlink, hdf5_runtime->hid_hash, ref);
+ free(ref);
+ }
+
+ return;
+}
+
+/* compare function for sorting file records by descending rank */
+static int hdf5_record_compare(const void* a_p, const void* b_p)
+{
+ const struct darshan_hdf5_file* a = a_p;
+ const struct darshan_hdf5_file* b = b_p;
+
+ if(a->rank < b->rank)
+ return 1;
+ if(a->rank > b->rank)
+ return -1;
+
+ return 0;
+}
+
+static void hdf5_record_reduction_op(void* infile_v, void* inoutfile_v,
+ int *len, MPI_Datatype *datatype)
+{
+ struct darshan_hdf5_file tmp_file;
+ struct darshan_hdf5_file *infile = infile_v;
+ struct darshan_hdf5_file *inoutfile = inoutfile_v;
+ int i, j;
+
+ assert(hdf5_runtime);
+
+ for(i=0; i<*len; i++)
+ {
+ memset(&tmp_file, 0, sizeof(struct darshan_hdf5_file));
+ tmp_file.f_id = infile->f_id;
+ tmp_file.rank = -1;
+
+ /* sum */
+ for(j=HDF5_OPENS; j<=HDF5_OPENS; j++)
+ {
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
+ }
+
+ /* min non-zero (if available) value */
+ for(j=HDF5_F_OPEN_TIMESTAMP; j<=HDF5_F_OPEN_TIMESTAMP; j++)
+ {
+ if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)
+ tmp_file.fcounters[j] = inoutfile->fcounters[j];
+ else
+ tmp_file.fcounters[j] = infile->fcounters[j];
+ }
+
+ /* max */
+ for(j=HDF5_F_CLOSE_TIMESTAMP; j<=HDF5_F_CLOSE_TIMESTAMP; j++)
+ {
+ if(infile->fcounters[j] > inoutfile->fcounters[j])
+ tmp_file.fcounters[j] = infile->fcounters[j];
+ else
+ tmp_file.fcounters[j] = inoutfile->fcounters[j];
+ }
+
+ /* update pointers */
+ *inoutfile = tmp_file;
+ inoutfile++;
+ infile++;
+ }
+
+ return;
+}
+
+/************************************************************************
+ * Functions exported by HDF5 module for coordinating with darshan-core *
+ ************************************************************************/
+
+static void hdf5_begin_shutdown()
+{
+ assert(hdf5_runtime);
+
+ HDF5_LOCK();
+ /* disable further instrumentation while Darshan shuts down */
+ instrumentation_disabled = 1;
+ HDF5_UNLOCK();
+
+ return;
+}
+
+static void hdf5_get_output_data(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **hdf5_buf,
+ int *hdf5_buf_sz)
+{
+ struct hdf5_file_runtime *file;
+ int i;
+ struct darshan_hdf5_file *red_send_buf = NULL;
+ struct darshan_hdf5_file *red_recv_buf = NULL;
+ MPI_Datatype red_type;
+ MPI_Op red_op;
+
+ assert(hdf5_runtime);
+
+ /* if there are globally shared files, do a shared file reduction */
+ if(shared_rec_count)
+ {
+ /* necessary initialization of shared records */
+ for(i = 0; i < shared_rec_count; i++)
+ {
+ HASH_FIND(hlink, hdf5_runtime->file_hash, &shared_recs[i],
+ sizeof(darshan_record_id), file);
+ assert(file);
+
+ file->file_record->rank = -1;
+ }
+
+ /* sort the array of files descending by rank so that we get all of the
+ * shared files (marked by rank -1) in a contiguous portion at end
+ * of the array
+ */
+ qsort(hdf5_runtime->file_record_array, hdf5_runtime->file_array_ndx,
+ sizeof(struct darshan_hdf5_file), hdf5_record_compare);
+
+ /* make *send_buf point to the shared files at the end of sorted array */
+ red_send_buf =
+ &(hdf5_runtime->file_record_array[hdf5_runtime->file_array_ndx-shared_rec_count]);
+
+ /* allocate memory for the reduction output on rank 0 */
+ if(my_rank == 0)
+ {
+ red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_hdf5_file));
+ if(!red_recv_buf)
+ return;
+ }
+
+ /* construct a datatype for a HDF5 file record. This is serving no purpose
+ * except to make sure we can do a reduction on proper boundaries
+ */
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_hdf5_file),
+ MPI_BYTE, &red_type);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);
+
+ /* register a HDF5 file record reduction operator */
+ DARSHAN_MPI_CALL(PMPI_Op_create)(hdf5_record_reduction_op, 1, &red_op);
+
+ /* reduce shared HDF5 file records */
+ DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,
+ shared_rec_count, red_type, red_op, 0, mod_comm);
+
+ /* clean up reduction state */
+ if(my_rank == 0)
+ {
+ int tmp_ndx = hdf5_runtime->file_array_ndx - shared_rec_count;
+ memcpy(&(hdf5_runtime->file_record_array[tmp_ndx]), red_recv_buf,
+ shared_rec_count * sizeof(struct darshan_hdf5_file));
+ free(red_recv_buf);
+ }
+ else
+ {
+ hdf5_runtime->file_array_ndx -= shared_rec_count;
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);
+ }
+
+ *hdf5_buf = (void *)(hdf5_runtime->file_record_array);
+ *hdf5_buf_sz = hdf5_runtime->file_array_ndx * sizeof(struct darshan_hdf5_file);
+
+ return;
+}
+
+static void hdf5_shutdown()
+{
+ struct hdf5_file_runtime_ref *ref, *tmp;
+
+ assert(hdf5_runtime);
+
+ HASH_ITER(hlink, hdf5_runtime->hid_hash, ref, tmp)
+ {
+ HASH_DELETE(hlink, hdf5_runtime->hid_hash, ref);
+ free(ref);
+ }
+
+ HASH_CLEAR(hlink, hdf5_runtime->file_hash); /* these entries are freed all at once below */
+
+ free(hdf5_runtime->file_runtime_array);
+ free(hdf5_runtime->file_record_array);
+ free(hdf5_runtime);
+ hdf5_runtime = NULL;
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-runtime/lib/darshan-mpiio.c b/darshan-runtime/lib/darshan-mpiio.c
index 0cba6da..c823851 100644
--- a/darshan-runtime/lib/darshan-mpiio.c
+++ b/darshan-runtime/lib/darshan-mpiio.c
@@ -15,16 +15,13 @@
#include <time.h>
#include <stdlib.h>
#include <errno.h>
-#include <sys/uio.h>
-#include <sys/mman.h>
#include <search.h>
#include <assert.h>
-#include <libgen.h>
-#include <aio.h>
#define __USE_GNU
#include <pthread.h>
#include "uthash.h"
+
#include "darshan.h"
#include "darshan-mpiio-log-format.h"
#include "darshan-dynamic.h"
@@ -55,11 +52,12 @@
struct mpiio_file_runtime
{
struct darshan_mpiio_file* file_record;
- double last_mpi_meta_end; /* end time of last MPI meta op (so far) */
- /* TODO: any stateful (but not intended for persistent storage in the log)
- * information about MPI-IO access. If we don't have any then this struct
- * could be eliminated.
- */
+ enum darshan_io_type last_io_type;
+ double last_meta_end;
+ double last_read_end;
+ double last_write_end;
+ void *access_root;
+ int access_count;
UT_hash_handle hlink;
};
@@ -89,10 +87,14 @@ struct mpiio_file_runtime
struct mpiio_file_runtime_ref
{
struct mpiio_file_runtime* file;
- MPI_File *fh;
+ MPI_File fh;
UT_hash_handle hlink;
};
+/* The mpiio_runtime structure maintains necessary state for storing
+ * MPI-IO file records and for coordinating with darshan-core at
+ * shutdown time.
+ */
struct mpiio_runtime
{
struct mpiio_file_runtime* file_runtime_array;
@@ -101,8 +103,6 @@ struct mpiio_runtime
int file_array_ndx;
struct mpiio_file_runtime* file_hash;
struct mpiio_file_runtime_ref* fh_hash;
- void *red_buf;
- int shared_rec_count;
};
static struct mpiio_runtime *mpiio_runtime = NULL;
@@ -110,22 +110,79 @@ static pthread_mutex_t mpiio_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER
static int instrumentation_disabled = 0;
static int my_rank = -1;
-#define MPIIO_LOCK() pthread_mutex_lock(&mpiio_runtime_mutex)
-#define MPIIO_UNLOCK() pthread_mutex_unlock(&mpiio_runtime_mutex)
-
static void mpiio_runtime_initialize(void);
-static void mpiio_begin_shutdown(void);
-static void mpiio_shutdown(void);
-static void mpiio_get_output_data(
- void **buffer,
- int *size);
-static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File *fh);
static struct mpiio_file_runtime* mpiio_file_by_name(const char *name);
+static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File fh);
+static struct mpiio_file_runtime* mpiio_file_by_fh(MPI_File fh);
+static void mpiio_file_close_fh(MPI_File fh);
+static int mpiio_record_compare(const void* a, const void* b);
static void mpiio_record_reduction_op(void* infile_v, void* inoutfile_v,
int *len, MPI_Datatype *datatype);
-static void mpiio_setup_reduction(darshan_record_id *shared_recs,
- int *shared_rec_count, void **send_buf, void **recv_buf, int *rec_size);
-static int mpiio_file_compare(const void* a, const void* b);
+static void mpiio_shared_record_variance(MPI_Comm mod_comm,
+ struct darshan_mpiio_file *inrec_array, struct darshan_mpiio_file *outrec_array,
+ int shared_rec_count);
+
+static void mpiio_begin_shutdown(void);
+static void mpiio_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **mpiio_buf, int *mpiio_buf_sz);
+static void mpiio_shutdown(void);
+
+#define MPIIO_LOCK() pthread_mutex_lock(&mpiio_runtime_mutex)
+#define MPIIO_UNLOCK() pthread_mutex_unlock(&mpiio_runtime_mutex)
+
+#define MPIIO_RECORD_READ(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \
+ struct mpiio_file_runtime* file; \
+ int size = 0; \
+ double __elapsed = __tm2-__tm1; \
+ if(__ret != MPI_SUCCESS) break; \
+ file = mpiio_file_by_fh(__fh); \
+ if(!file) break; \
+ DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size); \
+ size = size * __count; \
+ DARSHAN_BUCKET_INC(&(file->file_record->counters[MPIIO_SIZE_READ_AGG_0_100]), size); \
+ darshan_common_val_counter(&file->access_root, &file->access_count, size); \
+ file->file_record->counters[MPIIO_BYTES_READ] += size; \
+ file->file_record->counters[__counter] += 1; \
+ if(file->last_io_type == DARSHAN_IO_WRITE) \
+ file->file_record->counters[MPIIO_RW_SWITCHES] += 1; \
+ file->last_io_type = DARSHAN_IO_READ; \
+ if(file->file_record->fcounters[MPIIO_F_READ_START_TIMESTAMP] == 0) \
+ file->file_record->fcounters[MPIIO_F_READ_START_TIMESTAMP] = __tm1; \
+ file->file_record->fcounters[MPIIO_F_READ_END_TIMESTAMP] = __tm2; \
+ if(file->file_record->fcounters[MPIIO_F_MAX_READ_TIME] < __elapsed) { \
+ file->file_record->fcounters[MPIIO_F_MAX_READ_TIME] = __elapsed; \
+ file->file_record->counters[MPIIO_MAX_READ_TIME_SIZE] = size; } \
+ DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[MPIIO_F_READ_TIME], __tm1, __tm2, file->last_read_end); \
+} while(0)
+
+#define MPIIO_RECORD_WRITE(__ret, __fh, __count, __datatype, __counter, __tm1, __tm2) do { \
+ struct mpiio_file_runtime* file; \
+ int size = 0; \
+ double __elapsed = __tm2-__tm1; \
+ if(__ret != MPI_SUCCESS) break; \
+ file = mpiio_file_by_fh(__fh); \
+ if(!file) break; \
+ DARSHAN_MPI_CALL(PMPI_Type_size)(__datatype, &size); \
+ size = size * __count; \
+ DARSHAN_BUCKET_INC(&(file->file_record->counters[MPIIO_SIZE_WRITE_AGG_0_100]), size); \
+ darshan_common_val_counter(&file->access_root, &file->access_count, size); \
+ file->file_record->counters[MPIIO_BYTES_WRITTEN] += size; \
+ file->file_record->counters[__counter] += 1; \
+ if(file->last_io_type == DARSHAN_IO_READ) \
+ file->file_record->counters[MPIIO_RW_SWITCHES] += 1; \
+ file->last_io_type = DARSHAN_IO_WRITE; \
+ if(file->file_record->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] == 0) \
+ file->file_record->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] = __tm1; \
+ file->file_record->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] = __tm2; \
+ if(file->file_record->fcounters[MPIIO_F_MAX_WRITE_TIME] < __elapsed) { \
+ file->file_record->fcounters[MPIIO_F_MAX_WRITE_TIME] = __elapsed; \
+ file->file_record->counters[MPIIO_MAX_WRITE_TIME_SIZE] = size; } \
+ DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[MPIIO_F_WRITE_TIME], __tm1, __tm2, file->last_write_end); \
+} while(0)
+
+/**********************************************************
+ * Wrappers for MPI-IO functions of interest *
+ **********************************************************/
#ifdef HAVE_MPIIO_CONST
int MPI_File_open(MPI_Comm comm, const char *filename, int amode, MPI_Info info, MPI_File *fh)
@@ -158,43 +215,615 @@ int MPI_File_open(MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_F
filename = tmp + 1;
}
- file = mpiio_file_by_name_setfh(filename, fh);
+ file = mpiio_file_by_name_setfh(filename, (*fh));
if(file)
{
- file->file_record->rank = my_rank;
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record, tm1, tm2, file->last_mpi_meta_end, DARSHAN_MPIIO_F_META_TIME);
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, DARSHAN_MPIIO_F_OPEN_TIMESTAMP) == 0)
- DARSHAN_COUNTER_F_SET(file->file_record, DARSHAN_MPIIO_F_OPEN_TIMESTAMP,
- tm1);
+ file->file_record->counters[MPIIO_MODE] = amode;
DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);
if(comm_size == 1)
{
- DARSHAN_COUNTER_INC(file->file_record, DARSHAN_MPIIO_INDEP_OPENS, 1);
+ file->file_record->counters[MPIIO_INDEP_OPENS] += 1;
}
else
{
- DARSHAN_COUNTER_INC(file->file_record, DARSHAN_MPIIO_COLL_OPENS, 1);
+ file->file_record->counters[MPIIO_COLL_OPENS] += 1;
}
if(info != MPI_INFO_NULL)
{
- DARSHAN_COUNTER_INC(file->file_record, DARSHAN_MPIIO_HINTS, 1);
+ file->file_record->counters[MPIIO_HINTS] += 1;
}
+ if(file->file_record->fcounters[MPIIO_F_OPEN_TIMESTAMP] == 0)
+ file->file_record->fcounters[MPIIO_F_OPEN_TIMESTAMP] = tm1;
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[MPIIO_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ }
+
+ MPIIO_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int MPI_File_read(MPI_File fh, void *buf, int count,
+ MPI_Datatype datatype, MPI_Status *status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read)(fh, buf, count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write(MPI_File fh, const void *buf, int count,
+ MPI_Datatype datatype, MPI_Status *status)
+#else
+int MPI_File_write(MPI_File fh, void *buf, int count,
+ MPI_Datatype datatype, MPI_Status *status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write)(fh, buf, count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_at(MPI_File fh, MPI_Offset offset, void *buf,
+ int count, MPI_Datatype datatype, MPI_Status *status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_at)(fh, offset, buf,
+ count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_at(MPI_File fh, MPI_Offset offset, const void *buf,
+ int count, MPI_Datatype datatype, MPI_Status *status)
+#else
+int MPI_File_write_at(MPI_File fh, MPI_Offset offset, void *buf,
+ int count, MPI_Datatype datatype, MPI_Status *status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_at)(fh, offset, buf,
+ count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_all)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_all(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+#else
+int MPI_File_write_all(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_all)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_at_all(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype, MPI_Status * status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all)(fh, offset, buf,
+ count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, const void * buf,
+ int count, MPI_Datatype datatype, MPI_Status * status)
+#else
+int MPI_File_write_at_all(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype, MPI_Status * status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all)(fh, offset, buf,
+ count, datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_shared)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_INDEP_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_shared(MPI_File fh, const void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+#else
+int MPI_File_write_shared(MPI_File fh, void * buf, int count, MPI_Datatype datatype, MPI_Status *status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_shared)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_INDEP_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_ordered(MPI_File fh, void * buf, int count,
+ MPI_Datatype datatype, MPI_Status * status)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_COLL_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_ordered(MPI_File fh, const void * buf, int count,
+ MPI_Datatype datatype, MPI_Status * status)
+#else
+int MPI_File_write_ordered(MPI_File fh, void * buf, int count,
+ MPI_Datatype datatype, MPI_Status * status)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered)(fh, buf, count,
+ datatype, status);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_COLL_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_all_begin)(fh, buf, count, datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_all_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)
+#else
+int MPI_File_write_all_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_all_begin)(fh, buf, count, datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_at_all_begin)(fh, offset, buf,
+ count, datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, const void * buf,
+ int count, MPI_Datatype datatype)
+#else
+int MPI_File_write_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_at_all_begin)(fh, offset,
+ buf, count, datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_read_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_read_ordered_begin)(fh, buf, count,
+ datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_SPLIT_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_write_ordered_begin(MPI_File fh, const void * buf, int count, MPI_Datatype datatype)
+#else
+int MPI_File_write_ordered_begin(MPI_File fh, void * buf, int count, MPI_Datatype datatype)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_write_ordered_begin)(fh, buf, count,
+ datatype);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_SPLIT_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_iread(MPI_File fh, void * buf, int count, MPI_Datatype datatype, __D_MPI_REQUEST * request)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iread)(fh, buf, count, datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_iwrite(MPI_File fh, const void * buf, int count,
+ MPI_Datatype datatype, __D_MPI_REQUEST * request)
+#else
+int MPI_File_iwrite(MPI_File fh, void * buf, int count,
+ MPI_Datatype datatype, __D_MPI_REQUEST * request)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iwrite)(fh, buf, count, datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_iread_at(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iread_at)(fh, offset, buf, count,
+ datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, const void * buf,
+ int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)
+#else
+int MPI_File_iwrite_at(MPI_File fh, MPI_Offset offset, void * buf,
+ int count, MPI_Datatype datatype, __D_MPI_REQUEST *request)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_at)(fh, offset, buf,
+ count, datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_iread_shared(MPI_File fh, void * buf, int count,
+ MPI_Datatype datatype, __D_MPI_REQUEST * request)
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iread_shared)(fh, buf, count,
+ datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_READ(ret, fh, count, datatype, MPIIO_NB_READS, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_iwrite_shared(MPI_File fh, const void * buf, int count,
+ MPI_Datatype datatype, __D_MPI_REQUEST * request)
+#else
+int MPI_File_iwrite_shared(MPI_File fh, void * buf, int count,
+ MPI_Datatype datatype, __D_MPI_REQUEST * request)
+#endif
+{
+ int ret;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_iwrite_shared)(fh, buf, count,
+ datatype, request);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ MPIIO_RECORD_WRITE(ret, fh, count, datatype, MPIIO_NB_WRITES, tm1, tm2);
+ MPIIO_UNLOCK();
+ return(ret);
+}
+
+int MPI_File_sync(MPI_File fh)
+{
+ int ret;
+ struct mpiio_file_runtime* file;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_sync)(fh);
+ tm2 = darshan_core_wtime();
+
+ if(ret == MPI_SUCCESS)
+ {
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ file = mpiio_file_by_fh(fh);
+ if(file)
+ {
+ file->file_record->counters[MPIIO_SYNCS] += 1;
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[MPIIO_F_WRITE_TIME],
+ tm1, tm2, file->last_write_end);
}
+ MPIIO_UNLOCK();
+ }
+
+ return(ret);
+}
+
+#ifdef HAVE_MPIIO_CONST
+int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype,
+ MPI_Datatype filetype, const char *datarep, MPI_Info info)
+#else
+int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype,
+ MPI_Datatype filetype, char *datarep, MPI_Info info)
+#endif
+{
+ int ret;
+ struct mpiio_file_runtime* file;
+ double tm1, tm2;
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_set_view)(fh, disp, etype, filetype,
+ datarep, info);
+ tm2 = darshan_core_wtime();
+
+ if(ret == MPI_SUCCESS)
+ {
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ file = mpiio_file_by_fh(fh);
+ if(file)
+ {
+ file->file_record->counters[MPIIO_VIEWS] += 1;
+ if(info != MPI_INFO_NULL)
+ {
+ file->file_record->counters[MPIIO_HINTS] += 1;
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[MPIIO_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ }
+ }
MPIIO_UNLOCK();
}
return(ret);
}
+int MPI_File_close(MPI_File *fh)
+{
+ int ret;
+ struct mpiio_file_runtime* file;
+ MPI_File tmp_fh = *fh;
+ double tm1, tm2;
+
+ tm1 = darshan_core_wtime();
+ ret = DARSHAN_MPI_CALL(PMPI_File_close)(fh);
+ tm2 = darshan_core_wtime();
+
+ MPIIO_LOCK();
+ mpiio_runtime_initialize();
+ file = mpiio_file_by_fh(tmp_fh);
+ if(file)
+ {
+ file->file_record->fcounters[MPIIO_F_CLOSE_TIMESTAMP] =
+ darshan_core_wtime();
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[MPIIO_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ mpiio_file_close_fh(tmp_fh);
+ }
+ MPIIO_UNLOCK();
+
+ return(ret);
+}
+
+/***********************************************************
+ * Internal functions for manipulating MPI-IO module state *
+ ***********************************************************/
+
+/* initialize data structures and register with darshan-core component */
static void mpiio_runtime_initialize()
{
int mem_limit;
struct darshan_module_funcs mpiio_mod_fns =
{
.begin_shutdown = &mpiio_begin_shutdown,
- .setup_reduction = &mpiio_setup_reduction,
- .record_reduction_op = &mpiio_record_reduction_op,
.get_output_data = &mpiio_get_output_data,
.shutdown = &mpiio_shutdown
};
@@ -207,6 +836,7 @@ static void mpiio_runtime_initialize()
darshan_core_register_module(
DARSHAN_MPIIO_MOD,
&mpiio_mod_fns,
+ &my_rank,
&mem_limit,
NULL);
@@ -239,65 +869,6 @@ static void mpiio_runtime_initialize()
memset(mpiio_runtime->file_record_array, 0, mpiio_runtime->file_array_size *
sizeof(struct darshan_mpiio_file));
- /* TODO: can we move this out of here? perhaps register_module returns rank? */
- DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &my_rank);
-
- return;
-}
-
-static void mpiio_begin_shutdown()
-{
- assert(mpiio_runtime);
-
- MPIIO_LOCK();
- instrumentation_disabled = 1;
- MPIIO_UNLOCK();
-
- return;
-}
-
-static void mpiio_get_output_data(
- void **buffer,
- int *size)
-{
- assert(mpiio_runtime);
-
- /* TODO: clean up reduction stuff */
- if(my_rank == 0)
- {
- int tmp_ndx = mpiio_runtime->file_array_ndx - mpiio_runtime->shared_rec_count;
- memcpy(&(mpiio_runtime->file_record_array[tmp_ndx]), mpiio_runtime->red_buf,
- mpiio_runtime->shared_rec_count * sizeof(struct darshan_mpiio_file));
- free(mpiio_runtime->red_buf);
- }
- else
- {
- mpiio_runtime->file_array_ndx -= mpiio_runtime->shared_rec_count;
- }
-
- *buffer = (void *)(mpiio_runtime->file_record_array);
- *size = mpiio_runtime->file_array_ndx * sizeof(struct darshan_mpiio_file);
-
- return;
-}
-
-static void mpiio_shutdown()
-{
- struct mpiio_file_runtime_ref *ref, *tmp;
-
- HASH_ITER(hlink, mpiio_runtime->fh_hash, ref, tmp)
- {
- HASH_DELETE(hlink, mpiio_runtime->fh_hash, ref);
- free(ref);
- }
-
- HASH_CLEAR(hlink, mpiio_runtime->file_hash); /* these entries are freed all at once below */
-
- free(mpiio_runtime->file_runtime_array);
- free(mpiio_runtime->file_record_array);
- free(mpiio_runtime);
- mpiio_runtime = NULL;
-
return;
}
@@ -339,6 +910,7 @@ static struct mpiio_file_runtime* mpiio_file_by_name(const char *name)
file = &(mpiio_runtime->file_runtime_array[mpiio_runtime->file_array_ndx]);
file->file_record = &(mpiio_runtime->file_record_array[mpiio_runtime->file_array_ndx]);
file->file_record->f_id = file_id;
+ file->file_record->rank = my_rank;
/* add new record to file hash table */
HASH_ADD(hlink, mpiio_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);
@@ -354,7 +926,7 @@ static struct mpiio_file_runtime* mpiio_file_by_name(const char *name)
/* get an MPIIO file record for the given file path, and also create a
* reference structure using the corresponding file handle
*/
-static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File *fh)
+static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI_File fh)
{
struct mpiio_file_runtime* file;
struct mpiio_file_runtime_ref* ref;
@@ -394,6 +966,56 @@ static struct mpiio_file_runtime* mpiio_file_by_name_setfh(const char* name, MPI
return(file);
}
+/* get an MPIIO file record for the given file handle */
+static struct mpiio_file_runtime* mpiio_file_by_fh(MPI_File fh)
+{
+ struct mpiio_file_runtime_ref* ref;
+
+ if(!mpiio_runtime || instrumentation_disabled)
+ return(NULL);
+
+ /* search hash table for existing file ref for this file handle */
+ HASH_FIND(hlink, mpiio_runtime->fh_hash, &fh, sizeof(fh), ref);
+ if(ref)
+ return(ref->file);
+
+ return(NULL);
+}
+
+/* free up reference data structures for the given file handle */
+static void mpiio_file_close_fh(MPI_File fh)
+{
+ struct mpiio_file_runtime_ref* ref;
+
+ if(!mpiio_runtime || instrumentation_disabled)
+ return;
+
+ /* search hash table for this fd */
+ HASH_FIND(hlink, mpiio_runtime->fh_hash, &fh, sizeof(fh), ref);
+ if(ref)
+ {
+ /* we have a reference, delete it */
+ HASH_DELETE(hlink, mpiio_runtime->fh_hash, ref);
+ free(ref);
+ }
+
+ return;
+}
+
+/* compare function for sorting file records by descending rank */
+static int mpiio_record_compare(const void* a_p, const void* b_p)
+{
+ const struct darshan_mpiio_file* a = a_p;
+ const struct darshan_mpiio_file* b = b_p;
+
+ if(a->rank < b->rank)
+ return 1;
+ if(a->rank > b->rank)
+ return -1;
+
+ return 0;
+}
+
static void mpiio_record_reduction_op(
void* infile_v,
void* inoutfile_v,
@@ -403,7 +1025,7 @@ static void mpiio_record_reduction_op(
struct darshan_mpiio_file tmp_file;
struct darshan_mpiio_file *infile = infile_v;
struct darshan_mpiio_file *inoutfile = inoutfile_v;
- int i, j;
+ int i, j, k;
assert(mpiio_runtime);
@@ -415,19 +1037,58 @@ static void mpiio_record_reduction_op(
tmp_file.rank = -1;
/* sum */
- for(j=DARSHAN_MPIIO_INDEP_OPENS; j<=DARSHAN_MPIIO_HINTS; j++)
+ for(j=MPIIO_INDEP_OPENS; j<=MPIIO_VIEWS; j++)
+ {
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
+ }
+
+ tmp_file.counters[MPIIO_MODE] = infile->counters[MPIIO_MODE];
+
+ /* sum */
+ for(j=MPIIO_BYTES_READ; j<=MPIIO_RW_SWITCHES; j++)
+ {
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
+ }
+
+ /* skip MPIIO_MAX_*_TIME_SIZE; handled in floating point section */
+
+ for(j=MPIIO_SIZE_READ_AGG_0_100; j<=MPIIO_SIZE_WRITE_AGG_1G_PLUS; j++)
{
tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
- /* sum (floating point) */
- for(j=DARSHAN_MPIIO_F_META_TIME; j<=DARSHAN_MPIIO_F_META_TIME; j++)
+ /* first collapse any duplicates */
+ for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)
+ {
+ for(k=MPIIO_ACCESS1_ACCESS; k<=MPIIO_ACCESS4_ACCESS; k++)
+ {
+ if(infile->counters[j] == inoutfile->counters[k])
+ {
+ infile->counters[j+4] += inoutfile->counters[k+4];
+ inoutfile->counters[k] = 0;
+ inoutfile->counters[k+4] = 0;
+ }
+ }
+ }
+
+ /* first set */
+ for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)
+ {
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[MPIIO_ACCESS1_ACCESS]),
+ &(tmp_file.counters[MPIIO_ACCESS1_COUNT]), infile->counters[j],
+ infile->counters[j+4]);
+ }
+
+ /* second set */
+ for(j=MPIIO_ACCESS1_ACCESS; j<=MPIIO_ACCESS4_ACCESS; j++)
{
- tmp_file.fcounters[j] = infile->fcounters[j] + inoutfile->fcounters[j];
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[MPIIO_ACCESS1_ACCESS]),
+ &(tmp_file.counters[MPIIO_ACCESS1_COUNT]), inoutfile->counters[j],
+ inoutfile->counters[j+4]);
}
/* min non-zero (if available) value */
- for(j=DARSHAN_MPIIO_F_OPEN_TIMESTAMP; j<=DARSHAN_MPIIO_F_OPEN_TIMESTAMP; j++)
+ for(j=MPIIO_F_OPEN_TIMESTAMP; j<=MPIIO_F_WRITE_START_TIMESTAMP; j++)
{
if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)
tmp_file.fcounters[j] = inoutfile->fcounters[j];
@@ -435,6 +1096,96 @@ static void mpiio_record_reduction_op(
tmp_file.fcounters[j] = infile->fcounters[j];
}
+ /* max */
+ for(j=MPIIO_F_READ_END_TIMESTAMP; j<= MPIIO_F_CLOSE_TIMESTAMP; j++)
+ {
+ if(infile->fcounters[j] > inoutfile->fcounters[j])
+ tmp_file.fcounters[j] = infile->fcounters[j];
+ else
+ tmp_file.fcounters[j] = inoutfile->fcounters[j];
+ }
+
+ /* sum */
+ for(j=MPIIO_F_READ_TIME; j<=MPIIO_F_META_TIME; j++)
+ {
+ tmp_file.counters[j] = infile->fcounters[j] + inoutfile->fcounters[j];
+ }
+
+ /* max (special case) */
+ if(infile->fcounters[MPIIO_F_MAX_READ_TIME] >
+ inoutfile->fcounters[MPIIO_F_MAX_READ_TIME])
+ {
+ tmp_file.fcounters[MPIIO_F_MAX_READ_TIME] =
+ infile->fcounters[MPIIO_F_MAX_READ_TIME];
+ tmp_file.counters[MPIIO_MAX_READ_TIME_SIZE] =
+ infile->counters[MPIIO_MAX_READ_TIME_SIZE];
+ }
+ else
+ {
+ tmp_file.fcounters[MPIIO_F_MAX_READ_TIME] =
+ inoutfile->fcounters[MPIIO_F_MAX_READ_TIME];
+ tmp_file.counters[MPIIO_MAX_READ_TIME_SIZE] =
+ inoutfile->counters[MPIIO_MAX_READ_TIME_SIZE];
+ }
+
+ if(infile->fcounters[MPIIO_F_MAX_WRITE_TIME] >
+ inoutfile->fcounters[MPIIO_F_MAX_WRITE_TIME])
+ {
+ tmp_file.fcounters[MPIIO_F_MAX_WRITE_TIME] =
+ infile->fcounters[MPIIO_F_MAX_WRITE_TIME];
+ tmp_file.counters[MPIIO_MAX_WRITE_TIME_SIZE] =
+ infile->counters[MPIIO_MAX_WRITE_TIME_SIZE];
+ }
+ else
+ {
+ tmp_file.fcounters[MPIIO_F_MAX_WRITE_TIME] =
+ inoutfile->fcounters[MPIIO_F_MAX_WRITE_TIME];
+ tmp_file.counters[MPIIO_MAX_WRITE_TIME_SIZE] =
+ inoutfile->counters[MPIIO_MAX_WRITE_TIME_SIZE];
+ }
+
+ /* min (zeroes are ok here; some procs don't do I/O) */
+ if(infile->fcounters[MPIIO_F_FASTEST_RANK_TIME] <
+ inoutfile->fcounters[MPIIO_F_FASTEST_RANK_TIME])
+ {
+ tmp_file.counters[MPIIO_FASTEST_RANK] =
+ infile->counters[MPIIO_FASTEST_RANK];
+ tmp_file.counters[MPIIO_FASTEST_RANK_BYTES] =
+ infile->counters[MPIIO_FASTEST_RANK_BYTES];
+ tmp_file.fcounters[MPIIO_F_FASTEST_RANK_TIME] =
+ infile->fcounters[MPIIO_F_FASTEST_RANK_TIME];
+ }
+ else
+ {
+ tmp_file.counters[MPIIO_FASTEST_RANK] =
+ inoutfile->counters[MPIIO_FASTEST_RANK];
+ tmp_file.counters[MPIIO_FASTEST_RANK_BYTES] =
+ inoutfile->counters[MPIIO_FASTEST_RANK_BYTES];
+ tmp_file.fcounters[MPIIO_F_FASTEST_RANK_TIME] =
+ inoutfile->fcounters[MPIIO_F_FASTEST_RANK_TIME];
+ }
+
+ /* max */
+ if(infile->fcounters[MPIIO_F_SLOWEST_RANK_TIME] >
+ inoutfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME])
+ {
+ tmp_file.counters[MPIIO_SLOWEST_RANK] =
+ infile->counters[MPIIO_SLOWEST_RANK];
+ tmp_file.counters[MPIIO_SLOWEST_RANK_BYTES] =
+ infile->counters[MPIIO_SLOWEST_RANK_BYTES];
+ tmp_file.fcounters[MPIIO_F_SLOWEST_RANK_TIME] =
+ infile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];
+ }
+ else
+ {
+ tmp_file.counters[MPIIO_SLOWEST_RANK] =
+ inoutfile->counters[MPIIO_SLOWEST_RANK];
+ tmp_file.counters[MPIIO_SLOWEST_RANK_BYTES] =
+ inoutfile->counters[MPIIO_SLOWEST_RANK_BYTES];
+ tmp_file.fcounters[MPIIO_F_SLOWEST_RANK_TIME] =
+ inoutfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];
+ }
+
/* update pointers */
*inoutfile = tmp_file;
inoutfile++;
@@ -444,68 +1195,251 @@ static void mpiio_record_reduction_op(
return;
}
-static void mpiio_setup_reduction(
- darshan_record_id *shared_recs,
- int *shared_rec_count,
- void **send_buf,
- void **recv_buf,
- int *rec_size)
+static void mpiio_shared_record_variance(MPI_Comm mod_comm,
+ struct darshan_mpiio_file *inrec_array, struct darshan_mpiio_file *outrec_array,
+ int shared_rec_count)
{
- struct mpiio_file_runtime *file;
+ MPI_Datatype var_dt;
+ MPI_Op var_op;
int i;
- int count;
+ struct darshan_variance_dt *var_send_buf = NULL;
+ struct darshan_variance_dt *var_recv_buf = NULL;
- assert(mpiio_runtime);
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_variance_dt),
+ MPI_BYTE, &var_dt);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&var_dt);
+
+ DARSHAN_MPI_CALL(PMPI_Op_create)(darshan_variance_reduce, 1, &var_op);
+
+ var_send_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));
+ if(!var_send_buf)
+ return;
+
+ if(my_rank == 0)
+ {
+ var_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));
+
+ if(!var_recv_buf)
+ return;
+ }
+
+ /* get total i/o time variances for shared records */
+
+ for(i=0; i<shared_rec_count; i++)
+ {
+ var_send_buf[i].n = 1;
+ var_send_buf[i].S = 0;
+ var_send_buf[i].T = inrec_array[i].fcounters[MPIIO_F_READ_TIME] +
+ inrec_array[i].fcounters[MPIIO_F_WRITE_TIME] +
+ inrec_array[i].fcounters[MPIIO_F_META_TIME];
+ }
- /* necessary initialization of shared records (e.g., change rank to -1) */
- for(i = 0,count = 0; i < *shared_rec_count; i++)
+ DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,
+ var_dt, var_op, 0, mod_comm);
+
+ if(my_rank == 0)
{
- HASH_FIND(hlink, mpiio_runtime->file_hash, &shared_recs[i],
- sizeof(darshan_record_id), file);
- if (!file) {
- continue;
+ for(i=0; i<shared_rec_count; i++)
+ {
+ outrec_array[i].fcounters[MPIIO_F_VARIANCE_RANK_TIME] =
+ (var_recv_buf[i].S / var_recv_buf[i].n);
}
- count++;
- file->file_record->rank = -1;
}
- /* sort the array of files descending by rank so that we get all of the
- * shared files (marked by rank -1) in a contiguous portion at end
- * of the array
- */
- qsort(mpiio_runtime->file_record_array, mpiio_runtime->file_array_ndx,
- sizeof(struct darshan_mpiio_file), mpiio_file_compare);
+ /* get total bytes moved variances for shared records */
- /* make *send_buf point to the shared files at the end of sorted array */
- *send_buf =
- &(mpiio_runtime->file_record_array[mpiio_runtime->file_array_ndx-(count)]);
+ for(i=0; i<shared_rec_count; i++)
+ {
+ var_send_buf[i].n = 1;
+ var_send_buf[i].S = 0;
+ var_send_buf[i].T = (double)
+ inrec_array[i].counters[MPIIO_BYTES_READ] +
+ inrec_array[i].counters[MPIIO_BYTES_WRITTEN];
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,
+ var_dt, var_op, 0, mod_comm);
- /* allocate memory for the reduction output on rank 0 */
- if((my_rank == 0) && (count > 0))
+ if(my_rank == 0)
{
- *recv_buf = malloc(count * sizeof(struct darshan_mpiio_file));
- mpiio_runtime->red_buf = *recv_buf;
+ for(i=0; i<shared_rec_count; i++)
+ {
+ outrec_array[i].fcounters[MPIIO_F_VARIANCE_RANK_BYTES] =
+ (var_recv_buf[i].S / var_recv_buf[i].n);
+ }
}
- *rec_size = sizeof(struct darshan_mpiio_file);
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&var_dt);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&var_op);
+ free(var_send_buf);
+ free(var_recv_buf);
+
+ return;
+}
+
+/**************************************************************************
+ * Functions exported by MPI-IO module for coordinating with darshan-core *
+ **************************************************************************/
+
+static void mpiio_begin_shutdown()
+{
+ assert(mpiio_runtime);
- mpiio_runtime->shared_rec_count = count;
+ MPIIO_LOCK();
+ /* disable further instrumentation while Darshan shuts down */
+ instrumentation_disabled = 1;
+ MPIIO_UNLOCK();
return;
}
-/* compare function for sorting file records by descending rank */
-static int mpiio_file_compare(const void* a_p, const void* b_p)
+static void mpiio_get_output_data(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **mpiio_buf,
+ int *mpiio_buf_sz)
{
- const struct darshan_mpiio_file* a = a_p;
- const struct darshan_mpiio_file* b = b_p;
+ struct mpiio_file_runtime *file;
+ struct mpiio_file_runtime* tmp;
+ int i;
+ double mpiio_time;
+ void *red_send_buf = NULL;
+ void *red_recv_buf = NULL;
+ MPI_Datatype red_type;
+ MPI_Op red_op;
- if(a->rank < b->rank)
- return 1;
- if(a->rank > b->rank)
- return -1;
+ assert(mpiio_runtime);
- return 0;
+ /* go through and set the 4 most common access sizes for MPI-IO */
+ for(i = 0; i < mpiio_runtime->file_array_ndx; i++)
+ {
+ tmp = &(mpiio_runtime->file_runtime_array[i]);
+
+ /* common access sizes */
+ darshan_walk_common_vals(tmp->access_root,
+ &(tmp->file_record->counters[MPIIO_ACCESS1_ACCESS]),
+ &(tmp->file_record->counters[MPIIO_ACCESS1_COUNT]));
+ }
+
+ /* if there are globally shared files, do a shared file reduction */
+ if(shared_rec_count)
+ {
+ /* necessary initialization of shared records */
+ for(i = 0; i < shared_rec_count; i++)
+ {
+ HASH_FIND(hlink, mpiio_runtime->file_hash, &shared_recs[i],
+ sizeof(darshan_record_id), file);
+ assert(file);
+
+ mpiio_time =
+ file->file_record->fcounters[MPIIO_F_READ_TIME] +
+ file->file_record->fcounters[MPIIO_F_WRITE_TIME] +
+ file->file_record->fcounters[MPIIO_F_META_TIME];
+
+ /* initialize fastest/slowest info prior to the reduction */
+ file->file_record->counters[MPIIO_FASTEST_RANK] =
+ file->file_record->rank;
+ file->file_record->counters[MPIIO_FASTEST_RANK_BYTES] =
+ file->file_record->counters[MPIIO_BYTES_READ] +
+ file->file_record->counters[MPIIO_BYTES_WRITTEN];
+ file->file_record->fcounters[MPIIO_F_FASTEST_RANK_TIME] =
+ mpiio_time;
+
+ /* until reduction occurs, we assume that this rank is both
+ * the fastest and slowest. It is up to the reduction operator
+ * to find the true min and max.
+ */
+ file->file_record->counters[MPIIO_SLOWEST_RANK] =
+ file->file_record->counters[MPIIO_FASTEST_RANK];
+ file->file_record->counters[MPIIO_SLOWEST_RANK_BYTES] =
+ file->file_record->counters[MPIIO_FASTEST_RANK_BYTES];
+ file->file_record->fcounters[MPIIO_F_SLOWEST_RANK_TIME] =
+ file->file_record->fcounters[MPIIO_F_FASTEST_RANK_TIME];
+
+ file->file_record->rank = -1;
+ }
+
+ /* sort the array of files descending by rank so that we get all of the
+ * shared files (marked by rank -1) in a contiguous portion at end
+ * of the array
+ */
+ qsort(mpiio_runtime->file_record_array, mpiio_runtime->file_array_ndx,
+ sizeof(struct darshan_mpiio_file), mpiio_record_compare);
+
+ /* make *send_buf point to the shared files at the end of sorted array */
+ red_send_buf =
+ &(mpiio_runtime->file_record_array[mpiio_runtime->file_array_ndx-shared_rec_count]);
+
+ /* allocate memory for the reduction output on rank 0 */
+ if(my_rank == 0)
+ {
+ red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_mpiio_file));
+ if(!red_recv_buf)
+ return;
+ }
+
+ /* construct a datatype for a MPIIO file record. This is serving no purpose
+ * except to make sure we can do a reduction on proper boundaries
+ */
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_mpiio_file),
+ MPI_BYTE, &red_type);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);
+
+ /* register a MPIIO file record reduction operator */
+ DARSHAN_MPI_CALL(PMPI_Op_create)(mpiio_record_reduction_op, 1, &red_op);
+
+ /* reduce shared MPIIO file records */
+ DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,
+ shared_rec_count, red_type, red_op, 0, mod_comm);
+
+ /* get the time and byte variances for shared files */
+ mpiio_shared_record_variance(mod_comm, red_send_buf, red_recv_buf,
+ shared_rec_count);
+
+ /* clean up reduction state */
+ if(my_rank == 0)
+ {
+ int tmp_ndx = mpiio_runtime->file_array_ndx - shared_rec_count;
+ memcpy(&(mpiio_runtime->file_record_array[tmp_ndx]), red_recv_buf,
+ shared_rec_count * sizeof(struct darshan_mpiio_file));
+ free(red_recv_buf);
+ }
+ else
+ {
+ mpiio_runtime->file_array_ndx -= shared_rec_count;
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);
+ }
+
+ *mpiio_buf = (void *)(mpiio_runtime->file_record_array);
+ *mpiio_buf_sz = mpiio_runtime->file_array_ndx * sizeof(struct darshan_mpiio_file);
+
+ return;
+}
+
+static void mpiio_shutdown()
+{
+ struct mpiio_file_runtime_ref *ref, *tmp;
+
+ assert(mpiio_runtime);
+
+ HASH_ITER(hlink, mpiio_runtime->fh_hash, ref, tmp)
+ {
+ HASH_DELETE(hlink, mpiio_runtime->fh_hash, ref);
+ free(ref);
+ }
+
+ HASH_CLEAR(hlink, mpiio_runtime->file_hash); /* these entries are freed all at once below */
+
+ free(mpiio_runtime->file_runtime_array);
+ free(mpiio_runtime->file_record_array);
+ free(mpiio_runtime);
+ mpiio_runtime = NULL;
+
+ return;
}
/*
diff --git a/darshan-runtime/lib/darshan-null.c b/darshan-runtime/lib/darshan-null.c
index 0a864e2..ed9eb7d 100644
--- a/darshan-runtime/lib/darshan-null.c
+++ b/darshan-runtime/lib/darshan-null.c
@@ -28,18 +28,12 @@
* that may be reused and expanded on by developers adding new instrumentation modules.
*/
-/* TODO: this probably shouldn't be here -- LD_PRELOADing POSIX wrappers will cause MPI linker dependency */
-#ifdef DARSHAN_PRELOAD
-extern double (*__real_PMPI_Comm_rank)(MPI_Comm comm, int *rank);
-#endif
-
/* The DARSHAN_FORWARD_DECL macro (defined in darshan.h) is used to provide forward
* declarations for wrapped funcions, regardless if Darshan is used with statically
* or dynamically linked executables.
*/
DARSHAN_FORWARD_DECL(foo, int, (const char *name, int arg1, int arg2));
-
/* The null_record_runtime structure maintains necessary runtime metadata
* for a "NULL" module data record (darshan_null_record structure, defined
* in darshan-null-log-format.h). This metadata assists with the instrumenting
@@ -125,7 +119,8 @@ static struct null_record_runtime* null_record_by_name(const char *name);
/* forward declaration for module functions needed to interface with darshan-core */
static void null_begin_shutdown(void);
-static void null_get_output_data(void **buffer, int *size);
+static void null_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **null_buf, int *null_buf_sz);
static void null_shutdown(void);
/* macros for obtaining/releasing the "NULL" module lock */
@@ -145,13 +140,13 @@ static void null_shutdown(void);
rec = null_record_by_name(__name); \
if(!rec) break; \
/* increment counter indicating number of calls to 'bar' */ \
- DARSHAN_COUNTER_INC(rec->record_p, NULL_BARS, 1); \
+ rec->record_p->counters[NULL_BARS] += 1; \
/* store data value for most recent call to 'bar' */ \
- DARSHAN_COUNTER_SET(rec->record_p, NULL_BAR_DAT, __dat); \
+ rec->record_p->counters[NULL_BAR_DAT] = __dat; \
/* store timestamp of most recent call to 'bar' */ \
- DARSHAN_COUNTER_F_SET(rec->record_p, NULL_F_BAR_TIMESTAMP, __tm1); \
+ rec->record_p->fcounters[NULL_F_BAR_TIMESTAMP] = __tm1; \
/* store duration of most recent call to 'bar' */ \
- DARSHAN_COUNTER_F_SET(rec->record_p, NULL_F_BAR_DURATION, elapsed); \
+ rec->record_p->fcounters[NULL_F_BAR_DURATION] = elapsed; \
} while(0)
/**********************************************************
@@ -209,8 +204,6 @@ static void null_runtime_initialize()
struct darshan_module_funcs null_mod_fns =
{
.begin_shutdown = &null_begin_shutdown,
- .setup_reduction = NULL, /* "NULL" module does not do reductions */
- .record_reduction_op = NULL, /* "NULL" module does not do reductions */
.get_output_data = &null_get_output_data,
.shutdown = &null_shutdown
};
@@ -224,6 +217,7 @@ static void null_runtime_initialize()
darshan_core_register_module(
DARSHAN_NULL_MOD, /* Darshan module identifier, defined in darshan-log-format.h */
&null_mod_fns,
+ &my_rank,
&mem_limit,
NULL);
@@ -262,9 +256,6 @@ static void null_runtime_initialize()
memset(null_runtime->record_array, 0, null_runtime->rec_array_size *
sizeof(struct darshan_null_record));
- /* TODO: we should move this out of here.. perhaps register_module returns rank? */
- DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &my_rank);
-
return;
}
@@ -342,17 +333,30 @@ static void null_begin_shutdown()
/* Pass output data for the "NULL" module back to darshan-core to log to file. */
static void null_get_output_data(
- void **buffer,
- int *size)
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **null_buf,
+ int *null_buf_sz)
{
assert(null_runtime);
+ /* NOTE: this function can be used to run collective operations prior to
+ * shutting down the module, as implied by the MPI communicator passed in
+ * as the first agrument. Typically, module developers will want to run a
+ * reduction on shared data records (passed in in the 'shared_recs' array),
+ * but other collective routines can be run here as well. For a detailed
+ * example illustrating how to run shared file reductions, consider the
+ * POSIX or MPIIO instrumentation modules, as they both implement this
+ * functionality.
+ */
+
/* Just set the output buffer to point at the array of the "NULL" module's
* I/O records, and set the output size according to the number of records
* currently being tracked.
*/
- *buffer = (void *)(null_runtime->record_array);
- *size = null_runtime->rec_array_ndx * sizeof(struct darshan_null_record);
+ *null_buf = (void *)(null_runtime->record_array);
+ *null_buf_sz = null_runtime->rec_array_ndx * sizeof(struct darshan_null_record);
return;
}
diff --git a/darshan-runtime/lib/darshan-pnetcdf-stubs.c b/darshan-runtime/lib/darshan-pnetcdf-stubs.c
new file mode 100644
index 0000000..5c54336
--- /dev/null
+++ b/darshan-runtime/lib/darshan-pnetcdf-stubs.c
@@ -0,0 +1,77 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+/* This file contains stubs for the ncmpi functions intercepted by Darshan.
+ * They are defined as weak symbols in order to satisfy dependencies of the
+ * pnetcdf wrappers in cases where pnetcdf is not being used.
+ */
+
+#include "darshan-runtime-config.h"
+#include <stdio.h>
+#include <pthread.h>
+#include <string.h>
+#include "mpi.h"
+#include "darshan.h"
+
+int ncmpi_create(MPI_Comm comm, const char *path,
+ int cmode, MPI_Info info, int *ncidp) __attribute__((weak));
+
+int ncmpi_create(MPI_Comm comm, const char *path,
+ int cmode, MPI_Info info, int *ncidp)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan ncmpi_create() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+}
+
+int ncmpi_open(MPI_Comm comm, const char *path,
+ int omode, MPI_Info info, int *ncidp) __attribute__((weak));
+
+int ncmpi_open(MPI_Comm comm, const char *path,
+ int omode, MPI_Info info, int *ncidp)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan ncmpi_open() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+}
+
+int ncmpi_close(int ncid) __attribute__((weak));
+
+int ncmpi_close(int ncid)
+{
+ int rank;
+
+ MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+ if(rank == 0)
+ {
+ fprintf(stderr, "WARNING: Darshan ncmpi_close() stub called; this is probably the result of a link-time problem.\n");
+ }
+
+ return(-1);
+
+}
+
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-runtime/lib/darshan-pnetcdf.c b/darshan-runtime/lib/darshan-pnetcdf.c
new file mode 100644
index 0000000..5f4c0d7
--- /dev/null
+++ b/darshan-runtime/lib/darshan-pnetcdf.c
@@ -0,0 +1,588 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#include "darshan-runtime-config.h"
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <stdarg.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <search.h>
+#include <assert.h>
+#define __USE_GNU
+#include <pthread.h>
+
+#include "uthash.h"
+
+#include "darshan.h"
+#include "darshan-pnetcdf-log-format.h"
+#include "darshan-dynamic.h"
+
+DARSHAN_FORWARD_DECL(ncmpi_create, int, (MPI_Comm comm, const char *path, int cmode, MPI_Info info, int *ncidp));
+DARSHAN_FORWARD_DECL(ncmpi_open, int, (MPI_Comm comm, const char *path, int omode, MPI_Info info, int *ncidp));
+DARSHAN_FORWARD_DECL(ncmpi_close, int, (int ncid));
+
+/* structure to track i/o stats for a given PNETCDF file at runtime */
+struct pnetcdf_file_runtime
+{
+ struct darshan_pnetcdf_file* file_record;
+ UT_hash_handle hlink;
+};
+
+/* structure to associate a PNETCDF ncid with an existing file runtime structure */
+struct pnetcdf_file_runtime_ref
+{
+ struct pnetcdf_file_runtime* file;
+ int ncid;
+ UT_hash_handle hlink;
+};
+
+/* necessary state for storing PNETCDF file records and coordinating with
+ * darshan-core at shutdown time
+ */
+struct pnetcdf_runtime
+{
+ struct pnetcdf_file_runtime* file_runtime_array;
+ struct darshan_pnetcdf_file* file_record_array;
+ int file_array_size;
+ int file_array_ndx;
+ struct pnetcdf_file_runtime *file_hash;
+ struct pnetcdf_file_runtime_ref* ncid_hash;
+};
+
+static struct pnetcdf_runtime *pnetcdf_runtime = NULL;
+static pthread_mutex_t pnetcdf_runtime_mutex = PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP;
+static int instrumentation_disabled = 0;
+static int my_rank = -1;
+
+static void pnetcdf_runtime_initialize(void);
+static struct pnetcdf_file_runtime* pnetcdf_file_by_name(const char *name);
+static struct pnetcdf_file_runtime* pnetcdf_file_by_name_setncid(const char* name, int ncid);
+static struct pnetcdf_file_runtime* pnetcdf_file_by_ncid(int ncid);
+static void pnetcdf_file_close_ncid(int ncid);
+static int pnetcdf_record_compare(const void* a, const void* b);
+static void pnetcdf_record_reduction_op(void* infile_v, void* inoutfile_v,
+ int *len, MPI_Datatype *datatype);
+
+static void pnetcdf_begin_shutdown(void);
+static void pnetcdf_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **pnetcdf_buf, int *pnetcdf_buf_sz);
+static void pnetcdf_shutdown(void);
+
+#define PNETCDF_LOCK() pthread_mutex_lock(&pnetcdf_runtime_mutex)
+#define PNETCDF_UNLOCK() pthread_mutex_unlock(&pnetcdf_runtime_mutex)
+
+/*********************************************************
+ * Wrappers for PNETCDF functions of interest *
+ *********************************************************/
+
+int DARSHAN_DECL(ncmpi_create)(MPI_Comm comm, const char *path,
+ int cmode, MPI_Info info, int *ncidp)
+{
+ int ret;
+ struct pnetcdf_file_runtime* file;
+ char* tmp;
+ int comm_size;
+ double tm1;
+
+ MAP_OR_FAIL(ncmpi_create);
+
+ tm1 = darshan_core_wtime();
+ ret = __real_ncmpi_create(comm, path, cmode, info, ncidp);
+ if(ret == 0)
+ {
+ /* use ROMIO approach to strip prefix if present */
+ /* strip off prefix if there is one, but only skip prefixes
+ * if they are greater than length one to allow for windows
+ * drive specifications (e.g. c:\...)
+ */
+ tmp = strchr(path, ':');
+ if (tmp > path + 1) {
+ path = tmp + 1;
+ }
+
+ PNETCDF_LOCK();
+ pnetcdf_runtime_initialize();
+ file = pnetcdf_file_by_name_setncid(path, (*ncidp));
+ if(file)
+ {
+ if(file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] == 0)
+ file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] = tm1;
+ DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);
+ if(comm_size == 1)
+ {
+ file->file_record->counters[PNETCDF_INDEP_OPENS] += 1;
+ }
+ else
+ {
+ file->file_record->counters[PNETCDF_COLL_OPENS] += 1;
+ }
+ }
+ PNETCDF_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(ncmpi_open)(MPI_Comm comm, const char *path,
+ int omode, MPI_Info info, int *ncidp)
+{
+ int ret;
+ struct pnetcdf_file_runtime* file;
+ char* tmp;
+ int comm_size;
+ double tm1;
+
+ MAP_OR_FAIL(ncmpi_open);
+
+ tm1 = darshan_core_wtime();
+ ret = __real_ncmpi_open(comm, path, omode, info, ncidp);
+ if(ret == 0)
+ {
+ /* use ROMIO approach to strip prefix if present */
+ /* strip off prefix if there is one, but only skip prefixes
+ * if they are greater than length one to allow for windows
+ * drive specifications (e.g. c:\...)
+ */
+ tmp = strchr(path, ':');
+ if (tmp > path + 1) {
+ path = tmp + 1;
+ }
+
+ PNETCDF_LOCK();
+ pnetcdf_runtime_initialize();
+ file = pnetcdf_file_by_name_setncid(path, (*ncidp));
+ if(file)
+ {
+ if(file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] == 0)
+ file->file_record->fcounters[PNETCDF_F_OPEN_TIMESTAMP] = tm1;
+ DARSHAN_MPI_CALL(PMPI_Comm_size)(comm, &comm_size);
+ if(comm_size == 1)
+ {
+ file->file_record->counters[PNETCDF_INDEP_OPENS] += 1;
+ }
+ else
+ {
+ file->file_record->counters[PNETCDF_COLL_OPENS] += 1;
+ }
+ }
+ PNETCDF_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(ncmpi_close)(int ncid)
+{
+ struct pnetcdf_file_runtime* file;
+ int ret;
+
+ MAP_OR_FAIL(ncmpi_close);
+
+ ret = __real_ncmpi_close(ncid);
+
+ PNETCDF_LOCK();
+ pnetcdf_runtime_initialize();
+ file = pnetcdf_file_by_ncid(ncid);
+ if(file)
+ {
+ file->file_record->fcounters[PNETCDF_F_CLOSE_TIMESTAMP] =
+ darshan_core_wtime();
+ pnetcdf_file_close_ncid(ncid);
+ }
+ PNETCDF_UNLOCK();
+
+ return(ret);
+}
+
+/************************************************************
+ * Internal functions for manipulating PNETCDF module state *
+ ************************************************************/
+
+/* initialize internal PNETCDF module data strucutres and register with darshan-core */
+static void pnetcdf_runtime_initialize()
+{
+ int mem_limit;
+ struct darshan_module_funcs pnetcdf_mod_fns =
+ {
+ .begin_shutdown = &pnetcdf_begin_shutdown,
+ .get_output_data = &pnetcdf_get_output_data,
+ .shutdown = &pnetcdf_shutdown
+ };
+
+ /* don't do anything if already initialized or instrumenation is disabled */
+ if(pnetcdf_runtime || instrumentation_disabled)
+ return;
+
+ /* register pnetcdf module with darshan-core */
+ darshan_core_register_module(
+ DARSHAN_PNETCDF_MOD,
+ &pnetcdf_mod_fns,
+ &my_rank,
+ &mem_limit,
+ NULL);
+
+ /* return if no memory assigned by darshan-core */
+ if(mem_limit == 0)
+ return;
+
+ pnetcdf_runtime = malloc(sizeof(*pnetcdf_runtime));
+ if(!pnetcdf_runtime)
+ return;
+ memset(pnetcdf_runtime, 0, sizeof(*pnetcdf_runtime));
+
+ /* set maximum number of file records according to max memory limit */
+ /* NOTE: maximum number of records is based on the size of a pnetcdf file record */
+ /* TODO: should we base memory usage off file record or total runtime structure sizes? */
+ pnetcdf_runtime->file_array_size = mem_limit / sizeof(struct darshan_pnetcdf_file);
+ pnetcdf_runtime->file_array_ndx = 0;
+
+ /* allocate array of runtime file records */
+ pnetcdf_runtime->file_runtime_array = malloc(pnetcdf_runtime->file_array_size *
+ sizeof(struct pnetcdf_file_runtime));
+ pnetcdf_runtime->file_record_array = malloc(pnetcdf_runtime->file_array_size *
+ sizeof(struct darshan_pnetcdf_file));
+ if(!pnetcdf_runtime->file_runtime_array || !pnetcdf_runtime->file_record_array)
+ {
+ pnetcdf_runtime->file_array_size = 0;
+ return;
+ }
+ memset(pnetcdf_runtime->file_runtime_array, 0, pnetcdf_runtime->file_array_size *
+ sizeof(struct pnetcdf_file_runtime));
+ memset(pnetcdf_runtime->file_record_array, 0, pnetcdf_runtime->file_array_size *
+ sizeof(struct darshan_pnetcdf_file));
+
+ return;
+}
+
+/* get a PNETCDF file record for the given file path */
+static struct pnetcdf_file_runtime* pnetcdf_file_by_name(const char *name)
+{
+ struct pnetcdf_file_runtime *file = NULL;
+ char *newname = NULL;
+ darshan_record_id file_id;
+
+ if(!pnetcdf_runtime || instrumentation_disabled)
+ return(NULL);
+
+ newname = darshan_clean_file_path(name);
+ if(!newname)
+ newname = (char*)name;
+
+ /* get a unique id for this file from darshan core */
+ darshan_core_register_record(
+ (void*)newname,
+ strlen(newname),
+ 1,
+ DARSHAN_PNETCDF_MOD,
+ &file_id,
+ NULL);
+
+ /* search the hash table for this file record, and return if found */
+ HASH_FIND(hlink, pnetcdf_runtime->file_hash, &file_id, sizeof(darshan_record_id), file);
+ if(file)
+ {
+ if(newname != name)
+ free(newname);
+ return(file);
+ }
+
+ if(pnetcdf_runtime->file_array_ndx < pnetcdf_runtime->file_array_size);
+ {
+ /* no existing record, assign a new file record from the global array */
+ file = &(pnetcdf_runtime->file_runtime_array[pnetcdf_runtime->file_array_ndx]);
+ file->file_record = &(pnetcdf_runtime->file_record_array[pnetcdf_runtime->file_array_ndx]);
+ file->file_record->f_id = file_id;
+ file->file_record->rank = my_rank;
+
+ /* add new record to file hash table */
+ HASH_ADD(hlink, pnetcdf_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);
+
+ pnetcdf_runtime->file_array_ndx++;
+ }
+
+ if(newname != name)
+ free(newname);
+ return(file);
+}
+
+/* get a PNETCDF file record for the given file path, and also create a
+ * reference structure using the returned ncid
+ */
+static struct pnetcdf_file_runtime* pnetcdf_file_by_name_setncid(const char* name, int ncid)
+{
+ struct pnetcdf_file_runtime* file;
+ struct pnetcdf_file_runtime_ref* ref;
+
+ if(!pnetcdf_runtime || instrumentation_disabled)
+ return(NULL);
+
+ /* find file record by name first */
+ file = pnetcdf_file_by_name(name);
+
+ if(!file)
+ return(NULL);
+
+ /* search hash table for existing file ref for this ncid */
+ HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);
+ if(ref)
+ {
+ /* we have a reference. Make sure it points to the correct file
+ * and return it
+ */
+ ref->file = file;
+ return(file);
+ }
+
+ /* if we hit this point, then we don't have a reference for this ncid
+ * in the table yet. Add it.
+ */
+ ref = malloc(sizeof(*ref));
+ if(!ref)
+ return(NULL);
+ memset(ref, 0, sizeof(*ref));
+
+ ref->file = file;
+ ref->ncid = ncid;
+ HASH_ADD(hlink, pnetcdf_runtime->ncid_hash, ncid, sizeof(int), ref);
+
+ return(file);
+}
+
+/* get a PNETCDF file record for the given ncid */
+static struct pnetcdf_file_runtime* pnetcdf_file_by_ncid(int ncid)
+{
+ struct pnetcdf_file_runtime_ref* ref;
+
+ if(!pnetcdf_runtime || instrumentation_disabled)
+ return(NULL);
+
+ /* search hash table for existing file ref for this ncid */
+ HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);
+ if(ref)
+ return(ref->file);
+
+ return(NULL);
+}
+
+/* free up PNETCDF reference data structures for the given ncid */
+static void pnetcdf_file_close_ncid(int ncid)
+{
+ struct pnetcdf_file_runtime_ref* ref;
+
+ if(!pnetcdf_runtime || instrumentation_disabled)
+ return;
+
+ /* search hash table for this ncid */
+ HASH_FIND(hlink, pnetcdf_runtime->ncid_hash, &ncid, sizeof(int), ref);
+ if(ref)
+ {
+ /* we have a reference, delete it */
+ HASH_DELETE(hlink, pnetcdf_runtime->ncid_hash, ref);
+ free(ref);
+ }
+
+ return;
+}
+
+/* compare function for sorting file records by descending rank */
+static int pnetcdf_record_compare(const void* a_p, const void* b_p)
+{
+ const struct darshan_pnetcdf_file* a = a_p;
+ const struct darshan_pnetcdf_file* b = b_p;
+
+ if(a->rank < b->rank)
+ return 1;
+ if(a->rank > b->rank)
+ return -1;
+
+ return 0;
+}
+
+static void pnetcdf_record_reduction_op(void* infile_v, void* inoutfile_v,
+ int *len, MPI_Datatype *datatype)
+{
+ struct darshan_pnetcdf_file tmp_file;
+ struct darshan_pnetcdf_file *infile = infile_v;
+ struct darshan_pnetcdf_file *inoutfile = inoutfile_v;
+ int i, j;
+
+ assert(pnetcdf_runtime);
+
+ for(i=0; i<*len; i++)
+ {
+ memset(&tmp_file, 0, sizeof(struct darshan_pnetcdf_file));
+ tmp_file.f_id = infile->f_id;
+ tmp_file.rank = -1;
+
+ /* sum */
+ for(j=PNETCDF_INDEP_OPENS; j<=PNETCDF_COLL_OPENS; j++)
+ {
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
+ }
+
+ /* min non-zero (if available) value */
+ for(j=PNETCDF_F_OPEN_TIMESTAMP; j<=PNETCDF_F_OPEN_TIMESTAMP; j++)
+ {
+ if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)
+ tmp_file.fcounters[j] = inoutfile->fcounters[j];
+ else
+ tmp_file.fcounters[j] = infile->fcounters[j];
+ }
+
+ /* max */
+ for(j=PNETCDF_F_CLOSE_TIMESTAMP; j<=PNETCDF_F_CLOSE_TIMESTAMP; j++)
+ {
+ if(infile->fcounters[j] > inoutfile->fcounters[j])
+ tmp_file.fcounters[j] = infile->fcounters[j];
+ else
+ tmp_file.fcounters[j] = inoutfile->fcounters[j];
+ }
+
+ /* update pointers */
+ *inoutfile = tmp_file;
+ inoutfile++;
+ infile++;
+ }
+
+ return;
+}
+
+/***************************************************************************
+ * Functions exported by PNETCDF module for coordinating with darshan-core *
+ ***************************************************************************/
+
+static void pnetcdf_begin_shutdown()
+{
+ assert(pnetcdf_runtime);
+
+ PNETCDF_LOCK();
+ /* disable further instrumentation while Darshan shuts down */
+ instrumentation_disabled = 1;
+ PNETCDF_UNLOCK();
+
+ return;
+}
+
+static void pnetcdf_get_output_data(
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **pnetcdf_buf,
+ int *pnetcdf_buf_sz)
+{
+ struct pnetcdf_file_runtime *file;
+ int i;
+ struct darshan_pnetcdf_file *red_send_buf = NULL;
+ struct darshan_pnetcdf_file *red_recv_buf = NULL;
+ MPI_Datatype red_type;
+ MPI_Op red_op;
+
+ assert(pnetcdf_runtime);
+
+ /* if there are globally shared files, do a shared file reduction */
+ if(shared_rec_count)
+ {
+ /* necessary initialization of shared records */
+ for(i = 0; i < shared_rec_count; i++)
+ {
+ HASH_FIND(hlink, pnetcdf_runtime->file_hash, &shared_recs[i],
+ sizeof(darshan_record_id), file);
+ assert(file);
+
+ file->file_record->rank = -1;
+ }
+
+ /* sort the array of files descending by rank so that we get all of the
+ * shared files (marked by rank -1) in a contiguous portion at end
+ * of the array
+ */
+ qsort(pnetcdf_runtime->file_record_array, pnetcdf_runtime->file_array_ndx,
+ sizeof(struct darshan_pnetcdf_file), pnetcdf_record_compare);
+
+ /* make *send_buf point to the shared files at the end of sorted array */
+ red_send_buf =
+ &(pnetcdf_runtime->file_record_array[pnetcdf_runtime->file_array_ndx-shared_rec_count]);
+
+ /* allocate memory for the reduction output on rank 0 */
+ if(my_rank == 0)
+ {
+ red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_pnetcdf_file));
+ if(!red_recv_buf)
+ return;
+ }
+
+ /* construct a datatype for a PNETCDF file record. This is serving no purpose
+ * except to make sure we can do a reduction on proper boundaries
+ */
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_pnetcdf_file),
+ MPI_BYTE, &red_type);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);
+
+ /* register a PNETCDF file record reduction operator */
+ DARSHAN_MPI_CALL(PMPI_Op_create)(pnetcdf_record_reduction_op, 1, &red_op);
+
+ /* reduce shared PNETCDF file records */
+ DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,
+ shared_rec_count, red_type, red_op, 0, mod_comm);
+
+ /* clean up reduction state */
+ if(my_rank == 0)
+ {
+ int tmp_ndx = pnetcdf_runtime->file_array_ndx - shared_rec_count;
+ memcpy(&(pnetcdf_runtime->file_record_array[tmp_ndx]), red_recv_buf,
+ shared_rec_count * sizeof(struct darshan_pnetcdf_file));
+ free(red_recv_buf);
+ }
+ else
+ {
+ pnetcdf_runtime->file_array_ndx -= shared_rec_count;
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);
+ }
+
+ *pnetcdf_buf = (void *)(pnetcdf_runtime->file_record_array);
+ *pnetcdf_buf_sz = pnetcdf_runtime->file_array_ndx * sizeof(struct darshan_pnetcdf_file);
+
+ return;
+}
+
+static void pnetcdf_shutdown()
+{
+ struct pnetcdf_file_runtime_ref *ref, *tmp;
+
+ assert(pnetcdf_runtime);
+
+ HASH_ITER(hlink, pnetcdf_runtime->ncid_hash, ref, tmp)
+ {
+ HASH_DELETE(hlink, pnetcdf_runtime->ncid_hash, ref);
+ free(ref);
+ }
+
+ HASH_CLEAR(hlink, pnetcdf_runtime->file_hash); /* these entries are freed all at once below */
+
+ free(pnetcdf_runtime->file_runtime_array);
+ free(pnetcdf_runtime->file_record_array);
+ free(pnetcdf_runtime);
+ pnetcdf_runtime = NULL;
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-runtime/lib/darshan-posix.c b/darshan-runtime/lib/darshan-posix.c
index 2f03f7a..0ef1a08 100644
--- a/darshan-runtime/lib/darshan-posix.c
+++ b/darshan-runtime/lib/darshan-posix.c
@@ -27,8 +27,11 @@
#include <pthread.h>
#include "uthash.h"
+#include "utlist.h"
+
#include "darshan.h"
#include "darshan-posix-log-format.h"
+#include "darshan-dynamic.h"
#ifndef HAVE_OFF64_T
typedef int64_t off64_t;
@@ -37,11 +40,6 @@ typedef int64_t off64_t;
#define aiocb64 aiocb
#endif
-/* TODO: this probably shouldn't be here long term -- MPI symbols mess up LD_PRELOAD */
-#ifdef DARSHAN_PRELOAD
-extern double (*__real_PMPI_Comm_rank)(MPI_Comm comm, int *rank);
-#endif
-
/* TODO: more libc, fgetc, etc etc etc. */
DARSHAN_FORWARD_DECL(open, int, (const char *path, int flags, ...));
@@ -79,31 +77,21 @@ DARSHAN_FORWARD_DECL(fsync, int, (int fd));
DARSHAN_FORWARD_DECL(fdatasync, int, (int fd));
DARSHAN_FORWARD_DECL(close, int, (int fd));
DARSHAN_FORWARD_DECL(fclose, int, (FILE *fp));
-/* TODO aio */
-/* TODO listio */
-
-/* maximum number of access sizes and stride sizes that darshan will track
- * per file at runtime; at log time they will be reduced into the 4 most
- * frequently occurring ones
- */
-#define POSIX_MAX_ACCESS_COUNT_RUNTIME 32
-
-enum posix_io_type
+DARSHAN_FORWARD_DECL(aio_read, int, (struct aiocb *aiocbp));
+DARSHAN_FORWARD_DECL(aio_write, int, (struct aiocb *aiocbp));
+DARSHAN_FORWARD_DECL(aio_read64, int, (struct aiocb64 *aiocbp));
+DARSHAN_FORWARD_DECL(aio_write64, int, (struct aiocb64 *aiocbp));
+DARSHAN_FORWARD_DECL(aio_return, ssize_t, (struct aiocb *aiocbp));
+DARSHAN_FORWARD_DECL(aio_return64, ssize_t, (struct aiocb64 *aiocbp));
+DARSHAN_FORWARD_DECL(lio_listio, int, (int mode, struct aiocb *const aiocb_list[], int nitems, struct sigevent *sevp));
+DARSHAN_FORWARD_DECL(lio_listio64, int, (int mode, struct aiocb64 *const aiocb_list[], int nitems, struct sigevent *sevp));
+
+/* struct to track information about aio operations in flight */
+struct posix_aio_tracker
{
- POSIX_READ = 1,
- POSIX_WRITE = 2,
-};
-
-enum posix_counter_type
-{
- POSIX_COUNTER_ACCESS,
- POSIX_COUNTER_STRIDE
-};
-
-struct posix_access_counter
-{
- int64_t size;
- int freq;
+ double tm1;
+ void *aiocbp;
+ struct posix_aio_tracker* next;
};
/* The posix_file_runtime structure maintains necessary runtime metadata
@@ -135,7 +123,7 @@ struct posix_file_runtime
int64_t offset;
int64_t last_byte_read;
int64_t last_byte_written;
- enum posix_io_type last_io_type;
+ enum darshan_io_type last_io_type;
double last_meta_end;
double last_read_end;
double last_write_end;
@@ -143,6 +131,7 @@ struct posix_file_runtime
int access_count;
void* stride_root;
int stride_count;
+ struct posix_aio_tracker* aio_list;
UT_hash_handle hlink;
};
@@ -188,8 +177,6 @@ struct posix_runtime
int file_array_ndx;
struct posix_file_runtime* file_hash;
struct posix_file_runtime_ref* fd_hash;
- void *red_buf;
- int shared_rec_count;
};
static struct posix_runtime *posix_runtime = NULL;
@@ -198,59 +185,28 @@ static int instrumentation_disabled = 0;
static int my_rank = -1;
static int darshan_mem_alignment = 1;
-/* global variables for determining 4 most common accesses/strides */
-static struct posix_file_runtime* walker_file = NULL;
-static int walker_validx;
-static int walker_cntidx;
-
static void posix_runtime_initialize(void);
static struct posix_file_runtime* posix_file_by_name(const char *name);
static struct posix_file_runtime* posix_file_by_name_setfd(const char* name, int fd);
static struct posix_file_runtime* posix_file_by_fd(int fd);
static void posix_file_close_fd(int fd);
-static void posix_access_counter(struct posix_file_runtime* file, ssize_t size,
- enum posix_counter_type type);
-static void posix_access_walker(const void* nodep, const VISIT which, const int depth);
-static void posix_walk_file_accesses(void);
-static int posix_access_compare(const void* a_p, const void* b_p);
-static int posix_file_compare(const void* a, const void* b);
-
-static void posix_begin_shutdown(void);
-static void posix_setup_reduction(darshan_record_id *shared_recs, int *shared_rec_count,
- void **send_buf, void **recv_buf, int *rec_size);
+static void posix_aio_tracker_add(int fd, void *aiocbp);
+static struct posix_aio_tracker* posix_aio_tracker_del(int fd, void *aiocbp);
+static int posix_record_compare(const void* a, const void* b);
static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
int *len, MPI_Datatype *datatype);
-static void posix_get_output_data(void **buffer, int *size);
+static void posix_shared_record_variance(MPI_Comm mod_comm,
+ struct darshan_posix_file *inrec_array, struct darshan_posix_file *outrec_array,
+ int shared_rec_count);
+
+static void posix_begin_shutdown(void);
+static void posix_get_output_data(MPI_Comm mod_comm, darshan_record_id *shared_recs,
+ int shared_rec_count, void **posix_buf, int *posix_buf_sz);
static void posix_shutdown(void);
#define POSIX_LOCK() pthread_mutex_lock(&posix_runtime_mutex)
#define POSIX_UNLOCK() pthread_mutex_unlock(&posix_runtime_mutex)
-#define POSIX_COMMON_COUNTER_INC(__rec_p, __value, __count, __validx, __cntidx) do {\
- int i; \
- int set = 0; \
- int64_t min = DARSHAN_COUNTER_VALUE(__rec_p, __cntidx); \
- int min_index = 0; \
- if(__value == 0) break; \
- for(i=0; i<4; i++) { \
- /* increment bucket if already exists */ \
- if(DARSHAN_COUNTER_VALUE(__rec_p, __validx + i) == __value) { \
- DARSHAN_COUNTER_INC(__rec_p, __cntidx + i, __count); \
- set = 1; \
- break; \
- } \
- /* otherwise find the least frequently used bucket */ \
- else if(DARSHAN_COUNTER_VALUE(__rec_p, __cntidx + i) < min) { \
- min = DARSHAN_COUNTER_VALUE(__rec_p, __cntidx + i); \
- min_index = i; \
- } \
- } \
- if(!set && (__count > min)) { \
- DARSHAN_COUNTER_SET(__rec_p, __cntidx+min_index, __count); \
- DARSHAN_COUNTER_SET(__rec_p, __validx+min_index, __value); \
- } \
-} while(0)
-
#define POSIX_RECORD_OPEN(__ret, __path, __mode, __stream_flag, __tm1, __tm2) do { \
struct posix_file_runtime* file; \
char* exclude; \
@@ -264,19 +220,18 @@ static void posix_shutdown(void);
if(exclude) break; \
file = posix_file_by_name_setfd(__path, __ret); \
if(!file) break; \
- file->file_record->rank = my_rank; \
if(__mode) \
- DARSHAN_COUNTER_SET(file->file_record, POSIX_MODE, __mode); \
+ file->file_record->counters[POSIX_MODE] = __mode; \
file->offset = 0; \
file->last_byte_written = 0; \
file->last_byte_read = 0; \
if(__stream_flag)\
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FOPENS, 1); \
+ file->file_record->counters[POSIX_FOPENS] += 1; \
else \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_OPENS, 1); \
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, POSIX_F_OPEN_TIMESTAMP) == 0) \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_OPEN_TIMESTAMP, __tm1); \
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record, __tm1, __tm2, file->last_meta_end, POSIX_F_META_TIME); \
+ file->file_record->counters[POSIX_OPENS] += 1; \
+ if(file->file_record->fcounters[POSIX_F_OPEN_TIMESTAMP] == 0) \
+ file->file_record->fcounters[POSIX_F_OPEN_TIMESTAMP] = __tm1; \
+ DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_META_TIME], __tm1, __tm2, file->last_meta_end); \
} while(0)
#define POSIX_RECORD_READ(__ret, __fd, __pread_flag, __pread_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \
@@ -293,9 +248,9 @@ static void posix_shutdown(void);
else \
this_offset = file->offset; \
if(this_offset > file->last_byte_read) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_SEQ_READS, 1); \
+ file->file_record->counters[POSIX_SEQ_READS] += 1; \
if(this_offset == (file->last_byte_read + 1)) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_CONSEC_READS, 1); \
+ file->file_record->counters[POSIX_CONSEC_READS] += 1; \
if(this_offset > 0 && this_offset > file->last_byte_read \
&& file->last_byte_read != 0) \
stride = this_offset - file->last_byte_read - 1; \
@@ -303,30 +258,31 @@ static void posix_shutdown(void);
stride = 0; \
file->last_byte_read = this_offset + __ret - 1; \
file->offset = this_offset + __ret; \
- DARSHAN_COUNTER_MAX(file->file_record, POSIX_MAX_BYTE_READ, (this_offset + __ret - 1)); \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_BYTES_READ, __ret); \
- if(__stream_flag)\
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FREADS, 1); \
- else\
- DARSHAN_COUNTER_INC(file->file_record, POSIX_READS, 1); \
- DARSHAN_BUCKET_INC(file->file_record, POSIX_SIZE_READ_0_100, __ret); \
- posix_access_counter(file, __ret, POSIX_COUNTER_ACCESS); \
- posix_access_counter(file, stride, POSIX_COUNTER_STRIDE); \
+ if(file->file_record->counters[POSIX_MAX_BYTE_READ] < (this_offset + __ret - 1)) \
+ file->file_record->counters[POSIX_MAX_BYTE_READ] = (this_offset + __ret - 1); \
+ file->file_record->counters[POSIX_BYTES_READ] += __ret; \
+ if(__stream_flag) \
+ file->file_record->counters[POSIX_FREADS] += 1; \
+ else \
+ file->file_record->counters[POSIX_READS] += 1; \
+ DARSHAN_BUCKET_INC(&(file->file_record->counters[POSIX_SIZE_READ_0_100]), __ret); \
+ darshan_common_val_counter(&file->access_root, &file->access_count, __ret); \
+ darshan_common_val_counter(&file->stride_root, &file->stride_count, stride); \
if(!__aligned) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_MEM_NOT_ALIGNED, 1); \
- file_alignment = DARSHAN_COUNTER_VALUE(file->file_record, POSIX_FILE_ALIGNMENT); \
+ file->file_record->counters[POSIX_MEM_NOT_ALIGNED] += 1; \
+ file_alignment = file->file_record->counters[POSIX_FILE_ALIGNMENT]; \
if(file_alignment > 0 && (this_offset % file_alignment) != 0) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FILE_NOT_ALIGNED, 1); \
- if(file->last_io_type == POSIX_WRITE) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_RW_SWITCHES, 1); \
- file->last_io_type = POSIX_READ; \
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record, __tm1, __tm2, file->last_read_end, POSIX_F_READ_TIME); \
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, POSIX_F_READ_START_TIMESTAMP) == 0) \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_READ_START_TIMESTAMP, __tm1); \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_READ_END_TIMESTAMP, __tm2); \
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, POSIX_F_MAX_READ_TIME) < __elapsed){ \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_MAX_READ_TIME, __elapsed); \
- DARSHAN_COUNTER_SET(file->file_record, POSIX_MAX_READ_TIME_SIZE, __ret); } \
+ file->file_record->counters[POSIX_FILE_NOT_ALIGNED] += 1; \
+ if(file->last_io_type == DARSHAN_IO_WRITE) \
+ file->file_record->counters[POSIX_RW_SWITCHES] += 1; \
+ file->last_io_type = DARSHAN_IO_READ; \
+ if(file->file_record->fcounters[POSIX_F_READ_START_TIMESTAMP] == 0) \
+ file->file_record->fcounters[POSIX_F_READ_START_TIMESTAMP] = __tm1; \
+ file->file_record->fcounters[POSIX_F_READ_END_TIMESTAMP] = __tm2; \
+ if(file->file_record->fcounters[POSIX_F_MAX_READ_TIME] < __elapsed) { \
+ file->file_record->fcounters[POSIX_F_MAX_READ_TIME] = __elapsed; \
+ file->file_record->counters[POSIX_MAX_READ_TIME_SIZE] = __ret; } \
+ DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_READ_TIME], __tm1, __tm2, file->last_read_end); \
} while(0)
#define POSIX_RECORD_WRITE(__ret, __fd, __pwrite_flag, __pwrite_offset, __aligned, __stream_flag, __tm1, __tm2) do{ \
@@ -343,9 +299,9 @@ static void posix_shutdown(void);
else \
this_offset = file->offset; \
if(this_offset > file->last_byte_written) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_SEQ_WRITES, 1); \
+ file->file_record->counters[POSIX_SEQ_WRITES] += 1; \
if(this_offset == (file->last_byte_written + 1)) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_CONSEC_WRITES, 1); \
+ file->file_record->counters[POSIX_CONSEC_WRITES] += 1; \
if(this_offset > 0 && this_offset > file->last_byte_written \
&& file->last_byte_written != 0) \
stride = this_offset - file->last_byte_written - 1; \
@@ -353,30 +309,31 @@ static void posix_shutdown(void);
stride = 0; \
file->last_byte_written = this_offset + __ret - 1; \
file->offset = this_offset + __ret; \
- DARSHAN_COUNTER_MAX(file->file_record, POSIX_MAX_BYTE_WRITTEN, (this_offset + __ret - 1)); \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_BYTES_WRITTEN, __ret); \
+ if(file->file_record->counters[POSIX_MAX_BYTE_WRITTEN] < (this_offset + __ret - 1)) \
+ file->file_record->counters[POSIX_MAX_BYTE_WRITTEN] = (this_offset + __ret - 1); \
+ file->file_record->counters[POSIX_BYTES_WRITTEN] += __ret; \
if(__stream_flag) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FWRITES, 1); \
+ file->file_record->counters[POSIX_FWRITES] += 1; \
else \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_WRITES, 1); \
- DARSHAN_BUCKET_INC(file->file_record, POSIX_SIZE_WRITE_0_100, __ret); \
- posix_access_counter(file, __ret, POSIX_COUNTER_ACCESS); \
- posix_access_counter(file, stride, POSIX_COUNTER_STRIDE); \
+ file->file_record->counters[POSIX_WRITES] += 1; \
+ DARSHAN_BUCKET_INC(&(file->file_record->counters[POSIX_SIZE_WRITE_0_100]), __ret); \
+ darshan_common_val_counter(&file->access_root, &file->access_count, __ret); \
+ darshan_common_val_counter(&file->stride_root, &file->stride_count, stride); \
if(!__aligned) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_MEM_NOT_ALIGNED, 1); \
- file_alignment = DARSHAN_COUNTER_VALUE(file->file_record, POSIX_FILE_ALIGNMENT); \
+ file->file_record->counters[POSIX_MEM_NOT_ALIGNED] += 1; \
+ file_alignment = file->file_record->counters[POSIX_FILE_ALIGNMENT]; \
if(file_alignment > 0 && (this_offset % file_alignment) != 0) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FILE_NOT_ALIGNED, 1); \
- if(file->last_io_type == POSIX_READ) \
- DARSHAN_COUNTER_INC(file->file_record, POSIX_RW_SWITCHES, 1); \
- file->last_io_type = POSIX_WRITE; \
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record, __tm1, __tm2, file->last_write_end, POSIX_F_WRITE_TIME); \
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, POSIX_F_WRITE_START_TIMESTAMP) == 0) \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_WRITE_START_TIMESTAMP, __tm1); \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_WRITE_END_TIMESTAMP, __tm2); \
- if(DARSHAN_COUNTER_F_VALUE(file->file_record, POSIX_F_MAX_WRITE_TIME) < __elapsed){ \
- DARSHAN_COUNTER_F_SET(file->file_record, POSIX_F_MAX_WRITE_TIME, __elapsed); \
- DARSHAN_COUNTER_SET(file->file_record, POSIX_MAX_WRITE_TIME_SIZE, __ret); } \
+ file->file_record->counters[POSIX_FILE_NOT_ALIGNED] += 1; \
+ if(file->last_io_type == DARSHAN_IO_READ) \
+ file->file_record->counters[POSIX_RW_SWITCHES] += 1; \
+ file->last_io_type = DARSHAN_IO_WRITE; \
+ if(file->file_record->fcounters[POSIX_F_WRITE_START_TIMESTAMP] == 0) \
+ file->file_record->fcounters[POSIX_F_WRITE_START_TIMESTAMP] = __tm1; \
+ file->file_record->fcounters[POSIX_F_WRITE_END_TIMESTAMP] = __tm2; \
+ if(file->file_record->fcounters[POSIX_F_MAX_WRITE_TIME] < __elapsed) { \
+ file->file_record->fcounters[POSIX_F_MAX_WRITE_TIME] = __elapsed; \
+ file->file_record->counters[POSIX_MAX_WRITE_TIME_SIZE] = __ret; } \
+ DARSHAN_TIMER_INC_NO_OVERLAP(file->file_record->fcounters[POSIX_F_WRITE_TIME], __tm1, __tm2, file->last_write_end); \
} while(0)
#define POSIX_LOOKUP_RECORD_STAT(__path, __statbuf, __tm1, __tm2) do { \
@@ -397,9 +354,8 @@ static void posix_shutdown(void);
} while(0)
#define POSIX_RECORD_STAT(__file, __statbuf, __tm1, __tm2) do { \
- (__file)->file_record->rank = my_rank; \
- DARSHAN_COUNTER_F_INC_NO_OVERLAP((__file)->file_record, __tm1, __tm2, (__file)->last_meta_end, POSIX_F_META_TIME); \
- DARSHAN_COUNTER_INC((__file)->file_record, POSIX_STATS, 1); \
+ DARSHAN_TIMER_INC_NO_OVERLAP((__file)->file_record->fcounters[POSIX_F_META_TIME], __tm1, __tm2, (__file)->last_meta_end); \
+ (__file)->file_record->counters[POSIX_STATS] += 1; \
} while(0)
/**********************************************************
@@ -906,9 +862,10 @@ off_t DARSHAN_DECL(lseek)(int fd, off_t offset, int whence)
if(file)
{
file->offset = ret;
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_meta_end, POSIX_F_META_TIME);
- DARSHAN_COUNTER_INC(file->file_record, POSIX_SEEKS, 1);
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ file->file_record->counters[POSIX_SEEKS] += 1;
}
POSIX_UNLOCK();
}
@@ -936,9 +893,10 @@ off_t DARSHAN_DECL(lseek64)(int fd, off_t offset, int whence)
if(file)
{
file->offset = ret;
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_meta_end, POSIX_F_META_TIME);
- DARSHAN_COUNTER_INC(file->file_record, POSIX_SEEKS, 1);
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ file->file_record->counters[POSIX_SEEKS] += 1;
}
POSIX_UNLOCK();
}
@@ -966,9 +924,10 @@ int DARSHAN_DECL(fseek)(FILE *stream, long offset, int whence)
if(file)
{
file->offset = ftell(stream);
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_meta_end, POSIX_F_META_TIME);
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FSEEKS, 1);
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
+ file->file_record->counters[POSIX_FSEEKS] += 1;
}
POSIX_UNLOCK();
}
@@ -1135,7 +1094,7 @@ void* DARSHAN_DECL(mmap)(void *addr, size_t length, int prot, int flags,
file = posix_file_by_fd(fd);
if(file)
{
- DARSHAN_COUNTER_INC(file->file_record, POSIX_MMAPS, 1);
+ file->file_record->counters[POSIX_MMAPS] += 1;
}
POSIX_UNLOCK();
@@ -1159,7 +1118,7 @@ void* DARSHAN_DECL(mmap64)(void *addr, size_t length, int prot, int flags,
file = posix_file_by_fd(fd);
if(file)
{
- DARSHAN_COUNTER_INC(file->file_record, POSIX_MMAPS, 1);
+ file->file_record->counters[POSIX_MMAPS] += 1;
}
POSIX_UNLOCK();
@@ -1186,9 +1145,10 @@ int DARSHAN_DECL(fsync)(int fd)
file = posix_file_by_fd(fd);
if(file)
{
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_write_end, POSIX_F_WRITE_TIME);
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FSYNCS, 1);
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_WRITE_TIME],
+ tm1, tm2, file->last_write_end);
+ file->file_record->counters[POSIX_FSYNCS] += 1;
}
POSIX_UNLOCK();
@@ -1215,9 +1175,10 @@ int DARSHAN_DECL(fdatasync)(int fd)
file = posix_file_by_fd(fd);
if(file)
{
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_write_end, POSIX_F_WRITE_TIME);
- DARSHAN_COUNTER_INC(file->file_record, POSIX_FDSYNCS, 1);
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_WRITE_TIME],
+ tm1, tm2, file->last_write_end);
+ file->file_record->counters[POSIX_FDSYNCS] += 1;
}
POSIX_UNLOCK();
@@ -1243,10 +1204,11 @@ int DARSHAN_DECL(close)(int fd)
{
file->last_byte_written = 0;
file->last_byte_read = 0;
- DARSHAN_COUNTER_F_SET(file->file_record,
- POSIX_F_CLOSE_TIMESTAMP, darshan_core_wtime());
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_meta_end, POSIX_F_META_TIME);
+ file->file_record->fcounters[POSIX_F_CLOSE_TIMESTAMP] =
+ darshan_core_wtime();
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
posix_file_close_fd(fd);
}
POSIX_UNLOCK();
@@ -1274,10 +1236,11 @@ int DARSHAN_DECL(fclose)(FILE *fp)
{
file->last_byte_written = 0;
file->last_byte_read = 0;
- DARSHAN_COUNTER_F_SET(file->file_record,
- POSIX_F_CLOSE_TIMESTAMP, darshan_core_wtime());
- DARSHAN_COUNTER_F_INC_NO_OVERLAP(file->file_record,
- tm1, tm2, file->last_meta_end, POSIX_F_META_TIME);
+ file->file_record->fcounters[POSIX_F_CLOSE_TIMESTAMP] =
+ darshan_core_wtime();
+ DARSHAN_TIMER_INC_NO_OVERLAP(
+ file->file_record->fcounters[POSIX_F_META_TIME],
+ tm1, tm2, file->last_meta_end);
posix_file_close_fd(fd);
}
POSIX_UNLOCK();
@@ -1285,6 +1248,200 @@ int DARSHAN_DECL(fclose)(FILE *fp)
return(ret);
}
+int DARSHAN_DECL(aio_read)(struct aiocb *aiocbp)
+{
+ int ret;
+
+ MAP_OR_FAIL(aio_read);
+
+ ret = __real_aio_read(aiocbp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(aio_write)(struct aiocb *aiocbp)
+{
+ int ret;
+
+ MAP_OR_FAIL(aio_write);
+
+ ret = __real_aio_write(aiocbp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(aio_read64)(struct aiocb64 *aiocbp)
+{
+ int ret;
+
+ MAP_OR_FAIL(aio_read64);
+
+ ret = __real_aio_read64(aiocbp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(aio_write64)(struct aiocb64 *aiocbp)
+{
+ int ret;
+
+ MAP_OR_FAIL(aio_write64);
+
+ ret = __real_aio_write64(aiocbp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ posix_aio_tracker_add(aiocbp->aio_fildes, aiocbp);
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
+ssize_t DARSHAN_DECL(aio_return)(struct aiocb *aiocbp)
+{
+ int ret;
+ double tm2;
+ struct posix_aio_tracker *tmp;
+ int aligned_flag = 0;
+
+ MAP_OR_FAIL(aio_return);
+
+ ret = __real_aio_return(aiocbp);
+ tm2 = darshan_core_wtime();
+
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ tmp = posix_aio_tracker_del(aiocbp->aio_fildes, aiocbp);
+ if (tmp)
+ {
+ if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)
+ aligned_flag = 1;
+ if(aiocbp->aio_lio_opcode == LIO_WRITE)
+ {
+ POSIX_RECORD_WRITE(ret, aiocbp->aio_fildes,
+ 1, aiocbp->aio_offset, aligned_flag, 0,
+ tmp->tm1, tm2);
+ }
+ else if(aiocbp->aio_lio_opcode == LIO_READ)
+ {
+ POSIX_RECORD_READ(ret, aiocbp->aio_fildes,
+ 1, aiocbp->aio_offset, aligned_flag, 0,
+ tmp->tm1, tm2);
+ }
+ free(tmp);
+ }
+ POSIX_UNLOCK();
+
+ return(ret);
+}
+
+ssize_t DARSHAN_DECL(aio_return64)(struct aiocb64 *aiocbp)
+{
+ int ret;
+ double tm2;
+ struct posix_aio_tracker *tmp;
+ int aligned_flag = 0;
+
+ MAP_OR_FAIL(aio_return64);
+
+ ret = __real_aio_return64(aiocbp);
+ tm2 = darshan_core_wtime();
+
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ tmp = posix_aio_tracker_del(aiocbp->aio_fildes, aiocbp);
+ if (tmp)
+ {
+ if((unsigned long)aiocbp->aio_buf % darshan_mem_alignment == 0)
+ aligned_flag = 1;
+ if(aiocbp->aio_lio_opcode == LIO_WRITE)
+ {
+ POSIX_RECORD_WRITE(ret, aiocbp->aio_fildes,
+ 1, aiocbp->aio_offset, aligned_flag, 0,
+ tmp->tm1, tm2);
+ }
+ else if(aiocbp->aio_lio_opcode == LIO_READ)
+ {
+ POSIX_RECORD_READ(ret, aiocbp->aio_fildes,
+ 1, aiocbp->aio_offset, aligned_flag, 0,
+ tmp->tm1, tm2);
+ }
+ free(tmp);
+ }
+ POSIX_UNLOCK();
+
+ return(ret);
+}
+
+int DARSHAN_DECL(lio_listio)(int mode, struct aiocb *const aiocb_list[],
+ int nitems, struct sigevent *sevp)
+{
+ int ret;
+ int i;
+
+ MAP_OR_FAIL(lio_listio);
+
+ ret = __real_lio_listio(mode, aiocb_list, nitems, sevp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ for(i = 0; i < nitems; i++)
+ {
+ posix_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);
+ }
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
+int DARSHAN_DECL(lio_listio64)(int mode, struct aiocb64 *const aiocb_list[],
+ int nitems, struct sigevent *sevp)
+{
+ int ret;
+ int i;
+
+ MAP_OR_FAIL(lio_listio64);
+
+ ret = __real_lio_listio64(mode, aiocb_list, nitems, sevp);
+ if(ret == 0)
+ {
+ POSIX_LOCK();
+ posix_runtime_initialize();
+ for(i = 0; i < nitems; i++)
+ {
+ posix_aio_tracker_add(aiocb_list[i]->aio_fildes, aiocb_list[i]);
+ }
+ POSIX_UNLOCK();
+ }
+
+ return(ret);
+}
+
/**********************************************************
* Internal functions for manipulating POSIX module state *
**********************************************************/
@@ -1296,8 +1453,6 @@ static void posix_runtime_initialize()
struct darshan_module_funcs posix_mod_fns =
{
.begin_shutdown = &posix_begin_shutdown,
- .setup_reduction = &posix_setup_reduction,
- .record_reduction_op = &posix_record_reduction_op,
.get_output_data = &posix_get_output_data,
.shutdown = &posix_shutdown
};
@@ -1310,6 +1465,7 @@ static void posix_runtime_initialize()
darshan_core_register_module(
DARSHAN_POSIX_MOD,
&posix_mod_fns,
+ &my_rank,
&mem_limit,
&darshan_mem_alignment);
@@ -1343,9 +1499,6 @@ static void posix_runtime_initialize()
memset(posix_runtime->file_record_array, 0, posix_runtime->file_array_size *
sizeof(struct darshan_posix_file));
- /* TODO: can we move this out of here? perhaps register_module returns rank? */
- DARSHAN_MPI_CALL(PMPI_Comm_rank)(MPI_COMM_WORLD, &my_rank);
-
return;
}
@@ -1388,8 +1541,9 @@ static struct posix_file_runtime* posix_file_by_name(const char *name)
file = &(posix_runtime->file_runtime_array[posix_runtime->file_array_ndx]);
file->file_record = &(posix_runtime->file_record_array[posix_runtime->file_array_ndx]);
file->file_record->f_id = file_id;
- DARSHAN_COUNTER_SET(file->file_record, POSIX_MEM_ALIGNMENT, darshan_mem_alignment);
- DARSHAN_COUNTER_SET(file->file_record, POSIX_FILE_ALIGNMENT, file_alignment);
+ file->file_record->rank = my_rank;
+ file->file_record->counters[POSIX_MEM_ALIGNMENT] = darshan_mem_alignment;
+ file->file_record->counters[POSIX_FILE_ALIGNMENT] = file_alignment;
/* add new record to file hash table */
HASH_ADD(hlink, posix_runtime->file_hash, file_record->f_id, sizeof(darshan_record_id), file);
@@ -1481,136 +1635,8 @@ static void posix_file_close_fd(int fd)
return;
}
-static void posix_access_counter(struct posix_file_runtime* file, ssize_t size,
- enum posix_counter_type type)
-{
- struct posix_access_counter* counter;
- struct posix_access_counter* found;
- void* tmp;
- void** root;
- int* count;
- struct posix_access_counter tmp_counter;
-
- /* don't count sizes or strides of 0 */
- if(size == 0)
- return;
-
- switch(type)
- {
- case POSIX_COUNTER_ACCESS:
- root = &file->access_root;
- count = &file->access_count;
- break;
- case POSIX_COUNTER_STRIDE:
- root = &file->stride_root;
- count = &file->stride_count;
- break;
- default:
- return;
- }
-
- /* check to see if this size is already recorded */
- tmp_counter.size = size;
- tmp_counter.freq = 1;
- tmp = tfind(&tmp_counter, root, posix_access_compare);
- if(tmp)
- {
- found = *(struct posix_access_counter**)tmp;
- found->freq++;
- return;
- }
-
- /* we can add a new one as long as we haven't hit the limit */
- if(*count < POSIX_MAX_ACCESS_COUNT_RUNTIME)
- {
- counter = malloc(sizeof(*counter));
- if(!counter)
- {
- return;
- }
-
- counter->size = size;
- counter->freq = 1;
-
- tmp = tsearch(counter, root, posix_access_compare);
- found = *(struct posix_access_counter**)tmp;
- /* if we get a new answer out here we are in trouble; this was
- * already checked with the tfind()
- */
- assert(found == counter);
-
- (*count)++;
- }
-
- return;
-}
-
-static void posix_access_walker(const void* nodep, const VISIT which, const int depth)
-{
- struct posix_access_counter* counter;
-
- switch (which)
- {
- case postorder:
- case leaf:
- counter = *(struct posix_access_counter**)nodep;
- POSIX_COMMON_COUNTER_INC(walker_file->file_record, counter->size, counter->freq,
- walker_validx, walker_cntidx);
- default:
- break;
- }
-
- return;
-};
-
-/* posix_walk_file_accesses()
- *
- * goes through runtime collections of accesses sizes and chooses the 4 most
- * common for logging
- */
-static void posix_walk_file_accesses()
-{
- int i;
-
- for(i=0; i<posix_runtime->file_array_ndx; i++)
- {
- /* walk trees for both access sizes and stride sizes to pick 4 most
- * common of each
- */
-
- /* NOTE: setting global variables here for cp_access_walker() */
- walker_file = &posix_runtime->file_runtime_array[i];
- walker_validx = POSIX_ACCESS1_ACCESS;
- walker_cntidx = POSIX_ACCESS1_COUNT;
- twalk(walker_file->access_root,
- posix_access_walker);
- tdestroy(walker_file->access_root, free);
-
- walker_validx = POSIX_STRIDE1_STRIDE;
- walker_cntidx = POSIX_STRIDE1_COUNT;
- twalk(walker_file->stride_root,
- posix_access_walker);
- tdestroy(walker_file->stride_root, free);
- }
-
- return;
-}
-
-static int posix_access_compare(const void* a_p, const void* b_p)
-{
- const struct posix_access_counter* a = a_p;
- const struct posix_access_counter* b = b_p;
-
- if(a->size < b->size)
- return(-1);
- if(a->size > b->size)
- return(1);
- return(0);
-}
-
-
/* compare function for sorting file records by descending rank */
-static int posix_file_compare(const void* a_p, const void* b_p)
+static int posix_record_compare(const void* a_p, const void* b_p)
{
const struct darshan_posix_file* a = a_p;
const struct darshan_posix_file* b = b_p;
@@ -1623,114 +1649,56 @@ static int posix_file_compare(const void* a_p, const void* b_p)
return 0;
}
-/************************************************************************
- * Functions exported by this module for coordinating with darshan-core *
- ************************************************************************/
-
-static void posix_begin_shutdown()
-{
- assert(posix_runtime);
-
- POSIX_LOCK();
-
- /* go through file access data for each record and set the 4 most common
- * stride/access size counters.
- */
- posix_walk_file_accesses();
-
- /* disable further instrumentation while Darshan shuts down */
- instrumentation_disabled = 1;
- POSIX_UNLOCK();
-
- return;
-}
-
-static void posix_setup_reduction(
- darshan_record_id *shared_recs,
- int *shared_rec_count,
- void **send_buf,
- void **recv_buf,
- int *rec_size)
+/* finds the tracker structure for a given aio operation, removes it from
+ * the linked list for the darshan_file structure, and returns a pointer.
+ *
+ * returns NULL if aio operation not found
+ */
+static struct posix_aio_tracker* posix_aio_tracker_del(int fd, void *aiocbp)
{
- struct posix_file_runtime *file;
- int i;
- double posix_time;
- int count;
-
- assert(posix_runtime);
+ struct posix_aio_tracker *tracker = NULL, *iter, *tmp;
+ struct posix_file_runtime* file;
- /* necessary initialization of shared records (e.g., change rank to -1) */
- for(i = 0,count = 0; i < *shared_rec_count; i++)
+ file = posix_file_by_fd(fd);
+ if (file)
{
- HASH_FIND(hlink, posix_runtime->file_hash, &shared_recs[i],
- sizeof(darshan_record_id), file);
- if (!file) {
- continue;
+ LL_FOREACH_SAFE(file->aio_list, iter, tmp)
+ {
+ if (iter->aiocbp == aiocbp)
+ {
+ LL_DELETE(file->aio_list, iter);
+ tracker = iter;
+ break;
+ }
}
- count++;
-
- posix_time =
- file->file_record->fcounters[POSIX_F_READ_TIME] +
- file->file_record->fcounters[POSIX_F_WRITE_TIME] +
- file->file_record->fcounters[POSIX_F_META_TIME];
-
- /* initialize fastest/slowest info prior to the reduction */
- file->file_record->counters[POSIX_FASTEST_RANK] =
- file->file_record->rank;
- file->file_record->counters[POSIX_FASTEST_RANK_BYTES] =
- file->file_record->counters[POSIX_BYTES_READ] +
- file->file_record->counters[POSIX_BYTES_WRITTEN];
- file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME] =
- posix_time;
-
- /* until reduction occurs, we assume that this rank is both
- * the fastest and slowest. It is up to the reduction operator
- * to find the true min and max.
- */
- file->file_record->counters[POSIX_SLOWEST_RANK] =
- file->file_record->counters[POSIX_FASTEST_RANK];
- file->file_record->counters[POSIX_SLOWEST_RANK_BYTES] =
- file->file_record->counters[POSIX_FASTEST_RANK_BYTES];
- file->file_record->fcounters[POSIX_F_SLOWEST_RANK_TIME] =
- file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME];
-
- file->file_record->rank = -1;
}
- /* sort the array of files descending by rank so that we get all of the
- * shared files (marked by rank -1) in a contiguous portion at end
- * of the array
- */
- qsort(posix_runtime->file_record_array, posix_runtime->file_array_ndx,
- sizeof(struct darshan_posix_file), posix_file_compare);
+ return(tracker);
+}
- /* make *send_buf point to the shared files at the end of sorted array */
- *send_buf =
- &(posix_runtime->file_record_array[posix_runtime->file_array_ndx-(count)]);
+/* adds a tracker for the given aio operation */
+static void posix_aio_tracker_add(int fd, void *aiocbp)
+{
+ struct posix_aio_tracker* tracker;
+ struct posix_file_runtime* file;
- /* allocate memory for the reduction output on rank 0 */
- if((my_rank == 0) && (count > 0))
+ file = posix_file_by_fd(fd);
+ if (file)
{
- printf("shared count = %d\n", count);
- *recv_buf = malloc(count * sizeof(struct darshan_posix_file));
- posix_runtime->red_buf = *recv_buf;
- printf("recv_buf = %p\n", *recv_buf);
-// if(!(*recv_buf))
-// return;
+ tracker = malloc(sizeof(*tracker));
+ if (tracker)
+ {
+ tracker->tm1 = darshan_core_wtime();
+ tracker->aiocbp = aiocbp;
+ LL_PREPEND(file->aio_list, tracker);
+ }
}
- *rec_size = sizeof(struct darshan_posix_file);
-
- posix_runtime->shared_rec_count = count;
-
return;
}
-static void posix_record_reduction_op(
- void* infile_v,
- void* inoutfile_v,
- int *len,
- MPI_Datatype *datatype)
+static void posix_record_reduction_op(void* infile_v, void* inoutfile_v,
+ int *len, MPI_Datatype *datatype)
{
struct darshan_posix_file tmp_file;
struct darshan_posix_file *infile = infile_v;
@@ -1742,7 +1710,6 @@ static void posix_record_reduction_op(
for(i=0; i<*len; i++)
{
memset(&tmp_file, 0, sizeof(struct darshan_posix_file));
-
tmp_file.f_id = infile->f_id;
tmp_file.rank = -1;
@@ -1752,7 +1719,10 @@ static void posix_record_reduction_op(
tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
- tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
+ if(POSIX_FILE_PARTIAL(infile))
+ tmp_file.counters[POSIX_MODE] = inoutfile->counters[POSIX_MODE];
+ else
+ tmp_file.counters[POSIX_MODE] = infile->counters[POSIX_MODE];
/* sum */
for(j=POSIX_BYTES_READ; j<=POSIX_BYTES_WRITTEN; j++)
@@ -1772,26 +1742,30 @@ static void posix_record_reduction_op(
/* sum */
for(j=POSIX_CONSEC_READS; j<=POSIX_MEM_NOT_ALIGNED; j++)
{
- tmp_file.counters[j] = infile->counters[j] +
- inoutfile->counters[j];
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
- tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
- tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
+ if(POSIX_FILE_PARTIAL(infile))
+ tmp_file.counters[POSIX_MEM_ALIGNMENT] = inoutfile->counters[POSIX_MEM_ALIGNMENT];
+ else
+ tmp_file.counters[POSIX_MEM_ALIGNMENT] = infile->counters[POSIX_MEM_ALIGNMENT];
/* sum */
for(j=POSIX_FILE_NOT_ALIGNED; j<=POSIX_FILE_NOT_ALIGNED; j++)
{
- tmp_file.counters[j] = infile->counters[j] +
- inoutfile->counters[j];
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
+ if(POSIX_FILE_PARTIAL(infile))
+ tmp_file.counters[POSIX_FILE_ALIGNMENT] = inoutfile->counters[POSIX_FILE_ALIGNMENT];
+ else
+ tmp_file.counters[POSIX_FILE_ALIGNMENT] = infile->counters[POSIX_FILE_ALIGNMENT];
+
/* skip POSIX_MAX_*_TIME_SIZE; handled in floating point section */
for(j=POSIX_SIZE_READ_0_100; j<=POSIX_SIZE_WRITE_1G_PLUS; j++)
{
- tmp_file.counters[j] = infile->counters[j] +
- inoutfile->counters[j];
+ tmp_file.counters[j] = infile->counters[j] + inoutfile->counters[j];
}
/* first collapse any duplicates */
@@ -1811,14 +1785,16 @@ static void posix_record_reduction_op(
/* first set */
for(j=POSIX_STRIDE1_STRIDE; j<=POSIX_STRIDE4_STRIDE; j++)
{
- POSIX_COMMON_COUNTER_INC(&tmp_file, infile->counters[j],
- infile->counters[j+4], POSIX_STRIDE1_STRIDE, POSIX_STRIDE1_COUNT);
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_STRIDE1_STRIDE]),
+ &(tmp_file.counters[POSIX_STRIDE1_COUNT]), infile->counters[j],
+ infile->counters[j+4]);
}
/* second set */
for(j=POSIX_STRIDE1_STRIDE; j<=POSIX_STRIDE4_STRIDE; j++)
{
- POSIX_COMMON_COUNTER_INC(&tmp_file, inoutfile->counters[j],
- inoutfile->counters[j+4], POSIX_STRIDE1_STRIDE, POSIX_STRIDE1_COUNT);
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_STRIDE1_STRIDE]),
+ &(tmp_file.counters[POSIX_STRIDE1_COUNT]), inoutfile->counters[j],
+ inoutfile->counters[j+4]);
}
/* same for access counts */
@@ -1840,20 +1816,23 @@ static void posix_record_reduction_op(
/* first set */
for(j=POSIX_ACCESS1_ACCESS; j<=POSIX_ACCESS4_ACCESS; j++)
{
- POSIX_COMMON_COUNTER_INC(&tmp_file, infile->counters[j],
- infile->counters[j+4], POSIX_ACCESS1_ACCESS, POSIX_ACCESS1_COUNT);
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_ACCESS1_ACCESS]),
+ &(tmp_file.counters[POSIX_ACCESS1_COUNT]), infile->counters[j],
+ infile->counters[j+4]);
}
/* second set */
for(j=POSIX_ACCESS1_ACCESS; j<=POSIX_ACCESS4_ACCESS; j++)
{
- POSIX_COMMON_COUNTER_INC(&tmp_file, inoutfile->counters[j],
- inoutfile->counters[j+4], POSIX_ACCESS1_ACCESS, POSIX_ACCESS1_COUNT);
+ DARSHAN_COMMON_VAL_COUNTER_INC(&(tmp_file.counters[POSIX_ACCESS1_ACCESS]),
+ &(tmp_file.counters[POSIX_ACCESS1_COUNT]), inoutfile->counters[j],
+ inoutfile->counters[j+4]);
}
/* min non-zero (if available) value */
for(j=POSIX_F_OPEN_TIMESTAMP; j<=POSIX_F_WRITE_START_TIMESTAMP; j++)
{
- if(infile->fcounters[j] > inoutfile->fcounters[j] && inoutfile->fcounters[j] > 0)
+ if(infile->fcounters[j] > inoutfile->fcounters[j] &&
+ inoutfile->fcounters[j] > 0)
tmp_file.fcounters[j] = inoutfile->fcounters[j];
else
tmp_file.fcounters[j] = infile->fcounters[j];
@@ -1958,28 +1937,233 @@ static void posix_record_reduction_op(
return;
}
+static void posix_shared_record_variance(MPI_Comm mod_comm,
+ struct darshan_posix_file *inrec_array, struct darshan_posix_file *outrec_array,
+ int shared_rec_count)
+{
+ MPI_Datatype var_dt;
+ MPI_Op var_op;
+ int i;
+ struct darshan_variance_dt *var_send_buf = NULL;
+ struct darshan_variance_dt *var_recv_buf = NULL;
+
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_variance_dt),
+ MPI_BYTE, &var_dt);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&var_dt);
+
+ DARSHAN_MPI_CALL(PMPI_Op_create)(darshan_variance_reduce, 1, &var_op);
+
+ var_send_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));
+ if(!var_send_buf)
+ return;
+
+ if(my_rank == 0)
+ {
+ var_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_variance_dt));
+
+ if(!var_recv_buf)
+ return;
+ }
+
+ /* get total i/o time variances for shared records */
+
+ for(i=0; i<shared_rec_count; i++)
+ {
+ var_send_buf[i].n = 1;
+ var_send_buf[i].S = 0;
+ var_send_buf[i].T = inrec_array[i].fcounters[POSIX_F_READ_TIME] +
+ inrec_array[i].fcounters[POSIX_F_WRITE_TIME] +
+ inrec_array[i].fcounters[POSIX_F_META_TIME];
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,
+ var_dt, var_op, 0, mod_comm);
+
+ if(my_rank == 0)
+ {
+ for(i=0; i<shared_rec_count; i++)
+ {
+ outrec_array[i].fcounters[POSIX_F_VARIANCE_RANK_TIME] =
+ (var_recv_buf[i].S / var_recv_buf[i].n);
+ }
+ }
+
+ /* get total bytes moved variances for shared records */
+
+ for(i=0; i<shared_rec_count; i++)
+ {
+ var_send_buf[i].n = 1;
+ var_send_buf[i].S = 0;
+ var_send_buf[i].T = (double)
+ inrec_array[i].counters[POSIX_BYTES_READ] +
+ inrec_array[i].counters[POSIX_BYTES_WRITTEN];
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Reduce)(var_send_buf, var_recv_buf, shared_rec_count,
+ var_dt, var_op, 0, mod_comm);
+
+ if(my_rank == 0)
+ {
+ for(i=0; i<shared_rec_count; i++)
+ {
+ outrec_array[i].fcounters[POSIX_F_VARIANCE_RANK_BYTES] =
+ (var_recv_buf[i].S / var_recv_buf[i].n);
+ }
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&var_dt);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&var_op);
+ free(var_send_buf);
+ free(var_recv_buf);
+
+ return;
+}
+
+/************************************************************************
+ * Functions exported by this module for coordinating with darshan-core *
+ ************************************************************************/
+
+static void posix_begin_shutdown()
+{
+ assert(posix_runtime);
+
+ POSIX_LOCK();
+ /* disable further instrumentation while Darshan shuts down */
+ instrumentation_disabled = 1;
+ POSIX_UNLOCK();
+
+ return;
+}
+
static void posix_get_output_data(
- void **buffer,
- int *size)
+ MPI_Comm mod_comm,
+ darshan_record_id *shared_recs,
+ int shared_rec_count,
+ void **posix_buf,
+ int *posix_buf_sz)
{
+ struct posix_file_runtime *file;
+ struct posix_file_runtime *tmp;
+ int i;
+ double posix_time;
+ struct darshan_posix_file *red_send_buf = NULL;
+ struct darshan_posix_file *red_recv_buf = NULL;
+ MPI_Datatype red_type;
+ MPI_Op red_op;
+
assert(posix_runtime);
- /* TODO: cleaner way to do this? */
- /* clean up reduction state */
- if((my_rank == 0) && (posix_runtime->red_buf))
+ /* go through file access data for each record and set the 4 most common
+ * stride/access size counters.
+ */
+ for(i = 0; i < posix_runtime->file_array_ndx; i++)
{
- int tmp_ndx = posix_runtime->file_array_ndx - posix_runtime->shared_rec_count;
- memcpy(&(posix_runtime->file_record_array[tmp_ndx]), posix_runtime->red_buf,
- posix_runtime->shared_rec_count * sizeof(struct darshan_posix_file));
- free(posix_runtime->red_buf);
+ tmp = &(posix_runtime->file_runtime_array[i]);
+
+ /* common accesses */
+ darshan_walk_common_vals(tmp->access_root,
+ &(tmp->file_record->counters[POSIX_ACCESS1_ACCESS]),
+ &(tmp->file_record->counters[POSIX_ACCESS1_COUNT]));
+ /* common strides */
+ darshan_walk_common_vals(tmp->stride_root,
+ &(tmp->file_record->counters[POSIX_STRIDE1_STRIDE]),
+ &(tmp->file_record->counters[POSIX_STRIDE1_COUNT]));
}
- else
+
+ /* if there are globally shared files, do a shared file reduction */
+ if(shared_rec_count)
{
- posix_runtime->file_array_ndx -= posix_runtime->shared_rec_count;
+ /* necessary initialization of shared records */
+ for(i = 0; i < shared_rec_count; i++)
+ {
+ HASH_FIND(hlink, posix_runtime->file_hash, &shared_recs[i],
+ sizeof(darshan_record_id), file);
+ assert(file);
+
+ posix_time =
+ file->file_record->fcounters[POSIX_F_READ_TIME] +
+ file->file_record->fcounters[POSIX_F_WRITE_TIME] +
+ file->file_record->fcounters[POSIX_F_META_TIME];
+
+ /* initialize fastest/slowest info prior to the reduction */
+ file->file_record->counters[POSIX_FASTEST_RANK] =
+ file->file_record->rank;
+ file->file_record->counters[POSIX_FASTEST_RANK_BYTES] =
+ file->file_record->counters[POSIX_BYTES_READ] +
+ file->file_record->counters[POSIX_BYTES_WRITTEN];
+ file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME] =
+ posix_time;
+
+ /* until reduction occurs, we assume that this rank is both
+ * the fastest and slowest. It is up to the reduction operator
+ * to find the true min and max.
+ */
+ file->file_record->counters[POSIX_SLOWEST_RANK] =
+ file->file_record->counters[POSIX_FASTEST_RANK];
+ file->file_record->counters[POSIX_SLOWEST_RANK_BYTES] =
+ file->file_record->counters[POSIX_FASTEST_RANK_BYTES];
+ file->file_record->fcounters[POSIX_F_SLOWEST_RANK_TIME] =
+ file->file_record->fcounters[POSIX_F_FASTEST_RANK_TIME];
+
+ file->file_record->rank = -1;
+ }
+
+ /* sort the array of files descending by rank so that we get all of the
+ * shared files (marked by rank -1) in a contiguous portion at end
+ * of the array
+ */
+ qsort(posix_runtime->file_record_array, posix_runtime->file_array_ndx,
+ sizeof(struct darshan_posix_file), posix_record_compare);
+
+ /* make *send_buf point to the shared files at the end of sorted array */
+ red_send_buf =
+ &(posix_runtime->file_record_array[posix_runtime->file_array_ndx-shared_rec_count]);
+
+ /* allocate memory for the reduction output on rank 0 */
+ if(my_rank == 0)
+ {
+ red_recv_buf = malloc(shared_rec_count * sizeof(struct darshan_posix_file));
+ if(!red_recv_buf)
+ return;
+ }
+
+ /* construct a datatype for a POSIX file record. This is serving no purpose
+ * except to make sure we can do a reduction on proper boundaries
+ */
+ DARSHAN_MPI_CALL(PMPI_Type_contiguous)(sizeof(struct darshan_posix_file),
+ MPI_BYTE, &red_type);
+ DARSHAN_MPI_CALL(PMPI_Type_commit)(&red_type);
+
+ /* register a POSIX file record reduction operator */
+ DARSHAN_MPI_CALL(PMPI_Op_create)(posix_record_reduction_op, 1, &red_op);
+
+ /* reduce shared POSIX file records */
+ DARSHAN_MPI_CALL(PMPI_Reduce)(red_send_buf, red_recv_buf,
+ shared_rec_count, red_type, red_op, 0, mod_comm);
+
+ /* get the time and byte variances for shared files */
+ posix_shared_record_variance(mod_comm, red_send_buf, red_recv_buf,
+ shared_rec_count);
+
+ /* clean up reduction state */
+ if(my_rank == 0)
+ {
+ int tmp_ndx = posix_runtime->file_array_ndx - shared_rec_count;
+ memcpy(&(posix_runtime->file_record_array[tmp_ndx]), red_recv_buf,
+ shared_rec_count * sizeof(struct darshan_posix_file));
+ free(red_recv_buf);
+ }
+ else
+ {
+ posix_runtime->file_array_ndx -= shared_rec_count;
+ }
+
+ DARSHAN_MPI_CALL(PMPI_Type_free)(&red_type);
+ DARSHAN_MPI_CALL(PMPI_Op_free)(&red_op);
}
- *buffer = (void *)(posix_runtime->file_record_array);
- *size = posix_runtime->file_array_ndx * sizeof(struct darshan_posix_file);
+ *posix_buf = (void *)(posix_runtime->file_record_array);
+ *posix_buf_sz = posix_runtime->file_array_ndx * sizeof(struct darshan_posix_file);
return;
}
diff --git a/darshan-test/regression/run-all.sh b/darshan-test/regression/run-all.sh
index 97607cc..cb03fcc 100755
--- a/darshan-test/regression/run-all.sh
+++ b/darshan-test/regression/run-all.sh
@@ -17,7 +17,7 @@ DARSHAN_TESTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
export DARSHAN_TESTDIR
# check darshan path
-if [ ! -x $DARSHAN_PATH/bin/darshan-posix-parser ]; then
+if [ ! -x $DARSHAN_PATH/bin/darshan-parser ]; then
echo "Error: $DARSHAN_PATH doesn't contain a valid Darshan install." 1>&2
exit 1
fi
diff --git a/darshan-test/regression/test-cases/cxxpi.sh b/darshan-test/regression/test-cases/cxxpi.sh
index eda00eb..e7d7f47 100755
--- a/darshan-test/regression/test-cases/cxxpi.sh
+++ b/darshan-test/regression/test-cases/cxxpi.sh
@@ -21,7 +21,7 @@ if [ $? -ne 0 ]; then
fi
# parse log
-$DARSHAN_PATH/bin/darshan-posix-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
+$DARSHAN_PATH/bin/darshan-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
if [ $? -ne 0 ]; then
echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
exit 1
@@ -29,12 +29,7 @@ fi
# check results
# in this case we want to confirm that the open counts are zero; cxxpi does not do any IO
-#POSIX_OPENS=`grep CP_POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 4`
-#if [ "$POSIX_OPENS"x != ""x ]; then
-# echo "Error: Found unexpected POSIX open count of $POSIX_OPENS" 1>&2
-# exit 1
-#fi
-POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -d : -f 2 |xargs`
+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
if [ "$POSIX_OPENS"x != ""x ]; then
echo "Error: Found unexpected POSIX open count of $POSIX_OPENS" 1>&2
exit 1
diff --git a/darshan-test/regression/test-cases/fperf-f77.sh b/darshan-test/regression/test-cases/fperf-f77.sh
index 7159572..6a73390 100755
--- a/darshan-test/regression/test-cases/fperf-f77.sh
+++ b/darshan-test/regression/test-cases/fperf-f77.sh
@@ -21,31 +21,23 @@ if [ $? -ne 0 ]; then
fi
# parse log
-$DARSHAN_PATH/bin/darshan-posix-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
-if [ $? -ne 0 ]; then
- echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
- exit 1
-fi
-$DARSHAN_PATH/bin/darshan-mpiio-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}-mpiio.darshan.txt
+$DARSHAN_PATH/bin/darshan-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
if [ $? -ne 0 ]; then
echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
exit 1
fi
-
# check results
# in this case we want to confirm that both the MPI and POSIX open counters were triggered
-POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $POSIX_OPENS -gt 0 ]; then
+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$POSIX_OPENS" -gt 0 ]; then
echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2
exit 1
fi
-MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}-mpiio.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $MPI_OPENS -gt 0 ]; then
+MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$MPI_OPENS" -gt 0 ]; then
echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2
exit 1
fi
-
-
exit 0
diff --git a/darshan-test/regression/test-cases/fperf-f90.sh b/darshan-test/regression/test-cases/fperf-f90.sh
index 0bdf420..fc166b6 100755
--- a/darshan-test/regression/test-cases/fperf-f90.sh
+++ b/darshan-test/regression/test-cases/fperf-f90.sh
@@ -21,31 +21,23 @@ if [ $? -ne 0 ]; then
fi
# parse log
-$DARSHAN_PATH/bin/darshan-posix-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
-if [ $? -ne 0 ]; then
- echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
- exit 1
-fi
-$DARSHAN_PATH/bin/darshan-mpiio-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}-mpiio.darshan.txt
+$DARSHAN_PATH/bin/darshan-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
if [ $? -ne 0 ]; then
echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
exit 1
fi
-
# check results
# in this case we want to confirm that both the MPI and POSIX open counters were triggered
-POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $POSIX_OPENS -gt 0 ]; then
+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$POSIX_OPENS" -gt 0 ]; then
echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2
exit 1
fi
-MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}-mpiio.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $MPI_OPENS -gt 0 ]; then
+MPI_OPENS=`grep COLL_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$MPI_OPENS" -gt 0 ]; then
echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2
exit 1
fi
-
-
exit 0
diff --git a/darshan-test/regression/test-cases/mpi-io-test.sh b/darshan-test/regression/test-cases/mpi-io-test.sh
index 63d04b0..52dd387 100755
--- a/darshan-test/regression/test-cases/mpi-io-test.sh
+++ b/darshan-test/regression/test-cases/mpi-io-test.sh
@@ -21,12 +21,7 @@ if [ $? -ne 0 ]; then
fi
# parse log
-$DARSHAN_PATH/bin/darshan-posix-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
-if [ $? -ne 0 ]; then
- echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
- exit 1
-fi
-$DARSHAN_PATH/bin/darshan-mpiio-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}-mpiio.darshan.txt
+$DARSHAN_PATH/bin/darshan-parser $DARSHAN_LOGFILE > $DARSHAN_TMP/${PROG}.darshan.txt
if [ $? -ne 0 ]; then
echo "Error: failed to parse ${DARSHAN_LOGFILE}" 1>&2
exit 1
@@ -34,16 +29,15 @@ fi
# check results
# in this case we want to confirm that both the MPI and POSIX open counters were triggered
-POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $POSIX_OPENS -gt 0 ]; then
+POSIX_OPENS=`grep POSIX_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$POSIX_OPENS" -gt 0 ]; then
echo "Error: POSIX open count of $POSIX_OPENS is incorrect" 1>&2
exit 1
fi
-MPI_OPENS=`grep INDEP_OPENS $DARSHAN_TMP/${PROG}-mpiio.darshan.txt |cut -d : -f 2 |xargs`
-if [ ! $MPI_OPENS -gt 0 ]; then
+MPI_OPENS=`grep INDEP_OPENS $DARSHAN_TMP/${PROG}.darshan.txt |cut -f 5`
+if [ ! "$MPI_OPENS" -gt 0 ]; then
echo "Error: MPI open count of $MPI_OPENS is incorrect" 1>&2
exit 1
fi
-
exit 0
diff --git a/darshan-util/Makefile.in b/darshan-util/Makefile.in
index 20544f1..f1add4b 100644
--- a/darshan-util/Makefile.in
+++ b/darshan-util/Makefile.in
@@ -1,4 +1,4 @@
-all: libdarshan-util.a darshan-base-parser darshan-posix-parser darshan-mpiio-parser darshan-bgq-parser
+all: libdarshan-util.a darshan-analyzer darshan-convert darshan-parser jenkins-hash-gen
DESTDIR =
srcdir = @srcdir@
@@ -13,6 +13,9 @@ libdir = $(DESTDIR)@libdir@
pkgconfigdir = $(DESTDIR)$(libdir)/pkgconfig
DARSHAN_LOG_FORMAT = $(srcdir)/../darshan-log-format.h
+DARSHAN_MOD_LOG_FORMATS = $(srcdir)/../darshan-posix-log-format.h $(srcdir)/../darshan-mpiio-log-format.h $(srcdir)/../darshan-hdf5-log-format.h $(srcdir)/../darshan-pnetcdf-log-format.h
+DARSHAN_MOD_LOGUTIL_HEADERS = darshan-posix-logutils.h darshan-mpiio-logutils.h darshan-hdf5-logutils.h darshan-pnetcdf-logutils.h
+
DARSHAN_ENABLE_SHARED=@DARSHAN_ENABLE_SHARED@
VPATH = $(srcdir)
@@ -56,73 +59,70 @@ darshan-mpiio-logutils.o: darshan-mpiio-logutils.c darshan-logutils.h darshan-mp
darshan-mpiio-logutils.po: darshan-mpiio-logutils.c darshan-logutils.h darshan-mpiio-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h | uthash-1.9.2
$(CC) $(CFLAGS_SHARED) -c $< -o $@
-libdarshan-util.so: darshan-logutils.po darshan-posix-logutils.po darshan-mpiio-logutils.po
- $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ $(LIBS)
-
-libdarshan-util.a: darshan-logutils.o darshan-posix-logutils.o darshan-mpiio-logutils.o
- ar rcs libdarshan-util.a $^
-
-jenkins: util/bin/jenkins.o lookup3.o
- $(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ lookup3.o $(LIBS)
+darshan-hdf5-logutils.o: darshan-hdf5-logutils.c darshan-logutils.h darshan-hdf5-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS) -c $< -o $@
+darshan-hdf5-logutils.po: darshan-hdf5-logutils.c darshan-logutils.h darshan-hdf5-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-hdf5-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
-darshan-base-parser: darshan-base-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
+darshan-pnetcdf-logutils.o: darshan-pnetcdf-logutils.c darshan-logutils.h darshan-pnetcdf-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS) -c $< -o $@
+darshan-pnetcdf-logutils.po: darshan-pnetcdf-logutils.c darshan-logutils.h darshan-pnetcdf-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-pnetcdf-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
-darshan-posix-parser: darshan-posix-parser.c darshan-logutils.h darshan-posix-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-posix-log-format.h libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
+darshan-bgq-logutils.o: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS) -c $< -o $@
+darshan-bgq-logutils.po: darshan-bgq-logutils.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h | uthash-1.9.2
+ $(CC) $(CFLAGS_SHARED) -c $< -o $@
-darshan-mpiio-parser: darshan-mpiio-parser.c darshan-logutils.h darshan-mpiio-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-mpiio-log-format.h libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
-darshan-bgq-parser: darshan-bgq-parser.c darshan-logutils.h darshan-bgq-logutils.h $(DARSHAN_LOG_FORMAT) $(srcdir)/../darshan-bgq-log-format.h libdarshan-util.a | uthash-1.9.2
- $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
+libdarshan-util.so: darshan-logutils.po darshan-posix-logutils.po darshan-mpiio-logutils.po darshan-hdf5-logutils.po darshan-pnetcdf-logutils.po darshan-bgq-logutils.po
+ $(CC) $(CFLAGS_SHARED) $(LDFLAGS) -o $@ $^ $(LIBS)
+
+libdarshan-util.a: darshan-logutils.o darshan-posix-logutils.o darshan-mpiio-logutils.o darshan-hdf5-logutils.o darshan-pnetcdf-logutils.o darshan-bgq-logutils.o
+ ar rcs libdarshan-util.a $^
-#darshan-parser: darshan-parser.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o | uthash-1.9.2
-# $(CC) $(CFLAGS) $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS)
+jenkins-hash-gen: jenkins-hash-gen.c lookup3.o
+ $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
-#darshan-convert: darshan-convert.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o lookup3.o
-# $(CC) $(CFLAGS) $(LDFLAGS) $< darshan-logutils.o lookup3.o -o $@ $(LIBS)
+lookup3.o: lookup3.c
+ $(CC) $(CFLAGS) -c $< -o $@
-#darshan-analyzer: darshan-analyzer.c $(DARSHAN_LOG_FORMAT) darshan-logutils.h darshan-logutils.o
-# $(CC) $(CFLAGS) $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS)
+darshan-analyzer: darshan-analyzer.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
-#darshan-log-params: darshan-log-params.c $(DARSHAN_LOG_FORMAT)
-# $(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ $(LIBS)
+darshan-convert: darshan-convert.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) libdarshan-util.a lookup3.o | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
#darshan-diff: darshan-diff.o $(DARSHAN_LOG_FORMAT) darshan-logutils.o darshan-logutils.h
# $(CC) $(CFLAGS) $(LDFLAGS) $< darshan-logutils.o -o $@ $(LIBS)
#darshan-diff.o: darshan-diff.c
# $(CC) $(CFLAGS) -c $< -o $@
+darshan-parser: darshan-parser.c darshan-logutils.h $(DARSHAN_LOG_FORMAT) $(DARSHAN_MOD_LOGUTIL_HEADERS) $(DARSHAN_MOD_LOG_FORMATS) libdarshan-util.a | uthash-1.9.2
+ $(CC) $(CFLAGS) $(LDFLAGS) $^ -o $@ $(LIBS)
+
#test/gztest: test/gztest.c mktestdir
# $(CC) $(CFLAGS) $(LDFLAGS) -lz $< -o $@
#test/gz-bench: test/gz-bench.c mktestdir
# $(CC) $(CFLAGS) $(LDFLAGS) -lz $< -o $@
-lookup3.o: lookup3.c
- $(CC) $(CFLAGS) -c $< -o $@
-
install:: all
install -d $(bindir)
install -d $(libdir)
install -d $(includedir)
install -d $(pkgconfigdir)
-# install -m 755 darshan-parser $(bindir)
- install -m 755 darshan-base-parser $(bindir)
- install -m 755 darshan-posix-parser $(bindir)
- install -m 755 darshan-mpiio-parser $(bindir)
- install -m 755 darshan-bgq-parser $(bindir)
-# install -m 755 darshan-convert $(bindir)
+ install -m 755 darshan-analyzer $(bindir)
+ install -m 755 darshan-convert $(bindir)
# install -m 755 darshan-diff $(bindir)
-# install -m 755 darshan-analyzer $(bindir)
+ install -m 755 darshan-parser $(bindir)
# install -m 755 $(srcdir)/darshan-summary-per-file.sh $(bindir)
install -m 755 libdarshan-util.a $(libdir)
ifeq ($(DARSHAN_ENABLE_SHARED),1)
install -m 755 libdarshan-util.so $(libdir)
endif
-# install -m 644 $(srcdir)/darshan-logutils.h $(includedir)
-# install -m 644 $(DARSHAN_LOG_FORMAT) $(includedir)
+ install -m 644 $(srcdir)/darshan-logutils.h $(includedir)
+ install -m 644 $(DARSHAN_LOG_FORMAT) $(includedir)
# install -m 755 darshan-job-summary/bin/darshan-job-summary.pl $(bindir)
# install -d $(libdir)/TeX
# install -m 644 $(srcdir)/darshan-job-summary/lib/TeX/Encode.pm $(libdir)/TeX/
@@ -135,7 +135,7 @@ endif
clean::
- rm -f *.o *.a darshan-base-parser darshan-posix-parser darshan-mpiio-parser
+ rm -f *.o *.a darshan-analyzer darshan-convert darshan-parser jenkins-hash-gen
distclean:: clean
rm -f darshan-runtime-config.h aclocal.m4 autom4te.cache/* config.status config.log Makefile util/bin/darshan-job-summary.pl
diff --git a/darshan-util/darshan-analyzer.c b/darshan-util/darshan-analyzer.c
index f16d98f..a9be4b2 100644
--- a/darshan-util/darshan-analyzer.c
+++ b/darshan-util/darshan-analyzer.c
@@ -1,6 +1,13 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
+#include <assert.h>
#include <ftw.h>
#include <zlib.h>
@@ -11,10 +18,8 @@
#define BUCKET3 0.60
#define BUCKET4 0.80
-char * base = NULL;
-
-int total_single = 0;
-int total_multi = 0;
+int total_shared = 0;
+int total_fpp = 0;
int total_mpio = 0;
int total_pnet = 0;
int total_hdf5 = 0;
@@ -25,67 +30,92 @@ int bucket2 = 0;
int bucket3 = 0;
int bucket4 = 0;
int bucket5 = 0;
-int fail = 0;
-int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_pnet, int *used_hdf5, int *used_multi, int *used_single)
+int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_pnet, int *used_hdf5, int *used_shared, int *used_fpp)
{
- struct darshan_job job;
- struct darshan_file cp_file;
- char tmp_string[4096];
- darshan_fd zfile;
int ret;
+ darshan_fd file;
+ struct darshan_header header;
+ struct darshan_job job;
+ struct darshan_mod_logutil_funcs *psx_mod = mod_logutils[DARSHAN_POSIX_MOD];
+ struct darshan_posix_file *psx_buf, *psx_buf_p;
+ int psx_buf_sz, psx_buf_bytes_left;
+ struct darshan_posix_file *psx_rec;
+ darshan_record_id rec_id;
int f_count;
double total_io_time;
double total_job_time;
- zfile = darshan_log_open(fname, "r");
- if (zfile == NULL)
+ assert(psx_mod);
+
+ file = darshan_log_open(fname);
+ if (file == NULL)
{
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", fname);
+ fprintf(stderr, "darshan_log_open() failed to open %s.\n", fname);
return -1;
}
- ret = darshan_log_getjob(zfile, &job);
+ ret = darshan_log_getheader(file, &header);
if (ret < 0)
{
- perror("darshan_log_getjob");
- fprintf(stderr, "%s\n", fname);
- darshan_log_close(zfile);
+ fprintf(stderr, "darshan_log_getheader() failed on file %s.\n", fname);
+ darshan_log_close(file);
return -1;
}
- ret = darshan_log_getexe(zfile, tmp_string);
+ ret = darshan_log_getjob(file, &job);
if (ret < 0)
{
- perror("darshan_log_getexe");
- fprintf(stderr, "%s\n", fname);
- darshan_log_close(zfile);
+ fprintf(stderr, "darshan_log_getjob() failed on file %s.\n", fname);
+ darshan_log_close(file);
+ return -1;
+ }
+
+ psx_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+ psx_buf = malloc(psx_buf_sz);
+ if (!psx_buf)
+ {
+ darshan_log_close(file);
+ return -1;
+ }
+
+ ret = darshan_log_getmod(file, DARSHAN_POSIX_MOD, (void *)psx_buf, &psx_buf_sz);
+ if (ret < 0)
+ {
+ fprintf(stderr, "darshan_log_getmod() failed on file %s.\n", fname);
+ darshan_log_close(file);
return -1;
}
f_count = 0;
total_io_time = 0.0;
- while ((ret = darshan_log_getfile(zfile, &job, &cp_file)) == 1)
+ psx_buf_bytes_left = psx_buf_sz;
+ psx_buf_p = psx_buf;
+ while(psx_buf_bytes_left)
{
+ ret = psx_mod->log_get_record((void **)&psx_buf_p, &psx_buf_bytes_left,
+ (void **)&psx_rec, &rec_id, file->swap_flag);
+
f_count += 1;
- if (cp_file.rank == -1)
- *used_single = 1;
+ if (psx_rec->rank == -1)
+ *used_shared = 1;
else
- *used_multi = 1;
-
- *used_mpio += cp_file.counters[CP_INDEP_OPENS];
- *used_mpio += cp_file.counters[CP_COLL_OPENS];
- *used_pnet += cp_file.counters[CP_INDEP_NC_OPENS];
- *used_pnet += cp_file.counters[CP_COLL_NC_OPENS];
- *used_hdf5 += cp_file.counters[CP_HDF5_OPENS];
+ *used_fpp = 1;
- total_io_time += cp_file.fcounters[CP_F_POSIX_READ_TIME];
- total_io_time += cp_file.fcounters[CP_F_POSIX_WRITE_TIME];
- total_io_time += cp_file.fcounters[CP_F_POSIX_META_TIME];
+ total_io_time += (psx_rec->fcounters[POSIX_F_READ_TIME] +
+ psx_rec->fcounters[POSIX_F_WRITE_TIME] +
+ psx_rec->fcounters[POSIX_F_META_TIME]);
}
+ if (header.mod_map[DARSHAN_MPIIO_MOD].len > 0)
+ *used_mpio += 1;
+ if (header.mod_map[DARSHAN_HDF5_MOD].len > 0)
+ *used_hdf5 += 1;
+ if (header.mod_map[DARSHAN_PNETCDF_MOD].len > 0)
+ *used_pnet += 1;
+
total_job_time = (double)job.end_time - (double)job.start_time;
if (total_job_time < 1.0)
{
@@ -101,7 +131,8 @@ int process_log(const char *fname, double *io_ratio, int *used_mpio, int *used_p
*io_ratio = 0.0;
}
- darshan_log_close(zfile);
+ free(psx_buf);
+ darshan_log_close(file);
return 0;
}
@@ -112,20 +143,20 @@ int tree_walk (const char *fpath, const struct stat *sb, int typeflag)
int used_mpio = 0;
int used_pnet = 0;
int used_hdf5 = 0;
- int used_multi = 0;
- int used_single = 0;
+ int used_shared = 0;
+ int used_fpp = 0;
if (typeflag != FTW_F) return 0;
- process_log(fpath,&io_ratio,&used_mpio,&used_pnet,&used_hdf5,&used_multi,&used_single);
+ process_log(fpath,&io_ratio,&used_mpio,&used_pnet,&used_hdf5,&used_shared,&used_fpp);
total_count++;
if (used_mpio > 0) total_mpio++;
if (used_pnet > 0) total_pnet++;
if (used_hdf5 > 0) total_hdf5++;
- if (used_single > 0) total_single++;
- if (used_multi > 0) total_multi++;
+ if (used_shared > 0) total_shared++;
+ if (used_fpp > 0) total_fpp++;
if (io_ratio <= BUCKET1)
bucket1++;
@@ -137,22 +168,18 @@ int tree_walk (const char *fpath, const struct stat *sb, int typeflag)
bucket4++;
else if (io_ratio > BUCKET4)
bucket5++;
- else
- {
- printf("iorat: %lf\n", io_ratio);
- fail++;
- }
return 0;
}
int main(int argc, char **argv)
{
+ char * base = NULL;
int ret = 0;
if(argc != 2)
{
- fprintf(stderr, "Error: bad arguments.\n");
+ fprintf(stderr, "Error: directory of Darshan logs required as argument.\n");
return(-1);
}
@@ -165,13 +192,13 @@ int main(int argc, char **argv)
return(-1);
}
- printf (" log: %s\n", base);
- printf (" total: %d\n", total_count);
- printf ("single: %lf [%d]\n", (double)total_single/(double)total_count, total_single);
- printf (" multi: %lf [%d]\n", (double)total_multi/(double)total_count, total_multi);
- printf (" mpio: %lf [%d]\n", (double)total_mpio/(double)total_count, total_mpio);
- printf (" pnet: %lf [%d]\n", (double)total_pnet/(double)total_count, total_pnet);
- printf (" hdf5: %lf [%d]\n", (double)total_hdf5/(double)total_count, total_hdf5);
+ printf ("log dir: %s\n", base);
+ printf (" total: %d\n", total_count);
+ printf (" shared: %lf [%d]\n", (double)total_shared/(double)total_count, total_shared);
+ printf (" fpp: %lf [%d]\n", (double)total_fpp/(double)total_count, total_fpp);
+ printf (" mpio: %lf [%d]\n", (double)total_mpio/(double)total_count, total_mpio);
+ printf (" pnet: %lf [%d]\n", (double)total_pnet/(double)total_count, total_pnet);
+ printf (" hdf5: %lf [%d]\n", (double)total_hdf5/(double)total_count, total_hdf5);
printf ("%.2lf-%.2lf: %d\n", (double)0.0, (double)BUCKET1, bucket1);
printf ("%.2lf-%.2lf: %d\n", (double)BUCKET1, (double)BUCKET2, bucket2);
printf ("%.2lf-%.2lf: %d\n", (double)BUCKET2, (double)BUCKET3, bucket3);
@@ -179,3 +206,12 @@ int main(int argc, char **argv)
printf ("%.2lf-%.2lf: %d\n", (double)BUCKET4, (double)100.0, bucket5);
return 0;
}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-util/darshan-base-parser.c b/darshan-util/darshan-base-parser.c
deleted file mode 100644
index 0f86f49..0000000
--- a/darshan-util/darshan-base-parser.c
+++ /dev/null
@@ -1,180 +0,0 @@
-/*
- * Copyright (C) 2015 University of Chicago.
- * See COPYRIGHT notice in top-level directory.
- *
- */
-
-#include <stdio.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <string.h>
-#include <zlib.h>
-#include <time.h>
-#include <stdlib.h>
-#include <getopt.h>
-#include <assert.h>
-
-#include "darshan-logutils.h"
-#include "darshan-posix-logutils.h"
-#include "uthash-1.9.2/src/uthash.h"
-
-int main(int argc, char **argv)
-{
- int ret;
- int i;
- char *filename;
- char tmp_string[4096];
- darshan_fd fd;
- struct darshan_header header;
- struct darshan_job job;
- struct darshan_record_ref *rec_hash = NULL;
- struct darshan_record_ref *ref, *tmp;
- int mount_count;
- char** mnt_pts;
- char** fs_types;
- time_t tmp_time = 0;
- char *token;
- char *save;
- char buffer[DARSHAN_JOB_METADATA_LEN];
-
- assert(argc == 2);
- filename = argv[1];
-
- struct stat sbuf;
- stat(filename, &sbuf);
-
- fd = darshan_log_open(filename, "r");
- if(!fd)
- {
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", filename);
- return(-1);
- }
-
- /* read darshan log header */
- ret = darshan_log_getheader(fd, &header);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getheader() failed to read log header.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* read darshan job info */
- ret = darshan_log_getjob(fd, &job);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getjob() failed to read job data.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* get the original command line for this job */
- ret = darshan_log_getexe(fd, tmp_string);
- if(ret < 0)
- {
- fprintf(stderr, "Error: unable to read trailing job information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print job summary */
- printf("# darshan log version: %s\n", header.version_string);
- printf("# size of POSIX file statistics: %zu bytes\n", sizeof(struct darshan_posix_file));
- printf("# size of job statistics: %zu bytes\n", sizeof(struct darshan_job));
- printf("# exe: %s\n", tmp_string);
- printf("# uid: %" PRId64 "\n", job.uid);
- printf("# jobid: %" PRId64 "\n", job.jobid);
- printf("# start_time: %" PRId64 "\n", job.start_time);
- tmp_time += job.start_time;
- printf("# start_time_asci: %s", ctime(&tmp_time));
- printf("# end_time: %" PRId64 "\n", job.end_time);
- tmp_time = 0;
- tmp_time += job.end_time;
- printf("# end_time_asci: %s", ctime(&tmp_time));
- printf("# nprocs: %" PRId64 "\n", job.nprocs);
- printf("# run time: %" PRId64 "\n", job.end_time - job.start_time + 1);
- for(token=strtok_r(job.metadata, "\n", &save);
- token != NULL;
- token=strtok_r(NULL, "\n", &save))
- {
- char *key;
- char *value;
- /* NOTE: we intentionally only split on the first = character.
- * There may be additional = characters in the value portion
- * (for example, when storing mpi-io hints).
- */
- strcpy(buffer, token);
- key = buffer;
- value = index(buffer, '=');
- if(!value)
- continue;
- /* convert = to a null terminator to split key and value */
- value[0] = '\0';
- value++;
- printf("# metadata: %s = %s\n", key, value);
- }
-
- /* print out each module found in the log */
- printf("\n# instrumenation modules registered with darshan\n");
- printf("# -------------------------------------------------------\n");
- for(i=0;i < DARSHAN_MAX_MODS; i++)
- {
- if(header.mod_map[i].len > 0)
- {
- printf("# %s:\tsize=%"PRIu64"\n", darshan_module_names[i], header.mod_map[i].len);
- }
-
- }
-
- /* get the mount information for this log */
- ret = darshan_log_getmounts(fd, &mnt_pts, &fs_types, &mount_count);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmounts() failed to read mount information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print table of mounted file systems */
- printf("\n# mounted file systems (mount point and fs type)\n");
- printf("# -------------------------------------------------------\n");
- for(i=0; i<mount_count; i++)
- {
- printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);
- }
-
- /* read hash of darshan records */
- ret = darshan_log_gethash(fd, &rec_hash);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmap() failed to read record map.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print info for each record stored in the log */
- printf("\n# records registered with Darshan\n");
- printf("# -------------------------------------------------------\n");
- HASH_ITER(hlink, rec_hash, ref, tmp)
- {
- printf("# ID: %"PRIu64"\tName: %s\n", ref->rec.id, ref->rec.name);
- }
-
- /* free mount info */
- for(i=0; i<mount_count; i++)
- {
- free(mnt_pts[i]);
- free(fs_types[i]);
- }
- if(mount_count > 0)
- {
- free(mnt_pts);
- free(fs_types);
- }
-
- darshan_log_close(fd);
-
- return(0);
-}
diff --git a/darshan-util/darshan-bgq-logutils.c b/darshan-util/darshan-bgq-logutils.c
index f96ec79..c538e20 100644
--- a/darshan-util/darshan-bgq-logutils.c
+++ b/darshan-util/darshan-bgq-logutils.c
@@ -19,33 +19,80 @@
#include "darshan-bgq-logutils.h"
-int darshan_log_get_bgq_file(darshan_fd fd, struct darshan_bgq_record *file)
+/* counter name strings for the POSIX module */
+#define X(a, b) #a,
+char *bgq_counter_names[] = {
+ BGQ_COUNTERS
+};
+
+char *bgq_f_counter_names[] = {
+ BGQ_F_COUNTERS
+};
+#undef X
+
+static int darshan_log_get_bgq_file(void** psx_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag);
+static void darshan_log_print_bgq_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+struct darshan_mod_logutil_funcs bgq_logutils =
+{
+ .log_get_record = &darshan_log_get_bgq_file,
+ .log_print_record = &darshan_log_print_bgq_file,
+};
+
+static int darshan_log_get_bgq_file(void** psx_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag)
{
int i;
- int ret;
+ struct darshan_bgq_record *file = (struct darshan_bgq_record *)
+ (*psx_buf_p);
+
+ if(*bytes_left < sizeof(struct darshan_bgq_record))
+ return(-1);
+
+ if(byte_swap_flag)
+ {
+ /* swap bytes if necessary */
+ DARSHAN_BSWAP64(&file->f_id);
+ DARSHAN_BSWAP64(&file->rank);
+ for(i=0; i<BGQ_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->counters[i]);
+ for(i=0; i<BGQ_F_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->fcounters[i]);
+ }
- /* reset file record, so that diff compares against a zero'd out record
- * if file is missing
- */
- memset(file, 0, sizeof(*file));
+ /* update/set output variables */
+ *file_rec = (void *)file;
+ *rec_id = file->f_id;
+ *psx_buf_p = (file + 1); /* increment input buf by size of file record */
+ *bytes_left -= sizeof(struct darshan_bgq_record);
+
+ return(0);
+}
+
+static void darshan_log_print_bgq_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ int i;
+ struct darshan_bgq_record *bgq_file_rec =
+ (struct darshan_bgq_record *)file_rec;
+
+ for(i=0; i<BGQ_NUM_INDICES; i++)
+ {
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ bgq_file_rec->rank, bgq_file_rec->f_id, bgq_counter_names[i],
+ bgq_file_rec->counters[i], file_name, mnt_pt, fs_type);
+ }
- ret = darshan_log_get_moddat(fd, DARSHAN_BGQ_MOD,
- (void *)file, sizeof(*file));
- if(ret == 1)
+ for(i=0; i<BGQ_F_NUM_INDICES; i++)
{
- if(fd->swap_flag)
- {
- /* swap bytes if necessary */
- DARSHAN_BSWAP64(&file->f_id);
- DARSHAN_BSWAP64(&file->rank);
- for(i=0; i<BGQ_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->counters[i]);
- for(i=0; i<BGQ_F_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->fcounters[i]);
- }
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_BGQ_MOD],
+ bgq_file_rec->rank, bgq_file_rec->f_id, bgq_f_counter_names[i],
+ bgq_file_rec->fcounters[i], file_name, mnt_pt, fs_type);
}
- return(ret);
+ return;
}
/*
diff --git a/darshan-util/darshan-bgq-logutils.h b/darshan-util/darshan-bgq-logutils.h
index 66ae974..a8bbb3a 100644
--- a/darshan-util/darshan-bgq-logutils.h
+++ b/darshan-util/darshan-bgq-logutils.h
@@ -10,6 +10,9 @@
#include "darshan-logutils.h"
#include "darshan-bgq-log-format.h"
-int darshan_log_get_bgq_file(darshan_fd fd, struct darshan_bgq_record *file);
+extern char *bgq_counter_names[];
+extern char *bgq_f_counter_names[];
+
+extern struct darshan_mod_logutil_funcs bgq_logutils;
#endif
diff --git a/darshan-util/darshan-convert-logs.pl b/darshan-util/darshan-convert-logs.pl
index 1132a0f..2987b7c 100755
--- a/darshan-util/darshan-convert-logs.pl
+++ b/darshan-util/darshan-convert-logs.pl
@@ -20,7 +20,7 @@
#
my $darshan_convert = "./darshan-convert";
-my $jenkins = "./jenkins";
+my $jenkins_hash_gen = "./jenkins-hash-gen";
sub load_annotations($$)
{
@@ -80,16 +80,17 @@ sub main()
$day = $3;
$logname = $4;
}
- $hashed_fname = `$jenkins --64 --key $hash_key $logname`;
+ $hashed_fname = `$jenkins_hash_gen --64 --key $hash_key $logname`;
chomp($hashed_fname);
@args = ("$darshan_convert",
+ "--bzip2",
"--obfuscate",
"--key=$hash_key",
"--annotate=$annotation",
"--reset-md",
"$logfile",
- "$output_path/$year/$month/$day/$hashed_fname.bz2");
+ "$output_path/$year/$month/$day/$hashed_fname.darshan");
$rc = system(@args);
if ($rc) {
print("$hashed_fname\t$logfile:failed:$rc\n");
diff --git a/darshan-util/darshan-convert.c b/darshan-util/darshan-convert.c
index fb39570..df5cd53 100644
--- a/darshan-util/darshan-convert.c
+++ b/darshan-util/darshan-convert.c
@@ -1,6 +1,7 @@
/*
- * (C) 2011 by Argonne National Laboratory.
- * See COPYRIGHT in top-level directory.
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
*/
#include <stdio.h>
@@ -25,6 +26,7 @@ int usage (char *exename)
fprintf(stderr, "Usage: %s [options] <infile> <outfile>\n", exename);
fprintf(stderr, " Converts darshan log from infile to outfile.\n");
fprintf(stderr, " rewrites the log file into the newest format.\n");
+ fprintf(stderr, " --bzip2 Use bzip2 compression instead of zlib.\n");
fprintf(stderr, " --obfuscate Obfuscate items in the log.\n");
fprintf(stderr, " --key <key> Key to use when obfuscating.\n");
fprintf(stderr, " --annotate <string> Additional metadata to add.\n");
@@ -35,15 +37,15 @@ int usage (char *exename)
}
void parse_args (int argc, char **argv, char **infile, char **outfile,
- int *obfuscate, int *reset_md, int *key, char **annotate, uint64_t* hash)
+ int *bzip2, int *obfuscate, int *reset_md, int *key,
+ char **annotate, uint64_t* hash)
{
int index;
int ret;
- *reset_md = 0;
-
static struct option long_opts[] =
{
+ {"bzip2", 0, NULL, 'b'},
{"annotate", 1, NULL, 'a'},
{"obfuscate", 0, NULL, 'o'},
{"reset-md", 0, NULL, 'r'},
@@ -53,6 +55,10 @@ void parse_args (int argc, char **argv, char **infile, char **outfile,
{ 0, 0, 0, 0 }
};
+ *bzip2 = 0;
+ *obfuscate = 0;
+ *reset_md = 0;
+ *key = 0;
*hash = 0;
while(1)
@@ -63,6 +69,9 @@ void parse_args (int argc, char **argv, char **infile, char **outfile,
switch(c)
{
+ case 'b':
+ *bzip2 = 1;
+ break;
case 'a':
*annotate = optarg;
break;
@@ -129,13 +138,21 @@ void obfuscate_exe(int key, char *exe)
return;
}
-void obfuscate_file(int key, struct darshan_file *file)
+void obfuscate_filenames(int key, struct darshan_record_ref *rec_hash)
{
+ struct darshan_record_ref *ref, *tmp;
uint32_t hashed;
+ char tmp_string[32];
- hashed = darshan_hashlittle(file->name_suffix, sizeof(file->name_suffix), key);
- memset(file->name_suffix, 0, sizeof(file->name_suffix));
- sprintf(file->name_suffix, "%u", hashed);
+ HASH_ITER(hlink, rec_hash, ref, tmp)
+ {
+ hashed = darshan_hashlittle(ref->rec.name, strlen(ref->rec.name), key);
+ sprintf(tmp_string, "%u", hashed);
+ free(ref->rec.name);
+ ref->rec.name = malloc(strlen(tmp_string));
+ assert(ref->rec.name);
+ memcpy(ref->rec.name, tmp_string, strlen(tmp_string));
+ }
return;
}
@@ -175,7 +192,24 @@ void add_annotation (char *annotation,
{
fprintf(stderr,
"not enough space left in metadata for: current=%s token=%s (remain=%d:need=%d)\n",
- job->metadata, token, remaining-1, strlen(token)+1);
+ job->metadata, token, remaining-1, (int)strlen(token)+1);
+ }
+ }
+
+ return;
+}
+
+static void remove_hash_recs(struct darshan_record_ref **rec_hash, darshan_record_id hash)
+{
+ struct darshan_record_ref *ref, *tmp;
+
+ HASH_ITER(hlink, *rec_hash, ref, tmp)
+ {
+ if(ref->rec.id != hash)
+ {
+ HASH_DELETE(hlink, *rec_hash, ref);
+ free(ref->rec.name);
+ free(ref);
}
}
@@ -187,53 +221,68 @@ int main(int argc, char **argv)
int ret;
char *infile_name;
char *outfile_name;
+ struct darshan_header header;
struct darshan_job job;
- struct darshan_file cp_file;
- char tmp_string[4096];
+ char tmp_string[4096] = {0};
darshan_fd infile;
darshan_fd outfile;
int i;
int mount_count;
- int64_t* devs;
char** mnt_pts;
char** fs_types;
- int last_rank = 0;
- int obfuscate = 0;
- int key = 0;
+ struct darshan_record_ref *rec_hash = NULL;
+ struct darshan_record_ref *ref, *tmp;
+ char *mod_buf;
+ int mod_buf_sz;
+ enum darshan_comp_type comp_type;
+ int bzip2;
+ int obfuscate;
+ int key;
char *annotation = NULL;
- uint64_t hash;
- int reset_md = 0;
+ darshan_record_id hash;
+ int reset_md;
- parse_args(argc, argv, &infile_name, &outfile_name, &obfuscate, &reset_md, &key, &annotation, &hash);
+ parse_args(argc, argv, &infile_name, &outfile_name, &bzip2, &obfuscate,
+ &reset_md, &key, &annotation, &hash);
- infile = darshan_log_open(infile_name, "r");
+ infile = darshan_log_open(infile_name);
if(!infile)
{
fprintf(stderr, "darshan_log_open() failed to open %s\n.", infile_name);
return(-1);
}
- /* TODO: safety check that outfile_name doesn't exist; we don't want to
- * overwrite something by accident.
- */
- outfile = darshan_log_open(outfile_name, "w");
+ comp_type = bzip2 ? comp_type = DARSHAN_BZIP2_COMP : DARSHAN_ZLIB_COMP;
+ outfile = darshan_log_create(outfile_name, comp_type);
if(!outfile)
{
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", outfile_name);
+ fprintf(stderr, "darshan_log_create() failed to create %s\n.", outfile_name);
+ darshan_log_close(infile);
return(-1);
}
- /* TODO: for now this tool is just reading the input file and throwing
- * away the data. Need to write the log_put*() functions and use this
- * program as a test harness
+ /* read header from input file */
+ ret = darshan_log_getheader(infile, &header);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to read header from input log file %s.\n", infile_name);
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
+ }
+
+ /* NOTE: we do not write the header to the output file until the end, as
+ * the mapping data stored in this structure may change in the conversion
+ * process (particularly, if we are converting between libz/bz2 compression)
*/
-
+
/* read job info */
ret = darshan_log_getjob(infile, &job);
if(ret < 0)
{
fprintf(stderr, "Error: unable to read job information from log file.\n");
darshan_log_close(infile);
+ darshan_log_close(outfile);
return(-1);
}
@@ -245,6 +294,7 @@ int main(int argc, char **argv)
if (ret < 0)
{
fprintf(stderr, "Error: unable to write job information to log file.\n");
+ darshan_log_close(infile);
darshan_log_close(outfile);
return(-1);
}
@@ -254,6 +304,7 @@ int main(int argc, char **argv)
{
fprintf(stderr, "Error: unable to read trailing job information.\n");
darshan_log_close(infile);
+ darshan_log_close(outfile);
return(-1);
}
@@ -263,68 +314,150 @@ int main(int argc, char **argv)
if(ret < 0)
{
fprintf(stderr, "Error: unable to write trailing job information.\n");
+ darshan_log_close(infile);
darshan_log_close(outfile);
return(-1);
}
-
- ret = darshan_log_getmounts(infile, &devs, &mnt_pts, &fs_types, &mount_count);
+
+ ret = darshan_log_getmounts(infile, &mnt_pts, &fs_types, &mount_count);
if(ret < 0)
{
fprintf(stderr, "Error: unable to read trailing job information.\n");
darshan_log_close(infile);
+ darshan_log_close(outfile);
return(-1);
}
- ret = darshan_log_putmounts(outfile, devs, mnt_pts, fs_types, mount_count);
+ ret = darshan_log_putmounts(outfile, mnt_pts, fs_types, mount_count);
if(ret < 0)
{
fprintf(stderr, "Error: unable to write mount information.\n");
+ darshan_log_close(infile);
darshan_log_close(outfile);
return(-1);
}
- ret = darshan_log_getfile(infile, &job, &cp_file);
+ ret = darshan_log_gethash(infile, &rec_hash);
if(ret < 0)
{
- fprintf(stderr, "Error: failed to process log file.\n");
- fflush(stderr);
+ fprintf(stderr, "Error: unable to read darshan record hash.\n");
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
}
- if(ret == 0)
+
+ /* NOTE: obfuscating filepaths breaks the ability to map files
+ * to the corresponding FS & mount info maintained by darshan
+ */
+ if(obfuscate) obfuscate_filenames(key, rec_hash);
+ if(hash) remove_hash_recs(&rec_hash, hash);
+
+ ret = darshan_log_puthash(outfile, rec_hash);
+ if(ret < 0)
{
- goto done;
+ fprintf(stderr, "Error: unable to write darshan record hash.\n");
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
}
- do
+ mod_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);
+ if(!mod_buf)
+ return(-1);
+
+ for(i=0; i<DARSHAN_MAX_MODS; i++)
{
- if(cp_file.rank != -1 && cp_file.rank < last_rank)
+ int mod_bytes_left;
+ int mod_bytes_left_save;
+ void *mod_buf_p;
+ void *rec_p = NULL;
+ darshan_record_id rec_id;
+
+ memset(mod_buf, 0, DARSHAN_DEF_COMP_BUF_SZ);
+ mod_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+
+ /* check each module for any data */
+ ret = darshan_log_getmod(infile, i, mod_buf, &mod_buf_sz);
+ if(ret < 0)
{
- fprintf(stderr, "Error: log file contains out of order rank data.\n");
- fflush(stderr);
+ fprintf(stderr, "Error: failed to get module %s data.\n",
+ darshan_module_names[i]);
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
return(-1);
}
- if(cp_file.rank != -1)
- last_rank = cp_file.rank;
+ else if(ret == 0)
+ {
+ /* skip modules not present in log file */
+ continue;
+ }
- if(!hash || hash == cp_file.hash)
+ /* skip modules with no defined logutil handlers */
+ if(!mod_logutils[i])
{
- if (obfuscate) obfuscate_file(key, &cp_file);
+ fprintf(stderr, "Warning: no log utility handlers defined "
+ "for module %s, SKIPPING\n", darshan_module_names[i]);
+ continue;
+ }
+
+ /* we have module data to convert */
+ /* NOTE: it is necessary to iterate through each module's
+ * records to correct any endianness issues before writing
+ * this data back to file
+ */
+ mod_bytes_left = mod_buf_sz;
+ mod_buf_p = mod_buf;
+ while(mod_bytes_left > 0)
+ {
+ mod_bytes_left_save = mod_bytes_left;
+ ret = mod_logutils[i]->log_get_record(&mod_buf_p, &mod_bytes_left,
+ &rec_p, &rec_id, infile->swap_flag);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: failed to parse module %s data record\n",
+ darshan_module_names[i]);
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
+ }
- ret = darshan_log_putfile(outfile, &job, &cp_file);
- if (ret < 0)
+ if(hash == rec_id)
{
- fprintf(stderr, "Error: failed to write file record.\n");
+ mod_buf_p = rec_p;
+ mod_buf_sz = mod_bytes_left_save - mod_bytes_left;
break;
}
+ else if(mod_bytes_left == 0)
+ {
+ mod_buf_p = mod_buf;
+ }
}
- } while((ret = darshan_log_getfile(infile, &job, &cp_file)) == 1);
+ ret = darshan_log_putmod(outfile, i, mod_buf_p, mod_buf_sz);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: failed to put module %s data.\n",
+ darshan_module_names[i]);
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
+ }
+ }
+ free(mod_buf);
+
+ /* write header to output file */
+ ret = darshan_log_putheader(outfile);
if(ret < 0)
{
- fprintf(stderr, "Error: failed to process log file.\n");
- fflush(stderr);
+ fprintf(stderr, "Error: unable to write header to output log file %s.\n", outfile_name);
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+ return(-1);
}
-done:
+ darshan_log_close(infile);
+ darshan_log_close(outfile);
+
for(i=0; i<mount_count; i++)
{
free(mnt_pts[i]);
@@ -332,15 +465,25 @@ done:
}
if(mount_count > 0)
{
- free(devs);
free(mnt_pts);
free(fs_types);
}
-
- darshan_log_close(infile);
- darshan_log_close(outfile);
+
+ HASH_ITER(hlink, rec_hash, ref, tmp)
+ {
+ HASH_DELETE(hlink, rec_hash, ref);
+ free(ref->rec.name);
+ free(ref);
+ }
return(ret);
}
-
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-util/darshan-hdf5-logutils.c b/darshan-util/darshan-hdf5-logutils.c
new file mode 100644
index 0000000..e4d6368
--- /dev/null
+++ b/darshan-util/darshan-hdf5-logutils.c
@@ -0,0 +1,105 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#define _GNU_SOURCE
+#include "darshan-util-config.h"
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <errno.h>
+
+#include "darshan-hdf5-logutils.h"
+
+/* counter name strings for the HDF5 module */
+#define X(a) #a,
+char *hdf5_counter_names[] = {
+ HDF5_COUNTERS
+};
+
+char *hdf5_f_counter_names[] = {
+ HDF5_F_COUNTERS
+};
+#undef X
+
+static int darshan_log_get_hdf5_file(void** hdf5_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag);
+static void darshan_log_print_hdf5_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+struct darshan_mod_logutil_funcs hdf5_logutils =
+{
+ .log_get_record = &darshan_log_get_hdf5_file,
+ .log_print_record = &darshan_log_print_hdf5_file,
+};
+
+static int darshan_log_get_hdf5_file(void** hdf5_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag)
+{
+ int i;
+ struct darshan_hdf5_file *file = (struct darshan_hdf5_file *)
+ (*hdf5_buf_p);
+
+ if(*bytes_left < sizeof(struct darshan_hdf5_file))
+ return(-1);
+
+ if(byte_swap_flag)
+ {
+ /* swap bytes if necessary */
+ DARSHAN_BSWAP64(&file->f_id);
+ DARSHAN_BSWAP64(&file->rank);
+ for(i=0; i<HDF5_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->counters[i]);
+ for(i=0; i<HDF5_F_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->fcounters[i]);
+ }
+
+ /* update/set output variables */
+ *file_rec = (void *)file;
+ *rec_id = file->f_id;
+ *hdf5_buf_p = (file + 1); /* increment input buf by size of file record */
+ *bytes_left -= sizeof(struct darshan_hdf5_file);
+
+ return(0);
+}
+
+static void darshan_log_print_hdf5_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ int i;
+ struct darshan_hdf5_file *hdf5_file_rec =
+ (struct darshan_hdf5_file *)file_rec;
+
+ for(i=0; i<HDF5_NUM_INDICES; i++)
+ {
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ hdf5_file_rec->rank, hdf5_file_rec->f_id, hdf5_counter_names[i],
+ hdf5_file_rec->counters[i], file_name, mnt_pt, fs_type);
+ }
+
+ for(i=0; i<HDF5_F_NUM_INDICES; i++)
+ {
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_HDF5_MOD],
+ hdf5_file_rec->rank, hdf5_file_rec->f_id, hdf5_f_counter_names[i],
+ hdf5_file_rec->fcounters[i], file_name, mnt_pt, fs_type);
+ }
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-util/darshan-hdf5-logutils.h b/darshan-util/darshan-hdf5-logutils.h
new file mode 100644
index 0000000..91ebd3f
--- /dev/null
+++ b/darshan-util/darshan-hdf5-logutils.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_HDF5_LOG_UTILS_H
+#define __DARSHAN_HDF5_LOG_UTILS_H
+
+#include "darshan-logutils.h"
+#include "darshan-hdf5-log-format.h"
+
+extern char *hdf5_counter_names[];
+extern char *hdf5_f_counter_names[];
+
+extern struct darshan_mod_logutil_funcs hdf5_logutils;
+
+#endif
diff --git a/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in b/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in
index 1663244..13041ed 100755
--- a/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in
+++ b/darshan-util/darshan-job-summary/bin/darshan-job-summary.pl.in
@@ -1,6 +1,6 @@
#!/usr/bin/perl -w
#
-# (C) 2009 by Argonne National Laboratory.
+# (C) 2015 by Argonne National Laboratory.
# See COPYRIGHT in top-level directory.
#
@@ -18,1315 +18,6 @@ use English;
use Number::Bytes::Human qw(format_bytes);
use POSIX qw(strftime);
-#
-# system commands used
-#
-my $darshan_parser = "$PREFIX/bin/darshan-parser";
-my $pdflatex = "pdflatex";
-my $epstopdf = "epstopdf";
-my $cp = "cp";
-my $mv = "mv";
-my $gnuplot;
-# Prefer gnuplot installed with darshan, if present.
-if(-x "$PREFIX/bin/gnuplot")
-{
- $gnuplot = "$PREFIX/bin/gnuplot";
-}
-else
-{
- $gnuplot = "gnuplot";
-}
-
-my $orig_dir = getcwd;
-my $output_file = "summary.pdf";
-my $verbose_flag = 0;
-my $input_file = "";
-my %access_hash = ();
-my @access_size = ();
-my %hash_files = ();
-my $size_est_flag = 0;
-my $read_interval_overflow_flag = 0;
-my $write_interval_overflow_flag = 0;
-
-# data structures for calculating performance
-my %hash_unique_file_time = ();
-my $shared_file_time = 0;
-my $total_job_bytes = 0;
-
-process_args();
-
-check_prereqs();
-
-my $tmp_dir = tempdir( CLEANUP => !$verbose_flag );
-if ($verbose_flag)
-{
- print "verbose: $tmp_dir\n";
-}
-
-
-open(TRACE, "$darshan_parser $input_file |") || die("Can't execute \"$darshan_parser $input_file\": $!\n");
-
-open(FA_READ, ">$tmp_dir/file-access-read.dat") || die("error opening output file: $!\n");
-open(FA_WRITE, ">$tmp_dir/file-access-write.dat") || die("error opening output file: $!\n");
-open(FA_READ_SH, ">$tmp_dir/file-access-read-sh.dat") || die("error opening output file: $!\n");
-open(FA_WRITE_SH, ">$tmp_dir/file-access-write-sh.dat") || die("error opening output file: $!\n");
-
-my $last_read_start = 0;
-my $last_write_start = 0;
-
-my $cumul_read_indep = 0;
-my $cumul_read_bytes_indep = 0;
-
-my $cumul_write_indep = 0;
-my $cumul_write_bytes_indep = 0;
-
-my $cumul_read_shared = 0;
-my $cumul_read_bytes_shared = 0;
-
-my $cumul_write_shared = 0;
-my $cumul_write_bytes_shared = 0;
-
-my $cumul_meta_shared = 0;
-my $cumul_meta_indep = 0;
-
-my $first_data_line = 1;
-my $current_rank = 0;
-my $current_hash = 0;
-my %file_record_hash = ();
-
-my %fs_data = ();
-
-while ($line = <TRACE>) {
- chomp($line);
-
- if ($line =~ /^\s*$/) {
- # ignore blank lines
- }
- elsif ($line =~ /^#/) {
- if ($line =~ /^# exe: /) {
- ($junk, $cmdline) = split(':', $line, 2);
- # add escape characters if needed for special characters in
- # command line
- $cmdline = encode('latex', $cmdline);
- }
- if ($line =~ /^# nprocs: /) {
- ($junk, $nprocs) = split(':', $line, 2);
- $procreads[$nprocs] = 0;
- }
- if ($line =~ /^# run time: /) {
- ($junk, $runtime) = split(':', $line, 2);
- }
- if ($line =~ /^# start_time: /) {
- ($junk, $starttime) = split(':', $line, 2);
- }
- if ($line =~ /^# uid: /) {
- ($junk, $uid) = split(':', $line, 2);
- }
- if ($line =~ /^# jobid: /) {
- ($junk, $jobid) = split(':', $line, 2);
- }
- if ($line =~ /^# darshan log version: /) {
- ($junk, $version) = split(':', $line, 2);
- $version =~ s/^\s+//;
- ($major, $minor) = split(/\./, $version, 2);
- }
- }
- else {
- # parse line
- @fields = split(/[\t ]+/, $line);
-
- # encode the file system name to protect against special characters
- $fields[5] = encode('latex', $fields[5]);
-
- # is this our first piece of data?
- if($first_data_line)
- {
- $current_rank = $fields[0];
- $current_hash = $fields[1];
- $first_data_line = 0;
- }
-
- # is this a new file record?
- if($fields[0] != $current_rank || $fields[1] != $current_hash)
- {
- # process previous record
- process_file_record($current_rank, $current_hash, \%file_record_hash);
-
- # reset variables for next record
- $current_rank = $fields[0];
- $current_hash = $fields[1];
- %file_record_hash = ();
- $file_record_hash{CP_NAME_SUFFIX} = $fields[4];
- }
-
- $file_record_hash{$fields[2]} = $fields[3];
-
- $summary{$fields[2]} += $fields[3];
-
- # record per-process POSIX read count
- if ($fields[2] eq "CP_POSIX_READS" || $fields[2] eq "CP_POSIX_FREADS") {
- if ($fields[0] == -1) {
- $procreads[$nprocs] += $fields[3];
- }
- else {
- $procreads[$fields[0]] += $fields[3];
- }
- }
-
- # record per-proces POSIX write count
- if ($fields[2] eq "CP_POSIX_WRITES" || $fields[2] eq "CP_POSIX_FWRITES") {
- if ($fields[0] == -1) {
- $procwrites[$nprocs] += $fields[3];
- }
- else {
- $procwrites[$fields[0]] += $fields[3];
- }
- }
-
- # seperate accumulators for independent and shared reads and writes
- if ($fields[2] eq "CP_F_POSIX_READ_TIME" && $fields[0] == -1){
- $cumul_read_shared += $fields[3];
- }
- if ($fields[2] eq "CP_F_POSIX_READ_TIME" && $fields[0] != -1){
- $cumul_read_indep += $fields[3];
- }
- if ($fields[2] eq "CP_F_POSIX_WRITE_TIME" && $fields[0] == -1){
- $cumul_write_shared += $fields[3];
- }
- if ($fields[2] eq "CP_F_POSIX_WRITE_TIME" && $fields[0] != -1){
- $cumul_write_indep += $fields[3];
- }
-
- if ($fields[2] eq "CP_F_POSIX_META_TIME" && $fields[0] == -1){
- $cumul_meta_shared += $fields[3];
- }
- if ($fields[2] eq "CP_F_POSIX_META_TIME" && $fields[0] != -1){
- $cumul_meta_indep += $fields[3];
- }
-
- if ((($fields[2] eq "CP_BYTES_READ") or
- ($fields[2] eq "CP_BYTES_WRITTEN")) and
- not defined($fs_data{$fields[5]}))
- {
- $fs_data{$fields[5]} = [0,0];
- }
-
- if ($fields[2] eq "CP_BYTES_READ" && $fields[0] == -1){
- $cumul_read_bytes_shared += $fields[3];
- $fs_data{$fields[5]}->[0] += $fields[3];
- }
- if ($fields[2] eq "CP_BYTES_READ" && $fields[0] != -1){
- $cumul_read_bytes_indep += $fields[3];
- $fs_data{$fields[5]}->[0] += $fields[3];
- }
- if ($fields[2] eq "CP_BYTES_WRITTEN" && $fields[0] == -1){
- $cumul_write_bytes_shared += $fields[3];
- $fs_data{$fields[5]}->[1] += $fields[3];
- }
- if ($fields[2] eq "CP_BYTES_WRITTEN" && $fields[0] != -1){
- $cumul_write_bytes_indep += $fields[3];
- $fs_data{$fields[5]}->[1] += $fields[3];
- }
-
- # record start and end of reads and writes
-
- if ($fields[2] eq "CP_F_READ_START_TIMESTAMP") {
- # store until we find the end
- # adjust for systems that give absolute time stamps
- $last_read_start = $fields[3];
- }
- if ($fields[2] eq "CP_F_READ_END_TIMESTAMP" && $fields[3] != 0) {
- # assume we got the read start already
- my $xdelta = $fields[3] - $last_read_start;
- # adjust for systems that have absolute time stamps
- if($last_read_start > $starttime) {
- $last_read_start -= $starttime;
- }
- if($fields[3] > $runtime && !$read_interval_overflow_flag)
- {
- $read_interval_overflow_flag = 1;
- print "Warning: detected read access at time $fields[3] but runtime is only $runtime seconds.\n";
- }
- if($fields[0] == -1){
- print FA_READ_SH "$last_read_start\t0\t$xdelta\t0\n";
- }
- else{
- print FA_READ "$last_read_start\t$fields[0]\t$xdelta\t0\n";
- }
- }
- if ($fields[2] eq "CP_F_WRITE_START_TIMESTAMP") {
- # store until we find the end
- $last_write_start = $fields[3];
- }
- if ($fields[2] eq "CP_F_WRITE_END_TIMESTAMP" && $fields[3] != 0) {
- # assume we got the write start already
- my $xdelta = $fields[3] - $last_write_start;
- # adjust for systems that have absolute time stamps
- if($last_write_start > $starttime) {
- $last_write_start -= $starttime;
- }
- if($fields[3] > $runtime && !$write_interval_overflow_flag)
- {
- $write_interval_overflow_flag = 1;
- print "Warning: detected write access at time $fields[3] but runtime is only $runtime seconds.\n";
- }
- if($fields[0] == -1){
- print FA_WRITE_SH "$last_write_start\t0\t$xdelta\t0\n";
- }
- else{
- print FA_WRITE "$last_write_start\t$fields[0]\t$xdelta\t0\n";
- }
- }
-
- if ($fields[2] =~ /^CP_ACCESS(.)_ACCESS/) {
- $access_size[$1] = $fields[3];
- }
- if ($fields[2] =~ /^CP_ACCESS(.)_COUNT/) {
- my $tmp_access_size = $access_size[$1];
- if(defined $access_hash{$tmp_access_size}){
- $access_hash{$tmp_access_size} += $fields[3];
- }
- else{
- $access_hash{$tmp_access_size} = $fields[3];
- }
- }
- }
-}
-
-close(TRACE) || die "darshan-parser failure: $! $?";
-
-#
-# Exit out if there are no actual file accesses
-#
-if ($first_data_line)
-{
- $strtm = strftime("%a %b %e %H:%M:%S %Y", localtime($starttime));
-
- print "This darshan log has no file records. No summary was produced.\n";
- print " jobid:$jobid\n";
- print " uid:$uid\n";
- print "starttime: $strtm ($starttime )\n";
- print " runtime:$runtime (seconds)\n";
- print " nprocs:$nprocs\n";
- print " version: $version\n";
- exit(1);
-}
-
-# process last file record
-$file_record_hash{CP_NAME_SUFFIX} = $fields[4];
-process_file_record($current_rank, $current_hash, \%file_record_hash);
-
-# Fudge one point at the end to make xrange match in read and write plots.
-# For some reason I can't get the xrange command to work. -Phil
-print FA_READ "$runtime\t-1\t0\t0\n";
-print FA_WRITE "$runtime\t-1\t0\t0\n";
-print FA_READ_SH "$runtime\t0\t0\t0\n";
-print FA_WRITE_SH "$runtime\t0\t0\t0\n";
-close(FA_READ);
-close(FA_WRITE);
-close(FA_READ_SH);
-close(FA_WRITE_SH);
-
-# counts of operations
-open(COUNTS, ">$tmp_dir/counts.dat") || die("error opening output file: $!\n");
-print COUNTS "# P=POSIX, MI=MPI-IO indep., MC=MPI-IO coll., R=read, W=write\n";
-print COUNTS "# PR, MIR, MCR, PW, MIW, MCW, Popen, Pseek, Pstat\n";
-my $total_posix_opens = $summary{CP_POSIX_OPENS} + $summary{CP_POSIX_FOPENS};
-my $total_syncs = $summary{CP_POSIX_FSYNCS} + $summary{CP_POSIX_FDSYNCS};
-print COUNTS "Read, ", $summary{CP_POSIX_READS} + $summary{CP_POSIX_FREADS}, ", ",
- $summary{CP_INDEP_READS}, ", ", $summary{CP_COLL_READS}, "\n",
- "Write, ", $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FWRITES}, ", ",
- $summary{CP_INDEP_WRITES}, ", ", $summary{CP_COLL_WRITES}, "\n",
- "Open, ", $total_posix_opens, ", ", $summary{CP_INDEP_OPENS},", ",
- $summary{CP_COLL_OPENS}, "\n",
- "Stat, ", $summary{CP_POSIX_STATS}, ", 0, 0\n",
- "Seek, ", $summary{CP_POSIX_SEEKS}, ", 0, 0\n",
- "Mmap, ", $summary{CP_POSIX_MMAPS}, ", 0, 0\n",
- "Fsync, ", $total_syncs, ", 0, 0\n";
-close COUNTS;
-
-# histograms of reads and writes
-open (HIST, ">$tmp_dir/hist.dat") || die("error opening output file: $!\n");
-print HIST "# size_range read write\n";
-print HIST "0-100, ", $summary{CP_SIZE_READ_0_100}, ", ",
- $summary{CP_SIZE_WRITE_0_100}, "\n";
-print HIST "101-1K, ", $summary{CP_SIZE_READ_100_1K}, ", ",
- $summary{CP_SIZE_WRITE_100_1K}, "\n";
-print HIST "1K-10K, ", $summary{CP_SIZE_READ_1K_10K}, ", ",
- $summary{CP_SIZE_WRITE_1K_10K}, "\n";
-print HIST "10K-100K, ", $summary{CP_SIZE_READ_10K_100K}, ", ",
- $summary{CP_SIZE_WRITE_10K_100K}, "\n";
-print HIST "100K-1M, ", $summary{CP_SIZE_READ_100K_1M}, ", ",
- $summary{CP_SIZE_WRITE_100K_1M}, "\n";
-print HIST "1M-4M, ", $summary{CP_SIZE_READ_1M_4M}, ", ",
- $summary{CP_SIZE_WRITE_1M_4M}, "\n";
-print HIST "4M-10M, ", $summary{CP_SIZE_READ_4M_10M}, ", ",
- $summary{CP_SIZE_WRITE_4M_10M}, "\n";
-print HIST "10M-100M, ", $summary{CP_SIZE_READ_10M_100M}, ", ",
- $summary{CP_SIZE_WRITE_10M_100M}, "\n";
-print HIST "100M-1G, ", $summary{CP_SIZE_READ_100M_1G}, ", ",
- $summary{CP_SIZE_WRITE_100M_1G}, "\n";
-print HIST "1G+, ", $summary{CP_SIZE_READ_1G_PLUS}, ", ",
- $summary{CP_SIZE_WRITE_1G_PLUS}, "\n";
-close HIST;
-
-# sequential and consecutive accesses
-open (PATTERN, ">$tmp_dir/pattern.dat") || die("error opening output file: $!\n");
-print PATTERN "# op total sequential consecutive\n";
-print PATTERN "Read, ", $summary{CP_POSIX_READS} + $summary{CP_POSIX_FREADS}, ", ",
- $summary{CP_SEQ_READS}, ", ", $summary{CP_CONSEC_READS}, "\n";
-print PATTERN "Write, ", $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FWRITES}, ", ",
- $summary{CP_SEQ_WRITES}, ", ", $summary{CP_CONSEC_WRITES}, "\n";
-close PATTERN;
-
-# aligned I/O
-open (ALIGN, ">$tmp_dir/align.dat") || die("error opening output file: $!\n");
-print ALIGN "# total unaligned_mem unaligned_file align_mem align_file\n";
-print ALIGN $summary{CP_POSIX_READS} + $summary{CP_POSIX_WRITES} + $summary{CP_POSIX_FREADS} + $summary{CP_POSIX_FWRITES}
-, ", ",
- $summary{CP_MEM_NOT_ALIGNED}, ", ", $summary{CP_FILE_NOT_ALIGNED}, "\n";
-close ALIGN;
-
-# MPI types
-open (TYPES, ">$tmp_dir/types.dat") || die("error opening output file: $!\n");
-print TYPES "# type use_count\n";
-print TYPES "Named, ", $summary{CP_COMBINER_NAMED}, "\n";
-print TYPES "Dup, ", $summary{CP_COMBINER_DUP}, "\n";
-print TYPES "Contig, ", $summary{CP_COMBINER_CONTIGUOUS}, "\n";
-print TYPES "Vector, ", $summary{CP_COMBINER_VECTOR}, "\n";
-print TYPES "HvecInt, ", $summary{CP_COMBINER_HVECTOR_INTEGER}, "\n";
-print TYPES "Hvector, ", $summary{CP_COMBINER_HVECTOR}, "\n";
-print TYPES "Indexed, ", $summary{CP_COMBINER_INDEXED}, "\n";
-print TYPES "HindInt, ", $summary{CP_COMBINER_HINDEXED_INTEGER}, "\n";
-print TYPES "Hindexed, ", $summary{CP_COMBINER_HINDEXED}, "\n";
-print TYPES "IndBlk, ", $summary{CP_COMBINER_INDEXED_BLOCK}, "\n";
-print TYPES "StructInt, ", $summary{CP_COMBINER_STRUCT_INTEGER}, "\n";
-print TYPES "Struct, ", $summary{CP_COMBINER_STRUCT}, "\n";
-print TYPES "Subarray, ", $summary{CP_COMBINER_SUBARRAY}, "\n";
-print TYPES "Darray, ", $summary{CP_COMBINER_DARRAY}, "\n";
-print TYPES "F90Real, ", $summary{CP_COMBINER_F90_REAL}, "\n";
-print TYPES "F90Complex, ", $summary{CP_COMBINER_F90_COMPLEX}, "\n";
-print TYPES "F90Int, ", $summary{CP_COMBINER_F90_INTEGER}, "\n";
-print TYPES "Resized, ", $summary{CP_COMBINER_RESIZED}, "\n";
-close TYPES;
-
-# generate histogram of process I/O counts
-#
-# NOTE: NEED TO FILL IN ACTUAL WRITE DATA!!!
-#
-$minprocread = (defined $procreads[0]) ? $procreads[0] : 0;
-$maxprocread = (defined $procreads[0]) ? $procreads[0] : 0;
-for ($i=1; $i < $nprocs; $i++) {
- $rdi = (defined $procreads[$i]) ? $procreads[$i] : 0;
- $minprocread = ($rdi > $minprocread) ? $minprocread : $rdi;
- $maxprocread = ($rdi < $maxprocread) ? $maxprocread : $rdi;
-}
-$minprocread += $procreads[$nprocs];
-$maxprocread += $procreads[$nprocs];
-# print "$minprocread $maxprocread\n";
-
- at bucket = ( 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 );
-
-for ($i=0; $i < $nprocs; $i++) {
- $mysize = ((defined $procreads[$i]) ? $procreads[$i] : 0) +
- $procreads[$nprocs];
- $mysize -= $minprocread;
- $mybucket = ($mysize > 0) ?
- (($mysize * 10) / ($maxprocread - $minprocread)) : 0;
- $bucket[$mybucket]++;
-}
-
-open(IODIST, ">$tmp_dir/iodist.dat") || die("error opening output file: $!\n");
-print IODIST "# bucket n_procs_rd n_procs_wr\n";
-print IODIST "# NOTE: WRITES ARE A COPY OF READS FOR NOW!!!\n";
-
-$bucketsize = $maxprocread - $minprocread / 10;
-# TODO: do writes also, is dropping a 0 in for now
-for ($i=0; $i < 10; $i++) {
- print IODIST $bucketsize * $i + $minprocread, "-",
- $bucketsize * ($i+1) + $minprocread, ", ", $bucket[$i], ", 0\n";
-}
-close IODIST;
-
-# generate title for summary
-($executable, $junk) = split(' ', $cmdline, 2);
- at parts = split('/', $executable);
-$cmd = $parts[$#parts];
-
- at timearray = localtime($starttime);
-$year = $timearray[5] + 1900;
-$mon = $timearray[4] + 1;
-$mday = $timearray[3];
-
-open(TITLE, ">$tmp_dir/title.tex") || die("error opening output file:$!\n");
-print TITLE "
-\\rhead{\\thepage\\ of \\pageref{LastPage}}
-\\chead[
-\\large $cmd ($mon/$mday/$year)
-]
-{
-\\large $cmd ($mon/$mday/$year)
-}
-\\cfoot[
-\\scriptsize{$cmdline}
-]
-{
-\\scriptsize{$cmdline}
-}
-";
-close TITLE;
-
-open(TABLES, ">$tmp_dir/job-table.tex") || die("error opening output file:$!\n");
-print TABLES "
-\\begin{tabular}{|p{.47\\columnwidth}|p{.35\\columnwidth}|p{.47\\columnwidth}|p{.6\\columnwidth}|}
-\\hline
-jobid: $jobid \& uid: $uid \& nprocs: $nprocs \& runtime: $runtime seconds\\\\
-\\hline
-\\end{tabular}
-";
-close TABLES;
-
-open(TABLES, ">$tmp_dir/access-table.tex") || die("error opening output file:$!\n");
-print TABLES "
-\\begin{tabular}{r|r}
-\\multicolumn{2}{c}{ } \\\\
-\\multicolumn{2}{c}{Most Common Access Sizes} \\\\
-\\hline
-access size \& count \\\\
-\\hline
-\\hline
-";
-
-# sort access sizes (descending)
-my $i = 0;
-foreach $value (sort {$access_hash{$b} <=> $access_hash{$a} } keys %access_hash) {
- if($i == 4) {
- last;
- }
- if($access_hash{$value} == 0) {
- last;
- }
- print TABLES "$value \& $access_hash{$value} \\\\\n";
- $i++;
-}
-
-print TABLES "
-\\hline
-\\end{tabular}
-";
-close TABLES;
-
-open(TABLES, ">$tmp_dir/file-count-table.tex") || die("error opening output file:$!\n");
-print TABLES "
-\\begin{tabular}{r|r|r|r}
-\\multicolumn{4}{c}{ } \\\\
-\\multicolumn{4}{c}{File Count Summary} \\\\
-";
-if($size_est_flag == 1)
-{
-print TABLES "
-\\multicolumn{4}{c}{(estimated by I/O access offsets)} \\\\
-";
-}
-print TABLES "
-\\hline
-type \& number of files \& avg. size \& max size \\\\
-\\hline
-\\hline
-";
-my $counter;
-my $sum;
-my $max;
-my $key;
-my $avg;
-
-$counter = 0;
-$sum = 0;
-$max = 0;
-foreach $key (keys %hash_files) {
- $counter++;
- if($hash_files{$key}{'min_open_size'} >
- $hash_files{$key}{'max_size'})
- {
- $sum += $hash_files{$key}{'min_open_size'};
- if($hash_files{$key}{'min_open_size'} > $max)
- {
- $max = $hash_files{$key}{'min_open_size'};
- }
- }
- else
- {
- $sum += $hash_files{$key}{'max_size'};
- if($hash_files{$key}{'max_size'} > $max)
- {
- $max = $hash_files{$key}{'max_size'};
- }
- }
-}
-if($counter > 0) { $avg = $sum / $counter; }
-else { $avg = 0; }
-$avg = format_bytes($avg);
-$max = format_bytes($max);
-print TABLES "total opened \& $counter \& $avg \& $max \\\\\n";
-
-$counter = 0;
-$sum = 0;
-$max = 0;
-foreach $key (keys %hash_files) {
- if($hash_files{$key}{'was_read'} && !($hash_files{$key}{'was_written'}))
- {
- $counter++;
- if($hash_files{$key}{'min_open_size'} >
- $hash_files{$key}{'max_size'})
- {
- $sum += $hash_files{$key}{'min_open_size'};
- if($hash_files{$key}{'min_open_size'} > $max)
- {
- $max = $hash_files{$key}{'min_open_size'};
- }
- }
- else
- {
- $sum += $hash_files{$key}{'max_size'};
- if($hash_files{$key}{'max_size'} > $max)
- {
- $max = $hash_files{$key}{'max_size'};
- }
- }
- }
-}
-if($counter > 0) { $avg = $sum / $counter; }
-else { $avg = 0; }
-$avg = format_bytes($avg);
-$max = format_bytes($max);
-print TABLES "read-only files \& $counter \& $avg \& $max \\\\\n";
-
-$counter = 0;
-$sum = 0;
-$max = 0;
-foreach $key (keys %hash_files) {
- if(!($hash_files{$key}{'was_read'}) && $hash_files{$key}{'was_written'})
- {
- $counter++;
- if($hash_files{$key}{'min_open_size'} >
- $hash_files{$key}{'max_size'})
- {
- $sum += $hash_files{$key}{'min_open_size'};
- if($hash_files{$key}{'min_open_size'} > $max)
- {
- $max = $hash_files{$key}{'min_open_size'};
- }
- }
- else
- {
- $sum += $hash_files{$key}{'max_size'};
- if($hash_files{$key}{'max_size'} > $max)
- {
- $max = $hash_files{$key}{'max_size'};
- }
- }
- }
-}
-if($counter > 0) { $avg = $sum / $counter; }
-else { $avg = 0; }
-$avg = format_bytes($avg);
-$max = format_bytes($max);
-print TABLES "write-only files \& $counter \& $avg \& $max \\\\\n";
-
-$counter = 0;
-$sum = 0;
-$max = 0;
-foreach $key (keys %hash_files) {
- if($hash_files{$key}{'was_read'} && $hash_files{$key}{'was_written'})
- {
- $counter++;
- if($hash_files{$key}{'min_open_size'} >
- $hash_files{$key}{'max_size'})
- {
- $sum += $hash_files{$key}{'min_open_size'};
- if($hash_files{$key}{'min_open_size'} > $max)
- {
- $max = $hash_files{$key}{'min_open_size'};
- }
- }
- else
- {
- $sum += $hash_files{$key}{'max_size'};
- if($hash_files{$key}{'max_size'} > $max)
- {
- $max = $hash_files{$key}{'max_size'};
- }
- }
- }
-}
-if($counter > 0) { $avg = $sum / $counter; }
-else { $avg = 0; }
-$avg = format_bytes($avg);
-$max = format_bytes($max);
-print TABLES "read/write files \& $counter \& $avg \& $max \\\\\n";
-
-$counter = 0;
-$sum = 0;
-$max = 0;
-foreach $key (keys %hash_files) {
- if($hash_files{$key}{'was_written'} &&
- $hash_files{$key}{'min_open_size'} == 0 &&
- $hash_files{$key}{'max_size'} > 0)
- {
- $counter++;
- if($hash_files{$key}{'min_open_size'} >
- $hash_files{$key}{'max_size'})
- {
- $sum += $hash_files{$key}{'min_open_size'};
- if($hash_files{$key}{'min_open_size'} > $max)
- {
- $max = $hash_files{$key}{'min_open_size'};
- }
- }
- else
- {
- $sum += $hash_files{$key}{'max_size'};
- if($hash_files{$key}{'max_size'} > $max)
- {
- $max = $hash_files{$key}{'max_size'};
- }
- }
- }
-}
-if($counter > 0) { $avg = $sum / $counter; }
-else { $avg = 0; }
-$avg = format_bytes($avg);
-$max = format_bytes($max);
-print TABLES "created files \& $counter \& $avg \& $max \\\\\n";
-
-print TABLES "
-\\hline
-\\end{tabular}
-";
-close(TABLES);
-
-
-#
-# Generate Per Filesystem Data
-#
-open(TABLES, ">$tmp_dir/fs-data-table.tex") || die("error opening output files:$!\n");
-if (($major > 1) or ($minor > 23))
-{
- print TABLES "
- \\begin{tabular}{c|r|r|r|r}
- \\multicolumn{5}{c}{ } \\\\
- \\multicolumn{5}{c}{Data Transfer Per Filesystem} \\\\
- \\hline
- \\multirow{2}{*}{File System} \& \\multicolumn{2}{c}{Write} \\vline \& \\multicolumn{2}{c}{Read} \\\\
- \\cline{2-5}
- \& MiB \& Ratio \& MiB \& Ratio \\\\\
- \\hline
- \\hline
- ";
- foreach $key (keys %fs_data)
- {
- my $wr_total_mb = ($fs_data{$key}->[1] / (1024*1024));
- my $rd_total_mb = ($fs_data{$key}->[0] / (1024*1024));
- my $wr_total_rt;
-
- if ($cumul_write_bytes_shared+$cumul_write_bytes_indep)
- {
- $wr_total_rt = ($fs_data{$key}->[1] / ($cumul_write_bytes_shared + $cumul_write_bytes_indep));
- }
- else
- {
- $wr_total_rt = 0;
- }
-
- my $rd_total_rt;
- if ($cumul_read_bytes_shared+$cumul_read_bytes_indep)
- {
- $rd_total_rt = ($fs_data{$key}->[0] / ($cumul_read_bytes_shared + $cumul_read_bytes_indep));
- }
- else
- {
- $rd_total_rt = 0;
- }
-
- printf TABLES "%s \& %.5f \& %.5f \& %.5f \& %.5f \\\\\n",
- $key, $wr_total_mb, $wr_total_rt, $rd_total_mb, $rd_total_rt;
-}
-print TABLES "
-\\hline
-\\end{tabular}
-";
-}
-else
-{
- print TABLES "
-\\rule{0in}{1in}
-\\parbox{5in}{Log versions prior to 1.24 do not support per-filesystem data.}
-";
-}
-close(TABLES);
-
-
-open(TIME, ">$tmp_dir/time-summary.dat") || die("error opening output file:$!\n");
-print TIME "# <type>, <app time>, <read>, <write>, <meta>\n";
-print TIME "POSIX, ", ((($runtime * $nprocs - $summary{CP_F_POSIX_READ_TIME} -
- $summary{CP_F_POSIX_WRITE_TIME} -
- $summary{CP_F_POSIX_META_TIME})/($runtime * $nprocs)) * 100);
-print TIME ", ", (($summary{CP_F_POSIX_READ_TIME}/($runtime * $nprocs))*100);
-print TIME ", ", (($summary{CP_F_POSIX_WRITE_TIME}/($runtime * $nprocs))*100);
-print TIME ", ", (($summary{CP_F_POSIX_META_TIME}/($runtime * $nprocs))*100), "\n";
-print TIME "MPI-IO, ", ((($runtime * $nprocs - $summary{CP_F_MPI_READ_TIME} -
- $summary{CP_F_MPI_WRITE_TIME} -
- $summary{CP_F_MPI_META_TIME})/($runtime * $nprocs)) * 100);
-print TIME ", ", (($summary{CP_F_MPI_READ_TIME}/($runtime * $nprocs))*100);
-print TIME ", ", (($summary{CP_F_MPI_WRITE_TIME}/($runtime * $nprocs))*100);
-print TIME ", ", (($summary{CP_F_MPI_META_TIME}/($runtime * $nprocs))*100), "\n";
-close TIME;
-
-# copy template files to tmp tmp_dir
-system "$cp $PREFIX/share/*.gplt $tmp_dir/";
-system "$cp $PREFIX/share/*.tex $tmp_dir/";
-
-# generate template for file access plot (we have to set range)
-my $ymax = $nprocs;
-my $yinc = int($nprocs / 8);
-if($yinc == 0) {$yinc=1;}
-my $ymaxtic = $nprocs-1;
-open(FILEACC, ">$tmp_dir/file-access-read-eps.gplt") || die("error opening output file:$!\n");
-print FILEACC "#!/usr/bin/gnuplot -persist
-
-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in
-set output \"file-access-read.eps\"
-set ylabel \"MPI rank\"
-set xlabel \"hours:minutes:seconds\"
-set xdata time
-set timefmt \"%s\"
-set format x \"%H:%M:%S\"
-set yrange [-1:$ymax]
-set title \"Timespan from first to last read access on independent files\"
-set xrange [\"0\":\"$runtime\"]
-set ytics 0,$yinc,$ymaxtic
-#set ytics -1,1
-set lmargin 4
-
-# color blindness work around
-set style line 2 lc 3
-set style line 3 lc 4
-set style line 4 lc 5
-set style line 5 lc 2
-set style increment user
-
-# lw 3 to make lines thicker...
-# note that writes are slightly offset for better visibility
-plot \"file-access-read.dat\" using 1:2:3:4 with vectors nohead filled notitle
-";
-close FILEACC;
-
-open(FILEACC, ">$tmp_dir/file-access-write-eps.gplt") || die("error opening output file:$!\n");
-print FILEACC "#!/usr/bin/gnuplot -persist
-
-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in
-set output \"file-access-write.eps\"
-set ylabel \"MPI rank\"
-set xlabel \"hours:minutes:seconds\"
-set xdata time
-set timefmt \"%s\"
-set format x \"%H:%M:%S\"
-set title \"Timespan from first to last write access on independent files\"
-set yrange [-1:$ymax]
-set xrange [\"0\":\"$runtime\"]
-#set ytics -1,1
-set ytics 0,$yinc,$ymaxtic
-set lmargin 4
-
-# color blindness work around
-set style line 2 lc 3
-set style line 3 lc 4
-set style line 4 lc 5
-set style line 5 lc 2
-set style increment user
-
-# lw 3 to make lines thicker...
-plot \"file-access-write.dat\" using 1:2:3:4 with vectors nohead filled lt 2 notitle
-";
-close FILEACC;
-
-open(FILEACC, ">$tmp_dir/file-access-shared-eps.gplt") || die("error opening output file:$!\n");
-print FILEACC "#!/usr/bin/gnuplot -persist
-
-set terminal postscript eps color solid font \"Helvetica\" 18 size 10in,2.5in
-set output \"file-access-shared.eps\"
-set xlabel \"hours:minutes:seconds\"
-set xdata time
-set timefmt \"%s\"
-set format x \"%H:%M:%S\"
-unset ytics
-set ylabel \"All processes\"
-set xrange [\"0\":\"$runtime\"]
-set yrange [-1:1]
-set title \"Timespan from first to last access on files shared by all processes\"
-set lmargin 4
-
-# color blindness work around
-set style line 2 lc 3
-set style line 3 lc 4
-set style line 4 lc 5
-set style line 5 lc 2
-set style increment user
-
-plot \"file-access-read-sh.dat\" using 1:2:3:4 with vectors nohead filled lw 10 title \"read\", \\
-\"file-access-write-sh.dat\" using 1:((\$2)-.2):3:4 with vectors nohead filled lw 10 title \"write\"
-";
-close FILEACC;
-
-$cumul_read_indep /= $nprocs;
-$cumul_read_bytes_indep /= $nprocs;
-$cumul_read_bytes_indep /= 1048576.0;
-
-$cumul_write_indep /= $nprocs;
-$cumul_write_bytes_indep /= $nprocs;
-$cumul_write_bytes_indep /= 1048576.0;
-
-$cumul_read_shared /= $nprocs;
-$cumul_read_bytes_shared /= $nprocs;
-$cumul_read_bytes_shared /= 1048576.0;
-
-$cumul_write_shared /= $nprocs;
-$cumul_write_bytes_shared /= $nprocs;
-$cumul_write_bytes_shared /= 1048576.0;
-
-$cumul_meta_shared /= $nprocs;
-$cumul_meta_indep /= $nprocs;
-
-open(FILEACC, ">$tmp_dir/file-access-table.tex") || die("error opening output file:$!\n");
-print FILEACC "
-\\begin{tabular}{l|p{1.7in}r}
-\\multicolumn{3}{c}{Average I/O per process} \\\\
-\\hline
- \& Cumulative time spent in I/O functions (seconds) \& Amount of I/O (MB) \\\\
-\\hline
-\\hline
-";
-
-# printf to get consistent precision in output
-printf(FILEACC "Independent reads \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\",
- $cumul_read_indep, $cumul_read_bytes_indep);
-printf(FILEACC "Independent writes \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\",
- $cumul_write_indep, $cumul_write_bytes_indep);
-printf(FILEACC "Independent metadata \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{N/A} \\\\",
- $cumul_meta_indep);
-printf(FILEACC "Shared reads \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\",
- $cumul_read_shared, $cumul_read_bytes_shared);
-printf(FILEACC "Shared writes \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{%f} \\\\",
- $cumul_write_shared, $cumul_write_bytes_shared);
-printf(FILEACC "Shared metadata \& \\multicolumn{1}{r}{%f} \& \\multicolumn{1}{r}{N/A} \\\\",
- $cumul_meta_shared);
-
-print FILEACC "
-\\hline
-\\end{tabular}
-";
-close(FILEACC);
-
-#
-# Variance Data
-#
-open(VARP, ">$tmp_dir/variance-table.tex") || die("error opening output file:$!\n");
-print VARP "
-\\begin{tabular}{c|r|r|r|r|r|r|r|r|r}
-\\multicolumn{10}{c}{} \\\\
-\\multicolumn{10}{c}{Variance in Shared Files} \\\\
-\\hline
-File \& Processes \& \\multicolumn{3}{c}{Fastest} \\vline \&
-\\multicolumn{3}{c}{Slowest} \\vline \& \\multicolumn{2}{c}{\$\\sigma\$} \\\\
-\\cline{3-10}
-Suffix \& \& Rank \& Time \& Bytes \& Rank \& Time \& Bytes \& Time \& Bytes \\\\
-\\hline
-\\hline
-";
-
-my $curcount = 1;
-foreach $key (sort { $hash_files{$b}{'slowest_time'} <=> $hash_files{$a}{'slowest_time'} } keys %hash_files) {
-
- if ($curcount > 20) { last; }
-
- if ($hash_files{$key}{'procs'} > 1)
- {
- my $vt = sprintf("%.3g", sqrt($hash_files{$key}{'variance_time'}));
- my $vb = sprintf("%.3g", sqrt($hash_files{$key}{'variance_bytes'}));
- my $fast_bytes = format_bytes($hash_files{$key}{'fastest_bytes'});
- my $slow_bytes = format_bytes($hash_files{$key}{'slowest_bytes'});
- my $name = encode('latex', $hash_files{$key}{'name'});
-
- print VARP "
- $name \&
- $hash_files{$key}{'procs'} \&
- $hash_files{$key}{'fastest_rank'} \&
- $hash_files{$key}{'fastest_time'} \&
- $fast_bytes \&
- $hash_files{$key}{'slowest_rank'} \&
- $hash_files{$key}{'slowest_time'} \&
- $slow_bytes \&
- $vt \&
- $vb \\\\
- ";
- $curcount++;
- }
-}
-
-print VARP "
-\\hline
-\\end{tabular}
-";
-close(VARP);
-
-# calculate performance
-##########################################################################
-
-# what was the slowest time by any proc for unique file access?
-my $slowest_uniq_time = 0;
-if(keys %hash_unique_file_time > 0)
-{
- $slowest_uniq_time < $_ and $slowest_uniq_time = $_ for values %hash_unique_file_time;
-}
-print("Slowest unique file time: $slowest_uniq_time\n");
-print("Slowest shared file time: $shared_file_time\n");
-print("Total bytes read and written by app (may be incorrect): $total_job_bytes\n");
-my $tmp_total_time = $slowest_uniq_time+$shared_file_time;
-print("Total absolute I/O time: $tmp_total_time\n");
-
-# move to tmp_dir
-chdir $tmp_dir;
-
-# execute gnuplot scripts
-system "$gnuplot counts-eps.gplt";
-system "$epstopdf counts.eps";
-system "$gnuplot hist-eps.gplt";
-system "$epstopdf hist.eps";
-system "$gnuplot pattern-eps.gplt";
-system "$epstopdf pattern.eps";
-system "$gnuplot time-summary-eps.gplt";
-system "$epstopdf time-summary.eps";
-system "$gnuplot file-access-read-eps.gplt";
-system "$epstopdf file-access-read.eps";
-system "$gnuplot file-access-write-eps.gplt";
-system "$epstopdf file-access-write.eps";
-system "$gnuplot file-access-shared-eps.gplt";
-system "$epstopdf file-access-shared.eps";
-
-#system "gnuplot align-pdf.gplt";
-#system "gnuplot iodist-pdf.gplt";
-#system "gnuplot types-pdf.gplt";
-
-# generate summary PDF
-# NOTE: an autoconf test determines if -halt-on-error is available and sets
-# __CP_PDFLATEX_HALT_ON_ERROR accordingly
-$system_rc = system "$pdflatex @__CP_PDFLATEX_HALT_ON_ERROR@ summary.tex > latex.output";
-if($system_rc)
-{
- print("LaTeX generation (phase1) failed [$system_rc], aborting summary creation.\n");
- print("error log:\n");
- system("tail latex.output");
- exit(1);
-}
-$system_rc = system "$pdflatex @__CP_PDFLATEX_HALT_ON_ERROR@ summary.tex > latex.output2";
-if($system_rc)
-{
- print("LaTeX generation (phase2) failed [$system_rc], aborting summary creation.\n");
- print("error log:\n");
- system("tail latex.output2");
- exit(1);
-}
-
-# get back out of tmp dir and grab results
-chdir $orig_dir;
-system "$mv $tmp_dir/summary.pdf $output_file";
-
-
-sub process_file_record
-{
- my $rank = $_[0];
- my $hash = $_[1];
- my(%file_record) = %{$_[2]};
-
- if($file_record{'CP_INDEP_OPENS'} == 0 &&
- $file_record{'CP_COLL_OPENS'} == 0 &&
- $file_record{'CP_POSIX_OPENS'} == 0 &&
- $file_record{'CP_POSIX_FOPENS'} == 0)
- {
- # file wasn't really opened, just stat probably
- return;
- }
-
- # record smallest open time size reported by any rank
- if(!defined($hash_files{$hash}{'min_open_size'}) ||
- $hash_files{$hash}{'min_open_size'} >
- $file_record{'CP_SIZE_AT_OPEN'})
- {
- # size at open will be set to -1 if the darshan library was not
- # configured to stat files at open time
- if($file_record{'CP_SIZE_AT_OPEN'} < 0)
- {
- $hash_files{$hash}{'min_open_size'} = 0;
- # set flag indicating that file sizes are estimated
- $size_est_flag = 1;
- }
- else
- {
- $hash_files{$hash}{'min_open_size'} =
- $file_record{'CP_SIZE_AT_OPEN'};
- }
- }
-
- # record largest size that the file reached at any rank
- if(!defined($hash_files{$hash}{'max_size'}) ||
- $hash_files{$hash}{'max_size'} <
- ($file_record{'CP_MAX_BYTE_READ'} + 1))
- {
- $hash_files{$hash}{'max_size'} =
- $file_record{'CP_MAX_BYTE_READ'} + 1;
- }
- if(!defined($hash_files{$hash}{'max_size'}) ||
- $hash_files{$hash}{'max_size'} <
- ($file_record{'CP_MAX_BYTE_WRITTEN'} + 1))
- {
- $hash_files{$hash}{'max_size'} =
- $file_record{'CP_MAX_BYTE_WRITTEN'} + 1;
- }
-
- # make sure there is an initial value for read and write flags
- if(!defined($hash_files{$hash}{'was_read'}))
- {
- $hash_files{$hash}{'was_read'} = 0;
- }
- if(!defined($hash_files{$hash}{'was_written'}))
- {
- $hash_files{$hash}{'was_written'} = 0;
- }
-
- if($file_record{'CP_INDEP_OPENS'} > 0 ||
- $file_record{'CP_COLL_OPENS'} > 0)
- {
- # mpi file
- if($file_record{'CP_INDEP_READS'} > 0 ||
- $file_record{'CP_COLL_READS'} > 0 ||
- $file_record{'CP_SPLIT_READS'} > 0 ||
- $file_record{'CP_NB_READS'} > 0)
- {
- # data was read from the file
- $hash_files{$hash}{'was_read'} = 1;
- }
- if($file_record{'CP_INDEP_WRITES'} > 0 ||
- $file_record{'CP_COLL_WRITES'} > 0 ||
- $file_record{'CP_SPLIT_WRITES'} > 0 ||
- $file_record{'CP_NB_WRITES'} > 0)
- {
- # data was written to the file
- $hash_files{$hash}{'was_written'} = 1;
- }
- }
- else
- {
- # posix file
- if($file_record{'CP_POSIX_READS'} > 0 ||
- $file_record{'CP_POSIX_FREADS'} > 0)
- {
- # data was read from the file
- $hash_files{$hash}{'was_read'} = 1;
- }
- if($file_record{'CP_POSIX_WRITES'} > 0 ||
- $file_record{'CP_POSIX_FWRITES'} > 0)
- {
- # data was written to the file
- $hash_files{$hash}{'was_written'} = 1;
- }
- }
-
- $hash_files{$hash}{'name'} = $file_record{CP_NAME_SUFFIX};
-
- if ($rank == -1)
- {
- $hash_files{$hash}{'procs'} = $nprocs;
- $hash_files{$hash}{'slowest_rank'} = $file_record{'CP_SLOWEST_RANK'};
- $hash_files{$hash}{'slowest_time'} = $file_record{'CP_F_SLOWEST_RANK_TIME'};
- $hash_files{$hash}{'slowest_bytes'} = $file_record{'CP_SLOWEST_RANK_BYTES'};
- $hash_files{$hash}{'fastest_rank'} = $file_record{'CP_FASTEST_RANK'};
- $hash_files{$hash}{'fastest_time'} = $file_record{'CP_F_FASTEST_RANK_TIME'};
- $hash_files{$hash}{'fastest_bytes'} = $file_record{'CP_FASTEST_RANK_BYTES'};
- $hash_files{$hash}{'variance_time'} = $file_record{'CP_F_VARIANCE_RANK_TIME'};
- $hash_files{$hash}{'variance_bytes'} = $file_record{'CP_F_VARIANCE_RANK_BYTES'};
- }
- else
- {
- my $total_time = $file_record{'CP_F_POSIX_META_TIME'} +
- $file_record{'CP_F_POSIX_READ_TIME'} +
- $file_record{'CP_F_POSIX_WRITE_TIME'};
-
- my $total_bytes = $file_record{'CP_BYTES_READ'} +
- $file_record{'CP_BYTES_WRITTEN'};
-
- if(!defined($hash_files{$hash}{'slowest_time'}) ||
- $hash_files{$hash}{'slowest_time'} < $total_time)
- {
- $hash_files{$hash}{'slowest_time'} = $total_time;
- $hash_files{$hash}{'slowest_rank'} = $rank;
- $hash_files{$hash}{'slowest_bytes'} = $total_bytes;
- }
-
- if(!defined($hash_files{$hash}{'fastest_time'}) ||
- $hash_files{$hash}{'fastest_time'} > $total_time)
- {
- $hash_files{$hash}{'fastest_time'} = $total_time;
- $hash_files{$hash}{'fastest_rank'} = $rank;
- $hash_files{$hash}{'fastest_bytes'} = $total_bytes;
- }
-
- if(!defined($hash_files{$hash}{'variance_time_S'}))
- {
- $hash_files{$hash}{'variance_time_S'} = 0;
- $hash_files{$hash}{'variance_time_T'} = $total_time;
- $hash_files{$hash}{'variance_time_n'} = 1;
- $hash_files{$hash}{'variance_bytes_S'} = 0;
- $hash_files{$hash}{'variance_bytes_T'} = $total_bytes;
- $hash_files{$hash}{'variance_bytes_n'} = 1;
- $hash_files{$hash}{'procs'} = 1;
- $hash_files{$hash}{'variance_time'} = 0;
- $hash_files{$hash}{'variance_bytes'} = 0;
- }
- else
- {
- my $n = $hash_files{$hash}{'variance_time_n'};
- my $m = 1;
- my $T = $hash_files{$hash}{'variance_time_T'};
- $hash_files{$hash}{'variance_time_S'} += ($m/($n*($n+$m)))*(($n/$m)*$total_time - $T)*(($n/$m)*$total_time - $T);
- $hash_files{$hash}{'variance_time_T'} += $total_time;
- $hash_files{$hash}{'variance_time_n'} += 1;
-
- $hash_files{$hash}{'variance_time'} = $hash_files{$hash}{'variance_time_S'} / $hash_files{$hash}{'variance_time_n'};
-
- $n = $hash_files{$hash}{'variance_bytes_n'};
- $m = 1;
- $T = $hash_files{$hash}{'variance_bytes_T'};
- $hash_files{$hash}{'variance_bytes_S'} += ($m/($n*($n+$m)))*(($n/$m)*$total_bytes - $T)*(($n/$m)*$total_bytes - $T);
- $hash_files{$hash}{'variance_bytes_T'} += $total_bytes;
- $hash_files{$hash}{'variance_bytes_n'} += 1;
-
- $hash_files{$hash}{'variance_bytes'} = $hash_files{$hash}{'variance_bytes_S'} / $hash_files{$hash}{'variance_bytes_n'};
-
- $hash_files{$hash}{'procs'} = $hash_files{$hash}{'variance_time_n'};
- }
- }
-
- # if this is a non-shared file, then add the time spent here to the
- # total for that particular rank
- if ($rank != -1)
- {
- # is it mpi-io or posix?
- if($file_record{CP_INDEP_OPENS} > 0 ||
- $file_record{CP_COLL_OPENS} > 0)
- {
- # add up mpi times
- if(defined($hash_unique_file_time{$rank}))
- {
- $hash_unique_file_time{$rank} +=
- $file_record{CP_F_MPI_META_TIME} +
- $file_record{CP_F_MPI_READ_TIME} +
- $file_record{CP_F_MPI_WRITE_TIME};
- }
- else
- {
- $hash_unique_file_time{$rank} =
- $file_record{CP_F_MPI_META_TIME} +
- $file_record{CP_F_MPI_READ_TIME} +
- $file_record{CP_F_MPI_WRITE_TIME};
- }
- }
- else
- {
- # add up posix times
- if(defined($hash_unique_file_time{$rank}))
- {
- $hash_unique_file_time{$rank} +=
- $file_record{CP_F_POSIX_META_TIME} +
- $file_record{CP_F_POSIX_READ_TIME} +
- $file_record{CP_F_POSIX_WRITE_TIME};
- }
- else
- {
- $hash_unique_file_time{$rank} =
- $file_record{CP_F_POSIX_META_TIME} +
- $file_record{CP_F_POSIX_READ_TIME} +
- $file_record{CP_F_POSIX_WRITE_TIME};
- }
- }
- }
- else
- {
-
- # cumulative time spent on shared files by slowest proc
- if($major > 1)
- {
- # new file format
- $shared_file_time += $file_record{'CP_F_SLOWEST_RANK_TIME'};
- }
- else
- {
- # old file format. Guess time spent as duration between first open
- # and last io
- if($file_record{'CP_F_READ_END_TIMESTAMP'} >
- $file_record{'CP_F_WRITE_END_TIMESTAMP'})
- {
- # be careful of files that were opened but not read or
- # written
- if($file_record{'CP_F_READ_END_TIMESTAMP'} > $file_record{'CP_F_OPEN_TIMESTAMP'}) {
- $shared_file_time += $file_record{'CP_F_READ_END_TIMESTAMP'} -
- $file_record{'CP_F_OPEN_TIMESTAMP'};
- }
- }
- else
- {
- if($file_record{'CP_F_WRITE_END_TIMESTAMP'} > $file_record{'CP_F_OPEN_TIMESTAMP'}) {
- $shared_file_time += $file_record{'CP_F_WRITE_END_TIMESTAMP'} -
- $file_record{'CP_F_OPEN_TIMESTAMP'};
- }
- }
- }
- }
-
- my $mpi_did_read =
- $file_record{'CP_INDEP_READS'} +
- $file_record{'CP_COLL_READS'} +
- $file_record{'CP_NB_READS'} +
- $file_record{'CP_SPLIT_READS'};
-
- # add up how many bytes were transferred
- if(($file_record{CP_INDEP_OPENS} > 0 ||
- $file_record{CP_COLL_OPENS} > 0) && (!($mpi_did_read)))
- {
- # mpi file that was only written; disregard any read accesses that
- # may have been performed for sieving at the posix level
- $total_job_bytes += $file_record{'CP_BYTES_WRITTEN'};
- }
- else
- {
- # normal case
- $total_job_bytes += $file_record{'CP_BYTES_WRITTEN'} +
- $file_record{'CP_BYTES_READ'};
- }
-
- # TODO
- # (detect mpi or posix and):
- # - sum meta time per rank for uniq files
- # - sum io time per rank for uniq files
- # - sum time from first open to last io for shared files
- # - sum meta time/nprocs for shared files
- # - sum io time/nprocs for shared files
-
- # TODO: ideas
- # graph time spent performing I/O per rank
- # for rank that spent the most time performing I/O:
- # - meta on ro files, meta on wo files, read time, write time
- # table with nfiles accessed, ro, wo, rw, created
-}
sub process_args
{
@@ -1408,9 +99,9 @@ sub check_prereqs
print("error: failed to execute $gnuplot.\n");
exit(1);
}
-
+
$output =~ /gnuplot (\d+)\.(\d+)/;
- if($1 < 4 || $2 < 2)
+ if($1 < 4 || ($1 < 5 && $2 < 2))
{
print("error: detected $gnuplot version $1.$2, but darshan-job-summary requires at least 4.2.\n");
exit(1);
@@ -1419,41 +110,6 @@ sub check_prereqs
return;
}
-#
-# Execute which to see if the binary can be found in
-# the users path.
-#
-sub checkbin($)
-{
- my $binname = shift;
- my $rc;
-
- # save stdout/err
- open(SAVEOUT, ">&STDOUT");
- open(SAVEERR, ">&STDERR");
-
- # redirect stdout/error
- open(STDERR, '>/dev/null');
- open(STDOUT, '>/dev/null');
- $rc = system("which $binname");
- if ($rc)
- {
- $rc = 1;
- }
- close(STDOUT);
- close(STDERR);
-
- # suppress perl warning
- select(SAVEERR);
- select(SAVEOUT);
-
- # restore stdout/err
- open(STDOUT, ">&SAVEOUT");
- open(STDERR, ">&SAVEERR");
-
- return $rc;
-}
-
sub print_help
{
print <<EOF;
diff --git a/darshan-util/darshan-load-mysql.c b/darshan-util/darshan-load-mysql.c
deleted file mode 100644
index 8f9c2d8..0000000
--- a/darshan-util/darshan-load-mysql.c
+++ /dev/null
@@ -1,509 +0,0 @@
-#include <stdlib.h>
-#include <stdio.h>
-#include <string.h>
-#include <ftw.h>
-#include <libgen.h>
-#include <mysql.h>
-#include <regex.h>
-#include <sys/types.h>
-#include <getopt.h>
-#include <readline/readline.h>
-
-#include "darshan-logutils.h"
-
-#define MAXSQL (1024*1024)
-#define STOPWALK (1)
-#define CONTWALK (0)
-
-#define OPT_HOST (1)
-#define OPT_USER (2)
-#define OPT_PASS (3)
-#define OPT_DB (4)
-#define OPT_PATH (5)
-
-const char *insert_job_fmt = "insert into %s values('%d','%s','%s','%s',\
-'%d','%ld','%ld','%d')";
-const char *insert_mnt_fmt = "insert into %s values('%d','%d','%" PRId64 "','%s','%s')";
-const char *insert_file_fmt = "insert into %s values('%d','%ld','%" PRId64 "','%d',\
-'%s',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "','%" PRId64 "',\
-'%.16lf','%.16lf','%.16lf','%.16lf','%.16lf',\
-'%.16lf','%.16lf','%.16lf','%.16lf','%.16lf',\
-'%.16lf','%.16lf','%.16lf','%.16lf')";
-
-MYSQL *mysql = NULL;
-int debug = 0;
-
-int tree_walk (const char *fpath, const struct stat *sb, int typeflag)
-{
- struct darshan_file file;
- struct darshan_job job;
- darshan_fd dfile = NULL;
- int ret;
- int rc;
- int nofiles;
- char exe[4096];
- char *base;
- char *dash;
- char *username;
- char *jobid;
- char *sqlstmt = NULL;
- int count;
- int i;
- int64_t *devs;
- char **mnts;
- char **fstypes;
- regex_t regex;
- regmatch_t match[1];
- char *filepath = NULL;
- int prev_rank;
-
- rc = CONTWALK;
- count = 0;
-
- /* Only Process Files */
- if (typeflag != FTW_F) return CONTWALK;
-
- sqlstmt = malloc(MAXSQL);
- if (!sqlstmt)
- {
- perror("malloc");
- rc = STOPWALK;
- goto exit;
- }
-
- filepath = strdup(fpath);
- if (!filepath)
- {
- perror("strdup");
- rc = STOPWALK;
- goto exit;
- }
-
- /* Process Log Files */
- dfile = darshan_log_open(fpath, "r");
- if (dfile == NULL)
- {
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", fpath);
- rc = CONTWALK;
- goto exit;
- }
-
- ret = darshan_log_getjob(dfile, &job);
- if (ret < 0)
- {
- perror("darshan_log_getjob");
- fprintf(stderr, "%s\n", fpath);
- rc = CONTWALK;
- goto exit;
- }
-
- memset(exe, 0, sizeof(exe));
-
- ret = darshan_log_getexe(dfile, exe, &nofiles);
- if (ret < 0)
- {
- perror("darshan_log_getexe");
- fprintf(stderr, "%s\n", fpath);
- rc = CONTWALK;
- goto exit;
- }
-
- base = basename(filepath);
- username = base;
- dash = index(base, '_');
- *dash = '\0';
- jobid = dash+1;
-
- /*
- * Find jobid for log file name
- */
- ret = regcomp(®ex, "_id[[:digit:]]+_", REG_EXTENDED);
- if (ret != 0)
- {
- char buf[256];
- regerror(ret, ®ex, buf, sizeof(buf));
- fprintf(stderr, "regcomp: %s\n", buf);
- rc = STOPWALK;
- goto exit;
- }
-
- ret = regexec(®ex, jobid, 1, match, 0);
- if (ret != 0)
- {
- char buf[256];
- regerror(ret, ®ex, buf, sizeof(buf));
- fprintf(stderr, "regexec: %s\n", buf);
- rc = STOPWALK;
- goto exit;
- }
-
- regfree(®ex);
-
- dash = jobid;
- jobid += (match[0].rm_so + 3);
- dash += (match[0].rm_eo - 1);
- *dash = 0;
-
- /*
- * Insert Job Record
- */
- snprintf(sqlstmt, MAXSQL, insert_job_fmt, "darshan_job_intrepid",
- atoi(jobid), username, job.version_string, exe, job.uid,
- job.start_time, job.end_time, job.nprocs);
-
- if (debug) printf("sql: %s\n", sqlstmt);
- ret = mysql_query(mysql, sqlstmt);
- if (ret)
- {
- fprintf(stderr, "log not processed: %s [mysql: %d (%s)] : \
-jobid=%d start_time=%ld\n",
- fpath, mysql_errno(mysql), mysql_error(mysql),
- atoi(jobid), job.start_time);
- rc = CONTWALK;
- goto exit;
- }
-
- /*
- * Insert MountPoint Record (if present)
- */
- ret = darshan_log_getmounts(dfile,&devs,&mnts,&fstypes,&count,&nofiles);
- if (ret < 0)
- {
- perror("darshan_log_getmounts");
- fprintf(stderr, "%s\n", fpath);
- rc = STOPWALK;
- goto exit;
- }
-
- for (i=0; (i<count); i++)
- {
- snprintf(sqlstmt,MAXSQL,insert_mnt_fmt, "darshan_mountpoints_intrepid",
- atoi(jobid), job.start_time, devs[i], mnts[i], fstypes[i]);
-
- if (debug) printf("sql: %s\n", sqlstmt);
- ret = mysql_query(mysql, sqlstmt);
- if (ret)
- {
- fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),
- mysql_error(mysql));
- rc = STOPWALK;
- goto exit;
- }
- }
-
- /*
- * Insert File Records (if present)
- */
- if (!nofiles)
- {
- while ((ret = darshan_log_getfile(dfile, &job, &file)) == 1)
- {
- /*
- * Work around issue where bogus file data was in older logs.
- * Bogus data was files taht were 'stat'd but not opened.
- */
- if (file.rank != -1 && file.rank < prev_rank)
- {
- continue;
- }
- if (file.rank != -1)
- prev_rank = file.rank;
-
- snprintf(sqlstmt, MAXSQL, insert_file_fmt, "darshan_file_intrepid",
- atoi(jobid), job.start_time, file.hash, file.rank, file.name_suffix,
- file.counters[CP_INDEP_OPENS],
- file.counters[CP_COLL_OPENS],
- file.counters[CP_INDEP_READS],
- file.counters[CP_INDEP_WRITES],
- file.counters[CP_COLL_READS],
- file.counters[CP_COLL_WRITES],
- file.counters[CP_SPLIT_READS],
- file.counters[CP_SPLIT_WRITES],
- file.counters[CP_NB_READS],
- file.counters[CP_NB_WRITES],
- file.counters[CP_SYNCS],
- file.counters[CP_POSIX_READS],
- file.counters[CP_POSIX_WRITES],
- file.counters[CP_POSIX_OPENS],
- file.counters[CP_POSIX_SEEKS],
- file.counters[CP_POSIX_STATS],
- file.counters[CP_POSIX_MMAPS],
- file.counters[CP_POSIX_FREADS],
- file.counters[CP_POSIX_FWRITES],
- file.counters[CP_POSIX_FOPENS],
- file.counters[CP_POSIX_FSEEKS],
- file.counters[CP_POSIX_FSYNCS],
- file.counters[CP_POSIX_FDSYNCS],
- file.counters[CP_INDEP_NC_OPENS],
- file.counters[CP_COLL_NC_OPENS],
- file.counters[CP_HDF5_OPENS],
- file.counters[CP_COMBINER_NAMED],
- file.counters[CP_COMBINER_DUP],
- file.counters[CP_COMBINER_CONTIGUOUS],
- file.counters[CP_COMBINER_VECTOR],
- file.counters[CP_COMBINER_HVECTOR_INTEGER],
- file.counters[CP_COMBINER_HVECTOR],
- file.counters[CP_COMBINER_INDEXED],
- file.counters[CP_COMBINER_HINDEXED_INTEGER],
- file.counters[CP_COMBINER_HINDEXED],
- file.counters[CP_COMBINER_INDEXED_BLOCK],
- file.counters[CP_COMBINER_STRUCT_INTEGER],
- file.counters[CP_COMBINER_STRUCT],
- file.counters[CP_COMBINER_SUBARRAY],
- file.counters[CP_COMBINER_DARRAY],
- file.counters[CP_COMBINER_F90_REAL],
- file.counters[CP_COMBINER_F90_COMPLEX],
- file.counters[CP_COMBINER_F90_INTEGER],
- file.counters[CP_COMBINER_RESIZED],
- file.counters[CP_HINTS],
- file.counters[CP_VIEWS],
- file.counters[CP_MODE],
- file.counters[CP_BYTES_READ],
- file.counters[CP_BYTES_WRITTEN],
- file.counters[CP_MAX_BYTE_READ],
- file.counters[CP_MAX_BYTE_WRITTEN],
- file.counters[CP_CONSEC_READS],
- file.counters[CP_CONSEC_WRITES],
- file.counters[CP_SEQ_READS],
- file.counters[CP_SEQ_WRITES],
- file.counters[CP_RW_SWITCHES],
- file.counters[CP_MEM_NOT_ALIGNED],
- file.counters[CP_MEM_ALIGNMENT],
- file.counters[CP_FILE_NOT_ALIGNED],
- file.counters[CP_FILE_ALIGNMENT],
- file.counters[CP_MAX_READ_TIME_SIZE],
- file.counters[CP_MAX_WRITE_TIME_SIZE],
- file.counters[CP_SIZE_READ_0_100],
- file.counters[CP_SIZE_READ_100_1K],
- file.counters[CP_SIZE_READ_1K_10K],
- file.counters[CP_SIZE_READ_10K_100K],
- file.counters[CP_SIZE_READ_100K_1M],
- file.counters[CP_SIZE_READ_1M_4M],
- file.counters[CP_SIZE_READ_4M_10M],
- file.counters[CP_SIZE_READ_10M_100M],
- file.counters[CP_SIZE_READ_100M_1G],
- file.counters[CP_SIZE_READ_1G_PLUS],
- file.counters[CP_SIZE_WRITE_0_100],
- file.counters[CP_SIZE_WRITE_100_1K],
- file.counters[CP_SIZE_WRITE_1K_10K],
- file.counters[CP_SIZE_WRITE_10K_100K],
- file.counters[CP_SIZE_WRITE_100K_1M],
- file.counters[CP_SIZE_WRITE_1M_4M],
- file.counters[CP_SIZE_WRITE_4M_10M],
- file.counters[CP_SIZE_WRITE_10M_100M],
- file.counters[CP_SIZE_WRITE_100M_1G],
- file.counters[CP_SIZE_WRITE_1G_PLUS],
- file.counters[CP_SIZE_READ_AGG_0_100],
- file.counters[CP_SIZE_READ_AGG_100_1K],
- file.counters[CP_SIZE_READ_AGG_1K_10K],
- file.counters[CP_SIZE_READ_AGG_10K_100K],
- file.counters[CP_SIZE_READ_AGG_100K_1M],
- file.counters[CP_SIZE_READ_AGG_1M_4M],
- file.counters[CP_SIZE_READ_AGG_4M_10M],
- file.counters[CP_SIZE_READ_AGG_10M_100M],
- file.counters[CP_SIZE_READ_AGG_100M_1G],
- file.counters[CP_SIZE_READ_AGG_1G_PLUS],
- file.counters[CP_SIZE_WRITE_AGG_0_100],
- file.counters[CP_SIZE_WRITE_AGG_100_1K],
- file.counters[CP_SIZE_WRITE_AGG_1K_10K],
- file.counters[CP_SIZE_WRITE_AGG_10K_100K],
- file.counters[CP_SIZE_WRITE_AGG_100K_1M],
- file.counters[CP_SIZE_WRITE_AGG_1M_4M],
- file.counters[CP_SIZE_WRITE_AGG_4M_10M],
- file.counters[CP_SIZE_WRITE_AGG_10M_100M],
- file.counters[CP_SIZE_WRITE_AGG_100M_1G],
- file.counters[CP_SIZE_WRITE_AGG_1G_PLUS],
- file.counters[CP_EXTENT_READ_0_100],
- file.counters[CP_EXTENT_READ_100_1K],
- file.counters[CP_EXTENT_READ_1K_10K],
- file.counters[CP_EXTENT_READ_10K_100K],
- file.counters[CP_EXTENT_READ_100K_1M],
- file.counters[CP_EXTENT_READ_1M_4M],
- file.counters[CP_EXTENT_READ_4M_10M],
- file.counters[CP_EXTENT_READ_10M_100M],
- file.counters[CP_EXTENT_READ_100M_1G],
- file.counters[CP_EXTENT_READ_1G_PLUS],
- file.counters[CP_EXTENT_WRITE_0_100],
- file.counters[CP_EXTENT_WRITE_100_1K],
- file.counters[CP_EXTENT_WRITE_1K_10K],
- file.counters[CP_EXTENT_WRITE_10K_100K],
- file.counters[CP_EXTENT_WRITE_100K_1M],
- file.counters[CP_EXTENT_WRITE_1M_4M],
- file.counters[CP_EXTENT_WRITE_4M_10M],
- file.counters[CP_EXTENT_WRITE_10M_100M],
- file.counters[CP_EXTENT_WRITE_100M_1G],
- file.counters[CP_EXTENT_WRITE_1G_PLUS],
- file.counters[CP_STRIDE1_STRIDE],
- file.counters[CP_STRIDE2_STRIDE],
- file.counters[CP_STRIDE3_STRIDE],
- file.counters[CP_STRIDE4_STRIDE],
- file.counters[CP_STRIDE1_COUNT],
- file.counters[CP_STRIDE2_COUNT],
- file.counters[CP_STRIDE3_COUNT],
- file.counters[CP_STRIDE4_COUNT],
- file.counters[CP_ACCESS1_ACCESS],
- file.counters[CP_ACCESS2_ACCESS],
- file.counters[CP_ACCESS3_ACCESS],
- file.counters[CP_ACCESS4_ACCESS],
- file.counters[CP_ACCESS1_COUNT],
- file.counters[CP_ACCESS2_COUNT],
- file.counters[CP_ACCESS3_COUNT],
- file.counters[CP_ACCESS4_COUNT],
- file.counters[CP_DEVICE],
- file.counters[CP_SIZE_AT_OPEN],
- file.fcounters[CP_F_OPEN_TIMESTAMP],
- file.fcounters[CP_F_READ_START_TIMESTAMP],
- file.fcounters[CP_F_WRITE_START_TIMESTAMP],
- file.fcounters[CP_F_CLOSE_TIMESTAMP],
- file.fcounters[CP_F_READ_END_TIMESTAMP],
- file.fcounters[CP_F_WRITE_END_TIMESTAMP],
- file.fcounters[CP_F_POSIX_READ_TIME],
- file.fcounters[CP_F_POSIX_WRITE_TIME],
- file.fcounters[CP_F_POSIX_META_TIME],
- file.fcounters[CP_F_MPI_META_TIME],
- file.fcounters[CP_F_MPI_READ_TIME],
- file.fcounters[CP_F_MPI_WRITE_TIME],
- file.fcounters[CP_F_MAX_READ_TIME],
- file.fcounters[CP_F_MAX_WRITE_TIME]);
-
- if (debug) printf("sql: %s\n", sqlstmt);
- ret = mysql_query(mysql, sqlstmt);
- if (ret)
- {
- fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),
- mysql_error(mysql));
- rc = STOPWALK;
- goto exit;
- }
- }
- }
-
-exit:
- if (dfile) darshan_log_close(dfile);
-
- if (count > 0)
- {
- for(i=0; i<count; i++)
- {
- if (mnts[i]) free(mnts[i]);
- if (fstypes[i]) free(fstypes[i]);
- }
- if (devs) free(devs);
- if (mnts) free(mnts);
- if (fstypes) free(fstypes);
- }
-
- if (sqlstmt)
- {
- free(sqlstmt);
- }
-
- if (filepath)
- {
- free(filepath);
- }
-
- return rc;
-}
-
-int main (int argc, char **argv)
-{
- char base[256] = "";
- char host[256] = "";
- char user[256] = "";
- char pass[256] = "";
- char db[256] = "";
- int ret = 0;
-
- while(1)
- {
- static struct option options[] = {
- {"host", 1, NULL, OPT_HOST},
- {"user", 1, NULL, OPT_USER},
- {"pass", 1, NULL, OPT_PASS},
- {"db", 1, NULL, OPT_DB},
- {"path", 1, NULL, OPT_PATH},
- {NULL, 0, NULL, 0}
- };
- int o = getopt_long(argc, argv, "", options, NULL);
-
- if (o == -1) break;
-
- switch(o)
- {
- case OPT_HOST:
- strncpy(host, optarg, sizeof(host));
- break;
- case OPT_USER:
- strncpy(user, optarg, sizeof(user));
- break;
- case OPT_PASS:
- strncpy(pass, optarg, sizeof(pass));
- break;
- case OPT_DB:
- strncpy(db, optarg, sizeof(db));
- break;
- case OPT_PATH:
- strncpy(base, optarg, sizeof(base));
- break;
- }
- }
-
- if (strcmp(pass, "") == 0)
- {
- char *line = readline(NULL);
- if (line)
- {
- strncpy(pass, line, sizeof(pass));
- }
- }
-
- mysql = mysql_init(NULL);
- if (mysql == NULL)
- {
- fprintf(stderr, "mysql_init failed");
- exit(-1);
- }
-
- mysql = mysql_real_connect(mysql, host, user, pass, db, 0, NULL, 0);
- if (mysql == NULL)
- {
- fprintf(stderr, "mysql_real_connect");
- exit(-1);
- }
-
- /* Turn off auto commits, hopefuly for performance sake */
- (void) mysql_autocommit(mysql, 0);
-
- ret = ftw(base, tree_walk, 512);
- if(ret != 0)
- {
- fprintf(stderr, "Error: failed to walk path: %s\n", base);
- return(-1);
- }
-
- ret = mysql_commit(mysql);
- if (ret)
- {
- fprintf(stderr, "mysql: %d (%s)\n", mysql_errno(mysql),
- mysql_error(mysql));
- }
-
- mysql_close(mysql);
-
- return 0;
-}
diff --git a/darshan-util/darshan-log-params.c b/darshan-util/darshan-log-params.c
deleted file mode 100644
index 22645e5..0000000
--- a/darshan-util/darshan-log-params.c
+++ /dev/null
@@ -1,14 +0,0 @@
-#include <stdio.h>
-#include "darshan-log-format.h"
-
-int main(int argc, char **argv)
-{
- printf("version: %s\n", CP_VERSION);
- printf("CP_JOB_RECORD_SIZE: %d\n", CP_JOB_RECORD_SIZE);
- printf("CP_EXE_LEN: %zu\n", CP_EXE_LEN);
- printf("CP_FILE_RECORD_SIZE: %zu\n", CP_FILE_RECORD_SIZE);
- printf("CP_NAME_SUFFIX_LEN: %d\n", CP_NAME_SUFFIX_LEN);
- printf("CP_NUM_INDICES: %d\n", CP_NUM_INDICES);
- printf("CP_F_NUM_INDICES: %d\n", CP_F_NUM_INDICES);
- return(0);
-}
diff --git a/darshan-util/darshan-logutils.c b/darshan-util/darshan-logutils.c
index a94eda3..3a4327e 100644
--- a/darshan-util/darshan-logutils.c
+++ b/darshan-util/darshan-logutils.c
@@ -22,39 +22,95 @@
static int darshan_log_seek(darshan_fd fd, off_t offset);
static int darshan_log_read(darshan_fd fd, void *buf, int len);
static int darshan_log_write(darshan_fd fd, void *buf, int len);
+static int darshan_decompress_buf(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz,
+ enum darshan_comp_type comp_type);
+static int darshan_compress_buf(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz,
+ enum darshan_comp_type comp_type);
+static int darshan_zlib_decomp(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz);
+static int darshan_zlib_comp(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz);
+#ifdef HAVE_LIBBZ2
+static int darshan_bzip2_decomp(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz);
+static int darshan_bzip2_comp(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz);
+#endif
+
+
+/* TODO: can we make this s.t. we don't care about ordering (i.e., X macro it ) */
+struct darshan_mod_logutil_funcs *mod_logutils[DARSHAN_MAX_MODS] =
+{
+ NULL, /* NULL */
+ &posix_logutils, /* POSIX */
+ &mpiio_logutils, /* MPI-IO */
+ &hdf5_logutils, /* HDF5 */
+ &pnetcdf_logutils, /* PNETCDF */
+ &bgq_logutils, /* BG/Q */
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+ NULL
+};
/* darshan_log_open()
*
- * open a darshan log file for reading/writing
+ * open an existing darshan log file for reading only
*
* returns 0 on success, -1 on failure
*/
-darshan_fd darshan_log_open(const char *name, const char *mode)
+darshan_fd darshan_log_open(const char *name)
{
darshan_fd tmp_fd;
- /* we only allows "w" or "r" modes, nothing fancy */
- assert(strlen(mode) == 1);
- assert(mode[0] == 'r' || mode[0] == 'w');
-
tmp_fd = malloc(sizeof(*tmp_fd));
if(!tmp_fd)
return(NULL);
memset(tmp_fd, 0, sizeof(*tmp_fd));
- if(strcmp(mode, "r") == 0)
- {
- /* TODO: check for bz2 */
- }
-
- if(strcmp(mode, "w") == 0)
+ tmp_fd->fildes = open(name, O_RDONLY);
+ if(tmp_fd->fildes < 0)
{
- /* TODO: check for bz2 */
+ perror("darshan_log_open: ");
+ free(tmp_fd);
+ tmp_fd = NULL;
}
- tmp_fd->gzf = gzopen(name, mode);
- if(!tmp_fd->gzf)
+ return(tmp_fd);
+}
+
+/* darshan_log_create()
+ *
+ * create a darshan log file for writing with the given compression method
+ *
+ * returns 0 on success, -1 on failure
+ */
+darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type)
+{
+ darshan_fd tmp_fd;
+
+ tmp_fd = malloc(sizeof(*tmp_fd));
+ if(!tmp_fd)
+ return(NULL);
+ memset(tmp_fd, 0, sizeof(*tmp_fd));
+
+ /* TODO: permissions when creating? umask */
+ /* when writing, we create the log file making sure not to overwrite
+ * an existing log
+ */
+ tmp_fd->comp_type = comp_type;
+ tmp_fd->fildes = open(name, O_WRONLY | O_CREAT | O_EXCL, 0400);
+ if(tmp_fd->fildes < 0)
{
+ perror("darshan_log_create: ");
free(tmp_fd);
tmp_fd = NULL;
}
@@ -66,6 +122,7 @@ darshan_fd darshan_log_open(const char *name, const char *mode)
*
* read the header of the darshan log and set internal data structures
* NOTE: this function must be called before reading other portions of the log
+ * NOTE: this is the only portion of the darshan log which is uncompressed
*
* returns 0 on success, -1 on failure
*/
@@ -81,11 +138,11 @@ int darshan_log_getheader(darshan_fd fd, struct darshan_header *header)
return(-1);
}
- /* read header from log file */
+ /* read uncompressed header from log file */
ret = darshan_log_read(fd, header, sizeof(*header));
- if(ret < sizeof(*header))
+ if(ret != sizeof(*header))
{
- fprintf(stderr, "Error: invalid darshan log file (failed to read header).\n");
+ fprintf(stderr, "Error: failed to read darshan log file header.\n");
return(-1);
}
@@ -122,13 +179,57 @@ int darshan_log_getheader(darshan_fd fd, struct darshan_header *header)
}
}
+ fd->comp_type = header->comp_type;
+
/* save the mapping of data within log file to this file descriptor */
+ fd->job_map.off = sizeof(struct darshan_header);
+ fd->job_map.len = header->rec_map.off - fd->job_map.off;
memcpy(&fd->rec_map, &header->rec_map, sizeof(struct darshan_log_map));
memcpy(&fd->mod_map, &header->mod_map, DARSHAN_MAX_MODS * sizeof(struct darshan_log_map));
return(0);
}
+/* darshan_log_putheader()
+ *
+ * write a darshan header to log file
+ * NOTE: the header is not passed in, and is instead built using
+ * contents of the given file descriptor
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_putheader(darshan_fd fd)
+{
+ struct darshan_header header;
+ int ret;
+
+ ret = darshan_log_seek(fd, 0);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to seek in darshan log file.\n");
+ return(-1);
+ }
+
+ memset(&header, 0, sizeof(header));
+ strcpy(header.version_string, DARSHAN_LOG_VERSION);
+ header.magic_nr = DARSHAN_MAGIC_NR;
+ header.comp_type = fd->comp_type;
+
+ /* copy the mapping information to the header */
+ memcpy(&header.rec_map, &fd->rec_map, sizeof(struct darshan_log_map));
+ memcpy(&header.mod_map, &fd->mod_map, DARSHAN_MAX_MODS * sizeof(struct darshan_log_map));
+
+ /* write header to file */
+ ret = darshan_log_write(fd, &header, sizeof(header));
+ if(ret != sizeof(header))
+ {
+ fprintf(stderr, "Error: failed to write Darshan log file header.\n");
+ return(-1);
+ }
+
+ return(0);
+}
+
/* darshan_log_getjob()
*
* read job level metadata from the darshan log file
@@ -137,22 +238,47 @@ int darshan_log_getheader(darshan_fd fd, struct darshan_header *header)
*/
int darshan_log_getjob(darshan_fd fd, struct darshan_job *job)
{
+ char *comp_buf;
+ char job_buf[DARSHAN_JOB_RECORD_SIZE] = {0};
+ int job_buf_sz = DARSHAN_JOB_RECORD_SIZE;
int ret;
- ret = darshan_log_seek(fd, sizeof(struct darshan_header));
+ assert(fd->job_map.len > 0 && fd->job_map.off > 0);
+
+ /* allocate buffer to store compressed job info */
+ comp_buf = malloc(fd->job_map.len);
+ if(!comp_buf)
+ return(-1);
+
+ ret = darshan_log_seek(fd, fd->job_map.off);
if(ret < 0)
{
fprintf(stderr, "Error: unable to seek in darshan log file.\n");
+ free(comp_buf);
return(-1);
}
- /* read the job data from the log file */
- ret = darshan_log_read(fd, job, sizeof(*job));
- if(ret < sizeof(*job))
+ /* read the compressed job data from the log file */
+ ret = darshan_log_read(fd, comp_buf, fd->job_map.len);
+ if(ret != fd->job_map.len)
+ {
+ fprintf(stderr, "Error: failed to read darshan log file job data.\n");
+ free(comp_buf);
+ return(-1);
+ }
+
+ /* decompress the job data */
+ ret = darshan_decompress_buf(comp_buf, fd->job_map.len,
+ job_buf, &job_buf_sz, fd->comp_type);
+ if(ret < 0)
{
- fprintf(stderr, "Error: invalid darshan log file (failed to read job data).\n");
+ fprintf(stderr, "Error: failed to decompress darshan job data.\n");
+ free(comp_buf);
return(-1);
}
+ free(comp_buf);
+
+ memcpy(job, job_buf, sizeof(*job));
if(fd->swap_flag)
{
@@ -164,71 +290,174 @@ int darshan_log_getjob(darshan_fd fd, struct darshan_job *job)
DARSHAN_BSWAP64(&job->jobid);
}
+ /* save trailing exe & mount information, so it can be retrieved later */
+ if(!fd->exe_mnt_data)
+ fd->exe_mnt_data = malloc(DARSHAN_EXE_LEN+1);
+ if(!fd->exe_mnt_data)
+ return(-1);
+ memcpy(fd->exe_mnt_data, &job_buf[sizeof(*job)], DARSHAN_EXE_LEN+1);
+
return(0);
}
-int darshan_log_getexe(darshan_fd fd, char *buf)
+/* darshan_log_putjob()
+ *
+ * write job level metadat to darshan log file
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_putjob(darshan_fd fd, struct darshan_job *job)
{
- int tmp_off = sizeof(struct darshan_header) + sizeof(struct darshan_job);
+ struct darshan_job job_copy;
+ char *comp_buf;
+ int comp_buf_sz;
+ int len;
int ret;
- char *newline;
- ret = darshan_log_seek(fd, tmp_off);
+ ret = darshan_log_seek(fd, sizeof(struct darshan_header));
if(ret < 0)
{
fprintf(stderr, "Error: unable to seek in darshan log file.\n");
return(-1);
}
- /* read the trailing exe data from the darshan log */
- ret = darshan_log_read(fd, buf, DARSHAN_EXE_LEN+1);
- if(ret < DARSHAN_EXE_LEN+1)
+ memset(&job_copy, 0, sizeof(*job));
+ memcpy(&job_copy, job, sizeof(*job));
+
+ /* check for newline in existing metadata, add if needed */
+ len = strlen(job_copy.metadata);
+ if(len > 0 && len < DARSHAN_JOB_METADATA_LEN)
+ {
+ if(job_copy.metadata[len-1] != '\n')
+ {
+ job_copy.metadata[len] = '\n';
+ job_copy.metadata[len+1] = '\0';
+ }
+ }
+
+ comp_buf = malloc(sizeof(*job));
+ if(!comp_buf)
+ return(-1);
+ comp_buf_sz = sizeof(*job);
+
+ /* compress the job data */
+ ret = darshan_compress_buf((char*)&job_copy, sizeof(*job),
+ comp_buf, &comp_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: failed to decompress darshan job data.\n");
+ free(comp_buf);
+ return(-1);
+ }
+
+ /* write job data to file */
+ ret = darshan_log_write(fd, comp_buf, comp_buf_sz);
+ if(ret != comp_buf_sz)
{
- fprintf(stderr, "Error: invalid darshan log file (failed to read exe string).\n");
+ fprintf(stderr, "Error: failed to write darshan log file job data.\n");
+ free(comp_buf);
return(-1);
}
- /* mount info is stored after the exe string, so truncate there */
- newline = strchr(buf, '\n');
+ free(comp_buf);
+ return(0);
+}
+
+/* darshan_log_getexe()
+ *
+ * reads the application exe name from darshan log file
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_getexe(darshan_fd fd, char *buf)
+{
+ char *newline;
+ int ret;
+
+ /* if the exe/mount data has not been saved yet, read in the job info */
+ if(!fd->exe_mnt_data)
+ {
+ struct darshan_job job;
+ ret = darshan_log_getjob(fd, &job);
+
+ if(ret < 0 || !fd->exe_mnt_data)
+ return(-1);
+ }
+
+ /* exe string is located before the first line break */
+ newline = strchr(fd->exe_mnt_data, '\n');
+
+ /* copy over the exe string */
if(newline)
- *newline = '\0';
+ memcpy(buf, fd->exe_mnt_data, (newline - fd->exe_mnt_data));
return (0);
}
+/* darshan_log_putexe()
+ *
+ * wrties the application exe name to darshan log file
+ * NOTE: this needs to be called immediately following put_job as it
+ * expects the final pointer to be positioned immediately following
+ * the darshan job information
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_putexe(darshan_fd fd, char *buf)
+{
+ int len;
+ int ret;
+ char comp_buf[DARSHAN_EXE_LEN] = {0};
+ int comp_buf_sz = DARSHAN_EXE_LEN;
+
+ len = strlen(buf);
+
+ /* compress the input exe string */
+ ret = darshan_compress_buf(buf, len, comp_buf, &comp_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to compress exe string.\n");
+ return(-1);
+ }
+
+ ret = darshan_log_write(fd, comp_buf, comp_buf_sz);
+ if(ret != comp_buf_sz)
+ {
+ fprintf(stderr, "Error: failed to write exe string to darshan log file.\n");
+ return(-1);
+ }
+
+ return(0);
+}
+
/* darshan_log_getmounts()
*
- * retrieves mount table information from the log. Note that mnt_pts and
+ * retrieves mount table information from the log. Note that mnt_pts and
* fs_types are arrays that will be allocated by the function and must be
- * freed by the caller. count will indicate the size of the arrays
+ * freed by the caller. count will indicate the size of the arrays
+ *
+ * returns 0 on success, -1 on failure
*/
int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
char*** fs_types, int* count)
{
- int tmp_off = sizeof(struct darshan_header) + sizeof(struct darshan_job);
- int ret;
char *pos;
int array_index = 0;
- char buf[DARSHAN_EXE_LEN+1];
+ int ret;
- ret = darshan_log_seek(fd, tmp_off);
- if(ret < 0)
+ /* if the exe/mount data has not been saved yet, read in the job info */
+ if(!fd->exe_mnt_data)
{
- fprintf(stderr, "Error: unable to seek in darshan log file.\n");
- return(-1);
- }
+ struct darshan_job job;
+ ret = darshan_log_getjob(fd, &job);
- /* read the trailing mount data from the darshan log */
- ret = darshan_log_read(fd, buf, DARSHAN_EXE_LEN+1);
- if(ret < DARSHAN_EXE_LEN+1)
- {
- fprintf(stderr, "Error: invalid darshan log file (failed to read mount info).\n");
- return(-1);
+ if(ret < 0 || !fd->exe_mnt_data)
+ return(-1);
}
/* count entries */
*count = 0;
- pos = buf;
+ pos = fd->exe_mnt_data;
while((pos = strchr(pos, '\n')) != NULL)
{
pos++;
@@ -250,7 +479,7 @@ int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
/* work backwards through the table and parse each line (except for
* first, which holds command line information)
*/
- while((pos = strrchr(buf, '\n')) != NULL)
+ while((pos = strrchr(fd->exe_mnt_data, '\n')) != NULL)
{
/* overestimate string lengths */
(*mnt_pts)[array_index] = malloc(DARSHAN_EXE_LEN);
@@ -262,7 +491,7 @@ int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
(*mnt_pts)[array_index]);
if(ret != 2)
{
- fprintf(stderr, "Error: poorly formatted mount table in log file.\n");
+ fprintf(stderr, "Error: poorly formatted mount table in darshan log file.\n");
return(-1);
}
pos--;
@@ -273,6 +502,54 @@ int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
return(0);
}
+/* darshan_log_putmounts()
+ *
+ * writes mount information to the darshan log file
+ * NOTE: this function call should follow immediately after the call
+ * to darshan_log_putexe(), as it assumes the darshan log file pointer
+ * is pointing to the offset immediately following the exe string
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_putmounts(darshan_fd fd, char** mnt_pts, char** fs_types, int count)
+{
+ int i;
+ char line[1024];
+ char mnt_dat[DARSHAN_EXE_LEN] = {0};
+ int mnt_dat_sz = 0;
+ char comp_buf[DARSHAN_EXE_LEN] = {0};
+ int comp_buf_sz = DARSHAN_EXE_LEN;
+ char *tmp;
+ int ret;
+
+ /* write each mount entry to file */
+ tmp = mnt_dat;
+ for(i=count-1; i>=0; i--)
+ {
+ sprintf(line, "\n%s\t%s", fs_types[i], mnt_pts[i]);
+
+ memcpy(tmp, line, strlen(line));
+ tmp += strlen(line);
+ mnt_dat_sz += strlen(line);
+ }
+
+ ret = darshan_compress_buf(mnt_dat, mnt_dat_sz, comp_buf, &comp_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to compress mount data.\n");
+ return(-1);
+ }
+
+ ret = darshan_log_write(fd, comp_buf, comp_buf_sz);
+ if (ret != comp_buf_sz)
+ {
+ fprintf(stderr, "Error: failed to write darshan log mount data.\n");
+ return(-1);
+ }
+
+ return(0);
+}
+
/* darshan_log_gethash()
*
* read the hash of records from the darshan log file
@@ -281,8 +558,9 @@ int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
*/
int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)
{
+ char *comp_buf;
char *hash_buf;
- int hash_buf_sz = fd->rec_map.len;
+ int hash_buf_sz;
char *buf_ptr;
darshan_record_id *rec_id_ptr;
uint32_t *path_len_ptr;
@@ -290,24 +568,46 @@ int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)
struct darshan_record_ref *ref;
int ret;
- hash_buf = malloc(hash_buf_sz);
- if(!hash_buf)
+ /* allocate buffers to store compressed and decompressed record
+ * hash data
+ */
+ comp_buf = malloc(fd->rec_map.len);
+ hash_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);
+ if(!comp_buf || !hash_buf)
return(-1);
+ hash_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+ memset(hash_buf, 0, DARSHAN_DEF_COMP_BUF_SZ);
ret = darshan_log_seek(fd, fd->rec_map.off);
if(ret < 0)
{
fprintf(stderr, "Error: unable to seek in darshan log file.\n");
+ free(comp_buf);
+ free(hash_buf);
return(-1);
}
/* read the record hash from the log file */
- ret = darshan_log_read(fd, hash_buf, fd->rec_map.len);
- if(ret < fd->rec_map.len)
+ ret = darshan_log_read(fd, comp_buf, fd->rec_map.len);
+ if(ret != fd->rec_map.len)
+ {
+ fprintf(stderr, "Error: failed to read record hash from darshan log file.\n");
+ free(comp_buf);
+ free(hash_buf);
+ return(-1);
+ }
+
+ /* decompress the record hash buffer */
+ ret = darshan_decompress_buf(comp_buf, fd->rec_map.len,
+ hash_buf, &hash_buf_sz, fd->comp_type);
+ if(ret < 0)
{
- fprintf(stderr, "Error: invalid darshan log file (failed to read record hash).\n");
+ fprintf(stderr, "Error: unable to decompress darshan job data.\n");
+ free(comp_buf);
+ free(hash_buf);
return(-1);
}
+ free(comp_buf);
buf_ptr = hash_buf;
while(buf_ptr < (hash_buf + hash_buf_sz))
@@ -321,7 +621,6 @@ int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)
path_len_ptr = (uint32_t *)buf_ptr;
buf_ptr += sizeof(uint32_t);
path_ptr = (char *)buf_ptr;
- buf_ptr += *path_len_ptr;
if(fd->swap_flag)
{
@@ -336,12 +635,14 @@ int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)
ref = malloc(sizeof(*ref));
if(!ref)
{
+ free(hash_buf);
return(-1);
}
ref->rec.name = malloc(*path_len_ptr + 1);
if(!ref->rec.name)
{
free(ref);
+ free(hash_buf);
return(-1);
}
@@ -353,48 +654,238 @@ int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash)
/* add this record to the hash */
HASH_ADD(hlink, *hash, rec.id, sizeof(darshan_record_id), ref);
}
+
+ buf_ptr += *path_len_ptr;
}
+ free(hash_buf);
return(0);
}
-int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id,
- void *moddat_buf, int moddat_buf_sz)
+/* darshan_log_puthash()
+ *
+ * writes the hash table of records to the darshan log file
+ * NOTE: this function call should follow immediately after the call
+ * to darshan_log_putmounts(), as it assumes the darshan log file pointer
+ * is pointing to the offset immediately following the mount information
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash)
{
- int mod_buf_end = fd->mod_map[mod_id].off + fd->mod_map[mod_id].len;
+ size_t hash_buf_sz;
+ char *hash_buf;
+ char *hash_buf_off;
+ struct darshan_record_ref *ref, *tmp;
+ uint32_t name_len;
+ size_t record_sz;
+ char *comp_buf;
+ int comp_buf_sz;
int ret;
- if(!fd->mod_map[mod_id].len || fd->pos == mod_buf_end)
- return(0); /* no (more) data corresponding to this mod_id */
+ /* allocate a buffer to store 2 MiB worth of record data */
+ /* NOTE: this buffer may be reallocated if estimate is too small */
+ hash_buf_sz = 2 * 1024 * 1024;
+ hash_buf = malloc(hash_buf_sz);
+ if(!hash_buf)
+ {
+ return(-1);
+ }
- /* only seek to start of module data if current log file position
- * is not within the given mod_id's range. This allows one to
- * repeatedly call this function and get chunks of a module's
- * data piecemeal.
- */
- if((fd->pos < fd->mod_map[mod_id].off) || (fd->pos > mod_buf_end))
+ /* serialize the record hash into a buffer for writing */
+ hash_buf_off = hash_buf;
+ HASH_ITER(hlink, hash, ref, tmp)
{
- ret = darshan_log_seek(fd, fd->mod_map[mod_id].off);
- if(ret < 0)
+ name_len = strlen(ref->rec.name);
+ record_sz = sizeof(darshan_record_id) + sizeof(uint32_t) + name_len;
+ /* make sure there is room in the buffer for this record */
+ if((hash_buf_off + record_sz) > (hash_buf + hash_buf_sz))
{
- fprintf(stderr, "Error: unable to seek in darshan log file.\n");
- return(-1);
+ char *tmp_buf;
+ size_t old_buf_sz;
+
+ /* if no room, reallocate the hash buffer at twice the current size */
+ old_buf_sz = hash_buf_off - hash_buf;
+ hash_buf_sz *= 2;
+ tmp_buf = malloc(hash_buf_sz);
+ if(!tmp_buf)
+ {
+ free(hash_buf);
+ return(-1);
+ }
+
+ memcpy(tmp_buf, hash_buf, old_buf_sz);
+ free(hash_buf);
+ hash_buf = tmp_buf;
+ hash_buf_off = hash_buf + old_buf_sz;
}
+
+ /* now serialize the record into the hash buffer.
+ * NOTE: darshan record hash serialization method:
+ * ... darshan_record_id | (uint32_t) path_len | path ...
+ */
+ *((darshan_record_id *)hash_buf_off) = ref->rec.id;
+ hash_buf_off += sizeof(darshan_record_id);
+ *((uint32_t *)hash_buf_off) = name_len;
+ hash_buf_off += sizeof(uint32_t);
+ memcpy(hash_buf_off, ref->rec.name, name_len);
+ hash_buf_off += name_len;
}
+ hash_buf_sz = hash_buf_off - hash_buf;
- /* read the record hash from the log file */
- ret = darshan_log_read(fd, moddat_buf, moddat_buf_sz);
- if(ret != moddat_buf_sz)
+ comp_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);
+ if(!comp_buf)
+ return(-1);
+ comp_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+
+ /* compress the record hash */
+ ret = darshan_compress_buf(hash_buf, hash_buf_sz,
+ comp_buf, &comp_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to compress darshan record hash.\n");
+ free(comp_buf);
+ free(hash_buf);
+ return(-1);
+ }
+
+ /* set the appropriate mapping info for the record hash in the file descriptor */
+ fd->rec_map.off = fd->pos;
+ fd->rec_map.len = comp_buf_sz;
+
+ /* write the record hash to file */
+ ret = darshan_log_write(fd, comp_buf, comp_buf_sz);
+ if(ret != comp_buf_sz)
+ {
+ fprintf(stderr, "Error: failed to write record hash to darshan log file.\n");
+ free(comp_buf);
+ free(hash_buf);
+ return(-1);
+ }
+
+ free(comp_buf);
+ free(hash_buf);
+
+ return(0);
+}
+
+/* darshan_log_getmod()
+ *
+ * returns 1 on successful read of module data, 0 on no data, -1 on failure
+ */
+int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,
+ void *mod_buf, int *mod_buf_sz)
+{
+ char *comp_buf;
+ int ret;
+
+ if(mod_id < 0 || mod_id >= DARSHAN_MAX_MODS)
+ {
+ fprintf(stderr, "Error: invalid Darshan module id.\n");
+ return(-1);
+ }
+
+ if(fd->mod_map[mod_id].len == 0)
+ return(0); /* no data corresponding to this mod_id */
+
+ comp_buf = malloc(fd->mod_map[mod_id].len);
+ if(!comp_buf)
+ return(-1);
+
+ ret = darshan_log_seek(fd, fd->mod_map[mod_id].off);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to seek in darshan log file.\n");
+ free(comp_buf);
+ return(ret);
+ }
+
+ /* read this module's data from the log file */
+ ret = darshan_log_read(fd, comp_buf, fd->mod_map[mod_id].len);
+ if(ret != fd->mod_map[mod_id].len)
{
fprintf(stderr,
- "Error: invalid darshan log file (failed to read module %s data).\n",
+ "Error: failed to read module %s data from darshan log file.\n",
+ darshan_module_names[mod_id]);
+ free(comp_buf);
+ return(-1);
+ }
+
+ /* decompress this module's data */
+ ret = darshan_decompress_buf(comp_buf, fd->mod_map[mod_id].len,
+ mod_buf, mod_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to decompress module %s data.\n",
darshan_module_names[mod_id]);
+ free(comp_buf);
return(-1);
}
+ free(comp_buf);
return(1);
}
+/* darshan_log_putmod()
+ *
+ * write a chunk of module data to the darshan log file
+ * NOTE: this function call should be called directly after the
+ * put_hash() function, as it expects the file pointer to be
+ * positioned directly past the record hash location. Also,
+ * for a set of modules with data to write to file, this function
+ * should be called in order of increasing module identifiers,
+ * as the darshan log file format expects this ordering.
+ *
+ * returns 0 on success, -1 on failure
+ */
+int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,
+ void *mod_buf, int mod_buf_sz)
+{
+ char *comp_buf;
+ int comp_buf_sz;
+ int ret;
+
+ if(mod_id < 0 || mod_id >= DARSHAN_MAX_MODS)
+ {
+ fprintf(stderr, "Error: invalid Darshan module id.\n");
+ return(-1);
+ }
+
+ comp_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);
+ if(!comp_buf)
+ return(-1);
+ comp_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+
+ /* compress the module's data */
+ ret = darshan_compress_buf(mod_buf, mod_buf_sz,
+ comp_buf, &comp_buf_sz, fd->comp_type);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to compress module %s data.\n",
+ darshan_module_names[mod_id]);
+ free(comp_buf);
+ return(-1);
+ }
+
+ /* set the appropriate mapping information for this module */
+ fd->mod_map[mod_id].off = fd->pos;
+ fd->mod_map[mod_id].len = comp_buf_sz;
+
+ /* write the module chunk to the log file */
+ ret = darshan_log_write(fd, comp_buf, comp_buf_sz);
+ if(ret != comp_buf_sz)
+ {
+ fprintf(stderr,
+ "Error: failed to write module %s data to darshan log file.\n",
+ darshan_module_names[mod_id]);
+ free(comp_buf);
+ return(-1);
+ }
+
+ free(comp_buf);
+ return(0);
+}
+
/* darshan_log_close()
*
* close an open darshan file descriptor
@@ -403,10 +894,11 @@ int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id,
*/
void darshan_log_close(darshan_fd fd)
{
- if(fd->gzf)
- gzclose(fd->gzf);
+ if(fd->fildes)
+ close(fd->fildes);
- /* TODO: check bz2 */
+ if(fd->exe_mnt_data)
+ free(fd->exe_mnt_data);
free(fd);
@@ -419,66 +911,296 @@ void darshan_log_close(darshan_fd fd)
*/
static int darshan_log_seek(darshan_fd fd, off_t offset)
{
- z_off_t zoff = 0;
- z_off_t zoff_ret = 0;
+ off_t ret_off;
if(fd->pos == offset)
return(0);
- if(fd->gzf)
+ ret_off = lseek(fd->fildes, offset, SEEK_SET);
+ if(ret_off == offset)
{
- zoff += offset;
- zoff_ret = gzseek(fd->gzf, zoff, SEEK_SET);
- if(zoff_ret == zoff)
- {
- fd->pos = offset;
- return(0);
- }
- return(-1);
+ fd->pos = offset;
+ return(0);
}
- /* TODO: check bz2 */
-
return(-1);
}
+/* return amount read on success, 0 on EOF, -1 on failure.
+ */
+static int darshan_log_read(darshan_fd fd, void* buf, int len)
+{
+ int ret;
+
+ /* read data from the log file using the given map */
+ ret = read(fd->fildes, buf, len);
+ if(ret > 0)
+ fd->pos += ret;
+
+ return(ret);
+}
+
/* return amount written on success, -1 on failure.
*/
static int darshan_log_write(darshan_fd fd, void* buf, int len)
{
int ret;
- if(fd->gzf)
+ ret = write(fd->fildes, buf, len);
+ if(ret > 0)
+ fd->pos += ret;
+
+ return(ret);
+}
+
+/* return 0 on successful decompression, -1 on failure
+ */
+static int darshan_decompress_buf(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz,
+ enum darshan_comp_type comp_type)
+{
+ int ret;
+
+ switch(comp_type)
{
- ret = gzwrite(fd->gzf, buf, len);
- if(ret > 0)
- fd->pos += ret;
- return(ret);
+ case DARSHAN_ZLIB_COMP:
+ ret = darshan_zlib_decomp(comp_buf, comp_buf_sz,
+ decomp_buf, inout_decomp_buf_sz);
+ break;
+#ifdef HAVE_LIBBZ2
+ case DARSHAN_BZIP2_COMP:
+ ret = darshan_bzip2_decomp(comp_buf, comp_buf_sz,
+ decomp_buf, inout_decomp_buf_sz);
+ break;
+#endif
+ default:
+ fprintf(stderr, "Error: invalid decompression method.\n");
+ return(-1);
}
- /* TODO: check bz2 */
+ return(ret);
+}
+
+static int darshan_compress_buf(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz,
+ enum darshan_comp_type comp_type)
+{
+ int ret;
+
+ switch(comp_type)
+ {
+ case DARSHAN_ZLIB_COMP:
+ ret = darshan_zlib_comp(decomp_buf, decomp_buf_sz,
+ comp_buf, inout_comp_buf_sz);
+ break;
+#ifdef HAVE_LIBBZ2
+ case DARSHAN_BZIP2_COMP:
+ ret = darshan_bzip2_comp(decomp_buf, decomp_buf_sz,
+ comp_buf, inout_comp_buf_sz);
+ break;
+#endif
+ default:
+ fprintf(stderr, "Error: invalid compression method.\n");
+ return(-1);
+ }
+
+ return(ret);
- return(-1);
}
-/* return amount read on success, 0 on EOF, -1 on failure.
- */
-static int darshan_log_read(darshan_fd fd, void* buf, int len)
+static int darshan_zlib_decomp(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz)
{
int ret;
+ int total_out = 0;
+ z_stream tmp_stream;
+
+ memset(&tmp_stream, 0, sizeof(tmp_stream));
+ tmp_stream.zalloc = Z_NULL;
+ tmp_stream.zfree = Z_NULL;
+ tmp_stream.opaque = Z_NULL;
+ tmp_stream.next_in = (unsigned char*)comp_buf;
+ tmp_stream.avail_in = comp_buf_sz;
+ tmp_stream.next_out = (unsigned char*)decomp_buf;
+ tmp_stream.avail_out = *inout_decomp_buf_sz;
+
+ /* initialize the zlib decompression parameters */
+ /* TODO: check these parameters? */
+ //ret = inflateInit2(&tmp_stream, 31);
+ ret = inflateInit(&tmp_stream);
+ if(ret != Z_OK)
+ {
+ return(-1);
+ }
- if(fd->gzf)
+ /* while we have not finished consuming all of the compressed input data */
+ while(tmp_stream.avail_in)
{
- ret = gzread(fd->gzf, buf, len);
- if(ret > 0)
- fd->pos += ret;
- return(ret);
+ if(tmp_stream.avail_out == 0)
+ {
+ /* We ran out of buffer space for decompression. In theory,
+ * we could just alloc more space, but probably just easier
+ * to bump up the default size of the output buffer.
+ */
+ inflateEnd(&tmp_stream);
+ return(-1);
+ }
+
+ /* decompress data */
+ ret = inflate(&tmp_stream, Z_FINISH);
+ if(ret != Z_STREAM_END)
+ {
+ inflateEnd(&tmp_stream);
+ return(-1);
+ }
+
+ total_out += tmp_stream.total_out;
+ if(tmp_stream.avail_in)
+ inflateReset(&tmp_stream);
}
+ inflateEnd(&tmp_stream);
- /* TODO: check bz2 */
+ *inout_decomp_buf_sz = total_out;
+ return(0);
+}
- return(-1);
+static int darshan_zlib_comp(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz)
+{
+ int ret;
+ z_stream tmp_stream;
+
+ memset(&tmp_stream, 0, sizeof(tmp_stream));
+ tmp_stream.zalloc = Z_NULL;
+ tmp_stream.zfree = Z_NULL;
+ tmp_stream.opaque = Z_NULL;
+ tmp_stream.next_in = (unsigned char*)decomp_buf;
+ tmp_stream.avail_in = decomp_buf_sz;
+ tmp_stream.next_out = (unsigned char*)comp_buf;
+ tmp_stream.avail_out = *inout_comp_buf_sz;
+
+ ret = deflateInit(&tmp_stream, Z_DEFAULT_COMPRESSION);
+ if(ret != Z_OK)
+ {
+ return(-1);
+ }
+
+ /* compress data */
+ ret = deflate(&tmp_stream, Z_FINISH);
+ if(ret != Z_STREAM_END)
+ {
+ deflateEnd(&tmp_stream);
+ return(-1);
+ }
+ deflateEnd(&tmp_stream);
+
+ *inout_comp_buf_sz = tmp_stream.total_out;
+ return(0);
+}
+
+#ifdef HAVE_LIBBZ2
+static int darshan_bzip2_decomp(char* comp_buf, int comp_buf_sz,
+ char* decomp_buf, int* inout_decomp_buf_sz)
+{
+ int ret;
+ int total_out = 0;
+ bz_stream tmp_stream;
+
+ memset(&tmp_stream, 0, sizeof(tmp_stream));
+ tmp_stream.bzalloc = NULL;
+ tmp_stream.bzfree = NULL;
+ tmp_stream.opaque = NULL;
+ tmp_stream.next_in = comp_buf;
+ tmp_stream.avail_in = comp_buf_sz;
+ tmp_stream.next_out = decomp_buf;
+ tmp_stream.avail_out = *inout_decomp_buf_sz;
+
+ ret = BZ2_bzDecompressInit(&tmp_stream, 1, 0);
+ if(ret != BZ_OK)
+ {
+ return(-1);
+ }
+
+ /* while we have not finished consuming all of the uncompressed input data */
+ while(tmp_stream.avail_in)
+ {
+ if(tmp_stream.avail_out == 0)
+ {
+ /* We ran out of buffer space for decompression. In theory,
+ * we could just alloc more space, but probably just easier
+ * to bump up the default size of the output buffer.
+ */
+ BZ2_bzDecompressEnd(&tmp_stream);
+ return(-1);
+ }
+
+ /* decompress data */
+ ret = BZ2_bzDecompress(&tmp_stream);
+ if(ret != BZ_STREAM_END)
+ {
+ BZ2_bzDecompressEnd(&tmp_stream);
+ return(-1);
+ }
+
+ assert(tmp_stream.total_out_hi32 == 0);
+ total_out += tmp_stream.total_out_lo32;
+ if(tmp_stream.avail_in)
+ {
+ /* reinitialize bzip2 stream, we have more data to
+ * decompress
+ */
+ BZ2_bzDecompressEnd(&tmp_stream);
+ ret = BZ2_bzDecompressInit(&tmp_stream, 1, 0);
+ if(ret != BZ_OK)
+ {
+ return(-1);
+ }
+ }
+ }
+ BZ2_bzDecompressEnd(&tmp_stream);
+
+ *inout_decomp_buf_sz = total_out;
+ return(0);
+}
+
+static int darshan_bzip2_comp(char* decomp_buf, int decomp_buf_sz,
+ char* comp_buf, int* inout_comp_buf_sz)
+{
+ int ret;
+ bz_stream tmp_stream;
+
+ memset(&tmp_stream, 0, sizeof(tmp_stream));
+ tmp_stream.bzalloc = NULL;
+ tmp_stream.bzfree = NULL;
+ tmp_stream.opaque = NULL;
+ tmp_stream.next_in = decomp_buf;
+ tmp_stream.avail_in = decomp_buf_sz;
+ tmp_stream.next_out = comp_buf;
+ tmp_stream.avail_out = *inout_comp_buf_sz;
+
+ ret = BZ2_bzCompressInit(&tmp_stream, 9, 1, 30);
+ if(ret != BZ_OK)
+ {
+ return(-1);
+ }
+
+ /* compress data */
+ do
+ {
+ ret = BZ2_bzCompress(&tmp_stream, BZ_FINISH);
+ if(ret < 0)
+ {
+ BZ2_bzCompressEnd(&tmp_stream);
+ return(-1);
+ }
+ } while (ret != BZ_STREAM_END);
+ BZ2_bzCompressEnd(&tmp_stream);
+
+ assert(tmp_stream.total_out_hi32 == 0);
+ *inout_comp_buf_sz = tmp_stream.total_out_lo32;
+ return(0);
}
+#endif
/*
* Local variables:
diff --git a/darshan-util/darshan-logutils.h b/darshan-util/darshan-logutils.h
index b6bb37a..f188575 100644
--- a/darshan-util/darshan-logutils.h
+++ b/darshan-util/darshan-logutils.h
@@ -12,20 +12,26 @@
#include <bzlib.h>
#endif
-#include "darshan-log-format.h"
#include "uthash-1.9.2/src/uthash.h"
-/* default to a compression buffer size of 4 MiB */
-#define DARSHAN_DEF_DECOMP_BUF_SZ (4*1024*1024)
+#include "darshan-log-format.h"
+
+/* default to a buffer size of 4 MiB for compression/decompression */
+#define DARSHAN_DEF_COMP_BUF_SZ (4*1024*1024)
-/* TODO: we need to refactor s.t. utilities don't know implementation
- of this, but module-specific functions do */
+/* TODO: can we refactor this def out of header? modules currently poke at swap_flag
+ * directly, but other than that there's no reason for another module to know this
+ * definition.
+ */
struct darshan_fd_s
{
- gzFile gzf;
+ int fildes;
int64_t pos;
+ enum darshan_comp_type comp_type;
char version[8];
int swap_flag;
+ char *exe_mnt_data;
+ struct darshan_log_map job_map;
struct darshan_log_map rec_map;
struct darshan_log_map mod_map[DARSHAN_MAX_MODS];
};
@@ -37,30 +43,70 @@ struct darshan_record_ref
UT_hash_handle hlink;
};
-darshan_fd darshan_log_open(const char *name, const char* mode);
+struct darshan_mod_logutil_funcs
+{
+ int (*log_get_record)(
+ void** mod_buf_p,
+ int* mod_bytes_left,
+ void** file_rec,
+ darshan_record_id* rec_id,
+ int byte_swap_flag
+ );
+ void (*log_print_record)(
+ void *file_rec,
+ char *file_name,
+ char *mnt_pt,
+ char *fs_type
+ );
+};
+
+extern struct darshan_mod_logutil_funcs *mod_logutils[DARSHAN_MAX_MODS];
+
+#include "darshan-posix-logutils.h"
+#include "darshan-mpiio-logutils.h"
+#include "darshan-hdf5-logutils.h"
+#include "darshan-pnetcdf-logutils.h"
+#include "darshan-bgq-logutils.h"
+
+darshan_fd darshan_log_open(const char *name);
+darshan_fd darshan_log_create(const char *name, enum darshan_comp_type comp_type);
int darshan_log_getheader(darshan_fd fd, struct darshan_header *header);
+int darshan_log_putheader(darshan_fd fd);
int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);
+int darshan_log_putjob(darshan_fd fd, struct darshan_job *job);
int darshan_log_getexe(darshan_fd fd, char *buf);
+int darshan_log_putexe(darshan_fd fd, char *buf);
int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
char*** fs_types, int* count);
+int darshan_log_putmounts(darshan_fd fd, char** mnt_pts,
+ char** fs_types, int count);
int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);
-int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id,
- void *moddat_buf, int moddat_buf_sz);
+int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash);
+int darshan_log_getmod(darshan_fd fd, darshan_module_id mod_id,
+ void *mod_buf, int *mod_buf_sz);
+int darshan_log_putmod(darshan_fd fd, darshan_module_id mod_id,
+ void *mod_buf, int mod_buf_sz);
void darshan_log_close(darshan_fd file);
-/* convenience macros for printing out counters */
-#define CP_PRINT_HEADER() printf("#<rank>\t<file>\t<counter>\t<value>\t<name suffix>\t<mount pt>\t<fs type>\n")
-#define CP_PRINT(__job, __file, __counter, __mnt_pt, __fs_type) do {\
- printf("%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t...%s\t%s\t%s\n", \
- (__file)->rank, (__file)->hash, darshan_names[__counter], \
- (__file)->counters[__counter], (__file)->name_suffix, \
- __mnt_pt, __fs_type); \
+/* convenience macros for printing Darshan counters */
+#define DARSHAN_PRINT_HEADER() \
+ printf("\n#<module>\t<rank>\t<file>\t<counter>\t<value>" \
+ "\t<file name>\t<mount pt>\t<fs type>\n")
+
+#define DARSHAN_COUNTER_PRINT(__mod_name, __rank, __file_id, \
+ __counter, __counter_val, __file_name, \
+ __mnt_pt, __fs_type) do { \
+ printf("%s\t%" PRId64 "\t%" PRIu64 "\t%s\t%" PRId64 "\t%s\t%s\t%s\n", \
+ __mod_name, __rank, __file_id, __counter, __counter_val, \
+ __file_name, __mnt_pt, __fs_type); \
} while(0)
-#define CP_F_PRINT(__job, __file, __counter, __mnt_pt, __fs_type) do {\
- printf("%" PRId64 "\t%" PRIu64 "\t%s\t%f\t...%s\t%s\t%s\n", \
- (__file)->rank, (__file)->hash, darshan_f_names[__counter], \
- (__file)->fcounters[__counter], (__file)->name_suffix, \
- __mnt_pt, __fs_type); \
+
+#define DARSHAN_F_COUNTER_PRINT(__mod_name, __rank, __file_id, \
+ __counter, __counter_val, __file_name, \
+ __mnt_pt, __fs_type) do { \
+ printf("%s\t%" PRId64 "\t%" PRIu64 "\t%s\t%f\t%s\t%s\t%s\n", \
+ __mod_name, __rank, __file_id, __counter, __counter_val, \
+ __file_name, __mnt_pt, __fs_type); \
} while(0)
/* naive byte swap implementation */
diff --git a/darshan-util/darshan-mpiio-logutils.c b/darshan-util/darshan-mpiio-logutils.c
index 33fc1c3..f6c30e6 100644
--- a/darshan-util/darshan-mpiio-logutils.c
+++ b/darshan-util/darshan-mpiio-logutils.c
@@ -19,33 +19,80 @@
#include "darshan-mpiio-logutils.h"
-int darshan_log_get_mpiio_file(darshan_fd fd, struct darshan_mpiio_file *file)
+/* counter name strings for the MPI-IO module */
+#define X(a) #a,
+char *mpiio_counter_names[] = {
+ MPIIO_COUNTERS
+};
+
+char *mpiio_f_counter_names[] = {
+ MPIIO_F_COUNTERS
+};
+#undef X
+
+static int darshan_log_get_mpiio_file(void** mpiio_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag);
+static void darshan_log_print_mpiio_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+struct darshan_mod_logutil_funcs mpiio_logutils =
+{
+ .log_get_record = &darshan_log_get_mpiio_file,
+ .log_print_record = &darshan_log_print_mpiio_file,
+};
+
+static int darshan_log_get_mpiio_file(void** mpiio_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag)
{
int i;
- int ret;
+ struct darshan_mpiio_file *file = (struct darshan_mpiio_file *)
+ (*mpiio_buf_p);
+
+ if(*bytes_left < sizeof(struct darshan_mpiio_file))
+ return(-1);
+
+ if(byte_swap_flag)
+ {
+ /* swap bytes if necessary */
+ DARSHAN_BSWAP64(&file->f_id);
+ DARSHAN_BSWAP64(&file->rank);
+ for(i=0; i<MPIIO_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->counters[i]);
+ for(i=0; i<MPIIO_F_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->fcounters[i]);
+ }
- /* reset file record, so that diff compares against a zero'd out record
- * if file is missing
- */
- memset(file, 0, sizeof(*file));
+ /* update/set output variables */
+ *file_rec = (void *)file;
+ *rec_id = file->f_id;
+ *mpiio_buf_p = (file + 1); /* increment input buf by size of file record */
+ *bytes_left -= sizeof(struct darshan_mpiio_file);
+
+ return(0);
+}
+
+static void darshan_log_print_mpiio_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ int i;
+ struct darshan_mpiio_file *mpiio_file_rec =
+ (struct darshan_mpiio_file *)file_rec;
+
+ for(i=0; i<MPIIO_NUM_INDICES; i++)
+ {
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ mpiio_file_rec->rank, mpiio_file_rec->f_id, mpiio_counter_names[i],
+ mpiio_file_rec->counters[i], file_name, mnt_pt, fs_type);
+ }
- ret = darshan_log_get_moddat(fd, DARSHAN_MPIIO_MOD,
- (void *)file, sizeof(*file));
- if(ret == 1)
+ for(i=0; i<MPIIO_F_NUM_INDICES; i++)
{
- if(fd->swap_flag)
- {
- /* swap bytes if necessary */
- DARSHAN_BSWAP64(&file->f_id);
- DARSHAN_BSWAP64(&file->rank);
- for(i=0; i<DARSHAN_MPIIO_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->counters[i]);
- for(i=0; i<DARSHAN_MPIIO_F_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->fcounters[i]);
- }
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_MPIIO_MOD],
+ mpiio_file_rec->rank, mpiio_file_rec->f_id, mpiio_f_counter_names[i],
+ mpiio_file_rec->fcounters[i], file_name, mnt_pt, fs_type);
}
- return(ret);
+ return;
}
/*
diff --git a/darshan-util/darshan-mpiio-logutils.h b/darshan-util/darshan-mpiio-logutils.h
index 965225f..3732ba5 100644
--- a/darshan-util/darshan-mpiio-logutils.h
+++ b/darshan-util/darshan-mpiio-logutils.h
@@ -10,6 +10,9 @@
#include "darshan-logutils.h"
#include "darshan-mpiio-log-format.h"
-int darshan_log_get_mpiio_file(darshan_fd fd, struct darshan_mpiio_file *file);
+extern char *mpiio_counter_names[];
+extern char *mpiio_f_counter_names[];
+
+extern struct darshan_mod_logutil_funcs mpiio_logutils;
#endif
diff --git a/darshan-util/darshan-mpiio-parser.c b/darshan-util/darshan-mpiio-parser.c
deleted file mode 100644
index 683194f..0000000
--- a/darshan-util/darshan-mpiio-parser.c
+++ /dev/null
@@ -1,214 +0,0 @@
-/*
- * Copyright (C) 2015 University of Chicago.
- * See COPYRIGHT notice in top-level directory.
- *
- */
-
-#include <stdio.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <string.h>
-#include <zlib.h>
-#include <time.h>
-#include <stdlib.h>
-#include <getopt.h>
-#include <assert.h>
-
-#include "darshan-logutils.h"
-#include "darshan-mpiio-logutils.h"
-#include "uthash-1.9.2/src/uthash.h"
-
-int main(int argc, char **argv)
-{
- int ret;
- int i;
- char *filename;
- char tmp_string[4096];
- darshan_fd fd;
- struct darshan_header header;
- struct darshan_job job;
- struct darshan_record_ref *rec_hash = NULL;
- struct darshan_record_ref *ref;
- int mount_count;
- char** mnt_pts;
- char** fs_types;
- time_t tmp_time = 0;
- char *token;
- char *save;
- char buffer[DARSHAN_JOB_METADATA_LEN];
- struct darshan_mpiio_file next_file;
-
- assert(argc == 2);
- filename = argv[1];
-
- struct stat sbuf;
- stat(filename, &sbuf);
-
- fd = darshan_log_open(filename, "r");
- if(!fd)
- {
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", filename);
- return(-1);
- }
-
- /**************************************************************/
- /* TODO: some of this code should be shared or in a separate executable
- * instead of repeated in each module parser
- */
-
- /* read darshan log header */
- ret = darshan_log_getheader(fd, &header);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getheader() failed to read log header.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* read darshan job info */
- ret = darshan_log_getjob(fd, &job);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getjob() failed to read job data.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* get the original command line for this job */
- ret = darshan_log_getexe(fd, tmp_string);
- if(ret < 0)
- {
- fprintf(stderr, "Error: unable to read trailing job information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print job summary */
- printf("# darshan log version: %s\n", header.version_string);
- printf("# size of MPIIO file statistics: %zu bytes\n", sizeof(struct darshan_mpiio_file));
- printf("# size of job statistics: %zu bytes\n", sizeof(struct darshan_job));
- printf("# exe: %s\n", tmp_string);
- printf("# uid: %" PRId64 "\n", job.uid);
- printf("# jobid: %" PRId64 "\n", job.jobid);
- printf("# start_time: %" PRId64 "\n", job.start_time);
- tmp_time += job.start_time;
- printf("# start_time_asci: %s", ctime(&tmp_time));
- printf("# end_time: %" PRId64 "\n", job.end_time);
- tmp_time = 0;
- tmp_time += job.end_time;
- printf("# end_time_asci: %s", ctime(&tmp_time));
- printf("# nprocs: %" PRId64 "\n", job.nprocs);
- printf("# run time: %" PRId64 "\n", job.end_time - job.start_time + 1);
- for(token=strtok_r(job.metadata, "\n", &save);
- token != NULL;
- token=strtok_r(NULL, "\n", &save))
- {
- char *key;
- char *value;
- /* NOTE: we intentionally only split on the first = character.
- * There may be additional = characters in the value portion
- * (for example, when storing mpi-io hints).
- */
- strcpy(buffer, token);
- key = buffer;
- value = index(buffer, '=');
- if(!value)
- continue;
- /* convert = to a null terminator to split key and value */
- value[0] = '\0';
- value++;
- printf("# metadata: %s = %s\n", key, value);
- }
-
- /* get the mount information for this log */
- ret = darshan_log_getmounts(fd, &mnt_pts, &fs_types, &mount_count);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmounts() failed to read mount information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print table of mounted file systems */
- printf("\n# mounted file systems (mount point and fs type)\n");
- printf("# -------------------------------------------------------\n");
- for(i=0; i<mount_count; i++)
- {
- printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);
- }
-
- /* read hash of darshan records */
- ret = darshan_log_gethash(fd, &rec_hash);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmap() failed to read record map.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* end TODO */
- /*******************************************/
-
- printf("\n*** FILE RECORD DATA ***\n");
-
- ret = darshan_log_get_mpiio_file(fd, &next_file);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_get_mpiio_file() failed to read next record.\n");
- darshan_log_close(fd);
- return(-1);
- }
- if(ret == 0)
- {
- printf("# no files opened.\n");
- darshan_log_close(fd);
- return(0);
- }
-
- /* loop over each stored MPIIO file record and print counters */
- i = 1;
- do
- {
- /* get the pathname for this record */
- HASH_FIND(hlink, rec_hash, &next_file.f_id, sizeof(darshan_record_id), ref);
- assert(ref);
-
- printf("\tRecord %d: id=%"PRIu64" (path=%s, rank=%"PRId64")\n",
- i, next_file.f_id, ref->rec.name, next_file.rank);
- /* TODO: does it make sense to put these in a header or something?
- * Down side of listing them here is ordering dependency between enum
- * in header and names here.
- */
- printf(
- "\t\tMPIIO_INDEP_OPENS:\t%"PRIu64"\n"
- "\t\tMPIIO_COLL_OPENS:\t%"PRIu64"\n"
- "\t\tMPIIO_HINTS:\t%"PRIu64"\n"
- "\t\tMPIIO_F_META_TIME:\t%lf\n"
- "\t\tMPIIO_F_OPEN_TIMESTAMP:\t%lf\n",
- next_file.counters[DARSHAN_MPIIO_INDEP_OPENS],
- next_file.counters[DARSHAN_MPIIO_COLL_OPENS],
- next_file.counters[DARSHAN_MPIIO_HINTS],
- next_file.fcounters[DARSHAN_MPIIO_F_META_TIME],
- next_file.fcounters[DARSHAN_MPIIO_F_OPEN_TIMESTAMP]);
-
- i++;
- } while((ret = darshan_log_get_mpiio_file(fd, &next_file)) == 1);
-
- /* free mount info */
- for(i=0; i<mount_count; i++)
- {
- free(mnt_pts[i]);
- free(fs_types[i]);
- }
- if(mount_count > 0)
- {
- free(mnt_pts);
- free(fs_types);
- }
-
- darshan_log_close(fd);
-
- return(0);
-}
diff --git a/darshan-util/darshan-parser.c b/darshan-util/darshan-parser.c
index 02171a5..266033a 100644
--- a/darshan-util/darshan-parser.c
+++ b/darshan-util/darshan-parser.c
@@ -1,6 +1,7 @@
/*
- * (C) 2009 by Argonne National Laboratory.
- * See COPYRIGHT in top-level directory.
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
*/
#include <stdio.h>
@@ -15,10 +16,10 @@
#include <getopt.h>
#include <assert.h>
-#include "darshan-logutils.h"
-
#include "uthash-1.9.2/src/uthash.h"
+#include "darshan-logutils.h"
+
/*
* Options
*/
@@ -41,7 +42,6 @@
#define FILETYPE_PARTSHARED (1 << 2)
#define max(a,b) (((a) > (b)) ? (a) : (b))
-#define max3(a,b,c) (((a) > (b)) ? (((a) > (c)) ? (a) : (c)) : (((b) > (c)) ? (b) : (c)))
/*
* Datatypes
@@ -49,36 +49,14 @@
typedef struct hash_entry_s
{
UT_hash_handle hlink;
- uint64_t hash;
+ darshan_record_id rec_id;
int64_t type;
int64_t procs;
- int64_t counters[CP_NUM_INDICES];
- double fcounters[CP_F_NUM_INDICES];
+ void *rec_dat;
double cumul_time;
- double meta_time;
double slowest_time;
- char name_suffix[CP_NAME_SUFFIX_LEN+1];
} hash_entry_t;
-typedef struct perf_data_s
-{
- int64_t total_bytes;
- double slowest_rank_time;
- double slowest_rank_meta_time;
- int slowest_rank_rank;
- double shared_time_by_cumul;
- double shared_time_by_open;
- double shared_time_by_open_lastio;
- double shared_time_by_slowest;
- double shared_meta_time;
- double agg_perf_by_cumul;
- double agg_perf_by_open;
- double agg_perf_by_open_lastio;
- double agg_perf_by_slowest;
- double *rank_cumul_io_time;
- double *rank_cumul_md_time;
-} perf_data_t;
-
typedef struct file_data_s
{
int64_t total;
@@ -101,15 +79,41 @@ typedef struct file_data_s
int64_t shared_max;
} file_data_t;
+typedef struct perf_data_s
+{
+ int64_t total_bytes;
+ double slowest_rank_time;
+ double slowest_rank_meta_time;
+ int slowest_rank_rank;
+ double shared_time_by_cumul;
+ double shared_time_by_open;
+ double shared_time_by_open_lastio;
+ double shared_time_by_slowest;
+ double shared_meta_time;
+ double agg_perf_by_cumul;
+ double agg_perf_by_open;
+ double agg_perf_by_open_lastio;
+ double agg_perf_by_slowest;
+ double *rank_cumul_io_time;
+ double *rank_cumul_md_time;
+} perf_data_t;
+
/*
* Prototypes
*/
-void accum_perf(struct darshan_file *, hash_entry_t *, perf_data_t *);
-void calc_perf(struct darshan_job *, hash_entry_t *, perf_data_t *);
+void posix_accum_file(struct darshan_posix_file *pfile, hash_entry_t *hfile, int64_t nprocs);
+void posix_accum_perf(struct darshan_posix_file *pfile, perf_data_t *pdata);
+void posix_calc_file(hash_entry_t *file_hash, file_data_t *fdata);
+void posix_print_total_file(struct darshan_posix_file *pfile);
+void posix_file_list(hash_entry_t *file_hash, struct darshan_record_ref *rec_hash, int detail_flag);
+
+void mpiio_accum_file(struct darshan_mpiio_file *mfile, hash_entry_t *hfile, int64_t nprocs);
+void mpiio_accum_perf(struct darshan_mpiio_file *mfile, perf_data_t *pdata);
+void mpiio_calc_file(hash_entry_t *file_hash, file_data_t *fdata);
+void mpiio_print_total_file(struct darshan_mpiio_file *mfile);
+void mpiio_file_list(hash_entry_t *file_hash, struct darshan_record_ref *rec_hash, int detail_flag);
-void accum_file(struct darshan_job *, struct darshan_file *, hash_entry_t *, file_data_t *);
-void calc_file(struct darshan_job *, hash_entry_t *, file_data_t *);
-void file_list(struct darshan_job *, hash_entry_t *, int);
+void calc_perf(perf_data_t *pdata, int64_t nprocs);
int usage (char *exename)
{
@@ -191,65 +195,92 @@ int main(int argc, char **argv)
{
int ret;
int mask;
+ int i, j;
char *filename;
+ char tmp_string[4096] = {0};
+ darshan_fd fd;
+ struct darshan_header header;
struct darshan_job job;
- struct darshan_file cp_file;
- char tmp_string[4096];
- time_t tmp_time = 0;
- darshan_fd file;
- int i;
+ struct darshan_record_ref *rec_hash = NULL;
+ struct darshan_record_ref *ref, *tmp_ref;
int mount_count;
- int64_t* devs;
char** mnt_pts;
char** fs_types;
- int last_rank = 0;
+ time_t tmp_time = 0;
char *token;
char *save;
char buffer[DARSHAN_JOB_METADATA_LEN];
+ int empty_mods = 0;
+ char *mod_buf;
+ int mod_buf_sz;
hash_entry_t *file_hash = NULL;
hash_entry_t *curr = NULL;
- hash_entry_t *tmp = NULL;
+ hash_entry_t *tmp_file = NULL;
hash_entry_t total;
- perf_data_t pdata;
file_data_t fdata;
+ perf_data_t pdata;
- memset(&pdata, 0, sizeof(pdata));
memset(&total, 0, sizeof(total));
+ memset(&fdata, 0, sizeof(fdata));
+ memset(&pdata, 0, sizeof(pdata));
mask = parse_args(argc, argv, &filename);
- file = darshan_log_open(filename, "r");
- if(!file)
+ fd = darshan_log_open(filename);
+ if(!fd)
{
fprintf(stderr, "darshan_log_open() failed to open %s\n.", filename);
return(-1);
}
-
- /* read job info */
- ret = darshan_log_getjob(file, &job);
+
+ /* read darshan log header */
+ ret = darshan_log_getheader(fd, &header);
if(ret < 0)
{
- fprintf(stderr, "Error: unable to read job information from log file.\n");
- darshan_log_close(file);
+ fprintf(stderr, "darshan_log_getheader() failed to read log header.\n");
+ darshan_log_close(fd);
return(-1);
}
- /* warn user about any missing information in this log format */
- darshan_log_print_version_warnings(&job);
+ /* read darshan job info */
+ ret = darshan_log_getjob(fd, &job);
+ if(ret < 0)
+ {
+ fprintf(stderr, "darshan_log_getjob() failed to read job data.\n");
+ darshan_log_close(fd);
+ return(-1);
+ }
- ret = darshan_log_getexe(file, tmp_string);
+ /* get the original command line for this job */
+ ret = darshan_log_getexe(fd, tmp_string);
if(ret < 0)
{
fprintf(stderr, "Error: unable to read trailing job information.\n");
- darshan_log_close(file);
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* get the mount information for this log */
+ ret = darshan_log_getmounts(fd, &mnt_pts, &fs_types, &mount_count);
+ if(ret < 0)
+ {
+ fprintf(stderr, "darshan_log_getmounts() failed to read mount information.\n");
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* read hash of darshan records */
+ ret = darshan_log_gethash(fd, &rec_hash);
+ if(ret < 0)
+ {
+ fprintf(stderr, "darshan_log_getmap() failed to read record map.\n");
+ darshan_log_close(fd);
return(-1);
}
/* print job summary */
- printf("# darshan log version: %s\n", job.version_string);
- printf("# size of file statistics: %zu bytes\n", sizeof(cp_file));
- printf("# size of job statistics: %zu bytes\n", sizeof(job));
+ printf("# darshan log version: %s\n", header.version_string);
printf("# exe: %s\n", tmp_string);
printf("# uid: %" PRId64 "\n", job.uid);
printf("# jobid: %" PRId64 "\n", job.jobid);
@@ -282,98 +313,39 @@ int main(int argc, char **argv)
value++;
printf("# metadata: %s = %s\n", key, value);
}
-
- /* print table of mounted file systems */
- ret = darshan_log_getmounts(file, &devs, &mnt_pts, &fs_types, &mount_count);
- printf("\n# mounted file systems (device, mount point, and fs type)\n");
+
+ /* print breakdown of each log file component's contribution to file size */
+ printf("\n# log file component sizes (compressed)\n");
printf("# -------------------------------------------------------\n");
- for(i=0; i<mount_count; i++)
- {
- printf("# mount entry: %" PRId64 "\t%s\t%s\n", devs[i], mnt_pts[i], fs_types[i]);
- }
-
- /* try to retrieve first record (may not exist) */
- ret = darshan_log_getfile(file, &job, &cp_file);
- if(ret < 0)
+ printf("# header: %zu bytes (uncompressed)\n", sizeof(struct darshan_header));
+ printf("# job data: %zu bytes\n", header.rec_map.off - sizeof(struct darshan_header));
+ printf("# record table: %zu bytes\n", header.rec_map.len);
+ for(i=0; i<DARSHAN_MAX_MODS; i++)
{
- fprintf(stderr, "Error: failed to parse log file.\n");
- fflush(stderr);
- return(-1);
- }
- if(ret == 0)
- {
- /* it looks like the app didn't open any files */
- printf("# no files opened.\n");
- darshan_log_close(file);
- return(0);
+ if(header.mod_map[i].len)
+ {
+ printf("# %s module: %zu bytes\n", darshan_module_names[i],
+ header.mod_map[i].len);
+ }
}
- if ((mask & OPTION_BASE))
+ /* print table of mounted file systems */
+ printf("\n# mounted file systems (mount point and fs type)\n");
+ printf("# -------------------------------------------------------\n");
+ for(i=0; i<mount_count; i++)
{
- printf("\n# description of columns:\n");
- printf("# <rank>: MPI rank. -1 indicates that the file is shared\n");
- printf("# across all processes and statistics are aggregated.\n");
- printf("# <file>: hash of file path. 0 indicates that statistics\n");
- printf("# are condensed to refer to all files opened at the given\n");
- printf("# process.\n");
- printf("# <counter> and <value>: statistical counters.\n");
- printf("# A value of -1 indicates that Darshan could not monitor\n");
- printf("# that counter, and its value should be ignored.\n");
- printf("# <name suffix>: last %d characters of file name.\n", CP_NAME_SUFFIX_LEN);
- printf("# <mount pt>: mount point that the file resides on.\n");
- printf("# <fs type>: type of file system that the file resides on.\n");
- printf("\n# description of counters:\n");
- printf("# CP_POSIX_*: posix operation counts.\n");
- printf("# CP_COLL_*: MPI collective operation counts.\n");
- printf("# CP_INDEP_*: MPI independent operation counts.\n");
- printf("# CP_SPIT_*: MPI split collective operation counts.\n");
- printf("# CP_NB_*: MPI non blocking operation counts.\n");
- printf("# READS,WRITES,OPENS,SEEKS,STATS, and MMAPS are types of operations.\n");
- printf("# CP_*_NC_OPENS: number of indep. and collective pnetcdf opens.\n");
- printf("# CP_HDF5_OPENS: number of hdf5 opens.\n");
- printf("# CP_COMBINER_*: combiner counts for MPI mem and file datatypes.\n");
- printf("# CP_HINTS: number of times MPI hints were used.\n");
- printf("# CP_VIEWS: number of times MPI file views were used.\n");
- printf("# CP_MODE: mode that file was opened in.\n");
- printf("# CP_BYTES_*: total bytes read and written.\n");
- printf("# CP_MAX_BYTE_*: highest offset byte read and written.\n");
- printf("# CP_CONSEC_*: number of exactly adjacent reads and writes.\n");
- printf("# CP_SEQ_*: number of reads and writes from increasing offsets.\n");
- printf("# CP_RW_SWITCHES: number of times access alternated between read and write.\n");
- printf("# CP_*_ALIGNMENT: memory and file alignment.\n");
- printf("# CP_*_NOT_ALIGNED: number of reads and writes that were not aligned.\n");
- printf("# CP_MAX_*_TIME_SIZE: size of the slowest read and write operations.\n");
- printf("# CP_SIZE_READ_*: histogram of read access sizes.\n");
- printf("# CP_SIZE_READ_AGG_*: histogram of MPI datatype total sizes.\n");
- printf("# CP_EXTENT_READ_*: histogram of MPI datatype extents.\n");
- printf("# CP_STRIDE*_STRIDE: the four most common strides detected.\n");
- printf("# CP_STRIDE*_COUNT: count of the four most common strides.\n");
- printf("# CP_ACCESS*_ACCESS: the four most common access sizes.\n");
- printf("# CP_ACCESS*_COUNT: count of the four most common access sizes.\n");
- printf("# CP_DEVICE: File system identifier.\n");
- printf("# CP_SIZE_AT_OPEN: size of file when first opened.\n");
- printf("# CP_*_RANK_BYTES: fastest, slowest and variance of bytes transfer.\n");
- printf("# CP_F_OPEN_TIMESTAMP: timestamp of first open (mpi or posix).\n");
- printf("# CP_F_*_START_TIMESTAMP: timestamp of first read/write (mpi or posix).\n");
- printf("# CP_F_*_END_TIMESTAMP: timestamp of last read/write (mpi or posix).\n");
- printf("# CP_F_CLOSE_TIMESTAMP: timestamp of last close (mpi or posix).\n");
- printf("# CP_F_POSIX_READ/WRITE_TIME: cumulative time spent in posix reads or writes.\n");
- printf("# CP_F_MPI_READ/WRITE_TIME: cumulative time spent in mpi-io reads or writes.\n");
- printf("# CP_F_POSIX_META_TIME: cumulative time spent in posix open, close, fsync, stat and seek, .\n");
- printf("# CP_F_MPI_META_TIME: cumulative time spent in mpi-io open, close, set_view, and sync.\n");
- printf("# CP_MAX_*_TIME: duration of the slowest read and write operations.\n");
- printf("# CP_*_RANK_TIME: fastest, slowest variance of transfer time. Note that these counters show MPI-IO time for files accessed using MPI-IO, and POSIX time otherwise.\n");
-
- printf("\n");
- CP_PRINT_HEADER();
+ printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);
}
+ mod_buf = malloc(DARSHAN_DEF_COMP_BUF_SZ);
+ if(!mod_buf)
+ return(-1);
+
pdata.rank_cumul_io_time = malloc(sizeof(double)*job.nprocs);
pdata.rank_cumul_md_time = malloc(sizeof(double)*job.nprocs);
if (!pdata.rank_cumul_io_time || !pdata.rank_cumul_md_time)
{
- perror("malloc failed");
- darshan_log_close(file);
+ darshan_log_close(fd);
return(-1);
}
else
@@ -382,169 +354,285 @@ int main(int argc, char **argv)
memset(pdata.rank_cumul_md_time, 0, sizeof(double)*job.nprocs);
}
- do
+ for(i=0; i<DARSHAN_MAX_MODS; i++)
{
- char* mnt_pt = NULL;
- char* fs_type = NULL;
- hash_entry_t *hfile = NULL;
-
- if(cp_file.rank != -1 && cp_file.rank < last_rank)
+ int mod_bytes_left;
+ void *mod_buf_p;
+ void *rec_p = NULL;
+ darshan_record_id rec_id;
+ void *save_io, *save_md;
+
+ /* reset data structures for each module */
+ mod_buf_sz = DARSHAN_DEF_COMP_BUF_SZ;
+ memset(mod_buf, 0, DARSHAN_DEF_COMP_BUF_SZ);
+
+ /* check each module for any data */
+ ret = darshan_log_getmod(fd, i, mod_buf, &mod_buf_sz);
+ if(ret < 0)
{
- fprintf(stderr, "Error: log file contains out of order rank data.\n");
- fflush(stderr);
+ fprintf(stderr, "Error: failed to get module %s data\n",
+ darshan_module_names[i]);
+ darshan_log_close(fd);
return(-1);
}
- if(cp_file.rank != -1)
- last_rank = cp_file.rank;
-
- for(i=0; i<mount_count; i++)
+ else if(ret == 0)
{
- if(cp_file.counters[CP_DEVICE] == devs[i])
- {
- mnt_pt = mnt_pts[i];
- fs_type = fs_types[i];
- break;
- }
+ /* skip modules not present in log file */
+ empty_mods++;
+ continue;
+ }
+
+ /* skip modules with no defined logutil handlers */
+ if(!mod_logutils[i])
+ {
+ fprintf(stderr, "Warning: no log utility handlers defined "
+ "for module %s, SKIPPING\n", darshan_module_names[i]);
+ continue;
}
- if(!mnt_pt)
- mnt_pt = "UNKNOWN";
- if(!fs_type)
- fs_type = "UNKNOWN";
- HASH_FIND(hlink,file_hash,&cp_file.hash,sizeof(int64_t),hfile);
- if (!hfile)
+ printf("\n# *******************************************************\n");
+ printf("# %s module data\n", darshan_module_names[i]);
+ printf("# *******************************************************\n");
+
+ if(mask & OPTION_BASE)
{
- hfile = (hash_entry_t*) malloc(sizeof(*hfile));
- if (!hfile)
+ /* TODO: does each module print header of what each counter means??? */
+ DARSHAN_PRINT_HEADER();
+ }
+
+ /* this module has data to be parsed and printed */
+ mod_bytes_left = mod_buf_sz;
+ mod_buf_p = mod_buf;
+
+ /* loop over each of this module's records and print them */
+ while (mod_bytes_left > 0)
+ {
+ char *mnt_pt = NULL;
+ char *fs_type = NULL;
+ hash_entry_t *hfile = NULL;
+
+ ret = mod_logutils[i]->log_get_record(&mod_buf_p, &mod_bytes_left,
+ &rec_p, &rec_id, fd->swap_flag);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: failed to parse module %s data record\n",
+ darshan_module_names[i]);
+ darshan_log_close(fd);
+ return(-1);
+ }
+
+ /* get the pathname for this record */
+ HASH_FIND(hlink, rec_hash, &rec_id, sizeof(darshan_record_id), ref);
+ assert(ref);
+
+ /* get mount point and fs type associated with this record */
+ for(j=0; j<mount_count; j++)
+ {
+ if(strncmp(mnt_pts[j], ref->rec.name, strlen(mnt_pts[j])) == 0)
+ {
+ mnt_pt = mnt_pts[j];
+ fs_type = fs_types[j];
+ break;
+ }
+ }
+ if(!mnt_pt)
+ mnt_pt = "UNKNOWN";
+ if(!fs_type)
+ fs_type = "UNKNOWN";
+
+ if(mask & OPTION_BASE)
{
- fprintf(stderr,"malloc failure");
- exit(1);
+ /* print the corresponding module data for this record */
+ mod_logutils[i]->log_print_record(rec_p, ref->rec.name,
+ mnt_pt, fs_type);
}
- /* init */
- memset(hfile, 0, sizeof(*hfile));
- hfile->hash = cp_file.hash;
- memcpy(hfile->name_suffix, cp_file.name_suffix, CP_NAME_SUFFIX_LEN+1);
- hfile->type = 0;
- hfile->procs = 0;
- hfile->cumul_time = 0.0;
- hfile->meta_time = 0.0;
- hfile->slowest_time = 0.0;
-
- HASH_ADD(hlink,file_hash,hash,sizeof(int64_t),hfile);
+ /* we calculate more detailed stats for POSIX and MPI-IO modules,
+ * if the parser is executed with more than the base option
+ */
+ if(i != DARSHAN_POSIX_MOD && i != DARSHAN_MPIIO_MOD)
+ continue;
+
+ HASH_FIND(hlink, file_hash, &rec_id, sizeof(darshan_record_id), hfile);
+ if(!hfile)
+ {
+ hfile = malloc(sizeof(*hfile));
+ if(!hfile)
+ return(-1);
+
+ /* init */
+ memset(hfile, 0, sizeof(*hfile));
+ hfile->rec_id = rec_id;
+ hfile->type = 0;
+ hfile->procs = 0;
+ hfile->rec_dat = NULL;
+ hfile->cumul_time = 0.0;
+ hfile->slowest_time = 0.0;
+
+ HASH_ADD(hlink, file_hash, rec_id, sizeof(darshan_record_id), hfile);
+ }
+
+ if(i == DARSHAN_POSIX_MOD)
+ {
+ posix_accum_file((struct darshan_posix_file*)rec_p, &total, job.nprocs);
+ posix_accum_file((struct darshan_posix_file*)rec_p, hfile, job.nprocs);
+ posix_accum_perf((struct darshan_posix_file*)rec_p, &pdata);
+ }
+ else if(i == DARSHAN_MPIIO_MOD)
+ {
+ mpiio_accum_file((struct darshan_mpiio_file*)rec_p, &total, job.nprocs);
+ mpiio_accum_file((struct darshan_mpiio_file*)rec_p, hfile, job.nprocs);
+ mpiio_accum_perf((struct darshan_mpiio_file*)rec_p, &pdata);
+ }
}
- accum_file(&job, &cp_file, &total, NULL);
- accum_file(&job, &cp_file, hfile, &fdata);
- accum_perf(&cp_file, hfile, &pdata);
+ /* we calculate more detailed stats for POSIX and MPI-IO modules,
+ * if the parser is executed with more than the base option
+ */
+ if(i != DARSHAN_POSIX_MOD && i != DARSHAN_MPIIO_MOD)
+ continue;
- if ((mask & OPTION_BASE))
+ /* Total Calc */
+ if(mask & OPTION_TOTAL)
{
- for(i=0; i<CP_NUM_INDICES; i++)
+ if(i == DARSHAN_POSIX_MOD)
{
- CP_PRINT(&job, &cp_file, i, mnt_pt, fs_type);
+ posix_print_total_file((struct darshan_posix_file*)total.rec_dat);
}
- for(i=0; i<CP_F_NUM_INDICES; i++)
+ else if(i == DARSHAN_MPIIO_MOD)
{
- CP_F_PRINT(&job, &cp_file, i, mnt_pt, fs_type);
+ mpiio_print_total_file((struct darshan_mpiio_file*)total.rec_dat);
}
}
- }while((ret = darshan_log_getfile(file, &job, &cp_file)) == 1);
- /* Total Calc */
- if ((mask & OPTION_TOTAL))
- {
- for(i=0; i<CP_NUM_INDICES; i++)
+ /* File Calc */
+ if(mask & OPTION_FILE)
{
- printf("total_%s: %" PRId64 "\n",
- darshan_names[i], total.counters[i]);
+ if(i == DARSHAN_POSIX_MOD)
+ {
+ posix_calc_file(file_hash, &fdata);
+ }
+ else if(i == DARSHAN_MPIIO_MOD)
+ {
+ mpiio_calc_file(file_hash, &fdata);
+ }
+
+ printf("\n# files\n");
+ printf("# -----\n");
+ printf("# total: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.total,
+ fdata.total_size,
+ fdata.total_max);
+ printf("# read_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.read_only,
+ fdata.read_only_size,
+ fdata.read_only_max);
+ printf("# write_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.write_only,
+ fdata.write_only_size,
+ fdata.write_only_max);
+ printf("# read_write: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.read_write,
+ fdata.read_write_size,
+ fdata.read_write_max);
+ printf("# unique: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.unique,
+ fdata.unique_size,
+ fdata.unique_max);
+ printf("# shared: %" PRId64 " %" PRId64 " %" PRId64 "\n",
+ fdata.shared,
+ fdata.shared_size,
+ fdata.shared_max);
}
- for(i=0; i<CP_F_NUM_INDICES; i++)
+
+ /* Perf Calc */
+ if(mask & OPTION_PERF)
{
- printf("total_%s: %lf\n",
- darshan_f_names[i], total.fcounters[i]);
+ calc_perf(&pdata, job.nprocs);
+
+ printf("\n# performance\n");
+ printf("# -----------\n");
+ printf("# total_bytes: %" PRId64 "\n", pdata.total_bytes);
+ printf("#\n");
+ printf("# I/O timing for unique files (seconds):\n");
+ printf("# ...........................\n");
+ printf("# unique files: slowest_rank_io_time: %lf\n", pdata.slowest_rank_time);
+ printf("# unique files: slowest_rank_meta_time: %lf\n", pdata.slowest_rank_meta_time);
+ printf("# unique files: slowest rank: %d\n", pdata.slowest_rank_rank);
+ printf("#\n");
+ printf("# I/O timing for shared files (seconds):\n");
+ printf("# (multiple estimates shown; time_by_slowest is generally the most accurate)\n");
+ printf("# ...........................\n");
+ printf("# shared files: time_by_cumul_io_only: %lf\n", pdata.shared_time_by_cumul);
+ printf("# shared files: time_by_cumul_meta_only: %lf\n", pdata.shared_meta_time);
+ printf("# shared files: time_by_open: %lf\n", pdata.shared_time_by_open);
+ printf("# shared files: time_by_open_lastio: %lf\n", pdata.shared_time_by_open_lastio);
+ printf("# shared files: time_by_slowest: %lf\n", pdata.shared_time_by_slowest);
+ printf("#\n");
+ printf("# Aggregate performance, including both shared and unique files (MiB/s):\n");
+ printf("# (multiple estimates shown; agg_perf_by_slowest is generally the most accurate)\n");
+ printf("# ...........................\n");
+ printf("# agg_perf_by_cumul: %lf\n", pdata.agg_perf_by_cumul);
+ printf("# agg_perf_by_open: %lf\n", pdata.agg_perf_by_open);
+ printf("# agg_perf_by_open_lastio: %lf\n", pdata.agg_perf_by_open_lastio);
+ printf("# agg_perf_by_slowest: %lf\n", pdata.agg_perf_by_slowest);
}
- }
- /* Perf Calc */
- calc_perf(&job, file_hash, &pdata);
- if ((mask & OPTION_PERF))
- {
- printf("\n# performance\n");
- printf("# -----------\n");
- printf("# total_bytes: %" PRId64 "\n", pdata.total_bytes);
- printf("#\n");
- printf("# I/O timing for unique files (seconds):\n");
- printf("# ...........................\n");
- printf("# unique files: slowest_rank_io_time: %lf\n", pdata.slowest_rank_time);
- printf("# unique files: slowest_rank_meta_time: %lf\n", pdata.slowest_rank_meta_time);
- printf("# unique files: slowest rank: %d\n", pdata.slowest_rank_rank);
- printf("#\n");
- printf("# I/O timing for shared files (seconds):\n");
- printf("# (multiple estimates shown; time_by_slowest is generally the most accurate)\n");
- printf("# ...........................\n");
- printf("# shared files: time_by_cumul_io_only: %lf\n", pdata.shared_time_by_cumul);
- printf("# shared files: time_by_cumul_meta_only: %lf\n", pdata.shared_meta_time);
- printf("# shared files: time_by_open: %lf\n", pdata.shared_time_by_open);
- printf("# shared files: time_by_open_lastio: %lf\n", pdata.shared_time_by_open_lastio);
- printf("# shared files: time_by_slowest: %lf\n", pdata.shared_time_by_slowest);
- printf("#\n");
- printf("# Aggregate performance, including both shared and unique files (MiB/s):\n");
- printf("# (multiple estimates shown; agg_perf_by_slowest is generally the most accurate)\n");
- printf("# ...........................\n");
- printf("# agg_perf_by_cumul: %lf\n", pdata.agg_perf_by_cumul);
- printf("# agg_perf_by_open: %lf\n", pdata.agg_perf_by_open);
- printf("# agg_perf_by_open_lastio: %lf\n", pdata.agg_perf_by_open_lastio);
- printf("# agg_perf_by_slowest: %lf\n", pdata.agg_perf_by_slowest);
- }
+ if((mask & OPTION_FILE_LIST) || (mask & OPTION_FILE_LIST_DETAILED))
+ {
+ if(i == DARSHAN_POSIX_MOD)
+ {
+ if(mask & OPTION_FILE_LIST_DETAILED)
+ posix_file_list(file_hash, rec_hash, 1);
+ else
+ posix_file_list(file_hash, rec_hash, 0);
+ }
+ else if(i == DARSHAN_MPIIO_MOD)
+ {
+ if(mask & OPTION_FILE_LIST_DETAILED)
+ mpiio_file_list(file_hash, rec_hash, 1);
+ else
+ mpiio_file_list(file_hash, rec_hash, 0);
+ }
+ }
- /* File Calc */
- calc_file(&job, file_hash, &fdata);
- if ((mask & OPTION_FILE))
- {
- printf("\n# files\n");
- printf("# -----\n");
- printf("# total: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.total,
- fdata.total_size,
- fdata.total_max);
- printf("# read_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.read_only,
- fdata.read_only_size,
- fdata.read_only_max);
- printf("# write_only: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.write_only,
- fdata.write_only_size,
- fdata.write_only_max);
- printf("# read_write: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.read_write,
- fdata.read_write_size,
- fdata.read_write_max);
- printf("# unique: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.unique,
- fdata.unique_size,
- fdata.unique_max);
- printf("# shared: %" PRId64 " %" PRId64 " %" PRId64 "\n",
- fdata.shared,
- fdata.shared_size,
- fdata.shared_max);
+ /* reset data structures for next module */
+ if(total.rec_dat) free(total.rec_dat);
+ memset(&total, 0, sizeof(total));
+ memset(&fdata, 0, sizeof(fdata));
+ save_io = pdata.rank_cumul_io_time;
+ save_md = pdata.rank_cumul_md_time;
+ memset(&pdata, 0, sizeof(pdata));
+ memset(save_io, 0, sizeof(double)*job.nprocs);
+ memset(save_md, 0, sizeof(double)*job.nprocs);
+ pdata.rank_cumul_io_time = save_io;
+ pdata.rank_cumul_md_time = save_md;
+
+ HASH_ITER(hlink, file_hash, curr, tmp_file)
+ {
+ HASH_DELETE(hlink, file_hash, curr);
+ if(curr->rec_dat) free(curr->rec_dat);
+ free(curr);
+ }
}
+ if(empty_mods == DARSHAN_MAX_MODS)
+ printf("\n# no module data available.\n");
- if ((mask & OPTION_FILE_LIST) || mask & OPTION_FILE_LIST_DETAILED)
- {
- if(mask & OPTION_FILE_LIST_DETAILED)
- file_list(&job, file_hash, 1);
- else
- file_list(&job, file_hash, 0);
- }
+ darshan_log_close(fd);
- if(ret < 0)
+ free(mod_buf);
+ free(pdata.rank_cumul_io_time);
+ free(pdata.rank_cumul_md_time);
+
+ /* free record hash data */
+ HASH_ITER(hlink, rec_hash, ref, tmp_ref)
{
- fprintf(stderr, "Error: failed to parse log file.\n");
- fflush(stderr);
- return(-1);
+ HASH_DELETE(hlink, rec_hash, ref);
+ free(ref->rec.name);
+ free(ref);
}
+ /* free mount info */
for(i=0; i<mount_count; i++)
{
free(mnt_pts[i]);
@@ -552,75 +640,44 @@ int main(int argc, char **argv)
}
if(mount_count > 0)
{
- free(devs);
free(mnt_pts);
free(fs_types);
}
-
- darshan_log_close(file);
-
- HASH_ITER(hlink, file_hash, curr, tmp)
- {
- HASH_DELETE(hlink, file_hash, curr);
- free(curr);
- }
return(0);
}
-void accum_file(struct darshan_job *job,
- struct darshan_file *dfile,
- hash_entry_t *hfile,
- file_data_t *fdata)
+void posix_accum_file(struct darshan_posix_file *pfile,
+ hash_entry_t *hfile,
+ int64_t nprocs)
{
- int i;
+ int i, j;
+ int set;
+ int min_ndx;
+ int64_t min;
+ struct darshan_posix_file* tmp;
hfile->procs += 1;
- if (dfile->rank == -1)
+ if(pfile->rank == -1)
{
- if(job->version_string[0] == '1')
- {
- hfile->slowest_time =
- max((dfile->fcounters[CP_F_READ_END_TIMESTAMP]
- - dfile->fcounters[CP_F_OPEN_TIMESTAMP]),
- (dfile->fcounters[CP_F_WRITE_END_TIMESTAMP]
- - dfile->fcounters[CP_F_OPEN_TIMESTAMP]));
- if(hfile->slowest_time < 0)
- hfile->slowest_time = 0;
- }
- else
- {
- hfile->slowest_time = dfile->fcounters[CP_F_SLOWEST_RANK_TIME];
- }
+ hfile->slowest_time = pfile->fcounters[POSIX_F_SLOWEST_RANK_TIME];
}
else
{
- if(dfile->counters[CP_INDEP_OPENS] || dfile->counters[CP_COLL_OPENS])
- {
- /* MPI file */
- hfile->slowest_time = max(hfile->slowest_time,
- (dfile->fcounters[CP_F_MPI_META_TIME] +
- dfile->fcounters[CP_F_MPI_READ_TIME] +
- dfile->fcounters[CP_F_MPI_WRITE_TIME]));
- }
- else
- {
- /* POSIX file */
- hfile->slowest_time = max(hfile->slowest_time,
- (dfile->fcounters[CP_F_POSIX_META_TIME] +
- dfile->fcounters[CP_F_POSIX_READ_TIME] +
- dfile->fcounters[CP_F_POSIX_WRITE_TIME]));
- }
+ hfile->slowest_time = max(hfile->slowest_time,
+ (pfile->fcounters[POSIX_F_META_TIME] +
+ pfile->fcounters[POSIX_F_READ_TIME] +
+ pfile->fcounters[POSIX_F_WRITE_TIME]));
}
- if (dfile->rank == -1)
+ if(pfile->rank == -1)
{
- hfile->procs = job->nprocs;
+ hfile->procs = nprocs;
hfile->type |= FILETYPE_SHARED;
}
- else if (hfile->procs > 1)
+ else if(hfile->procs > 1)
{
hfile->type &= (~FILETYPE_UNIQUE);
hfile->type |= FILETYPE_PARTSHARED;
@@ -630,137 +687,163 @@ void accum_file(struct darshan_job *job,
hfile->type |= FILETYPE_UNIQUE;
}
- if(dfile->counters[CP_INDEP_OPENS] || dfile->counters[CP_COLL_OPENS])
- {
- hfile->cumul_time += dfile->fcounters[CP_F_MPI_META_TIME] +
- dfile->fcounters[CP_F_MPI_READ_TIME] +
- dfile->fcounters[CP_F_MPI_WRITE_TIME];
- }
- else
+ hfile->cumul_time += pfile->fcounters[POSIX_F_META_TIME] +
+ pfile->fcounters[POSIX_F_READ_TIME] +
+ pfile->fcounters[POSIX_F_WRITE_TIME];
+
+ if(hfile->rec_dat == NULL)
{
- hfile->cumul_time += dfile->fcounters[CP_F_POSIX_META_TIME] +
- dfile->fcounters[CP_F_POSIX_READ_TIME] +
- dfile->fcounters[CP_F_POSIX_WRITE_TIME];
+ hfile->rec_dat = malloc(sizeof(struct darshan_posix_file));
+ assert(hfile->rec_dat);
+ memset(hfile->rec_dat, 0, sizeof(struct darshan_posix_file));
}
+ tmp = (struct darshan_posix_file*)hfile->rec_dat;
- for (i = 0; i < CP_NUM_INDICES; i++)
+ for(i = 0; i < POSIX_NUM_INDICES; i++)
{
switch(i)
{
- case CP_DEVICE:
- case CP_MODE:
- case CP_MEM_ALIGNMENT:
- case CP_FILE_ALIGNMENT:
- if(CP_FILE_PARTIAL(hfile))
- hfile->counters[i] = dfile->counters[i];
+ case POSIX_MODE:
+ case POSIX_MEM_ALIGNMENT:
+ case POSIX_FILE_ALIGNMENT:
+ if(POSIX_FILE_PARTIAL(tmp))
+ tmp->counters[i] = pfile->counters[i];
break;
- case CP_SIZE_AT_OPEN:
- if (hfile->counters[i] == -1)
- {
- hfile->counters[i] = dfile->counters[i];
- }
- if (hfile->counters[i] > dfile->counters[i] && !CP_FILE_PARTIAL(dfile))
+ case POSIX_MAX_BYTE_READ:
+ case POSIX_MAX_BYTE_WRITTEN:
+ if (tmp->counters[i] < pfile->counters[i])
{
- hfile->counters[i] = dfile->counters[i];
+ tmp->counters[i] = pfile->counters[i];
}
break;
- case CP_MAX_BYTE_READ:
- case CP_MAX_BYTE_WRITTEN:
- if (hfile->counters[i] < dfile->counters[i])
- {
- hfile->counters[i] = dfile->counters[i];
- }
- break;
-
- case CP_STRIDE1_STRIDE:
- case CP_STRIDE2_STRIDE:
- case CP_STRIDE3_STRIDE:
- case CP_STRIDE4_STRIDE:
- case CP_ACCESS1_ACCESS:
- case CP_ACCESS2_ACCESS:
- case CP_ACCESS3_ACCESS:
- case CP_ACCESS4_ACCESS:
+ case POSIX_STRIDE1_STRIDE:
+ case POSIX_STRIDE2_STRIDE:
+ case POSIX_STRIDE3_STRIDE:
+ case POSIX_STRIDE4_STRIDE:
+ case POSIX_ACCESS1_ACCESS:
+ case POSIX_ACCESS2_ACCESS:
+ case POSIX_ACCESS3_ACCESS:
+ case POSIX_ACCESS4_ACCESS:
/*
* do nothing here because these will be stored
* when the _COUNT is accessed.
*/
break;
-
- case CP_STRIDE1_COUNT:
- case CP_STRIDE2_COUNT:
- case CP_STRIDE3_COUNT:
- case CP_STRIDE4_COUNT:
- case CP_ACCESS1_COUNT:
- case CP_ACCESS2_COUNT:
- case CP_ACCESS3_COUNT:
- case CP_ACCESS4_COUNT:
- if (hfile->counters[i] < dfile->counters[i])
+ case POSIX_STRIDE1_COUNT:
+ case POSIX_STRIDE2_COUNT:
+ case POSIX_STRIDE3_COUNT:
+ case POSIX_STRIDE4_COUNT:
+ set = 0;
+ min_ndx = POSIX_STRIDE1_COUNT;
+ min = tmp->counters[min_ndx];
+ for(j = POSIX_STRIDE1_COUNT; j <= POSIX_STRIDE4_COUNT; j++)
+ {
+ if(tmp->counters[j-4] == pfile->counters[i-4])
+ {
+ tmp->counters[j] += pfile->counters[i];
+ set = 1;
+ break;
+ }
+ if(tmp->counters[j] < min)
+ {
+ min_ndx = j;
+ min = tmp->counters[j];
+ }
+ }
+ if(!set && (pfile->counters[i] > min))
{
- hfile->counters[i] = dfile->counters[i];
- hfile->counters[i-4] = dfile->counters[i-4];
+ tmp->counters[min_ndx] = pfile->counters[i];
+ tmp->counters[min_ndx-4] = pfile->counters[i-4];
}
break;
- case CP_FASTEST_RANK:
- case CP_SLOWEST_RANK:
- case CP_FASTEST_RANK_BYTES:
- case CP_SLOWEST_RANK_BYTES:
- hfile->counters[i] = 0;
+ case POSIX_ACCESS1_COUNT:
+ case POSIX_ACCESS2_COUNT:
+ case POSIX_ACCESS3_COUNT:
+ case POSIX_ACCESS4_COUNT:
+ set = 0;
+ min_ndx = POSIX_ACCESS1_COUNT;
+ min = tmp->counters[min_ndx];
+ for(j = POSIX_ACCESS1_COUNT; j <= POSIX_ACCESS4_COUNT; j++)
+ {
+ if(tmp->counters[j-4] == pfile->counters[i-4])
+ {
+ tmp->counters[j] += pfile->counters[i];
+ set = 1;
+ break;
+ }
+ if(tmp->counters[j] < min)
+ {
+ min_ndx = j;
+ min = tmp->counters[j];
+ }
+ }
+ if(!set && (pfile->counters[i] > min))
+ {
+ tmp->counters[i] = pfile->counters[i];
+ tmp->counters[i-4] = pfile->counters[i-4];
+ }
break;
- case CP_MAX_READ_TIME_SIZE:
- case CP_MAX_WRITE_TIME_SIZE:
+ case POSIX_FASTEST_RANK:
+ case POSIX_SLOWEST_RANK:
+ case POSIX_FASTEST_RANK_BYTES:
+ case POSIX_SLOWEST_RANK_BYTES:
+ tmp->counters[i] = 0;
+ break;
+ case POSIX_MAX_READ_TIME_SIZE:
+ case POSIX_MAX_WRITE_TIME_SIZE:
break;
default:
- hfile->counters[i] += dfile->counters[i];
+ tmp->counters[i] += pfile->counters[i];
break;
}
}
- for (i = 0; i < CP_F_NUM_INDICES; i++)
+ for(i = 0; i < POSIX_F_NUM_INDICES; i++)
{
switch(i)
{
- case CP_F_OPEN_TIMESTAMP:
- case CP_F_READ_START_TIMESTAMP:
- case CP_F_WRITE_START_TIMESTAMP:
- if(hfile->fcounters[i] == 0 ||
- hfile->fcounters[i] > dfile->fcounters[i])
+ case POSIX_F_OPEN_TIMESTAMP:
+ case POSIX_F_READ_START_TIMESTAMP:
+ case POSIX_F_WRITE_START_TIMESTAMP:
+ if(tmp->fcounters[i] == 0 ||
+ tmp->fcounters[i] > pfile->fcounters[i])
{
- hfile->fcounters[i] = dfile->fcounters[i];
+ tmp->fcounters[i] = pfile->fcounters[i];
}
break;
- case CP_F_CLOSE_TIMESTAMP:
- case CP_F_READ_END_TIMESTAMP:
- case CP_F_WRITE_END_TIMESTAMP:
- if(hfile->fcounters[i] == 0 ||
- hfile->fcounters[i] < dfile->fcounters[i])
+ case POSIX_F_READ_END_TIMESTAMP:
+ case POSIX_F_WRITE_END_TIMESTAMP:
+ case POSIX_F_CLOSE_TIMESTAMP:
+ if(tmp->fcounters[i] == 0 ||
+ tmp->fcounters[i] < pfile->fcounters[i])
{
- hfile->fcounters[i] = dfile->fcounters[i];
+ tmp->fcounters[i] = pfile->fcounters[i];
}
break;
- case CP_F_FASTEST_RANK_TIME:
- case CP_F_SLOWEST_RANK_TIME:
- case CP_F_VARIANCE_RANK_TIME:
- case CP_F_VARIANCE_RANK_BYTES:
- hfile->fcounters[i] = 0;
+ case POSIX_F_FASTEST_RANK_TIME:
+ case POSIX_F_SLOWEST_RANK_TIME:
+ case POSIX_F_VARIANCE_RANK_TIME:
+ case POSIX_F_VARIANCE_RANK_BYTES:
+ tmp->fcounters[i] = 0;
break;
- case CP_F_MAX_READ_TIME:
- if (hfile->fcounters[i] > dfile->fcounters[i])
+ case POSIX_F_MAX_READ_TIME:
+ if(tmp->fcounters[i] < pfile->fcounters[i])
{
- hfile->fcounters[i] = dfile->fcounters[i];
- hfile->counters[CP_MAX_READ_TIME_SIZE] =
- dfile->counters[CP_MAX_READ_TIME_SIZE];
+ tmp->fcounters[i] = pfile->fcounters[i];
+ tmp->counters[POSIX_MAX_READ_TIME_SIZE] =
+ pfile->counters[POSIX_MAX_READ_TIME_SIZE];
}
break;
- case CP_F_MAX_WRITE_TIME:
- if (hfile->fcounters[i] > dfile->fcounters[i])
+ case POSIX_F_MAX_WRITE_TIME:
+ if(tmp->fcounters[i] < pfile->fcounters[i])
{
- hfile->fcounters[i] = dfile->fcounters[i];
- hfile->counters[CP_MAX_WRITE_TIME_SIZE] =
- dfile->counters[CP_MAX_WRITE_TIME_SIZE];
+ tmp->fcounters[i] = pfile->fcounters[i];
+ tmp->counters[POSIX_MAX_WRITE_TIME_SIZE] =
+ pfile->counters[POSIX_MAX_WRITE_TIME_SIZE];
}
break;
default:
- hfile->fcounters[i] += dfile->fcounters[i];
+ tmp->fcounters[i] += pfile->fcounters[i];
break;
}
}
@@ -768,187 +851,255 @@ void accum_file(struct darshan_job *job,
return;
}
-void file_list(struct darshan_job *djob, hash_entry_t *file_hash, int detail_flag)
+void mpiio_accum_file(struct darshan_mpiio_file *mfile,
+ hash_entry_t *hfile,
+ int64_t nprocs)
{
- hash_entry_t *curr = NULL;
- hash_entry_t *tmp = NULL;
- char* type;
- int i;
+ int i, j;
+ int set;
+ int min_ndx;
+ int64_t min;
+ struct darshan_mpiio_file* tmp;
- /* TODO: list of columns:
- *
- * normal mode?
- * - hash
- * - suffix
- * - MPI or POSIX
- * - nprocs
- * - slowest I/O time
- * - average cumulative I/O time
- *
- * detailed mode?
- * - first open
- * - first read
- * - first write
- * - last close
- * - last read
- * - last write
- * - MPI indep opens
- * - MPI coll opens
- * - POSIX opens
- * - r histogram (POSIX)
- * - w histogram (POSIX)
- */
+ hfile->procs += 1;
- if(detail_flag)
- printf("\n# Per-file summary of I/O activity (detailed).\n");
+ if(mfile->rank == -1)
+ {
+ hfile->slowest_time = mfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];
+ }
else
- printf("\n# Per-file summary of I/O activity.\n");
+ {
+ hfile->slowest_time = max(hfile->slowest_time,
+ (mfile->fcounters[MPIIO_F_META_TIME] +
+ mfile->fcounters[MPIIO_F_READ_TIME] +
+ mfile->fcounters[MPIIO_F_WRITE_TIME]));
+ }
- printf("# <hash>: hash of file name\n");
- printf("# <suffix>: last %d characters of file name\n", CP_NAME_SUFFIX_LEN);
- printf("# <type>: MPI or POSIX\n");
- printf("# <nprocs>: number of processes that opened the file\n");
- printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");
- printf("# <avg>: average time in seconds consumed in IO per process\n");
- if(detail_flag)
+ if(mfile->rank == -1)
{
- printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");
- printf("# <end_{open/read/write}>: end timestamp of last open, read, or write\n");
- printf("# <mpi_indep_opens>: independent MPI_File_open calls\n");
- printf("# <mpi_coll_opens>: collective MPI_File_open calls\n");
- printf("# <posix_opens>: POSIX open calls\n");
- printf("# <CP_SIZE_READ_*>: POSIX read size histogram\n");
- printf("# <CP_SIZE_WRITE_*>: POSIX write size histogram\n");
+ hfile->procs = nprocs;
+ hfile->type |= FILETYPE_SHARED;
+
}
-
- printf("\n# <hash>\t<suffix>\t<type>\t<nprocs>\t<slowest>\t<avg>");
- if(detail_flag)
+ else if(hfile->procs > 1)
{
- printf("\t<start_open>\t<start_read>\t<start_write>");
- printf("\t<end_open>\t<end_read>\t<end_write>");
- printf("\t<mpi_indep_opens>\t<mpi_coll_opens>\t<posix_opens>");
- for(i=CP_SIZE_READ_0_100; i<= CP_SIZE_WRITE_1G_PLUS; i++)
- printf("\t%s", darshan_names[i]);
+ hfile->type &= (~FILETYPE_UNIQUE);
+ hfile->type |= FILETYPE_PARTSHARED;
+ }
+ else
+ {
+ hfile->type |= FILETYPE_UNIQUE;
}
- printf("\n");
- HASH_ITER(hlink, file_hash, curr, tmp)
+ hfile->cumul_time += mfile->fcounters[MPIIO_F_META_TIME] +
+ mfile->fcounters[MPIIO_F_READ_TIME] +
+ mfile->fcounters[MPIIO_F_WRITE_TIME];
+
+ if(hfile->rec_dat == NULL)
{
- if(curr->counters[CP_INDEP_OPENS] || curr->counters[CP_COLL_OPENS])
- type = "MPI";
- else
- type = "POSIX";
+ hfile->rec_dat = malloc(sizeof(struct darshan_mpiio_file));
+ assert(hfile->rec_dat);
+ memset(hfile->rec_dat, 0, sizeof(struct darshan_mpiio_file));
+ }
+ tmp = (struct darshan_mpiio_file*)hfile->rec_dat;
- printf("%" PRIu64 "\t%s\t%s\t%" PRId64 "\t%f\t%f",
- curr->hash,
- curr->name_suffix,
- type,
- curr->procs,
- curr->slowest_time,
- curr->cumul_time/(double)curr->procs);
- if(detail_flag)
+ for(i = 0; i < MPIIO_NUM_INDICES; i++)
+ {
+ switch(i)
{
- for(i=CP_F_OPEN_TIMESTAMP; i<=CP_F_WRITE_END_TIMESTAMP; i++)
+ case MPIIO_MODE:
+ tmp->counters[i] = mfile->counters[i];
+ break;
+ case MPIIO_ACCESS1_ACCESS:
+ case MPIIO_ACCESS2_ACCESS:
+ case MPIIO_ACCESS3_ACCESS:
+ case MPIIO_ACCESS4_ACCESS:
+ /*
+ * do nothing here because these will be stored
+ * when the _COUNT is accessed.
+ */
+ break;
+ case MPIIO_ACCESS1_COUNT:
+ case MPIIO_ACCESS2_COUNT:
+ case MPIIO_ACCESS3_COUNT:
+ case MPIIO_ACCESS4_COUNT:
+ set = 0;
+ min_ndx = MPIIO_ACCESS1_COUNT;
+ min = tmp->counters[min_ndx];
+ for(j = MPIIO_ACCESS1_COUNT; j <= MPIIO_ACCESS4_COUNT; j++)
{
- printf("\t%f", curr->fcounters[i]);
+ if(tmp->counters[j-4] == mfile->counters[i-4])
+ {
+ tmp->counters[j] += mfile->counters[i];
+ set = 1;
+ break;
+ }
+ if(tmp->counters[j] < min)
+ {
+ min_ndx = j;
+ min = tmp->counters[j];
+ }
}
- printf("\t%" PRId64 "\t%" PRId64 "\t%" PRId64, curr->counters[CP_INDEP_OPENS], curr->counters[CP_COLL_OPENS], curr->counters[CP_POSIX_OPENS]);
- for(i=CP_SIZE_READ_0_100; i<= CP_SIZE_WRITE_1G_PLUS; i++)
- printf("\t%" PRId64, curr->counters[i]);
+ if(!set && (mfile->counters[i] > min))
+ {
+ tmp->counters[i] = mfile->counters[i];
+ tmp->counters[i-4] = mfile->counters[i-4];
+ }
+ break;
+ case MPIIO_FASTEST_RANK:
+ case MPIIO_SLOWEST_RANK:
+ case MPIIO_FASTEST_RANK_BYTES:
+ case MPIIO_SLOWEST_RANK_BYTES:
+ tmp->counters[i] = 0;
+ break;
+ case MPIIO_MAX_READ_TIME_SIZE:
+ case MPIIO_MAX_WRITE_TIME_SIZE:
+ break;
+ default:
+ tmp->counters[i] += mfile->counters[i];
+ break;
+ }
+ }
+
+ for(i = 0; i < MPIIO_F_NUM_INDICES; i++)
+ {
+ switch(i)
+ {
+ case MPIIO_F_OPEN_TIMESTAMP:
+ case MPIIO_F_READ_START_TIMESTAMP:
+ case MPIIO_F_WRITE_START_TIMESTAMP:
+ if(tmp->fcounters[i] == 0 ||
+ tmp->fcounters[i] > mfile->fcounters[i])
+ {
+ tmp->fcounters[i] = mfile->fcounters[i];
+ }
+ break;
+ case MPIIO_F_READ_END_TIMESTAMP:
+ case MPIIO_F_WRITE_END_TIMESTAMP:
+ case MPIIO_F_CLOSE_TIMESTAMP:
+ if(tmp->fcounters[i] == 0 ||
+ tmp->fcounters[i] < mfile->fcounters[i])
+ {
+ tmp->fcounters[i] = mfile->fcounters[i];
+ }
+ break;
+ case MPIIO_F_FASTEST_RANK_TIME:
+ case MPIIO_F_SLOWEST_RANK_TIME:
+ case MPIIO_F_VARIANCE_RANK_TIME:
+ case MPIIO_F_VARIANCE_RANK_BYTES:
+ tmp->fcounters[i] = 0;
+ break;
+ case MPIIO_F_MAX_READ_TIME:
+ if(tmp->fcounters[i] < mfile->fcounters[i])
+ {
+ tmp->fcounters[i] = mfile->fcounters[i];
+ tmp->counters[MPIIO_MAX_READ_TIME_SIZE] =
+ mfile->counters[MPIIO_MAX_READ_TIME_SIZE];
+ }
+ break;
+ case MPIIO_F_MAX_WRITE_TIME:
+ if(tmp->fcounters[i] < mfile->fcounters[i])
+ {
+ tmp->fcounters[i] = mfile->fcounters[i];
+ tmp->counters[MPIIO_MAX_WRITE_TIME_SIZE] =
+ mfile->counters[MPIIO_MAX_WRITE_TIME_SIZE];
+ }
+ break;
+ default:
+ tmp->fcounters[i] += mfile->fcounters[i];
+ break;
}
- printf("\n");
}
return;
}
-void calc_file(struct darshan_job *djob,
- hash_entry_t *file_hash,
- file_data_t *fdata)
+void posix_accum_perf(struct darshan_posix_file *pfile,
+ perf_data_t *pdata)
{
- hash_entry_t *curr = NULL;
- hash_entry_t *tmp = NULL;
+ pdata->total_bytes += pfile->counters[POSIX_BYTES_READ] +
+ pfile->counters[POSIX_BYTES_WRITTEN];
- memset(fdata, 0, sizeof(*fdata));
-
- HASH_ITER(hlink, file_hash, curr, tmp)
+ /*
+ * Calculation of Shared File Time
+ * Four Methods!!!!
+ * by_cumul: sum time counters and divide by nprocs
+ * (inaccurate if lots of variance between procs)
+ * by_open: difference between timestamp of open and close
+ * (inaccurate if file is left open without i/o happening)
+ * by_open_lastio: difference between timestamp of open and the
+ * timestamp of last i/o
+ * (similar to above but fixes case where file is left
+ * open after io is complete)
+ * by_slowest: use slowest rank time from log data
+ * (most accurate but requires newer log version)
+ */
+ if(pfile->rank == -1)
{
- int64_t max;
- int64_t r;
- int64_t w;
-
- max = max3(curr->counters[CP_SIZE_AT_OPEN],
- curr->counters[CP_MAX_BYTE_READ],
- curr->counters[CP_MAX_BYTE_WRITTEN]);
-
- r = (curr->counters[CP_POSIX_READS]+
- curr->counters[CP_POSIX_FREADS]+
- curr->counters[CP_INDEP_READS]+
- curr->counters[CP_COLL_READS]+
- curr->counters[CP_SPLIT_READS]+
- curr->counters[CP_NB_READS]);
-
- w = (curr->counters[CP_POSIX_WRITES]+
- curr->counters[CP_POSIX_FWRITES]+
- curr->counters[CP_INDEP_WRITES]+
- curr->counters[CP_COLL_WRITES]+
- curr->counters[CP_SPLIT_WRITES]+
- curr->counters[CP_NB_WRITES]);
-
- fdata->total += 1;
- fdata->total_size += max;
- fdata->total_max = max(fdata->total_max, max);
-
- if (r && !w)
+ /* by_open */
+ if(pfile->fcounters[POSIX_F_CLOSE_TIMESTAMP] >
+ pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])
{
- fdata->read_only += 1;
- fdata->read_only_size += max;
- fdata->read_only_max = max(fdata->read_only_max, max);
+ pdata->shared_time_by_open +=
+ pfile->fcounters[POSIX_F_CLOSE_TIMESTAMP] -
+ pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];
}
- if (!r && w)
+ /* by_open_lastio */
+ if(pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] >
+ pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP])
{
- fdata->write_only += 1;
- fdata->write_only_size += max;
- fdata->write_only_max = max(fdata->write_only_max, max);
+ /* be careful: file may have been opened but not read or written */
+ if(pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] > pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])
+ {
+ pdata->shared_time_by_open_lastio +=
+ pfile->fcounters[POSIX_F_READ_END_TIMESTAMP] -
+ pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];
+ }
}
-
- if (r && w)
+ else
{
- fdata->read_write += 1;
- fdata->read_write_size += max;
- fdata->read_write_max = max(fdata->read_write_max, max);
+ /* be careful: file may have been opened but not read or written */
+ if(pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP] > pfile->fcounters[POSIX_F_OPEN_TIMESTAMP])
+ {
+ pdata->shared_time_by_open_lastio +=
+ pfile->fcounters[POSIX_F_WRITE_END_TIMESTAMP] -
+ pfile->fcounters[POSIX_F_OPEN_TIMESTAMP];
+ }
}
- if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))
- {
- fdata->shared += 1;
- fdata->shared_size += max;
- fdata->shared_max = max(fdata->shared_max, max);
- }
+ pdata->shared_time_by_cumul +=
+ pfile->fcounters[POSIX_F_META_TIME] +
+ pfile->fcounters[POSIX_F_READ_TIME] +
+ pfile->fcounters[POSIX_F_WRITE_TIME];
+ pdata->shared_meta_time += pfile->fcounters[POSIX_F_META_TIME];
- if ((curr->type & (FILETYPE_UNIQUE)))
- {
- fdata->unique += 1;
- fdata->unique_size += max;
- fdata->unique_max = max(fdata->unique_max, max);
- }
+ /* by_slowest */
+ pdata->shared_time_by_slowest +=
+ pfile->fcounters[POSIX_F_SLOWEST_RANK_TIME];
+ }
+
+ /*
+ * Calculation of Unique File Time
+ * record the data for each file and sum it
+ */
+ else
+ {
+ pdata->rank_cumul_io_time[pfile->rank] +=
+ (pfile->fcounters[POSIX_F_META_TIME] +
+ pfile->fcounters[POSIX_F_READ_TIME] +
+ pfile->fcounters[POSIX_F_WRITE_TIME]);
+ pdata->rank_cumul_md_time[pfile->rank] += pfile->fcounters[POSIX_F_META_TIME];
}
return;
}
-void accum_perf(struct darshan_file *dfile,
- hash_entry_t *hfile,
- perf_data_t *pdata)
+void mpiio_accum_perf(struct darshan_mpiio_file *mfile,
+ perf_data_t *pdata)
{
- int64_t mpi_file;
-
- pdata->total_bytes += dfile->counters[CP_BYTES_READ] +
- dfile->counters[CP_BYTES_WRITTEN];
-
- mpi_file = dfile->counters[CP_INDEP_OPENS] +
- dfile->counters[CP_COLL_OPENS];
+ pdata->total_bytes += mfile->counters[MPIIO_BYTES_READ] +
+ mfile->counters[MPIIO_BYTES_WRITTEN];
/*
* Calculation of Shared File Time
@@ -964,61 +1115,49 @@ void accum_perf(struct darshan_file *dfile,
* by_slowest: use slowest rank time from log data
* (most accurate but requires newer log version)
*/
- if (dfile->rank == -1)
+ if(mfile->rank == -1)
{
- /* by_open (same for MPI or POSIX) */
- if (dfile->fcounters[CP_F_CLOSE_TIMESTAMP] >
- dfile->fcounters[CP_F_OPEN_TIMESTAMP])
+ /* by_open */
+ if(mfile->fcounters[MPIIO_F_CLOSE_TIMESTAMP] >
+ mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])
{
pdata->shared_time_by_open +=
- dfile->fcounters[CP_F_CLOSE_TIMESTAMP] -
- dfile->fcounters[CP_F_OPEN_TIMESTAMP];
+ mfile->fcounters[MPIIO_F_CLOSE_TIMESTAMP] -
+ mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];
}
- /* by_open_lastio (same for MPI or POSIX) */
- if (dfile->fcounters[CP_F_READ_END_TIMESTAMP] >
- dfile->fcounters[CP_F_WRITE_END_TIMESTAMP])
+ /* by_open_lastio */
+ if(mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] >
+ mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP])
{
/* be careful: file may have been opened but not read or written */
- if(dfile->fcounters[CP_F_READ_END_TIMESTAMP] > dfile->fcounters[CP_F_OPEN_TIMESTAMP])
+ if(mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] > mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])
{
pdata->shared_time_by_open_lastio +=
- dfile->fcounters[CP_F_READ_END_TIMESTAMP] -
- dfile->fcounters[CP_F_OPEN_TIMESTAMP];
+ mfile->fcounters[MPIIO_F_READ_END_TIMESTAMP] -
+ mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];
}
}
else
{
/* be careful: file may have been opened but not read or written */
- if(dfile->fcounters[CP_F_WRITE_END_TIMESTAMP] > dfile->fcounters[CP_F_OPEN_TIMESTAMP])
+ if(mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] > mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP])
{
pdata->shared_time_by_open_lastio +=
- dfile->fcounters[CP_F_WRITE_END_TIMESTAMP] -
- dfile->fcounters[CP_F_OPEN_TIMESTAMP];
+ mfile->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] -
+ mfile->fcounters[MPIIO_F_OPEN_TIMESTAMP];
}
}
- /* by_cumul */
- if (mpi_file)
- {
- pdata->shared_time_by_cumul +=
- dfile->fcounters[CP_F_MPI_META_TIME] +
- dfile->fcounters[CP_F_MPI_READ_TIME] +
- dfile->fcounters[CP_F_MPI_WRITE_TIME];
- pdata->shared_meta_time += dfile->fcounters[CP_F_MPI_META_TIME];
- }
- else
- {
- pdata->shared_time_by_cumul +=
- dfile->fcounters[CP_F_POSIX_META_TIME] +
- dfile->fcounters[CP_F_POSIX_READ_TIME] +
- dfile->fcounters[CP_F_POSIX_WRITE_TIME];
- pdata->shared_meta_time += dfile->fcounters[CP_F_POSIX_META_TIME];
- }
+ pdata->shared_time_by_cumul +=
+ mfile->fcounters[MPIIO_F_META_TIME] +
+ mfile->fcounters[MPIIO_F_READ_TIME] +
+ mfile->fcounters[MPIIO_F_WRITE_TIME];
+ pdata->shared_meta_time += mfile->fcounters[MPIIO_F_META_TIME];
- /* by_slowest (same for MPI or POSIX) */
+ /* by_slowest */
pdata->shared_time_by_slowest +=
- dfile->fcounters[CP_F_SLOWEST_RANK_TIME];
+ mfile->fcounters[MPIIO_F_SLOWEST_RANK_TIME];
}
/*
@@ -1027,38 +1166,169 @@ void accum_perf(struct darshan_file *dfile,
*/
else
{
- if (mpi_file)
+ pdata->rank_cumul_io_time[mfile->rank] +=
+ (mfile->fcounters[MPIIO_F_META_TIME] +
+ mfile->fcounters[MPIIO_F_READ_TIME] +
+ mfile->fcounters[MPIIO_F_WRITE_TIME]);
+ pdata->rank_cumul_md_time[mfile->rank] += mfile->fcounters[MPIIO_F_META_TIME];
+ }
+
+ return;
+}
+
+void posix_calc_file(hash_entry_t *file_hash,
+ file_data_t *fdata)
+{
+ hash_entry_t *curr = NULL;
+ hash_entry_t *tmp = NULL;
+ struct darshan_posix_file *file_rec;
+
+ memset(fdata, 0, sizeof(*fdata));
+ HASH_ITER(hlink, file_hash, curr, tmp)
+ {
+ int64_t bytes;
+ int64_t r;
+ int64_t w;
+
+ file_rec = (struct darshan_posix_file*)curr->rec_dat;
+ assert(file_rec);
+
+ bytes = file_rec->counters[POSIX_BYTES_READ] +
+ file_rec->counters[POSIX_BYTES_WRITTEN];
+
+ r = (file_rec->counters[POSIX_READS]+
+ file_rec->counters[POSIX_FREADS]);
+
+ w = (file_rec->counters[POSIX_WRITES]+
+ file_rec->counters[POSIX_FWRITES]);
+
+ fdata->total += 1;
+ fdata->total_size += bytes;
+ fdata->total_max = max(fdata->total_max, bytes);
+
+ if (r && !w)
+ {
+ fdata->read_only += 1;
+ fdata->read_only_size += bytes;
+ fdata->read_only_max = max(fdata->read_only_max, bytes);
+ }
+
+ if (!r && w)
+ {
+ fdata->write_only += 1;
+ fdata->write_only_size += bytes;
+ fdata->write_only_max = max(fdata->write_only_max, bytes);
+ }
+
+ if (r && w)
+ {
+ fdata->read_write += 1;
+ fdata->read_write_size += bytes;
+ fdata->read_write_max = max(fdata->read_write_max, bytes);
+ }
+
+ if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))
+ {
+ fdata->shared += 1;
+ fdata->shared_size += bytes;
+ fdata->shared_max = max(fdata->shared_max, bytes);
+ }
+
+ if ((curr->type & (FILETYPE_UNIQUE)))
+ {
+ fdata->unique += 1;
+ fdata->unique_size += bytes;
+ fdata->unique_max = max(fdata->unique_max, bytes);
+ }
+ }
+
+ return;
+}
+
+void mpiio_calc_file(hash_entry_t *file_hash,
+ file_data_t *fdata)
+{
+ hash_entry_t *curr = NULL;
+ hash_entry_t *tmp = NULL;
+ struct darshan_mpiio_file *file_rec;
+
+ memset(fdata, 0, sizeof(*fdata));
+ HASH_ITER(hlink, file_hash, curr, tmp)
+ {
+ int64_t bytes;
+ int64_t r;
+ int64_t w;
+
+ file_rec = (struct darshan_mpiio_file*)curr->rec_dat;
+ assert(file_rec);
+
+ bytes = file_rec->counters[MPIIO_BYTES_READ] +
+ file_rec->counters[MPIIO_BYTES_WRITTEN];
+
+ r = (file_rec->counters[MPIIO_INDEP_READS]+
+ file_rec->counters[MPIIO_COLL_READS] +
+ file_rec->counters[MPIIO_SPLIT_READS] +
+ file_rec->counters[MPIIO_NB_READS]);
+
+ w = (file_rec->counters[MPIIO_INDEP_WRITES]+
+ file_rec->counters[MPIIO_COLL_WRITES] +
+ file_rec->counters[MPIIO_SPLIT_WRITES] +
+ file_rec->counters[MPIIO_NB_WRITES]);
+
+ fdata->total += 1;
+ fdata->total_size += bytes;
+ fdata->total_max = max(fdata->total_max, bytes);
+
+ if (r && !w)
{
- pdata->rank_cumul_io_time[dfile->rank] += dfile->fcounters[CP_F_MPI_META_TIME] +
- dfile->fcounters[CP_F_MPI_READ_TIME] +
- dfile->fcounters[CP_F_MPI_WRITE_TIME];
- pdata->rank_cumul_md_time[dfile->rank] += dfile->fcounters[CP_F_MPI_META_TIME];
+ fdata->read_only += 1;
+ fdata->read_only_size += bytes;
+ fdata->read_only_max = max(fdata->read_only_max, bytes);
}
- else
+
+ if (!r && w)
{
- pdata->rank_cumul_io_time[dfile->rank] += dfile->fcounters[CP_F_POSIX_META_TIME] +
- dfile->fcounters[CP_F_POSIX_READ_TIME] +
- dfile->fcounters[CP_F_POSIX_WRITE_TIME];
- pdata->rank_cumul_md_time[dfile->rank] += dfile->fcounters[CP_F_POSIX_META_TIME];
+ fdata->write_only += 1;
+ fdata->write_only_size += bytes;
+ fdata->write_only_max = max(fdata->write_only_max, bytes);
+ }
+ if (r && w)
+ {
+ fdata->read_write += 1;
+ fdata->read_write_size += bytes;
+ fdata->read_write_max = max(fdata->read_write_max, bytes);
+ }
+
+ if ((curr->type & (FILETYPE_SHARED|FILETYPE_PARTSHARED)))
+ {
+ fdata->shared += 1;
+ fdata->shared_size += bytes;
+ fdata->shared_max = max(fdata->shared_max, bytes);
+ }
+
+ if ((curr->type & (FILETYPE_UNIQUE)))
+ {
+ fdata->unique += 1;
+ fdata->unique_size += bytes;
+ fdata->unique_max = max(fdata->unique_max, bytes);
}
}
return;
}
-void calc_perf(struct darshan_job *djob,
- hash_entry_t *hash_rank_uniq,
- perf_data_t *pdata)
+void calc_perf(perf_data_t *pdata,
+ int64_t nprocs)
{
int64_t i;
pdata->shared_time_by_cumul =
- pdata->shared_time_by_cumul / (double)djob->nprocs;
+ pdata->shared_time_by_cumul / (double)nprocs;
- pdata->shared_meta_time = pdata->shared_meta_time / (double)djob->nprocs;
+ pdata->shared_meta_time = pdata->shared_meta_time / (double)nprocs;
- for (i=0; i<djob->nprocs; i++)
+ for (i=0; i<nprocs; i++)
{
if (pdata->rank_cumul_io_time[i] > pdata->slowest_rank_time)
{
@@ -1091,3 +1361,232 @@ void calc_perf(struct darshan_job *djob,
return;
}
+
+void posix_print_total_file(struct darshan_posix_file *pfile)
+{
+ int i;
+ printf("\n");
+ for(i = 0; i < POSIX_NUM_INDICES; i++)
+ {
+ printf("total_%s: %"PRId64"\n",
+ posix_counter_names[i], pfile->counters[i]);
+ }
+ for(i = 0; i < POSIX_F_NUM_INDICES; i++)
+ {
+ printf("total_%s: %lf\n",
+ posix_f_counter_names[i], pfile->fcounters[i]);
+ }
+ return;
+}
+
+void mpiio_print_total_file(struct darshan_mpiio_file *mfile)
+{
+ int i;
+ printf("\n");
+ for(i = 0; i < MPIIO_NUM_INDICES; i++)
+ {
+ printf("total_%s: %"PRId64"\n",
+ mpiio_counter_names[i], mfile->counters[i]);
+ }
+ for(i = 0; i < MPIIO_F_NUM_INDICES; i++)
+ {
+ printf("total_%s: %lf\n",
+ mpiio_f_counter_names[i], mfile->fcounters[i]);
+ }
+ return;
+}
+
+void posix_file_list(hash_entry_t *file_hash,
+ struct darshan_record_ref *rec_hash,
+ int detail_flag)
+{
+ hash_entry_t *curr = NULL;
+ hash_entry_t *tmp = NULL;
+ struct darshan_posix_file *file_rec = NULL;
+ struct darshan_record_ref *ref = NULL;
+ int i;
+
+ /* list of columns:
+ *
+ * normal mode
+ * - file id
+ * - file name
+ * - nprocs
+ * - slowest I/O time
+ * - average cumulative I/O time
+ *
+ * detailed mode
+ * - first open
+ * - first read
+ * - first write
+ * - last read
+ * - last write
+ * - last close
+ * - POSIX opens
+ * - r histogram
+ * - w histogram
+ */
+
+ if(detail_flag)
+ printf("\n# Per-file summary of I/O activity (detailed).\n");
+ else
+ printf("\n# Per-file summary of I/O activity.\n");
+
+ printf("# <record_id>: darshan record id for this file\n");
+ printf("# <file_name>: file name\n");
+ printf("# <nprocs>: number of processes that opened the file\n");
+ printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");
+ printf("# <avg>: average time in seconds consumed in IO per process\n");
+ if(detail_flag)
+ {
+ printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");
+ printf("# <end_{read/write/close}>: end timestamp of last read, write, or close\n");
+ printf("# <posix_opens>: POSIX open calls\n");
+ printf("# <POSIX_SIZE_READ_*>: POSIX read size histogram\n");
+ printf("# <POSIX_SIZE_WRITE_*>: POSIX write size histogram\n");
+ }
+
+ printf("\n# <file_id>\t<file_name>\t<nprocs>\t<slowest>\t<avg>");
+ if(detail_flag)
+ {
+ printf("\t<start_open>\t<start_read>\t<start_write>");
+ printf("\t<end_read>\t<end_write>\t<end_close>\t<posix_opens>");
+ for(i=POSIX_SIZE_READ_0_100; i<= POSIX_SIZE_WRITE_1G_PLUS; i++)
+ printf("\t<%s>", posix_counter_names[i]);
+ }
+ printf("\n");
+
+ HASH_ITER(hlink, file_hash, curr, tmp)
+ {
+ file_rec = (struct darshan_posix_file*)curr->rec_dat;
+ assert(file_rec);
+
+ HASH_FIND(hlink, rec_hash, &(curr->rec_id), sizeof(darshan_record_id), ref);
+ assert(ref);
+
+ printf("%" PRIu64 "\t%s\t%" PRId64 "\t%f\t%f",
+ curr->rec_id,
+ ref->rec.name,
+ curr->procs,
+ curr->slowest_time,
+ curr->cumul_time/(double)curr->procs);
+
+ if(detail_flag)
+ {
+ for(i=POSIX_F_OPEN_TIMESTAMP; i<=POSIX_F_CLOSE_TIMESTAMP; i++)
+ {
+ printf("\t%f", file_rec->fcounters[i]);
+ }
+ printf("\t%" PRId64, file_rec->counters[POSIX_OPENS]);
+ for(i=POSIX_SIZE_READ_0_100; i<= POSIX_SIZE_WRITE_1G_PLUS; i++)
+ printf("\t%" PRId64, file_rec->counters[i]);
+ }
+ printf("\n");
+ }
+
+ return;
+}
+
+void mpiio_file_list(hash_entry_t *file_hash,
+ struct darshan_record_ref *rec_hash,
+ int detail_flag)
+{
+ hash_entry_t *curr = NULL;
+ hash_entry_t *tmp = NULL;
+ struct darshan_mpiio_file *file_rec = NULL;
+ struct darshan_record_ref *ref = NULL;
+ int i;
+
+ /* list of columns:
+ *
+ * normal mode
+ * - file id
+ * - file name
+ * - nprocs
+ * - slowest I/O time
+ * - average cumulative I/O time
+ *
+ * detailed mode
+ * - first open
+ * - first read
+ * - first write
+ * - last read
+ * - last write
+ * - last close
+ * - MPI indep opens
+ * - MPI coll opens
+ * - r histogram
+ * - w histogram
+ */
+
+ if(detail_flag)
+ printf("\n# Per-file summary of I/O activity (detailed).\n");
+ else
+ printf("\n# Per-file summary of I/O activity.\n");
+
+ printf("# <record_id>: darshan record id for this file\n");
+ printf("# <file_name>: file name\n");
+ printf("# <nprocs>: number of processes that opened the file\n");
+ printf("# <slowest>: (estimated) time in seconds consumed in IO by slowest process\n");
+ printf("# <avg>: average time in seconds consumed in IO per process\n");
+ if(detail_flag)
+ {
+ printf("# <start_{open/read/write}>: start timestamp of first open, read, or write\n");
+ printf("# <end_{read/write/close}>: end timestamp of last read, write, or close\n");
+ printf("# <mpi_indep_opens>: independent MPI_File_open calls\n");
+ printf("# <mpi_coll_opens>: collective MPI_File_open calls\n");
+ printf("# <MPIIO_SIZE_READ_AGG_*>: MPI-IO aggregate read size histogram\n");
+ printf("# <MPIIO_SIZE_WRITE_AGG_*>: MPI-IO aggregate write size histogram\n");
+ }
+
+ printf("\n# <file_id>\t<file_name>\t<nprocs>\t<slowest>\t<avg>");
+ if(detail_flag)
+ {
+ printf("\t<start_open>\t<start_read>\t<start_write>");
+ printf("\t<end_read>\t<end_write>\t<end_close>");
+ printf("\t<mpi_indep_opens>\t<mpi_coll_opens>");
+ for(i=MPIIO_SIZE_READ_AGG_0_100; i<= MPIIO_SIZE_WRITE_AGG_1G_PLUS; i++)
+ printf("\t<%s>", mpiio_counter_names[i]);
+ }
+ printf("\n");
+
+ HASH_ITER(hlink, file_hash, curr, tmp)
+ {
+ file_rec = (struct darshan_mpiio_file*)curr->rec_dat;
+ assert(file_rec);
+
+ HASH_FIND(hlink, rec_hash, &(curr->rec_id), sizeof(darshan_record_id), ref);
+ assert(ref);
+
+ printf("%" PRIu64 "\t%s\t%" PRId64 "\t%f\t%f",
+ curr->rec_id,
+ ref->rec.name,
+ curr->procs,
+ curr->slowest_time,
+ curr->cumul_time/(double)curr->procs);
+
+ if(detail_flag)
+ {
+ for(i=MPIIO_F_OPEN_TIMESTAMP; i<=MPIIO_F_CLOSE_TIMESTAMP; i++)
+ {
+ printf("\t%f", file_rec->fcounters[i]);
+ }
+ printf("\t%" PRId64 "\t%" PRId64, file_rec->counters[MPIIO_INDEP_OPENS],
+ file_rec->counters[MPIIO_COLL_OPENS]);
+ for(i=MPIIO_SIZE_READ_AGG_0_100; i<= MPIIO_SIZE_WRITE_AGG_1G_PLUS; i++)
+ printf("\t%" PRId64, file_rec->counters[i]);
+ }
+ printf("\n");
+ }
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-util/darshan-pnetcdf-logutils.c b/darshan-util/darshan-pnetcdf-logutils.c
new file mode 100644
index 0000000..d37d21f
--- /dev/null
+++ b/darshan-util/darshan-pnetcdf-logutils.c
@@ -0,0 +1,105 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#define _GNU_SOURCE
+#include "darshan-util-config.h"
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <inttypes.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <errno.h>
+
+#include "darshan-pnetcdf-logutils.h"
+
+/* counter name strings for the PNETCDF module */
+#define X(a) #a,
+char *pnetcdf_counter_names[] = {
+ PNETCDF_COUNTERS
+};
+
+char *pnetcdf_f_counter_names[] = {
+ PNETCDF_F_COUNTERS
+};
+#undef X
+
+static int darshan_log_get_pnetcdf_file(void** pnetcdf_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag);
+static void darshan_log_print_pnetcdf_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+struct darshan_mod_logutil_funcs pnetcdf_logutils =
+{
+ .log_get_record = &darshan_log_get_pnetcdf_file,
+ .log_print_record = &darshan_log_print_pnetcdf_file,
+};
+
+static int darshan_log_get_pnetcdf_file(void** pnetcdf_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag)
+{
+ int i;
+ struct darshan_pnetcdf_file *file = (struct darshan_pnetcdf_file *)
+ (*pnetcdf_buf_p);
+
+ if(*bytes_left < sizeof(struct darshan_pnetcdf_file))
+ return(-1);
+
+ if(byte_swap_flag)
+ {
+ /* swap bytes if necessary */
+ DARSHAN_BSWAP64(&file->f_id);
+ DARSHAN_BSWAP64(&file->rank);
+ for(i=0; i<PNETCDF_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->counters[i]);
+ for(i=0; i<PNETCDF_F_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->fcounters[i]);
+ }
+
+ /* update/set output variables */
+ *file_rec = (void *)file;
+ *rec_id = file->f_id;
+ *pnetcdf_buf_p = (file + 1); /* increment input buf by size of file record */
+ *bytes_left -= sizeof(struct darshan_pnetcdf_file);
+
+ return(0);
+}
+
+static void darshan_log_print_pnetcdf_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ int i;
+ struct darshan_pnetcdf_file *pnetcdf_file_rec =
+ (struct darshan_pnetcdf_file *)file_rec;
+
+ for(i=0; i<PNETCDF_NUM_INDICES; i++)
+ {
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ pnetcdf_file_rec->rank, pnetcdf_file_rec->f_id, pnetcdf_counter_names[i],
+ pnetcdf_file_rec->counters[i], file_name, mnt_pt, fs_type);
+ }
+
+ for(i=0; i<PNETCDF_F_NUM_INDICES; i++)
+ {
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_PNETCDF_MOD],
+ pnetcdf_file_rec->rank, pnetcdf_file_rec->f_id, pnetcdf_f_counter_names[i],
+ pnetcdf_file_rec->fcounters[i], file_name, mnt_pt, fs_type);
+ }
+
+ return;
+}
+
+/*
+ * Local variables:
+ * c-indent-level: 4
+ * c-basic-offset: 4
+ * End:
+ *
+ * vim: ts=8 sts=4 sw=4 expandtab
+ */
diff --git a/darshan-util/darshan-pnetcdf-logutils.h b/darshan-util/darshan-pnetcdf-logutils.h
new file mode 100644
index 0000000..9645717
--- /dev/null
+++ b/darshan-util/darshan-pnetcdf-logutils.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright (C) 2015 University of Chicago.
+ * See COPYRIGHT notice in top-level directory.
+ *
+ */
+
+#ifndef __DARSHAN_PNETCDF_LOG_UTILS_H
+#define __DARSHAN_PNETCDF_LOG_UTILS_H
+
+#include "darshan-logutils.h"
+#include "darshan-pnetcdf-log-format.h"
+
+extern char *pnetcdf_counter_names[];
+extern char *pnetcdf_f_counter_names[];
+
+extern struct darshan_mod_logutil_funcs pnetcdf_logutils;
+
+#endif
diff --git a/darshan-util/darshan-posix-logutils.c b/darshan-util/darshan-posix-logutils.c
index 51ba78d..5a02610 100644
--- a/darshan-util/darshan-posix-logutils.c
+++ b/darshan-util/darshan-posix-logutils.c
@@ -19,33 +19,80 @@
#include "darshan-posix-logutils.h"
-int darshan_log_get_posix_file(darshan_fd fd, struct darshan_posix_file *file)
+/* counter name strings for the POSIX module */
+#define X(a) #a,
+char *posix_counter_names[] = {
+ POSIX_COUNTERS
+};
+
+char *posix_f_counter_names[] = {
+ POSIX_F_COUNTERS
+};
+#undef X
+
+static int darshan_log_get_posix_file(void** psx_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag);
+static void darshan_log_print_posix_file(void *file_rec,
+ char *file_name, char *mnt_pt, char *fs_type);
+
+struct darshan_mod_logutil_funcs posix_logutils =
+{
+ .log_get_record = &darshan_log_get_posix_file,
+ .log_print_record = &darshan_log_print_posix_file,
+};
+
+static int darshan_log_get_posix_file(void** psx_buf_p, int* bytes_left,
+ void** file_rec, darshan_record_id* rec_id, int byte_swap_flag)
{
int i;
- int ret;
+ struct darshan_posix_file *file = (struct darshan_posix_file *)
+ (*psx_buf_p);
+
+ if(*bytes_left < sizeof(struct darshan_posix_file))
+ return(-1);
+
+ if(byte_swap_flag)
+ {
+ /* swap bytes if necessary */
+ DARSHAN_BSWAP64(&file->f_id);
+ DARSHAN_BSWAP64(&file->rank);
+ for(i=0; i<POSIX_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->counters[i]);
+ for(i=0; i<POSIX_F_NUM_INDICES; i++)
+ DARSHAN_BSWAP64(&file->fcounters[i]);
+ }
- /* reset file record, so that diff compares against a zero'd out record
- * if file is missing
- */
- memset(file, 0, sizeof(*file));
+ /* update/set output variables */
+ *file_rec = (void *)file;
+ *rec_id = file->f_id;
+ *psx_buf_p = (file + 1); /* increment input buf by size of file record */
+ *bytes_left -= sizeof(struct darshan_posix_file);
+
+ return(0);
+}
+
+static void darshan_log_print_posix_file(void *file_rec, char *file_name,
+ char *mnt_pt, char *fs_type)
+{
+ int i;
+ struct darshan_posix_file *posix_file_rec =
+ (struct darshan_posix_file *)file_rec;
+
+ for(i=0; i<POSIX_NUM_INDICES; i++)
+ {
+ DARSHAN_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ posix_file_rec->rank, posix_file_rec->f_id, posix_counter_names[i],
+ posix_file_rec->counters[i], file_name, mnt_pt, fs_type);
+ }
- ret = darshan_log_get_moddat(fd, DARSHAN_POSIX_MOD,
- (void *)file, sizeof(*file));
- if(ret == 1)
+ for(i=0; i<POSIX_F_NUM_INDICES; i++)
{
- if(fd->swap_flag)
- {
- /* swap bytes if necessary */
- DARSHAN_BSWAP64(&file->f_id);
- DARSHAN_BSWAP64(&file->rank);
- for(i=0; i<POSIX_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->counters[i]);
- for(i=0; i<POSIX_F_NUM_INDICES; i++)
- DARSHAN_BSWAP64(&file->fcounters[i]);
- }
+ DARSHAN_F_COUNTER_PRINT(darshan_module_names[DARSHAN_POSIX_MOD],
+ posix_file_rec->rank, posix_file_rec->f_id, posix_f_counter_names[i],
+ posix_file_rec->fcounters[i], file_name, mnt_pt, fs_type);
}
- return(ret);
+ return;
}
/*
diff --git a/darshan-util/darshan-posix-logutils.h b/darshan-util/darshan-posix-logutils.h
index 59b307e..b641dbd 100644
--- a/darshan-util/darshan-posix-logutils.h
+++ b/darshan-util/darshan-posix-logutils.h
@@ -10,6 +10,9 @@
#include "darshan-logutils.h"
#include "darshan-posix-log-format.h"
-int darshan_log_get_posix_file(darshan_fd fd, struct darshan_posix_file *file);
+extern char *posix_counter_names[];
+extern char *posix_f_counter_names[];
+
+extern struct darshan_mod_logutil_funcs posix_logutils;
#endif
diff --git a/darshan-util/darshan-posix-parser.c b/darshan-util/darshan-posix-parser.c
deleted file mode 100644
index 9c9dbd0..0000000
--- a/darshan-util/darshan-posix-parser.c
+++ /dev/null
@@ -1,354 +0,0 @@
-/*
- * Copyright (C) 2015 University of Chicago.
- * See COPYRIGHT notice in top-level directory.
- *
- */
-
-#include <stdio.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <string.h>
-#include <zlib.h>
-#include <time.h>
-#include <stdlib.h>
-#include <getopt.h>
-#include <assert.h>
-
-#include "darshan-logutils.h"
-#include "darshan-posix-logutils.h"
-#include "uthash-1.9.2/src/uthash.h"
-
-int main(int argc, char **argv)
-{
- int ret;
- int i;
- char *filename;
- char tmp_string[4096];
- darshan_fd fd;
- struct darshan_header header;
- struct darshan_job job;
- struct darshan_record_ref *rec_hash = NULL;
- struct darshan_record_ref *ref;
- int mount_count;
- char** mnt_pts;
- char** fs_types;
- time_t tmp_time = 0;
- char *token;
- char *save;
- char buffer[DARSHAN_JOB_METADATA_LEN];
- struct darshan_posix_file next_file;
-
- assert(argc == 2);
- filename = argv[1];
-
- struct stat sbuf;
- stat(filename, &sbuf);
-
- fd = darshan_log_open(filename, "r");
- if(!fd)
- {
- fprintf(stderr, "darshan_log_open() failed to open %s\n.", filename);
- return(-1);
- }
-
- /* read darshan log header */
- ret = darshan_log_getheader(fd, &header);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getheader() failed to read log header.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* read darshan job info */
- ret = darshan_log_getjob(fd, &job);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getjob() failed to read job data.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* get the original command line for this job */
- ret = darshan_log_getexe(fd, tmp_string);
- if(ret < 0)
- {
- fprintf(stderr, "Error: unable to read trailing job information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print job summary */
- printf("# darshan log version: %s\n", header.version_string);
- printf("# size of POSIX file statistics: %zu bytes\n", sizeof(struct darshan_posix_file));
- printf("# size of job statistics: %zu bytes\n", sizeof(struct darshan_job));
- printf("# exe: %s\n", tmp_string);
- printf("# uid: %" PRId64 "\n", job.uid);
- printf("# jobid: %" PRId64 "\n", job.jobid);
- printf("# start_time: %" PRId64 "\n", job.start_time);
- tmp_time += job.start_time;
- printf("# start_time_asci: %s", ctime(&tmp_time));
- printf("# end_time: %" PRId64 "\n", job.end_time);
- tmp_time = 0;
- tmp_time += job.end_time;
- printf("# end_time_asci: %s", ctime(&tmp_time));
- printf("# nprocs: %" PRId64 "\n", job.nprocs);
- printf("# run time: %" PRId64 "\n", job.end_time - job.start_time + 1);
- for(token=strtok_r(job.metadata, "\n", &save);
- token != NULL;
- token=strtok_r(NULL, "\n", &save))
- {
- char *key;
- char *value;
- /* NOTE: we intentionally only split on the first = character.
- * There may be additional = characters in the value portion
- * (for example, when storing mpi-io hints).
- */
- strcpy(buffer, token);
- key = buffer;
- value = index(buffer, '=');
- if(!value)
- continue;
- /* convert = to a null terminator to split key and value */
- value[0] = '\0';
- value++;
- printf("# metadata: %s = %s\n", key, value);
- }
-
- /* get the mount information for this log */
- ret = darshan_log_getmounts(fd, &mnt_pts, &fs_types, &mount_count);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmounts() failed to read mount information.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- /* print table of mounted file systems */
- printf("\n# mounted file systems (mount point and fs type)\n");
- printf("# -------------------------------------------------------\n");
- for(i=0; i<mount_count; i++)
- {
- printf("# mount entry:\t%s\t%s\n", mnt_pts[i], fs_types[i]);
- }
-
- /* read hash of darshan records */
- ret = darshan_log_gethash(fd, &rec_hash);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_getmap() failed to read record map.\n");
- darshan_log_close(fd);
- return(-1);
- }
-
- printf("\n*** FILE RECORD DATA ***\n");
-
- ret = darshan_log_get_posix_file(fd, &next_file);
- if(ret < 0)
- {
- fprintf(stderr, "darshan_log_get_posix_file() failed to read next record.\n");
- darshan_log_close(fd);
- return(-1);
- }
- if(ret == 0)
- {
- printf("# no files opened.\n");
- darshan_log_close(fd);
- return(0);
- }
-
- /* loop over each stored POSIX file record and print counters */
- i = 1;
- do
- {
- /* get the pathname for this record */
- HASH_FIND(hlink, rec_hash, &next_file.f_id, sizeof(darshan_record_id), ref);
- assert(ref);
-
- printf("\tRecord %d: id=%"PRIu64" (path=%s, rank=%"PRId64")\n",
- i, next_file.f_id, ref->rec.name, next_file.rank);
- printf(
- "\t\tPOSIX_OPENS:\t%"PRIu64"\n"
- "\t\tPOSIX_READS:\t%"PRIu64"\n"
- "\t\tPOSIX_WRITES:\t%"PRIu64"\n"
- "\t\tPOSIX_SEEKS:\t%"PRIu64"\n"
- "\t\tPOSIX_STATS:\t%"PRIu64"\n"
- "\t\tPOSIX_MMAPS:\t%"PRIu64"\n"
- "\t\tPOSIX_FOPENS:\t%"PRIu64"\n"
- "\t\tPOSIX_FREADS:\t%"PRIu64"\n"
- "\t\tPOSIX_FWRITES:\t%"PRIu64"\n"
- "\t\tPOSIX_FSEEKS:\t%"PRIu64"\n"
- "\t\tPOSIX_FSYNCS:\t%"PRIu64"\n"
- "\t\tPOSIX_FDSYNCS:\t%"PRIu64"\n"
- "\t\tPOSIX_MODE:\t%"PRIu64"\n"
- "\t\tPOSIX_BYTES_READ:\t%"PRIu64"\n"
- "\t\tPOSIX_BYTES_WRITTEN:\t%"PRIu64"\n"
- "\t\tPOSIX_MAX_BYTE_READ:\t%"PRIu64"\n"
- "\t\tPOSIX_MAX_BYTE_WRITTEN:\t%"PRIu64"\n"
- "\t\tPOSIX_CONSEC_READS:\t%"PRIu64"\n"
- "\t\tPOSIX_CONSEC_WRITES:\t%"PRIu64"\n"
- "\t\tPOSIX_SEQ_READS:\t%"PRIu64"\n"
- "\t\tPOSIX_SEQ_WRITES:\t%"PRIu64"\n"
- "\t\tPOSIX_RW_SWITCHES:\t%"PRIu64"\n"
- "\t\tPOSIX_MEM_NOT_ALIGNED:\t%"PRIu64"\n"
- "\t\tPOSIX_MEM_ALIGNMENT:\t%"PRIu64"\n"
- "\t\tPOSIX_FILE_NOT_ALIGNED:\t%"PRIu64"\n"
- "\t\tPOSIX_FILE_ALIGNMENT:\t%"PRIu64"\n"
- "\t\tPOSIX_MAX_READ_TIME_SIZE:\t%"PRIu64"\n"
- "\t\tPOSIX_MAX_WRITE_TIME_SIZE:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_0_100:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_100_1K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_1K_10K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_10K_100K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_100K_1M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_1M_4M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_4M_10M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_10M_100M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_100M_1G:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_READ_1G_PLUS:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_0_100:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_100_1K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_1K_10K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_10K_100K:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_100K_1M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_1M_4M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_4M_10M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_10M_100M:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_100M_1G:\t%"PRIu64"\n"
- "\t\tPOSIX_SIZE_WRITE_1G_PLUS:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE1_STRIDE:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE2_STRIDE:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE3_STRIDE:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE4_STRIDE:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE1_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE2_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE3_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_STRIDE4_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS1_ACCESS:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS2_ACCESS:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS3_ACCESS:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS4_ACCESS:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS1_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS2_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS3_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_ACCESS4_COUNT:\t%"PRIu64"\n"
- "\t\tPOSIX_FASTEST_RANK:\t%"PRIu64"\n"
- "\t\tPOSIX_FASTEST_RANK_BYTES:\t%"PRIu64"\n"
- "\t\tPOSIX_SLOWEST_RANK:\t%"PRIu64"\n"
- "\t\tPOSIX_SLOWEST_RANK_BYTES:\t%"PRIu64"\n"
- "\t\tPOSIX_F_OPEN_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_READ_START_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_WRITE_START_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_READ_END_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_WRITE_END_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_CLOSE_TIMESTAMP:\t%lf\n"
- "\t\tPOSIX_F_READ_TIME:\t%lf\n"
- "\t\tPOSIX_F_WRITE_TIME:\t%lf\n"
- "\t\tPOSIX_F_META_TIME:\t%lf\n"
- "\t\tPOSIX_F_MAX_READ_TIME:\t%lf\n"
- "\t\tPOSIX_F_MAX_WRITE_TIME:\t%lf\n"
- "\t\tPOSIX_F_FASTEST_RANK_TIME:\t%lf\n"
- "\t\tPOSIX_F_SLOWEST_RANK_TIME:\t%lf\n",
- next_file.counters[POSIX_OPENS],
- next_file.counters[POSIX_READS],
- next_file.counters[POSIX_WRITES],
- next_file.counters[POSIX_SEEKS],
- next_file.counters[POSIX_STATS],
- next_file.counters[POSIX_MMAPS],
- next_file.counters[POSIX_FOPENS],
- next_file.counters[POSIX_FREADS],
- next_file.counters[POSIX_FWRITES],
- next_file.counters[POSIX_FSEEKS],
- next_file.counters[POSIX_FSYNCS],
- next_file.counters[POSIX_FDSYNCS],
- next_file.counters[POSIX_MODE],
- next_file.counters[POSIX_BYTES_READ],
- next_file.counters[POSIX_BYTES_WRITTEN],
- next_file.counters[POSIX_MAX_BYTE_READ],
- next_file.counters[POSIX_MAX_BYTE_WRITTEN],
- next_file.counters[POSIX_CONSEC_READS],
- next_file.counters[POSIX_CONSEC_WRITES],
- next_file.counters[POSIX_SEQ_READS],
- next_file.counters[POSIX_SEQ_WRITES],
- next_file.counters[POSIX_RW_SWITCHES],
- next_file.counters[POSIX_MEM_NOT_ALIGNED],
- next_file.counters[POSIX_MEM_ALIGNMENT],
- next_file.counters[POSIX_FILE_NOT_ALIGNED],
- next_file.counters[POSIX_FILE_ALIGNMENT],
- next_file.counters[POSIX_MAX_READ_TIME_SIZE],
- next_file.counters[POSIX_MAX_WRITE_TIME_SIZE],
- next_file.counters[POSIX_SIZE_READ_0_100],
- next_file.counters[POSIX_SIZE_READ_100_1K],
- next_file.counters[POSIX_SIZE_READ_1K_10K],
- next_file.counters[POSIX_SIZE_READ_10K_100K],
- next_file.counters[POSIX_SIZE_READ_100K_1M],
- next_file.counters[POSIX_SIZE_READ_1M_4M],
- next_file.counters[POSIX_SIZE_READ_4M_10M],
- next_file.counters[POSIX_SIZE_READ_10M_100M],
- next_file.counters[POSIX_SIZE_READ_100M_1G],
- next_file.counters[POSIX_SIZE_READ_1G_PLUS],
- next_file.counters[POSIX_SIZE_WRITE_0_100],
- next_file.counters[POSIX_SIZE_WRITE_100_1K],
- next_file.counters[POSIX_SIZE_WRITE_1K_10K],
- next_file.counters[POSIX_SIZE_WRITE_10K_100K],
- next_file.counters[POSIX_SIZE_WRITE_100K_1M],
- next_file.counters[POSIX_SIZE_WRITE_1M_4M],
- next_file.counters[POSIX_SIZE_WRITE_4M_10M],
- next_file.counters[POSIX_SIZE_WRITE_10M_100M],
- next_file.counters[POSIX_SIZE_WRITE_100M_1G],
- next_file.counters[POSIX_SIZE_WRITE_1G_PLUS],
- next_file.counters[POSIX_STRIDE1_STRIDE],
- next_file.counters[POSIX_STRIDE2_STRIDE],
- next_file.counters[POSIX_STRIDE3_STRIDE],
- next_file.counters[POSIX_STRIDE4_STRIDE],
- next_file.counters[POSIX_STRIDE1_COUNT],
- next_file.counters[POSIX_STRIDE2_COUNT],
- next_file.counters[POSIX_STRIDE3_COUNT],
- next_file.counters[POSIX_STRIDE4_COUNT],
- next_file.counters[POSIX_ACCESS1_ACCESS],
- next_file.counters[POSIX_ACCESS2_ACCESS],
- next_file.counters[POSIX_ACCESS3_ACCESS],
- next_file.counters[POSIX_ACCESS4_ACCESS],
- next_file.counters[POSIX_ACCESS1_COUNT],
- next_file.counters[POSIX_ACCESS2_COUNT],
- next_file.counters[POSIX_ACCESS3_COUNT],
- next_file.counters[POSIX_ACCESS4_COUNT],
- next_file.counters[POSIX_FASTEST_RANK],
- next_file.counters[POSIX_FASTEST_RANK_BYTES],
- next_file.counters[POSIX_SLOWEST_RANK],
- next_file.counters[POSIX_SLOWEST_RANK_BYTES],
- next_file.fcounters[POSIX_F_OPEN_TIMESTAMP],
- next_file.fcounters[POSIX_F_READ_START_TIMESTAMP],
- next_file.fcounters[POSIX_F_WRITE_START_TIMESTAMP],
- next_file.fcounters[POSIX_F_READ_END_TIMESTAMP],
- next_file.fcounters[POSIX_F_WRITE_END_TIMESTAMP],
- next_file.fcounters[POSIX_F_CLOSE_TIMESTAMP],
- next_file.fcounters[POSIX_F_READ_TIME],
- next_file.fcounters[POSIX_F_WRITE_TIME],
- next_file.fcounters[POSIX_F_META_TIME],
- next_file.fcounters[POSIX_F_MAX_READ_TIME],
- next_file.fcounters[POSIX_F_MAX_WRITE_TIME],
- next_file.fcounters[POSIX_F_FASTEST_RANK_TIME],
- next_file.fcounters[POSIX_F_SLOWEST_RANK_TIME]);
-
- i++;
- } while((ret = darshan_log_get_posix_file(fd, &next_file)) == 1);
-
- /* free mount info */
- for(i=0; i<mount_count; i++)
- {
- free(mnt_pts[i]);
- free(fs_types[i]);
- }
- if(mount_count > 0)
- {
- free(mnt_pts);
- free(fs_types);
- }
-
- darshan_log_close(fd);
-
- return(0);
-}
diff --git a/darshan-util/darshan-job-summary/bin/jenkins.c b/darshan-util/jenkins-hash-gen.c
similarity index 100%
rename from darshan-util/darshan-job-summary/bin/jenkins.c
rename to darshan-util/jenkins-hash-gen.c
diff --git a/doc/darshan-dev-modular-runtime.png b/doc/darshan-dev-modular-runtime.png
index 86a8f34..4d60cf0 100644
Binary files a/doc/darshan-dev-modular-runtime.png and b/doc/darshan-dev-modular-runtime.png differ
diff --git a/doc/darshan-modularization.txt b/doc/darshan-modularization.txt
index 08304f1..5564516 100644
--- a/doc/darshan-modularization.txt
+++ b/doc/darshan-modularization.txt
@@ -74,8 +74,7 @@ In general, instrumentation modules are composed of:
* internal functions for initializing and maintaining internal data structures and module-specific
I/O characterization data;
-* a set of functions for interfacing with the Darshan runtime environment, including an optional
- reduction operation to condense I/O data shared on all processes into a single data record.
+* a set of functions for interfacing with the Darshan runtime environment
A block diagram illustrating the interaction of an example POSIX instrumentation module and the
Darshan runtime environment is given below in Figure 1.
@@ -83,6 +82,30 @@ Darshan runtime environment is given below in Figure 1.
.Darshan runtime environment
image::darshan-dev-modular-runtime.png[align="center"]
+As shown in Figure 1, the Darshan runtime environment is just a library (libdarshan) which
+intercepts and instruments functions of interest made by an application to existing system
+libraries. Two primary components of this library are `darshan-core` and `darshan-common`.
+`darshan-core` is the central component which manages the initialization/shutdown of Darshan,
+coordinates with active instrumentation modules, and writes I/O characterization logs to disk,
+among other things. `darshan-core` intercepts `MPI_Init()` to initialize key internal data
+stuctures and intercepts `MPI_Finalize()` to initiate Darshan's shutdown process. `darshan-common`
+simply provides module developers with functionality that is likely to be reused across modules
+to minimize development and maintenance costs. Instrumentation modules must utilize `darshan-core`
+to register themselves and corresponding I/O records with Darshan so they can be added to the
+output I/O characterization. While not shown in Figure 1, numerous modules can be registered
+with Darshan at any given time and Darshan is capable of correlating records between these
+modules.
+
+In the next three subsections, we describe instrumentation modules, the `darshan-core` component,
+and the `darshan-common` component in more detail. In
+link:darshan-modularization.html#_adding_new_instrumentation_modules[Section 4], we provide the
+required steps for integrating new instrumentation modules into Darshan. This section also includes
+details on an example module that can serve as a minimal starting point for new module implementations.
+In link:darshan-modularization.html#_shared_record_reductions[Section 5], we provide details on
+implementing a shared record reduction operation within an instrumentation module. This optional
+mechanism allows modules to compress records which are shared across all processes of an application,
+minimizing the size of the resulting I/O characterization log.
+
==== Instrumentation modules
The wrapper functions used to intercept I/O function calls of interest are central to the design of
@@ -106,6 +129,12 @@ environment to coordinate with modules while shutting down:
struct darshan_module_funcs
{
void (*begin_shutdown)(void);
+ void (*get_output_data)(
+ void** buf,
+ int* size
+ );
+ void (*shutdown)(void);
+ /* OPTIONAL: shared record reductions ignored by setting setup_reduction to NULL */
void (*setup_reduction)(
darshan_record_id *shared_recs,
int *shared_rec_count,
@@ -113,17 +142,13 @@ struct darshan_module_funcs
void **recv_buf,
int *rec_size
);
+ /* OPTIONAL: shared record reductions ignored by setting record_reduction_op to NULL */
void (*record_reduction_op)(
void* a,
void* b,
int *len,
MPI_Datatype *datatype
);
- void (*get_output_data)(
- void** buf,
- int* size
- );
- void (*shutdown)(void);
};
`begin_shutdown()`
@@ -134,30 +159,6 @@ to ensure data consistency and avoid other race conditions. This function also s
opportunity for a module to modify internal data structures prior to a possible reduction of shared
data.
-`setup_reduction()`
-
-An optional feature provided to instrumentation modules it the ability to run reduction operations
-on I/O data records which are shared across all application processes (e.g., data records for a
-shared file). This reduction is done to minimize the size of the resulting I/O characterization,
-by aggregating shared records into a single data record.
-
-This function allows modules to setup internal data structures to run a reduction operation
-on data records which are shared across all application processes. Module developers can bypass
-the shared record reduction mechanism by setting the `setup_reduction` function pointer equal to `NULL`.
-This is helpful in initial prototyping of a module, or in the case where a module would not maintain
-I/O data which is shared across all processes.
-
-The shared record reduction mechanism is described in detail
-link:darshan-modularization.html#_shared_record_reductions[here].
-
-`record_reduction_op()`
-
-This function implements the actual shared record reduction operation. Module developers can bypass
-the shared record reduction mechanism by setting the `record_reduction_op` pointer equal to `NULL`.
-
-The shared record reduction mechanism is described in detail
-link:darshan-modularization.html#_shared_record_reductions[here].
-
`get_output_data()`
This function is responsible for passing back a single buffer storing all data this module is
@@ -173,6 +174,11 @@ characterization.
This function is a signal from Darshan that it is safe to shutdown. It should clean up and free
all internal data structures.
+`setup_reduction()` and `record_reduction_op()` are optional function pointers which can be
+implemented to utilize Darshan's shared I/O record reduction mechanism, described in detail in
+link:darshan-modularization.html#_shared_record_reductions[Section 5]. Module developers can
+bypass this mechanism by setting these function pointers to NULL.
+
==== darshan-core
Within darshan-runtime, the darshan-core component manages the initialization and shutdown of the
@@ -476,8 +482,8 @@ Since Darshan perfers to aggregate data records which are shared across all proc
data record, module developers should consider implementing this functionality eventually, though it
is not strictly required.
-As mentioned previously, module developers must provide implementations for the `begin_reduction()`
-and `record_reduction_op` functions in the darshan_module_funcs structure to leverage Darshan's
+As mentioned previously, module developers must provide implementations for the `setup_reduction()`
+and `record_reduction_op()` functions in the darshan_module_funcs structure to leverage Darshan's
shared record reduction mechanism. These functions are described in detail as follows:
[source,c]
hooks/post-receive
--
More information about the Darshan-commits
mailing list