<html lang='en'>
<head>
<meta content='text/html; charset=utf-8' http-equiv='Content-Type'>
<title>
GitLab
</title>
</meta>
</head>
<style>
img {
max-width: 100%;
height: auto;
}
p.details {
font-style:italic;
color:#777
}
.footer p {
font-size:small;
color:#777
}
pre.commit-message {
white-space: pre-wrap;
}
.file-stats a {
text-decoration: none;
}
.file-stats .new-file {
color: #090;
}
.file-stats .deleted-file {
color: #B00;
}
</style>
<body>
<div class='content'>
<h3>Shane Snyder pushed to branch mmap-dev at <a href="https://xgitlab.cels.anl.gov/darshan/darshan">darshan / darshan</a></h3>
<h4>
Commits:
</h4>
<ul>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/c9618c8a4c290acbe2c7f2d436815b8d2d459626">c9618c8a</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-04T13:52:19Z</i>
</div>
<pre class='commit-message'>runtime now stores mnt & exe data at all ranks</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/4ac7590597e0bdb2e7982597e12f53315a6d2ab7">4ac75905</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-04T15:18:06Z</i>
</div>
<pre class='commit-message'>modify hash write routine to use new serialization</pre>
</li>
<li>
<strong><a href="https://xgitlab.cels.anl.gov/darshan/darshan/commit/66a83b0382dade07f600a43bf50707be39e63ef2">66a83b03</a></strong>
<div>
<span>by Shane Snyder</span>
<i>at 2015-12-07T13:25:52Z</i>
</div>
<pre class='commit-message'>first cut at the stitch logs utility
Currently, this tool can take a set of temporary, uncompressed log
files on a given node, and combine into a single darshan log.
Next step would be to modify this code to perform shared file
reduction logic to combine per-node log files into per-job log
files.</pre>
</li>
</ul>
<h4>3 changed files:</h4>
<ul>
<li class='file-stats'>
<a href='#diff-0'>
darshan-runtime/lib/darshan-core.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-1'>
darshan-util/darshan-logutils.c
</a>
</li>
<li class='file-stats'>
<a href='#diff-2'>
darshan-util/darshan-stitch-logs.c
</a>
</li>
</ul>
<h4>Changes:</h4>
<li id='diff-0'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/9193b34aaf7e9f9d1e5361dfb44667d14277f23f...66a83b0382dade07f600a43bf50707be39e63ef2#diff-0'>
<strong>
darshan-runtime/lib/darshan-core.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-runtime/lib/darshan-core.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-runtime/lib/darshan-core.c
</span><span style="color: #aaaaaa">@@ -91,8 +91,6 @@ static int mnt_data_count = 0;
</span> /* prototypes for internal helper functions */
static void darshan_log_record_hints_and_ver(
struct darshan_core_runtime* core);
-static void darshan_get_exe_and_mounts_root(
<span style="color: #000000;background-color: #ffdddd">- struct darshan_core_runtime *core, int argc, char **argv);
</span> static void darshan_get_exe_and_mounts(
struct darshan_core_runtime *core, int argc, char **argv);
static void darshan_block_size_from_path(
<span style="color: #aaaaaa">@@ -470,12 +468,12 @@ static void add_entry(char* buf, int* space_left, struct mntent *entry)
</span> return;
}
-/* darshan_get_exe_and_mounts_root()
<span style="color: #000000;background-color: #ddffdd">+/* darshan_get_exe_and_mounts()
</span> *
* collects command line and list of mounted file systems into a string that
* will be stored with the job-level metadata
*/
-static void darshan_get_exe_and_mounts_root(struct darshan_core_runtime *core,
<span style="color: #000000;background-color: #ddffdd">+static void darshan_get_exe_and_mounts(struct darshan_core_runtime *core,
</span> int argc, char **argv)
{
FILE* tab;
<span style="color: #aaaaaa">@@ -581,7 +579,7 @@ static void darshan_get_exe_and_mounts_root(struct darshan_core_runtime *core,
</span> }
endmntent(tab);
<span style="color: #000000;background-color: #ffdddd">- /* Sort mount points in order of longest path to shortest path. This is
</span><span style="color: #000000;background-color: #ddffdd">+ /* sort mount points in order of longest path to shortest path. This is
</span> * necessary so that if we try to match file paths to mount points later
* we don't match on "/" every time.
*/
<span style="color: #aaaaaa">@@ -589,29 +587,6 @@ static void darshan_get_exe_and_mounts_root(struct darshan_core_runtime *core,
</span> return;
}
-/* darshan_get_exe_and_mounts()
<span style="color: #000000;background-color: #ffdddd">- *
- * collects command line and list of mounted file systems into a string that
- * will be stored with the job-level metadata
- */
</span>-static void darshan_get_exe_and_mounts(struct darshan_core_runtime *core,
<span style="color: #000000;background-color: #ffdddd">- int argc, char **argv)
</span>-{
<span style="color: #000000;background-color: #ffdddd">- if(my_rank == 0)
- {
- darshan_get_exe_and_mounts_root(core, argc, argv);
- }
</span>-
<span style="color: #000000;background-color: #ffdddd">- /* broadcast mount count to all nodes */
- DARSHAN_MPI_CALL(PMPI_Bcast)(&mnt_data_count, 1, MPI_INT, 0,
- MPI_COMM_WORLD);
- /* broadcast mount data to all nodes */
- DARSHAN_MPI_CALL(PMPI_Bcast)(mnt_data_array,
- mnt_data_count*sizeof(mnt_data_array[0]), MPI_BYTE, 0, MPI_COMM_WORLD);
</span>-
<span style="color: #000000;background-color: #ffdddd">- return;
</span>-}
-
static void darshan_add_record_hashref(struct darshan_core_runtime *core,
char *name, darshan_record_id id, struct darshan_core_record_ref **ref)
{
</code></pre>
<br>
</li>
<li id='diff-1'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/9193b34aaf7e9f9d1e5361dfb44667d14277f23f...66a83b0382dade07f600a43bf50707be39e63ef2#diff-1'>
<strong>
darshan-util/darshan-logutils.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-logutils.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-logutils.c
</span><span style="color: #aaaaaa">@@ -638,13 +638,12 @@ int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash)
</span> int hash_buf_sz;
struct darshan_record_ref *ref, *tmp;
char *buf_ptr;
<span style="color: #000000;background-color: #ffdddd">- int path_len;
</span> int wrote;
assert(state);
/* allocate memory for largest possible hash record */
<span style="color: #000000;background-color: #ffdddd">- hash_buf_sz = sizeof(darshan_record_id) + sizeof(uint32_t) + PATH_MAX;
</span><span style="color: #000000;background-color: #ddffdd">+ hash_buf_sz = sizeof(darshan_record_id) + PATH_MAX + 1;
</span> hash_buf = malloc(hash_buf_sz);
if(!hash_buf)
return(-1);
<span style="color: #aaaaaa">@@ -654,18 +653,15 @@ int darshan_log_puthash(darshan_fd fd, struct darshan_record_ref *hash)
</span> HASH_ITER(hlink, hash, ref, tmp)
{
buf_ptr = hash_buf;
<span style="color: #000000;background-color: #ffdddd">- path_len = strlen(ref->rec.name);
</span>
/* the hash buffer has space to serialize this record
* NOTE: darshan record hash serialization method:
<span style="color: #000000;background-color: #ffdddd">- * ... darshan_record_id | (uint32_t) path_len | path ...
</span><span style="color: #000000;background-color: #ddffdd">+ * ... darshan_record_id | path '\0' ...
</span> */
*((darshan_record_id *)buf_ptr) = ref->rec.id;
buf_ptr += sizeof(darshan_record_id);
<span style="color: #000000;background-color: #ffdddd">- *((uint32_t *)buf_ptr) = path_len;
- buf_ptr += sizeof(uint32_t);
- memcpy(buf_ptr, ref->rec.name, path_len);
- buf_ptr += path_len;
</span><span style="color: #000000;background-color: #ddffdd">+ strcpy(buf_ptr, ref->rec.name);
+ buf_ptr += strlen(ref->rec.name) + 1;
</span>
/* write this hash entry to log file */
wrote = darshan_log_dzwrite(fd, DARSHAN_REC_MAP_REGION_ID,
</code></pre>
<br>
</li>
<li id='diff-2'>
<a href='https://xgitlab.cels.anl.gov/darshan/darshan/compare/9193b34aaf7e9f9d1e5361dfb44667d14277f23f...66a83b0382dade07f600a43bf50707be39e63ef2#diff-2'>
<strong>
darshan-util/darshan-stitch-logs.c
</strong>
</a>
<hr>
<pre class="highlight"><code><span style="color: #000000;background-color: #ffdddd">--- a/darshan-util/darshan-stitch-logs.c
</span><span style="color: #000000;background-color: #ddffdd">+++ b/darshan-util/darshan-stitch-logs.c
</span><span style="color: #aaaaaa">@@ -1,12 +1,13 @@
</span>-#include <stdio.h>
<span style="color: #000000;background-color: #ddffdd">+ #include <stdio.h>
</span> #include <stdlib.h>
#include <fcntl.h>
#include <glob.h>
#include <string.h>
-#include <assert.h>
#include "darshan-logutils.h"
<span style="color: #000000;background-color: #ddffdd">+#define DEF_MOD_BUF_SIZE 1024 /* 1 KiB is enough for all current mod records ... */
+
</span> int logfile_path_comp(const void *a, const void *b)
{
char *pathA = *(char **)a;
<span style="color: #aaaaaa">@@ -30,30 +31,30 @@ int logfile_path_comp(const void *a, const void *b)
</span>
int main(int argc, char *argv[])
{
<span style="color: #000000;background-color: #ffdddd">- darshan_fd out_fd;
- char *out_logname; /* XXX default + configurable? */
- int ret;
- int i, j;
</span> char *tmplog_dir;
int job_id;
glob_t globbuf;
char glob_pstr[512];
<span style="color: #000000;background-color: #ffdddd">- int tmp_fd;
- struct darshan_job in_job, out_job;
- char in_exe_mnt[DARSHAN_EXE_LEN+1];
- char *mnt_s, *pos;
- char out_exe[DARSHAN_EXE_LEN+1];
- char **out_mnt_pts;
- char **out_fs_types;
- int out_mnt_count = 0;
</span><span style="color: #000000;background-color: #ddffdd">+ char *stitch_logname = "/tmp/test123.darshan"; /* XXX default + configurable? */
+ darshan_fd in_fd, stitch_fd;
+ struct darshan_job in_job, stitch_job;
+ char stitch_exe[DARSHAN_EXE_LEN+1];
+ char **stitch_mnt_pts;
+ char **stitch_fs_types;
+ int stitch_mnt_count = 0;
+ struct darshan_record_ref *in_hash = NULL;
+ struct darshan_record_ref *stitch_hash = NULL;
+ struct darshan_record_ref *ref, *tmp, *found;
+ darshan_record_id rec_id;
+ char *mod_buf;
+ int i, j;
+ int ret;
</span>
/* TODO: are there any checks we should do to ensure tmp logs belong to the same job */
/* we can't specifically check the job id, since the pid is used if no job scheduler */
/* TODO: how do we set the output logfile name to be unique, and have necessary semantic info contained */
<span style="color: #000000;background-color: #ffdddd">- /* TODO: more thorough way of cleaning up (free, close, etc.) */
</span>-
if(argc != 3)
{
fprintf(stderr, "Usage: ./darshan-stitch-tmplogs <tmp_dir> <job_id>\n"
<span style="color: #aaaaaa">@@ -83,235 +84,241 @@ int main(int argc, char *argv[])
</span> */
qsort(globbuf.gl_pathv, globbuf.gl_pathc, sizeof(char *), logfile_path_comp);
<span style="color: #000000;background-color: #ffdddd">- memset(&out_job, 0, sizeof(struct darshan_job));
</span><span style="color: #000000;background-color: #ddffdd">+ memset(&stitch_job, 0, sizeof(struct darshan_job));
</span>
/* first pass at stitching together logs:
<span style="color: #000000;background-color: #ffdddd">- * - compose a final job-level metadata structure
</span><span style="color: #000000;background-color: #ddffdd">+ * - compose output job-level metadata structure (including exe & mount data)
+ * - compose output record_id->file_name mapping
</span> */
for(i = 0; i < globbuf.gl_pathc; i++)
{
memset(&in_job, 0, sizeof(struct darshan_job));
<span style="color: #000000;background-color: #ffdddd">- tmp_fd = open(globbuf.gl_pathv[i], O_RDONLY);
- if(tmp_fd < 0)
</span><span style="color: #000000;background-color: #ddffdd">+ in_fd = darshan_log_open(globbuf.gl_pathv[i]);
+ if(in_fd == NULL)
</span> {
fprintf(stderr,
<span style="color: #000000;background-color: #ffdddd">- "Error: unable to open Darshan log file %s.\n",
</span><span style="color: #000000;background-color: #ddffdd">+ "Error: unable to open input Darshan log file %s.\n",
</span> globbuf.gl_pathv[i]);
<span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
</span> return(-1);
}
/* read job-level metadata from the input file */
<span style="color: #000000;background-color: #ffdddd">- ret = pread(tmp_fd, &in_job, sizeof(struct darshan_job), 0);
- if(ret < sizeof(struct darshan_job))
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_getjob(in_fd, &in_job);
+ if(ret < 0)
</span> {
fprintf(stderr,
<span style="color: #000000;background-color: #ffdddd">- "Error: unable to read job data from Darshan log file %s\n",
</span><span style="color: #000000;background-color: #ddffdd">+ "Error: unable to read job data from input Darshan log file %s.\n",
</span> globbuf.gl_pathv[i]);
<span style="color: #000000;background-color: #ddffdd">+ darshan_log_close(in_fd);
+ globfree(&globbuf);
</span> return(-1);
}
if(i == 0)
{
<span style="color: #000000;background-color: #ffdddd">- /* get all job data from the first input log */
- memcpy(&out_job, &in_job, sizeof(struct darshan_job));
</span><span style="color: #000000;background-color: #ddffdd">+ /* get job data, exe, & mounts directly from the first input log */
+ memcpy(&stitch_job, &in_job, sizeof(struct darshan_job));
</span>
<span style="color: #000000;background-color: #ffdddd">- ret = pread(tmp_fd, in_exe_mnt, DARSHAN_EXE_LEN+1, sizeof(struct darshan_job));
- if(ret < DARSHAN_EXE_LEN+1)
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_getexe(in_fd, stitch_exe);
+ if(ret < 0)
</span> {
fprintf(stderr,
<span style="color: #000000;background-color: #ffdddd">- "Error: unable to read exe & mount data from Darshan log file %s\n",
</span><span style="color: #000000;background-color: #ddffdd">+ "Error: unable to read exe string from Darshan log file %s.\n",
</span> globbuf.gl_pathv[i]);
<span style="color: #000000;background-color: #ddffdd">+ darshan_log_close(in_fd);
+ globfree(&globbuf);
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- /* get the exe string */
- mnt_s = strchr(in_exe_mnt, '\n');
- memcpy(out_exe, in_exe_mnt, mnt_s-in_exe_mnt);
- out_exe[mnt_s-in_exe_mnt] = '\0';
</span>-
<span style="color: #000000;background-color: #ffdddd">- /* build mount structures */
- pos = mnt_s;
- while((pos = strchr(pos, '\n')) != NULL)
- {
- pos++;
- out_mnt_count++;
- }
</span>-
<span style="color: #000000;background-color: #ffdddd">- if(out_mnt_count)
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_getmounts(in_fd, &stitch_mnt_pts,
+ &stitch_fs_types, &stitch_mnt_count);
+ if(ret < 0)
</span> {
<span style="color: #000000;background-color: #ffdddd">- out_mnt_pts = malloc(out_mnt_count * sizeof(char *));
- assert(out_mnt_pts);
- out_fs_types = malloc(out_mnt_count * sizeof(char *));
- assert(out_fs_types);
</span>-
<span style="color: #000000;background-color: #ffdddd">- /* work backwards through the table and parse each line (except for
- * first, which holds command line information)
- */
- j = 0;
- while((pos = strrchr(in_exe_mnt, '\n')) != NULL)
- {
- /* overestimate string lengths */
- out_mnt_pts[j] = malloc(DARSHAN_EXE_LEN);
- assert(out_mnt_pts[j]);
- out_fs_types[j] = malloc(DARSHAN_EXE_LEN);
- assert(out_fs_types[j]);
</span>-
<span style="color: #000000;background-color: #ffdddd">- ret = sscanf(++pos, "%s\t%s", out_fs_types[j], out_mnt_pts[j]);
- if(ret != 2)
- {
- fprintf(stderr,
- "Error: poorly formatted mount table in darshan log file %s.\n",
- globbuf.gl_pathv[i]);
- return(-1);
- }
- pos--;
- *pos = '\0';
- j++;
- }
</span><span style="color: #000000;background-color: #ddffdd">+ fprintf(stderr,
+ "Error: unable to read mount info from Darshan log file %s.\n",
+ globbuf.gl_pathv[i]);
+ darshan_log_close(in_fd);
+ globfree(&globbuf);
+ return(-1);
</span> }
}
else
{
<span style="color: #000000;background-color: #ffdddd">- /* potentially update job timestamps */
- if(in_job.start_time < out_job.start_time)
- out_job.start_time = in_job.start_time;
- if(in_job.end_time > out_job.end_time)
- out_job.end_time = in_job.end_time;
</span><span style="color: #000000;background-color: #ddffdd">+ /* potentially update job timestamps using remaining logs */
+ if(in_job.start_time < stitch_job.start_time)
+ stitch_job.start_time = in_job.start_time;
+ if(in_job.end_time > stitch_job.end_time)
+ stitch_job.end_time = in_job.end_time;
</span> }
<span style="color: #000000;background-color: #ffdddd">- close(tmp_fd);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_gethash(in_fd, &in_hash);
+ if(ret < 0)
+ {
+ fprintf(stderr,
+ "Error: unable to read job data from input Darshan log file %s.\n",
+ globbuf.gl_pathv[i]);
+ darshan_log_close(in_fd);
+ globfree(&globbuf);
+ return(-1);
+ }
+
+ /* iterate the input hash, copying over record_id->file_name mappings
+ * that have not already been copied to the output hash
+ */
+ HASH_ITER(hlink, in_hash, ref, tmp)
+ {
+ HASH_FIND(hlink, stitch_hash, &(ref->rec.id),
+ sizeof(darshan_record_id), found);
+ if(!found)
+ {
+ HASH_ADD(hlink, stitch_hash, rec.id,
+ sizeof(darshan_record_id), ref);
+ }
+ else if(strcmp(ref->rec.name, found->rec.name))
+ {
+ fprintf(stderr,
+ "Error: invalid Darshan record table entry.\n");
+ darshan_log_close(in_fd);
+ globfree(&globbuf);
+ return(-1);
+ }
+ }
+
+ darshan_log_close(in_fd);
</span> }
/* create the output "stitched together" log */
<span style="color: #000000;background-color: #ffdddd">- out_fd = darshan_log_create("/tmp/test123.darshan", DARSHAN_ZLIB_COMP, 1);
- if(out_fd == NULL)
</span><span style="color: #000000;background-color: #ddffdd">+ stitch_fd = darshan_log_create(stitch_logname, DARSHAN_ZLIB_COMP, 1);
+ if(stitch_fd == NULL)
</span> {
fprintf(stderr, "Error: unable to create output darshan log.\n");
<span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
</span> return(-1);
}
/* write the darshan job info, exe string, and mount data to output file */
<span style="color: #000000;background-color: #ffdddd">- ret = darshan_log_putjob(out_fd, &out_job);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_putjob(stitch_fd, &stitch_job);
</span> if(ret < 0)
{
fprintf(stderr, "Error: unable to write job data to output darshan log.\n");
<span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- ret = darshan_log_putexe(out_fd, out_exe);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_putexe(stitch_fd, stitch_exe);
</span> if(ret < 0)
{
fprintf(stderr, "Error: unable to write exe string to output darshan log.\n");
<span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- ret = darshan_log_putmounts(out_fd, out_mnt_pts, out_fs_types, out_mnt_count);
</span><span style="color: #000000;background-color: #ddffdd">+ ret = darshan_log_putmounts(stitch_fd, stitch_mnt_pts, stitch_fs_types, stitch_mnt_count);
</span> if(ret < 0)
{
fprintf(stderr, "Error: unable to write mount data to output darshan log.\n");
<span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
</span> return(-1);
}
<span style="color: #000000;background-color: #ffdddd">- void *mod_buf = malloc(2*1024*1024);
- assert(mod_buf);
</span><span style="color: #000000;background-color: #ddffdd">+ /* write the stitched together table of records to output file */
+ ret = darshan_log_puthash(stitch_fd, stitch_hash);
+ if(ret < 0)
+ {
+ fprintf(stderr, "Error: unable to write record table to output darshan log.\n");
+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
+ return(-1);
+ }
</span>
<span style="color: #000000;background-color: #ffdddd">- /* second pass at stitching together logs:
- * - append module data to output log file
- */
- for(i = 0; i < globbuf.gl_pathc; i++)
</span><span style="color: #000000;background-color: #ddffdd">+ mod_buf = malloc(DEF_MOD_BUF_SIZE);
+ if(!mod_buf)
</span> {
<span style="color: #000000;background-color: #ffdddd">- tmp_fd = open(globbuf.gl_pathv[i], O_RDONLY);
- if(tmp_fd < 0)
</span><span style="color: #000000;background-color: #ddffdd">+ globfree(&globbuf);
+ darshan_log_close(stitch_fd);
+ unlink(stitch_logname);
+ return(-1);
+ }
+ memset(mod_buf, 0, DEF_MOD_BUF_SIZE);
+
+ for(i = 0; i < DARSHAN_MPIIO_MOD; i++)
+ {
+ if(!mod_logutils[i]) continue;
+
+#if 0
+ /* XXX first build shared record list? */
+ for(j = 0; j < globbuf.gl_pathc; j++)
</span> {
<span style="color: #000000;background-color: #ffdddd">- fprintf(stderr,
- "Error: unable to open Darshan log file %s.\n",
- globbuf.gl_pathv[i]);
- return(-1);
</span><span style="color: #000000;background-color: #ddffdd">+
</span> }
<span style="color: #000000;background-color: #ffdddd">- ret = pread(tmp_fd, mod_buf, 2*1024*1024, 4*1024);
- if(ret < (2*1024*1024))
</span><span style="color: #000000;background-color: #ddffdd">+ /* XXX second aggregate shared records ? */
+ for(j = 0; j < globbuf.gl_pathc; j++)
</span> {
<span style="color: #000000;background-color: #ffdddd">- fprintf(stderr,
- "Error: unable to read module data from Darshan log file %s\n",
- globbuf.gl_pathv[i]);
- return(-1);
</span><span style="color: #000000;background-color: #ddffdd">+
</span> }
<span style="color: #000000;background-color: #ddffdd">+#endif
</span>
<span style="color: #000000;background-color: #ffdddd">- /* iterate and write file records */
- void *mod_buf_p = mod_buf;
- while(1)
</span><span style="color: #000000;background-color: #ddffdd">+ /* XXX third write each rank's blobs, with rank 0 writing the shared ones ? */
+ for(j = 0; j < globbuf.gl_pathc; j++)
</span> {
<span style="color: #000000;background-color: #ffdddd">-
</span>-#if 0
<span style="color: #000000;background-color: #ffdddd">- ret = darshan_log_put_posix_file(out_fd, VOID*);
</span><span style="color: #000000;background-color: #ddffdd">+ in_fd = darshan_log_open(globbuf.gl_pathv[j]);
+ if(in_fd == NULL)
+ {
+ fprintf(stderr,
+ "Error: unable to open input Darshan log file %s.\n",
+ globbuf.gl_pathv[j]);
+ free(mod_buf);
+ globfree(&globbuf);
+ darshan_log_close(in_fd);
+ unlink(stitch_logname);
+ return(-1);
+ }
+
+ /* loop over module records and write them to output file */
+ while((ret = mod_logutils[i]->log_get_record(in_fd, mod_buf, &rec_id)) == 1)
+ {
+ ret = mod_logutils[i]->log_put_record(stitch_fd, mod_buf);
+ if(ret < 0)
+ {
+ fprintf(stderr,
+ "Error: unable to write %s module record to output log file %s.\n",
+ darshan_module_names[i], globbuf.gl_pathv[j]);
+ free(mod_buf);
+ darshan_log_close(in_fd);
+ unlink(stitch_logname);
+ return(-1);
+ }
+
+ memset(mod_buf, 0, DEF_MOD_BUF_SIZE);
+ }
</span> if(ret < 0)
{
fprintf(stderr,
<span style="color: #000000;background-color: #ffdddd">- "Error: unable to write module record to output darshan log.\n");
</span><span style="color: #000000;background-color: #ddffdd">+ "Error: unable to read %s module record from input log file %s.\n",
+ darshan_module_names[i], globbuf.gl_pathv[j]);
+ free(mod_buf);
+ darshan_log_close(in_fd);
+ unlink(stitch_logname);
</span> return(-1);
}
-#endif
<span style="color: #000000;background-color: #ddffdd">+ darshan_log_close(in_fd);
</span> }
-
<span style="color: #000000;background-color: #ffdddd">- close(tmp_fd);
</span> }
<span style="color: #000000;background-color: #ffdddd">- darshan_log_close(out_fd);
</span>-
<span style="color: #000000;background-color: #ffdddd">- free(mod_buf);
</span><span style="color: #000000;background-color: #ddffdd">+ darshan_log_close(stitch_fd);
</span> globfree(&globbuf);
<span style="color: #000000;background-color: #ddffdd">+ free(mod_buf);
</span>
return(0);
}
-
-#if 0
<span style="color: #000000;background-color: #ffdddd">- /* print out job info to sanity check */
- time_t tmp_time = 0;
- char *token;
- char *save;
- char buffer[DARSHAN_JOB_METADATA_LEN];
- #include <time.h>
- printf("# exe: %s\n", out_exe);
- printf("# uid: %" PRId64 "\n", out_job.uid);
- printf("# jobid: %" PRId64 "\n", out_job.jobid);
- printf("# start_time: %" PRId64 "\n", out_job.start_time);
- tmp_time += out_job.start_time;
- printf("# start_time_asci: %s", ctime(&tmp_time));
- printf("# end_time: %" PRId64 "\n", out_job.end_time);
- tmp_time = 0;
- tmp_time += out_job.end_time;
- printf("# end_time_asci: %s", ctime(&tmp_time));
- printf("# nprocs: %" PRId64 "\n", out_job.nprocs);
- printf("# run time: %" PRId64 "\n", out_job.end_time - out_job.start_time + 1);
- for(token=strtok_r(out_job.metadata, "\n", &save);
- token != NULL;
- token=strtok_r(NULL, "\n", &save))
- {
- char *key;
- char *value;
- /* NOTE: we intentionally only split on the first = character.
- * There may be additional = characters in the value portion
- * (for example, when storing mpi-io hints).
- */
- strcpy(buffer, token);
- key = buffer;
- value = index(buffer, '=');
- if(!value)
- continue;
- /* convert = to a null terminator to split key and value */
- value[0] = '\0';
- value++;
- printf("# metadata: %s = %s\n", key, value);
- }
</span>-
<span style="color: #000000;background-color: #ffdddd">- /* print table of mounted file systems */
- printf("\n# mounted file systems (mount point and fs type)\n");
- printf("# -------------------------------------------------------\n");
- for(j=0; j<out_mnt_count; j++)
- {
- printf("# mount entry:\t%s\t%s\n", out_mnt_pts[j], out_fs_types[j]);
- }
</span>-#endif
</code></pre>
<br>
</li>
</div>
<div class='footer' style='margin-top: 10px;'>
<p>
—
<br>
<a href="https://xgitlab.cels.anl.gov/darshan/darshan/compare/9193b34aaf7e9f9d1e5361dfb44667d14277f23f...66a83b0382dade07f600a43bf50707be39e63ef2">View it on GitLab</a>
<script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","action":{"@type":"ViewAction","name":["merge_requests","issues","commit"],"url":"https://xgitlab.cels.anl.gov/darshan/darshan/compare/9193b34aaf7e9f9d1e5361dfb44667d14277f23f...66a83b0382dade07f600a43bf50707be39e63ef2"}}</script>
</p>
</div>
</body>
</html>