[Darshan-commits] [Git][darshan/darshan][master] 4 commits: omit outputting with old-style fopen counters

[Shane Snyder]

Shane Snyder pushed to branch master at darshan / darshan


Commits:
615f5396 by Phil Carns at 2018-04-02T16:59:35-04:00
omit outputting with old-style fopen counters

- these cannot be easily translated into Darshan 3.1.x format
- fixes #241

- - - - -
b4ec5840 by Shane Snyder at 2018-06-11T15:22:34-05:00
Merge branch 'master' into 'issue-241-fopen-upconvert'

# Conflicts:
#   ChangeLog
- - - - -
1980d236 by Shane Snyder at 2018-06-11T15:25:47-05:00
add another comment for more details

- - - - -
2c40940f by Shane Snyder at 2018-06-11T15:26:32-05:00
Merge branch 'issue-241-fopen-upconvert' into 'master'

omit outputting with old-style fopen counters

Closes #241

See merge request darshan/darshan!22
- - - - -


2 changed files:

- ChangeLog
- darshan-util/darshan-posix-logutils.c


Changes:

=====================================
ChangeLog
=====================================
--- a/ChangeLog
+++ b/ChangeLog
@@ -4,6 +4,9 @@ Darshan Release Change Log
 
 Darshan-3.1.7
 =============
+* bug fix for darshan-parser backwards compatibility: Darshan logs generated
+  by Darshan versions prior to 3.1.0 may have included stdio counters that
+  were not properly up-converted.  Reported by Teng Wang.
 * bug fix to MiB reported in I/O performance estimate of
   darshan-job-summary.pl when both posix and stdio access is present
 


=====================================
darshan-util/darshan-posix-logutils.c
=====================================
--- a/darshan-util/darshan-posix-logutils.c
+++ b/darshan-util/darshan-posix-logutils.c
@@ -86,13 +86,27 @@ static int darshan_log_get_posix_file(darshan_fd fd, void** posix_buf_p)
         if(fd->mod_ver[DARSHAN_POSIX_MOD] == 1)
         {
             rec_len = DARSHAN_POSIX_FILE_SIZE_1;
-            ret = darshan_log_get_mod(fd, DARSHAN_POSIX_MOD, scratch, rec_len);
+            int64_t *fopen_counter;
+            
+            /* This version of the posix module had some stdio counters
+             * mixed in with the posix counters.  If the fopen counters are
+             * 0, then we can simply update the record format to skip those
+             * counters.  If the fopen counters are non-zero, then we omit
+             * the entire record because there is no clean way to properly
+             * up-convert it.
+             */
+            dest_p = scratch + (sizeof(struct darshan_base_record) + 
+                (6 * sizeof(int64_t)));
+            fopen_counter = (int64_t*)dest_p;
+            do
+            {
+                /* pull POSIX records until we find one that doesn't have STDIO data */
+                ret = darshan_log_get_mod(fd, DARSHAN_POSIX_MOD, scratch, rec_len);
+            } while(ret == rec_len && *fopen_counter > 0);
             if(ret != rec_len)
                 goto exit;
 
             /* upconvert version 1 to version 2 in-place */
-            dest_p = scratch + (sizeof(struct darshan_base_record) + 
-                (6 * sizeof(int64_t)));
             src_p = dest_p + (4 * sizeof(int64_t));
             len = rec_len - (src_p - scratch);
             memmove(dest_p, src_p, len);
@@ -230,9 +244,9 @@ static void darshan_log_print_posix_description(int ver)
     if(ver <= 1)
     {
         printf("\n# WARNING: POSIX module log format version 1 has the following limitations:\n");
-        printf("# - Partial instrumentation of stdio stream I/O functions not parsable by Darshan versions >= 3.1.0\n");
-        printf("#     * Using darshan-logutils versions < 3.1.0, this data can be found in the following POSIX counters:\n");
-        printf("#         * POSIX_FOPENS, POSIX_FREADS, POSIX_FWRITES, POSIX_FSEEKS\n");
+        printf("# - Darshan version 3.1.0 and earlier had only partial instrumentation of stdio stream I/O functions.\n");
+        printf("#   File records with stdio counters present will be omitted from output.\n");
+        printf("#   Use darshan-logutils < 3.1.0 to retrieve those records.\n");
     }
     if(ver <= 2)
     {



View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/81683bb50128ef220a8be8d52fa6d85cbb686ed6...2c40940ff465c5d0822cb099e8273102d30e2748

---
View it on GitLab: https://xgitlab.cels.anl.gov/darshan/darshan/compare/81683bb50128ef220a8be8d52fa6d85cbb686ed6...2c40940ff465c5d0822cb099e8273102d30e2748
You're receiving this email because of your account on xgitlab.cels.anl.gov.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-commits/attachments/20180611/fd5b80c7/attachment-0001.html>