<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks for the suggestion, Sam! I'll make sure that particular warning is cleared up in our next release, because that is confusing (since, as you point out, all of our other utilities work directly on Darshan log files, not text output).</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We also will have new Python bindings to our C utility library in our next release, that should allow us to simplify things like dxt_analyzer -- that would allow us to skip the step of converting the log to text, as Python applications would be able to operate
on Darshan logs directly. We'll make sure to keep the community posted about developments on that front.
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
--Shane<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on behalf of Omlin Samuel <omlin@cscs.ch><br>
<b>Sent:</b> Tuesday, October 13, 2020 6:01 AM<br>
<b>To:</b> Harms, Kevin <harms@alcf.anl.gov>; Carns, Philip H. <carns@mcs.anl.gov>; darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Darshan-users] dxt_analyzer.py: 'invalid continuation byte' error</font>
<div> </div>
</div>
<style>
<!--
.x_EmailQuote
{margin-left:1pt;
padding-left:4pt;
border-left:#800000 2px solid}
-->
</style>
<div>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Hi Kevin,</p>
<p><br>
</p>
<p>thanks a lot for this clarification on the usage of the DXT-related tools. This worked for me as well. Thus, it is only a documentation issue. Calling `<span>dxt_analyzer.py --help</span>` prints:<br>
```<br>
</p>
<div>-i DXT_LOGNAME, --input DXT_LOGNAME<br>
dxt log path<br>
<span>```</span></div>
<div>It would be good probably to indicate here that this input file should be the output of the
<font size="2"><span style="font-size:10pt">darshan-dxt-parser</span></font> and not a .darshan file, in particular as, e.g.,
<span>darshan-job-summary.pl takes directly a .darshan file as input.</span></div>
<div><span><br>
</span></div>
<div><span>Thanks a lot!</span></div>
<div><span><br>
</span></div>
<div><span>Sam</span></div>
<div><span><br>
</span></div>
<div><span>PS: <br>
> <font size="2"><span style="font-size:10pt">Looking at the plot from run, it looks like you were possibly doing I/O almost constantly during the run.</span></font><br>
</span></div>
<div><span>- Yes this is correct: the profiled application is an I/O-performance test.</span><br>
</div>
<p></p>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Harms, Kevin <harms@alcf.anl.gov><br>
<b>Sent:</b> Tuesday, October 13, 2020 1:19:02 AM<br>
<b>To:</b> Omlin Samuel; Carns, Philip H.; darshan-users@lists.mcs.anl.gov<br>
<b>Subject:</b> Re: [Darshan-users] dxt_analyzer.py: 'invalid continuation byte' error</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt">
<div class="x_PlainText">Sam,<br>
<br>
I think there was just a misunderstanding on how to use the tools. Please follow this:<br>
<br>
# first parse the DXT log from the darshan log.<br>
$ ./darshan-dxt-parser ./omlins_a.out_id1174947_10-9-40448-1694713884344482404_1602235659.darshan > parsed-dxt.txt<br>
<br>
# then run the analyzer<br>
$ python3 dxt_analyzer.py -i parsed-dxt.txt<br>
<br>
# this produces a PDF with the plot<br>
$ ls *.pdf<br>
dxt_plot.pdf<br>
<br>
Looking at the plot from run, it looks like you were possibly doing I/O almost constantly during the run.<br>
<br>
kevin<br>
<br>
________________________________________<br>
From: Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on behalf of Omlin Samuel <omlin@cscs.ch><br>
Sent: Monday, October 12, 2020 8:10 AM<br>
To: Carns, Philip H.; darshan-users@lists.mcs.anl.gov<br>
Subject: Re: [Darshan-users] dxt_analyzer.py: 'invalid continuation byte' error<br>
<br>
Hi Phil,<br>
<br>
<br>
thanks a lot. You can find the file attached.<br>
<br>
<br>
Cheers,<br>
<br>
<br>
Sam<br>
<br>
________________________________<br>
From: Carns, Philip H. <carns@mcs.anl.gov><br>
Sent: Friday, October 9, 2020 10:37:41 PM<br>
To: Omlin Samuel; darshan-users@lists.mcs.anl.gov<br>
Subject: Re: [Darshan-users] dxt_analyzer.py: 'invalid continuation byte' error<br>
<br>
Hi Sam,<br>
<br>
I'm not sure off hand what would cause that. Can you share the log file by any chance (you can send it to me directly if you would prefer not to post it on the mailing list)? I can try to reproduce on some other systems and see if that narrows down the problem.<br>
<br>
thanks,<br>
-Phil<br>
________________________________<br>
From: Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on behalf of Omlin Samuel <omlin@cscs.ch><br>
Sent: Friday, October 9, 2020 10:18 AM<br>
To: darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
Subject: [Darshan-users] dxt_analyzer.py: 'invalid continuation byte' error<br>
<br>
Hi all,<br>
<br>
I have tried the following command both on Ubuntu 16.04 (python 3.5) and on SUSE Linux Enterprise Server 15 SP1 (python 3.6) and got the same error:<br>
```<br>
$ python3 /usr/local/bin/dxt_analyzer.py -i omlins_a.out_id1174947_10-9-40448-1694713884344482404_1602235659.darshan<br>
Traceback (most recent call last):<br>
File "/usr/local/bin/dxt_analyzer.py", line 264, in <module><br>
for line in infile:<br>
File "/usr/lib/python3.5/codecs.py", line 321, in decode<br>
(result, consumed) = self._buffer_decode(data, self.errors, final)<br>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 24: invalid continuation byte<br>
```<br>
<br>
With python 2.7.12 (on Ubuntu), I did not get the error, but the generated plot was empty.<br>
<br>
Note that the input file does contain values generated by DXT:<br>
```<br>
$ darshan-dxt-parser omlins_a.out_id1174947_10-9-40448-1694713884344482404_1602235659.darshan | tail -3<br>
X_POSIX 7 read 197 211527180650 1073741824 804.8381 806.1340 [ 0]<br>
X_POSIX 7 read 198 212600922683 1073741824 806.4404 807.7277 [ 0]<br>
X_POSIX 7 read 199 213674664716 1073741824 808.0719 809.3472 [ 0]<br>
```<br>
<br>
Can you fix this / help me to get this working?<br>
<br>
Thanks!!<br>
<br>
Sam<br>
<br>
<br>
<br>
<br>
</div>
</span></font></div>
</body>
</html>