<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi Wadud,<br>
<br>
Could you try regenerating the summary report, but pass the
'--verbose' flag to darshan-job-summary script?<br>
<br>
That should create a directory in /tmp that stores all the temporary
files used by that script. Included will be a 'summary.log' file
that contains the output from each step of generating the summary.
Do you happen to see any errors or indication of what went wrong in
there? It could be that you are missing some LaTeX or gnuplot
prerequisite packages or something like that.<br>
<br>
Thanks,<br>
--Shane<br>
<br>
<div class="moz-cite-prefix">On 11/30/2016 05:33 AM, Wadud Miah
wrote:<br>
</div>
<blockquote
cite="mid:AM4PR0501MB2707D59DE69B123AA8B7BA40C08C0@AM4PR0501MB2707.eurprd05.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Hi
again Shane,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">When
I try to create the job summary report, the command gets
stuck:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">[wadud@WM-porlock
~]$ darshan-job-summary.pl
pr1ebt05_bt.A.9.mpi_io_full_id2483888_11-29-65132-8929929721473377761_1.darshan<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Slowest
unique file time: 0.001923<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Slowest
shared file time: 0.130845<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Total
bytes read and written by app (may be incorrect): 104858212<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Total
absolute I/O time: 0.132768<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">**NOTE:
above shared and unique file times calculated using MPI-IO
timers if MPI-IO interface used on a given file, POSIX
timers otherwise.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">This
has happened also on my laptop. When I run “top -H” I don’t
see the process doing anything. Any help will be
appreciated.
<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Wadud.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"
lang="EN-US">From:</span></b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"
lang="EN-US"> Shane Snyder [<a class="moz-txt-link-freetext" href="mailto:ssnyder@mcs.anl.gov">mailto:ssnyder@mcs.anl.gov</a>]
<br>
<b>Sent:</b> 29 November 2016 20:50<br>
<b>To:</b> Wadud Miah <a class="moz-txt-link-rfc2396E" href="mailto:wadud.miah@nag.co.uk"><wadud.miah@nag.co.uk></a>;
<a class="moz-txt-link-abbreviated" href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
<b>Subject:</b> Re: [Darshan-users] Darshan trace file
creation<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Wadud,<br>
<br>
I'm assuming this is a Fortran app, right?<br>
<br>
If so, I believe this is tied back to a known issue with Intel
MPI's Fortran bindings. Long story short, these bindings are
calling the PMPI variants of the MPI function calls directly
which bypasses any profiling tool that wraps the MPI interface
at the C layer (e.g., Darshan). See this post for more
details: <a moz-do-not-send="true"
href="https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/279445">https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/279445</a><br>
<br>
I haven't personally confirmed this on Intel MPI 5.x versions
(the post I reference above is regarding a 4.x version of
Intel MPI), but I believe other users have reported the
problem still exists.<br>
<br>
It should be possible to work around the issue by modifying
Darshan to provide an additional interface for intercepting
Fortran MPI calls natively at the Fortran layer, but we have
not undertaken this development effort as of yet. Lacking
that, I'm not sure what, if any, other solutions there are,
unfortunately.<br>
<br>
Thanks,<br>
--Shane<o:p></o:p></p>
<div>
<p class="MsoNormal">On 11/28/2016 02:58 PM, Wadud Miah wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div id="divtagdefaultwrapper">
<div>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Hello,
<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">I
built Darshan with Intel MPI 5.1.3 and gcc 4.3.4 with
the following configure flags:<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><code><span
style="font-size:11.0pt">./configure
--prefix=$HOME/darshan-3.1.2-impi --with-mem-align=8
--with-jobid-env=NONE
--with-log-path-by-env=DARSHAN_LOGPATH</span></code><span
style="font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">which
built fine. I set
</span><code><span style="font-size:10.0pt">DARSHAN_LOGPATH</span></code><span
style="font-family:"Calibri",sans-serif"> to
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span
style="font-family:"Calibri",sans-serif">
and I created the directory
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span
style="font-family:"Calibri",sans-serif">. I
then executed my MPI code which does I/O using:<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><code><span
style="font-size:11.0pt">mpirun -n 9 -env LD_PRELOAD
/home/pr1ebt00/pr1ebt05/darshan-3.1.2-impi/lib/libdarshan.so
./bt.A.9.mpi_io_full</span></code><span
style="font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">in
the same directory that has the
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span
style="font-family:"Calibri",sans-serif">
directory. However, I do not see any tracefile in the
directory
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span
style="font-family:"Calibri",sans-serif">
and no error messages. Any help will be greatly
appreciated.
<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Thanks
in advance,
<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Wadud.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div id="Signature">
<div id="divtagdefaultwrapper">
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">---------------------------------------<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Dr.
Wadud Miah<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Computational
Scientist<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">Numerical
Algorithms Group<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif">01865
518035<o:p></o:p></span></p>
<p style="background:white"><span
style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr size="2" width="100%" align="center">
</div>
<p>The Numerical Algorithms Group Ltd is a company registered
in England and Wales with company number 1249803. The
registered office is:<o:p></o:p></p>
<p>Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United
Kingdom.<o:p></o:p></p>
<p> <o:p></o:p></p>
<p>This e-mail has been scanned for all viruses by Microsoft
Office 365. <o:p></o:p></p>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Darshan-users mailing list<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
________________________________________________________________________<br>
This e-mail has been scanned for all viruses by Star.<br>
________________________________________________________________________<o:p></o:p></p>
</div>
<hr width="100%">
<p>The Numerical Algorithms Group Ltd is a company registered in
England and Wales with company number 1249803. The registered
office is:</p>
<p>Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United
Kingdom.</p>
<p> </p>
<p>This e-mail has been scanned for all viruses by Microsoft
Office 365. </p>
<hr width="100%">
</blockquote>
<br>
</body>
</html>