<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
code
{mso-style-priority:99;
font-family:"Courier New";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Hi Shane,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Many thanks for letting me know about Intel MPI’s issue with Fortran. Yes, this is for a Fortran code. It seems like Intel MPI version
2017.0.098 has fixed this issue as it is now creating trace files. You may now close this ticket.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US">Wadud.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> Shane Snyder [mailto:ssnyder@mcs.anl.gov]
<br>
<b>Sent:</b> 29 November 2016 20:50<br>
<b>To:</b> Wadud Miah <wadud.miah@nag.co.uk>; darshan-users@lists.mcs.anl.gov<br>
<b>Subject:</b> Re: [Darshan-users] Darshan trace file creation<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Wadud,<br>
<br>
I'm assuming this is a Fortran app, right?<br>
<br>
If so, I believe this is tied back to a known issue with Intel MPI's Fortran bindings. Long story short, these bindings are calling the PMPI variants of the MPI function calls directly which bypasses any profiling tool that wraps the MPI interface at the C
layer (e.g., Darshan). See this post for more details: <a href="https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/279445">
https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/279445</a><br>
<br>
I haven't personally confirmed this on Intel MPI 5.x versions (the post I reference above is regarding a 4.x version of Intel MPI), but I believe other users have reported the problem still exists.<br>
<br>
It should be possible to work around the issue by modifying Darshan to provide an additional interface for intercepting Fortran MPI calls natively at the Fortran layer, but we have not undertaken this development effort as of yet. Lacking that, I'm not sure
what, if any, other solutions there are, unfortunately.<br>
<br>
Thanks,<br>
--Shane<o:p></o:p></p>
<div>
<p class="MsoNormal">On 11/28/2016 02:58 PM, Wadud Miah wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div id="divtagdefaultwrapper">
<div>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Hello,
<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">I built Darshan with Intel MPI 5.1.3 and gcc 4.3.4 with the following configure flags:<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><code><span style="font-size:11.0pt">./configure --prefix=$HOME/darshan-3.1.2-impi --with-mem-align=8 --with-jobid-env=NONE --with-log-path-by-env=DARSHAN_LOGPATH</span></code><span style="font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">which built fine. I set
</span><code><span style="font-size:10.0pt">DARSHAN_LOGPATH</span></code><span style="font-family:"Calibri",sans-serif"> to
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span style="font-family:"Calibri",sans-serif"> and I created the directory
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span style="font-family:"Calibri",sans-serif">. I then executed my MPI code which does I/O using:<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><code><span style="font-size:11.0pt">mpirun -n 9 -env LD_PRELOAD /home/pr1ebt00/pr1ebt05/darshan-3.1.2-impi/lib/libdarshan.so ./bt.A.9.mpi_io_full</span></code><span style="font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">in the same directory that has the
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span style="font-family:"Calibri",sans-serif"> directory. However, I do not see any tracefile in the directory
</span><code><span style="font-size:10.0pt">darshan-trace-dir</span></code><span style="font-family:"Calibri",sans-serif"> and no error messages. Any help will be greatly appreciated.
<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Thanks in advance,
<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Wadud.<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div id="Signature">
<div id="divtagdefaultwrapper">
<p style="background:white"><span style="font-family:"Calibri",sans-serif">---------------------------------------<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Dr. Wadud Miah<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Computational Scientist<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">Numerical Algorithms Group<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif">01865 518035<o:p></o:p></span></p>
<p style="background:white"><span style="font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p>The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is:<o:p></o:p></p>
<p>Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.<o:p></o:p></p>
<p> <o:p></o:p></p>
<p>This e-mail has been scanned for all viruses by Microsoft Office 365. <o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Darshan-users mailing list<o:p></o:p></pre>
<pre><a href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><o:p></o:p></pre>
<pre><a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
________________________________________________________________________<br>
This e-mail has been scanned for all viruses by Star.<br>
________________________________________________________________________<o:p></o:p></p>
</div>
<hr width="100%">
<p>The Numerical Algorithms Group Ltd is a company registered in England and Wales with company number 1249803. The registered office is:</p>
<p>Wilkinson House, Jordan Hill Road, Oxford OX2 8DR, United Kingdom.</p>
<p> </p>
<p>This e-mail has been scanned for all viruses by Microsoft Office 365. </p>
<hr width="100%">
</body>
</html>