<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Arial;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt">Hi Phil,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Awesome, your workaround fixed the issue.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Regarding the linker, we are under CLE6Up5, and SUSE12 SP3 on the front end node, we are using xalt ( the wrapper to track library usage at linking and execution does not conflict with darshan), so the ld
used is xalt wrapper and it works perfectly with both version.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">which ld<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">/sw/xc40cle6/xalt/1.1.2/sles12.3_cce8.6.5/bin/ld<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">ld --version<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">GNU ld (GNU Binutils) 2.26.0.20160224<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Copyright (C) 2015 Free Software Foundation, Inc.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">This program is free software; you may redistribute it under the terms of<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">the GNU General Public License version 3 or (at your option) a later version.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">This program has absolutely no warranty.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">I really appreciate your kind and prompt help.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt">Bilel <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">-------<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Bilel Hadri, PhD<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Computational Scientist<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">KAUST Supercomputing Lab<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Al Khawarizmi Bldg. (1) Office 126<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">4700 King Abdullah University of Science and Technology<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Thuwal 23955-6900<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Kingdom of Saudi Arabia<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Office Phone: +966 12 808 0654<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Cell Phone: + 966 544 700 893<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Phil Carns <carns@mcs.anl.gov><br>
<b>Date: </b>Tuesday, February 27, 2018 at 6:18 PM<br>
<b>To: </b>"bilel.hadri@kaust.edu.sa" <bilel.hadri@kaust.edu.sa>, "darshan-users@lists.mcs.anl.gov" <darshan-users@lists.mcs.anl.gov><br>
<b>Subject: </b>Re: [Darshan-users] darshan3.1.5 issue on Cray XC40 cle6.up05<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On 02/27/2018 12:34 AM, Bilel Hadri wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Phil,
<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Sure, test_scalapack.f90 call ScaLAPACK routines and uses inside MPI library. The output of the failing ftn command with the craype-verbose flag<o:p></o:p></p>
</blockquote>
<p><o:p> </o:p></p>
<p>Thanks Bilel. The petsc example helps too. This clarifies that the trigger on Cray up05 is to compile a program that doesn't directly use MPI symbols on its own but instead links in another library that indirectly uses MPI symbols on its behalf.<o:p></o:p></p>
<p>This isn't an unusual use case (scalapack and petsc have done this forever), and the linker options in Darshan are set up to handle it, but something has obviously changed in the linker in this respect in this update.<o:p></o:p></p>
<p>We are still trying to work out how to reproduce this but it may be a little bit. In the mean time would you be willing to try something in your 3.1.5 install to help debug?
<o:p></o:p></p>
<p>Can you manually edit /sw/xc40cle6/darshan/3.1.5/sles12.3_gcc7.2.0/share/ld-opts/darshan-base-ld-opts and add this line to the top of the file, then try again to compile your examples with 3.1.5 loaded?<o:p></o:p></p>
<p>--undefined=__wrap_MPI_Init<o:p></o:p></p>
<p>By my (admittedly slightly murky) interpretation of the link order and the flags that we are setting, that shouldn't be necessary, but it would be helpful to see what it does for your example.<o:p></o:p></p>
<p>thanks,<o:p></o:p></p>
<p>-Phil<o:p></o:p></p>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1">This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.<br>
</font>
</body>
</html>