<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Hi Abid,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Sorry for the late response, I’m on vacations with limited access to my computer.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Let’s try to figure out what changed since your earlier messages about NA errors. Which libraries were rebuilt?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Matthieu<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Abid Malik <abidmuslim@gmail.com><br>
<b>Date: </b>Wednesday, 6 November 2019 at 17:56<br>
<b>To: </b>"Dorier, Matthieu" <mdorier@anl.gov><br>
<b>Cc: </b>"Carns, Philip H." <carns@mcs.anl.gov>, "Latham, Robert J. via mochi-devel" <mochi-devel@lists.mcs.anl.gov><br>
<b>Subject: </b>Re: [Mochi-devel] running mochi examples<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">Hi Matthieu,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">For simple cases, I am using <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">gcc test.server.c -o server $(pkg-config --libs margo)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">I pretty much tried everything. I just installed libfabric manually. I am not sure what to try next.<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Nov 6, 2019 at 12:07 PM Dorier, Matthieu <<a href="mailto:mdorier@anl.gov">mdorier@anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Hi Abid,
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">At this top of my head this could be a problem with the order in which libraries are linked. Could you send your CMakeLists.txt file?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Matthieu</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>> on behalf of Abid Malik via mochi-devel <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Reply to: </b>Abid Malik <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Date: </b>Tuesday, 5 November 2019 at 00:01<br>
<b>To: </b>"Carns, Philip H." <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>><br>
<b>Cc: </b>"Latham, Robert J. via mochi-devel" <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [Mochi-devel] running mochi examples</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">Thanks.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">I am getting the following linking error:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Linking C executable 01_margo_hello_server<br>
/usr/bin/ld: CMakeFiles/01_margo_hello_server.dir/hello_server.c.o: undefined reference to symbol 'ABT_thread_create'<br>
/gpfs/alpine/csc299/world-shared/amalik/mochi/spack/opt/spack/linux-rhel7-power9le/gcc-6.4.0/argobots-develop-uwvdjjdzrkn475ax7bw4w7hztawzidg3/lib/libabt.so.0: error adding symbols: DSO missing from command line<br>
/usr/bin/sha1sum: 01_margo_hello_server: No such file or directory<br>
collect2: error: ld returned 1 exit status<br>
make[2]: *** [01_margo_hello_server] Error 1<br>
make[1]: *** [CMakeFiles/01_margo_hello_server.dir/all] Error 2<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">make: *** [all] Error 2</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">currently, I have the following:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">Currently Loaded Modules:<br>
1) xl/16.1.1-3 6) darshan-runtime/3.1.7 11) boost-1.70.0-gcc-6.4.0-dw376xg<br>
2) spectrum-mpi/10.3.0.1-20190611 7) DefApps 12) libfabric-1.8.1-gcc-6.4.0-d4kxa4p<br>
3) hsi/5.0.2.p5 8) argobots-develop-gcc-6.4.0-uwvdjjd 13) mercury-master-gcc-6.4.0-nbp34rm<br>
4) xalt/1.1.4 9) bzip2-1.0.8-gcc-6.4.0-mdsmice 14) margo-0.5.2-gcc-6.4.0-gniufvt<br>
5) lsf-tools/2.0 10) zlib-1.2.11-gcc-6.4.0-nbedgs2 15) cmake/3.15.2</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">Argobot is there. Am I missing something?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New""> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Courier New"">Abid</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Mon, Nov 4, 2019 at 5:10 PM Carns, Philip H. <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:12.0pt;color:black">For the transport it should work Ok to just use "verbs://" now (the trailing colon and slashes are important, though). Mercury will automatically convert that to the more verbose string internally.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:12.0pt;color:black">thanks,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:12.0pt;color:black">-Phil</span><o:p></o:p></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="gmail-m_4718529552322078054gmail-m_8051748058728153858divRplyFwdMsg">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="color:black">From:</span></b><span style="color:black"> mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>>
on behalf of Latham, Robert J. via mochi-devel <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Sent:</b> Monday, November 4, 2019 3:11 PM<br>
<b>To:</b> <a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a> <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>>;
<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a> <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Subject:</b> Re: [Mochi-devel] running mochi examples</span> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Mon, 2019-11-04 at 07:37 -0500, Abid Malik via mochi-devel wrote:<br>
> <br>
> Hello,<br>
> <br>
> I am trying to play with the mochi examples at <br>
> <br>
> <a href="https://xgitlab.cels.anl.gov/sds/sds-examples.git" target="_blank">https://xgitlab.cels.anl.gov/sds/sds-examples.git</a><br>
> <br>
> I am trying the example on summit:<br>
> <br>
> /mercury/01_hello/01_hg_hello_server<br>
> <br>
> How should I start the server/client? I am trying it on login2 node<br>
> and trying to initiate it like:<br>
> <br>
> ./01_hg_hello_server bmi+tcp://login2:1234<br>
<br>
I see you are using spack. If so, I think you would have had to go out<br>
of your way to request the 'bmi' transport. You probably built<br>
libfabric. You can find out for sure with 'spack find'<br>
<br>
You tell margo (mercury, really) which protocol to use but the string<br>
for "libfabric using the verbs provider" is a little odd:<br>
<br>
"ofi+verbs;ofi_rxm://"<br>
<br>
> and getting errors.<br>
> <br>
> # NA -- Error -- /tmp/amalik/spack-stage/spack-stage-mercury-master-<br>
> nbp34rmg6rkc2ayiqxxxuupamvmdusr7/spack-src/src/na/na.c:281<br>
> # NA_Initialize_opt(): Specified class name does not support<br>
> requested protocol<br>
<br>
This says mercury did not recognize 'bmi+tcp'. You'd get the same<br>
error if you requested "moon_rock".<br>
<br>
I don't have a lot of guidance to offer you (yet). I just got started<br>
on summit a few days ago. When I try to take the advice I gave you I<br>
get an error, too:<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server verbs<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na.c:279<br>
# NA_Initialize_opt(): Specified class name does not support requested<br>
protocol<br>
<br>
That's what we expect. margo/mercury don't know what 'verbs'<br>
is. Instead, we need to tell margo/mercury to use OFI (libfabric):<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server "ofi+verbs;ofi_rxm://"<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na_ofi.c:1406<br>
# na_ofi_getinfo(): fi_getinfo() failed, rc: -61(No data available)<br>
<br>
Uh, I'm still working on that one...<br>
<br>
==rob<br>
<br>
_______________________________________________<br>
mochi-devel mailing list<br>
<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel</a><br>
<a href="https://www.mcs.anl.gov/research/projects/mochi" target="_blank">https://www.mcs.anl.gov/research/projects/mochi</a><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Abid M. Malik<br>
******************************************************<br>
"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>
"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>
"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>
"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br>
<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Abid M. Malik<br>
******************************************************<br>
"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>
"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>
"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>
"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br>
<o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>