<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Abid followed up some more off-list; I think his code building now but he still has some run time problems to work through.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
I have a quick follow up on one of the tangents in this discussion; the issue that was preventing "verbs://" from working correctly as the transport name has been fixed (you will need to uninstall and then re-install the mercury package to pick this up):</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<a href="https://github.com/mercury-hpc/mercury/issues/322" style="">https://github.com/mercury-hpc/mercury/issues/322</a></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
That wasn't the core problem here, but something confusing that we hit along the way.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
-Phil<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Dorier, Matthieu <mdorier@anl.gov><br>
<b>Sent:</b> Sunday, November 10, 2019 3:06 PM<br>
<b>To:</b> Abid Malik <abidmuslim@gmail.com><br>
<b>Cc:</b> Carns, Philip H. <carns@mcs.anl.gov>; Latham, Robert J. via mochi-devel <mochi-devel@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Mochi-devel] running mochi examples</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p.x_msonormal0, li.x_msonormal0, div.x_msonormal0
{margin-right:0cm;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle18
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:72.0pt 72.0pt 72.0pt 72.0pt}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-GB" link="blue" vlink="purple">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span lang="EN-US" style="">Hi Abid,</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style=""> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="">Sorry for the late response, I’m on vacations with limited access to my computer.</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="">Let’s try to figure out what changed since your earlier messages about NA errors. Which libraries were rebuilt?</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style=""> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="">Thanks,</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style=""> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="">Matthieu</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style=""> </span></p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0cm 0cm 0cm">
<p class="x_MsoNormal"><b><span style="font-size:12.0pt; color:black">From: </span>
</b><span style="font-size:12.0pt; color:black">Abid Malik <abidmuslim@gmail.com><br>
<b>Date: </b>Wednesday, 6 November 2019 at 17:56<br>
<b>To: </b>"Dorier, Matthieu" <mdorier@anl.gov><br>
<b>Cc: </b>"Carns, Philip H." <carns@mcs.anl.gov>, "Latham, Robert J. via mochi-devel" <mochi-devel@lists.mcs.anl.gov><br>
<b>Subject: </b>Re: [Mochi-devel] running mochi examples</span></p>
</div>
<div>
<p class="x_MsoNormal"> </p>
</div>
<div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New"">Hi Matthieu,</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New"">For simple cases, I am using </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New"">gcc test.server.c -o server $(pkg-config --libs margo)</span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal"><span style="font-size:12.0pt; font-family:"Courier New"">I pretty much tried everything. I just installed libfabric manually. I am not sure what to try next.</span></p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div>
<div>
<p class="x_MsoNormal">On Wed, Nov 6, 2019 at 12:07 PM Dorier, Matthieu <<a href="mailto:mdorier@anl.gov">mdorier@anl.gov</a>> wrote:</p>
</div>
<blockquote style="border:none; border-left:solid #CCCCCC 1.0pt; padding:0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-right:0cm">
<div>
<div>
<p class="x_MsoNormal" style=""><span lang="EN-US">Hi Abid, </span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US"> </span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US">At this top of my head this could be a problem with the order in which libraries are linked. Could you send your CMakeLists.txt file?</span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US">Thanks,</span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US"> </span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US">Matthieu</span></p>
<p class="x_MsoNormal" style=""><span lang="EN-US"> </span></p>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0cm 0cm 0cm">
<p class="x_MsoNormal" style=""><b><span style="font-size:12.0pt; color:black">From:
</span></b><span style="font-size:12.0pt; color:black">mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>> on behalf of Abid Malik via mochi-devel <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Reply to: </b>Abid Malik <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Date: </b>Tuesday, 5 November 2019 at 00:01<br>
<b>To: </b>"Carns, Philip H." <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>><br>
<b>Cc: </b>"Latham, Robert J. via mochi-devel" <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [Mochi-devel] running mochi examples</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""> </p>
</div>
<div>
<div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
</div>
<div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">Thanks.</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">I am getting the following linking error:</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<p class="x_MsoNormal" style="">Linking C executable 01_margo_hello_server<br>
/usr/bin/ld: CMakeFiles/01_margo_hello_server.dir/hello_server.c.o: undefined reference to symbol 'ABT_thread_create'<br>
/gpfs/alpine/csc299/world-shared/amalik/mochi/spack/opt/spack/linux-rhel7-power9le/gcc-6.4.0/argobots-develop-uwvdjjdzrkn475ax7bw4w7hztawzidg3/lib/libabt.so.0: error adding symbols: DSO missing from command line<br>
/usr/bin/sha1sum: 01_margo_hello_server: No such file or directory<br>
collect2: error: ld returned 1 exit status<br>
make[2]: *** [01_margo_hello_server] Error 1<br>
make[1]: *** [CMakeFiles/01_margo_hello_server.dir/all] Error 2</p>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Arial",sans-serif">make: *** [all] Error 2</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">currently, I have the following:</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">Currently Loaded Modules:<br>
1) xl/16.1.1-3 6) darshan-runtime/3.1.7 11) boost-1.70.0-gcc-6.4.0-dw376xg<br>
2) spectrum-mpi/10.3.0.1-20190611 7) DefApps 12) libfabric-1.8.1-gcc-6.4.0-d4kxa4p<br>
3) hsi/5.0.2.p5 8) argobots-develop-gcc-6.4.0-uwvdjjd 13) mercury-master-gcc-6.4.0-nbp34rm<br>
4) xalt/1.1.4 9) bzip2-1.0.8-gcc-6.4.0-mdsmice 14) margo-0.5.2-gcc-6.4.0-gniufvt<br>
5) lsf-tools/2.0 10) zlib-1.2.11-gcc-6.4.0-nbedgs2 15) cmake/3.15.2</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">Argobot is there. Am I missing something?</span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New""> </span></p>
</div>
<div>
<p class="x_MsoNormal" style=""><span style="font-size:12.0pt; font-family:"Courier New"">Abid</span></p>
</div>
<p class="x_MsoNormal" style=""> </p>
</div>
<div>
<p class="x_MsoNormal" style=""> </p>
</div>
<p class="x_MsoNormal" style=""> </p>
<div>
<div>
<p class="x_MsoNormal" style="">On Mon, Nov 4, 2019 at 5:10 PM Carns, Philip H. <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>> wrote:</p>
</div>
<blockquote style="border:none; border-left:solid #CCCCCC 1.0pt; padding:0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; margin-bottom:5.0pt">
<div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:12.0pt; color:black">For the transport it should work Ok to just use "verbs://" now (the trailing colon and slashes are important, though). Mercury will automatically convert that to the
more verbose string internally.</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:12.0pt; color:black"> </span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:12.0pt; color:black">thanks,</span></p>
</div>
<div>
<p class="x_MsoNormal" style="background:white"><span style="font-size:12.0pt; color:black">-Phil</span></p>
</div>
<div class="x_MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="x_gmail-m_4718529552322078054gmail-m_8051748058728153858divRplyFwdMsg">
<p class="x_MsoNormal" style=""><b><span style="color:black">From:</span></b><span style="color:black"> mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>> on behalf of Latham, Robert
J. via mochi-devel <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Sent:</b> Monday, November 4, 2019 3:11 PM<br>
<b>To:</b> <a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a> <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>>;
<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a> <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Subject:</b> Re: [Mochi-devel] running mochi examples</span> </p>
<div>
<p class="x_MsoNormal" style=""> </p>
</div>
</div>
<div>
<div>
<p class="x_MsoNormal" style="">On Mon, 2019-11-04 at 07:37 -0500, Abid Malik via mochi-devel wrote:<br>
> <br>
> Hello,<br>
> <br>
> I am trying to play with the mochi examples at <br>
> <br>
> <a href="https://xgitlab.cels.anl.gov/sds/sds-examples.git" target="_blank">https://xgitlab.cels.anl.gov/sds/sds-examples.git</a><br>
> <br>
> I am trying the example on summit:<br>
> <br>
> /mercury/01_hello/01_hg_hello_server<br>
> <br>
> How should I start the server/client? I am trying it on login2 node<br>
> and trying to initiate it like:<br>
> <br>
> ./01_hg_hello_server bmi+tcp://login2:1234<br>
<br>
I see you are using spack. If so, I think you would have had to go out<br>
of your way to request the 'bmi' transport. You probably built<br>
libfabric. You can find out for sure with 'spack find'<br>
<br>
You tell margo (mercury, really) which protocol to use but the string<br>
for "libfabric using the verbs provider" is a little odd:<br>
<br>
"ofi+verbs;ofi_rxm://"<br>
<br>
> and getting errors.<br>
> <br>
> # NA -- Error -- /tmp/amalik/spack-stage/spack-stage-mercury-master-<br>
> nbp34rmg6rkc2ayiqxxxuupamvmdusr7/spack-src/src/na/na.c:281<br>
> # NA_Initialize_opt(): Specified class name does not support<br>
> requested protocol<br>
<br>
This says mercury did not recognize 'bmi+tcp'. You'd get the same<br>
error if you requested "moon_rock".<br>
<br>
I don't have a lot of guidance to offer you (yet). I just got started<br>
on summit a few days ago. When I try to take the advice I gave you I<br>
get an error, too:<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server verbs<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na.c:279<br>
# NA_Initialize_opt(): Specified class name does not support requested<br>
protocol<br>
<br>
That's what we expect. margo/mercury don't know what 'verbs'<br>
is. Instead, we need to tell margo/mercury to use OFI (libfabric):<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server "ofi+verbs;ofi_rxm://"<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na_ofi.c:1406<br>
# na_ofi_getinfo(): fi_getinfo() failed, rc: -61(No data available)<br>
<br>
Uh, I'm still working on that one...<br>
<br>
==rob<br>
<br>
_______________________________________________<br>
mochi-devel mailing list<br>
<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel</a><br>
<a href="https://www.mcs.anl.gov/research/projects/mochi" target="_blank">https://www.mcs.anl.gov/research/projects/mochi</a></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="x_MsoNormal" style=""><br clear="all">
</p>
<div>
<p class="x_MsoNormal" style=""> </p>
</div>
<p class="x_MsoNormal" style="">-- </p>
<div>
<div>
<div>
<p class="x_MsoNormal" style="">Abid M. Malik<br>
******************************************************<br>
"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>
"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>
"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>
"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br>
</p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="x_MsoNormal"><br clear="all">
</p>
<div>
<p class="x_MsoNormal"> </p>
</div>
<p class="x_MsoNormal">-- </p>
<div>
<div>
<div>
<p class="x_MsoNormal">Abid M. Malik<br>
******************************************************<br>
"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>
"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>
"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>
"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br>
</p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>