<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace;font-size:small">Hi Matthieu,</div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small"><br></div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small">For simple cases, I am using </div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small"><br></div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small">gcc test.server.c -o server $(pkg-config --libs margo)<br></div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small"><br></div><div class="gmail_default" style="font-family:courier new,monospace;font-size:small">I pretty much tried everything. I just installed libfabric manually. I am not sure what to try next.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 6, 2019 at 12:07 PM Dorier, Matthieu <<a href="mailto:mdorier@anl.gov">mdorier@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-GB">
<div class="gmail-m_4718529552322078054WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi Abid,
<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">At this top of my head this could be a problem with the order in which libraries are linked. Could you send your CMakeLists.txt file?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US">Matthieu<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black">mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>> on behalf of Abid Malik via mochi-devel <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Reply to: </b>Abid Malik <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Date: </b>Tuesday, 5 November 2019 at 00:01<br>
<b>To: </b>"Carns, Philip H." <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>><br>
<b>Cc: </b>"Latham, Robert J. via mochi-devel" <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Subject: </b>Re: [Mochi-devel] running mochi examples<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">Thanks.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">I am getting the following linking error:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<p class="MsoNormal">Linking C executable 01_margo_hello_server<br>
/usr/bin/ld: CMakeFiles/01_margo_hello_server.dir/hello_server.c.o: undefined reference to symbol 'ABT_thread_create'<br>
/gpfs/alpine/csc299/world-shared/amalik/mochi/spack/opt/spack/linux-rhel7-power9le/gcc-6.4.0/argobots-develop-uwvdjjdzrkn475ax7bw4w7hztawzidg3/lib/libabt.so.0: error adding symbols: DSO missing from command line<br>
/usr/bin/sha1sum: 01_margo_hello_server: No such file or directory<br>
collect2: error: ld returned 1 exit status<br>
make[2]: *** [01_margo_hello_server] Error 1<br>
make[1]: *** [CMakeFiles/01_margo_hello_server.dir/all] Error 2<u></u><u></u></p>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:Arial,sans-serif">make: *** [all] Error 2</span><span style="font-size:12pt;font-family:"Courier New""><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">currently, I have the following:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">Currently Loaded Modules:<br>
1) xl/16.1.1-3 6) darshan-runtime/3.1.7 11) boost-1.70.0-gcc-6.4.0-dw376xg<br>
2) spectrum-mpi/10.3.0.1-20190611 7) DefApps 12) libfabric-1.8.1-gcc-6.4.0-d4kxa4p<br>
3) hsi/5.0.2.p5 8) argobots-develop-gcc-6.4.0-uwvdjjd 13) mercury-master-gcc-6.4.0-nbp34rm<br>
4) xalt/1.1.4 9) bzip2-1.0.8-gcc-6.4.0-mdsmice 14) margo-0.5.2-gcc-6.4.0-gniufvt<br>
5) lsf-tools/2.0 10) zlib-1.2.11-gcc-6.4.0-nbedgs2 15) cmake/3.15.2<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">Argobot is there. Am I missing something?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Courier New"">Abid<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Nov 4, 2019 at 5:10 PM Carns, Philip H. <<a href="mailto:carns@mcs.anl.gov" target="_blank">carns@mcs.anl.gov</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:12pt;color:black">For the transport it should work Ok to just use "verbs://" now (the trailing colon and slashes are important, though). Mercury will automatically convert that to the
more verbose string internally.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:12pt;color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:12pt;color:black">thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:12pt;color:black">-Phil<u></u><u></u></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="gmail-m_4718529552322078054gmail-m_8051748058728153858divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> mochi-devel <<a href="mailto:mochi-devel-bounces@lists.mcs.anl.gov" target="_blank">mochi-devel-bounces@lists.mcs.anl.gov</a>> on behalf of Latham, Robert J. via mochi-devel
<<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>><br>
<b>Sent:</b> Monday, November 4, 2019 3:11 PM<br>
<b>To:</b> <a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a> <<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a>>;
<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a> <<a href="mailto:abidmuslim@gmail.com" target="_blank">abidmuslim@gmail.com</a>><br>
<b>Subject:</b> Re: [Mochi-devel] running mochi examples</span> <u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">On Mon, 2019-11-04 at 07:37 -0500, Abid Malik via mochi-devel wrote:<br>
> <br>
> Hello,<br>
> <br>
> I am trying to play with the mochi examples at <br>
> <br>
> <a href="https://xgitlab.cels.anl.gov/sds/sds-examples.git" target="_blank">https://xgitlab.cels.anl.gov/sds/sds-examples.git</a><br>
> <br>
> I am trying the example on summit:<br>
> <br>
> /mercury/01_hello/01_hg_hello_server<br>
> <br>
> How should I start the server/client? I am trying it on login2 node<br>
> and trying to initiate it like:<br>
> <br>
> ./01_hg_hello_server bmi+tcp://login2:1234<br>
<br>
I see you are using spack. If so, I think you would have had to go out<br>
of your way to request the 'bmi' transport. You probably built<br>
libfabric. You can find out for sure with 'spack find'<br>
<br>
You tell margo (mercury, really) which protocol to use but the string<br>
for "libfabric using the verbs provider" is a little odd:<br>
<br>
"ofi+verbs;ofi_rxm://"<br>
<br>
> and getting errors.<br>
> <br>
> # NA -- Error -- /tmp/amalik/spack-stage/spack-stage-mercury-master-<br>
> nbp34rmg6rkc2ayiqxxxuupamvmdusr7/spack-src/src/na/na.c:281<br>
> # NA_Initialize_opt(): Specified class name does not support<br>
> requested protocol<br>
<br>
This says mercury did not recognize 'bmi+tcp'. You'd get the same<br>
error if you requested "moon_rock".<br>
<br>
I don't have a lot of guidance to offer you (yet). I just got started<br>
on summit a few days ago. When I try to take the advice I gave you I<br>
get an error, too:<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server verbs<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na.c:279<br>
# NA_Initialize_opt(): Specified class name does not support requested<br>
protocol<br>
<br>
That's what we expect. margo/mercury don't know what 'verbs'<br>
is. Instead, we need to tell margo/mercury to use OFI (libfabric):<br>
<br>
$ jsrun -r1 -n1 ./margo-example-server "ofi+verbs;ofi_rxm://"<br>
# NA -- Error -- /tmp/robl/spack-stage/spack-stage-mercury-master-<br>
c6nwqortoqjsyviyinlg5t6sfay6wv7l/spack-src/src/na/na_ofi.c:1406<br>
# na_ofi_getinfo(): fi_getinfo() failed, rc: -61(No data available)<br>
<br>
Uh, I'm still working on that one...<br>
<br>
==rob<br>
<br>
_______________________________________________<br>
mochi-devel mailing list<br>
<a href="mailto:mochi-devel@lists.mcs.anl.gov" target="_blank">mochi-devel@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel</a><br>
<a href="https://www.mcs.anl.gov/research/projects/mochi" target="_blank">https://www.mcs.anl.gov/research/projects/mochi</a><u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">Abid M. Malik<br>
******************************************************<br>
"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>
"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>
"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>
"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br>
<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Abid M. Malik<br>******************************************************<br>"I have learned silence from the talkative, toleration from the intolerant, and kindness from the unkind"---Gibran<br>"Success is not for the chosen few, but for the few who choose" --- John Maxwell<br>"Being a good person does not depend on your religion or status in life, your race or skin color, political views or culture. IT DEPENDS ON HOW GOOD YOU TREAT OTHERS"--- Abid<br>"The Universe is talking to us, and the language of the Universe is mathematics."----Abid<br> </div></div></div>