<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Hi Pradeep,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
The key error in that backtrace is this one: "<span style="font-size:11.0pt; font-family:"Georgia",serif">na_sm_put(): process_vm_writev() failed (Operation not permitted)</span>"</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
This means that your processes don't have sufficient permission to access each other's memory. This permission is usually set automatically on commercial Linux clusters, but if you are on a laptop or a home-grown system, you might need to do this (as root)
on the node:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
echo 0 > /proc/sys/kernel/yama/ptrace_scope</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
That setting will be lost on reboot, but there is probably a way to make it persistent; I just don't know off the top of my head. At any rate you can try it by just doing the above before deciding if you want to make that change permanent.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
-Phil<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> mochi-devel <mochi-devel-bounces@lists.mcs.anl.gov> on behalf of Pradeep Subedi <ps917@ored.rutgers.edu><br>
<b>Sent:</b> Monday, February 10, 2020 4:06 PM<br>
<b>To:</b> mochi-devel@lists.mcs.anl.gov <mochi-devel@lists.mcs.anl.gov><br>
<b>Subject:</b> [Mochi-devel] margo_bulk_transfer() not working on sm transport</font>
<div> </div>
</div>
<div>
<div dir="auto" style="color:rgb(33,33,33); background-color:rgb(255,255,255); text-align:left">
<span style="font-family:Georgia,serif; font-size:11pt">Hi,</span><br>
</div>
<div id="x_id-804bc796-1da3-4769-a48c-f778aa5579e5" class="x_ms-outlook-mobile-reference-message" dir="auto" style="text-align:left">
<div class="x_WordSection1">
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">I am currently working on developing a mochi service provider. The provider I developed works on tcp and sockets transport layer, but when I use sm transport, I get errors.</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">On the client side the handle is created as:
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">hret = margo_bulk_create(provider->client->mid, 1, (void**)(&data), &rdma_size,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> HG_BULK_WRITE_ONLY, &in.handle);</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">On the server side the handle is created as</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> hret = margo_bulk_create(mid, 1, (void**)&buffer, &size,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> HG_BULK_READ_ONLY, &bulk_handle);</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">The server fails on :</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">hret = margo_bulk_transfer(mid, HG_BULK_PUSH, info->addr, in.handle, 0,</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> bulk_handle, 0, size);
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">with following errors:</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># NA -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/na/na_sm.c:3741</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># na_sm_put(): process_vm_writev() failed (Operation not permitted)</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:829</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># hg_bulk_transfer_pieces(): Could not transfer data (NA_PROTOCOL_ERROR)</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:988</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># hg_bulk_transfer(): Could not transfer data pieces</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:1829</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"># HG_Bulk_transfer_id(): Could not start transfer of bulk data
</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">Is this a known issue with shared memory transport for margo_bulk_transfer or should I be doing something different?</span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif"> </span></p>
<p class="x_MsoNormal"><span style="font-size:11.0pt; font-family:"Georgia",serif">Thanks,</span></p>
<p class="x_MsoNormal"><span style="font-family:Georgia,serif; color:black">Pradeep Subedi</span></p>
<p class="x_MsoNormal"> </p>
</div>
<br>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Georgia}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle17
{font-family:"Georgia",serif;
color:windowtext}
.x_MsoChpDefault
{font-family:"Calibri",sans-serif}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
-->
</style></div>
</body>
</html>