<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Just to clarify, are you doing anything explicit to spawn a new process (i.e., fork) ahead of the call to execl? My understanding is that execl replaces the calling process, so generally speaking it shouldn't result in 2 processes (MPI one + one for system
tasks)?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
--Shane<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Adrian Jackson <a.jackson@epcc.ed.ac.uk><br>
<b>Sent:</b> Tuesday, July 12, 2022 10:58 AM<br>
<b>To:</b> Snyder, Shane <ssnyder@mcs.anl.gov>; darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Darshan-users] darshan execl issue</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi Shane,<br>
<br>
Thanks for the reply. This is spawning an additional process that runs <br>
for a bit then ends, the original MPI processes are still all there, <br>
it's just there is an additional one for a bit. The spawned process <br>
doesn't do any MPI, it's just doing some system interaction stuff. If <br>
Darshan intercepts or is triggered by a process ending I think that <br>
would explain it.<br>
<br>
It's something we can work around anyway, we can just not use Darshan <br>
for this exectuable, I was just checking if it's expected behaviour with <br>
Darshan or whether we'd stumbled across "an accidental feature" :)<br>
<br>
cheers<br>
<br>
adrianj<br>
<br>
On 12/07/2022 16:53, Snyder, Shane wrote:<br>
> This email was sent to you by someone outside the University.<br>
> You should only click on links or attachments if you are certain that <br>
> the email is genuine and the content is safe.<br>
> Hi Adrian,<br>
> <br>
> So you are replacing one of the MPI processes in MPI_COMM_WOLRD with a <br>
> new process? In that case, it is probably that this new replacing <br>
> process is not calling MPI_Finalize which ultimately causes Darshan to <br>
> hang -- Darshan is intercepting the shutdown call and performing some <br>
> collective operations for MPI applications, and if one of the ranks <br>
> disappears these calls will likely just hang. If that's the issue, you <br>
> could probably reproduce without using Darshan by having your MPI <br>
> processes run a collective on MPI_COMM_WORLD (like a barrier) _after_ <br>
> the execl call.<br>
> <br>
> A couple of different ideas:<br>
> <br>
> * If possible, it might be worth trying to fork ahead of the execl<br>
> call so that you still have all MPI processes hanging around at<br>
> shutdown time?<br>
> * You may be able to run Darshan in non-MPI mode at runtime (using<br>
> 'export DARSHAN_ENABLE_NONMPI=1') to workaround this problem. This<br>
> would prevent Darshan from running collectives at shutdown time, but<br>
> will result in a different log file for each process in your<br>
> application.<br>
> <br>
> Thanks,<br>
> --Shane<br>
> ------------------------------------------------------------------------<br>
> *From:* Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on <br>
> behalf of Adrian Jackson <a.jackson@epcc.ed.ac.uk><br>
> *Sent:* Tuesday, July 12, 2022 8:13 AM<br>
> *To:* darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
> *Subject:* [Darshan-users] darshan execl issue<br>
> Hi,<br>
> <br>
> I've encountered an issue using Darshan (3.3.1) with a code which calls<br>
> execl from one MPI process. Using with Darshan the MPI run just hangs.<br>
> Is spawning processes from a subset of MPI processes an issue for<br>
> Darshan? I would say that I can still spawn processes (i.e. using fork)<br>
> and it seems to work, but using execl doesn't.<br>
> <br>
> cheers<br>
> <br>
> adrianj<br>
> --<br>
> Tel: +44 131 6506470 skype: remoteadrianj<br>
> The University of Edinburgh is a charitable body, registered in <br>
> Scotland, with registration number SC005336. Is e buidheann carthannais <br>
> a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh <br>
> SC005336.<br>
> _______________________________________________<br>
> Darshan-users mailing list<br>
> Darshan-users@lists.mcs.anl.gov<br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a>
<br>
> <<a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a>><br>
<br>
-- <br>
Tel: +44 131 6506470 skype: remoteadrianj<br>
</div>
</span></font></div>
</body>
</html>