<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Hi Piero,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
In the serial case, is the rank that's doing I/O still using MPI-IO, or is it making calls directly to POSIX in that case?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
The Darshan log for the serial case doesn't show any MPI-IO activity, but I'm not sure if that's accurate, or if it's an indication that we missed some instrumentation.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
-Phil<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on behalf of Piero LANUCARA <p.lanucara@cineca.it><br>
<b>Sent:</b> Wednesday, February 12, 2020 5:29 AM<br>
<b>To:</b> Snyder, Shane <ssnyder@mcs.anl.gov>; Harms, Kevin <harms@alcf.anl.gov><br>
<b>Cc:</b> darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Darshan-users] Darshan & EPCC benchio different behaviour</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi Shane, Kevin<br>
<br>
thanks for the update.<br>
<br>
I attached a new upated files (log and pdf) to this email.<br>
<br>
Also, the log from BENCHIO is attached.<br>
<br>
thanks again<br>
<br>
regards<br>
<br>
Piero<br>
<br>
<br>
Il 11/02/2020 20:15, Shane Snyder ha scritto:<br>
> Definitely looks like something strange is happening when Darshan is <br>
> estimating the time spent in I/O operations (as seen in the very first <br>
> figure, observed write time barely even registers) in the serial case, <br>
> which it is ultimately used to provide the performance estimate.<br>
><br>
> If you could provide them, the raw Darshan logs would be really <br>
> helpful. That should make it clear whether it's an instrumentation <br>
> issue (i.e., under accounting for time spent in I/O operations at <br>
> runtime) or if its an issue with the heuristics in the PDF summary <br>
> tool you are using, as Kevin points out. If it's the latter, having an <br>
> example log to test modifications to our heuristics would be very <br>
> helpful to us.<br>
><br>
> Thanks,<br>
> --Shane<br>
><br>
> On 2/11/20 8:36 AM, Harms, Kevin wrote:<br>
>> Piero,<br>
>><br>
>> the performance estimate is based on heuristics, it's possible the <br>
>> 'serial' model is breaking some assumptions about how the I/O is <br>
>> done. Is every rank opening the file, but only rank 0 is doing actual <br>
>> I/O?<br>
>><br>
>> If possible, you could provide the log and we could check to see <br>
>> what the counters look like.<br>
>><br>
>> kevin<br>
>><br>
>> ________________________________________<br>
>> From: Piero LANUCARA <p.lanucara@cineca.it><br>
>> Sent: Tuesday, February 11, 2020 2:28 AM<br>
>> To: Harms, Kevin<br>
>> Cc: darshan-users@lists.mcs.anl.gov<br>
>> Subject: Re: [Darshan-users] Darshan & EPCC benchio different behaviour<br>
>><br>
>> Hi Kevin<br>
>><br>
>> first of all thanks for the investigation..I did some futher test and it<br>
>> seems like the issue may appear using Fortran (MPI, mainly IntelMPI) <br>
>> codes.<br>
>><br>
>> Is this information useful?<br>
>><br>
>> regards<br>
>> Piero<br>
>> Il 07/02/2020 16:07, Harms, Kevin ha scritto:<br>
>>> Piero,<br>
>>><br>
>>> just to confirm, the serial case is still running in parallel, <br>
>>> 36 processes, but the I/O is only from rank 0?<br>
>>><br>
>>> kevin<br>
>>><br>
>>> ________________________________________<br>
>>> From: Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on <br>
>>> behalf of Piero LANUCARA <p.lanucara@cineca.it><br>
>>> Sent: Wednesday, February 5, 2020 4:56 AM<br>
>>> To: darshan-users@lists.mcs.anl.gov<br>
>>> Subject: Re: [Darshan-users] Darshan & EPCC benchio different behaviour<br>
>>><br>
>>> p.s<br>
>>><br>
>>> to be more "verbose" I add to the discussion:<br>
>>><br>
>>> Darshan output for the "serial" run (serial.pdf)<br>
>>><br>
>>> Darshan output for the MPI-IO run (mpiio.pdf)<br>
>>><br>
>>> benchio output for "serial" run (serial.out)<br>
>>><br>
>>> benchio output for "MPI-IO" run (mpi-io.out)<br>
>>><br>
>>> thanks<br>
>>><br>
>>> Piero<br>
>>><br>
>>> Il 04/02/2020 19:44, Piero LANUCARA ha scritto:<br>
>>>> Dear all<br>
>>>><br>
>>>> I'm using Darshan to measure EPCC benchio benchmark<br>
>>>> (<a href="https://github.com/EPCCed/benchio">https://github.com/EPCCed/benchio</a>) behaviour on a given x86 Tier1<br>
>>>> machine.<br>
>>>><br>
>>>> running two benchio tests (MPI-IO and serial) a different behaviour<br>
>>>> appear<br>
>>>><br>
>>>> while Darhsan pdf log file is able to recover the estimated time and<br>
>>>> bandwidth in the MPI-IO case, the "serial" run is completely<br>
>>>> underestimated by Darshan (the time and bandwidth are less/greater<br>
>>>> than benchio output).<br>
>>>><br>
>>>> Suggestions are welcomed<br>
>>>><br>
>>>> thanks<br>
>>>><br>
>>>> Piero<br>
>>>><br>
>>>> _______________________________________________<br>
>>>> Darshan-users mailing list<br>
>>>> Darshan-users@lists.mcs.anl.gov<br>
>>>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
>> _______________________________________________<br>
>> Darshan-users mailing list<br>
>> Darshan-users@lists.mcs.anl.gov<br>
>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
><br>
> _______________________________________________<br>
> Darshan-users mailing list<br>
> Darshan-users@lists.mcs.anl.gov<br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
</div>
</span></font></div>
</body>
</html>