<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
<p>Hi Phil.....POSIX</p>
<p>this is a well known benchmark ....you can easily verify it!</p>
<p><br>
</p>
<p>by the way it's something like that</p>
<p><br>
</p>
<p>! Serial write is unconditionally compiled<br>
subroutine serialwrite(filename, iodata, n1, n2, n3, cartcomm)<br>
<br>
character*(*) :: filename<br>
<br>
integer :: n1, n2, n3<br>
double precision, dimension(0:n1+1,0:n2+1,0:n3+1) :: iodata<br>
<br>
integer :: cartcomm, ierr, rank, size<br>
integer, parameter :: iounit = 10<br>
<br>
integer :: i<br>
<br>
call MPI_Comm_size(cartcomm, size, ierr)<br>
call MPI_Comm_rank(cartcomm, rank, ierr)<br>
<br>
! Write same amount of data as the parallel write but do it all
from rank 0<br>
! This is just to get a baseline figure for serial IO performance
- note<br>
! that the contents of the file will be differnent from the
parallel calls<br>
<br>
if (rank == 0) then<br>
<br>
open(file=filename, unit=iounit, access='stream')<br>
<br>
do i = 1, size<br>
write(unit=iounit) iodata(1:n1, 1:n2, 1:n3)<br>
end do<br>
<br>
close(iounit)<br>
<br>
end if<br>
<br>
end subroutine serialwrite<br>
<br>
</p>
<p><br>
</p>
<p>Piero<br>
</p>
<div class="moz-cite-prefix">Il 12/02/2020 14:13, Carns, Philip H.
ha scritto:<br>
</div>
<blockquote type="cite"
cite="mid:BN8PR09MB3620817B6D3219B10D247577F71B0@BN8PR09MB3620.namprd09.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
Hi Piero,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
In the serial case, is the rank that's doing I/O still using
MPI-IO, or is it making calls directly to POSIX in that case?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
The Darshan log for the serial case doesn't show any MPI-IO
activity, but I'm not sure if that's accurate, or if it's an
indication that we missed some instrumentation.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
-Phil<br>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
Darshan-users <a class="moz-txt-link-rfc2396E" href="mailto:darshan-users-bounces@lists.mcs.anl.gov"><darshan-users-bounces@lists.mcs.anl.gov></a>
on behalf of Piero LANUCARA <a class="moz-txt-link-rfc2396E" href="mailto:p.lanucara@cineca.it"><p.lanucara@cineca.it></a><br>
<b>Sent:</b> Wednesday, February 12, 2020 5:29 AM<br>
<b>To:</b> Snyder, Shane <a class="moz-txt-link-rfc2396E" href="mailto:ssnyder@mcs.anl.gov"><ssnyder@mcs.anl.gov></a>; Harms,
Kevin <a class="moz-txt-link-rfc2396E" href="mailto:harms@alcf.anl.gov"><harms@alcf.anl.gov></a><br>
<b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a>
<a class="moz-txt-link-rfc2396E" href="mailto:darshan-users@lists.mcs.anl.gov"><darshan-users@lists.mcs.anl.gov></a><br>
<b>Subject:</b> Re: [Darshan-users] Darshan & EPCC benchio
different behaviour</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span
style="font-size:11pt;">
<div class="PlainText">Hi Shane, Kevin<br>
<br>
thanks for the update.<br>
<br>
I attached a new upated files (log and pdf) to this email.<br>
<br>
Also, the log from BENCHIO is attached.<br>
<br>
thanks again<br>
<br>
regards<br>
<br>
Piero<br>
<br>
<br>
Il 11/02/2020 20:15, Shane Snyder ha scritto:<br>
> Definitely looks like something strange is happening
when Darshan is <br>
> estimating the time spent in I/O operations (as seen
in the very first <br>
> figure, observed write time barely even registers) in
the serial case, <br>
> which it is ultimately used to provide the
performance estimate.<br>
><br>
> If you could provide them, the raw Darshan logs would
be really <br>
> helpful. That should make it clear whether it's an
instrumentation <br>
> issue (i.e., under accounting for time spent in I/O
operations at <br>
> runtime) or if its an issue with the heuristics in
the PDF summary <br>
> tool you are using, as Kevin points out. If it's the
latter, having an <br>
> example log to test modifications to our heuristics
would be very <br>
> helpful to us.<br>
><br>
> Thanks,<br>
> --Shane<br>
><br>
> On 2/11/20 8:36 AM, Harms, Kevin wrote:<br>
>> Piero,<br>
>><br>
>> the performance estimate is based on
heuristics, it's possible the <br>
>> 'serial' model is breaking some assumptions about
how the I/O is <br>
>> done. Is every rank opening the file, but only
rank 0 is doing actual <br>
>> I/O?<br>
>><br>
>> If possible, you could provide the log and we
could check to see <br>
>> what the counters look like.<br>
>><br>
>> kevin<br>
>><br>
>> ________________________________________<br>
>> From: Piero LANUCARA <a class="moz-txt-link-rfc2396E" href="mailto:p.lanucara@cineca.it"><p.lanucara@cineca.it></a><br>
>> Sent: Tuesday, February 11, 2020 2:28 AM<br>
>> To: Harms, Kevin<br>
>> Cc: <a class="moz-txt-link-abbreviated" href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
>> Subject: Re: [Darshan-users] Darshan & EPCC
benchio different behaviour<br>
>><br>
>> Hi Kevin<br>
>><br>
>> first of all thanks for the investigation..I did
some futher test and it<br>
>> seems like the issue may appear using Fortran
(MPI, mainly IntelMPI) <br>
>> codes.<br>
>><br>
>> Is this information useful?<br>
>><br>
>> regards<br>
>> Piero<br>
>> Il 07/02/2020 16:07, Harms, Kevin ha scritto:<br>
>>> Piero,<br>
>>><br>
>>> just to confirm, the serial case is still
running in parallel, <br>
>>> 36 processes, but the I/O is only from rank
0?<br>
>>><br>
>>> kevin<br>
>>><br>
>>> ________________________________________<br>
>>> From: Darshan-users
<a class="moz-txt-link-rfc2396E" href="mailto:darshan-users-bounces@lists.mcs.anl.gov"><darshan-users-bounces@lists.mcs.anl.gov></a> on <br>
>>> behalf of Piero LANUCARA
<a class="moz-txt-link-rfc2396E" href="mailto:p.lanucara@cineca.it"><p.lanucara@cineca.it></a><br>
>>> Sent: Wednesday, February 5, 2020 4:56 AM<br>
>>> To: <a class="moz-txt-link-abbreviated" href="mailto:darshan-users@lists.mcs.anl.gov">darshan-users@lists.mcs.anl.gov</a><br>
>>> Subject: Re: [Darshan-users] Darshan &
EPCC benchio different behaviour<br>
>>><br>
>>> p.s<br>
>>><br>
>>> to be more "verbose" I add to the discussion:<br>
>>><br>
>>> Darshan output for the "serial" run
(serial.pdf)<br>
>>><br>
>>> Darshan output for the MPI-IO run (mpiio.pdf)<br>
>>><br>
>>> benchio output for "serial" run (serial.out)<br>
>>><br>
>>> benchio output for "MPI-IO" run (mpi-io.out)<br>
>>><br>
>>> thanks<br>
>>><br>
>>> Piero<br>
>>><br>
>>> Il 04/02/2020 19:44, Piero LANUCARA ha
scritto:<br>
>>>> Dear all<br>
>>>><br>
>>>> I'm using Darshan to measure EPCC benchio
benchmark<br>
>>>> (<a
href="https://github.com/EPCCed/benchio"
moz-do-not-send="true">https://github.com/EPCCed/benchio</a>)
behaviour on a given x86 Tier1<br>
>>>> machine.<br>
>>>><br>
>>>> running two benchio tests (MPI-IO and
serial) a different behaviour<br>
>>>> appear<br>
>>>><br>
>>>> while Darhsan pdf log file is able to
recover the estimated time and<br>
>>>> bandwidth in the MPI-IO case, the
"serial" run is completely<br>
>>>> underestimated by Darshan (the time and
bandwidth are less/greater<br>
>>>> than benchio output).<br>
>>>><br>
>>>> Suggestions are welcomed<br>
>>>><br>
>>>> thanks<br>
>>>><br>
>>>> Piero<br>
>>>><br>
>>>>
_______________________________________________<br>
>>>> Darshan-users mailing list<br>
>>>> <a class="moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><br>
>>>> <a
href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users"
moz-do-not-send="true">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
>> _______________________________________________<br>
>> Darshan-users mailing list<br>
>> <a class="moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><br>
>> <a
href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users"
moz-do-not-send="true">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
><br>
> _______________________________________________<br>
> Darshan-users mailing list<br>
> <a class="moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a><br>
> <a
href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users"
moz-do-not-send="true">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a><br>
</div>
</span></font></div>
</blockquote>
</body>
</html>