<div dir="ltr"><div class="gmail_default" style="font-family:"comic sans ms",sans-serif"><font color="#38761d" style="background-color:rgb(255,255,255)">Hi Si,</font></div><div class="gmail_default" style="font-family:"comic sans ms",sans-serif"><font color="#38761d" style="background-color:rgb(255,255,255)"><br></font></div><div class="gmail_default" style="font-family:"comic sans ms",sans-serif"><font color="#38761d" style="background-color:rgb(255,255,255)">I have confirmed using the test case in</font></div><div class="gmail_default" style="font-family:"comic sans ms",sans-serif">





<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Monaco"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font color="#38761d">/scratch1/02503/edwardsj/impi_bugtest</font></span></p><p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Monaco"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures;background-color:rgb(255,255,255)"><font color="#38761d">that this is indeed the problem.</font></span></p></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 5, 2019 at 8:33 PM Jim Edwards <<a href="mailto:jedwards@ucar.edu">jedwards@ucar.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:"comic sans ms",sans-serif;color:rgb(56,118,29)">Thanks Wei-keng I'll try the test program - but I know I have the 0 length arrays - this is the same app we found the bug in mpich with...</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 5, 2019 at 8:18 PM Wei-keng Liao <<a href="mailto:wkliao@eecs.northwestern.edu" target="_blank">wkliao@eecs.northwestern.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Does your I/O pattern result in some of the MPI processes making<br>
zero-length read requests? That is the condition to run into this bug.<br>
<br>
I suggest you to run the test program given in that github issue on Frontera.<br>
If a similar error message beginning with "ADIOI_LUSTRE_IOCONTIG(228)” appears,<br>
then you got your confirmation.<br>
<br>
Wei-keng<br>
<br>
> On Sep 5, 2019, at 8:17 PM, Jim Edwards <<a href="mailto:jedwards@ucar.edu" target="_blank">jedwards@ucar.edu</a>> wrote:<br>
> <br>
> Hi Si, <br>
> <br>
> Can you contact the impi developers and see if this could be the issue and if we could get a fix quickly if it is.   <br>
> <br>
> Wei-keng - thanks!  <br>
> <br>
> On Thu, Sep 5, 2019 at 7:06 PM Wei-keng Liao <<a href="mailto:wkliao@eecs.northwestern.edu" target="_blank">wkliao@eecs.northwestern.edu</a>> wrote:<br>
> I guess you are probably hit with the same issue reported in<br>
> <a href="https://github.com/pmodels/mpich/pull/3634" rel="noreferrer" target="_blank">https://github.com/pmodels/mpich/pull/3634</a><br>
> <br>
> It has been fixed in the MPICH master branch.<br>
> <br>
> Wei-keng<br>
> <br>
> > On Sep 5, 2019, at 7:43 PM, Jim Edwards via parallel-netcdf <<a href="mailto:parallel-netcdf@lists.mcs.anl.gov" target="_blank">parallel-netcdf@lists.mcs.anl.gov</a>> wrote:<br>
> > <br>
> > I'm using a new system TACC Frontera and having problems on the first read - I get:<br>
> > [15065] ADIOI_LUSTRE_IOCONTIG(228): Other I/O error Cannot allocate memory<br>
> >  : Unknown error occurs in reading file<br>
> > <br>
> > Just wondering if anyone has seen this before?<br>
> > <br>
> > -- <br>
> > Jim Edwards<br>
> > <br>
> > CESM Software Engineer<br>
> > National Center for Atmospheric Research<br>
> > Boulder, CO <br>
> <br>
> <br>
> <br>
> -- <br>
> Jim Edwards<br>
> <br>
> CESM Software Engineer<br>
> National Center for Atmospheric Research<br>
> Boulder, CO <br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_-8328649107625825942gmail_signature"><div dir="ltr"><div><div><div>Jim Edwards<br><br></div><font size="1">CESM Software Engineer<br></font></div><font size="1">National Center for Atmospheric Research<br></font></div><font size="1">Boulder, CO</font> <br></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div><div>Jim Edwards<br><br></div><font size="1">CESM Software Engineer<br></font></div><font size="1">National Center for Atmospheric Research<br></font></div><font size="1">Boulder, CO</font> <br></div></div>