<div dir="ltr">Sorry, I did not mean any disrespect to ORNL. I (my colleague) was painting with a broad brush and our experience is not with scientists. And I am sure that our experience with ORNL management does not reflect the attitude of all of ORNL management. I'm glad to hear that there are important project to ORNL that are behind PETSc.<div><br></div><div>Our project has been told _explicitly_ to seek alternatives to PETSc because they say PETSC will not run well on SUMMIT (something about Barry not liking threads). We have had to put risk mitigation material in proposals to the effect "we will use hypre if PETSc does not work". Nothing against hypre but this is silly.<div><br></div><div>Mark</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 27, 2015 at 2:57 PM, Bartlett, Roscoe A. <span dir="ltr"><<a href="mailto:bartlettra@ornl.gov" target="_blank">bartlettra@ornl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If ORNL hates PETSc, then they forgot to tell the CASL codes
</span><span style="font-size:11.0pt;font-family:Wingdings;color:#1f497d">J</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">-Ross<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><a name="14bcc9d1a8c7dc95__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></a></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:ideas-team-bounces@lists.mcs.anl.gov" target="_blank">ideas-team-bounces@lists.mcs.anl.gov</a> [mailto:<a href="mailto:ideas-team-bounces@lists.mcs.anl.gov" target="_blank">ideas-team-bounces@lists.mcs.anl.gov</a>]
<b>On Behalf Of </b>Mark Adams<br>
<b>Sent:</b> Friday, February 27, 2015 2:06 PM<br>
<b>To:</b> Todd Gamblin<br>
<b>Cc:</b> petsc-dev; <a href="mailto:ideas-team@lists.mcs.anl.gov" target="_blank">ideas-team@lists.mcs.anl.gov</a><br>
<b>Subject:</b> Re: [Ideas-team] [petsc-dev] Seeking OLCF users complaining about poor build times<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">FWIW, I have some log files from Edison and Titan in the last week using thread safe configuration, SuperLU, hypre, & Metis. Its not clear to me if the external packages were built before. That is, I am not sure these were clean builds.
Like Nathan Titan is good:<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">11m config<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">3min make<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">and Edison is not good:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">1hr 34min config<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">17 min make<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">And unless you are doing git bisect who cares if it take more than a minute you have a context switch anyway. And I have probably configure PETSc at least 20 times this past week on Edison and Hopper.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">BTW, One of my PIs told me recently "ORNL hates PETSc", and I said PETSc is like Broccoli, you like it or you don't, and what if next week ORNL hates MPI? (my PI seemed to appreciate that) ... or FORTRAN (I should have said that too).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Mark<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Fri, Feb 27, 2015 at 1:10 PM, Todd Gamblin <<a href="mailto:tgamblin@llnl.gov" target="_blank">tgamblin@llnl.gov</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">CMake test stuff too, so it has the same problem. At the very least it<br>
tests the compiler id and ends up creating a bunch of directories and<br>
files in the CMakeFiles directory.<br>
<br>
So you're still in the same boat with CMake... But you don't have to<br>
maintain your own elaborate build system on the side.<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On 2/27/15, 10:06 AM, "Barry Smith" <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
<br>
><br>
>> On Feb 27, 2015, at 12:00 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br>
>><br>
>> Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> writes:<br>
>>> Actually the parallel compiles of the 1000+ files on the "regular"<br>
>>> filesystems at ANL and LBL is taking less than 2 minutes so I can't<br>
>>> blame the filesystem bandwidth.<br>
>><br>
>> I think bandwidth is adequate, but latency (especially for metadata) is<br>
>> rather high. Normal make uses parallelism to mitigate, but configure is<br>
>> sequential, so gets hit harder.<br>
><br>
> Yup, that was my conclusion. So the solution is 1) apply pressure to<br>
>improve latency on these systems a bit and 2) incorporate more<br>
>parallelism in ./configure without making it even more complicated. Or<br>
>switch to cmake where you don't test anything but just read the machines<br>
>capabilities from an outdated database :-).<br>
><br>
> Barry<br>
><br>
><br>
<br>
<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>