<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body dir="auto">
<div><br>
<br>
--Amneet Bhalla<span style="font-size: 13pt;"> </span></div>
<div><br>
On Jan 16, 2016, at 10:21 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div><span></span><br>
<blockquote type="cite"><span>On Jan 16, 2016, at 7:12 AM, Griffith, Boyce Eugene <<a href="mailto:boyceg@email.unc.edu">boyceg@email.unc.edu</a>> wrote:</span><br>
</blockquote>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>On Jan 16, 2016, at 12:34 AM, Bhalla, Amneet Pal S <<a href="mailto:amneetb@live.unc.edu">amneetb@live.unc.edu</a>> wrote:</span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span></span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span></span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span></span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite"><span>On Jan 15, 2016, at 5:40 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:</span><br>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite"><span></span><br>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite"><span>I am inclined to try</span><br>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite"><span>Barry's experiment first, since this may have bugs that we have not yet discovered.</span><br>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span></span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>Ok, I tried Barry’s suggestion. The runtime for PetscOptionsFindPair_Private() fell from 14% to mere 1.6%.</span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>If I am getting it right, it’s the petsc options in the KSPSolve() that is sucking up nontrivial amount of time (14 - 1.6)</span><br>
</blockquote>
</blockquote>
<blockquote type="cite">
<blockquote type="cite"><span>and not KSPSetFromOptions() itself (1.6%). </span><br>
</blockquote>
</blockquote>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite"><span>Barry / Matt / Jed, if we were using KSPReset here and reusing KSPs, would that also bypass these calls to PetscOptionsXXX?</span><br>
</blockquote>
<span></span><br>
<span> No that is a different issue.</span><br>
<span></span><br>
<span> In the short term I recommend when running optimized/production you work with a PETSc with those Options checking in KSPSolve commented out, you don't use them anyways*. Since you are using ASM with many subdomains there are many "fast" calls to KSPSolve
which is why for your particular case the the PetscOptionsFindPair_Private takes so much time.</span><br>
<span></span><br>
<span> Now that you have eliminated this issue I would be very interested in seeing the HPCToolKit or Instruments profiling of the code to see hot spots in the PETSc solver configuration you are using. Thanks
</span><br>
</div>
</blockquote>
<div><br>
</div>
Barry --- the best way and the least back and forth way would be if I can send you the files (maybe off-list) that you can view in HPCViewer, which is a light weight java script app. You can view which the calling context (which petsc function calls which internal
petsc routine) in a cascade form. If I send you an excel sheet, it would be in a flat view and not that useful for serious profiling.
<div><br>
</div>
<div>Let me know if you would like to try that. <br>
<blockquote type="cite">
<div><span></span><br>
<span> Barry</span><br>
<span></span><br>
<span>* Eventually we'll switch to a KSPPreSolveMonitorSet() and KSPPostSolveMonitorSet() model to eliminate this overhead but still have the functionality.</span><br>
<span></span><br>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite"><span>Thanks,</span><br>
</blockquote>
<blockquote type="cite"><span></span><br>
</blockquote>
<blockquote type="cite"><span>-- Boyce</span><br>
</blockquote>
<span></span><br>
</div>
</blockquote>
</div>
</body>
</html>