[petsc-users] Amortizing calls to PetscOptionsFindPair_Private()

[Griffith, Boyce Eugene]

On Jan 16, 2016, at 4:00 PM, Bhalla, Amneet Pal S <amneetb at live.unc.edu<mailto:amneetb at live.unc.edu>> wrote:



--Amneet Bhalla

On Jan 16, 2016, at 10:21 AM, Barry Smith <bsmith at mcs.anl.gov<mailto:bsmith at mcs.anl.gov>> wrote:


On Jan 16, 2016, at 7:12 AM, Griffith, Boyce Eugene <boyceg at email.unc.edu<mailto:boyceg at email.unc.edu>> wrote:


On Jan 16, 2016, at 12:34 AM, Bhalla, Amneet Pal S <amneetb at live.unc.edu<mailto:amneetb at live.unc.edu>> wrote:



On Jan 15, 2016, at 5:40 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:

I am inclined to try
Barry's experiment first, since this may have bugs that we have not yet discovered.

Ok, I tried Barry’s suggestion. The runtime for PetscOptionsFindPair_Private() fell from 14% to mere 1.6%.
If I am getting it right, it’s the petsc options in the KSPSolve() that is sucking up nontrivial amount of time (14 - 1.6)
and not KSPSetFromOptions() itself (1.6%).

Barry / Matt / Jed, if we were using KSPReset here and reusing KSPs, would that also bypass these calls to PetscOptionsXXX?

 No that is a different issue.

  In the short term I recommend when running optimized/production you work with a PETSc with those Options checking in KSPSolve commented out, you don't use them anyways*.  Since you are using ASM with many subdomains there are many "fast" calls to KSPSolve which is why for your particular case the the PetscOptionsFindPair_Private takes so much time.

 Now that you have eliminated this issue I would be very interested in seeing the HPCToolKit or Instruments profiling of the code to see  hot spots in the PETSc solver configuration you are using. Thanks

Barry --- the best way and the least back and forth way would be if I can send you the files (maybe off-list) that you can view in HPCViewer, which is a light weight java script app. You can view which the calling context (which petsc function calls which internal petsc routine) in a cascade form. If I send you an excel sheet, it would be in a flat view and not that useful for serious profiling.

Amneet, can you just run with OS X Instruments, which Barry already knows how to use (right Barry?)? :-)

Thanks,

-- Boyce


Let me know if you would like to try that.

  Barry

* Eventually we'll switch to a KSPPreSolveMonitorSet() and KSPPostSolveMonitorSet() model to eliminate this overhead but still have the functionality.


Thanks,

-- Boyce


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160116/ba68dace/attachment.html>