[petsc-users] Amortizing calls to PetscOptionsFindPair_Private()

[Barry Smith]

> On Jan 16, 2016, at 7:12 AM, Griffith, Boyce Eugene <boyceg at email.unc.edu> wrote:
> 
> 
>> On Jan 16, 2016, at 12:34 AM, Bhalla, Amneet Pal S <amneetb at live.unc.edu> wrote:
>> 
>> 
>> 
>>> On Jan 15, 2016, at 5:40 PM, Matthew Knepley <knepley at gmail.com> wrote:
>>> 
>>> I am inclined to try
>>> Barry's experiment first, since this may have bugs that we have not yet discovered.
>> 
>> Ok, I tried Barry’s suggestion. The runtime for PetscOptionsFindPair_Private() fell from 14% to mere 1.6%.
>> If I am getting it right, it’s the petsc options in the KSPSolve() that is sucking up nontrivial amount of time (14 - 1.6)
>> and not KSPSetFromOptions() itself (1.6%). 
> 
> Barry / Matt / Jed, if we were using KSPReset here and reusing KSPs, would that also bypass these calls to PetscOptionsXXX?

  No that is a different issue.

   In the short term I recommend when running optimized/production you work with a PETSc with those Options checking in KSPSolve commented out, you don't use them anyways*.  Since you are using ASM with many subdomains there are many "fast" calls to KSPSolve which is why for your particular case the the PetscOptionsFindPair_Private takes so much time.

  Now that you have eliminated this issue I would be very interested in seeing the HPCToolKit or Instruments profiling of the code to see  hot spots in the PETSc solver configuration you are using. Thanks 

   Barry

* Eventually we'll switch to a KSPPreSolveMonitorSet() and KSPPostSolveMonitorSet() model to eliminate this overhead but still have the functionality.

> 
> Thanks,
> 
> -- Boyce