[petsc-users] Amortizing calls to PetscOptionsFindPair_Private()

[Barry Smith]

> On Jan 16, 2016, at 3:10 PM, Griffith, Boyce Eugene <boyceg at email.unc.edu> wrote:
> 
> 
>> On Jan 16, 2016, at 4:06 PM, Bhalla, Amneet Pal S <amneetb at live.unc.edu> wrote:
>> 
>> Does Instruments save results somewhere (like in a cascade view) that I can send to Barry?
> 
> Yes --- "save as..." will save the current trace, and then you can open it back up.

  Either way is fine so long as I don't have to install a ton of stuff; which it sounds like I won't.

  Barry

> 
> -- Boyce
> 
>> --Amneet Bhalla 
>> 
>> On Jan 16, 2016, at 1:04 PM, Griffith, Boyce Eugene <boyceg at email.unc.edu> wrote:
>> 
>>> 
>>>> On Jan 16, 2016, at 4:00 PM, Bhalla, Amneet Pal S <amneetb at live.unc.edu> wrote:
>>>> 
>>>> 
>>>> 
>>>> --Amneet Bhalla 
>>>> 
>>>> On Jan 16, 2016, at 10:21 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>> 
>>>>> 
>>>>>> On Jan 16, 2016, at 7:12 AM, Griffith, Boyce Eugene <boyceg at email.unc.edu> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Jan 16, 2016, at 12:34 AM, Bhalla, Amneet Pal S <amneetb at live.unc.edu> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Jan 15, 2016, at 5:40 PM, Matthew Knepley <knepley at gmail.com> wrote:
>>>>>>>> 
>>>>>>>> I am inclined to try
>>>>>>>> Barry's experiment first, since this may have bugs that we have not yet discovered.
>>>>>>> 
>>>>>>> Ok, I tried Barry’s suggestion. The runtime for PetscOptionsFindPair_Private() fell from 14% to mere 1.6%.
>>>>>>> If I am getting it right, it’s the petsc options in the KSPSolve() that is sucking up nontrivial amount of time (14 - 1.6)
>>>>>>> and not KSPSetFromOptions() itself (1.6%). 
>>>>>> 
>>>>>> Barry / Matt / Jed, if we were using KSPReset here and reusing KSPs, would that also bypass these calls to PetscOptionsXXX?
>>>>> 
>>>>>  No that is a different issue.
>>>>> 
>>>>>   In the short term I recommend when running optimized/production you work with a PETSc with those Options checking in KSPSolve commented out, you don't use them anyways*.  Since you are using ASM with many subdomains there are many "fast" calls to KSPSolve which is why for your particular case the the PetscOptionsFindPair_Private takes so much time.
>>>>> 
>>>>>  Now that you have eliminated this issue I would be very interested in seeing the HPCToolKit or Instruments profiling of the code to see  hot spots in the PETSc solver configuration you are using. Thanks 
>>>> 
>>>> Barry --- the best way and the least back and forth way would be if I can send you the files (maybe off-list) that you can view in HPCViewer, which is a light weight java script app. You can view which the calling context (which petsc function calls which internal petsc routine) in a cascade form. If I send you an excel sheet, it would be in a flat view and not that useful for serious profiling.
>>> 
>>> Amneet, can you just run with OS X Instruments, which Barry already knows how to use (right Barry?)? :-)
>>> 
>>> Thanks,
>>> 
>>> -- Boyce
>>> 
>>>> 
>>>> Let me know if you would like to try that. 
>>>>> 
>>>>>   Barry
>>>>> 
>>>>> * Eventually we'll switch to a KSPPreSolveMonitorSet() and KSPPostSolveMonitorSet() model to eliminate this overhead but still have the functionality.
>>>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> -- Boyce
>>>>> 
>>> 
>