[petsc-users] Amortizing calls to PetscOptionsFindPair_Private()

[Barry Smith]

  Amneet,

    Thanks for bringing this to our attention. The long term design goal in PETSc is that the PetscOptions... calls are all made from the XXXSetFromOptions() calls and not within the numerical solver portions. Unfortunately this is not as easy to do as it might seem; hence there are a bunch of them scattered within the solver portions. 

   In particular the worse culprit is KSPSolve(). You can run the following experiment: edit src/ksp/ksp/interface/itfunc.c and locate the function KSPSolve() now comment out all the lines with the work Option in them (I count about 17 of them) now do make gnumake in that directory (of course with optimized build) then rerun your exact same code that you report for from below.

   How much faster is the total time and how much percentage are the troublesome Options calls now? In other words how much does this change help? A dramatic difference would motivate us to fix this problem sooner rather than later.

  Barry

> On Jan 15, 2016, at 5:33 PM, Bhalla, Amneet Pal S <amneetb at live.unc.edu> wrote:
> 
> 
> Hi Barry,
> 
> In our code at each timestep we build MG level smoothers using PETSc KSP solvers. We are using a PETSc function KSPSetFromOptions()
> after we set some default values to the KSP. However, the profiler is showing that PetscOptionsFindPair_Private() is taking about 14% of total runtime.
> We ran the code for 100 timesteps, and preconditioner is built everytime step. I am posting a sequence of calls to KSPSolve_Richardson that shows
> getting PETScOptions adds up to a lot of cost
> 
> "KSPSolve_Richardson" 	       8.80e+05 12.7% 	
>   "PCApplyBAorAB" 	               5.25e+05  7.6% 	
>     "PCApply" 	                               4.85e+05  7.0% 	
>       "PCApply_ASM" 	               4.85e+05  7.0% 	
>         "KSPSolve" 	                       4.53e+05  6.6% 	
>           "KSPSolve_PREONLY"                      2.06e+05  3.0% 	
>           "PetscObjectViewFromOptions" 	       3.19e+04  0.5% 	
>           "PetscObjectViewFromOptions" 	       2.39e+04  0.3% 	
>           "PetscOptionsGetBool" 	               2.39e+04  0.3% 	
>           "PetscOptionsHasName" 	               2.39e+04  0.3% 	
>          "PetscOptionsGetBool" 	               2.39e+04  0.3% 	
>          "PetscOptionsHasName" 	               1.60e+04  0.2% 	
>          "PetscOptionsGetBool" 	               1.60e+04  0.2% 	
>          "PetscObjectViewFromOptions" 	       1.60e+04  0.2% 	
>          "KSPReasonViewFromOptions" 	       1.60e+04  0.2% 	
>          "PetscOptionsGetBool" 	               1.56e+04  0.2% 	
>          "PetscObjectViewFromOptions" 	        7.98e+03  0.1% 	
>          "KSPSetUpOnBlocks" 	                        7.98e+03  0.1% 	
>          "PetscOptionsGetBool" 	                7.98e+03  0.1% 	
>          "VecSet" 	                                        7.97e+03  0.1% 	
>          "PetscObjectViewFromOptions" 	        7.92e+03  0.1%
> 
> Do you have some suggestions as to doing it in a fast way -- maybe parsing options only once in the simulation and making populating KSP 
> options essentially a no-op?
> 
> Thanks,
> --Amneet