[petsc-users] strong-scaling vs weak-scaling

[Justin Chang]

Matt,

So is the "solve phase" going to be KSPSolve() - PCSetUp()?

In other words, if I want to look at time/iterations, should it just be
over KSPSolve or should I exclude the PC setup?

Justin


On Wed, Aug 31, 2016 at 5:13 AM, Matthew Knepley <knepley at gmail.com> wrote:

> On Wed, Aug 31, 2016 at 2:01 AM, Justin Chang <jychang48 at gmail.com> wrote:
>
>> Attached is the -log_view output (from firedrake). Event Stage 1:
>> Linear_solver is where I assemble and solve the linear system of equations.
>>
>> I am using the HYPRE BoomerAMG preconditioner so log_view cannot "see
>> into" the exact steps, but based on what it can see, how do I distinguish
>> between these various setup and timing phases?
>>
>> For example, when I look at these lines:
>>
>> PCSetUp                1 1.0 2.2858e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
>> 0.0e+00  9  0  0  0  0  11  0  0  0  0     0
>> PCApply               38 1.0 1.4102e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
>> 0.0e+00 56  0  0  0  0  66  0  0  0  0     0
>> KSPSetUp               1 1.0 9.9111e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
>> 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
>> KSPSolve               1 1.0 1.7529e+01 1.0 2.44e+09 1.0 0.0e+00 0.0e+00
>> 0.0e+00 70  7  0  0  0  82  7  0  0  0   139
>> SNESSolve              1 1.0 2.1056e+01 1.0 3.75e+10 1.0 0.0e+00 0.0e+00
>> 0.0e+00 84100  0  0  0  99100  0  0  0  1781
>> SNESFunctionEval       1 1.0 1.0763e+00 1.0 1.07e+10 1.0 0.0e+00 0.0e+00
>> 0.0e+00  4 29  0  0  0   5 29  0  0  0  9954
>> SNESJacobianEval       1 1.0 2.4495e+00 1.0 2.43e+10 1.0 0.0e+00 0.0e+00
>> 0.0e+00 10 65  0  0  0  12 65  0  0  0  9937
>>
>> So how do I break down "mesh setup", "matrix setup", and "solve time"
>> phases? I am guessing "PCSetUp" has to do with one of the first two phases,
>> but how would I categorize the rest of the events? I see that HYPRE doesn't
>> have as much information as the other PCs like GAMG and ML but can one
>> still breakdown the timing phases through log_view alone?
>>
>
> 1) It looks like you call PCSetUp() yourself, since otherwise KSPSetUp()
> would contain that time. Notice that you can ignore KSPSetUp() here.
>
> 2) The setup time is usually KSPSetUp(), but if here you add to it
> PCSetUp() since you called it.
>
> 3) The solve time for SNES can be split into
>
>   a) KSPSolve() for the update calculation
>
>   b) SNESFunctionEval, SNESJacobianEval for everything else (conv check,
> line search, J calc, etc.) or you can just take SNESSolve() - KSPSolve()
>
>  4) Note that PCApply() is most of KSPSolve(), which is generally good
>
>   Thanks,
>
>      Matt
>
>
>> Thanks,
>> Justin
>>
>> On Tue, Aug 30, 2016 at 11:14 PM, Jed Brown <jed at jedbrown.org> wrote:
>>
>>> Mark Adams <mfadams at lbl.gov> writes:
>>>
>>> >>
>>> >>
>>> >> Anyway, what I really wanted to say is, it's good to know that these
>>> >> "dynamic range/performance spectrum/static scaling" plots are
>>> designed to
>>> >> go past the sweet spots. I also agree that it would be interesting to
>>> see a
>>> >> time vs dofs*iterations/time plot. Would it then also be useful to
>>> look at
>>> >> the step to setting up the preconditioner?
>>> >>
>>> >>
>>> > Yes, I generally split up timing between "mesh setup" (symbolic
>>> > factorization of LU), "matrix setup" (eg, factorizations), and solve
>>> time.
>>> > The degree of amortization that you get for the two setup phases
>>> depends on
>>> > your problem and so it is useful to separate them.
>>>
>>> Right, there is nothing wrong with splitting up the phases, but if you
>>> never show a spectrum for the total, then I will be suspicious.  And if
>>> you only show "per iteration" instead of for a complete solve, then I
>>> will assume that you're only doing that because convergence is unusably
>>> slow.
>>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160831/85697d07/attachment-0001.html>