<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Aug 31, 2016 at 5:23 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div>Matt,<br><br></div>So is the "solve phase" going to be KSPSolve() - PCSetUp()?<br></div></div></div></blockquote><div><br></div><div>Setup Phase: KSPSetUp + PCSetup</div><div><br></div><div>Solve Phase: SNESSolve<br></div><div> This contains <span style="color:rgb(80,0,80)">SNESFunctionEval, </span><span style="color:rgb(80,0,80)">SNESJacobianEval, KSPSolve</span></div><div><span style="color:rgb(80,0,80)"><br></span></div><div><span style="color:rgb(80,0,80)"> Matt</span></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div></div>In other words, if I want to look at time/iterations, should it just be over KSPSolve or should I exclude the PC setup?<span class=""><font color="#888888"><br><br></font></span></div><span class=""><font color="#888888">Justin</font></span><div><div class="h5"><br><div><div><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 31, 2016 at 5:13 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Wed, Aug 31, 2016 at 2:01 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div>Attached is the -log_view output (from firedrake). Event Stage 1: Linear_solver is where I assemble and solve the linear system of equations.<br><br></div>I am using the HYPRE BoomerAMG preconditioner so log_view cannot "see into" the exact steps, but based on what it can see, how do I distinguish between these various setup and timing phases?<br><br></div>For example, when I look at these lines:<br><br>PCSetUp 1 1.0 2.2858e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 9 0 0 0 0 11 0 0 0 0 0<br>PCApply 38 1.0 1.4102e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 56 0 0 0 0 66 0 0 0 0 0<br>KSPSetUp 1 1.0 9.9111e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>KSPSolve 1 1.0 1.7529e+01 1.0 2.44e+09 1.0 0.0e+00 0.0e+00 0.0e+00 70 7 0 0 0 82 7 0 0 0 139<br>SNESSolve 1 1.0 2.1056e+01 1.0 3.75e+10 1.0 0.0e+00 0.0e+00 0.0e+00 84100 0 0 0 99100 0 0 0 1781<br>SNESFunctionEval 1 1.0 1.0763e+00 1.0 1.07e+10 1.0 0.0e+00 0.0e+00 0.0e+00 4 29 0 0 0 5 29 0 0 0 9954<br>SNESJacobianEval 1 1.0 2.4495e+00 1.0 2.43e+10 1.0 0.0e+00 0.0e+00 0.0e+00 10 65 0 0 0 12 65 0 0 0 9937<br><br></div>So how do I break down "mesh setup", "matrix setup", and "solve time" phases? I am guessing "PCSetUp" has to do with one of the first two phases, but how would I categorize the rest of the events? I see that HYPRE doesn't have as much information as the other PCs like GAMG and ML but can one still breakdown the timing phases through log_view alone?<br></div></div></blockquote><div><br></div></span><div>1) It looks like you call PCSetUp() yourself, since otherwise KSPSetUp() would contain that time. Notice that you can ignore KSPSetUp() here.</div><div><br></div><div>2) The setup time is usually KSPSetUp(), but if here you add to it PCSetUp() since you called it.</div><div><br></div><div>3) The solve time for SNES can be split into</div><div><br></div><div> a) KSPSolve() for the update calculation</div><div><br></div><div> b) SNESFunctionEval, SNESJacobianEval for everything else (conv check, line search, J calc, etc.) or you can just take SNESSolve() - KSPSolve()</div><div><br></div><div> 4) Note that PCApply() is most of KSPSolve(), which is generally good</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Thanks,<br></div><div>Justin<br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 30, 2016 at 11:14 PM, Jed Brown <span dir="ltr"><<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> writes:<br>
<br>
>><br>
>><br>
>> Anyway, what I really wanted to say is, it's good to know that these<br>
>> "dynamic range/performance spectrum/static scaling" plots are designed to<br>
>> go past the sweet spots. I also agree that it would be interesting to see a<br>
>> time vs dofs*iterations/time plot. Would it then also be useful to look at<br>
>> the step to setting up the preconditioner?<br>
>><br>
>><br>
> Yes, I generally split up timing between "mesh setup" (symbolic<br>
> factorization of LU), "matrix setup" (eg, factorizations), and solve time.<br>
> The degree of amortization that you get for the two setup phases depends on<br>
> your problem and so it is useful to separate them.<br>
<br>
</span>Right, there is nothing wrong with splitting up the phases, but if you<br>
never show a spectrum for the total, then I will be suspicious. And if<br>
you only show "per iteration" instead of for a complete solve, then I<br>
will assume that you're only doing that because convergence is unusably<br>
slow.<br>
</blockquote></div><br></div>
</div></div></blockquote></span></div><br><br clear="all"><span><div><br></div>-- <br><div data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</span></div></div>
</blockquote></div><br></div></div></div></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>