<div dir="ltr">Barry, thanks for your comments and advise.<div>Lagging Jacobian evaluation certainly helped (just tested it).</div><div>However, eventually the finite differencing for Jacobian evaluation should be replaced with some sort of approximated Jacobian evaluation subroutine.</div><div>So at this moment I don't worry too much on its cost.</div><div><br></div><div>Thanks again,</div><div><br></div><div>Ling</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 19, 2017 at 12:02 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
> On May 19, 2017, at 9:25 AM, Zou, Ling <<a href="mailto:ling.zou@inl.gov">ling.zou@inl.gov</a>> wrote:<br>
><br>
> Hi All,<br>
><br>
> In terms of code performance, sometimes people would ask for info about total non-linear iteration numbers, total linear iteration numbers, etc. I suppose all these could be found in the log summary. For the attached log summary, can I say?<br>
> total non-linear iteration number = 573<br>
> total linear iteration number = 2321<br>
<br>
</span> Yes,<br>
<br>
The log file is kind of funny. It spends 62% of the time in MatFDColorApply() which is computing the Jacobian via differencing and coloring, this is a lot of time. You might consider lagging the Jacobian; that is not recompute the Jacobian for each new linear solve. You can use -snes_lag_jacobian 2 or -snes_lag_jacobian 3 etc and see how this affects the run time.<br>
<br>
Barry<br>
<br>
<br>
<br>
<br>
> MatMult MF<br>
<div class="HOEnZb"><div class="h5">><br>
> Thank you.<br>
><br>
> Ling<br>
><br>
><br>
><br>
> ------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<br>
> Event Count Time (sec) Flops --- Global --- --- Stage --- Total<br>
> Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s<br>
> ------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<br>
><br>
> --- Event Stage 0: Main Stage<br>
><br>
> VecDot 607 1.0 1.8729e-04 1.0 1.95e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1040<br>
> VecMDot 2321 1.0 1.1075e-03 1.0 2.87e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 5 0 0 0 0 5 0 0 0 2590<br>
> VecNorm 5422 1.0 1.2229e-03 1.0 1.74e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 1423<br>
> VecScale 5822 1.0 1.2764e-03 1.0 9.37e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 734<br>
> VecCopy 14334 1.0 1.8302e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> VecSet 1231 1.0 3.0700e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> VecAXPY 14961 1.0 3.3679e-03 1.0 4.82e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 0 9 0 0 0 1430<br>
> VecWAXPY 20842 1.0 5.5537e-03 1.0 4.00e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 8 0 0 0 0 8 0 0 0 721<br>
> VecMAXPY 2894 1.0 1.4292e-03 1.0 3.62e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 7 0 0 0 2536<br>
> VecSetRandom 34 1.0 1.0322e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> VecReduceArith 1146 1.0 2.9907e-04 1.0 3.68e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 1230<br>
> VecReduceComm 573 1.0 9.2384e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> VecNormalize 2894 1.0 1.9604e-03 1.0 1.39e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 712<br>
> SNESJacobianEval 573 1.0 2.2410e+00 1.0 5.60e+06 1.0 0.0e+00 0.0e+00 0.0e+00 62 11 0 0 0 62 11 0 0 0 2<br>
> MatMult MF 2928 1.0 5.7922e-01 1.0 2.83e+06 1.0 0.0e+00 0.0e+00 0.0e+00 16 5 0 0 0 16 5 0 0 0 5<br>
> MatMult 2928 1.0 5.7963e-01 1.0 2.83e+06 1.0 0.0e+00 0.0e+00 0.0e+00 16 5 0 0 0 16 5 0 0 0 5<br>
> MatSolve 2894 1.0 7.3171e-03 1.0 1.76e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 33 0 0 0 0 33 0 0 0 2405<br>
> MatLUFactorNum 573 1.0 1.4733e-02 1.0 1.71e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 32 0 0 0 0 32 0 0 0 1158<br>
> MatILUFactorSym 1 1.0 2.4543e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatAssemblyBegin 1147 1.0 7.9204e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatAssemblyEnd 1147 1.0 2.5825e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatGetRowIJ 2 1.0 6.3280e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatGetOrdering 1 1.0 1.5754e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatZeroEntries 573 1.0 5.7120e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatFDColorCreate 1 1.0 2.0541e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatFDColorSetUp 1 1.0 1.6103e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> MatFDColorApply 573 1.0 2.2386e+00 1.0 5.60e+06 1.0 0.0e+00 0.0e+00 0.0e+00 62 11 0 0 0 62 11 0 0 0 2<br>
> MatFDColorFunc 11460 1.0 2.2264e+00 1.0 1.85e+06 1.0 0.0e+00 0.0e+00 0.0e+00 62 3 0 0 0 62 3 0 0 0 1<br>
> MatColoringApply 1 1.0 3.9990e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> KSPGMRESOrthog 2321 1.0 2.9685e-03 1.0 5.75e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 11 0 0 0 0 11 0 0 0 1935<br>
> KSPSetUp 573 1.0 2.3291e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0<br>
> KSPSolve 573 1.0 5.0164e-01 1.0 4.50e+07 1.0 0.0e+00 0.0e+00 0.0e+00 14 85 0 0 0 14 85 0 0 0 90<br>
> PCSetUp 573 1.0 1.5172e-02 1.0 1.71e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 32 0 0 0 0 32 0 0 0 1124<br>
> PCApply 2894 1.0 7.8614e-03 1.0 1.76e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 33 0 0 0 0 33 0 0 0 2239<br>
> ------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<br>
><br>
<br>
</div></div></blockquote></div><br></div>