[petsc-users] SNES stuck with small residual

Thu Sep 24 18:27:12 CDT 2015

On Thu, Sep 24, 2015 at 6:05 PM, Gideon Simpson <gideon.simpson at gmail.com>
wrote:

> Nothing springs to mind in terms of the computation.  The only non-trivial
> calculation is the use of PetscPowScalar; everything else is
> straightforward +/-*.  The problem does have 10^7 unknowns; could that be
> an issue?
>

The unknowns alone would not cause this. You could have catastrophic
cancellation in your evaluation, even just using -. I am
suspicious of this explanation, but Barry is right that is can be tested
fairly easily by using 128.

   Matt

> -gideon
>
> On Sep 24, 2015, at 7:01 PM, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Thu, Sep 24, 2015 at 5:44 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>
>>   I completely disagree with Matt, looking at the numbers one sees very
>> typical Newton behavior
>>
>> a little slow at the beginning because it is not in the quadratic basin
>>
>> 0 SNES Function norm 5.979113140046e+02
>>   1 SNES Function norm 5.106334303399e+02
>>   2 SNES Function norm 4.522093337153e+02
>>   3 SNES Function norm 4.042911112972e+02
>>   4 SNES Function norm 3.618952927134e+02
>>   5 SNES Function norm 3.239307814846e+02
>>   6 SNES Function norm 2.899625386925e+02
>>   7 SNES Function norm 2.596235386610e+02
>>   8 SNES Function norm 2.178613867932e+02
>>   9 SNES Function norm 1.541279494063e+02
>>  10 SNES Function norm 1.361917300390e+02
>>
>>  suddenly fast convergence as it gets in the quadratic basin and behaves
>> like Newton should behave
>>
>>  11 SNES Function norm 6.951420638296e+00
>>  12 SNES Function norm 3.249486326694e-03
>>  13 SNES Function norm 4.590368792536e-05
>>
>
> Not exactly quadratic, but definitely faster than beforeIt still looks
> about linear with c = 0.01. However,
> Barry might be right since this rapid convergence arrests suddenly.
>
> Can you think of a reason why your function evaluation would only be
> single precision?
>
>    Matt
>
>
>> no more convergence as it "runs out of digits"
>>
>>  14 SNES Function norm 4.422797479565e-05
>>  15 SNES Function norm 4.371345326958e-05
>>  16 SNES Function norm 4.348609553058e-05
>>  17 SNES Function norm 4.324907311342e-05
>>  18 SNES Function norm 4.324904992696e-05
>>  19 SNES Function norm 4.324904992565e-05
>>
>> The only difference between this run and most other runs of Newton (when
>> it works in the standard way) is that it "runs out of digits" early (after
>> only reducing the residual norm by 1e-7 when for super nice problems it can
>> do like 1e-12).
>>
>> I don't think your Jacobian is crap, I don't think the Jacobian direction
>> is crap and the Jacobian is not not terribly ill-conditioned since the
>> direct solver reduces the residual norm by 1e-8.
>>
>> My recommendation when Newton "runs out of digits" is to try using quad
>> precision. With PETSc you just need a recent version of the GNU compilers
>> and the configure options --with-precision=__float128
>> --download-f2cblaslapack   plus you need to make sure your code always uses
>> PetscReal and PetscScalar instead of "double" or whatever and whenever you
>> call elementary functions like sin() you call PetscSinReal() or
>> PetscSinScalar() etc so that it uses the quad precision version (if you
>> miss one of these you will never see the quad precision).
>>
>>   Barry
>>
>> What does it mean for Newton to "run out of digits"; this is not a
>> technical term but I believe it is related to how accurate your function
>> evaluations can be (I think some people call this noise) when expressed
>> using floating point numbers. Perhaps it is something like "ill-conditioned
>> functions". If the function evaluations are only valid for 8 digits then it
>> is not reasonable to expect Newton (or any floating point algorithm) to
>> squeeze the norm of the function values to 1e.-12.
>>
>>
>>
>>
>>
>> > On Sep 24, 2015, at 5:28 PM, Matthew Knepley <knepley at gmail.com> wrote:
>> >
>> > On Thu, Sep 24, 2015 at 5:23 PM, Gideon Simpson <
>> gideon.simpson at gmail.com> wrote:
>> > Just wondering if anyone has any suggestions for this scenario:
>> >
>> > I have seen this happen a bunch. Your Newton directions are crap. That
>> means
>> >
>> >   a) you have a crap Jacobian
>> >
>> >   b) your Jacobian direction is crap (unlikely here)
>> >
>> >   c) your Jacobian is very ill-conditioned
>> >
>> > Something to try in order to see what is happening is to start doing
>> Picard when Newton stalls. If it
>> > converges, then you have something like c) I think. We can try to
>> mitigate this by NPC. Something
>> > as simple as Newton + NCG or Newton + NRichardson can work.
>> >
>> >   Matt
>> >
>> >   0 SNES Function norm 5.979113140046e+02
>> >     0 KSP preconditioned resid norm 1.609660749629e+03 true resid norm
>> 5.979113140046e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.083640787055e-06 true resid norm
>> 4.070012556358e-04 ||r(i)||/||b|| 6.807050579289e-07
>> >   1 SNES Function norm 5.106334303399e+02
>> >     0 KSP preconditioned resid norm 1.304622870569e+03 true resid norm
>> 5.106334303399e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.542022975926e-06 true resid norm
>> 3.284828506619e-04 ||r(i)||/||b|| 6.432850478341e-07
>> >   2 SNES Function norm 4.522093337153e+02
>> >     0 KSP preconditioned resid norm 1.017788935768e+03 true resid norm
>> 4.522093337153e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.338757544659e-06 true resid norm
>> 2.556045980853e-04 ||r(i)||/||b|| 5.652351223829e-07
>> >   3 SNES Function norm 4.042911112972e+02
>> >     0 KSP preconditioned resid norm 7.996536035697e+02 true resid norm
>> 4.042911112972e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 8.612178921208e-07 true resid norm
>> 2.026756625099e-04 ||r(i)||/||b|| 5.013112008810e-07
>> >   4 SNES Function norm 3.618952927134e+02
>> >     0 KSP preconditioned resid norm 6.392806337991e+02 true resid norm
>> 3.618952927134e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 5.099112497936e-07 true resid norm
>> 1.605851784448e-04 ||r(i)||/||b|| 4.437338138353e-07
>> >   5 SNES Function norm 3.239307814846e+02
>> >     0 KSP preconditioned resid norm 5.205889419531e+02 true resid norm
>> 3.239307814846e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 3.364948858968e-07 true resid norm
>> 1.326652268287e-04 ||r(i)||/||b|| 4.095480714142e-07
>> >   6 SNES Function norm 2.899625386925e+02
>> >     0 KSP preconditioned resid norm 4.305631057215e+02 true resid norm
>> 2.899625386925e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 2.472775427751e-07 true resid norm
>> 1.094956512355e-04 ||r(i)||/||b|| 3.776199909453e-07
>> >   7 SNES Function norm 2.596235386610e+02
>> >     0 KSP preconditioned resid norm 3.606693697722e+02 true resid norm
>> 2.596235386610e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.804499181789e-07 true resid norm
>> 9.162646660567e-05 ||r(i)||/||b|| 3.529204904849e-07
>> >   8 SNES Function norm 2.178613867932e+02
>> >     0 KSP preconditioned resid norm 2.791607776778e+02 true resid norm
>> 2.178613867932e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.269331257925e-07 true resid norm
>> 7.151934347788e-05 ||r(i)||/||b|| 3.282791160499e-07
>> >   9 SNES Function norm 1.541279494063e+02
>> >     0 KSP preconditioned resid norm 1.781925534106e+02 true resid norm
>> 1.541279494063e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.095285568305e-07 true resid norm
>> 4.530373014757e-05 ||r(i)||/||b|| 2.939358521416e-07
>> >  10 SNES Function norm 1.361917300390e+02
>> >     0 KSP preconditioned resid norm 4.801281559143e+01 true resid norm
>> 1.361917300390e+02 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.158004981151e-07 true resid norm
>> 1.216364616255e-05 ||r(i)||/||b|| 8.931266354477e-08
>> >  11 SNES Function norm 6.951420638296e+00
>> >     0 KSP preconditioned resid norm 2.808725633599e+00 true resid norm
>> 6.951420638296e+00 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 2.361764665651e-09 true resid norm
>> 7.088079744061e-07 ||r(i)||/||b|| 1.019659162188e-07
>> >  12 SNES Function norm 3.249486326694e-03
>> >     0 KSP preconditioned resid norm 1.388995922430e-03 true resid norm
>> 3.249486326694e-03 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 1.757348050151e-12 true resid norm
>> 3.473453149942e-10 ||r(i)||/||b|| 1.068923762322e-07
>> >  13 SNES Function norm 4.590368792536e-05
>> >     0 KSP preconditioned resid norm 4.360884570719e-06 true resid norm
>> 4.590368792536e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 3.926789585311e-15 true resid norm
>> 1.102338507065e-12 ||r(i)||/||b|| 2.401416001385e-08
>> >  14 SNES Function norm 4.422797479565e-05
>> >     0 KSP preconditioned resid norm 5.831925614907e-06 true resid norm
>> 4.422797479565e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 5.584144701532e-15 true resid norm
>> 1.478006791605e-12 ||r(i)||/||b|| 3.341791701821e-08
>> >  15 SNES Function norm 4.371345326958e-05
>> >     0 KSP preconditioned resid norm 4.897666437748e-06 true resid norm
>> 4.371345326958e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 4.080247182926e-15 true resid norm
>> 1.236587295819e-12 ||r(i)||/||b|| 2.828848336902e-08
>> >  16 SNES Function norm 4.348609553058e-05
>> >     0 KSP preconditioned resid norm 3.738450099362e-06 true resid norm
>> 4.348609553058e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 3.655406222015e-15 true resid norm
>> 9.403440338690e-13 ||r(i)||/||b|| 2.162401619174e-08
>> >  17 SNES Function norm 4.324907311342e-05
>> >     0 KSP preconditioned resid norm 5.296206632183e-06 true resid norm
>> 4.324907311342e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 5.018283198612e-15 true resid norm
>> 1.334862856428e-12 ||r(i)||/||b|| 3.086454252850e-08
>> >  18 SNES Function norm 4.324904992696e-05
>> >     0 KSP preconditioned resid norm 5.289306428405e-06 true resid norm
>> 4.324904992696e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 5.691985954854e-15 true resid norm
>> 1.335680822478e-12 ||r(i)||/||b|| 3.088347200075e-08
>> >  19 SNES Function norm 4.324904992565e-05
>> >     0 KSP preconditioned resid norm 5.289311834250e-06 true resid norm
>> 4.324904992565e-05 ||r(i)||/||b|| 1.000000000000e+00
>> >     1 KSP preconditioned resid norm 5.455286406170e-15 true resid norm
>> 1.337387995906e-12 ||r(i)||/||b|| 3.092294508677e-08
>> >
>> >
>> > At this point, it terminates with a diverged line search.
>> >
>> >
>> > -gideon
>> >
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> > -- Norbert Wiener
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150924/35c4fef2/attachment-0001.html>