<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Sep 24, 2015 at 5:44 PM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
I completely disagree with Matt, looking at the numbers one sees very typical Newton behavior<br>
<br>
a little slow at the beginning because it is not in the quadratic basin<br>
<br>
0 SNES Function norm 5.979113140046e+02<br>
1 SNES Function norm 5.106334303399e+02<br>
2 SNES Function norm 4.522093337153e+02<br>
3 SNES Function norm 4.042911112972e+02<br>
4 SNES Function norm 3.618952927134e+02<br>
5 SNES Function norm 3.239307814846e+02<br>
6 SNES Function norm 2.899625386925e+02<br>
7 SNES Function norm 2.596235386610e+02<br>
8 SNES Function norm 2.178613867932e+02<br>
9 SNES Function norm 1.541279494063e+02<br>
10 SNES Function norm 1.361917300390e+02<br>
<br>
suddenly fast convergence as it gets in the quadratic basin and behaves like Newton should behave<br>
<br>
11 SNES Function norm 6.951420638296e+00<br>
12 SNES Function norm 3.249486326694e-03<br>
13 SNES Function norm 4.590368792536e-05<br></blockquote><div><br></div><div>Not exactly quadratic, but definitely faster than beforeIt still looks about linear with c = 0.01. However,</div><div>Barry might be right since this rapid convergence arrests suddenly.</div><div><br></div><div>Can you think of a reason why your function evaluation would only be single precision?</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
no more convergence as it "runs out of digits"<br>
<br>
14 SNES Function norm 4.422797479565e-05<br>
15 SNES Function norm 4.371345326958e-05<br>
16 SNES Function norm 4.348609553058e-05<br>
17 SNES Function norm 4.324907311342e-05<br>
18 SNES Function norm 4.324904992696e-05<br>
19 SNES Function norm 4.324904992565e-05<br>
<br>
The only difference between this run and most other runs of Newton (when it works in the standard way) is that it "runs out of digits" early (after only reducing the residual norm by 1e-7 when for super nice problems it can do like 1e-12).<br>
<br>
I don't think your Jacobian is crap, I don't think the Jacobian direction is crap and the Jacobian is not not terribly ill-conditioned since the direct solver reduces the residual norm by 1e-8.<br>
<br>
My recommendation when Newton "runs out of digits" is to try using quad precision. With PETSc you just need a recent version of the GNU compilers and the configure options --with-precision=__float128 --download-f2cblaslapack plus you need to make sure your code always uses PetscReal and PetscScalar instead of "double" or whatever and whenever you call elementary functions like sin() you call PetscSinReal() or PetscSinScalar() etc so that it uses the quad precision version (if you miss one of these you will never see the quad precision).<br>
<br>
Barry<br>
<br>
What does it mean for Newton to "run out of digits"; this is not a technical term but I believe it is related to how accurate your function evaluations can be (I think some people call this noise) when expressed using floating point numbers. Perhaps it is something like "ill-conditioned functions". If the function evaluations are only valid for 8 digits then it is not reasonable to expect Newton (or any floating point algorithm) to squeeze the norm of the function values to 1e.-12.<br>
<br>
<br>
<br>
<br>
<br>
> On Sep 24, 2015, at 5:28 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:<br>
><br>
> On Thu, Sep 24, 2015 at 5:23 PM, Gideon Simpson <<a href="mailto:gideon.simpson@gmail.com">gideon.simpson@gmail.com</a>> wrote:<br>
> Just wondering if anyone has any suggestions for this scenario:<br>
><br>
> I have seen this happen a bunch. Your Newton directions are crap. That means<br>
><br>
> a) you have a crap Jacobian<br>
><br>
> b) your Jacobian direction is crap (unlikely here)<br>
><br>
> c) your Jacobian is very ill-conditioned<br>
><br>
> Something to try in order to see what is happening is to start doing Picard when Newton stalls. If it<br>
> converges, then you have something like c) I think. We can try to mitigate this by NPC. Something<br>
> as simple as Newton + NCG or Newton + NRichardson can work.<br>
><br>
> Matt<br>
><br>
> 0 SNES Function norm 5.979113140046e+02<br>
> 0 KSP preconditioned resid norm 1.609660749629e+03 true resid norm 5.979113140046e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.083640787055e-06 true resid norm 4.070012556358e-04 ||r(i)||/||b|| 6.807050579289e-07<br>
> 1 SNES Function norm 5.106334303399e+02<br>
> 0 KSP preconditioned resid norm 1.304622870569e+03 true resid norm 5.106334303399e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.542022975926e-06 true resid norm 3.284828506619e-04 ||r(i)||/||b|| 6.432850478341e-07<br>
> 2 SNES Function norm 4.522093337153e+02<br>
> 0 KSP preconditioned resid norm 1.017788935768e+03 true resid norm 4.522093337153e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.338757544659e-06 true resid norm 2.556045980853e-04 ||r(i)||/||b|| 5.652351223829e-07<br>
> 3 SNES Function norm 4.042911112972e+02<br>
> 0 KSP preconditioned resid norm 7.996536035697e+02 true resid norm 4.042911112972e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 8.612178921208e-07 true resid norm 2.026756625099e-04 ||r(i)||/||b|| 5.013112008810e-07<br>
> 4 SNES Function norm 3.618952927134e+02<br>
> 0 KSP preconditioned resid norm 6.392806337991e+02 true resid norm 3.618952927134e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 5.099112497936e-07 true resid norm 1.605851784448e-04 ||r(i)||/||b|| 4.437338138353e-07<br>
> 5 SNES Function norm 3.239307814846e+02<br>
> 0 KSP preconditioned resid norm 5.205889419531e+02 true resid norm 3.239307814846e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 3.364948858968e-07 true resid norm 1.326652268287e-04 ||r(i)||/||b|| 4.095480714142e-07<br>
> 6 SNES Function norm 2.899625386925e+02<br>
> 0 KSP preconditioned resid norm 4.305631057215e+02 true resid norm 2.899625386925e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 2.472775427751e-07 true resid norm 1.094956512355e-04 ||r(i)||/||b|| 3.776199909453e-07<br>
> 7 SNES Function norm 2.596235386610e+02<br>
> 0 KSP preconditioned resid norm 3.606693697722e+02 true resid norm 2.596235386610e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.804499181789e-07 true resid norm 9.162646660567e-05 ||r(i)||/||b|| 3.529204904849e-07<br>
> 8 SNES Function norm 2.178613867932e+02<br>
> 0 KSP preconditioned resid norm 2.791607776778e+02 true resid norm 2.178613867932e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.269331257925e-07 true resid norm 7.151934347788e-05 ||r(i)||/||b|| 3.282791160499e-07<br>
> 9 SNES Function norm 1.541279494063e+02<br>
> 0 KSP preconditioned resid norm 1.781925534106e+02 true resid norm 1.541279494063e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.095285568305e-07 true resid norm 4.530373014757e-05 ||r(i)||/||b|| 2.939358521416e-07<br>
> 10 SNES Function norm 1.361917300390e+02<br>
> 0 KSP preconditioned resid norm 4.801281559143e+01 true resid norm 1.361917300390e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.158004981151e-07 true resid norm 1.216364616255e-05 ||r(i)||/||b|| 8.931266354477e-08<br>
> 11 SNES Function norm 6.951420638296e+00<br>
> 0 KSP preconditioned resid norm 2.808725633599e+00 true resid norm 6.951420638296e+00 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 2.361764665651e-09 true resid norm 7.088079744061e-07 ||r(i)||/||b|| 1.019659162188e-07<br>
> 12 SNES Function norm 3.249486326694e-03<br>
> 0 KSP preconditioned resid norm 1.388995922430e-03 true resid norm 3.249486326694e-03 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 1.757348050151e-12 true resid norm 3.473453149942e-10 ||r(i)||/||b|| 1.068923762322e-07<br>
> 13 SNES Function norm 4.590368792536e-05<br>
> 0 KSP preconditioned resid norm 4.360884570719e-06 true resid norm 4.590368792536e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 3.926789585311e-15 true resid norm 1.102338507065e-12 ||r(i)||/||b|| 2.401416001385e-08<br>
> 14 SNES Function norm 4.422797479565e-05<br>
> 0 KSP preconditioned resid norm 5.831925614907e-06 true resid norm 4.422797479565e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 5.584144701532e-15 true resid norm 1.478006791605e-12 ||r(i)||/||b|| 3.341791701821e-08<br>
> 15 SNES Function norm 4.371345326958e-05<br>
> 0 KSP preconditioned resid norm 4.897666437748e-06 true resid norm 4.371345326958e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 4.080247182926e-15 true resid norm 1.236587295819e-12 ||r(i)||/||b|| 2.828848336902e-08<br>
> 16 SNES Function norm 4.348609553058e-05<br>
> 0 KSP preconditioned resid norm 3.738450099362e-06 true resid norm 4.348609553058e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 3.655406222015e-15 true resid norm 9.403440338690e-13 ||r(i)||/||b|| 2.162401619174e-08<br>
> 17 SNES Function norm 4.324907311342e-05<br>
> 0 KSP preconditioned resid norm 5.296206632183e-06 true resid norm 4.324907311342e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 5.018283198612e-15 true resid norm 1.334862856428e-12 ||r(i)||/||b|| 3.086454252850e-08<br>
> 18 SNES Function norm 4.324904992696e-05<br>
> 0 KSP preconditioned resid norm 5.289306428405e-06 true resid norm 4.324904992696e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 5.691985954854e-15 true resid norm 1.335680822478e-12 ||r(i)||/||b|| 3.088347200075e-08<br>
> 19 SNES Function norm 4.324904992565e-05<br>
> 0 KSP preconditioned resid norm 5.289311834250e-06 true resid norm 4.324904992565e-05 ||r(i)||/||b|| 1.000000000000e+00<br>
> 1 KSP preconditioned resid norm 5.455286406170e-15 true resid norm 1.337387995906e-12 ||r(i)||/||b|| 3.092294508677e-08<br>
><br>
><br>
> At this point, it terminates with a diverged line search.<br>
><br>
><br>
> -gideon<br>
><br>
><br>
><br>
<span class="HOEnZb"><font color="#888888">><br>
> --<br>
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
> -- Norbert Wiener<br>
<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>