<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 1, 2017 at 1:46 PM, Jed Brown <span dir="ltr"><<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-">Mark Adams <<a href="mailto:mfadams@lbl.gov">mfadams@lbl.gov</a>> writes:<br>
<br>
> I have added some code in a TS post step method to look at the number of<br>
> nonlinear iterations and cut the time step if it took too many SNES<br>
> iterations. That helped but now I want to go one step further and recover<br>
> from a failed time step (see appended error message).<br>
><br>
> How can/should I recover from a SNES failure inside of TS? I could just put<br>
> TSSolve in a loop and if TS failed because of the solver, then reduce the<br>
> time step and try to recover the state, and try again.<br>
><br>
> Thoughts?<br>
><br>
> Thanks,<br>
> Mark<br>
><br>
> Note, this is a direct solver and the linear solver is diverging? I've<br>
> seen the same thing with the maximum its in SNES and failed linesearch.<br>
><br>
> 23 TS dt 0.0025 time 0.1025<br>
> 0 SNES Function norm 9.997473571601e+04<br>
> 1 SNES Function norm 2.295851692533e-01<br>
> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1<br>
> [0]PETSC ERROR: --------------------- Error Message<br>
> ------------------------------<wbr>------------------------------<wbr>--<br>
> [0]PETSC ERROR:<br>
> [0]PETSC ERROR: TSStep has failed due to DIVERGED_NONLINEAR_SOLVE, increase<br>
> -ts_max_snes_failures or make negative to attempt recovery<br>
<br>
</span>Did you increase -ts_max_snes_failures (TSSetMaxSNESFailures)?<br></blockquote><div><br></div><div>Ah, read the error message :) with a negative value I get this.</div><div><br></div><div>TS can not, or is not going to try, to reconstruct my state and reduce the time step. So I think I need to do this myself.</div><div><br></div><div>I guess just making loop and testing in the TSConvergedReason, is the way to go. We already do something like this for mesh adpativity.</div><div><br>Thanks,</div><div><br></div><div><div>23 TS dt 0.0025 time 0.1025</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> 1 SNES Function norm 2.295851692533e-01 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 1</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div> 0 SNES Function norm 9.997473571601e+04 </div><div> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0</div><div>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div>[0]PETSC ERROR: </div><div>[0]PETSC ERROR: TSStep has failed due to DIVERGED_STEP_REJECTED</div><div>[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</div><div>[0]PETSC ERROR: Petsc Development GIT revision: v3.8-53-ga2d93b2d43 GIT Date: 2017-10-09 12:56:49 -0400</div><div>[0]PETSC ERROR: ./ex48 on a arch-macosx-gnu-O named MarksMac-5.local by markadams Wed Nov 1 13:51:24 2017</div><div>[0]PETSC ERROR: Configure options --with-cc=clang --with-cc++=clang++ COPTFLAGS="-O2 -g -mavx2" CXXOPTFLAGS="-O2 -g -mavx2" FOPTFLAGS="-O2 -g -mavx2" --download-mpich=1 --download-parmetis=1 --download-metis=1 --download-hypre=1 --download-ml=1 --download-triangle=1 --download-ctetgen=1 --download-p4est=1 --download-fftw --download-mumps=1 --download-scalapack=1 --with-x=0 --download-superlu_dist --download-superlu --download-ctetgen --with-debugging=0 --download-hdf5=1 PETSC_ARCH=arch-macosx-gnu-O --download-chaco --with-viewfromoptions=1</div><div>[0]PETSC ERROR: #1 TSStep() line 4130 in /Users/markadams/Codes/petsc/src/ts/interface/ts.c</div><div>[0]PETSC ERROR: #2 TSSolve() line 4374 in /Users/markadams/Codes/petsc/src/ts/interface/ts.c</div><div>*********** DIVERGED_STEP_REJECTED ierr=91 reason=-2 </div><div>L_2 Diff: 1.32241e+07 at time 0.1025</div></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="gmail-HOEnZb"><div class="gmail-h5"><br>
> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for<br>
> trouble shooting.<br>
> [0]PETSC ERROR: Petsc Development GIT revision: v3.8-53-ga2d93b2d43 GIT<br>
> Date: 2017-10-09 12:56:49 -0400<br>
> [0]PETSC ERROR: ./ex48 on a arch-macosx-gnu-O named MarksMac-5.local by<br>
> markadams Wed Nov 1 13:17:15 2017<br>
> [0]PETSC ERROR: Configure options --with-cc=clang --with-cc++=clang++<br>
> COPTFLAGS="-O2 -g -mavx2" CXXOPTFLAGS="-O2 -g -mavx2" FOPTFLAGS="-O2 -g<br>
> -mavx2" --download-mpich=1 --download-parmetis=1 --download-metis=1<br>
> --download-hypre=1 --download-ml=1 --download-triangle=1<br>
> --download-ctetgen=1 --download-p4est=1 --download-fftw --download-mumps=1<br>
> --download-scalapack=1 --with-x=0 --download-superlu_dist<br>
> --download-superlu --download-ctetgen --with-debugging=0 --download-hdf5=1<br>
> PETSC_ARCH=arch-macosx-gnu-O --download-chaco --with-viewfromoptions=1<br>
> [0]PETSC ERROR: #1 TSStep() line 4129 in<br>
> /Users/markadams/Codes/petsc/<wbr>src/ts/interface/ts.c<br>
> [0]PETSC ERROR: #2 TSSolve() line 4374 in<br>
> /Users/markadams/Codes/petsc/<wbr>src/ts/interface/ts.c<br>
> [0]PETSC ERROR: #3 main() line 1161 in /Users/markadams/Codes/mhd/<wbr>src/ex48.c<br>
</div></div></blockquote></div><br></div></div>