<div dir="ltr">Let me get more debugging information first. code were implemented in a bunch of different files. simply copy-and-paste will make it very messy.</div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Sep 17, 2017 at 1:38 AM, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Send a code fragment that reproduces the problem; you must be calling on the wrong KSP or at the wrong place in the code<br>
<div class="HOEnZb"><div class="h5"><br>
> On Sep 17, 2017, at 12:33 AM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
><br>
> Call KSPView(). this is I have from the output<br>
><br>
> for my COARSE grid:<br>
><br>
> imin = 4 imax = 41<br>
> jmin = 4 jmax = 8<br>
> kmin = 4 kmax = 20<br>
> KSP Object: 24 MPI processes<br>
> type not yet set<br>
> maximum iterations=20, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using DEFAULT norm type for convergence test<br>
> PC Object: 24 MPI processes<br>
> type not yet set<br>
> PC has not been set up so information may be incomplete<br>
> KSP Object: 24 MPI processes<br>
> type not yet set<br>
> maximum iterations=20, initial guess is zero<br>
> tolerances: relative=1e-14, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using DEFAULT norm type for convergence test<br>
> PC Object: 24 MPI processes<br>
> type not yet set<br>
> PC has not been set up so information may be incomplete<br>
><br>
> Using Neumann Solver<br>
> 0 KSP Residual norm 4.933753594518e+00<br>
> 2 KSP Residual norm 1.028883218492e-03<br>
> 4 KSP Residual norm 2.980337307701e-06<br>
> 6 KSP Residual norm 2.976110274418e-06<br>
> 8 KSP Residual norm 2.976110265786e-06<br>
> 10 KSP Residual norm 2.976110265696e-06<br>
> 12 KSP Residual norm 2.976158755617e-06<br>
> 14 KSP Residual norm 2.976038363683e-06<br>
> 16 KSP Residual norm 2.976102606899e-06<br>
> 18 KSP Residual norm 2.976429345213e-06<br>
> Linear solve did not converge due to DIVERGED_ITS iterations 18<br>
><br>
> The solution diverges! The residual is 2.976429e-06. Solve again using GMRES!<br>
> 0 KSP Residual norm 4.932103470452e+00<br>
> 1 KSP Residual norm 4.586081825060e-01<br>
> 2 KSP Residual norm 4.702606826126e-02<br>
> 3 KSP Residual norm 5.717152414461e-03<br>
> 4 KSP Residual norm 5.122354290806e-04<br>
> 5 KSP Residual norm 4.682702445806e-05<br>
> 6 KSP Residual norm 4.368929895884e-06<br>
> 7 KSP Residual norm 4.908451624092e-07<br>
> 8 KSP Residual norm 4.758895647974e-08<br>
> 9 KSP Residual norm 3.831683374269e-09<br>
> 10 KSP Residual norm 2.844848230467e-10<br>
> 11 KSP Residual norm 2.252748969152e-11<br>
> 12 KSP Residual norm 1.432700892770e-12<br>
> 13 KSP Residual norm 9.569163228927e-14<br>
> 14 KSP Residual norm 1.129667714748e-14<br>
> Linear solve converged due to CONVERGED_RTOL iterations 14<br>
><br>
> In poisson_func(): num_iter = 14, rel_residual = 1.129668e-14<br>
><br>
> for my REFINE grid:<br>
><br>
> imin = 4 imax = 28<br>
> jmin = 4 jmax = 10<br>
> kmin = 4 kmax = 28<br>
> KSP Object: 96 MPI processes<br>
> type not yet set<br>
> maximum iterations=20, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using DEFAULT norm type for convergence test<br>
> PC Object: 96 MPI processes<br>
> type not yet set<br>
> PC has not been set up so information may be incomplete<br>
> KSP Object: 96 MPI processes<br>
> type not yet set<br>
> maximum iterations=20, initial guess is zero<br>
> tolerances: relative=1e-14, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using DEFAULT norm type for convergence test<br>
> PC Object: 96 MPI processes<br>
> type not yet set<br>
> PC has not been set up so information may be incomplete<br>
><br>
> On Sun, Sep 17, 2017 at 1:22 AM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
> Thanks!<br>
><br>
> On Sun, Sep 17, 2017 at 1:20 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
><br>
> > On Sep 17, 2017, at 12:17 AM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
> ><br>
> > OK. I will check the output Calling KSPView()<br>
> ><br>
> > I am experimenting with -start_in_debugger. it's for serial not for parallel, right?<br>
><br>
> It can be for parallel, opens an xterm with the debugger for each MPI process. You need to type cont in all the windows<br>
><br>
><br>
> ><br>
> > On Sun, Sep 17, 2017 at 1:12 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> ><br>
> > Should work in the code also. Call KSPSetTolerances() and then call KSPView() to see what it is set to<br>
> ><br>
> > > On Sep 17, 2017, at 12:11 AM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
> > ><br>
> > > Yes. but only works via command line option.before was set up in the code.<br>
> > ><br>
> > > On Sun, Sep 17, 2017 at 12:44 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> > ><br>
> > > > On Sep 16, 2017, at 11:33 PM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
> > > ><br>
> > > > thanks.<br>
> > > ><br>
> > > > Yes. KSPSetTolerances is used.<br>
> > > ><br>
> > > > tolerance is set to be 1.e-14<br>
> > ><br>
> > > There is the rtol and atol are you sure you set the atol to 1e-14?<br>
> > ><br>
> > > ><br>
> > > > I was thinking if there is a debug option I could use from PETSc to get some insight? GDB was used.<br>
> > ><br>
> > > You can use the command line option -start_in_debugger<br>
> > ><br>
> > ><br>
> > > ><br>
> > > > On Sun, Sep 17, 2017 at 12:25 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>> wrote:<br>
> > > ><br>
> > > > > On Sep 16, 2017, at 10:39 PM, Hao Zhang <<a href="mailto:hbcbh1999@gmail.com">hbcbh1999@gmail.com</a>> wrote:<br>
> > > > ><br>
> > > > > hi,<br>
> > > > ><br>
> > > > > I am using KSPBCGSL method and HYPRE boomeramg precondtioner passed into KSPsolve for poisson equation as part of incompressible NS.<br>
> > > > ><br>
> > > > > I have two questions<br>
> > > > > Q1 is number of iterations for the same solver. max_iter = 20 for the same poisson function<br>
> > > > ><br>
> > > > > KSPGetIterationNumber() is used to set up max_iter = 20<br>
> > > ><br>
> > > > What do you mean, KSPGetIterationNumber() tells you the current number of iterations. You cannot use it to set the maximum number of iterations. Use KSPSetTolerances() or -ksp_max_it <its> to set the number<br>
> > > > ><br>
> > > > ><br>
> > > > > 0 KSP Residual norm 4.512245447770e-04<br>
> > > > > 2 KSP Residual norm 6.396709069731e-07<br>
> > > > > 4 KSP Residual norm 1.757784220489e-10<br>
> > > > > 6 KSP Residual norm 1.501464709144e-14<br>
> > > > > 8 KSP Residual norm 1.376381429122e-18<br>
> > > > > 10 KSP Residual norm 1.072650548945e-19<br>
> > > > > In poisson_func(): num_iter = 10, rel_residual = 1.072651e-19<br>
> > > > ><br>
> > > > > and the very same function at the next time step gives: remind you guys max_iter = 20. I assume the number of iterations should stop at num_iter = 2.<br>
> > > ><br>
> > > > Why would it stop at num_iter 2. The number of iterations allowed is the same for each solve, it does not accumulate over all solves.<br>
> > > ><br>
> > > > > but the solver seems to keep iterating util hitting the max_iter.<br>
> > > > > 0 KSP Residual norm 5.107005838093e-16<br>
> > > > > 2 KSP Residual norm 3.353634821198e-20<br>
> > > > > 4 KSP Residual norm 1.835096039266e-23<br>
> > > > > 6 KSP Residual norm 1.645496102409e-23<br>
> > > > > 8 KSP Residual norm 1.645496099837e-23<br>
> > > > > 10 KSP Residual norm 1.645496099836e-23<br>
> > > > > 12 KSP Residual norm 1.645496099530e-23<br>
> > > > > 14 KSP Residual norm 1.645496034473e-23<br>
> > > > > 16 KSP Residual norm 1.645461130961e-23<br>
> > > > > 18 KSP Residual norm 1.645460451075e-23<br>
> > > > > In poisson_func(): num_iter = 18, rel_residual = 1.645460e-23<br>
> > > ><br>
> > > > But the way it is almost never realistic to try to get residual norms to be this small. You might consider using something like -ksp_atol 1.e-16 or something<br>
> > > ><br>
> > > > ><br>
> > > > > Q2 is: it seems HYPRE with KSPsolve works with coarse mesh but when I double the mesh size. the solve halted indefinitely. ksp_monitor didn't produce any results.<br>
> > > > > is there a way to understand what's happening?<br>
> > > ><br>
> > > > You could run in the debugger and use control c to interrupt the code when it is "hanging" and type bt to see the stack trace for the hanging.<br>
> > > ><br>
> > > ><br>
> > > > ><br>
> > > > > --<br>
> > > > > Hao Zhang<br>
> > > > > Dept. of Applid Mathematics and Statistics,<br>
> > > > > Stony Brook University,<br>
> > > > > Stony Brook, New York, 11790<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > --<br>
> > > > Hao Zhang<br>
> > > > Dept. of Applid Mathematics and Statistics,<br>
> > > > Stony Brook University,<br>
> > > > Stony Brook, New York, 11790<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > --<br>
> > > Hao Zhang<br>
> > > Dept. of Applid Mathematics and Statistics,<br>
> > > Stony Brook University,<br>
> > > Stony Brook, New York, 11790<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Hao Zhang<br>
> > Dept. of Applid Mathematics and Statistics,<br>
> > Stony Brook University,<br>
> > Stony Brook, New York, 11790<br>
><br>
><br>
><br>
><br>
> --<br>
> Hao Zhang<br>
> Dept. of Applid Mathematics and Statistics,<br>
> Stony Brook University,<br>
> Stony Brook, New York, 11790<br>
><br>
><br>
><br>
> --<br>
> Hao Zhang<br>
> Dept. of Applid Mathematics and Statistics,<br>
> Stony Brook University,<br>
> Stony Brook, New York, 11790<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Hao Zhang<br>Dept. of Applid Mathematics and Statistics,<br>Stony Brook University, <br>Stony Brook, New York, 11790</div>
</div>