[petsc-users] different convergence behaviour

[Barry Smith]

  It is not uncommon for an iterative linear solver to work fine for some time steps but then start to perform poorly at a later timestep because the physics (mathematically the conditioning or eigenstructure of the Jacobian) changes over time; perhaps becomes singular. Another possibility is the trajectory of the solution is very sensitive to the solution of the nonlinear problem at each time step so that an iterative linear solver and a direct linear solver result in very difficult physical solutions after many time steps. In other words after many time-steps the computed solutions can be very different and if the computed solution for the iterative linear solver is eventually "non-physical" or ill-conditioned the nonlinear solver could break down.

  Please run with the iterative solver (that eventually breaks) with the option -ksp_monitor_true_solution -ksp_converged_reason and and send ALL the output (it will be very large, don't worry about it). Then we can see if the linear solver is breaking down. Note that by default PETSc linear solvers do not generate an error that stops the program if the linear solve fails, hence your NR code should call KSPGetConvergedReason() after EVERY linear solve and if the reason is negative your code needs to do something different since the linear solve failed and your code should not just keep on running NR.

  Barry


> On Jul 12, 2016, at 9:52 AM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Tue, Jul 12, 2016 at 8:44 AM, Hoang Giang Bui <hgbk2008 at gmail.com> wrote:
> Hi Matt
> 
> 1) In the log you sent, the linear solver converges due to the Relative Tolerance, 1.0e-9, not the breakdown tolerance 1e-30. Change rtol will affect the convergence.
> 
> Sorry i got it wrong in the previous email, the ksp_rtol 1.0e-12 DOES affect the convergence, and it took more iterations. But the simulation still failed at a definite time step.
> 
> 2) What do you mean  -fieldsplit_wp_ksp_rtol 1.0e-8 does not work? ALWAYS send the view output.
> 
> In the log file I sent previously, the line
> 
>     KSP Object:    (fieldsplit_wp_)     8 MPI processes
>       type: preonly
>       maximum iterations=10000, initial guess is zero
>       tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
>       left preconditioning
>       using NONE norm type for convergence test
> 
> impressed me that the rtol for fieldsplit_wp is still 1.0e-5
> 
> KSP "preonly" does no iterations, so it does not read the tolerance. If you want to lower the tolerance,
> choose a solver like GMRES
> 
>   -fieldsplit_wp_ksp_type gmres -fieldsplit_wp_ksp_rtol 1e-8
>  
> 3) I can't tell you anything about Newton convergence if you do not send the output, -snes_monitor -snes_view
> 
> I did not yet use SNES, instead using my NR iterator so I have no view for SNES.
> 
> It is hard to debug an iteration which we did not code. It could be you have a bug. If not, then very small changes in
> the iterates are making a difference, which means your Jacobians are close to singular. A problem reformulation would
> probably help more than solver tweaking.
> 
>   Thanks,
> 
>     Matt
>  
> 4) If there is a difference between LU and an iterative solver with residual 1e-9, then your system is very ill-conditioned.
> Yes it is ill-conditioned
> 
> 
> 
> 
> 
> 
> 
> Giang
> 
> On Tue, Jul 12, 2016 at 2:49 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Tue, Jul 12, 2016 at 7:42 AM, Hoang Giang Bui <hgbk2008 at gmail.com> wrote:
> Hello
> 
> I encountered different convergence behaviour of Newton Raphson when using different solver settings with PETSc
> 
> For the first solver configuration, I used direct solver
> -ksp_type preonly
> -pc_type lu
> -pc_factor_mat_solver_package mumps
> -mat_mumps_icntl_1 6
> -mat_mumps_icntl_4 3
> -mat_mumps_icntl_7 4
> -mat_mumps_icntl_14 40
> -mat_mumps_icntl_23 0
> 
> The simulation can run completely and the NR typically converged after 6/7 iterations. Of course, it's very slow. For the second solver configuration:
> -ksp_type gmres
> -ksp_max_it 300
> -ksp_gmres_restart 300
> -ksp_gmres_modifiedgramschmidt
> -pc_view
> -pc_fieldsplit_type multiplicative
> -fieldsplit_u_pc_type hypre
> -fieldsplit_u_pc_hypre_type boomeramg
> -fieldsplit_u_pc_hypre_boomeramg_coarsen_type PMIS
> -fieldsplit_u_pc_hypre_boomeramg_strong_threshold 0.6
> -fieldsplit_u_pc_hypre_boomeramg_max_levels 25
> -fieldsplit_wp_ksp_rtol 1.0e-8
> -fieldsplit_wp_pc_type hypre
> -fieldsplit_wp_pc_hypre_type boomeramg
> -fieldsplit_wp_pc_hypre_boomeramg_coarsen_type PMIS
> -fieldsplit_wp_pc_hypre_boomeramg_strong_threshold 0.6
> -fieldsplit_wp_pc_hypre_boomeramg_max_levels 25
> 
> The solver runs much faster, but the NR does not converge in 30 iterations after some time steps. I thought setting the solver tolerance -ksp_rtol 1.0e-12 but it doesn't help much because GMRES already terminate with tolerance 1e-30 (see sample log file). Can we set the tolerance of the sub-ksp of the Fieldsplit? I tried -fieldsplit_wp_ksp_rtol 1.0e-8 but it doesn't work.
> 
> 1) In the log you sent, the linear solver converges due to the Relative Tolerance, 1.0e-9, not the breakdown tolerance 1e-30. Change rtol will affect the convergence.
> 
> 2) What do you mean  -fieldsplit_wp_ksp_rtol 1.0e-8 does not work? ALWAYS send the view output.
> 
> 3) I can't tell you anything about Newton convergence if you do not send the output, -snes_monitor -snes_view
> 
> 4) If there is a difference between LU and an iterative solver with residual 1e-9, then your system is very ill-conditioned.
> 
>   Thanks,
> 
>      Matt
>  
> Sorry this problem is run with many time steps and is quite big so I cannot reproduce in a simple test case.
> 
> Giang
> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener