[petsc-users] Preconditioning in SNES

Tue Nov 1 10:21:10 CDT 2011

On Tue, Nov 1, 2011 at 2:10 PM, Konstantinos Kontzialis <
ckontzialis at lycos.com> wrote:

> Dear all,
>
>  I use SNES to solve an implicit time marching formulation for the NS
> equations of gas dynamics using the discontinuous Galerkin discretization.
> I use a matrix-free formulation with a preconditioner coreesponding to the
> Jacobian of the system, but during the Newton solution just from the 1st
> step I get the following:
>
>  0 KSP preconditioned resid norm 9.356767598746e+00 true resid norm
> 9.356767598746e+00 ||Ae||/||Ax|| 1.000000000000e+00
>  1 KSP preconditioned resid norm 4.281908138930e-15 true resid norm
> 3.150842395998e-15 ||Ae||/||Ax|| 3.367447532223e-16
> Linear solve converged due to CONVERGED_RTOL iterations 1
>
>
> Timestep   0: dt = 0.002, T = 0, Res[rho] = 1.22465e-16, Res[rhou] =
> 0.00215905, Res[rhov] = 1.61374e-08, Res[E] = 2.78002e-07, CFL = 399.998
>
> /*********************Stage 1 of SSPIRK (6,4)******************/
>
>  0 SNES Function norm 1.155072464398e-02
>    0 KSP preconditioned resid norm 1.155072464398e-02 true resid norm
> 1.155072464398e-02 ||Ae||/||Ax|| 1.000000000000e+00
>    1 KSP preconditioned resid norm 1.155070298335e-02 true resid norm
> 1.155070298335e-02 ||Ae||/||Ax|| 9.999981247380e-01
>    2 KSP preconditioned resid norm 1.152933291306e-02 true resid norm
> 1.152933291544e-02 ||Ae||/||Ax|| 9.981480184831e-01
>    3 KSP preconditioned resid norm 9.177957248260e-03 true resid norm
> 9.177957556426e-03 ||Ae||/||Ax|| 7.945785082156e-01
>    4 KSP preconditioned resid norm 3.124174018085e-03 true resid norm
> 3.124174393942e-03 ||Ae||/||Ax|| 2.704743200305e-01
>    5 KSP preconditioned resid norm 6.978400843850e-04 true resid norm
> 6.978392323251e-04 ||Ae||/||Ax|| 6.041519072039e-02
>    6 KSP preconditioned resid norm 1.634324558019e-04 true resid norm
> 1.634328735266e-04 ||Ae||/||Ax|| 1.414914462632e-02
>    7 KSP preconditioned resid norm 9.422855713525e-05 true resid norm
> 9.422715588996e-05 ||Ae||/||Ax|| 8.157683504215e-03
> .
> .
> .
> .
> .
> .
> .
>  1603 KSP preconditioned resid norm 1.191037802130e-07 true resid norm
> 1.191038670563e-07 ||Ae||/||Ax|| 1.031137618870e-05
>  1604 KSP preconditioned resid norm 1.178729142296e-07 true resid norm
> 1.178730336572e-07 ||Ae||/||Ax|| 1.020481721193e-05
>  1605 KSP preconditioned resid norm 1.169791191707e-07 true resid norm
> 1.169793038395e-07 ||Ae||/||Ax|| 1.012744286138e-05
>  1606 KSP preconditioned resid norm 1.168865445541e-07 true resid norm
> 1.168870972685e-07 ||Ae||/||Ax|| 1.011946010931e-05
>  1607 KSP preconditioned resid norm 1.168834953500e-07 true resid norm
> 1.168834210021e-07 ||Ae||/||Ax|| 1.011914183782e-05
>  1608 KSP preconditioned resid norm 1.168824468217e-07 true resid norm
> 1.168821544127e-07 ||Ae||/||Ax|| 1.011903218328e-05
>  1609 KSP preconditioned resid norm 1.168617721831e-07 true resid norm
> 1.168618273848e-07 ||Ae||/||Ax|| 1.011727237786e-05
>  1610 KSP preconditioned resid norm 1.165091490989e-07 true resid norm
> 1.165091854357e-07 ||Ae||/||Ax|| 1.008674252281e-05
>  1611 KSP preconditioned resid norm 1.153333136618e-07 true resid norm
> 1.153329197440e-07 ||Ae||/||Ax|| 9.984907726463e-06
> Linear solve converged due to CONVERGED_RTOL iterations 1611
>
>  1 SNES Function norm 1.156248777455e-07
>    0 KSP preconditioned resid norm 1.156248777455e-07 true resid norm
> 1.156248777455e-07 ||Ae||/||Ax|| 1.000000000000e+00
>    1 KSP preconditioned resid norm 1.154422371390e-07 true resid norm
> 1.161710852713e-07 ||Ae||/||Ax|| 1.004723961974e+00
>    2 KSP preconditioned resid norm 1.152905844799e-07 true resid norm
> 2.668696384811e-04 ||Ae||/||Ax|| 2.308064178614e+03
>    3 KSP preconditioned resid norm 1.151761139639e-07 true resid norm
> 1.126358239931e-03 ||Ae||/||Ax|| 9.741486969699e+03
>    4 KSP preconditioned resid norm 1.151754181824e-07 true resid norm
> 1.108271364124e-03 ||Ae||/||Ax|| 9.585059770296e+03
>    5 KSP preconditioned resid norm 1.151747447109e-07 true resid norm
> 1.133976467714e-03 ||Ae||/||Ax|| 9.807374414786e+03
>    6 KSP preconditioned resid norm 1.151536921969e-07 true resid norm
> 1.682604476358e-03 ||Ae||/||Ax|| 1.455227031730e+04
>    7 KSP preconditioned resid norm 1.151536741521e-07 true resid norm
> 1.675362372285e-03 ||Ae||/||Ax|| 1.448963583747e+04
>    8 KSP preconditioned resid norm 1.151516417557e-07 true resid norm
> 1.739170524328e-03 ||Ae||/||Ax|| 1.504149070891e+04
>    9 KSP preconditioned resid norm 1.151458368640e-07 true resid norm
> 1.888742801931e-03 ||Ae||/||Ax|| 1.633509015325e+04
>   10 KSP preconditioned resid norm 1.151432970406e-07 true resid norm
> 1.936535093080e-03 ||Ae||/||Ax|| 1.674842932455e+04
>   11 KSP preconditioned resid norm 1.151415856844e-07 true resid norm
> 1.943578776829e-03 ||Ae||/||Ax|| 1.680934773489e+04
>   12 KSP preconditioned resid norm 1.151415106457e-07 true resid norm
> 1.941675149163e-03 ||Ae||/||Ax|| 1.679288391064e+04
>   13 KSP preconditioned resid norm 1.151072722808e-07 true resid norm
> 2.055181937477e-03 ||Ae||/||Ax|| 1.777456527997e+04
>   14 KSP preconditioned resid norm 1.150661328335e-07 true resid norm
> 2.247389279980e-03 ||Ae||/||Ax|| 1.943690081062e+04
>   15 KSP preconditioned resid norm 1.150587945589e-07 true resid norm
> 2.349201250473e-03 ||Ae||/||Ax|| 2.031743770266e+04
>   16 KSP preconditioned resid norm 1.150293904255e-07 true resid norm
> 2.708861286410e-03 ||Ae||/||Ax|| 2.342801427537e+04
>   17 KSP preconditioned resid norm 1.149883485190e-07 true resid norm
> 3.425777483190e-03 ||Ae||/||Ax|| 2.962837712771e+04
>   18 KSP preconditioned resid norm 1.149835570884e-07 true resid norm
> 3.492000995212e-03 ||Ae||/||Ax|| 3.020112162105e+04
>   19 KSP preconditioned resid norm 1.149457801834e-07 true resid norm
> 4.258473277795e-03 ||Ae||/||Ax|| 3.683007810108e+04
>   20 KSP preconditioned resid norm 1.149279955651e-07 true resid norm
> 4.574312834982e-03 ||Ae||/||Ax|| 3.956166634874e+04
>   21 KSP preconditioned resid norm 1.148902843639e-07 true resid norm
> 5.376279701577e-03 ||Ae||/||Ax|| 4.649760333939e+04
>   22 KSP preconditioned resid norm 1.148568926668e-07 true resid norm
> 6.095225432786e-03 ||Ae||/||Ax|| 5.271551894049e+04
>   23 KSP preconditioned resid norm 1.148172845230e-07 true resid norm
> 6.957862632489e-03 ||Ae||/||Ax|| 6.017617288039e+04
>   24 KSP preconditioned resid norm 1.147830369881e-07 true resid norm
> 7.726330820912e-03 ||Ae||/||Ax|| 6.682239126701e+04
>   25 KSP preconditioned resid norm 1.147438237790e-07 true resid norm
> 8.597270798069e-03 ||Ae||/||Ax|| 7.435485308784e+04
>   26 KSP preconditioned resid norm 1.147090792721e-07 true resid norm
> 9.388950565795e-03 ||Ae||/||Ax|| 8.120182048070e+04
>   27 KSP preconditioned resid norm 1.146703081864e-07 true resid norm
> 1.025969917305e-02 ||Ae||/||Ax|| 8.873262720872e+04
>   28 KSP preconditioned resid norm 1.146352282540e-07 true resid norm
> 1.106505484202e-02 ||Ae||/||Ax|| 9.569787279149e+04
>   29 KSP preconditioned resid norm 1.145968062554e-07 true resid norm
> 1.193384225768e-02 ||Ae||/||Ax|| 1.032117178445e+05
>   30 KSP preconditioned resid norm 1.274732710949e-02 true resid norm
> 1.274732710949e-02 ||Ae||/||Ax|| 1.102472699478e+05
> Linear solve did not converge due to DIVERGED_DTOL iterations 30
>
> As you may see on the 2nd Newton iteration the linear system solution does
> not converge. I use the following arguments to start the computation:
>
> mpiexec -n 8 ./hoac blasius -llf_flux -n_out 1 -end_time 10000.0 -implicit
> -implicit_type 6 -pc_type asm -sub_pc_type ilu -snes_mf_operator
> -snes_max_fail 500 -snes_monitor -snes_stol 1.0e-50 -ksp_right_pc
> -snes_converged_reason -ksp_gmres_restart 30 -snes_max_linear_solve_fail
> 500 -sub_pc_factor_levels 2 -snes_max_it 1000 -sub_pc_factor_mat_ordering_
> **type rcm -dt 2.e-3 -snes_rtol 1.0e-8 -gl -snes_converged_reason
> -ksp_converged_reason -ksp_monitor_true_residual
>
>
> Why is this happening? What might I be doing wrong? Any suggestions or
> guide lines?
>

It could be that:

  a) Your system is really ill-conditioned, and the PC is not that good

  b) Your system has a null space, and the initial guess had no component
in it

  c) Your Jacobian is not actually the Jacobian of the system, so the PC is
not good

  d) The FD approximation to the Jacobian action is not good

When you have so many different possible problems, the right thing to do is
simplify the system until you can narrow down the cause

  a) Run in serial

  b) Shrink the problem

  c) Fully form the Jacobian using -snes_fd

  d) Run with -pc_type lu

     Matt

> Thank you,
>
> Kostas
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20111101/12b4e430/attachment.htm>