[petsc-users] Preconditioning in SNES

Tue Nov 1 09:10:52 CDT 2011

Dear all,

  I use SNES to solve an implicit time marching formulation for the NS 
equations of gas dynamics using the discontinuous Galerkin 
discretization. I use a matrix-free formulation with a preconditioner 
coreesponding to the Jacobian of the system, but during the Newton 
solution just from the 1st step I get the following:

  0 KSP preconditioned resid norm 9.356767598746e+00 true resid norm 
9.356767598746e+00 ||Ae||/||Ax|| 1.000000000000e+00
   1 KSP preconditioned resid norm 4.281908138930e-15 true resid norm 
3.150842395998e-15 ||Ae||/||Ax|| 3.367447532223e-16
Linear solve converged due to CONVERGED_RTOL iterations 1

Timestep   0: dt = 0.002, T = 0, Res[rho] = 1.22465e-16, Res[rhou] = 
0.00215905, Res[rhov] = 1.61374e-08, Res[E] = 2.78002e-07, CFL = 399.998

/*********************Stage 1 of SSPIRK (6,4)******************/

   0 SNES Function norm 1.155072464398e-02
     0 KSP preconditioned resid norm 1.155072464398e-02 true resid norm 
1.155072464398e-02 ||Ae||/||Ax|| 1.000000000000e+00
     1 KSP preconditioned resid norm 1.155070298335e-02 true resid norm 
1.155070298335e-02 ||Ae||/||Ax|| 9.999981247380e-01
     2 KSP preconditioned resid norm 1.152933291306e-02 true resid norm 
1.152933291544e-02 ||Ae||/||Ax|| 9.981480184831e-01
     3 KSP preconditioned resid norm 9.177957248260e-03 true resid norm 
9.177957556426e-03 ||Ae||/||Ax|| 7.945785082156e-01
     4 KSP preconditioned resid norm 3.124174018085e-03 true resid norm 
3.124174393942e-03 ||Ae||/||Ax|| 2.704743200305e-01
     5 KSP preconditioned resid norm 6.978400843850e-04 true resid norm 
6.978392323251e-04 ||Ae||/||Ax|| 6.041519072039e-02
     6 KSP preconditioned resid norm 1.634324558019e-04 true resid norm 
1.634328735266e-04 ||Ae||/||Ax|| 1.414914462632e-02
     7 KSP preconditioned resid norm 9.422855713525e-05 true resid norm 
9.422715588996e-05 ||Ae||/||Ax|| 8.157683504215e-03
.
.
.
.
.
.
.
   1603 KSP preconditioned resid norm 1.191037802130e-07 true resid norm 
1.191038670563e-07 ||Ae||/||Ax|| 1.031137618870e-05
   1604 KSP preconditioned resid norm 1.178729142296e-07 true resid norm 
1.178730336572e-07 ||Ae||/||Ax|| 1.020481721193e-05
   1605 KSP preconditioned resid norm 1.169791191707e-07 true resid norm 
1.169793038395e-07 ||Ae||/||Ax|| 1.012744286138e-05
   1606 KSP preconditioned resid norm 1.168865445541e-07 true resid norm 
1.168870972685e-07 ||Ae||/||Ax|| 1.011946010931e-05
   1607 KSP preconditioned resid norm 1.168834953500e-07 true resid norm 
1.168834210021e-07 ||Ae||/||Ax|| 1.011914183782e-05
   1608 KSP preconditioned resid norm 1.168824468217e-07 true resid norm 
1.168821544127e-07 ||Ae||/||Ax|| 1.011903218328e-05
   1609 KSP preconditioned resid norm 1.168617721831e-07 true resid norm 
1.168618273848e-07 ||Ae||/||Ax|| 1.011727237786e-05
   1610 KSP preconditioned resid norm 1.165091490989e-07 true resid norm 
1.165091854357e-07 ||Ae||/||Ax|| 1.008674252281e-05
   1611 KSP preconditioned resid norm 1.153333136618e-07 true resid norm 
1.153329197440e-07 ||Ae||/||Ax|| 9.984907726463e-06
Linear solve converged due to CONVERGED_RTOL iterations 1611

  1 SNES Function norm 1.156248777455e-07
     0 KSP preconditioned resid norm 1.156248777455e-07 true resid norm 
1.156248777455e-07 ||Ae||/||Ax|| 1.000000000000e+00
     1 KSP preconditioned resid norm 1.154422371390e-07 true resid norm 
1.161710852713e-07 ||Ae||/||Ax|| 1.004723961974e+00
     2 KSP preconditioned resid norm 1.152905844799e-07 true resid norm 
2.668696384811e-04 ||Ae||/||Ax|| 2.308064178614e+03
     3 KSP preconditioned resid norm 1.151761139639e-07 true resid norm 
1.126358239931e-03 ||Ae||/||Ax|| 9.741486969699e+03
     4 KSP preconditioned resid norm 1.151754181824e-07 true resid norm 
1.108271364124e-03 ||Ae||/||Ax|| 9.585059770296e+03
     5 KSP preconditioned resid norm 1.151747447109e-07 true resid norm 
1.133976467714e-03 ||Ae||/||Ax|| 9.807374414786e+03
     6 KSP preconditioned resid norm 1.151536921969e-07 true resid norm 
1.682604476358e-03 ||Ae||/||Ax|| 1.455227031730e+04
     7 KSP preconditioned resid norm 1.151536741521e-07 true resid norm 
1.675362372285e-03 ||Ae||/||Ax|| 1.448963583747e+04
     8 KSP preconditioned resid norm 1.151516417557e-07 true resid norm 
1.739170524328e-03 ||Ae||/||Ax|| 1.504149070891e+04
     9 KSP preconditioned resid norm 1.151458368640e-07 true resid norm 
1.888742801931e-03 ||Ae||/||Ax|| 1.633509015325e+04
    10 KSP preconditioned resid norm 1.151432970406e-07 true resid norm 
1.936535093080e-03 ||Ae||/||Ax|| 1.674842932455e+04
    11 KSP preconditioned resid norm 1.151415856844e-07 true resid norm 
1.943578776829e-03 ||Ae||/||Ax|| 1.680934773489e+04
    12 KSP preconditioned resid norm 1.151415106457e-07 true resid norm 
1.941675149163e-03 ||Ae||/||Ax|| 1.679288391064e+04
    13 KSP preconditioned resid norm 1.151072722808e-07 true resid norm 
2.055181937477e-03 ||Ae||/||Ax|| 1.777456527997e+04
    14 KSP preconditioned resid norm 1.150661328335e-07 true resid norm 
2.247389279980e-03 ||Ae||/||Ax|| 1.943690081062e+04
    15 KSP preconditioned resid norm 1.150587945589e-07 true resid norm 
2.349201250473e-03 ||Ae||/||Ax|| 2.031743770266e+04
    16 KSP preconditioned resid norm 1.150293904255e-07 true resid norm 
2.708861286410e-03 ||Ae||/||Ax|| 2.342801427537e+04
    17 KSP preconditioned resid norm 1.149883485190e-07 true resid norm 
3.425777483190e-03 ||Ae||/||Ax|| 2.962837712771e+04
    18 KSP preconditioned resid norm 1.149835570884e-07 true resid norm 
3.492000995212e-03 ||Ae||/||Ax|| 3.020112162105e+04
    19 KSP preconditioned resid norm 1.149457801834e-07 true resid norm 
4.258473277795e-03 ||Ae||/||Ax|| 3.683007810108e+04
    20 KSP preconditioned resid norm 1.149279955651e-07 true resid norm 
4.574312834982e-03 ||Ae||/||Ax|| 3.956166634874e+04
    21 KSP preconditioned resid norm 1.148902843639e-07 true resid norm 
5.376279701577e-03 ||Ae||/||Ax|| 4.649760333939e+04
    22 KSP preconditioned resid norm 1.148568926668e-07 true resid norm 
6.095225432786e-03 ||Ae||/||Ax|| 5.271551894049e+04
    23 KSP preconditioned resid norm 1.148172845230e-07 true resid norm 
6.957862632489e-03 ||Ae||/||Ax|| 6.017617288039e+04
    24 KSP preconditioned resid norm 1.147830369881e-07 true resid norm 
7.726330820912e-03 ||Ae||/||Ax|| 6.682239126701e+04
    25 KSP preconditioned resid norm 1.147438237790e-07 true resid norm 
8.597270798069e-03 ||Ae||/||Ax|| 7.435485308784e+04
    26 KSP preconditioned resid norm 1.147090792721e-07 true resid norm 
9.388950565795e-03 ||Ae||/||Ax|| 8.120182048070e+04
    27 KSP preconditioned resid norm 1.146703081864e-07 true resid norm 
1.025969917305e-02 ||Ae||/||Ax|| 8.873262720872e+04
    28 KSP preconditioned resid norm 1.146352282540e-07 true resid norm 
1.106505484202e-02 ||Ae||/||Ax|| 9.569787279149e+04
    29 KSP preconditioned resid norm 1.145968062554e-07 true resid norm 
1.193384225768e-02 ||Ae||/||Ax|| 1.032117178445e+05
    30 KSP preconditioned resid norm 1.274732710949e-02 true resid norm 
1.274732710949e-02 ||Ae||/||Ax|| 1.102472699478e+05
Linear solve did not converge due to DIVERGED_DTOL iterations 30

As you may see on the 2nd Newton iteration the linear system solution 
does not converge. I use the following arguments to start the computation:

mpiexec -n 8 ./hoac blasius -llf_flux -n_out 1 -end_time 10000.0 
-implicit -implicit_type 6 -pc_type asm -sub_pc_type ilu 
-snes_mf_operator -snes_max_fail 500 -snes_monitor -snes_stol 1.0e-50 
-ksp_right_pc -snes_converged_reason -ksp_gmres_restart 30 
-snes_max_linear_solve_fail 500 -sub_pc_factor_levels 2 -snes_max_it 
1000 -sub_pc_factor_mat_ordering_type rcm -dt 2.e-3 -snes_rtol 1.0e-8 
-gl -snes_converged_reason -ksp_converged_reason -ksp_monitor_true_residual

Why is this happening? What might I be doing wrong? Any suggestions or 
guide lines?

Thank you,

Kostas