On Tue, Nov 1, 2011 at 2:10 PM, Konstantinos Kontzialis <span dir="ltr"><<a href="mailto:ckontzialis@lycos.com">ckontzialis@lycos.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Dear all,<br>
<br>
I use SNES to solve an implicit time marching formulation for the NS equations of gas dynamics using the discontinuous Galerkin discretization. I use a matrix-free formulation with a preconditioner coreesponding to the Jacobian of the system, but during the Newton solution just from the 1st step I get the following:<br>
<br>
0 KSP preconditioned resid norm 9.356767598746e+00 true resid norm 9.356767598746e+00 ||Ae||/||Ax|| 1.000000000000e+00<br>
1 KSP preconditioned resid norm 4.281908138930e-15 true resid norm 3.150842395998e-15 ||Ae||/||Ax|| 3.367447532223e-16<br>
Linear solve converged due to CONVERGED_RTOL iterations 1<br>
<br>
<br>
Timestep 0: dt = 0.002, T = 0, Res[rho] = 1.22465e-16, Res[rhou] = 0.00215905, Res[rhov] = 1.61374e-08, Res[E] = 2.78002e-07, CFL = 399.998<br>
<br>
/*********************Stage 1 of SSPIRK (6,4)******************/<br>
<br>
0 SNES Function norm 1.155072464398e-02<br>
0 KSP preconditioned resid norm 1.155072464398e-02 true resid norm 1.155072464398e-02 ||Ae||/||Ax|| 1.000000000000e+00<br>
1 KSP preconditioned resid norm 1.155070298335e-02 true resid norm 1.155070298335e-02 ||Ae||/||Ax|| 9.999981247380e-01<br>
2 KSP preconditioned resid norm 1.152933291306e-02 true resid norm 1.152933291544e-02 ||Ae||/||Ax|| 9.981480184831e-01<br>
3 KSP preconditioned resid norm 9.177957248260e-03 true resid norm 9.177957556426e-03 ||Ae||/||Ax|| 7.945785082156e-01<br>
4 KSP preconditioned resid norm 3.124174018085e-03 true resid norm 3.124174393942e-03 ||Ae||/||Ax|| 2.704743200305e-01<br>
5 KSP preconditioned resid norm 6.978400843850e-04 true resid norm 6.978392323251e-04 ||Ae||/||Ax|| 6.041519072039e-02<br>
6 KSP preconditioned resid norm 1.634324558019e-04 true resid norm 1.634328735266e-04 ||Ae||/||Ax|| 1.414914462632e-02<br>
7 KSP preconditioned resid norm 9.422855713525e-05 true resid norm 9.422715588996e-05 ||Ae||/||Ax|| 8.157683504215e-03<br>
.<br>
.<br>
.<br>
.<br>
.<br>
.<br>
.<br>
1603 KSP preconditioned resid norm 1.191037802130e-07 true resid norm 1.191038670563e-07 ||Ae||/||Ax|| 1.031137618870e-05<br>
1604 KSP preconditioned resid norm 1.178729142296e-07 true resid norm 1.178730336572e-07 ||Ae||/||Ax|| 1.020481721193e-05<br>
1605 KSP preconditioned resid norm 1.169791191707e-07 true resid norm 1.169793038395e-07 ||Ae||/||Ax|| 1.012744286138e-05<br>
1606 KSP preconditioned resid norm 1.168865445541e-07 true resid norm 1.168870972685e-07 ||Ae||/||Ax|| 1.011946010931e-05<br>
1607 KSP preconditioned resid norm 1.168834953500e-07 true resid norm 1.168834210021e-07 ||Ae||/||Ax|| 1.011914183782e-05<br>
1608 KSP preconditioned resid norm 1.168824468217e-07 true resid norm 1.168821544127e-07 ||Ae||/||Ax|| 1.011903218328e-05<br>
1609 KSP preconditioned resid norm 1.168617721831e-07 true resid norm 1.168618273848e-07 ||Ae||/||Ax|| 1.011727237786e-05<br>
1610 KSP preconditioned resid norm 1.165091490989e-07 true resid norm 1.165091854357e-07 ||Ae||/||Ax|| 1.008674252281e-05<br>
1611 KSP preconditioned resid norm 1.153333136618e-07 true resid norm 1.153329197440e-07 ||Ae||/||Ax|| 9.984907726463e-06<br>
Linear solve converged due to CONVERGED_RTOL iterations 1611<br>
<br>
1 SNES Function norm 1.156248777455e-07<br>
0 KSP preconditioned resid norm 1.156248777455e-07 true resid norm 1.156248777455e-07 ||Ae||/||Ax|| 1.000000000000e+00<br>
1 KSP preconditioned resid norm 1.154422371390e-07 true resid norm 1.161710852713e-07 ||Ae||/||Ax|| 1.004723961974e+00<br>
2 KSP preconditioned resid norm 1.152905844799e-07 true resid norm 2.668696384811e-04 ||Ae||/||Ax|| 2.308064178614e+03<br>
3 KSP preconditioned resid norm 1.151761139639e-07 true resid norm 1.126358239931e-03 ||Ae||/||Ax|| 9.741486969699e+03<br>
4 KSP preconditioned resid norm 1.151754181824e-07 true resid norm 1.108271364124e-03 ||Ae||/||Ax|| 9.585059770296e+03<br>
5 KSP preconditioned resid norm 1.151747447109e-07 true resid norm 1.133976467714e-03 ||Ae||/||Ax|| 9.807374414786e+03<br>
6 KSP preconditioned resid norm 1.151536921969e-07 true resid norm 1.682604476358e-03 ||Ae||/||Ax|| 1.455227031730e+04<br>
7 KSP preconditioned resid norm 1.151536741521e-07 true resid norm 1.675362372285e-03 ||Ae||/||Ax|| 1.448963583747e+04<br>
8 KSP preconditioned resid norm 1.151516417557e-07 true resid norm 1.739170524328e-03 ||Ae||/||Ax|| 1.504149070891e+04<br>
9 KSP preconditioned resid norm 1.151458368640e-07 true resid norm 1.888742801931e-03 ||Ae||/||Ax|| 1.633509015325e+04<br>
10 KSP preconditioned resid norm 1.151432970406e-07 true resid norm 1.936535093080e-03 ||Ae||/||Ax|| 1.674842932455e+04<br>
11 KSP preconditioned resid norm 1.151415856844e-07 true resid norm 1.943578776829e-03 ||Ae||/||Ax|| 1.680934773489e+04<br>
12 KSP preconditioned resid norm 1.151415106457e-07 true resid norm 1.941675149163e-03 ||Ae||/||Ax|| 1.679288391064e+04<br>
13 KSP preconditioned resid norm 1.151072722808e-07 true resid norm 2.055181937477e-03 ||Ae||/||Ax|| 1.777456527997e+04<br>
14 KSP preconditioned resid norm 1.150661328335e-07 true resid norm 2.247389279980e-03 ||Ae||/||Ax|| 1.943690081062e+04<br>
15 KSP preconditioned resid norm 1.150587945589e-07 true resid norm 2.349201250473e-03 ||Ae||/||Ax|| 2.031743770266e+04<br>
16 KSP preconditioned resid norm 1.150293904255e-07 true resid norm 2.708861286410e-03 ||Ae||/||Ax|| 2.342801427537e+04<br>
17 KSP preconditioned resid norm 1.149883485190e-07 true resid norm 3.425777483190e-03 ||Ae||/||Ax|| 2.962837712771e+04<br>
18 KSP preconditioned resid norm 1.149835570884e-07 true resid norm 3.492000995212e-03 ||Ae||/||Ax|| 3.020112162105e+04<br>
19 KSP preconditioned resid norm 1.149457801834e-07 true resid norm 4.258473277795e-03 ||Ae||/||Ax|| 3.683007810108e+04<br>
20 KSP preconditioned resid norm 1.149279955651e-07 true resid norm 4.574312834982e-03 ||Ae||/||Ax|| 3.956166634874e+04<br>
21 KSP preconditioned resid norm 1.148902843639e-07 true resid norm 5.376279701577e-03 ||Ae||/||Ax|| 4.649760333939e+04<br>
22 KSP preconditioned resid norm 1.148568926668e-07 true resid norm 6.095225432786e-03 ||Ae||/||Ax|| 5.271551894049e+04<br>
23 KSP preconditioned resid norm 1.148172845230e-07 true resid norm 6.957862632489e-03 ||Ae||/||Ax|| 6.017617288039e+04<br>
24 KSP preconditioned resid norm 1.147830369881e-07 true resid norm 7.726330820912e-03 ||Ae||/||Ax|| 6.682239126701e+04<br>
25 KSP preconditioned resid norm 1.147438237790e-07 true resid norm 8.597270798069e-03 ||Ae||/||Ax|| 7.435485308784e+04<br>
26 KSP preconditioned resid norm 1.147090792721e-07 true resid norm 9.388950565795e-03 ||Ae||/||Ax|| 8.120182048070e+04<br>
27 KSP preconditioned resid norm 1.146703081864e-07 true resid norm 1.025969917305e-02 ||Ae||/||Ax|| 8.873262720872e+04<br>
28 KSP preconditioned resid norm 1.146352282540e-07 true resid norm 1.106505484202e-02 ||Ae||/||Ax|| 9.569787279149e+04<br>
29 KSP preconditioned resid norm 1.145968062554e-07 true resid norm 1.193384225768e-02 ||Ae||/||Ax|| 1.032117178445e+05<br>
30 KSP preconditioned resid norm 1.274732710949e-02 true resid norm 1.274732710949e-02 ||Ae||/||Ax|| 1.102472699478e+05<br>
Linear solve did not converge due to DIVERGED_DTOL iterations 30<br>
<br>
As you may see on the 2nd Newton iteration the linear system solution does not converge. I use the following arguments to start the computation:<br>
<br>
mpiexec -n 8 ./hoac blasius -llf_flux -n_out 1 -end_time 10000.0 -implicit -implicit_type 6 -pc_type asm -sub_pc_type ilu -snes_mf_operator -snes_max_fail 500 -snes_monitor -snes_stol 1.0e-50 -ksp_right_pc -snes_converged_reason -ksp_gmres_restart 30 -snes_max_linear_solve_fail 500 -sub_pc_factor_levels 2 -snes_max_it 1000 -sub_pc_factor_mat_ordering_<u></u>type rcm -dt 2.e-3 -snes_rtol 1.0e-8 -gl -snes_converged_reason -ksp_converged_reason -ksp_monitor_true_residual<br>
<br>
<br>
Why is this happening? What might I be doing wrong? Any suggestions or guide lines?<br></blockquote><div><br></div><div>It could be that:</div><div><br></div><div> a) Your system is really ill-conditioned, and the PC is not that good</div>
<div><br></div><div> b) Your system has a null space, and the initial guess had no component in it</div><div><br></div><div> c) Your Jacobian is not actually the Jacobian of the system, so the PC is not good</div><div><br>
</div><div> d) The FD approximation to the Jacobian action is not good</div><div><br></div><div>When you have so many different possible problems, the right thing to do is simplify the system until you can narrow down the cause</div>
<div><br></div><div> a) Run in serial</div><div><br></div><div> b) Shrink the problem</div><div><br></div><div> c) Fully form the Jacobian using -snes_fd</div><div><br></div><div> d) Run with -pc_type lu</div><div><br>
</div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Thank you,<br>
<br>
Kostas<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>