<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Mar 26, 2015 at 11:31 AM, Manav Bhatia <span dir="ltr"><<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Ok, so I ran my fluids problem with -pc_type jacobi. This time it did not return with “inf”, but there was no convergence. </div></div></blockquote><div><br></div><div>Good. This seems to confirm that ILU(0) was the problem.</div><div><br></div><div>It is not a surprise that Jacobi is not an effective preconditioner. The second step that we always recommend is to look in</div><div>the literature and find some preconditioners that people say work for your problem, or for something like it.</div><div><br></div><div>With PETSc, it is simple to reproduce preconditioners from the literature, for example SIMPLE from Patankar, and then</div><div>optimize them for your problem.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Here is the first 10 iterations: </div><div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 0 KSP preconditioned resid norm 7.840061446913e+07 true resid norm 2.709083260443e+06 ||r(i)||/||b|| 1.000000000000e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 1 KSP preconditioned resid norm 7.838673732620e+07 true resid norm 2.709686618724e+06 ||r(i)||/||b|| 1.000222716773e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 2 KSP preconditioned resid norm 7.838673500462e+07 true resid norm 2.709685645990e+06 ||r(i)||/||b|| 1.000222357709e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 3 KSP preconditioned resid norm 7.838080292706e+07 true resid norm 2.709691690751e+06 ||r(i)||/||b|| 1.000224589003e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 4 KSP preconditioned resid norm 7.837871329477e+07 true resid norm 2.709550983943e+06 ||r(i)||/||b|| 1.000172650101e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 5 KSP preconditioned resid norm 7.837788814104e+07 true resid norm 2.709528734769e+06 ||r(i)||/||b|| 1.000164437296e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 6 KSP preconditioned resid norm 7.837690087318e+07 true resid norm 2.709494763671e+06 ||r(i)||/||b|| 1.000151897594e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 7 KSP preconditioned resid norm 7.836061163366e+07 true resid norm 2.709622438419e+06 ||r(i)||/||b|| 1.000199025989e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 8 KSP preconditioned resid norm 7.835981946243e+07 true resid norm 2.709587889985e+06 ||r(i)||/||b|| 1.000186273176e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 9 KSP preconditioned resid norm 7.828072351639e+07 true resid norm 2.710062032107e+06 ||r(i)||/||b|| 1.000361292574e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)"> 10 KSP preconditioned resid norm 7.828054329393e+07 true resid norm 2.710056988152e+06 ||r(i)||/||b|| 1.000359430706e+00</div></div><div><br></div><div>and here are the last 10 iterations: </div><div><br></div><div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">991 KSP preconditioned resid norm 7.801323853477e+07 true resid norm 2.710182819026e+06 ||r(i)||/||b|| 1.000405878475e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">992 KSP preconditioned resid norm 7.801323853477e+07 true resid norm 2.710182817684e+06 ||r(i)||/||b|| 1.000405877980e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">993 KSP preconditioned resid norm 7.801323853476e+07 true resid norm 2.710182817272e+06 ||r(i)||/||b|| 1.000405877828e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">994 KSP preconditioned resid norm 7.801323853470e+07 true resid norm 2.710182825637e+06 ||r(i)||/||b|| 1.000405880916e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">995 KSP preconditioned resid norm 7.801323853470e+07 true resid norm 2.710182825705e+06 ||r(i)||/||b|| 1.000405880941e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">996 KSP preconditioned resid norm 7.801323853470e+07 true resid norm 2.710182826222e+06 ||r(i)||/||b|| 1.000405881131e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">997 KSP preconditioned resid norm 7.801323853469e+07 true resid norm 2.710182826506e+06 ||r(i)||/||b|| 1.000405881236e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">998 KSP preconditioned resid norm 7.801323853424e+07 true resid norm 2.710182832779e+06 ||r(i)||/||b|| 1.000405883552e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">999 KSP preconditioned resid norm 7.801323852813e+07 true resid norm 2.710182815446e+06 ||r(i)||/||b|| 1.000405877154e+00</div><div style="margin:0px;font-size:10px;font-family:Monaco;color:rgb(245,245,245);background-color:rgb(0,0,0)">1000 KSP preconditioned resid norm 7.801323817948e+07 true resid norm 2.710182749096e+06 ||r(i)||/||b|| 1.000405852662e+00</div></div><div><br></div><div>Any recommendations? </div><div><br></div><div>Thanks,</div><div>Manav</div><div><div class="h5"><div><br></div><br><div><blockquote type="cite"><div>On Mar 26, 2015, at 11:28 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:</div><br><div><br> This is fine. PCGAMG does algebraic multigrid so the mesh doesn't matter in its use.<br><br> Barry<br><br><blockquote type="cite">On Mar 26, 2015, at 11:20 AM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br><br>Thanks. <br>Quick question (out of ignorance): does it matter that the HEX8 may still be arranged in an unstructured fashion? Meaning, that although I use brick elements, my grid does not have a structured grid appearance.<br><br>-Manav<br><br><br><blockquote type="cite">On Mar 26, 2015, at 11:14 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br><br><br><blockquote type="cite">On Mar 26, 2015, at 10:51 AM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br><br>Barry, <br><br>On a related note, I have another elasticity problem that I am trying to solver with HEX8 elements. It is an isotropic solid structure. Do you have a recommended preconditioned for this problem? <br></blockquote><br> Yes, this one clearly requires PCGAMG. Make sure you read all the docs on PCGMAG, you will need to supply either the coordinates with PCSetCoordiantes() or the near null space of the operator. Unfortunately our documentation for this sucks and everyone refuses to improve it.<br><br>Barry<br><br><br><br><blockquote type="cite"><br> The problem has about 770K dofs and the default ILU(0) has not done well. I have also tried ILU(1), and that too has been unhelpful. I am observing stagnation of residuals after a drop of couple of magnitudes. <br><br> Any recommendations would be greatly appreciated. <br><br>Thanks,<br>Manav<br><br><br><br><br><br><br><blockquote type="cite">On Mar 26, 2015, at 10:35 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br><br><br><blockquote type="cite">On Mar 26, 2015, at 10:19 AM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br><br>Thanks, Barry. I will try that. <br><br>This is Euler flow equations discretized with SUPG. The mesh is made of 4-noded tetrahedra. The flow parameters correspond to transonic flow. <br></blockquote><br>Yes, ILU could easily fail on this and really isn't appropriate. Likely you should be using PCFIELDSPLIT for preconditioning.<br><br>Barry<br><br><blockquote type="cite"><br>-Manav<br><br><br><blockquote type="cite">On Mar 26, 2015, at 10:17 AM, Barry Smith <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br><br><br>The default preconditioner with ILU(0) on each process is not appropriate for your problem and is producing overflow. Try -sub_pc_type lu and see if that produces a different result. <br><br>Is this a Stokes-like problem?<br><br>Barry<br><br><blockquote type="cite">On Mar 26, 2015, at 10:10 AM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br><br>Thanks, Matt. <br><br>Following is the output with: -ksp_monitor_lg_residualnorm -ksp_log -ksp_view -ksp_monitor_true_residual -ksp_converged_reason<br><br>0 KSP preconditioned resid norm inf true resid norm 2.709083260443e+06 ||r(i)||/||b|| 1.000000000000e+00<br>Linear solve did not converge due to DIVERGED_NANORINF iterations 0<br>KSP Object: 12 MPI processes<br>type: gmres<br>GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>GMRES: happy breakdown tolerance 1e-30<br>maximum iterations=1000<br>tolerances: relative=1e-10, absolute=1e-50, divergence=10000<br>left preconditioning<br>using nonzero initial guess<br>using PRECONDITIONED norm type for convergence test<br>PC Object: 12 MPI processes<br>type: bjacobi<br>block Jacobi: number of blocks = 12<br>Local solve is same for all blocks, in the following KSP and PC objects:<br>KSP Object: (sub_) 1 MPI processes<br>type: preonly<br>maximum iterations=10000, initial guess is zero<br>tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>left preconditioning<br>using NONE norm type for convergence test<br>PC Object: (sub_) 1 MPI processes<br>type: ilu<br>ILU: out-of-place factorization<br>0 levels of fill<br>tolerance for zero pivot 2.22045e-14<br>using diagonal shift on blocks to prevent zero pivot [INBLOCKS]<br>matrix ordering: natural<br>factor fill ratio given 1, needed 1<br> Factored matrix follows:<br> Mat Object: 1 MPI processes<br> type: seqaij<br> rows=667070, cols=667070<br> package used to perform factorization: petsc<br> total: nonzeros=4.6765e+07, allocated nonzeros=4.6765e+07<br> total number of mallocs used during MatSetValues calls =0<br> using I-node routines: found 133414 nodes, limit used is 5<br>linear system matrix = precond matrix:<br>Mat Object: () 1 MPI processes<br>type: seqaij<br>rows=667070, cols=667070<br>total: nonzeros=4.6765e+07, allocated nonzeros=5.473e+07<br>total number of mallocs used during MatSetValues calls =0<br> using I-node routines: found 133414 nodes, limit used is 5<br>linear system matrix = precond matrix:<br>Mat Object: () 12 MPI processes<br>type: mpiaij<br>rows=6723030, cols=6723030<br>total: nonzeros=4.98852e+08, allocated nonzeros=5.38983e+08<br>total number of mallocs used during MatSetValues calls =0<br>using I-node (on process 0) routines: found 133414 nodes, limit used is 5<br><br><br>Anything jumps out at you as odd? <br><br>-Manav<br><br><br><br><blockquote type="cite">On Mar 26, 2015, at 9:34 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br><br>On Thu, Mar 26, 2015 at 9:21 AM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br>Hi,<br><br>I am using the KSP linear solver for my system of equations, without any command line options at this point. I have checked that the L1 norms of my system matrix and the force vector are finite values, but the KSP solver is returning with an “inf” residual in the very first iteration.<br><br>The problem has 6.7M dofs and I have tried this on multiple machines with different number of nodes with the same result.<br><br>Is there a reason why the solver would return after the first iteration with an inf?<br><br>I am not sure on where to start debugging this case, so I would appreciate any pointers.<br><br>For all solver questions, we want to see the output of<br><br>-ksp_view -ksp_monitor_true_residual -ksp_converged_reason<br><br>The problem here would be that there is an error, so we would never see the output<br>of -ksp_view and know what solver you are using. If you are using something complex,<br>can you try using<br><br>-pc_type jacobi<br><br>and send the output from the options above? Then we can figure out why the other solver<br>gets an inf.<br><br>Thanks,<br><br>Matt<br><br>Thanks,<br>Manav<br><br><br><br><br><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br></blockquote><br></blockquote><br></blockquote><br></blockquote><br></blockquote><br></blockquote><br></blockquote><br></blockquote><br></div></blockquote></div><br></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>