<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Thank you Barry, your suggestion really helped speed up the program. The maximum number of iterations is 48. I still don't know what the asm pre-conditioner is but I guess I just need to read the manual. I'm trying to add code to do what you suggested automatically, and I found that I can add:</div><div><br></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; color: rgb(38, 126, 61); "><span class="Apple-style-span" style="color: rgb(0, 0, 0); "> PC <span style="color: #4d8186">pc</span>;</span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: #4d8186">ierr</span> = KSPGetPC(ksp_,&<span style="color: #4d8186">pc</span>); <span style="color: #794830">CHKERR</span>(<span style="color: #4d8186">ierr</span>);</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 18px/normal Menlo; "> <span style="color: #4d8186">ierr</span> = PCSetType(<span style="color: #4d8186">pc</span>, PCASM); <span style="color: #794830">CHKERR</span>(<span style="color: #4d8186">ierr</span>);</div></div><div><br></div><div>However, I cannot find the function to replace the -sub_pc_type. Can you point me where to look? My system may not be symmetric so I can't use the cg option.</div><div><br></div><div>Thanks again for your response.</div><div><br></div><div>a˛</div><div><br></div><br><div><div>On Apr 26, 2011, at 2:23 PM, Barry Smith wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div><br><blockquote type="cite">System solved in 3664 iterations... 42.683 s<br></blockquote><br> The preconditioner is simply not up to the task it has been assigned. This number of iterations is problematic.<br><br> Have you tried -pc_type asm -sub_pc_type lu If that works well you can try -pc_type asm -sub_pc_type ilu and see if that still works. <br><br> If the matrix is indeed symmetric positive definite you will want to use -ksp_type cg<br><br><br><br> Barry<br><br><br>On Apr 26, 2011, at 1:37 AM, Alejandro Marcos Aragón wrote:<br><br><blockquote type="cite">Hi all,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I'm using the standard configuration of the KSP solver, but the time it takes to solve a large system of equations is increasing (because of the increasing number of iterations?). These are my timing lines and the log from the KSP solver in two consecutive solves:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">[1] Solving system... 0.154203 s<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">KSP Object:<br></blockquote><blockquote type="cite"> type: gmres<br></blockquote><blockquote type="cite"> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br></blockquote><blockquote type="cite"> GMRES: happy breakdown tolerance 1e-30<br></blockquote><blockquote type="cite"> maximum iterations=10000<br></blockquote><blockquote type="cite"> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br></blockquote><blockquote type="cite"> left preconditioning<br></blockquote><blockquote type="cite"> using nonzero initial guess<br></blockquote><blockquote type="cite"> using PRECONDITIONED norm type for convergence test<br></blockquote><blockquote type="cite">PC Object:<br></blockquote><blockquote type="cite"> type: bjacobi<br></blockquote><blockquote type="cite"> block Jacobi: number of blocks = 3<br></blockquote><blockquote type="cite"> Local solve is same for all blocks, in the following KSP and PC objects:<br></blockquote><blockquote type="cite"> KSP Object:(sub_)<br></blockquote><blockquote type="cite"> type: preonly<br></blockquote><blockquote type="cite"> maximum iterations=10000, initial guess is zero<br></blockquote><blockquote type="cite"> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br></blockquote><blockquote type="cite"> left preconditioning<br></blockquote><blockquote type="cite"> using PRECONDITIONED norm type for convergence test<br></blockquote><blockquote type="cite"> PC Object:(sub_)<br></blockquote><blockquote type="cite"> type: ilu<br></blockquote><blockquote type="cite"> ILU: out-of-place factorization<br></blockquote><blockquote type="cite"> 0 levels of fill<br></blockquote><blockquote type="cite"> tolerance for zero pivot 1e-12<br></blockquote><blockquote type="cite"> using diagonal shift to prevent zero pivot<br></blockquote><blockquote type="cite"> matrix ordering: natural<br></blockquote><blockquote type="cite"> factor fill ratio given 1, needed 1<br></blockquote><blockquote type="cite"> Factored matrix follows:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=seqaij, rows=2020, cols=2020<br></blockquote><blockquote type="cite"> package used to perform factorization: petsc<br></blockquote><blockquote type="cite"> total: nonzeros=119396, allocated nonzeros=163620<br></blockquote><blockquote type="cite"> using I-node routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"> linear system matrix = precond matrix:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=seqaij, rows=2020, cols=2020<br></blockquote><blockquote type="cite"> total: nonzeros=119396, allocated nonzeros=163620<br></blockquote><blockquote type="cite"> using I-node routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"> linear system matrix = precond matrix:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=mpiaij, rows=6058, cols=6058<br></blockquote><blockquote type="cite"> total: nonzeros=365026, allocated nonzeros=509941<br></blockquote><blockquote type="cite"> using I-node (on process 0) routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">[1] System solved in 51 iterations... 0.543215 s<br></blockquote><blockquote type="cite">...<br></blockquote><blockquote type="cite">...<br></blockquote><blockquote type="cite">...<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">[1] Solving system... 0.302414 s<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">KSP Object:<br></blockquote><blockquote type="cite"> type: gmres<br></blockquote><blockquote type="cite"> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br></blockquote><blockquote type="cite"> GMRES: happy breakdown tolerance 1e-30<br></blockquote><blockquote type="cite"> maximum iterations=10000<br></blockquote><blockquote type="cite"> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br></blockquote><blockquote type="cite"> left preconditioning<br></blockquote><blockquote type="cite"> using nonzero initial guess<br></blockquote><blockquote type="cite"> using PRECONDITIONED norm type for convergence test<br></blockquote><blockquote type="cite">PC Object:<br></blockquote><blockquote type="cite"> type: bjacobi<br></blockquote><blockquote type="cite"> block Jacobi: number of blocks = 3<br></blockquote><blockquote type="cite"> Local solve is same for all blocks, in the following KSP and PC objects:<br></blockquote><blockquote type="cite"> KSP Object:(sub_)<br></blockquote><blockquote type="cite"> type: preonly<br></blockquote><blockquote type="cite"> maximum iterations=10000, initial guess is zero<br></blockquote><blockquote type="cite"> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br></blockquote><blockquote type="cite"> left preconditioning<br></blockquote><blockquote type="cite"> using PRECONDITIONED norm type for convergence test<br></blockquote><blockquote type="cite"> PC Object:(sub_)<br></blockquote><blockquote type="cite"> type: ilu<br></blockquote><blockquote type="cite"> ILU: out-of-place factorization<br></blockquote><blockquote type="cite"> 0 levels of fill<br></blockquote><blockquote type="cite"> tolerance for zero pivot 1e-12<br></blockquote><blockquote type="cite"> using diagonal shift to prevent zero pivot<br></blockquote><blockquote type="cite"> matrix ordering: natural<br></blockquote><blockquote type="cite"> factor fill ratio given 1, needed 1<br></blockquote><blockquote type="cite"> Factored matrix follows:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=seqaij, rows=2020, cols=2020<br></blockquote><blockquote type="cite"> package used to perform factorization: petsc<br></blockquote><blockquote type="cite"> total: nonzeros=119396, allocated nonzeros=163620<br></blockquote><blockquote type="cite"> using I-node routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"> linear system matrix = precond matrix:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=seqaij, rows=2020, cols=2020<br></blockquote><blockquote type="cite"> total: nonzeros=119396, allocated nonzeros=163620<br></blockquote><blockquote type="cite"> using I-node routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"> linear system matrix = precond matrix:<br></blockquote><blockquote type="cite"> Matrix Object:<br></blockquote><blockquote type="cite"> type=mpiaij, rows=6058, cols=6058<br></blockquote><blockquote type="cite"> total: nonzeros=365026, allocated nonzeros=509941<br></blockquote><blockquote type="cite"> using I-node (on process 0) routines: found 676 nodes, limit used is 5<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">[1] System solved in 3664 iterations... 42.683 s<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">As you can see, the second iteration takes more than 40 seconds to solve. Could some explain why this is happening and why he number of iterations is increasing dramatically between solves? Thank you all,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Alejandro M. Aragón, Ph.D.<br></blockquote><br></div></blockquote></div><br></body></html>