<div><div dir="auto">Use -ksp_type fgmres if your inner ksp solvers are gmres. Maybe that will help?</div></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 18, 2019 at 1:33 PM Rossi, Simone via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div id="m_-4869738905073706766divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Thanks Barry.</p>
<p style="margin-top:0;margin-bottom:0">Let me know if you can spot anything out of the ksp_view</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p>KSP Object: 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=5000, nonzero initial guess</p>
<p> tolerances: relative=0.001, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p>PC Object: 1 MPI processes</p>
<p> type: fieldsplit</p>
<p> FieldSplit with MULTIPLICATIVE composition: total splits = 2, blocksize = 2</p>
<p> Solver info for each split is in the following KSP objects:</p>
<p> Split number 0 Fields 0</p>
<p> KSP Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=10000, initial guess is zero</p>
<p> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p> PC Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: hypre</p>
<p> HYPRE BoomerAMG preconditioning</p>
<p> Cycle type V</p>
<p> Maximum number of levels 25</p>
<p> Maximum number of iterations PER hypre call 1</p>
<p> Convergence tolerance PER hypre call 0.</p>
<p> Threshold for strong coupling 0.25</p>
<p> Interpolation truncation factor 0.</p>
<p> Interpolation: max elements per row 0</p>
<p> Number of levels of aggressive coarsening 0</p>
<p> Number of paths for aggressive coarsening 1</p>
<p> Maximum row sums 0.9</p>
<p> Sweeps down 1</p>
<p> Sweeps up 1</p>
<p> Sweeps on coarse 1</p>
<p> Relax down symmetric-SOR/Jacobi</p>
<p> Relax up symmetric-SOR/Jacobi</p>
<p> Relax on coarse Gaussian-elimination</p>
<p> Relax weight (all) 1.</p>
<p> Outer relax weight (all) 1.</p>
<p> Using CF-relaxation</p>
<p> Not using more complex smoothers.</p>
<p> Measure type local</p>
<p> Coarsen type Falgout</p>
<p> Interpolation type classical</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=35937, cols=35937</p>
<p> total: nonzeros=912673, allocated nonzeros=912673</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> not using I-node routines</p>
<p> Split number 1 Fields 1</p>
<p> KSP Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=10000, initial guess is zero</p>
<p> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p> PC Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: hypre</p>
<p> HYPRE BoomerAMG preconditioning</p>
<p> Cycle type V</p>
<p> Maximum number of levels 25</p>
<p> Maximum number of iterations PER hypre call 1</p>
<p> Convergence tolerance PER hypre call 0.</p>
<p> Threshold for strong coupling 0.25</p>
<p> Interpolation truncation factor 0.</p>
<p> Interpolation: max elements per row 0</p>
<p> Number of levels of aggressive coarsening 0</p>
<p> Number of paths for aggressive coarsening 1</p>
<p> Maximum row sums 0.9</p>
<p> Sweeps down 1</p>
<p> Sweeps up 1</p>
<p> Sweeps on coarse 1</p>
<p> Relax down symmetric-SOR/Jacobi</p>
<p> Relax up symmetric-SOR/Jacobi</p>
<p> Relax on coarse Gaussian-elimination</p>
<p> Relax weight (all) 1.</p>
<p> Outer relax weight (all) 1.</p>
<p> Using CF-relaxation</p>
<p> Not using more complex smoothers.</p>
<p> Measure type local</p>
<p> Coarsen type Falgout</p>
<p> Interpolation type classical</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=35937, cols=35937</p>
<p> total: nonzeros=912673, allocated nonzeros=912673</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> not using I-node routines</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: () 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=71874, cols=71874</p>
<p> total: nonzeros=3650692, allocated nonzeros=3650692</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> using I-node routines: found 35937 nodes, limit used is 5</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
</div>
<hr style="display:inline-block;width:98%">
<div id="m_-4869738905073706766divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>><br>
<b>Sent:</b> Monday, March 18, 2019 3:27:13 PM<br>
<b>To:</b> Rossi, Simone<br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] BJACOBI with FIELDSPLIT</font>
<div> </div>
</div></div><div dir="ltr">
<div class="m_-4869738905073706766BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="m_-4869738905073706766PlainText"><br>
Simone,<br>
<br>
This is indeed surprising, given the block structure of the matrix and the exact block solves we'd expect the solver to converge after the application of the preconditioner. Please send the output of -ksp_view
<br>
<br>
Barry<br>
<br>
Also if you are willing to share your test code we can try running it to determine why it doesn't converge immediately.<br>
<br>
<br>
> On Mar 18, 2019, at 2:14 PM, Rossi, Simone via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
> <br>
> Dear all,<br>
> I'm debugging my application in which I'm trying to use the FIELDSPLIT preconditioner for solving a 2x2 block matrix.<br>
> <br>
> Currently I'm testing the preconditioner on a decoupled system where I solve two identical and independent Poisson problems. Using the default fieldsplit type (multiplicative), I'm expecting the method to be equivalent to a Block Jacobi solver.
<br>
> Setting <br>
> -ksp_rtol 1e-6<br>
> while using gmres/hypre on each subblock with<br>
> -fieldsplit_0_ksp_rtol 1e-12<br>
> -fieldsplit_1_ksp_rtol 1e-12<br>
> I'm expecting to converge in 1 iteration with a single solve for each block.<br>
> <br>
> Asking to output the iteration count for the subblocks with<br>
> -ksp_converged_reason<br>
> -fieldsplit_0_ksp_converged_reason<br>
> -fieldsplit_1_ksp_converged_reason<br>
> revealed that the outer solver converges in 1 iteration, but each block is solved for 3 times.<br>
> This is the output I get:<br>
> <br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> 0 KSP preconditioned resid norm 9.334948012657e+01 true resid norm 1.280164130222e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> <br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> 1 KSP preconditioned resid norm 1.518151977611e-11 true resid norm 8.123270435936e-12 ||r(i)||/||b|| 6.345491366429e-14<br>
> <br>
> Linear solve converged due to CONVERGED_RTOL iterations 1<br>
> <br>
> <br>
> Are the subblocks actually solved for multiple times at every outer iteration? <br>
> <br>
> Thanks for the help,<br>
> <br>
> Simone <br>
<br>
</div>
</span></font></div>
</div>
</blockquote></div></div>