<div dir="ltr"><div dir="ltr">On Mon, Mar 18, 2019 at 3:56 PM Rossi, Simone via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div id="gmail-m_-878005859810095852divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p style="margin-top:0px;margin-bottom:0px">To follow up on that: when would you want to use gmres instead of fgmres in the outer ksp?</p></div></div></blockquote><div><br></div><div>The difference here is just that FGMRES is right-preconditioned by default, so you do not get the extra application. I think</div><div>if you use the regular monitor, -ksp_monitor, you will not see 2 applications.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div id="gmail-m_-878005859810095852divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p style="margin-top:0px;margin-bottom:0px">Thanks again for the help,</p>
<p style="margin-top:0px;margin-bottom:0px">Simone</p>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-878005859810095852divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Rossi, Simone<br>
<b>Sent:</b> Monday, March 18, 2019 3:43:04 PM<br>
<b>To:</b> Justin Chang<br>
<b>Cc:</b> Smith, Barry F.; <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] BJACOBI with FIELDSPLIT</font>
<div> </div>
</div>
<div dir="ltr">
<div id="gmail-m_-878005859810095852x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top:0px;margin-bottom:0px">Thanks, using fgmres it does work as expected. </p>
<p style="margin-top:0px;margin-bottom:0px">I thought gmres would do the same since I'm solving the subblocks "exactly".</p>
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
<p style="margin-top:0px;margin-bottom:0px">Simone</p>
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-878005859810095852x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Justin Chang <<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>><br>
<b>Sent:</b> Monday, March 18, 2019 3:38:34 PM<br>
<b>To:</b> Rossi, Simone<br>
<b>Cc:</b> Smith, Barry F.; <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] BJACOBI with FIELDSPLIT</font>
<div> </div>
</div>
<div>
<div>
<div dir="auto">Use -ksp_type fgmres if your inner ksp solvers are gmres. Maybe that will help?</div>
</div>
<div><br>
<div class="gmail-m_-878005859810095852x_x_gmail_quote">
<div dir="ltr" class="gmail-m_-878005859810095852x_x_gmail_attr">On Mon, Mar 18, 2019 at 1:33 PM Rossi, Simone via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="gmail-m_-878005859810095852x_x_gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div id="gmail-m_-878005859810095852x_x_m_-4869738905073706766divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top:0px;margin-bottom:0px">Thanks Barry.</p>
<p style="margin-top:0px;margin-bottom:0px">Let me know if you can spot anything out of the ksp_view</p>
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
<p>KSP Object: 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=5000, nonzero initial guess</p>
<p> tolerances: relative=0.001, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p>PC Object: 1 MPI processes</p>
<p> type: fieldsplit</p>
<p> FieldSplit with MULTIPLICATIVE composition: total splits = 2, blocksize = 2</p>
<p> Solver info for each split is in the following KSP objects:</p>
<p> Split number 0 Fields 0</p>
<p> KSP Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=10000, initial guess is zero</p>
<p> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p> PC Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: hypre</p>
<p> HYPRE BoomerAMG preconditioning</p>
<p> Cycle type V</p>
<p> Maximum number of levels 25</p>
<p> Maximum number of iterations PER hypre call 1</p>
<p> Convergence tolerance PER hypre call 0.</p>
<p> Threshold for strong coupling 0.25</p>
<p> Interpolation truncation factor 0.</p>
<p> Interpolation: max elements per row 0</p>
<p> Number of levels of aggressive coarsening 0</p>
<p> Number of paths for aggressive coarsening 1</p>
<p> Maximum row sums 0.9</p>
<p> Sweeps down 1</p>
<p> Sweeps up 1</p>
<p> Sweeps on coarse 1</p>
<p> Relax down symmetric-SOR/Jacobi</p>
<p> Relax up symmetric-SOR/Jacobi</p>
<p> Relax on coarse Gaussian-elimination</p>
<p> Relax weight (all) 1.</p>
<p> Outer relax weight (all) 1.</p>
<p> Using CF-relaxation</p>
<p> Not using more complex smoothers.</p>
<p> Measure type local</p>
<p> Coarsen type Falgout</p>
<p> Interpolation type classical</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: (fieldsplit_0_) 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=35937, cols=35937</p>
<p> total: nonzeros=912673, allocated nonzeros=912673</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> not using I-node routines</p>
<p> Split number 1 Fields 1</p>
<p> KSP Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: gmres</p>
<p> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</p>
<p> happy breakdown tolerance 1e-30</p>
<p> maximum iterations=10000, initial guess is zero</p>
<p> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</p>
<p> left preconditioning</p>
<p> using PRECONDITIONED norm type for convergence test</p>
<p> PC Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: hypre</p>
<p> HYPRE BoomerAMG preconditioning</p>
<p> Cycle type V</p>
<p> Maximum number of levels 25</p>
<p> Maximum number of iterations PER hypre call 1</p>
<p> Convergence tolerance PER hypre call 0.</p>
<p> Threshold for strong coupling 0.25</p>
<p> Interpolation truncation factor 0.</p>
<p> Interpolation: max elements per row 0</p>
<p> Number of levels of aggressive coarsening 0</p>
<p> Number of paths for aggressive coarsening 1</p>
<p> Maximum row sums 0.9</p>
<p> Sweeps down 1</p>
<p> Sweeps up 1</p>
<p> Sweeps on coarse 1</p>
<p> Relax down symmetric-SOR/Jacobi</p>
<p> Relax up symmetric-SOR/Jacobi</p>
<p> Relax on coarse Gaussian-elimination</p>
<p> Relax weight (all) 1.</p>
<p> Outer relax weight (all) 1.</p>
<p> Using CF-relaxation</p>
<p> Not using more complex smoothers.</p>
<p> Measure type local</p>
<p> Coarsen type Falgout</p>
<p> Interpolation type classical</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: (fieldsplit_1_) 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=35937, cols=35937</p>
<p> total: nonzeros=912673, allocated nonzeros=912673</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> not using I-node routines</p>
<p> linear system matrix = precond matrix:</p>
<p> Mat Object: () 1 MPI processes</p>
<p> type: seqaij</p>
<p> rows=71874, cols=71874</p>
<p> total: nonzeros=3650692, allocated nonzeros=3650692</p>
<p> total number of mallocs used during MatSetValues calls =0</p>
<p> using I-node routines: found 35937 nodes, limit used is 5</p>
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-878005859810095852x_x_m_-4869738905073706766divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>><br>
<b>Sent:</b> Monday, March 18, 2019 3:27:13 PM<br>
<b>To:</b> Rossi, Simone<br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] BJACOBI with FIELDSPLIT</font>
<div> </div>
</div>
</div>
<div dir="ltr">
<div class="gmail-m_-878005859810095852x_x_m_-4869738905073706766BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="gmail-m_-878005859810095852x_x_m_-4869738905073706766PlainText"><br>
Simone,<br>
<br>
This is indeed surprising, given the block structure of the matrix and the exact block solves we'd expect the solver to converge after the application of the preconditioner. Please send the output of -ksp_view
<br>
<br>
Barry<br>
<br>
Also if you are willing to share your test code we can try running it to determine why it doesn't converge immediately.<br>
<br>
<br>
> On Mar 18, 2019, at 2:14 PM, Rossi, Simone via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
> <br>
> Dear all,<br>
> I'm debugging my application in which I'm trying to use the FIELDSPLIT preconditioner for solving a 2x2 block matrix.<br>
> <br>
> Currently I'm testing the preconditioner on a decoupled system where I solve two identical and independent Poisson problems. Using the default fieldsplit type (multiplicative), I'm expecting the method to be equivalent to a Block Jacobi solver.
<br>
> Setting <br>
> -ksp_rtol 1e-6<br>
> while using gmres/hypre on each subblock with<br>
> -fieldsplit_0_ksp_rtol 1e-12<br>
> -fieldsplit_1_ksp_rtol 1e-12<br>
> I'm expecting to converge in 1 iteration with a single solve for each block.<br>
> <br>
> Asking to output the iteration count for the subblocks with<br>
> -ksp_converged_reason<br>
> -fieldsplit_0_ksp_converged_reason<br>
> -fieldsplit_1_ksp_converged_reason<br>
> revealed that the outer solver converges in 1 iteration, but each block is solved for 3 times.<br>
> This is the output I get:<br>
> <br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> 0 KSP preconditioned resid norm 9.334948012657e+01 true resid norm 1.280164130222e+02 ||r(i)||/||b|| 1.000000000000e+00<br>
> <br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 7<br>
> 1 KSP preconditioned resid norm 1.518151977611e-11 true resid norm 8.123270435936e-12 ||r(i)||/||b|| 6.345491366429e-14<br>
> <br>
> Linear solve converged due to CONVERGED_RTOL iterations 1<br>
> <br>
> <br>
> Are the subblocks actually solved for multiple times at every outer iteration? <br>
> <br>
> Thanks for the help,<br>
> <br>
> Simone <br>
<br>
</div>
</span></font></div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>