[petsc-users] Understanding inner vs outer fieldsplit convergence
Matthew Knepley
knepley at gmail.com
Thu Jan 26 21:50:07 CST 2017
On Thu, Jan 26, 2017 at 9:36 PM, David Knezevic <david.knezevic at akselos.com>
wrote:
> On Thu, Jan 26, 2017 at 6:50 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>>
>> David, with Schur complement preconditioning the nesting can be rather
>> complicated and hard to track. We need to know exactly what monitors you
>> have turned on and the output from -ksp_view in order to understand why you
>> are seeing this (seemingly) strange effect. Send all command line arguments
>> and if you are running a PETSc example.
>>
>
> Hi Barry,
>
> Thanks for your comments. The command line arguments I'm using are:
>
> -ksp_view -ksp_monitor -ksp_type cg -pc_type fieldsplit
> -pc_fieldsplit_type schur -fieldsplit_block_2_ksp_monitor
> -pc_fieldsplit_type schur -fieldsplit_block_1_pc_type cholesky
> -fieldsplit_block_1_ksp_type preonly -fieldsplit_block_2_pc_type cholesky
> -fieldsplit_block_2_ksp_type minres
>
preonly messes with monitors because "nothing is supposed to happen" so ksp
monitors don't fire. I would start with -ksp_type richardson -ksp_max_it 1
Matt
> Note that I cleaned up my code a bit (I was doing some things via code
> previously, but now I'm doing it all via command line arguments) and now I
> can't replicate the strange effect that I reported in my previous email, so
> I guess that must have been a bug at my end, sorry about that! (I'm using
> my own example here, not a PETSc example.)
>
> I've pasted the output that I get below, and it doesn't have any repeated
> inner iterations anymore, so it looks good to me now.
>
> Thanks,
> David
>
> Residual norms for fieldsplit_block_2_ solve.
> 0 KSP Residual norm 4.742303891404e+01
> 1 KSP Residual norm 2.909253505627e-01
> 2 KSP Residual norm 9.891933795043e-02
> 3 KSP Residual norm 7.147789520726e-02
> 4 KSP Residual norm 1.668752975827e-02
> 5 KSP Residual norm 5.020016959860e-03
> 6 KSP Residual norm 2.939648855177e-03
> 7 KSP Residual norm 2.847937179284e-03
> 8 KSP Residual norm 2.840502402597e-03
> 9 KSP Residual norm 2.831875522390e-03
> 10 KSP Residual norm 2.688309294147e-03
> 11 KSP Residual norm 1.364866662320e-03
> 12 KSP Residual norm 1.351460340317e-03
> 13 KSP Residual norm 9.154713357796e-06
> 0 KSP Residual norm 2.254632359666e+02
> Residual norms for fieldsplit_block_2_ solve.
> 0 KSP Residual norm 1.562796164027e-05
> 1 KSP Residual norm 1.562568942204e-05
> 2 KSP Residual norm 1.559644955712e-05
> 3 KSP Residual norm 4.739731314562e-06
> 4 KSP Residual norm 4.099203626623e-06
> 5 KSP Residual norm 4.077340563505e-06
> 6 KSP Residual norm 4.069934161136e-06
> 7 KSP Residual norm 4.069081720832e-06
> 8 KSP Residual norm 4.056610946411e-06
> 9 KSP Residual norm 4.049977739107e-06
> 10 KSP Residual norm 6.930727940912e-07
> 11 KSP Residual norm 2.525753277129e-07
> 12 KSP Residual norm 2.490106038182e-07
> 13 KSP Residual norm 2.117790443270e-07
> 14 KSP Residual norm 4.278506164384e-08
> 15 KSP Residual norm 4.258945198001e-08
> 16 KSP Residual norm 4.229334447981e-08
> 17 KSP Residual norm 4.411718412575e-09
> 18 KSP Residual norm 2.538786960344e-10
> 19 KSP Residual norm 1.938707294127e-11
> 1 KSP Residual norm 8.266789276240e-01
> Residual norms for fieldsplit_block_2_ solve.
> 0 KSP Residual norm 9.262098141749e-08
> 1 KSP Residual norm 5.654510775654e-10
> 2 KSP Residual norm 1.850940554946e-10
> 3 KSP Residual norm 1.215872907413e-10
> 4 KSP Residual norm 3.606362942651e-11
> 5 KSP Residual norm 9.320062914875e-12
> 6 KSP Residual norm 5.745183182943e-12
> 7 KSP Residual norm 5.567033341218e-12
> 8 KSP Residual norm 5.549943035966e-12
> 9 KSP Residual norm 5.535981844425e-12
> 10 KSP Residual norm 5.145409580427e-12
> 11 KSP Residual norm 2.666485042347e-12
> 12 KSP Residual norm 2.642698026095e-12
> 13 KSP Residual norm 2.351888597601e-14
> 2 KSP Residual norm 4.408818087377e-07
> Residual norms for fieldsplit_block_2_ solve.
> 0 KSP Residual norm 6.995890260994e-14
> 1 KSP Residual norm 6.992908711883e-14
> 2 KSP Residual norm 6.975805821489e-14
> 3 KSP Residual norm 3.895698269070e-14
> 4 KSP Residual norm 1.087381803170e-14
> 5 KSP Residual norm 9.649779038703e-15
> 6 KSP Residual norm 9.639503278208e-15
> 7 KSP Residual norm 9.632063248075e-15
> 8 KSP Residual norm 9.628902454482e-15
> 9 KSP Residual norm 9.422064141948e-15
> 10 KSP Residual norm 7.978243292438e-15
> 11 KSP Residual norm 1.840364998277e-15
> 12 KSP Residual norm 1.313440209393e-15
> 13 KSP Residual norm 1.237317810391e-15
> 14 KSP Residual norm 3.450624715175e-16
> 15 KSP Residual norm 2.677943917771e-16
> 16 KSP Residual norm 2.477309550239e-16
> 17 KSP Residual norm 2.379077634512e-16
> 18 KSP Residual norm 2.195084093320e-17
> 19 KSP Residual norm 3.440267981045e-18
> 20 KSP Residual norm 1.678278636525e-19
> 3 KSP Residual norm 2.982422976816e-09
> Residual norms for fieldsplit_block_2_ solve.
> 0 KSP Residual norm 7.797636927781e-16
> 1 KSP Residual norm 4.773690049123e-18
> 2 KSP Residual norm 1.562856784196e-18
> 3 KSP Residual norm 1.029604947226e-18
> 4 KSP Residual norm 3.077276285355e-19
> 5 KSP Residual norm 7.977399411645e-20
> 6 KSP Residual norm 4.916849936508e-20
> 7 KSP Residual norm 4.764440575250e-20
> 8 KSP Residual norm 4.678272883449e-20
> 9 KSP Residual norm 4.672106002328e-20
> 10 KSP Residual norm 4.395115290861e-20
> 11 KSP Residual norm 2.248590969592e-20
> 12 KSP Residual norm 2.226836892279e-20
> 13 KSP Residual norm 3.483134646974e-22
> 4 KSP Residual norm 3.713946450555e-15
> KSP Object: 1 MPI processes
> type: cg
> maximum iterations=1000, initial guess is zero
> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: fieldsplit
> FieldSplit with Schur preconditioner, factorization FULL
> Preconditioner for the Schur complement formed from A11
> Split info:
> Split number 0 Defined by IS
> Split number 1 Defined by IS
> KSP solver for A00 block
> KSP Object: (fieldsplit_block_1_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (fieldsplit_block_1_) 1 MPI processes
> type: cholesky
> Cholesky: out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: natural
> factor fill ratio given 5., needed 1.14458
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=30, cols=30
> package used to perform factorization: petsc
> total: nonzeros=285, allocated nonzeros=285
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: (fieldsplit_block_1_) 1 MPI processes
> type: seqaij
> rows=30, cols=30
> total: nonzeros=468, allocated nonzeros=468
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 10 nodes, limit used is 5
> KSP solver for S = A11 - A10 inv(A00) A01
> KSP Object: (fieldsplit_block_2_) 1 MPI processes
> type: minres
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: (fieldsplit_block_2_) 1 MPI processes
> type: cholesky
> Cholesky: out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: natural
> factor fill ratio given 5., needed 23.1555
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=10512, cols=10512
> package used to perform factorization: petsc
> total: nonzeros=4435992, allocated nonzeros=4435992
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix followed by preconditioner matrix:
> Mat Object: (fieldsplit_block_2_) 1 MPI processes
> type: schurcomplement
> rows=10512, cols=10512
> Schur complement A11 - A10 inv(A00) A01
> A11
> Mat Object: (fieldsplit_block_2_)
> 1 MPI processes
> type: seqaij
> rows=10512, cols=10512
> total: nonzeros=372636, allocated nonzeros=372636
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 3504 nodes, limit used is 5
> A10
> Mat Object: 1 MPI processes
> type: seqaij
> rows=10512, cols=30
> total: nonzeros=1872, allocated nonzeros=1872
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2143 nodes, limit used is 5
> KSP of A00
> KSP Object: (fieldsplit_block_1_)
> 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50,
> divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (fieldsplit_block_1_)
> 1 MPI processes
> type: cholesky
> Cholesky: out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: natural
> factor fill ratio given 5., needed 1.14458
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=30, cols=30
> package used to perform factorization: petsc
> total: nonzeros=285, allocated nonzeros=285
> total number of mallocs used during MatSetValues
> calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: (fieldsplit_block_1_)
> 1 MPI processes
> type: seqaij
> rows=30, cols=30
> total: nonzeros=468, allocated nonzeros=468
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 10 nodes, limit used is 5
> A01
> Mat Object: 1 MPI processes
> type: seqaij
> rows=30, cols=10512
> total: nonzeros=1872, allocated nonzeros=1872
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 9 nodes, limit used is 5
> Mat Object: (fieldsplit_block_2_) 1 MPI processes
> type: seqaij
> rows=10512, cols=10512
> total: nonzeros=372636, allocated nonzeros=372636
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 3504 nodes, limit used is 5
> linear system matrix = precond matrix:
> Mat Object: () 1 MPI processes
> type: seqaij
> rows=10542, cols=10542
> total: nonzeros=376848, allocated nonzeros=377064
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 3514 nodes, limit used is 5
>
>
>
>
>
>>
>>
>>
>> > On Jan 26, 2017, at 1:26 PM, David Knezevic <david.knezevic at akselos.com>
>> wrote:
>> >
>> > I'm exploring fieldsplit with Schur (this continues from some emails I
>> sent a few weeks ago about this topic), and I had a quick question about
>> the inner vs outer convergence.
>> >
>> > I've pasted the output below from "-ksp_monitor
>> -fieldsplit_FE_split_ksp_monitor", and I'm just wondering about why the
>> second outer iteration has two inner iteration loops, whereas all the other
>> outer iterations have one inner iteration loop? I assume it is something to
>> do with a convergence tolerance, but it's not clear to me which tolerance
>> would control that.
>> >
>> > Thanks,
>> > David
>> >
>> > ------------------------------------------------------------------
>> >
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 4.742303891408e+01
>> > 1 KSP Residual norm 2.909253505630e-01
>> > 2 KSP Residual norm 9.891933795059e-02
>> > 3 KSP Residual norm 7.147789520745e-02
>> > 4 KSP Residual norm 1.668752967907e-02
>> > 5 KSP Residual norm 5.019869896662e-03
>> > 6 KSP Residual norm 2.848579237244e-03
>> > 7 KSP Residual norm 2.847897269641e-03
>> > 8 KSP Residual norm 2.840502392022e-03
>> > 9 KSP Residual norm 2.831875522381e-03
>> > 10 KSP Residual norm 2.688309287993e-03
>> > 11 KSP Residual norm 1.351494303229e-03
>> > 12 KSP Residual norm 1.350874246297e-03
>> > 13 KSP Residual norm 9.154691604943e-06
>> > 0 KSP Residual norm 2.254632353893e+02
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 4.742303891408e+01
>> > 1 KSP Residual norm 2.909253505630e-01
>> > 2 KSP Residual norm 9.891933795059e-02
>> > 3 KSP Residual norm 7.147789520745e-02
>> > 4 KSP Residual norm 1.668752967907e-02
>> > 5 KSP Residual norm 5.019869896662e-03
>> > 6 KSP Residual norm 2.848579237244e-03
>> > 7 KSP Residual norm 2.847897269641e-03
>> > 8 KSP Residual norm 2.840502392022e-03
>> > 9 KSP Residual norm 2.831875522381e-03
>> > 10 KSP Residual norm 2.688309287993e-03
>> > 11 KSP Residual norm 1.351494303229e-03
>> > 12 KSP Residual norm 1.350874246297e-03
>> > 13 KSP Residual norm 9.154691604943e-06
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 1.554697370480e-05
>> > 1 KSP Residual norm 1.554471967929e-05
>> > 2 KSP Residual norm 1.551293889691e-05
>> > 3 KSP Residual norm 8.031337431574e-06
>> > 4 KSP Residual norm 4.137185786243e-06
>> > 5 KSP Residual norm 4.066606123330e-06
>> > 6 KSP Residual norm 4.051107282928e-06
>> > 7 KSP Residual norm 4.047442850256e-06
>> > 8 KSP Residual norm 4.047129984657e-06
>> > 9 KSP Residual norm 4.030697964677e-06
>> > 10 KSP Residual norm 2.882383190940e-06
>> > 11 KSP Residual norm 3.325005138484e-07
>> > 12 KSP Residual norm 2.107354774516e-07
>> > 13 KSP Residual norm 2.107005548204e-07
>> > 14 KSP Residual norm 4.399320792736e-08
>> > 15 KSP Residual norm 4.236902403786e-08
>> > 16 KSP Residual norm 2.932877082709e-08
>> > 17 KSP Residual norm 3.881909203171e-09
>> > 18 KSP Residual norm 1.107791399514e-09
>> > 19 KSP Residual norm 2.645048006100e-11
>> > 1 KSP Residual norm 8.266776463696e-01
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 9.262528453386e-08
>> > 1 KSP Residual norm 5.683232925010e-10
>> > 2 KSP Residual norm 1.915223168286e-10
>> > 3 KSP Residual norm 1.397893184942e-10
>> > 4 KSP Residual norm 1.691441435404e-11
>> > 5 KSP Residual norm 6.138315243419e-12
>> > 6 KSP Residual norm 5.576043830003e-12
>> > 7 KSP Residual norm 5.574440028225e-12
>> > 8 KSP Residual norm 5.559544964428e-12
>> > 9 KSP Residual norm 5.539862581746e-12
>> > 10 KSP Residual norm 5.258329460152e-12
>> > 11 KSP Residual norm 2.643581511791e-12
>> > 12 KSP Residual norm 2.641293392449e-12
>> > 13 KSP Residual norm 2.354608977643e-14
>> > 2 KSP Residual norm 4.450925351013e-07
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 6.653681330477e-14
>> > 1 KSP Residual norm 6.650750698147e-14
>> > 2 KSP Residual norm 6.111123464526e-14
>> > 3 KSP Residual norm 2.026817941567e-14
>> > 4 KSP Residual norm 9.604999144183e-15
>> > 5 KSP Residual norm 9.208296307424e-15
>> > 6 KSP Residual norm 9.196769686859e-15
>> > 7 KSP Residual norm 9.185058975459e-15
>> > 8 KSP Residual norm 9.180207477303e-15
>> > 9 KSP Residual norm 8.991574890909e-15
>> > 10 KSP Residual norm 8.032736869820e-15
>> > 11 KSP Residual norm 1.536409278928e-15
>> > 12 KSP Residual norm 1.177374264280e-15
>> > 13 KSP Residual norm 1.175712092044e-15
>> > 14 KSP Residual norm 2.572275406087e-16
>> > 15 KSP Residual norm 2.548423809711e-16
>> > 16 KSP Residual norm 8.616505207588e-17
>> > 17 KSP Residual norm 7.563053994201e-18
>> > 18 KSP Residual norm 6.807636198601e-18
>> > 19 KSP Residual norm 9.747028518744e-19
>> > 20 KSP Residual norm 2.419807103570e-21
>> > 3 KSP Residual norm 2.986369469883e-09
>> > Residual norms for fieldsplit_FE_split_ solve.
>> > 0 KSP Residual norm 7.813223137340e-16
>> > 1 KSP Residual norm 4.793103235095e-18
>> > 2 KSP Residual norm 1.615526128222e-18
>> > 3 KSP Residual norm 1.179102504397e-18
>> > 4 KSP Residual norm 1.427467627551e-19
>> > 5 KSP Residual norm 5.177440470993e-20
>> > 6 KSP Residual norm 4.703763659148e-20
>> > 7 KSP Residual norm 4.701953228322e-20
>> > 8 KSP Residual norm 4.689269668869e-20
>> > 9 KSP Residual norm 4.672625361251e-20
>> > 10 KSP Residual norm 4.435174006113e-20
>> > 11 KSP Residual norm 2.229156843383e-20
>> > 12 KSP Residual norm 2.228887211080e-20
>> > 13 KSP Residual norm 3.492936921635e-22
>> > 4 KSP Residual norm 3.753341263086e-15
>>
>>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170126/7358c67f/attachment-0001.html>
More information about the petsc-users
mailing list