<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Jan 18, 2017 at 4:42 AM, Klaij, Christiaan <span dir="ltr"><<a href="mailto:C.Klaij@marin.nl" target="_blank">C.Klaij@marin.nl</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks Lawrence, that nicely explains the unexpected behaviour!<br>
<br>
I guess in general there ought to be getters for the four<br>
ksp(A00)'s that occur in the full factorization.</blockquote><div><br></div><div>Yes, we will fix it. I think that the default retrieval should get the 00 block, not the inner as well.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
Chris<br>
<br>
<br>
dr. ir. Christiaan Klaij | CFD Researcher | Research & Development<br>
MARIN | T <a href="tel:%2B31%20317%2049%2033%2044" value="+31317493344">+31 317 49 33 44</a> | mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a> | <a href="http://www.marin.nl" rel="noreferrer" target="_blank">http://www.marin.nl</a><br>
<br>
</span>MARIN news: <a href="http://www.marin.nl/web/News/News-items/Verification-and-validation-exercises-for-flow-around-KVLCC2-tanker.htm" rel="noreferrer" target="_blank">http://www.marin.nl/web/News/<wbr>News-items/Verification-and-<wbr>validation-exercises-for-flow-<wbr>around-KVLCC2-tanker.htm</a><br>
<br>
______________________________<wbr>__________<br>
From: Lawrence Mitchell <<a href="mailto:lawrence.mitchell@imperial.ac.uk">lawrence.mitchell@imperial.<wbr>ac.uk</a>><br>
Sent: Wednesday, January 18, 2017 10:59 AM<br>
To: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
Cc: <a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>; Klaij, Christiaan<br>
<div class="HOEnZb"><div class="h5">Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
<br>
On 18/01/17 08:40, Klaij, Christiaan wrote:<br>
> Barry,<br>
><br>
> I've managed to replicate the problem with 3.7.4<br>
> snes/examples/tutorials/ex70.<wbr>c. Basically I've added<br>
> KSPGetTotalIterations to main (file is attached):<br>
<br>
PCFieldSplitGetSubKSP returns, in the Schur case:<br>
<br>
MatSchurComplementGet(pc-><wbr>schur, &ksp);<br>
<br>
in subksp[0]<br>
<br>
and<br>
<br>
pc->schur in subksp[1]<br>
<br>
In your case, subksp[0] is the (preonly) approximation to A^{-1} *inside*<br>
<br>
S = D - C A_inner^{-1} B<br>
<br>
And subksp[1] is the approximation to S^{-1}.<br>
<br>
Since each application of S to a vector (required in S^{-1}) requires<br>
one application of A^{-1}, because you use 225 iterations in total to<br>
invert S, you also use 225 applications of the KSP on A_inner.<br>
<br>
There doesn't appear to be a way to get the KSP used for A^{-1} if<br>
you've asked for different approximations to A^{-1} in the 0,0 block<br>
and inside S.<br>
<br>
Cheers,<br>
<br>
Lawrence<br>
<br>
> $ diff -u ex70.c.bak ex70.c<br>
> --- ex70.c.bak2017-01-18 09:25:46.286174830 +0100<br>
> +++ ex70.c2017-01-18 09:03:40.904483434 +0100<br>
> @@ -669,6 +669,10 @@<br>
> KSP ksp;<br>
> PetscErrorCode ierr;<br>
><br>
> + KSP *subksp;<br>
> + PC pc;<br>
> + PetscInt numsplit = 1, nusediter_vv, nusediter_pp;<br>
> +<br>
> ierr = PetscInitialize(&argc, &argv, NULL, help);CHKERRQ(ierr);<br>
> s.nx = 4;<br>
> s.ny = 6;<br>
> @@ -690,6 +694,13 @@<br>
> ierr = StokesSetupPC(&s, ksp);CHKERRQ(ierr);<br>
> ierr = KSPSolve(ksp, s.b, s.x);CHKERRQ(ierr);<br>
><br>
> + ierr = KSPGetPC(ksp, &pc);CHKERRQ(ierr);<br>
> + ierr = PCFieldSplitGetSubKSP(pc,&<wbr>numsplit,&subksp); CHKERRQ(ierr);<br>
> + ierr = KSPGetTotalIterations(subksp[<wbr>0],&nusediter_vv); CHKERRQ(ierr);<br>
> + ierr = KSPGetTotalIterations(subksp[<wbr>1],&nusediter_pp); CHKERRQ(ierr);<br>
> + ierr = PetscPrintf(PETSC_COMM_WORLD," total u solves = %i\n", nusediter_vv); CHKERRQ(ierr);<br>
> + ierr = PetscPrintf(PETSC_COMM_WORLD," total p solves = %i\n", nusediter_pp); CHKERRQ(ierr);<br>
> +<br>
> /* don't trust, verify! */<br>
> ierr = StokesCalcResidual(&s);<wbr>CHKERRQ(ierr);<br>
> ierr = StokesCalcError(&s);CHKERRQ(<wbr>ierr);<br>
><br>
> Now run as follows:<br>
><br>
> $ mpirun -n 2 ./ex70 -ksp_type fgmres -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_schur_fact_type lower -fieldsplit_0_ksp_type gmres -fieldsplit_0_pc_type bjacobi -fieldsplit_1_pc_type jacobi -fieldsplit_1_inner_ksp_type preonly -fieldsplit_1_inner_pc_type jacobi -fieldsplit_1_upper_ksp_type preonly -fieldsplit_1_upper_pc_type jacobi -fieldsplit_0_ksp_converged_<wbr>reason -fieldsplit_1_ksp_converged_<wbr>reason<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 14<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 14<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 16<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 16<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 17<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 18<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 20<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 21<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 23<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
> Linear fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 5<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
> total u solves = 225<br>
> total p solves = 225<br>
> residual u = 9.67257e-06<br>
> residual p = 5.42082e-07<br>
> residual [u,p] = 9.68775e-06<br>
> discretization error u = 0.0106464<br>
> discretization error p = 1.85907<br>
> discretization error [u,p] = 1.8591<br>
><br>
> So here again the total of 225 is correct for p, but for u it<br>
> should be 60. Hope this helps you find the problem.<br>
><br>
> Chris<br>
><br>
><br>
><br>
> dr. ir. Christiaan Klaij | CFD Researcher | Research & Development<br>
> MARIN | T +31 317 49 33 44 | mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a> | <a href="http://www.marin.nl" rel="noreferrer" target="_blank">http://www.marin.nl</a><br>
><br>
> MARIN news: <a href="http://www.marin.nl/web/News/News-items/Few-places-left-for-Offshore-and-Ship-hydrodynamics-courses.htm" rel="noreferrer" target="_blank">http://www.marin.nl/web/News/<wbr>News-items/Few-places-left-<wbr>for-Offshore-and-Ship-<wbr>hydrodynamics-courses.htm</a><br>
><br>
> ______________________________<wbr>__________<br>
> From: Klaij, Christiaan<br>
> Sent: Tuesday, January 17, 2017 8:45 AM<br>
> To: Barry Smith<br>
> Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
> Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
><br>
> Well, that's it, all the rest was hard coded. Here's the relevant part of the code:<br>
><br>
> CALL PCSetType(pc_system,<wbr>PCFIELDSPLIT,ierr); CHKERRQ(ierr)<br>
> CALL PCFieldSplitSetType(pc_system,<wbr>PC_COMPOSITE_SCHUR,ierr); CHKERRQ(ierr)<br>
> CALL PCFieldSplitSetIS(pc_system,"<wbr>0",isgs(1),ierr); CHKERRQ(ierr)<br>
> CALL PCFieldSplitSetIS(pc_system,"<wbr>1",isgs(2),ierr); CHKERRQ(ierr)<br>
> CALL PCFieldSplitSetSchurFactType(<wbr>pc_system,PC_FIELDSPLIT_SCHUR_<wbr>FACT_FULL,ierr);CHKERRQ(ierr)<br>
> CALL PCFieldSplitSetSchurPre(pc_<wbr>system,PC_FIELDSPLIT_SCHUR_<wbr>PRE_SELFP,PETSC_NULL_OBJECT,<wbr>ierr);CHKERRQ(ierr)<br>
><br>
> CALL KSPSetTolerances(ksp_system,<wbr>tol,PETSC_DEFAULT_REAL,PETSC_<wbr>DEFAULT_REAL,maxiter,ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>0_ksp_rtol","0.01",ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_ksp_rtol","0.01",ierr); CHKERRQ(ierr)<br>
><br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>0_ksp_pc_side","right",ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_ksp_pc_side","right",ierr); CHKERRQ(ierr)<br>
><br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>0_ksp_type","gmres",ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_upper_ksp_type","preonly",<wbr>ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_upper_pc_type","jacobi",<wbr>ierr); CHKERRQ(ierr)<br>
><br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_inner_ksp_type","preonly",<wbr>ierr); CHKERRQ(ierr)<br>
> CALL PetscOptionsSetValue(PETSC_<wbr>NULL_OBJECT,"-sys_fieldsplit_<wbr>1_inner_pc_type","jacobi",<wbr>ierr); CHKERRQ(ierr)<br>
><br>
> ______________________________<wbr>__________<br>
> From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>><br>
> Sent: Monday, January 16, 2017 9:28 PM<br>
> To: Klaij, Christiaan<br>
> Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
> Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
><br>
> Please send all the command line options you use.<br>
><br>
><br>
>> On Jan 16, 2017, at 1:47 AM, Klaij, Christiaan <<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a>> wrote:<br>
>><br>
>> Barry,<br>
>><br>
>> Sure, here's the output with:<br>
>><br>
>> -sys_ksp_view -sys_ksp_converged_reason -sys_fieldsplit_0_ksp_<wbr>converged_reason -sys_fieldsplit_1_ksp_<wbr>converged_reason<br>
>><br>
>> (In my previous email, I rearranged 0 & 1 for easy summing.)<br>
>><br>
>> Chris<br>
>><br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 2<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 6<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 2<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 3<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>> Linear sys_ solve converged due to CONVERGED_RTOL iterations 6<br>
>> KSP Object:(sys_) 1 MPI processes<br>
>> type: fgmres<br>
>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
>> GMRES: happy breakdown tolerance 1e-30<br>
>> maximum iterations=300, initial guess is zero<br>
>> tolerances: relative=0.01, absolute=1e-50, divergence=10000.<br>
>> right preconditioning<br>
>> using UNPRECONDITIONED norm type for convergence test<br>
>> PC Object:(sys_) 1 MPI processes<br>
>> type: fieldsplit<br>
>> FieldSplit with Schur preconditioner, factorization FULL<br>
>> Preconditioner for the Schur complement formed from Sp, an assembled approximation to S, which uses (lumped, if requested) A00's diagonal's inverse<br>
>> Split info:<br>
>> Split number 0 Defined by IS<br>
>> Split number 1 Defined by IS<br>
>> KSP solver for A00 block<br>
>> KSP Object: (sys_fieldsplit_0_) 1 MPI processes<br>
>> type: gmres<br>
>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
>> GMRES: happy breakdown tolerance 1e-30<br>
>> maximum iterations=10000, initial guess is zero<br>
>> tolerances: relative=0.01, absolute=1e-50, divergence=10000.<br>
>> right preconditioning<br>
>> using UNPRECONDITIONED norm type for convergence test<br>
>> PC Object: (sys_fieldsplit_0_) 1 MPI processes<br>
>> type: ilu<br>
>> ILU: out-of-place factorization<br>
>> 0 levels of fill<br>
>> tolerance for zero pivot 2.22045e-14<br>
>> matrix ordering: natural<br>
>> factor fill ratio given 1., needed 1.<br>
>> Factored matrix follows:<br>
>> Mat Object: 1 MPI processes<br>
>> type: seqaij<br>
>> rows=9600, cols=9600<br>
>> package used to perform factorization: petsc<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> linear system matrix = precond matrix:<br>
>> Mat Object: (sys_fieldsplit_0_) 1 MPI processes<br>
>> type: seqaij<br>
>> rows=9600, cols=9600<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> KSP solver for upper A00 in upper triangular factor<br>
>> KSP Object: (sys_fieldsplit_1_upper_) 1 MPI processes<br>
>> type: preonly<br>
>> maximum iterations=10000, initial guess is zero<br>
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
>> left preconditioning<br>
>> using NONE norm type for convergence test<br>
>> PC Object: (sys_fieldsplit_1_upper_) 1 MPI processes<br>
>> type: jacobi<br>
>> linear system matrix = precond matrix:<br>
>> Mat Object: (sys_fieldsplit_0_) 1 MPI processes<br>
>> type: seqaij<br>
>> rows=9600, cols=9600<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> KSP solver for S = A11 - A10 inv(A00) A01<br>
>> KSP Object: (sys_fieldsplit_1_) 1 MPI processes<br>
>> type: gmres<br>
>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
>> GMRES: happy breakdown tolerance 1e-30<br>
>> maximum iterations=10000, initial guess is zero<br>
>> tolerances: relative=0.01, absolute=1e-50, divergence=10000.<br>
>> right preconditioning<br>
>> using UNPRECONDITIONED norm type for convergence test<br>
>> PC Object: (sys_fieldsplit_1_) 1 MPI processes<br>
>> type: ilu<br>
>> ILU: out-of-place factorization<br>
>> 0 levels of fill<br>
>> tolerance for zero pivot 2.22045e-14<br>
>> matrix ordering: natural<br>
>> factor fill ratio given 1., needed 1.<br>
>> Factored matrix follows:<br>
>> Mat Object: 1 MPI processes<br>
>> type: seqaij<br>
>> rows=3200, cols=3200<br>
>> package used to perform factorization: petsc<br>
>> total: nonzeros=40404, allocated nonzeros=40404<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> linear system matrix followed by preconditioner matrix:<br>
>> Mat Object: (sys_fieldsplit_1_) 1 MPI processes<br>
>> type: schurcomplement<br>
>> rows=3200, cols=3200<br>
>> Schur complement A11 - A10 inv(A00) A01<br>
>> A11<br>
>> Mat Object: (sys_fieldsplit_1_) 1 MPI processes<br>
>> type: seqaij<br>
>> rows=3200, cols=3200<br>
>> total: nonzeros=40404, allocated nonzeros=40404<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> A10<br>
>> Mat Object: 1 MPI processes<br>
>> type: seqaij<br>
>> rows=3200, cols=9600<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> KSP of A00<br>
>> KSP Object: (sys_fieldsplit_1_inner_) 1 MPI processes<br>
>> type: preonly<br>
>> maximum iterations=10000, initial guess is zero<br>
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
>> left preconditioning<br>
>> using NONE norm type for convergence test<br>
>> PC Object: (sys_fieldsplit_1_inner_) 1 MPI processes<br>
>> type: jacobi<br>
>> linear system matrix = precond matrix:<br>
>> Mat Object: (sys_fieldsplit_0_) 1 MPI processes<br>
>> type: seqaij<br>
>> rows=9600, cols=9600<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> A01<br>
>> Mat Object: 1 MPI processes<br>
>> type: seqaij<br>
>> rows=9600, cols=3200<br>
>> total: nonzeros=47280, allocated nonzeros=47280<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> Mat Object: 1 MPI processes<br>
>> type: seqaij<br>
>> rows=3200, cols=3200<br>
>> total: nonzeros=40404, allocated nonzeros=40404<br>
>> total number of mallocs used during MatSetValues calls =0<br>
>> not using I-node routines<br>
>> linear system matrix followed by preconditioner matrix:<br>
>> Mat Object: 1 MPI processes<br>
>> type: nest<br>
>> rows=12800, cols=12800<br>
>> Matrix object:<br>
>> type=nest, rows=2, cols=2<br>
>> MatNest structure:<br>
>> (0,0) : prefix="mom_", type=seqaij, rows=9600, cols=9600<br>
>> (0,1) : prefix="grad_", type=seqaij, rows=9600, cols=3200<br>
>> (1,0) : prefix="div_", type=seqaij, rows=3200, cols=9600<br>
>> (1,1) : prefix="stab_", type=seqaij, rows=3200, cols=3200<br>
>> Mat Object: 1 MPI processes<br>
>> type: nest<br>
>> rows=12800, cols=12800<br>
>> Matrix object:<br>
>> type=nest, rows=2, cols=2<br>
>> MatNest structure:<br>
>> (0,0) : prefix="sys_fieldsplit_0_", type=seqaij, rows=9600, cols=9600<br>
>> (0,1) : type=seqaij, rows=9600, cols=3200<br>
>> (1,0) : type=seqaij, rows=3200, cols=9600<br>
>> (1,1) : prefix="sys_fieldsplit_1_", type=seqaij, rows=3200, cols=3200<br>
>> nusediter_vv 37<br>
>> nusediter_pp 37<br>
>><br>
>><br>
>><br>
>> dr. ir. Christiaan Klaij | CFD Researcher | Research & Development<br>
>> MARIN | T +31 317 49 33 44 | mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a> | <a href="http://www.marin.nl" rel="noreferrer" target="_blank">http://www.marin.nl</a><br>
>><br>
>> MARIN news: <a href="http://www.marin.nl/web/News/News-items/The-Ocean-Cleanup-testing-continues.htm" rel="noreferrer" target="_blank">http://www.marin.nl/web/News/<wbr>News-items/The-Ocean-Cleanup-<wbr>testing-continues.htm</a><br>
>><br>
>> ______________________________<wbr>__________<br>
>> From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>><br>
>> Sent: Friday, January 13, 2017 7:51 PM<br>
>> To: Klaij, Christiaan<br>
>> Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
>> Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
>><br>
>> Yes, I would have expected this to work. Could you send the output from -ksp_view in this case?<br>
>><br>
>><br>
>>> On Jan 13, 2017, at 3:46 AM, Klaij, Christiaan <<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a>> wrote:<br>
>>><br>
>>> Barry,<br>
>>><br>
>>> It's been a while but I'm finally using this function in<br>
>>> 3.7.4. Is it supposed to work with fieldsplit? Here's why.<br>
>>><br>
>>> I'm solving a Navier-Stokes system with fieldsplit (pc has one<br>
>>> velocity solve and one pressure solve) and trying to retrieve the<br>
>>> totals like this:<br>
>>><br>
>>> CALL KSPSolve(ksp_system,rr_system,<wbr>xx_system,ierr); CHKERRQ(ierr)<br>
>>> CALL PCFieldSplitGetSubKSP(pc_<wbr>system,numsplit,subksp,ierr); CHKERRQ(ierr)<br>
>>> CALL KSPGetTotalIterations(subksp(<wbr>1),nusediter_vv,ierr); CHKERRQ(ierr)<br>
>>> CALL KSPGetTotalIterations(subksp(<wbr>2),nusediter_pp,ierr); CHKERRQ(ierr)<br>
>>> print *, 'nusediter_vv', nusediter_vv<br>
>>> print *, 'nusediter_pp', nusediter_pp<br>
>>><br>
>>> Running the code shows this surprise:<br>
>>><br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 1<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 2<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 2<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 7<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>>> Linear sys_fieldsplit_0_ solve converged due to CONVERGED_RTOL iterations 8<br>
>>><br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 22<br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 6<br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 3<br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>>> Linear sys_fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 2<br>
>>><br>
>>> nusediter_vv 37<br>
>>> nusediter_pp 37<br>
>>><br>
>>> So the value of nusediter_pp is indeed 37, but for nusediter_vv<br>
>>> it should be 66. Any idea what went wrong?<br>
>>><br>
>>> Chris<br>
>>><br>
>>><br>
>>><br>
>>> dr. ir. Christiaan Klaij | CFD Researcher | Research & Development<br>
>>> MARIN | T +31 317 49 33 44 | mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a> | <a href="http://www.marin.nl" rel="noreferrer" target="_blank">http://www.marin.nl</a><br>
>>><br>
>>> MARIN news: <a href="http://www.marin.nl/web/News/News-items/MARIN-wishes-you-a-challenging-inspiring-2017.htm" rel="noreferrer" target="_blank">http://www.marin.nl/web/News/<wbr>News-items/MARIN-wishes-you-a-<wbr>challenging-inspiring-2017.htm</a><br>
>>><br>
>>> ______________________________<wbr>__________<br>
>>> From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>><br>
>>> Sent: Saturday, April 11, 2015 12:27 AM<br>
>>> To: Klaij, Christiaan<br>
>>> Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
>>> Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
>>><br>
>>> Chris,<br>
>>><br>
>>> I have added KSPGetTotalIterations() to the branch barry/add-ksp-total-<wbr>iterations/master and next. After tests it will go into master<br>
>>><br>
>>> Barry<br>
>>><br>
>>>> On Apr 10, 2015, at 8:07 AM, Klaij, Christiaan <<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a>> wrote:<br>
>>>><br>
>>>> Barry,<br>
>>>><br>
>>>> Sure, I can call PCFieldSplitGetSubKSP() to get the fieldsplit_0<br>
>>>> ksp and then KSPGetIterationNumber, but what does this number<br>
>>>> mean?<br>
>>>><br>
>>>> It appears to be the number of iterations of the last time that<br>
>>>> the subsystem was solved, right? If so, this corresponds to the<br>
>>>> last iteration of the coupled system, how about all the previous<br>
>>>> iterations?<br>
>>>><br>
>>>> Chris<br>
>>>> ______________________________<wbr>__________<br>
>>>> From: Barry Smith <<a href="mailto:bsmith@mcs.anl.gov">bsmith@mcs.anl.gov</a>><br>
>>>> Sent: Friday, April 10, 2015 2:48 PM<br>
>>>> To: Klaij, Christiaan<br>
>>>> Cc: <a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><br>
>>>> Subject: Re: [petsc-users] monitoring the convergence of fieldsplit 0 and 1<br>
>>>><br>
>>>> Chris,<br>
>>>><br>
>>>> It appears you should call PCFieldSplitGetSubKSP() and then get the information you want out of the individual KSPs. If this doesn't work please let us know.<br>
>>>><br>
>>>> Barry<br>
>>>><br>
>>>>> On Apr 10, 2015, at 6:48 AM, Klaij, Christiaan <<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a>> wrote:<br>
>>>>><br>
>>>>> A question when using PCFieldSplit: for each linear iteration of<br>
>>>>> the system, how many iterations for fielsplit 0 and 1?<br>
>>>>><br>
>>>>> One way to find out is to run with -ksp_monitor,<br>
>>>>> -fieldsplit_0_ksp_monitor and -fieldsplit_0_ksp_monitor. This<br>
>>>>> gives the complete convergence history.<br>
>>>>><br>
>>>>> Another way, suggested by Matt, is to use -ksp_monitor,<br>
>>>>> -fieldsplit_0_ksp_converged_<wbr>reason and<br>
>>>>> -fieldsplit_1_ksp_converged_<wbr>reason. This gives only the totals<br>
>>>>> for fieldsplit 0 and 1 (but without saying for which one).<br>
>>>>><br>
>>>>> Both ways require to somehow process the output, which is a bit<br>
>>>>> inconvenient. Could KSPGetResidualHistory perhaps return (some)<br>
>>>>> information on the subsystems' convergence for processing inside<br>
>>>>> the code?<br>
>>>>><br>
>>>>> Chris<br>
>>>>><br>
>>>>><br>
>>>>> dr. ir. Christiaan Klaij<br>
>>>>> CFD Researcher<br>
>>>>> Research & Development<br>
>>>>> E mailto:<a href="mailto:C.Klaij@marin.nl">C.Klaij@marin.nl</a><br>
>>>>> T <a href="tel:%2B31%20317%2049%2033%2044" value="+31317493344">+31 317 49 33 44</a><br>
>>>>><br>
>>>>><br>
>>>>> MARIN<br>
>>>>> 2, Haagsteeg, P.O. Box 28, 6700 AA Wageningen, The Netherlands<br>
>>>>> T <a href="tel:%2B31%20317%2049%2039%2011" value="+31317493911">+31 317 49 39 11</a>, F <a href="tel:%2B31%20317%2049%2032%2045" value="+31317493245">+31 317 49 32 45</a>, I <a href="http://www.marin.nl" rel="noreferrer" target="_blank">www.marin.nl</a><br>
>>>>><br>
>>>><br>
>>><br>
>><br>
><br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>