[petsc-users] Confusing Schur preconditioner behaviour
Justin Chang
jychang48 at gmail.com
Mon Mar 18 16:01:17 CDT 2019
Colin,
1) What equations are you solving?
2) In your second case, you set the outer ksp to preonly, thus we are
unable to see the ksp_monitor for the (firedrake_0_) solver. Set it to
gmres and see if you have a similar output to your first case:
0 KSP preconditioned resid norm 4.985448866758e+00 true resid norm
1.086016610848e-03 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 1.245615753306e-13 true resid norm
2.082000915439e-14 ||r(i)||/||b|| 1.917098591903e-11
Because according to the first ksp_view output, after one lu sweep for the
(firedrake_0_fieldsplit_1_) solver. That is, going from:
0 KSP preconditioned resid norm 8.819238435108e-02 true resid norm
1.797571993221e-01 ||r(i)||/||b|| 1.000000000000e+00
to
1 KSP preconditioned resid norm 1.025167319984e-02 true resid norm
3.340583874349e-02 ||r(i)||/||b|| 1.858386694356e-01
appeared to give you an exact schur complement.
Justin
On Mon, Mar 18, 2019 at 2:18 PM Cotter, Colin J via petsc-users <
petsc-users at mcs.anl.gov> wrote:
> Sorry, just to clarify, in the second case I see several *inner*
> iterations, even though I'm using LU on a supposedly exact Schur complement
> as the preconditioner for the Schur system.
> ------------------------------
> *From:* petsc-users <petsc-users-bounces at mcs.anl.gov> on behalf of
> Cotter, Colin J via petsc-users <petsc-users at mcs.anl.gov>
> *Sent:* 18 March 2019 20:14:48
> *To:* petsc-users at mcs.anl.gov
> *Subject:* [petsc-users] Confusing Schur preconditioner behaviour
>
>
> Dear petsc-users,
>
> I'm solving a 2x2 block system, for which I can construct the Schur
> complement analytically (through compatible FEM stuff),
>
> which I can pass as the preconditioning matrix.
>
>
> When using gmres on the outer iteration, and preonly+lu on the inner
> iterations with a Schur complement preconditioner,
>
> I see convergence in 1 iteration as expected. However, when I set gmres+lu
> on the inner iteration for S, I see several iterations.
>
>
> This seems strange to me, as the first result seems to confirm that I have
> an exact Schur complement, but the second result
>
> implies not.
>
> What could be going on here?
>
>
> I've appended output to the bottom of this message, first the preonly+lu
> and then for gmres+lu.
>
>
> all the best
>
> --Colin
>
>
> Linear firedrake_0_fieldsplit_1_ solve converged due to CONVERGED_ITS
> iterations 1
> Residual norms for firedrake_0_ solve.
> 0 KSP preconditioned resid norm 4.985448866758e+00 true resid norm
> 1.086016610848e-03 ||r(i)||/||b|| 1.000000000000e+00
> Linear firedrake_0_fieldsplit_1_ solve converged due to CONVERGED_ITS
> iterations 1
> 1 KSP preconditioned resid norm 1.245615753306e-13 true resid norm
> 2.082000915439e-14 ||r(i)||/||b|| 1.917098591903e-11
> KSP Object: (firedrake_0_) 1 MPI processes
> type: gmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-07, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: (firedrake_0_) 1 MPI processes
> type: fieldsplit
> FieldSplit with Schur preconditioner, factorization FULL
> Preconditioner for the Schur complement formed from A11
> Split info:
> Split number 0 Defined by IS
> Split number 1 Defined by IS
> KSP solver for A00 block
> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> package used to perform factorization: petsc
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> linear system matrix = precond matrix:
> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> KSP solver for S = A11 - A10 inv(A00) A01
> KSP Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 5.09173
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> package used to perform factorization: petsc
> total: nonzeros=1360836, allocated nonzeros=1360836
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> linear system matrix followed by preconditioner matrix:
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: schurcomplement
> rows=15360, cols=15360
> Schur complement A11 - A10 inv(A00) A01
> A11
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> total: nonzeros=267264, allocated nonzeros=267264
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> A10
> Mat Object: 1 MPI processes
> type: seqaij
> rows=15360, cols=6144
> total: nonzeros=73728, allocated nonzeros=73728
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> KSP of A00
> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50,
> divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> package used to perform factorization: petsc
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues
> calls =0
> using I-node routines: found 2048 nodes, limit
> used is 5
> linear system matrix = precond matrix:
> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is
> 5
> A01
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=15360
> total: nonzeros=73728, allocated nonzeros=73728
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> total: nonzeros=267264, allocated nonzeros=267264
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> linear system matrix followed by preconditioner matrix:
> Mat Object: (firedrake_0_) 1 MPI processes
> type: nest
> rows=21504, cols=21504
> Matrix object:
> type=nest, rows=2, cols=2
> MatNest structure:
> (0,0) : type=seqaij, rows=15360, cols=15360
> (0,1) : type=seqaij, rows=15360, cols=6144
> (1,0) : type=seqaij, rows=6144, cols=15360
> (1,1) : type=seqaij, rows=6144, cols=6144
> Mat Object: (firedrake_0_) 1 MPI processes
> type: nest
> rows=21504, cols=21504
> Matrix object:
> type=nest, rows=2, cols=2
> MatNest structure:
> (0,0) : prefix="firedrake_0_fieldsplit_1_", type=seqaij,
> rows=15360, cols=15360
> (0,1) : type=seqaij, rows=15360, cols=6144
> (1,0) : type=seqaij, rows=6144, cols=15360
> (1,1) : prefix="firedrake_0_fieldsplit_0_", type=seqaij,
> rows=6144, cols=6144
>
> =====
>
>
> Residual norms for firedrake_0_fieldsplit_1_ solve.
> 0 KSP preconditioned resid norm 8.819238435108e-02 true resid norm
> 1.797571993221e-01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 1.025167319984e-02 true resid norm
> 3.340583874349e-02 ||r(i)||/||b|| 1.858386694356e-01
> 2 KSP preconditioned resid norm 1.235104644359e-03 true resid norm
> 8.148396804822e-03 ||r(i)||/||b|| 4.533001646417e-02
> 3 KSP preconditioned resid norm 1.624748553125e-04 true resid norm
> 1.612221957927e-03 ||r(i)||/||b|| 8.968886720573e-03
> 4 KSP preconditioned resid norm 2.233373761266e-05 true resid norm
> 3.292437172839e-04 ||r(i)||/||b|| 1.831602397710e-03
> 5 KSP preconditioned resid norm 1.895393184017e-06 true resid norm
> 4.091207337005e-05 ||r(i)||/||b|| 2.275962994770e-04
> 6 KSP preconditioned resid norm 1.699212495729e-07 true resid norm
> 3.851173419652e-06 ||r(i)||/||b|| 2.142430697728e-05
> Linear firedrake_0_fieldsplit_1_ solve converged due to CONVERGED_RTOL
> iterations 6
> KSP Object: (firedrake_0_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_) 1 MPI processes
> type: fieldsplit
> FieldSplit with Schur preconditioner, factorization FULL
> Preconditioner for the Schur complement formed from A11
> Split info:
> Split number 0 Defined by IS
> Split number 1 Defined by IS
> KSP solver for A00 block
> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> package used to perform factorization: petsc
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> linear system matrix = precond matrix:
> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> KSP solver for S = A11 - A10 inv(A00) A01
> KSP Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: gmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 5.09173
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> package used to perform factorization: petsc
> total: nonzeros=1360836, allocated nonzeros=1360836
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> linear system matrix followed by preconditioner matrix:
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: schurcomplement
> rows=15360, cols=15360
> Schur complement A11 - A10 inv(A00) A01
> A11
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> total: nonzeros=267264, allocated nonzeros=267264
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> A10
> Mat Object: 1 MPI processes
> type: seqaij
> rows=15360, cols=6144
> total: nonzeros=73728, allocated nonzeros=73728
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> KSP of A00
> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50,
> divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: lu
> out-of-place factorization
> tolerance for zero pivot 2.22045e-14
> matrix ordering: nd
> factor fill ratio given 5., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> package used to perform factorization: petsc
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues
> calls =0
> using I-node routines: found 2048 nodes, limit
> used is 5
> linear system matrix = precond matrix:
> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
> type: seqaij
> rows=6144, cols=6144
> total: nonzeros=18432, allocated nonzeros=18432
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is
> 5
> A01
> Mat Object: 1 MPI processes
> type: seqaij
> rows=6144, cols=15360
> total: nonzeros=73728, allocated nonzeros=73728
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 2048 nodes, limit used is 5
> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
> type: seqaij
> rows=15360, cols=15360
> total: nonzeros=267264, allocated nonzeros=267264
> total number of mallocs used during MatSetValues calls =0
> using I-node routines: found 5120 nodes, limit used is 5
> linear system matrix followed by preconditioner matrix:
> Mat Object: (firedrake_0_) 1 MPI processes
> type: nest
> rows=21504, cols=21504
> Matrix object:
> type=nest, rows=2, cols=2
> MatNest structure:
> (0,0) : type=seqaij, rows=15360, cols=15360
> (0,1) : type=seqaij, rows=15360, cols=6144
> (1,0) : type=seqaij, rows=6144, cols=15360
> (1,1) : type=seqaij, rows=6144, cols=6144
> Mat Object: (firedrake_0_) 1 MPI processes
> type: nest
> rows=21504, cols=21504
> Matrix object:
> type=nest, rows=2, cols=2
> MatNest structure:
> (0,0) : prefix="firedrake_0_fieldsplit_1_", type=seqaij,
> rows=15360, cols=15360
> (0,1) : type=seqaij, rows=15360, cols=6144
> (1,0) : type=seqaij, rows=6144, cols=15360
> (1,1) : prefix="firedrake_0_fieldsplit_0_", type=seqaij,
> rows=6144, cols=6144
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190318/a30a7ed5/attachment-0001.html>
More information about the petsc-users
mailing list