[petsc-users] Confusing Schur preconditioner behaviour
Dave May
dave.mayhem23 at gmail.com
Mon Mar 18 18:04:23 CDT 2019
On Mon, 18 Mar 2019 at 22:53, Dave May <dave.mayhem23 at gmail.com> wrote:
>
>
> On Mon, 18 Mar 2019 at 20:14, Cotter, Colin J via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Dear petsc-users,
>>
>> I'm solving a 2x2 block system, for which I can construct the Schur
>> complement analytically (through compatible FEM stuff),
>>
>> which I can pass as the preconditioning matrix.
>>
>
> In the KSP view your sent I see lines
> "Preconditioner for the Schur complement formed from A11"
> which indicates your custom Schur complement preconditioner is not being
> used.
>
> It doesn't exactly explain what you observe but the solver appears to not
> be configured as you described / expected.
>
Oh - are your defining the exact Schur operator within the preconditioning
matrix?
If you are doing that, then you need to tell fieldsplit to use the Amat to
define the splits otherwise it will define the Schur compliment as
S = B22 - B21 inv(B11) B12
preconditiones with B22, where as what you want is
S = A22 - A21 inv(A11) A12
preconditioned with B22.
If your operators are set up this way and you didn't indicate to use Amat
to define S this would definitely explain why preonly works but iterating
on Schur does not.
>
> Thanks
> Dave
>
>
>
>> When using gmres on the outer iteration, and preonly+lu on the inner
>> iterations with a Schur complement preconditioner,
>>
>> I see convergence in 1 iteration as expected. However, when I set
>> gmres+lu on the inner iteration for S, I see several iterations.
>>
>>
>> This seems strange to me, as the first result seems to confirm that I
>> have an exact Schur complement, but the second result
>>
>> implies not.
>>
>> What could be going on here?
>>
>>
>> I've appended output to the bottom of this message, first the preonly+lu
>> and then for gmres+lu.
>>
>>
>> all the best
>>
>> --Colin
>>
>>
>> Linear firedrake_0_fieldsplit_1_ solve converged due to CONVERGED_ITS
>> iterations 1
>> Residual norms for firedrake_0_ solve.
>> 0 KSP preconditioned resid norm 4.985448866758e+00 true resid norm
>> 1.086016610848e-03 ||r(i)||/||b|| 1.000000000000e+00
>> Linear firedrake_0_fieldsplit_1_ solve converged due to CONVERGED_ITS
>> iterations 1
>> 1 KSP preconditioned resid norm 1.245615753306e-13 true resid norm
>> 2.082000915439e-14 ||r(i)||/||b|| 1.917098591903e-11
>> KSP Object: (firedrake_0_) 1 MPI processes
>> type: gmres
>> restart=30, using Classical (unmodified) Gram-Schmidt
>> Orthogonalization with no iterative refinement
>> happy breakdown tolerance 1e-30
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-07, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using PRECONDITIONED norm type for convergence test
>> PC Object: (firedrake_0_) 1 MPI processes
>> type: fieldsplit
>> FieldSplit with Schur preconditioner, factorization FULL
>> Preconditioner for the Schur complement formed from A11
>> Split info:
>> Split number 0 Defined by IS
>> Split number 1 Defined by IS
>> KSP solver for A00 block
>> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 1.
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> package used to perform factorization: petsc
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> linear system matrix = precond matrix:
>> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> KSP solver for S = A11 - A10 inv(A00) A01
>> KSP Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 5.09173
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> package used to perform factorization: petsc
>> total: nonzeros=1360836, allocated nonzeros=1360836
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> linear system matrix followed by preconditioner matrix:
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: schurcomplement
>> rows=15360, cols=15360
>> Schur complement A11 - A10 inv(A00) A01
>> A11
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> total: nonzeros=267264, allocated nonzeros=267264
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> A10
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=15360, cols=6144
>> total: nonzeros=73728, allocated nonzeros=73728
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> KSP of A00
>> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50,
>> divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 1.
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> package used to perform factorization: petsc
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues
>> calls =0
>> using I-node routines: found 2048 nodes, limit
>> used is 5
>> linear system matrix = precond matrix:
>> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls
>> =0
>> using I-node routines: found 2048 nodes, limit used
>> is 5
>> A01
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=15360
>> total: nonzeros=73728, allocated nonzeros=73728
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> total: nonzeros=267264, allocated nonzeros=267264
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> linear system matrix followed by preconditioner matrix:
>> Mat Object: (firedrake_0_) 1 MPI processes
>> type: nest
>> rows=21504, cols=21504
>> Matrix object:
>> type=nest, rows=2, cols=2
>> MatNest structure:
>> (0,0) : type=seqaij, rows=15360, cols=15360
>> (0,1) : type=seqaij, rows=15360, cols=6144
>> (1,0) : type=seqaij, rows=6144, cols=15360
>> (1,1) : type=seqaij, rows=6144, cols=6144
>> Mat Object: (firedrake_0_) 1 MPI processes
>> type: nest
>> rows=21504, cols=21504
>> Matrix object:
>> type=nest, rows=2, cols=2
>> MatNest structure:
>> (0,0) : prefix="firedrake_0_fieldsplit_1_", type=seqaij,
>> rows=15360, cols=15360
>> (0,1) : type=seqaij, rows=15360, cols=6144
>> (1,0) : type=seqaij, rows=6144, cols=15360
>> (1,1) : prefix="firedrake_0_fieldsplit_0_", type=seqaij,
>> rows=6144, cols=6144
>>
>> =====
>>
>>
>> Residual norms for firedrake_0_fieldsplit_1_ solve.
>> 0 KSP preconditioned resid norm 8.819238435108e-02 true resid norm
>> 1.797571993221e-01 ||r(i)||/||b|| 1.000000000000e+00
>> 1 KSP preconditioned resid norm 1.025167319984e-02 true resid norm
>> 3.340583874349e-02 ||r(i)||/||b|| 1.858386694356e-01
>> 2 KSP preconditioned resid norm 1.235104644359e-03 true resid norm
>> 8.148396804822e-03 ||r(i)||/||b|| 4.533001646417e-02
>> 3 KSP preconditioned resid norm 1.624748553125e-04 true resid norm
>> 1.612221957927e-03 ||r(i)||/||b|| 8.968886720573e-03
>> 4 KSP preconditioned resid norm 2.233373761266e-05 true resid norm
>> 3.292437172839e-04 ||r(i)||/||b|| 1.831602397710e-03
>> 5 KSP preconditioned resid norm 1.895393184017e-06 true resid norm
>> 4.091207337005e-05 ||r(i)||/||b|| 2.275962994770e-04
>> 6 KSP preconditioned resid norm 1.699212495729e-07 true resid norm
>> 3.851173419652e-06 ||r(i)||/||b|| 2.142430697728e-05
>> Linear firedrake_0_fieldsplit_1_ solve converged due to
>> CONVERGED_RTOL iterations 6
>> KSP Object: (firedrake_0_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_) 1 MPI processes
>> type: fieldsplit
>> FieldSplit with Schur preconditioner, factorization FULL
>> Preconditioner for the Schur complement formed from A11
>> Split info:
>> Split number 0 Defined by IS
>> Split number 1 Defined by IS
>> KSP solver for A00 block
>> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 1.
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> package used to perform factorization: petsc
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> linear system matrix = precond matrix:
>> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> KSP solver for S = A11 - A10 inv(A00) A01
>> KSP Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: gmres
>> restart=30, using Classical (unmodified) Gram-Schmidt
>> Orthogonalization with no iterative refinement
>> happy breakdown tolerance 1e-30
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>> left preconditioning
>> using PRECONDITIONED norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 5.09173
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> package used to perform factorization: petsc
>> total: nonzeros=1360836, allocated nonzeros=1360836
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> linear system matrix followed by preconditioner matrix:
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: schurcomplement
>> rows=15360, cols=15360
>> Schur complement A11 - A10 inv(A00) A01
>> A11
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> total: nonzeros=267264, allocated nonzeros=267264
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> A10
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=15360, cols=6144
>> total: nonzeros=73728, allocated nonzeros=73728
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> KSP of A00
>> KSP Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: preonly
>> maximum iterations=10000, initial guess is zero
>> tolerances: relative=1e-05, absolute=1e-50,
>> divergence=10000.
>> left preconditioning
>> using NONE norm type for convergence test
>> PC Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: lu
>> out-of-place factorization
>> tolerance for zero pivot 2.22045e-14
>> matrix ordering: nd
>> factor fill ratio given 5., needed 1.
>> Factored matrix follows:
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> package used to perform factorization: petsc
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues
>> calls =0
>> using I-node routines: found 2048 nodes, limit
>> used is 5
>> linear system matrix = precond matrix:
>> Mat Object: (firedrake_0_fieldsplit_0_) 1 MPI processes
>> type: seqaij
>> rows=6144, cols=6144
>> total: nonzeros=18432, allocated nonzeros=18432
>> total number of mallocs used during MatSetValues calls
>> =0
>> using I-node routines: found 2048 nodes, limit used
>> is 5
>> A01
>> Mat Object: 1 MPI processes
>> type: seqaij
>> rows=6144, cols=15360
>> total: nonzeros=73728, allocated nonzeros=73728
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 2048 nodes, limit used is 5
>> Mat Object: (firedrake_0_fieldsplit_1_) 1 MPI processes
>> type: seqaij
>> rows=15360, cols=15360
>> total: nonzeros=267264, allocated nonzeros=267264
>> total number of mallocs used during MatSetValues calls =0
>> using I-node routines: found 5120 nodes, limit used is 5
>> linear system matrix followed by preconditioner matrix:
>> Mat Object: (firedrake_0_) 1 MPI processes
>> type: nest
>> rows=21504, cols=21504
>> Matrix object:
>> type=nest, rows=2, cols=2
>> MatNest structure:
>> (0,0) : type=seqaij, rows=15360, cols=15360
>> (0,1) : type=seqaij, rows=15360, cols=6144
>> (1,0) : type=seqaij, rows=6144, cols=15360
>> (1,1) : type=seqaij, rows=6144, cols=6144
>> Mat Object: (firedrake_0_) 1 MPI processes
>> type: nest
>> rows=21504, cols=21504
>> Matrix object:
>> type=nest, rows=2, cols=2
>> MatNest structure:
>> (0,0) : prefix="firedrake_0_fieldsplit_1_", type=seqaij,
>> rows=15360, cols=15360
>> (0,1) : type=seqaij, rows=15360, cols=6144
>> (1,0) : type=seqaij, rows=6144, cols=15360
>> (1,1) : prefix="firedrake_0_fieldsplit_0_", type=seqaij,
>> rows=6144, cols=6144
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190319/2ab071e4/attachment-0001.html>
More information about the petsc-users
mailing list