[petsc-users] PCASMType
Barry Smith
bsmith at mcs.anl.gov
Thu Aug 4 19:42:38 CDT 2016
History,
1) I originally implemented the ASM with one subdomain per process
2) easily extended to support multiple domain per process
3) added -pc_asm_type restrict etc but it only worked for one subdomain per process because it took advantage of the fact that
restrict etc could be achieved by simply dropping the parallel communication in the vector scatters
4) Matt didn't like the restriction to one process per subdomain so he added an additional argument to PCASMSetLocalSubdomains() that allowed passing in the overlapping and non-overlapping regions of each domain (foolishly calling the non-overlapping index set is_local even though local has nothing to do with), so that the restrict etc could be handled.
Unfortunately IMHO Matt made a mess of things because if you use things like -pc_asm_blocks n or -pc_asm_overlap 1 etc it does not handle the -pc_asm_type restrict since it cannot track the is vs is_local. The code needs to be refactored so that things like -pc_asm_blocks and -pc_asm_overlap 1 can track the is vs is_local index sets properly when the -pc_asm_type is set. Also the name is_local needs to be changed to something meaningfully like is_nonoverlapping This refactoring would also result in easier cleaner code then is currently there.
So basically until the PCASM is refactored properly to handle restrict etc you are stuck with being able to use the restrict etc ONLY if you specifically supply the overlapping and non overlapping domains yourself with PCASMSetLocalSubdomains and curse at Matt everyday like we all do.
Barry
> On Aug 4, 2016, at 11:57 AM, Boyce Griffith <griffith at cims.nyu.edu> wrote:
>
> With PETSc 3.7.2, changing the ASM type does not seem to have any effect for serial jobs. E.g. using ksp/examples/tutorials/ex2:
>
> $ ./ex2 -m 32 -n 32 -pc_type asm -pc_asm_blocks 8 -ksp_view -ksp_monitor_true_residual -pc_asm_type basic
> 0 KSP preconditioned resid norm 7.227368482718e+00 true resid norm 1.166190378969e+01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 2.680806990415e+00 true resid norm 3.440170689976e+00 ||r(i)||/||b|| 2.949922029898e-01
> 2 KSP preconditioned resid norm 1.537918148738e+00 true resid norm 1.676069837125e+00 ||r(i)||/||b|| 1.437218028335e-01
> 3 KSP preconditioned resid norm 1.025836188311e+00 true resid norm 1.127193636507e+00 ||r(i)||/||b|| 9.665605692131e-02
> 4 KSP preconditioned resid norm 7.465807140977e-01 true resid norm 7.063999312309e-01 ||r(i)||/||b|| 6.057329437543e-02
> 5 KSP preconditioned resid norm 5.806789214514e-01 true resid norm 5.248222587585e-01 ||r(i)||/||b|| 4.500313741419e-02
> 6 KSP preconditioned resid norm 4.709931353737e-01 true resid norm 4.142974151217e-01 ||r(i)||/||b|| 3.552571026079e-02
> 7 KSP preconditioned resid norm 4.123520053531e-01 true resid norm 3.428306005637e-01 ||r(i)||/||b|| 2.939748147011e-02
> 8 KSP preconditioned resid norm 3.623327779165e-01 true resid norm 3.188334125692e-01 ||r(i)||/||b|| 2.733973957589e-02
> 9 KSP preconditioned resid norm 2.416851769562e-01 true resid norm 3.243055253185e-01 ||r(i)||/||b|| 2.780896937301e-02
> 10 KSP preconditioned resid norm 1.215247217085e-01 true resid norm 1.564954190637e-01 ||r(i)||/||b|| 1.341937147536e-02
> 11 KSP preconditioned resid norm 6.497470510985e-02 true resid norm 8.673119491739e-02 ||r(i)||/||b|| 7.437138608026e-03
> 12 KSP preconditioned resid norm 3.246036728276e-02 true resid norm 4.591618222897e-02 ||r(i)||/||b|| 3.937280143707e-03
> 13 KSP preconditioned resid norm 1.435458873790e-02 true resid norm 1.985166297182e-02 ||r(i)||/||b|| 1.702266056197e-03
> 14 KSP preconditioned resid norm 6.960292630481e-03 true resid norm 9.778813102206e-03 ||r(i)||/||b|| 8.385263057007e-04
> 15 KSP preconditioned resid norm 3.604789592169e-03 true resid norm 4.449537275931e-03 ||r(i)||/||b|| 3.815446736805e-04
> 16 KSP preconditioned resid norm 2.347335356674e-03 true resid norm 2.530581623538e-03 ||r(i)||/||b|| 2.169955840122e-04
> 17 KSP preconditioned resid norm 1.606443885426e-03 true resid norm 1.835719869720e-03 ||r(i)||/||b|| 1.574116801875e-04
> 18 KSP preconditioned resid norm 8.685134492648e-04 true resid norm 1.250934036178e-03 ||r(i)||/||b|| 1.072667086555e-04
> 19 KSP preconditioned resid norm 4.160062164697e-04 true resid norm 5.589755065266e-04 ||r(i)||/||b|| 4.793175424931e-05
> 20 KSP preconditioned resid norm 2.264387793673e-04 true resid norm 2.733953335794e-04 ||r(i)||/||b|| 2.344345644671e-05
> 21 KSP preconditioned resid norm 1.572062148327e-04 true resid norm 1.655081111255e-04 ||r(i)||/||b|| 1.419220344382e-05
> 22 KSP preconditioned resid norm 1.046603808940e-04 true resid norm 1.170344120764e-04 ||r(i)||/||b|| 1.003561804204e-05
> 23 KSP preconditioned resid norm 7.323396102053e-05 true resid norm 9.226622743289e-05 ||r(i)||/||b|| 7.911763730589e-06
> 24 KSP preconditioned resid norm 3.961819569402e-05 true resid norm 5.397694170644e-05 ||r(i)||/||b|| 4.628484566487e-06
> KSP Object: 1 MPI processes
> type: gmres
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
> GMRES: happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=9.18274e-06, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: asm
> Additive Schwarz: total subdomain blocks = 8, amount of overlap = 1
> Additive Schwarz: restriction/interpolation type - BASIC
> Local solve is same for all blocks, in the following KSP and PC objects:
> KSP Object: (sub_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: icc
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> using Manteuffel shift [POSITIVE_DEFINITE]
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=160, cols=160
> package used to perform factorization: petsc
> total: nonzeros=443, allocated nonzeros=443
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=160, cols=160
> total: nonzeros=726, allocated nonzeros=726
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=1024, cols=1024
> total: nonzeros=4992, allocated nonzeros=5120
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> Norm of error 0.000445244 iterations 24
> $ ./ex2 -m 32 -n 32 -pc_type asm -pc_asm_blocks 8 -ksp_view -ksp_monitor_true_residual -pc_asm_type restrict
> 0 KSP preconditioned resid norm 7.227368482718e+00 true resid norm 1.166190378969e+01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 2.680806990415e+00 true resid norm 3.440170689976e+00 ||r(i)||/||b|| 2.949922029898e-01
> 2 KSP preconditioned resid norm 1.537918148738e+00 true resid norm 1.676069837125e+00 ||r(i)||/||b|| 1.437218028335e-01
> 3 KSP preconditioned resid norm 1.025836188311e+00 true resid norm 1.127193636507e+00 ||r(i)||/||b|| 9.665605692131e-02
> 4 KSP preconditioned resid norm 7.465807140977e-01 true resid norm 7.063999312309e-01 ||r(i)||/||b|| 6.057329437543e-02
> 5 KSP preconditioned resid norm 5.806789214514e-01 true resid norm 5.248222587585e-01 ||r(i)||/||b|| 4.500313741419e-02
> 6 KSP preconditioned resid norm 4.709931353737e-01 true resid norm 4.142974151217e-01 ||r(i)||/||b|| 3.552571026079e-02
> 7 KSP preconditioned resid norm 4.123520053531e-01 true resid norm 3.428306005637e-01 ||r(i)||/||b|| 2.939748147011e-02
> 8 KSP preconditioned resid norm 3.623327779165e-01 true resid norm 3.188334125692e-01 ||r(i)||/||b|| 2.733973957589e-02
> 9 KSP preconditioned resid norm 2.416851769562e-01 true resid norm 3.243055253185e-01 ||r(i)||/||b|| 2.780896937301e-02
> 10 KSP preconditioned resid norm 1.215247217085e-01 true resid norm 1.564954190637e-01 ||r(i)||/||b|| 1.341937147536e-02
> 11 KSP preconditioned resid norm 6.497470510985e-02 true resid norm 8.673119491739e-02 ||r(i)||/||b|| 7.437138608026e-03
> 12 KSP preconditioned resid norm 3.246036728276e-02 true resid norm 4.591618222897e-02 ||r(i)||/||b|| 3.937280143707e-03
> 13 KSP preconditioned resid norm 1.435458873790e-02 true resid norm 1.985166297182e-02 ||r(i)||/||b|| 1.702266056197e-03
> 14 KSP preconditioned resid norm 6.960292630481e-03 true resid norm 9.778813102206e-03 ||r(i)||/||b|| 8.385263057007e-04
> 15 KSP preconditioned resid norm 3.604789592169e-03 true resid norm 4.449537275931e-03 ||r(i)||/||b|| 3.815446736805e-04
> 16 KSP preconditioned resid norm 2.347335356674e-03 true resid norm 2.530581623538e-03 ||r(i)||/||b|| 2.169955840122e-04
> 17 KSP preconditioned resid norm 1.606443885426e-03 true resid norm 1.835719869720e-03 ||r(i)||/||b|| 1.574116801875e-04
> 18 KSP preconditioned resid norm 8.685134492648e-04 true resid norm 1.250934036178e-03 ||r(i)||/||b|| 1.072667086555e-04
> 19 KSP preconditioned resid norm 4.160062164697e-04 true resid norm 5.589755065266e-04 ||r(i)||/||b|| 4.793175424931e-05
> 20 KSP preconditioned resid norm 2.264387793673e-04 true resid norm 2.733953335794e-04 ||r(i)||/||b|| 2.344345644671e-05
> 21 KSP preconditioned resid norm 1.572062148327e-04 true resid norm 1.655081111255e-04 ||r(i)||/||b|| 1.419220344382e-05
> 22 KSP preconditioned resid norm 1.046603808940e-04 true resid norm 1.170344120764e-04 ||r(i)||/||b|| 1.003561804204e-05
> 23 KSP preconditioned resid norm 7.323396102053e-05 true resid norm 9.226622743289e-05 ||r(i)||/||b|| 7.911763730589e-06
> 24 KSP preconditioned resid norm 3.961819569402e-05 true resid norm 5.397694170644e-05 ||r(i)||/||b|| 4.628484566487e-06
> KSP Object: 1 MPI processes
> type: gmres
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
> GMRES: happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=9.18274e-06, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: asm
> Additive Schwarz: total subdomain blocks = 8, amount of overlap = 1
> Additive Schwarz: restriction/interpolation type - RESTRICT
> Local solve is same for all blocks, in the following KSP and PC objects:
> KSP Object: (sub_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: icc
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> using Manteuffel shift [POSITIVE_DEFINITE]
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=160, cols=160
> package used to perform factorization: petsc
> total: nonzeros=443, allocated nonzeros=443
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=160, cols=160
> total: nonzeros=726, allocated nonzeros=726
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=1024, cols=1024
> total: nonzeros=4992, allocated nonzeros=5120
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> Norm of error 0.000445244 iterations 24
> $ ./ex2 -m 32 -n 32 -pc_type asm -pc_asm_blocks 8 -ksp_view -ksp_monitor_true_residual -pc_asm_type interpolate
> 0 KSP preconditioned resid norm 7.227368482718e+00 true resid norm 1.166190378969e+01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 2.680806990415e+00 true resid norm 3.440170689976e+00 ||r(i)||/||b|| 2.949922029898e-01
> 2 KSP preconditioned resid norm 1.537918148738e+00 true resid norm 1.676069837125e+00 ||r(i)||/||b|| 1.437218028335e-01
> 3 KSP preconditioned resid norm 1.025836188311e+00 true resid norm 1.127193636507e+00 ||r(i)||/||b|| 9.665605692131e-02
> 4 KSP preconditioned resid norm 7.465807140977e-01 true resid norm 7.063999312309e-01 ||r(i)||/||b|| 6.057329437543e-02
> 5 KSP preconditioned resid norm 5.806789214514e-01 true resid norm 5.248222587585e-01 ||r(i)||/||b|| 4.500313741419e-02
> 6 KSP preconditioned resid norm 4.709931353737e-01 true resid norm 4.142974151217e-01 ||r(i)||/||b|| 3.552571026079e-02
> 7 KSP preconditioned resid norm 4.123520053531e-01 true resid norm 3.428306005637e-01 ||r(i)||/||b|| 2.939748147011e-02
> 8 KSP preconditioned resid norm 3.623327779165e-01 true resid norm 3.188334125692e-01 ||r(i)||/||b|| 2.733973957589e-02
> 9 KSP preconditioned resid norm 2.416851769562e-01 true resid norm 3.243055253185e-01 ||r(i)||/||b|| 2.780896937301e-02
> 10 KSP preconditioned resid norm 1.215247217085e-01 true resid norm 1.564954190637e-01 ||r(i)||/||b|| 1.341937147536e-02
> 11 KSP preconditioned resid norm 6.497470510985e-02 true resid norm 8.673119491739e-02 ||r(i)||/||b|| 7.437138608026e-03
> 12 KSP preconditioned resid norm 3.246036728276e-02 true resid norm 4.591618222897e-02 ||r(i)||/||b|| 3.937280143707e-03
> 13 KSP preconditioned resid norm 1.435458873790e-02 true resid norm 1.985166297182e-02 ||r(i)||/||b|| 1.702266056197e-03
> 14 KSP preconditioned resid norm 6.960292630481e-03 true resid norm 9.778813102206e-03 ||r(i)||/||b|| 8.385263057007e-04
> 15 KSP preconditioned resid norm 3.604789592169e-03 true resid norm 4.449537275931e-03 ||r(i)||/||b|| 3.815446736805e-04
> 16 KSP preconditioned resid norm 2.347335356674e-03 true resid norm 2.530581623538e-03 ||r(i)||/||b|| 2.169955840122e-04
> 17 KSP preconditioned resid norm 1.606443885426e-03 true resid norm 1.835719869720e-03 ||r(i)||/||b|| 1.574116801875e-04
> 18 KSP preconditioned resid norm 8.685134492648e-04 true resid norm 1.250934036178e-03 ||r(i)||/||b|| 1.072667086555e-04
> 19 KSP preconditioned resid norm 4.160062164697e-04 true resid norm 5.589755065266e-04 ||r(i)||/||b|| 4.793175424931e-05
> 20 KSP preconditioned resid norm 2.264387793673e-04 true resid norm 2.733953335794e-04 ||r(i)||/||b|| 2.344345644671e-05
> 21 KSP preconditioned resid norm 1.572062148327e-04 true resid norm 1.655081111255e-04 ||r(i)||/||b|| 1.419220344382e-05
> 22 KSP preconditioned resid norm 1.046603808940e-04 true resid norm 1.170344120764e-04 ||r(i)||/||b|| 1.003561804204e-05
> 23 KSP preconditioned resid norm 7.323396102053e-05 true resid norm 9.226622743289e-05 ||r(i)||/||b|| 7.911763730589e-06
> 24 KSP preconditioned resid norm 3.961819569402e-05 true resid norm 5.397694170644e-05 ||r(i)||/||b|| 4.628484566487e-06
> KSP Object: 1 MPI processes
> type: gmres
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
> GMRES: happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=9.18274e-06, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: asm
> Additive Schwarz: total subdomain blocks = 8, amount of overlap = 1
> Additive Schwarz: restriction/interpolation type - INTERPOLATE
> Local solve is same for all blocks, in the following KSP and PC objects:
> KSP Object: (sub_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: icc
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> using Manteuffel shift [POSITIVE_DEFINITE]
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=160, cols=160
> package used to perform factorization: petsc
> total: nonzeros=443, allocated nonzeros=443
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=160, cols=160
> total: nonzeros=726, allocated nonzeros=726
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=1024, cols=1024
> total: nonzeros=4992, allocated nonzeros=5120
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> Norm of error 0.000445244 iterations 24
> $ ./ex2 -m 32 -n 32 -pc_type asm -pc_asm_blocks 8 -ksp_view -ksp_monitor_true_residual -pc_asm_type none
> 0 KSP preconditioned resid norm 7.227368482718e+00 true resid norm 1.166190378969e+01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 2.680806990415e+00 true resid norm 3.440170689976e+00 ||r(i)||/||b|| 2.949922029898e-01
> 2 KSP preconditioned resid norm 1.537918148738e+00 true resid norm 1.676069837125e+00 ||r(i)||/||b|| 1.437218028335e-01
> 3 KSP preconditioned resid norm 1.025836188311e+00 true resid norm 1.127193636507e+00 ||r(i)||/||b|| 9.665605692131e-02
> 4 KSP preconditioned resid norm 7.465807140977e-01 true resid norm 7.063999312309e-01 ||r(i)||/||b|| 6.057329437543e-02
> 5 KSP preconditioned resid norm 5.806789214514e-01 true resid norm 5.248222587585e-01 ||r(i)||/||b|| 4.500313741419e-02
> 6 KSP preconditioned resid norm 4.709931353737e-01 true resid norm 4.142974151217e-01 ||r(i)||/||b|| 3.552571026079e-02
> 7 KSP preconditioned resid norm 4.123520053531e-01 true resid norm 3.428306005637e-01 ||r(i)||/||b|| 2.939748147011e-02
> 8 KSP preconditioned resid norm 3.623327779165e-01 true resid norm 3.188334125692e-01 ||r(i)||/||b|| 2.733973957589e-02
> 9 KSP preconditioned resid norm 2.416851769562e-01 true resid norm 3.243055253185e-01 ||r(i)||/||b|| 2.780896937301e-02
> 10 KSP preconditioned resid norm 1.215247217085e-01 true resid norm 1.564954190637e-01 ||r(i)||/||b|| 1.341937147536e-02
> 11 KSP preconditioned resid norm 6.497470510985e-02 true resid norm 8.673119491739e-02 ||r(i)||/||b|| 7.437138608026e-03
> 12 KSP preconditioned resid norm 3.246036728276e-02 true resid norm 4.591618222897e-02 ||r(i)||/||b|| 3.937280143707e-03
> 13 KSP preconditioned resid norm 1.435458873790e-02 true resid norm 1.985166297182e-02 ||r(i)||/||b|| 1.702266056197e-03
> 14 KSP preconditioned resid norm 6.960292630481e-03 true resid norm 9.778813102206e-03 ||r(i)||/||b|| 8.385263057007e-04
> 15 KSP preconditioned resid norm 3.604789592169e-03 true resid norm 4.449537275931e-03 ||r(i)||/||b|| 3.815446736805e-04
> 16 KSP preconditioned resid norm 2.347335356674e-03 true resid norm 2.530581623538e-03 ||r(i)||/||b|| 2.169955840122e-04
> 17 KSP preconditioned resid norm 1.606443885426e-03 true resid norm 1.835719869720e-03 ||r(i)||/||b|| 1.574116801875e-04
> 18 KSP preconditioned resid norm 8.685134492648e-04 true resid norm 1.250934036178e-03 ||r(i)||/||b|| 1.072667086555e-04
> 19 KSP preconditioned resid norm 4.160062164697e-04 true resid norm 5.589755065266e-04 ||r(i)||/||b|| 4.793175424931e-05
> 20 KSP preconditioned resid norm 2.264387793673e-04 true resid norm 2.733953335794e-04 ||r(i)||/||b|| 2.344345644671e-05
> 21 KSP preconditioned resid norm 1.572062148327e-04 true resid norm 1.655081111255e-04 ||r(i)||/||b|| 1.419220344382e-05
> 22 KSP preconditioned resid norm 1.046603808940e-04 true resid norm 1.170344120764e-04 ||r(i)||/||b|| 1.003561804204e-05
> 23 KSP preconditioned resid norm 7.323396102053e-05 true resid norm 9.226622743289e-05 ||r(i)||/||b|| 7.911763730589e-06
> 24 KSP preconditioned resid norm 3.961819569402e-05 true resid norm 5.397694170644e-05 ||r(i)||/||b|| 4.628484566487e-06
> KSP Object: 1 MPI processes
> type: gmres
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
> GMRES: happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=9.18274e-06, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: asm
> Additive Schwarz: total subdomain blocks = 8, amount of overlap = 1
> Additive Schwarz: restriction/interpolation type - NONE
> Local solve is same for all blocks, in the following KSP and PC objects:
> KSP Object: (sub_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: icc
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> using Manteuffel shift [POSITIVE_DEFINITE]
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=160, cols=160
> package used to perform factorization: petsc
> total: nonzeros=443, allocated nonzeros=443
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=160, cols=160
> total: nonzeros=726, allocated nonzeros=726
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=1024, cols=1024
> total: nonzeros=4992, allocated nonzeros=5120
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> Norm of error 0.000445244 iterations 24
>
> Is this setting meant only to affect how communication between processors is handled?
>
> Also, the MULTIPLICATIVE variant does not seem to behave as I would expect --- for this same example, if you switch from ADDITIVE to MULTIPLICATIVE, the solver converges slightly more slowly:
>
> $ ./ex2 -m 32 -n 32 -pc_type asm -pc_asm_blocks 8 -ksp_view -ksp_monitor_true_residual -pc_asm_local_type MULTIPLICATIVE
> 0 KSP preconditioned resid norm 7.467363913958e+00 true resid norm 1.166190378969e+01 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP preconditioned resid norm 2.878371937592e+00 true resid norm 3.646367718253e+00 ||r(i)||/||b|| 3.126734522949e-01
> 2 KSP preconditioned resid norm 1.666575161021e+00 true resid norm 1.940699059619e+00 ||r(i)||/||b|| 1.664135714560e-01
> 3 KSP preconditioned resid norm 1.086140238220e+00 true resid norm 1.191473615464e+00 ||r(i)||/||b|| 1.021680196433e-01
> 4 KSP preconditioned resid norm 7.939217314942e-01 true resid norm 8.059317628307e-01 ||r(i)||/||b|| 6.910807852344e-02
> 5 KSP preconditioned resid norm 6.265169154675e-01 true resid norm 5.942294290555e-01 ||r(i)||/||b|| 5.095475316653e-02
> 6 KSP preconditioned resid norm 5.164999302721e-01 true resid norm 4.585844476718e-01 ||r(i)||/||b|| 3.932329197203e-02
> 7 KSP preconditioned resid norm 4.472399844370e-01 true resid norm 3.884049472908e-01 ||r(i)||/||b|| 3.330544946136e-02
> 8 KSP preconditioned resid norm 3.445446366213e-01 true resid norm 4.008290378967e-01 ||r(i)||/||b|| 3.437080644166e-02
> 9 KSP preconditioned resid norm 1.987509894375e-01 true resid norm 2.619628925380e-01 ||r(i)||/||b|| 2.246313271505e-02
> 10 KSP preconditioned resid norm 1.084551743751e-01 true resid norm 1.354891040098e-01 ||r(i)||/||b|| 1.161809481995e-02
> 11 KSP preconditioned resid norm 6.108303419460e-02 true resid norm 7.252267103275e-02 ||r(i)||/||b|| 6.218767736436e-03
> 12 KSP preconditioned resid norm 3.641579250431e-02 true resid norm 4.069996187932e-02 ||r(i)||/||b|| 3.489992938829e-03
> 13 KSP preconditioned resid norm 2.424898818735e-02 true resid norm 2.469590201945e-02 ||r(i)||/||b|| 2.117656127577e-03
> 14 KSP preconditioned resid norm 1.792399391125e-02 true resid norm 1.622090905110e-02 ||r(i)||/||b|| 1.390931475995e-03
> 15 KSP preconditioned resid norm 1.320657155648e-02 true resid norm 1.336753101147e-02 ||r(i)||/||b|| 1.146256327657e-03
> 16 KSP preconditioned resid norm 7.398524571182e-03 true resid norm 9.747691680405e-03 ||r(i)||/||b|| 8.358576657974e-04
> 17 KSP preconditioned resid norm 3.043993613039e-03 true resid norm 3.848714422908e-03 ||r(i)||/||b|| 3.300245390731e-04
> 18 KSP preconditioned resid norm 1.767867968946e-03 true resid norm 1.736586340170e-03 ||r(i)||/||b|| 1.489110501585e-04
> 19 KSP preconditioned resid norm 1.088792656005e-03 true resid norm 1.307506936484e-03 ||r(i)||/||b|| 1.121177948355e-04
> 20 KSP preconditioned resid norm 4.622653682144e-04 true resid norm 5.718427718734e-04 ||r(i)||/||b|| 4.903511315013e-05
> 21 KSP preconditioned resid norm 2.591703287585e-04 true resid norm 2.690982547548e-04 ||r(i)||/||b|| 2.307498497738e-05
> 22 KSP preconditioned resid norm 1.596527181997e-04 true resid norm 1.715846687846e-04 ||r(i)||/||b|| 1.471326396435e-05
> 23 KSP preconditioned resid norm 1.006766623019e-04 true resid norm 1.044525361282e-04 ||r(i)||/||b|| 8.956731080268e-06
> 24 KSP preconditioned resid norm 5.349814270060e-05 true resid norm 6.598682341705e-05 ||r(i)||/||b|| 5.658323427037e-06
> KSP Object: 1 MPI processes
> type: gmres
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
> GMRES: happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=9.18274e-06, absolute=1e-50, divergence=10000.
> left preconditioning
> using PRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: asm
> Additive Schwarz: total subdomain blocks = 8, amount of overlap = 1
> Additive Schwarz: restriction/interpolation type - BASIC
> Additive Schwarz: local solve composition type - MULTIPLICATIVE
> Local solve is same for all blocks, in the following KSP and PC objects:
> KSP Object: (sub_) 1 MPI processes
> type: preonly
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
> left preconditioning
> using NONE norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: icc
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> using Manteuffel shift [POSITIVE_DEFINITE]
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqsbaij
> rows=160, cols=160
> package used to perform factorization: petsc
> total: nonzeros=443, allocated nonzeros=443
> total number of mallocs used during MatSetValues calls =0
> block size is 1
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=160, cols=160
> total: nonzeros=726, allocated nonzeros=726
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=1024, cols=1024
> total: nonzeros=4992, allocated nonzeros=5120
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> Norm of error 0.000292304 iterations 24
>
> Thanks,
>
> -- Boyce
>
More information about the petsc-users
mailing list