[petsc-users] ASM vs GASM
Mark Adams
mfadams at lbl.gov
Tue Apr 30 11:31:05 CDT 2019
When I said it was singular I was looking at "preconditioned residual norm
to an rtol of 1e-12. If I look at the true residual norm, however, it
stagnates around 1e-4."
This is not what I am seeing in this output. It is just a poor PC. The big
drop in the residual at the beginning is suspicious. When you solve this
problem well have you checked that you are getting a good solution? That
is, do you check that your model is not messed up, like a bad mesh.
On Tue, Apr 30, 2019 at 11:35 AM Boyce Griffith via petsc-users <
petsc-users at mcs.anl.gov> wrote:
>
>
> On Apr 30, 2019, at 11:23 AM, Fande Kong <fdkong.jd at gmail.com> wrote:
>
>
>
> On Tue, Apr 30, 2019 at 7:40 AM Boyce Griffith via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>>
>>
>> On Apr 30, 2019, at 9:06 AM, Mark Adams <mfadams at lbl.gov> wrote:
>>
>>
>>>
>>>
>>> Allowing GASM to construct the "outer" subdomains from the
>>> non-overlapping "inner" subdomains, and using "exact" subdomain solvers
>>> (subdomain KSPs are using FGMRES+ILU with an rtol of 1e-12), I get
>>> convergence in ~2 iterations in the preconditioned residual norm to an rtol
>>> of 1e-12. If I look at the true residual norm, however, it stagnates around
>>> 1e-4.
>>>
>>>
>> That PC is singular.
>>
>>
>> Yes. I am confused about why GASM is giving a singular PC but ASM is not
>> with the same subdomains.
>>
>
> We could not tell much without the info of detailed solver setup. The
> overlapping function was implemented by me and Dmitry a couple of years
> ago, and it is trick when a subdomain is shared by multiple cores. Do you
> mind to start from our example? Or setup an example for us to demonstrate
> your issues?
>
> At least please use "-ksp_view" (linear solver) or "-snes_view" (nonlinear
> solver) to print more information that will help us a bit.
>
>
> Here you go:
>
> $ ./main -d 3 -n 10 -ksp_type fgmres -sub_ksp_type fgmres -sub_pc_type ilu
> -sub_ksp_rtol 1.0e-12 -ksp_converged_reason -ksp_monitor_true_residual
> -ksp_max_it 10 -ksp_view
> make: `main' is up to date.
> Running ./main -d 3 -n 10 -ksp_type fgmres -sub_ksp_type fgmres
> -sub_pc_type ilu -sub_ksp_rtol 1.0e-12 -ksp_converged_reason
> -ksp_monitor_true_residual -ksp_max_it 10 -ksp_view
>
> 0 KSP unpreconditioned resid norm 5.249964981356e+02 true resid norm
> 5.249964981356e+02 ||r(i)||/||b|| 1.000000000000e+00
> 1 KSP unpreconditioned resid norm 9.316296223724e-02 true resid norm
> 9.316296223724e-02 ||r(i)||/||b|| 1.774544450641e-04
> 2 KSP unpreconditioned resid norm 9.314881028141e-02 true resid norm
> 9.314881028141e-02 ||r(i)||/||b|| 1.774274887779e-04
> 3 KSP unpreconditioned resid norm 9.299990517422e-02 true resid norm
> 9.299918556770e-02 ||r(i)||/||b|| 1.771424874222e-04
> 4 KSP unpreconditioned resid norm 9.224468272306e-02 true resid norm
> 9.393543403858e-02 ||r(i)||/||b|| 1.789258297382e-04
> 5 KSP unpreconditioned resid norm 9.150828598034e-02 true resid norm
> 9.511673987375e-02 ||r(i)||/||b|| 1.811759510997e-04
> 6 KSP unpreconditioned resid norm 9.078924839691e-02 true resid norm
> 1.013093335976e-01 ||r(i)||/||b|| 1.929714463951e-04
> 7 KSP unpreconditioned resid norm 9.008689850931e-02 true resid norm
> 1.011099594157e-01 ||r(i)||/||b|| 1.925916835155e-04
> 8 KSP unpreconditioned resid norm 8.940060065590e-02 true resid norm
> 1.090779251949e-01 ||r(i)||/||b|| 2.077688624253e-04
> 9 KSP unpreconditioned resid norm 8.872975256529e-02 true resid norm
> 1.102873098599e-01 ||r(i)||/||b|| 2.100724676289e-04
> 10 KSP unpreconditioned resid norm 8.807378313465e-02 true resid norm
> 1.071996745064e-01 ||r(i)||/||b|| 2.041912182026e-04
> Linear solve did not converge due to DIVERGED_ITS iterations 10
> KSP Object: 1 MPI processes
> type: fgmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10, nonzero initial guess
> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
> right preconditioning
> using UNPRECONDITIONED norm type for convergence test
> PC Object: 1 MPI processes
> type: gasm
> Restriction/interpolation type: RESTRICT
> requested amount of overlap = 0
> total number of subdomains = 2
> max number of local subdomains = 2
> [0|1] number of locally-supported subdomains = 2
> Subdomain solver info is as follows:
> - - - - - - - - - - - - - - - - - -
> [0|1] (subcomm [0|1]) local subdomain number 0, local size = 484
> KSP Object: (sub_) 1 MPI processes
> type: fgmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
> right preconditioning
> using UNPRECONDITIONED norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: ilu
> out-of-place factorization
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=484, cols=484
> package used to perform factorization: petsc
> total: nonzeros=15376, allocated nonzeros=15376
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: () 1 MPI processes
> type: seqaij
> rows=484, cols=484
> total: nonzeros=15376, allocated nonzeros=15376
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> - - - - - - - - - - - - - - - - - -
> [0|1] (subcomm [0|1]) local subdomain number 1, local size = 121
> KSP Object: (sub_) 1 MPI processes
> type: fgmres
> restart=30, using Classical (unmodified) Gram-Schmidt
> Orthogonalization with no iterative refinement
> happy breakdown tolerance 1e-30
> maximum iterations=10000, initial guess is zero
> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.
> right preconditioning
> using UNPRECONDITIONED norm type for convergence test
> PC Object: (sub_) 1 MPI processes
> type: ilu
> out-of-place factorization
> 0 levels of fill
> tolerance for zero pivot 2.22045e-14
> matrix ordering: natural
> factor fill ratio given 1., needed 1.
> Factored matrix follows:
> Mat Object: 1 MPI processes
> type: seqaij
> rows=121, cols=121
> package used to perform factorization: petsc
> total: nonzeros=961, allocated nonzeros=961
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> linear system matrix = precond matrix:
> Mat Object: () 1 MPI processes
> type: seqaij
> rows=121, cols=121
> total: nonzeros=961, allocated nonzeros=961
> total number of mallocs used during MatSetValues calls =0
> not using I-node routines
> - - - - - - - - - - - - - - - - - -
> *[0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------*
> [0]PETSC ERROR: Petsc has generated inconsistent data
> [0]PETSC ERROR: Called more times than PetscViewerASCIIPushSynchronized()
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [0]PETSC ERROR: Petsc Release Version 3.11.1, unknown
> [0]PETSC ERROR: ./main on a darwin-dbg named boycesimacwork.dhcp.unc.edu
> by boyceg Tue Apr 30 11:29:02 2019
> [0]PETSC ERROR: Configure options --CC=mpicc --CXX=mpicxx --FC=mpif90
> --PETSC_ARCH=darwin-dbg --with-debugging=1 --with-c++-support=1
> --with-hypre=1 --download-hypre=1 --with-hdf5=1 --with-hdf5-dir=/usr/local
> [0]PETSC ERROR: #1 PetscViewerASCIIPopSynchronized() line 438 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/sys/classes/viewer/impls/ascii/filev.c
> [0]PETSC ERROR: #2 PCView_GASM() line 251 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/pc/impls/gasm/gasm.c
> [0]PETSC ERROR: #3 PCView() line 1651 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/pc/interface/precon.c
> [0]PETSC ERROR: #4 KSPView() line 213 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itcreate.c
> [0]PETSC ERROR: #5 PetscObjectView() line 100 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/sys/objects/destroy.c
> [0]PETSC ERROR: #6 ObjectView() line 14 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: #7 KSPSolve() line 831 in
> /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itfunc.c
>
> One difference between them is that the default GASM overlap is 0 (
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGASMSetOverlap.html),
>> but the default ASM overlap is 1 (
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCASMSetOverlap.html
>> ).
>>
>
> For your particular application, not sure you need any overlap since there
> are two different PDEs on two different subdomains. It may be fine to run
> the PC without the overlapping domain. It is a definitely interesting
> application for GASM.
>
> Fande,
>
>
>
>> However, changing the GASM overlap does not make any difference in the
>> convergence history.
>>
>> -- Boyce
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190430/e505c820/attachment-0001.html>
More information about the petsc-users
mailing list