<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Apr 30, 2019, at 11:23 AM, Fande Kong <<a href="mailto:fdkong.jd@gmail.com" class="">fdkong.jd@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div dir="ltr" class="gmail_attr">On Tue, Apr 30, 2019 at 7:40 AM Boyce Griffith via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Apr 30, 2019, at 9:06 AM, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>> wrote:</div><br class="gmail-m_1664224320951973972Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class=""><br class=""><br class="">Allowing GASM to construct the "outer" subdomains from the non-overlapping "inner" subdomains, and using "exact" subdomain solvers (subdomain KSPs are using FGMRES+ILU with an rtol of 1e-12), I get convergence in ~2 iterations in the preconditioned residual norm to an rtol of 1e-12. If I look at the true residual norm, however, it stagnates around 1e-4.<br class=""><br class=""></blockquote><div class=""><br class=""></div><div class="">That PC is singular. </div></div></div></div></blockquote></div><div class=""><br class=""></div>Yes. I am confused about why GASM is giving a singular PC but ASM is not with the same subdomains.</div></blockquote><div class=""><br class=""></div><div class="">We could not tell much without the info of detailed solver setup. The overlapping function was implemented by me and Dmitry a couple of years ago, and it is trick when a subdomain is shared by multiple cores. Do you mind to start from our example? Or setup an example for us to demonstrate your issues? </div><div class=""><br class=""></div><div class="">At least please use "-ksp_view" (linear solver) or "-snes_view" (nonlinear solver) to print more information that will help us a bit.</div></div></div></blockquote><div><br class=""></div>Here you go:</div><div><br class=""></div><div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">$ ./main -d 3 -n 10 -ksp_type fgmres -sub_ksp_type fgmres -sub_pc_type ilu -sub_ksp_rtol 1.0e-12 -ksp_converged_reason -ksp_monitor_true_residual -ksp_max_it 10 -ksp_view</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">make: `main' is up to date.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Running ./main -d 3 -n 10 -ksp_type fgmres -sub_ksp_type fgmres -sub_pc_type ilu -sub_ksp_rtol 1.0e-12 -ksp_converged_reason -ksp_monitor_true_residual -ksp_max_it 10 -ksp_view</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098); min-height: 15px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 KSP unpreconditioned resid norm 5.249964981356e+02 true resid norm 5.249964981356e+02 ||r(i)||/||b|| 1.000000000000e+00</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 1 KSP unpreconditioned resid norm 9.316296223724e-02 true resid norm 9.316296223724e-02 ||r(i)||/||b|| 1.774544450641e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 2 KSP unpreconditioned resid norm 9.314881028141e-02 true resid norm 9.314881028141e-02 ||r(i)||/||b|| 1.774274887779e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 3 KSP unpreconditioned resid norm 9.299990517422e-02 true resid norm 9.299918556770e-02 ||r(i)||/||b|| 1.771424874222e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 4 KSP unpreconditioned resid norm 9.224468272306e-02 true resid norm 9.393543403858e-02 ||r(i)||/||b|| 1.789258297382e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 5 KSP unpreconditioned resid norm 9.150828598034e-02 true resid norm 9.511673987375e-02 ||r(i)||/||b|| 1.811759510997e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 6 KSP unpreconditioned resid norm 9.078924839691e-02 true resid norm 1.013093335976e-01 ||r(i)||/||b|| 1.929714463951e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 7 KSP unpreconditioned resid norm 9.008689850931e-02 true resid norm 1.011099594157e-01 ||r(i)||/||b|| 1.925916835155e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 8 KSP unpreconditioned resid norm 8.940060065590e-02 true resid norm 1.090779251949e-01 ||r(i)||/||b|| 2.077688624253e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 9 KSP unpreconditioned resid norm 8.872975256529e-02 true resid norm 1.102873098599e-01 ||r(i)||/||b|| 2.100724676289e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 10 KSP unpreconditioned resid norm 8.807378313465e-02 true resid norm 1.071996745064e-01 ||r(i)||/||b|| 2.041912182026e-04</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Linear solve did not converge due to DIVERGED_ITS iterations 10</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">KSP Object: 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: fgmres</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> happy breakdown tolerance 1e-30</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> maximum iterations=10, nonzero initial guess</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> right preconditioning</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> using UNPRECONDITIONED norm type for convergence test</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">PC Object: 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: gasm</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Restriction/interpolation type: RESTRICT</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> requested amount of overlap = 0</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total number of subdomains = 2</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> max number of local subdomains = 2</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> [0|1] number of locally-supported subdomains = 2</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Subdomain solver info is as follows:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> - - - - - - - - - - - - - - - - - -</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> [0|1] (subcomm [0|1]) local subdomain number 0, local size = 484</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> KSP Object: (sub_) 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: fgmres</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> happy breakdown tolerance 1e-30</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> maximum iterations=10000, initial guess is zero</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> right preconditioning</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> using UNPRECONDITIONED norm type for convergence test</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> PC Object: (sub_) 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: ilu</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> out-of-place factorization</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 levels of fill</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> tolerance for zero pivot 2.22045e-14</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> matrix ordering: natural</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> factor fill ratio given 1., needed 1.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Factored matrix follows:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Mat Object: 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: seqaij</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> rows=484, cols=484</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> package used to perform factorization: petsc</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total: nonzeros=15376, allocated nonzeros=15376</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total number of mallocs used during MatSetValues calls =0</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> not using I-node routines</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> linear system matrix = precond matrix:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Mat Object: () 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: seqaij</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> rows=484, cols=484</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total: nonzeros=15376, allocated nonzeros=15376</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total number of mallocs used during MatSetValues calls =0</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> not using I-node routines</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> - - - - - - - - - - - - - - - - - -</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> [0|1] (subcomm [0|1]) local subdomain number 1, local size = 121</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> KSP Object: (sub_) 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: fgmres</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> happy breakdown tolerance 1e-30</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> maximum iterations=10000, initial guess is zero</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> right preconditioning</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> using UNPRECONDITIONED norm type for convergence test</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> PC Object: (sub_) 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: ilu</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> out-of-place factorization</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> 0 levels of fill</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> tolerance for zero pivot 2.22045e-14</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> matrix ordering: natural</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> factor fill ratio given 1., needed 1.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Factored matrix follows:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Mat Object: 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: seqaij</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> rows=121, cols=121</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> package used to perform factorization: petsc</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total: nonzeros=961, allocated nonzeros=961</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total number of mallocs used during MatSetValues calls =0</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> not using I-node routines</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> linear system matrix = precond matrix:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> Mat Object: () 1 MPI processes</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> type: seqaij</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> rows=121, cols=121</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total: nonzeros=961, allocated nonzeros=961</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> total number of mallocs used during MatSetValues calls =0</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> not using I-node routines</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> - - - - - - - - - - - - - - - - - -</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(255, 59, 30); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><b class="">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</b></span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: Petsc has generated inconsistent data</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: Called more times than PetscViewerASCIIPushSynchronized()</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" class="">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: Petsc Release Version 3.11.1, unknown </span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: ./main on a darwin-dbg named <a href="http://boycesimacwork.dhcp.unc.edu" class="">boycesimacwork.dhcp.unc.edu</a> by boyceg Tue Apr 30 11:29:02 2019</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: Configure options --CC=mpicc --CXX=mpicxx --FC=mpif90 --PETSC_ARCH=darwin-dbg --with-debugging=1 --with-c++-support=1 --with-hypre=1 --download-hypre=1 --with-hdf5=1 --with-hdf5-dir=/usr/local</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #1 PetscViewerASCIIPopSynchronized() line 438 in /Users/boyceg/sfw/petsc/petsc-maint/src/sys/classes/viewer/impls/ascii/filev.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #2 PCView_GASM() line 251 in /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/pc/impls/gasm/gasm.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #3 PCView() line 1651 in /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/pc/interface/precon.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #4 KSPView() line 213 in /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itcreate.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #5 PetscObjectView() line 100 in /Users/boyceg/sfw/petsc/petsc-maint/src/sys/objects/destroy.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #6 ObjectView() line 14 in /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itfunc.c</span></div><div style="margin: 0px; font-stretch: normal; font-size: 13px; line-height: normal; font-family: Menlo; color: rgb(244, 244, 244); background-color: rgba(0, 0, 0, 0.85098);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[0]PETSC ERROR: #7 KSPSolve() line 831 in /Users/boyceg/sfw/petsc/petsc-maint/src/ksp/ksp/interface/itfunc.c</span></div><div class=""><br class=""></div><blockquote type="cite" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><div class="">One difference between them is that the default GASM overlap is 0 (<a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGASMSetOverlap.html" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCGASMSetOverlap.html</a>), but the default ASM overlap is 1 (<a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCASMSetOverlap.html" target="_blank" class="">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCASMSetOverlap.html</a>).</div></div></blockquote><div class=""><br class=""></div><div class="">For your particular application, not sure you need any overlap since there are two different PDEs on two different subdomains. It may be fine to run the PC without the overlapping domain. It is a definitely interesting application for GASM. </div><div class=""><br class=""></div><div class="">Fande,</div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><div class="">However, changing the GASM overlap does not make any difference in the convergence history.</div><div class=""><div class=""><br class=""></div><div class="">-- Boyce</div></div></div></blockquote></div></blockquote></div><br class=""></body></html>