<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 9, 2017 at 2:19 PM, David Nolte <span dir="ltr"><<a href="mailto:dnolte@dim.uchile.cl" target="_blank">dnolte@dim.uchile.cl</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
Hi Mark,<br>
<br>
thanks for clarifying.<br>
When I wrote the initial question I had somehow overlooked the fact
that the GAMG standard smoother was Chebychev while ML uses SOR. All
the other comments concerning threshold etc were based on this
mistake.<br>
<br>
The following settings work quite well, of course LU is used on the
coarse level.<span class="gmail-"><br>
<br>
-pc_type gamg<br>
-pc_gamg_type agg<br>
-pc_gamg_threshold 0.03<br></span>
-pc_gamg_square_graph 10 # no effect ?<span class="gmail-"><br>
-pc_gamg_sym_graph<br>
-mg_levels_ksp_type richardson<br>
-mg_levels_pc_type sor<br>
<br></span>
-pc_gamg_agg_nsmooths 0 does not seem to improve the convergence.<br></div></blockquote><div><br></div><div>Looks reasonable. And this smoothing is good for elliptic operators convergence but it makes the operator more expensive. It's worth doing for elliptic operators but in my experience not for others. If you convergence rate does not change then you probably want -pc_gamg_agg_nsmooths 0. This is a cheaper (if smoothing does not help convergence a lot), simpler method and want to use it.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<br>
The ksp view now looks like this: (does this seem reasonable?)<br>
<br>
<br>
KSP Object: 4 MPI processes<span class="gmail-"><br>
type: fgmres<br>
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt
Orthogonalization with no iterative refinement<br>
GMRES: happy breakdown tolerance 1e-30<br>
maximum iterations=10000<br>
tolerances: relative=1e-06, absolute=1e-50, divergence=10000.<br>
right preconditioning<br>
using nonzero initial guess<br>
using UNPRECONDITIONED norm type for convergence test<br></span>
PC Object: 4 MPI processes<br>
type: gamg<br>
MG: type is MULTIPLICATIVE, levels=5 cycles=v<span class="gmail-"><br>
Cycles per PCApply=1<br>
Using Galerkin computed coarse grid matrices<br>
GAMG specific options<br></span>
Threshold for dropping small values from graph 0.03<br>
AGG specific options<br>
Symmetric graph true<span class="gmail-"><br>
Coarse grid solver -- level ------------------------------<wbr>-<br></span>
KSP Object: (mg_coarse_) 4 MPI processes<span class="gmail-"><br>
type: preonly<br>
maximum iterations=10000, initial guess is zero<br>
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
left preconditioning<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_coarse_) 4 MPI processes<br>
type: bjacobi<br>
block Jacobi: number of blocks = 4<br>
Local solve is same for all blocks, in the following KSP and
PC objects:<br>
KSP Object: (mg_coarse_sub_) 1 MPI processes<br>
type: preonly<br>
maximum iterations=1, initial guess is zero<span class="gmail-"><br>
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000.<br>
left preconditioning<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_coarse_sub_) 1 MPI processes<span class="gmail-"><br>
type: lu<br>
LU: out-of-place factorization<br>
tolerance for zero pivot 2.22045e-14<br>
using diagonal shift on blocks to prevent zero pivot
[INBLOCKS]<br>
matrix ordering: nd<br></span>
factor fill ratio given 5., needed 1.<span class="gmail-"><br>
Factored matrix follows:<br>
Mat Object: 1 MPI processes<br>
type: seqaij<br></span>
rows=38, cols=38<span class="gmail-"><br>
package used to perform factorization: petsc<br></span>
total: nonzeros=1444, allocated nonzeros=1444<span class="gmail-"><br>
total number of mallocs used during MatSetValues
calls =0<br></span>
using I-node routines: found 8 nodes, limit used
is 5<span class="gmail-"><br>
linear system matrix = precond matrix:<br>
Mat Object: 1 MPI processes<br>
type: seqaij<br></span>
rows=38, cols=38<br>
total: nonzeros=1444, allocated nonzeros=1444<span class="gmail-"><br>
total number of mallocs used during MatSetValues calls =0<br></span>
using I-node routines: found 8 nodes, limit used is 5<span class="gmail-"><br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<br>
type: mpiaij<br>
rows=38, cols=38<br>
total: nonzeros=1444, allocated nonzeros=1444<span class="gmail-"><br>
total number of mallocs used during MatSetValues calls =0<br></span>
using I-node (on process 0) routines: found 8 nodes, limit
used is 5<span class="gmail-"><br>
Down solver (pre-smoother) on level 1
------------------------------<wbr>-<br></span>
KSP Object: (mg_levels_1_) 4 MPI processes<span class="gmail-"><br>
type: richardson<br>
Richardson: damping factor=1.<br>
maximum iterations=2<br>
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
left preconditioning<br>
using nonzero initial guess<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_levels_1_) 4 MPI processes<span class="gmail-"><br>
type: sor<br>
SOR: type = local_symmetric, iterations = 1, local
iterations = 1, omega = 1.<br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<br>
type: mpiaij<br>
rows=168, cols=168<br>
total: nonzeros=19874, allocated nonzeros=19874<span class="gmail-"><br>
total number of mallocs used during MatSetValues calls =0<br></span>
using I-node (on process 0) routines: found 17 nodes,
limit used is 5<span class="gmail-"><br>
Up solver (post-smoother) same as down solver (pre-smoother)<br></span><span class="gmail-">
Down solver (pre-smoother) on level 2
------------------------------<wbr>-<br></span>
KSP Object: (mg_levels_2_) 4 MPI processes<span class="gmail-"><br>
type: richardson<br>
Richardson: damping factor=1.<br>
maximum iterations=2<br>
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
left preconditioning<br>
using nonzero initial guess<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_levels_2_) 4 MPI processes<span class="gmail-"><br>
type: sor<br>
SOR: type = local_symmetric, iterations = 1, local
iterations = 1, omega = 1.<br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<br>
type: mpiaij<br>
rows=3572, cols=3572<br>
total: nonzeros=963872, allocated nonzeros=963872<span class="gmail-"><br>
total number of mallocs used during MatSetValues calls =0<br>
not using I-node (on process 0) routines<br>
Up solver (post-smoother) same as down solver (pre-smoother)<br></span>
Down solver (pre-smoother) on level 3
------------------------------<wbr>-<br>
KSP Object: (mg_levels_3_) 4 MPI processes<span class="gmail-"><br>
type: richardson<br>
Richardson: damping factor=1.<br>
maximum iterations=2<br>
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
left preconditioning<br>
using nonzero initial guess<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_levels_3_) 4 MPI processes<span class="gmail-"><br>
type: sor<br>
SOR: type = local_symmetric, iterations = 1, local
iterations = 1, omega = 1.<br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<br>
type: mpiaij<br>
rows=21179, cols=21179<br>
total: nonzeros=1060605, allocated nonzeros=1060605<span class="gmail-"><br>
total number of mallocs used during MatSetValues calls =0<br>
not using I-node (on process 0) routines<br>
Up solver (post-smoother) same as down solver (pre-smoother)<br></span>
Down solver (pre-smoother) on level 4
------------------------------<wbr>-<br>
KSP Object: (mg_levels_4_) 4 MPI processes<span class="gmail-"><br>
type: richardson<br>
Richardson: damping factor=1.<br>
maximum iterations=2<br>
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
left preconditioning<br>
using nonzero initial guess<br>
using NONE norm type for convergence test<br></span>
PC Object: (mg_levels_4_) 4 MPI processes<span class="gmail-"><br>
type: sor<br>
SOR: type = local_symmetric, iterations = 1, local
iterations = 1, omega = 1.<br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<span class="gmail-"><br>
type: mpiaij<br>
rows=1745224, cols=1745224<br>
total: nonzeros=99452608, allocated nonzeros=99452608<br>
total number of mallocs used during MatSetValues calls =0<br></span>
using I-node (on process 0) routines: found 254433 nodes,
limit used is 5<span class="gmail-"><br>
Up solver (post-smoother) same as down solver (pre-smoother)<br>
linear system matrix = precond matrix:<br></span>
Mat Object: 4 MPI processes<span class="gmail-"><br>
type: mpiaij<br>
rows=1745224, cols=1745224<br>
total: nonzeros=99452608, allocated nonzeros=99452608<br>
total number of mallocs used during MatSetValues calls =0<br></span>
using I-node (on process 0) routines: found 254433 nodes,
limit used is 5<br>
<br>
<br>
<br>
<br>
Thanks, David<div><div class="gmail-h5"><br>
<br>
<br>
<br>
<div class="gmail-m_-3326292587204676525moz-cite-prefix">On 11/08/2017 10:11 PM, Mark Adams
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Nov 1, 2017 at 5:45 PM, David
Nolte <span dir="ltr"><<a href="mailto:dnolte@dim.uchile.cl" target="_blank">dnolte@dim.uchile.cl</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Thanks
Barry.<br>
By simply replacing chebychev by richardson I get similar
performance<br>
with GAMG and ML </blockquote>
<div><br>
</div>
<div>That too (I assumed you were using the same, I could
not see cheby in your view data).</div>
<div><br>
</div>
<div>I guess SOR works for the coarse grid solver because
the coarse grid is small. It should help using lu.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">(GAMG
even slightly faster):<br>
</blockquote>
<div><br>
</div>
<div>This is "random" fluctuations.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-pc_type<br>
gamg <wbr> <wbr> <wbr> <wbr> <wbr> <br>
<br>
-pc_gamg_type<br>
agg <wbr> <wbr> <wbr> <wbr> <br>
<br>
-pc_gamg_threshold<br>
0.03 <wbr> <wbr> <wbr> <wbr> <br>
<br>
-pc_gamg_square_graph 10<br>
-pc_gamg_sym_graph<br>
-mg_levels_ksp_type<br>
richardson <wbr> <wbr> <wbr> <wbr> <br>
<br>
-mg_levels_pc_type sor<br>
<br>
Is it still true that I need to set "-pc_gamg_sym_graph"
if the matrix<br>
is asymmetric? </blockquote>
<div><br>
</div>
<div>yes,</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">For
serial runs it doesn't seem to matter, </blockquote>
<div><br>
</div>
<div>yes,</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">but in<br>
parallel the PC setup hangs (after calls of<br>
PCGAMGFilterGraph()) if -pc_gamg_sym_graph is not set.<br>
</blockquote>
<div><br>
</div>
<div>yep,</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span class="gmail-m_-3326292587204676525HOEnZb"><font color="#888888"><br>
David<br>
</font></span>
<div class="gmail-m_-3326292587204676525HOEnZb">
<div class="gmail-m_-3326292587204676525h5"><br>
<br>
On 10/21/2017 12:10 AM, Barry Smith wrote:<br>
> David,<br>
><br>
> GAMG picks the number of levels based on how
the coarsening process etc proceeds. You cannot
hardwire it to a particular value. You can run with
-info to get more info potentially on the decisions
GAMG is making.<br>
><br>
> Barry<br>
><br>
>> On Oct 20, 2017, at 2:06 PM, David Nolte <<a href="mailto:dnolte@dim.uchile.cl" target="_blank">dnolte@dim.uchile.cl</a>>
wrote:<br>
>><br>
>> PS: I didn't realize at first, it looks as if
the -pc_mg_levels 3 option<br>
>> was not taken into account:<br>
>> type: gamg<br>
>> MG: type is MULTIPLICATIVE, levels=1
cycles=v<br>
>><br>
>><br>
>><br>
>> On 10/20/2017 03:32 PM, David Nolte wrote:<br>
>>> Dear all,<br>
>>><br>
>>> I have some problems using GAMG as a
preconditioner for (F)GMRES.<br>
>>> Background: I am solving the
incompressible, unsteady Navier-Stokes<br>
>>> equations with a coupled mixed FEM
approach, using P1/P1 elements for<br>
>>> velocity and pressure on an unstructured
tetrahedron mesh with about<br>
>>> 2mio DOFs (and up to 15mio). The method
is stabilized with SUPG/PSPG,<br>
>>> hence, no zeros on the diagonal of the
pressure block. Time<br>
>>> discretization with semi-implicit
backward Euler. The flow is a<br>
>>> convection dominated flow through a
nozzle.<br>
>>><br>
>>> So far, for this setup, I have been quite
happy with a simple FGMRES/ML<br>
>>> solver for the full system (rather
bruteforce, I admit, but much faster<br>
>>> than any block/Schur preconditioners I
tried):<br>
>>><br>
>>> -ksp_converged_reason<br>
>>> -ksp_monitor_true_residual<br>
>>> -ksp_type fgmres<br>
>>> -ksp_rtol 1.0e-6<br>
>>> -ksp_initial_guess_nonzero<br>
>>><br>
>>> -pc_type ml<br>
>>> -pc_ml_Threshold 0.03<br>
>>> -pc_ml_maxNlevels 3<br>
>>><br>
>>> This setup converges in ~100 iterations
(see below the ksp_view output)<br>
>>> to rtol:<br>
>>><br>
>>> 119 KSP unpreconditioned resid norm
4.004030812027e-05 true resid norm<br>
>>> 4.004030812037e-05 ||r(i)||/||b||
1.621791251517e-06<br>
>>> 120 KSP unpreconditioned resid norm
3.256863709982e-05 true resid norm<br>
>>> 3.256863709982e-05 ||r(i)||/||b||
1.319158947617e-06<br>
>>> 121 KSP unpreconditioned resid norm
2.751959681502e-05 true resid norm<br>
>>> 2.751959681503e-05 ||r(i)||/||b||
1.114652795021e-06<br>
>>> 122 KSP unpreconditioned resid norm
2.420611122789e-05 true resid norm<br>
>>> 2.420611122788e-05 ||r(i)||/||b||
9.804434897105e-07<br>
>>><br>
>>><br>
>>> Now I'd like to try GAMG instead of ML.
However, I don't know how to set<br>
>>> it up to get similar performance.<br>
>>> The obvious/naive<br>
>>><br>
>>> -pc_type gamg<br>
>>> -pc_gamg_type agg<br>
>>><br>
>>> # with and without<br>
>>> -pc_gamg_threshold 0.03<br>
>>> -pc_mg_levels 3<br>
>>><br>
>>> converges very slowly on 1 proc and much
worse on 8 (~200k dofs per<br>
>>> proc), for instance:<br>
>>> np = 1:<br>
>>> 980 KSP unpreconditioned resid norm
1.065009356215e-02 true resid norm<br>
>>> 1.065009356215e-02 ||r(i)||/||b||
4.532259705508e-04<br>
>>> 981 KSP unpreconditioned resid norm
1.064978578182e-02 true resid norm<br>
>>> 1.064978578182e-02 ||r(i)||/||b||
4.532128726342e-04<br>
>>> 982 KSP unpreconditioned resid norm
1.064956706598e-02 true resid norm<br>
>>> 1.064956706598e-02 ||r(i)||/||b||
4.532035649508e-04<br>
>>><br>
>>> np = 8:<br>
>>> 980 KSP unpreconditioned resid norm
3.179946748495e-02 true resid norm<br>
>>> 3.179946748495e-02 ||r(i)||/||b||
1.353259896710e-03<br>
>>> 981 KSP unpreconditioned resid norm
3.179946748317e-02 true resid norm<br>
>>> 3.179946748317e-02 ||r(i)||/||b||
1.353259896634e-03<br>
>>> 982 KSP unpreconditioned resid norm
3.179946748317e-02 true resid norm<br>
>>> 3.179946748317e-02 ||r(i)||/||b||
1.353259896634e-03<br>
>>><br>
>>> A very high threshold seems to improve
the GAMG PC, for instance with<br>
>>> 0.75 I get convergence to rtol=1e-6 after
744 iterations.<br>
>>> What else should I try?<br>
>>><br>
>>> I would very much appreciate any advice
on configuring GAMG and<br>
>>> differences w.r.t ML to be taken into
account (not a multigrid expert<br>
>>> though).<br>
>>><br>
>>> Thanks, best wishes<br>
>>> David<br>
>>><br>
>>><br>
>>> ------<br>
>>> ksp_view for -pc_type gamg
-pc_gamg_threshold 0.75 -pc_mg_levels 3<br>
>>><br>
>>> KSP Object: 1 MPI processes<br>
>>> type: fgmres<br>
>>> GMRES: restart=30, using Classical
(unmodified) Gram-Schmidt<br>
>>> Orthogonalization with no iterative
refinement<br>
>>> GMRES: happy breakdown tolerance
1e-30<br>
>>> maximum iterations=10000<br>
>>> tolerances: relative=1e-06,
absolute=1e-50, divergence=10000.<br>
>>> right preconditioning<br>
>>> using nonzero initial guess<br>
>>> using UNPRECONDITIONED norm type for
convergence test<br>
>>> PC Object: 1 MPI processes<br>
>>> type: gamg<br>
>>> MG: type is MULTIPLICATIVE, levels=1
cycles=v<br>
>>> Cycles per PCApply=1<br>
>>> Using Galerkin computed coarse grid
matrices<br>
>>> GAMG specific options<br>
>>> Threshold for dropping small
values from graph 0.75<br>
>>> AGG specific options<br>
>>> Symmetric graph false<br>
>>> Coarse grid solver -- level
------------------------------<wbr>-<br>
>>> KSP Object: (mg_levels_0_) 1
MPI processes<br>
>>> type: preonly<br>
>>> maximum iterations=2, initial guess
is zero<br>
>>> tolerances: relative=1e-05,
absolute=1e-50, divergence=10000.<br>
>>> left preconditioning<br>
>>> using NONE norm type for
convergence test<br>
>>> PC Object: (mg_levels_0_) 1
MPI processes<br>
>>> type: sor<br>
>>> SOR: type = local_symmetric,
iterations = 1, local iterations =<br>
>>> 1, omega = 1.<br>
>>> linear system matrix = precond
matrix:<br>
>>> Mat Object: 1 MPI processes<br>
>>> type: seqaij<br>
>>> rows=1745224, cols=1745224<br>
>>> total: nonzeros=99452608,
allocated nonzeros=99452608<br>
>>> total number of mallocs used
during MatSetValues calls =0<br>
>>> using I-node routines: found
1037847 nodes, limit used is 5<br>
>>> linear system matrix = precond matrix:<br>
>>> Mat Object: 1 MPI processes<br>
>>> type: seqaij<br>
>>> rows=1745224, cols=1745224<br>
>>> total: nonzeros=99452608, allocated
nonzeros=99452608<br>
>>> total number of mallocs used during
MatSetValues calls =0<br>
>>> using I-node routines: found
1037847 nodes, limit used is 5<br>
>>><br>
>>><br>
>>> ------<br>
>>> ksp_view for -pc_type ml:<br>
>>><br>
>>> KSP Object: 8 MPI processes<br>
>>> type: fgmres<br>
>>> GMRES: restart=30, using Classical
(unmodified) Gram-Schmidt<br>
>>> Orthogonalization with no iterative
refinement<br>
>>> GMRES: happy breakdown tolerance
1e-30<br>
>>> maximum iterations=10000<br>
>>> tolerances: relative=1e-06,
absolute=1e-50, divergence=10000.<br>
>>> right preconditioning<br>
>>> using nonzero initial guess<br>
>>> using UNPRECONDITIONED norm type for
convergence test<br>
>>> PC Object: 8 MPI processes<br>
>>> type: ml<br>
>>> MG: type is MULTIPLICATIVE, levels=3
cycles=v<br>
>>> Cycles per PCApply=1<br>
>>> Using Galerkin computed coarse grid
matrices<br>
>>> Coarse grid solver -- level
------------------------------<wbr>-<br>
>>> KSP Object: (mg_coarse_) 8 MPI
processes<br>
>>> type: preonly<br>
>>> maximum iterations=10000, initial
guess is zero<br>
>>> tolerances: relative=1e-05,
absolute=1e-50, divergence=10000.<br>
>>> left preconditioning<br>
>>> using NONE norm type for
convergence test<br>
>>> PC Object: (mg_coarse_) 8 MPI
processes<br>
>>> type: redundant<br>
>>> Redundant preconditioner: First
(color=0) of 8 PCs follows<br>
>>> KSP Object:
(mg_coarse_redundant_) 1 MPI processes<br>
>>> type: preonly<br>
>>> maximum iterations=10000,
initial guess is zero<br>
>>> tolerances: relative=1e-05,
absolute=1e-50, divergence=10000.<br>
>>> left preconditioning<br>
>>> using NONE norm type for
convergence test<br>
>>> PC Object:
(mg_coarse_redundant_) 1 MPI processes<br>
>>> type: lu<br>
>>> LU: out-of-place
factorization<br>
>>> tolerance for zero pivot
2.22045e-14<br>
>>> using diagonal shift on
blocks to prevent zero pivot [INBLOCKS]<br>
>>> matrix ordering: nd<br>
>>> factor fill ratio given 5.,
needed 10.4795<br>
>>> Factored matrix follows:<br>
>>> Mat Object:
1 MPI processes<br>
>>> type: seqaij<br>
>>> rows=6822, cols=6822<br>
>>> package used to perform
factorization: petsc<br>
>>> total:
nonzeros=9575688, allocated nonzeros=9575688<br>
>>> total number of mallocs
used during MatSetValues calls =0<br>
>>> not using I-node
routines<br>
>>> linear system matrix = precond
matrix:<br>
>>> Mat Object: 1 MPI
processes<br>
>>> type: seqaij<br>
>>> rows=6822, cols=6822<br>
>>> total: nonzeros=913758,
allocated nonzeros=913758<br>
>>> total number of mallocs used
during MatSetValues calls =0<br>
>>> not using I-node routines<br>
>>> linear system matrix = precond
matrix:<br>
>>> Mat Object: 8 MPI processes<br>
>>> type: mpiaij<br>
>>> rows=6822, cols=6822<br>
>>> total: nonzeros=913758, allocated
nonzeros=913758<br>
>>> total number of mallocs used
during MatSetValues calls =0<br>
>>> not using I-node (on process 0)
routines<br>
>>> Down solver (pre-smoother) on level 1
------------------------------<wbr>-<br>
>>> KSP Object: (mg_levels_1_) 8
MPI processes<br>
>>> type: richardson<br>
>>> Richardson: damping factor=1.<br>
>>> maximum iterations=2<br>
>>> tolerances: relative=1e-05,
absolute=1e-50, divergence=10000.<br>
>>> left preconditioning<br>
>>> using nonzero initial guess<br>
>>> using NONE norm type for
convergence test<br>
>>> PC Object: (mg_levels_1_) 8
MPI processes<br>
>>> type: sor<br>
>>> SOR: type = local_symmetric,
iterations = 1, local iterations =<br>
>>> 1, omega = 1.<br>
>>> linear system matrix = precond
matrix:<br>
>>> Mat Object: 8 MPI processes<br>
>>> type: mpiaij<br>
>>> rows=67087, cols=67087<br>
>>> total: nonzeros=9722749,
allocated nonzeros=9722749<br>
>>> total number of mallocs used
during MatSetValues calls =0<br>
>>> not using I-node (on process 0)
routines<br>
>>> Up solver (post-smoother) same as down
solver (pre-smoother)<br>
>>> Down solver (pre-smoother) on level 2
------------------------------<wbr>-<br>
>>> KSP Object: (mg_levels_2_) 8
MPI processes<br>
>>> type: richardson<br>
>>> Richardson: damping factor=1.<br>
>>> maximum iterations=2<br>
>>> tolerances: relative=1e-05,
absolute=1e-50, divergence=10000.<br>
>>> left preconditioning<br>
>>> using nonzero initial guess<br>
>>> using NONE norm type for
convergence test<br>
>>> PC Object: (mg_levels_2_) 8
MPI processes<br>
>>> type: sor<br>
>>> SOR: type = local_symmetric,
iterations = 1, local iterations =<br>
>>> 1, omega = 1.<br>
>>> linear system matrix = precond
matrix:<br>
>>> Mat Object: 8 MPI processes<br>
>>> type: mpiaij<br>
>>> rows=1745224, cols=1745224<br>
>>> total: nonzeros=99452608,
allocated nonzeros=99452608<br>
>>> total number of mallocs used
during MatSetValues calls =0<br>
>>> using I-node (on process 0)
routines: found 126690 nodes,<br>
>>> limit used is 5<br>
>>> Up solver (post-smoother) same as down
solver (pre-smoother)<br>
>>> linear system matrix = precond matrix:<br>
>>> Mat Object: 8 MPI processes<br>
>>> type: mpiaij<br>
>>> rows=1745224, cols=1745224<br>
>>> total: nonzeros=99452608, allocated
nonzeros=99452608<br>
>>> total number of mallocs used during
MatSetValues calls =0<br>
>>> using I-node (on process 0)
routines: found 126690 nodes, limit<br>
>>> used is 5<br>
>>><br>
<br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div></div>