[petsc-users] Convergence of AMG
Manav Bhatia
bhatiamanav at gmail.com
Wed Oct 31 14:42:13 CDT 2018
Here are the updates. I did not find the options to make much difference in the results.
I noticed this message in the GAMG output for cases 2, 3: HARD stop of coarsening on level 3. Grid too small: 1 block nodes
Is this implying that the mesh on level 3 could not be coarsened towards levels 4/5?
In one of your earlier emails you had mentioned about reducing the rate of coarsening. What is the option to do that.
Common options:
-pc_mg_levels 5 -mg_levels_ksp_max_it 4 -pc_gamg_square_graph 0 -pc_gamg_threshold 0. -mg_levels_ksp_type richardson -gamg_est_ksp_type cg
Case# | #levels | #KSP Iters | Extra Options
————————————————————————————
1 | 5 | 67 |
2 | 5 | 67 | -pc_gamg_agg_nsmooths 2
3 | 5 | 67 | -pc_gamg_agg_nsmooths 2 -mg_levels_esteig_ksp_type cg
Regards,
Manav
1) -pc_mg_levels 5 -mg_levels_ksp_max_it 4 -pc_gamg_square_graph 0 -pc_gamg_threshold 0. -mg_levels_ksp_type richardson -gamg_est_ksp_type cg
Blackbird-3:tmp manav$ cat t | grep "KSP\ Res"
0 KSP Residual norm 1.851603973086e+00
1 KSP Residual norm 1.236968440850e+00
2 KSP Residual norm 9.390455456383e-01
3 KSP Residual norm 6.995550854087e-01
4 KSP Residual norm 5.376014874545e-01
5 KSP Residual norm 4.224938459190e-01
6 KSP Residual norm 3.299526332446e-01
7 KSP Residual norm 2.598913867489e-01
8 KSP Residual norm 2.021899918256e-01
9 KSP Residual norm 1.569629696968e-01
10 KSP Residual norm 1.199340095037e-01
11 KSP Residual norm 8.839078696637e-02
12 KSP Residual norm 6.343737778919e-02
13 KSP Residual norm 4.618676240943e-02
14 KSP Residual norm 3.423870718445e-02
15 KSP Residual norm 2.476201782979e-02
16 KSP Residual norm 1.816680574952e-02
17 KSP Residual norm 1.313655583815e-02
18 KSP Residual norm 9.537218399283e-03
19 KSP Residual norm 7.142389309622e-03
20 KSP Residual norm 5.254424711175e-03
21 KSP Residual norm 3.884143778641e-03
22 KSP Residual norm 2.972078747664e-03
23 KSP Residual norm 2.291218731265e-03
24 KSP Residual norm 1.750029682894e-03
25 KSP Residual norm 1.336782384261e-03
26 KSP Residual norm 1.008407782667e-03
27 KSP Residual norm 7.491621913428e-04
28 KSP Residual norm 5.407480688723e-04
29 KSP Residual norm 3.885903074862e-04
30 KSP Residual norm 2.823392094831e-04
31 KSP Residual norm 2.334379417608e-04
32 KSP Residual norm 1.862232013173e-04
33 KSP Residual norm 1.436841687740e-04
34 KSP Residual norm 1.071549728624e-04
35 KSP Residual norm 8.090249549012e-05
36 KSP Residual norm 6.198056235196e-05
37 KSP Residual norm 4.807122983322e-05
38 KSP Residual norm 3.749190126381e-05
39 KSP Residual norm 2.856634096713e-05
40 KSP Residual norm 2.143362174581e-05
41 KSP Residual norm 1.620193111270e-05
42 KSP Residual norm 1.205751485823e-05
43 KSP Residual norm 8.472022598540e-06
44 KSP Residual norm 5.899591838872e-06
45 KSP Residual norm 4.023678186795e-06
46 KSP Residual norm 2.700561982264e-06
47 KSP Residual norm 1.868774325367e-06
48 KSP Residual norm 1.291496854620e-06
49 KSP Residual norm 9.216156351135e-07
50 KSP Residual norm 6.624151736445e-07
51 KSP Residual norm 4.847699010466e-07
52 KSP Residual norm 3.717034432412e-07
53 KSP Residual norm 2.944641488187e-07
54 KSP Residual norm 2.405160144034e-07
55 KSP Residual norm 1.930168301359e-07
56 KSP Residual norm 1.543411662956e-07
57 KSP Residual norm 1.219117666559e-07
58 KSP Residual norm 1.018761570699e-07
59 KSP Residual norm 8.612996296470e-08
60 KSP Residual norm 7.804919225099e-08
61 KSP Residual norm 7.130576810131e-08
62 KSP Residual norm 5.229870149728e-08
63 KSP Residual norm 4.015915040910e-08
64 KSP Residual norm 3.139220241645e-08
65 KSP Residual norm 2.447134574457e-08
66 KSP Residual norm 1.875885438105e-08
67 KSP Residual norm 1.422423827541e-08
Blackbird-3:tmp manav$ cat t | grep "GAMG"
[0] PCSetUp_GAMG(): level 0) N=543606, n data rows=6, n data cols=6, nnz/row (ave)=54, np=4
[0] PCGAMGFilterGraph(): 98.7979% nnz after filtering, with threshold 0., 8.95943 nnz ave. (N=90601)
[0] PCGAMGProlongator_AGG(): New grid 16817 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.150898e+00 min=3.523136e-02 PC=jacobi
[0] PCSetUp_GAMG(): 1) N=100902, n data cols=6, nnz/row (ave)=103, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 17.2399 nnz ave. (N=16817)
[0] PCGAMGProlongator_AGG(): New grid 2021 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=1.875419e+00 min=3.291195e-02 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=12126, n data cols=6, nnz/row (ave)=172, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 28.7325 nnz ave. (N=2021)
[0] PCGAMGProlongator_AGG(): New grid 161 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.227706e+00 min=4.290859e-02 PC=jacobi
[0] PCSetUp_GAMG(): 3) N=966, n data cols=6, nnz/row (ave)=176, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 28.2353 nnz ave. (N=161)
[0] PCGAMGProlongator_AGG(): New grid 14 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.199096e+00 min=7.296159e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 18 with simple aggregation
[0] PCSetUp_GAMG(): 4) N=84, n data cols=6, nnz/row (ave)=82, 1 active pes
[0] PCSetUp_GAMG(): 5 levels, grid complexity = 1.43482
KSP Object: 4 MPI processes
type: gmres
restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
happy breakdown tolerance 1e-30
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-08, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
PC Object: 4 MPI processes
type: gamg
type is MULTIPLICATIVE, levels=5 cycles=v
Cycles per PCApply=1
Using externally compute Galerkin coarse grid matrices
GAMG specific options
Threshold for dropping small values in graph on each level = 0. 0. 0.
Threshold scaling factor for each level not specified = 1.
AGG specific options
[1] PetscCommDuplicate(): Using internal PETSc communicator 4504571456 140452555073200
[2] PetscCommDuplicate(): Using internal PETSc communicator 4504063552 140197931234784
[3] PetscCommDuplicate(): Using internal PETSc communicator 4542250560 140215134305744
Symmetric graph false
Number of levels to square graph 0
Number smoothing steps 1
Coarse grid solver -- level -------------------------------
KSP Object: (mg_coarse_) 4 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_) 4 MPI processes
type: bjacobi
number of blocks = 4
Local solve is same for all blocks, in the following KSP and PC objects:
[0] PetscCommDuplicate(): Using internal PETSc communicator 4530183744 140522297539408
KSP Object: (mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_sub_) 1 MPI processes
type: lu
out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.01042
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=84, cols=84, bs=6
package used to perform factorization: petsc
total: nonzeros=6984, allocated nonzeros=6984
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 19 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=84, cols=84, bs=6
total: nonzeros=6912, allocated nonzeros=6912
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 22 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=84, cols=84, bs=6
total: nonzeros=6912, allocated nonzeros=6912
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 22 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (mg_levels_1_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_1_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=966, cols=966, bs=6
total: nonzeros=170460, allocated nonzeros=170460
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 68 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (mg_levels_2_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_2_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=12126, cols=12126, bs=6
total: nonzeros=2091780, allocated nonzeros=2091780
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 1000 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 3 -------------------------------
KSP Object: (mg_levels_3_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_3_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=100902, cols=100902, bs=6
total: nonzeros=10438380, allocated nonzeros=10438380
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 8420 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 4 -------------------------------
KSP Object: (mg_levels_4_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_4_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
2) -pc_mg_levels 5 -mg_levels_ksp_max_it 4 -pc_gamg_square_graph 0 -pc_gamg_threshold 0. -mg_levels_ksp_type richardson -gamg_est_ksp_type cg -pc_gamg_agg_nsmooths 2
Dhcp-90-164:tmp manav$ less t | grep "KSP\ Res"
0 KSP Residual norm 2.267825771593e+00
1 KSP Residual norm 1.408716287389e+00
2 KSP Residual norm 1.051634554284e+00
3 KSP Residual norm 7.943327803314e-01
4 KSP Residual norm 5.985495977659e-01
5 KSP Residual norm 4.876074034827e-01
6 KSP Residual norm 4.032803415998e-01
7 KSP Residual norm 3.151459268079e-01
8 KSP Residual norm 2.287875309266e-01
9 KSP Residual norm 1.689560628134e-01
10 KSP Residual norm 1.265410177028e-01
11 KSP Residual norm 9.716400303328e-02
12 KSP Residual norm 7.305887324105e-02
13 KSP Residual norm 5.252262598629e-02
14 KSP Residual norm 3.665923907231e-02
15 KSP Residual norm 2.602482063395e-02
16 KSP Residual norm 1.874425155866e-02
17 KSP Residual norm 1.338002525296e-02
18 KSP Residual norm 9.461558101831e-03
19 KSP Residual norm 6.860781203061e-03
20 KSP Residual norm 5.293924529472e-03
21 KSP Residual norm 4.001605646043e-03
22 KSP Residual norm 2.932783008322e-03
23 KSP Residual norm 2.190592529117e-03
24 KSP Residual norm 1.651488369324e-03
25 KSP Residual norm 1.242272218904e-03
26 KSP Residual norm 9.109308493484e-04
27 KSP Residual norm 6.734018279380e-04
28 KSP Residual norm 5.045747272596e-04
29 KSP Residual norm 3.708717871933e-04
30 KSP Residual norm 2.700583924223e-04
31 KSP Residual norm 2.279903216897e-04
32 KSP Residual norm 1.860788922727e-04
33 KSP Residual norm 1.393442047104e-04
34 KSP Residual norm 1.056758942826e-04
35 KSP Residual norm 8.292722358648e-05
36 KSP Residual norm 6.249605418622e-05
37 KSP Residual norm 4.515823660374e-05
38 KSP Residual norm 3.341965987471e-05
39 KSP Residual norm 2.455167026138e-05
40 KSP Residual norm 1.799936419620e-05
41 KSP Residual norm 1.301185237440e-05
42 KSP Residual norm 9.185857856804e-06
43 KSP Residual norm 6.639180846148e-06
44 KSP Residual norm 4.796369056787e-06
45 KSP Residual norm 3.517639548008e-06
46 KSP Residual norm 2.533810036316e-06
47 KSP Residual norm 1.776841834240e-06
48 KSP Residual norm 1.216140909196e-06
49 KSP Residual norm 8.985883410485e-07
50 KSP Residual norm 7.081872318657e-07
51 KSP Residual norm 5.298199755216e-07
52 KSP Residual norm 3.780633027727e-07
53 KSP Residual norm 2.776467936978e-07
54 KSP Residual norm 2.145923065527e-07
55 KSP Residual norm 1.847249755132e-07
56 KSP Residual norm 1.602876793711e-07
57 KSP Residual norm 1.405461845847e-07
58 KSP Residual norm 1.210529511057e-07
59 KSP Residual norm 1.040909071752e-07
60 KSP Residual norm 8.799454707008e-08
61 KSP Residual norm 7.574693709284e-08
62 KSP Residual norm 5.902264183743e-08
63 KSP Residual norm 4.928450943916e-08
64 KSP Residual norm 4.103274222934e-08
65 KSP Residual norm 3.333493557786e-08
66 KSP Residual norm 2.697332570486e-08
67 KSP Residual norm 2.047460738099e-08
Dhcp-90-164:tmp manav$ less t | grep "GAMG"
[0] PCSetUp_GAMG(): level 0) N=543606, n data rows=6, n data cols=6, nnz/row (ave)=54, np=4
[0] PCGAMGFilterGraph(): 98.7979% nnz after filtering, with threshold 0., 8.95943 nnz ave. (N=90601)
[0] PCGAMGProlongator_AGG(): New grid 16817 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.150898e+00 min=3.523136e-02 PC=jacobi
[0] PCSetUp_GAMG(): 1) N=100902, n data cols=6, nnz/row (ave)=207, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 34.5235 nnz ave. (N=16817)
[0] PCGAMGProlongator_AGG(): New grid 1041 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.387504e+00 min=1.113469e-01 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=6246, n data cols=6, nnz/row (ave)=398, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 65.9061 nnz ave. (N=1041)
[0] PCGAMGProlongator_AGG(): New grid 38 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.352516e+00 min=4.279834e-02 PC=jacobi
[0] PCSetUp_GAMG(): 3) N=228, n data cols=6, nnz/row (ave)=207, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 33.875 nnz ave. (N=38)
[0] PCGAMGProlongator_AGG(): New grid 1 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.109888e+00 min=7.508213e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 0 with simple aggregation
[0] PCSetUp_GAMG(): 4) N=6, n data cols=6, nnz/row (ave)=6, 1 active pes
[0] PCSetUp_GAMG(): HARD stop of coarsening on level 3. Grid too small: 1 block nodes
[0] PCSetUp_GAMG(): 5 levels, grid complexity = 1.80239
KSP Object: 4 MPI processes
type: gmres
restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
happy breakdown tolerance 1e-30
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-08, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
PC Object: 4 MPI processes
type: gamg
type is MULTIPLICATIVE, levels=5 cycles=v
Cycles per PCApply=1
Using externally compute Galerkin coarse grid matrices
GAMG specific options
Threshold for dropping small values in graph on each level = 0. 0. 0.
Threshold scaling factor for each level not specified = 1.
AGG specific options
Symmetric graph false
Number of levels to square graph 0
Number smoothing steps 2
Coarse grid solver -- level -------------------------------
KSP Object: (mg_coarse_) 4 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_) 4 MPI processes
type: bjacobi
number of blocks = 4
Local solve is same for all blocks, in the following KSP and PC objects:
[0] PetscCommDuplicate(): Using internal PETSc communicator 4570619456 140283319649920
[2] PetscCommDuplicate(): Using internal PETSc communicator 4506250816 140419971129472
KSP Object: (mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_sub_) 1 MPI processes
type: lu
out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=6, cols=6, bs=6
package used to perform factorization: petsc
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 2 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=6, cols=6, bs=6
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 2 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=6, cols=6, bs=6
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 2 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (mg_levels_1_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_1_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=228, cols=228, bs=6
total: nonzeros=47232, allocated nonzeros=47232
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 10 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (mg_levels_2_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_2_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=6246, cols=6246, bs=6
total: nonzeros=2489868, allocated nonzeros=2489868
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 490 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 3 -------------------------------
KSP Object: (mg_levels_3_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_3_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=100902, cols=100902, bs=6
total: nonzeros=20912652, allocated nonzeros=20912652
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 8420 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 4 -------------------------------
KSP Object: (mg_levels_4_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_4_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
2) -pc_mg_levels 5 -mg_levels_ksp_max_it 4 -pc_gamg_square_graph 0 -pc_gamg_threshold 0. -mg_levels_ksp_type richardson -gamg_est_ksp_type cg -pc_gamg_agg_nsmooths 2 -mg_levels_esteig_ksp_type cg
Dhcp-90-164:tmp manav$ less t | grep "KSP\ Res"
0 KSP Residual norm 2.267825771593e+00
1 KSP Residual norm 1.408716287387e+00
2 KSP Residual norm 1.051634554280e+00
3 KSP Residual norm 7.943327803270e-01
4 KSP Residual norm 5.985495977612e-01
5 KSP Residual norm 4.876074034779e-01
6 KSP Residual norm 4.032803415953e-01
7 KSP Residual norm 3.151459268039e-01
8 KSP Residual norm 2.287875309236e-01
9 KSP Residual norm 1.689560628111e-01
10 KSP Residual norm 1.265410177010e-01
11 KSP Residual norm 9.716400303191e-02
12 KSP Residual norm 7.305887323998e-02
13 KSP Residual norm 5.252262598550e-02
14 KSP Residual norm 3.665923907174e-02
15 KSP Residual norm 2.602482063353e-02
16 KSP Residual norm 1.874425155833e-02
17 KSP Residual norm 1.338002525269e-02
18 KSP Residual norm 9.461558101624e-03
19 KSP Residual norm 6.860781202901e-03
20 KSP Residual norm 5.293924529345e-03
21 KSP Residual norm 4.001605645944e-03
22 KSP Residual norm 2.932783008245e-03
23 KSP Residual norm 2.190592529059e-03
24 KSP Residual norm 1.651488369281e-03
25 KSP Residual norm 1.242272218876e-03
26 KSP Residual norm 9.109308493356e-04
27 KSP Residual norm 6.734018279390e-04
28 KSP Residual norm 5.045747272693e-04
29 KSP Residual norm 3.708717872080e-04
30 KSP Residual norm 2.700584035265e-04
31 KSP Residual norm 2.279903320914e-04
32 KSP Residual norm 1.860789013071e-04
33 KSP Residual norm 1.393442086232e-04
34 KSP Residual norm 1.056758941655e-04
35 KSP Residual norm 8.292722294552e-05
36 KSP Residual norm 6.249605489796e-05
37 KSP Residual norm 4.515823692238e-05
38 KSP Residual norm 3.341965924406e-05
39 KSP Residual norm 2.455167014490e-05
40 KSP Residual norm 1.799936521170e-05
41 KSP Residual norm 1.301185398833e-05
42 KSP Residual norm 9.185859481763e-06
43 KSP Residual norm 6.639182448645e-06
44 KSP Residual norm 4.796370558780e-06
45 KSP Residual norm 3.517640894677e-06
46 KSP Residual norm 2.533811059265e-06
47 KSP Residual norm 1.776842623521e-06
48 KSP Residual norm 1.216141654930e-06
49 KSP Residual norm 8.985890975369e-07
50 KSP Residual norm 7.081880149660e-07
51 KSP Residual norm 5.298207972657e-07
52 KSP Residual norm 3.780642754235e-07
53 KSP Residual norm 2.776479493176e-07
54 KSP Residual norm 2.145935700364e-07
55 KSP Residual norm 1.847262262413e-07
56 KSP Residual norm 1.602888215732e-07
57 KSP Residual norm 1.405471647242e-07
58 KSP Residual norm 1.210537184709e-07
59 KSP Residual norm 1.040914613543e-07
60 KSP Residual norm 8.800584999158e-08
61 KSP Residual norm 7.575320990546e-08
62 KSP Residual norm 5.902566537130e-08
63 KSP Residual norm 4.928732270849e-08
64 KSP Residual norm 4.103605656966e-08
65 KSP Residual norm 3.333879521091e-08
66 KSP Residual norm 2.697680027664e-08
67 KSP Residual norm 2.047615127912e-08
Dhcp-90-164:tmp manav$ less t | grep "GAMG"
[0] PCSetUp_GAMG(): level 0) N=543606, n data rows=6, n data cols=6, nnz/row (ave)=54, np=4
[0] PCGAMGFilterGraph(): 98.7979% nnz after filtering, with threshold 0., 8.95943 nnz ave. (N=90601)
[0] PCGAMGProlongator_AGG(): New grid 16817 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.150898e+00 min=3.523136e-02 PC=jacobi
[0] PCSetUp_GAMG(): 1) N=100902, n data cols=6, nnz/row (ave)=207, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 34.5235 nnz ave. (N=16817)
[0] PCGAMGProlongator_AGG(): New grid 1041 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.387504e+00 min=1.113469e-01 PC=jacobi
[0] PCSetUp_GAMG(): 2) N=6246, n data cols=6, nnz/row (ave)=398, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 65.9061 nnz ave. (N=1041)
[0] PCGAMGProlongator_AGG(): New grid 38 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.352516e+00 min=4.279834e-02 PC=jacobi
[0] PCSetUp_GAMG(): 3) N=228, n data cols=6, nnz/row (ave)=207, 4 active pes
[0] PCGAMGFilterGraph(): 100.% nnz after filtering, with threshold 0., 33.875 nnz ave. (N=38)
[0] PCGAMGProlongator_AGG(): New grid 1 nodes
[0] PCGAMGOptProlongator_AGG(): Smooth P0: max eigen=2.109888e+00 min=7.508213e-02 PC=jacobi
[0] PCGAMGCreateLevel_GAMG(): Number of equations (loc) 0 with simple aggregation
[0] PCSetUp_GAMG(): 4) N=6, n data cols=6, nnz/row (ave)=6, 1 active pes
[0] PCSetUp_GAMG(): HARD stop of coarsening on level 3. Grid too small: 1 block nodes
[0] PCSetUp_GAMG(): 5 levels, grid complexity = 1.80239
KSP Object: 4 MPI processes
type: gmres
restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement
happy breakdown tolerance 1e-30
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-08, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
PC Object: 4 MPI processes
type: gamg
type is MULTIPLICATIVE, levels=5 cycles=v
Cycles per PCApply=1
Using externally compute Galerkin coarse grid matrices
GAMG specific options
Threshold for dropping small values in graph on each level = 0. 0. 0.
Threshold scaling factor for each level not specified = 1.
AGG specific options
Symmetric graph false
Number of levels to square graph 0
Number smoothing steps 2
Coarse grid solver -- level -------------------------------
KSP Object: (mg_coarse_) 4 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_) 4 MPI processes
type: bjacobi
number of blocks = 4
Local solve is same for all blocks, in the following KSP and PC objects:
[0] PetscCommDuplicate(): Using internal PETSc communicator 4358889024 140338640884000
KSP Object: (mg_coarse_sub_) 1 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_sub_) 1 MPI processes
type: lu
out-of-place factorization
tolerance for zero pivot 2.22045e-14
using diagonal shift on blocks to prevent zero pivot [INBLOCKS]
matrix ordering: nd
factor fill ratio given 5., needed 1.
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqaij
rows=6, cols=6, bs=6
package used to perform factorization: petsc
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 2 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 1 MPI processes
type: seqaij
rows=6, cols=6, bs=6
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node routines: found 2 nodes, limit used is 5
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=6, cols=6, bs=6
total: nonzeros=36, allocated nonzeros=36
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 2 nodes, limit used is 5
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (mg_levels_1_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_1_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=228, cols=228, bs=6
total: nonzeros=47232, allocated nonzeros=47232
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 10 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (mg_levels_2_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_2_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=6246, cols=6246, bs=6
total: nonzeros=2489868, allocated nonzeros=2489868
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 490 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 3 -------------------------------
KSP Object: (mg_levels_3_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_3_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: 4 MPI processes
type: mpiaij
rows=100902, cols=100902, bs=6
total: nonzeros=20912652, allocated nonzeros=20912652
total number of mallocs used during MatSetValues calls =0
using nonscalable MatPtAP() implementation
using I-node (on process 0) routines: found 8420 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 4 -------------------------------
KSP Object: (mg_levels_4_) 4 MPI processes
type: richardson
damping factor=1.
maximum iterations=4, nonzero initial guess
tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_levels_4_) 4 MPI processes
type: sor
type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Mat Object: () 4 MPI processes
type: mpiaij
rows=543606, cols=543606, bs=6
total: nonzeros=29224836, allocated nonzeros=29302596
total number of mallocs used during MatSetValues calls =0
has attached near null space
using I-node (on process 0) routines: found 45644 nodes, limit used is 5
> On Oct 30, 2018, at 1:51 PM, Mark Adams <mfadams at lbl.gov> wrote:
>
> The last two cases were cut off but that is OK.
>
> Lets just do -pc_gamg_square_graph 0 and please add/do -ksp_view -mg_levels_ksp_type richardson -pc_mg_levels 5 -gamg_est_ksp_type cg
>
> Send the grep GAMG again, and the ksp_view output.
>
> And then do this again with: -pc_gamg_agg_nsmooths
>
> I think the eigen estimators are messed up on the coarse grids.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20181031/9aadee68/attachment-0001.html>
More information about the petsc-users
mailing list