[petsc-users] CPR-AMG: SNES with two cores worse than with one
Robert Annewandter
robert.annewandter at opengosim.com
Thu Jul 6 14:03:04 CDT 2017
Hi all,
I like to understand why the SNES of my CPR-AMG Two-Stage Preconditioner
(with KSPFGMRES + multipl. PCComposite (PCGalerkin with KSPGMRES +
BoomerAMG, PCBJacobi + PCLU init) on a 24,000 x 24,000 matrix) struggles
to converge when using two cores instead of one. Because of the adaptive
time stepping of the Newton, this leads to severe cuts in time step.
This is how I run it with two cores
mpirun \
-n 2 pflotran \
-pflotranin het.pflinput \
-ksp_monitor_true_residual \
-flow_snes_view \
-flow_snes_converged_reason \
-flow_sub_1_pc_type bjacobi \
-flow_sub_1_sub_pc_type lu \
-flow_sub_1_sub_pc_factor_pivot_in_blocks true\
-flow_sub_1_sub_pc_factor_nonzeros_along_diagonal \
-options_left \
-log_summary \
-info
With one core I get (after grepping the crap away from -info):
Step 32 Time= 1.80000E+01
[...]
0 2r: 1.58E-02 2x: 0.00E+00 2u: 0.00E+00 ir: 7.18E-03 iu: 0.00E+00
rsn: 0
[0] SNESComputeJacobian(): Rebuilding preconditioner
Residual norms for flow_ solve.
0 KSP unpreconditioned resid norm 1.581814306485e-02 true resid norm
1.581814306485e-02 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 5.697603110484e+07 true resid norm
5.175721849125e+03 ||r(i)||/||b|| 5.037527476892e+03
1 KSP preconditioned resid norm 5.041509073319e+06 true resid norm
3.251596928176e+02 ||r(i)||/||b|| 3.164777657484e+02
2 KSP preconditioned resid norm 1.043761838360e+06 true resid norm
8.957519558348e+01 ||r(i)||/||b|| 8.718349288342e+01
3 KSP preconditioned resid norm 1.129189815646e+05 true resid norm
2.722436912053e+00 ||r(i)||/||b|| 2.649746479496e+00
4 KSP preconditioned resid norm 8.829637298082e+04 true resid norm
8.026373593492e+00 ||r(i)||/||b|| 7.812065388300e+00
5 KSP preconditioned resid norm 6.506021637694e+04 true resid norm
3.479889319880e+00 ||r(i)||/||b|| 3.386974527698e+00
6 KSP preconditioned resid norm 6.392263200180e+04 true resid norm
3.819202631980e+00 ||r(i)||/||b|| 3.717228003987e+00
7 KSP preconditioned resid norm 2.464946645480e+04 true resid norm
7.329964753388e-01 ||r(i)||/||b|| 7.134251013911e-01
8 KSP preconditioned resid norm 2.603879153772e+03 true resid norm
2.035525412004e-02 ||r(i)||/||b|| 1.981175861414e-02
9 KSP preconditioned resid norm 1.774410462754e+02 true resid norm
3.001214973121e-03 ||r(i)||/||b|| 2.921081026352e-03
10 KSP preconditioned resid norm 1.664227038378e+01 true resid norm
3.413136309181e-04 ||r(i)||/||b|| 3.322003855903e-04
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
1.131868956745e+00 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 2.067297386780e+07 at iteration 11
11 KSP preconditioned resid norm 1.131868956745e+00 true resid norm
1.526261825526e-05 ||r(i)||/||b|| 1.485509868409e-05
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
2.148515820410e-14 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 1.581814306485e-02 at iteration 1
1 KSP unpreconditioned resid norm 2.148515820410e-14 true resid norm
2.148698024622e-14 ||r(i)||/||b|| 1.358375642332e-12
[0] SNESSolve_NEWTONLS(): iter=0, linear solve iterations=1
[0] SNESNEWTONLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX||
3.590873180642e-01 near zero implies inconsistent rhs
[0] SNESSolve_NEWTONLS(): fnorm=1.5818143064846742e-02,
gnorm=1.0695649833687331e-02, ynorm=4.6826522561266171e+02, lssucceed=0
[0] SNESConvergedDefault(): Converged due to small update length:
4.682652256127e+02 < 1.000000000000e-05 * 3.702480426117e+09
1 2r: 1.07E-02 2x: 3.70E+09 2u: 4.68E+02 ir: 5.05E-03 iu: 4.77E+01
rsn: stol
Nonlinear flow_ solve converged due to CONVERGED_SNORM_RELATIVE iterations 1
But with two cores I get:
Step 32 Time= 1.80000E+01
[...]
0 2r: 6.16E-03 2x: 0.00E+00 2u: 0.00E+00 ir: 3.63E-03 iu: 0.00E+00
rsn: 0
[0] SNESComputeJacobian(): Rebuilding preconditioner
Residual norms for flow_ solve.
0 KSP unpreconditioned resid norm 6.162760088924e-03 true resid norm
6.162760088924e-03 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 8.994949630499e+08 true resid norm
7.982144380936e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.950556502615e+08 true resid norm
1.550138696155e+00 ||r(i)||/||b|| 1.942007839218e+00
2 KSP preconditioned resid norm 1.044849684205e+08 true resid norm
2.166193480531e+00 ||r(i)||/||b|| 2.713798920631e+00
3 KSP preconditioned resid norm 8.209708619718e+06 true resid norm
3.076045005154e-01 ||r(i)||/||b|| 3.853657436340e-01
4 KSP preconditioned resid norm 3.027461352422e+05 true resid norm
1.207731865714e-02 ||r(i)||/||b|| 1.513041869549e-02
5 KSP preconditioned resid norm 1.595302164817e+04 true resid norm
4.123713694368e-04 ||r(i)||/||b|| 5.166172769585e-04
6 KSP preconditioned resid norm 1.898935810797e+03 true resid norm
8.275885058330e-05 ||r(i)||/||b|| 1.036799719897e-04
7 KSP preconditioned resid norm 1.429881682558e+02 true resid norm
4.751240525466e-06 ||r(i)||/||b|| 5.952335987324e-06
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
8.404003313455e+00 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 8.994949630499e+08 at iteration 8
8 KSP preconditioned resid norm 8.404003313455e+00 true resid norm
3.841921844578e-07 ||r(i)||/||b|| 4.813145016211e-07
1 KSP unpreconditioned resid norm 6.162162548202e-03 true resid norm
6.162162548202e-03 ||r(i)||/||b|| 9.999030400804e-01
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 4.360556381209e+07 true resid norm
1.000000245433e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 5.385519331932e+06 true resid norm
8.785183939860e-02 ||r(i)||/||b|| 8.785181783689e-02
2 KSP preconditioned resid norm 4.728931283459e+05 true resid norm
2.008708805316e-02 ||r(i)||/||b|| 2.008708312313e-02
3 KSP preconditioned resid norm 2.734215698319e+04 true resid norm
6.418720397673e-03 ||r(i)||/||b|| 6.418718822309e-03
4 KSP preconditioned resid norm 1.002270029334e+04 true resid norm
4.040289515991e-03 ||r(i)||/||b|| 4.040288524372e-03
5 KSP preconditioned resid norm 1.321280190971e+03 true resid norm
1.023292238313e-04 ||r(i)||/||b|| 1.023291987163e-04
6 KSP preconditioned resid norm 6.594292964815e+01 true resid norm
1.877106733170e-06 ||r(i)||/||b|| 1.877106272467e-06
7 KSP preconditioned resid norm 7.816325147216e+00 true resid norm
2.552611664980e-07 ||r(i)||/||b|| 2.552611038486e-07
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
6.391568446109e-01 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 4.360556381209e+07 at iteration 8
8 KSP preconditioned resid norm 6.391568446109e-01 true resid norm
1.680724939670e-08 ||r(i)||/||b|| 1.680724527166e-08
2 KSP unpreconditioned resid norm 4.328902922753e-07 true resid norm
4.328902922752e-07 ||r(i)||/||b|| 7.024292460341e-05
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 8.794597825780e+08 true resid norm
1.000000094566e+00 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 8.609906572102e+08 true resid norm
2.965044981249e+00 ||r(i)||/||b|| 2.965044700856e+00
2 KSP preconditioned resid norm 9.318108989314e+07 true resid norm
1.881262939380e+00 ||r(i)||/||b|| 1.881262761477e+00
3 KSP preconditioned resid norm 6.908723262483e+06 true resid norm
2.639592490398e-01 ||r(i)||/||b|| 2.639592240782e-01
4 KSP preconditioned resid norm 2.651677791227e+05 true resid norm
9.736480169584e-03 ||r(i)||/||b|| 9.736479248845e-03
5 KSP preconditioned resid norm 1.192178471172e+04 true resid norm
3.082839752692e-04 ||r(i)||/||b|| 3.082839461160e-04
6 KSP preconditioned resid norm 1.492201446262e+03 true resid norm
4.633866284506e-05 ||r(i)||/||b|| 4.633865846301e-05
7 KSP preconditioned resid norm 1.160670017241e+02 true resid norm
2.821157348522e-06 ||r(i)||/||b|| 2.821157081737e-06
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
6.447568262216e+00 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 8.794597825780e+08 at iteration 8
8 KSP preconditioned resid norm 6.447568262216e+00 true resid norm
1.516068561348e-07 ||r(i)||/||b|| 1.516068417980e-07
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
6.135731709822e-15 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 6.162760088924e-03 at iteration 3
3 KSP unpreconditioned resid norm 6.135731709822e-15 true resid norm
1.142020328809e-14 ||r(i)||/||b|| 1.853098793933e-12
[0] SNESSolve_NEWTONLS(): iter=0, linear solve iterations=3
[0] SNESNEWTONLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX||
1.998388224666e-02 near zero implies inconsistent rhs
[0] SNESSolve_NEWTONLS(): fnorm=6.1627600889243711e-03,
gnorm=1.0406503258190572e-02, ynorm=6.2999025681245366e+04, lssucceed=0
1 2r: 1.04E-02 2x: 3.70E+09 2u: 6.30E+04 ir: 6.54E-03 iu: 5.00E+04
rsn: 0
[0] SNESComputeJacobian(): Rebuilding preconditioner
Residual norms for flow_ solve.
0 KSP unpreconditioned resid norm 1.040650325819e-02 true resid norm
1.040650325819e-02 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 6.758906811264e+07 true resid norm
9.814998431686e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.503922806424e+06 true resid norm
2.275130113021e-01 ||r(i)||/||b|| 2.318013730574e-01
2 KSP preconditioned resid norm 3.316753614870e+05 true resid norm
3.820733530238e-02 ||r(i)||/||b|| 3.892750016040e-02
3 KSP preconditioned resid norm 2.956751700483e+04 true resid norm
2.143772538677e-03 ||r(i)||/||b|| 2.184180215207e-03
4 KSP preconditioned resid norm 1.277067042524e+03 true resid norm
9.093614251311e-05 ||r(i)||/||b|| 9.265018547485e-05
5 KSP preconditioned resid norm 1.060996002446e+02 true resid norm
1.042893700050e-05 ||r(i)||/||b|| 1.062551061326e-05
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
5.058127343285e+00 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 6.758906811264e+07 at iteration 6
6 KSP preconditioned resid norm 5.058127343285e+00 true resid norm
4.054770602120e-07 ||r(i)||/||b|| 4.131198420807e-07
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
4.449606189225e-10 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 1.040650325819e-02 at iteration 1
1 KSP unpreconditioned resid norm 4.449606189225e-10 true resid norm
4.449606189353e-10 ||r(i)||/||b|| 4.275793779098e-08
[0] SNESSolve_NEWTONLS(): iter=1, linear solve iterations=1
[0] SNESNEWTONLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX||
4.300066663571e-02 near zero implies inconsistent rhs
[0] SNESSolve_NEWTONLS(): fnorm=1.0406503258190572e-02,
gnorm=7.3566280848133728e-02, ynorm=7.9500485128639993e+04, lssucceed=0
2 2r: 7.36E-02 2x: 3.70E+09 2u: 7.95E+04 ir: 4.62E-02 iu: 5.00E+04
rsn: 0
[0] SNESComputeJacobian(): Rebuilding preconditioner
Residual norms for flow_ solve.
0 KSP unpreconditioned resid norm 7.356628084813e-02 true resid norm
7.356628084813e-02 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 7.253424029194e+06 true resid norm
9.647008645250e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 7.126940190688e+06 true resid norm
1.228009197928e+00 ||r(i)||/||b|| 1.272942984800e+00
2 KSP preconditioned resid norm 9.391591432635e+05 true resid norm
7.804929162756e-01 ||r(i)||/||b|| 8.090517433711e-01
3 KSP preconditioned resid norm 6.538499674761e+04 true resid norm
5.503467432893e-02 ||r(i)||/||b|| 5.704843475602e-02
4 KSP preconditioned resid norm 1.593713396575e+04 true resid norm
8.902701363763e-02 ||r(i)||/||b|| 9.228457951208e-02
5 KSP preconditioned resid norm 4.837260621464e+02 true resid norm
2.966772992825e-03 ||r(i)||/||b|| 3.075329464213e-03
6 KSP preconditioned resid norm 1.681372767335e+02 true resid norm
5.312467443025e-04 ||r(i)||/||b|| 5.506854651406e-04
7 KSP preconditioned resid norm 1.271478850717e+01 true resid norm
2.123810020488e-05 ||r(i)||/||b|| 2.201521838103e-05
8 KSP preconditioned resid norm 1.262723712696e+00 true resid norm
1.150572715331e-06 ||r(i)||/||b|| 1.192673042641e-06
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
9.053072585125e-02 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 7.253424029194e+06 at iteration 9
9 KSP preconditioned resid norm 9.053072585125e-02 true resid norm
9.475050575058e-08 ||r(i)||/||b|| 9.821749853747e-08
1 KSP unpreconditioned resid norm 8.171589173162e-03 true resid norm
8.171589173162e-03 ||r(i)||/||b|| 1.110779161180e-01
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 4.345765068989e+07 true resid norm
9.999992231691e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 5.388715093466e+06 true resid norm
8.125387327699e-02 ||r(i)||/||b|| 8.125393639755e-02
2 KSP preconditioned resid norm 4.763725726436e+05 true resid norm
2.464285618036e-02 ||r(i)||/||b|| 2.464287532371e-02
3 KSP preconditioned resid norm 2.287746683380e+04 true resid norm
7.224823080100e-03 ||r(i)||/||b|| 7.224828692570e-03
4 KSP preconditioned resid norm 4.872858764091e+03 true resid norm
3.972261388893e-03 ||r(i)||/||b|| 3.972264474670e-03
5 KSP preconditioned resid norm 8.670449895323e+02 true resid norm
2.359005963873e-04 ||r(i)||/||b|| 2.359007796423e-04
6 KSP preconditioned resid norm 4.252589693890e+01 true resid norm
1.471904261226e-06 ||r(i)||/||b|| 1.471905404648e-06
7 KSP preconditioned resid norm 5.128476471782e+00 true resid norm
1.643725157865e-07 ||r(i)||/||b|| 1.643726434763e-07
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
4.311901915856e-01 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 4.345765068989e+07 at iteration 8
8 KSP preconditioned resid norm 4.311901915856e-01 true resid norm
1.166123921637e-08 ||r(i)||/||b|| 1.166124827519e-08
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
2.373662391739e-09 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 7.356628084813e-02 at iteration 2
2 KSP unpreconditioned resid norm 2.373662391739e-09 true resid norm
2.373662391658e-09 ||r(i)||/||b|| 3.226562990941e-08
[0] SNESSolve_NEWTONLS(): iter=2, linear solve iterations=2
[0] SNESNEWTONLSCheckResidual_Private(): ||J^T(F-Ax)||/||F-AX||
4.343326231305e-02 near zero implies inconsistent rhs
[0] SNESSolve_NEWTONLS(): fnorm=7.3566280848133728e-02,
gnorm=7.2259942496422647e-02, ynorm=6.3156901950486099e+04, lssucceed=0
3 2r: 7.23E-02 2x: 3.70E+09 2u: 6.32E+04 ir: 4.52E-02 iu: 5.00E+04
rsn: 0
[0] SNESComputeJacobian(): Rebuilding preconditioner
Residual norms for flow_ solve.
0 KSP unpreconditioned resid norm 7.225994249642e-02 true resid norm
7.225994249642e-02 ||r(i)||/||b|| 1.000000000000e+00
Residual norms for flow_sub_0_galerkin_ solve.
0 KSP preconditioned resid norm 7.705582590638e+05 true resid norm
9.649751442741e-01 ||r(i)||/||b|| 1.000000000000e+00
1 KSP preconditioned resid norm 2.444424220392e+04 true resid norm
8.243110200738e-03 ||r(i)||/||b|| 8.542303135630e-03
2 KSP preconditioned resid norm 2.080899648412e+03 true resid norm
7.642343147053e-04 ||r(i)||/||b|| 7.919730567570e-04
3 KSP preconditioned resid norm 9.911171129874e+02 true resid norm
5.904182179180e-05 ||r(i)||/||b|| 6.118481096859e-05
4 KSP preconditioned resid norm 5.258230282482e+02 true resid norm
2.043366677644e-04 ||r(i)||/||b|| 2.117532964210e-04
5 KSP preconditioned resid norm 5.522830460456e+01 true resid norm
1.710780366056e-05 ||r(i)||/||b|| 1.772875059225e-05
6 KSP preconditioned resid norm 5.922280741715e+00 true resid norm
1.543198740828e-06 ||r(i)||/||b|| 1.599210870855e-06
7 KSP preconditioned resid norm 3.339500859115e-01 true resid norm
1.221335666427e-07 ||r(i)||/||b|| 1.265665414984e-07
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
3.329208597672e-02 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 7.705582590638e+05 at iteration 8
8 KSP preconditioned resid norm 3.329208597672e-02 true resid norm
9.758240835324e-09 ||r(i)||/||b|| 1.011242713683e-08
[0] KSPConvergedDefault(): Linear solver has converged. Residual norm
2.697128456432e-11 is less than relative tolerance 1.000000000000e-07
times initial right hand side norm 7.225994249642e-02 at iteration 1
1 KSP unpreconditioned resid norm 2.697128456432e-11 true resid norm
2.697128457142e-11 ||r(i)||/||b|| 3.732536124389e-10
[0] SNESSolve_NEWTONLS(): iter=3, linear solve iterations=1
[0] SNESNEWTONLSCheckResidual_Private():
||J^T(F-Ax)||/||F-AX||4.329227684222e-02 near zero implies inconsistent rhs
[0] SNESSolve_NEWTONLS(): fnorm=7.2259942496422647e-02,
gnorm=5.4435602192925014e-01, ynorm=2.7049750229137400e+04, lssucceed=0
[0] SNESConvergedDefault(): Converged due to small update length:
2.704975022914e+04 < 1.000000000000e-05 * 3.702469482296e+09
4 2r: 5.44E-01 2x: 3.70E+09 2u: 2.70E+04 ir: 3.84E-01 iu: 2.34E+04
rsn: stol
Nonlinear flow_ solve converged due to CONVERGED_SNORM_RELATIVE iterations 4
As the simulation advances this behaviour leads to frequent time step
cuts because of 8 subsequently failed Newton iterations, which brings
the simulation practically to a halt.
Is the Block Jacobi not a good choice? Better ASM with huge overlap? Or
is there something wrong with my RHS? Maybe the SNES, SNESLS, KSP
tolerances need better tuning?
Grateful for any clarifying words!
Robert
My SNES_view is:
SNES Object: (flow_) 2 MPI processes
type: newtonls
maximum iterations=8, maximum function evaluations=10000
tolerances: relative=1e-05, absolute=1e-05, solution=1e-05
total number of linear solver iterations=1
total number of function evaluations=2
norm schedule ALWAYS
SNESLineSearch Object: (flow_) 2 MPI processes
type: basic
maxstep=1.000000e+08, minlambda=1.000000e-05
tolerances: relative=1.000000e-05, absolute=1.000000e-05,
lambda=1.000000e-08
maximum iterations=40
using user-defined precheck step
KSP Object: (flow_) 2 MPI processes
type: fgmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt
Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=200, initial guess is zero
tolerances: relative=1e-07, absolute=1e-50, divergence=10000.
right preconditioning
using UNPRECONDITIONED norm type for convergence test
PC Object: (flow_) 2 MPI processes
type: composite
Composite PC type - MULTIPLICATIVE
PCs on composite preconditioner follow
---------------------------------
PC Object: (flow_sub_0_) 2 MPI processes
type: galerkin
Galerkin PC
KSP on Galerkin follow
---------------------------------
KSP Object: (flow_sub_0_galerkin_) 2 MPI processes
type: gmres
GMRES: restart=30, using Classical (unmodified) Gram-Schmidt
Orthogonalization with no iterative refinement
GMRES: happy breakdown tolerance 1e-30
maximum iterations=200, initial guess is zero
tolerances: relative=1e-07, absolute=1e-50, divergence=10000.
left preconditioning
using PRECONDITIONED norm type for convergence test
PC Object: (flow_sub_0_galerkin_) 2 MPI processes
type: hypre
HYPRE BoomerAMG preconditioning
HYPRE BoomerAMG: Cycle type V
HYPRE BoomerAMG: Maximum number of levels 25
HYPRE BoomerAMG: Maximum number of iterations PER hypre call 1
HYPRE BoomerAMG: Convergence tolerance PER hypre call 0.
HYPRE BoomerAMG: Threshold for strong coupling 0.25
HYPRE BoomerAMG: Interpolation truncation factor 0.
HYPRE BoomerAMG: Interpolation: max elements per row 0
HYPRE BoomerAMG: Number of levels of aggressive coarsening 0
HYPRE BoomerAMG: Number of paths for aggressive coarsening 1
HYPRE BoomerAMG: Maximum row sums 0.9
HYPRE BoomerAMG: Sweeps down 1
HYPRE BoomerAMG: Sweeps up 1
HYPRE BoomerAMG: Sweeps on coarse 1
HYPRE BoomerAMG: Relax down symmetric-SOR/Jacobi
HYPRE BoomerAMG: Relax up symmetric-SOR/Jacobi
HYPRE BoomerAMG: Relax on coarse Gaussian-elimination
HYPRE BoomerAMG: Relax weight (all) 1.
HYPRE BoomerAMG: Outer relax weight (all) 1.
HYPRE BoomerAMG: Using CF-relaxation
HYPRE BoomerAMG: Not using more complex smoothers.
HYPRE BoomerAMG: Measure type local
HYPRE BoomerAMG: Coarsen type Falgout
HYPRE BoomerAMG: Interpolation type classical
linear system matrix = precond matrix:
Mat Object: 2 MPI processes
type: mpiaij
rows=8000, cols=8000
total: nonzeros=53600, allocated nonzeros=53600
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
linear system matrix = precond matrix:
Mat Object: (flow_) 2 MPI processes
type: mpibaij
rows=24000, cols=24000, bs=3
total: nonzeros=482400, allocated nonzeros=482400
total number of mallocs used during MatSetValues calls =0
PC Object: (flow_sub_1_) 2 MPI processes
type: bjacobi
block Jacobi: number of blocks = 2
Local solve is same for all blocks, in the following KSP and
PC objects:
KSP Object: (flow_sub_1_sub_) 1 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50,
divergence=10000. <------ not working: -flow_sub_1_sub_ksp_rtol 1e-7
left preconditioning
using NONE norm type for convergence test
PC Object: (flow_sub_1_sub_) 1 MPI processes
type: lu
out-of-place factorization
tolerance for zero pivot 2.22045e-14
matrix ordering: nd
factor fill ratio given 5., needed 18.3108
Factored matrix follows:
Mat Object: 1 MPI processes
type: seqbaij
rows=12000, cols=12000, bs=3
package used to perform factorization: petsc
total: nonzeros=4350654, allocated nonzeros=4350654
total number of mallocs used during MatSetValues calls =0
block size is 3
linear system matrix = precond matrix:
Mat Object: (flow_) 1 MPI processes
type: seqbaij
rows=12000, cols=12000, bs=3
total: nonzeros=237600, allocated nonzeros=237600
total number of mallocs used during MatSetValues calls =0
block size is 3
linear system matrix = precond matrix:
Mat Object: (flow_) 2 MPI processes
type: mpibaij
rows=24000, cols=24000, bs=3
total: nonzeros=482400, allocated nonzeros=482400
total number of mallocs used during MatSetValues calls =0
---------------------------------
linear system matrix = precond matrix:
Mat Object: (flow_) 2 MPI processes
type: mpibaij
rows=24000, cols=24000, bs=3
total: nonzeros=482400, allocated nonzeros=482400
total number of mallocs used during MatSetValues calls =0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170706/6d2d3d2d/attachment-0001.html>
More information about the petsc-users
mailing list