[petsc-users] How to speed up geometric multigrid

Michele Rosso mrosso at uci.edu
Mon Sep 23 10:55:12 CDT 2013


Hi,

I am successfully using PETSc to solve a 3D Poisson's equation with CG + 
MG .  Such equation arises from a projection algorithm for a multiphase 
incompressible flow simulation.
I set up the solver as I was suggested to do in a previous thread 
(title: "GAMG speed") and run a test case (liquid droplet with surface 
tension falling under the effect of gravity in a quiescent fluid).
The solution of the Poisson Equation via multigrid is correct but it 
becomes progressively slower and slower as the simulation progresses (I 
am performing successive solves) due to an increase in the number of 
iterations.
Since the solution of the Poisson equation is mission-critical, I need 
to speed it up as much as I can.
Could you please help me out with this?

I run the test case with the following options:

-pc_type mg  -pc_mg_galerkin  -pc_mg_levels 5   -mg_levels_ksp_type 
richardson -mg_levels_ksp_max_it 1
-mg_coarse_pc_type lu   -mg_coarse_pc_factor_mat_solver_package 
superlu_dist
-log_summary -ksp_view  -ksp_monitor_true_residual  -options_left

Please find the diagnostic for the final solve in the attached file 
"final.txt'.
Thank you,

Michele
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130923/30e04403/attachment-0001.html>
-------------- next part --------------
  0 KSP unpreconditioned resid norm 8.471082897815e-07 true resid norm 8.471082897815e-07 ||r(i)||/||b|| 1.000000000000e+00
  1 KSP unpreconditioned resid norm 5.889629222055e-07 true resid norm 5.889629222055e-07 ||r(i)||/||b|| 6.952628481034e-01
  2 KSP unpreconditioned resid norm 4.929221177373e-07 true resid norm 4.929221177373e-07 ||r(i)||/||b|| 5.818879636563e-01
  3 KSP unpreconditioned resid norm 4.077087883813e-07 true resid norm 4.077087883813e-07 ||r(i)||/||b|| 4.812947686847e-01
  4 KSP unpreconditioned resid norm 3.532777909896e-07 true resid norm 3.532777909896e-07 ||r(i)||/||b|| 4.170397046649e-01
  5 KSP unpreconditioned resid norm 3.133843759979e-07 true resid norm 3.133843759979e-07 ||r(i)||/||b|| 3.699460621247e-01
  6 KSP unpreconditioned resid norm 2.664884465415e-07 true resid norm 2.664884465415e-07 ||r(i)||/||b|| 3.145860449675e-01
  7 KSP unpreconditioned resid norm 2.266816102745e-07 true resid norm 2.266816102745e-07 ||r(i)||/||b|| 2.675946074532e-01
  8 KSP unpreconditioned resid norm 1.909025017919e-07 true resid norm 1.909025017919e-07 ||r(i)||/||b|| 2.253578486892e-01
  9 KSP unpreconditioned resid norm 1.635648456077e-07 true resid norm 1.635648456077e-07 ||r(i)||/||b|| 1.930861113989e-01
 10 KSP unpreconditioned resid norm 1.434980659348e-07 true resid norm 1.434980659348e-07 ||r(i)||/||b|| 1.693975465307e-01
 11 KSP unpreconditioned resid norm 1.271661365357e-07 true resid norm 1.271661365357e-07 ||r(i)||/||b|| 1.501179224306e-01
 12 KSP unpreconditioned resid norm 1.132794401526e-07 true resid norm 1.132794401526e-07 ||r(i)||/||b|| 1.337248631835e-01
 13 KSP unpreconditioned resid norm 1.002788428906e-07 true resid norm 1.002788428906e-07 ||r(i)||/||b|| 1.183778320909e-01
 14 KSP unpreconditioned resid norm 8.753317847137e-08 true resid norm 8.753317847137e-08 ||r(i)||/||b|| 1.033317458078e-01
 15 KSP unpreconditioned resid norm 7.770742874496e-08 true resid norm 7.770742874496e-08 ||r(i)||/||b|| 9.173257974492e-02
 16 KSP unpreconditioned resid norm 6.961935247483e-08 true resid norm 6.961935247483e-08 ||r(i)||/||b|| 8.218471394346e-02
 17 KSP unpreconditioned resid norm 6.471790768651e-08 true resid norm 6.471790768651e-08 ||r(i)||/||b|| 7.639862396248e-02
 18 KSP unpreconditioned resid norm 6.162477500458e-08 true resid norm 6.162477500458e-08 ||r(i)||/||b|| 7.274722222407e-02
 19 KSP unpreconditioned resid norm 6.116147101803e-08 true resid norm 6.116147101803e-08 ||r(i)||/||b|| 7.220029806792e-02
 20 KSP unpreconditioned resid norm 6.307214323139e-08 true resid norm 6.307214323139e-08 ||r(i)||/||b|| 7.445582104698e-02
 21 KSP unpreconditioned resid norm 6.569260578724e-08 true resid norm 6.569260578724e-08 ||r(i)||/||b|| 7.754924202688e-02
 22 KSP unpreconditioned resid norm 7.002116374918e-08 true resid norm 7.002116374918e-08 ||r(i)||/||b|| 8.265904677576e-02
 23 KSP unpreconditioned resid norm 7.648471462197e-08 true resid norm 7.648471462197e-08 ||r(i)||/||b|| 9.028918208521e-02
 24 KSP unpreconditioned resid norm 8.218354530044e-08 true resid norm 8.218354530044e-08 ||r(i)||/||b|| 9.701657543883e-02
 25 KSP unpreconditioned resid norm 8.777006719029e-08 true resid norm 8.777006719029e-08 ||r(i)||/||b|| 1.036113897704e-01
 26 KSP unpreconditioned resid norm 9.074938460336e-08 true resid norm 9.074938460336e-08 ||r(i)||/||b|| 1.071284341070e-01
 27 KSP unpreconditioned resid norm 9.311175500020e-08 true resid norm 9.311175500020e-08 ||r(i)||/||b|| 1.099171807470e-01
 28 KSP unpreconditioned resid norm 9.391328578136e-08 true resid norm 9.391328578136e-08 ||r(i)||/||b|| 1.108633771080e-01
 29 KSP unpreconditioned resid norm 9.397746582366e-08 true resid norm 9.397746582366e-08 ||r(i)||/||b|| 1.109391407891e-01
 30 KSP unpreconditioned resid norm 9.269761294239e-08 true resid norm 9.269761294239e-08 ||r(i)||/||b|| 1.094282915899e-01
 31 KSP unpreconditioned resid norm 9.037592803258e-08 true resid norm 9.037592803258e-08 ||r(i)||/||b|| 1.066875736228e-01
 32 KSP unpreconditioned resid norm 8.739586906294e-08 true resid norm 8.739586906294e-08 ||r(i)||/||b|| 1.031696538886e-01
 33 KSP unpreconditioned resid norm 8.294737580326e-08 true resid norm 8.294737580326e-08 ||r(i)||/||b|| 9.791826712575e-02
 34 KSP unpreconditioned resid norm 7.825811344542e-08 true resid norm 7.825811344542e-08 ||r(i)||/||b|| 9.238265566450e-02
 35 KSP unpreconditioned resid norm 7.370584204445e-08 true resid norm 7.370584204445e-08 ||r(i)||/||b|| 8.700876019459e-02
 36 KSP unpreconditioned resid norm 6.944350218542e-08 true resid norm 6.944350218542e-08 ||r(i)||/||b|| 8.197712503006e-02
 37 KSP unpreconditioned resid norm 6.611084651652e-08 true resid norm 6.611084651652e-08 ||r(i)||/||b|| 7.804296961086e-02
 38 KSP unpreconditioned resid norm 6.155389358744e-08 true resid norm 6.155389358744e-08 ||r(i)||/||b|| 7.266354765968e-02
 39 KSP unpreconditioned resid norm 5.877895869357e-08 true resid norm 5.877895869357e-08 ||r(i)||/||b|| 6.938777415191e-02
 40 KSP unpreconditioned resid norm 5.544264371508e-08 true resid norm 5.544264371508e-08 ||r(i)||/||b|| 6.544929896671e-02
 41 KSP unpreconditioned resid norm 5.273241362566e-08 true resid norm 5.273241362566e-08 ||r(i)||/||b|| 6.224990861471e-02
 42 KSP unpreconditioned resid norm 5.119250885272e-08 true resid norm 5.119250885272e-08 ||r(i)||/||b|| 6.043207163741e-02
 43 KSP unpreconditioned resid norm 4.982771093310e-08 true resid norm 4.982771093310e-08 ||r(i)||/||b|| 5.882094595716e-02
 44 KSP unpreconditioned resid norm 4.811237834099e-08 true resid norm 4.811237834099e-08 ||r(i)||/||b|| 5.679601878692e-02
 45 KSP unpreconditioned resid norm 4.628304375229e-08 true resid norm 4.628304375229e-08 ||r(i)||/||b|| 5.463651378530e-02
 46 KSP unpreconditioned resid norm 4.476871541552e-08 true resid norm 4.476871541552e-08 ||r(i)||/||b|| 5.284886944864e-02
 47 KSP unpreconditioned resid norm 4.419875774021e-08 true resid norm 4.419875774021e-08 ||r(i)||/||b|| 5.217604204016e-02
 48 KSP unpreconditioned resid norm 4.359181679552e-08 true resid norm 4.359181679552e-08 ||r(i)||/||b|| 5.145955637710e-02
 49 KSP unpreconditioned resid norm 4.304314318039e-08 true resid norm 4.304314318039e-08 ||r(i)||/||b|| 5.081185451685e-02
 50 KSP unpreconditioned resid norm 4.347264822128e-08 true resid norm 4.347264822128e-08 ||r(i)||/||b|| 5.131887946993e-02
 51 KSP unpreconditioned resid norm 4.378084979329e-08 true resid norm 4.378084979329e-08 ||r(i)||/||b|| 5.168270730132e-02
 52 KSP unpreconditioned resid norm 4.459872124486e-08 true resid norm 4.459872124486e-08 ||r(i)||/||b|| 5.264819360505e-02
 53 KSP unpreconditioned resid norm 4.586835225867e-08 true resid norm 4.586835225867e-08 ||r(i)||/||b|| 5.414697602652e-02
 54 KSP unpreconditioned resid norm 4.759826693288e-08 true resid norm 4.759826693288e-08 ||r(i)||/||b|| 5.618911714955e-02
 55 KSP unpreconditioned resid norm 4.951983455569e-08 true resid norm 4.951983455569e-08 ||r(i)||/||b|| 5.845750201366e-02
 56 KSP unpreconditioned resid norm 5.077348189601e-08 true resid norm 5.077348189601e-08 ||r(i)||/||b|| 5.993741592247e-02
 57 KSP unpreconditioned resid norm 5.187532881197e-08 true resid norm 5.187532881197e-08 ||r(i)||/||b|| 6.123813146174e-02
 58 KSP unpreconditioned resid norm 5.194414285622e-08 true resid norm 5.194414285622e-08 ||r(i)||/||b|| 6.131936552011e-02
 59 KSP unpreconditioned resid norm 5.114127723640e-08 true resid norm 5.114127723640e-08 ||r(i)||/||b|| 6.037159339994e-02
 60 KSP unpreconditioned resid norm 4.987798575055e-08 true resid norm 4.987798575055e-08 ||r(i)||/||b|| 5.888029470638e-02
 61 KSP unpreconditioned resid norm 4.837751000107e-08 true resid norm 4.837751000107e-08 ||r(i)||/||b|| 5.710900316363e-02
 62 KSP unpreconditioned resid norm 4.735998413707e-08 true resid norm 4.735998413707e-08 ||r(i)||/||b|| 5.590782749782e-02
 63 KSP unpreconditioned resid norm 4.609537746550e-08 true resid norm 4.609537746550e-08 ||r(i)||/||b|| 5.441497624511e-02
 64 KSP unpreconditioned resid norm 4.470951308242e-08 true resid norm 4.470951308242e-08 ||r(i)||/||b|| 5.277898188667e-02
 65 KSP unpreconditioned resid norm 4.325085133153e-08 true resid norm 4.325085133153e-08 ||r(i)||/||b|| 5.105705120970e-02
 66 KSP unpreconditioned resid norm 4.181200034623e-08 true resid norm 4.181200034623e-08 ||r(i)||/||b|| 4.935850687640e-02
 67 KSP unpreconditioned resid norm 4.054904820539e-08 true resid norm 4.054904820539e-08 ||r(i)||/||b|| 4.786760877508e-02
 68 KSP unpreconditioned resid norm 3.941638106292e-08 true resid norm 3.941638106292e-08 ||r(i)||/||b|| 4.653051037086e-02
 69 KSP unpreconditioned resid norm 3.818351979063e-08 true resid norm 3.818351979063e-08 ||r(i)||/||b|| 4.507513413720e-02
 70 KSP unpreconditioned resid norm 3.732269485449e-08 true resid norm 3.732269485449e-08 ||r(i)||/||b|| 4.405894181973e-02
 71 KSP unpreconditioned resid norm 3.638709503712e-08 true resid norm 3.638709503712e-08 ||r(i)||/||b|| 4.295447875562e-02
 72 KSP unpreconditioned resid norm 3.507340393789e-08 true resid norm 3.507340393789e-08 ||r(i)||/||b|| 4.140368399292e-02
 73 KSP unpreconditioned resid norm 3.378527900460e-08 true resid norm 3.378527900460e-08 ||r(i)||/||b|| 3.988306974699e-02
 74 KSP unpreconditioned resid norm 3.213894671931e-08 true resid norm 3.213894671931e-08 ||r(i)||/||b|| 3.793959651558e-02
 75 KSP unpreconditioned resid norm 3.071349886547e-08 true resid norm 3.071349886547e-08 ||r(i)||/||b|| 3.625687439960e-02
 76 KSP unpreconditioned resid norm 2.982607199289e-08 true resid norm 2.982607199289e-08 ||r(i)||/||b|| 3.520927885216e-02
 77 KSP unpreconditioned resid norm 2.896588396864e-08 true resid norm 2.896588396864e-08 ||r(i)||/||b|| 3.419383840065e-02
 78 KSP unpreconditioned resid norm 2.849350659098e-08 true resid norm 2.849350659098e-08 ||r(i)||/||b|| 3.363620322772e-02
 79 KSP unpreconditioned resid norm 2.839644438060e-08 true resid norm 2.839644438060e-08 ||r(i)||/||b|| 3.352162258726e-02
 80 KSP unpreconditioned resid norm 2.878125026793e-08 true resid norm 2.878125026793e-08 ||r(i)||/||b|| 3.397588078775e-02
 81 KSP unpreconditioned resid norm 2.927107179400e-08 true resid norm 2.927107179400e-08 ||r(i)||/||b|| 3.455410854444e-02
 82 KSP unpreconditioned resid norm 2.995930520842e-08 true resid norm 2.995930520842e-08 ||r(i)||/||b|| 3.536655888016e-02
 83 KSP unpreconditioned resid norm 3.045914214927e-08 true resid norm 3.045914214927e-08 ||r(i)||/||b|| 3.595660969995e-02
 84 KSP unpreconditioned resid norm 3.095556579777e-08 true resid norm 3.095556579777e-08 ||r(i)||/||b|| 3.654263117382e-02
 85 KSP unpreconditioned resid norm 3.108374161273e-08 true resid norm 3.108374161273e-08 ||r(i)||/||b|| 3.669394100812e-02
 86 KSP unpreconditioned resid norm 3.098332255348e-08 true resid norm 3.098332255348e-08 ||r(i)||/||b|| 3.657539765249e-02
 87 KSP unpreconditioned resid norm 3.090888002411e-08 true resid norm 3.090888002411e-08 ||r(i)||/||b|| 3.648751924276e-02
 88 KSP unpreconditioned resid norm 3.060254382784e-08 true resid norm 3.060254382784e-08 ||r(i)||/||b|| 3.612589346250e-02
 89 KSP unpreconditioned resid norm 3.054287138188e-08 true resid norm 3.054287138188e-08 ||r(i)||/||b|| 3.605545093858e-02
 90 KSP unpreconditioned resid norm 3.019326857855e-08 true resid norm 3.019326857855e-08 ||r(i)||/||b|| 3.564274950767e-02
 91 KSP unpreconditioned resid norm 2.981415246582e-08 true resid norm 2.981415246582e-08 ||r(i)||/||b|| 3.519520800996e-02
 92 KSP unpreconditioned resid norm 2.934425853265e-08 true resid norm 2.934425853265e-08 ||r(i)||/||b|| 3.464050451002e-02
 93 KSP unpreconditioned resid norm 2.892868376837e-08 true resid norm 2.892868376837e-08 ||r(i)||/||b|| 3.414992406205e-02
 94 KSP unpreconditioned resid norm 2.807160358194e-08 true resid norm 2.807160358194e-08 ||r(i)||/||b|| 3.313815237150e-02
 95 KSP unpreconditioned resid norm 2.733196031734e-08 true resid norm 2.733196031734e-08 ||r(i)||/||b|| 3.226501339561e-02
 96 KSP unpreconditioned resid norm 2.674172249913e-08 true resid norm 2.674172249913e-08 ||r(i)||/||b|| 3.156824554984e-02
 97 KSP unpreconditioned resid norm 2.570861397552e-08 true resid norm 2.570861397552e-08 ||r(i)||/||b|| 3.034867476288e-02
 98 KSP unpreconditioned resid norm 2.480290643102e-08 true resid norm 2.480290643102e-08 ||r(i)||/||b|| 2.927949912687e-02
 99 KSP unpreconditioned resid norm 2.393472264734e-08 true resid norm 2.393472264734e-08 ||r(i)||/||b|| 2.825461978836e-02
100 KSP unpreconditioned resid norm 2.275435374520e-08 true resid norm 2.275435374520e-08 ||r(i)||/||b|| 2.686121009520e-02
101 KSP unpreconditioned resid norm 2.178902400109e-08 true resid norm 2.178902400109e-08 ||r(i)||/||b|| 2.572165125041e-02
102 KSP unpreconditioned resid norm 2.056965586266e-08 true resid norm 2.056965586266e-08 ||r(i)||/||b|| 2.428220348069e-02
103 KSP unpreconditioned resid norm 1.971345133315e-08 true resid norm 1.971345133315e-08 ||r(i)||/||b|| 2.327146549143e-02
104 KSP unpreconditioned resid norm 1.874079176158e-08 true resid norm 1.874079176158e-08 ||r(i)||/||b|| 2.212325388341e-02
105 KSP unpreconditioned resid norm 1.786435683475e-08 true resid norm 1.786435683475e-08 ||r(i)||/||b|| 2.108863418083e-02
106 KSP unpreconditioned resid norm 1.706457796840e-08 true resid norm 1.706457796840e-08 ||r(i)||/||b|| 2.014450593183e-02
107 KSP unpreconditioned resid norm 1.630473998948e-08 true resid norm 1.630473998948e-08 ||r(i)||/||b|| 1.924752736594e-02
108 KSP unpreconditioned resid norm 1.574283867690e-08 true resid norm 1.574283867690e-08 ||r(i)||/||b|| 1.858421038585e-02
109 KSP unpreconditioned resid norm 1.556968009361e-08 true resid norm 1.556968009361e-08 ||r(i)||/||b|| 1.837979899551e-02
110 KSP unpreconditioned resid norm 1.523973136297e-08 true resid norm 1.523973136297e-08 ||r(i)||/||b|| 1.799029893439e-02
111 KSP unpreconditioned resid norm 1.488557248475e-08 true resid norm 1.488557248475e-08 ||r(i)||/||b|| 1.757221911804e-02
112 KSP unpreconditioned resid norm 1.449864006948e-08 true resid norm 1.449864006948e-08 ||r(i)||/||b|| 1.711545057978e-02
113 KSP unpreconditioned resid norm 1.432126672372e-08 true resid norm 1.432126672372e-08 ||r(i)||/||b|| 1.690606371874e-02
114 KSP unpreconditioned resid norm 1.415386315852e-08 true resid norm 1.415386315852e-08 ||r(i)||/||b|| 1.670844605023e-02
115 KSP unpreconditioned resid norm 1.382237564251e-08 true resid norm 1.382237564251e-08 ||r(i)||/||b|| 1.631712947358e-02
116 KSP unpreconditioned resid norm 1.352938132525e-08 true resid norm 1.352938132525e-08 ||r(i)||/||b|| 1.597125360294e-02
117 KSP unpreconditioned resid norm 1.332023721901e-08 true resid norm 1.332023721901e-08 ||r(i)||/||b|| 1.572436178431e-02
118 KSP unpreconditioned resid norm 1.294810005035e-08 true resid norm 1.294810005035e-08 ||r(i)||/||b|| 1.528505883668e-02
119 KSP unpreconditioned resid norm 1.260835085133e-08 true resid norm 1.260835085133e-08 ||r(i)||/||b|| 1.488398945380e-02
120 KSP unpreconditioned resid norm 1.232815412303e-08 true resid norm 1.232815412303e-08 ||r(i)||/||b|| 1.455322096566e-02
121 KSP unpreconditioned resid norm 1.200898521152e-08 true resid norm 1.200898521152e-08 ||r(i)||/||b|| 1.417644633677e-02
122 KSP unpreconditioned resid norm 1.180686235502e-08 true resid norm 1.180686235502e-08 ||r(i)||/||b|| 1.393784300949e-02
123 KSP unpreconditioned resid norm 1.152359469099e-08 true resid norm 1.152359469099e-08 ||r(i)||/||b|| 1.360344932282e-02
124 KSP unpreconditioned resid norm 1.129846801969e-08 true resid norm 1.129846801969e-08 ||r(i)||/||b|| 1.333769030003e-02
125 KSP unpreconditioned resid norm 1.081947011589e-08 true resid norm 1.081947011589e-08 ||r(i)||/||b|| 1.277223968459e-02
126 KSP unpreconditioned resid norm 1.036954512419e-08 true resid norm 1.036954512419e-08 ||r(i)||/||b|| 1.224110925283e-02
127 KSP unpreconditioned resid norm 9.744507014406e-09 true resid norm 9.744507014406e-09 ||r(i)||/||b|| 1.150326012855e-02
128 KSP unpreconditioned resid norm 9.160148503681e-09 true resid norm 9.160148503681e-09 ||r(i)||/||b|| 1.081343272658e-02
129 KSP unpreconditioned resid norm 8.713396021123e-09 true resid norm 8.713396021123e-09 ||r(i)||/||b|| 1.028604739941e-02
130 KSP unpreconditioned resid norm 8.065538786935e-09 true resid norm 8.065538786935e-09 ||r(i)||/||b|| 9.521260604138e-03
131 KSP unpreconditioned resid norm 7.600710595219e-09 true resid norm 7.600710595219e-09 ||r(i)||/||b|| 8.972537144194e-03
132 KSP unpreconditioned resid norm 7.042499013017e-09 true resid norm 7.042499013017e-09 ||r(i)||/||b|| 8.313575841447e-03
133 KSP unpreconditioned resid norm 6.488293059262e-09 true resid norm 6.488293059262e-09 ||r(i)||/||b|| 7.659343129478e-03
134 KSP unpreconditioned resid norm 6.095008004956e-09 true resid norm 6.095008004956e-09 ||r(i)||/||b|| 7.195075385850e-03
135 KSP unpreconditioned resid norm 5.689629867887e-09 true resid norm 5.689629867887e-09 ||r(i)||/||b|| 6.716531919850e-03
136 KSP unpreconditioned resid norm 5.350918350727e-09 true resid norm 5.350918350727e-09 ||r(i)||/||b|| 6.316687506514e-03
137 KSP unpreconditioned resid norm 5.077782670826e-09 true resid norm 5.077782670826e-09 ||r(i)||/||b|| 5.994254491520e-03
138 KSP unpreconditioned resid norm 4.693456261445e-09 true resid norm 4.693456261445e-09 ||r(i)||/||b|| 5.540562308339e-03
139 KSP unpreconditioned resid norm 4.327021695310e-09 true resid norm 4.327021695310e-09 ||r(i)||/||b|| 5.107991206681e-03
140 KSP unpreconditioned resid norm 4.022960135943e-09 true resid norm 4.022960135943e-09 ||r(i)||/||b|| 4.749050604830e-03
141 KSP unpreconditioned resid norm 3.803745635547e-09 true resid norm 3.803745635547e-09 ||r(i)||/||b|| 4.490270820662e-03
142 KSP unpreconditioned resid norm 3.581362358291e-09 true resid norm 3.581362358291e-09 ||r(i)||/||b|| 4.227750337817e-03
143 KSP unpreconditioned resid norm 3.408132039231e-09 true resid norm 3.408132039231e-09 ||r(i)||/||b|| 4.023254264351e-03
144 KSP unpreconditioned resid norm 3.344832032939e-09 true resid norm 3.344832032939e-09 ||r(i)||/||b|| 3.948529454011e-03
145 KSP unpreconditioned resid norm 3.208107857745e-09 true resid norm 3.208107857745e-09 ||r(i)||/||b|| 3.787128394851e-03
146 KSP unpreconditioned resid norm 3.087343355611e-09 true resid norm 3.087343355611e-09 ||r(i)||/||b|| 3.644567516164e-03
147 KSP unpreconditioned resid norm 3.022538018497e-09 true resid norm 3.022538018497e-09 ||r(i)||/||b|| 3.568065682932e-03
148 KSP unpreconditioned resid norm 2.923230289033e-09 true resid norm 2.923230289033e-09 ||r(i)||/||b|| 3.450834237246e-03
149 KSP unpreconditioned resid norm 2.833109002568e-09 true resid norm 2.833109002568e-09 ||r(i)||/||b|| 3.344447264586e-03
150 KSP unpreconditioned resid norm 2.753952392615e-09 true resid norm 2.753952392615e-09 ||r(i)||/||b|| 3.251003945818e-03
151 KSP unpreconditioned resid norm 2.677124193740e-09 true resid norm 2.677124193740e-09 ||r(i)||/||b|| 3.160309285168e-03
152 KSP unpreconditioned resid norm 2.578412706791e-09 true resid norm 2.578412706791e-09 ||r(i)||/||b|| 3.043781695792e-03
153 KSP unpreconditioned resid norm 2.449680459273e-09 true resid norm 2.449680459273e-09 ||r(i)||/||b|| 2.891815000305e-03
154 KSP unpreconditioned resid norm 2.350739446063e-09 true resid norm 2.350739446063e-09 ||r(i)||/||b|| 2.775016458249e-03
155 KSP unpreconditioned resid norm 2.230331194218e-09 true resid norm 2.230331194218e-09 ||r(i)||/||b|| 2.632876128261e-03
156 KSP unpreconditioned resid norm 2.095716328619e-09 true resid norm 2.095716328619e-09 ||r(i)||/||b|| 2.473965080851e-03
157 KSP unpreconditioned resid norm 1.967063208093e-09 true resid norm 1.967063208093e-09 ||r(i)||/||b|| 2.322091793720e-03
158 KSP unpreconditioned resid norm 1.861768684115e-09 true resid norm 1.861768684115e-09 ||r(i)||/||b|| 2.197793017225e-03
159 KSP unpreconditioned resid norm 1.789027091537e-09 true resid norm 1.789027091537e-09 ||r(i)||/||b|| 2.111922540622e-03
160 KSP unpreconditioned resid norm 1.684922987165e-09 true resid norm 1.684922987165e-09 ||r(i)||/||b|| 1.989029038542e-03
161 KSP unpreconditioned resid norm 1.582609227035e-09 true resid norm 1.582609227035e-09 ||r(i)||/||b|| 1.868249013882e-03
162 KSP unpreconditioned resid norm 1.501597611234e-09 true resid norm 1.501597611234e-09 ||r(i)||/||b|| 1.772615885534e-03
163 KSP unpreconditioned resid norm 1.395819010765e-09 true resid norm 1.395819010765e-09 ||r(i)||/||b|| 1.647745663219e-03
164 KSP unpreconditioned resid norm 1.271906790435e-09 true resid norm 1.271906790435e-09 ||r(i)||/||b|| 1.501468945326e-03
165 KSP unpreconditioned resid norm 1.160968729258e-09 true resid norm 1.160968729258e-09 ||r(i)||/||b|| 1.370508048690e-03
166 KSP unpreconditioned resid norm 1.086312221042e-09 true resid norm 1.086312221042e-09 ||r(i)||/||b|| 1.282377039802e-03
167 KSP unpreconditioned resid norm 1.010733613894e-09 true resid norm 1.010733613894e-09 ||r(i)||/||b|| 1.193157505465e-03
168 KSP unpreconditioned resid norm 9.659506407625e-10 true resid norm 9.659506407625e-10 ||r(i)||/||b|| 1.140291804973e-03
169 KSP unpreconditioned resid norm 9.142920667557e-10 true resid norm 9.142920667557e-10 ||r(i)||/||b|| 1.079309549658e-03
170 KSP unpreconditioned resid norm 8.777007956409e-10 true resid norm 8.777007956409e-10 ||r(i)||/||b|| 1.036114043775e-03
171 KSP unpreconditioned resid norm 8.651938742751e-10 true resid norm 8.651938742751e-10 ||r(i)||/||b|| 1.021349790472e-03
172 KSP unpreconditioned resid norm 8.449669697471e-10 true resid norm 8.449669697471e-10 ||r(i)||/||b|| 9.974722003548e-04
173 KSP unpreconditioned resid norm 8.226753443690e-10 true resid norm 8.226753443690e-10 ||r(i)||/||b|| 9.711572349047e-04
174 KSP unpreconditioned resid norm 8.119036890805e-10 true resid norm 8.119036890805e-10 ||r(i)||/||b|| 9.584414399840e-04
175 KSP unpreconditioned resid norm 7.969494091924e-10 true resid norm 7.969494091924e-10 ||r(i)||/||b|| 9.407881126958e-04
176 KSP unpreconditioned resid norm 7.927024508324e-10 true resid norm 7.927024508324e-10 ||r(i)||/||b|| 9.357746351850e-04
177 KSP unpreconditioned resid norm 7.828725447824e-10 true resid norm 7.828725447824e-10 ||r(i)||/||b|| 9.241705626377e-04
178 KSP unpreconditioned resid norm 7.494508082515e-10 true resid norm 7.494508082515e-10 ||r(i)||/||b|| 8.847166499160e-04
179 KSP unpreconditioned resid norm 7.231444464622e-10 true resid norm 7.231444464622e-10 ||r(i)||/||b|| 8.536623418581e-04
180 KSP unpreconditioned resid norm 7.011791261722e-10 true resid norm 7.011791261722e-10 ||r(i)||/||b|| 8.277325751978e-04
181 KSP unpreconditioned resid norm 6.696139551270e-10 true resid norm 6.696139551271e-10 ||r(i)||/||b|| 7.904703131872e-04
182 KSP unpreconditioned resid norm 6.451235118753e-10 true resid norm 6.451235118753e-10 ||r(i)||/||b|| 7.615596726621e-04
183 KSP unpreconditioned resid norm 6.306916481370e-10 true resid norm 6.306916481370e-10 ||r(i)||/||b|| 7.445230506476e-04
184 KSP unpreconditioned resid norm 6.100227035205e-10 true resid norm 6.100227035205e-10 ||r(i)||/||b|| 7.201236381217e-04
185 KSP unpreconditioned resid norm 5.919504539386e-10 true resid norm 5.919504539386e-10 ||r(i)||/||b|| 6.987895893349e-04
186 KSP unpreconditioned resid norm 5.856700542909e-10 true resid norm 5.856700542909e-10 ||r(i)||/||b|| 6.913756615957e-04
187 KSP unpreconditioned resid norm 5.715591865793e-10 true resid norm 5.715591865793e-10 ||r(i)||/||b|| 6.747179710952e-04
188 KSP unpreconditioned resid norm 5.547066799722e-10 true resid norm 5.547066799722e-10 ||r(i)||/||b|| 6.548238125674e-04
189 KSP unpreconditioned resid norm 5.480604885087e-10 true resid norm 5.480604885087e-10 ||r(i)||/||b|| 6.469780724848e-04
190 KSP unpreconditioned resid norm 5.296236594065e-10 true resid norm 5.296236594065e-10 ||r(i)||/||b|| 6.252136424531e-04
191 KSP unpreconditioned resid norm 5.105824359200e-10 true resid norm 5.105824359201e-10 ||r(i)||/||b|| 6.027357329389e-04
192 KSP unpreconditioned resid norm 4.831649558134e-10 true resid norm 4.831649558134e-10 ||r(i)||/||b|| 5.703697645764e-04
193 KSP unpreconditioned resid norm 4.690710598203e-10 true resid norm 4.690710598203e-10 ||r(i)||/||b|| 5.537321089625e-04
194 KSP unpreconditioned resid norm 4.547545034138e-10 true resid norm 4.547545034138e-10 ||r(i)||/||b|| 5.368316057102e-04
195 KSP unpreconditioned resid norm 4.328632397413e-10 true resid norm 4.328632397413e-10 ||r(i)||/||b|| 5.109892618959e-04
196 KSP unpreconditioned resid norm 4.079089522157e-10 true resid norm 4.079089522157e-10 ||r(i)||/||b|| 4.815310594126e-04
197 KSP unpreconditioned resid norm 3.873059797615e-10 true resid norm 3.873059797615e-10 ||r(i)||/||b|| 4.572095261415e-04
198 KSP unpreconditioned resid norm 3.729969299802e-10 true resid norm 3.729969299802e-10 ||r(i)||/||b|| 4.403178843598e-04
199 KSP unpreconditioned resid norm 3.477970037895e-10 true resid norm 3.477970037895e-10 ||r(i)||/||b|| 4.105697087196e-04
200 KSP unpreconditioned resid norm 3.282214610955e-10 true resid norm 3.282214610955e-10 ||r(i)||/||b|| 3.874610425312e-04
201 KSP unpreconditioned resid norm 3.120709893337e-10 true resid norm 3.120709893337e-10 ||r(i)||/||b|| 3.683956267436e-04
202 KSP unpreconditioned resid norm 2.888671401828e-10 true resid norm 2.888671401828e-10 ||r(i)||/||b|| 3.410037933371e-04
203 KSP unpreconditioned resid norm 2.762162725563e-10 true resid norm 2.762162725563e-10 ||r(i)||/||b|| 3.260696133992e-04
204 KSP unpreconditioned resid norm 2.633334545155e-10 true resid norm 2.633334545155e-10 ||r(i)||/||b|| 3.108616191011e-04
205 KSP unpreconditioned resid norm 2.491344685484e-10 true resid norm 2.491344685484e-10 ||r(i)||/||b|| 2.940999061792e-04
206 KSP unpreconditioned resid norm 2.386964029507e-10 true resid norm 2.386964029507e-10 ||r(i)||/||b|| 2.817779094244e-04
207 KSP unpreconditioned resid norm 2.274820848858e-10 true resid norm 2.274820848858e-10 ||r(i)||/||b|| 2.685395570199e-04
208 KSP unpreconditioned resid norm 2.212409903975e-10 true resid norm 2.212409903975e-10 ||r(i)||/||b|| 2.611720284954e-04
209 KSP unpreconditioned resid norm 2.114227895993e-10 true resid norm 2.114227895993e-10 ||r(i)||/||b|| 2.495817738413e-04
210 KSP unpreconditioned resid norm 2.048332706014e-10 true resid norm 2.048332706014e-10 ||r(i)||/||b|| 2.418029348459e-04
211 KSP unpreconditioned resid norm 1.975994847684e-10 true resid norm 1.975994847684e-10 ||r(i)||/||b|| 2.332635474732e-04
212 KSP unpreconditioned resid norm 1.907641063078e-10 true resid norm 1.907641063078e-10 ||r(i)||/||b|| 2.251944746722e-04
213 KSP unpreconditioned resid norm 1.835317861753e-10 true resid norm 1.835317861753e-10 ||r(i)||/||b|| 2.166568175394e-04
214 KSP unpreconditioned resid norm 1.783343277288e-10 true resid norm 1.783343277288e-10 ||r(i)||/||b|| 2.105212873962e-04
215 KSP unpreconditioned resid norm 1.737252962117e-10 true resid norm 1.737252962116e-10 ||r(i)||/||b|| 2.050803873687e-04
216 KSP unpreconditioned resid norm 1.681090858385e-10 true resid norm 1.681090858385e-10 ||r(i)||/||b|| 1.984505261799e-04
217 KSP unpreconditioned resid norm 1.634109844504e-10 true resid norm 1.634109844504e-10 ||r(i)||/||b|| 1.929044803617e-04
218 KSP unpreconditioned resid norm 1.596828500439e-10 true resid norm 1.596828500438e-10 ||r(i)||/||b|| 1.885034675850e-04
219 KSP unpreconditioned resid norm 1.559655758914e-10 true resid norm 1.559655758914e-10 ||r(i)||/||b|| 1.841152751930e-04
220 KSP unpreconditioned resid norm 1.497441081261e-10 true resid norm 1.497441081261e-10 ||r(i)||/||b|| 1.767709157523e-04
221 KSP unpreconditioned resid norm 1.440543864995e-10 true resid norm 1.440543864995e-10 ||r(i)||/||b|| 1.700542755126e-04
222 KSP unpreconditioned resid norm 1.406455332877e-10 true resid norm 1.406455332877e-10 ||r(i)||/||b|| 1.660301699136e-04
223 KSP unpreconditioned resid norm 1.352855224988e-10 true resid norm 1.352855224988e-10 ||r(i)||/||b|| 1.597027489055e-04
224 KSP unpreconditioned resid norm 1.318566764812e-10 true resid norm 1.318566764812e-10 ||r(i)||/||b|| 1.556550420670e-04
225 KSP unpreconditioned resid norm 1.247714207143e-10 true resid norm 1.247714207143e-10 ||r(i)||/||b|| 1.472909924497e-04
226 KSP unpreconditioned resid norm 1.170326734811e-10 true resid norm 1.170326734811e-10 ||r(i)||/||b|| 1.381555049016e-04
227 KSP unpreconditioned resid norm 1.104850176393e-10 true resid norm 1.104850176393e-10 ||r(i)||/||b|| 1.304260848017e-04
228 KSP unpreconditioned resid norm 1.057860311059e-10 true resid norm 1.057860311059e-10 ||r(i)||/||b|| 1.248789940814e-04
229 KSP unpreconditioned resid norm 1.010024307373e-10 true resid norm 1.010024307373e-10 ||r(i)||/||b|| 1.192320178609e-04
230 KSP unpreconditioned resid norm 9.517109911279e-11 true resid norm 9.517109911278e-11 ||r(i)||/||b|| 1.123482089136e-04
231 KSP unpreconditioned resid norm 8.906657903584e-11 true resid norm 8.906657903583e-11 ||r(i)||/||b|| 1.051419046540e-04
232 KSP unpreconditioned resid norm 8.571475780780e-11 true resid norm 8.571475780779e-11 ||r(i)||/||b|| 1.011851245487e-04
233 KSP unpreconditioned resid norm 8.105732120348e-11 true resid norm 8.105732120347e-11 ||r(i)||/||b|| 9.568708296360e-05
234 KSP unpreconditioned resid norm 7.805220078644e-11 true resid norm 7.805220078644e-11 ||r(i)||/||b|| 9.213957852611e-05
235 KSP unpreconditioned resid norm 7.445788494535e-11 true resid norm 7.445788494536e-11 ||r(i)||/||b|| 8.789653677520e-05
236 KSP unpreconditioned resid norm 7.257750273074e-11 true resid norm 7.257750273076e-11 ||r(i)||/||b|| 8.567677073433e-05
237 KSP unpreconditioned resid norm 7.062425062277e-11 true resid norm 7.062425062278e-11 ||r(i)||/||b|| 8.337098275948e-05
238 KSP unpreconditioned resid norm 6.715189342905e-11 true resid norm 6.715189342907e-11 ||r(i)||/||b|| 7.927191155973e-05
239 KSP unpreconditioned resid norm 6.623125011338e-11 true resid norm 6.623125011340e-11 ||r(i)||/||b|| 7.818510444572e-05
240 KSP unpreconditioned resid norm 6.442916815518e-11 true resid norm 6.442916815520e-11 ||r(i)||/||b|| 7.605777080971e-05
241 KSP unpreconditioned resid norm 6.297584870411e-11 true resid norm 6.297584870414e-11 ||r(i)||/||b|| 7.434214664619e-05
242 KSP unpreconditioned resid norm 6.218304192586e-11 true resid norm 6.218304192588e-11 ||r(i)||/||b|| 7.340624885388e-05
243 KSP unpreconditioned resid norm 6.119793248448e-11 true resid norm 6.119793248450e-11 ||r(i)||/||b|| 7.224334034115e-05
244 KSP unpreconditioned resid norm 6.039481879983e-11 true resid norm 6.039481879984e-11 ||r(i)||/||b|| 7.129527538376e-05
245 KSP unpreconditioned resid norm 6.025966944200e-11 true resid norm 6.025966944201e-11 ||r(i)||/||b|| 7.113573337542e-05
246 KSP unpreconditioned resid norm 5.948714963964e-11 true resid norm 5.948714963965e-11 ||r(i)||/||b|| 7.022378408667e-05
247 KSP unpreconditioned resid norm 5.875133821595e-11 true resid norm 5.875133821596e-11 ||r(i)||/||b|| 6.935516854771e-05
248 KSP unpreconditioned resid norm 5.865559932945e-11 true resid norm 5.865559932945e-11 ||r(i)||/||b|| 6.924215007337e-05
249 KSP unpreconditioned resid norm 5.807527161438e-11 true resid norm 5.807527161438e-11 ||r(i)||/||b|| 6.855708097174e-05
250 KSP unpreconditioned resid norm 5.729567969355e-11 true resid norm 5.729567969356e-11 ||r(i)||/||b|| 6.763678314178e-05
251 KSP unpreconditioned resid norm 5.638458710165e-11 true resid norm 5.638458710166e-11 ||r(i)||/||b|| 6.656125052938e-05
252 KSP unpreconditioned resid norm 5.502345910189e-11 true resid norm 5.502345910190e-11 ||r(i)||/||b|| 6.495445714041e-05
253 KSP unpreconditioned resid norm 5.380650850111e-11 true resid norm 5.380650850112e-11 ||r(i)||/||b|| 6.351786324155e-05
254 KSP unpreconditioned resid norm 5.122861296752e-11 true resid norm 5.122861296753e-11 ||r(i)||/||b|| 6.047469206179e-05
255 KSP unpreconditioned resid norm 4.940773406195e-11 true resid norm 4.940773406196e-11 ||r(i)||/||b|| 5.832516888095e-05
256 KSP unpreconditioned resid norm 4.709810110736e-11 true resid norm 4.709810110736e-11 ||r(i)||/||b|| 5.559867808578e-05
257 KSP unpreconditioned resid norm 4.445230653863e-11 true resid norm 4.445230653862e-11 ||r(i)||/||b|| 5.247535300367e-05
258 KSP unpreconditioned resid norm 4.227887565137e-11 true resid norm 4.227887565136e-11 ||r(i)||/||b|| 4.990964692632e-05
259 KSP unpreconditioned resid norm 3.927450241359e-11 true resid norm 3.927450241358e-11 ||r(i)||/||b|| 4.636302452395e-05
260 KSP unpreconditioned resid norm 3.754948266816e-11 true resid norm 3.754948266814e-11 ||r(i)||/||b|| 4.432666179884e-05
261 KSP unpreconditioned resid norm 3.517864153420e-11 true resid norm 3.517864153417e-11 ||r(i)||/||b|| 4.152791556703e-05
262 KSP unpreconditioned resid norm 3.347259122434e-11 true resid norm 3.347259122432e-11 ||r(i)||/||b|| 3.951394600678e-05
263 KSP unpreconditioned resid norm 3.122604584625e-11 true resid norm 3.122604584623e-11 ||r(i)||/||b|| 3.686192925144e-05
264 KSP unpreconditioned resid norm 2.921844421531e-11 true resid norm 2.921844421529e-11 ||r(i)||/||b|| 3.449198239204e-05
265 KSP unpreconditioned resid norm 2.807535371206e-11 true resid norm 2.807535371204e-11 ||r(i)||/||b|| 3.314257934990e-05
266 KSP unpreconditioned resid norm 2.679326847398e-11 true resid norm 2.679326847397e-11 ||r(i)||/||b|| 3.162909488335e-05
267 KSP unpreconditioned resid norm 2.620100211750e-11 true resid norm 2.620100211750e-11 ||r(i)||/||b|| 3.092993237530e-05
268 KSP unpreconditioned resid norm 2.480717416211e-11 true resid norm 2.480717416212e-11 ||r(i)||/||b|| 2.928453712632e-05
269 KSP unpreconditioned resid norm 2.421629839252e-11 true resid norm 2.421629839252e-11 ||r(i)||/||b|| 2.858701618747e-05
270 KSP unpreconditioned resid norm 2.361009921624e-11 true resid norm 2.361009921625e-11 ||r(i)||/||b|| 2.787140617210e-05
271 KSP unpreconditioned resid norm 2.251561534423e-11 true resid norm 2.251561534425e-11 ||r(i)||/||b|| 2.657938260769e-05
272 KSP unpreconditioned resid norm 2.149855566100e-11 true resid norm 2.149855566102e-11 ||r(i)||/||b|| 2.537875726203e-05
273 KSP unpreconditioned resid norm 2.108747915310e-11 true resid norm 2.108747915312e-11 ||r(i)||/||b|| 2.489348694552e-05
274 KSP unpreconditioned resid norm 2.035331893385e-11 true resid norm 2.035331893387e-11 ||r(i)||/||b|| 2.402682063130e-05
275 KSP unpreconditioned resid norm 1.941734891236e-11 true resid norm 1.941734891238e-11 ||r(i)||/||b|| 2.292192054618e-05
276 KSP unpreconditioned resid norm 1.853704982428e-11 true resid norm 1.853704982430e-11 ||r(i)||/||b|| 2.188273925295e-05
277 KSP unpreconditioned resid norm 1.792368946256e-11 true resid norm 1.792368946257e-11 ||r(i)||/||b|| 2.115867555398e-05
278 KSP unpreconditioned resid norm 1.760944257573e-11 true resid norm 1.760944257573e-11 ||r(i)||/||b|| 2.078771130935e-05
279 KSP unpreconditioned resid norm 1.714545149771e-11 true resid norm 1.714545149771e-11 ||r(i)||/||b|| 2.023997605092e-05
280 KSP unpreconditioned resid norm 1.671909267963e-11 true resid norm 1.671909267963e-11 ||r(i)||/||b|| 1.973666517175e-05
281 KSP unpreconditioned resid norm 1.609444716094e-11 true resid norm 1.609444716094e-11 ||r(i)||/||b|| 1.899927949600e-05
282 KSP unpreconditioned resid norm 1.566178459113e-11 true resid norm 1.566178459112e-11 ||r(i)||/||b|| 1.848852712227e-05
283 KSP unpreconditioned resid norm 1.514203649442e-11 true resid norm 1.514203649441e-11 ||r(i)||/||b|| 1.787497144942e-05
284 KSP unpreconditioned resid norm 1.456362863785e-11 true resid norm 1.456362863785e-11 ||r(i)||/||b|| 1.719216871506e-05
285 KSP unpreconditioned resid norm 1.376505865010e-11 true resid norm 1.376505865009e-11 ||r(i)||/||b|| 1.624946753105e-05
286 KSP unpreconditioned resid norm 1.303680225659e-11 true resid norm 1.303680225658e-11 ||r(i)||/||b|| 1.538977060411e-05
287 KSP unpreconditioned resid norm 1.225258977795e-11 true resid norm 1.225258977795e-11 ||r(i)||/||b|| 1.446401826750e-05
288 KSP unpreconditioned resid norm 1.121424657129e-11 true resid norm 1.121424657129e-11 ||r(i)||/||b|| 1.323826800725e-05
289 KSP unpreconditioned resid norm 1.054203483048e-11 true resid norm 1.054203483047e-11 ||r(i)||/||b|| 1.244473104282e-05
290 KSP unpreconditioned resid norm 9.820019392645e-12 true resid norm 9.820019392636e-12 ||r(i)||/||b|| 1.159240148054e-05
291 KSP unpreconditioned resid norm 9.223807475811e-12 true resid norm 9.223807475802e-12 ||r(i)||/||b|| 1.088858129128e-05
292 KSP unpreconditioned resid norm 8.482817720280e-12 true resid norm 8.482817720271e-12 ||r(i)||/||b|| 1.001385280087e-05
293 KSP unpreconditioned resid norm 8.015330479160e-12 true resid norm 8.015330479152e-12 ||r(i)||/||b|| 9.461990368692e-06
KSP Object: 128 MPI processes
  type: cg
  maximum iterations=10000, initial guess is zero
  tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
  left preconditioning
  has attached null space
  using UNPRECONDITIONED norm type for convergence test
PC Object: 128 MPI processes
  type: mg
    MG: type is MULTIPLICATIVE, levels=5 cycles=v
      Cycles per PCApply=1
      Using Galerkin computed coarse grid matrices
  Coarse grid solver -- level -------------------------------
    KSP Object:    (mg_coarse_)     128 MPI processes
      type: preonly
      maximum iterations=1, initial guess is zero
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using NONE norm type for convergence test
    PC Object:    (mg_coarse_)     128 MPI processes
      type: lu
        LU: out-of-place factorization
        tolerance for zero pivot 2.22045e-14
        matrix ordering: natural
        factor fill ratio given 0, needed 0
          Factored matrix follows:
            Matrix Object:             128 MPI processes
              type: mpiaij
              rows=1024, cols=1024
              package used to perform factorization: superlu_dist
              total: nonzeros=0, allocated nonzeros=0
              total number of mallocs used during MatSetValues calls =0
                SuperLU_DIST run parameters:
                  Process grid nprow 16 x npcol 8 
                  Equilibrate matrix TRUE 
                  Matrix input mode 1 
                  Replace tiny pivots TRUE 
                  Use iterative refinement FALSE 
                  Processors in row 16 col partition 8 
                  Row permutation LargeDiag 
                  Column permutation METIS_AT_PLUS_A
                  Parallel symbolic factorization FALSE 
                  Repeated factorization SamePattern_SameRowPerm
      linear system matrix = precond matrix:
      Matrix Object:       128 MPI processes
        type: mpiaij
        rows=1024, cols=1024
        total: nonzeros=27648, allocated nonzeros=27648
        total number of mallocs used during MatSetValues calls =0
          not using I-node (on process 0) routines
  Down solver (pre-smoother) on level 1 -------------------------------
    KSP Object:    (mg_levels_1_)     128 MPI processes
      type: richardson
        Richardson: damping factor=1
      maximum iterations=1
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using nonzero initial guess
      using NONE norm type for convergence test
    PC Object:    (mg_levels_1_)     128 MPI processes
      type: sor
        SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
      linear system matrix = precond matrix:
      Matrix Object:       128 MPI processes
        type: mpiaij
        rows=8192, cols=8192
        total: nonzeros=221184, allocated nonzeros=221184
        total number of mallocs used during MatSetValues calls =0
          using I-node (on process 0) routines: found 16 nodes, limit used is 5
  Up solver (post-smoother) same as down solver (pre-smoother)
  Down solver (pre-smoother) on level 2 -------------------------------
    KSP Object:    (mg_levels_2_)     128 MPI processes
      type: richardson
        Richardson: damping factor=1
      maximum iterations=1
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using nonzero initial guess
      using NONE norm type for convergence test
    PC Object:    (mg_levels_2_)     128 MPI processes
      type: sor
        SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
      linear system matrix = precond matrix:
      Matrix Object:       128 MPI processes
        type: mpiaij
        rows=65536, cols=65536
        total: nonzeros=1769472, allocated nonzeros=1769472
        total number of mallocs used during MatSetValues calls =0
          not using I-node (on process 0) routines
  Up solver (post-smoother) same as down solver (pre-smoother)
  Down solver (pre-smoother) on level 3 -------------------------------
    KSP Object:    (mg_levels_3_)     128 MPI processes
      type: richardson
        Richardson: damping factor=1
      maximum iterations=1
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using nonzero initial guess
      using NONE norm type for convergence test
    PC Object:    (mg_levels_3_)     128 MPI processes
      type: sor
        SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
      linear system matrix = precond matrix:
      Matrix Object:       128 MPI processes
        type: mpiaij
        rows=524288, cols=524288
        total: nonzeros=14155776, allocated nonzeros=14155776
        total number of mallocs used during MatSetValues calls =0
          not using I-node (on process 0) routines
  Up solver (post-smoother) same as down solver (pre-smoother)
  Down solver (pre-smoother) on level 4 -------------------------------
    KSP Object:    (mg_levels_4_)     128 MPI processes
      type: richardson
        Richardson: damping factor=1
      maximum iterations=1
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000
      left preconditioning
      using nonzero initial guess
      using NONE norm type for convergence test
    PC Object:    (mg_levels_4_)     128 MPI processes
      type: sor
        SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
      linear system matrix = precond matrix:
      Matrix Object:       128 MPI processes
        type: mpiaij
        rows=4194304, cols=4194304
        total: nonzeros=29360128, allocated nonzeros=29360128
        total number of mallocs used during MatSetValues calls =0
  Up solver (post-smoother) same as down solver (pre-smoother)
  linear system matrix = precond matrix:
  Matrix Object:   128 MPI processes
    type: mpiaij
    rows=4194304, cols=4194304
    total: nonzeros=29360128, allocated nonzeros=29360128
    total number of mallocs used during MatSetValues calls =0
 
************************************************************************************************************************
***             WIDEN YOUR WINDOW TO 120 CHARACTERS.  Use 'enscript -r -fCourier9' to print this document            ***
************************************************************************************************************************

---------------------------------------------- PETSc Performance Summary: ----------------------------------------------

./hit on a interlagos-64idx-pgi-opt named nid25475 with 128 processors, by Unknown Fri Sep 20 12:56:00 2013
Using Petsc Release Version 3.4.2, Jul, 02, 2013 

                         Max       Max/Min        Avg      Total 
Time (sec):           3.240e+03      1.00000   3.240e+03
Objects:              5.352e+05      1.00000   5.352e+05
Flops:                4.067e+11      1.00000   4.067e+11  5.206e+13
Flops/sec:            1.255e+08      1.00000   1.255e+08  1.606e+10
MPI Messages:         7.811e+06      1.00000   7.811e+06  9.998e+08
MPI Message Lengths:  2.990e+10      1.00000   3.827e+03  3.827e+12
MPI Reductions:       1.522e+06      1.00000

Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
                            e.g., VecAXPY() for real vectors of length N --> 2N flops
                            and VecAXPY() for complex vectors of length N --> 8N flops

Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --
                        Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total 
 0:      Main Stage: 3.2403e+03 100.0%  5.2056e+13 100.0%  9.998e+08 100.0%  3.827e+03      100.0%  1.522e+06 100.0% 

------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
   Count: number of times phase was executed
   Time and Flops: Max - maximum over all processors
                   Ratio - ratio of maximum to minimum over all processors
   Mess: number of messages sent
   Avg. len: average message length (bytes)
   Reduct: number of global reductions
   Global: entire computation
   Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
      %T - percent time in this phase         %f - percent flops in this phase
      %M - percent messages in this phase     %L - percent message lengths in this phase
      %R - percent reductions in this phase
   Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event                Count      Time (sec)     Flops                             --- Global ---  --- Stage ---   Total
                   Max Ratio  Max     Ratio   Max  Ratio  Mess   Avg len Reduct  %T %f %M %L %R  %T %f %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------

--- Event Stage 0: Main Stage

VecTDot           177386 1.0 5.7980e+01 2.1 1.16e+10 1.0 0.0e+00 0.0e+00 1.8e+05  1  3  0  0 12   1  3  0  0 12 25664
VecNorm           179519 1.0 4.6141e+01 3.5 1.18e+10 1.0 0.0e+00 0.0e+00 1.8e+05  1  3  0  0 12   1  3  0  0 12 32637
VecScale          354772 1.0 2.3496e+00 1.5 1.00e+09 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0 54733
VecCopy           179519 1.0 8.4071e+00 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
VecSet           1243857 1.0 1.8819e+01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  1  0  0  0  0   1  0  0  0  0     0
VecAXPY           177386 1.0 2.6131e+01 1.2 1.16e+10 1.0 0.0e+00 0.0e+00 0.0e+00  1  3  0  0  0   1  3  0  0  0 56944
VecAYPX           532158 1.0 4.1760e+01 1.3 1.20e+10 1.0 0.0e+00 0.0e+00 0.0e+00  1  3  0  0  0   1  3  0  0  0 36832
VecScatterBegin  1598611 1.0 9.1558e+01 1.3 0.00e+00 0.0 1.0e+09 3.8e+03 0.0e+00  3  0100100  0   3  0100100  0     0
VecScatterEnd    1598611 1.0 2.5825e+02 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  6  0  0  0  0   6  0  0  0  0     0
MatMult           532869 1.0 1.0230e+03 1.1 1.36e+11 1.0 4.1e+08 6.2e+03 0.0e+00 29 33 41 66  0  29 33 41 66  0 16969
MatMultAdd        354772 1.0 1.6270e+02 1.1 2.24e+10 1.0 1.4e+08 9.9e+02 0.0e+00  5  6 14  4  0   5  6 14  4  0 17634
MatMultTranspose  354776 1.0 1.8670e+02 1.3 2.24e+10 1.0 1.4e+08 9.9e+02 0.0e+00  5  6 14  4  0   5  6 14  4  0 15367
MatSolve           88693 1.0 1.6765e+02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  4  0  0  0  0   4  0  0  0  0     0
MatSOR            709544 1.0 1.3796e+03 1.1 1.79e+11 1.0 3.2e+08 3.2e+03 7.1e+05 41 44 32 27 47  41 44 32 27 47 16627
MatLUFactorSym         1 1.0 6.4993e-04 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatLUFactorNum         1 1.0 1.2938e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatAssemblyBegin     728 1.0 2.8366e+0018.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.4e+03  0  0  0  0  0   0  0  0  0  0     0
MatAssemblyEnd       728 1.0 2.5708e+00 1.2 0.00e+00 0.0 1.2e+04 1.1e+03 7.2e+01  0  0  0  0  0   0  0  0  0  0     0
MatGetRowIJ            1 1.0 5.0068e-06 5.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatGetOrdering         1 1.0 2.0981e-05 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatView             4977 1.0 1.3075e+00 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+03  0  0  0  0  0   0  0  0  0  0     0
MatPtAP                4 1.0 2.1368e-01 1.0 5.11e+06 1.0 2.5e+04 6.0e+03 1.0e+02  0  0  0  0  0   0  0  0  0  0  3058
MatPtAPSymbolic        4 1.0 1.5158e-01 1.1 0.00e+00 0.0 1.5e+04 7.8e+03 6.0e+01  0  0  0  0  0   0  0  0  0  0     0
MatPtAPNumeric         4 1.0 7.0144e-02 1.1 5.11e+06 1.0 9.7e+03 3.1e+03 4.0e+01  0  0  0  0  0   0  0  0  0  0  9316
MatGetLocalMat         4 1.0 2.1817e-02 3.3 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatGetBrAoCol          4 1.0 2.9885e-02 3.2 0.00e+00 0.0 1.1e+04 8.4e+03 8.0e+00  0  0  0  0  0   0  0  0  0  0     0
MatGetSymTrans         8 1.0 9.0678e-03 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
KSPSetUp               6 1.0 1.0205e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 2.2e+01  0  0  0  0  0   0  0  0  0  0     0
KSPSolve             711 1.0 3.0209e+03 1.0 4.07e+11 1.0 1.0e+09 3.8e+03 1.5e+06 93100100100 99  93100100100 99 17232
PCSetUp                1 1.0 4.2324e-01 1.0 5.36e+06 1.0 3.4e+04 4.6e+03 3.0e+02  0  0  0  0  0   0  0  0  0  0  1620
PCApply            88693 1.0 2.2119e+03 1.0 2.87e+11 1.0 9.1e+08 2.6e+03 7.1e+05 67 71 91 61 47  67 71 91 61 47 16614
------------------------------------------------------------------------------------------------------------------------

Memory usage is given in bytes:

Object Type          Creations   Destructions     Memory  Descendants' Mem.
Reports information only for process 0.

--- Event Stage 0: Main Stage

              Vector 535068          535068  74824032416     0
      Vector Scatter    19             19        22572     0
              Matrix    38             38     14004608     0
   Matrix Null Space     1              1          652     0
    Distributed Mesh     5              5       830792     0
     Bipartite Graph    10             10         8560     0
           Index Set    47             47       534480     0
   IS L to G Mapping     5              5       405756     0
       Krylov Solver     7              7         9536     0
     DMKSP interface     3              3         2088     0
      Preconditioner     7              7         7352     0
              Viewer     1              0            0     0
========================================================================================================================
Average time to get PetscTime(): 4.05312e-07
Average time for MPI_Barrier(): 1.32084e-05
Average time for zero size MPI_Send(): 2.75671e-06
#PETSc Option Table entries:
-ksp_monitor_true_residual
-ksp_view
-log_summary
-mg_coarse_pc_factor_mat_solver_package superlu_dist
-mg_coarse_pc_type lu
-mg_levels_ksp_max_it 1
-mg_levels_ksp_type richardson
-options_left
-pc_mg_galerkin
-pc_mg_levels 5
-pc_type mg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8
Configure run at: Wed Aug 28 23:25:43 2013
Configure options: --known-level1-dcache-size=16384 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=4 --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=0 --known-mpi-c-double-complex=0 --with-batch="1 " --known-mpi-shared="0 " --known-memcmp-ok  --with-blas-lapack-lib="-L/opt/acml/5.3.0/pgi64/lib  -lacml" --COPTFLAGS="-O3 -fastsse" --FOPTFLAGS="-O3 -fastsse" --CXXOPTFLAGS="-O3 -fastsse" --with-x="0 " --with-debugging="0 " --with-clib-autodetect="0 " --with-cxxlib-autodetect="0 " --with-fortranlib-autodetect="0 " --with-shared-libraries=0 --with-dynamic-loading=0 --with-mpi-compilers="1 " --known-mpi-shared-libraries=0 --with-64-bit-indices --download-blacs="1 " --download-scalapack="1 " --download-superlu_dist="1 " --download-metis="1 " --download-parmetis="1 " --with-cc=cc --with-cxx=CC --with-fc=ftn PETSC_ARCH=interlagos-64idx-pgi-opt
-----------------------------------------
Libraries compiled on Wed Aug 28 23:25:43 2013 on h2ologin3 
Machine characteristics: Linux-2.6.32.59-0.7-default-x86_64-with-SuSE-11-x86_64
Using PETSc directory: /mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2
Using PETSc arch: interlagos-64idx-pgi-opt
-----------------------------------------

Using C compiler: cc  -O3 -fastsse  ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: ftn  -O3 -fastsse   ${FOPTFLAGS} ${FFLAGS} 
-----------------------------------------

Using include paths: -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/include -I/opt/cray/udreg/2.3.2-1.0402.7311.2.1.gem/include -I/opt/cray/ugni/5.0-1.0402.7128.7.6.gem/include -I/opt/cray/pmi/4.0.1-1.0000.9421.73.3.gem/include -I/opt/cray/dmapp/4.0.1-1.0402.7439.5.1.gem/include -I/opt/cray/gni-headers/2.1-1.0402.7082.6.2.gem/include -I/opt/cray/xpmem/0.1-2.0402.44035.2.1.gem/include -I/opt/cray/rca/1.0.0-2.0402.42153.2.106.gem/include -I/opt/cray-hss-devel/7.0.0/include -I/opt/cray/krca/1.0.0-2.0402.42157.2.94.gem/include -I/opt/cray/mpt/6.0.1/gni/mpich2-pgi/121/include -I/opt/acml/5.3.0/pgi64_fma4/include -I/opt/cray/libsci/12.1.01/pgi/121/interlagos/include -I/opt/fftw/3.3.0.3/interlagos/include -I/usr/include/alps -I/opt/pgi/13.6.0/linux86-64/13.6/include -I/opt/cray/xe-sysroot/4.2.24/usr/include
-----------------------------------------

Using C linker: cc
Using Fortran linker: ftn
Using libraries: -Wl,-rpath,/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -L/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -lpetsc -Wl,-rpath,/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -L/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -lsuperlu_dist_3.3 -L/opt/acml/5.3.0/pgi64/lib -lacml -lpthread -lparmetis -lmetis -ldl 
-----------------------------------------

#PETSc Option Table entries:
-ksp_monitor_true_residual
-ksp_view
-log_summary
-mg_coarse_pc_factor_mat_solver_package superlu_dist
-mg_coarse_pc_type lu
-mg_levels_ksp_max_it 1
-mg_levels_ksp_type richardson
-options_left
-pc_mg_galerkin
-pc_mg_levels 5
-pc_type mg
#End of PETSc Option Table entries
There are no unused options.


More information about the petsc-users mailing list