[petsc-users] GAMG speed
Michele Rosso
mrosso at uci.edu
Fri Sep 20 17:55:30 CDT 2013
Hi Jed,
I set-up and tested the solve for the multi-phase test case (liquid
droplet with surface tension falling under the effect of gravity in a
quiescent fluid). The solution of the Poisson Equation via multigrid is
correct but it becomes progressively slower and slower as the simulation
progresses (I am performing successive solves). Could you please help me
out in speeding this up?
I run the test case with the options:
-pc_type mg -pc_mg_galerkin -pc_mg_levels 5 -mg_levels_ksp_type
richardson -mg_levels_ksp_max_it 1
-mg_coarse_pc_type lu -mg_coarse_pc_factor_mat_solver_package superlu_dist
-log_summary -ksp_view -ksp_monitor_true_residual -options_left
Please find the diagnostic for the final solve in the attached file
"final.txt'.
Thank you,
Michele
On 08/14/2013 04:35 AM, Jed Brown wrote:
> Please always use "reply-all" so that your messages go to the list.
> This is standard mailing list etiquette. It is important to preserve
> threading for people who find this discussion later and so that we do
> not waste our time re-answering the same questions that have already
> been answered in private side-conversations. You'll likely get an
> answer faster that way too.
>
> Michele Rosso <mrosso at uci.edu> writes:
>
>> Jed,
>>
>> thank you very much for the detailed analysis.
>> I confirm that
>>
>> -log_summary -ksp_monitor -ksp_view -ksp_converged_reason -pc_type mg
>> -pc_mg_galerkin -pc_mg_levels 5 -mg_levels_ksp_type richardson
>> -mg_levels_ksp_max_it 1
>>
>> results in a faster solve with the 256^3 grid (it finally beats CG + ICC).
>>
>> Yes, I perform different solves: I setup matrix and KSP only once at the beginning of the run and then I re-use them
>> at each time step. The rhs term changes during the simulation though. For now (single phase flow) the matrix does not change,
>> but I will be dealing soon with a multiphase flow and thus not even the matrix values will be constant in time (it will be a variable coefficients Poisson Equation).
> Okay, the coefficient variation from the multiphase flow can drastically
> change the characteristics of the solve.
>
>> I need to solve up to the discretization error, so maybe FMG is worth a try.
>> The matrix-free approach is appealing given that the Poisson solver is really mission critical (basically it accounts for most of the simulation time).
>> I will use the level set method and ghost fluid method in order to account for the discontinuities at the interface between phases: the computation of the matrix and rhs
>> values will be influenced by such methods so my only concern is to be sure matrix-free can be used in these circumstances.
> Matrix-free can be used in principle, but those problems can be several
> orders of magnitude more ill-conditioned, so don't invest any more time
> on it right now. Get the discretization set up using assembled
> matrices, then go through the options we've tried to find an efficient
> solver. The best choice will likely depend on the details of the
> formulation, the types of fluids involved, and the geometric
> configuration of the fluids.
>
>> I do not have any prior experience with matrix-free methods so I will have to rely on your assistance for this.
>> Thank you very much.
>>
>> Michele
>>
>>
>>
>>
>> On 08/13/2013 08:36 PM, Jed Brown wrote:
>>> Michele Rosso <mrosso at uci.edu> writes:
>>>
>>>> Hi Jed,
>>>>
>>>> I attached the output for both the runs you suggested. At the beginning
>>>> of each file I included the options I used.
>>>>
>>>> On a side note, I tried to run with a grid of 256^3 (exactly as before)
>>>> but with less levels, i.e. 3 instead of 4 or 5.
>>>> My system stops the run because of an Out Of Memory condition. It is
>>>> really odd since I have not changed anything except
>>>> - pc_mg_levels. I cannot send you any output since there is none. Do
>>>> you have any guess where the problem comes from?
>>> The selected algorithm does a direct solve on the coarse grid. Each
>>> time you reduce the number of levels, the coarse grid size grows by a
>>> factor of 8. Going from 5 to 3 levels is going from a 16^3 coarse grid
>>> to a 64^3 coarse grid. Applying a direct solver to the latter ends up
>>> using a lot of memory. I think this is not worth bothering with and it
>>> might even be (slightly) faster to use 6 levels. That is not where the
>>> time is being spent.
>>>
>>>> -log_summary -ksp_monitor -ksp_view -ksp_converged_reason -pc_type mg
>>>> -pc_mg_galerkin -pc_mg_levels 5 -mg_levels_ksp_type richardson
>>>> -mg_levels_ksp_max_it 1
>>>>
>>>>
>>>>
>>>> 0 KSP Residual norm 3.653965664551e-05
>>>> 1 KSP Residual norm 1.910638846094e-06
>>>> 2 KSP Residual norm 8.690440116045e-08
>>>> 3 KSP Residual norm 3.732213639394e-09
>>>> 4 KSP Residual norm 1.964855338020e-10
>>> This converges well.
>>>
>>>> Max Max/Min Avg Total
>>>> Time (sec): 4.048e+00 1.00012 4.048e+00
>>>> Objects: 2.490e+02 1.00000 2.490e+02
>>>> Flops: 2.663e+08 1.00000 2.663e+08 2.130e+09
>>>> Flops/sec: 6.579e+07 1.00012 6.579e+07 5.263e+08
>>>> MPI Messages: 6.820e+02 1.00000 6.820e+02 5.456e+03
>>>> MPI Message Lengths: 8.245e+06 1.00000 1.209e+04 6.596e+07
>>>> MPI Reductions: 4.580e+02 1.00000
>>>> VecTDot 12 1.0 2.9428e-02 1.2 6.29e+06 1.0 0.0e+00 0.0e+00 1.2e+01 1 2 0 0 3 1 2 0 0 3 1710
>>>> VecNorm 9 1.0 1.0796e-02 1.2 4.72e+06 1.0 0.0e+00 0.0e+00 9.0e+00 0 2 0 0 2 0 2 0 0 2 3497
>>>> VecScale 24 1.0 2.4652e-04 1.1 1.99e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6442
>>>> VecCopy 3 1.0 5.0740e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> VecSet 116 1.0 1.4349e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> VecAXPY 12 1.0 2.8027e-02 1.0 6.29e+06 1.0 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 1796
>>>> VecAYPX 29 1.0 3.0655e-02 1.4 4.16e+06 1.0 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 1085
>>>> VecScatterBegin 123 1.0 3.5391e-02 1.1 0.00e+00 0.0 3.5e+03 1.2e+04 0.0e+00 1 0 65 66 0 1 0 65 66 0 0
>>>> VecScatterEnd 123 1.0 2.5395e-02 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> MatMult 31 1.0 2.3556e-01 1.0 5.62e+07 1.0 1.0e+03 2.3e+04 0.0e+00 6 21 19 36 0 6 21 19 36 0 1908
>>>> MatMultAdd 24 1.0 5.9044e-02 1.0 1.21e+07 1.0 5.8e+02 2.8e+03 0.0e+00 1 5 11 2 0 1 5 11 2 0 1644
>>>> MatMultTranspose 28 1.0 7.4601e-02 1.1 1.42e+07 1.0 6.7e+02 2.8e+03 0.0e+00 2 5 12 3 0 2 5 12 3 0 1518
>>>> MatSolve 6 1.0 3.8311e-03 1.0 1.44e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 3006
>>>> MatSOR 48 1.0 5.8050e-01 1.0 1.01e+08 1.0 8.6e+02 1.5e+04 4.8e+01 14 38 16 19 10 14 38 16 19 11 1390
>>> Most of the solve time is in MatSOR and MatMult. That's expected since
>>> the subdomains are pretty big.
>>>
>>>> MatLUFactorSym 1 1.0 3.0620e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+00 0 0 0 0 1 0 0 0 0 1 0
>>>> MatLUFactorNum 1 1.0 2.4665e-02 1.0 1.95e+07 1.0 0.0e+00 0.0e+00 0.0e+00 1 7 0 0 0 1 7 0 0 0 6329
>>>> MatAssemblyBegin 20 1.0 2.4351e-02 6.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.2e+01 0 0 0 0 5 0 0 0 0 5 0
>>>> MatAssemblyEnd 20 1.0 1.3176e-01 1.0 0.00e+00 0.0 5.6e+02 2.1e+03 7.2e+01 3 0 10 2 16 3 0 10 2 16 0
>>>> MatGetRowIJ 1 1.0 1.1516e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> MatGetOrdering 1 1.0 4.1008e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> MatView 16 1.3 1.0209e-03 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 3 0 0 0 0 3 0
>>>> MatPtAP 4 1.0 6.4001e-01 1.0 4.06e+07 1.0 1.1e+03 1.7e+04 1.0e+02 16 15 21 30 22 16 15 21 30 22 507
>>> MatPtAP dominates the setup time. For profiling, you could register a
>>> stage (PetscLogStageRegister) and time the setup separately from the
>>> solve.
>>>
>>>> MatPtAPSymbolic 4 1.0 3.7003e-01 1.0 0.00e+00 0.0 7.2e+02 2.0e+04 6.0e+01 9 0 13 22 13 9 0 13 22 13 0
>>>> MatPtAPNumeric 4 1.0 2.7004e-01 1.0 4.06e+07 1.0 4.2e+02 1.2e+04 4.0e+01 7 15 8 8 9 7 15 8 8 9 1202
>>>> MatGetRedundant 1 1.0 7.9393e-04 1.0 0.00e+00 0.0 1.7e+02 7.1e+03 4.0e+00 0 0 3 2 1 0 0 3 2 1 0
>>>> MatGetLocalMat 4 1.0 3.9521e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 1 0 0 0 2 1 0 0 0 2 0
>>>> MatGetBrAoCol 4 1.0 1.7719e-02 1.0 0.00e+00 0.0 4.3e+02 2.7e+04 8.0e+00 0 0 8 18 2 0 0 8 18 2 0
>>>> MatGetSymTrans 8 1.0 1.3007e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
>>>> KSPSetUp 7 1.0 1.3097e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 2.4e+01 0 0 0 0 5 0 0 0 0 5 0
>>>> KSPSolve 2 1.0 1.0450e+00 1.0 2.04e+08 1.0 3.4e+03 1.2e+04 7.5e+01 26 77 62 60 16 26 77 62 60 16 1563
>>>> PCSetUp 1 1.0 8.6248e-01 1.0 6.21e+07 1.0 1.9e+03 1.1e+04 3.2e+02 21 23 35 32 69 21 23 35 32 69 576
>>>> PCApply 6 1.0 8.4384e-01 1.0 1.61e+08 1.0 3.2e+03 9.0e+03 4.8e+01 21 60 59 44 10 21 60 59 44 11 1523
>>> Do you know why there are 6 PCApply events? With four iterations of the
>>> Krylov method, there should be only 5 events. Oh, it looks like you do
>>> two solves. Is one of those with a different system?
>>>
>>> Recall that the old KSPSolve time was over 3.35 seconds.
>>>
>>>> -log_summary -ksp_monitor -ksp_view -ksp_converged_reason -pc_type mg
>>>> -pc_mg_galerkin -pc_mg_levels 5 -mg_levels_ksp_type richardson
>>>> -mg_levels_ksp_max_it 1 -pc_mg_type full
>>>>
>>>> 0 KSP Residual norm 3.654533581988e-05
>>>> 1 KSP Residual norm 8.730776244351e-07
>>>> 2 KSP Residual norm 3.474626061661e-08
>>>> 3 KSP Residual norm 1.813665557493e-09
>>> This converges slightly faster, but ends up not paying off.
>>>
>>>> Time (sec): 4.261e+00 1.00012 4.261e+00
>>>> Objects: 2.950e+02 1.00000 2.950e+02
>>>> Flops: 3.322e+08 1.00000 3.322e+08 2.658e+09
>>>> Flops/sec: 7.797e+07 1.00012 7.796e+07 6.237e+08
>>>> MPI Messages: 1.442e+03 1.00000 1.442e+03 1.154e+04
>>>> MPI Message Lengths: 1.018e+07 1.00000 7.057e+03 8.141e+07
>>>> MPI Reductions: 5.460e+02 1.00000
>>> More messages, more work, etc., so not better.
>>>
>>>> KSPSolve 2 1.0 1.2287e+00 1.0 2.70e+08 1.0 9.5e+03 5.8e+03 1.6e+02 29 81 82 68 30 29 81 82 68 30 1758
>>>> PCSetUp 1 1.0 8.6414e-01 1.0 6.21e+07 1.0 1.9e+03 1.1e+04 3.2e+02 20 19 17 26 58 20 19 17 26 58 575
>>>> PCApply 5 1.0 1.0571e+00 1.0 2.33e+08 1.0 9.3e+03 4.9e+03 1.4e+02 24 70 81 56 26 24 70 81 56 26 1764
>>> It's still entirely possible that you can make Full MG beat V-cycles,
>>> especially if you only need to converge up to discretization error. By
>>> my figures, your good solver takes 12 work units to converge well below
>>> discretization error (after Galerkin setup, but maybe you only need to
>>> do that once?). If you only need to equal truncation error, this can be
>>> brought down to about 5 (probably at best a 2x speedup in parallel).
>>> This would involve a high-order (cubic) FMG prolongation.
>>>
>>> Alternatively, you can speed up the implementation (and significantly
>>> reduce memory usage) by creating geometric coarse levels and a
>>> matrix-free implementation of MatSOR and MatMult. (The matrices are
>>> great for experimenting, but if this solver is mission critical and
>>> still a bottleneck, the matrix is an inefficient way to represent the
>>> operator since it has very low arithmetic intensity/requires a lot of
>>> memory bandwidth.) I predict you can probably speed up the solve by
>>> perhaps another factor of 2 with a good matrix-free FMG implementation.
>>> Do you want to go down this path?
-------------- next part --------------
0 KSP unpreconditioned resid norm 8.471082897815e-07 true resid norm 8.471082897815e-07 ||r(i)||/||b|| 1.000000000000e+00
1 KSP unpreconditioned resid norm 5.889629222055e-07 true resid norm 5.889629222055e-07 ||r(i)||/||b|| 6.952628481034e-01
2 KSP unpreconditioned resid norm 4.929221177373e-07 true resid norm 4.929221177373e-07 ||r(i)||/||b|| 5.818879636563e-01
3 KSP unpreconditioned resid norm 4.077087883813e-07 true resid norm 4.077087883813e-07 ||r(i)||/||b|| 4.812947686847e-01
4 KSP unpreconditioned resid norm 3.532777909896e-07 true resid norm 3.532777909896e-07 ||r(i)||/||b|| 4.170397046649e-01
5 KSP unpreconditioned resid norm 3.133843759979e-07 true resid norm 3.133843759979e-07 ||r(i)||/||b|| 3.699460621247e-01
6 KSP unpreconditioned resid norm 2.664884465415e-07 true resid norm 2.664884465415e-07 ||r(i)||/||b|| 3.145860449675e-01
7 KSP unpreconditioned resid norm 2.266816102745e-07 true resid norm 2.266816102745e-07 ||r(i)||/||b|| 2.675946074532e-01
8 KSP unpreconditioned resid norm 1.909025017919e-07 true resid norm 1.909025017919e-07 ||r(i)||/||b|| 2.253578486892e-01
9 KSP unpreconditioned resid norm 1.635648456077e-07 true resid norm 1.635648456077e-07 ||r(i)||/||b|| 1.930861113989e-01
10 KSP unpreconditioned resid norm 1.434980659348e-07 true resid norm 1.434980659348e-07 ||r(i)||/||b|| 1.693975465307e-01
11 KSP unpreconditioned resid norm 1.271661365357e-07 true resid norm 1.271661365357e-07 ||r(i)||/||b|| 1.501179224306e-01
12 KSP unpreconditioned resid norm 1.132794401526e-07 true resid norm 1.132794401526e-07 ||r(i)||/||b|| 1.337248631835e-01
13 KSP unpreconditioned resid norm 1.002788428906e-07 true resid norm 1.002788428906e-07 ||r(i)||/||b|| 1.183778320909e-01
14 KSP unpreconditioned resid norm 8.753317847137e-08 true resid norm 8.753317847137e-08 ||r(i)||/||b|| 1.033317458078e-01
15 KSP unpreconditioned resid norm 7.770742874496e-08 true resid norm 7.770742874496e-08 ||r(i)||/||b|| 9.173257974492e-02
16 KSP unpreconditioned resid norm 6.961935247483e-08 true resid norm 6.961935247483e-08 ||r(i)||/||b|| 8.218471394346e-02
17 KSP unpreconditioned resid norm 6.471790768651e-08 true resid norm 6.471790768651e-08 ||r(i)||/||b|| 7.639862396248e-02
18 KSP unpreconditioned resid norm 6.162477500458e-08 true resid norm 6.162477500458e-08 ||r(i)||/||b|| 7.274722222407e-02
19 KSP unpreconditioned resid norm 6.116147101803e-08 true resid norm 6.116147101803e-08 ||r(i)||/||b|| 7.220029806792e-02
20 KSP unpreconditioned resid norm 6.307214323139e-08 true resid norm 6.307214323139e-08 ||r(i)||/||b|| 7.445582104698e-02
21 KSP unpreconditioned resid norm 6.569260578724e-08 true resid norm 6.569260578724e-08 ||r(i)||/||b|| 7.754924202688e-02
22 KSP unpreconditioned resid norm 7.002116374918e-08 true resid norm 7.002116374918e-08 ||r(i)||/||b|| 8.265904677576e-02
23 KSP unpreconditioned resid norm 7.648471462197e-08 true resid norm 7.648471462197e-08 ||r(i)||/||b|| 9.028918208521e-02
24 KSP unpreconditioned resid norm 8.218354530044e-08 true resid norm 8.218354530044e-08 ||r(i)||/||b|| 9.701657543883e-02
25 KSP unpreconditioned resid norm 8.777006719029e-08 true resid norm 8.777006719029e-08 ||r(i)||/||b|| 1.036113897704e-01
26 KSP unpreconditioned resid norm 9.074938460336e-08 true resid norm 9.074938460336e-08 ||r(i)||/||b|| 1.071284341070e-01
27 KSP unpreconditioned resid norm 9.311175500020e-08 true resid norm 9.311175500020e-08 ||r(i)||/||b|| 1.099171807470e-01
28 KSP unpreconditioned resid norm 9.391328578136e-08 true resid norm 9.391328578136e-08 ||r(i)||/||b|| 1.108633771080e-01
29 KSP unpreconditioned resid norm 9.397746582366e-08 true resid norm 9.397746582366e-08 ||r(i)||/||b|| 1.109391407891e-01
30 KSP unpreconditioned resid norm 9.269761294239e-08 true resid norm 9.269761294239e-08 ||r(i)||/||b|| 1.094282915899e-01
31 KSP unpreconditioned resid norm 9.037592803258e-08 true resid norm 9.037592803258e-08 ||r(i)||/||b|| 1.066875736228e-01
32 KSP unpreconditioned resid norm 8.739586906294e-08 true resid norm 8.739586906294e-08 ||r(i)||/||b|| 1.031696538886e-01
33 KSP unpreconditioned resid norm 8.294737580326e-08 true resid norm 8.294737580326e-08 ||r(i)||/||b|| 9.791826712575e-02
34 KSP unpreconditioned resid norm 7.825811344542e-08 true resid norm 7.825811344542e-08 ||r(i)||/||b|| 9.238265566450e-02
35 KSP unpreconditioned resid norm 7.370584204445e-08 true resid norm 7.370584204445e-08 ||r(i)||/||b|| 8.700876019459e-02
36 KSP unpreconditioned resid norm 6.944350218542e-08 true resid norm 6.944350218542e-08 ||r(i)||/||b|| 8.197712503006e-02
37 KSP unpreconditioned resid norm 6.611084651652e-08 true resid norm 6.611084651652e-08 ||r(i)||/||b|| 7.804296961086e-02
38 KSP unpreconditioned resid norm 6.155389358744e-08 true resid norm 6.155389358744e-08 ||r(i)||/||b|| 7.266354765968e-02
39 KSP unpreconditioned resid norm 5.877895869357e-08 true resid norm 5.877895869357e-08 ||r(i)||/||b|| 6.938777415191e-02
40 KSP unpreconditioned resid norm 5.544264371508e-08 true resid norm 5.544264371508e-08 ||r(i)||/||b|| 6.544929896671e-02
41 KSP unpreconditioned resid norm 5.273241362566e-08 true resid norm 5.273241362566e-08 ||r(i)||/||b|| 6.224990861471e-02
42 KSP unpreconditioned resid norm 5.119250885272e-08 true resid norm 5.119250885272e-08 ||r(i)||/||b|| 6.043207163741e-02
43 KSP unpreconditioned resid norm 4.982771093310e-08 true resid norm 4.982771093310e-08 ||r(i)||/||b|| 5.882094595716e-02
44 KSP unpreconditioned resid norm 4.811237834099e-08 true resid norm 4.811237834099e-08 ||r(i)||/||b|| 5.679601878692e-02
45 KSP unpreconditioned resid norm 4.628304375229e-08 true resid norm 4.628304375229e-08 ||r(i)||/||b|| 5.463651378530e-02
46 KSP unpreconditioned resid norm 4.476871541552e-08 true resid norm 4.476871541552e-08 ||r(i)||/||b|| 5.284886944864e-02
47 KSP unpreconditioned resid norm 4.419875774021e-08 true resid norm 4.419875774021e-08 ||r(i)||/||b|| 5.217604204016e-02
48 KSP unpreconditioned resid norm 4.359181679552e-08 true resid norm 4.359181679552e-08 ||r(i)||/||b|| 5.145955637710e-02
49 KSP unpreconditioned resid norm 4.304314318039e-08 true resid norm 4.304314318039e-08 ||r(i)||/||b|| 5.081185451685e-02
50 KSP unpreconditioned resid norm 4.347264822128e-08 true resid norm 4.347264822128e-08 ||r(i)||/||b|| 5.131887946993e-02
51 KSP unpreconditioned resid norm 4.378084979329e-08 true resid norm 4.378084979329e-08 ||r(i)||/||b|| 5.168270730132e-02
52 KSP unpreconditioned resid norm 4.459872124486e-08 true resid norm 4.459872124486e-08 ||r(i)||/||b|| 5.264819360505e-02
53 KSP unpreconditioned resid norm 4.586835225867e-08 true resid norm 4.586835225867e-08 ||r(i)||/||b|| 5.414697602652e-02
54 KSP unpreconditioned resid norm 4.759826693288e-08 true resid norm 4.759826693288e-08 ||r(i)||/||b|| 5.618911714955e-02
55 KSP unpreconditioned resid norm 4.951983455569e-08 true resid norm 4.951983455569e-08 ||r(i)||/||b|| 5.845750201366e-02
56 KSP unpreconditioned resid norm 5.077348189601e-08 true resid norm 5.077348189601e-08 ||r(i)||/||b|| 5.993741592247e-02
57 KSP unpreconditioned resid norm 5.187532881197e-08 true resid norm 5.187532881197e-08 ||r(i)||/||b|| 6.123813146174e-02
58 KSP unpreconditioned resid norm 5.194414285622e-08 true resid norm 5.194414285622e-08 ||r(i)||/||b|| 6.131936552011e-02
59 KSP unpreconditioned resid norm 5.114127723640e-08 true resid norm 5.114127723640e-08 ||r(i)||/||b|| 6.037159339994e-02
60 KSP unpreconditioned resid norm 4.987798575055e-08 true resid norm 4.987798575055e-08 ||r(i)||/||b|| 5.888029470638e-02
61 KSP unpreconditioned resid norm 4.837751000107e-08 true resid norm 4.837751000107e-08 ||r(i)||/||b|| 5.710900316363e-02
62 KSP unpreconditioned resid norm 4.735998413707e-08 true resid norm 4.735998413707e-08 ||r(i)||/||b|| 5.590782749782e-02
63 KSP unpreconditioned resid norm 4.609537746550e-08 true resid norm 4.609537746550e-08 ||r(i)||/||b|| 5.441497624511e-02
64 KSP unpreconditioned resid norm 4.470951308242e-08 true resid norm 4.470951308242e-08 ||r(i)||/||b|| 5.277898188667e-02
65 KSP unpreconditioned resid norm 4.325085133153e-08 true resid norm 4.325085133153e-08 ||r(i)||/||b|| 5.105705120970e-02
66 KSP unpreconditioned resid norm 4.181200034623e-08 true resid norm 4.181200034623e-08 ||r(i)||/||b|| 4.935850687640e-02
67 KSP unpreconditioned resid norm 4.054904820539e-08 true resid norm 4.054904820539e-08 ||r(i)||/||b|| 4.786760877508e-02
68 KSP unpreconditioned resid norm 3.941638106292e-08 true resid norm 3.941638106292e-08 ||r(i)||/||b|| 4.653051037086e-02
69 KSP unpreconditioned resid norm 3.818351979063e-08 true resid norm 3.818351979063e-08 ||r(i)||/||b|| 4.507513413720e-02
70 KSP unpreconditioned resid norm 3.732269485449e-08 true resid norm 3.732269485449e-08 ||r(i)||/||b|| 4.405894181973e-02
71 KSP unpreconditioned resid norm 3.638709503712e-08 true resid norm 3.638709503712e-08 ||r(i)||/||b|| 4.295447875562e-02
72 KSP unpreconditioned resid norm 3.507340393789e-08 true resid norm 3.507340393789e-08 ||r(i)||/||b|| 4.140368399292e-02
73 KSP unpreconditioned resid norm 3.378527900460e-08 true resid norm 3.378527900460e-08 ||r(i)||/||b|| 3.988306974699e-02
74 KSP unpreconditioned resid norm 3.213894671931e-08 true resid norm 3.213894671931e-08 ||r(i)||/||b|| 3.793959651558e-02
75 KSP unpreconditioned resid norm 3.071349886547e-08 true resid norm 3.071349886547e-08 ||r(i)||/||b|| 3.625687439960e-02
76 KSP unpreconditioned resid norm 2.982607199289e-08 true resid norm 2.982607199289e-08 ||r(i)||/||b|| 3.520927885216e-02
77 KSP unpreconditioned resid norm 2.896588396864e-08 true resid norm 2.896588396864e-08 ||r(i)||/||b|| 3.419383840065e-02
78 KSP unpreconditioned resid norm 2.849350659098e-08 true resid norm 2.849350659098e-08 ||r(i)||/||b|| 3.363620322772e-02
79 KSP unpreconditioned resid norm 2.839644438060e-08 true resid norm 2.839644438060e-08 ||r(i)||/||b|| 3.352162258726e-02
80 KSP unpreconditioned resid norm 2.878125026793e-08 true resid norm 2.878125026793e-08 ||r(i)||/||b|| 3.397588078775e-02
81 KSP unpreconditioned resid norm 2.927107179400e-08 true resid norm 2.927107179400e-08 ||r(i)||/||b|| 3.455410854444e-02
82 KSP unpreconditioned resid norm 2.995930520842e-08 true resid norm 2.995930520842e-08 ||r(i)||/||b|| 3.536655888016e-02
83 KSP unpreconditioned resid norm 3.045914214927e-08 true resid norm 3.045914214927e-08 ||r(i)||/||b|| 3.595660969995e-02
84 KSP unpreconditioned resid norm 3.095556579777e-08 true resid norm 3.095556579777e-08 ||r(i)||/||b|| 3.654263117382e-02
85 KSP unpreconditioned resid norm 3.108374161273e-08 true resid norm 3.108374161273e-08 ||r(i)||/||b|| 3.669394100812e-02
86 KSP unpreconditioned resid norm 3.098332255348e-08 true resid norm 3.098332255348e-08 ||r(i)||/||b|| 3.657539765249e-02
87 KSP unpreconditioned resid norm 3.090888002411e-08 true resid norm 3.090888002411e-08 ||r(i)||/||b|| 3.648751924276e-02
88 KSP unpreconditioned resid norm 3.060254382784e-08 true resid norm 3.060254382784e-08 ||r(i)||/||b|| 3.612589346250e-02
89 KSP unpreconditioned resid norm 3.054287138188e-08 true resid norm 3.054287138188e-08 ||r(i)||/||b|| 3.605545093858e-02
90 KSP unpreconditioned resid norm 3.019326857855e-08 true resid norm 3.019326857855e-08 ||r(i)||/||b|| 3.564274950767e-02
91 KSP unpreconditioned resid norm 2.981415246582e-08 true resid norm 2.981415246582e-08 ||r(i)||/||b|| 3.519520800996e-02
92 KSP unpreconditioned resid norm 2.934425853265e-08 true resid norm 2.934425853265e-08 ||r(i)||/||b|| 3.464050451002e-02
93 KSP unpreconditioned resid norm 2.892868376837e-08 true resid norm 2.892868376837e-08 ||r(i)||/||b|| 3.414992406205e-02
94 KSP unpreconditioned resid norm 2.807160358194e-08 true resid norm 2.807160358194e-08 ||r(i)||/||b|| 3.313815237150e-02
95 KSP unpreconditioned resid norm 2.733196031734e-08 true resid norm 2.733196031734e-08 ||r(i)||/||b|| 3.226501339561e-02
96 KSP unpreconditioned resid norm 2.674172249913e-08 true resid norm 2.674172249913e-08 ||r(i)||/||b|| 3.156824554984e-02
97 KSP unpreconditioned resid norm 2.570861397552e-08 true resid norm 2.570861397552e-08 ||r(i)||/||b|| 3.034867476288e-02
98 KSP unpreconditioned resid norm 2.480290643102e-08 true resid norm 2.480290643102e-08 ||r(i)||/||b|| 2.927949912687e-02
99 KSP unpreconditioned resid norm 2.393472264734e-08 true resid norm 2.393472264734e-08 ||r(i)||/||b|| 2.825461978836e-02
100 KSP unpreconditioned resid norm 2.275435374520e-08 true resid norm 2.275435374520e-08 ||r(i)||/||b|| 2.686121009520e-02
101 KSP unpreconditioned resid norm 2.178902400109e-08 true resid norm 2.178902400109e-08 ||r(i)||/||b|| 2.572165125041e-02
102 KSP unpreconditioned resid norm 2.056965586266e-08 true resid norm 2.056965586266e-08 ||r(i)||/||b|| 2.428220348069e-02
103 KSP unpreconditioned resid norm 1.971345133315e-08 true resid norm 1.971345133315e-08 ||r(i)||/||b|| 2.327146549143e-02
104 KSP unpreconditioned resid norm 1.874079176158e-08 true resid norm 1.874079176158e-08 ||r(i)||/||b|| 2.212325388341e-02
105 KSP unpreconditioned resid norm 1.786435683475e-08 true resid norm 1.786435683475e-08 ||r(i)||/||b|| 2.108863418083e-02
106 KSP unpreconditioned resid norm 1.706457796840e-08 true resid norm 1.706457796840e-08 ||r(i)||/||b|| 2.014450593183e-02
107 KSP unpreconditioned resid norm 1.630473998948e-08 true resid norm 1.630473998948e-08 ||r(i)||/||b|| 1.924752736594e-02
108 KSP unpreconditioned resid norm 1.574283867690e-08 true resid norm 1.574283867690e-08 ||r(i)||/||b|| 1.858421038585e-02
109 KSP unpreconditioned resid norm 1.556968009361e-08 true resid norm 1.556968009361e-08 ||r(i)||/||b|| 1.837979899551e-02
110 KSP unpreconditioned resid norm 1.523973136297e-08 true resid norm 1.523973136297e-08 ||r(i)||/||b|| 1.799029893439e-02
111 KSP unpreconditioned resid norm 1.488557248475e-08 true resid norm 1.488557248475e-08 ||r(i)||/||b|| 1.757221911804e-02
112 KSP unpreconditioned resid norm 1.449864006948e-08 true resid norm 1.449864006948e-08 ||r(i)||/||b|| 1.711545057978e-02
113 KSP unpreconditioned resid norm 1.432126672372e-08 true resid norm 1.432126672372e-08 ||r(i)||/||b|| 1.690606371874e-02
114 KSP unpreconditioned resid norm 1.415386315852e-08 true resid norm 1.415386315852e-08 ||r(i)||/||b|| 1.670844605023e-02
115 KSP unpreconditioned resid norm 1.382237564251e-08 true resid norm 1.382237564251e-08 ||r(i)||/||b|| 1.631712947358e-02
116 KSP unpreconditioned resid norm 1.352938132525e-08 true resid norm 1.352938132525e-08 ||r(i)||/||b|| 1.597125360294e-02
117 KSP unpreconditioned resid norm 1.332023721901e-08 true resid norm 1.332023721901e-08 ||r(i)||/||b|| 1.572436178431e-02
118 KSP unpreconditioned resid norm 1.294810005035e-08 true resid norm 1.294810005035e-08 ||r(i)||/||b|| 1.528505883668e-02
119 KSP unpreconditioned resid norm 1.260835085133e-08 true resid norm 1.260835085133e-08 ||r(i)||/||b|| 1.488398945380e-02
120 KSP unpreconditioned resid norm 1.232815412303e-08 true resid norm 1.232815412303e-08 ||r(i)||/||b|| 1.455322096566e-02
121 KSP unpreconditioned resid norm 1.200898521152e-08 true resid norm 1.200898521152e-08 ||r(i)||/||b|| 1.417644633677e-02
122 KSP unpreconditioned resid norm 1.180686235502e-08 true resid norm 1.180686235502e-08 ||r(i)||/||b|| 1.393784300949e-02
123 KSP unpreconditioned resid norm 1.152359469099e-08 true resid norm 1.152359469099e-08 ||r(i)||/||b|| 1.360344932282e-02
124 KSP unpreconditioned resid norm 1.129846801969e-08 true resid norm 1.129846801969e-08 ||r(i)||/||b|| 1.333769030003e-02
125 KSP unpreconditioned resid norm 1.081947011589e-08 true resid norm 1.081947011589e-08 ||r(i)||/||b|| 1.277223968459e-02
126 KSP unpreconditioned resid norm 1.036954512419e-08 true resid norm 1.036954512419e-08 ||r(i)||/||b|| 1.224110925283e-02
127 KSP unpreconditioned resid norm 9.744507014406e-09 true resid norm 9.744507014406e-09 ||r(i)||/||b|| 1.150326012855e-02
128 KSP unpreconditioned resid norm 9.160148503681e-09 true resid norm 9.160148503681e-09 ||r(i)||/||b|| 1.081343272658e-02
129 KSP unpreconditioned resid norm 8.713396021123e-09 true resid norm 8.713396021123e-09 ||r(i)||/||b|| 1.028604739941e-02
130 KSP unpreconditioned resid norm 8.065538786935e-09 true resid norm 8.065538786935e-09 ||r(i)||/||b|| 9.521260604138e-03
131 KSP unpreconditioned resid norm 7.600710595219e-09 true resid norm 7.600710595219e-09 ||r(i)||/||b|| 8.972537144194e-03
132 KSP unpreconditioned resid norm 7.042499013017e-09 true resid norm 7.042499013017e-09 ||r(i)||/||b|| 8.313575841447e-03
133 KSP unpreconditioned resid norm 6.488293059262e-09 true resid norm 6.488293059262e-09 ||r(i)||/||b|| 7.659343129478e-03
134 KSP unpreconditioned resid norm 6.095008004956e-09 true resid norm 6.095008004956e-09 ||r(i)||/||b|| 7.195075385850e-03
135 KSP unpreconditioned resid norm 5.689629867887e-09 true resid norm 5.689629867887e-09 ||r(i)||/||b|| 6.716531919850e-03
136 KSP unpreconditioned resid norm 5.350918350727e-09 true resid norm 5.350918350727e-09 ||r(i)||/||b|| 6.316687506514e-03
137 KSP unpreconditioned resid norm 5.077782670826e-09 true resid norm 5.077782670826e-09 ||r(i)||/||b|| 5.994254491520e-03
138 KSP unpreconditioned resid norm 4.693456261445e-09 true resid norm 4.693456261445e-09 ||r(i)||/||b|| 5.540562308339e-03
139 KSP unpreconditioned resid norm 4.327021695310e-09 true resid norm 4.327021695310e-09 ||r(i)||/||b|| 5.107991206681e-03
140 KSP unpreconditioned resid norm 4.022960135943e-09 true resid norm 4.022960135943e-09 ||r(i)||/||b|| 4.749050604830e-03
141 KSP unpreconditioned resid norm 3.803745635547e-09 true resid norm 3.803745635547e-09 ||r(i)||/||b|| 4.490270820662e-03
142 KSP unpreconditioned resid norm 3.581362358291e-09 true resid norm 3.581362358291e-09 ||r(i)||/||b|| 4.227750337817e-03
143 KSP unpreconditioned resid norm 3.408132039231e-09 true resid norm 3.408132039231e-09 ||r(i)||/||b|| 4.023254264351e-03
144 KSP unpreconditioned resid norm 3.344832032939e-09 true resid norm 3.344832032939e-09 ||r(i)||/||b|| 3.948529454011e-03
145 KSP unpreconditioned resid norm 3.208107857745e-09 true resid norm 3.208107857745e-09 ||r(i)||/||b|| 3.787128394851e-03
146 KSP unpreconditioned resid norm 3.087343355611e-09 true resid norm 3.087343355611e-09 ||r(i)||/||b|| 3.644567516164e-03
147 KSP unpreconditioned resid norm 3.022538018497e-09 true resid norm 3.022538018497e-09 ||r(i)||/||b|| 3.568065682932e-03
148 KSP unpreconditioned resid norm 2.923230289033e-09 true resid norm 2.923230289033e-09 ||r(i)||/||b|| 3.450834237246e-03
149 KSP unpreconditioned resid norm 2.833109002568e-09 true resid norm 2.833109002568e-09 ||r(i)||/||b|| 3.344447264586e-03
150 KSP unpreconditioned resid norm 2.753952392615e-09 true resid norm 2.753952392615e-09 ||r(i)||/||b|| 3.251003945818e-03
151 KSP unpreconditioned resid norm 2.677124193740e-09 true resid norm 2.677124193740e-09 ||r(i)||/||b|| 3.160309285168e-03
152 KSP unpreconditioned resid norm 2.578412706791e-09 true resid norm 2.578412706791e-09 ||r(i)||/||b|| 3.043781695792e-03
153 KSP unpreconditioned resid norm 2.449680459273e-09 true resid norm 2.449680459273e-09 ||r(i)||/||b|| 2.891815000305e-03
154 KSP unpreconditioned resid norm 2.350739446063e-09 true resid norm 2.350739446063e-09 ||r(i)||/||b|| 2.775016458249e-03
155 KSP unpreconditioned resid norm 2.230331194218e-09 true resid norm 2.230331194218e-09 ||r(i)||/||b|| 2.632876128261e-03
156 KSP unpreconditioned resid norm 2.095716328619e-09 true resid norm 2.095716328619e-09 ||r(i)||/||b|| 2.473965080851e-03
157 KSP unpreconditioned resid norm 1.967063208093e-09 true resid norm 1.967063208093e-09 ||r(i)||/||b|| 2.322091793720e-03
158 KSP unpreconditioned resid norm 1.861768684115e-09 true resid norm 1.861768684115e-09 ||r(i)||/||b|| 2.197793017225e-03
159 KSP unpreconditioned resid norm 1.789027091537e-09 true resid norm 1.789027091537e-09 ||r(i)||/||b|| 2.111922540622e-03
160 KSP unpreconditioned resid norm 1.684922987165e-09 true resid norm 1.684922987165e-09 ||r(i)||/||b|| 1.989029038542e-03
161 KSP unpreconditioned resid norm 1.582609227035e-09 true resid norm 1.582609227035e-09 ||r(i)||/||b|| 1.868249013882e-03
162 KSP unpreconditioned resid norm 1.501597611234e-09 true resid norm 1.501597611234e-09 ||r(i)||/||b|| 1.772615885534e-03
163 KSP unpreconditioned resid norm 1.395819010765e-09 true resid norm 1.395819010765e-09 ||r(i)||/||b|| 1.647745663219e-03
164 KSP unpreconditioned resid norm 1.271906790435e-09 true resid norm 1.271906790435e-09 ||r(i)||/||b|| 1.501468945326e-03
165 KSP unpreconditioned resid norm 1.160968729258e-09 true resid norm 1.160968729258e-09 ||r(i)||/||b|| 1.370508048690e-03
166 KSP unpreconditioned resid norm 1.086312221042e-09 true resid norm 1.086312221042e-09 ||r(i)||/||b|| 1.282377039802e-03
167 KSP unpreconditioned resid norm 1.010733613894e-09 true resid norm 1.010733613894e-09 ||r(i)||/||b|| 1.193157505465e-03
168 KSP unpreconditioned resid norm 9.659506407625e-10 true resid norm 9.659506407625e-10 ||r(i)||/||b|| 1.140291804973e-03
169 KSP unpreconditioned resid norm 9.142920667557e-10 true resid norm 9.142920667557e-10 ||r(i)||/||b|| 1.079309549658e-03
170 KSP unpreconditioned resid norm 8.777007956409e-10 true resid norm 8.777007956409e-10 ||r(i)||/||b|| 1.036114043775e-03
171 KSP unpreconditioned resid norm 8.651938742751e-10 true resid norm 8.651938742751e-10 ||r(i)||/||b|| 1.021349790472e-03
172 KSP unpreconditioned resid norm 8.449669697471e-10 true resid norm 8.449669697471e-10 ||r(i)||/||b|| 9.974722003548e-04
173 KSP unpreconditioned resid norm 8.226753443690e-10 true resid norm 8.226753443690e-10 ||r(i)||/||b|| 9.711572349047e-04
174 KSP unpreconditioned resid norm 8.119036890805e-10 true resid norm 8.119036890805e-10 ||r(i)||/||b|| 9.584414399840e-04
175 KSP unpreconditioned resid norm 7.969494091924e-10 true resid norm 7.969494091924e-10 ||r(i)||/||b|| 9.407881126958e-04
176 KSP unpreconditioned resid norm 7.927024508324e-10 true resid norm 7.927024508324e-10 ||r(i)||/||b|| 9.357746351850e-04
177 KSP unpreconditioned resid norm 7.828725447824e-10 true resid norm 7.828725447824e-10 ||r(i)||/||b|| 9.241705626377e-04
178 KSP unpreconditioned resid norm 7.494508082515e-10 true resid norm 7.494508082515e-10 ||r(i)||/||b|| 8.847166499160e-04
179 KSP unpreconditioned resid norm 7.231444464622e-10 true resid norm 7.231444464622e-10 ||r(i)||/||b|| 8.536623418581e-04
180 KSP unpreconditioned resid norm 7.011791261722e-10 true resid norm 7.011791261722e-10 ||r(i)||/||b|| 8.277325751978e-04
181 KSP unpreconditioned resid norm 6.696139551270e-10 true resid norm 6.696139551271e-10 ||r(i)||/||b|| 7.904703131872e-04
182 KSP unpreconditioned resid norm 6.451235118753e-10 true resid norm 6.451235118753e-10 ||r(i)||/||b|| 7.615596726621e-04
183 KSP unpreconditioned resid norm 6.306916481370e-10 true resid norm 6.306916481370e-10 ||r(i)||/||b|| 7.445230506476e-04
184 KSP unpreconditioned resid norm 6.100227035205e-10 true resid norm 6.100227035205e-10 ||r(i)||/||b|| 7.201236381217e-04
185 KSP unpreconditioned resid norm 5.919504539386e-10 true resid norm 5.919504539386e-10 ||r(i)||/||b|| 6.987895893349e-04
186 KSP unpreconditioned resid norm 5.856700542909e-10 true resid norm 5.856700542909e-10 ||r(i)||/||b|| 6.913756615957e-04
187 KSP unpreconditioned resid norm 5.715591865793e-10 true resid norm 5.715591865793e-10 ||r(i)||/||b|| 6.747179710952e-04
188 KSP unpreconditioned resid norm 5.547066799722e-10 true resid norm 5.547066799722e-10 ||r(i)||/||b|| 6.548238125674e-04
189 KSP unpreconditioned resid norm 5.480604885087e-10 true resid norm 5.480604885087e-10 ||r(i)||/||b|| 6.469780724848e-04
190 KSP unpreconditioned resid norm 5.296236594065e-10 true resid norm 5.296236594065e-10 ||r(i)||/||b|| 6.252136424531e-04
191 KSP unpreconditioned resid norm 5.105824359200e-10 true resid norm 5.105824359201e-10 ||r(i)||/||b|| 6.027357329389e-04
192 KSP unpreconditioned resid norm 4.831649558134e-10 true resid norm 4.831649558134e-10 ||r(i)||/||b|| 5.703697645764e-04
193 KSP unpreconditioned resid norm 4.690710598203e-10 true resid norm 4.690710598203e-10 ||r(i)||/||b|| 5.537321089625e-04
194 KSP unpreconditioned resid norm 4.547545034138e-10 true resid norm 4.547545034138e-10 ||r(i)||/||b|| 5.368316057102e-04
195 KSP unpreconditioned resid norm 4.328632397413e-10 true resid norm 4.328632397413e-10 ||r(i)||/||b|| 5.109892618959e-04
196 KSP unpreconditioned resid norm 4.079089522157e-10 true resid norm 4.079089522157e-10 ||r(i)||/||b|| 4.815310594126e-04
197 KSP unpreconditioned resid norm 3.873059797615e-10 true resid norm 3.873059797615e-10 ||r(i)||/||b|| 4.572095261415e-04
198 KSP unpreconditioned resid norm 3.729969299802e-10 true resid norm 3.729969299802e-10 ||r(i)||/||b|| 4.403178843598e-04
199 KSP unpreconditioned resid norm 3.477970037895e-10 true resid norm 3.477970037895e-10 ||r(i)||/||b|| 4.105697087196e-04
200 KSP unpreconditioned resid norm 3.282214610955e-10 true resid norm 3.282214610955e-10 ||r(i)||/||b|| 3.874610425312e-04
201 KSP unpreconditioned resid norm 3.120709893337e-10 true resid norm 3.120709893337e-10 ||r(i)||/||b|| 3.683956267436e-04
202 KSP unpreconditioned resid norm 2.888671401828e-10 true resid norm 2.888671401828e-10 ||r(i)||/||b|| 3.410037933371e-04
203 KSP unpreconditioned resid norm 2.762162725563e-10 true resid norm 2.762162725563e-10 ||r(i)||/||b|| 3.260696133992e-04
204 KSP unpreconditioned resid norm 2.633334545155e-10 true resid norm 2.633334545155e-10 ||r(i)||/||b|| 3.108616191011e-04
205 KSP unpreconditioned resid norm 2.491344685484e-10 true resid norm 2.491344685484e-10 ||r(i)||/||b|| 2.940999061792e-04
206 KSP unpreconditioned resid norm 2.386964029507e-10 true resid norm 2.386964029507e-10 ||r(i)||/||b|| 2.817779094244e-04
207 KSP unpreconditioned resid norm 2.274820848858e-10 true resid norm 2.274820848858e-10 ||r(i)||/||b|| 2.685395570199e-04
208 KSP unpreconditioned resid norm 2.212409903975e-10 true resid norm 2.212409903975e-10 ||r(i)||/||b|| 2.611720284954e-04
209 KSP unpreconditioned resid norm 2.114227895993e-10 true resid norm 2.114227895993e-10 ||r(i)||/||b|| 2.495817738413e-04
210 KSP unpreconditioned resid norm 2.048332706014e-10 true resid norm 2.048332706014e-10 ||r(i)||/||b|| 2.418029348459e-04
211 KSP unpreconditioned resid norm 1.975994847684e-10 true resid norm 1.975994847684e-10 ||r(i)||/||b|| 2.332635474732e-04
212 KSP unpreconditioned resid norm 1.907641063078e-10 true resid norm 1.907641063078e-10 ||r(i)||/||b|| 2.251944746722e-04
213 KSP unpreconditioned resid norm 1.835317861753e-10 true resid norm 1.835317861753e-10 ||r(i)||/||b|| 2.166568175394e-04
214 KSP unpreconditioned resid norm 1.783343277288e-10 true resid norm 1.783343277288e-10 ||r(i)||/||b|| 2.105212873962e-04
215 KSP unpreconditioned resid norm 1.737252962117e-10 true resid norm 1.737252962116e-10 ||r(i)||/||b|| 2.050803873687e-04
216 KSP unpreconditioned resid norm 1.681090858385e-10 true resid norm 1.681090858385e-10 ||r(i)||/||b|| 1.984505261799e-04
217 KSP unpreconditioned resid norm 1.634109844504e-10 true resid norm 1.634109844504e-10 ||r(i)||/||b|| 1.929044803617e-04
218 KSP unpreconditioned resid norm 1.596828500439e-10 true resid norm 1.596828500438e-10 ||r(i)||/||b|| 1.885034675850e-04
219 KSP unpreconditioned resid norm 1.559655758914e-10 true resid norm 1.559655758914e-10 ||r(i)||/||b|| 1.841152751930e-04
220 KSP unpreconditioned resid norm 1.497441081261e-10 true resid norm 1.497441081261e-10 ||r(i)||/||b|| 1.767709157523e-04
221 KSP unpreconditioned resid norm 1.440543864995e-10 true resid norm 1.440543864995e-10 ||r(i)||/||b|| 1.700542755126e-04
222 KSP unpreconditioned resid norm 1.406455332877e-10 true resid norm 1.406455332877e-10 ||r(i)||/||b|| 1.660301699136e-04
223 KSP unpreconditioned resid norm 1.352855224988e-10 true resid norm 1.352855224988e-10 ||r(i)||/||b|| 1.597027489055e-04
224 KSP unpreconditioned resid norm 1.318566764812e-10 true resid norm 1.318566764812e-10 ||r(i)||/||b|| 1.556550420670e-04
225 KSP unpreconditioned resid norm 1.247714207143e-10 true resid norm 1.247714207143e-10 ||r(i)||/||b|| 1.472909924497e-04
226 KSP unpreconditioned resid norm 1.170326734811e-10 true resid norm 1.170326734811e-10 ||r(i)||/||b|| 1.381555049016e-04
227 KSP unpreconditioned resid norm 1.104850176393e-10 true resid norm 1.104850176393e-10 ||r(i)||/||b|| 1.304260848017e-04
228 KSP unpreconditioned resid norm 1.057860311059e-10 true resid norm 1.057860311059e-10 ||r(i)||/||b|| 1.248789940814e-04
229 KSP unpreconditioned resid norm 1.010024307373e-10 true resid norm 1.010024307373e-10 ||r(i)||/||b|| 1.192320178609e-04
230 KSP unpreconditioned resid norm 9.517109911279e-11 true resid norm 9.517109911278e-11 ||r(i)||/||b|| 1.123482089136e-04
231 KSP unpreconditioned resid norm 8.906657903584e-11 true resid norm 8.906657903583e-11 ||r(i)||/||b|| 1.051419046540e-04
232 KSP unpreconditioned resid norm 8.571475780780e-11 true resid norm 8.571475780779e-11 ||r(i)||/||b|| 1.011851245487e-04
233 KSP unpreconditioned resid norm 8.105732120348e-11 true resid norm 8.105732120347e-11 ||r(i)||/||b|| 9.568708296360e-05
234 KSP unpreconditioned resid norm 7.805220078644e-11 true resid norm 7.805220078644e-11 ||r(i)||/||b|| 9.213957852611e-05
235 KSP unpreconditioned resid norm 7.445788494535e-11 true resid norm 7.445788494536e-11 ||r(i)||/||b|| 8.789653677520e-05
236 KSP unpreconditioned resid norm 7.257750273074e-11 true resid norm 7.257750273076e-11 ||r(i)||/||b|| 8.567677073433e-05
237 KSP unpreconditioned resid norm 7.062425062277e-11 true resid norm 7.062425062278e-11 ||r(i)||/||b|| 8.337098275948e-05
238 KSP unpreconditioned resid norm 6.715189342905e-11 true resid norm 6.715189342907e-11 ||r(i)||/||b|| 7.927191155973e-05
239 KSP unpreconditioned resid norm 6.623125011338e-11 true resid norm 6.623125011340e-11 ||r(i)||/||b|| 7.818510444572e-05
240 KSP unpreconditioned resid norm 6.442916815518e-11 true resid norm 6.442916815520e-11 ||r(i)||/||b|| 7.605777080971e-05
241 KSP unpreconditioned resid norm 6.297584870411e-11 true resid norm 6.297584870414e-11 ||r(i)||/||b|| 7.434214664619e-05
242 KSP unpreconditioned resid norm 6.218304192586e-11 true resid norm 6.218304192588e-11 ||r(i)||/||b|| 7.340624885388e-05
243 KSP unpreconditioned resid norm 6.119793248448e-11 true resid norm 6.119793248450e-11 ||r(i)||/||b|| 7.224334034115e-05
244 KSP unpreconditioned resid norm 6.039481879983e-11 true resid norm 6.039481879984e-11 ||r(i)||/||b|| 7.129527538376e-05
245 KSP unpreconditioned resid norm 6.025966944200e-11 true resid norm 6.025966944201e-11 ||r(i)||/||b|| 7.113573337542e-05
246 KSP unpreconditioned resid norm 5.948714963964e-11 true resid norm 5.948714963965e-11 ||r(i)||/||b|| 7.022378408667e-05
247 KSP unpreconditioned resid norm 5.875133821595e-11 true resid norm 5.875133821596e-11 ||r(i)||/||b|| 6.935516854771e-05
248 KSP unpreconditioned resid norm 5.865559932945e-11 true resid norm 5.865559932945e-11 ||r(i)||/||b|| 6.924215007337e-05
249 KSP unpreconditioned resid norm 5.807527161438e-11 true resid norm 5.807527161438e-11 ||r(i)||/||b|| 6.855708097174e-05
250 KSP unpreconditioned resid norm 5.729567969355e-11 true resid norm 5.729567969356e-11 ||r(i)||/||b|| 6.763678314178e-05
251 KSP unpreconditioned resid norm 5.638458710165e-11 true resid norm 5.638458710166e-11 ||r(i)||/||b|| 6.656125052938e-05
252 KSP unpreconditioned resid norm 5.502345910189e-11 true resid norm 5.502345910190e-11 ||r(i)||/||b|| 6.495445714041e-05
253 KSP unpreconditioned resid norm 5.380650850111e-11 true resid norm 5.380650850112e-11 ||r(i)||/||b|| 6.351786324155e-05
254 KSP unpreconditioned resid norm 5.122861296752e-11 true resid norm 5.122861296753e-11 ||r(i)||/||b|| 6.047469206179e-05
255 KSP unpreconditioned resid norm 4.940773406195e-11 true resid norm 4.940773406196e-11 ||r(i)||/||b|| 5.832516888095e-05
256 KSP unpreconditioned resid norm 4.709810110736e-11 true resid norm 4.709810110736e-11 ||r(i)||/||b|| 5.559867808578e-05
257 KSP unpreconditioned resid norm 4.445230653863e-11 true resid norm 4.445230653862e-11 ||r(i)||/||b|| 5.247535300367e-05
258 KSP unpreconditioned resid norm 4.227887565137e-11 true resid norm 4.227887565136e-11 ||r(i)||/||b|| 4.990964692632e-05
259 KSP unpreconditioned resid norm 3.927450241359e-11 true resid norm 3.927450241358e-11 ||r(i)||/||b|| 4.636302452395e-05
260 KSP unpreconditioned resid norm 3.754948266816e-11 true resid norm 3.754948266814e-11 ||r(i)||/||b|| 4.432666179884e-05
261 KSP unpreconditioned resid norm 3.517864153420e-11 true resid norm 3.517864153417e-11 ||r(i)||/||b|| 4.152791556703e-05
262 KSP unpreconditioned resid norm 3.347259122434e-11 true resid norm 3.347259122432e-11 ||r(i)||/||b|| 3.951394600678e-05
263 KSP unpreconditioned resid norm 3.122604584625e-11 true resid norm 3.122604584623e-11 ||r(i)||/||b|| 3.686192925144e-05
264 KSP unpreconditioned resid norm 2.921844421531e-11 true resid norm 2.921844421529e-11 ||r(i)||/||b|| 3.449198239204e-05
265 KSP unpreconditioned resid norm 2.807535371206e-11 true resid norm 2.807535371204e-11 ||r(i)||/||b|| 3.314257934990e-05
266 KSP unpreconditioned resid norm 2.679326847398e-11 true resid norm 2.679326847397e-11 ||r(i)||/||b|| 3.162909488335e-05
267 KSP unpreconditioned resid norm 2.620100211750e-11 true resid norm 2.620100211750e-11 ||r(i)||/||b|| 3.092993237530e-05
268 KSP unpreconditioned resid norm 2.480717416211e-11 true resid norm 2.480717416212e-11 ||r(i)||/||b|| 2.928453712632e-05
269 KSP unpreconditioned resid norm 2.421629839252e-11 true resid norm 2.421629839252e-11 ||r(i)||/||b|| 2.858701618747e-05
270 KSP unpreconditioned resid norm 2.361009921624e-11 true resid norm 2.361009921625e-11 ||r(i)||/||b|| 2.787140617210e-05
271 KSP unpreconditioned resid norm 2.251561534423e-11 true resid norm 2.251561534425e-11 ||r(i)||/||b|| 2.657938260769e-05
272 KSP unpreconditioned resid norm 2.149855566100e-11 true resid norm 2.149855566102e-11 ||r(i)||/||b|| 2.537875726203e-05
273 KSP unpreconditioned resid norm 2.108747915310e-11 true resid norm 2.108747915312e-11 ||r(i)||/||b|| 2.489348694552e-05
274 KSP unpreconditioned resid norm 2.035331893385e-11 true resid norm 2.035331893387e-11 ||r(i)||/||b|| 2.402682063130e-05
275 KSP unpreconditioned resid norm 1.941734891236e-11 true resid norm 1.941734891238e-11 ||r(i)||/||b|| 2.292192054618e-05
276 KSP unpreconditioned resid norm 1.853704982428e-11 true resid norm 1.853704982430e-11 ||r(i)||/||b|| 2.188273925295e-05
277 KSP unpreconditioned resid norm 1.792368946256e-11 true resid norm 1.792368946257e-11 ||r(i)||/||b|| 2.115867555398e-05
278 KSP unpreconditioned resid norm 1.760944257573e-11 true resid norm 1.760944257573e-11 ||r(i)||/||b|| 2.078771130935e-05
279 KSP unpreconditioned resid norm 1.714545149771e-11 true resid norm 1.714545149771e-11 ||r(i)||/||b|| 2.023997605092e-05
280 KSP unpreconditioned resid norm 1.671909267963e-11 true resid norm 1.671909267963e-11 ||r(i)||/||b|| 1.973666517175e-05
281 KSP unpreconditioned resid norm 1.609444716094e-11 true resid norm 1.609444716094e-11 ||r(i)||/||b|| 1.899927949600e-05
282 KSP unpreconditioned resid norm 1.566178459113e-11 true resid norm 1.566178459112e-11 ||r(i)||/||b|| 1.848852712227e-05
283 KSP unpreconditioned resid norm 1.514203649442e-11 true resid norm 1.514203649441e-11 ||r(i)||/||b|| 1.787497144942e-05
284 KSP unpreconditioned resid norm 1.456362863785e-11 true resid norm 1.456362863785e-11 ||r(i)||/||b|| 1.719216871506e-05
285 KSP unpreconditioned resid norm 1.376505865010e-11 true resid norm 1.376505865009e-11 ||r(i)||/||b|| 1.624946753105e-05
286 KSP unpreconditioned resid norm 1.303680225659e-11 true resid norm 1.303680225658e-11 ||r(i)||/||b|| 1.538977060411e-05
287 KSP unpreconditioned resid norm 1.225258977795e-11 true resid norm 1.225258977795e-11 ||r(i)||/||b|| 1.446401826750e-05
288 KSP unpreconditioned resid norm 1.121424657129e-11 true resid norm 1.121424657129e-11 ||r(i)||/||b|| 1.323826800725e-05
289 KSP unpreconditioned resid norm 1.054203483048e-11 true resid norm 1.054203483047e-11 ||r(i)||/||b|| 1.244473104282e-05
290 KSP unpreconditioned resid norm 9.820019392645e-12 true resid norm 9.820019392636e-12 ||r(i)||/||b|| 1.159240148054e-05
291 KSP unpreconditioned resid norm 9.223807475811e-12 true resid norm 9.223807475802e-12 ||r(i)||/||b|| 1.088858129128e-05
292 KSP unpreconditioned resid norm 8.482817720280e-12 true resid norm 8.482817720271e-12 ||r(i)||/||b|| 1.001385280087e-05
293 KSP unpreconditioned resid norm 8.015330479160e-12 true resid norm 8.015330479152e-12 ||r(i)||/||b|| 9.461990368692e-06
KSP Object: 128 MPI processes
type: cg
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
has attached null space
using UNPRECONDITIONED norm type for convergence test
PC Object: 128 MPI processes
type: mg
MG: type is MULTIPLICATIVE, levels=5 cycles=v
Cycles per PCApply=1
Using Galerkin computed coarse grid matrices
Coarse grid solver -- level -------------------------------
KSP Object: (mg_coarse_) 128 MPI processes
type: preonly
maximum iterations=1, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using NONE norm type for convergence test
PC Object: (mg_coarse_) 128 MPI processes
type: lu
LU: out-of-place factorization
tolerance for zero pivot 2.22045e-14
matrix ordering: natural
factor fill ratio given 0, needed 0
Factored matrix follows:
Matrix Object: 128 MPI processes
type: mpiaij
rows=1024, cols=1024
package used to perform factorization: superlu_dist
total: nonzeros=0, allocated nonzeros=0
total number of mallocs used during MatSetValues calls =0
SuperLU_DIST run parameters:
Process grid nprow 16 x npcol 8
Equilibrate matrix TRUE
Matrix input mode 1
Replace tiny pivots TRUE
Use iterative refinement FALSE
Processors in row 16 col partition 8
Row permutation LargeDiag
Column permutation METIS_AT_PLUS_A
Parallel symbolic factorization FALSE
Repeated factorization SamePattern_SameRowPerm
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=1024, cols=1024
total: nonzeros=27648, allocated nonzeros=27648
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Down solver (pre-smoother) on level 1 -------------------------------
KSP Object: (mg_levels_1_) 128 MPI processes
type: richardson
Richardson: damping factor=1
maximum iterations=1
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (mg_levels_1_) 128 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=8192, cols=8192
total: nonzeros=221184, allocated nonzeros=221184
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 16 nodes, limit used is 5
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 2 -------------------------------
KSP Object: (mg_levels_2_) 128 MPI processes
type: richardson
Richardson: damping factor=1
maximum iterations=1
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (mg_levels_2_) 128 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=65536, cols=65536
total: nonzeros=1769472, allocated nonzeros=1769472
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 3 -------------------------------
KSP Object: (mg_levels_3_) 128 MPI processes
type: richardson
Richardson: damping factor=1
maximum iterations=1
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (mg_levels_3_) 128 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=524288, cols=524288
total: nonzeros=14155776, allocated nonzeros=14155776
total number of mallocs used during MatSetValues calls =0
not using I-node (on process 0) routines
Up solver (post-smoother) same as down solver (pre-smoother)
Down solver (pre-smoother) on level 4 -------------------------------
KSP Object: (mg_levels_4_) 128 MPI processes
type: richardson
Richardson: damping factor=1
maximum iterations=1
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using nonzero initial guess
using NONE norm type for convergence test
PC Object: (mg_levels_4_) 128 MPI processes
type: sor
SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=4194304, cols=4194304
total: nonzeros=29360128, allocated nonzeros=29360128
total number of mallocs used during MatSetValues calls =0
Up solver (post-smoother) same as down solver (pre-smoother)
linear system matrix = precond matrix:
Matrix Object: 128 MPI processes
type: mpiaij
rows=4194304, cols=4194304
total: nonzeros=29360128, allocated nonzeros=29360128
total number of mallocs used during MatSetValues calls =0
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./hit on a interlagos-64idx-pgi-opt named nid25475 with 128 processors, by Unknown Fri Sep 20 12:56:00 2013
Using Petsc Release Version 3.4.2, Jul, 02, 2013
Max Max/Min Avg Total
Time (sec): 3.240e+03 1.00000 3.240e+03
Objects: 5.352e+05 1.00000 5.352e+05
Flops: 4.067e+11 1.00000 4.067e+11 5.206e+13
Flops/sec: 1.255e+08 1.00000 1.255e+08 1.606e+10
MPI Messages: 7.811e+06 1.00000 7.811e+06 9.998e+08
MPI Message Lengths: 2.990e+10 1.00000 3.827e+03 3.827e+12
MPI Reductions: 1.522e+06 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 3.2403e+03 100.0% 5.2056e+13 100.0% 9.998e+08 100.0% 3.827e+03 100.0% 1.522e+06 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %f - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecTDot 177386 1.0 5.7980e+01 2.1 1.16e+10 1.0 0.0e+00 0.0e+00 1.8e+05 1 3 0 0 12 1 3 0 0 12 25664
VecNorm 179519 1.0 4.6141e+01 3.5 1.18e+10 1.0 0.0e+00 0.0e+00 1.8e+05 1 3 0 0 12 1 3 0 0 12 32637
VecScale 354772 1.0 2.3496e+00 1.5 1.00e+09 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 54733
VecCopy 179519 1.0 8.4071e+00 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 1243857 1.0 1.8819e+01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0
VecAXPY 177386 1.0 2.6131e+01 1.2 1.16e+10 1.0 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 56944
VecAYPX 532158 1.0 4.1760e+01 1.3 1.20e+10 1.0 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 36832
VecScatterBegin 1598611 1.0 9.1558e+01 1.3 0.00e+00 0.0 1.0e+09 3.8e+03 0.0e+00 3 0100100 0 3 0100100 0 0
VecScatterEnd 1598611 1.0 2.5825e+02 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 6 0 0 0 0 6 0 0 0 0 0
MatMult 532869 1.0 1.0230e+03 1.1 1.36e+11 1.0 4.1e+08 6.2e+03 0.0e+00 29 33 41 66 0 29 33 41 66 0 16969
MatMultAdd 354772 1.0 1.6270e+02 1.1 2.24e+10 1.0 1.4e+08 9.9e+02 0.0e+00 5 6 14 4 0 5 6 14 4 0 17634
MatMultTranspose 354776 1.0 1.8670e+02 1.3 2.24e+10 1.0 1.4e+08 9.9e+02 0.0e+00 5 6 14 4 0 5 6 14 4 0 15367
MatSolve 88693 1.0 1.6765e+02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 4 0 0 0 0 0
MatSOR 709544 1.0 1.3796e+03 1.1 1.79e+11 1.0 3.2e+08 3.2e+03 7.1e+05 41 44 32 27 47 41 44 32 27 47 16627
MatLUFactorSym 1 1.0 6.4993e-04 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatLUFactorNum 1 1.0 1.2938e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyBegin 728 1.0 2.8366e+0018.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.4e+03 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 728 1.0 2.5708e+00 1.2 0.00e+00 0.0 1.2e+04 1.1e+03 7.2e+01 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 5.0068e-06 5.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 2.0981e-05 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatView 4977 1.0 1.3075e+00 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+03 0 0 0 0 0 0 0 0 0 0 0
MatPtAP 4 1.0 2.1368e-01 1.0 5.11e+06 1.0 2.5e+04 6.0e+03 1.0e+02 0 0 0 0 0 0 0 0 0 0 3058
MatPtAPSymbolic 4 1.0 1.5158e-01 1.1 0.00e+00 0.0 1.5e+04 7.8e+03 6.0e+01 0 0 0 0 0 0 0 0 0 0 0
MatPtAPNumeric 4 1.0 7.0144e-02 1.1 5.11e+06 1.0 9.7e+03 3.1e+03 4.0e+01 0 0 0 0 0 0 0 0 0 0 9316
MatGetLocalMat 4 1.0 2.1817e-02 3.3 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetBrAoCol 4 1.0 2.9885e-02 3.2 0.00e+00 0.0 1.1e+04 8.4e+03 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetSymTrans 8 1.0 9.0678e-03 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 6 1.0 1.0205e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 2.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 711 1.0 3.0209e+03 1.0 4.07e+11 1.0 1.0e+09 3.8e+03 1.5e+06 93100100100 99 93100100100 99 17232
PCSetUp 1 1.0 4.2324e-01 1.0 5.36e+06 1.0 3.4e+04 4.6e+03 3.0e+02 0 0 0 0 0 0 0 0 0 0 1620
PCApply 88693 1.0 2.2119e+03 1.0 2.87e+11 1.0 9.1e+08 2.6e+03 7.1e+05 67 71 91 61 47 67 71 91 61 47 16614
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 535068 535068 74824032416 0
Vector Scatter 19 19 22572 0
Matrix 38 38 14004608 0
Matrix Null Space 1 1 652 0
Distributed Mesh 5 5 830792 0
Bipartite Graph 10 10 8560 0
Index Set 47 47 534480 0
IS L to G Mapping 5 5 405756 0
Krylov Solver 7 7 9536 0
DMKSP interface 3 3 2088 0
Preconditioner 7 7 7352 0
Viewer 1 0 0 0
========================================================================================================================
Average time to get PetscTime(): 4.05312e-07
Average time for MPI_Barrier(): 1.32084e-05
Average time for zero size MPI_Send(): 2.75671e-06
#PETSc Option Table entries:
-ksp_monitor_true_residual
-ksp_view
-log_summary
-mg_coarse_pc_factor_mat_solver_package superlu_dist
-mg_coarse_pc_type lu
-mg_levels_ksp_max_it 1
-mg_levels_ksp_type richardson
-options_left
-pc_mg_galerkin
-pc_mg_levels 5
-pc_type mg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8
Configure run at: Wed Aug 28 23:25:43 2013
Configure options: --known-level1-dcache-size=16384 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=4 --known-memcmp-ok=1 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=0 --known-mpi-c-double-complex=0 --with-batch="1 " --known-mpi-shared="0 " --known-memcmp-ok --with-blas-lapack-lib="-L/opt/acml/5.3.0/pgi64/lib -lacml" --COPTFLAGS="-O3 -fastsse" --FOPTFLAGS="-O3 -fastsse" --CXXOPTFLAGS="-O3 -fastsse" --with-x="0 " --with-debugging="0 " --with-clib-autodetect="0 " --with-cxxlib-autodetect="0 " --with-fortranlib-autodetect="0 " --with-shared-libraries=0 --with-dynamic-loading=0 --with-mpi-compilers="1 " --known-mpi-shared-libraries=0 --with-64-bit-indices --download-blacs="1 " --download-scalapack="1 " --download-superlu_dist="1 " --download-metis="1 " --download-parmetis="1 " --with-cc=cc --with-cxx=CC --with-fc=ftn PETSC_ARCH=interlagos-64idx-pgi-opt
-----------------------------------------
Libraries compiled on Wed Aug 28 23:25:43 2013 on h2ologin3
Machine characteristics: Linux-2.6.32.59-0.7-default-x86_64-with-SuSE-11-x86_64
Using PETSc directory: /mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2
Using PETSc arch: interlagos-64idx-pgi-opt
-----------------------------------------
Using C compiler: cc -O3 -fastsse ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: ftn -O3 -fastsse ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/include -I/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/include -I/opt/cray/udreg/2.3.2-1.0402.7311.2.1.gem/include -I/opt/cray/ugni/5.0-1.0402.7128.7.6.gem/include -I/opt/cray/pmi/4.0.1-1.0000.9421.73.3.gem/include -I/opt/cray/dmapp/4.0.1-1.0402.7439.5.1.gem/include -I/opt/cray/gni-headers/2.1-1.0402.7082.6.2.gem/include -I/opt/cray/xpmem/0.1-2.0402.44035.2.1.gem/include -I/opt/cray/rca/1.0.0-2.0402.42153.2.106.gem/include -I/opt/cray-hss-devel/7.0.0/include -I/opt/cray/krca/1.0.0-2.0402.42157.2.94.gem/include -I/opt/cray/mpt/6.0.1/gni/mpich2-pgi/121/include -I/opt/acml/5.3.0/pgi64_fma4/include -I/opt/cray/libsci/12.1.01/pgi/121/interlagos/include -I/opt/fftw/3.3.0.3/interlagos/include -I/usr/include/alps -I/opt/pgi/13.6.0/linux86-64/13.6/include -I/opt/cray/xe-sysroot/4.2.24/usr/include
-----------------------------------------
Using C linker: cc
Using Fortran linker: ftn
Using libraries: -Wl,-rpath,/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -L/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -lpetsc -Wl,-rpath,/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -L/mnt/a/u/sciteam/mrosso/LIBS/petsc-3.4.2/interlagos-64idx-pgi-opt/lib -lsuperlu_dist_3.3 -L/opt/acml/5.3.0/pgi64/lib -lacml -lpthread -lparmetis -lmetis -ldl
-----------------------------------------
#PETSc Option Table entries:
-ksp_monitor_true_residual
-ksp_view
-log_summary
-mg_coarse_pc_factor_mat_solver_package superlu_dist
-mg_coarse_pc_type lu
-mg_levels_ksp_max_it 1
-mg_levels_ksp_type richardson
-options_left
-pc_mg_galerkin
-pc_mg_levels 5
-pc_type mg
#End of PETSc Option Table entries
There are no unused options.
More information about the petsc-users
mailing list