From filippo.leonardi at sam.math.ethz.ch Wed Oct 1 01:01:45 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Wed, 1 Oct 2014 08:01:45 +0200 Subject: [petsc-users] Scaling/Preconditioners for Poisson equation In-Reply-To: References: <2490546.DNVhllGaLT@besikovitch-ii> <2401575.mGBRNJJlEm@besikovitch-ii> Message-ID: <1712698.oSvEgN0riB@besikovitch-ii> I am actually having hard time figuring out where I am spending my time. Reading the report I am spending time on KSPSolve and PCApply (e+02). Since the number of those operations is well under control. I guess is some communication that is the bottleneck. The lines: VecScatterBegin 4097 1.0 2.5168e+01 3.2 0.00e+00 0.0 2.9e+09 3.7e+01 0.0e+00 3 0 87 39 0 10 0100100 0 0 VecScatterEnd 4097 1.0 1.7736e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 25 0 0 0 0 88 0 0 0 0 0 are probably what is slowing down the solution. Also, times do not add up properly, especially in KSPSolve. PS: until now I was using outputs in VTK. I guess it is better to output in PEtsc binary? Is it better to output from PETSC_COMM_SELF (i.e. each processor individually)? Using Petsc Release Version 3.3.0, Patch 3, Wed Aug 29 11:26:24 CDT 2012 Max Max/Min Avg Total Time (sec): 6.719e+02 1.00001 6.719e+02 Objects: 2.456e+03 1.00000 2.456e+03 Flops: 1.251e+10 1.00000 1.251e+10 5.123e+13 Flops/sec: 1.861e+07 1.00001 1.861e+07 7.624e+10 MPI Messages: 9.041e+05 1.09958 8.223e+05 3.368e+09 MPI Message Lengths: 4.299e+10 735.09182 8.386e+01 2.825e+11 MPI Reductions: 4.827e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.0203e+01 1.5% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 1: StepStage: 1.3436e-02 0.0% 1.4496e+10 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2: ConvStage: 3.3163e-01 0.0% 7.4088e+10 0.1% 2.949e+06 0.1% 7.173e+00 8.6% 0.000e+00 0.0% 3: ProjStage: 9.3354e-02 0.0% 3.7581e+09 0.0% 3.932e+05 0.0% 9.564e-01 1.1% 0.000e+00 0.0% 4: IoStage: 3.6459e+02 54.3% 0.0000e+00 0.0% 4.587e+05 0.0% 2.622e+01 31.3% 1.000e+02 2.1% 5: KspStage: 9.4361e+01 14.0% 4.5060e+13 88.0% 4.444e+08 13.2% 1.667e+01 19.9% 3.081e+03 63.8% 6: SolvAlloc: 2.2626e+00 0.3% 0.0000e+00 0.0% 3.604e+05 0.0% 1.217e-01 0.1% 3.330e+02 6.9% 7: SolvSolve: 1.0235e+01 1.5% 1.7448e+09 0.0% 9.830e+04 0.0% 2.391e-01 0.3% 1.200e+01 0.2% 8: SolvDeall: 1.5842e-02 0.0% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 9: MG Apply: 1.8982e+02 28.3% 6.0759e+12 11.9% 2.919e+09 86.7% 3.249e+01 38.7% 1.300e+03 26.9% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: StepStage VecAXPY 48 1.0 1.5407e-02 2.0 3.54e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 99100 0 0 0 940849 --- Event Stage 2: ConvStage VecCopy 72 1.0 2.6107e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 7 0 0 0 0 0 VecAXPY 72 1.0 2.2448e-02 1.4 4.72e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 5 26 0 0 0 860999 VecAXPBYCZ 84 1.0 4.1915e-02 1.3 1.10e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 11 61 0 0 0 1075909 VecPointwiseMult 72 1.0 2.5126e-02 1.2 2.36e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 7 13 0 0 0 384609 VecScatterBegin 120 1.0 6.1090e-02 1.9 0.00e+00 0.0 2.9e+06 8.2e+03 0.0e+00 0 0 0 9 0 12 0100100 0 0 VecScatterEnd 120 1.0 1.1047e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 29 0 0 0 0 0 --- Event Stage 3: ProjStage VecScale 4 1.0 2.4414e-04 2.2 1.31e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 14 0 0 0 2199023 VecCopy 12 1.0 5.6450e-03 6.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 5 0 0 0 0 0 VecAXPY 12 1.0 3.9425e-03 8.0 7.86e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 3 86 0 0 0 817054 VecScatterBegin 16 1.0 7.6387e-03 1.8 0.00e+00 0.0 3.9e+05 8.2e+03 0.0e+00 0 0 0 1 0 6 0100100 0 0 VecScatterEnd 16 1.0 2.1989e-02 3.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 13 0 0 0 0 0 --- Event Stage 4: IoStage VecView 20 1.0 2.6112e+0211197.9 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+01 21 0 0 0 1 39 0 0 0 40 0 VecCopy 20 1.0 4.1862e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 12 1.0 3.5555e-03 1.8 0.00e+00 0.0 2.9e+05 8.2e+03 0.0e+00 0 0 0 1 0 0 0 64 3 0 0 VecScatterEnd 12 1.0 7.5151e+0160547.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 6 0 0 0 0 0 --- Event Stage 5: KspStage VecMDot 21 1.0 4.4578e-02 2.7 3.67e+06 1.0 0.0e+00 0.0e+00 2.1e+01 0 0 0 0 0 0 0 0 0 1 337210 VecNorm 57 1.0 4.0502e-02 2.6 3.74e+06 1.0 0.0e+00 0.0e+00 5.7e+01 0 0 0 0 1 0 0 0 0 2 377777 VecScale 26 1.0 1.3385e-03 1.1 8.52e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2607178 VecCopy 31 1.0 7.8409e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 156 1.0 6.1290e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 26 1.0 3.2058e-03 1.7 1.70e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2177108 VecAYPX 26 1.0 4.8437e-03 1.6 8.52e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 720452 VecMAXPY 47 1.0 1.8381e-02 1.3 8.72e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1942289 VecScatterBegin 97 1.0 8.3990e-03 1.4 0.00e+00 0.0 1.9e+06 5.2e+03 0.0e+00 0 0 0 3 0 0 0 0 17 0 0 VecScatterEnd 97 1.0 5.4450e-02 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 26 1.0 1.4583e-02 2.5 2.56e+06 1.0 0.0e+00 0.0e+00 2.6e+01 0 0 0 0 1 0 0 0 0 1 717896 MatMult 47 1.0 1.6683e-01 1.3 2.16e+07 1.0 1.2e+06 8.2e+03 0.0e+00 0 0 0 3 0 0 0 0 17 0 529370 MatMultTranspose 25 1.0 7.7603e-03 1.2 1.26e+06 1.0 7.2e+05 2.6e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 667090 MatLUFactorSym 5 1.0 9.5924e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.5e+01 0 0 0 0 0 1 0 0 0 0 0 MatLUFactorNum 5 1.0 3.3532e+01 1.0 1.09e+10 1.0 0.0e+00 0.0e+00 0.0e+00 5 87 0 0 0 35 99 0 0 0 1333065 MatAssemblyBegin 130 1.0 7.9255e-01 4.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.5e+02 0 0 0 0 3 1 0 0 0 5 0 MatAssemblyEnd 130 1.0 1.9086e-01 1.3 0.00e+00 0.0 6.8e+06 4.5e+01 4.0e+02 0 0 0 0 8 0 0 2 1 13 0 MatGetRowIJ 5 1.0 6.3694e-03 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 5 1.0 4.2130e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 0 0 0 0 0 0 0 MatView 70 1.2 2.7097e-02 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+01 0 0 0 0 1 0 0 0 0 2 0 MatPtAP 25 1.0 3.6540e+00 1.0 4.55e+07 1.0 1.5e+07 8.7e+02 6.8e+02 1 0 0 5 14 4 0 3 23 22 51010 MatPtAPSymbolic 25 1.0 8.0998e-01 1.0 0.00e+00 0.0 1.4e+07 6.9e+02 6.2e+02 0 0 0 3 13 1 0 3 17 20 0 MatPtAPNumeric 25 1.0 2.8595e+00 1.0 4.55e+07 1.0 7.2e+05 4.5e+03 5.0e+01 0 0 0 1 1 3 0 0 6 2 65182 MatGetRedundant 5 1.0 6.1452e+00 1.1 0.00e+00 0.0 2.5e+08 1.2e+02 2.0e+01 1 0 7 10 0 6 0 57 52 1 0 MatGetLocalMat 25 1.0 6.3210e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+01 0 0 0 0 1 0 0 0 0 2 0 MatGetBrAoCol 25 1.0 1.1783e-01 1.9 0.00e+00 0.0 6.8e+06 1.2e+03 5.0e+01 0 0 0 3 1 0 0 2 14 2 0 MatGetSymTrans 50 1.0 9.1975e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 21 1.0 4.8957e-02 2.3 7.34e+06 1.0 0.0e+00 0.0e+00 2.1e+01 0 0 0 0 0 0 0 0 0 1 614106 KSPSetUp 40 1.0 1.0306e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+02 0 0 0 0 4 0 0 0 0 6 0 KSPSolve 5 1.0 2.8417e+02 1.0 1.25e+10 1.0 3.4e+09 4.9e+01 4.3e+03 42100100 59 90 301113757295140 179947 PCSetUp 5 1.0 9.4727e+01 1.0 1.10e+10 1.0 4.4e+08 1.1e+02 2.7e+03 14 88 13 17 56 100100100 83 87 473904 PCApply 26 1.0 1.9041e+02 1.0 1.48e+09 1.0 2.9e+09 3.7e+01 1.3e+03 28 12 87 39 27 201 13657195 42 31910 MGSetup Level 0 1 1.0 5.4520e+01 1.0 2.18e+09 1.0 8.4e+07 7.3e+01 2.7e+01 8 17 2 2 1 58 20 19 11 1 163977 MGSetup Level 1 1 1.0 1.5559e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 2 1 1.0 1.3490e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 3 1 1.0 1.4379e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 4 1 1.0 2.0809e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 5 1 1.0 2.0609e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 0 1 1.0 8.8773e+00 1.0 2.18e+09 1.0 8.4e+07 7.3e+01 2.7e+01 1 17 2 2 1 9 20 19 11 1 1007064 MGSetup Level 1 1 1.0 1.1740e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 2 1 1.0 1.1399e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 3 1 1.0 1.2438e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 4 1 1.0 1.1642e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 5 1 1.0 1.8711e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 0 1 1.0 8.8315e+00 1.0 2.18e+09 1.0 8.4e+07 7.3e+01 2.7e+01 1 17 2 2 1 9 20 19 11 1 1012286 MGSetup Level 1 1 1.0 2.6200e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 2 1 1.0 1.3020e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 3 1 1.0 1.1592e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 4 1 1.0 1.1511e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 5 1 1.0 1.7359e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 0 1 1.0 9.0600e+00 1.0 2.18e+09 1.0 8.4e+07 7.3e+01 2.7e+01 1 17 2 2 1 9 20 19 11 1 986750 MGSetup Level 1 1 1.0 1.3521e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 2 1 1.0 1.0810e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 3 1 1.0 1.3070e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 4 1 1.0 1.2069e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 5 1 1.0 1.9269e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 0 1 1.0 9.1177e+00 1.0 2.18e+09 1.0 8.4e+07 7.3e+01 2.7e+01 1 17 2 2 1 10 20 19 11 1 980513 MGSetup Level 1 1 1.0 1.4212e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 2 1 1.0 1.2362e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 3 1 1.0 1.2891e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 4 1 1.0 1.1790e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MGSetup Level 5 1 1.0 1.7860e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 --- Event Stage 6: SolvAlloc VecSet 50 1.0 1.4619e-01 7.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 4 0 0 0 0 0 MatAssemblyBegin 4 1.0 5.9544e-02 6.1 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 0 1 0 0 0 2 0 MatAssemblyEnd 4 1.0 3.3315e-02 1.1 0.00e+00 0.0 9.8e+04 2.0e+03 1.6e+01 0 0 0 0 0 1 0 27 49 5 0 --- Event Stage 7: SolvSolve VecNorm 12 1.0 2.0508e+013529.3 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 2 0 0 0 0 100 0 0 0100 0 VecScale 1 1.0 8.3923e-05 2.9 3.28e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 8 0 0 0 1599290 VecCopy 6 1.0 2.3437e-03 6.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 6 1.0 2.1067e-03 6.7 3.93e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 92 0 0 0 764531 VecScatterBegin 4 1.0 3.1297e-03 1.8 0.00e+00 0.0 9.8e+04 8.2e+03 0.0e+00 0 0 0 0 0 0 0100100 0 0 VecScatterEnd 4 1.0 5.5318e-03 5.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 --- Event Stage 8: SolvDeall --- Event Stage 9: MG Apply VecMDot 250 1.0 2.0555e-01 2.2 2.06e+07 1.0 0.0e+00 0.0e+00 2.5e+02 0 0 0 0 5 0 1 0 0 19 410404 VecNorm 275 1.0 4.2012e-02 1.6 4.12e+06 1.0 0.0e+00 0.0e+00 2.8e+02 0 0 0 0 6 0 0 0 0 21 401611 VecScale 1835 1.0 1.5452e-02 1.3 6.51e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1725703 VecCopy 285 1.0 2.6994e-03 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 1793 1.0 7.0033e-03 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 3170 1.0 3.3056e-02 1.3 1.86e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 2298583 VecAYPX 3250 1.0 5.5980e-02 1.2 1.21e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 885314 VecMAXPY 275 1.0 3.9382e-02 1.1 2.43e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 2 0 0 0 2531654 VecScatterBegin 4097 1.0 2.5168e+01 3.2 0.00e+00 0.0 2.9e+09 3.7e+01 0.0e+00 3 0 87 39 0 10 0100100 0 0 VecScatterEnd 4097 1.0 1.7736e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 25 0 0 0 0 88 0 0 0 0 0 VecNormalize 275 1.0 4.6273e-02 1.5 6.18e+06 1.0 0.0e+00 0.0e+00 2.8e+02 0 0 0 0 6 0 0 0 0 21 546952 MatMult 2875 1.0 1.8335e+01 3.4 2.20e+08 1.0 2.8e+08 3.1e+02 0.0e+00 1 2 8 30 0 5 15 9 78 0 49193 MatMultAdd 390 1.0 9.9083e+0018.4 7.51e+06 1.0 1.1e+07 1.2e+02 0.0e+00 1 0 0 0 0 2 1 0 1 0 3105 MatMultTranspose 520 1.0 1.5965e+0010.4 1.41e+07 1.0 1.5e+07 1.5e+02 0.0e+00 0 0 0 1 0 0 1 1 2 0 36130 MatSolve 156 1.0 2.1702e+00 1.6 9.77e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 8 0 0 0 1 66 0 0 0 1844070 MatSOR 2615 1.0 1.1141e+00 1.1 1.78e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 1 12 0 0 0 655558 MatView 25 1.0 5.5473e-03 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.5e+01 0 0 0 0 1 0 0 0 0 2 0 KSPGMRESOrthog 250 1.0 2.3644e-01 1.9 4.12e+07 1.0 0.0e+00 0.0e+00 2.5e+02 0 0 0 0 5 0 3 0 0 19 713578 KSPSetUp 25 1.0 9.4339e-01 4.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.5e+02 0 0 0 0 5 0 0 0 0 19 0 KSPSolve 936 1.0 1.8826e+02 1.1 1.43e+09 1.0 2.9e+09 3.3e+01 1.3e+03 27 11 85 34 27 97 96 98 87100 31071 PCApply 2771 1.0 1.8160e+02 1.1 1.16e+09 1.0 2.6e+09 8.0e+00 0.0e+00 26 9 78 7 0 92 78 90 19 0 26058 MGSmooth Level 0 24 1.0 2.5841e+01 1.1 1.50e+08 1.0 4.0e+08 8.0e+00 0.0e+00 4 1 12 1 0 13 10 14 3 0 23826 MGSmooth Level 1 40 1.0 2.3332e+0016.6 7.33e+04 1.0 1.2e+07 1.7e+01 5.2e+01 0 0 0 0 1 0 0 0 0 4 129 MGResid Level 1 20 1.0 8.2547e-0217.7 8.80e+03 1.0 2.1e+06 1.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 437 MGInterp Level 1 48 1.0 2.4154e+00221.0 2.59e+03 1.0 1.4e+06 8.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4 MGSmooth Level 2 32 1.0 6.3778e-0112.5 5.90e+05 1.0 1.0e+07 4.7e+01 5.2e+01 0 0 0 0 1 0 0 0 0 4 3789 MGResid Level 2 16 1.0 5.0234e-0213.1 5.63e+04 1.0 1.7e+06 4.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 4592 MGInterp Level 2 40 1.0 4.2881e-01131.6 1.73e+04 1.0 1.1e+06 2.2e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 165 MGSmooth Level 3 24 1.0 5.3460e-02 1.2 4.16e+06 1.0 8.0e+06 1.5e+02 5.2e+01 0 0 0 0 1 0 0 0 1 4 319012 MGResid Level 3 12 1.0 6.1336e-03 2.1 3.38e+05 1.0 1.3e+06 1.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 225664 MGInterp Level 3 32 1.0 2.5873e-0180.0 1.11e+05 1.0 9.2e+05 7.0e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 1751 MGSmooth Level 4 16 1.0 1.0150e-01 1.1 2.58e+07 1.0 5.9e+06 5.3e+02 5.2e+01 0 0 0 1 1 0 2 0 3 4 1039338 MGResid Level 4 8 1.0 5.6934e-03 1.4 1.80e+06 1.0 8.5e+05 5.3e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 1296577 MGInterp Level 4 24 1.0 2.4087e-0152.2 6.64e+05 1.0 6.9e+05 2.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 11283 MGSmooth Level 5 8 1.0 3.3459e-01 1.0 4.47e+07 1.0 8.6e+05 8.2e+03 5.2e+01 0 0 0 2 1 0 3 0 6 4 547503 MGResid Level 5 4 1.0 1.2396e-02 1.3 1.97e+06 1.0 9.8e+04 8.2e+03 0.0e+00 0 0 0 0 0 0 0 0 1 0 649658 MGInterp Level 5 16 1.0 3.7598e-0119.2 3.54e+06 1.0 4.6e+05 9.3e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 38554 MGSmooth Level 0 30 1.0 3.2777e+01 1.1 1.88e+08 1.0 5.0e+08 8.0e+00 0.0e+00 5 2 15 1 0 16 13 17 4 0 23480 MGSmooth Level 1 50 1.0 3.3483e+0019.5 8.95e+04 1.0 1.5e+07 1.7e+01 5.2e+01 0 0 0 0 1 1 0 1 0 4 110 MGResid Level 1 25 1.0 1.6569e-0130.9 1.10e+04 1.0 2.7e+06 1.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 272 MGInterp Level 1 60 1.0 3.3231e+00680.2 3.24e+03 1.0 1.7e+06 8.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4 MGSmooth Level 2 40 1.0 7.9050e-0114.3 7.18e+05 1.0 1.2e+07 4.7e+01 5.2e+01 0 0 0 0 1 0 0 0 1 4 3721 MGResid Level 2 20 1.0 3.5753e-02 7.5 7.04e+04 1.0 2.1e+06 4.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 8065 MGInterp Level 2 50 1.0 5.5985e-01146.2 2.16e+04 1.0 1.4e+06 2.2e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 158 MGSmooth Level 3 30 1.0 6.2155e-02 1.3 5.03e+06 1.0 9.7e+06 1.5e+02 5.2e+01 0 0 0 1 1 0 0 0 1 4 331765 MGResid Level 3 15 1.0 1.1305e-02 3.7 4.22e+05 1.0 1.6e+06 1.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 153048 MGInterp Level 3 40 1.0 3.8209e-01101.7 1.38e+05 1.0 1.1e+06 7.0e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 1482 MGSmooth Level 4 20 1.0 1.1585e-01 1.1 3.08e+07 1.0 7.0e+06 5.3e+02 5.2e+01 0 0 0 1 1 0 2 0 3 4 1087811 MGResid Level 4 10 1.0 6.8018e-03 1.3 2.25e+06 1.0 1.1e+06 5.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 1356613 MGInterp Level 4 30 1.0 1.5521e-0128.1 8.29e+05 1.0 8.6e+05 2.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 21889 MGSmooth Level 5 10 1.0 3.2689e-01 1.0 5.11e+07 1.0 1.0e+06 8.2e+03 5.2e+01 0 0 0 3 1 0 3 0 8 4 640367 MGResid Level 5 5 1.0 1.6725e-02 1.4 2.46e+06 1.0 1.2e+05 8.2e+03 0.0e+00 0 0 0 0 0 0 0 0 1 0 601888 MGInterp Level 5 20 1.0 2.4798e-0110.0 4.42e+06 1.0 5.7e+05 9.3e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 73069 MGSmooth Level 0 30 1.0 3.5707e+01 1.1 1.88e+08 1.0 5.0e+08 8.0e+00 0.0e+00 5 2 15 1 0 18 13 17 4 0 21554 MGSmooth Level 1 50 1.0 3.3039e+0011.8 8.95e+04 1.0 1.5e+07 1.7e+01 5.2e+01 0 0 0 0 1 1 0 1 0 4 111 MGResid Level 1 25 1.0 1.3634e-0121.7 1.10e+04 1.0 2.7e+06 1.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 330 MGInterp Level 1 60 1.0 2.7394e+00527.8 3.24e+03 1.0 1.7e+06 8.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 5 MGSmooth Level 2 40 1.0 9.1679e-0111.0 7.18e+05 1.0 1.2e+07 4.7e+01 5.2e+01 0 0 0 0 1 0 0 0 1 4 3208 MGResid Level 2 20 1.0 7.1785e-0214.7 7.04e+04 1.0 2.1e+06 4.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 4017 MGInterp Level 2 50 1.0 5.1115e-01147.6 2.16e+04 1.0 1.4e+06 2.2e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 173 MGSmooth Level 3 30 1.0 6.1442e-02 1.2 5.03e+06 1.0 9.7e+06 1.5e+02 5.2e+01 0 0 0 1 1 0 0 0 1 4 335617 MGResid Level 3 15 1.0 7.9360e-03 2.5 4.22e+05 1.0 1.6e+06 1.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 218013 MGInterp Level 3 40 1.0 2.6162e-0167.2 1.38e+05 1.0 1.1e+06 7.0e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 2164 MGSmooth Level 4 20 1.0 1.1088e-01 1.1 3.08e+07 1.0 7.0e+06 5.3e+02 5.2e+01 0 0 0 1 1 0 2 0 3 4 1136519 MGResid Level 4 10 1.0 6.8250e-03 1.3 2.25e+06 1.0 1.1e+06 5.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 1352016 MGInterp Level 4 30 1.0 2.2836e-0138.8 8.29e+05 1.0 8.6e+05 2.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 14878 MGSmooth Level 5 10 1.0 3.2905e-01 1.0 5.11e+07 1.0 1.0e+06 8.2e+03 5.2e+01 0 0 0 3 1 0 3 0 8 4 636160 MGResid Level 5 5 1.0 1.5521e-02 1.3 2.46e+06 1.0 1.2e+05 8.2e+03 0.0e+00 0 0 0 0 0 0 0 0 1 0 648570 MGInterp Level 5 20 1.0 2.9873e-0112.1 4.42e+06 1.0 5.7e+05 9.3e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 60654 MGSmooth Level 0 36 1.0 4.3941e+01 1.1 2.25e+08 1.0 6.0e+08 8.0e+00 0.0e+00 6 2 18 2 0 22 15 21 4 0 21018 MGSmooth Level 1 60 1.0 3.7174e+0013.1 1.06e+05 1.0 1.8e+07 1.7e+01 5.2e+01 0 0 1 0 1 1 0 1 0 4 117 MGResid Level 1 30 1.0 1.6295e-0120.9 1.32e+04 1.0 3.2e+06 1.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 332 MGInterp Level 1 72 1.0 3.5079e+00490.6 3.89e+03 1.0 2.1e+06 8.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 5 MGSmooth Level 2 48 1.0 1.2006e+0017.2 8.46e+05 1.0 1.5e+07 4.7e+01 5.2e+01 0 0 0 0 1 0 0 0 1 4 2887 MGResid Level 2 24 1.0 6.0709e-02 9.8 8.45e+04 1.0 2.6e+06 4.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 5700 MGInterp Level 2 60 1.0 8.0356e-01207.5 2.59e+04 1.0 1.7e+06 2.2e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 132 MGSmooth Level 3 36 1.0 7.0747e-02 1.2 5.91e+06 1.0 1.1e+07 1.5e+02 5.2e+01 0 0 0 1 1 0 0 0 2 4 341890 MGResid Level 3 18 1.0 9.8090e-03 2.2 5.07e+05 1.0 1.9e+06 1.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 211660 MGInterp Level 3 48 1.0 2.5153e-0147.9 1.66e+05 1.0 1.4e+06 7.0e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 2701 MGSmooth Level 4 24 1.0 1.2681e-01 1.1 3.58e+07 1.0 8.2e+06 5.3e+02 5.2e+01 0 0 0 2 1 0 2 0 4 4 1155582 MGResid Level 4 12 1.0 7.9920e-03 1.2 2.70e+06 1.0 1.3e+06 5.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 1385501 MGInterp Level 4 36 1.0 1.6041e-0122.1 9.95e+05 1.0 1.0e+06 2.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 25416 MGSmooth Level 5 12 1.0 3.6647e-01 1.0 5.75e+07 1.0 1.2e+06 8.2e+03 5.2e+01 0 0 0 3 1 0 4 0 9 4 642901 MGResid Level 5 6 1.0 1.9434e-02 1.4 2.95e+06 1.0 1.5e+05 8.2e+03 0.0e+00 0 0 0 0 0 0 0 0 1 0 621563 MGInterp Level 5 24 1.0 2.6055e-01 8.5 5.31e+06 1.0 6.9e+05 9.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 83451 MGSmooth Level 0 36 1.0 4.5240e+01 1.2 2.25e+08 1.0 6.0e+08 8.0e+00 0.0e+00 6 2 18 2 0 23 15 21 4 0 20414 MGSmooth Level 1 60 1.0 4.3619e+0020.0 1.06e+05 1.0 1.8e+07 1.7e+01 5.2e+01 0 0 1 0 1 1 0 1 0 4 99 MGResid Level 1 30 1.0 1.4730e-0120.5 1.32e+04 1.0 3.2e+06 1.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 367 MGInterp Level 1 72 1.0 3.5583e+00610.7 3.89e+03 1.0 2.1e+06 8.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 4 MGSmooth Level 2 48 1.0 8.7782e-0110.1 8.46e+05 1.0 1.5e+07 4.7e+01 5.2e+01 0 0 0 0 1 0 0 0 1 4 3948 MGResid Level 2 24 1.0 4.1538e-02 6.8 8.45e+04 1.0 2.6e+06 4.7e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 8330 MGInterp Level 2 60 1.0 4.0416e-0178.0 2.59e+04 1.0 1.7e+06 2.2e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 263 MGSmooth Level 3 36 1.0 7.2884e-02 1.3 5.91e+06 1.0 1.1e+07 1.5e+02 5.2e+01 0 0 0 1 1 0 0 0 2 4 331863 MGResid Level 3 18 1.0 1.1254e-02 2.6 5.07e+05 1.0 1.9e+06 1.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 184487 MGInterp Level 3 48 1.0 2.9326e-0165.5 1.66e+05 1.0 1.4e+06 7.0e+01 0.0e+00 0 0 0 0 0 0 0 0 0 0 2317 MGSmooth Level 4 24 1.0 1.2857e-01 1.1 3.58e+07 1.0 8.2e+06 5.3e+02 5.2e+01 0 0 0 2 1 0 2 0 4 4 1139850 MGResid Level 4 12 1.0 1.0152e-02 1.6 2.70e+06 1.0 1.3e+06 5.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 1090681 MGInterp Level 4 36 1.0 1.6758e-0124.0 9.95e+05 1.0 1.0e+06 2.5e+02 0.0e+00 0 0 0 0 0 0 0 0 0 0 24328 MGSmooth Level 5 12 1.0 3.6468e-01 1.0 5.75e+07 1.0 1.2e+06 8.2e+03 5.2e+01 0 0 0 3 1 0 4 0 9 4 646050 MGResid Level 5 6 1.0 1.8821e-02 1.4 2.95e+06 1.0 1.5e+05 8.2e+03 0.0e+00 0 0 0 0 0 0 0 0 1 0 641815 MGInterp Level 5 24 1.0 2.4655e-01 8.2 5.31e+06 1.0 6.9e+05 9.3e+02 0.0e+00 0 0 0 0 0 0 0 0 1 0 88189 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 0 0 0 --- Event Stage 1: StepStage --- Event Stage 2: ConvStage --- Event Stage 3: ProjStage Container 0 24 13152 0 Vector 0 616 41290592 0 Vector Scatter 0 128 132608 0 Matrix 0 168 175179360 0 Distributed Mesh 0 40 886464 0 Bipartite Graph 0 80 54720 0 Index Set 0 12 140000 0 IS L to G Mapping 0 60 490736 0 Krylov Solver 28 52 708640 0 Preconditioner 28 52 44640 0 --- Event Stage 4: IoStage Vector 20 20 5272800 0 Viewer 20 20 13760 0 --- Event Stage 5: KspStage Container 30 0 0 0 Vector 629 232 15473320 0 Vector Scatter 185 25 25900 0 Matrix 210 0 0 0 Distributed Mesh 50 0 0 0 Bipartite Graph 100 0 0 0 Index Set 435 420 824140 0 IS L to G Mapping 75 0 0 0 Krylov Solver 30 0 0 0 Preconditioner 30 0 0 0 Viewer 5 5 3560 0 --- Event Stage 6: SolvAlloc Vector 140 6 8848 0 Vector Scatter 6 0 0 0 Matrix 6 0 0 0 Matrix Null Space 1 0 0 0 Distributed Mesh 2 0 0 0 Bipartite Graph 4 0 0 0 Index Set 14 14 372400 0 IS L to G Mapping 3 0 0 0 Krylov Solver 7 0 0 0 Preconditioner 7 0 0 0 Viewer 1 0 0 0 --- Event Stage 7: SolvSolve Matrix Null Space 0 1 580 0 Krylov Solver 7 7 8192 0 Preconditioner 7 7 6416 0 --- Event Stage 8: SolvDeall Container 0 6 3288 0 Vector 0 282 46205592 0 Vector Scatter 0 33 34188 0 Matrix 0 45 48414516 0 Distributed Mesh 0 10 221616 0 Bipartite Graph 0 20 13680 0 Index Set 0 3 35000 0 IS L to G Mapping 0 15 122684 0 Krylov Solver 0 13 177160 0 Preconditioner 0 13 11160 0 --- Event Stage 9: MG Apply Vector 375 0 0 0 ======================================================================================================================== Average time to get PetscTime(): 1.90735e-07 Average time for MPI_Barrier(): 4.88281e-05 Average time for zero size MPI_Send(): 1.68436e-06 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor -ksp_monitor_true_residual -ksp_view -log_summary -options_left -pc_mg_galerkin -pc_mg_levels 6 -pc_mg_type full -pc_type mg On Tuesday 30 September 2014 02:13:55 Barry Smith wrote: > http://lmgtfy.com/?q=HPGMG ... > > On Sep 30, 2014, at 1:59 AM, Filippo Leonardi wrote: > > Thank you everybody, number of iterations seems now to be under control, > > I'll run some scaling test and hope for the best. > > > > Now I am intrigued, is there, by any chance, any reference I can look up > > for this? -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From chris at bpi.cam.ac.uk Wed Oct 1 03:51:53 2014 From: chris at bpi.cam.ac.uk (Chris Richardson) Date: Wed, 01 Oct 2014 09:51:53 +0100 Subject: [petsc-users] =?utf-8?q?PETSC_ERROR=3A_Can_only_handle_MPIU=5FREA?= =?utf-8?q?L_or_MPIU=5FCOMPLEX_with_ML_preconditioner?= Message-ID: I have just run into this error after pulling the master branch of petsc, when using the ML preconditioner in FEniCS. It looks like a type error, but I guess FEniCS uses PETScInt, so should be safe? PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types Previously, it worked, so I think it might be due to a recent change? Here is my configure script for petsc... I don't think I am doing anything weird. ./configure \ --with-c-support --with-clanguage=C++ --with-debugging=0 --with-shared-libraries=1 \ --with-mpi-dir=$MPICH_DIR \ --with-blas-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] \ --with-lapack-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] \ --with-scalapack-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] --with-scalapack-include=$CRAY_LIBSCI_PREFIX_DIR/include --with-scalapack=1\ --with-blacs-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] --with-blacs-include=$CRAY_LIBSCI_PREFIX_DIR/include --with-blacs=1\ --with-fortran-interfaces=1 \ --with-ptscotch=1 --download-ptscotch=1 \ --with-mumps=1 --download-mumps=1 \ --with-hypre=1 --download-hypre=1 \ --with-superlu_dist=1 --download-superlu_dist=1 \ --with-suitesparse=1 --download-suitesparse=1 \ --with-metis=1 --download-metis=1 \ --with-ml=1 --download-ml=1 \ --with-parmetis=1 --download-parmetis=1 From knepley at gmail.com Wed Oct 1 05:33:17 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 1 Oct 2014 05:33:17 -0500 Subject: [petsc-users] PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX with ML preconditioner In-Reply-To: References: Message-ID: On Wed, Oct 1, 2014 at 3:51 AM, Chris Richardson wrote: > > I have just run into this error after pulling the master branch of petsc, > when using the ML preconditioner in FEniCS. > It looks like a type error, but I guess FEniCS uses PETScInt, so should be > safe? > > > PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types > We need to see the entire error (with the stack). Thanks, Matt > Previously, it worked, so I think it might be due to a recent change? > Here is my configure script for petsc... I don't think I am doing anything > weird. > > ./configure \ > --with-c-support --with-clanguage=C++ --with-debugging=0 > --with-shared-libraries=1 \ > --with-mpi-dir=$MPICH_DIR \ > --with-blas-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] \ > --with-lapack-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] \ > --with-scalapack-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] > --with-scalapack-include=$CRAY_LIBSCI_PREFIX_DIR/include > --with-scalapack=1\ > --with-blacs-lib=[$CRAY_LIBSCI_PREFIX_DIR/lib/libsci_gnu.so] > --with-blacs-include=$CRAY_LIBSCI_PREFIX_DIR/include --with-blacs=1\ > --with-fortran-interfaces=1 \ > --with-ptscotch=1 --download-ptscotch=1 \ > --with-mumps=1 --download-mumps=1 \ > --with-hypre=1 --download-hypre=1 \ > --with-superlu_dist=1 --download-superlu_dist=1 \ > --with-suitesparse=1 --download-suitesparse=1 \ > --with-metis=1 --download-metis=1 \ > --with-ml=1 --download-ml=1 \ > --with-parmetis=1 --download-parmetis=1 > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bpi.cam.ac.uk Wed Oct 1 07:03:11 2014 From: chris at bpi.cam.ac.uk (Chris Richardson) Date: Wed, 01 Oct 2014 13:03:11 +0100 Subject: [petsc-users] =?utf-8?q?PETSC_ERROR=3A_Can_only_handle_MPIU=5FREA?= =?utf-8?q?L_or_MPIU=5FCOMPLEX_with_ML_preconditioner?= In-Reply-To: References: Message-ID: On 01/10/2014 11:33, Matthew Knepley wrote: > On Wed, Oct 1, 2014 at 3:51 AM, Chris Richardson > wrote: > >> I have just run into this error after pulling the master branch of >> petsc, when using the ML preconditioner in FEniCS. >> It looks like a type error, but I guess FEniCS uses PETScInt, so >> should be safe? >> >> PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types > > We need to see the entire error (with the stack). > OK, this kind of thing:- [1]PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types ^C Program received signal SIGINT, Interrupt. 0x00007ffff64d9653 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 81 ../sysdeps/unix/syscall-template.S: No such file or directory. (gdb) up #1 0x00007ffff72c8591 in ?? () from /usr/lib/libmpi.so.1 (gdb) up #2 0x00007ffff72ca787 in opal_event_base_loop () from /usr/lib/libmpi.so.1 (gdb) up #3 0x00007ffff72eda3e in opal_progress () from /usr/lib/libmpi.so.1 (gdb) up #4 0x00007fffb42884d5 in ?? () from /usr/lib/openmpi/lib/openmpi/mca_grpcomm_bad.so (gdb) up #5 0x00007ffff723d02a in ompi_mpi_finalize () from /usr/lib/libmpi.so.1 (gdb) up #6 0x00007ffff7766b12 in dolfin::SubSystemsManager::finalize_mpi () at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:249 249 MPI_Finalize(); (gdb) up #7 0x00007ffff7766cde in dolfin::SubSystemsManager::finalize () at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:219 219 finalize_mpi(); (gdb) up #8 0x00007ffff641a149 in __run_exit_handlers (status=1, listp=0x7ffff679d6c8 <__exit_funcs>, run_list_atexit=run_list_atexit at entry=true) at exit.c:82 82 exit.c: No such file or directory. (gdb) up #9 0x00007ffff641a195 in __GI_exit (status=) at exit.c:104 104 in exit.c (gdb) up #10 0x00007ffff729cc00 in orte_ess_base_app_abort () from /usr/lib/libmpi.so.1 (gdb) up #11 0x00007ffff729c2a9 in orte_errmgr_base_error_abort () from /usr/lib/libmpi.so.1 (gdb) up #12 0x00007ffff723bb69 in ompi_mpi_abort () from /usr/lib/libmpi.so.1 (gdb) up #13 0x00007ffff37b9ee3 in PetscSum_Local (in=0x230f4520, out=0x7fffffffc650, cnt=, datatype=0x7fffffffc428) at /opt/packages/src/petsc/src/sys/objects/pinit.c:302 302 MPI_Abort(MPI_COMM_WORLD,1); (gdb) up #14 0x00007fffb10dd4dc in mca_coll_basic_scan_intra () from /usr/lib/openmpi/lib/openmpi/mca_coll_basic.so (gdb) up #15 0x00007fffb08c7e19 in mca_coll_sync_scan () from /usr/lib/openmpi/lib/openmpi/mca_coll_sync.so (gdb) up #16 0x00007ffff725835d in PMPI_Scan () from /usr/lib/libmpi.so.1 (gdb) up #17 0x00007ffff413ec45 in MatWrapML_MPIAIJ (newmat=0x16a39228, reuse=MAT_INITIAL_MATRIX, mlmat=0x7ed6090) at /opt/packages/src/petsc/src/ksp/pc/impls/ml/ml.c:422 422 ierr = MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); (gdb) list 417 ierr = MatSetSizes(A,m,n,PETSC_DECIDE,PETSC_DECIDE);CHKERRQ(ierr); 418 ierr = MatSetType(A,MATMPIAIJ);CHKERRQ(ierr); 419 /* keep track of block size for A matrices */ 420 ierr = MatSetBlockSize (A,mlmat->num_PDEs);CHKERRQ(ierr); 421 ierr = PetscMalloc3(m,&nnzA,m,&nnzB,m,&nnz);CHKERRQ(ierr); 422 ierr = MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); 423 rstart -= m; 424 425 for (i=0; i References: <2490546.DNVhllGaLT@besikovitch-ii> <2401575.mGBRNJJlEm@besikovitch-ii> <1712698.oSvEgN0riB@besikovitch-ii> Message-ID: <871tqs0yjc.fsf@jedbrown.org> Filippo Leonardi writes: > I am actually having hard time figuring out where I am spending my time. > > Reading the report I am spending time on KSPSolve and PCApply (e+02). Since > the number of those operations is well under control. I guess is some > communication that is the bottleneck. > > The lines: > VecScatterBegin 4097 1.0 2.5168e+01 3.2 0.00e+00 0.0 2.9e+09 3.7e+01 > 0.0e+00 3 0 87 39 0 10 0100100 0 0 > VecScatterEnd 4097 1.0 1.7736e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 25 0 0 0 0 88 0 0 0 0 0 > are probably what is slowing down the solution. How large is your problem? > Also, times do not add up properly, especially in KSPSolve. Timings are inclusive, not exclusive. I don't know what is going on with the having the same event appear many times within the stage, but I remember fixing an issue that might have caused that years ago and I'd appreciate it if you would upgrade to the current version of PETSc. How large is your problem size and how many processors are you running on? > PS: until now I was using outputs in VTK. I guess it is better to output in > PEtsc binary? Yes. You can use the binary-appended VTK viewer (please upgrade to current version of PETSc) or the PETSc binary viewer. > Is it better to output from PETSC_COMM_SELF (i.e. each processor > individually)? No, use collective IO. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From filippo.leonardi at sam.math.ethz.ch Wed Oct 1 09:28:00 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Wed, 1 Oct 2014 16:28:00 +0200 Subject: [petsc-users] Scaling/Preconditioners for Poisson equation In-Reply-To: <871tqs0yjc.fsf@jedbrown.org> References: <2490546.DNVhllGaLT@besikovitch-ii> <1712698.oSvEgN0riB@besikovitch-ii> <871tqs0yjc.fsf@jedbrown.org> Message-ID: <6279409.IsbNUDEfFT@besikovitch-ii> On Wednesday 01 October 2014 07:23:03 Jed Brown wrote: > Filippo Leonardi writes: > > I am actually having hard time figuring out where I am spending my time. > > > > Reading the report I am spending time on KSPSolve and PCApply (e+02). > > Since > > the number of those operations is well under control. I guess is some > > communication that is the bottleneck. > > > > The lines: > > VecScatterBegin 4097 1.0 2.5168e+01 3.2 0.00e+00 0.0 2.9e+09 3.7e+01 > > 0.0e+00 3 0 87 39 0 10 0100100 0 0 > > VecScatterEnd 4097 1.0 1.7736e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 > > 0.0e+00 25 0 0 0 0 88 0 0 0 0 0 > > are probably what is slowing down the solution. > > How large is your problem? 512^3 cells with 1 dof on 4096 processors > > > Also, times do not add up properly, especially in KSPSolve. > > Timings are inclusive, not exclusive. > > I don't know what is going on with the having the same event appear many > times within the stage, but I remember fixing an issue that might have > caused that years ago and I'd appreciate it if you would upgrade to the > current version of PETSc. I'll try if I can get some new version on the cluster I'm running. On my laptop, using petsc 3.5, I get something like: --- Event Stage 5: KspStage VecMDot 1953 1.0 3.0402e-01 1.3 3.91e+07 1.0 0.0e+00 0.0e+00 2.0e+03 1 1 0 0 1 2 2 0 0 2 514 VecNorm 5445 1.0 3.6542e-01 1.1 4.46e+07 1.0 0.0e+00 0.0e+00 5.4e+03 1 1 0 0 3 3 2 0 0 4 488 VecScale 2465 1.0 1.8466e-02 1.1 1.01e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2187 VecCopy 2979 1.0 3.5844e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 9134 1.0 2.7974e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 2465 1.0 3.2354e-02 1.9 2.02e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 2497 VecAYPX 2466 1.0 2.7912e-02 1.5 1.01e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1448 VecMAXPY 4418 1.0 4.7874e-02 1.1 9.42e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 4 0 0 0 7871 VecScatterBegin 6471 1.0 3.4048e-02 1.2 0.00e+00 0.0 4.8e+04 7.9e+02 0.0e+00 0 0 4 5 0 0 0 20 25 0 0 VecScatterEnd 6471 1.0 1.3429e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 VecNormalize 2466 1.0 2.5448e-01 1.0 3.03e+07 1.0 0.0e+00 0.0e+00 2.5e+03 1 0 0 0 1 2 1 0 0 2 476 MatMult 4419 1.0 3.9537e-01 1.2 1.63e+08 1.0 3.5e+04 1.0e+03 0.0e+00 1 2 3 5 0 3 7 15 24 0 1648 MatMultTranspose 1026 1.0 4.6899e-02 1.5 1.18e+07 1.0 1.2e+04 1.3e+02 0.0e+00 0 0 1 0 0 0 1 5 1 0 1008 MatLUFactorSym 513 1.0 7.8421e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 6 0 0 0 0 0 MatLUFactorNum 513 1.0 2.2127e+00 1.0 1.81e+09 1.0 0.0e+00 0.0e+00 0.0e+00 7 23 0 0 0 16 78 0 0 0 3269 MatConvert 513 1.0 4.4040e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 4617 1.0 4.1133e-01 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 4.1e+03 1 0 0 0 2 3 0 0 0 3 0 MatAssemblyEnd 4617 1.0 7.7171e-01 1.0 0.00e+00 0.0 4.9e+04 5.2e+01 1.6e+04 3 0 4 0 9 6 0 21 2 13 0 MatGetRowIJ 513 1.0 8.5605e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetSubMatrice 513 1.0 3.5439e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+03 1 0 0 0 1 3 0 0 0 2 0 MatGetOrdering 513 1.0 4.3043e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 3 0 0 0 0 0 MatPtAP 1026 1.0 3.1987e+00 1.0 1.29e+08 1.0 1.0e+05 4.1e+02 1.7e+04 11 2 8 5 10 24 6 44 28 14 162 MatPtAPSymbolic 1026 1.0 1.9393e+00 1.0 0.00e+00 0.0 6.8e+04 4.6e+02 7.2e+03 7 0 5 4 4 15 0 28 20 6 0 MatPtAPNumeric 1026 1.0 1.2644e+00 1.0 1.29e+08 1.0 3.7e+04 3.1e+02 1.0e+04 4 2 3 1 6 10 6 15 7 8 409 MatGetRedundant 513 1.0 4.0419e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+03 1 0 0 0 1 3 0 0 0 2 0 MatMPIConcateSeq 513 1.0 4.4424e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetLocalMat 1026 1.0 1.5555e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 MatGetBrAoCol 1026 1.0 1.7637e-01 1.3 0.00e+00 0.0 3.1e+04 8.0e+02 0.0e+00 1 0 2 3 0 1 0 13 16 0 0 MatGetSymTrans 2052 1.0 4.6286e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 1953 1.0 3.2750e-01 1.2 7.82e+07 1.0 0.0e+00 0.0e+00 2.0e+03 1 1 0 0 1 2 3 0 0 2 955 KSPSetUp 2565 1.0 8.8727e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.1e+04 3 0 0 0 6 7 0 0 0 9 0 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer KSPSolve 513 1.0 2.7668e+01 1.0 7.65e+09 1.0 1.3e+06 5.7e+02 1.8e+05 94 98 95 90 99 209328528467143 1106 PCSetUp 513 1.0 1.0804e+01 1.0 1.95e+09 1.0 2.0e+05 5.8e+02 9.0e+04 36 25 15 15 51 81 84 85 76 73 722 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer PCApply 2466 1.0 1.4483e+01 1.0 5.32e+09 1.0 1.0e+06 5.5e+02 5.2e+04 49 68 77 71 30 109228428367 43 1469 MGSetup Level 0 513 1.0 4.3083e+00 1.0 1.81e+09 1.0 2.1e+04 1.3e+03 1.0e+04 14 23 2 3 6 32 78 9 18 8 1679 MGSetup Level 1 513 1.0 3.4038e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 3.1e+03 1 0 0 0 2 2 0 0 0 3 0 MGSetup Level 2 513 1.0 5.1557e-01 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 3.1e+03 2 0 0 0 2 4 0 0 0 3 0 > > How large is your problem size and how many processors are you running on? > > > PS: until now I was using outputs in VTK. I guess it is better to output > > in > > PEtsc binary? > > Yes. You can use the binary-appended VTK viewer (please upgrade to > current version of PETSc) or the PETSc binary viewer. > > > Is it better to output from PETSC_COMM_SELF (i.e. each processor > > individually)? > > No, use collective IO. Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Oct 1 09:32:36 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 09:32:36 -0500 Subject: [petsc-users] PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX with ML preconditioner In-Reply-To: References: Message-ID: Recommend running under valgrind. http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind On Oct 1, 2014, at 7:03 AM, Chris Richardson wrote: > On 01/10/2014 11:33, Matthew Knepley wrote: >> On Wed, Oct 1, 2014 at 3:51 AM, Chris Richardson >> wrote: >>> I have just run into this error after pulling the master branch of >>> petsc, when using the ML preconditioner in FEniCS. >>> It looks like a type error, but I guess FEniCS uses PETScInt, so >>> should be safe? >>> PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types >> We need to see the entire error (with the stack). > > OK, this kind of thing:- > > > [1]PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types > ^C > Program received signal SIGINT, Interrupt. > 0x00007ffff64d9653 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81 > 81 ../sysdeps/unix/syscall-template.S: No such file or directory. > (gdb) up > #1 0x00007ffff72c8591 in ?? () from /usr/lib/libmpi.so.1 > (gdb) up > #2 0x00007ffff72ca787 in opal_event_base_loop () from /usr/lib/libmpi.so.1 > (gdb) up > #3 0x00007ffff72eda3e in opal_progress () from /usr/lib/libmpi.so.1 > (gdb) up > #4 0x00007fffb42884d5 in ?? () > from /usr/lib/openmpi/lib/openmpi/mca_grpcomm_bad.so > (gdb) up > #5 0x00007ffff723d02a in ompi_mpi_finalize () from /usr/lib/libmpi.so.1 > (gdb) up > #6 0x00007ffff7766b12 in dolfin::SubSystemsManager::finalize_mpi () > at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:249 > 249 MPI_Finalize(); > (gdb) up > #7 0x00007ffff7766cde in dolfin::SubSystemsManager::finalize () > at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:219 > 219 finalize_mpi(); > (gdb) up > #8 0x00007ffff641a149 in __run_exit_handlers (status=1, > listp=0x7ffff679d6c8 <__exit_funcs>, > run_list_atexit=run_list_atexit at entry=true) at exit.c:82 > 82 exit.c: No such file or directory. > (gdb) up > #9 0x00007ffff641a195 in __GI_exit (status=) at exit.c:104 > 104 in exit.c > (gdb) up > #10 0x00007ffff729cc00 in orte_ess_base_app_abort () from /usr/lib/libmpi.so.1 > (gdb) up > #11 0x00007ffff729c2a9 in orte_errmgr_base_error_abort () > from /usr/lib/libmpi.so.1 > (gdb) up > #12 0x00007ffff723bb69 in ompi_mpi_abort () from /usr/lib/libmpi.so.1 > (gdb) up > #13 0x00007ffff37b9ee3 in PetscSum_Local (in=0x230f4520, out=0x7fffffffc650, > cnt=, datatype=0x7fffffffc428) > at /opt/packages/src/petsc/src/sys/objects/pinit.c:302 > 302 MPI_Abort(MPI_COMM_WORLD,1); > (gdb) up > #14 0x00007fffb10dd4dc in mca_coll_basic_scan_intra () > from /usr/lib/openmpi/lib/openmpi/mca_coll_basic.so > (gdb) up > #15 0x00007fffb08c7e19 in mca_coll_sync_scan () > from /usr/lib/openmpi/lib/openmpi/mca_coll_sync.so > (gdb) up > #16 0x00007ffff725835d in PMPI_Scan () from /usr/lib/libmpi.so.1 > (gdb) up > #17 0x00007ffff413ec45 in MatWrapML_MPIAIJ (newmat=0x16a39228, > reuse=MAT_INITIAL_MATRIX, mlmat=0x7ed6090) > at /opt/packages/src/petsc/src/ksp/pc/impls/ml/ml.c:422 > 422 ierr = MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); > (gdb) list > 417 ierr = MatSetSizes(A,m,n,PETSC_DECIDE,PETSC_DECIDE);CHKERRQ(ierr); > 418 ierr = MatSetType(A,MATMPIAIJ);CHKERRQ(ierr); > 419 /* keep track of block size for A matrices */ > 420 ierr = MatSetBlockSize (A,mlmat->num_PDEs);CHKERRQ(ierr); > 421 ierr = PetscMalloc3(m,&nnzA,m,&nnzB,m,&nnz);CHKERRQ(ierr); > 422 ierr = MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); > 423 rstart -= m; > 424 > 425 for (i=0; i 426 row = gordering[i] - rstart; > (gdb) From knepley at gmail.com Wed Oct 1 09:49:31 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 1 Oct 2014 09:49:31 -0500 Subject: [petsc-users] PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX with ML preconditioner In-Reply-To: References: Message-ID: On Wed, Oct 1, 2014 at 9:32 AM, Barry Smith wrote: > > Recommend running under valgrind. > http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind Barry, it looks like this commit introduced it https://bitbucket.org/petsc/petsc/commits/7be6b909d0172a4ed9ffbe3833ba966bb47c2681#Lsrc/ksp/pc/impls/ml/ml.cT422 and it should fail according to https://bitbucket.org/petsc/petsc/src/643e5bfe5b452fa5bc89efaa4c603d713f2a6f29/src/sys/objects/pinit.c?at=master#cl-284 Why don't the nightly tests fail? Matt On Oct 1, 2014, at 7:03 AM, Chris Richardson wrote: > > > On 01/10/2014 11:33, Matthew Knepley wrote: > >> On Wed, Oct 1, 2014 at 3:51 AM, Chris Richardson > >> wrote: > >>> I have just run into this error after pulling the master branch of > >>> petsc, when using the ML preconditioner in FEniCS. > >>> It looks like a type error, but I guess FEniCS uses PETScInt, so > >>> should be safe? > >>> PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types > >> We need to see the entire error (with the stack). > > > > OK, this kind of thing:- > > > > > > [1]PETSC ERROR: Can only handle MPIU_REAL or MPIU_COMPLEX data types > > ^C > > Program received signal SIGINT, Interrupt. > > 0x00007ffff64d9653 in epoll_wait () at > ../sysdeps/unix/syscall-template.S:81 > > 81 ../sysdeps/unix/syscall-template.S: No such file or directory. > > (gdb) up > > #1 0x00007ffff72c8591 in ?? () from /usr/lib/libmpi.so.1 > > (gdb) up > > #2 0x00007ffff72ca787 in opal_event_base_loop () from > /usr/lib/libmpi.so.1 > > (gdb) up > > #3 0x00007ffff72eda3e in opal_progress () from /usr/lib/libmpi.so.1 > > (gdb) up > > #4 0x00007fffb42884d5 in ?? () > > from /usr/lib/openmpi/lib/openmpi/mca_grpcomm_bad.so > > (gdb) up > > #5 0x00007ffff723d02a in ompi_mpi_finalize () from /usr/lib/libmpi.so.1 > > (gdb) up > > #6 0x00007ffff7766b12 in dolfin::SubSystemsManager::finalize_mpi () > > at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:249 > > 249 MPI_Finalize(); > > (gdb) up > > #7 0x00007ffff7766cde in dolfin::SubSystemsManager::finalize () > > at /opt/packages/src/dolfin/dolfin/common/SubSystemsManager.cpp:219 > > 219 finalize_mpi(); > > (gdb) up > > #8 0x00007ffff641a149 in __run_exit_handlers (status=1, > > listp=0x7ffff679d6c8 <__exit_funcs>, > > run_list_atexit=run_list_atexit at entry=true) at exit.c:82 > > 82 exit.c: No such file or directory. > > (gdb) up > > #9 0x00007ffff641a195 in __GI_exit (status=) at > exit.c:104 > > 104 in exit.c > > (gdb) up > > #10 0x00007ffff729cc00 in orte_ess_base_app_abort () from > /usr/lib/libmpi.so.1 > > (gdb) up > > #11 0x00007ffff729c2a9 in orte_errmgr_base_error_abort () > > from /usr/lib/libmpi.so.1 > > (gdb) up > > #12 0x00007ffff723bb69 in ompi_mpi_abort () from /usr/lib/libmpi.so.1 > > (gdb) up > > #13 0x00007ffff37b9ee3 in PetscSum_Local (in=0x230f4520, > out=0x7fffffffc650, > > cnt=, datatype=0x7fffffffc428) > > at /opt/packages/src/petsc/src/sys/objects/pinit.c:302 > > 302 MPI_Abort(MPI_COMM_WORLD,1); > > (gdb) up > > #14 0x00007fffb10dd4dc in mca_coll_basic_scan_intra () > > from /usr/lib/openmpi/lib/openmpi/mca_coll_basic.so > > (gdb) up > > #15 0x00007fffb08c7e19 in mca_coll_sync_scan () > > from /usr/lib/openmpi/lib/openmpi/mca_coll_sync.so > > (gdb) up > > #16 0x00007ffff725835d in PMPI_Scan () from /usr/lib/libmpi.so.1 > > (gdb) up > > #17 0x00007ffff413ec45 in MatWrapML_MPIAIJ (newmat=0x16a39228, > > reuse=MAT_INITIAL_MATRIX, mlmat=0x7ed6090) > > at /opt/packages/src/petsc/src/ksp/pc/impls/ml/ml.c:422 > > 422 ierr = > MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); > > (gdb) list > > 417 ierr = > MatSetSizes(A,m,n,PETSC_DECIDE,PETSC_DECIDE);CHKERRQ(ierr); > > 418 ierr = MatSetType(A,MATMPIAIJ);CHKERRQ(ierr); > > 419 /* keep track of block size for A matrices */ > > 420 ierr = MatSetBlockSize (A,mlmat->num_PDEs);CHKERRQ(ierr); > > 421 ierr = PetscMalloc3(m,&nnzA,m,&nnzB,m,&nnz);CHKERRQ(ierr); > > 422 ierr = > MPI_Scan(&m,&rstart,1,MPIU_INT,MPIU_SUM,mlmat->comm->USR_com);CHKERRQ(ierr); > > 423 rstart -= m; > > 424 > > 425 for (i=0; i > 426 row = gordering[i] - rstart; > > (gdb) > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From epscodes at gmail.com Wed Oct 1 14:51:16 2014 From: epscodes at gmail.com (Xiangdong) Date: Wed, 1 Oct 2014 15:51:16 -0400 Subject: [petsc-users] examples of automatic differentiation with PETSc Message-ID: Hello everyone, Could you please point me to examples of using automatic differentiation for Jacobian calculation in PETSc? Chapter 9 of the manual briefly talked about adifor, but seems not quite complete. Thank you. Xiangdong -------------- next part -------------- An HTML attachment was scrubbed... URL: From chaw0023 at umn.edu Wed Oct 1 15:06:35 2014 From: chaw0023 at umn.edu (Saurabh Chawdhary) Date: Wed, 01 Oct 2014 15:06:35 -0500 Subject: [petsc-users] Default solver type for ksp? Message-ID: <542C5ECB.7060103@umn.edu> I understand that we can chose on of many solvers from KSPType form here . But if do not set any option to the type which one does it chose by default? I could not find the answer in manual or web search. It would be great if someone can share the knowledge. Thanks, Saurabh -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 1 15:09:13 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 15:09:13 -0500 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: References: Message-ID: Sorry that functionality is no longer in PETSc. On Oct 1, 2014, at 2:51 PM, Xiangdong wrote: > Hello everyone, > > Could you please point me to examples of using automatic differentiation for Jacobian calculation in PETSc? Chapter 9 of the manual briefly talked about adifor, but seems not quite complete. > > Thank you. > > Xiangdong From epscodes at gmail.com Wed Oct 1 15:13:44 2014 From: epscodes at gmail.com (Xiangdong) Date: Wed, 1 Oct 2014 16:13:44 -0400 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: References: Message-ID: Is there any reason that AD is not supported in PETSc? Is it because the ADIFOR/ADIC library itself? Thank you. Xiangdong On Wed, Oct 1, 2014 at 4:09 PM, Barry Smith wrote: > > > Sorry that functionality is no longer in PETSc. > > > On Oct 1, 2014, at 2:51 PM, Xiangdong wrote: > > > Hello everyone, > > > > Could you please point me to examples of using automatic differentiation > for Jacobian calculation in PETSc? Chapter 9 of the manual briefly talked > about adifor, but seems not quite complete. > > > > Thank you. > > > > Xiangdong > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 1 15:14:43 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 15:14:43 -0500 Subject: [petsc-users] Default solver type for ksp? In-Reply-To: <542C5ECB.7060103@umn.edu> References: <542C5ECB.7060103@umn.edu> Message-ID: Never guess or rely on possible wrong documentation or other peoples responses -ksp_view or KSPView(ksp,0); Barry On Oct 1, 2014, at 3:06 PM, Saurabh Chawdhary wrote: > I understand that we can chose on of many solvers from KSPType form here. But if do not set any option to the type which one does it chose by default? I could not find the answer in manual or web search. It would be great if someone can share the knowledge. > > Thanks, > Saurabh From jed at jedbrown.org Wed Oct 1 15:14:49 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 01 Oct 2014 15:14:49 -0500 Subject: [petsc-users] Default solver type for ksp? In-Reply-To: <542C5ECB.7060103@umn.edu> References: <542C5ECB.7060103@umn.edu> Message-ID: <87lhozy2bq.fsf@jedbrown.org> Saurabh Chawdhary writes: > I understand that we can chose on of many solvers from KSPType form here > . > But if do not set any option to the type which one does it chose by > default? GMRES > I could not find the answer in manual or web search. -ksp_view -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Wed Oct 1 15:16:20 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 01 Oct 2014 15:16:20 -0500 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: References: Message-ID: <87iok3y297.fsf@jedbrown.org> Xiangdong writes: > Is there any reason that AD is not supported in PETSc? Is it because the > ADIFOR/ADIC library itself? Those libraries are not maintained and have license problems. You're welcome to use an AD tool yourself. The support in PETSc was mostly just making simple examples cleaner; it wasn't widely used in real applications. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Oct 1 15:23:40 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 15:23:40 -0500 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: <87iok3y297.fsf@jedbrown.org> References: <87iok3y297.fsf@jedbrown.org> Message-ID: On Oct 1, 2014, at 3:16 PM, Jed Brown wrote: > Xiangdong writes: > >> Is there any reason that AD is not supported in PETSc? Is it because the >> ADIFOR/ADIC library itself? > > Those libraries are not maintained and have license problems. You're > welcome to use an AD tool yourself. The support in PETSc was mostly > just making simple examples cleaner; it wasn't widely used in real > applications. Wasn?t used at all in anything but trivial codes. From chaw0023 at umn.edu Wed Oct 1 15:53:45 2014 From: chaw0023 at umn.edu (Saurabh Chawdhary) Date: Wed, 01 Oct 2014 15:53:45 -0500 Subject: [petsc-users] Default solver type for ksp? In-Reply-To: <87lhozy2bq.fsf@jedbrown.org> References: <542C5ECB.7060103@umn.edu> <87lhozy2bq.fsf@jedbrown.org> Message-ID: <542C69D9.6060106@umn.edu> Thanks a lot guys. I also did -ksp_view and saw it was GMRES. ___________________________________ KSP Object: 32 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=500, initial guess is zero tolerances: relative=1e-10, absolute=1e-10, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test _____________________________________ On Wednesday, October 01, 2014 3:14:49 PM, Jed Brown wrote: > Saurabh Chawdhary writes: > >> I understand that we can chose on of many solvers from KSPType form here >> . >> But if do not set any option to the type which one does it chose by >> default? > > GMRES > >> I could not find the answer in manual or web search. > > -ksp_view From gaetank at gmail.com Wed Oct 1 16:09:40 2014 From: gaetank at gmail.com (Gaetan Kenway) Date: Wed, 1 Oct 2014 17:09:40 -0400 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: References: <87iok3y297.fsf@jedbrown.org> Message-ID: HI Xiangdong We have use the Tapenade AD package to compute sparse matrix jacobians for CFD problems. We use the result from the AD to populate a PETSc sparse matrix and then use PETSc to solve the resulting linear system. Hope that helps. Gaetan On Wed, Oct 1, 2014 at 4:23 PM, Barry Smith wrote: > > On Oct 1, 2014, at 3:16 PM, Jed Brown wrote: > > > Xiangdong writes: > > > >> Is there any reason that AD is not supported in PETSc? Is it because the > >> ADIFOR/ADIC library itself? > > > > Those libraries are not maintained and have license problems. You're > > welcome to use an AD tool yourself. The support in PETSc was mostly > > just making simple examples cleaner; it wasn't widely used in real > > applications. > > Wasn?t used at all in anything but trivial codes. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 1 16:17:05 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 16:17:05 -0500 Subject: [petsc-users] examples of automatic differentiation with PETSc In-Reply-To: References: <87iok3y297.fsf@jedbrown.org> Message-ID: Thanks. Note that us removing the ?simple minded? AD stuff from PETSc doesn?t mean we think AD is a bad idea. It can be an effective way to generate Jacobians. Barry On Oct 1, 2014, at 4:09 PM, Gaetan Kenway wrote: > HI Xiangdong > > We have use the Tapenade AD package to compute sparse matrix jacobians for CFD problems. We use the result from the AD to populate a PETSc sparse matrix and then use PETSc to solve the resulting linear system. > > Hope that helps. > > Gaetan > > On Wed, Oct 1, 2014 at 4:23 PM, Barry Smith wrote: > > On Oct 1, 2014, at 3:16 PM, Jed Brown wrote: > > > Xiangdong writes: > > > >> Is there any reason that AD is not supported in PETSc? Is it because the > >> ADIFOR/ADIC library itself? > > > > Those libraries are not maintained and have license problems. You're > > welcome to use an AD tool yourself. The support in PETSc was mostly > > just making simple examples cleaner; it wasn't widely used in real > > applications. > > Wasn?t used at all in anything but trivial codes. > > > > From bsmith at mcs.anl.gov Wed Oct 1 21:42:39 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 1 Oct 2014 21:42:39 -0500 Subject: [petsc-users] include/finclude/petscsysdef.h and daimag In-Reply-To: <1FDD0BF6-7CB3-42AB-B928-1AECE701B705@gmail.com> References: <1FDD0BF6-7CB3-42AB-B928-1AECE701B705@gmail.com> Message-ID: Randy, Thanks for the report. I am not sure why we use the d.. version of any of those functions. I have eliminated them in the branch barry/remove-dreal and merged that to next. If all the testing looks good on it we will put it into maint (the next patch) and master. Barry On Sep 29, 2014, at 3:42 PM, Randall Mackie wrote: > I recently ran into an issue with include/finclude/petscsysdef.h and the definition of PetscImaginaryPart, which is defined as daimag(a) in the case PETSC_MISSING_DREAL is not defined. > > > 1) As far as I know, daimag is not a valid fortran statement, and I suspect that here you might want dimag. > > > 2) That being said, I was wondering why all my compiles using gfortran worked fine and didn't complain, whereas an Intel 2015 compilation did complain about daimag. > > Turns out Intel and gfortran have different behaviors for the dreal test in config/BuildSystem/config/types.py. > Gfortran gives an *error* saying the argument in the call to dreal should be COMPLEX(8) and not REAL(4), whereas Intel just *warns* that the argument data type is incompatible with the intrinsic procedure. Since gfortran does not compile the dreal code test, it sets PETSC_MISSING_DREAL to 1, and uses aimag. > > But I'm curious just what exactly you are trying to test for? DREAL is a valid extension if the argument is double complex. So dreal(dcmplx(3.0,0.0)) will work just fine, but your test of dreal(3.0) is not a valid statement and should fail. > > Furthermore, I'm not sure I understand the need for the dreal stuff, since real, conjg, aimag all return values of the same kind as the argument for the case of complex variables. > > > Thanks, Randy Mackie From balay at mcs.anl.gov Thu Oct 2 08:16:46 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 08:16:46 -0500 Subject: [petsc-users] petsc git next branch *is* unwound! Message-ID: petsc git users, If you are using next branch from git repository for integration or testing or working with it in some form - i.e if you have 'next' listed by: git branch Please delete and recreate it. i.e: git checkout master git branch -D next git checkout next >From my check - the following feature branches are currently in next (not yet merged to master). Feature branch owners, please remerge your feature branches into to next - as you see fit. $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) origin/barry/cleanup-blaslapack origin/barry/remove-dreal origin/barry/remove-sidl origin/barry/reuse-hwloc origin/dmeiser/pullrequest-fix-cusp-bjacobi2 origin/dmeiser/pullrequest-remove-thrust-py origin/jed/sr-driver4 origin/jed/vecghostgetvalues origin/karlrupp/fix-viennacl-veccopy origin/karpeev/fix-ksp-pcgasm origin/knepley/feature-optcontrol-mg origin/knepley/feature-plex-generator-args origin/knepley/feature-plex-nasm origin/knepley/fix-configure-petsc-dep origin/knepley/fix-quadrature-order origin/madams/sr-driver4 origin/maint origin/mlange05/fix-gmsh-coordinates origin/next origin/paulmullowney/aijcusp-ellpack-fix origin/prbrune/sf-examplefix origin/prbrune/snes-ex19vtkoutput origin/sarich/fix-tao-jbearing-bqpip origin/shri/ts-dae-semi-explicit origin/shri/ts-is-for-differential-variables origin/shri/ts-powergrid origin/stefano_zampini/matis-preallocation origin/stefano_zampini/pcbddc-changeofbasis origin/stefano_zampini/pcbddc_deluxe origin/tisaac/feature-fe-geom-dim origin/tisaac/plex-flexible-projection [obviously remove maint, next from the above list] The previous 'next' branch is currently available as 'next-oct-2014' - if you need it for any checks/debugging. Satish From balay at mcs.anl.gov Thu Oct 2 08:38:31 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 08:38:31 -0500 Subject: [petsc-users] petsc git next branch *is* unwound! In-Reply-To: References: Message-ID: On Thu, 2 Oct 2014, Satish Balay wrote: > petsc git users, > > If you are using next branch from git repository for integration or > testing or working with it in some form - i.e if you have 'next' > listed by: > > git branch > > Please delete and recreate it. i.e: > > git checkout master > git branch -D next > git checkout next > > From my check - the following feature branches are currently in next > (not yet merged to master). > > Feature branch owners, please remerge your feature branches into to > next - as you see fit. BTW: Some of you might want to rebase/cleanup some of the feature branches before merging to new next. Also some of these feature branches are from pre-petsc-3.5 timeframe. So a rebase to latest master might be something to consider. [if they are still in line for a merge to master] Satish > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > origin/barry/cleanup-blaslapack > origin/barry/remove-dreal > origin/barry/remove-sidl > origin/barry/reuse-hwloc > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > origin/dmeiser/pullrequest-remove-thrust-py > origin/jed/sr-driver4 > origin/jed/vecghostgetvalues > origin/karlrupp/fix-viennacl-veccopy > origin/karpeev/fix-ksp-pcgasm > origin/knepley/feature-optcontrol-mg > origin/knepley/feature-plex-generator-args > origin/knepley/feature-plex-nasm > origin/knepley/fix-configure-petsc-dep > origin/knepley/fix-quadrature-order > origin/madams/sr-driver4 > origin/maint > origin/mlange05/fix-gmsh-coordinates > origin/next > origin/paulmullowney/aijcusp-ellpack-fix > origin/prbrune/sf-examplefix > origin/prbrune/snes-ex19vtkoutput > origin/sarich/fix-tao-jbearing-bqpip > origin/shri/ts-dae-semi-explicit > origin/shri/ts-is-for-differential-variables > origin/shri/ts-powergrid > origin/stefano_zampini/matis-preallocation > origin/stefano_zampini/pcbddc-changeofbasis > origin/stefano_zampini/pcbddc_deluxe > origin/tisaac/feature-fe-geom-dim > origin/tisaac/plex-flexible-projection > > [obviously remove maint, next from the above list] > > The previous 'next' branch is currently available as 'next-oct-2014' - > if you need it for any checks/debugging. > > Satish > > > > From hillsmattc at outlook.com Thu Oct 2 08:51:31 2014 From: hillsmattc at outlook.com (Matthew Hills) Date: Thu, 2 Oct 2014 15:51:31 +0200 Subject: [petsc-users] Scalsca support Message-ID: Hi PETSc Team, I'm attempting to use Scalsca to analyze the MPI communications in a program called SESKA. I'm struggling to find a means of doing so, do you perhaps have any ideas? I currently configure with: ./configure --with-mpi=1 --CFLAGS=-pg -CXX_CXXFLAGS=-pg --FFLAGS=-pg --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-f-blas-lapack=${PETSC_DIR}/externalpackages/fblaslapacklinpack-3.1.1.tar.gz --with-parmetis=1 --download-parmetis=${PETSC_DIR}/externalpackages/parmetis-4.0.2-p3.tar.gz --with-mumps=1 --download-mumps=${PETSC_DIR}/externalpackages/MUMPS_4.10.0-p3.tar.gz --with-scalapack=1 --download-scalapack=${PETSC_DIR}/externalpackages/scalapack-2.0.2.tgz --download-blacs=${PETSC_DIR}/externalpackages/blacs-dev.tar.gz --with-superlu_dist=1 --download-superlu_dist=${PETSC_DIR}/externalpackages/superlu_dist_3.1.tar.gz --with-metis --download-metis=${PETSC_DIR}/externalpackages/metis-5.0.2-p3.tar.gz --with-sowing=${PETSC_DIR}/externalpackages/sowing-1.1.16d.tar.gz --with-c2html=0 --with-shared-libraries=1 --download-mpe=${SESKADIR}/packages/downloads/mpe2-1.3.0.tar.gz --download-mpich=${SESKADIR}/packages/downloads/mpich-3.0.4.tar.gz --with-debugging=1 The compiler flags are used to generate compiler based profiles. MPE is being used with the PETSc logger to also create a profile but this information is proving insufficient (not displaying MPI communications/barriers). I execute with: mpiexec -n $1 seska -log_mpe petsc_mpe Regards, Matthew Hills -------------- next part -------------- An HTML attachment was scrubbed... URL: From hillsmattc at outlook.com Thu Oct 2 09:07:11 2014 From: hillsmattc at outlook.com (Matthew Hills) Date: Thu, 2 Oct 2014 16:07:11 +0200 Subject: [petsc-users] Scalsca support In-Reply-To: References: Message-ID: * typo: Scalasca From: hillsmattc at outlook.com To: petsc-users at mcs.anl.gov Date: Thu, 2 Oct 2014 15:51:31 +0200 Subject: [petsc-users] Scalsca support Hi PETSc Team, I'm attempting to use Scalsca to analyze the MPI communications in a program called SESKA. I'm struggling to find a means of doing so, do you perhaps have any ideas? I currently configure with: ./configure --with-mpi=1 --CFLAGS=-pg -CXX_CXXFLAGS=-pg --FFLAGS=-pg --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-f-blas-lapack=${PETSC_DIR}/externalpackages/fblaslapacklinpack-3.1.1.tar.gz --with-parmetis=1 --download-parmetis=${PETSC_DIR}/externalpackages/parmetis-4.0.2-p3.tar.gz --with-mumps=1 --download-mumps=${PETSC_DIR}/externalpackages/MUMPS_4.10.0-p3.tar.gz --with-scalapack=1 --download-scalapack=${PETSC_DIR}/externalpackages/scalapack-2.0.2.tgz --download-blacs=${PETSC_DIR}/externalpackages/blacs-dev.tar.gz --with-superlu_dist=1 --download-superlu_dist=${PETSC_DIR}/externalpackages/superlu_dist_3.1.tar.gz --with-metis --download-metis=${PETSC_DIR}/externalpackages/metis-5.0.2-p3.tar.gz --with-sowing=${PETSC_DIR}/externalpackages/sowing-1.1.16d.tar.gz --with-c2html=0 --with-shared-libraries=1 --download-mpe=${SESKADIR}/packages/downloads/mpe2-1.3.0.tar.gz --download-mpich=${SESKADIR}/packages/downloads/mpich-3.0.4.tar.gz --with-debugging=1 The compiler flags are used to generate compiler based profiles. MPE is being used with the PETSc logger to also create a profile but this information is proving insufficient (not displaying MPI communications/barriers). I execute with: mpiexec -n $1 seska -log_mpe petsc_mpe Regards, Matthew Hills -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 2 11:05:37 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 2 Oct 2014 11:05:37 -0500 Subject: [petsc-users] petsc git next branch *is* unwound! In-Reply-To: References: Message-ID: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Why does it say "Already up-to-date.? ? It?s like those things are already in next? Why isn?t it putting a bunch of stuff into next? Confusing Barry ~/Src/petsc next $ git checkout master Switched to branch 'master' Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded. (use "git pull" to update your local branch) ~/Src/petsc master $ git branch -D next Deleted branch next (was 5fad734). ~/Src/petsc master $ git checkout next Branch next set up to track remote branch next from origin. Switched to a new branch 'next' ~/Src/petsc next $ git pull From bitbucket.org:petsc/petsc + 5fad734...6e2aac5 next -> origin/next (forced update) * [new branch] next-oct-2014 -> origin/next-oct-2014 Already up-to-date! Merge made by the 'recursive' strategy. ~/Src/petsc next $ git merge barry/cleanup-blaslapack Already up-to-date. ~/Src/petsc next $ git merge barry/remove-dreal Already up-to-date. ~/Src/petsc next $ git merge barry/remove-sidl Already up-to-date. ~/Src/petsc next $ git merge barry/reuse-hwloc Already up-to-date. ~/Src/petsc next $ git push Counting objects: 1, done. Writing objects: 100% (1/1), 248 bytes | 0 bytes/s, done. Total 1 (delta 0), reused 0 (delta 0) To git at bitbucket.org:petsc/petsc.git 6e2aac5..8c34c3a next -> next On Oct 2, 2014, at 8:16 AM, Satish Balay wrote: > petsc git users, > > If you are using next branch from git repository for integration or > testing or working with it in some form - i.e if you have 'next' > listed by: > > git branch > > Please delete and recreate it. i.e: > > git checkout master > git branch -D next > git checkout next > > From my check - the following feature branches are currently in next > (not yet merged to master). > > Feature branch owners, please remerge your feature branches into to > next - as you see fit. > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > origin/barry/cleanup-blaslapack > origin/barry/remove-dreal > origin/barry/remove-sidl > origin/barry/reuse-hwloc > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > origin/dmeiser/pullrequest-remove-thrust-py > origin/jed/sr-driver4 > origin/jed/vecghostgetvalues > origin/karlrupp/fix-viennacl-veccopy > origin/karpeev/fix-ksp-pcgasm > origin/knepley/feature-optcontrol-mg > origin/knepley/feature-plex-generator-args > origin/knepley/feature-plex-nasm > origin/knepley/fix-configure-petsc-dep > origin/knepley/fix-quadrature-order > origin/madams/sr-driver4 > origin/maint > origin/mlange05/fix-gmsh-coordinates > origin/next > origin/paulmullowney/aijcusp-ellpack-fix > origin/prbrune/sf-examplefix > origin/prbrune/snes-ex19vtkoutput > origin/sarich/fix-tao-jbearing-bqpip > origin/shri/ts-dae-semi-explicit > origin/shri/ts-is-for-differential-variables > origin/shri/ts-powergrid > origin/stefano_zampini/matis-preallocation > origin/stefano_zampini/pcbddc-changeofbasis > origin/stefano_zampini/pcbddc_deluxe > origin/tisaac/feature-fe-geom-dim > origin/tisaac/plex-flexible-projection > > [obviously remove maint, next from the above list] > > The previous 'next' branch is currently available as 'next-oct-2014' - > if you need it for any checks/debugging. > > Satish > > > From balay at mcs.anl.gov Thu Oct 2 11:08:28 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 11:08:28 -0500 Subject: [petsc-users] petsc git next branch *is* unwound! In-Reply-To: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: Sorry - perhaps the instructions should have been: git checkout master git branch -D next git fetch git checkout next Satish On Thu, 2 Oct 2014, Barry Smith wrote: > > Why does it say "Already up-to-date.? ? It?s like those things are already in next? Why isn?t it putting a bunch of stuff into next? Confusing > > > Barry > > > ~/Src/petsc next $ git checkout master > Switched to branch 'master' > Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded. > (use "git pull" to update your local branch) > ~/Src/petsc master $ git branch -D next > Deleted branch next (was 5fad734). > ~/Src/petsc master $ git checkout next > Branch next set up to track remote branch next from origin. > Switched to a new branch 'next' > ~/Src/petsc next $ git pull > From bitbucket.org:petsc/petsc > + 5fad734...6e2aac5 next -> origin/next (forced update) > * [new branch] next-oct-2014 -> origin/next-oct-2014 > Already up-to-date! > Merge made by the 'recursive' strategy. > ~/Src/petsc next $ git merge barry/cleanup-blaslapack > Already up-to-date. > ~/Src/petsc next $ git merge barry/remove-dreal > Already up-to-date. > ~/Src/petsc next $ git merge barry/remove-sidl > Already up-to-date. > ~/Src/petsc next $ git merge barry/reuse-hwloc > Already up-to-date. > ~/Src/petsc next $ git push > Counting objects: 1, done. > Writing objects: 100% (1/1), 248 bytes | 0 bytes/s, done. > Total 1 (delta 0), reused 0 (delta 0) > To git at bitbucket.org:petsc/petsc.git > 6e2aac5..8c34c3a next -> next > > > > > On Oct 2, 2014, at 8:16 AM, Satish Balay wrote: > > > petsc git users, > > > > If you are using next branch from git repository for integration or > > testing or working with it in some form - i.e if you have 'next' > > listed by: > > > > git branch > > > > Please delete and recreate it. i.e: > > > > git checkout master > > git branch -D next > > git checkout next > > > > From my check - the following feature branches are currently in next > > (not yet merged to master). > > > > Feature branch owners, please remerge your feature branches into to > > next - as you see fit. > > > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > > origin/barry/cleanup-blaslapack > > origin/barry/remove-dreal > > origin/barry/remove-sidl > > origin/barry/reuse-hwloc > > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > > origin/dmeiser/pullrequest-remove-thrust-py > > origin/jed/sr-driver4 > > origin/jed/vecghostgetvalues > > origin/karlrupp/fix-viennacl-veccopy > > origin/karpeev/fix-ksp-pcgasm > > origin/knepley/feature-optcontrol-mg > > origin/knepley/feature-plex-generator-args > > origin/knepley/feature-plex-nasm > > origin/knepley/fix-configure-petsc-dep > > origin/knepley/fix-quadrature-order > > origin/madams/sr-driver4 > > origin/maint > > origin/mlange05/fix-gmsh-coordinates > > origin/next > > origin/paulmullowney/aijcusp-ellpack-fix > > origin/prbrune/sf-examplefix > > origin/prbrune/snes-ex19vtkoutput > > origin/sarich/fix-tao-jbearing-bqpip > > origin/shri/ts-dae-semi-explicit > > origin/shri/ts-is-for-differential-variables > > origin/shri/ts-powergrid > > origin/stefano_zampini/matis-preallocation > > origin/stefano_zampini/pcbddc-changeofbasis > > origin/stefano_zampini/pcbddc_deluxe > > origin/tisaac/feature-fe-geom-dim > > origin/tisaac/plex-flexible-projection > > > > [obviously remove maint, next from the above list] > > > > The previous 'next' branch is currently available as 'next-oct-2014' - > > if you need it for any checks/debugging. > > > > Satish > > > > > > > > From jed at jedbrown.org Thu Oct 2 11:09:04 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 11:09:04 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: <87fvf6v4gv.fsf@jedbrown.org> Barry Smith writes: > Why does it say "Already up-to-date.? ? It?s like those things are already in next? Why isn?t it putting a bunch of stuff into next? Confusing > > > Barry > > > ~/Src/petsc next $ git checkout master > Switched to branch 'master' > Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded. > (use "git pull" to update your local branch) > ~/Src/petsc master $ git branch -D next > Deleted branch next (was 5fad734). > ~/Src/petsc master $ git checkout next > Branch next set up to track remote branch next from origin. > Switched to a new branch 'next' Satish forgot to tell you to "git fetch" before "git checkout next". -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From balay at mcs.anl.gov Thu Oct 2 11:26:25 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 11:26:25 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: balay at asterix /home/balay/petsc (master=) $ git co next Switched to branch 'next' Your branch is behind 'origin/next' by 1963 commits, and can be fast-forwarded. (use "git pull" to update your local branch) balay at asterix /home/balay/petsc (next<) Did old next get remerged into new next? Should I rewind it again? Satish On Thu, 2 Oct 2014, Satish Balay wrote: > Sorry - perhaps the instructions should have been: > > git checkout master > git branch -D next > git fetch > git checkout next > > Satish > > On Thu, 2 Oct 2014, Barry Smith wrote: > > > > > Why does it say "Already up-to-date.? ? It?s like those things are already in next? Why isn?t it putting a bunch of stuff into next? Confusing > > > > > > Barry > > > > > > ~/Src/petsc next $ git checkout master > > Switched to branch 'master' > > Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded. > > (use "git pull" to update your local branch) > > ~/Src/petsc master $ git branch -D next > > Deleted branch next (was 5fad734). > > ~/Src/petsc master $ git checkout next > > Branch next set up to track remote branch next from origin. > > Switched to a new branch 'next' > > ~/Src/petsc next $ git pull > > From bitbucket.org:petsc/petsc > > + 5fad734...6e2aac5 next -> origin/next (forced update) > > * [new branch] next-oct-2014 -> origin/next-oct-2014 > > Already up-to-date! > > Merge made by the 'recursive' strategy. > > ~/Src/petsc next $ git merge barry/cleanup-blaslapack > > Already up-to-date. > > ~/Src/petsc next $ git merge barry/remove-dreal > > Already up-to-date. > > ~/Src/petsc next $ git merge barry/remove-sidl > > Already up-to-date. > > ~/Src/petsc next $ git merge barry/reuse-hwloc > > Already up-to-date. > > ~/Src/petsc next $ git push > > Counting objects: 1, done. > > Writing objects: 100% (1/1), 248 bytes | 0 bytes/s, done. > > Total 1 (delta 0), reused 0 (delta 0) > > To git at bitbucket.org:petsc/petsc.git > > 6e2aac5..8c34c3a next -> next > > > > > > > > > > On Oct 2, 2014, at 8:16 AM, Satish Balay wrote: > > > > > petsc git users, > > > > > > If you are using next branch from git repository for integration or > > > testing or working with it in some form - i.e if you have 'next' > > > listed by: > > > > > > git branch > > > > > > Please delete and recreate it. i.e: > > > > > > git checkout master > > > git branch -D next > > > git checkout next > > > > > > From my check - the following feature branches are currently in next > > > (not yet merged to master). > > > > > > Feature branch owners, please remerge your feature branches into to > > > next - as you see fit. > > > > > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > > > origin/barry/cleanup-blaslapack > > > origin/barry/remove-dreal > > > origin/barry/remove-sidl > > > origin/barry/reuse-hwloc > > > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > > > origin/dmeiser/pullrequest-remove-thrust-py > > > origin/jed/sr-driver4 > > > origin/jed/vecghostgetvalues > > > origin/karlrupp/fix-viennacl-veccopy > > > origin/karpeev/fix-ksp-pcgasm > > > origin/knepley/feature-optcontrol-mg > > > origin/knepley/feature-plex-generator-args > > > origin/knepley/feature-plex-nasm > > > origin/knepley/fix-configure-petsc-dep > > > origin/knepley/fix-quadrature-order > > > origin/madams/sr-driver4 > > > origin/maint > > > origin/mlange05/fix-gmsh-coordinates > > > origin/next > > > origin/paulmullowney/aijcusp-ellpack-fix > > > origin/prbrune/sf-examplefix > > > origin/prbrune/snes-ex19vtkoutput > > > origin/sarich/fix-tao-jbearing-bqpip > > > origin/shri/ts-dae-semi-explicit > > > origin/shri/ts-is-for-differential-variables > > > origin/shri/ts-powergrid > > > origin/stefano_zampini/matis-preallocation > > > origin/stefano_zampini/pcbddc-changeofbasis > > > origin/stefano_zampini/pcbddc_deluxe > > > origin/tisaac/feature-fe-geom-dim > > > origin/tisaac/plex-flexible-projection > > > > > > [obviously remove maint, next from the above list] > > > > > > The previous 'next' branch is currently available as 'next-oct-2014' - > > > if you need it for any checks/debugging. > > > > > > Satish > > > > > > > > > > > > > > From bsmith at mcs.anl.gov Thu Oct 2 11:29:56 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 2 Oct 2014 11:29:56 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: Too late. I likely screwed up the new next. On Oct 2, 2014, at 11:26 AM, Satish Balay wrote: > balay at asterix /home/balay/petsc (master=) > $ git co next > Switched to branch 'next' > Your branch is behind 'origin/next' by 1963 commits, and can be fast-forwarded. > (use "git pull" to update your local branch) > balay at asterix /home/balay/petsc (next<) > > > Did old next get remerged into new next? Should I rewind it again? > > Satish > > > On Thu, 2 Oct 2014, Satish Balay wrote: > >> Sorry - perhaps the instructions should have been: >> >> git checkout master >> git branch -D next >> git fetch >> git checkout next >> >> Satish >> >> On Thu, 2 Oct 2014, Barry Smith wrote: >> >>> >>> Why does it say "Already up-to-date.? ? It?s like those things are already in next? Why isn?t it putting a bunch of stuff into next? Confusing >>> >>> >>> Barry >>> >>> >>> ~/Src/petsc next $ git checkout master >>> Switched to branch 'master' >>> Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded. >>> (use "git pull" to update your local branch) >>> ~/Src/petsc master $ git branch -D next >>> Deleted branch next (was 5fad734). >>> ~/Src/petsc master $ git checkout next >>> Branch next set up to track remote branch next from origin. >>> Switched to a new branch 'next' >>> ~/Src/petsc next $ git pull >>> From bitbucket.org:petsc/petsc >>> + 5fad734...6e2aac5 next -> origin/next (forced update) >>> * [new branch] next-oct-2014 -> origin/next-oct-2014 >>> Already up-to-date! >>> Merge made by the 'recursive' strategy. >>> ~/Src/petsc next $ git merge barry/cleanup-blaslapack >>> Already up-to-date. >>> ~/Src/petsc next $ git merge barry/remove-dreal >>> Already up-to-date. >>> ~/Src/petsc next $ git merge barry/remove-sidl >>> Already up-to-date. >>> ~/Src/petsc next $ git merge barry/reuse-hwloc >>> Already up-to-date. >>> ~/Src/petsc next $ git push >>> Counting objects: 1, done. >>> Writing objects: 100% (1/1), 248 bytes | 0 bytes/s, done. >>> Total 1 (delta 0), reused 0 (delta 0) >>> To git at bitbucket.org:petsc/petsc.git >>> 6e2aac5..8c34c3a next -> next >>> >>> >>> >>> >>> On Oct 2, 2014, at 8:16 AM, Satish Balay wrote: >>> >>>> petsc git users, >>>> >>>> If you are using next branch from git repository for integration or >>>> testing or working with it in some form - i.e if you have 'next' >>>> listed by: >>>> >>>> git branch >>>> >>>> Please delete and recreate it. i.e: >>>> >>>> git checkout master >>>> git branch -D next >>>> git checkout next >>>> >>>> From my check - the following feature branches are currently in next >>>> (not yet merged to master). >>>> >>>> Feature branch owners, please remerge your feature branches into to >>>> next - as you see fit. >>>> >>>> $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) >>>> origin/barry/cleanup-blaslapack >>>> origin/barry/remove-dreal >>>> origin/barry/remove-sidl >>>> origin/barry/reuse-hwloc >>>> origin/dmeiser/pullrequest-fix-cusp-bjacobi2 >>>> origin/dmeiser/pullrequest-remove-thrust-py >>>> origin/jed/sr-driver4 >>>> origin/jed/vecghostgetvalues >>>> origin/karlrupp/fix-viennacl-veccopy >>>> origin/karpeev/fix-ksp-pcgasm >>>> origin/knepley/feature-optcontrol-mg >>>> origin/knepley/feature-plex-generator-args >>>> origin/knepley/feature-plex-nasm >>>> origin/knepley/fix-configure-petsc-dep >>>> origin/knepley/fix-quadrature-order >>>> origin/madams/sr-driver4 >>>> origin/maint >>>> origin/mlange05/fix-gmsh-coordinates >>>> origin/next >>>> origin/paulmullowney/aijcusp-ellpack-fix >>>> origin/prbrune/sf-examplefix >>>> origin/prbrune/snes-ex19vtkoutput >>>> origin/sarich/fix-tao-jbearing-bqpip >>>> origin/shri/ts-dae-semi-explicit >>>> origin/shri/ts-is-for-differential-variables >>>> origin/shri/ts-powergrid >>>> origin/stefano_zampini/matis-preallocation >>>> origin/stefano_zampini/pcbddc-changeofbasis >>>> origin/stefano_zampini/pcbddc_deluxe >>>> origin/tisaac/feature-fe-geom-dim >>>> origin/tisaac/plex-flexible-projection >>>> >>>> [obviously remove maint, next from the above list] >>>> >>>> The previous 'next' branch is currently available as 'next-oct-2014' - >>>> if you need it for any checks/debugging. >>>> >>>> Satish >>>> >>>> >>>> >>> >>> >> From jed at jedbrown.org Thu Oct 2 11:30:30 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 11:30:30 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: <87a95ev3h5.fsf@jedbrown.org> Satish Balay writes: > balay at asterix /home/balay/petsc (master=) > $ git co next > Switched to branch 'next' > Your branch is behind 'origin/next' by 1963 commits, and can be fast-forwarded. > (use "git pull" to update your local branch) > balay at asterix /home/balay/petsc (next<) > > > Did old next get remerged into new next? Should I rewind it again? https://bitbucket.org/petsc/petsc/commits/8c34c3ad2d52157761586f2bb6d4466aeb01b511 Barry should $ git config pull.ff only so he can stop making undesirable merges from upstream. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Thu Oct 2 11:34:30 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 11:34:30 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> Message-ID: <874mvmv3ah.fsf@jedbrown.org> Barry Smith writes: > Too late. I likely screwed up the new next. fixed now -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From balay at mcs.anl.gov Thu Oct 2 11:37:22 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 11:37:22 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <874mvmv3ah.fsf@jedbrown.org> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <874mvmv3ah.fsf@jedbrown.org> Message-ID: On Thu, 2 Oct 2014, Jed Brown wrote: > Barry Smith writes: > > > Too late. I likely screwed up the new next. > > fixed now And everyone who had the git repo should do the following (again - if its done already) git checkout master git branch -D next git fetch git checkout next Satish From bsmith at mcs.anl.gov Thu Oct 2 11:37:39 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 2 Oct 2014 11:37:39 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <87a95ev3h5.fsf@jedbrown.org> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <87a95ev3h5.fsf@jedbrown.org> Message-ID: Done, but note in my previous email with a full cut and paste of what I did I never got any messages about misbehavior On Oct 2, 2014, at 11:30 AM, Jed Brown wrote: > Satish Balay writes: > >> balay at asterix /home/balay/petsc (master=) >> $ git co next >> Switched to branch 'next' >> Your branch is behind 'origin/next' by 1963 commits, and can be fast-forwarded. >> (use "git pull" to update your local branch) >> balay at asterix /home/balay/petsc (next<) >> >> >> Did old next get remerged into new next? Should I rewind it again? > > https://bitbucket.org/petsc/petsc/commits/8c34c3ad2d52157761586f2bb6d4466aeb01b511 > > Barry should > > $ git config pull.ff only > > so he can stop making undesirable merges from upstream. From bsmith at mcs.anl.gov Thu Oct 2 11:39:35 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 2 Oct 2014 11:39:35 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <874mvmv3ah.fsf@jedbrown.org> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <874mvmv3ah.fsf@jedbrown.org> Message-ID: So should I redo the process Satish said or is my current next ok? (Seems hard to believe) On Oct 2, 2014, at 11:34 AM, Jed Brown wrote: > Barry Smith writes: > >> Too late. I likely screwed up the new next. > > fixed now From jed at jedbrown.org Thu Oct 2 11:45:35 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 11:45:35 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <874mvmv3ah.fsf@jedbrown.org> Message-ID: <87vbo2to7k.fsf@jedbrown.org> Barry Smith writes: > So should I redo the process Satish said or is my current next ok? (Seems hard to believe) Redo it. And set pull.ff=only. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Thu Oct 2 11:47:57 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 11:47:57 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <87a95ev3h5.fsf@jedbrown.org> Message-ID: <87sij6to3m.fsf@jedbrown.org> Barry Smith writes: > Done, but note in my previous email with a full cut and paste of what I did I never got any messages about misbehavior The following looks to me like jumbled output. | ~/Src/petsc next $ git pull | From bitbucket.org:petsc/petsc | + 5fad734...6e2aac5 next -> origin/next (forced update) | * [new branch] next-oct-2014 -> origin/next-oct-2014 | Already up-to-date! | Merge made by the 'recursive' strategy. | ~/Src/petsc next $ It doesn't make sense to say "Already up-to-date" and that it made a merge. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From bsmith at mcs.anl.gov Thu Oct 2 11:53:15 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 2 Oct 2014 11:53:15 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <87sij6to3m.fsf@jedbrown.org> References: <8831BC4F-D55A-4BAE-9EFB-F2FF1FF43A73@mcs.anl.gov> <87a95ev3h5.fsf@jedbrown.org> <87sij6to3m.fsf@jedbrown.org> Message-ID: It is not jumbled output. It is the output I got On Oct 2, 2014, at 11:47 AM, Jed Brown wrote: > Barry Smith writes: > >> Done, but note in my previous email with a full cut and paste of what I did I never got any messages about misbehavior > > The following looks to me like jumbled output. > > | ~/Src/petsc next $ git pull > | From bitbucket.org:petsc/petsc > | + 5fad734...6e2aac5 next -> origin/next (forced update) > | * [new branch] next-oct-2014 -> origin/next-oct-2014 > | Already up-to-date! > | Merge made by the 'recursive' strategy. > | ~/Src/petsc next $ > > It doesn't make sense to say "Already up-to-date" and that it made a merge. From wdn at lanl.gov Thu Oct 2 11:57:56 2014 From: wdn at lanl.gov (Nystrom, William David) Date: Thu, 2 Oct 2014 16:57:56 +0000 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: , Message-ID: On my local machine I have my master, bare git repo with the following config file which Jed suggested a year or so ago: % cat config [core] repositoryformatversion = 0 filemode = true bare = true sharedrepository = 1 [receive] denyNonFastforwards = true [remote "origin"] url = https://bitbucket.org/petsc/petsc fetch = +refs/heads/maint:refs/heads/maint fetch = +refs/heads/master:refs/heads/master fetch = +refs/heads/next:refs/heads/next [remote "artab"] url = ssh://ml-fey.lanl.gov/usr/projects/artab/users/wdn/repos/git/petsc fetch = +refs/*:refs/* mirror = true And for my workflow, I stay in sync with the official PETSc git repo by periodically executing the following command: git remote -v update origin And then I have various git clones from my master git repo where I do actual work. Do I need to do anything special now that the "next" branch has been unwound with either my master git repo or my git clones? In my master git repo, I have 3 additional branches - wdn_mods_next, wdn_mods_master and wdn_mods_maint where I keep any of my own modifications to those respective branches. And I keep wdn_mods_next in sync with next by periodically executing the following commands in a git clone with the wdn_mods_next branch checked out: git pull origin next resolve any conflicts and commit git push origin wdn_mods_next:wdn_mods_next So, if I do the following set of steps, should I expect them to work now that next has been unwound? 1. cd to my master bare petsc repo git remote -v update origin 2. cd to my git clone of petsc with wdn_mods_next branch checked out git pull origin next resolve any conflicts and commit git push origin wdn_mods_next:wdn_mods_next Thanks, Dave -- Dave Nystrom LANL HPC-5 Phone: 505-667-7913 Email: wdn at lanl.gov Smail: Mail Stop B272 Group HPC-5 Los Alamos National Laboratory Los Alamos, NM 87545 ________________________________________ From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Satish Balay [balay at mcs.anl.gov] Sent: Thursday, October 02, 2014 7:38 AM To: petsc-dev at mcs.anl.gov; petsc-users at mcs.anl.gov Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! On Thu, 2 Oct 2014, Satish Balay wrote: > petsc git users, > > If you are using next branch from git repository for integration or > testing or working with it in some form - i.e if you have 'next' > listed by: > > git branch > > Please delete and recreate it. i.e: > > git checkout master > git branch -D next > git checkout next > > From my check - the following feature branches are currently in next > (not yet merged to master). > > Feature branch owners, please remerge your feature branches into to > next - as you see fit. BTW: Some of you might want to rebase/cleanup some of the feature branches before merging to new next. Also some of these feature branches are from pre-petsc-3.5 timeframe. So a rebase to latest master might be something to consider. [if they are still in line for a merge to master] Satish > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > origin/barry/cleanup-blaslapack > origin/barry/remove-dreal > origin/barry/remove-sidl > origin/barry/reuse-hwloc > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > origin/dmeiser/pullrequest-remove-thrust-py > origin/jed/sr-driver4 > origin/jed/vecghostgetvalues > origin/karlrupp/fix-viennacl-veccopy > origin/karpeev/fix-ksp-pcgasm > origin/knepley/feature-optcontrol-mg > origin/knepley/feature-plex-generator-args > origin/knepley/feature-plex-nasm > origin/knepley/fix-configure-petsc-dep > origin/knepley/fix-quadrature-order > origin/madams/sr-driver4 > origin/maint > origin/mlange05/fix-gmsh-coordinates > origin/next > origin/paulmullowney/aijcusp-ellpack-fix > origin/prbrune/sf-examplefix > origin/prbrune/snes-ex19vtkoutput > origin/sarich/fix-tao-jbearing-bqpip > origin/shri/ts-dae-semi-explicit > origin/shri/ts-is-for-differential-variables > origin/shri/ts-powergrid > origin/stefano_zampini/matis-preallocation > origin/stefano_zampini/pcbddc-changeofbasis > origin/stefano_zampini/pcbddc_deluxe > origin/tisaac/feature-fe-geom-dim > origin/tisaac/plex-flexible-projection > > [obviously remove maint, next from the above list] > > The previous 'next' branch is currently available as 'next-oct-2014' - > if you need it for any checks/debugging. > > Satish > > > > From robertrezk at yahoo.ca Thu Oct 2 12:02:32 2014 From: robertrezk at yahoo.ca (Robert Rezk) Date: Thu, 2 Oct 2014 10:02:32 -0700 Subject: [petsc-users] Save matrix to binary file on sub directory Message-ID: <1412269352.41936.YahooMailNeo@web124504.mail.ne1.yahoo.com> I am using the pets viewer and I want to save binary file to sub directory. I tried to save file to the current directory and it work fine. when I try to use file name like this "./xyz_dir/file_name" it doesn't work. Best Regards Robert Andrawis -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Thu Oct 2 12:04:49 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 12:04:49 -0500 Subject: [petsc-users] Save matrix to binary file on sub directory In-Reply-To: <1412269352.41936.YahooMailNeo@web124504.mail.ne1.yahoo.com> References: <1412269352.41936.YahooMailNeo@web124504.mail.ne1.yahoo.com> Message-ID: <87mw9etnbi.fsf@jedbrown.org> Robert Rezk writes: > I am using the pets viewer and I want to save binary file to sub directory. > > I tried to save file to the current directory and it work fine. > > when I try to use file name like this "./xyz_dir/file_name" it doesn't work. What symptoms do you see? Can you reproduce with an example? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From thronesf at gmail.com Thu Oct 2 12:08:56 2014 From: thronesf at gmail.com (Sharp Stone) Date: Thu, 2 Oct 2014 13:08:56 -0400 Subject: [petsc-users] How to use In-Reply-To: References: Message-ID: Matt, Thank you very much! The ex19 used the Gauss-Seidel method. But I'm a little bit wonder whether we have any examples of Jacobi method to solve such linear systems "(mat)A dot (vec)X = (vec)B" with dof>1? Thanks in advance! On Tue, Sep 30, 2014 at 10:15 PM, Matthew Knepley wrote: > On Tue, Sep 30, 2014 at 9:13 PM, Sharp Stone wrote: > >> Hi all, >> >> I have four differential equations to be solved with a linear sparse >> matrix system that has the form of "(mat)A dot (vec)X = (vec)B". For each >> node, I have dof=4. I found few tutorials or examples on the KSPSolve with >> dof>1, and I do know it's possible to solve this problem with Petsc. Are >> there any sources illustrating ksp solver with dof>1? Many thanks! >> > > Yes, try SNES ex19. > > Thanks, > > Matt > > >> Sorry for the stupid questions. >> >> -- >> Best regards, >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 2 12:11:36 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 2 Oct 2014 12:11:36 -0500 Subject: [petsc-users] How to use In-Reply-To: References: Message-ID: On Thu, Oct 2, 2014 at 12:08 PM, Sharp Stone wrote: > Matt, > > Thank you very much! > > The ex19 used the Gauss-Seidel method. But I'm a little bit wonder whether > we have any examples of Jacobi method to solve such linear systems "(mat)A > dot (vec)X = (vec)B" with dof>1? Thanks in advance! > Try -pc_type jacobi Matt > On Tue, Sep 30, 2014 at 10:15 PM, Matthew Knepley > wrote: > >> On Tue, Sep 30, 2014 at 9:13 PM, Sharp Stone wrote: >> >>> Hi all, >>> >>> I have four differential equations to be solved with a linear sparse >>> matrix system that has the form of "(mat)A dot (vec)X = (vec)B". For each >>> node, I have dof=4. I found few tutorials or examples on the KSPSolve with >>> dof>1, and I do know it's possible to solve this problem with Petsc. Are >>> there any sources illustrating ksp solver with dof>1? Many thanks! >>> >> >> Yes, try SNES ex19. >> >> Thanks, >> >> Matt >> >> >>> Sorry for the stupid questions. >>> >>> -- >>> Best regards, >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Thu Oct 2 12:12:05 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 12:12:05 -0500 Subject: [petsc-users] How to use In-Reply-To: References: Message-ID: <87k34itmze.fsf@jedbrown.org> Sharp Stone writes: > Matt, > > Thank you very much! > > The ex19 used the Gauss-Seidel method. But I'm a little bit wonder whether > we have any examples of Jacobi method to solve such linear systems "(mat)A > dot (vec)X = (vec)B" with dof>1? Thanks in advance! ex19 does whatever you want: -pc_type jacobi and the like. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From robertrezk at yahoo.ca Thu Oct 2 12:17:41 2014 From: robertrezk at yahoo.ca (Robert Rezk) Date: Thu, 2 Oct 2014 10:17:41 -0700 Subject: [petsc-users] Save matrix to binary file on sub directory In-Reply-To: <87mw9etnbi.fsf@jedbrown.org> References: <1412269352.41936.YahooMailNeo@web124504.mail.ne1.yahoo.com> <87mw9etnbi.fsf@jedbrown.org> Message-ID: <1412270261.53059.YahooMailNeo@web124502.mail.ne1.yahoo.com> This is the code I used PetscViewer viewer; PetscErrorCode ierr; PetscViewerBinaryOpen(comm_,File_Name.c_str(),FILE_MODE_WRITE,&viewer); ierr = MatView(*petsc_matrix, viewer); PetscViewerDestroy(&viewer); and I can't see any output or crash Best Regards Robert Andrawis ________________________________ From: Jed Brown To: Robert Rezk ; "petsc-users at mcs.anl.gov" Sent: Thursday, October 2, 2014 1:04 PM Subject: Re: [petsc-users] Save matrix to binary file on sub directory Robert Rezk writes: > I am using the pets viewer and I want to save binary file to sub directory. > > I tried to save file to the current directory and it work fine. > > when I try to use file name like this "./xyz_dir/file_name" it doesn't work. What symptoms do you see? Can you reproduce with an example? -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Thu Oct 2 12:26:15 2014 From: thronesf at gmail.com (Sharp Stone) Date: Thu, 2 Oct 2014 13:26:15 -0400 Subject: [petsc-users] How to use In-Reply-To: <87k34itmze.fsf@jedbrown.org> References: <87k34itmze.fsf@jedbrown.org> Message-ID: Hi Matt and Jed, Thank you very much for your replies. In source code ex19, the routine SNESSetNGS() is used, and no other options or Jacobi routines are used. How can we change the iteration method? The routine SNESSetFromOptions(snes) could do this for the user? If I understand correctly, petsc would use Gauss-Seidel or Jacobi method as designated with even the same source code? Thank you in advance! On Thu, Oct 2, 2014 at 1:12 PM, Jed Brown wrote: > Sharp Stone writes: > > > Matt, > > > > Thank you very much! > > > > The ex19 used the Gauss-Seidel method. But I'm a little bit wonder > whether > > we have any examples of Jacobi method to solve such linear systems > "(mat)A > > dot (vec)X = (vec)B" with dof>1? Thanks in advance! > > ex19 does whatever you want: -pc_type jacobi and the like. > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From patrick.sanan at gmail.com Thu Oct 2 12:34:55 2014 From: patrick.sanan at gmail.com (Patrick Sanan) Date: Thu, 02 Oct 2014 19:34:55 +0200 Subject: [petsc-users] How to use In-Reply-To: References: <87k34itmze.fsf@jedbrown.org> Message-ID: <542D8CBF.2030109@gmail.com> On 10/2/14 7:26 PM, Sharp Stone wrote: > Hi Matt and Jed, > > Thank you very much for your replies. In source code ex19, the routine > SNESSetNGS() is used, and no other options or Jacobi routines are > used. How can we change the iteration method? The routine > SNESSetFromOptions(snes) could do this for the user? If I understand > correctly, petsc would use Gauss-Seidel or Jacobi method as designated > with even the same source code? Yes, and this applies for other objects as well. The important thing, in general, is to call XXXSetFromOptions after you have set any default options with XXXSetYYY, but before you call XXXSetUp (which does things like allocate memory). This allows you to set whichever defaults make sense, and also control things from the command line. > > Thank you in advance! > > On Thu, Oct 2, 2014 at 1:12 PM, Jed Brown > wrote: > > Sharp Stone > writes: > > > Matt, > > > > Thank you very much! > > > > The ex19 used the Gauss-Seidel method. But I'm a little bit > wonder whether > > we have any examples of Jacobi method to solve such linear > systems "(mat)A > > dot (vec)X = (vec)B" with dof>1? Thanks in advance! > > ex19 does whatever you want: -pc_type jacobi and the like. > > > > > -- > Best regards, > > Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulhuaizhang at gmail.com Thu Oct 2 12:52:29 2014 From: paulhuaizhang at gmail.com (paul zhang) Date: Thu, 2 Oct 2014 13:52:29 -0400 Subject: [petsc-users] More stable solver Message-ID: Hello, I am modelling a hypersonic computational fluid dynamic problem, where the matrix A is quite sparse and stiff. The solution is x=[rho_1, rho_2, .., rho_11, u, v, w, T], where rho_s is the density for each of the species, some of which can reach 1.E-5, while some others can be below 1.E-20. As I ran my case, I found the number of iterations oscillated after being steady for a long time. For instance, the number of iterations was like 10 for 2000 steps, then it goes up to 50 for no reason. Then the code blows up. My code has being debugged for months. It should be fine. I was wondering if there is some other method that can be more stable than FGMRES or some method similar in terms of stability that I can try. Some code pieces are attached for checking. Many thanks, Paul void petsc_init(void) { vector::iterator cit; vector::iterator it; //Create nonlinear solver context KSPCreate(PETSC_COMM_WORLD,&ksp); VecCreateMPI(PETSC_COMM_WORLD,grid[gid].cellCount*nVars,grid[gid].globalCellCount*nVars,&rhs); VecSetFromOptions(rhs); VecDuplicate(rhs,&deltaU); if (ps_step_max>1) { VecDuplicate(rhs,&soln_n); VecDuplicate(rhs,&pseudo_delta); VecDuplicate(rhs,&pseudo_right); } VecSet(rhs,0.); VecSet(deltaU,0.); vector diagonal_nonzeros, off_diagonal_nonzeros; int nextCellCount; // Calculate space necessary for matrix memory allocation for (cit=grid[gid].cell.begin();cit!=grid[gid].cell.end();cit++) { nextCellCount=0; for (it=(*cit).faces.begin();it!=(*cit).faces.end();it++) { if (grid[gid].face[*it].bc==INTERNAL_FACE) { nextCellCount++; } } for (int i=0;i -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Thu Oct 2 12:56:28 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 12:56:28 -0500 Subject: [petsc-users] More stable solver In-Reply-To: References: Message-ID: <8738b6tkxf.fsf@jedbrown.org> paul zhang writes: > Hello, > > I am modelling a hypersonic computational fluid dynamic problem, where the > matrix A is quite sparse and stiff. The solution is x=[rho_1, rho_2, .., > rho_11, u, v, w, T], where rho_s is the density for each of the species, > some of which can reach 1.E-5, while some others can be below 1.E-20. > As I ran my case, I found the number of iterations oscillated after being > steady for a long time. For instance, the number of iterations was like 10 > for 2000 steps, then it goes up to 50 for no reason. Computers usually don't do things "for no reason". I suggest that you write out the problem state immediately before and after the problem gets more difficult. It can be useful to visualize that state, but also to debug convergence using -ksp_monitor_true_residual, -ksp_monitor_singular_value and similar. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From thronesf at gmail.com Thu Oct 2 13:57:03 2014 From: thronesf at gmail.com (Sharp Stone) Date: Thu, 2 Oct 2014 14:57:03 -0400 Subject: [petsc-users] How to use In-Reply-To: References: <87k34itmze.fsf@jedbrown.org> Message-ID: Hi all, I'm sorry I'm still confusing that if I have calculated the Jacobian elements, how can I go ahead to solve the linear system with dof>1 at each node? The example code ex19 solves the linear system with Guass-Seidel method, and now I do want to use Jacobian matrix because I've found the Jacobian elements. Any ideas? Many thanks! On Thu, Oct 2, 2014 at 1:26 PM, Sharp Stone wrote: > Hi Matt and Jed, > > Thank you very much for your replies. In source code ex19, the routine > SNESSetNGS() is used, and no other options or Jacobi routines are used. How > can we change the iteration method? The routine SNESSetFromOptions(snes) > could do this for the user? If I understand correctly, petsc would use > Gauss-Seidel or Jacobi method as designated with even the same source code? > > Thank you in advance! > > On Thu, Oct 2, 2014 at 1:12 PM, Jed Brown wrote: > >> Sharp Stone writes: >> >> > Matt, >> > >> > Thank you very much! >> > >> > The ex19 used the Gauss-Seidel method. But I'm a little bit wonder >> whether >> > we have any examples of Jacobi method to solve such linear systems >> "(mat)A >> > dot (vec)X = (vec)B" with dof>1? Thanks in advance! >> >> ex19 does whatever you want: -pc_type jacobi and the like. >> > > > > -- > Best regards, > > Feng > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Thu Oct 2 14:02:05 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 14:02:05 -0500 Subject: [petsc-users] How to use In-Reply-To: References: <87k34itmze.fsf@jedbrown.org> Message-ID: <87r3yqs3bm.fsf@jedbrown.org> Sharp Stone writes: > Hi all, > > I'm sorry I'm still confusing that if I have calculated the Jacobian > elements, how can I go ahead to solve the linear system with dof>1 at each > node? The example code ex19 solves the linear system with Guass-Seidel > method, As we just said, the same code (ex19.c) can use many different methods. The default method is Newton-Krylov with block Jacobi preconditioning. Run with -snes_view to see details. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From thronesf at gmail.com Thu Oct 2 14:06:22 2014 From: thronesf at gmail.com (Sharp Stone) Date: Thu, 2 Oct 2014 15:06:22 -0400 Subject: [petsc-users] How to use In-Reply-To: <87r3yqs3bm.fsf@jedbrown.org> References: <87k34itmze.fsf@jedbrown.org> <87r3yqs3bm.fsf@jedbrown.org> Message-ID: Hi Jed, Thank you very much! On Thu, Oct 2, 2014 at 3:02 PM, Jed Brown wrote: > Sharp Stone writes: > > > Hi all, > > > > I'm sorry I'm still confusing that if I have calculated the Jacobian > > elements, how can I go ahead to solve the linear system with dof>1 at > each > > node? The example code ex19 solves the linear system with Guass-Seidel > > method, > > As we just said, the same code (ex19.c) can use many different methods. > The default method is Newton-Krylov with block Jacobi preconditioning. > Run with -snes_view to see details. > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 2 14:44:06 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 2 Oct 2014 14:44:06 -0500 Subject: [petsc-users] Save matrix to binary file on sub directory In-Reply-To: <1412270261.53059.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1412269352.41936.YahooMailNeo@web124504.mail.ne1.yahoo.com> <87mw9etnbi.fsf@jedbrown.org> <1412270261.53059.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: On Thu, Oct 2, 2014 at 12:17 PM, Robert Rezk wrote: > This is the code I used > PetscViewer viewer; > PetscErrorCode ierr; > > PetscViewerBinaryOpen(comm_,File_Name.c_str(),FILE_MODE_WRITE,&viewer); > > Can you check all return codes with CHKERRQ(ierr)? Thanks, Matt ierr = MatView(*petsc_matrix, viewer); > > PetscViewerDestroy(&viewer); > > and I can't see any output or crash > > Best Regards > Robert Andrawis > > > > ------------------------------ > *From:* Jed Brown > *To:* Robert Rezk ; "petsc-users at mcs.anl.gov" < > petsc-users at mcs.anl.gov> > *Sent:* Thursday, October 2, 2014 1:04 PM > *Subject:* Re: [petsc-users] Save matrix to binary file on sub directory > > Robert Rezk writes: > > > > I am using the pets viewer and I want to save binary file to sub > directory. > > > > I tried to save file to the current directory and it work fine. > > > > when I try to use file name like this "./xyz_dir/file_name" it doesn't > work. > > > What symptoms do you see? Can you reproduce with an example? > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Thu Oct 2 15:29:46 2014 From: thronesf at gmail.com (Sharp Stone) Date: Thu, 2 Oct 2014 16:29:46 -0400 Subject: [petsc-users] How to use In-Reply-To: <87r3yqs3bm.fsf@jedbrown.org> References: <87k34itmze.fsf@jedbrown.org> <87r3yqs3bm.fsf@jedbrown.org> Message-ID: Hi all, In the ex19, specifically from Lines 581-587, about the "invert the system" part, 581: /* invert the system:582: [ dfu / du 0 0 0 ][yu] = [fu]583: [ 0 dfv / dv 0 0 ][yv] [fv]584: [ dfo / du dfo / dv dfo / do 0 ][yo] [fo]585: [ dft / du dft / dv 0 dft / dt ][yt] [ft]586: by simple back-substitution587: */ I realized u and v are independent variables, but even so, the first two rows should be both containing dfu/dv and dfv/du terms just like the 3rd and 4th rows, do I miss some points? Thank you in advance! On Thu, Oct 2, 2014 at 3:02 PM, Jed Brown wrote: > Sharp Stone writes: > > > Hi all, > > > > I'm sorry I'm still confusing that if I have calculated the Jacobian > > elements, how can I go ahead to solve the linear system with dof>1 at > each > > node? The example code ex19 solves the linear system with Guass-Seidel > > method, > > As we just said, the same code (ex19.c) can use many different methods. > The default method is Newton-Krylov with block Jacobi preconditioning. > Run with -snes_view to see details. > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From wdn at lanl.gov Thu Oct 2 15:35:48 2014 From: wdn at lanl.gov (Nystrom, William David) Date: Thu, 2 Oct 2014 20:35:48 +0000 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: , , Message-ID: What is meant by the phrase "petsc git next branch is unwound"? What git commands are run to do this? And how is the next branch different when it is unwound from immediately before it is unwound? Thanks, Dave -- Dave Nystrom LANL HPC-5 Phone: 505-667-7913 Email: wdn at lanl.gov Smail: Mail Stop B272 Group HPC-5 Los Alamos National Laboratory Los Alamos, NM 87545 ________________________________________ From: Nystrom, William David Sent: Thursday, October 02, 2014 10:57 AM To: petsc-users; petsc-dev at mcs.anl.gov Cc: Nystrom, William David Subject: RE: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! On my local machine I have my master, bare git repo with the following config file which Jed suggested a year or so ago: % cat config [core] repositoryformatversion = 0 filemode = true bare = true sharedrepository = 1 [receive] denyNonFastforwards = true [remote "origin"] url = https://bitbucket.org/petsc/petsc fetch = +refs/heads/maint:refs/heads/maint fetch = +refs/heads/master:refs/heads/master fetch = +refs/heads/next:refs/heads/next [remote "artab"] url = ssh://ml-fey.lanl.gov/usr/projects/artab/users/wdn/repos/git/petsc fetch = +refs/*:refs/* mirror = true And for my workflow, I stay in sync with the official PETSc git repo by periodically executing the following command: git remote -v update origin And then I have various git clones from my master git repo where I do actual work. Do I need to do anything special now that the "next" branch has been unwound with either my master git repo or my git clones? In my master git repo, I have 3 additional branches - wdn_mods_next, wdn_mods_master and wdn_mods_maint where I keep any of my own modifications to those respective branches. And I keep wdn_mods_next in sync with next by periodically executing the following commands in a git clone with the wdn_mods_next branch checked out: git pull origin next resolve any conflicts and commit git push origin wdn_mods_next:wdn_mods_next So, if I do the following set of steps, should I expect them to work now that next has been unwound? 1. cd to my master bare petsc repo git remote -v update origin 2. cd to my git clone of petsc with wdn_mods_next branch checked out git pull origin next resolve any conflicts and commit git push origin wdn_mods_next:wdn_mods_next Thanks, Dave -- Dave Nystrom LANL HPC-5 Phone: 505-667-7913 Email: wdn at lanl.gov Smail: Mail Stop B272 Group HPC-5 Los Alamos National Laboratory Los Alamos, NM 87545 ________________________________________ From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Satish Balay [balay at mcs.anl.gov] Sent: Thursday, October 02, 2014 7:38 AM To: petsc-dev at mcs.anl.gov; petsc-users at mcs.anl.gov Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! On Thu, 2 Oct 2014, Satish Balay wrote: > petsc git users, > > If you are using next branch from git repository for integration or > testing or working with it in some form - i.e if you have 'next' > listed by: > > git branch > > Please delete and recreate it. i.e: > > git checkout master > git branch -D next > git checkout next > > From my check - the following feature branches are currently in next > (not yet merged to master). > > Feature branch owners, please remerge your feature branches into to > next - as you see fit. BTW: Some of you might want to rebase/cleanup some of the feature branches before merging to new next. Also some of these feature branches are from pre-petsc-3.5 timeframe. So a rebase to latest master might be something to consider. [if they are still in line for a merge to master] Satish > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > origin/barry/cleanup-blaslapack > origin/barry/remove-dreal > origin/barry/remove-sidl > origin/barry/reuse-hwloc > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > origin/dmeiser/pullrequest-remove-thrust-py > origin/jed/sr-driver4 > origin/jed/vecghostgetvalues > origin/karlrupp/fix-viennacl-veccopy > origin/karpeev/fix-ksp-pcgasm > origin/knepley/feature-optcontrol-mg > origin/knepley/feature-plex-generator-args > origin/knepley/feature-plex-nasm > origin/knepley/fix-configure-petsc-dep > origin/knepley/fix-quadrature-order > origin/madams/sr-driver4 > origin/maint > origin/mlange05/fix-gmsh-coordinates > origin/next > origin/paulmullowney/aijcusp-ellpack-fix > origin/prbrune/sf-examplefix > origin/prbrune/snes-ex19vtkoutput > origin/sarich/fix-tao-jbearing-bqpip > origin/shri/ts-dae-semi-explicit > origin/shri/ts-is-for-differential-variables > origin/shri/ts-powergrid > origin/stefano_zampini/matis-preallocation > origin/stefano_zampini/pcbddc-changeofbasis > origin/stefano_zampini/pcbddc_deluxe > origin/tisaac/feature-fe-geom-dim > origin/tisaac/plex-flexible-projection > > [obviously remove maint, next from the above list] > > The previous 'next' branch is currently available as 'next-oct-2014' - > if you need it for any checks/debugging. > > Satish > > > > From jed at jedbrown.org Thu Oct 2 16:26:01 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 16:26:01 -0500 Subject: [petsc-users] How to use In-Reply-To: References: <87k34itmze.fsf@jedbrown.org> <87r3yqs3bm.fsf@jedbrown.org> Message-ID: <87fvf6rwnq.fsf@jedbrown.org> Sharp Stone writes: > Hi all, > > In the ex19, specifically from Lines 581-587, about the "invert the system" > part, > > 581: /* invert the system:582: [ dfu / du > 0 0 0 ][yu] = [fu]583: [ 0 dfv / > dv 0 0 ][yv] [fv]584: [ dfo / du dfo / dv > dfo / do 0 ][yo] [fo]585: [ dft / du dft / dv > 0 dft / dt ][yt] [ft]586: by simple > back-substitution587: */ > > I realized u and v are independent variables, but even so, the first two > rows should be both containing dfu/dv and dfv/du terms just like the 3rd > and 4th rows, do I miss some points? Look at the equations defining the residual (e.g., line 337). Those terms are zero. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Thu Oct 2 16:29:27 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 16:29:27 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: Message-ID: <87d2aarwi0.fsf@jedbrown.org> "Nystrom, William David" writes: > What is meant by the phrase "petsc git next branch is unwound"? What git > commands are run to do this? And how is the next branch different when > it is unwound from immediately before it is unwound? git checkout next git reset --hard master # discarding anything in 'next' git push origin +next # force push So the history of branch testing in 'next' was discarded, but 'master' is not changed. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From balay at mcs.anl.gov Thu Oct 2 17:33:02 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Thu, 2 Oct 2014 17:33:02 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: , Message-ID: You would have to delete both 'next' and 'wdn_mods_next' branches from all your clones - and recreate them. Wrt recreating 'next' [as indicated in my previous e-mail] the following should be sufficient: git fetch origin # add the extra option 'origin' because you have multiple remotes git checkout next Wrt recreating wdn_mods_next - presumably you have 'wdn_mods' in some local branch - that you keep arround that you merge to next to get wdn_mods_next.. git checkout next git checkout -b wdn_mods_next git merge wdn_mods If not - perhaps you can cherry/pick/rebase as needed. Or you can create it by merging 'wdn_mods_master' into 'next' [assuming the 'wdn_mods' part is same for all the 3 branches] git checkout next git checkout -b wdn_mods_next git merge wdn_mods_master Satish On Thu, 2 Oct 2014, Nystrom, William David wrote: > On my local machine I have my master, bare git repo with the following config file > which Jed suggested a year or so ago: > > % cat config > [core] > repositoryformatversion = 0 > filemode = true > bare = true > sharedrepository = 1 > [receive] > denyNonFastforwards = true > [remote "origin"] > url = https://bitbucket.org/petsc/petsc > fetch = +refs/heads/maint:refs/heads/maint > fetch = +refs/heads/master:refs/heads/master > fetch = +refs/heads/next:refs/heads/next > [remote "artab"] > url = ssh://ml-fey.lanl.gov/usr/projects/artab/users/wdn/repos/git/petsc > fetch = +refs/*:refs/* > mirror = true > > And for my workflow, I stay in sync with the official PETSc git repo by periodically > executing the following command: > > git remote -v update origin > > And then I have various git clones from my master git repo where I do actual > work. > > Do I need to do anything special now that the "next" branch has been unwound > with either my master git repo or my git clones? In my master git repo, I have > 3 additional branches - wdn_mods_next, wdn_mods_master and wdn_mods_maint > where I keep any of my own modifications to those respective branches. And I > keep wdn_mods_next in sync with next by periodically executing the following > commands in a git clone with the wdn_mods_next branch checked out: > > git pull origin next > > resolve any conflicts and commit > > git push origin wdn_mods_next:wdn_mods_next > > So, if I do the following set of steps, should I expect them to work now that next > has been unwound? > > 1. cd to my master bare petsc repo > git remote -v update origin > > 2. cd to my git clone of petsc with wdn_mods_next branch checked out > git pull origin next > resolve any conflicts and commit > git push origin wdn_mods_next:wdn_mods_next > > Thanks, > > Dave > > -- > Dave Nystrom > LANL HPC-5 > Phone: 505-667-7913 > Email: wdn at lanl.gov > Smail: Mail Stop B272 > Group HPC-5 > Los Alamos National Laboratory > Los Alamos, NM 87545 > > > ________________________________________ > From: petsc-dev-bounces at mcs.anl.gov [petsc-dev-bounces at mcs.anl.gov] on behalf of Satish Balay [balay at mcs.anl.gov] > Sent: Thursday, October 02, 2014 7:38 AM > To: petsc-dev at mcs.anl.gov; petsc-users at mcs.anl.gov > Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! > > On Thu, 2 Oct 2014, Satish Balay wrote: > > > petsc git users, > > > > If you are using next branch from git repository for integration or > > testing or working with it in some form - i.e if you have 'next' > > listed by: > > > > git branch > > > > Please delete and recreate it. i.e: > > > > git checkout master > > git branch -D next > > git checkout next > > > > From my check - the following feature branches are currently in next > > (not yet merged to master). > > > > Feature branch owners, please remerge your feature branches into to > > next - as you see fit. > > BTW: Some of you might want to rebase/cleanup some of the feature branches > before merging to new next. > > Also some of these feature branches are from pre-petsc-3.5 timeframe. > So a rebase to latest master might be something to consider. [if they > are still in line for a merge to master] > > Satish > > > > > $ comm -12 <(git branch -r --merged next | sort) <(git branch -r --no-merged master | sort) > > origin/barry/cleanup-blaslapack > > origin/barry/remove-dreal > > origin/barry/remove-sidl > > origin/barry/reuse-hwloc > > origin/dmeiser/pullrequest-fix-cusp-bjacobi2 > > origin/dmeiser/pullrequest-remove-thrust-py > > origin/jed/sr-driver4 > > origin/jed/vecghostgetvalues > > origin/karlrupp/fix-viennacl-veccopy > > origin/karpeev/fix-ksp-pcgasm > > origin/knepley/feature-optcontrol-mg > > origin/knepley/feature-plex-generator-args > > origin/knepley/feature-plex-nasm > > origin/knepley/fix-configure-petsc-dep > > origin/knepley/fix-quadrature-order > > origin/madams/sr-driver4 > > origin/maint > > origin/mlange05/fix-gmsh-coordinates > > origin/next > > origin/paulmullowney/aijcusp-ellpack-fix > > origin/prbrune/sf-examplefix > > origin/prbrune/snes-ex19vtkoutput > > origin/sarich/fix-tao-jbearing-bqpip > > origin/shri/ts-dae-semi-explicit > > origin/shri/ts-is-for-differential-variables > > origin/shri/ts-powergrid > > origin/stefano_zampini/matis-preallocation > > origin/stefano_zampini/pcbddc-changeofbasis > > origin/stefano_zampini/pcbddc_deluxe > > origin/tisaac/feature-fe-geom-dim > > origin/tisaac/plex-flexible-projection > > > > [obviously remove maint, next from the above list] > > > > The previous 'next' branch is currently available as 'next-oct-2014' - > > if you need it for any checks/debugging. > > > > Satish > > > > > > > > > > From jed at jedbrown.org Thu Oct 2 17:39:50 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 02 Oct 2014 17:39:50 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: Message-ID: <871tqqrt8p.fsf@jedbrown.org> Satish Balay writes: > You would have to delete both 'next' and 'wdn_mods_next' branches from > all your clones - and recreate them. Do your normal "git remote update" (or git fetch) in your "master" clone (confusing to use the same name to refer to a repository and a branch). I'll call the repository Mirror. Anyway, you should see that all three branches are updated. 'maint' and 'master' should fast-forward, while 'next' will say "forced update". After that, recreate 'next' on the clones just like in Satish's original email. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From zonexo at gmail.com Thu Oct 2 19:25:44 2014 From: zonexo at gmail.com (TAY wee-beng) Date: Fri, 03 Oct 2014 08:25:44 +0800 Subject: [petsc-users] Check for inf or nan in PETSc matrix Message-ID: <542DED08.40903@gmail.com> Hi, I have a PETSc matrix. How can I check for inf or nan in its values? I understand that there's a subroutine "PetscIsInfOrNanScalar" but not sure how to use it. -- Thank you. Yours sincerely, TAY wee-beng From zonexo at gmail.com Fri Oct 3 02:13:20 2014 From: zonexo at gmail.com (TAY wee-beng) Date: Fri, 03 Oct 2014 15:13:20 +0800 Subject: [petsc-users] How to check where code hangs Message-ID: <542E4C90.8050302@gmail.com> Hi, This qn is not PETSc related but I hope to get some ans from experienced users. I'm running an MPI code which uses 144 cpu. It aborts after a short while. I'm trying to find out where it aborts exactly. However, being an MPI code, it seems quite difficult. I used: call MPI_Barrier(MPI_COMM_WORLD,ierr); if (myid==0) print *, "xxx" where xxx = 1,2,3 .... If it prints up to 3, I know it aborts between 3 and 4. However, it doesn't seem to work as supposed. I wonder why. Also is there a better way to do this? -- Thank you. Yours sincerely, TAY wee-beng From bsmith at mcs.anl.gov Fri Oct 3 07:22:54 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 3 Oct 2014 07:22:54 -0500 Subject: [petsc-users] How to check where code hangs In-Reply-To: <542E4C90.8050302@gmail.com> References: <542E4C90.8050302@gmail.com> Message-ID: You really have to use a debugger. When it is ?hanging? you interrupt it and type bt. Or if you are lucky you can use total view. Unfortunately getting access to the debugger on a batch system is often difficult. If your nodes have access to X windows and xterm you can run the PETSc program with, for example, -start_in_debugger -display $DISPLAY -debugger_nodes 0 to have only the 0th MPI process start up the debugger. You can also use -debugger_nodes 0,2,8 for example to start a debugger on 3 different MPI processes. Barry On Oct 3, 2014, at 2:13 AM, TAY wee-beng wrote: > Hi, > > This qn is not PETSc related but I hope to get some ans from experienced users. > > I'm running an MPI code which uses 144 cpu. It aborts after a short while. I'm trying to find out where it aborts exactly. > > However, being an MPI code, it seems quite difficult. > > I used: > > call MPI_Barrier(MPI_COMM_WORLD,ierr); if (myid==0) print *, "xxx" > > where xxx = 1,2,3 .... > > If it prints up to 3, I know it aborts between 3 and 4. However, it doesn't seem to work as supposed. > > I wonder why. > > Also is there a better way to do this? > > -- > Thank you. > > Yours sincerely, > > TAY wee-beng > From asmund.ervik at ntnu.no Fri Oct 3 10:22:23 2014 From: asmund.ervik at ntnu.no (=?utf-8?B?w4VzbXVuZCBFcnZpaw==?=) Date: Fri, 3 Oct 2014 15:22:23 +0000 Subject: [petsc-users] Penalization vs. setting interior points for Laplace eqn Message-ID: <0E576811AB298343AC632BBCAAEFC37945DC4A6B@WAREHOUSE08.win.ntnu.no> Dear PETSc users, (This may not be very PETSc-specific, but here goes.) I'm using KSP for solving two equations of Poisson/Laplace type. First I solve a Laplace equation in a box with Dirichlet BCs on the top/bottom and Neumann on the sides, and discontinuous coefficients inside the box. This gives me an electric potential. I then use this as one input in a two-phase incompressible Navier-Stokes solver in the same box domain, again with discontinuous coefficients, where the electric potential contributes to part of the pressure jump across the phase boundaries. Here I solve a pressure Poisson equation. This code gives good results which agree with theoretical results and experiments. Now I want to do simulations in a complex domain. I have already implemented L2 penalization for the flow part, without an electric field, and this works well. Now I want to be able to set a constant potential in the part of the domain where I apply penalization. Does it make sense to do penalization for the Laplace equation somehow, or should I just set the potential values on the grid points where I want to? Regards, ?smund From bsmith at mcs.anl.gov Fri Oct 3 10:33:03 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 3 Oct 2014 10:33:03 -0500 Subject: [petsc-users] Penalization vs. setting interior points for Laplace eqn In-Reply-To: <0E576811AB298343AC632BBCAAEFC37945DC4A6B@WAREHOUSE08.win.ntnu.no> References: <0E576811AB298343AC632BBCAAEFC37945DC4A6B@WAREHOUSE08.win.ntnu.no> Message-ID: <35D07FB8-31C4-441B-AF54-787568798C2C@mcs.anl.gov> I think you can just set the values there. The advantage is simplicity plus it won?t mess up the convergence of the iterative methods as a penalization likely would. Barry On Oct 3, 2014, at 10:22 AM, ?smund Ervik wrote: > Dear PETSc users, > > (This may not be very PETSc-specific, but here goes.) > > I'm using KSP for solving two equations of Poisson/Laplace type. First I solve a Laplace equation in a box with Dirichlet BCs on the top/bottom and Neumann on the sides, and discontinuous coefficients inside the box. This gives me an electric potential. I then use this as one input in a two-phase incompressible Navier-Stokes solver in the same box domain, again with discontinuous coefficients, where the electric potential contributes to part of the pressure jump across the phase boundaries. Here I solve a pressure Poisson equation. This code gives good results which agree with theoretical results and experiments. > > Now I want to do simulations in a complex domain. I have already implemented L2 penalization for the flow part, without an electric field, and this works well. Now I want to be able to set a constant potential in the part of the domain where I apply penalization. Does it make sense to do penalization for the Laplace equation somehow, or should I just set the potential values on the grid points where I want to? > > Regards, > ?smund From bsmith at mcs.anl.gov Fri Oct 3 13:20:53 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 3 Oct 2014 13:20:53 -0500 Subject: [petsc-users] Check for inf or nan in PETSc matrix In-Reply-To: <542DED08.40903@gmail.com> References: <542DED08.40903@gmail.com> Message-ID: <1F437C1F-55A0-405F-A2E0-8D0772113415@mcs.anl.gov> It is possible to check an individual number to see if it has a Nan or Inf with PetscIsInfOrNanScalar() but we don?t have code for checking in a matrix directly. For vectors you can use VecGetArrayRead() and loop over the local entries with PetscIsInfOrNanScalar() For dense matrices you can use MatDenseGetArray() or MatSeqAIJGetArray(). For MPIAIJ matrices you can use MatMPIAIJGetSeqAIJ() and then do the test on each of the two SeqAIJ matrices. You can do the same thing with BAIJ matrices by using BAIJ in the above calls instead of AIJ. Barry On Oct 2, 2014, at 7:25 PM, TAY wee-beng wrote: > Hi, > > I have a PETSc matrix. How can I check for inf or nan in its values? > > I understand that there's a subroutine "PetscIsInfOrNanScalar" but not sure how to use it. > > -- > Thank you. > > Yours sincerely, > > TAY wee-beng > From bsmith at mcs.anl.gov Sat Oct 4 14:27:54 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 4 Oct 2014 14:27:54 -0500 Subject: [petsc-users] Looking for images generated from PETSc simulations Message-ID: <99CE38AA-95BA-4E56-8123-0CCC36DECF5C@mcs.anl.gov> Users of PETSc, We are looking for images of simulations (and shorted movies) that benefited from the use of PETSc for our website as well as our various programming managers who provide support for the development of PETSc. If you have some images we?d appreciate you sending them or a link to them to petsc-maint at mcs.anl.gov with a very short description, maybe a URL, and your name and institution information. Thanks Barry You can also send us additions to our partial list of publications that use PETSc at http://www.mcs.anl.gov/petsc/publications/index.html From jed at jedbrown.org Sat Oct 4 18:56:49 2014 From: jed at jedbrown.org (Jed Brown) Date: Sat, 04 Oct 2014 17:56:49 -0600 Subject: [petsc-users] Penalization vs. setting interior points for Laplace eqn In-Reply-To: <35D07FB8-31C4-441B-AF54-787568798C2C@mcs.anl.gov> References: <0E576811AB298343AC632BBCAAEFC37945DC4A6B@WAREHOUSE08.win.ntnu.no> <35D07FB8-31C4-441B-AF54-787568798C2C@mcs.anl.gov> Message-ID: <87h9zjqtha.fsf@jedbrown.org> Barry Smith writes: > I think you can just set the values there. The advantage is > simplicity plus it won?t mess up the convergence of the iterative > methods as a penalization likely would. Write the trivial equations for those rows (u = "what_you_want") and have all the neighboring equations use what_you_want. That is "symmetric" removal of those equations. See this description with notation. https://scicomp.stackexchange.com/questions/3298/appropriate-space-for-weak-solutions-to-an-elliptical-pde-with-mixed-inhomogeneo/3300#3300 Of course you can use other methods, but I recommend the above because it maintains symmetry, decouples the boundary dofs, and does not introduce poor scaling. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From louis.moresi at unimelb.edu.au Sun Oct 5 13:21:21 2014 From: louis.moresi at unimelb.edu.au (Louis Moresi) Date: Sun, 5 Oct 2014 18:21:21 +0000 Subject: [petsc-users] [petsc-announce] Looking for images generated from PETSc simulations In-Reply-To: <99CE38AA-95BA-4E56-8123-0CCC36DECF5C@mcs.anl.gov> References: <99CE38AA-95BA-4E56-8123-0CCC36DECF5C@mcs.anl.gov> Message-ID: <52D8A1BE-0827-470F-B9E6-68B27EE90A36@unimelb.edu.au> http://youtu.be/cVulRP2tUGM Movie from a recent Nature paper using Underworld. More details available from www.moresi.info Louis Moresi +61 4 0333 1413 (au) +1 505 349 4425 (us) On 5 Oct 2014, at 07:36, Barry Smith > wrote: Users of PETSc, We are looking for images of simulations (and shorted movies) that benefited from the use of PETSc for our website as well as our various programming managers who provide support for the development of PETSc. If you have some images we?d appreciate you sending them or a link to them to petsc-maint at mcs.anl.gov with a very short description, maybe a URL, and your name and institution information. Thanks Barry You can also send us additions to our partial list of publications that use PETSc at http://www.mcs.anl.gov/petsc/publications/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek.gaston at inl.gov Sun Oct 5 19:15:45 2014 From: derek.gaston at inl.gov (Gaston, Derek R) Date: Sun, 5 Oct 2014 20:15:45 -0400 Subject: [petsc-users] [petsc-announce] Looking for images generated from PETSc simulations In-Reply-To: <52D8A1BE-0827-470F-B9E6-68B27EE90A36@unimelb.edu.au> References: <99CE38AA-95BA-4E56-8123-0CCC36DECF5C@mcs.anl.gov> <52D8A1BE-0827-470F-B9E6-68B27EE90A36@unimelb.edu.au> Message-ID: Barry, We have tons of nice pictures generated using MOOSE. Anything you see in these videos is up for grabs: https://www.youtube.com/watch?v=6shfkm3fnLY https://www.youtube.com/watch?v=V-2VfET8SNw https://www.youtube.com/watch?v=4xTfQxpGAI4 Tons more stills of nuclear fuel, geophysics, chemistry, microstructure evolution, etc. Let me know what you're interested in... Derek On Sun, Oct 5, 2014 at 2:21 PM, Louis Moresi wrote: > http://youtu.be/cVulRP2tUGM > > Movie from a recent Nature paper using Underworld. More details > available from www.moresi.info > > Louis Moresi > +61 4 0333 1413 (au) > +1 505 349 4425 (us) > > > > On 5 Oct 2014, at 07:36, Barry Smith wrote: > > > Users of PETSc, > > We are looking for images of simulations (and shorted movies) that > benefited from the use of PETSc for our website as well as our various > programming managers who provide support for the development of PETSc. If > you have some images we?d appreciate you sending them or a link to them to > petsc-maint at mcs.anl.gov with a very short description, maybe a URL, and > your name and institution information. > > Thanks > > Barry > > You can also send us additions to our partial list of publications that > use PETSc at http://www.mcs.anl.gov/petsc/publications/index.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From filippo.leonardi at sam.math.ethz.ch Mon Oct 6 07:23:39 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Mon, 6 Oct 2014 14:23:39 +0200 Subject: [petsc-users] Multigrid iterations for different versions of Petsc Message-ID: <44147693.kW2a8PGp4h@besikovitch-ii> Hi, Again me with multigrid. Sorry for the annoyance, but I am a bit worried about my multigrid solver: I still can't figure out where (if something) my code is doing wrong. I post the details of a run with my local machine and Petsc 3.5 compared with the runs done with Petsc 3.4.3 on a cluster (the most recent version available). The problem is that the run in the cluster (3.4.3) does twice many iteration per krylow solver (as the local 3.5) and goes even worse when increasing the mesh. Also, while on my local machine I get 7 iterations (which is quite nice), on the cluster I get 14 iteration, which is not what I expect from multigrid. Are there been any changes on the MG solver that can influence the behaviour of the solver? http://www.mcs.anl.gov/petsc/documentation/changes/35.html seems not to include anything relevant. Everything is the same: outer iteration with gmres, lu as coarse solver etc (all default), I just adapted the APIs. Thus I'd expect more or less the same result. Best, Filippo I attach the profiles. -------------- next part -------------- 8 processors, by lfilippo Mon Oct 6 13:53:27 2014 Using Petsc Release Version 3.4.3, Oct, 15, 2013 Max Max/Min Avg Total Time (sec): 1.294e+01 1.00000 1.294e+01 Objects: 9.510e+02 1.00000 9.510e+02 Flops: 2.150e+09 1.00000 2.150e+09 1.720e+10 Flops/sec: 1.662e+08 1.00000 1.662e+08 1.329e+09 MPI Messages: 7.237e+04 1.00166 7.227e+04 5.781e+05 MPI Message Lengths: 2.109e+08 1.42512 2.157e+03 1.247e+09 MPI Reductions: 1.333e+04 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.4267e-02 0.1% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 1: StepStage: 3.4571e-02 0.3% 7.0779e+07 0.4% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2: ConvStage: 6.8375e-01 5.3% 3.6176e+08 2.1% 7.200e+03 1.2% 2.040e+02 9.5% 0.000e+00 0.0% 3: ProjStage: 6.1173e-02 0.5% 1.8350e+07 0.1% 9.600e+02 0.2% 2.721e+01 1.3% 0.000e+00 0.0% 4: IoStage: 6.4588e-01 5.0% 0.0000e+00 0.0% 5.680e+02 0.1% 2.621e+02 12.2% 1.000e+02 0.8% 5: KspStage: 1.6636e+00 12.9% 3.4223e+09 19.9% 9.584e+03 1.7% 2.195e+02 10.2% 1.932e+03 14.5% 6: SolvAlloc: 2.6209e-01 2.0% 0.0000e+00 0.0% 2.560e+02 0.0% 1.383e+00 0.1% 3.350e+02 2.5% 7: SolvSolve: 6.9740e-02 0.5% 3.4079e+06 0.0% 1.520e+02 0.0% 4.168e+00 0.2% 3.200e+01 0.2% 8: SolvDeall: 1.5375e-03 0.0% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 9: MG Apply: 9.5009e+00 73.4% 1.3324e+10 77.5% 5.594e+05 96.8% 1.438e+03 66.7% 1.093e+04 82.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: StepStage VecAXPY 120 1.0 3.4878e-02 1.0 8.85e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 99100 0 0 0 2029 --- Event Stage 2: ConvStage VecCopy 180 1.0 6.4032e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 9 0 0 0 0 0 VecAXPY 180 1.0 4.5187e-02 1.2 1.18e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 6 26 0 0 0 2088 VecAXPBYCZ 210 1.0 1.0587e-01 1.1 2.75e+07 1.0 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 15 61 0 0 0 2080 VecPointwiseMult 180 1.0 5.6722e-02 1.0 5.90e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 8 13 0 0 0 832 VecScatterBegin 300 1.0 1.3938e-01 1.0 0.00e+00 0.0 7.2e+03 1.6e+04 0.0e+00 1 0 1 9 0 20 0100100 0 0 VecScatterEnd 300 1.0 3.1792e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 4 0 0 0 0 0 --- Event Stage 3: ProjStage VecScale 10 1.0 5.1546e-04 1.1 3.28e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 14 0 0 0 5086 VecCopy 30 1.0 1.4046e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 23 0 0 0 0 0 VecAXPY 30 1.0 6.9752e-03 1.1 1.97e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 11 86 0 0 0 2255 VecScatterBegin 40 1.0 1.6682e-02 1.0 0.00e+00 0.0 9.6e+02 1.6e+04 0.0e+00 0 0 0 1 0 27 0100100 0 0 VecScatterEnd 40 1.0 4.2324e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 6 0 0 0 0 0 --- Event Stage 4: IoStage VecView 20 1.0 3.5643e-0141.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+01 1 0 0 0 0 29 0 0 0 40 0 VecCopy 20 1.0 3.6664e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 0 0 0 0 0 VecScatterBegin 12 1.0 4.3378e-03 1.6 0.00e+00 0.0 2.9e+02 1.6e+04 0.0e+00 0 0 0 0 0 1 0 51 3 0 0 VecScatterEnd 12 1.0 9.9678e-02107.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 8 0 0 0 0 0 --- Event Stage 5: KspStage VecMDot 154 1.0 1.7173e-01 1.0 7.57e+07 1.0 0.0e+00 0.0e+00 1.5e+02 1 4 0 0 1 10 18 0 0 8 3526 VecNorm 341 1.0 2.2405e-02 1.6 2.23e+07 1.0 0.0e+00 0.0e+00 3.4e+02 0 1 0 0 3 1 5 0 0 18 7980 VecScale 165 1.0 7.3164e-03 1.0 5.41e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 5912 VecCopy 187 1.0 5.1092e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 3 0 0 0 0 0 VecSet 197 1.0 2.8651e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecAXPY 165 1.0 1.6271e-02 1.4 1.08e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 1 3 0 0 0 5317 VecAYPX 165 1.0 3.1347e-02 1.1 5.41e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 1 0 0 0 1380 VecMAXPY 319 1.0 3.8127e-01 1.0 1.61e+08 1.0 0.0e+00 0.0e+00 0.0e+00 3 8 0 0 0 22 38 0 0 0 3388 VecScatterBegin 327 1.0 3.0026e-02 1.2 0.00e+00 0.0 7.9e+03 1.6e+04 0.0e+00 0 0 1 10 0 2 0 82 99 0 0 VecScatterEnd 327 1.0 4.1978e-02 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecNormalize 165 1.0 1.5593e-02 1.1 1.62e+07 1.0 0.0e+00 0.0e+00 1.6e+02 0 1 0 0 1 1 4 0 0 9 8322 MatMult 319 1.0 6.8451e-01 1.0 1.46e+08 1.0 7.7e+03 1.6e+04 0.0e+00 5 7 1 10 0 41 34 80 99 0 1710 MatMultTranspose 4 1.0 1.7753e-03 1.0 2.53e+05 1.0 2.2e+02 3.2e+02 0.0e+00 0 0 0 0 0 0 0 2 0 0 1139 MatLUFactorSym 1 1.0 1.3900e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatLUFactorNum 1 1.0 1.3113e-04 1.1 3.87e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2360 MatAssemblyBegin 15 1.0 3.8953e-03 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 0 0 0 0 0 0 0 0 0 1 0 MatAssemblyEnd 15 1.0 1.3953e-02 1.0 0.00e+00 0.0 6.9e+02 4.3e+02 7.2e+01 0 0 0 0 1 1 0 7 0 4 0 MatGetRowIJ 1 1.0 2.7180e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 8.0824e-05 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 132 1.2 2.6198e-02 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 1.1e+02 0 0 0 0 1 1 0 0 0 6 0 MatGetRedundant 1 1.0 2.2316e-04 1.1 0.00e+00 0.0 1.7e+02 2.5e+02 4.0e+00 0 0 0 0 0 0 0 2 0 0 0 KSPGMRESOrthog 154 1.0 3.1410e-01 1.0 1.51e+08 1.0 0.0e+00 0.0e+00 1.5e+02 2 7 0 0 1 19 35 0 0 8 3856 KSPSetUp 7 1.0 4.5030e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 3.4e+01 0 0 0 0 0 0 0 0 0 2 0 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer KSPSolve 11 1.0 1.1139e+01 1.0 2.09e+09 1.0 5.7e+05 1.7e+03 1.3e+04 86 97 98 77 96 6694895937755660 1503 PCSetUp 1 1.0 6.2445e-02 1.0 2.91e+05 1.0 1.9e+03 6.7e+02 4.2e+02 0 0 0 0 3 4 0 20 1 22 37 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer PCApply 165 1.0 9.5051e+00 1.0 1.67e+09 1.0 5.6e+05 1.5e+03 1.1e+04 73 77 97 67 82 5713895837655566 1402 MGSetup Level 0 1 1.0 1.4729e-03 1.1 3.87e+04 1.0 3.6e+02 1.4e+02 4.9e+01 0 0 0 0 0 0 0 4 0 3 210 MGSetup Level 1 1 1.0 1.0180e-03 1.0 0.00e+00 0.0 4.8e+01 6.6e+01 2.8e+01 0 0 0 0 0 0 0 1 0 1 0 MGSetup Level 2 1 1.0 2.7621e-03 1.0 0.00e+00 0.0 4.8e+01 2.6e+02 2.8e+01 0 0 0 0 0 0 0 1 0 1 0 MGSetup Level 3 1 1.0 1.2487e-02 1.0 0.00e+00 0.0 4.8e+01 1.0e+03 2.8e+01 0 0 0 0 0 1 0 1 0 1 0 MGSetup Level 4 1 1.0 1.5512e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 --- Event Stage 6: SolvAlloc VecSet 50 1.0 1.9866e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 8 0 0 0 0 0 MatAssemblyBegin 4 1.0 4.7951e-03 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 0 1 0 0 0 2 0 MatAssemblyEnd 4 1.0 1.4767e-02 1.0 0.00e+00 0.0 9.6e+01 4.1e+03 1.6e+01 0 0 0 0 0 6 0 38 49 5 0 --- Event Stage 7: SolvSolve VecNorm 30 1.0 3.3298e-02 4.5 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+01 0 0 0 0 0 29 0 0 0 94 0 VecScale 1 1.0 5.7220e-05 1.1 3.28e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 8 0 0 0 4581 VecCopy 6 1.0 2.3477e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 3 0 0 0 0 0 VecSet 1 1.0 1.3099e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecAXPY 6 1.0 1.9960e-03 1.0 3.93e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 3 92 0 0 0 1576 VecScatterBegin 5 1.0 3.7007e-03 1.0 0.00e+00 0.0 1.5e+02 1.6e+04 0.0e+00 0 0 0 0 0 5 0100100 0 0 VecScatterEnd 5 1.0 1.1663e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 --- Event Stage 8: SolvDeall --- Event Stage 9: MG Apply VecMDot 40 1.0 9.2528e-03 1.1 4.12e+06 1.0 0.0e+00 0.0e+00 4.0e+01 0 0 0 0 0 0 0 0 0 0 3561 VecNorm 44 1.0 9.3055e-04 1.2 8.24e+05 1.0 0.0e+00 0.0e+00 4.4e+01 0 0 0 0 0 0 0 0 0 0 7081 VecScale 6644 1.0 4.3099e-02 1.0 2.86e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 5314 VecCopy 994 1.0 4.5226e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 8089 1.0 1.5699e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 13208 1.0 2.0364e-01 1.0 1.13e+08 1.0 0.0e+00 0.0e+00 0.0e+00 2 5 0 0 0 2 7 0 0 0 4440 VecAYPX 13860 1.0 3.0671e-01 1.1 7.67e+07 1.0 0.0e+00 0.0e+00 0.0e+00 2 4 0 0 0 3 5 0 0 0 2001 VecMAXPY 44 1.0 7.6602e-03 1.0 4.87e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 5083 VecScatterBegin 16874 1.0 1.8452e-01 1.0 0.00e+00 0.0 5.6e+05 1.5e+03 0.0e+00 1 0 97 67 0 2 0100100 0 0 VecScatterEnd 16874 1.0 2.6180e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 VecNormalize 44 1.0 1.5919e-03 1.1 1.24e+06 1.0 0.0e+00 0.0e+00 4.4e+01 0 0 0 0 0 0 0 0 0 0 6209 MatMult 11264 1.0 3.0948e+00 1.0 7.20e+08 1.0 2.9e+05 2.7e+03 0.0e+00 24 34 50 63 0 32 43 52 94 0 1862 MatMultAdd 1650 1.0 3.0787e-01 1.0 4.76e+07 1.0 9.2e+04 1.7e+02 0.0e+00 2 2 16 1 0 3 3 17 2 0 1237 MatMultTranspose 2310 1.0 5.8261e-01 1.0 8.93e+07 1.0 1.3e+05 2.1e+02 0.0e+00 4 4 22 2 0 6 5 23 3 0 1226 MatSolve 825 1.0 1.1252e-02 1.1 3.22e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2290 MatSOR 9944 1.0 4.4170e+00 1.0 5.77e+08 1.0 0.0e+00 0.0e+00 0.0e+00 34 27 0 0 0 46 35 0 0 0 1045 KSPGMRESOrthog 40 1.0 1.5266e-02 1.0 8.24e+06 1.0 0.0e+00 0.0e+00 4.0e+01 0 0 0 0 0 0 0 0 0 0 4316 KSPSetUp 4 1.0 2.8238e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+01 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 4125 1.0 7.7953e+00 1.0 1.39e+09 1.0 2.6e+05 2.5e+03 1.1e+04 60 65 45 53 82 82 83 47 80100 1424 PCApply 10769 1.0 4.4700e+00 1.0 5.80e+08 1.0 4.6e+04 6.4e+01 0.0e+00 34 27 8 0 0 47 35 8 0 0 1038 MGSmooth Level 0 825 1.0 6.5387e-02 1.0 3.22e+06 1.0 4.6e+04 6.4e+01 8.2e+02 1 0 8 0 6 1 0 8 0 8 394 MGSmooth Level 1 1320 1.0 1.4422e-01 1.0 7.16e+06 1.0 8.3e+04 2.6e+02 4.0e+03 1 0 14 2 30 2 0 15 3 37 397 MGResid Level 1 660 1.0 8.7240e-03 1.0 6.34e+05 1.0 1.6e+04 2.6e+02 0.0e+00 0 0 3 0 0 0 0 3 0 0 581 MGInterp Level 1 1650 1.0 3.5585e-02 1.1 7.02e+05 1.0 9.2e+04 2.2e+01 0.0e+00 0 0 16 0 0 0 0 17 0 0 158 MGSmooth Level 2 990 1.0 2.3586e-01 1.0 4.57e+07 1.0 6.4e+04 1.0e+03 3.0e+03 2 2 11 5 23 2 3 11 8 28 1549 MGResid Level 2 495 1.0 1.6258e-02 1.0 3.80e+06 1.0 1.2e+04 1.0e+03 0.0e+00 0 0 2 1 0 0 0 2 1 0 1871 MGInterp Level 2 1320 1.0 5.1451e-02 1.1 4.48e+06 1.0 7.4e+04 7.0e+01 0.0e+00 0 0 13 0 0 1 0 13 1 0 696 MGSmooth Level 3 660 1.0 8.7171e-01 1.0 2.54e+08 1.0 4.4e+04 4.1e+03 2.0e+03 7 12 8 14 15 9 15 8 22 19 2329 MGResid Level 3 330 1.0 5.7600e-02 1.0 2.03e+07 1.0 7.9e+03 4.1e+03 0.0e+00 0 1 1 3 0 1 1 1 4 0 2816 MGInterp Level 3 990 1.0 1.6762e-01 1.0 2.67e+07 1.0 5.5e+04 2.5e+02 0.0e+00 1 1 10 1 0 2 2 10 2 0 1274 MGSmooth Level 4 330 1.0 6.6037e+00 1.0 1.08e+09 1.0 2.4e+04 1.6e+04 1.0e+03 51 50 4 32 8 69 65 4 47 10 1305 MGResid Level 4 165 1.0 3.2559e-01 1.0 8.11e+07 1.0 4.0e+03 1.6e+04 0.0e+00 3 4 1 5 0 3 5 1 8 0 1993 MGInterp Level 4 660 1.0 9.2705e-01 1.0 1.41e+08 1.0 3.7e+04 9.3e+02 0.0e+00 7 7 6 3 0 10 8 7 4 0 1213 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 0 0 0 --- Event Stage 1: StepStage --- Event Stage 2: ConvStage --- Event Stage 3: ProjStage --- Event Stage 4: IoStage Vector 20 20 5273760 0 Viewer 20 20 14080 0 --- Event Stage 5: KspStage Container 1 0 0 0 Vector 450 360 87362432 0 Vector Scatter 31 4 4208 0 Matrix 29 0 0 0 Matrix Null Space 16 11 6556 0 Distributed Mesh 8 0 0 0 Bipartite Graph 16 0 0 0 Index Set 73 70 146048 0 IS L to G Mapping 12 0 0 0 Krylov Solver 4 0 0 0 DMKSP interface 2 0 0 0 Preconditioner 4 0 0 0 Viewer 11 11 8008 0 --- Event Stage 6: SolvAlloc Vector 141 6 9136 0 Vector Scatter 6 0 0 0 Matrix 6 0 0 0 Distributed Mesh 2 0 0 0 Bipartite Graph 4 0 0 0 Index Set 14 14 372624 0 IS L to G Mapping 3 0 0 0 Krylov Solver 7 0 0 0 DMKSP interface 1 0 0 0 Preconditioner 7 0 0 0 Viewer 1 0 0 0 --- Event Stage 7: SolvSolve Vector 1 0 0 0 --- Event Stage 8: SolvDeall Container 0 1 564 0 Vector 0 266 48795872 0 Vector Scatter 0 28 29456 0 Matrix 0 32 11933468 0 Matrix Null Space 0 5 2980 0 Distributed Mesh 0 8 209600 0 Bipartite Graph 0 16 12672 0 Index Set 0 3 2792 0 IS L to G Mapping 0 12 120752 0 Krylov Solver 0 11 146712 0 DMKSP interface 0 2 1296 0 Preconditioner 0 11 9840 0 --- Event Stage 9: MG Apply Vector 60 8 87232 0 ======================================================================================================================== Average time to get PetscTime(): 1.90735e-07 Average time for MPI_Barrier(): 4.19617e-06 Average time for zero size MPI_Send(): 2.77162e-06 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor -ksp_monitor_true_residual -ksp_view -log_summary -options_left -pc_mg_levels 5 -pc_mg_log -pc_mg_type full -pc_type mg -------------- next part -------------- 8 processors, by filippo Mon Oct 6 14:16:40 2014 Using Petsc Release Version 3.5.0, Jun, 30, 2014 Max Max/Min Avg Total Time (sec): 1.487e+01 1.00008 1.487e+01 Objects: 7.540e+02 1.00000 7.540e+02 Flops: 1.089e+09 1.00000 1.089e+09 8.713e+09 Flops/sec: 7.325e+07 1.00008 7.325e+07 5.860e+08 MPI Messages: 3.927e+04 1.00306 3.917e+04 3.134e+05 MPI Message Lengths: 1.552e+08 1.68209 2.556e+03 8.008e+08 MPI Reductions: 7.494e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 2.4696e-02 0.2% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 1: StepStage: 1.1863e-02 0.1% 7.0779e+07 0.8% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2: ConvStage: 1.1314e+00 7.6% 3.6176e+08 4.2% 7.200e+03 2.3% 3.765e+02 14.7% 0.000e+00 0.0% 3: ProjStage: 1.7104e-01 1.2% 1.8350e+07 0.2% 9.600e+02 0.3% 5.019e+01 2.0% 0.000e+00 0.0% 4: IoStage: 8.4624e-01 5.7% 0.0000e+00 0.0% 5.680e+02 0.2% 4.835e+02 18.9% 1.000e+02 1.3% 5: KspStage: 2.1664e+00 14.6% 1.2813e+09 14.7% 5.464e+03 1.7% 2.115e+02 8.3% 1.213e+03 16.2% 6: SolvAlloc: 4.8040e-01 3.2% 0.0000e+00 0.0% 1.920e+02 0.1% 2.511e+00 0.1% 2.260e+02 3.0% 7: SolvSolve: 1.5624e-01 1.1% 3.4079e+06 0.0% 1.200e+02 0.0% 7.529e+00 0.3% 3.000e+01 0.4% 8: SolvDeall: 1.4080e-03 0.0% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 9: MG Apply: 9.8788e+00 66.4% 6.9772e+09 80.1% 2.988e+05 95.4% 1.424e+03 55.7% 5.924e+03 79.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage --- Event Stage 1: StepStage VecAXPY 120 1.0 1.7724e-02 2.1 8.85e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 99100 0 0 0 3993 --- Event Stage 2: ConvStage VecCopy 180 1.0 8.1402e-02 8.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecAXPY 180 1.0 9.0203e-02 8.9 1.18e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 4 26 0 0 0 1046 VecAXPBYCZ 210 1.0 1.0767e-01 3.8 2.75e+07 1.0 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 7 61 0 0 0 2045 VecPointwiseMult 180 1.0 6.0939e-02 6.5 5.90e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 3 13 0 0 0 774 VecScatterBegin 300 1.0 7.6515e-02 1.7 0.00e+00 0.0 7.2e+03 1.6e+04 0.0e+00 0 0 2 15 0 5 0100100 0 0 VecScatterEnd 300 1.0 7.6575e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 62 0 0 0 0 0 --- Event Stage 3: ProjStage VecScale 10 1.0 4.2558e-04 1.2 3.28e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 14 0 0 0 6160 VecCopy 30 1.0 6.9132e-03 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecAXPY 30 1.0 2.9233e-03 1.5 1.97e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 1 86 0 0 0 5381 VecScatterBegin 40 1.0 1.8399e-02 3.3 0.00e+00 0.0 9.6e+02 1.6e+04 0.0e+00 0 0 0 2 0 5 0100100 0 0 VecScatterEnd 40 1.0 1.7596e-01 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 85 0 0 0 0 0 --- Event Stage 4: IoStage VecView 20 1.0 4.0467e-01 4.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+01 2 0 0 0 1 28 0 0 0 40 0 VecCopy 20 1.0 1.7228e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 12 1.0 1.8814e-03 1.4 0.00e+00 0.0 2.9e+02 1.6e+04 0.0e+00 0 0 0 1 0 0 0 51 3 0 0 VecScatterEnd 12 1.0 1.2537e-01 7.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 8 0 0 0 0 0 --- Event Stage 5: KspStage VecMDot 77 1.0 1.0097e-01 1.2 2.03e+07 1.0 0.0e+00 0.0e+00 7.7e+01 1 2 0 0 1 4 13 0 0 6 1604 VecNorm 187 1.0 2.6558e-01 1.2 1.23e+07 1.0 0.0e+00 0.0e+00 1.9e+02 2 1 0 0 2 11 8 0 0 15 369 VecScale 88 1.0 4.1835e-03 1.1 2.88e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 2 0 0 0 5514 VecCopy 110 1.0 9.3369e-03 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 122 1.0 6.4487e-03 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 88 1.0 5.9593e-03 1.4 5.77e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 4 0 0 0 7742 VecAYPX 88 1.0 5.0604e-03 1.2 2.88e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 2 0 0 0 4559 VecMAXPY 165 1.0 4.0621e-02 1.3 4.55e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 2 28 0 0 0 8970 VecScatterBegin 173 1.0 9.2487e-03 1.5 0.00e+00 0.0 4.2e+03 1.6e+04 0.0e+00 0 0 1 8 0 0 0 77 98 0 0 VecScatterEnd 173 1.0 5.2526e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 21 0 0 0 0 0 VecNormalize 88 1.0 9.7991e-02 1.4 8.65e+06 1.0 0.0e+00 0.0e+00 8.8e+01 1 1 0 0 1 4 5 0 0 7 706 MatMult 165 1.0 6.5082e-01 1.2 7.03e+07 1.0 4.0e+03 1.6e+04 0.0e+00 4 6 1 8 0 27 44 72 98 0 864 MatMultTranspose 4 1.0 3.4583e-03 2.3 2.53e+05 1.0 2.2e+02 3.2e+02 0.0e+00 0 0 0 0 0 0 0 4 0 0 585 MatLUFactorSym 1 1.0 7.8917e-05 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatLUFactorNum 1 1.0 8.7976e-05 1.6 3.87e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 3518 MatConvert 1 1.0 3.6001e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 15 1.0 7.3933e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 0 0 0 0 0 3 0 0 0 2 0 MatAssemblyEnd 15 1.0 4.1289e-02 1.4 0.00e+00 0.0 6.9e+02 4.3e+02 7.2e+01 0 0 0 0 1 2 0 13 0 6 0 MatGetRowIJ 1 1.0 9.0599e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetSubMatrice 1 1.0 1.0691e-03 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 4.8876e-05 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 132 1.2 4.2408e-01 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.1e+02 3 0 0 0 1 18 0 0 0 9 0 MatGetRedundant 1 1.0 1.1230e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatMPIConcateSeq 1 1.0 3.9101e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 77 1.0 1.1721e-01 1.2 4.05e+07 1.0 0.0e+00 0.0e+00 7.7e+01 1 4 0 0 1 5 25 0 0 6 2764 KSPSetUp 7 1.0 9.3307e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 3.4e+01 0 0 0 0 0 0 0 0 0 3 0 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer KSPSolve 11 1.0 1.1635e+01 1.0 1.03e+09 1.0 3.0e+05 1.7e+03 7.0e+03 78 95 97 64 94 5366455569773579 710 PCSetUp 1 1.0 1.4414e-01 1.0 2.91e+05 1.0 1.5e+03 8.3e+02 3.4e+02 1 0 0 0 5 7 0 27 2 28 16 Warning -- total time of even greater than time of entire stage -- something is wrong with the timer PCApply 88 1.0 9.8873e+00 1.0 8.72e+08 1.0 3.0e+05 1.5e+03 5.9e+03 66 80 95 56 79 4565455469673488 706 MGSetup Level 0 1 1.0 8.9178e-03 1.8 3.87e+04 1.0 1.9e+02 3.9e+01 3.8e+01 0 0 0 0 1 0 0 4 0 3 35 MGSetup Level 1 1 1.0 3.5300e-03 1.2 0.00e+00 0.0 4.8e+01 6.6e+01 2.4e+01 0 0 0 0 0 0 0 1 0 2 0 MGSetup Level 2 1 1.0 6.0401e-03 1.1 0.00e+00 0.0 4.8e+01 2.6e+02 2.4e+01 0 0 0 0 0 0 0 1 0 2 0 MGSetup Level 3 1 1.0 2.8425e-02 1.0 0.00e+00 0.0 4.8e+01 1.0e+03 2.4e+01 0 0 0 0 0 1 0 1 0 2 0 MGSetup Level 4 1 1.0 2.9070e-03 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 --- Event Stage 6: SolvAlloc VecSet 50 1.0 1.6383e-02 3.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 MatAssemblyBegin 4 1.0 1.1187e-01 4.9 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 0 15 0 0 0 4 0 MatAssemblyEnd 4 1.0 4.0808e-02 1.5 0.00e+00 0.0 9.6e+01 4.1e+03 1.6e+01 0 0 0 0 0 7 0 50 50 7 0 --- Event Stage 7: SolvSolve VecNorm 30 1.0 8.2951e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+01 0 0 0 0 0 42 0 0 0100 0 VecScale 1 1.0 5.8317e-0411.2 3.28e+04 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 8 0 0 0 450 VecCopy 6 1.0 6.0177e-04 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 1 1.0 2.5990e-03 9.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 6 1.0 5.6362e-04 1.9 3.93e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 92 0 0 0 5581 VecScatterBegin 5 1.0 9.3579e-04 1.5 0.00e+00 0.0 1.2e+02 2.0e+04 0.0e+00 0 0 0 0 0 1 0100100 0 0 VecScatterEnd 5 1.0 3.4877e-02 3.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 14 0 0 0 0 0 --- Event Stage 8: SolvDeall --- Event Stage 9: MG Apply VecMDot 40 1.0 5.9973e-02 1.8 4.12e+06 1.0 0.0e+00 0.0e+00 4.0e+01 0 0 0 0 1 1 0 0 0 1 549 VecNorm 44 1.0 4.3393e-02 1.8 8.24e+05 1.0 0.0e+00 0.0e+00 4.4e+01 0 0 0 0 1 0 0 0 0 1 152 VecScale 44 1.0 7.3981e-04 1.2 4.12e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4453 VecCopy 532 1.0 2.0494e-03 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 4316 1.0 3.5908e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 8 1.0 1.9002e-04 1.5 1.50e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6305 VecAYPX 7392 1.0 6.8383e-02 1.2 4.09e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 1 5 0 0 0 4787 VecAXPBYCZ 3520 1.0 4.7338e-02 1.1 7.52e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 7 0 0 0 0 9 0 0 0 12716 VecMAXPY 44 1.0 7.3493e-03 2.5 4.87e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 5298 VecScatterBegin 9020 1.0 6.5990e-02 1.1 0.00e+00 0.0 3.0e+05 1.5e+03 0.0e+00 0 0 95 56 0 1 0100100 0 0 VecScatterEnd 9020 1.0 4.4829e+00 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 29 0 0 0 0 44 0 0 0 0 0 VecNormalize 44 1.0 4.4158e-02 1.8 1.24e+06 1.0 0.0e+00 0.0e+00 4.4e+01 0 0 0 0 1 0 0 0 0 1 224 MatMult 6028 1.0 4.3947e+00 1.1 3.61e+08 1.0 1.6e+05 2.7e+03 0.0e+00 28 33 50 53 0 43 41 52 94 0 656 MatMultAdd 880 1.0 3.1981e-01 1.2 2.54e+07 1.0 4.9e+04 1.7e+02 0.0e+00 2 2 16 1 0 3 3 16 2 0 635 MatMultTranspose 1232 1.0 5.7993e-01 1.1 4.76e+07 1.0 6.9e+04 2.1e+02 0.0e+00 4 4 22 2 0 6 5 23 3 0 657 MatSolve 440 1.0 2.6267e-03 1.1 1.72e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 5232 MatSOR 5324 1.0 8.8669e-01 1.1 3.10e+08 1.0 0.0e+00 0.0e+00 0.0e+00 6 28 0 0 0 9 36 0 0 0 2800 MatResidual 880 1.0 5.4892e-01 1.1 5.27e+07 1.0 2.1e+04 2.9e+03 0.0e+00 4 5 7 8 0 5 6 7 14 0 768 KSPGMRESOrthog 40 1.0 6.2556e-02 1.7 8.24e+06 1.0 0.0e+00 0.0e+00 4.0e+01 0 1 0 0 1 1 1 0 0 1 1053 KSPSetUp 4 1.0 9.8751e-03 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+01 0 0 0 0 1 0 0 0 0 1 0 KSPSolve 2200 1.0 8.3100e+00 1.0 7.28e+08 1.0 1.4e+05 2.5e+03 5.9e+03 55 67 45 44 79 83 83 47 80100 700 PCApply 5764 1.0 9.4324e-01 1.1 3.12e+08 1.0 2.5e+04 6.4e+01 0.0e+00 6 29 8 0 0 9 36 8 0 0 2646 MGSmooth Level 0 440 1.0 1.2119e-01 1.1 1.72e+06 1.0 2.5e+04 6.4e+01 4.4e+02 1 0 8 0 6 1 0 8 0 7 113 MGSmooth Level 1 704 1.0 3.0677e-01 1.0 3.72e+06 1.0 4.5e+04 2.6e+02 2.2e+03 2 0 14 1 29 3 0 15 3 37 97 MGResid Level 1 352 1.0 3.7244e-02 1.4 3.15e+05 1.0 8.4e+03 2.6e+02 0.0e+00 0 0 3 0 0 0 0 3 0 0 68 MGInterp Level 1 880 1.0 9.5410e-02 1.2 3.75e+05 1.0 4.9e+04 2.2e+01 0.0e+00 1 0 16 0 0 1 0 16 0 0 31 MGSmooth Level 2 528 1.0 4.3912e-01 1.0 2.38e+07 1.0 3.4e+04 1.0e+03 1.6e+03 3 2 11 4 22 4 3 11 8 28 433 MGResid Level 2 264 1.0 4.4301e-02 1.4 1.89e+06 1.0 6.3e+03 1.0e+03 0.0e+00 0 0 2 1 0 0 0 2 1 0 342 MGInterp Level 2 704 1.0 1.1298e-01 1.1 2.39e+06 1.0 3.9e+04 7.0e+01 0.0e+00 1 0 13 0 0 1 0 13 1 0 169 MGSmooth Level 3 352 1.0 1.6617e+00 1.0 1.32e+08 1.0 2.3e+04 4.1e+03 1.1e+03 11 12 7 12 15 17 15 8 22 19 638 MGResid Level 3 176 1.0 1.2805e-01 1.2 1.01e+07 1.0 4.2e+03 4.1e+03 0.0e+00 1 1 1 2 0 1 1 1 4 0 631 MGInterp Level 3 528 1.0 2.5215e-01 1.2 1.42e+07 1.0 3.0e+04 2.5e+02 0.0e+00 2 1 9 1 0 2 2 10 2 0 452 MGSmooth Level 4 176 1.0 5.8620e+00 1.0 5.66e+08 1.0 1.3e+04 1.6e+04 5.8e+02 39 52 4 26 8 58 65 4 48 10 772 MGResid Level 4 88 1.0 3.4808e-01 1.0 4.04e+07 1.0 2.1e+03 1.6e+04 0.0e+00 2 4 1 4 0 3 5 1 8 0 928 MGInterp Level 4 352 1.0 7.0880e-01 1.2 7.50e+07 1.0 2.0e+04 9.3e+02 0.0e+00 4 7 6 2 0 7 9 7 4 0 846 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Viewer 1 0 0 0 --- Event Stage 1: StepStage --- Event Stage 2: ConvStage --- Event Stage 3: ProjStage --- Event Stage 4: IoStage Vector 20 20 5274080 0 Viewer 20 20 14400 0 --- Event Stage 5: KspStage Vector 286 206 46757776 0 Vector Scatter 23 4 116656 0 Matrix 30 0 0 0 Matrix Null Space 16 11 6644 0 Distributed Mesh 8 0 0 0 Star Forest Bipartite Graph 16 0 0 0 Discrete System 8 0 0 0 Index Set 51 46 146616 0 IS L to G Mapping 8 0 0 0 Krylov Solver 4 0 0 0 DMKSP interface 2 0 0 0 Preconditioner 4 0 0 0 Viewer 11 11 8272 0 --- Event Stage 6: SolvAlloc Vector 141 6 9232 0 Vector Scatter 4 0 0 0 Matrix 6 0 0 0 Distributed Mesh 2 0 0 0 Star Forest Bipartite Graph 4 0 0 0 Discrete System 2 0 0 0 Index Set 8 8 366784 0 IS L to G Mapping 2 0 0 0 Krylov Solver 7 0 0 0 DMKSP interface 1 0 0 0 Preconditioner 7 0 0 0 Viewer 1 0 0 0 --- Event Stage 7: SolvSolve Vector 1 0 0 0 --- Event Stage 8: SolvDeall Vector 0 256 46163472 0 Vector Scatter 0 20 96688 0 Matrix 0 33 11943444 0 Matrix Null Space 0 5 3020 0 Distributed Mesh 0 8 39232 0 Star Forest Bipartite Graph 0 16 13056 0 Discrete System 0 8 6400 0 Index Set 0 5 4472 0 IS L to G Mapping 0 8 61664 0 Krylov Solver 0 11 146808 0 DMKSP interface 0 2 1328 0 Preconditioner 0 11 10280 0 --- Event Stage 9: MG Apply Vector 60 8 87360 0 ======================================================================================================================== Average time to get PetscTime(): 0 Average time for MPI_Barrier(): 2.49863e-05 Average time for zero size MPI_Send(): 3.00109e-05 #PETSc Option Table entries: -ksp_converged_reason -ksp_monitor -ksp_monitor_true_residual -ksp_view -log_summary -options_left -pc_mg_levels 5 -pc_mg_log -pc_mg_type full -pc_type mg -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From bsmith at mcs.anl.gov Mon Oct 6 07:36:47 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 6 Oct 2014 07:36:47 -0500 Subject: [petsc-users] Multigrid iterations for different versions of Petsc In-Reply-To: <44147693.kW2a8PGp4h@besikovitch-ii> References: <44147693.kW2a8PGp4h@besikovitch-ii> Message-ID: First make sure both runs think they are doing the exact same solver options so run both with -ksp_view and make sure the output is the same. I note that the 3.5 run has KSPSolve 2200 1.0 8.3100e+00 1.0 7.28e+08 1.0 1.4e+05 2.5e+03 5.9e+03 55 67 45 44 79 83 83 47 80100 700 PCApply 5764 1.0 9.4324e-01 1.1 3.12e+08 1.0 2.5e+04 6.4e+01 0.0e+00 6 29 8 0 0 9 36 8 0 0 2646 while the 3.4 has KSPSolve 4125 1.0 7.7953e+00 1.0 1.39e+09 1.0 2.6e+05 2.5e+03 1.1e+04 60 65 45 53 82 82 83 47 80100 1424 PCApply 10769 1.0 4.4700e+00 1.0 5.80e+08 1.0 4.6e+04 6.4e+01 0.0e+00 34 27 8 0 0 47 35 8 0 0 1038 why twice as many KSPSolves? Are you calling the KSPSolve more ? Barry On Oct 6, 2014, at 7:23 AM, Filippo Leonardi wrote: > Hi, > > Again me with multigrid. Sorry for the annoyance, but I am a bit worried about > my multigrid solver: I still can't figure out where (if something) my code is > doing wrong. > > I post the details of a run with my local machine and Petsc 3.5 compared with > the runs done with Petsc 3.4.3 on a cluster (the most recent version > available). > > The problem is that the run in the cluster (3.4.3) does twice many iteration > per krylow solver (as the local 3.5) and goes even worse when increasing the > mesh. Also, while on my local machine I get 7 iterations (which is quite > nice), on the cluster I get 14 iteration, which is not what I expect from > multigrid. > > Are there been any changes on the MG solver that can influence the behaviour of > the solver? > > http://www.mcs.anl.gov/petsc/documentation/changes/35.html > seems not to include anything relevant. > > Everything is the same: outer iteration with gmres, lu as coarse solver etc > (all default), I just adapted the APIs. Thus I'd expect more or less the same > result. > > Best, > Filippo > > I attach the profiles.<3-4.txt><3-5.txt> From filippo.leonardi at sam.math.ethz.ch Mon Oct 6 08:15:25 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Mon, 6 Oct 2014 15:15:25 +0200 Subject: [petsc-users] Multigrid iterations for different versions of Petsc In-Reply-To: References: <44147693.kW2a8PGp4h@besikovitch-ii> Message-ID: <6282227.hIgEfibN7f@besikovitch-ii> On Monday 06 October 2014 07:36:47 Barry Smith wrote: > First make sure both runs think they are doing the exact same solver > options so run both with -ksp_view and make sure the output is the same. Ok, now I am really confused. I did a "diff" onto the ouput of kspview (attached) and the result is... weird. The size of the matrices is cubed for the version 3.4.3, although I am sure the mesh size and the initial matrix (BTW a DMDA get Jacobian) are the same in the two cases (I even output the result) (the code and the command line options are exactly the same except for API changes). Is this a bug? > > I note that the 3.5 run has > > KSPSolve 2200 1.0 8.3100e+00 1.0 7.28e+08 1.0 1.4e+05 2.5e+03 > 5.9e+03 55 67 45 44 79 83 83 47 80100 700 PCApply 5764 1.0 > 9.4324e-01 1.1 3.12e+08 1.0 2.5e+04 6.4e+01 0.0e+00 6 29 8 0 0 9 36 > 8 0 0 2646 > > while the 3.4 has > > KSPSolve 4125 1.0 7.7953e+00 1.0 1.39e+09 1.0 2.6e+05 2.5e+03 > 1.1e+04 60 65 45 53 82 82 83 47 80100 1424 PCApply 10769 1.0 > 4.4700e+00 1.0 5.80e+08 1.0 4.6e+04 6.4e+01 0.0e+00 34 27 8 0 0 47 35 > 8 0 0 1038 > > why twice as many KSPSolves? Are you calling the KSPSolve more ? I call KSPSolve 11 times here (I am looking at the stage 5), and then MG is calling KSPSolve by its own and the doubling of iterations is explained by the doubling of iterations. > > Barry > > On Oct 6, 2014, at 7:23 AM, Filippo Leonardi wrote: > > Hi, > > > > Again me with multigrid. Sorry for the annoyance, but I am a bit worried > > about my multigrid solver: I still can't figure out where (if something) > > my code is doing wrong. > > > > I post the details of a run with my local machine and Petsc 3.5 compared > > with the runs done with Petsc 3.4.3 on a cluster (the most recent version > > available). > > > > The problem is that the run in the cluster (3.4.3) does twice many > > iteration per krylow solver (as the local 3.5) and goes even worse when > > increasing the mesh. Also, while on my local machine I get 7 iterations > > (which is quite nice), on the cluster I get 14 iteration, which is not > > what I expect from multigrid. > > > > Are there been any changes on the MG solver that can influence the > > behaviour of the solver? > > > > http://www.mcs.anl.gov/petsc/documentation/changes/35.html > > seems not to include anything relevant. > > > > Everything is the same: outer iteration with gmres, lu as coarse solver > > etc > > (all default), I just adapted the APIs. Thus I'd expect more or less the > > same result. > > > > Best, > > Filippo > > > > I attach the profiles.<3-4.txt><3-5.txt> -------------- next part -------------- KSP Object: 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using PRECONDITIONED norm type for convergence test PC Object: 8 MPI processes type: mg MG: type is FULL, levels=5 cycles=v Not using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 8 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using NONE norm type for convergence test PC Object: (mg_coarse_) 8 MPI processes type: redundant Redundant preconditioner: First (color=0) of 8 PCs follows KSP Object: (mg_coarse_redundant_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_redundant_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 4.42857 Factored matrix follows: Matrix Object: 1 MPI processes type: seqaij rows=64, cols=64 package used to perform factorization: petsc total: nonzeros=1984, allocated nonzeros=1984 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Matrix Object: 1 MPI processes type: seqaij rows=64, cols=64 total: nonzeros=448, allocated nonzeros=448 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=64, cols=64 total: nonzeros=448, allocated nonzeros=448 total number of mallocs used during MatSetValues calls =0 Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.153663, max = 1.6903 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_1_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_1_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=512, cols=512 total: nonzeros=3584, allocated nonzeros=3584 total number of mallocs used during MatSetValues calls =0 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_1_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=512, cols=512 total: nonzeros=3584, allocated nonzeros=3584 total number of mallocs used during MatSetValues calls =0 Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.153015, max = 1.68317 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_2_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_2_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=4096, cols=4096 total: nonzeros=28672, allocated nonzeros=28672 total number of mallocs used during MatSetValues calls =0 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_2_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=4096, cols=4096 total: nonzeros=28672, allocated nonzeros=28672 total number of mallocs used during MatSetValues calls =0 Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.152678, max = 1.67946 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_3_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_3_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=32768, cols=32768 total: nonzeros=229376, allocated nonzeros=229376 total number of mallocs used during MatSetValues calls =0 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_3_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=32768, cols=32768 total: nonzeros=229376, allocated nonzeros=229376 total number of mallocs used during MatSetValues calls =0 Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.147343, max = 1.62077 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_4_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_4_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=262144, cols=262144 total: nonzeros=1835008, allocated nonzeros=1835008 total number of mallocs used during MatSetValues calls =0 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_4_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=262144, cols=262144 total: nonzeros=1835008, allocated nonzeros=1835008 total number of mallocs used during MatSetValues calls =0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Matrix Object: 8 MPI processes type: mpiaij rows=262144, cols=262144 total: nonzeros=1835008, allocated nonzeros=1835008 total number of mallocs used during MatSetValues calls =0 -------------- next part -------------- KSP Object: 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using PRECONDITIONED norm type for convergence test PC Object: 8 MPI processes type: mg MG: type is FULL, levels=5 cycles=v Not using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 8 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using NONE norm type for convergence test PC Object: (mg_coarse_) 8 MPI processes type: redundant Redundant preconditioner: First (color=0) of 8 PCs follows KSP Object: (mg_coarse_redundant_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_redundant_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 1.375 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=8, cols=8 package used to perform factorization: petsc total: nonzeros=44, allocated nonzeros=44 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=8, cols=8 total: nonzeros=32, allocated nonzeros=32 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=8, cols=8 total: nonzeros=32, allocated nonzeros=56 total number of mallocs used during MatSetValues calls =0 has attached null space Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.155469, max = 1.71016 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_1_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_1_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=64, cols=64 total: nonzeros=448, allocated nonzeros=448 total number of mallocs used during MatSetValues calls =0 has attached null space maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_1_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=64, cols=64 total: nonzeros=448, allocated nonzeros=448 total number of mallocs used during MatSetValues calls =0 has attached null space Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.15366, max = 1.69026 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_2_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_2_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=512, cols=512 total: nonzeros=3584, allocated nonzeros=3584 total number of mallocs used during MatSetValues calls =0 has attached null space maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_2_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=512, cols=512 total: nonzeros=3584, allocated nonzeros=3584 total number of mallocs used during MatSetValues calls =0 has attached null space Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.152956, max = 1.68252 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_3_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_3_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=4096, cols=4096 total: nonzeros=28672, allocated nonzeros=28672 total number of mallocs used during MatSetValues calls =0 has attached null space maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_3_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=4096, cols=4096 total: nonzeros=28672, allocated nonzeros=28672 total number of mallocs used during MatSetValues calls =0 has attached null space Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 8 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.149148, max = 1.64063 Chebyshev: estimated using: [0 0.1; 0 1.1] KSP Object: (mg_levels_4_est_) 8 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_4_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=32768, cols=32768 total: nonzeros=229376, allocated nonzeros=229376 total number of mallocs used during MatSetValues calls =0 has attached null space maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning has attached null space using nonzero initial guess using NONE norm type for convergence test PC Object: (mg_levels_4_) 8 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=32768, cols=32768 total: nonzeros=229376, allocated nonzeros=229376 total number of mallocs used during MatSetValues calls =0 has attached null space Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 8 MPI processes type: mpiaij rows=32768, cols=32768 total: nonzeros=229376, allocated nonzeros=229376 total number of mallocs used during MatSetValues calls =0 has attached null space -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From csp at info.szwgroup.com Tue Oct 7 01:05:04 2014 From: csp at info.szwgroup.com (=?utf-8?B?TXMuIEVsbGEgV2Vp?=) Date: Tue, 7 Oct 2014 14:05:04 +0800 (CST) Subject: [petsc-users] =?utf-8?q?Last_four_days_to_join_CSP_Focus_South_Af?= =?utf-8?q?rica_November_with_a_discount?= Message-ID: <20141007060504.F00552D348C2@mx7.easemaillist.com> An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Tue Oct 7 08:14:27 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Tue, 07 Oct 2014 15:14:27 +0200 Subject: [petsc-users] Matrix missing diagonal entry Message-ID: <1484883.lN7CaqHuu7@asaru> Hello, when I try to KSPSolve my matrix petsc prints out that a diagonal entry is missing: KSPSetOperators(solver, _matCLU.matrix, _matCLU.matrix ); KSPSolve(solver, vin.vector, vp.vector); And petsc is perfectly right about that specific entry being zero. So this may be more of a mathematical problem then pure petsc. The algorithm that I try to reimplement does the following: lu(_matrixCLU, _pivotsCLU); // Compute LU decomposition // Left out: Permutate in according to pivotsCLU forwardSubstitution(_matrixCLU, in, y); // CLU^-1 * in = y (lower triangle of CLU) backSubstitution(_matrixCLU, y, p); // CLU^-1 * y = p (upper triangle of CLU) multiply(_matrixA, p, out ); // out = A * p (not relevant here) I want to replace to lu, forward- and backwardSubstituation by using a petsc KSP solver. Before doing any of these operations _matrixCLU and _matCLU are equal: =============== CLU Before LU ======= ==== Dynamic Matrix of size 7 x 7 ==== 1.0000 0.3679 0.1353 0.3679 1.0000 0.0000 0.0000 0.3679 1.0000 0.3679 0.1353 1.0000 1.0000 0.0000 0.1353 0.3679 1.0000 0.3679 1.0000 1.0000 1.0000 0.3679 0.1353 0.3679 1.0000 1.0000 0.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 Mat Object:CLU 1 MPI processes type: seqsbaij row 0: (0, 1) (1, 0.367879) (2, 0.135335) (3, 0.367879) (4, 1) (5, 0) (6, 0) row 1: (1, 1) (2, 0.367879) (3, 0.135335) (4, 1) (5, 1) (6, 0) row 2: (2, 1) (3, 0.367879) (4, 1) (5, 1) (6, 1) row 3: (3, 1) (4, 1) (5, 0) (6, 1) row 4: row 5: row 6: LU decomposition of matrixCLU works, no rank deficiency (no 0 on the main diagonal after LU and I tested for full rank of matrixCLU). Petsc complains with this matrix: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Object is in wrong state [0]PETSC ERROR: Matrix is missing diagonal entry 4 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown [0]PETSC ERROR: ../build/debug/binprecice on a arch-linux2-c-debug named helium by lindnefn Mon Oct 6 16:26:49 2014 [0]PETSC ERROR: Configure options [0]PETSC ERROR: #1 MatICCFactorSymbolic_SeqSBAIJ() line 2430 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaijfact2.c [0]PETSC ERROR: #2 MatICCFactorSymbolic() line 6266 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c [0]PETSC ERROR: #3 PCSetup_ICC() line 20 in /data2/scratch/lindner/petsc/src/ksp/pc/impls/factor/icc/icc.c [0]PETSC ERROR: #4 PCSetUp() line 902 in /data2/scratch/lindner/petsc/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #5 KSPSetUp() line 305 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #6 KSPSolve() line 417 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #7 map() line 696 in src/mapping/PetRadialBasisFctMapping.hpp numerical equality test failed: 0.00000000000000000000e+00 instead of 1.00000000000000000000e+00 file: src/mapping/tests/PetRadialBasisFctMappingTest.cpp line: 196 statement: value==1.0 difference: 1.00000000000000000000e+00 What's indeed perfectly right. But what is the best way to cope with that? Thanks, Florian From bsmith at mcs.anl.gov Tue Oct 7 08:34:55 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 7 Oct 2014 08:34:55 -0500 Subject: [petsc-users] Matrix missing diagonal entry In-Reply-To: <1484883.lN7CaqHuu7@asaru> References: <1484883.lN7CaqHuu7@asaru> Message-ID: <0E439AAA-3314-418A-90C1-95DC74B840F6@mcs.anl.gov> PETSc has this ?extra? requirement that you provide values for the entire diagonal of the matrix, even those locations with zero. So make sure that you use MatSetValues() to put a zero on each diagonal entry that has zero. Also from below you are using ICC that is incomplete cholesky factorization. If you want to use a direct solver you should select -pc_type cholesky Barry On Oct 7, 2014, at 8:14 AM, Florian Lindner wrote: > Hello, > > when I try to KSPSolve my matrix petsc prints out that a diagonal entry is missing: > > KSPSetOperators(solver, _matCLU.matrix, _matCLU.matrix ); > KSPSolve(solver, vin.vector, vp.vector); > > And petsc is perfectly right about that specific entry being zero. So this may be more of a mathematical problem then pure petsc. > > The algorithm that I try to reimplement does the following: > > lu(_matrixCLU, _pivotsCLU); // Compute LU decomposition > // Left out: Permutate in according to pivotsCLU > forwardSubstitution(_matrixCLU, in, y); // CLU^-1 * in = y (lower triangle of CLU) > backSubstitution(_matrixCLU, y, p); // CLU^-1 * y = p (upper triangle of CLU) > multiply(_matrixA, p, out ); // out = A * p (not relevant here) > > I want to replace to lu, forward- and backwardSubstituation by using a petsc KSP solver. Before doing any of these operations _matrixCLU and _matCLU are equal: > > =============== CLU Before LU ======= > ==== Dynamic Matrix of size 7 x 7 ==== > 1.0000 0.3679 0.1353 0.3679 1.0000 0.0000 0.0000 > 0.3679 1.0000 0.3679 0.1353 1.0000 1.0000 0.0000 > 0.1353 0.3679 1.0000 0.3679 1.0000 1.0000 1.0000 > 0.3679 0.1353 0.3679 1.0000 1.0000 0.0000 1.0000 > 1.0000 1.0000 1.0000 1.0000 0.0000 0.0000 0.0000 > 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 0.0000 > 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 > > Mat Object:CLU 1 MPI processes > type: seqsbaij > row 0: (0, 1) (1, 0.367879) (2, 0.135335) (3, 0.367879) (4, 1) (5, 0) (6, 0) > row 1: (1, 1) (2, 0.367879) (3, 0.135335) (4, 1) (5, 1) (6, 0) > row 2: (2, 1) (3, 0.367879) (4, 1) (5, 1) (6, 1) > row 3: (3, 1) (4, 1) (5, 0) (6, 1) > row 4: > row 5: > row 6: > > LU decomposition of matrixCLU works, no rank deficiency (no 0 on the main diagonal after LU and I tested for full rank of matrixCLU). Petsc complains with this matrix: > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: Object is in wrong state > [0]PETSC ERROR: Matrix is missing diagonal entry 4 > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > [0]PETSC ERROR: ../build/debug/binprecice on a arch-linux2-c-debug named helium by lindnefn Mon Oct 6 16:26:49 2014 > [0]PETSC ERROR: Configure options > [0]PETSC ERROR: #1 MatICCFactorSymbolic_SeqSBAIJ() line 2430 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaijfact2.c > [0]PETSC ERROR: #2 MatICCFactorSymbolic() line 6266 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > [0]PETSC ERROR: #3 PCSetup_ICC() line 20 in /data2/scratch/lindner/petsc/src/ksp/pc/impls/factor/icc/icc.c > [0]PETSC ERROR: #4 PCSetUp() line 902 in /data2/scratch/lindner/petsc/src/ksp/pc/interface/precon.c > [0]PETSC ERROR: #5 KSPSetUp() line 305 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: #6 KSPSolve() line 417 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: #7 map() line 696 in src/mapping/PetRadialBasisFctMapping.hpp > numerical equality test failed: 0.00000000000000000000e+00 instead of 1.00000000000000000000e+00 > file: src/mapping/tests/PetRadialBasisFctMappingTest.cpp line: 196 > statement: value==1.0 > difference: 1.00000000000000000000e+00 > > > What's indeed perfectly right. But what is the best way to cope with that? > > Thanks, > Florian From salazardetroya at gmail.com Tue Oct 7 11:47:12 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Tue, 7 Oct 2014 11:47:12 -0500 Subject: [petsc-users] Error when trying to write solution to a file Message-ID: Hi all I'm solving a discretized PDE with TS. Inside my MonitorFunction(), hooked up with TSMonitorSet(), I process the solution at each time step to write another PETSc Vec. Then, I write this Vec to a matlab binary file. The name of the binary file is the time step. I save it inside of the folder "ForceHistory" // File name PetscViewer viewer; std::string filehistory("ForceHistory/"); std::ostringstream temp; temp << step; filehistory.append(temp.str()); filehistory.append(".mat"); // Open and set Binary file PetscViewerBinaryOpen(PETSC_COMM_WORLD,filehistory.c_str(),FILE_MODE_WRITE,&viewer); PetscViewerSetFormat(viewer, PETSC_VIEWER_BINARY_MATLAB); // Print VecView(Context->Felement,viewer); It works well until I reach the time step number 509. Beyond that time step, I get errors like this: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Unable to open file [0]PETSC ERROR: Cannot create file ForceHistory/758.mat for writing [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 [0]PETSC ERROR: ./beads on a linux-mpich named miguelito by miguel Tue Oct 7 11:20:51 2014 [0]PETSC ERROR: Configure options --download-hypre=yes --download-metis --download-ml=yes --download-mpich --download-mumps --download-parmetis --download-scalapack --download-superilu --with-blas-lib=/usr/lib/libblas.so --with-language=cxx --with-lapack-lib=/usr/lib/liblapack.so PETSC_ARCH=linux-mpich --download-chaco [0]PETSC ERROR: #2242 PetscViewerFileSetName_Binary() line 1102 in /home/miguel/petsc-3.5.2/src/sys/classes/viewer/impls/binary/binv.c [0]PETSC ERROR: #2243 PetscViewerFileSetName() line 624 in /home/miguel/petsc-3.5.2/src/sys/classes/viewer/impls/ascii/filev.c [0]PETSC ERROR: #2244 PetscViewerBinaryOpen() line 711 in /home/miguel/petsc-3.5.2/src/sys/classes/viewer/impls/binary/binv.c [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Write to file failed [0]PETSC ERROR: Error writing to file total size 2 err -1 wsize 8 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 [0]PETSC ERROR: ./beads on a linux-mpich named miguelito by miguel Tue Oct 7 11:20:51 2014 [0]PETSC ERROR: Configure options --download-hypre=yes --download-metis --download-ml=yes --download-mpich --download-mumps --download-parmetis --download-scalapack --download-superilu --with-blas-lib=/usr/lib/libblas.so --with-language=cxx --with-lapack-lib=/usr/lib/liblapack.so PETSC_ARCH=linux-mpich --download-chaco [0]PETSC ERROR: #2245 PetscBinaryWrite() line 401 in /home/miguel/petsc-3.5.2/src/sys/fileio/sysio.c [0]PETSC ERROR: #2246 PetscBinarySynchronizedWrite() line 656 in /home/miguel/petsc-3.5.2/src/sys/fileio/sysio.c [0]PETSC ERROR: #2247 PetscViewerBinaryWrite() line 817 in /home/miguel/petsc-3.5.2/src/sys/classes/viewer/impls/binary/binv.c [0]PETSC ERROR: #2248 VecView_MPI_Binary() line 372 in /home/miguel/petsc-3.5.2/src/vec/vec/impls/mpi/pdvec.c [0]PETSC ERROR: #2249 VecView_MPI() line 837 in /home/miguel/petsc-3.5.2/src/vec/vec/impls/mpi/pdvec.c [0]PETSC ERROR: #2250 VecView() line 604 in /home/miguel/petsc-3.5.2/src/vec/vec/interface/vector.c For some reason, it cannot create more files. It's not that I ran out of space in the disk or that no more files can't be created in that folder. Any suggestion? Thanks. Miguel -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 7 11:50:19 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 07 Oct 2014 10:50:19 -0600 Subject: [petsc-users] Error when trying to write solution to a file In-Reply-To: References: Message-ID: <87a957j038.fsf@jedbrown.org> Miguel Angel Salazar de Troya writes: > Hi all > > I'm solving a discretized PDE with TS. Inside my MonitorFunction(), hooked > up with TSMonitorSet(), I process the solution at each time step to write > another PETSc Vec. Then, I write this Vec to a matlab binary file. The name > of the binary file is the time step. I save it inside of the folder > "ForceHistory" > > // File name > PetscViewer viewer; > std::string filehistory("ForceHistory/"); > std::ostringstream temp; > temp << step; > filehistory.append(temp.str()); > filehistory.append(".mat"); > > // Open and set Binary file > PetscViewerBinaryOpen(PETSC_COMM_WORLD,filehistory.c_str(),FILE_MODE_WRITE,&viewer); > PetscViewerSetFormat(viewer, PETSC_VIEWER_BINARY_MATLAB); > > // Print > VecView(Context->Felement,viewer); Are you destroying the viewer (to close the file)? It looks like either a max number of file descriptors or a disk quota problem. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From salazardetroya at gmail.com Tue Oct 7 11:52:49 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Tue, 7 Oct 2014 11:52:49 -0500 Subject: [petsc-users] Error when trying to write solution to a file In-Reply-To: <87a957j038.fsf@jedbrown.org> References: <87a957j038.fsf@jedbrown.org> Message-ID: It was that, I totally forgot it. Thanks a lot! On Tue, Oct 7, 2014 at 11:50 AM, Jed Brown wrote: > Miguel Angel Salazar de Troya writes: > > > Hi all > > > > I'm solving a discretized PDE with TS. Inside my MonitorFunction(), > hooked > > up with TSMonitorSet(), I process the solution at each time step to write > > another PETSc Vec. Then, I write this Vec to a matlab binary file. The > name > > of the binary file is the time step. I save it inside of the folder > > "ForceHistory" > > > > // File name > > PetscViewer viewer; > > std::string filehistory("ForceHistory/"); > > std::ostringstream temp; > > temp << step; > > filehistory.append(temp.str()); > > filehistory.append(".mat"); > > > > // Open and set Binary file > > > PetscViewerBinaryOpen(PETSC_COMM_WORLD,filehistory.c_str(),FILE_MODE_WRITE,&viewer); > > PetscViewerSetFormat(viewer, PETSC_VIEWER_BINARY_MATLAB); > > > > // Print > > VecView(Context->Felement,viewer); > > Are you destroying the viewer (to close the file)? It looks like either > a max number of file descriptors or a disk quota problem. > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From jychang48 at gmail.com Tue Oct 7 22:09:29 2014 From: jychang48 at gmail.com (Justin Chang) Date: Tue, 7 Oct 2014 22:09:29 -0500 Subject: [petsc-users] Generating xdmf from h5 file. In-Reply-To: References: Message-ID: Hi Matt, Bumping this thread. I have a similar question. The lines you wrote for testnum 39 have also worked perfectly for me. However, when I change a couple parameters, namely not interpolating the mesh and using -petscspace_order 1: ./ex12 -run_type full -refinement_limit 0.015625 -petscspace_order 1 -pc_type gamg -ksp_rtol 1.0e-10 -ksp_monitor_short -ksp_converged_reason -snes_monitor_short -snes_converged_reason -dm_view hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append I get the following error after I run the xdmf script: Traceback (most recent call last): File "./bin/pythonscripts/petsc_gen_xdmf.py", line 223, in generateXdmf(sys.argv[1]) File "./bin/pythonscripts/petsc_gen_xdmf.py", line 218, in generateXdmf Xdmf(xdmfFilename).write(hdfFilename, topoPath, numCells, numCorners, cellDim, geomPath, numVertices, spaceDim, time, vfields, cfields) File "./bin/pythonscripts/petsc_gen_xdmf.py", line 176, in write self.writeSpaceGridHeader(fp, numCells, numCorners, cellDim, spaceDim) File "./bin/pythonscripts/petsc_gen_xdmf.py", line 75, in writeSpaceGridHeader ''' % (self.cellMap[cellDim][numCorners], numCells, "XYZ" if spaceDim > 2 else "XY")) Does the script so far only work for interpolated meshes/problems? Thanks, Justin On Sun, Sep 28, 2014 at 11:15 AM, subramanya sadasiva wrote: > Thanks Matt, > That worked perfectly. > > Subramanya > > ------------------------------ > Date: Fri, 26 Sep 2014 14:52:52 -0500 > Subject: Re: [petsc-users] Generating xdmf from h5 file. > From: knepley at gmail.com > To: potaman at outlook.com > CC: petsc-users at mcs.anl.gov > > > On Thu, Sep 25, 2014 at 10:29 PM, subramanya sadasiva > wrote: > > Hi Matt, > Sorry about that, > I changed > if 'time' in h5: > time = np.array(h5['time']).flatten() > else: > time = np.empty(1) > > The code now fails in the writeSpaceGridHeader function. with the error, > > > Traceback (most recent call last): > File > "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line > 232, in > generateXdmf(sys.argv[1]) > File > "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line > 227, in generateXdmf > Xdmf(xdmfFilename).write(hdfFilename, topoPath, numCells, numCorners, > cellDim, geomPath, numVertices, spaceDim, time, vfields, cfields) > File > "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line > 180, in write > self.writeSpaceGridHeader(fp, numCells, numCorners, cellDim, spaceDim) > File > "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line > 64, in writeSpaceGridHeader > print self.cellMap[cellDim][numCorners] > > The error is due to the fact that numCorners is set to be 1 , while > celldim=2. cellMap has the following elements. > {1: {1: 'Polyvertex', 2: 'Polyline'}, 2: {3: 'Triangle', 4: > 'Quadrilateral'}, 3: {8: 'Hexahedron', 4: 'Tetrahedron'}} > > > I also tried > ./ex12 -dm_view vtk:my.vtk:vtk_vtu . This doesn't seem to do anything. Is > there any specific option I need to build petsc with to get vtk output? My > current build has hdf5 and netcdf enabled. > > I made the change in a branch, and merged to 'next', so it will work if > you pull. Here is a test from ex12 (testnum 39 in builder.py): > > ./ex12 -run_type full -refinement_limit 0.015625 -interpolate 1 > -petscspace_order 2 -pc_type gamg -ksp_rtol 1.0e-10 -ksp_monitor_short > -ksp_converged_reason -snes_monitor_short -snes_converged_reason -dm_view > hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append > > which makes sol.h5. Then I run > > ./bin/pythonscripts/petsc_gen_xdmf.py sol.h5 > > which makes sol.xmf. I load it up in Paraview and it makes the attached > picture. > > Thanks, > > Matt > > > Thanks, > Subramanya > > Date: Wed, 24 Sep 2014 17:36:52 -0500 > Subject: Re: [petsc-users] Generating xdmf from h5 file. > From: knepley at gmail.com > To: potaman at outlook.com; petsc-maint at mcs.anl.gov; petsc-users at mcs.anl.gov > > On Wed, Sep 24, 2014 at 5:29 PM, subramanya sadasiva > wrote: > > Hi Matt, > That did not help. > > > That's not enough description to fix anything, and fixing it will require > programming. > > > Is there any other way to output the mesh to something that paraview can > view? I tried outputting the file to a vtk file using > ex12 -dm_view vtk:my.vtk:ascii_vtk > > which, I saw in another post on the forums, but that did not give me any > output. > > > This is mixing two different things. PETSc has a diagnostic ASCII vtk > output, so the type would be ascii, not vtk, > and format ascii_vtk . It also has a production VTU output, which is type > vtk with format vtk_vtu. > > Thanks, > > Matt > > > > Subramanya > > ------------------------------ > Date: Wed, 24 Sep 2014 17:19:51 -0500 > Subject: Re: [petsc-users] Generating xdmf from h5 file. > From: knepley at gmail.com > To: potaman at outlook.com > CC: petsc-users at mcs.anl.gov > > On Wed, Sep 24, 2014 at 5:08 PM, subramanya sadasiva > wrote: > > Hi, > i was trying to use petsc_gen_xdmf.py to convert a h5 file to a xdmf file. > The h5 file was generated by snes/ex12 which was run as, > > ex12 -dm_view hdf5:my.h5 > > When I do, > petsc_gen_xdmf.py my.h5 > > I get the following error, > > File "/home/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", > line 220, in > generateXdmf(sys.argv[1]) > File > "/home/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line > 208, in generateXdmf > time = np.array(h5['time']).flatten() > File "/usr/lib/python2.7/dist-packages/h5py/_hl/group.py", line 153, in > __getitem__ > oid = h5o.open(self.id, self._e(name), lapl=self._lapl) > File "h5o.pyx", line 173, in h5py.h5o.open (h5py/h5o.c:3403) > KeyError: "unable to open object (Symbol table: Can't open object)" > > I am not sure if the error is on my end. This is on Ubuntu 14.04 with the > serial version of hdf5. I built petsc with --download-hdf5, is it necessary > to use the same version of hdf5 to generate the xdmf file? > > > That code is alpha, and mainly built for me to experiment with an > application here, so it is not user-friendly. In your > HDF5 file, there is no 'time' since you are not running a TS. This access > to h5['time'] should just be protected, and > an empty array should be put in if its not there. > > Matt > > > Thanks > Subramanya > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Oct 8 02:15:21 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 8 Oct 2014 02:15:21 -0500 Subject: [petsc-users] Generating xdmf from h5 file. In-Reply-To: References: Message-ID: On Tue, Oct 7, 2014 at 10:09 PM, Justin Chang wrote: > Hi Matt, > > Bumping this thread. I have a similar question. The lines you wrote for > testnum 39 have also worked perfectly for me. However, when I change a > couple parameters, namely not interpolating the mesh and using > -petscspace_order 1: > > ./ex12 -run_type full -refinement_limit 0.015625 -petscspace_order 1 > -pc_type gamg -ksp_rtol 1.0e-10 -ksp_monitor_short -ksp_converged_reason > -snes_monitor_short -snes_converged_reason -dm_view hdf5:sol.h5 > -snes_view_solution hdf5:sol.h5::append > > I get the following error after I run the xdmf script: > > Traceback (most recent call last): > File "./bin/pythonscripts/petsc_gen_xdmf.py", line 223, in > generateXdmf(sys.argv[1]) > File "./bin/pythonscripts/petsc_gen_xdmf.py", line 218, in generateXdmf > Xdmf(xdmfFilename).write(hdfFilename, topoPath, numCells, numCorners, > cellDim, geomPath, numVertices, spaceDim, time, vfields, cfields) > File "./bin/pythonscripts/petsc_gen_xdmf.py", line 176, in write > self.writeSpaceGridHeader(fp, numCells, numCorners, cellDim, spaceDim) > File "./bin/pythonscripts/petsc_gen_xdmf.py", line 75, in > writeSpaceGridHeader > ''' % (self.cellMap[cellDim][numCorners], numCells, "XYZ" if spaceDim > > 2 else "XY")) > > Does the script so far only work for interpolated meshes/problems? > It should work for both. I think this is the same bug as reported yesterday by Michael Lange, that the coordinate Vec is sometimes missing the correct block size. I will track it down. Thanks, Matt > Thanks, > Justin > > On Sun, Sep 28, 2014 at 11:15 AM, subramanya sadasiva > wrote: > >> Thanks Matt, >> That worked perfectly. >> >> Subramanya >> >> ------------------------------ >> Date: Fri, 26 Sep 2014 14:52:52 -0500 >> Subject: Re: [petsc-users] Generating xdmf from h5 file. >> From: knepley at gmail.com >> To: potaman at outlook.com >> CC: petsc-users at mcs.anl.gov >> >> >> On Thu, Sep 25, 2014 at 10:29 PM, subramanya sadasiva < >> potaman at outlook.com> wrote: >> >> Hi Matt, >> Sorry about that, >> I changed >> if 'time' in h5: >> time = np.array(h5['time']).flatten() >> else: >> time = np.empty(1) >> >> The code now fails in the writeSpaceGridHeader function. with the >> error, >> >> >> Traceback (most recent call last): >> File >> "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 232, in >> generateXdmf(sys.argv[1]) >> File >> "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 227, in generateXdmf >> Xdmf(xdmfFilename).write(hdfFilename, topoPath, numCells, numCorners, >> cellDim, geomPath, numVertices, spaceDim, time, vfields, cfields) >> File >> "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 180, in write >> self.writeSpaceGridHeader(fp, numCells, numCorners, cellDim, spaceDim) >> File >> "/Users/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 64, in writeSpaceGridHeader >> print self.cellMap[cellDim][numCorners] >> >> The error is due to the fact that numCorners is set to be 1 , while >> celldim=2. cellMap has the following elements. >> {1: {1: 'Polyvertex', 2: 'Polyline'}, 2: {3: 'Triangle', 4: >> 'Quadrilateral'}, 3: {8: 'Hexahedron', 4: 'Tetrahedron'}} >> >> >> I also tried >> ./ex12 -dm_view vtk:my.vtk:vtk_vtu . This doesn't seem to do anything. >> Is there any specific option I need to build petsc with to get vtk output? >> My current build has hdf5 and netcdf enabled. >> >> I made the change in a branch, and merged to 'next', so it will work if >> you pull. Here is a test from ex12 (testnum 39 in builder.py): >> >> ./ex12 -run_type full -refinement_limit 0.015625 -interpolate 1 >> -petscspace_order 2 -pc_type gamg -ksp_rtol 1.0e-10 -ksp_monitor_short >> -ksp_converged_reason -snes_monitor_short -snes_converged_reason -dm_view >> hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append >> >> which makes sol.h5. Then I run >> >> ./bin/pythonscripts/petsc_gen_xdmf.py sol.h5 >> >> which makes sol.xmf. I load it up in Paraview and it makes the attached >> picture. >> >> Thanks, >> >> Matt >> >> >> Thanks, >> Subramanya >> >> Date: Wed, 24 Sep 2014 17:36:52 -0500 >> Subject: Re: [petsc-users] Generating xdmf from h5 file. >> From: knepley at gmail.com >> To: potaman at outlook.com; petsc-maint at mcs.anl.gov; petsc-users at mcs.anl.gov >> >> On Wed, Sep 24, 2014 at 5:29 PM, subramanya sadasiva > > wrote: >> >> Hi Matt, >> That did not help. >> >> >> That's not enough description to fix anything, and fixing it will require >> programming. >> >> >> Is there any other way to output the mesh to something that paraview can >> view? I tried outputting the file to a vtk file using >> ex12 -dm_view vtk:my.vtk:ascii_vtk >> >> which, I saw in another post on the forums, but that did not give me any >> output. >> >> >> This is mixing two different things. PETSc has a diagnostic ASCII vtk >> output, so the type would be ascii, not vtk, >> and format ascii_vtk . It also has a production VTU output, which is type >> vtk with format vtk_vtu. >> >> Thanks, >> >> Matt >> >> >> >> Subramanya >> >> ------------------------------ >> Date: Wed, 24 Sep 2014 17:19:51 -0500 >> Subject: Re: [petsc-users] Generating xdmf from h5 file. >> From: knepley at gmail.com >> To: potaman at outlook.com >> CC: petsc-users at mcs.anl.gov >> >> On Wed, Sep 24, 2014 at 5:08 PM, subramanya sadasiva > > wrote: >> >> Hi, >> i was trying to use petsc_gen_xdmf.py to convert a h5 file to a xdmf >> file. The h5 file was generated by snes/ex12 which was run as, >> >> ex12 -dm_view hdf5:my.h5 >> >> When I do, >> petsc_gen_xdmf.py my.h5 >> >> I get the following error, >> >> File >> "/home/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 220, in >> generateXdmf(sys.argv[1]) >> File >> "/home/ssadasiv/software/petsc/bin/pythonscripts/petsc_gen_xdmf.py", line >> 208, in generateXdmf >> time = np.array(h5['time']).flatten() >> File "/usr/lib/python2.7/dist-packages/h5py/_hl/group.py", line 153, in >> __getitem__ >> oid = h5o.open(self.id, self._e(name), lapl=self._lapl) >> File "h5o.pyx", line 173, in h5py.h5o.open (h5py/h5o.c:3403) >> KeyError: "unable to open object (Symbol table: Can't open object)" >> >> I am not sure if the error is on my end. This is on Ubuntu 14.04 with the >> serial version of hdf5. I built petsc with --download-hdf5, is it necessary >> to use the same version of hdf5 to generate the xdmf file? >> >> >> That code is alpha, and mainly built for me to experiment with an >> application here, so it is not user-friendly. In your >> HDF5 file, there is no 'time' since you are not running a TS. This access >> to h5['time'] should just be protected, and >> an empty array should be put in if its not there. >> >> Matt >> >> >> Thanks >> Subramanya >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From filippo.leonardi at sam.math.ethz.ch Wed Oct 8 04:01:01 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Wed, 8 Oct 2014 11:01:01 +0200 Subject: [petsc-users] PCMG behaviour on various levels Message-ID: <5294378.IbuJkrd7Lz@besikovitch-ii> Hi, Quick PCMG question. Default behaviour of PCMG (I'm using 3D DMDAs) is to use all processes for every level and redundantly solve the coarse level locally (to my knowledge). However, if I want to keep the level numbers to be log_c N for (c=2^d, d=dimension, N = dof of finest mesh) so that the coarse level is alway the same size (say N_0) I'll occur in scaling problems. At some point, if I increase N and the number of processors (I do N \sim #proc), I'll obtain too many processes solving the coarsest levels (to the point where each processor has less than 1 dof). Is PETSc doing this by default (and therefore, I am missing something?)? PCMGLevels is the right choice for this case. My idea is to subdivide PETSC_COMM_WORLD with Goup_Inlc groups. Have anyone tried/have an example for this (all I've got are bad failures)? My plan is solving a 3D ploblem scaling as folloiwng: 32^3 1 proc 4-6 levels 64^3 8 proc 5-7 levels 128^3 64 proc 6-8 levels .... Best, Filippo -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From jed at jedbrown.org Wed Oct 8 09:52:32 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 08 Oct 2014 08:52:32 -0600 Subject: [petsc-users] PCMG behaviour on various levels In-Reply-To: <5294378.IbuJkrd7Lz@besikovitch-ii> References: <5294378.IbuJkrd7Lz@besikovitch-ii> Message-ID: <87oatmbolr.fsf@jedbrown.org> Filippo Leonardi writes: > Hi, > > Quick PCMG question. > > Default behaviour of PCMG (I'm using 3D DMDAs) is to use all processes for > every level and redundantly solve the coarse level locally (to my knowledge). > > However, if I want to keep the level numbers to be log_c N for (c=2^d, > d=dimension, N = dof of finest mesh) so that the coarse level is alway the same > size (say N_0) I'll occur in scaling problems. At some point, if I increase N > and the number of processors (I do N \sim #proc), I'll obtain too many > processes solving the coarsest levels (to the point where each processor has > less than 1 dof). This is indeed a problem for large process counts and DMDA does not currently have a way to coarsen onto subcommunicators. I typically deal with this by switching to algebraic multigrid on the coarsest levels, e.g., -mg_coarse_pc_type gamg. GAMG manages aggregation to smaller process sets automatically. In more specific circumstances, such as HPGMG-FE, I stick with geometric multigrid and restrict the communicator on the coarse grids. The general approach could be ported to DMDA, though it is not trivial and would break some existing code (that assumes the restriction is distributed across the same process set). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From salazardetroya at gmail.com Wed Oct 8 11:38:35 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Wed, 8 Oct 2014 11:38:35 -0500 Subject: [petsc-users] When is the TSMonitor called? Message-ID: Hi all I want to save a certain quantity at the very beginning of each time step, before any TSFunction of any kind is called. Can I do this within the TSMonitor? It would also be ok to save that quantity at the very end of the time step. Thanks Miguel -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed Oct 8 12:16:29 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 08 Oct 2014 11:16:29 -0600 Subject: [petsc-users] When is the TSMonitor called? In-Reply-To: References: Message-ID: <871tqibhxu.fsf@jedbrown.org> Miguel Angel Salazar de Troya writes: > Hi all > > I want to save a certain quantity at the very beginning of each time step, > before any TSFunction of any kind is called. Can I do this within the > TSMonitor? It would also be ok to save that quantity at the very end of the > time step. This is the main time loop in TSSolve() while (!ts->reason) { ierr = TSMonitor(ts,ts->steps,ts->ptime,ts->vec_sol);CHKERRQ(ierr); ierr = TSStep(ts);CHKERRQ(ierr); ierr = TSPostStep(ts);CHKERRQ(ierr); } -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From haakon at hakostra.net Wed Oct 8 15:00:24 2014 From: haakon at hakostra.net (=?UTF-8?B?SMOla29uIFN0cmFuZGVuZXM=?=) Date: Wed, 08 Oct 2014 22:00:24 +0200 Subject: [petsc-users] 3D multi-dof DMDA and Poisson equation Message-ID: <543597D8.6070504@hakostra.net> Hi, I am making a new, simple incompressible CFD code. I have created a grid as a 3D DMDA, with ndof=4, one dof for each velocity component (u,v,w) and one for the pressure (p). Currently I am creating a forward Euler step, where the velocities are updated based on the velocities and pressure from the previous time step. That works great. Now I need to solve a Poisson equation for the pressure correction term, enforcing continuity onto my solution, and this is where I have a few questions for you: 1) How do I set up my equation system in the most efficient way? The vectors from the DMDA is not contiguous in a global sense, i.e. if I map the local (i,j,k) indices to a global index with globId = (info.mx*info.my)*k + (info-.mx)*j + i; globId is not gontignous on any process. How do I create my coefficient matrix, r.h.s. vector and solution vector? Is there some way I can make the local parts of the matrix and vectors mimic the decomposition from the DMDA? Or should I push all values into a different decomposition scheme, where each process holds one contigous slab of the equation system using the global ID's calculated above? Have I missed some really basic stuff here? 2) I know that my coefficient matrix from the Poisson equation is symmetric. Currently I am still creating and inserting values on both the upper and lower diagonal. Is this really necessary? Or is there a "smart" way I can save half the storage by only creating the upper or lower part? 3) I solve a transient problem, but my coefficient matrix is the same for all time steps and I only recompute the r.h.s. vector for each time step. I have got the impression that KSPsolve "finds out" if the A matrix have changed since last invocation or not, recalculating the preconditioner as necessary. But still I feel that I should utilize this property more. Do you have any tips in how I can use this in the most efficient way possible? For example really, really efficient, but expensive, preconditioners? Thanks in advance. Regards, H?kon Strandenes From knepley at gmail.com Wed Oct 8 15:04:28 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 8 Oct 2014 15:04:28 -0500 Subject: [petsc-users] 3D multi-dof DMDA and Poisson equation In-Reply-To: <543597D8.6070504@hakostra.net> References: <543597D8.6070504@hakostra.net> Message-ID: On Wed, Oct 8, 2014 at 3:00 PM, H?kon Strandenes wrote: > Hi, > > I am making a new, simple incompressible CFD code. I have created a grid > as a 3D DMDA, with ndof=4, one dof for each velocity component (u,v,w) and > one for the pressure (p). Currently I am creating a forward Euler step, > where the velocities are updated based on the velocities and pressure from > the previous time step. That works great. Now I need to solve a Poisson > equation for the pressure correction term, enforcing continuity onto my > solution, and this is where I have a few questions for you: > > 1) > How do I set up my equation system in the most efficient way? The vectors > from the DMDA is not contiguous in a global sense, i.e. if I map the local > (i,j,k) indices to a global index with > > globId = (info.mx*info.my)*k + (info-.mx)*j + i; > > globId is not gontignous on any process. How do I create my coefficient > matrix, r.h.s. vector and solution vector? Is there some way I can make the > local parts of the matrix and vectors mimic the decomposition from the > DMDA? Or should I push all values into a different decomposition scheme, > where each process holds one contigous slab of the equation system using > the global ID's calculated above? Have I missed some really basic stuff > here? > See SNES ex5 (which is Poisson when you throw away the extra e^u term). Hopefully this will explain your question about numbering. > 2) > I know that my coefficient matrix from the Poisson equation is symmetric. > Currently I am still creating and inserting values on both the upper and > lower diagonal. Is this really necessary? Or is there a "smart" way I can > save half the storage by only creating the upper or lower part? > You can save storage, but not work, so unless you are memory constrained, this is not worth doing. > 3) > I solve a transient problem, but my coefficient matrix is the same for all > time steps and I only recompute the r.h.s. vector for each time step. I > have got the impression that KSPsolve "finds out" if the A matrix have > changed since last invocation or not, recalculating the preconditioner as > necessary. But still I feel that I should utilize this property more. Do > you have any tips in how I can use this in the most efficient way possible? > For example really, really efficient, but expensive, preconditioners? > We should figure this out automatically. Please report if this does not happen for you. Also, we recommend using TS for timestepping (at least eventually). Thanks, Matt > Thanks in advance. > > Regards, > H?kon Strandenes > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed Oct 8 15:07:04 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 08 Oct 2014 14:07:04 -0600 Subject: [petsc-users] 3D multi-dof DMDA and Poisson equation In-Reply-To: <543597D8.6070504@hakostra.net> References: <543597D8.6070504@hakostra.net> Message-ID: <871tqi497b.fsf@jedbrown.org> H?kon Strandenes writes: > Hi, > > I am making a new, simple incompressible CFD code. I have created a grid > as a 3D DMDA, with ndof=4, one dof for each velocity component (u,v,w) > and one for the pressure (p). Currently I am creating a forward Euler > step, where the velocities are updated based on the velocities and > pressure from the previous time step. That works great. Now I need to > solve a Poisson equation for the pressure correction term, enforcing > continuity onto my solution, and this is where I have a few questions > for you: > > 1) > How do I set up my equation system in the most efficient way? The > vectors from the DMDA is not contiguous in a global sense, i.e. if I map > the local (i,j,k) indices to a global index with > > globId = (info.mx*info.my)*k + (info-.mx)*j + i; This is a "natural index", used in some cases for IO, but not for computation. > globId is not gontignous on any process. How do I create my coefficient > matrix, r.h.s. vector and solution vector? DMCreateGlobalVector() it will use what is described in the Users Manual as "PETSc ordering". Please read this section of the manual; I think it will answer your questions. > Is there some way I can make the local parts of the matrix and vectors > mimic the decomposition from the DMDA? Or should I push all values > into a different decomposition scheme, where each process holds one > contigous slab of the equation system using the global ID's calculated > above? Have I missed some really basic stuff here? > > 2) > I know that my coefficient matrix from the Poisson equation is > symmetric. Currently I am still creating and inserting values on both > the upper and lower diagonal. Is this really necessary? Or is there a > "smart" way I can save half the storage by only creating the upper or > lower part? -dm_mat_type sbaij > 3) > I solve a transient problem, but my coefficient matrix is the same for > all time steps and I only recompute the r.h.s. vector for each time > step. I have got the impression that KSPsolve "finds out" if the A > matrix have changed since last invocation or not, recalculating the > preconditioner as necessary. If you just call KSPSolve, it will reuse everything. Call KSPSetOperators (assuming you are manually doing the time discretization) if the matrix changes. > But still I feel that I should utilize this property more. Do you have > any tips in how I can use this in the most efficient way possible? For > example really, really efficient, but expensive, preconditioners? > > Thanks in advance. > > Regards, > H?kon Strandenes -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From natacha.bereux at gmail.com Thu Oct 9 03:12:39 2014 From: natacha.bereux at gmail.com (Natacha BEREUX) Date: Thu, 9 Oct 2014 10:12:39 +0200 Subject: [petsc-users] Block size Message-ID: Dear PETSc users, I am a bit confused about blocksizes in MPI matrices. I define a MPI matrix A with a certain blocksize, let say bs = 3. As far as I understand, the sparse pattern of the matrix A is made of square blocks of size 3x3, regardless of the actual values of the terms of A. Am I right ? The matrix A is distributed among several processors: does the local number of rows of A on each processor have to be a multiple of bs ? I want to use a multigrid preconditionner to solve the linear system of matrix A : A x = b Is it mandatory to define the right hand side b as a vector with the same blocksize bs=3 ? Thank you very much of your help, Best regards, Natacha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 9 03:19:16 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 9 Oct 2014 03:19:16 -0500 Subject: [petsc-users] Block size In-Reply-To: References: Message-ID: On Oct 9, 2014, at 3:12 AM, Natacha BEREUX wrote: > Dear PETSc users, > > I am a bit confused about blocksizes in MPI matrices. > > I define a MPI matrix A with a certain blocksize, let say bs = 3. > As far as I understand, the sparse pattern of the matrix A is made of square blocks of size 3x3, regardless of the actual values of the terms of A. > Am I right ? Only if you use the BAIJ or SBAIJ matrix format. If you use AIJ then it still stores the values without regard to the block size but it carries knowledge of the block size around and that gets used in some places such as the GAMG solver. > > The matrix A is distributed among several processors: does the local number of rows of A on each processor have to be a multiple of bs ? Correct > > I want to use a multigrid preconditionner to solve the linear system of matrix A : A x = b > Is it mandatory to define the right hand side b as a vector with the same blocksize bs=3 ? It probably doesn?t matter, but if you know something has an associated block size it is generally best to capture it as soon as you can in the problem. Barry > > Thank you very much of your help, > Best regards, > Natacha From Eric.Chamberland at giref.ulaval.ca Thu Oct 9 09:00:30 2014 From: Eric.Chamberland at giref.ulaval.ca (Eric Chamberland) Date: Thu, 09 Oct 2014 10:00:30 -0400 Subject: [petsc-users] Is there a bug in VecRestoreSubVector? Message-ID: <543694FE.8050500@giref.ulaval.ca> Hi, I noticed it because I got a strange behavior when computing y=Ax with (a wrongly) non-contiguous stride in parallel. Normally, the VecRestoreSubVector should have exited with an error ("Unhandled case, values have been changed and need to be copied back into X) but didn't, resulting in a silently wrong result (ouch). I corrected my code to define a contiguous stride, which "bypass" the bug in VecRestoreSubVector and now I hopefully compute the correct result. You may be interested by: https://bitbucket.org/placasse/petsc/commits/4e4323587ad9148c29ba9101e5030629f073700c By the way, if I have no local unknowns on a process, shall the stride *always* be considered locally contiguous? (this is my initial error on how to define a stride when no unknowns are locally owned). Thanks, Eric From jed at jedbrown.org Thu Oct 9 10:41:16 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 09 Oct 2014 09:41:16 -0600 Subject: [petsc-users] Is there a bug in VecRestoreSubVector? In-Reply-To: <543694FE.8050500@giref.ulaval.ca> References: <543694FE.8050500@giref.ulaval.ca> Message-ID: <87r3yh1c9v.fsf@jedbrown.org> Hi, Eric. Fabian reported this a couple days ago (all, petsc-users is preferred so that people can see recent discussion). Thanks for linking to Patrick's report. I have extended it to simplify the code a bit and reduce synchronization, but I don't have a test case handy and I'm about to board a plane. Could you try 'next' (or 'placasse/VecGetSubVector' in petsc.git, which has my follow-up commits) to let me know if it works for you. https://bitbucket.org/petsc/petsc/issue/75/vecrestoresubvector-silently-ignores-non Eric Chamberland writes: > Hi, > > I noticed it because I got a strange behavior when computing y=Ax with > (a wrongly) non-contiguous stride in parallel. Normally, the > VecRestoreSubVector should have exited with an error ("Unhandled case, > values have been changed and need to be copied back into X) but didn't, > resulting in a silently wrong result (ouch). > > I corrected my code to define a contiguous stride, which "bypass" the > bug in VecRestoreSubVector and now I hopefully compute the correct result. > > You may be interested by: > > > https://bitbucket.org/placasse/petsc/commits/4e4323587ad9148c29ba9101e5030629f073700c > > By the way, if I have no local unknowns on a process, shall the stride > *always* be considered locally contiguous? (this is my initial error on > how to define a stride when no unknowns are locally owned). > > Thanks, > > Eric -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From Eric.Chamberland at giref.ulaval.ca Thu Oct 9 11:06:44 2014 From: Eric.Chamberland at giref.ulaval.ca (Eric Chamberland) Date: Thu, 09 Oct 2014 12:06:44 -0400 Subject: [petsc-users] Is there a bug in VecRestoreSubVector? In-Reply-To: <87r3yh1c9v.fsf@jedbrown.org> References: <543694FE.8050500@giref.ulaval.ca> <87r3yh1c9v.fsf@jedbrown.org> Message-ID: <5436B294.2040604@giref.ulaval.ca> Hi Jed, I do not have a petsc/dev or petsc/next compiled (I just use petsc 3.5.2), but Patrick have one for sure. He should be able to try the patch sooner than I. Patrick, would you give it a try please? Thanks, Eric On 10/09/2014 11:41 AM, Jed Brown wrote: > Hi, Eric. Fabian reported this a couple days ago (all, petsc-users is > preferred so that people can see recent discussion). > > Thanks for linking to Patrick's report. I have extended it to simplify > the code a bit and reduce synchronization, but I don't have a test case > handy and I'm about to board a plane. Could you try 'next' (or > 'placasse/VecGetSubVector' in petsc.git, which has my follow-up commits) > to let me know if it works for you. > > https://bitbucket.org/petsc/petsc/issue/75/vecrestoresubvector-silently-ignores-non > > Eric Chamberland writes: > >> Hi, >> >> I noticed it because I got a strange behavior when computing y=Ax with >> (a wrongly) non-contiguous stride in parallel. Normally, the >> VecRestoreSubVector should have exited with an error ("Unhandled case, >> values have been changed and need to be copied back into X) but didn't, >> resulting in a silently wrong result (ouch). >> >> I corrected my code to define a contiguous stride, which "bypass" the >> bug in VecRestoreSubVector and now I hopefully compute the correct result. >> >> You may be interested by: >> >> >> https://bitbucket.org/placasse/petsc/commits/4e4323587ad9148c29ba9101e5030629f073700c >> >> By the way, if I have no local unknowns on a process, shall the stride >> *always* be considered locally contiguous? (this is my initial error on >> how to define a stride when no unknowns are locally owned). >> >> Thanks, >> >> Eric From jed at jedbrown.org Thu Oct 9 11:09:31 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 09 Oct 2014 10:09:31 -0600 Subject: [petsc-users] Is there a bug in VecRestoreSubVector? In-Reply-To: <5436B294.2040604@giref.ulaval.ca> References: <543694FE.8050500@giref.ulaval.ca> <87r3yh1c9v.fsf@jedbrown.org> <5436B294.2040604@giref.ulaval.ca> Message-ID: <829C52DA-2A51-42A9-B51B-97BE32D7D5E6@jedbrown.org> If you have a Git clone, you can make a temporary merge to see if it works. On October 9, 2014 10:06:44 AM MDT, Eric Chamberland wrote: >Hi Jed, > >I do not have a petsc/dev or petsc/next compiled (I just use petsc >3.5.2), but Patrick have one for sure. He should be able to try the >patch sooner than I. > >Patrick, would you give it a try please? > >Thanks, > >Eric > >On 10/09/2014 11:41 AM, Jed Brown wrote: >> Hi, Eric. Fabian reported this a couple days ago (all, petsc-users >is >> preferred so that people can see recent discussion). >> >> Thanks for linking to Patrick's report. I have extended it to >simplify >> the code a bit and reduce synchronization, but I don't have a test >case >> handy and I'm about to board a plane. Could you try 'next' (or >> 'placasse/VecGetSubVector' in petsc.git, which has my follow-up >commits) >> to let me know if it works for you. >> >> >https://bitbucket.org/petsc/petsc/issue/75/vecrestoresubvector-silently-ignores-non >> >> Eric Chamberland writes: >> >>> Hi, >>> >>> I noticed it because I got a strange behavior when computing y=Ax >with >>> (a wrongly) non-contiguous stride in parallel. Normally, the >>> VecRestoreSubVector should have exited with an error ("Unhandled >case, >>> values have been changed and need to be copied back into X) but >didn't, >>> resulting in a silently wrong result (ouch). >>> >>> I corrected my code to define a contiguous stride, which "bypass" >the >>> bug in VecRestoreSubVector and now I hopefully compute the correct >result. >>> >>> You may be interested by: >>> >>> >>> >https://bitbucket.org/placasse/petsc/commits/4e4323587ad9148c29ba9101e5030629f073700c >>> >>> By the way, if I have no local unknowns on a process, shall the >stride >>> *always* be considered locally contiguous? (this is my initial error >on >>> how to define a stride when no unknowns are locally owned). >>> >>> Thanks, >>> >>> Eric From carlesbona at gmail.com Thu Oct 9 12:25:47 2014 From: carlesbona at gmail.com (Carles Bona) Date: Thu, 9 Oct 2014 19:25:47 +0200 Subject: [petsc-users] Convergence of different fields/equations Message-ID: Hi all, I am currently running petsc to solve a time-dependent nonlinear system with default options (NewtonLS, etc.). I have a question regarding convergence. My application evolves different fields and equations, which might be in different units (like temperature and velocity) and might not involve the same order of derivatives (for example second derivatives, scaling as ~1/dx^2 or first derivatives, scaling as ~1/dx). My application also has a handicap: the tangent matrix is cumbersome to calculate. >From what I understand and I see in the source code, convergence norms are performed on the whole residual, that is, including all fields/equations. Am I wrong? My concern is, if I am using a relative convergence criteria for the residual norm, it might happen that I am solving one of my equations very nicely and another equation shouldn't be converging yet... But just because it's dealing with a much smaller residual than the other equation from the very beginning, the residual norm is not caring about it and it's showing convergence too early (specially if I don't run with the default parameters anymore and I ask for a relative convergence of 10^-3, for example). I have been looking at the documentation and mailing list and I haven't been able to find if there is a specific petsc option to prevent this from happening (other than setting a very restrictive relative convergence, which is painful for me as I don't have a nice tangent matrix) or if I should implement a convergence criteria myself, probably something like what has been suggested here: http://lists.mcs.anl.gov/pipermail/petsc-users/2014-August/022577.html Thanks in advance, Carles -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 9 13:24:53 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 9 Oct 2014 13:24:53 -0500 Subject: [petsc-users] Convergence of different fields/equations In-Reply-To: References: Message-ID: If your fields are nicely organized you can use VecStrideNorm() or VecStrideNormAll() to get the norms of particular fields of your residual. If they are not simply organized by stride you can use VecGetArrayRead()/VecRestoreArrayRead() to access all the fields directly and then compute the norms yourself (doing the MPI_Allreduce() to accumulate over the processes) by pulling out the entries of the vector that you need. Barry On Oct 9, 2014, at 12:25 PM, Carles Bona wrote: > Hi all, > > I am currently running petsc to solve a time-dependent nonlinear system with default options (NewtonLS, etc.). I have a question regarding convergence. My application evolves different fields and equations, which might be in different units (like temperature and velocity) and might not involve the same order of derivatives (for example second derivatives, scaling as ~1/dx^2 or first derivatives, scaling as ~1/dx). My application also has a handicap: the tangent matrix is cumbersome to calculate. > > From what I understand and I see in the source code, convergence norms are performed on the whole residual, that is, including all fields/equations. Am I wrong? > > My concern is, if I am using a relative convergence criteria for the residual norm, it might happen that I am solving one of my equations very nicely and another equation shouldn't be converging yet... But just because it's dealing with a much smaller residual than the other equation from the very beginning, the residual norm is not caring about it and it's showing convergence too early (specially if I don't run with the default parameters anymore and I ask for a relative convergence of 10^-3, for example). > > I have been looking at the documentation and mailing list and I haven't been able to find if there is a specific petsc option to prevent this from happening (other than setting a very restrictive relative convergence, which is painful for me as I don't have a nice tangent matrix) or if I should implement a convergence criteria myself, probably something like what has been suggested here: http://lists.mcs.anl.gov/pipermail/petsc-users/2014-August/022577.html > > Thanks in advance, > > Carles From hsahasra at purdue.edu Thu Oct 9 16:52:32 2014 From: hsahasra at purdue.edu (Harshad Sahasrabudhe) Date: Thu, 9 Oct 2014 17:52:32 -0400 Subject: [petsc-users] Scattering a sequential vector Message-ID: Hi, I need to scatter (distribute) a sequential vector on process 0 to all the processes according to a know set of global indices for each process. How can this be done using VecScatter? Thanks, Harshad -------------- next part -------------- An HTML attachment was scrubbed... URL: From popov at uni-mainz.de Thu Oct 9 16:57:41 2014 From: popov at uni-mainz.de (Anton Popov) Date: Thu, 9 Oct 2014 23:57:41 +0200 Subject: [petsc-users] Scattering a sequential vector In-Reply-To: References: Message-ID: <543704D5.2080205@uni-mainz.de> On 10/9/14 11:52 PM, Harshad Sahasrabudhe wrote: > Hi, > > I need to scatter (distribute) a sequential vector on process 0 to all > the processes according to a know set of global indices for each > process. How can this be done using VecScatter? > > Thanks, > Harshad looks like this is appropriate: http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToZero.html#VecScatterCreateToZero Anton -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 9 16:59:15 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 9 Oct 2014 16:59:15 -0500 Subject: [petsc-users] Scattering a sequential vector In-Reply-To: References: Message-ID: <80EFDABB-5AB8-4944-8053-2E46BE41303C@mcs.anl.gov> All processes except the 0th one create a sequential vector of size 0. All processes except 0 create two ISes of size 0. Process zero creates two ISes with the from locations (local in the sequential vector) and the to locations (global in the global vector) then all processes call VecScatterCreate() in the usual manner Barry On Oct 9, 2014, at 4:52 PM, Harshad Sahasrabudhe wrote: > Hi, > > I need to scatter (distribute) a sequential vector on process 0 to all the processes according to a know set of global indices for each process. How can this be done using VecScatter? > > Thanks, > Harshad From natacha.bereux at gmail.com Fri Oct 10 02:11:06 2014 From: natacha.bereux at gmail.com (Natacha BEREUX) Date: Fri, 10 Oct 2014 09:11:06 +0200 Subject: [petsc-users] Block size In-Reply-To: References: Message-ID: Thanks a lot for your answer! Regards, Natacha On Thu, Oct 9, 2014 at 10:19 AM, Barry Smith wrote: > > On Oct 9, 2014, at 3:12 AM, Natacha BEREUX > wrote: > > > Dear PETSc users, > > > > I am a bit confused about blocksizes in MPI matrices. > > > > I define a MPI matrix A with a certain blocksize, let say bs = 3. > > As far as I understand, the sparse pattern of the matrix A is made of > square blocks of size 3x3, regardless of the actual values of the terms of > A. > > Am I right ? > > Only if you use the BAIJ or SBAIJ matrix format. If you use AIJ then it > still stores the values without regard to the block size but it carries > knowledge of the block size around and that gets used in some places such > as the GAMG solver. > > > > The matrix A is distributed among several processors: does the local > number of rows of A on each processor have to be a multiple of bs ? > > Correct > > > > I want to use a multigrid preconditionner to solve the linear system of > matrix A : A x = b > > Is it mandatory to define the right hand side b as a vector with the > same blocksize bs=3 ? > > It probably doesn?t matter, but if you know something has an associated > block size it is generally best to capture it as soon as you can in the > problem. > > Barry > > > > > Thank you very much of your help, > > Best regards, > > Natacha > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wdn at lanl.gov Fri Oct 10 21:30:19 2014 From: wdn at lanl.gov (Nystrom, William David) Date: Sat, 11 Oct 2014 02:30:19 +0000 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <871tqqrt8p.fsf@jedbrown.org> References: , <871tqqrt8p.fsf@jedbrown.org> Message-ID: Jed, I don't think your reply below really does what I want to do. My wdn_mods_next branch is a branch that I maintain that was derived from the next branch. I do not merge changes from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of the official petsc next branch and I only update the next branch in my petsc clones by first performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint, master and next branches as well as containing any additional branches that I create. So, I update wdn_mods_next from the latest copy of next that I have by using the following command: git pull origin next in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is because my wdn_mods_next branch does not differ much from next - just a few hacks I have had to resort to. So here is one way I think I could accomplish my goal. In my bare, mirror repo execute git remote -v update origin At this point my bare mirror repo now has a copy of the new unwound next branch - if I am understanding correctly. Then in a clone of my mirror repo execute git checkout wdn_mods_next git reset --hard next git push origin +wdn_mods_next If I understand correctly, then my wdn_mods_next branch would be identical to the next branch and I would not have the changes I made to next that resulted in wdn_mods_next. Then I could untar a file of those changes on top of my clone with wdn_mods_next checked out and recommit those changes. This is about half a dozen files. But this seems like a hacky way to do business that is only reasonable because I have a small number of files. Isn't there a better way to do this? Perhaps you or Satish might ask why I am using next instead of master. The reason is that I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't want to wait for several days before it gets pulled into master and I can resume productive work. So maybe there is some flaw or weakness in my git work flow. I have been using git for several years now but am not a power user. I like git but do not really want to have to become a git guru. However, this petsc policy of periodically unwinding the next branch seems to always disrupt my workflow - so I have not found a robust way to deal with it yet. Dave ________________________________________ From: Jed Brown [jed at jedbrown.org] Sent: Thursday, October 02, 2014 4:39 PM To: Satish Balay; Nystrom, William David Cc: petsc-users; petsc-dev at mcs.anl.gov Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! Satish Balay writes: > You would have to delete both 'next' and 'wdn_mods_next' branches from > all your clones - and recreate them. Do your normal "git remote update" (or git fetch) in your "master" clone (confusing to use the same name to refer to a repository and a branch). I'll call the repository Mirror. Anyway, you should see that all three branches are updated. 'maint' and 'master' should fast-forward, while 'next' will say "forced update". After that, recreate 'next' on the clones just like in Satish's original email. From balay at mcs.anl.gov Fri Oct 10 22:54:26 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 10 Oct 2014 22:54:26 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: , <871tqqrt8p.fsf@jedbrown.org> Message-ID: I believe the workflow used by petsc is same as what most projects with git do - so there is nothing special about it. Try 'man gitworkflows' I think your reason for using next is flawed (you get more bugs than bugfixes on next - as its an integration branch - and features are merged here for testing purposes). But if thats what you want to do - thats fine. We can benift with the extra testing that you are now providing on next. And I believe git gives you tools to manage whatever workflow you choose. Generally we don't branch off next. New branches are created from master (or maint). This works well for new features workflow (or bug fix workflow). In your case - you'd like to maintain 'wdn_mods_next' branch - which is 'next' + local fixes that are specific to next (and not shared across other branches - like master). And these fixes are next specific - hence you want this branch to start off next. Presumably this is low over head stuff - and don't need permanant history for this (as we usually do with feature branches) For the above requirement - currently you do: a. branch wdn_mods_next from next b. add local commits in this branch c. merge latest next into wdn_mods_next. (as needed) Alternative is to use 'git rebase' to manage your local commits. This is not critical - but it lets you keep track of them more sanely. git checkout next git pull git checkout wdn_mods_next git rebase next [also check 'git rebase -i'] In the present situation - when next is unwound - you have to unwind and recreate wdn_mods_next. For this you'd like to extract your current modifications from wdn_mods_next - and apply to a new wdn_mods_next branch. git checkout wdn_mods_next git checkout -b wdn_mods_next-oct-2014 git checkout wdn_mods_next git reset --hard next # assuming you've already reset 'next' to latest git log origin/next-oct-2014..wdn_mods_next-oct-2014 git diff origin/next-oct-2014..wdn_mods_next-oct-2014 Now you can 'git cherrypick' the commits you need [from the above list] or grab the above diff - and apply to dn_mods_next(new) branch. for eg: git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > my_changes patch -Np1 < my_changes git commit Satish On Fri, 10 Oct 2014, Nystrom, William David wrote: > Jed, > > I don't think your reply below really does what I want to do. My wdn_mods_next branch is > a branch that I maintain that was derived from the next branch. I do not merge changes > from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of > the official petsc next branch and I only update the next branch in my petsc clones by first > performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint, > master and next branches as well as containing any additional branches that I create. > > So, I update wdn_mods_next from the latest copy of next that I have by using the following > command: > > git pull origin next > > in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and > has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is > because my wdn_mods_next branch does not differ much from next - just a few hacks I have > had to resort to. > > So here is one way I think I could accomplish my goal. In my bare, mirror repo execute > > git remote -v update origin > > At this point my bare mirror repo now has a copy of the new unwound next branch - if I am > understanding correctly. > > Then in a clone of my mirror repo execute > > git checkout wdn_mods_next > git reset --hard next > git push origin +wdn_mods_next > > If I understand correctly, then my wdn_mods_next branch would be identical to the next > branch and I would not have the changes I made to next that resulted in wdn_mods_next. > > Then I could untar a file of those changes on top of my clone with wdn_mods_next checked > out and recommit those changes. This is about half a dozen files. But this seems like a > hacky way to do business that is only reasonable because I have a small number of files. > > Isn't there a better way to do this? > > Perhaps you or Satish might ask why I am using next instead of master. The reason is that > I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't > want to wait for several days before it gets pulled into master and I can resume productive > work. > > So maybe there is some flaw or weakness in my git work flow. I have been using git for > several years now but am not a power user. I like git but do not really want to have to > become a git guru. However, this petsc policy of periodically unwinding the next branch > seems to always disrupt my workflow - so I have not found a robust way to deal with it > yet. > > Dave > > ________________________________________ > From: Jed Brown [jed at jedbrown.org] > Sent: Thursday, October 02, 2014 4:39 PM > To: Satish Balay; Nystrom, William David > Cc: petsc-users; petsc-dev at mcs.anl.gov > Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! > > Satish Balay writes: > > > You would have to delete both 'next' and 'wdn_mods_next' branches from > > all your clones - and recreate them. > > Do your normal "git remote update" (or git fetch) in your "master" clone > (confusing to use the same name to refer to a repository and a branch). > I'll call the repository Mirror. Anyway, you should see that all three > branches are updated. 'maint' and 'master' should fast-forward, while > 'next' will say "forced update". > > After that, recreate 'next' on the clones just like in Satish's original > email. > From dnystrom1 at comcast.net Fri Oct 10 23:26:51 2014 From: dnystrom1 at comcast.net (Dave Nystrom) Date: Fri, 10 Oct 2014 22:26:51 -0600 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: References: <871tqqrt8p.fsf@jedbrown.org> Message-ID: <21560.45451.738409.958412@naxion.lsscorp.com> Thanks for your detailed reply. Suppose I start working with wdn_mods_master instead of wdn_mods_next where wdn_mods_master is master + my local changes to master. Now, suppose I find a bug which I report. With the current petsc workflow, it seems like that bug would get fixed on next and then merged to master after some amount of time. Is that correct? If so, then I have to wait for that amount of time before I can have the fix. Or I would need to figure out how to pull it into my wdn_mods_master from next. And then most of the petsc features I am most interested in show up in next before master - like the stuff with cuda, threads and ViennaCL, etc. So I'm not sure how to really dispense with my use of and reliance on next. But I will give it more thought. And I will study the details of your reply below and see if I can upgrade my git skills. But I've never had to use the rebase, reset or cherry picking features of git before. So it will take take some study. Satish Balay writes: > I believe the workflow used by petsc is same as what most projects > with git do - so there is nothing special about it. > > Try 'man gitworkflows' I'll check this out. > I think your reason for using next is flawed (you get more bugs than > bugfixes on next - as its an integration branch - and features are > merged here for testing purposes). But if thats what you want to do - > thats fine. We can benefit with the extra testing that you are now > providing on next. And I believe git gives you tools to manage > whatever workflow you choose. > > Generally we don't branch off next. New branches are created from > master (or maint). This works well for new features workflow (or bug > fix workflow). > > In your case - you'd like to maintain 'wdn_mods_next' branch - which > is 'next' + local fixes that are specific to next (and not shared > across other branches - like master). And these fixes are next > specific - hence you want this branch to start off next. Presumably > this is low over head stuff - and don't need permanant history for > this (as we usually do with feature branches) It is low over head and I don't need a permanent history of petsc changes. > For the above requirement - currently you do: > > a. branch wdn_mods_next from next > b. add local commits in this branch > c. merge latest next into wdn_mods_next. (as needed) Right. > Alternative is to use 'git rebase' to manage your local commits. This > is not critical - but it lets you keep track of them more sanely. > > > git checkout next > git pull > git checkout wdn_mods_next > git rebase next > > [also check 'git rebase -i'] OK, I don't really know what rebase does - so I will have to study the documentation and my git book. > In the present situation - when next is unwound - you have to unwind > and recreate wdn_mods_next. For this you'd like to extract your > current modifications from wdn_mods_next - and apply to a new > wdn_mods_next branch. > > > git checkout wdn_mods_next > git checkout -b wdn_mods_next-oct-2014 > > > git checkout wdn_mods_next > git reset --hard next # assuming you've already reset 'next' to latest At the moment, I'm not sure the above git reset command will work - based on trying it and getting a git error message. > > git log origin/next-oct-2014..wdn_mods_next-oct-2014 > git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > > Now you can 'git cherrypick' the commits you need [from the above > list] or grab the above diff - and apply to dn_mods_next(new) branch. for eg: > > git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > my_changes > patch -Np1 < my_changes > > git commit This seems like a complicated procedure but maybe it will seem simpler and more intuitive if I study this more. Thanks, Dave > Satish > > On Fri, 10 Oct 2014, Nystrom, William David wrote: > > > Jed, > > > > I don't think your reply below really does what I want to do. My wdn_mods_next branch is > > a branch that I maintain that was derived from the next branch. I do not merge changes > > from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of > > the official petsc next branch and I only update the next branch in my petsc clones by first > > performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint, > > master and next branches as well as containing any additional branches that I create. > > > > So, I update wdn_mods_next from the latest copy of next that I have by using the following > > command: > > > > git pull origin next > > > > in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and > > has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is > > because my wdn_mods_next branch does not differ much from next - just a few hacks I have > > had to resort to. > > > > So here is one way I think I could accomplish my goal. In my bare, mirror repo execute > > > > git remote -v update origin > > > > At this point my bare mirror repo now has a copy of the new unwound next branch - if I am > > understanding correctly. > > > > Then in a clone of my mirror repo execute > > > > git checkout wdn_mods_next > > git reset --hard next > > git push origin +wdn_mods_next > > > > If I understand correctly, then my wdn_mods_next branch would be identical to the next > > branch and I would not have the changes I made to next that resulted in wdn_mods_next. > > > > Then I could untar a file of those changes on top of my clone with wdn_mods_next checked > > out and recommit those changes. This is about half a dozen files. But this seems like a > > hacky way to do business that is only reasonable because I have a small number of files. > > > > Isn't there a better way to do this? > > > > Perhaps you or Satish might ask why I am using next instead of master. The reason is that > > I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't > > want to wait for several days before it gets pulled into master and I can resume productive > > work. > > > > So maybe there is some flaw or weakness in my git work flow. I have been using git for > > several years now but am not a power user. I like git but do not really want to have to > > become a git guru. However, this petsc policy of periodically unwinding the next branch > > seems to always disrupt my workflow - so I have not found a robust way to deal with it > > yet. > > > > Dave > > > > ________________________________________ > > From: Jed Brown [jed at jedbrown.org] > > Sent: Thursday, October 02, 2014 4:39 PM > > To: Satish Balay; Nystrom, William David > > Cc: petsc-users; petsc-dev at mcs.anl.gov > > Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! > > > > Satish Balay writes: > > > > > You would have to delete both 'next' and 'wdn_mods_next' branches from > > > all your clones - and recreate them. > > > > Do your normal "git remote update" (or git fetch) in your "master" clone > > (confusing to use the same name to refer to a repository and a branch). > > I'll call the repository Mirror. Anyway, you should see that all three > > branches are updated. 'maint' and 'master' should fast-forward, while > > 'next' will say "forced update". > > > > After that, recreate 'next' on the clones just like in Satish's original > > email. > > From balay at mcs.anl.gov Sat Oct 11 00:34:20 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Sat, 11 Oct 2014 00:34:20 -0500 Subject: [petsc-users] [petsc-dev] petsc git next branch *is* unwound! In-Reply-To: <21560.45451.738409.958412@naxion.lsscorp.com> References: <871tqqrt8p.fsf@jedbrown.org> <21560.45451.738409.958412@naxion.lsscorp.com> Message-ID: On Fri, 10 Oct 2014, Dave Nystrom wrote: > Thanks for your detailed reply. Suppose I start working with wdn_mods_master > instead of wdn_mods_next where wdn_mods_master is master + my local changes > to master. Now, suppose I find a bug which I report. With the current petsc > workflow, it seems like that bug would get fixed on next and then merged to > master after some amount of time. Is that correct? Nope - the bug is fixed in a corresponding 'feature' branch [where the bug was originally introduced] or a new 'bug-fix' branch [created off latest master/maint] or directly in master/maint (if we think its a trivial fix that wont' break stuff). And then this branch is merged to next for further testing. Note: We don't develop (or make direct commits) on 'next'. It is an integration branch where feature/bugfix branches are merged into - and from where nightly tests are done. We find bugs with this testing - or we realize (very rarely) some of these feature branches won't graduate and make it to master [or need to be redone]. Hence its discard after every release. You are correct in the sense that most branches [sometimes bugfixes] have to stay in next for some time - before they get merged to master. And this 'time' 1 day or more [due to the way some of this testing is done - party the shedule - partly the amount of testing - and partly how we manually interpret test results and match them with these branches] > If so, then I have to > wait for that amount of time before I can have the fix. Or I would need to > figure out how to pull it into my wdn_mods_master from next. You can temporarily switch to the 'bug-fix' branch and use it - or merge it [but then be prepared to revert it - if this branch is rebased or discarded]. Yeah - its not ideal. But one of the points of git is to use/discard branches as you need. [trying to stick to one branch to track multi-branch development can be difficult] > And then most of the petsc features I am most interested in show up in next > before master - like the stuff with cuda, threads and ViennaCL, etc. So I'm > not sure how to really dispense with my use of and reliance on next. But I > will give it more thought. Yes - next is where the features show up first. But thats different then 'use next for bugfixes' - as 'next' is likely to have more new bugs [due to these new features] than bugfixes. > And I will study the details of your reply below and see if I can upgrade my > git skills. But I've never had to use the rebase, reset or cherry picking > features of git before. So it will take take some study. And based on the issue you are having with 'git reset --hard next' (the 'next' part - not the 'git reset' part) - an understanding of git branches is perhaps needed. Without it - its not easy to use git [or communicate about it] for eg: - understaning origin/branchname vs branchname [and how fetch/pull (and other basic operations) differ on origin/branchname vs branchname - use of commit-id vs branchname in various git commands - branchname vs tag - what is a 'new branch' - and what does 'delete branch' mean etc.. Satish > > Satish Balay writes: > > I believe the workflow used by petsc is same as what most projects > > with git do - so there is nothing special about it. > > > > Try 'man gitworkflows' > > I'll check this out. > > > I think your reason for using next is flawed (you get more bugs than > > bugfixes on next - as its an integration branch - and features are > > merged here for testing purposes). But if thats what you want to do - > > thats fine. We can benefit with the extra testing that you are now > > providing on next. And I believe git gives you tools to manage > > whatever workflow you choose. > > > > Generally we don't branch off next. New branches are created from > > master (or maint). This works well for new features workflow (or bug > > fix workflow). > > > > In your case - you'd like to maintain 'wdn_mods_next' branch - which > > is 'next' + local fixes that are specific to next (and not shared > > across other branches - like master). And these fixes are next > > specific - hence you want this branch to start off next. Presumably > > this is low over head stuff - and don't need permanant history for > > this (as we usually do with feature branches) > > It is low over head and I don't need a permanent history of petsc changes. > > > For the above requirement - currently you do: > > > > a. branch wdn_mods_next from next > > b. add local commits in this branch > > c. merge latest next into wdn_mods_next. (as needed) > > Right. > > > Alternative is to use 'git rebase' to manage your local commits. This > > is not critical - but it lets you keep track of them more sanely. > > > > > > git checkout next > > git pull > > git checkout wdn_mods_next > > git rebase next > > > > [also check 'git rebase -i'] > > OK, I don't really know what rebase does - so I will have to study the > documentation and my git book. > > > In the present situation - when next is unwound - you have to unwind > > and recreate wdn_mods_next. For this you'd like to extract your > > current modifications from wdn_mods_next - and apply to a new > > wdn_mods_next branch. > > > > > > git checkout wdn_mods_next > > git checkout -b wdn_mods_next-oct-2014 > > > > > > git checkout wdn_mods_next > > git reset --hard next # assuming you've already reset 'next' to latest > > At the moment, I'm not sure the above git reset command will work - based on > trying it and getting a git error message. > > > > > git log origin/next-oct-2014..wdn_mods_next-oct-2014 > > git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > > > > Now you can 'git cherrypick' the commits you need [from the above > > list] or grab the above diff - and apply to dn_mods_next(new) branch. for eg: > > > > git diff origin/next-oct-2014..wdn_mods_next-oct-2014 > my_changes > > patch -Np1 < my_changes > > > > git commit > > This seems like a complicated procedure but maybe it will seem simpler and > more intuitive if I study this more. > > Thanks, > > Dave > > > Satish > > > > On Fri, 10 Oct 2014, Nystrom, William David wrote: > > > > > Jed, > > > > > > I don't think your reply below really does what I want to do. My wdn_mods_next branch is > > > a branch that I maintain that was derived from the next branch. I do not merge changes > > > from wdn_mods_next into the next branch. The next branch is supposed to be a mirror of > > > the official petsc next branch and I only update the next branch in my petsc clones by first > > > performing a "git remote -v update origin" in my bare mirror repo which mirrors the petsc maint, > > > master and next branches as well as containing any additional branches that I create. > > > > > > So, I update wdn_mods_next from the latest copy of next that I have by using the following > > > command: > > > > > > git pull origin next > > > > > > in the petsc directory where I have wdn_mods_next checked out. Generally this is easy and > > > has no conflicts to resolve but on a rare occasion, I might have to resolve a conflict. This is > > > because my wdn_mods_next branch does not differ much from next - just a few hacks I have > > > had to resort to. > > > > > > So here is one way I think I could accomplish my goal. In my bare, mirror repo execute > > > > > > git remote -v update origin > > > > > > At this point my bare mirror repo now has a copy of the new unwound next branch - if I am > > > understanding correctly. > > > > > > Then in a clone of my mirror repo execute > > > > > > git checkout wdn_mods_next > > > git reset --hard next > > > git push origin +wdn_mods_next > > > > > > If I understand correctly, then my wdn_mods_next branch would be identical to the next > > > branch and I would not have the changes I made to next that resulted in wdn_mods_next. > > > > > > Then I could untar a file of those changes on top of my clone with wdn_mods_next checked > > > out and recommit those changes. This is about half a dozen files. But this seems like a > > > hacky way to do business that is only reasonable because I have a small number of files. > > > > > > Isn't there a better way to do this? > > > > > > Perhaps you or Satish might ask why I am using next instead of master. The reason is that > > > I am forever uncovering some bug in petsc that you guys fix on the next branch and I don't > > > want to wait for several days before it gets pulled into master and I can resume productive > > > work. > > > > > > So maybe there is some flaw or weakness in my git work flow. I have been using git for > > > several years now but am not a power user. I like git but do not really want to have to > > > become a git guru. However, this petsc policy of periodically unwinding the next branch > > > seems to always disrupt my workflow - so I have not found a robust way to deal with it > > > yet. > > > > > > Dave > > > > > > ________________________________________ > > > From: Jed Brown [jed at jedbrown.org] > > > Sent: Thursday, October 02, 2014 4:39 PM > > > To: Satish Balay; Nystrom, William David > > > Cc: petsc-users; petsc-dev at mcs.anl.gov > > > Subject: Re: [petsc-dev] [petsc-users] petsc git next branch *is* unwound! > > > > > > Satish Balay writes: > > > > > > > You would have to delete both 'next' and 'wdn_mods_next' branches from > > > > all your clones - and recreate them. > > > > > > Do your normal "git remote update" (or git fetch) in your "master" clone > > > (confusing to use the same name to refer to a repository and a branch). > > > I'll call the repository Mirror. Anyway, you should see that all three > > > branches are updated. 'maint' and 'master' should fast-forward, while > > > 'next' will say "forced update". > > > > > > After that, recreate 'next' on the clones just like in Satish's original > > > email. > > > > From krishns3 at my.erau.edu Sat Oct 11 22:21:56 2014 From: krishns3 at my.erau.edu (Krishnappa, Sanjay) Date: Sun, 12 Oct 2014 03:21:56 +0000 Subject: [petsc-users] Crow Instability Message-ID: <1413084116075.42467@my.erau.edu> Hi, I am looking for simulating crow instability using PETSc. I am very new to PETSc and I would like to know if there is any work/ Code that already exist relating to crow instability. Thanks in advance. regards Sanjay Krishnappa Graduate Research & Teaching Assistant Aerospace Department Embry-Riddle Aeronautical University Daytona Beach, FL -------------- next part -------------- An HTML attachment was scrubbed... URL: From Vincent.De-Groof at uibk.ac.at Mon Oct 13 10:18:52 2014 From: Vincent.De-Groof at uibk.ac.at (De Groof, Vincent Frans Maria) Date: Mon, 13 Oct 2014 15:18:52 +0000 Subject: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. Message-ID: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at> Hi, I am having a convergence problem in a nonlinear FE problem (we only use ksp from petsc, not snes). We don't have any problems using a direct solver with a cholesky factorization (Petsc factor, cholmod, mumps). Now we want to solve it using the bddc preconditioner. No problems on some problems. So far so good. Unfortunately, in some problems, the first increment in the newton iteration does not converge. It exits due to divergence (-4) after the first solve (0 iterations). This happens with CG, using the natural or preconditioned norm. Using the unpreconditioned norm, or minres, the problem converges with no problems. Checking the Dirichlet and Neumann errors of the subdomains (using -pc_bddc_check_level 1) delivers small errors for all subdomains (~1e-16 - 1e-11). Because minres works, and cg doesn't, it seems like I am missing a null space. But since the subdomain solvers work without problems and do not complain about a null pivot this seems unlikely. Also when I check the number of constraints on the subdomains, they should be fully-determined... any suggestions on what might be going wrong here? Am I really missing a nullspace? Or might there be something else going wrong? kind regards, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 13 10:23:56 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 13 Oct 2014 10:23:56 -0500 Subject: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. In-Reply-To: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at> References: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at> Message-ID: On Mon, Oct 13, 2014 at 10:18 AM, De Groof, Vincent Frans Maria < Vincent.De-Groof at uibk.ac.at> wrote: > Hi, > > > I am having a convergence problem in a nonlinear FE problem (we only use > ksp from petsc, not snes). We don't have any problems using a direct solver > with a cholesky factorization (Petsc factor, cholmod, mumps). Now we want > to solve it using the bddc preconditioner. No problems on some problems. So > far so good. > > > Unfortunately, in some problems, the first increment in the newton > iteration does not converge. It exits due to divergence (-4) after the > first solve (0 iterations). This happens with CG, using the natural or > preconditioned norm. Using the unpreconditioned norm, or minres, the > problem converges with no problems. Checking the Dirichlet and Neumann > errors of the subdomains (using -pc_bddc_check_level 1) delivers small > errors for all subdomains (~1e-16 - 1e-11). > > > Because minres works, and cg doesn't, it seems like I am missing a null > space. But since the subdomain solvers work without problems and do not > complain > MINRES works for symmetric indefinite problems, whereas CG needs SPD problems. Could your problem be indefinite? If not, and it is close, its possible that the preconditioned problem is indefinite. Matt > about a null pivot this seems unlikely. Also when I check the number of > constraints on the subdomains, they should be fully-determined... > > > > any suggestions on what might be going wrong here? Am I really missing a > nullspace? Or might there be something else going wrong? > > > > kind regards, > Vincent > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From Vincent.De-Groof at uibk.ac.at Mon Oct 13 11:24:37 2014 From: Vincent.De-Groof at uibk.ac.at (De Groof, Vincent Frans Maria) Date: Mon, 13 Oct 2014 16:24:37 +0000 Subject: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. In-Reply-To: References: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at>, Message-ID: <17A78B9D13564547AC894B88C1596747203AFD13@XMBX4.uibk.ac.at> But if it is indefinite, shouldn't the factorization algorithm complain about a zero pivot? Vincent ________________________________ Von: Matthew Knepley [knepley at gmail.com] Gesendet: Montag, 13. Oktober 2014 17:23 An: De Groof, Vincent Frans Maria Cc: petsc-users at mcs.anl.gov Betreff: Re: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. On Mon, Oct 13, 2014 at 10:18 AM, De Groof, Vincent Frans Maria > wrote: Hi, I am having a convergence problem in a nonlinear FE problem (we only use ksp from petsc, not snes). We don't have any problems using a direct solver with a cholesky factorization (Petsc factor, cholmod, mumps). Now we want to solve it using the bddc preconditioner. No problems on some problems. So far so good. Unfortunately, in some problems, the first increment in the newton iteration does not converge. It exits due to divergence (-4) after the first solve (0 iterations). This happens with CG, using the natural or preconditioned norm. Using the unpreconditioned norm, or minres, the problem converges with no problems. Checking the Dirichlet and Neumann errors of the subdomains (using -pc_bddc_check_level 1) delivers small errors for all subdomains (~1e-16 - 1e-11). Because minres works, and cg doesn't, it seems like I am missing a null space. But since the subdomain solvers work without problems and do not complain MINRES works for symmetric indefinite problems, whereas CG needs SPD problems. Could your problem be indefinite? If not, and it is close, its possible that the preconditioned problem is indefinite. Matt about a null pivot this seems unlikely. Also when I check the number of constraints on the subdomains, they should be fully-determined... any suggestions on what might be going wrong here? Am I really missing a nullspace? Or might there be something else going wrong? kind regards, Vincent -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 13 11:28:02 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 13 Oct 2014 11:28:02 -0500 Subject: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. In-Reply-To: <17A78B9D13564547AC894B88C1596747203AFD13@XMBX4.uibk.ac.at> References: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at> <17A78B9D13564547AC894B88C1596747203AFD13@XMBX4.uibk.ac.at> Message-ID: On Mon, Oct 13, 2014 at 11:24 AM, De Groof, Vincent Frans Maria < Vincent.De-Groof at uibk.ac.at> wrote: > > But if it is indefinite, shouldn't the factorization algorithm complain > about a zero pivot? > Not necessarily. There are plenty of factorizable indefinite systems. Here is one: an upper triangular matrix with both 1 and -1 on the diagonal. Matt > > Vincent > ------------------------------ > *Von:* Matthew Knepley [knepley at gmail.com] > *Gesendet:* Montag, 13. Oktober 2014 17:23 > *An:* De Groof, Vincent Frans Maria > *Cc:* petsc-users at mcs.anl.gov > *Betreff:* Re: [petsc-users] Convergence problem in nonlinear problem, > with PCBDDC. > > On Mon, Oct 13, 2014 at 10:18 AM, De Groof, Vincent Frans Maria < > Vincent.De-Groof at uibk.ac.at> wrote: > >> Hi, >> >> >> I am having a convergence problem in a nonlinear FE problem (we only >> use ksp from petsc, not snes). We don't have any problems using a direct >> solver with a cholesky factorization (Petsc factor, cholmod, mumps). Now we >> want to solve it using the bddc preconditioner. No problems on some >> problems. So far so good. >> >> >> Unfortunately, in some problems, the first increment in the newton >> iteration does not converge. It exits due to divergence (-4) after the >> first solve (0 iterations). This happens with CG, using the natural or >> preconditioned norm. Using the unpreconditioned norm, or minres, the >> problem converges with no problems. Checking the Dirichlet and Neumann >> errors of the subdomains (using -pc_bddc_check_level 1) delivers small >> errors for all subdomains (~1e-16 - 1e-11). >> >> >> Because minres works, and cg doesn't, it seems like I am missing a null >> space. But since the subdomain solvers work without problems and do not >> complain >> > > MINRES works for symmetric indefinite problems, whereas CG needs SPD > problems. Could your problem be indefinite? If not, and it is close, > its possible that the preconditioned problem is indefinite. > > Matt > > >> about a null pivot this seems unlikely. Also when I check the number >> of constraints on the subdomains, they should be fully-determined... >> >> >> >> any suggestions on what might be going wrong here? Am I really missing >> a nullspace? Or might there be something else going wrong? >> >> >> >> kind regards, >> Vincent >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Oct 13 11:31:44 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 13 Oct 2014 11:31:44 -0500 Subject: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. In-Reply-To: <17A78B9D13564547AC894B88C1596747203AFD13@XMBX4.uibk.ac.at> References: <17A78B9D13564547AC894B88C1596747203AFCF9@XMBX4.uibk.ac.at>, <17A78B9D13564547AC894B88C1596747203AFD13@XMBX4.uibk.ac.at> Message-ID: <1B1AF6A0-B7FC-4DE0-9ACF-E43856974EE2@mcs.anl.gov> On Oct 13, 2014, at 11:24 AM, De Groof, Vincent Frans Maria wrote: > > But if it is indefinite, shouldn't the factorization algorithm complain about a zero pivot? It may not. Cholesky generally just chugs along ?pushing? the zero pivot to the final entry at the lower right corner of the matrix, when it gets there if through roundoff the final value becomes slightly positive then you will never know that the problem is (technically) singular. So just cause Cholesky succeeds doesn?t mean that much. Barry > > > > Vincent > Von: Matthew Knepley [knepley at gmail.com] > Gesendet: Montag, 13. Oktober 2014 17:23 > An: De Groof, Vincent Frans Maria > Cc: petsc-users at mcs.anl.gov > Betreff: Re: [petsc-users] Convergence problem in nonlinear problem, with PCBDDC. > > On Mon, Oct 13, 2014 at 10:18 AM, De Groof, Vincent Frans Maria wrote: > Hi, > > > I am having a convergence problem in a nonlinear FE problem (we only use ksp from petsc, not snes). We don't have any problems using a direct solver with a cholesky factorization (Petsc factor, cholmod, mumps). Now we want to solve it using the bddc preconditioner. No problems on some problems. So far so good. > > > Unfortunately, in some problems, the first increment in the newton iteration does not converge. It exits due to divergence (-4) after the first solve (0 iterations). This happens with CG, using the natural or preconditioned norm. Using the unpreconditioned norm, or minres, the problem converges with no problems. Checking the Dirichlet and Neumann errors of the subdomains (using -pc_bddc_check_level 1) delivers small errors for all subdomains (~1e-16 - 1e-11). > > > Because minres works, and cg doesn't, it seems like I am missing a null space. But since the subdomain solvers work without problems and do not complain > > MINRES works for symmetric indefinite problems, whereas CG needs SPD problems. Could your problem be indefinite? If not, and it is close, > its possible that the preconditioned problem is indefinite. > > Matt > > about a null pivot this seems unlikely. Also when I check the number of constraints on the subdomains, they should be fully-determined... > > > > any suggestions on what might be going wrong here? Am I really missing a nullspace? Or might there be something else going wrong? > > > > kind regards, > Vincent > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener From davydden at gmail.com Mon Oct 13 12:16:10 2014 From: davydden at gmail.com (Denis Davydov) Date: Mon, 13 Oct 2014 19:16:10 +0200 Subject: [petsc-users] wrap MATMPIAIJ in MatShellSetOperation and use in SLEPc Message-ID: <67B2FCF8-0E22-4536-AA87-195D27928A6A@gmail.com> Dear all, I am solving a generalised eigenvalue problem S \cdot v = a M \cdot v (1) using Krylov-Schur eigensolver with the shift-and-invert transformation. Now I would like to expand the problem to the case [S + c^\ast \otimes c ] \cdot v = a M \cdot v (2), where ?c? is some vector. As a first step, I was trying to ?wrap" MATMPIAIJ matrix in the MatCreateShell() matrix with MatShellSetOperation() which then does multiplication. So far I do not add any action due to ?c? (e.g. c = 0), but only aim to workout how to approach the problem. Unfortunately, when I use shift-and-invert, i hit some errors (below). In another test problem without shift-and-invert, it seems to be working fine (I am still testing). Supposedly, the user-defined matrices are not supported in full in SLEPc and the error is triggered. I am quite sure it is related to the usage shift-and-invert, but I don?t know implementation details to tell why is it the case. Is there any other way I could consider the problem (2) within SLEPc with shift-and-invert or do some additional tricks to make it work within MatCreateShell()? p.s. this problem arises from the density functional theory with non-local pseudo-potentials, S is hermitian. Kind regards, Denis ? error ? [0]PETSC ERROR: No support for this operation for this object type! [0]PETSC ERROR: Not written for this matrix type! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.4.3, Oct, 15, 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ ? [0]PETSC ERROR: Configure options --prefix=/opt/gcc47-complex/petsc-3.4.3-complex PETSC_ARCH=darwin-release-c --with-mpi-dir=/opt/gcc47-complex/openmpi-1.6.5 --with-blas-lib=/opt/gcc47-complex/openblas-0.2.4/lib/libopenblas.dylib --with-lapack-lib=/opt/gcc47-complex/lapack-3.4.2/lib/liblapack.dylib --with-debugging=0 --with-x=0 --download-superlu_dist --download-metis --download-parmetis --with-clanguage=cxx --with-external-packages-dir=/opt/gcc47-complex/petsc_external_packages --with-scalar-type=complex [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatDuplicate() line 4061 in /petsc-3.4.3/src/mat/interface/matrix.c [0]PETSC ERROR: STMatGAXPY_Private() line 359 in src/st/interface/stsolve.c [0]PETSC ERROR: STSetUp_Sinvert() line 138 in src/st/impls/sinvert/sinvert.c [0]PETSC ERROR: STSetUp() line 290 in src/st/interface/stsolve.c [0]PETSC ERROR: EPSSetUp() line 215 in src/eps/interface/setup.c [0]PETSC ERROR: EPSSolve() line 90 in src/eps/interface/solve.c From jroman at dsic.upv.es Mon Oct 13 12:27:22 2014 From: jroman at dsic.upv.es (Jose E. Roman) Date: Mon, 13 Oct 2014 19:27:22 +0200 Subject: [petsc-users] wrap MATMPIAIJ in MatShellSetOperation and use in SLEPc In-Reply-To: <67B2FCF8-0E22-4536-AA87-195D27928A6A@gmail.com> References: <67B2FCF8-0E22-4536-AA87-195D27928A6A@gmail.com> Message-ID: El 13/10/2014, a las 19:16, Denis Davydov escribi?: > Dear all, > > I am solving a generalised eigenvalue problem > > S \cdot v = a M \cdot v (1) > > using Krylov-Schur eigensolver with the shift-and-invert transformation. > Now I would like to expand the problem to the case > > [S + c^\ast \otimes c ] \cdot v = a M \cdot v (2), > > where ?c? is some vector. > > As a first step, I was trying to ?wrap" MATMPIAIJ matrix in the MatCreateShell() matrix > with MatShellSetOperation() which then does multiplication. > So far I do not add any action due to ?c? (e.g. c = 0), but only aim to workout how to approach the problem. > Unfortunately, when I use shift-and-invert, i hit some errors (below). > In another test problem without shift-and-invert, it seems to be working fine (I am still testing). > Supposedly, the user-defined matrices are not supported in full in SLEPc and the error is triggered. > I am quite sure it is related to the usage shift-and-invert, but I don?t know implementation details to tell why is it the case. > > Is there any other way I could consider the problem (2) within SLEPc with shift-and-invert or > do some additional tricks to make it work within MatCreateShell()? > > p.s. this problem arises from the density functional theory with non-local pseudo-potentials, S is hermitian. > > Kind regards, > Denis > This case is discussed in section 3.4.2 of SLEPc's users guide. Basically, the default behaviour in shift-and-invert is that SLEPc builds matrix C=A-sigma*B (where sigma is the shift). In this case, the "A" matrix (the shell matrix in your case) needs two additional operations: duplicate and AXPY. You could implement these two operations in your shell matrix. You can also try with "-st_matmode inplace" in which case only AXPY is required. But probably what you want is to set "-st_matmode shell" so that you do not need any additional operation; the downside in all cases is that direct linear solves cannot be used and you must be cautious when using shift-and-invert with iterative linear solvers (see discussion at section 3.4.1). Jose From krishns3 at my.erau.edu Mon Oct 13 12:37:55 2014 From: krishns3 at my.erau.edu (Krishnappa, Sanjay) Date: Mon, 13 Oct 2014 17:37:55 +0000 Subject: [petsc-users] Crow Instability In-Reply-To: <1413084116075.42467@my.erau.edu> References: <1413084116075.42467@my.erau.edu> Message-ID: <1413221858864.73289@my.erau.edu> Hi, gentle reminder. regards Sanjay Krishnappa Graduate Research & Teaching Assistant Aerospace Department Embry-Riddle Aeronautical University Daytona Beach, FL? ________________________________ From: Krishnappa, Sanjay Sent: Saturday, October 11, 2014 9:21 PM To: petsc-users at mcs.anl.gov Subject: Crow Instability Hi, I am looking for simulating crow instability using PETSc. I am very new to PETSc and I would like to know if there is any work/ Code that already exist relating to crow instability. Thanks in advance. regards Sanjay Krishnappa Graduate Research & Teaching Assistant Aerospace Department Embry-Riddle Aeronautical University Daytona Beach, FL -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Mon Oct 13 12:49:36 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 13 Oct 2014 13:49:36 -0400 Subject: [petsc-users] Crow Instability In-Reply-To: <1413084116075.42467@my.erau.edu> References: <1413084116075.42467@my.erau.edu> Message-ID: <877g03oo5r.fsf@jedbrown.org> "Krishnappa, Sanjay" writes: > I am looking for simulating crow instability using PETSc. I am > very new to PETSc and I would like to know if there is any > work/ Code that already exist relating to crow > instability. Thanks in advance. PETSc is an algebraic solvers package, not a CFD package. Whether you want to simulate the stability diagrams from Crow's paper or long-time evolution, you'll need to choose a CFD package (or choose a method and implement it). It is a good problem for "high order" methods, if those are available to you, because physical dissipation is relatively low and the domain is simple (often a box). -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From fuentesdt at gmail.com Mon Oct 13 12:59:43 2014 From: fuentesdt at gmail.com (David Fuentes) Date: Mon, 13 Oct 2014 12:59:43 -0500 Subject: [petsc-users] PetscFEIntegrateResidual_OpenCL w/ viennacl Message-ID: Hi, when using the snes/ex12 example https://bitbucket.org/petsc/petsc/src/f5476a2b443e0eedb22d31cb1b2d85b9b918f73f/src/snes/examples/tutorials/ex12.c?at=master with -variable_coefficient field -petscfe_type opencl -mat_petscfe_type opencl and -dm_vec_type viennacl -dm_mat_type seqaijviennacl -mat_type seqaijviennacl will PetscFEIntegrateResidual_OpenCL transfer the residual vector directly to the viennacl datastructures on the GPU for the solve ? is the output residual of PetscFEIntegrateResidual_OpenCL a vector on the CPU ? thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 13 13:21:06 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 13 Oct 2014 13:21:06 -0500 Subject: [petsc-users] PetscFEIntegrateResidual_OpenCL w/ viennacl In-Reply-To: References: Message-ID: On Mon, Oct 13, 2014 at 12:59 PM, David Fuentes wrote: > Hi, > > when using the snes/ex12 example > https://bitbucket.org/petsc/petsc/src/f5476a2b443e0eedb22d31cb1b2d85b9b918f73f/src/snes/examples/tutorials/ex12.c?at=master > with > > -variable_coefficient field -petscfe_type opencl -mat_petscfe_type opencl > > and > > -dm_vec_type viennacl -dm_mat_type seqaijviennacl -mat_type seqaijviennacl > > will PetscFEIntegrateResidual_OpenCL transfer the residual vector directly > to the > viennacl datastructures on the GPU for the solve ? > is the output residual of PetscFEIntegrateResidual_OpenCL a vector on the > CPU ? > The answer is Yes, if I have not screwed anything up. I have not specifically checked, and this stuff is still fairly touchy. It should be easy to check by just forming one residual. There should be no entries in log_summary for the GPU communication. Karl, is this easy to check for you? I need to get everything reinstalled on a different machine. Thanks, Matt > thanks, > David > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From rupp at iue.tuwien.ac.at Mon Oct 13 14:16:37 2014 From: rupp at iue.tuwien.ac.at (Karl Rupp) Date: Mon, 13 Oct 2014 21:16:37 +0200 Subject: [petsc-users] PetscFEIntegrateResidual_OpenCL w/ viennacl In-Reply-To: References: Message-ID: <543C2515.3020608@iue.tuwien.ac.at> Hi David, > when using the snes/ex12 example > https://bitbucket.org/petsc/petsc/src/f5476a2b443e0eedb22d31cb1b2d85b9b918f73f/src/snes/examples/tutorials/ex12.c?at=master > with > > -variable_coefficient field -petscfe_type opencl -mat_petscfe_type opencl > > and > > -dm_vec_type viennacl -dm_mat_type seqaijviennacl -mat_type seqaijviennacl > > will PetscFEIntegrateResidual_OpenCL transfer the residual vector > directly to the > viennacl datastructures on the GPU for the solve ? No, unfortunately this doesn't work yet. The residual implementation is 'matrix free', so there is no matrix set up and *mat_type without effect. Setting -dm_vec_type viennacl will be supported in the future, but currently the OpenCL buffer is only set up to compute the residual. > is the output residual of PetscFEIntegrateResidual_OpenCL a vector on > the CPU ? yes, it is read back to elemVec[], which is passed to PetscFEIntegrateResidual_OpenCL() Best regards, Karli From knepley at gmail.com Mon Oct 13 14:22:44 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 13 Oct 2014 14:22:44 -0500 Subject: [petsc-users] PetscFEIntegrateResidual_OpenCL w/ viennacl In-Reply-To: <543C2515.3020608@iue.tuwien.ac.at> References: <543C2515.3020608@iue.tuwien.ac.at> Message-ID: On Mon, Oct 13, 2014 at 2:16 PM, Karl Rupp wrote: > Hi David, > > > when using the snes/ex12 example > >> https://bitbucket.org/petsc/petsc/src/f5476a2b443e0eedb22d31cb1b2d85 >> b9b918f73f/src/snes/examples/tutorials/ex12.c?at=master >> with >> >> -variable_coefficient field -petscfe_type opencl -mat_petscfe_type opencl >> >> and >> >> -dm_vec_type viennacl -dm_mat_type seqaijviennacl -mat_type >> seqaijviennacl >> >> will PetscFEIntegrateResidual_OpenCL transfer the residual vector >> directly to the >> viennacl datastructures on the GPU for the solve ? >> > > > No, unfortunately this doesn't work yet. The residual implementation is > 'matrix free', so there is no matrix set up and *mat_type without effect. > Setting > -dm_vec_type viennacl > will be supported in the future, but currently the OpenCL buffer is only > set up to compute the residual. > > > is the output residual of PetscFEIntegrateResidual_OpenCL a vector on >> the CPU ? >> > > yes, it is read back to elemVec[], which is passed to > PetscFEIntegrateResidual_OpenCL() > I forgot that assembly is still being done on the CPU, so elemVec[] is read back. However, the vector is OpenCL, so the assembled results will be on the GPU. I wrote code a long time ago to assemble on the GPU, but there was no speedup at all, so I threw it away. I am not sure it will ever make sense to do that. Matt > Best regards, > Karli > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From c.khroulev at gmail.com Mon Oct 13 18:04:16 2014 From: c.khroulev at gmail.com (Constantine Khroulev) Date: Mon, 13 Oct 2014 15:04:16 -0800 Subject: [petsc-users] -log_summary output in PETSc 3.5.2 Message-ID: Hello PETSc developers, Is there a way to get profiling stats similar to -log_summary, but in a format that can be processed programmatically? As far as I can tell, current -log_summary output is tricky to parse: whether output is space-separated depends on values that are printed. In addition to this, a long event name changes offsets from the beginning of the line for a given column, so extracting string subsets using offsets is also unreliable. Thanks! -- Constantine From graziosov at libero.it Mon Oct 13 18:23:37 2014 From: graziosov at libero.it (Valerio Grazioso) Date: Tue, 14 Oct 2014 01:23:37 +0200 Subject: [petsc-users] How to parallelize a linear system solver A*x=q in a scalar code Message-ID: Dear All, I have to parallelize an old fortran77-fortran90 cfd scalar code. Before starting from scratch, I decided to parallelize ?just? the pressure solver (for instance a linear system solver A*x=q). Because of the subroutines call tree, and in particular for the fact a lot of subroutines are doing some output, it would be of great help if I could let just one process run trough the main, leaving all the others on hold (with an MPI_Barrier), just to the point where I create an MPI matrix (an MPI vectors as well) with MatCreate, and then resolve the ?parallel? system. Is this plan feasible with petsc ? Thanks Valerio Grazioso From bsmith at mcs.anl.gov Mon Oct 13 18:26:48 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 13 Oct 2014 18:26:48 -0500 Subject: [petsc-users] -log_summary output in PETSc 3.5.2 In-Reply-To: References: Message-ID: <417F57FB-DF8B-4B03-8785-5FAE44EDC3B4@mcs.anl.gov> -log_view :filename:ASCII_INFO_DETAIL It saves information from each process in one file in Python so it is easy to process the summaries you want. Barry On Oct 13, 2014, at 6:04 PM, Constantine Khroulev wrote: > Hello PETSc developers, > > Is there a way to get profiling stats similar to -log_summary, but in > a format that can be processed programmatically? > > As far as I can tell, current -log_summary output is tricky to parse: > whether output is space-separated depends on values that are printed. > In addition to this, a long event name changes offsets from the > beginning of the line for a given column, so extracting string subsets > using offsets is also unreliable. > > Thanks! > -- > Constantine From c.khroulev at gmail.com Mon Oct 13 18:39:49 2014 From: c.khroulev at gmail.com (Constantine Khroulev) Date: Mon, 13 Oct 2014 15:39:49 -0800 Subject: [petsc-users] -log_summary output in PETSc 3.5.2 In-Reply-To: <417F57FB-DF8B-4B03-8785-5FAE44EDC3B4@mcs.anl.gov> References: <417F57FB-DF8B-4B03-8785-5FAE44EDC3B4@mcs.anl.gov> Message-ID: Barry, Thanks! Would you mind adding this to the manual? A minor comment: even though -log_view :filename:format *is* mentioned, supported formats are not documented in the PetscLogView manual page. -- Constantine On Mon, Oct 13, 2014 at 3:26 PM, Barry Smith wrote: > > -log_view :filename:ASCII_INFO_DETAIL > > It saves information from each process in one file in Python so it is easy to process the summaries you want. > > Barry > > > On Oct 13, 2014, at 6:04 PM, Constantine Khroulev wrote: > >> Hello PETSc developers, >> >> Is there a way to get profiling stats similar to -log_summary, but in >> a format that can be processed programmatically? >> >> As far as I can tell, current -log_summary output is tricky to parse: >> whether output is space-separated depends on values that are printed. >> In addition to this, a long event name changes offsets from the >> beginning of the line for a given column, so extracting string subsets >> using offsets is also unreliable. >> >> Thanks! >> -- >> Constantine > From jed at jedbrown.org Mon Oct 13 21:00:30 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 13 Oct 2014 22:00:30 -0400 Subject: [petsc-users] How to parallelize a linear system solver A*x=q in a scalar code In-Reply-To: References: Message-ID: <87mw8zl8ap.fsf@jedbrown.org> Valerio Grazioso writes: > Dear All, > I have to parallelize an old fortran77-fortran90 cfd scalar By "scalar", do you mean "serial"? > code. Before starting from scratch, I decided to parallelize ?just? > the pressure solver (for instance a linear system solver A*x=q). > Because of the subroutines call tree, and in particular for the fact a > lot of subroutines are doing some output, it would be of great help if > I could let just one process run trough the main, leaving all the > others on hold (with an MPI_Barrier), just to the point where I create > an MPI matrix (an MPI vectors as well) with MatCreate, and then > resolve the ?parallel? system. Is this plan feasible with petsc ? You can create parallel objects, but call MatSetValues and VecSetValues only from rank 0 (where the rest of the code runs). When done, call MatAssemblyBegin/MatAssemblyEnd and VecAssemblyBegin/VecAssemblyEnd collectively. PETSc will communicate the data where it needs to go. Be warned that this is totally non-scalable and cannot provide more than a small speedup. https://en.wikipedia.org/wiki/Amdahl%27s_law Indeed, depending on the problem configuration and the algorithms used by the current implementation, you might get more speedup in the by implementing a quality geometric multigrid solver in the serial code than by "parallelizing" only this bit. If the present code uses good algorithms, I would encourage doing the hard work of making it fully parallel as soon as possible. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From davydden at gmail.com Tue Oct 14 05:46:17 2014 From: davydden at gmail.com (Denis Davydov) Date: Tue, 14 Oct 2014 12:46:17 +0200 Subject: [petsc-users] wrap MATMPIAIJ in MatShellSetOperation and use in SLEPc In-Reply-To: References: <67B2FCF8-0E22-4536-AA87-195D27928A6A@gmail.com> Message-ID: <801DD168-2451-4A4F-8B99-9551B03B1CE7@gmail.com> Dear Jose, Thanks for your prompt answer. Now I see the problem. I will give it a try with "-st_matmode shell? first and see how it goes... Kind regards, Denis On 13 Oct 2014, at 19:27, Jose E. Roman wrote: > > This case is discussed in section 3.4.2 of SLEPc's users guide. Basically, the default behaviour in shift-and-invert is that SLEPc builds matrix C=A-sigma*B (where sigma is the shift). In this case, the "A" matrix (the shell matrix in your case) needs two additional operations: duplicate and AXPY. You could implement these two operations in your shell matrix. You can also try with "-st_matmode inplace" in which case only AXPY is required. But probably what you want is to set "-st_matmode shell" so that you do not need any additional operation; the downside in all cases is that direct linear solves cannot be used and you must be cautious when using shift-and-invert with iterative linear solvers (see discussion at section 3.4.1). > > Jose -------------- next part -------------- An HTML attachment was scrubbed... URL: From Vincent.De-Groof at uibk.ac.at Tue Oct 14 05:48:47 2014 From: Vincent.De-Groof at uibk.ac.at (De Groof, Vincent Frans Maria) Date: Tue, 14 Oct 2014 10:48:47 +0000 Subject: [petsc-users] PCBDDC - PCBDDCSetPrimalVerticesLocalIS Message-ID: <17A78B9D13564547AC894B88C1596747203AFD62@XMBX4.uibk.ac.at> Hi, I want to add a few primal vertices to the bddc preconditioner. But when checking the coarse space (using pc_bddc_check_level 1), nothing changes. Is there some special sequence that needs to be followed to add vertices? thanks, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From graziosov at libero.it Tue Oct 14 06:40:54 2014 From: graziosov at libero.it (Valerio Grazioso) Date: Tue, 14 Oct 2014 13:40:54 +0200 Subject: [petsc-users] How to parallelize a linear system solver A*x=q in a scalar code In-Reply-To: <87mw8zl8ap.fsf@jedbrown.org> References: <87mw8zl8ap.fsf@jedbrown.org> Message-ID: Dear Jed, thanks for the reply! >> Dear All, >> I have to parallelize an old fortran77-fortran90 cfd scalar > > By "scalar", do you mean "serial?? Yes, by scalar I mean serial. >> code. Before starting from scratch, I decided to parallelize ?just? >> the pressure solver (for instance a linear system solver A*x=q). >> Because of the subroutines call tree, and in particular for the fact a >> lot of subroutines are doing some output, it would be of great help if >> I could let just one process run trough the main, leaving all the >> others on hold (with an MPI_Barrier), just to the point where I create >> an MPI matrix (an MPI vectors as well) with MatCreate, and then >> resolve the ?parallel? system. Is this plan feasible with petsc ? > > You can create parallel objects, but call MatSetValues and VecSetValues > only from rank 0 (where the rest of the code runs). When done, call > MatAssemblyBegin/MatAssemblyEnd and VecAssemblyBegin/VecAssemblyEnd > collectively. PETSc will communicate the data where it needs to go. > Thanks for the advice. I?ll give it a try! > Be warned that this is totally non-scalable and cannot provide more than > a small speedup. > > https://en.wikipedia.org/wiki/Amdahl%27s_law I totally agree. But my idea was to ?use? this as an exercise: I have to get to know the serial code, and Petsc! > Indeed, depending on the problem configuration and the algorithms used > by the current implementation, you might get more speedup in the by > implementing a quality geometric multigrid solver in the serial code > than by "parallelizing" only this bit. > > If the present code uses good algorithms, I would encourage doing the > hard work of making it fully parallel as soon as possible. Yes, somehow I?m betting against the code: as a linear solver it uses a SOR iterative method (and I don?t think that the relaxation factor is well chosen). Just using Petsc?s ksp methods (including PC preconditioners) I hope to find faster solvers, even in the serial case. Regards Valerio From yoann.cheny at univ-lorraine.fr Tue Oct 14 07:27:29 2014 From: yoann.cheny at univ-lorraine.fr (Yoann Cheny) Date: Tue, 14 Oct 2014 14:27:29 +0200 (CEST) Subject: [petsc-users] preliminary questions before parallelizing structured solver In-Reply-To: <2015624195.1339280.1413281276225.JavaMail.zimbra@univ-lorraine.fr> Message-ID: <1612684620.1438643.1413289649413.JavaMail.zimbra@univ-lorraine.fr> Hi, I'm working on the parallelization of an existing 3D solver for incompressible Navier-Stokes equations based on a cut-cell method. The equations are discretized on a Cartesian staggered grid on which an arbitrary complex geomertry is represented by its level-set function such that the method preserves the usual Cartesian 7-points stencil. The time integration is performed with a semi-explicit projection method (AB/BDF2) and as a result a Poisson eq. (for the pressure) and a Helmoltz eq. (momentum) must be solved at each time step. Due to the data structure I plan to use DMDA object to build the linear system, am I right ? I read in the archives that dmda objects were not designed to handle with staggered grids, does it still hold ? And finally I plan to model my code on the ksp example ex46.c, is there any F90 version of it ? Any advice and suggestions will be greatly appreciated. Best regards, Yoann ================================================= Yoann CHENY Ma?tre de Conf?rences Enseignement : EEIGM 03 83 36 83 49 Recherche : LEMTA UMR 7563 CNRS 03 83 59 55 94 ================================================= -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 14 07:43:49 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 14 Oct 2014 07:43:49 -0500 Subject: [petsc-users] preliminary questions before parallelizing structured solver In-Reply-To: <1612684620.1438643.1413289649413.JavaMail.zimbra@univ-lorraine.fr> References: <2015624195.1339280.1413281276225.JavaMail.zimbra@univ-lorraine.fr> <1612684620.1438643.1413289649413.JavaMail.zimbra@univ-lorraine.fr> Message-ID: On Tue, Oct 14, 2014 at 7:27 AM, Yoann Cheny wrote: > Hi, > I'm working on the parallelization of an existing 3D solver > for incompressible Navier-Stokes equations based on a cut-cell method. > The equations are discretized on a Cartesian staggered grid on which an > arbitrary complex geomertry is represented by its level-set function such > that the method preserves the usual Cartesian 7-points stencil. > The time integration is performed with a semi-explicit projection method > (AB/BDF2) and as a result a Poisson eq. (for the pressure) and a Helmoltz > eq. (momentum) must be solved at each time step. > Due to the data structure I plan to use DMDA object to build the linear > system, am I right ? > This is definitely possible. > I read in the archives that dmda objects were not designed to handle with > staggered grids, does it still hold ? > Yes, it is not designed to handle this, so any option is somewhat messy. The easiest way to do a staggered discretization is to think of different variables being on shifted grids and use a bigger buffer region to handle the shift. > And finally I plan to model my code on the ksp example ex46.c, is there > any F90 version of it ? > KSP ex46 will not help you very much. SNES ex30 uses a staggered discretization and does the shifting. If you are really adventurous, you could try and use the new PetscSection layout over a DMDA, which allows variables anywhere (like faces). However, this is not tested at all, would require lots of debugging and programming, all to make your code somewhat nicer to look at and simpler on the boundaries. Unless you are up for a ton of development, I would stick with SNES ex30. Thanks, Matt Any advice and suggestions will be greatly appreciated. > Best regards, > Yoann > > ================================================= > Yoann CHENY > Ma?tre de Conf?rences > Enseignement : EEIGM 03 83 36 83 49 > Recherche : LEMTA UMR 7563 CNRS > 03 83 59 55 94 > ================================================= > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Tue Oct 14 10:00:55 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Tue, 14 Oct 2014 17:00:55 +0200 Subject: [petsc-users] Matrix missing diagonal entry In-Reply-To: <0E439AAA-3314-418A-90C1-95DC74B840F6@mcs.anl.gov> References: <1484883.lN7CaqHuu7@asaru> <0E439AAA-3314-418A-90C1-95DC74B840F6@mcs.anl.gov> Message-ID: <4942485.FqCg4qvntm@asaru> Am Dienstag, 7. Oktober 2014, 08:34:55 schrieb Barry Smith: > > PETSc has this ?extra? requirement that you provide values for the entire diagonal of the matrix, even those locations with zero. So make sure that you use MatSetValues() to put a zero on each diagonal entry that has zero. Thanks for that information. I came up with this scheme: _matCLU.assemble(MAT_FLUSH_ASSEMBLY); petsc::Vector zeros(_matCLU); // construct a vector like the given matrix MatDiagonalSet(_matCLU.matrix, zeros.vector, ADD_VALUES); _matCLU.assemble(MAT_FINAL_ASSEMBLY); (contains a bit of my petsc helper functions but should be clear I guess) To set all unset entries to zero. Do you see any problem with that? Best Regards, Florian > > Also from below you are using ICC that is incomplete cholesky factorization. If you want to use a direct solver you should select -pc_type cholesky > > > Barry > > On Oct 7, 2014, at 8:14 AM, Florian Lindner wrote: > > > Hello, > > > > when I try to KSPSolve my matrix petsc prints out that a diagonal entry is missing: > > > > KSPSetOperators(solver, _matCLU.matrix, _matCLU.matrix ); > > KSPSolve(solver, vin.vector, vp.vector); > > > > And petsc is perfectly right about that specific entry being zero. So this may be more of a mathematical problem then pure petsc. > > > > The algorithm that I try to reimplement does the following: > > > > lu(_matrixCLU, _pivotsCLU); // Compute LU decomposition > > // Left out: Permutate in according to pivotsCLU > > forwardSubstitution(_matrixCLU, in, y); // CLU^-1 * in = y (lower triangle of CLU) > > backSubstitution(_matrixCLU, y, p); // CLU^-1 * y = p (upper triangle of CLU) > > multiply(_matrixA, p, out ); // out = A * p (not relevant here) > > > > I want to replace to lu, forward- and backwardSubstituation by using a petsc KSP solver. Before doing any of these operations _matrixCLU and _matCLU are equal: > > > > =============== CLU Before LU ======= > > ==== Dynamic Matrix of size 7 x 7 ==== > > 1.0000 0.3679 0.1353 0.3679 1.0000 0.0000 0.0000 > > 0.3679 1.0000 0.3679 0.1353 1.0000 1.0000 0.0000 > > 0.1353 0.3679 1.0000 0.3679 1.0000 1.0000 1.0000 > > 0.3679 0.1353 0.3679 1.0000 1.0000 0.0000 1.0000 > > 1.0000 1.0000 1.0000 1.0000 0.0000 0.0000 0.0000 > > 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 0.0000 > > 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 > > > > Mat Object:CLU 1 MPI processes > > type: seqsbaij > > row 0: (0, 1) (1, 0.367879) (2, 0.135335) (3, 0.367879) (4, 1) (5, 0) (6, 0) > > row 1: (1, 1) (2, 0.367879) (3, 0.135335) (4, 1) (5, 1) (6, 0) > > row 2: (2, 1) (3, 0.367879) (4, 1) (5, 1) (6, 1) > > row 3: (3, 1) (4, 1) (5, 0) (6, 1) > > row 4: > > row 5: > > row 6: > > > > LU decomposition of matrixCLU works, no rank deficiency (no 0 on the main diagonal after LU and I tested for full rank of matrixCLU). Petsc complains with this matrix: > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > [0]PETSC ERROR: Object is in wrong state > > [0]PETSC ERROR: Matrix is missing diagonal entry 4 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > > [0]PETSC ERROR: ../build/debug/binprecice on a arch-linux2-c-debug named helium by lindnefn Mon Oct 6 16:26:49 2014 > > [0]PETSC ERROR: Configure options > > [0]PETSC ERROR: #1 MatICCFactorSymbolic_SeqSBAIJ() line 2430 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaijfact2.c > > [0]PETSC ERROR: #2 MatICCFactorSymbolic() line 6266 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > > [0]PETSC ERROR: #3 PCSetup_ICC() line 20 in /data2/scratch/lindner/petsc/src/ksp/pc/impls/factor/icc/icc.c > > [0]PETSC ERROR: #4 PCSetUp() line 902 in /data2/scratch/lindner/petsc/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #5 KSPSetUp() line 305 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #6 KSPSolve() line 417 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #7 map() line 696 in src/mapping/PetRadialBasisFctMapping.hpp > > numerical equality test failed: 0.00000000000000000000e+00 instead of 1.00000000000000000000e+00 > > file: src/mapping/tests/PetRadialBasisFctMappingTest.cpp line: 196 > > statement: value==1.0 > > difference: 1.00000000000000000000e+00 > > > > > > What's indeed perfectly right. But what is the best way to cope with that? > > > > Thanks, > > Florian > From bsmith at mcs.anl.gov Tue Oct 14 11:24:22 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 14 Oct 2014 11:24:22 -0500 Subject: [petsc-users] Matrix missing diagonal entry In-Reply-To: <4942485.FqCg4qvntm@asaru> References: <1484883.lN7CaqHuu7@asaru> <0E439AAA-3314-418A-90C1-95DC74B840F6@mcs.anl.gov> <4942485.FqCg4qvntm@asaru> Message-ID: Should be ok. On Oct 14, 2014, at 10:00 AM, Florian Lindner wrote: > Am Dienstag, 7. Oktober 2014, 08:34:55 schrieb Barry Smith: >> >> PETSc has this ?extra? requirement that you provide values for the entire diagonal of the matrix, even those locations with zero. So make sure that you use MatSetValues() to put a zero on each diagonal entry that has zero. > > Thanks for that information. I came up with this scheme: > > _matCLU.assemble(MAT_FLUSH_ASSEMBLY); > petsc::Vector zeros(_matCLU); // construct a vector like the given matrix > MatDiagonalSet(_matCLU.matrix, zeros.vector, ADD_VALUES); > _matCLU.assemble(MAT_FINAL_ASSEMBLY); > > (contains a bit of my petsc helper functions but should be clear I guess) > > To set all unset entries to zero. Do you see any problem with that? > > Best Regards, > Florian > >> >> Also from below you are using ICC that is incomplete cholesky factorization. If you want to use a direct solver you should select -pc_type cholesky >> >> >> Barry >> >> On Oct 7, 2014, at 8:14 AM, Florian Lindner wrote: >> >>> Hello, >>> >>> when I try to KSPSolve my matrix petsc prints out that a diagonal entry is missing: >>> >>> KSPSetOperators(solver, _matCLU.matrix, _matCLU.matrix ); >>> KSPSolve(solver, vin.vector, vp.vector); >>> >>> And petsc is perfectly right about that specific entry being zero. So this may be more of a mathematical problem then pure petsc. >>> >>> The algorithm that I try to reimplement does the following: >>> >>> lu(_matrixCLU, _pivotsCLU); // Compute LU decomposition >>> // Left out: Permutate in according to pivotsCLU >>> forwardSubstitution(_matrixCLU, in, y); // CLU^-1 * in = y (lower triangle of CLU) >>> backSubstitution(_matrixCLU, y, p); // CLU^-1 * y = p (upper triangle of CLU) >>> multiply(_matrixA, p, out ); // out = A * p (not relevant here) >>> >>> I want to replace to lu, forward- and backwardSubstituation by using a petsc KSP solver. Before doing any of these operations _matrixCLU and _matCLU are equal: >>> >>> =============== CLU Before LU ======= >>> ==== Dynamic Matrix of size 7 x 7 ==== >>> 1.0000 0.3679 0.1353 0.3679 1.0000 0.0000 0.0000 >>> 0.3679 1.0000 0.3679 0.1353 1.0000 1.0000 0.0000 >>> 0.1353 0.3679 1.0000 0.3679 1.0000 1.0000 1.0000 >>> 0.3679 0.1353 0.3679 1.0000 1.0000 0.0000 1.0000 >>> 1.0000 1.0000 1.0000 1.0000 0.0000 0.0000 0.0000 >>> 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 0.0000 >>> 0.0000 0.0000 1.0000 1.0000 0.0000 0.0000 0.0000 >>> >>> Mat Object:CLU 1 MPI processes >>> type: seqsbaij >>> row 0: (0, 1) (1, 0.367879) (2, 0.135335) (3, 0.367879) (4, 1) (5, 0) (6, 0) >>> row 1: (1, 1) (2, 0.367879) (3, 0.135335) (4, 1) (5, 1) (6, 0) >>> row 2: (2, 1) (3, 0.367879) (4, 1) (5, 1) (6, 1) >>> row 3: (3, 1) (4, 1) (5, 0) (6, 1) >>> row 4: >>> row 5: >>> row 6: >>> >>> LU decomposition of matrixCLU works, no rank deficiency (no 0 on the main diagonal after LU and I tested for full rank of matrixCLU). Petsc complains with this matrix: >>> >>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [0]PETSC ERROR: Object is in wrong state >>> [0]PETSC ERROR: Matrix is missing diagonal entry 4 >>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>> [0]PETSC ERROR: ../build/debug/binprecice on a arch-linux2-c-debug named helium by lindnefn Mon Oct 6 16:26:49 2014 >>> [0]PETSC ERROR: Configure options >>> [0]PETSC ERROR: #1 MatICCFactorSymbolic_SeqSBAIJ() line 2430 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaijfact2.c >>> [0]PETSC ERROR: #2 MatICCFactorSymbolic() line 6266 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c >>> [0]PETSC ERROR: #3 PCSetup_ICC() line 20 in /data2/scratch/lindner/petsc/src/ksp/pc/impls/factor/icc/icc.c >>> [0]PETSC ERROR: #4 PCSetUp() line 902 in /data2/scratch/lindner/petsc/src/ksp/pc/interface/precon.c >>> [0]PETSC ERROR: #5 KSPSetUp() line 305 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: #6 KSPSolve() line 417 in /data2/scratch/lindner/petsc/src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: #7 map() line 696 in src/mapping/PetRadialBasisFctMapping.hpp >>> numerical equality test failed: 0.00000000000000000000e+00 instead of 1.00000000000000000000e+00 >>> file: src/mapping/tests/PetRadialBasisFctMappingTest.cpp line: 196 >>> statement: value==1.0 >>> difference: 1.00000000000000000000e+00 >>> >>> >>> What's indeed perfectly right. But what is the best way to cope with that? >>> >>> Thanks, >>> Florian >> From thronesf at gmail.com Tue Oct 14 14:07:24 2014 From: thronesf at gmail.com (Sharp Stone) Date: Tue, 14 Oct 2014 15:07:24 -0400 Subject: [petsc-users] KSPsolve Problem Message-ID: Dear all, I'm trying to solve a linear system with petsc ksp solver. But after I called the kspsolve routine, I always got the "Vec entry at local location 0 is not-a-number or infinite at end of function" errors below. I checked the matrix and the vectors, and they are all numbers (and zeros). Is this a bug or I miss something? Thank you very much in advance! [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Floating point exception [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich [0]PETSC ERROR: #12 VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c [0]PETSC ERROR: #13 PCApply() line 442 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #14 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c [0]PETSC ERROR: #16 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c [0]PETSC ERROR: #18 PCApply() line 440 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #19 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c [0]PETSC ERROR: #22 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c -- Best regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 14 14:16:38 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 14 Oct 2014 14:16:38 -0500 Subject: [petsc-users] KSPsolve Problem In-Reply-To: References: Message-ID: On Tue, Oct 14, 2014 at 2:07 PM, Sharp Stone wrote: > Dear all, > > I'm trying to solve a linear system with petsc ksp solver. But after I > called the kspsolve routine, I always got the "Vec entry at local location > 0 is not-a-number or infinite at end of function" errors below. I checked > the matrix and the vectors, and they are all numbers (and zeros). Is this a > bug or I miss something? Thank you very much in advance! > Start simpler. Just use -pc_type lu -ksp_type gmres and see if you are getting a believable solution. Then back off on the PC and KSP. Matt > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [0]PETSC ERROR: Floating point exception > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or infinite > at end of function: Parameter number 3 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named > host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack > --download-mpich > > [0]PETSC ERROR: #12 VecValidValues() line 34 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > [0]PETSC ERROR: #13 PCApply() line 442 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > [0]PETSC ERROR: #16 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > [0]PETSC ERROR: #18 PCApply() line 440 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > [0]PETSC ERROR: #22 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > -- > Best regards, > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Oct 14 14:18:16 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 14 Oct 2014 14:18:16 -0500 Subject: [petsc-users] KSPsolve Problem In-Reply-To: References: Message-ID: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> I am guessing that the local solver is ILU(0) and that the ILU(0) factorization is creating a NaN or Inf due to a zero pivot. What happens if you run with the command line option -fp_trap ? Also if this is a small problem you can run with -mat_view to have it display the factored matrix. Barry On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: > Dear all, > > I'm trying to solve a linear system with petsc ksp solver. But after I called the kspsolve routine, I always got the "Vec entry at local location 0 is not-a-number or infinite at end of function" errors below. I checked the matrix and the vectors, and they are all numbers (and zeros). Is this a bug or I miss something? Thank you very much in advance! > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > [0]PETSC ERROR: Floating point exception > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich > > [0]PETSC ERROR: #12 VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > [0]PETSC ERROR: #13 PCApply() line 442 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > [0]PETSC ERROR: #16 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > [0]PETSC ERROR: #18 PCApply() line 440 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > > [0]PETSC ERROR: #22 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > -- > Best regards, > From thronesf at gmail.com Tue Oct 14 14:52:29 2014 From: thronesf at gmail.com (Sharp Stone) Date: Tue, 14 Oct 2014 15:52:29 -0400 Subject: [petsc-users] KSPsolve Problem In-Reply-To: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> References: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> Message-ID: Thank you for your replies. If I use -fp_trap, I still got the same type of errors as follows. [0]PETSC ERROR: VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 If use -pc_type lu -ksp_type gmres options, I got the zero pivot error as you suspected... Is there a way that can solve this problem? Thank you very much! [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Zero pivot in LU factorization: http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot [0]PETSC ERROR: Zero pivot row 52 value 0 tolerance 2.22045e-14 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 15:49:50 2014 [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich [0]PETSC ERROR: #49 MatPivotCheck_none() line 622 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h [0]PETSC ERROR: #50 MatPivotCheck() line 641 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h [0]PETSC ERROR: #51 MatLUFactorNumeric_SeqAIJ_Inode() line 1713 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/impls/aij/seq/inode.c [0]PETSC ERROR: #52 MatLUFactorNumeric() line 2894 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/interface/matrix.c [0]PETSC ERROR: #53 PCSetUp_LU() line 152 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/factor/lu/lu.c [0]PETSC ERROR: #54 PCSetUp() line 902 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #55 KSPSetUp() line 305 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #56 KSPSolve() line 417 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c On Tue, Oct 14, 2014 at 3:18 PM, Barry Smith wrote: > > I am guessing that the local solver is ILU(0) and that the ILU(0) > factorization is creating a NaN or Inf due to a zero pivot. What happens if > you run with the command line option -fp_trap ? > > Also if this is a small problem you can run with -mat_view to have it > display the factored matrix. > > Barry > > On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: > > > Dear all, > > > > I'm trying to solve a linear system with petsc ksp solver. But after I > called the kspsolve routine, I always got the "Vec entry at local location > 0 is not-a-number or infinite at end of function" errors below. I checked > the matrix and the vectors, and they are all numbers (and zeros). Is this a > bug or I miss something? Thank you very much in advance! > > > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > > > [0]PETSC ERROR: Floating point exception > > > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or > infinite at end of function: Parameter number 3 > > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named > host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack > --download-mpich > > > > [0]PETSC ERROR: #12 VecValidValues() line 34 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > > > [0]PETSC ERROR: #13 PCApply() line 442 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > > > [0]PETSC ERROR: #16 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > > > [0]PETSC ERROR: #18 PCApply() line 440 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > > > > > [0]PETSC ERROR: #22 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > -- > > Best regards, > > > > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 14 14:57:51 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 14 Oct 2014 14:57:51 -0500 Subject: [petsc-users] KSPsolve Problem In-Reply-To: References: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> Message-ID: On Tue, Oct 14, 2014 at 2:52 PM, Sharp Stone wrote: > Thank you for your replies. > > If I use -fp_trap, I still got the same type of errors as follows. > > [0]PETSC ERROR: VecValidValues() line 34 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c Vec > entry at local location 0 is not-a-number or infinite at end of function: > Parameter number 3 > > If use -pc_type lu -ksp_type gmres options, I got the zero pivot error as > you suspected... Is there a way that can solve this problem? Thank you very > much! > > You are trying to solve a singular system: https://www.khanacademy.org/math/precalculus/precalc-matrices/inverting_matrices/v/singular-matrices Its not even clear what a solution would mean. I suggest starting with a problem for which you know the answer. Matt > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > [0]PETSC ERROR: Zero pivot in LU factorization: > http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot > > [0]PETSC ERROR: Zero pivot row 52 value 0 tolerance 2.22045e-14 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named > host206-47.physci.fit.edu by GPL Tue Oct 14 15:49:50 2014 > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack > --download-mpich > > [0]PETSC ERROR: #49 MatPivotCheck_none() line 622 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > [0]PETSC ERROR: #50 MatPivotCheck() line 641 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > [0]PETSC ERROR: #51 MatLUFactorNumeric_SeqAIJ_Inode() line 1713 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/impls/aij/seq/inode.c > > [0]PETSC ERROR: #52 MatLUFactorNumeric() line 2894 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/interface/matrix.c > > [0]PETSC ERROR: #53 PCSetUp_LU() line 152 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/factor/lu/lu.c > > [0]PETSC ERROR: #54 PCSetUp() line 902 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #55 KSPSetUp() line 305 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #56 KSPSolve() line 417 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > On Tue, Oct 14, 2014 at 3:18 PM, Barry Smith wrote: > >> >> I am guessing that the local solver is ILU(0) and that the ILU(0) >> factorization is creating a NaN or Inf due to a zero pivot. What happens if >> you run with the command line option -fp_trap ? >> >> Also if this is a small problem you can run with -mat_view to have it >> display the factored matrix. >> >> Barry >> >> On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: >> >> > Dear all, >> > >> > I'm trying to solve a linear system with petsc ksp solver. But after I >> called the kspsolve routine, I always got the "Vec entry at local location >> 0 is not-a-number or infinite at end of function" errors below. I checked >> the matrix and the vectors, and they are all numbers (and zeros). Is this a >> bug or I miss something? Thank you very much in advance! >> > >> > [0]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> > >> > [0]PETSC ERROR: Floating point exception >> > >> > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or >> infinite at end of function: Parameter number 3 >> > >> > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html >> for trouble shooting. >> > >> > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 >> > >> > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named >> host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 >> > >> > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >> --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack >> --download-mpich >> > >> > [0]PETSC ERROR: #12 VecValidValues() line 34 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c >> > >> > [0]PETSC ERROR: #13 PCApply() line 442 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c >> > >> > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in >> /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h >> > >> > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c >> > >> > [0]PETSC ERROR: #16 KSPSolve() line 459 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c >> > >> > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c >> > >> > [0]PETSC ERROR: #18 PCApply() line 440 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c >> > >> > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in >> /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h >> > >> > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c >> > >> > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c >> > >> > >> > [0]PETSC ERROR: #22 KSPSolve() line 459 in >> /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c >> > >> > >> > -- >> > Best regards, >> > >> >> > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Oct 14 15:40:55 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 14 Oct 2014 15:40:55 -0500 Subject: [petsc-users] KSPsolve Problem In-Reply-To: References: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> Message-ID: <167E1190-4486-4E8B-AF57-680E806A7E2B@mcs.anl.gov> Is the matrix suppose to be singular for the problem you are solving? Perhaps the boundary conditions are incomplete or wrong? Barry On Oct 14, 2014, at 2:57 PM, Matthew Knepley wrote: > On Tue, Oct 14, 2014 at 2:52 PM, Sharp Stone wrote: > Thank you for your replies. > > If I use -fp_trap, I still got the same type of errors as follows. > > [0]PETSC ERROR: VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 > > If use -pc_type lu -ksp_type gmres options, I got the zero pivot error as you suspected... Is there a way that can solve this problem? Thank you very much! > > > You are trying to solve a singular system: > > https://www.khanacademy.org/math/precalculus/precalc-matrices/inverting_matrices/v/singular-matrices > > Its not even clear what a solution would mean. I suggest starting with a problem for which you know the answer. > > Matt > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > [0]PETSC ERROR: Zero pivot in LU factorization: http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot > > [0]PETSC ERROR: Zero pivot row 52 value 0 tolerance 2.22045e-14 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 15:49:50 2014 > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich > > [0]PETSC ERROR: #49 MatPivotCheck_none() line 622 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > [0]PETSC ERROR: #50 MatPivotCheck() line 641 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > [0]PETSC ERROR: #51 MatLUFactorNumeric_SeqAIJ_Inode() line 1713 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/impls/aij/seq/inode.c > > [0]PETSC ERROR: #52 MatLUFactorNumeric() line 2894 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/interface/matrix.c > > [0]PETSC ERROR: #53 PCSetUp_LU() line 152 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/factor/lu/lu.c > > [0]PETSC ERROR: #54 PCSetUp() line 902 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > [0]PETSC ERROR: #55 KSPSetUp() line 305 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > [0]PETSC ERROR: #56 KSPSolve() line 417 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > On Tue, Oct 14, 2014 at 3:18 PM, Barry Smith wrote: > > I am guessing that the local solver is ILU(0) and that the ILU(0) factorization is creating a NaN or Inf due to a zero pivot. What happens if you run with the command line option -fp_trap ? > > Also if this is a small problem you can run with -mat_view to have it display the factored matrix. > > Barry > > On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: > > > Dear all, > > > > I'm trying to solve a linear system with petsc ksp solver. But after I called the kspsolve routine, I always got the "Vec entry at local location 0 is not-a-number or infinite at end of function" errors below. I checked the matrix and the vectors, and they are all numbers (and zeros). Is this a bug or I miss something? Thank you very much in advance! > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > > > [0]PETSC ERROR: Floating point exception > > > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 > > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich > > > > [0]PETSC ERROR: #12 VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > > > [0]PETSC ERROR: #13 PCApply() line 442 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > > > [0]PETSC ERROR: #16 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > > > [0]PETSC ERROR: #18 PCApply() line 440 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > > > > > [0]PETSC ERROR: #22 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > -- > > Best regards, > > > > > > > -- > Best regards, > > Feng > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener From jychang48 at gmail.com Tue Oct 14 20:04:39 2014 From: jychang48 at gmail.com (Justin Chang) Date: Tue, 14 Oct 2014 20:04:39 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors Message-ID: Hi all, So I am writing a non-negative diffusion equation using DMPlex's FEM and Tao's SetVariableBounds functions. My code works really perfectly when I run it with one processor. However, once I use 2 or more processors, I get this error: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Nonconforming object sizes [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse and vector to != ctx from size) [1]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [1]PETSC ERROR: Nonconforming object sizes [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse and vector to != ctx from size) [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT Date: 2014-10-04 20:10:35 -0500 [1]PETSC ERROR: [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT Date: 2014-10-04 20:10:35 -0500 [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by justin Tue Oct 14 19:48:50 2014 [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by justin Tue Oct 14 19:48:50 2014 [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-fblaslapack --download-mpich --with-debugging=1 --download-metis --download-parmetis --download-triangle --with-cmake=cmake --download-ctetgen --download-superlu --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 -with-cmake=cmake [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-fblaslapack --download-mpich --with-debugging=1 --download-metis --download-parmetis --download-triangle --with-cmake=cmake --download-ctetgen --download-superlu --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 -with-cmake=cmake [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in /home/justin/petsc-master/src/vec/vec/utils/vscat.c #1 VecScatterBegin() line 1713 in /home/justin/petsc-master/src/vec/vec/utils/vscat.c [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in /home/justin/petsc-master/src/mat/interface/matrix.c #3 MatMultTranspose() line 2242 in /home/justin/petsc-master/src/mat/interface/matrix.c [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c [1]PETSC ERROR: #6 TaoSolve() line 190 in /home/justin/petsc-master/src/tao/interface/taosolver.c #6 TaoSolve() line 190 in /home/justin/petsc-master/src/tao/interface/taosolver.c [0]PETSC ERROR: #7 main() line 341 in /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c [1]PETSC ERROR: #7 main() line 341 in /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 [cli_1]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 [cli_0]: aborting job: application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 I have no idea how or why I am getting this error. What does this mean? My code is essentially built off of SNES ex12.c. The Jacobian matrix, residual vector, and solution vector were created using DMPlex and the built-in FEM functions within. The Hessian matrix and gradient vector were created by simple matmult() functions of the jacobian and residual. The lower bounds vector was created by duplicating the solution vector (initial guess set to zero). My FormFunctionGradient() is basically the same thing as in the maros.c example. can give more information if needed. Thanks, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 14 20:39:45 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 14 Oct 2014 20:39:45 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang wrote: > Hi all, > > So I am writing a non-negative diffusion equation using DMPlex's FEM and > Tao's SetVariableBounds functions. My code works really perfectly when I > run it with one processor. However, once I use 2 or more processors, I get > this error: > It looks like the problem is in the TAO definition, but you can check by just solving your problem with, for instance, BJacobi-LU in parallel. > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: Nonconforming object sizes > [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse and > vector to != ctx from size) > [1]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [1]PETSC ERROR: Nonconforming object sizes > [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse and > vector to != ctx from size) > [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT > Date: 2014-10-04 20:10:35 -0500 > [1]PETSC ERROR: [0]PETSC ERROR: See > http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT > Date: 2014-10-04 20:10:35 -0500 > [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by > justin Tue Oct 14 19:48:50 2014 > [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by > justin Tue Oct 14 19:48:50 2014 > [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --download-fblaslapack --download-mpich > --with-debugging=1 --download-metis --download-parmetis --download-triangle > --with-cmake=cmake --download-ctetgen --download-superlu > --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 > -with-cmake=cmake > [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --download-fblaslapack --download-mpich > --with-debugging=1 --download-metis --download-parmetis --download-triangle > --with-cmake=cmake --download-ctetgen --download-superlu > --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 > -with-cmake=cmake > [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in > /home/justin/petsc-master/src/vec/vec/utils/vscat.c > #1 VecScatterBegin() line 1713 in > /home/justin/petsc-master/src/vec/vec/utils/vscat.c > [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in > /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c > [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in > /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c > [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in > /home/justin/petsc-master/src/mat/interface/matrix.c > #3 MatMultTranspose() line 2242 in > /home/justin/petsc-master/src/mat/interface/matrix.c > [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in > /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c > [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in > /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c > [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in > /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c > [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in > /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c > [1]PETSC ERROR: #6 TaoSolve() line 190 in > /home/justin/petsc-master/src/tao/interface/taosolver.c > #6 TaoSolve() line 190 in > /home/justin/petsc-master/src/tao/interface/taosolver.c > [0]PETSC ERROR: #7 main() line 341 in > /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c > [1]PETSC ERROR: #7 main() line 341 in > /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c > [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message > -------send entire error message to petsc-maint at mcs.anl.gov---------- > ----------------End of Error Message -------send entire error message to > petsc-maint at mcs.anl.gov---------- > application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 > application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 > [cli_1]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 > [cli_0]: aborting job: > application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 > > > I have no idea how or why I am getting this error. What does this mean? > It looks like one dof is given to proc 0 which should live on proc 1. We have to look at the divisions made in the KKT solver. Can you send a small example? Matt > My code is essentially built off of SNES ex12.c. The Jacobian matrix, > residual vector, and solution vector were created using DMPlex and the > built-in FEM functions within. The Hessian matrix and gradient vector were > created by simple matmult() functions of the jacobian and residual. The > lower bounds vector was created by duplicating the solution vector (initial > guess set to zero). My FormFunctionGradient() is basically the same thing > as in the maros.c example. can give more information if needed. > > Thanks, > Justin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Tue Oct 14 21:22:34 2014 From: thronesf at gmail.com (Sharp Stone) Date: Tue, 14 Oct 2014 22:22:34 -0400 Subject: [petsc-users] KSPsolve Problem In-Reply-To: <167E1190-4486-4E8B-AF57-680E806A7E2B@mcs.anl.gov> References: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> <167E1190-4486-4E8B-AF57-680E806A7E2B@mcs.anl.gov> Message-ID: Thank you for your replies. I used the Dirichlet boundary condition, so I guess I may rule out the boundary problems. On Tue, Oct 14, 2014 at 4:40 PM, Barry Smith wrote: > > Is the matrix suppose to be singular for the problem you are solving? > Perhaps the boundary conditions are incomplete or wrong? > > Barry > > On Oct 14, 2014, at 2:57 PM, Matthew Knepley wrote: > > > On Tue, Oct 14, 2014 at 2:52 PM, Sharp Stone wrote: > > Thank you for your replies. > > > > If I use -fp_trap, I still got the same type of errors as follows. > > > > [0]PETSC ERROR: VecValidValues() line 34 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c Vec > entry at local location 0 is not-a-number or infinite at end of function: > Parameter number 3 > > > > If use -pc_type lu -ksp_type gmres options, I got the zero pivot error > as you suspected... Is there a way that can solve this problem? Thank you > very much! > > > > > > You are trying to solve a singular system: > > > > > https://www.khanacademy.org/math/precalculus/precalc-matrices/inverting_matrices/v/singular-matrices > > > > Its not even clear what a solution would mean. I suggest starting with a > problem for which you know the answer. > > > > Matt > > > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > > > [0]PETSC ERROR: Zero pivot in LU factorization: > http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot > > > > [0]PETSC ERROR: Zero pivot row 52 value 0 tolerance 2.22045e-14 > > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named > host206-47.physci.fit.edu by GPL Tue Oct 14 15:49:50 2014 > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack > --download-mpich > > > > [0]PETSC ERROR: #49 MatPivotCheck_none() line 622 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > > > [0]PETSC ERROR: #50 MatPivotCheck() line 641 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > > > [0]PETSC ERROR: #51 MatLUFactorNumeric_SeqAIJ_Inode() line 1713 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/impls/aij/seq/inode.c > > > > [0]PETSC ERROR: #52 MatLUFactorNumeric() line 2894 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/interface/matrix.c > > > > [0]PETSC ERROR: #53 PCSetUp_LU() line 152 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/factor/lu/lu.c > > > > [0]PETSC ERROR: #54 PCSetUp() line 902 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #55 KSPSetUp() line 305 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > [0]PETSC ERROR: #56 KSPSolve() line 417 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > On Tue, Oct 14, 2014 at 3:18 PM, Barry Smith wrote: > > > > I am guessing that the local solver is ILU(0) and that the ILU(0) > factorization is creating a NaN or Inf due to a zero pivot. What happens if > you run with the command line option -fp_trap ? > > > > Also if this is a small problem you can run with -mat_view to have it > display the factored matrix. > > > > Barry > > > > On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: > > > > > Dear all, > > > > > > I'm trying to solve a linear system with petsc ksp solver. But after I > called the kspsolve routine, I always got the "Vec entry at local location > 0 is not-a-number or infinite at end of function" errors below. I checked > the matrix and the vectors, and they are all numbers (and zeros). Is this a > bug or I miss something? Thank you very much in advance! > > > > > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > > > > > > [0]PETSC ERROR: Floating point exception > > > > > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or > infinite at end of function: Parameter number 3 > > > > > > [0]PETSC ERROR: See > http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named > host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ > --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack > --download-mpich > > > > > > [0]PETSC ERROR: #12 VecValidValues() line 34 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > > > > > [0]PETSC ERROR: #13 PCApply() line 442 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > > > > > [0]PETSC ERROR: #16 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > > > > > [0]PETSC ERROR: #18 PCApply() line 440 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in > /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > > > > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > > > > > > > > [0]PETSC ERROR: #22 KSPSolve() line 459 in > /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > > > > -- > > > Best regards, > > > > > > > > > > > > > -- > > Best regards, > > > > Feng > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Oct 14 21:31:35 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 14 Oct 2014 21:31:35 -0500 Subject: [petsc-users] KSPsolve Problem In-Reply-To: References: <2A907C67-2BC3-4832-B635-287D0B00CA3B@mcs.anl.gov> <167E1190-4486-4E8B-AF57-680E806A7E2B@mcs.anl.gov> Message-ID: <0B22AFC7-5C02-421C-B83D-EBFD0BD43EA4@mcs.anl.gov> How large is the matrix? You can save it to a file with the command line option -ksp_view_mat binary:binaryfilename this will create a file with the name binaryfilename and then load the matrix into Matlab using the the Matlab Script in $PETSC_DIR/share/petsc/matlab/PetscBinaryRead.m and compute its eigenvalues and eigenvectors to see what its null space is or just look at the matrix to see if something is missing in the matrix. Barry On Oct 14, 2014, at 9:22 PM, Sharp Stone wrote: > Thank you for your replies. > > I used the Dirichlet boundary condition, so I guess I may rule out the boundary problems. > > > On Tue, Oct 14, 2014 at 4:40 PM, Barry Smith wrote: > > Is the matrix suppose to be singular for the problem you are solving? Perhaps the boundary conditions are incomplete or wrong? > > Barry > > On Oct 14, 2014, at 2:57 PM, Matthew Knepley wrote: > > > On Tue, Oct 14, 2014 at 2:52 PM, Sharp Stone wrote: > > Thank you for your replies. > > > > If I use -fp_trap, I still got the same type of errors as follows. > > > > [0]PETSC ERROR: VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 > > > > If use -pc_type lu -ksp_type gmres options, I got the zero pivot error as you suspected... Is there a way that can solve this problem? Thank you very much! > > > > > > You are trying to solve a singular system: > > > > https://www.khanacademy.org/math/precalculus/precalc-matrices/inverting_matrices/v/singular-matrices > > > > Its not even clear what a solution would mean. I suggest starting with a problem for which you know the answer. > > > > Matt > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > > > [0]PETSC ERROR: Zero pivot in LU factorization: http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot > > > > [0]PETSC ERROR: Zero pivot row 52 value 0 tolerance 2.22045e-14 > > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 15:49:50 2014 > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich > > > > [0]PETSC ERROR: #49 MatPivotCheck_none() line 622 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > > > [0]PETSC ERROR: #50 MatPivotCheck() line 641 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/matimpl.h > > > > [0]PETSC ERROR: #51 MatLUFactorNumeric_SeqAIJ_Inode() line 1713 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/impls/aij/seq/inode.c > > > > [0]PETSC ERROR: #52 MatLUFactorNumeric() line 2894 in /Users/GPL/local/PETSc/petsc-3.5.2/src/mat/interface/matrix.c > > > > [0]PETSC ERROR: #53 PCSetUp_LU() line 152 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/factor/lu/lu.c > > > > [0]PETSC ERROR: #54 PCSetUp() line 902 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > [0]PETSC ERROR: #55 KSPSetUp() line 305 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > [0]PETSC ERROR: #56 KSPSolve() line 417 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > On Tue, Oct 14, 2014 at 3:18 PM, Barry Smith wrote: > > > > I am guessing that the local solver is ILU(0) and that the ILU(0) factorization is creating a NaN or Inf due to a zero pivot. What happens if you run with the command line option -fp_trap ? > > > > Also if this is a small problem you can run with -mat_view to have it display the factored matrix. > > > > Barry > > > > On Oct 14, 2014, at 2:07 PM, Sharp Stone wrote: > > > > > Dear all, > > > > > > I'm trying to solve a linear system with petsc ksp solver. But after I called the kspsolve routine, I always got the "Vec entry at local location 0 is not-a-number or infinite at end of function" errors below. I checked the matrix and the vectors, and they are all numbers (and zeros). Is this a bug or I miss something? Thank you very much in advance! > > > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > > > > > [0]PETSC ERROR: Floating point exception > > > > > > [0]PETSC ERROR: Vec entry at local location 0 is not-a-number or infinite at end of function: Parameter number 3 > > > > > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > > > > > [0]PETSC ERROR: Petsc Release Version 3.5.2, Sep, 08, 2014 > > > > > > [0]PETSC ERROR: ./pStreamer on a arch-darwin-cxx-debug named host206-47.physci.fit.edu by GPL Tue Oct 14 14:58:05 2014 > > > > > > [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-clanguage=cxx --download-fblaslapack --download-mpich > > > > > > [0]PETSC ERROR: #12 VecValidValues() line 34 in /Users/GPL/local/PETSc/petsc-3.5.2/src/vec/vec/interface/rvector.c > > > > > > [0]PETSC ERROR: #13 PCApply() line 442 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > > > [0]PETSC ERROR: #14 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > > > [0]PETSC ERROR: #15 KSPSolve_PREONLY() line 26 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/preonly/preonly.c > > > > > > [0]PETSC ERROR: #16 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > [0]PETSC ERROR: #17 PCApply_BJacobi_Singleblock() line 675 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/impls/bjacobi/bjacobi.c > > > > > > [0]PETSC ERROR: #18 PCApply() line 440 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/pc/interface/precon.c > > > > > > [0]PETSC ERROR: #19 KSP_PCApply() line 230 in /Users/GPL/local/PETSc/petsc-3.5.2/include/petsc-private/kspimpl.h > > > > > > [0]PETSC ERROR: #20 KSPInitialResidual() line 56 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itres.c > > > > > > [0]PETSC ERROR: #21 KSPSolve_BCGS() line 50 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/impls/bcgs/bcgs.c > > > > > > > > > [0]PETSC ERROR: #22 KSPSolve() line 459 in /Users/GPL/local/PETSc/petsc-3.5.2/src/ksp/ksp/interface/itfunc.c > > > > > > > > > -- > > > Best regards, > > > > > > > > > > > > > -- > > Best regards, > > > > Feng > > > > > > > > -- > > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > > -- Norbert Wiener > > > > > -- > Best regards, > > Feng From jychang48 at gmail.com Tue Oct 14 21:34:50 2014 From: jychang48 at gmail.com (Justin Chang) Date: Tue, 14 Oct 2014 21:34:50 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: Attached is the source code and the makefile used to compile/run the code. The source code is basically a dumbed down version of SNES ex12 plus necessary TAO routines. Thanks On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley wrote: > On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang wrote: > >> Hi all, >> >> So I am writing a non-negative diffusion equation using DMPlex's FEM and >> Tao's SetVariableBounds functions. My code works really perfectly when I >> run it with one processor. However, once I use 2 or more processors, I get >> this error: >> > > It looks like the problem is in the TAO definition, but you can check by > just solving your problem with, for instance, BJacobi-LU in parallel. > > >> [0]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [0]PETSC ERROR: Nonconforming object sizes >> [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse and >> vector to != ctx from size) >> [1]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [1]PETSC ERROR: Nonconforming object sizes >> [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse and >> vector to != ctx from size) >> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html >> for trouble shooting. >> [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT >> Date: 2014-10-04 20:10:35 -0500 >> [1]PETSC ERROR: [0]PETSC ERROR: See >> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. >> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT >> Date: 2014-10-04 20:10:35 -0500 >> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >> justin Tue Oct 14 19:48:50 2014 >> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >> justin Tue Oct 14 19:48:50 2014 >> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >> --with-fc=gfortran --download-fblaslapack --download-mpich >> --with-debugging=1 --download-metis --download-parmetis --download-triangle >> --with-cmake=cmake --download-ctetgen --download-superlu >> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >> -with-cmake=cmake >> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >> --with-fc=gfortran --download-fblaslapack --download-mpich >> --with-debugging=1 --download-metis --download-parmetis --download-triangle >> --with-cmake=cmake --download-ctetgen --download-superlu >> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >> -with-cmake=cmake >> [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in >> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >> #1 VecScatterBegin() line 1713 in >> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >> [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >> [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >> [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in >> /home/justin/petsc-master/src/mat/interface/matrix.c >> #3 MatMultTranspose() line 2242 in >> /home/justin/petsc-master/src/mat/interface/matrix.c >> [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in >> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >> [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in >> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >> [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >> [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >> [1]PETSC ERROR: #6 TaoSolve() line 190 in >> /home/justin/petsc-master/src/tao/interface/taosolver.c >> #6 TaoSolve() line 190 in >> /home/justin/petsc-master/src/tao/interface/taosolver.c >> [0]PETSC ERROR: #7 main() line 341 in >> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >> [1]PETSC ERROR: #7 main() line 341 in >> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >> [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message >> -------send entire error message to petsc-maint at mcs.anl.gov---------- >> ----------------End of Error Message -------send entire error message to >> petsc-maint at mcs.anl.gov---------- >> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >> [cli_1]: aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >> [cli_0]: aborting job: >> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >> >> >> I have no idea how or why I am getting this error. What does this mean? >> > > It looks like one dof is given to proc 0 which should live on proc 1. We > have to look at the divisions > made in the KKT solver. Can you send a small example? > > Matt > > >> My code is essentially built off of SNES ex12.c. The Jacobian matrix, >> residual vector, and solution vector were created using DMPlex and the >> built-in FEM functions within. The Hessian matrix and gradient vector were >> created by simple matmult() functions of the jacobian and residual. The >> lower bounds vector was created by duplicating the solution vector (initial >> guess set to zero). My FormFunctionGradient() is basically the same thing >> as in the maros.c example. can give more information if needed. >> >> Thanks, >> Justin >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.c Type: text/x-csrc Size: 14933 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: makefile Type: application/octet-stream Size: 741 bytes Desc: not available URL: From jason.sarich at gmail.com Tue Oct 14 22:12:54 2014 From: jason.sarich at gmail.com (Jason Sarich) Date: Tue, 14 Oct 2014 22:12:54 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: Hi Justin, If the only constraints you have are bounds on the variables, then you are much better off using the TAO algorithms tron or blmvm. IPM builds its own KKT matrix for a ksp solve that is undoubtably much less efficient than what will be used in tron (blmvm avoids ksp solves all together). That begin said, it looks like there's a bug in IPM and I'll try to track it down, thanks for the report. Jason On Tue, Oct 14, 2014 at 9:34 PM, Justin Chang wrote: > Attached is the source code and the makefile used to compile/run the > code. > > The source code is basically a dumbed down version of SNES ex12 plus > necessary TAO routines. > > Thanks > > On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley > wrote: > >> On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang >> wrote: >> >>> Hi all, >>> >>> So I am writing a non-negative diffusion equation using DMPlex's FEM >>> and Tao's SetVariableBounds functions. My code works really perfectly when >>> I run it with one processor. However, once I use 2 or more processors, I >>> get this error: >>> >> >> It looks like the problem is in the TAO definition, but you can check >> by just solving your problem with, for instance, BJacobi-LU in parallel. >> >> >>> [0]PETSC ERROR: --------------------- Error Message >>> -------------------------------------------------------------- >>> [0]PETSC ERROR: Nonconforming object sizes >>> [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse and >>> vector to != ctx from size) >>> [1]PETSC ERROR: --------------------- Error Message >>> -------------------------------------------------------------- >>> [1]PETSC ERROR: Nonconforming object sizes >>> [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse and >>> vector to != ctx from size) >>> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html >>> for trouble shooting. >>> [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT >>> Date: 2014-10-04 20:10:35 -0500 >>> [1]PETSC ERROR: [0]PETSC ERROR: See >>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>> shooting. >>> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT >>> Date: 2014-10-04 20:10:35 -0500 >>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >>> justin Tue Oct 14 19:48:50 2014 >>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >>> justin Tue Oct 14 19:48:50 2014 >>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>> --with-fc=gfortran --download-fblaslapack --download-mpich >>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>> --with-cmake=cmake --download-ctetgen --download-superlu >>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>> -with-cmake=cmake >>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>> --with-fc=gfortran --download-fblaslapack --download-mpich >>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>> --with-cmake=cmake --download-ctetgen --download-superlu >>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>> -with-cmake=cmake >>> [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in >>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>> #1 VecScatterBegin() line 1713 in >>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>> [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 >>> in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>> [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >>> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>> [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in >>> /home/justin/petsc-master/src/mat/interface/matrix.c >>> #3 MatMultTranspose() line 2242 in >>> /home/justin/petsc-master/src/mat/interface/matrix.c >>> [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>> [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>> [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>> [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>> [1]PETSC ERROR: #6 TaoSolve() line 190 in >>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>> #6 TaoSolve() line 190 in >>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>> [0]PETSC ERROR: #7 main() line 341 in >>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>> [1]PETSC ERROR: #7 main() line 341 in >>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>> [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message >>> -------send entire error message to petsc-maint at mcs.anl.gov---------- >>> ----------------End of Error Message -------send entire error message to >>> petsc-maint at mcs.anl.gov---------- >>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>> [cli_1]: aborting job: >>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>> [cli_0]: aborting job: >>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>> >>> >>> I have no idea how or why I am getting this error. What does this mean? >>> >> >> It looks like one dof is given to proc 0 which should live on proc 1. >> We have to look at the divisions >> made in the KKT solver. Can you send a small example? >> >> Matt >> >> >>> My code is essentially built off of SNES ex12.c. The Jacobian matrix, >>> residual vector, and solution vector were created using DMPlex and the >>> built-in FEM functions within. The Hessian matrix and gradient vector were >>> created by simple matmult() functions of the jacobian and residual. The >>> lower bounds vector was created by duplicating the solution vector (initial >>> guess set to zero). My FormFunctionGradient() is basically the same thing >>> as in the maros.c example. can give more information if needed. >>> >>> Thanks, >>> Justin >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jychang48 at gmail.com Tue Oct 14 23:58:13 2014 From: jychang48 at gmail.com (Justin Chang) Date: Tue, 14 Oct 2014 23:58:13 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: Hi Jason, Both of those algorithms resolved my problem. Thanks! Justin On Tue, Oct 14, 2014 at 10:12 PM, Jason Sarich wrote: > Hi Justin, > > If the only constraints you have are bounds on the variables, then you > are much better off using the TAO algorithms tron or blmvm. IPM builds its > own KKT matrix for a ksp solve that is undoubtably much less efficient than > what will be used in tron (blmvm avoids ksp solves all together). > > That begin said, it looks like there's a bug in IPM and I'll try to track > it down, thanks for the report. > > Jason > > > On Tue, Oct 14, 2014 at 9:34 PM, Justin Chang wrote: > >> Attached is the source code and the makefile used to compile/run the >> code. >> >> The source code is basically a dumbed down version of SNES ex12 plus >> necessary TAO routines. >> >> Thanks >> >> On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley >> wrote: >> >>> On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang >>> wrote: >>> >>>> Hi all, >>>> >>>> So I am writing a non-negative diffusion equation using DMPlex's FEM >>>> and Tao's SetVariableBounds functions. My code works really perfectly when >>>> I run it with one processor. However, once I use 2 or more processors, I >>>> get this error: >>>> >>> >>> It looks like the problem is in the TAO definition, but you can check >>> by just solving your problem with, for instance, BJacobi-LU in parallel. >>> >>> >>>> [0]PETSC ERROR: --------------------- Error Message >>>> -------------------------------------------------------------- >>>> [0]PETSC ERROR: Nonconforming object sizes >>>> [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse >>>> and vector to != ctx from size) >>>> [1]PETSC ERROR: --------------------- Error Message >>>> -------------------------------------------------------------- >>>> [1]PETSC ERROR: Nonconforming object sizes >>>> [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse >>>> and vector to != ctx from size) >>>> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html >>>> for trouble shooting. >>>> [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>> [1]PETSC ERROR: [0]PETSC ERROR: See >>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>>> shooting. >>>> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >>>> justin Tue Oct 14 19:48:50 2014 >>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by >>>> justin Tue Oct 14 19:48:50 2014 >>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>> -with-cmake=cmake >>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>> -with-cmake=cmake >>>> [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in >>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>> #1 VecScatterBegin() line 1713 in >>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>> [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 >>>> in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>> [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >>>> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>> [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in >>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>> #3 MatMultTranspose() line 2242 in >>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>> [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>> [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>> [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>> [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>> [1]PETSC ERROR: #6 TaoSolve() line 190 in >>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>> #6 TaoSolve() line 190 in >>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>> [0]PETSC ERROR: #7 main() line 341 in >>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>> [1]PETSC ERROR: #7 main() line 341 in >>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>> [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message >>>> -------send entire error message to petsc-maint at mcs.anl.gov---------- >>>> ----------------End of Error Message -------send entire error message >>>> to petsc-maint at mcs.anl.gov---------- >>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>> [cli_1]: aborting job: >>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>> [cli_0]: aborting job: >>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>> >>>> >>>> I have no idea how or why I am getting this error. What does this >>>> mean? >>>> >>> >>> It looks like one dof is given to proc 0 which should live on proc 1. >>> We have to look at the divisions >>> made in the KKT solver. Can you send a small example? >>> >>> Matt >>> >>> >>>> My code is essentially built off of SNES ex12.c. The Jacobian matrix, >>>> residual vector, and solution vector were created using DMPlex and the >>>> built-in FEM functions within. The Hessian matrix and gradient vector were >>>> created by simple matmult() functions of the jacobian and residual. The >>>> lower bounds vector was created by duplicating the solution vector (initial >>>> guess set to zero). My FormFunctionGradient() is basically the same thing >>>> as in the maros.c example. can give more information if needed. >>>> >>>> Thanks, >>>> Justin >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hariseldon99 at gmail.com Wed Oct 15 03:20:09 2014 From: hariseldon99 at gmail.com (Analabha Roy) Date: Wed, 15 Oct 2014 13:50:09 +0530 Subject: [petsc-users] TS example tutorial example 4 adapted - not parallelizing Message-ID: Hi all, I took the following petsc example (heat equation dynamics using DMDA) : http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/tutorials/ex4.c.html that compiles and runs in my system (Kubuntu 64-bit kernel 3.13.0-36-generic with petsc 3.5.2) both serially and in parallel as it should. I modified it to solve the quantum Schrodinger equation on a 1d lattice of unit size with a periodic time-dependent RHS. I am attaching the diff to this email and pasting it below. A prettier version of the diff is available online at : http://www.diffnow.com/?report=xhuvj The changed code is running okay when serial, but only shows part of the solution (presumably the part that the root process has) when parallel. I tried not to make any major structural changes other than modifying the RHS setting function and hard coding the solver routine (plus, I removed the 'exact solution' routine since my problem doesn't have one), but probably goofed somewhere with the communicators. Can someone help me figure it out? Regards, Analabha DIFF pasted (file also attached): 0a1,3 > //PROBLEMS: > //FIX PARALLELIZATION PROBLEM > 11,14c14,24 < Concepts: TS^time-dependent linear problems < Concepts: TS^heat equation < Concepts: TS^diffusion equation < Processors: n --- > Default problem parameters. These are overwritten by argument vectors > */ > #define DEFAULT_SIZE 100 > #define DEFAULT_ALPHA 0.3 > #define DEFAULT_AMPL 3.0 > #define DEFAULT_W 3.0 > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > /* > Default problem parameters. These are always hard coded and never > overwritten 15a26,29 > #define INIT_STEP 1E-4 > #define INIT_TIME 0.0 > #define ABSERROR 1E-9 > #define RELERROR 1E-9 19,31c33,39 < This program solves the one-dimensional heat equation (also called the < diffusion equation), < u_t = u_xx, < on the domain 0 <= x <= 1, with the boundary conditions < u(t,0) = 0, u(t,1) = 0, < and the initial condition < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). < This is a linear, second-order, parabolic equation. < < We discretize the right-hand side using finite differences with < uniform grid spacing h: < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) < We then demonstrate time evolution using the various TS methods by --- > This program solves the one-dimensional Schroedinger equation in a > tightly bound lattice with a periodic time dependence > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > on the domain 0 <= m < L, with the boundary conditions > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > This is a set of linear, first-order ODEs > The time evolution using the various TS methods can be run by 34,46d41 < < We compare the approximate solution with the exact solution, given by < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + < 3*exp(-4*pi*pi*t) * sin(2*pi*x) < < Notes: < This code demonstrates the TS solver interface to two variants of < linear problems, u_t = f(u,t), namely < - time-dependent f: f(u,t) is a function of t < - time-independent f: f(u,t) is simply f(u) < < The uniprocessor version of this code is ts/examples/tutorials/ex3.c < 59a55 > 64d59 < 75d69 < Vec solution; /* global exact solution vector */ 77d70 < PetscReal h; /* mesh width h = 1/(m-1) */ 79,80c72,77 < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ < PetscReal norm_2, norm_max; /* error norms */ --- > PetscViewer viewer1; /* viewer for the solution */ > PetscReal norm_2; /* wavefunction norm */ > Vec randoms; /* random numbers between -1 and 1 */ > PetscReal alpha; /* disorder strength */ > PetscReal h0; /* drive amplitude */ > PetscReal w; /* drive frequency */ 85a83 > extern PetscInt kdel (PetscInt, PetscInt); 87,88c85,86 < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void *); < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); --- > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > void *); 90d87 < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); 101,102c98,99 < PetscReal time_total_max = 1.0; /* default max total time */ < PetscInt time_steps_max = 100; /* default max timesteps */ --- > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time */ > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ 108,109c105,107 < PetscBool flg; < TSProblemType tsproblem = TS_LINEAR; --- > PetscReal alpha; /* Disorder strength */ > PetscReal h0; /* Drive amplitude */ > PetscReal w; /* drive frequency */ 119c117 < m = 60; --- > m = DEFAULT_SIZE; 121a120,132 > > h0 = DEFAULT_AMPL; > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > CHKERRQ (ierr); > > alpha = DEFAULT_ALPHA; > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > CHKERRQ (ierr); > > w = DEFAULT_W; > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > CHKERRQ (ierr); > 123a135 > 125,127c137,141 < appctx.h = 1.0 / (m - 1.0); < appctx.norm_2 = 0.0; < appctx.norm_max = 0.0; --- > appctx.alpha = alpha; > appctx.h0 = h0; > appctx.w = w; > > appctx.norm_2 = 1.0; 163d176 < create global work vector for storing exact solution. 167,168d179 < ierr = VecDuplicate (u, &appctx.solution); < CHKERRQ (ierr); 171c182 < Set up displays to show graphs of the solution and error --- > Set up displays to show graphs of the solution 173,189c184,195 < < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, < &appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, < &appctx.viewer2); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); --- > if (appctx.debug) > { > ierr = > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > CHKERRQ (ierr); > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > CHKERRQ (ierr); > ierr = PetscDrawSetDoubleBuffer (draw); > CHKERRQ (ierr); > } 197,202c203 < < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); < CHKERRQ (ierr); < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); < CHKERRQ (ierr); --- > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is always linear */ 205c206 < Set optional user-defined monitoring routine --- > Set user-defined monitoring routine 211d211 < 224,225c224,257 < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); --- > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Create random number context and generate random numbers > into application context > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscRandom rctx; /* random number generator context */ > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetFromOptions (rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > CHKERRQ (ierr); > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > CHKERRQ (ierr); > ierr = VecSetFromOptions (appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetRandom (appctx.randoms, rctx); > CHKERRQ (ierr); > ierr = VecScale (appctx.randoms, randomscale); > CHKERRQ (ierr); > ierr = PetscRandomDestroy (&rctx); > CHKERRQ (ierr); > > /* > For linear problems with a time-dependent f(u,t) in the equation > u_t = f(u,t), the user provides the discretized right-hand-side > as a time-dependent matrix. > */ > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); > CHKERRQ (ierr); > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); 227,254d258 < if (flg) < { < /* < For linear problems with a time-dependent f(u,t) in the equation < u_t = f(u,t), the user provides the discretized right-hand-side < as a time-dependent matrix. < */ < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); < CHKERRQ (ierr); < } < else < { < /* < For linear problems with a time-independent f(u) in the equation < u_t = f(u), the user provides the discretized right-hand-side < as a matrix only once, and then sets a null matrix evaluation < routine. < */ < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); < CHKERRQ (ierr); < } 256,266d259 < if (tsproblem == TS_NONLINEAR) < { < SNES snes; < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); < CHKERRQ (ierr); < ierr = TSGetSNES (ts, &snes); < CHKERRQ (ierr); < ierr = < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); < CHKERRQ (ierr); < } 272,273c265,266 < dt = appctx.h * appctx.h / 2.0; < ierr = TSSetInitialTimeStep (ts, 0.0, dt); --- > dt = INIT_STEP; > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); 281a275 > - Set default tolerances 287a282,285 > ierr = TSSetType (ts, TSRK); > CHKERRQ (ierr); > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) Prince-Dormand (4) > CHKERRQ (ierr); 289a288,291 > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > CHKERRQ (ierr); > 321,324c323,324 < PetscPrintf (PETSC_COMM_WORLD, < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", < (double) (appctx.norm_2 / steps), < (double) (appctx.norm_max / steps)); --- > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > (double) (appctx.norm_2)); 338,341c338,343 < ierr = PetscViewerDestroy (&appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDestroy (&appctx.viewer2); < CHKERRQ (ierr); --- > > if (appctx.debug) > { > ierr = PetscViewerDestroy (&appctx.viewer1); > CHKERRQ (ierr); > } 344,345d345 < ierr = VecDestroy (&appctx.solution); < CHKERRQ (ierr); 347a348,349 > ierr = VecDestroy (&appctx.randoms); > CHKERRQ (ierr); 377c379 < PetscScalar *u_localptr, h = appctx->h; --- > PetscScalar *u_localptr; 379a382 > PetscScalar zero = 0.0, one = 1.0; 394,395d396 < - Note that the Fortran interface to VecGetArray() differs from the < C version. See the users manual for details. 402,403c403,404 < directly into the array locations. Alternatively, we could use < VecSetValues() or VecSetValuesLocal(). --- > directly into the array locations. > This will set the midpoint to unity 406,408c407,410 < u_localptr[i - mybase] = < PetscSinScalar (PETSC_PI * i * 6. * h) + < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); --- > if (i - mybase == floor ((appctx->m) / 2)) > u_localptr[i - mybase] = one; > else > u_localptr[i - mybase] = zero; 421c423 < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); --- > ierr = PetscPrintf (appctx->comm, "initial vector\n"); 432,486d433 < #define __FUNCT__ "ExactSolution" < /* < ExactSolution - Computes the exact solution at a given time. < < Input Parameters: < t - current time < solution - vector in which exact solution will be computed < appctx - user-defined application context < < Output Parameter: < solution - vector with the newly computed exact solution < */ < PetscErrorCode < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) < { < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; < PetscInt i, mybase, myend; < PetscErrorCode ierr; < < /* < Determine starting and ending points of each processor's < range of grid values < */ < ierr = VecGetOwnershipRange (solution, &mybase, &myend); < CHKERRQ (ierr); < < /* < Get a pointer to vector data. < */ < ierr = VecGetArray (solution, &s_localptr); < CHKERRQ (ierr); < < /* < Simply write the solution directly into the array locations. < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). < */ < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); < sc1 = PETSC_PI * 6. * h; < sc2 = PETSC_PI * 2. * h; < for (i = mybase; i < myend; i++) < s_localptr[i - mybase] = < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; < < /* < Restore vector < */ < ierr = VecRestoreArray (solution, &s_localptr); < CHKERRQ (ierr); < return 0; < } < < /* --------------------------------------------------------------------- */ < #undef __FUNCT__ 490,491c437 < each timestep. This example plots the solution and computes the < error in two different norms. --- > each timestep. This example plots the solution and returns the norm 509c455,457 < PetscReal norm_2, norm_max; --- > PetscReal norm_2; > PetscScalar edge_mobility, variance; > PetscInt ix = (appctx->m) - 1; 512c460 < View a graph of the current iterate --- > Compute and store norm of wavefunction in norm_2 514c462 < ierr = VecView (u, appctx->viewer2); --- > ierr = VecNorm (u, NORM_2, &norm_2); 515a464 > appctx->norm_2 = norm_2; 518c467 < Compute the exact solution --- > Compute variance 520c469,476 < ierr = ExactSolution (time, appctx->solution, appctx); --- > > /* Build the lattice sites */ > Vec uabs, sites, sitesq; > ierr = VecDuplicate (u, &sites); > CHKERRQ (ierr); > ierr = VecDuplicate (u, &sitesq); > CHKERRQ (ierr); > ierr = VecDuplicate (u, &uabs); 523,526c479,483 < /* < Print debugging information if desired < */ < if (appctx->debug) --- > PetscInt c; > PetscScalar cdbl; > PetscScalar csq; > > for (c = 1; c < appctx->m; c++) 528c485,486 < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); --- > cdbl = c; > ierr = VecSetValues (sites, 1, &c, &cdbl, INSERT_VALUES); 530,534c488,489 < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > csq = c * c; > ierr = VecSetValues (sitesq, 1, &c, &csq, INSERT_VALUES); 538,541c493,495 < /* < Compute the 2-norm and max-norm of the error < */ < ierr = VecAXPY (appctx->solution, -1.0, u); --- > ierr = VecAssemblyBegin (sites); > CHKERRQ (ierr); > ierr = VecAssemblyEnd (sites); 543c497 < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); --- > ierr = VecAssemblyBegin (sitesq); 545,546c499 < norm_2 = PetscSqrtReal (appctx->h) * norm_2; < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); --- > ierr = VecAssemblyEnd (sitesq); 549,556c502,507 < /* < PetscPrintf() causes only the first processor in this < communicator to print the timestep information. < */ < ierr = < PetscPrintf (appctx->comm, < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", < step, (double) time, (double) norm_2, (double) norm_max); --- > ierr = VecCopy (u, uabs); > CHKERRQ (ierr); > ierr = VecAbs (uabs); > CHKERRQ (ierr); > > ierr = VecPointwiseMult (uabs, uabs, uabs); 558,559c509,520 < appctx->norm_2 += norm_2; < appctx->norm_max += norm_max; --- > > /*Compute \sum_m m^2 |u_m|^2 */ > PetscScalar msqbar; > ierr = VecTDot (uabs, sitesq, &msqbar); > CHKERRQ (ierr); > /*Compute (\sum_m m |u_m|^2)^2 */ > PetscScalar mbarsq; > ierr = VecTDot (uabs, sites, &mbarsq); > CHKERRQ (ierr); > mbarsq = mbarsq * mbarsq; > variance = msqbar - mbarsq; > variance = variance / ((appctx->m) * (appctx->m)); 562c523 < View a graph of the error --- > Compute Edge Mobility 564c525 < ierr = VecView (appctx->solution, appctx->viewer1); --- > ierr = VecGetValues (u, 1, &ix, &edge_mobility); 565a527,554 > PetscReal edge_mob_ampl = cabs (edge_mobility); > > if (time == INIT_TIME && !appctx->debug) > { > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " ###########################################\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " # time \t# edge_mob\t# variance #\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " ###########################################\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > > } > if (!appctx->debug) > { > ierr = > PetscPrintf (PETSC_COMM_WORLD, " # %3.5lf\t# %.5lf\t# %3.5lf #\n", > time, edge_mob_ampl, variance); > CHKERRQ (ierr); > } 572,574c561,568 < ierr = PetscPrintf (appctx->comm, "Error vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > /* > View a graph of the current iterate > */ > ierr = > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > norm_2); > > ierr = VecView (uabs, appctx->viewer1); 576a571,576 > ierr = VecDestroy (&sites); > CHKERRQ (ierr); > ierr = VecDestroy (&sitesq); > CHKERRQ (ierr); > ierr = VecDestroy (&uabs); > CHKERRQ (ierr); 583c583 < #define __FUNCT__ "RHSMatrixHeat" --- > #define __FUNCT__ "RHSMatrixSchrodinger" 585c585 < RHSMatrixHeat - User-provided routine to compute the right-hand-side --- > RHSMatrixSchrodinger - User-provided routine to compute the right-hand-side 599,600c599 < Notes: < RHSMatrixHeat computes entries for the locally owned part of the system. --- > RHSMatrixSchrodinger computes entries for the locally owned part of the system. 613c612 < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) --- > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) 618,619c617,629 < PetscInt i, mstart, mend, idx[3]; < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * stwo; --- > PetscInt m = appctx->m; > PetscScalar drive; > PetscReal h0 = appctx->h0; > PetscReal w = appctx->w; > PetscScalar *randoms_loc; > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); > /*Calculate drive amplitude at this time */ > > drive = w * t; > drive = PetscSinReal (drive); > drive = 2.0 * PETSC_i * h0 * drive; 623,625c633,641 < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ < < ierr = MatGetOwnershipRange (A, &mstart, &mend); --- > The Jacobian Matrix in our case is blocked > - The values are A_{lm} = i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i h(t) \delta_{lm} > - Thus the superdiagonal and subdiagonal elements are just i, and diagonal elements are random + drive > - The elements are set rowwise. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscInt rowstart, rowend, rowcount; > PetscScalar edge[2], middle[3]; > PetscInt idxedge[2], idxmiddle[3]; > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); 627,655c643 < < /* < Set matrix rows corresponding to boundary data < */ < < if (mstart == 0) < { /* first processor only */ < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); < CHKERRQ (ierr); < mstart++; < } < < if (mend == appctx->m) < { /* last processor only */ < mend--; < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); < CHKERRQ (ierr); < } < < /* < Set matrix rows corresponding to interior data. We construct the < matrix one row at a time. < */ < v[0] = sone; < v[1] = stwo; < v[2] = sone; < for (i = mstart; i < mend; i++) --- > for (rowcount = rowstart; rowcount < rowend; rowcount++) 657,661c645,677 < idx[0] = i - 1; < idx[1] = i; < idx[2] = i + 1; < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); < CHKERRQ (ierr); --- > if (rowcount == 0) //If it is the first row of the whole matrix > { > edge[0] = drive + randoms_loc[0]; > edge[1] = PETSC_i; > idxedge[0] = 0; > idxedge[1] = 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else if (rowcount == m - 1) //If it is the last row of the whole matrix > { > edge[0] = PETSC_i; > edge[1] = drive + randoms_loc[m - 1]; > idxedge[0] = m - 2; > idxedge[1] = m - 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else > { > middle[0] = PETSC_i; > middle[1] = drive + randoms_loc[rowcount]; > middle[2] = PETSC_i; > idxmiddle[0] = rowcount - 1; > idxmiddle[1] = rowcount; > idxmiddle[2] = rowcount + 1; > ierr = > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > INSERT_VALUES); > CHKERRQ (ierr); > } 683a700,701 > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); 689,691c707,709 < #define __FUNCT__ "RHSFunctionHeat" < PetscErrorCode < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void *ctx) --- > #define __FUNCT__ "kdel" > PetscInt > kdel (PetscInt i, PetscInt j) 693,704c711,714 < PetscErrorCode ierr; < Mat A; < < PetscFunctionBeginUser; < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); < CHKERRQ (ierr); < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); < CHKERRQ (ierr); < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ < ierr = MatMult (A, globalin, globalout); < CHKERRQ (ierr); < PetscFunctionReturn (0); --- > PetscInt value = 0; > if (i == j) > value = 1; > return value; -- --- *Analabha Roy* C.S.I.R Senior Research Associate Saha Institute of Nuclear Physics Section 1, Block AF Bidhannagar, Calcutta 700064 India *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com *Webpage*: http://www.ph.utexas.edu/~daneel/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 0a1,3 > //PROBLEMS: > //FIX PARALLELIZATION PROBLEM > 11,14c14,24 < Concepts: TS^time-dependent linear problems < Concepts: TS^heat equation < Concepts: TS^diffusion equation < Processors: n --- > Default problem parameters. These are overwritten by argument vectors > */ > #define DEFAULT_SIZE 100 > #define DEFAULT_ALPHA 0.3 > #define DEFAULT_AMPL 3.0 > #define DEFAULT_W 3.0 > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > /* > Default problem parameters. These are always hard coded and never > overwritten 15a26,29 > #define INIT_STEP 1E-4 > #define INIT_TIME 0.0 > #define ABSERROR 1E-9 > #define RELERROR 1E-9 19,31c33,39 < This program solves the one-dimensional heat equation (also called the < diffusion equation), < u_t = u_xx, < on the domain 0 <= x <= 1, with the boundary conditions < u(t,0) = 0, u(t,1) = 0, < and the initial condition < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). < This is a linear, second-order, parabolic equation. < < We discretize the right-hand side using finite differences with < uniform grid spacing h: < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) < We then demonstrate time evolution using the various TS methods by --- > This program solves the one-dimensional Schroedinger equation in a > tightly bound lattice with a periodic time dependence > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > on the domain 0 <= m < L, with the boundary conditions > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > This is a set of linear, first-order ODEs > The time evolution using the various TS methods can be run by 34,46d41 < < We compare the approximate solution with the exact solution, given by < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + < 3*exp(-4*pi*pi*t) * sin(2*pi*x) < < Notes: < This code demonstrates the TS solver interface to two variants of < linear problems, u_t = f(u,t), namely < - time-dependent f: f(u,t) is a function of t < - time-independent f: f(u,t) is simply f(u) < < The uniprocessor version of this code is ts/examples/tutorials/ex3.c < 59a55 > 64d59 < 75d69 < Vec solution; /* global exact solution vector */ 77d70 < PetscReal h; /* mesh width h = 1/(m-1) */ 79,80c72,77 < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ < PetscReal norm_2, norm_max; /* error norms */ --- > PetscViewer viewer1; /* viewer for the solution */ > PetscReal norm_2; /* wavefunction norm */ > Vec randoms; /* random numbers between -1 and 1 */ > PetscReal alpha; /* disorder strength */ > PetscReal h0; /* drive amplitude */ > PetscReal w; /* drive frequency */ 85a83 > extern PetscInt kdel (PetscInt, PetscInt); 87,88c85,86 < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void *); < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); --- > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > void *); 90d87 < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); 101,102c98,99 < PetscReal time_total_max = 1.0; /* default max total time */ < PetscInt time_steps_max = 100; /* default max timesteps */ --- > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time */ > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ 108,109c105,107 < PetscBool flg; < TSProblemType tsproblem = TS_LINEAR; --- > PetscReal alpha; /* Disorder strength */ > PetscReal h0; /* Drive amplitude */ > PetscReal w; /* drive frequency */ 119c117 < m = 60; --- > m = DEFAULT_SIZE; 121a120,132 > > h0 = DEFAULT_AMPL; > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > CHKERRQ (ierr); > > alpha = DEFAULT_ALPHA; > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > CHKERRQ (ierr); > > w = DEFAULT_W; > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > CHKERRQ (ierr); > 123a135 > 125,127c137,141 < appctx.h = 1.0 / (m - 1.0); < appctx.norm_2 = 0.0; < appctx.norm_max = 0.0; --- > appctx.alpha = alpha; > appctx.h0 = h0; > appctx.w = w; > > appctx.norm_2 = 1.0; 163d176 < create global work vector for storing exact solution. 167,168d179 < ierr = VecDuplicate (u, &appctx.solution); < CHKERRQ (ierr); 171c182 < Set up displays to show graphs of the solution and error --- > Set up displays to show graphs of the solution 173,189c184,195 < < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, < &appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, < &appctx.viewer2); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); --- > if (appctx.debug) > { > ierr = > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > CHKERRQ (ierr); > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > CHKERRQ (ierr); > ierr = PetscDrawSetDoubleBuffer (draw); > CHKERRQ (ierr); > } 197,202c203 < < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); < CHKERRQ (ierr); < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); < CHKERRQ (ierr); --- > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is always linear */ 205c206 < Set optional user-defined monitoring routine --- > Set user-defined monitoring routine 211d211 < 224,225c224,257 < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); --- > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Create random number context and generate random numbers > into application context > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscRandom rctx; /* random number generator context */ > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetFromOptions (rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > CHKERRQ (ierr); > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > CHKERRQ (ierr); > ierr = VecSetFromOptions (appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetRandom (appctx.randoms, rctx); > CHKERRQ (ierr); > ierr = VecScale (appctx.randoms, randomscale); > CHKERRQ (ierr); > ierr = PetscRandomDestroy (&rctx); > CHKERRQ (ierr); > > /* > For linear problems with a time-dependent f(u,t) in the equation > u_t = f(u,t), the user provides the discretized right-hand-side > as a time-dependent matrix. > */ > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); > CHKERRQ (ierr); > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); 227,254d258 < if (flg) < { < /* < For linear problems with a time-dependent f(u,t) in the equation < u_t = f(u,t), the user provides the discretized right-hand-side < as a time-dependent matrix. < */ < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); < CHKERRQ (ierr); < } < else < { < /* < For linear problems with a time-independent f(u) in the equation < u_t = f(u), the user provides the discretized right-hand-side < as a matrix only once, and then sets a null matrix evaluation < routine. < */ < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); < CHKERRQ (ierr); < } 256,266d259 < if (tsproblem == TS_NONLINEAR) < { < SNES snes; < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); < CHKERRQ (ierr); < ierr = TSGetSNES (ts, &snes); < CHKERRQ (ierr); < ierr = < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); < CHKERRQ (ierr); < } 272,273c265,266 < dt = appctx.h * appctx.h / 2.0; < ierr = TSSetInitialTimeStep (ts, 0.0, dt); --- > dt = INIT_STEP; > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); 281a275 > - Set default tolerances 287a282,285 > ierr = TSSetType (ts, TSRK); > CHKERRQ (ierr); > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) Prince-Dormand (4) > CHKERRQ (ierr); 289a288,291 > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > CHKERRQ (ierr); > 321,324c323,324 < PetscPrintf (PETSC_COMM_WORLD, < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", < (double) (appctx.norm_2 / steps), < (double) (appctx.norm_max / steps)); --- > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > (double) (appctx.norm_2)); 338,341c338,343 < ierr = PetscViewerDestroy (&appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDestroy (&appctx.viewer2); < CHKERRQ (ierr); --- > > if (appctx.debug) > { > ierr = PetscViewerDestroy (&appctx.viewer1); > CHKERRQ (ierr); > } 344,345d345 < ierr = VecDestroy (&appctx.solution); < CHKERRQ (ierr); 347a348,349 > ierr = VecDestroy (&appctx.randoms); > CHKERRQ (ierr); 377c379 < PetscScalar *u_localptr, h = appctx->h; --- > PetscScalar *u_localptr; 379a382 > PetscScalar zero = 0.0, one = 1.0; 394,395d396 < - Note that the Fortran interface to VecGetArray() differs from the < C version. See the users manual for details. 402,403c403,404 < directly into the array locations. Alternatively, we could use < VecSetValues() or VecSetValuesLocal(). --- > directly into the array locations. > This will set the midpoint to unity 406,408c407,410 < u_localptr[i - mybase] = < PetscSinScalar (PETSC_PI * i * 6. * h) + < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); --- > if (i - mybase == floor ((appctx->m) / 2)) > u_localptr[i - mybase] = one; > else > u_localptr[i - mybase] = zero; 421c423 < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); --- > ierr = PetscPrintf (appctx->comm, "initial vector\n"); 432,486d433 < #define __FUNCT__ "ExactSolution" < /* < ExactSolution - Computes the exact solution at a given time. < < Input Parameters: < t - current time < solution - vector in which exact solution will be computed < appctx - user-defined application context < < Output Parameter: < solution - vector with the newly computed exact solution < */ < PetscErrorCode < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) < { < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; < PetscInt i, mybase, myend; < PetscErrorCode ierr; < < /* < Determine starting and ending points of each processor's < range of grid values < */ < ierr = VecGetOwnershipRange (solution, &mybase, &myend); < CHKERRQ (ierr); < < /* < Get a pointer to vector data. < */ < ierr = VecGetArray (solution, &s_localptr); < CHKERRQ (ierr); < < /* < Simply write the solution directly into the array locations. < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). < */ < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); < sc1 = PETSC_PI * 6. * h; < sc2 = PETSC_PI * 2. * h; < for (i = mybase; i < myend; i++) < s_localptr[i - mybase] = < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; < < /* < Restore vector < */ < ierr = VecRestoreArray (solution, &s_localptr); < CHKERRQ (ierr); < return 0; < } < < /* --------------------------------------------------------------------- */ < #undef __FUNCT__ 490,491c437 < each timestep. This example plots the solution and computes the < error in two different norms. --- > each timestep. This example plots the solution and returns the norm 509c455,457 < PetscReal norm_2, norm_max; --- > PetscReal norm_2; > PetscScalar edge_mobility, variance; > PetscInt ix = (appctx->m) - 1; 512c460 < View a graph of the current iterate --- > Compute and store norm of wavefunction in norm_2 514c462 < ierr = VecView (u, appctx->viewer2); --- > ierr = VecNorm (u, NORM_2, &norm_2); 515a464 > appctx->norm_2 = norm_2; 518c467 < Compute the exact solution --- > Compute variance 520c469,476 < ierr = ExactSolution (time, appctx->solution, appctx); --- > > /* Build the lattice sites */ > Vec uabs, sites, sitesq; > ierr = VecDuplicate (u, &sites); > CHKERRQ (ierr); > ierr = VecDuplicate (u, &sitesq); > CHKERRQ (ierr); > ierr = VecDuplicate (u, &uabs); 523,526c479,483 < /* < Print debugging information if desired < */ < if (appctx->debug) --- > PetscInt c; > PetscScalar cdbl; > PetscScalar csq; > > for (c = 1; c < appctx->m; c++) 528c485,486 < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); --- > cdbl = c; > ierr = VecSetValues (sites, 1, &c, &cdbl, INSERT_VALUES); 530,534c488,489 < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > csq = c * c; > ierr = VecSetValues (sitesq, 1, &c, &csq, INSERT_VALUES); 538,541c493,495 < /* < Compute the 2-norm and max-norm of the error < */ < ierr = VecAXPY (appctx->solution, -1.0, u); --- > ierr = VecAssemblyBegin (sites); > CHKERRQ (ierr); > ierr = VecAssemblyEnd (sites); 543c497 < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); --- > ierr = VecAssemblyBegin (sitesq); 545,546c499 < norm_2 = PetscSqrtReal (appctx->h) * norm_2; < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); --- > ierr = VecAssemblyEnd (sitesq); 549,556c502,507 < /* < PetscPrintf() causes only the first processor in this < communicator to print the timestep information. < */ < ierr = < PetscPrintf (appctx->comm, < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", < step, (double) time, (double) norm_2, (double) norm_max); --- > ierr = VecCopy (u, uabs); > CHKERRQ (ierr); > ierr = VecAbs (uabs); > CHKERRQ (ierr); > > ierr = VecPointwiseMult (uabs, uabs, uabs); 558,559c509,520 < appctx->norm_2 += norm_2; < appctx->norm_max += norm_max; --- > > /*Compute \sum_m m^2 |u_m|^2 */ > PetscScalar msqbar; > ierr = VecTDot (uabs, sitesq, &msqbar); > CHKERRQ (ierr); > /*Compute (\sum_m m |u_m|^2)^2 */ > PetscScalar mbarsq; > ierr = VecTDot (uabs, sites, &mbarsq); > CHKERRQ (ierr); > mbarsq = mbarsq * mbarsq; > variance = msqbar - mbarsq; > variance = variance / ((appctx->m) * (appctx->m)); 562c523 < View a graph of the error --- > Compute Edge Mobility 564c525 < ierr = VecView (appctx->solution, appctx->viewer1); --- > ierr = VecGetValues (u, 1, &ix, &edge_mobility); 565a527,554 > PetscReal edge_mob_ampl = cabs (edge_mobility); > > if (time == INIT_TIME && !appctx->debug) > { > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " ###########################################\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " # time \t# edge_mob\t# variance #\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > ierr = > PetscPrintf (PETSC_COMM_WORLD, > " ###########################################\n", time, > edge_mob_ampl); > CHKERRQ (ierr); > > } > if (!appctx->debug) > { > ierr = > PetscPrintf (PETSC_COMM_WORLD, " # %3.5lf\t# %.5lf\t# %3.5lf #\n", > time, edge_mob_ampl, variance); > CHKERRQ (ierr); > } 572,574c561,568 < ierr = PetscPrintf (appctx->comm, "Error vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > /* > View a graph of the current iterate > */ > ierr = > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > norm_2); > > ierr = VecView (uabs, appctx->viewer1); 576a571,576 > ierr = VecDestroy (&sites); > CHKERRQ (ierr); > ierr = VecDestroy (&sitesq); > CHKERRQ (ierr); > ierr = VecDestroy (&uabs); > CHKERRQ (ierr); 583c583 < #define __FUNCT__ "RHSMatrixHeat" --- > #define __FUNCT__ "RHSMatrixSchrodinger" 585c585 < RHSMatrixHeat - User-provided routine to compute the right-hand-side --- > RHSMatrixSchrodinger - User-provided routine to compute the right-hand-side 599,600c599 < Notes: < RHSMatrixHeat computes entries for the locally owned part of the system. --- > RHSMatrixSchrodinger computes entries for the locally owned part of the system. 613c612 < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) --- > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) 618,619c617,629 < PetscInt i, mstart, mend, idx[3]; < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * stwo; --- > PetscInt m = appctx->m; > PetscScalar drive; > PetscReal h0 = appctx->h0; > PetscReal w = appctx->w; > PetscScalar *randoms_loc; > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); > /*Calculate drive amplitude at this time */ > > drive = w * t; > drive = PetscSinReal (drive); > drive = 2.0 * PETSC_i * h0 * drive; 623,625c633,641 < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ < < ierr = MatGetOwnershipRange (A, &mstart, &mend); --- > The Jacobian Matrix in our case is blocked > - The values are A_{lm} = i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i h(t) \delta_{lm} > - Thus the superdiagonal and subdiagonal elements are just i, and diagonal elements are random + drive > - The elements are set rowwise. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscInt rowstart, rowend, rowcount; > PetscScalar edge[2], middle[3]; > PetscInt idxedge[2], idxmiddle[3]; > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); 627,655c643 < < /* < Set matrix rows corresponding to boundary data < */ < < if (mstart == 0) < { /* first processor only */ < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); < CHKERRQ (ierr); < mstart++; < } < < if (mend == appctx->m) < { /* last processor only */ < mend--; < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); < CHKERRQ (ierr); < } < < /* < Set matrix rows corresponding to interior data. We construct the < matrix one row at a time. < */ < v[0] = sone; < v[1] = stwo; < v[2] = sone; < for (i = mstart; i < mend; i++) --- > for (rowcount = rowstart; rowcount < rowend; rowcount++) 657,661c645,677 < idx[0] = i - 1; < idx[1] = i; < idx[2] = i + 1; < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); < CHKERRQ (ierr); --- > if (rowcount == 0) //If it is the first row of the whole matrix > { > edge[0] = drive + randoms_loc[0]; > edge[1] = PETSC_i; > idxedge[0] = 0; > idxedge[1] = 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else if (rowcount == m - 1) //If it is the last row of the whole matrix > { > edge[0] = PETSC_i; > edge[1] = drive + randoms_loc[m - 1]; > idxedge[0] = m - 2; > idxedge[1] = m - 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else > { > middle[0] = PETSC_i; > middle[1] = drive + randoms_loc[rowcount]; > middle[2] = PETSC_i; > idxmiddle[0] = rowcount - 1; > idxmiddle[1] = rowcount; > idxmiddle[2] = rowcount + 1; > ierr = > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > INSERT_VALUES); > CHKERRQ (ierr); > } 683a700,701 > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); 689,691c707,709 < #define __FUNCT__ "RHSFunctionHeat" < PetscErrorCode < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void *ctx) --- > #define __FUNCT__ "kdel" > PetscInt > kdel (PetscInt i, PetscInt j) 693,704c711,714 < PetscErrorCode ierr; < Mat A; < < PetscFunctionBeginUser; < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); < CHKERRQ (ierr); < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); < CHKERRQ (ierr); < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ < ierr = MatMult (A, globalin, globalout); < CHKERRQ (ierr); < PetscFunctionReturn (0); --- > PetscInt value = 0; > if (i == j) > value = 1; > return value; From knepley at gmail.com Wed Oct 15 05:45:28 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 15 Oct 2014 05:45:28 -0500 Subject: [petsc-users] TS example tutorial example 4 adapted - not parallelizing In-Reply-To: References: Message-ID: On Wed, Oct 15, 2014 at 3:20 AM, Analabha Roy wrote: > > Hi all, > > > I took the following petsc example (heat equation dynamics using DMDA) : > > > http://www.mcs.anl.gov/petsc/petsc-current/src/ts/examples/tutorials/ex4.c.html > > that compiles and runs in my system (Kubuntu 64-bit kernel > 3.13.0-36-generic with petsc 3.5.2) both serially and in parallel as it > should. > > I modified it to solve the quantum Schrodinger equation on a 1d lattice of > unit size with a periodic time-dependent RHS. > > I am attaching the diff to this email and pasting it below. A prettier > version of the diff is available online at : > http://www.diffnow.com/?report=xhuvj > > The changed code is running okay when serial, but only shows part of the > solution (presumably the part that the root process has) when parallel. > > I tried not to make any major structural changes other than modifying the > RHS setting function and hard coding the solver routine (plus, I removed > the 'exact solution' routine since my problem doesn't have one), but > probably goofed somewhere with the communicators. > > Can someone help me figure it out? > I am not sure what you are doing at line 483. You seem to have all processes indexing a Vec at all entries, which cannot be correct. Matt > Regards, > Analabha > > > DIFF pasted (file also attached): > > > 0a1,3 > > //PROBLEMS: > > //FIX PARALLELIZATION PROBLEM > > > 11,14c14,24 > < Concepts: TS^time-dependent linear problems > < Concepts: TS^heat equation > < Concepts: TS^diffusion equation > < Processors: n > --- > > Default problem parameters. These are overwritten by argument vectors > > */ > > #define DEFAULT_SIZE 100 > > #define DEFAULT_ALPHA 0.3 > > #define DEFAULT_AMPL 3.0 > > #define DEFAULT_W 3.0 > > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > > /* > > Default problem parameters. These are always hard coded and never > > overwritten > 15a26,29 > > #define INIT_STEP 1E-4 > > #define INIT_TIME 0.0 > > #define ABSERROR 1E-9 > > #define RELERROR 1E-9 > 19,31c33,39 > < This program solves the one-dimensional heat equation (also called the > < diffusion equation), > < u_t = u_xx, > < on the domain 0 <= x <= 1, with the boundary conditions > < u(t,0) = 0, u(t,1) = 0, > < and the initial condition > < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). > < This is a linear, second-order, parabolic equation. > < > < We discretize the right-hand side using finite differences with > < uniform grid spacing h: > < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) > < We then demonstrate time evolution using the various TS methods by > --- > > This program solves the one-dimensional Schroedinger equation in a > > tightly bound lattice with a periodic time dependence > > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > > on the domain 0 <= m < L, with the boundary conditions > > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > > This is a set of linear, first-order ODEs > > The time evolution using the various TS methods can be run by > 34,46d41 > < > < We compare the approximate solution with the exact solution, given by > < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + > < 3*exp(-4*pi*pi*t) * sin(2*pi*x) > < > < Notes: > < This code demonstrates the TS solver interface to two variants of > < linear problems, u_t = f(u,t), namely > < - time-dependent f: f(u,t) is a function of t > < - time-independent f: f(u,t) is simply f(u) > < > < The uniprocessor version of this code is ts/examples/tutorials/ex3.c > < > 59a55 > > > 64d59 > < > 75d69 > < Vec solution; /* global exact solution vector */ > 77d70 > < PetscReal h; /* mesh width h = 1/(m-1) */ > 79,80c72,77 > < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ > < PetscReal norm_2, norm_max; /* error norms */ > --- > > PetscViewer viewer1; /* viewer for the solution */ > > PetscReal norm_2; /* wavefunction norm */ > > Vec randoms; /* random numbers between -1 and 1 */ > > PetscReal alpha; /* disorder strength */ > > PetscReal h0; /* drive amplitude */ > > PetscReal w; /* drive frequency */ > 85a83 > > extern PetscInt kdel (PetscInt, PetscInt); > 87,88c85,86 > < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void > *); > < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); > --- > > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > > void *); > 90d87 > < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); > 101,102c98,99 > < PetscReal time_total_max = 1.0; /* default max total time */ > < PetscInt time_steps_max = 100; /* default max timesteps */ > --- > > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time > */ > > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ > 108,109c105,107 > < PetscBool flg; > < TSProblemType tsproblem = TS_LINEAR; > --- > > PetscReal alpha; /* Disorder strength */ > > PetscReal h0; /* Drive amplitude */ > > PetscReal w; /* drive frequency */ > 119c117 > < m = 60; > --- > > m = DEFAULT_SIZE; > 121a120,132 > > > > h0 = DEFAULT_AMPL; > > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > > CHKERRQ (ierr); > > > > alpha = DEFAULT_ALPHA; > > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > > CHKERRQ (ierr); > > > > w = DEFAULT_W; > > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > > CHKERRQ (ierr); > > > 123a135 > > > 125,127c137,141 > < appctx.h = 1.0 / (m - 1.0); > < appctx.norm_2 = 0.0; > < appctx.norm_max = 0.0; > --- > > appctx.alpha = alpha; > > appctx.h0 = h0; > > appctx.w = w; > > > > appctx.norm_2 = 1.0; > 163d176 > < create global work vector for storing exact solution. > 167,168d179 > < ierr = VecDuplicate (u, &appctx.solution); > < CHKERRQ (ierr); > 171c182 > < Set up displays to show graphs of the solution and error > --- > > Set up displays to show graphs of the solution > 173,189c184,195 > < > < ierr = > < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, > < &appctx.viewer1); > < CHKERRQ (ierr); > < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > < CHKERRQ (ierr); > < ierr = PetscDrawSetDoubleBuffer (draw); > < CHKERRQ (ierr); > < ierr = > < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, > < &appctx.viewer2); > < CHKERRQ (ierr); > < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); > < CHKERRQ (ierr); > < ierr = PetscDrawSetDoubleBuffer (draw); > < CHKERRQ (ierr); > --- > > if (appctx.debug) > > { > > ierr = > > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > > CHKERRQ (ierr); > > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > > CHKERRQ (ierr); > > ierr = PetscDrawSetDoubleBuffer (draw); > > CHKERRQ (ierr); > > } > 197,202c203 > < > < flg = PETSC_FALSE; > < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); > < CHKERRQ (ierr); > < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); > < CHKERRQ (ierr); > --- > > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is > always linear */ > 205c206 > < Set optional user-defined monitoring routine > --- > > Set user-defined monitoring routine > 211d211 > < > 224,225c224,257 > < flg = PETSC_FALSE; > < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); > --- > > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Create random number context and generate random numbers > > into application context > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > > PetscRandom rctx; /* random number generator context */ > > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > > CHKERRQ (ierr); > > ierr = PetscRandomSetFromOptions (rctx); > > CHKERRQ (ierr); > > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > > CHKERRQ (ierr); > > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > > CHKERRQ (ierr); > > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > > CHKERRQ (ierr); > > ierr = VecSetFromOptions (appctx.randoms); > > CHKERRQ (ierr); > > ierr = VecSetRandom (appctx.randoms, rctx); > > CHKERRQ (ierr); > > ierr = VecScale (appctx.randoms, randomscale); > > CHKERRQ (ierr); > > ierr = PetscRandomDestroy (&rctx); > > CHKERRQ (ierr); > > > > /* > > For linear problems with a time-dependent f(u,t) in the equation > > u_t = f(u,t), the user provides the discretized right-hand-side > > as a time-dependent matrix. > > */ > > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > > CHKERRQ (ierr); > > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); > 227,254d258 > < if (flg) > < { > < /* > < For linear problems with a time-dependent f(u,t) in the equation > < u_t = f(u,t), the user provides the discretized right-hand-side > < as a time-dependent matrix. > < */ > < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > < CHKERRQ (ierr); > < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); > < CHKERRQ (ierr); > < } > < else > < { > < /* > < For linear problems with a time-independent f(u) in the equation > < u_t = f(u), the user provides the discretized right-hand-side > < as a matrix only once, and then sets a null matrix evaluation > < routine. > < */ > < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); > < CHKERRQ (ierr); > < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > < CHKERRQ (ierr); > < ierr = > < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); > < CHKERRQ (ierr); > < } > 256,266d259 > < if (tsproblem == TS_NONLINEAR) > < { > < SNES snes; > < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); > < CHKERRQ (ierr); > < ierr = TSGetSNES (ts, &snes); > < CHKERRQ (ierr); > < ierr = > < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); > < CHKERRQ (ierr); > < } > 272,273c265,266 > < dt = appctx.h * appctx.h / 2.0; > < ierr = TSSetInitialTimeStep (ts, 0.0, dt); > --- > > dt = INIT_STEP; > > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); > 281a275 > > - Set default tolerances > 287a282,285 > > ierr = TSSetType (ts, TSRK); > > CHKERRQ (ierr); > > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) > Prince-Dormand (4) > > CHKERRQ (ierr); > 289a288,291 > > > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > > CHKERRQ (ierr); > > > 321,324c323,324 > < PetscPrintf (PETSC_COMM_WORLD, > < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", > < (double) (appctx.norm_2 / steps), > < (double) (appctx.norm_max / steps)); > --- > > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > > (double) (appctx.norm_2)); > 338,341c338,343 > < ierr = PetscViewerDestroy (&appctx.viewer1); > < CHKERRQ (ierr); > < ierr = PetscViewerDestroy (&appctx.viewer2); > < CHKERRQ (ierr); > --- > > > > if (appctx.debug) > > { > > ierr = PetscViewerDestroy (&appctx.viewer1); > > CHKERRQ (ierr); > > } > 344,345d345 > < ierr = VecDestroy (&appctx.solution); > < CHKERRQ (ierr); > 347a348,349 > > ierr = VecDestroy (&appctx.randoms); > > CHKERRQ (ierr); > 377c379 > < PetscScalar *u_localptr, h = appctx->h; > --- > > PetscScalar *u_localptr; > 379a382 > > PetscScalar zero = 0.0, one = 1.0; > 394,395d396 > < - Note that the Fortran interface to VecGetArray() differs from the > < C version. See the users manual for details. > 402,403c403,404 > < directly into the array locations. Alternatively, we could use > < VecSetValues() or VecSetValuesLocal(). > --- > > directly into the array locations. > > This will set the midpoint to unity > 406,408c407,410 > < u_localptr[i - mybase] = > < PetscSinScalar (PETSC_PI * i * 6. * h) + > < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); > --- > > if (i - mybase == floor ((appctx->m) / 2)) > > u_localptr[i - mybase] = one; > > else > > u_localptr[i - mybase] = zero; > 421c423 > < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); > --- > > ierr = PetscPrintf (appctx->comm, "initial vector\n"); > 432,486d433 > < #define __FUNCT__ "ExactSolution" > < /* > < ExactSolution - Computes the exact solution at a given time. > < > < Input Parameters: > < t - current time > < solution - vector in which exact solution will be computed > < appctx - user-defined application context > < > < Output Parameter: > < solution - vector with the newly computed exact solution > < */ > < PetscErrorCode > < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) > < { > < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; > < PetscInt i, mybase, myend; > < PetscErrorCode ierr; > < > < /* > < Determine starting and ending points of each processor's > < range of grid values > < */ > < ierr = VecGetOwnershipRange (solution, &mybase, &myend); > < CHKERRQ (ierr); > < > < /* > < Get a pointer to vector data. > < */ > < ierr = VecGetArray (solution, &s_localptr); > < CHKERRQ (ierr); > < > < /* > < Simply write the solution directly into the array locations. > < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). > < */ > < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); > < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); > < sc1 = PETSC_PI * 6. * h; > < sc2 = PETSC_PI * 2. * h; > < for (i = mybase; i < myend; i++) > < s_localptr[i - mybase] = > < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + > < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; > < > < /* > < Restore vector > < */ > < ierr = VecRestoreArray (solution, &s_localptr); > < CHKERRQ (ierr); > < return 0; > < } > < > < /* --------------------------------------------------------------------- > */ > < #undef __FUNCT__ > 490,491c437 > < each timestep. This example plots the solution and computes the > < error in two different norms. > --- > > each timestep. This example plots the solution and returns the norm > 509c455,457 > < PetscReal norm_2, norm_max; > --- > > PetscReal norm_2; > > PetscScalar edge_mobility, variance; > > PetscInt ix = (appctx->m) - 1; > 512c460 > < View a graph of the current iterate > --- > > Compute and store norm of wavefunction in norm_2 > 514c462 > < ierr = VecView (u, appctx->viewer2); > --- > > ierr = VecNorm (u, NORM_2, &norm_2); > 515a464 > > appctx->norm_2 = norm_2; > 518c467 > < Compute the exact solution > --- > > Compute variance > 520c469,476 > < ierr = ExactSolution (time, appctx->solution, appctx); > --- > > > > /* Build the lattice sites */ > > Vec uabs, sites, sitesq; > > ierr = VecDuplicate (u, &sites); > > CHKERRQ (ierr); > > ierr = VecDuplicate (u, &sitesq); > > CHKERRQ (ierr); > > ierr = VecDuplicate (u, &uabs); > 523,526c479,483 > < /* > < Print debugging information if desired > < */ > < if (appctx->debug) > --- > > PetscInt c; > > PetscScalar cdbl; > > PetscScalar csq; > > > > for (c = 1; c < appctx->m; c++) > 528c485,486 > < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); > --- > > cdbl = c; > > ierr = VecSetValues (sites, 1, &c, &cdbl, INSERT_VALUES); > 530,534c488,489 > < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); > < CHKERRQ (ierr); > < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); > < CHKERRQ (ierr); > < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); > --- > > csq = c * c; > > ierr = VecSetValues (sitesq, 1, &c, &csq, INSERT_VALUES); > 538,541c493,495 > < /* > < Compute the 2-norm and max-norm of the error > < */ > < ierr = VecAXPY (appctx->solution, -1.0, u); > --- > > ierr = VecAssemblyBegin (sites); > > CHKERRQ (ierr); > > ierr = VecAssemblyEnd (sites); > 543c497 > < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); > --- > > ierr = VecAssemblyBegin (sitesq); > 545,546c499 > < norm_2 = PetscSqrtReal (appctx->h) * norm_2; > < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); > --- > > ierr = VecAssemblyEnd (sitesq); > 549,556c502,507 > < /* > < PetscPrintf() causes only the first processor in this > < communicator to print the timestep information. > < */ > < ierr = > < PetscPrintf (appctx->comm, > < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", > < step, (double) time, (double) norm_2, (double) norm_max); > --- > > ierr = VecCopy (u, uabs); > > CHKERRQ (ierr); > > ierr = VecAbs (uabs); > > CHKERRQ (ierr); > > > > ierr = VecPointwiseMult (uabs, uabs, uabs); > 558,559c509,520 > < appctx->norm_2 += norm_2; > < appctx->norm_max += norm_max; > --- > > > > /*Compute \sum_m m^2 |u_m|^2 */ > > PetscScalar msqbar; > > ierr = VecTDot (uabs, sitesq, &msqbar); > > CHKERRQ (ierr); > > /*Compute (\sum_m m |u_m|^2)^2 */ > > PetscScalar mbarsq; > > ierr = VecTDot (uabs, sites, &mbarsq); > > CHKERRQ (ierr); > > mbarsq = mbarsq * mbarsq; > > variance = msqbar - mbarsq; > > variance = variance / ((appctx->m) * (appctx->m)); > 562c523 > < View a graph of the error > --- > > Compute Edge Mobility > 564c525 > < ierr = VecView (appctx->solution, appctx->viewer1); > --- > > ierr = VecGetValues (u, 1, &ix, &edge_mobility); > 565a527,554 > > PetscReal edge_mob_ampl = cabs (edge_mobility); > > > > if (time == INIT_TIME && !appctx->debug) > > { > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, > > " ###########################################\n", time, > > edge_mob_ampl); > > CHKERRQ (ierr); > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, > > " # time \t# edge_mob\t# variance #\n", time, > > edge_mob_ampl); > > CHKERRQ (ierr); > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, > > " ###########################################\n", time, > > edge_mob_ampl); > > CHKERRQ (ierr); > > > > } > > if (!appctx->debug) > > { > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, " # %3.5lf\t# %.5lf\t# %3.5lf #\n", > > time, edge_mob_ampl, variance); > > CHKERRQ (ierr); > > } > 572,574c561,568 > < ierr = PetscPrintf (appctx->comm, "Error vector\n"); > < CHKERRQ (ierr); > < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); > --- > > /* > > View a graph of the current iterate > > */ > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > > norm_2); > > > > ierr = VecView (uabs, appctx->viewer1); > 576a571,576 > > ierr = VecDestroy (&sites); > > CHKERRQ (ierr); > > ierr = VecDestroy (&sitesq); > > CHKERRQ (ierr); > > ierr = VecDestroy (&uabs); > > CHKERRQ (ierr); > 583c583 > < #define __FUNCT__ "RHSMatrixHeat" > --- > > #define __FUNCT__ "RHSMatrixSchrodinger" > 585c585 > < RHSMatrixHeat - User-provided routine to compute the right-hand-side > --- > > RHSMatrixSchrodinger - User-provided routine to compute the > right-hand-side > 599,600c599 > < Notes: > < RHSMatrixHeat computes entries for the locally owned part of the > system. > --- > > RHSMatrixSchrodinger computes entries for the locally owned part of > the system. > 613c612 > < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) > --- > > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void > *ctx) > 618,619c617,629 > < PetscInt i, mstart, mend, idx[3]; > < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * > stwo; > --- > > PetscInt m = appctx->m; > > PetscScalar drive; > > PetscReal h0 = appctx->h0; > > PetscReal w = appctx->w; > > PetscScalar *randoms_loc; > > > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > > CHKERRQ (ierr); > > /*Calculate drive amplitude at this time */ > > > > drive = w * t; > > drive = PetscSinReal (drive); > > drive = 2.0 * PETSC_i * h0 * drive; > 623,625c633,641 > < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > < > < ierr = MatGetOwnershipRange (A, &mstart, &mend); > --- > > The Jacobian Matrix in our case is blocked > > - The values are A_{lm} = > i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i > h(t) \delta_{lm} > > - Thus the superdiagonal and subdiagonal elements are just i, and > diagonal elements are random + drive > > - The elements are set rowwise. > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > > PetscInt rowstart, rowend, rowcount; > > PetscScalar edge[2], middle[3]; > > PetscInt idxedge[2], idxmiddle[3]; > > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); > 627,655c643 > < > < /* > < Set matrix rows corresponding to boundary data > < */ > < > < if (mstart == 0) > < { /* first processor only */ > < v[0] = 1.0; > < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); > < CHKERRQ (ierr); > < mstart++; > < } > < > < if (mend == appctx->m) > < { /* last processor only */ > < mend--; > < v[0] = 1.0; > < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); > < CHKERRQ (ierr); > < } > < > < /* > < Set matrix rows corresponding to interior data. We construct the > < matrix one row at a time. > < */ > < v[0] = sone; > < v[1] = stwo; > < v[2] = sone; > < for (i = mstart; i < mend; i++) > --- > > for (rowcount = rowstart; rowcount < rowend; rowcount++) > 657,661c645,677 > < idx[0] = i - 1; > < idx[1] = i; > < idx[2] = i + 1; > < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); > < CHKERRQ (ierr); > --- > > if (rowcount == 0) //If it is the first row of the whole matrix > > { > > edge[0] = drive + randoms_loc[0]; > > edge[1] = PETSC_i; > > idxedge[0] = 0; > > idxedge[1] = 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > > CHKERRQ (ierr); > > } > > else if (rowcount == m - 1) //If it is the last row of the whole > matrix > > { > > edge[0] = PETSC_i; > > edge[1] = drive + randoms_loc[m - 1]; > > idxedge[0] = m - 2; > > idxedge[1] = m - 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > > CHKERRQ (ierr); > > } > > else > > { > > middle[0] = PETSC_i; > > middle[1] = drive + randoms_loc[rowcount]; > > middle[2] = PETSC_i; > > idxmiddle[0] = rowcount - 1; > > idxmiddle[1] = rowcount; > > idxmiddle[2] = rowcount + 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > > INSERT_VALUES); > > CHKERRQ (ierr); > > } > 683a700,701 > > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > > CHKERRQ (ierr); > 689,691c707,709 > < #define __FUNCT__ "RHSFunctionHeat" > < PetscErrorCode > < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void > *ctx) > --- > > #define __FUNCT__ "kdel" > > PetscInt > > kdel (PetscInt i, PetscInt j) > 693,704c711,714 > < PetscErrorCode ierr; > < Mat A; > < > < PetscFunctionBeginUser; > < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); > < CHKERRQ (ierr); > < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); > < CHKERRQ (ierr); > < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ > < ierr = MatMult (A, globalin, globalout); > < CHKERRQ (ierr); > < PetscFunctionReturn (0); > --- > > PetscInt value = 0; > > if (i == j) > > value = 1; > > return value; > > > > > > > > > > > -- > --- > *Analabha Roy* > C.S.I.R Senior Research Associate > > Saha Institute of Nuclear Physics > Section 1, Block AF > Bidhannagar, Calcutta 700064 > India > *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com > *Webpage*: http://www.ph.utexas.edu/~daneel/ > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From haakon at hakostra.net Wed Oct 15 08:41:21 2014 From: haakon at hakostra.net (=?UTF-8?B?SMOla29uIFN0cmFuZGVuZXM=?=) Date: Wed, 15 Oct 2014 15:41:21 +0200 Subject: [petsc-users] 3D multi-dof DMDA and Poisson equation In-Reply-To: <871tqi497b.fsf@jedbrown.org> References: <543597D8.6070504@hakostra.net> <871tqi497b.fsf@jedbrown.org> Message-ID: <543E7981.8010306@hakostra.net> Thanks for the reply, both Jed and Matthew. However, I am still a bit unsure on a small detail: I have created a 3D DMDA with ndof = 4, one for each velocity component (u,v,w) and one for the pressure (p). I only solve a Poisson equation to correct p, the velocities are solved explicitly. My question is this time: When I create a matrix with DMCreateMatrix(), I get four times as many rows as I really need, i.e. it assumes that I want to solve equations for all my DOF's. The same happens when I create a vector with DMCreateGlobalVector(), it gets one element for each dof. This is not desirable, since I only solve equations for one unknown per gridpoint. If my domain (for debugginf purposes) have 250 points, I want the local portion of my matrix to have 250 rows. Instead it gets 1000. Was I wrong when I created my DMDA with ndof = 4? Should I have created one DMDA for the velocity with ndof = 3 and one with ndof = 1 to use as a basis for the pressure solution? Or have I again missed something fundamentally? Regards, H?kon On 08. okt. 2014 22:07, Jed Brown wrote: > H?kon Strandenes writes: > >> Hi, >> >> I am making a new, simple incompressible CFD code. I have created a grid >> as a 3D DMDA, with ndof=4, one dof for each velocity component (u,v,w) >> and one for the pressure (p). Currently I am creating a forward Euler >> step, where the velocities are updated based on the velocities and >> pressure from the previous time step. That works great. Now I need to >> solve a Poisson equation for the pressure correction term, enforcing >> continuity onto my solution, and this is where I have a few questions >> for you: >> >> 1) >> How do I set up my equation system in the most efficient way? The >> vectors from the DMDA is not contiguous in a global sense, i.e. if I map >> the local (i,j,k) indices to a global index with >> >> globId = (info.mx*info.my)*k + (info-.mx)*j + i; > > This is a "natural index", used in some cases for IO, but not for > computation. > >> globId is not gontignous on any process. How do I create my coefficient >> matrix, r.h.s. vector and solution vector? > > DMCreateGlobalVector() > > it will use what is described in the Users Manual as "PETSc ordering". > Please read this section of the manual; I think it will answer your > questions. > >> Is there some way I can make the local parts of the matrix and vectors >> mimic the decomposition from the DMDA? Or should I push all values >> into a different decomposition scheme, where each process holds one >> contigous slab of the equation system using the global ID's calculated >> above? Have I missed some really basic stuff here? >> >> 2) >> I know that my coefficient matrix from the Poisson equation is >> symmetric. Currently I am still creating and inserting values on both >> the upper and lower diagonal. Is this really necessary? Or is there a >> "smart" way I can save half the storage by only creating the upper or >> lower part? > > -dm_mat_type sbaij > >> 3) >> I solve a transient problem, but my coefficient matrix is the same for >> all time steps and I only recompute the r.h.s. vector for each time >> step. I have got the impression that KSPsolve "finds out" if the A >> matrix have changed since last invocation or not, recalculating the >> preconditioner as necessary. > > If you just call KSPSolve, it will reuse everything. Call > KSPSetOperators (assuming you are manually doing the time > discretization) if the matrix changes. > >> But still I feel that I should utilize this property more. Do you have >> any tips in how I can use this in the most efficient way possible? For >> example really, really efficient, but expensive, preconditioners? >> >> Thanks in advance. >> >> Regards, >> H?kon Strandenes From dave.mayhem23 at gmail.com Wed Oct 15 09:14:45 2014 From: dave.mayhem23 at gmail.com (Dave May) Date: Wed, 15 Oct 2014 16:14:45 +0200 Subject: [petsc-users] 3D multi-dof DMDA and Poisson equation In-Reply-To: <543E7981.8010306@hakostra.net> References: <543597D8.6070504@hakostra.net> <871tqi497b.fsf@jedbrown.org> <543E7981.8010306@hakostra.net> Message-ID: For your pressue Poisson solve, you can use DMDAGetReducedDMDA to generate a DMDA with only 1 dof. This reduced DMDA will have the same parallel layout as your DMDA with 4 dofs. See http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMDAGetReducedDMDA.html Cheers, Dave On 15 October 2014 15:41, H?kon Strandenes wrote: > Thanks for the reply, both Jed and Matthew. However, I am still a bit > unsure on a small detail: > > I have created a 3D DMDA with ndof = 4, one for each velocity component > (u,v,w) and one for the pressure (p). I only solve a Poisson equation to > correct p, the velocities are solved explicitly. > > My question is this time: > When I create a matrix with DMCreateMatrix(), I get four times as many > rows as I really need, i.e. it assumes that I want to solve equations for > all my DOF's. The same happens when I create a vector with > DMCreateGlobalVector(), it gets one element for each dof. This is not > desirable, since I only solve equations for one unknown per gridpoint. If > my domain (for debugginf purposes) have 250 points, I want the local > portion of my matrix to have 250 rows. Instead it gets 1000. > > Was I wrong when I created my DMDA with ndof = 4? Should I have created > one DMDA for the velocity with ndof = 3 and one with ndof = 1 to use as a > basis for the pressure solution? Or have I again missed something > fundamentally? > > Regards, > H?kon > > > > On 08. okt. 2014 22:07, Jed Brown wrote: > >> H?kon Strandenes writes: >> >> Hi, >>> >>> I am making a new, simple incompressible CFD code. I have created a grid >>> as a 3D DMDA, with ndof=4, one dof for each velocity component (u,v,w) >>> and one for the pressure (p). Currently I am creating a forward Euler >>> step, where the velocities are updated based on the velocities and >>> pressure from the previous time step. That works great. Now I need to >>> solve a Poisson equation for the pressure correction term, enforcing >>> continuity onto my solution, and this is where I have a few questions >>> for you: >>> >>> 1) >>> How do I set up my equation system in the most efficient way? The >>> vectors from the DMDA is not contiguous in a global sense, i.e. if I map >>> the local (i,j,k) indices to a global index with >>> >>> globId = (info.mx*info.my)*k + (info-.mx)*j + i; >>> >> >> This is a "natural index", used in some cases for IO, but not for >> computation. >> >> globId is not gontignous on any process. How do I create my coefficient >>> matrix, r.h.s. vector and solution vector? >>> >> >> DMCreateGlobalVector() >> >> it will use what is described in the Users Manual as "PETSc ordering". >> Please read this section of the manual; I think it will answer your >> questions. >> >> Is there some way I can make the local parts of the matrix and vectors >>> mimic the decomposition from the DMDA? Or should I push all values >>> into a different decomposition scheme, where each process holds one >>> contigous slab of the equation system using the global ID's calculated >>> above? Have I missed some really basic stuff here? >>> >>> 2) >>> I know that my coefficient matrix from the Poisson equation is >>> symmetric. Currently I am still creating and inserting values on both >>> the upper and lower diagonal. Is this really necessary? Or is there a >>> "smart" way I can save half the storage by only creating the upper or >>> lower part? >>> >> >> -dm_mat_type sbaij >> >> 3) >>> I solve a transient problem, but my coefficient matrix is the same for >>> all time steps and I only recompute the r.h.s. vector for each time >>> step. I have got the impression that KSPsolve "finds out" if the A >>> matrix have changed since last invocation or not, recalculating the >>> preconditioner as necessary. >>> >> >> If you just call KSPSolve, it will reuse everything. Call >> KSPSetOperators (assuming you are manually doing the time >> discretization) if the matrix changes. >> >> But still I feel that I should utilize this property more. Do you have >>> any tips in how I can use this in the most efficient way possible? For >>> example really, really efficient, but expensive, preconditioners? >>> >>> Thanks in advance. >>> >>> Regards, >>> H?kon Strandenes >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed Oct 15 11:57:14 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 15 Oct 2014 11:57:14 -0500 Subject: [petsc-users] PCBDDC - PCBDDCSetPrimalVerticesLocalIS In-Reply-To: <17A78B9D13564547AC894B88C1596747203AFD62@XMBX4.uibk.ac.at> References: <17A78B9D13564547AC894B88C1596747203AFD62@XMBX4.uibk.ac.at> Message-ID: <87zjcxfez9.fsf@jedbrown.org> "De Groof, Vincent Frans Maria" writes: > I want to add a few primal vertices to the bddc preconditioner. But > when checking the coarse space (using pc_bddc_check_level 1), nothing > changes. Is there some special sequence that needs to be followed to > add vertices? Stefano, can you answer this? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From evanum at gmail.com Wed Oct 15 18:52:51 2014 From: evanum at gmail.com (Evan Um) Date: Wed, 15 Oct 2014 16:52:51 -0700 Subject: [petsc-users] Controlling decimal points when PETSC matrices and vectors are outputted. Message-ID: Dear users, Is there any way to control the precision (i.e. decimal points) of matrices and vectors when they are written into files? Otherwise, do I need to access elements of a matrix by myself and then write them with my own format? In advance, thanks for tour advice. Regards, Evan PetscViewerASCIIOpen(PETSC_COMM_WORLD,"A.out",&viewer); PetscViewerSetFormat(viewer, PETSC_VIEWER_ASCII_MATLAB); MatView(A_dt,viewer); -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 15 19:01:09 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 15 Oct 2014 19:01:09 -0500 Subject: [petsc-users] Controlling decimal points when PETSC matrices and vectors are outputted. In-Reply-To: References: Message-ID: Evan, There is very little control. But if you are reading the vectors/matrices into MATLAB or Python you should use the binary format; it is much faster and always preserves all the precision. Use PetscViewerBinaryOpen() then in MATLAB use PetscBinaryRead.m to read them in (it is in bin/matlab in PETSc 3.4 and earlier and in share/petsc/matlab in PETSc 3.5) to read into Python you can use bin/pythonscripts/petscBinaryIO.py Barry On Oct 15, 2014, at 6:52 PM, Evan Um wrote: > Dear users, > > Is there any way to control the precision (i.e. decimal points) of matrices and vectors when they are written into files? Otherwise, do I need to access elements of a matrix by myself and then write them with my own format? In advance, thanks for tour advice. > > Regards, > Evan > > > PetscViewerASCIIOpen(PETSC_COMM_WORLD,"A.out",&viewer); > PetscViewerSetFormat(viewer, PETSC_VIEWER_ASCII_MATLAB); > MatView(A_dt,viewer); From stefano.zampini at gmail.com Thu Oct 16 02:01:41 2014 From: stefano.zampini at gmail.com (Stefano Zampini) Date: Thu, 16 Oct 2014 10:01:41 +0300 Subject: [petsc-users] PCBDDC - PCBDDCSetPrimalVerticesLocalIS In-Reply-To: <87zjcxfez9.fsf@jedbrown.org> References: <17A78B9D13564547AC894B88C1596747203AFD62@XMBX4.uibk.ac.at> <87zjcxfez9.fsf@jedbrown.org> Message-ID: Vincent also wrote me directly. I'm forwarding my answer. Vincent, to my knowledge there are two cases when this can happen A) you specify indices (in local numbering) which are not lie on the interface B) you specify indices (in local numbering) which are on a part of the interface shared by 2 subregions only Note that A) is a natural requirement, whether B) is a bug in the current release. If you are in situation B), than you can either use 'master' where th ebug has been already fixed or (and this is the case if you are also using SLEPc), to simply comment lines 125, 127, 182 and 187 (4 lines in total) of src/ksp/pc/bddc/bddcgraph.c. This should solve your problem. Sorry for slow reply Stefano Il 15/ott/2014 22:04 "Jed Brown" ha scritto: > "De Groof, Vincent Frans Maria" writes: > > I want to add a few primal vertices to the bddc preconditioner. But > > when checking the coarse space (using pc_bddc_check_level 1), nothing > > changes. Is there some special sequence that needs to be followed to > > add vertices? > > Stefano, can you answer this? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hariseldon99 at gmail.com Thu Oct 16 02:48:50 2014 From: hariseldon99 at gmail.com (Analabha Roy) Date: Thu, 16 Oct 2014 13:18:50 +0530 Subject: [petsc-users] TS example tutorial example 4 adapted - not parallelizing In-Reply-To: References: Message-ID: Hi all, Thanks for the response. On Wed, Oct 15, 2014 at 4:15 PM, Matthew Knepley wrote: > > I am not sure what you are doing at line 483. You seem to have all > processes indexing a Vec > at all entries, which cannot be correct. > > Yes, you're right, of course. I have removed that bit (I'll do it in post-processing after I have dumped the full state data to file). In the meantime, the graphical output is still messed up when the code is run with debug flag in parallel. However, parallel graphical output works fine in the original ex4.c. I can;t figure out why... I have updated diff (link to pretty version ) attached and posted below. Thanks and Regards, Analabha 0a1,2 > //PROBLEMS: > //FIX PARALLELIZATION PROBLEM 3,8c5,11 < "Solves a simple time-dependent linear PDE (the heat equation).\n\ < Input parameters include:\n\ < -m , where = number of grid points\n\ < -time_dependent_rhs : Treat the problem as having a time-dependent right-hand side\n\ < -debug : Activate debugging printouts\n\ < -nox : Deactivate x-window graphics\n\n"; --- > "Solves a simple time-dependent Schroedinger equation for a TB model with periodic drive.\n\ > Apart from PetSc's routine options, input parameters include:\n\ > -m , where = lattice size\n\ > -a , where = drive amplitude\n\ > -d , where = disorder strength\n\ > -w , where = drive frequency\n\ > -debug : Activate debugging printouts\n\n"; 11,14c14 < Concepts: TS^time-dependent linear problems < Concepts: TS^heat equation < Concepts: TS^diffusion equation < Processors: n --- > Default problem parameters. These are overwritten by argument vectors 15a16,29 > #define DEFAULT_SIZE 100 > #define DEFAULT_ALPHA 0.3 > #define DEFAULT_AMPL 3.0 > #define DEFAULT_W 3.0 > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > /* > Default problem parameters. These are always hard coded and never > overwritten > */ > #define INIT_STEP 1E-4 > #define INIT_TIME 0.0 > #define ABSERROR 1E-9 > #define RELERROR 1E-9 19,31c33,39 < This program solves the one-dimensional heat equation (also called the < diffusion equation), < u_t = u_xx, < on the domain 0 <= x <= 1, with the boundary conditions < u(t,0) = 0, u(t,1) = 0, < and the initial condition < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). < This is a linear, second-order, parabolic equation. < < We discretize the right-hand side using finite differences with < uniform grid spacing h: < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) < We then demonstrate time evolution using the various TS methods by --- > This program solves the one-dimensional Schroedinger equation in a > tightly bound lattice with a periodic time dependence > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > on the domain 0 <= m < L, with the boundary conditions > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > This is a set of linear, first-order ODEs > The time evolution using the various TS methods can be run by 34,46d41 < < We compare the approximate solution with the exact solution, given by < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + < 3*exp(-4*pi*pi*t) * sin(2*pi*x) < < Notes: < This code demonstrates the TS solver interface to two variants of < linear problems, u_t = f(u,t), namely < - time-dependent f: f(u,t) is a function of t < - time-independent f: f(u,t) is simply f(u) < < The uniprocessor version of this code is ts/examples/tutorials/ex3.c < 59a55 > 64d59 < 75d69 < Vec solution; /* global exact solution vector */ 77d70 < PetscReal h; /* mesh width h = 1/(m-1) */ 79,80c72,77 < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ < PetscReal norm_2, norm_max; /* error norms */ --- > PetscViewer viewer1; /* viewer for the solution */ > PetscReal norm_2; /* wavefunction norm */ > Vec randoms; /* random numbers between -1 and 1 */ > PetscReal alpha; /* disorder strength */ > PetscReal h0; /* drive amplitude */ > PetscReal w; /* drive frequency */ 85a83 > extern PetscInt kdel (PetscInt, PetscInt); 87,88c85,86 < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void *); < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); --- > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > void *); 90d87 < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); 101,102c98,99 < PetscReal time_total_max = 1.0; /* default max total time */ < PetscInt time_steps_max = 100; /* default max timesteps */ --- > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time */ > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ 108,109c105,107 < PetscBool flg; < TSProblemType tsproblem = TS_LINEAR; --- > PetscReal alpha; /* Disorder strength */ > PetscReal h0; /* Drive amplitude */ > PetscReal w; /* drive frequency */ 119c117 < m = 60; --- > m = DEFAULT_SIZE; 121a120,132 > > h0 = DEFAULT_AMPL; > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > CHKERRQ (ierr); > > alpha = DEFAULT_ALPHA; > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > CHKERRQ (ierr); > > w = DEFAULT_W; > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > CHKERRQ (ierr); > 123a135 > 125,127c137,141 < appctx.h = 1.0 / (m - 1.0); < appctx.norm_2 = 0.0; < appctx.norm_max = 0.0; --- > appctx.alpha = alpha; > appctx.h0 = h0; > appctx.w = w; > > appctx.norm_2 = 1.0; 163d176 < create global work vector for storing exact solution. 167,168d179 < ierr = VecDuplicate (u, &appctx.solution); < CHKERRQ (ierr); 171c182 < Set up displays to show graphs of the solution and error --- > Set up displays to show graphs of the solution 173,189c184,195 < < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, < &appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, < &appctx.viewer2); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); --- > if (appctx.debug) > { > ierr = > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > CHKERRQ (ierr); > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > CHKERRQ (ierr); > ierr = PetscDrawSetDoubleBuffer (draw); > CHKERRQ (ierr); > } 197,202c203 < < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); < CHKERRQ (ierr); < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); < CHKERRQ (ierr); --- > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is always linear */ 205c206 < Set optional user-defined monitoring routine --- > Set user-defined monitoring routine 211d211 < 224,225c224,257 < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); --- > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Create random number context and generate random numbers > into application context > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscRandom rctx; /* random number generator context */ > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetFromOptions (rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > CHKERRQ (ierr); > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > CHKERRQ (ierr); > ierr = VecSetFromOptions (appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetRandom (appctx.randoms, rctx); > CHKERRQ (ierr); > ierr = VecScale (appctx.randoms, randomscale); > CHKERRQ (ierr); > ierr = PetscRandomDestroy (&rctx); > CHKERRQ (ierr); > > /* > For linear problems with a time-dependent f(u,t) in the equation > u_t = f(u,t), the user provides the discretized right-hand-side > as a time-dependent matrix. > */ > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); > CHKERRQ (ierr); > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); 227,254d258 < if (flg) < { < /* < For linear problems with a time-dependent f(u,t) in the equation < u_t = f(u,t), the user provides the discretized right-hand-side < as a time-dependent matrix. < */ < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); < CHKERRQ (ierr); < } < else < { < /* < For linear problems with a time-independent f(u) in the equation < u_t = f(u), the user provides the discretized right-hand-side < as a matrix only once, and then sets a null matrix evaluation < routine. < */ < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); < CHKERRQ (ierr); < } 256,266d259 < if (tsproblem == TS_NONLINEAR) < { < SNES snes; < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); < CHKERRQ (ierr); < ierr = TSGetSNES (ts, &snes); < CHKERRQ (ierr); < ierr = < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); < CHKERRQ (ierr); < } 272,273c265,266 < dt = appctx.h * appctx.h / 2.0; < ierr = TSSetInitialTimeStep (ts, 0.0, dt); --- > dt = INIT_STEP; > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); 281a275 > - Set default tolerances 287a282,285 > ierr = TSSetType (ts, TSRK); > CHKERRQ (ierr); > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) Prince-Dormand (4) > CHKERRQ (ierr); 289a288,291 > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > CHKERRQ (ierr); > 321,324c323,324 < PetscPrintf (PETSC_COMM_WORLD, < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", < (double) (appctx.norm_2 / steps), < (double) (appctx.norm_max / steps)); --- > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > (double) (appctx.norm_2)); 338,341c338,343 < ierr = PetscViewerDestroy (&appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDestroy (&appctx.viewer2); < CHKERRQ (ierr); --- > > if (appctx.debug) > { > ierr = PetscViewerDestroy (&appctx.viewer1); > CHKERRQ (ierr); > } 344,345d345 < ierr = VecDestroy (&appctx.solution); < CHKERRQ (ierr); 347a348,349 > ierr = VecDestroy (&appctx.randoms); > CHKERRQ (ierr); 377c379 < PetscScalar *u_localptr, h = appctx->h; --- > PetscScalar *u_localptr; 379a382 > PetscScalar zero = 0.0, one = 1.0; 394,395d396 < - Note that the Fortran interface to VecGetArray() differs from the < C version. See the users manual for details. 402,403c403,404 < directly into the array locations. Alternatively, we could use < VecSetValues() or VecSetValuesLocal(). --- > directly into the array locations. > This will set the midpoint to unity 406,408c407,410 < u_localptr[i - mybase] = < PetscSinScalar (PETSC_PI * i * 6. * h) + < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); --- > if (i - mybase == floor ((appctx->m) / 2)) > u_localptr[i - mybase] = one; > else > u_localptr[i - mybase] = zero; 421c423 < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); --- > ierr = PetscPrintf (appctx->comm, "initial vector\n"); 432,486d433 < #define __FUNCT__ "ExactSolution" < /* < ExactSolution - Computes the exact solution at a given time. < < Input Parameters: < t - current time < solution - vector in which exact solution will be computed < appctx - user-defined application context < < Output Parameter: < solution - vector with the newly computed exact solution < */ < PetscErrorCode < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) < { < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; < PetscInt i, mybase, myend; < PetscErrorCode ierr; < < /* < Determine starting and ending points of each processor's < range of grid values < */ < ierr = VecGetOwnershipRange (solution, &mybase, &myend); < CHKERRQ (ierr); < < /* < Get a pointer to vector data. < */ < ierr = VecGetArray (solution, &s_localptr); < CHKERRQ (ierr); < < /* < Simply write the solution directly into the array locations. < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). < */ < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); < sc1 = PETSC_PI * 6. * h; < sc2 = PETSC_PI * 2. * h; < for (i = mybase; i < myend; i++) < s_localptr[i - mybase] = < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; < < /* < Restore vector < */ < ierr = VecRestoreArray (solution, &s_localptr); < CHKERRQ (ierr); < return 0; < } < < /* --------------------------------------------------------------------- */ < #undef __FUNCT__ 490,491c437 < each timestep. This example plots the solution and computes the < error in two different norms. --- > each timestep. This example plots the solution and returns the norm 509,515c455 < PetscReal norm_2, norm_max; < < /* < View a graph of the current iterate < */ < ierr = VecView (u, appctx->viewer2); < CHKERRQ (ierr); --- > PetscReal norm_2; 518c458 < Compute the exact solution --- > Compute and store norm of wavefunction in norm_2 520c460 < ierr = ExactSolution (time, appctx->solution, appctx); --- > ierr = VecNorm (u, NORM_2, &norm_2); 521a462 > appctx->norm_2 = norm_2; 523,536d463 < /* < Print debugging information if desired < */ < if (appctx->debug) < { < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < } 538,541c465,467 < /* < Compute the 2-norm and max-norm of the error < */ < ierr = VecAXPY (appctx->solution, -1.0, u); --- > /* Build the lattice sites */ > Vec uabs; > ierr = VecDuplicate (u, &uabs); 543c469,470 < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); --- > > ierr = VecCopy (u, uabs); 545,546c472 < norm_2 = PetscSqrtReal (appctx->h) * norm_2; < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); --- > ierr = VecAbs (uabs); 549,556c475 < /* < PetscPrintf() causes only the first processor in this < communicator to print the timestep information. < */ < ierr = < PetscPrintf (appctx->comm, < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", < step, (double) time, (double) norm_2, (double) norm_max); --- > ierr = VecPointwiseMult (uabs, uabs, uabs); 558,559d476 < appctx->norm_2 += norm_2; < appctx->norm_max += norm_max; 561,565c478,483 < /* < View a graph of the error < */ < ierr = VecView (appctx->solution, appctx->viewer1); < CHKERRQ (ierr); --- > //if (!appctx->debug) > //{ > /* Create another viewer in application context and dump full output to it */ > //ierr = VecView (u, appctx->viewer2); > //CHKERRQ (ierr); > //} 572,574c490,497 < ierr = PetscPrintf (appctx->comm, "Error vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > /* > View a graph of the current iterate > */ > ierr = > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > norm_2); > > ierr = VecView (uabs, appctx->viewer1); 576a500,501 > ierr = VecDestroy (&uabs); > CHKERRQ (ierr); 583c508 < #define __FUNCT__ "RHSMatrixHeat" --- > #define __FUNCT__ "RHSMatrixSchrodinger" 585c510 < RHSMatrixHeat - User-provided routine to compute the right-hand-side --- > RHSMatrixSchrodinger - User-provided routine to compute the right-hand-side 599,600c524 < Notes: < RHSMatrixHeat computes entries for the locally owned part of the system. --- > RHSMatrixSchrodinger computes entries for the locally owned part of the system. 613c537 < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) --- > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) 618,619c542,554 < PetscInt i, mstart, mend, idx[3]; < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * stwo; --- > PetscInt m = appctx->m; > PetscScalar drive; > PetscReal h0 = appctx->h0; > PetscReal w = appctx->w; > PetscScalar *randoms_loc; > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); > /*Calculate drive amplitude at this time */ > > drive = w * t; > drive = PetscSinReal (drive); > drive = 2.0 * PETSC_i * h0 * drive; 623,625c558,566 < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ < < ierr = MatGetOwnershipRange (A, &mstart, &mend); --- > The Jacobian Matrix in our case is blocked > - The values are A_{lm} = i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i h(t) \delta_{lm} > - Thus the superdiagonal and subdiagonal elements are just i, and diagonal elements are random + drive > - The elements are set rowwise. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscInt rowstart, rowend, rowcount; > PetscScalar edge[2], middle[3]; > PetscInt idxedge[2], idxmiddle[3]; > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); 627,655c568 < < /* < Set matrix rows corresponding to boundary data < */ < < if (mstart == 0) < { /* first processor only */ < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); < CHKERRQ (ierr); < mstart++; < } < < if (mend == appctx->m) < { /* last processor only */ < mend--; < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); < CHKERRQ (ierr); < } < < /* < Set matrix rows corresponding to interior data. We construct the < matrix one row at a time. < */ < v[0] = sone; < v[1] = stwo; < v[2] = sone; < for (i = mstart; i < mend; i++) --- > for (rowcount = rowstart; rowcount < rowend; rowcount++) 657,661c570,602 < idx[0] = i - 1; < idx[1] = i; < idx[2] = i + 1; < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); < CHKERRQ (ierr); --- > if (rowcount == 0) //If it is the first row of the whole matrix > { > edge[0] = drive + randoms_loc[0]; > edge[1] = PETSC_i; > idxedge[0] = 0; > idxedge[1] = 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else if (rowcount == m - 1) //If it is the last row of the whole matrix > { > edge[0] = PETSC_i; > edge[1] = drive + randoms_loc[m - 1]; > idxedge[0] = m - 2; > idxedge[1] = m - 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else > { > middle[0] = PETSC_i; > middle[1] = drive + randoms_loc[rowcount]; > middle[2] = PETSC_i; > idxmiddle[0] = rowcount - 1; > idxmiddle[1] = rowcount; > idxmiddle[2] = rowcount + 1; > ierr = > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > INSERT_VALUES); > CHKERRQ (ierr); > } 683a625,626 > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); 689,691c632,634 < #define __FUNCT__ "RHSFunctionHeat" < PetscErrorCode < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void *ctx) --- > #define __FUNCT__ "kdel" > PetscInt > kdel (PetscInt i, PetscInt j) 693,704c636,639 < PetscErrorCode ierr; < Mat A; < < PetscFunctionBeginUser; < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); < CHKERRQ (ierr); < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); < CHKERRQ (ierr); < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ < ierr = MatMult (A, globalin, globalout); < CHKERRQ (ierr); < PetscFunctionReturn (0); --- > PetscInt value = 0; > if (i == j) > value = 1; > return value; > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- --- *Analabha Roy* C.S.I.R Senior Research Associate Saha Institute of Nuclear Physics Section 1, Block AF Bidhannagar, Calcutta 700064 India *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com *Webpage*: http://www.ph.utexas.edu/~daneel/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 0a1,2 > //PROBLEMS: > //FIX PARALLELIZATION PROBLEM 3,8c5,11 < "Solves a simple time-dependent linear PDE (the heat equation).\n\ < Input parameters include:\n\ < -m , where = number of grid points\n\ < -time_dependent_rhs : Treat the problem as having a time-dependent right-hand side\n\ < -debug : Activate debugging printouts\n\ < -nox : Deactivate x-window graphics\n\n"; --- > "Solves a simple time-dependent Schroedinger equation for a TB model with periodic drive.\n\ > Apart from PetSc's routine options, input parameters include:\n\ > -m , where = lattice size\n\ > -a , where = drive amplitude\n\ > -d , where = disorder strength\n\ > -w , where = drive frequency\n\ > -debug : Activate debugging printouts\n\n"; 11,14c14 < Concepts: TS^time-dependent linear problems < Concepts: TS^heat equation < Concepts: TS^diffusion equation < Processors: n --- > Default problem parameters. These are overwritten by argument vectors 15a16,29 > #define DEFAULT_SIZE 100 > #define DEFAULT_ALPHA 0.3 > #define DEFAULT_AMPL 3.0 > #define DEFAULT_W 3.0 > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > /* > Default problem parameters. These are always hard coded and never > overwritten > */ > #define INIT_STEP 1E-4 > #define INIT_TIME 0.0 > #define ABSERROR 1E-9 > #define RELERROR 1E-9 19,31c33,39 < This program solves the one-dimensional heat equation (also called the < diffusion equation), < u_t = u_xx, < on the domain 0 <= x <= 1, with the boundary conditions < u(t,0) = 0, u(t,1) = 0, < and the initial condition < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). < This is a linear, second-order, parabolic equation. < < We discretize the right-hand side using finite differences with < uniform grid spacing h: < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) < We then demonstrate time evolution using the various TS methods by --- > This program solves the one-dimensional Schroedinger equation in a > tightly bound lattice with a periodic time dependence > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > on the domain 0 <= m < L, with the boundary conditions > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > This is a set of linear, first-order ODEs > The time evolution using the various TS methods can be run by 34,46d41 < < We compare the approximate solution with the exact solution, given by < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + < 3*exp(-4*pi*pi*t) * sin(2*pi*x) < < Notes: < This code demonstrates the TS solver interface to two variants of < linear problems, u_t = f(u,t), namely < - time-dependent f: f(u,t) is a function of t < - time-independent f: f(u,t) is simply f(u) < < The uniprocessor version of this code is ts/examples/tutorials/ex3.c < 59a55 > 64d59 < 75d69 < Vec solution; /* global exact solution vector */ 77d70 < PetscReal h; /* mesh width h = 1/(m-1) */ 79,80c72,77 < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ < PetscReal norm_2, norm_max; /* error norms */ --- > PetscViewer viewer1; /* viewer for the solution */ > PetscReal norm_2; /* wavefunction norm */ > Vec randoms; /* random numbers between -1 and 1 */ > PetscReal alpha; /* disorder strength */ > PetscReal h0; /* drive amplitude */ > PetscReal w; /* drive frequency */ 85a83 > extern PetscInt kdel (PetscInt, PetscInt); 87,88c85,86 < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void *); < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); --- > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > void *); 90d87 < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); 101,102c98,99 < PetscReal time_total_max = 1.0; /* default max total time */ < PetscInt time_steps_max = 100; /* default max timesteps */ --- > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time */ > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ 108,109c105,107 < PetscBool flg; < TSProblemType tsproblem = TS_LINEAR; --- > PetscReal alpha; /* Disorder strength */ > PetscReal h0; /* Drive amplitude */ > PetscReal w; /* drive frequency */ 119c117 < m = 60; --- > m = DEFAULT_SIZE; 121a120,132 > > h0 = DEFAULT_AMPL; > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > CHKERRQ (ierr); > > alpha = DEFAULT_ALPHA; > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > CHKERRQ (ierr); > > w = DEFAULT_W; > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > CHKERRQ (ierr); > 123a135 > 125,127c137,141 < appctx.h = 1.0 / (m - 1.0); < appctx.norm_2 = 0.0; < appctx.norm_max = 0.0; --- > appctx.alpha = alpha; > appctx.h0 = h0; > appctx.w = w; > > appctx.norm_2 = 1.0; 163d176 < create global work vector for storing exact solution. 167,168d179 < ierr = VecDuplicate (u, &appctx.solution); < CHKERRQ (ierr); 171c182 < Set up displays to show graphs of the solution and error --- > Set up displays to show graphs of the solution 173,189c184,195 < < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, < &appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); < ierr = < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, < &appctx.viewer2); < CHKERRQ (ierr); < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); < CHKERRQ (ierr); < ierr = PetscDrawSetDoubleBuffer (draw); < CHKERRQ (ierr); --- > if (appctx.debug) > { > ierr = > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > CHKERRQ (ierr); > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > CHKERRQ (ierr); > ierr = PetscDrawSetDoubleBuffer (draw); > CHKERRQ (ierr); > } 197,202c203 < < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); < CHKERRQ (ierr); < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); < CHKERRQ (ierr); --- > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is always linear */ 205c206 < Set optional user-defined monitoring routine --- > Set user-defined monitoring routine 211d211 < 224,225c224,257 < flg = PETSC_FALSE; < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); --- > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > Create random number context and generate random numbers > into application context > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscRandom rctx; /* random number generator context */ > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetFromOptions (rctx); > CHKERRQ (ierr); > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > CHKERRQ (ierr); > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > CHKERRQ (ierr); > ierr = VecSetFromOptions (appctx.randoms); > CHKERRQ (ierr); > ierr = VecSetRandom (appctx.randoms, rctx); > CHKERRQ (ierr); > ierr = VecScale (appctx.randoms, randomscale); > CHKERRQ (ierr); > ierr = PetscRandomDestroy (&rctx); > CHKERRQ (ierr); > > /* > For linear problems with a time-dependent f(u,t) in the equation > u_t = f(u,t), the user provides the discretized right-hand-side > as a time-dependent matrix. > */ > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); > CHKERRQ (ierr); > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); 227,254d258 < if (flg) < { < /* < For linear problems with a time-dependent f(u,t) in the equation < u_t = f(u,t), the user provides the discretized right-hand-side < as a time-dependent matrix. < */ < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); < CHKERRQ (ierr); < } < else < { < /* < For linear problems with a time-independent f(u) in the equation < u_t = f(u), the user provides the discretized right-hand-side < as a matrix only once, and then sets a null matrix evaluation < routine. < */ < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); < CHKERRQ (ierr); < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, &appctx); < CHKERRQ (ierr); < ierr = < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); < CHKERRQ (ierr); < } 256,266d259 < if (tsproblem == TS_NONLINEAR) < { < SNES snes; < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); < CHKERRQ (ierr); < ierr = TSGetSNES (ts, &snes); < CHKERRQ (ierr); < ierr = < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); < CHKERRQ (ierr); < } 272,273c265,266 < dt = appctx.h * appctx.h / 2.0; < ierr = TSSetInitialTimeStep (ts, 0.0, dt); --- > dt = INIT_STEP; > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); 281a275 > - Set default tolerances 287a282,285 > ierr = TSSetType (ts, TSRK); > CHKERRQ (ierr); > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) Prince-Dormand (4) > CHKERRQ (ierr); 289a288,291 > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > CHKERRQ (ierr); > 321,324c323,324 < PetscPrintf (PETSC_COMM_WORLD, < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", < (double) (appctx.norm_2 / steps), < (double) (appctx.norm_max / steps)); --- > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > (double) (appctx.norm_2)); 338,341c338,343 < ierr = PetscViewerDestroy (&appctx.viewer1); < CHKERRQ (ierr); < ierr = PetscViewerDestroy (&appctx.viewer2); < CHKERRQ (ierr); --- > > if (appctx.debug) > { > ierr = PetscViewerDestroy (&appctx.viewer1); > CHKERRQ (ierr); > } 344,345d345 < ierr = VecDestroy (&appctx.solution); < CHKERRQ (ierr); 347a348,349 > ierr = VecDestroy (&appctx.randoms); > CHKERRQ (ierr); 377c379 < PetscScalar *u_localptr, h = appctx->h; --- > PetscScalar *u_localptr; 379a382 > PetscScalar zero = 0.0, one = 1.0; 394,395d396 < - Note that the Fortran interface to VecGetArray() differs from the < C version. See the users manual for details. 402,403c403,404 < directly into the array locations. Alternatively, we could use < VecSetValues() or VecSetValuesLocal(). --- > directly into the array locations. > This will set the midpoint to unity 406,408c407,410 < u_localptr[i - mybase] = < PetscSinScalar (PETSC_PI * i * 6. * h) + < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); --- > if (i - mybase == floor ((appctx->m) / 2)) > u_localptr[i - mybase] = one; > else > u_localptr[i - mybase] = zero; 421c423 < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); --- > ierr = PetscPrintf (appctx->comm, "initial vector\n"); 432,486d433 < #define __FUNCT__ "ExactSolution" < /* < ExactSolution - Computes the exact solution at a given time. < < Input Parameters: < t - current time < solution - vector in which exact solution will be computed < appctx - user-defined application context < < Output Parameter: < solution - vector with the newly computed exact solution < */ < PetscErrorCode < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) < { < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; < PetscInt i, mybase, myend; < PetscErrorCode ierr; < < /* < Determine starting and ending points of each processor's < range of grid values < */ < ierr = VecGetOwnershipRange (solution, &mybase, &myend); < CHKERRQ (ierr); < < /* < Get a pointer to vector data. < */ < ierr = VecGetArray (solution, &s_localptr); < CHKERRQ (ierr); < < /* < Simply write the solution directly into the array locations. < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). < */ < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); < sc1 = PETSC_PI * 6. * h; < sc2 = PETSC_PI * 2. * h; < for (i = mybase; i < myend; i++) < s_localptr[i - mybase] = < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; < < /* < Restore vector < */ < ierr = VecRestoreArray (solution, &s_localptr); < CHKERRQ (ierr); < return 0; < } < < /* --------------------------------------------------------------------- */ < #undef __FUNCT__ 490,491c437 < each timestep. This example plots the solution and computes the < error in two different norms. --- > each timestep. This example plots the solution and returns the norm 509,515c455 < PetscReal norm_2, norm_max; < < /* < View a graph of the current iterate < */ < ierr = VecView (u, appctx->viewer2); < CHKERRQ (ierr); --- > PetscReal norm_2; 518c458 < Compute the exact solution --- > Compute and store norm of wavefunction in norm_2 520c460 < ierr = ExactSolution (time, appctx->solution, appctx); --- > ierr = VecNorm (u, NORM_2, &norm_2); 521a462 > appctx->norm_2 = norm_2; 523,536d463 < /* < Print debugging information if desired < */ < if (appctx->debug) < { < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); < CHKERRQ (ierr); < } 538,541c465,467 < /* < Compute the 2-norm and max-norm of the error < */ < ierr = VecAXPY (appctx->solution, -1.0, u); --- > /* Build the lattice sites */ > Vec uabs; > ierr = VecDuplicate (u, &uabs); 543c469,470 < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); --- > > ierr = VecCopy (u, uabs); 545,546c472 < norm_2 = PetscSqrtReal (appctx->h) * norm_2; < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); --- > ierr = VecAbs (uabs); 549,556c475 < /* < PetscPrintf() causes only the first processor in this < communicator to print the timestep information. < */ < ierr = < PetscPrintf (appctx->comm, < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", < step, (double) time, (double) norm_2, (double) norm_max); --- > ierr = VecPointwiseMult (uabs, uabs, uabs); 558,559d476 < appctx->norm_2 += norm_2; < appctx->norm_max += norm_max; 561,565c478,483 < /* < View a graph of the error < */ < ierr = VecView (appctx->solution, appctx->viewer1); < CHKERRQ (ierr); --- > //if (!appctx->debug) > //{ > /* Create another viewer in application context and dump full output to it */ > //ierr = VecView (u, appctx->viewer2); > //CHKERRQ (ierr); > //} 572,574c490,497 < ierr = PetscPrintf (appctx->comm, "Error vector\n"); < CHKERRQ (ierr); < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); --- > /* > View a graph of the current iterate > */ > ierr = > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > norm_2); > > ierr = VecView (uabs, appctx->viewer1); 576a500,501 > ierr = VecDestroy (&uabs); > CHKERRQ (ierr); 583c508 < #define __FUNCT__ "RHSMatrixHeat" --- > #define __FUNCT__ "RHSMatrixSchrodinger" 585c510 < RHSMatrixHeat - User-provided routine to compute the right-hand-side --- > RHSMatrixSchrodinger - User-provided routine to compute the right-hand-side 599,600c524 < Notes: < RHSMatrixHeat computes entries for the locally owned part of the system. --- > RHSMatrixSchrodinger computes entries for the locally owned part of the system. 613c537 < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) --- > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) 618,619c542,554 < PetscInt i, mstart, mend, idx[3]; < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * stwo; --- > PetscInt m = appctx->m; > PetscScalar drive; > PetscReal h0 = appctx->h0; > PetscReal w = appctx->w; > PetscScalar *randoms_loc; > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); > /*Calculate drive amplitude at this time */ > > drive = w * t; > drive = PetscSinReal (drive); > drive = 2.0 * PETSC_i * h0 * drive; 623,625c558,566 < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ < < ierr = MatGetOwnershipRange (A, &mstart, &mend); --- > The Jacobian Matrix in our case is blocked > - The values are A_{lm} = i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i h(t) \delta_{lm} > - Thus the superdiagonal and subdiagonal elements are just i, and diagonal elements are random + drive > - The elements are set rowwise. > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ > PetscInt rowstart, rowend, rowcount; > PetscScalar edge[2], middle[3]; > PetscInt idxedge[2], idxmiddle[3]; > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); 627,655c568 < < /* < Set matrix rows corresponding to boundary data < */ < < if (mstart == 0) < { /* first processor only */ < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); < CHKERRQ (ierr); < mstart++; < } < < if (mend == appctx->m) < { /* last processor only */ < mend--; < v[0] = 1.0; < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); < CHKERRQ (ierr); < } < < /* < Set matrix rows corresponding to interior data. We construct the < matrix one row at a time. < */ < v[0] = sone; < v[1] = stwo; < v[2] = sone; < for (i = mstart; i < mend; i++) --- > for (rowcount = rowstart; rowcount < rowend; rowcount++) 657,661c570,602 < idx[0] = i - 1; < idx[1] = i; < idx[2] = i + 1; < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); < CHKERRQ (ierr); --- > if (rowcount == 0) //If it is the first row of the whole matrix > { > edge[0] = drive + randoms_loc[0]; > edge[1] = PETSC_i; > idxedge[0] = 0; > idxedge[1] = 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else if (rowcount == m - 1) //If it is the last row of the whole matrix > { > edge[0] = PETSC_i; > edge[1] = drive + randoms_loc[m - 1]; > idxedge[0] = m - 2; > idxedge[1] = m - 1; > ierr = > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > CHKERRQ (ierr); > } > else > { > middle[0] = PETSC_i; > middle[1] = drive + randoms_loc[rowcount]; > middle[2] = PETSC_i; > idxmiddle[0] = rowcount - 1; > idxmiddle[1] = rowcount; > idxmiddle[2] = rowcount + 1; > ierr = > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > INSERT_VALUES); > CHKERRQ (ierr); > } 683a625,626 > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > CHKERRQ (ierr); 689,691c632,634 < #define __FUNCT__ "RHSFunctionHeat" < PetscErrorCode < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void *ctx) --- > #define __FUNCT__ "kdel" > PetscInt > kdel (PetscInt i, PetscInt j) 693,704c636,639 < PetscErrorCode ierr; < Mat A; < < PetscFunctionBeginUser; < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); < CHKERRQ (ierr); < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); < CHKERRQ (ierr); < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ < ierr = MatMult (A, globalin, globalout); < CHKERRQ (ierr); < PetscFunctionReturn (0); --- > PetscInt value = 0; > if (i == j) > value = 1; > return value; From hariseldon99 at gmail.com Thu Oct 16 04:08:07 2014 From: hariseldon99 at gmail.com (Analabha Roy) Date: Thu, 16 Oct 2014 14:38:07 +0530 Subject: [petsc-users] TS example tutorial example 4 adapted - not parallelizing In-Reply-To: References: Message-ID: Hi all, Update: Turns out i set my initial condition wrong due to incorrectly using local indices while setting the value, and running it on multiple processor causes overstriding, setting all vector elements to zero. I have fixed it. Basically, line 407 changes from the local index (i-ibase) to the global index (just i). for (i = mybase; i < myend; i++) if (i == floor ((appctx->m) / 2)) u_localptr[i - mybase] = one; else u_localptr[i - mybase] = zero; A trivially silly thing I'm afraid. Thanks for pointing out my other unrelated error. I think I will compute the variance using MatMult() instead so as to keep everything collective. Basically set a diagonal matrix to X_{ii} = i and Xsq_{ii} = i^2 and compute expectation values using MatMult() and VecDot() Regards, Analabha On Thu, Oct 16, 2014 at 1:18 PM, Analabha Roy wrote: > Hi all, > > Thanks for the response. > > On Wed, Oct 15, 2014 at 4:15 PM, Matthew Knepley > wrote: >> >> I am not sure what you are doing at line 483. You seem to have all >> processes indexing a Vec >> at all entries, which cannot be correct. >> >> > Yes, you're right, of course. I have removed that bit (I'll do it in > post-processing after I have dumped the full state data to file). > > In the meantime, the graphical output is still messed up when the code is > run with debug flag in parallel. However, parallel graphical output works > fine in the original ex4.c. I can;t figure out why... > > I have updated diff (link to pretty version > ) attached and posted below. > > > Thanks and Regards, > Analabha > > 0a1,2 > > //PROBLEMS: > > //FIX PARALLELIZATION PROBLEM > 3,8c5,11 > < "Solves a simple time-dependent linear PDE (the heat equation).\n\ > < Input parameters include:\n\ > < -m , where = number of grid points\n\ > < -time_dependent_rhs : Treat the problem as having a time-dependent > right-hand side\n\ > < -debug : Activate debugging printouts\n\ > < -nox : Deactivate x-window graphics\n\n"; > --- > > "Solves a simple time-dependent Schroedinger equation for a TB model > with periodic drive.\n\ > > Apart from PetSc's routine options, input parameters include:\n\ > > -m , where = lattice size\n\ > > -a , where = drive amplitude\n\ > > -d , where = disorder strength\n\ > > -w , where = drive frequency\n\ > > -debug : Activate debugging printouts\n\n"; > 11,14c14 > < Concepts: TS^time-dependent linear problems > < Concepts: TS^heat equation > < Concepts: TS^diffusion equation > < Processors: n > --- > > Default problem parameters. These are overwritten by argument vectors > 15a16,29 > > #define DEFAULT_SIZE 100 > > #define DEFAULT_ALPHA 0.3 > > #define DEFAULT_AMPL 3.0 > > #define DEFAULT_W 3.0 > > #define TIME_TOTAL_MAX 100.0 /* default max total time */ > > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ > > /* > > Default problem parameters. These are always hard coded and never > > overwritten > > */ > > #define INIT_STEP 1E-4 > > #define INIT_TIME 0.0 > > #define ABSERROR 1E-9 > > #define RELERROR 1E-9 > 19,31c33,39 > < This program solves the one-dimensional heat equation (also called the > < diffusion equation), > < u_t = u_xx, > < on the domain 0 <= x <= 1, with the boundary conditions > < u(t,0) = 0, u(t,1) = 0, > < and the initial condition > < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). > < This is a linear, second-order, parabolic equation. > < > < We discretize the right-hand side using finite differences with > < uniform grid spacing h: > < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) > < We then demonstrate time evolution using the various TS methods by > --- > > This program solves the one-dimensional Schroedinger equation in a > > tightly bound lattice with a periodic time dependence > > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), > > on the domain 0 <= m < L, with the boundary conditions > > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 > > This is a set of linear, first-order ODEs > > The time evolution using the various TS methods can be run by > 34,46d41 > < > < We compare the approximate solution with the exact solution, given by > < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + > < 3*exp(-4*pi*pi*t) * sin(2*pi*x) > < > < Notes: > < This code demonstrates the TS solver interface to two variants of > < linear problems, u_t = f(u,t), namely > < - time-dependent f: f(u,t) is a function of t > < - time-independent f: f(u,t) is simply f(u) > < > < The uniprocessor version of this code is ts/examples/tutorials/ex3.c > < > 59a55 > > > 64d59 > < > 75d69 > < Vec solution; /* global exact solution vector */ > 77d70 > < PetscReal h; /* mesh width h = 1/(m-1) */ > 79,80c72,77 > < PetscViewer viewer1, viewer2; /* viewers for the solution and error */ > < PetscReal norm_2, norm_max; /* error norms */ > --- > > PetscViewer viewer1; /* viewer for the solution */ > > PetscReal norm_2; /* wavefunction norm */ > > Vec randoms; /* random numbers between -1 and 1 */ > > PetscReal alpha; /* disorder strength */ > > PetscReal h0; /* drive amplitude */ > > PetscReal w; /* drive frequency */ > 85a83 > > extern PetscInt kdel (PetscInt, PetscInt); > 87,88c85,86 > < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void > *); > < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); > --- > > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, Mat, > > void *); > 90d87 > < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); > 101,102c98,99 > < PetscReal time_total_max = 1.0; /* default max total time */ > < PetscInt time_steps_max = 100; /* default max timesteps */ > --- > > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time > */ > > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ > 108,109c105,107 > < PetscBool flg; > < TSProblemType tsproblem = TS_LINEAR; > --- > > PetscReal alpha; /* Disorder strength */ > > PetscReal h0; /* Drive amplitude */ > > PetscReal w; /* drive frequency */ > 119c117 > < m = 60; > --- > > m = DEFAULT_SIZE; > 121a120,132 > > > > h0 = DEFAULT_AMPL; > > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); > > CHKERRQ (ierr); > > > > alpha = DEFAULT_ALPHA; > > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); > > CHKERRQ (ierr); > > > > w = DEFAULT_W; > > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); > > CHKERRQ (ierr); > > > 123a135 > > > 125,127c137,141 > < appctx.h = 1.0 / (m - 1.0); > < appctx.norm_2 = 0.0; > < appctx.norm_max = 0.0; > --- > > appctx.alpha = alpha; > > appctx.h0 = h0; > > appctx.w = w; > > > > appctx.norm_2 = 1.0; > 163d176 > < create global work vector for storing exact solution. > 167,168d179 > < ierr = VecDuplicate (u, &appctx.solution); > < CHKERRQ (ierr); > 171c182 > < Set up displays to show graphs of the solution and error > --- > > Set up displays to show graphs of the solution > 173,189c184,195 > < > < ierr = > < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, > < &appctx.viewer1); > < CHKERRQ (ierr); > < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > < CHKERRQ (ierr); > < ierr = PetscDrawSetDoubleBuffer (draw); > < CHKERRQ (ierr); > < ierr = > < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, > < &appctx.viewer2); > < CHKERRQ (ierr); > < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); > < CHKERRQ (ierr); > < ierr = PetscDrawSetDoubleBuffer (draw); > < CHKERRQ (ierr); > --- > > if (appctx.debug) > > { > > ierr = > > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", > > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, > > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); > > CHKERRQ (ierr); > > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); > > CHKERRQ (ierr); > > ierr = PetscDrawSetDoubleBuffer (draw); > > CHKERRQ (ierr); > > } > 197,202c203 > < > < flg = PETSC_FALSE; > < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); > < CHKERRQ (ierr); > < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); > < CHKERRQ (ierr); > --- > > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is > always linear */ > 205c206 > < Set optional user-defined monitoring routine > --- > > Set user-defined monitoring routine > 211d211 > < > 224,225c224,257 > < flg = PETSC_FALSE; > < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); > --- > > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > Create random number context and generate random numbers > > into application context > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > > PetscRandom rctx; /* random number generator context */ > > PetscScalar randomscale = 2.0 * PETSC_i * alpha; > > > > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); > > CHKERRQ (ierr); > > ierr = PetscRandomSetFromOptions (rctx); > > CHKERRQ (ierr); > > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); > > CHKERRQ (ierr); > > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); > > CHKERRQ (ierr); > > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); > > CHKERRQ (ierr); > > ierr = VecSetFromOptions (appctx.randoms); > > CHKERRQ (ierr); > > ierr = VecSetRandom (appctx.randoms, rctx); > > CHKERRQ (ierr); > > ierr = VecScale (appctx.randoms, randomscale); > > CHKERRQ (ierr); > > ierr = PetscRandomDestroy (&rctx); > > CHKERRQ (ierr); > > > > /* > > For linear problems with a time-dependent f(u,t) in the equation > > u_t = f(u,t), the user provides the discretized right-hand-side > > as a time-dependent matrix. > > */ > > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > > CHKERRQ (ierr); > > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); > 227,254d258 > < if (flg) > < { > < /* > < For linear problems with a time-dependent f(u,t) in the equation > < u_t = f(u,t), the user provides the discretized right-hand-side > < as a time-dependent matrix. > < */ > < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > < CHKERRQ (ierr); > < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); > < CHKERRQ (ierr); > < } > < else > < { > < /* > < For linear problems with a time-independent f(u) in the equation > < u_t = f(u), the user provides the discretized right-hand-side > < as a matrix only once, and then sets a null matrix evaluation > < routine. > < */ > < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); > < CHKERRQ (ierr); > < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, > &appctx); > < CHKERRQ (ierr); > < ierr = > < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); > < CHKERRQ (ierr); > < } > 256,266d259 > < if (tsproblem == TS_NONLINEAR) > < { > < SNES snes; > < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); > < CHKERRQ (ierr); > < ierr = TSGetSNES (ts, &snes); > < CHKERRQ (ierr); > < ierr = > < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); > < CHKERRQ (ierr); > < } > 272,273c265,266 > < dt = appctx.h * appctx.h / 2.0; > < ierr = TSSetInitialTimeStep (ts, 0.0, dt); > --- > > dt = INIT_STEP; > > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); > 281a275 > > - Set default tolerances > 287a282,285 > > ierr = TSSetType (ts, TSRK); > > CHKERRQ (ierr); > > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta (5) > Prince-Dormand (4) > > CHKERRQ (ierr); > 289a288,291 > > > > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); > > CHKERRQ (ierr); > > > 321,324c323,324 > < PetscPrintf (PETSC_COMM_WORLD, > < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", > < (double) (appctx.norm_2 / steps), > < (double) (appctx.norm_max / steps)); > --- > > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", > > (double) (appctx.norm_2)); > 338,341c338,343 > < ierr = PetscViewerDestroy (&appctx.viewer1); > < CHKERRQ (ierr); > < ierr = PetscViewerDestroy (&appctx.viewer2); > < CHKERRQ (ierr); > --- > > > > if (appctx.debug) > > { > > ierr = PetscViewerDestroy (&appctx.viewer1); > > CHKERRQ (ierr); > > } > 344,345d345 > < ierr = VecDestroy (&appctx.solution); > < CHKERRQ (ierr); > 347a348,349 > > ierr = VecDestroy (&appctx.randoms); > > CHKERRQ (ierr); > 377c379 > < PetscScalar *u_localptr, h = appctx->h; > --- > > PetscScalar *u_localptr; > 379a382 > > PetscScalar zero = 0.0, one = 1.0; > 394,395d396 > < - Note that the Fortran interface to VecGetArray() differs from the > < C version. See the users manual for details. > 402,403c403,404 > < directly into the array locations. Alternatively, we could use > < VecSetValues() or VecSetValuesLocal(). > --- > > directly into the array locations. > > This will set the midpoint to unity > 406,408c407,410 > < u_localptr[i - mybase] = > < PetscSinScalar (PETSC_PI * i * 6. * h) + > < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); > --- > > if (i - mybase == floor ((appctx->m) / 2)) > > u_localptr[i - mybase] = one; > > else > > u_localptr[i - mybase] = zero; > 421c423 > < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); > --- > > ierr = PetscPrintf (appctx->comm, "initial vector\n"); > 432,486d433 > < #define __FUNCT__ "ExactSolution" > < /* > < ExactSolution - Computes the exact solution at a given time. > < > < Input Parameters: > < t - current time > < solution - vector in which exact solution will be computed > < appctx - user-defined application context > < > < Output Parameter: > < solution - vector with the newly computed exact solution > < */ > < PetscErrorCode > < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) > < { > < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; > < PetscInt i, mybase, myend; > < PetscErrorCode ierr; > < > < /* > < Determine starting and ending points of each processor's > < range of grid values > < */ > < ierr = VecGetOwnershipRange (solution, &mybase, &myend); > < CHKERRQ (ierr); > < > < /* > < Get a pointer to vector data. > < */ > < ierr = VecGetArray (solution, &s_localptr); > < CHKERRQ (ierr); > < > < /* > < Simply write the solution directly into the array locations. > < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). > < */ > < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); > < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); > < sc1 = PETSC_PI * 6. * h; > < sc2 = PETSC_PI * 2. * h; > < for (i = mybase; i < myend; i++) > < s_localptr[i - mybase] = > < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + > < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; > < > < /* > < Restore vector > < */ > < ierr = VecRestoreArray (solution, &s_localptr); > < CHKERRQ (ierr); > < return 0; > < } > < > < /* --------------------------------------------------------------------- > */ > < #undef __FUNCT__ > 490,491c437 > < each timestep. This example plots the solution and computes the > < error in two different norms. > --- > > each timestep. This example plots the solution and returns the norm > 509,515c455 > < PetscReal norm_2, norm_max; > < > < /* > < View a graph of the current iterate > < */ > < ierr = VecView (u, appctx->viewer2); > < CHKERRQ (ierr); > --- > > PetscReal norm_2; > 518c458 > < Compute the exact solution > --- > > Compute and store norm of wavefunction in norm_2 > 520c460 > < ierr = ExactSolution (time, appctx->solution, appctx); > --- > > ierr = VecNorm (u, NORM_2, &norm_2); > 521a462 > > appctx->norm_2 = norm_2; > 523,536d463 > < /* > < Print debugging information if desired > < */ > < if (appctx->debug) > < { > < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); > < CHKERRQ (ierr); > < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); > < CHKERRQ (ierr); > < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); > < CHKERRQ (ierr); > < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); > < CHKERRQ (ierr); > < } > 538,541c465,467 > < /* > < Compute the 2-norm and max-norm of the error > < */ > < ierr = VecAXPY (appctx->solution, -1.0, u); > --- > > /* Build the lattice sites */ > > Vec uabs; > > ierr = VecDuplicate (u, &uabs); > 543c469,470 > < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); > --- > > > > ierr = VecCopy (u, uabs); > 545,546c472 > < norm_2 = PetscSqrtReal (appctx->h) * norm_2; > < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); > --- > > ierr = VecAbs (uabs); > 549,556c475 > < /* > < PetscPrintf() causes only the first processor in this > < communicator to print the timestep information. > < */ > < ierr = > < PetscPrintf (appctx->comm, > < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", > < step, (double) time, (double) norm_2, (double) norm_max); > --- > > ierr = VecPointwiseMult (uabs, uabs, uabs); > 558,559d476 > < appctx->norm_2 += norm_2; > < appctx->norm_max += norm_max; > 561,565c478,483 > < /* > < View a graph of the error > < */ > < ierr = VecView (appctx->solution, appctx->viewer1); > < CHKERRQ (ierr); > --- > > //if (!appctx->debug) > > //{ > > /* Create another viewer in application context and dump full output > to it */ > > //ierr = VecView (u, appctx->viewer2); > > //CHKERRQ (ierr); > > //} > 572,574c490,497 > < ierr = PetscPrintf (appctx->comm, "Error vector\n"); > < CHKERRQ (ierr); > < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); > --- > > /* > > View a graph of the current iterate > > */ > > ierr = > > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, > > norm_2); > > > > ierr = VecView (uabs, appctx->viewer1); > 576a500,501 > > ierr = VecDestroy (&uabs); > > CHKERRQ (ierr); > 583c508 > < #define __FUNCT__ "RHSMatrixHeat" > --- > > #define __FUNCT__ "RHSMatrixSchrodinger" > 585c510 > < RHSMatrixHeat - User-provided routine to compute the right-hand-side > --- > > RHSMatrixSchrodinger - User-provided routine to compute the > right-hand-side > 599,600c524 > < Notes: > < RHSMatrixHeat computes entries for the locally owned part of the > system. > --- > > RHSMatrixSchrodinger computes entries for the locally owned part of > the system. > 613c537 > < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) > --- > > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void > *ctx) > 618,619c542,554 > < PetscInt i, mstart, mend, idx[3]; > < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * > stwo; > --- > > PetscInt m = appctx->m; > > PetscScalar drive; > > PetscReal h0 = appctx->h0; > > PetscReal w = appctx->w; > > PetscScalar *randoms_loc; > > > > ierr = VecGetArray (appctx->randoms, &randoms_loc); > > CHKERRQ (ierr); > > /*Calculate drive amplitude at this time */ > > > > drive = w * t; > > drive = PetscSinReal (drive); > > drive = 2.0 * PETSC_i * h0 * drive; > 623,625c558,566 > < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > < > < ierr = MatGetOwnershipRange (A, &mstart, &mend); > --- > > The Jacobian Matrix in our case is blocked > > - The values are A_{lm} = > i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i > h(t) \delta_{lm} > > - Thus the superdiagonal and subdiagonal elements are just i, and > diagonal elements are random + drive > > - The elements are set rowwise. > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > */ > > PetscInt rowstart, rowend, rowcount; > > PetscScalar edge[2], middle[3]; > > PetscInt idxedge[2], idxmiddle[3]; > > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); > 627,655c568 > < > < /* > < Set matrix rows corresponding to boundary data > < */ > < > < if (mstart == 0) > < { /* first processor only */ > < v[0] = 1.0; > < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); > < CHKERRQ (ierr); > < mstart++; > < } > < > < if (mend == appctx->m) > < { /* last processor only */ > < mend--; > < v[0] = 1.0; > < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); > < CHKERRQ (ierr); > < } > < > < /* > < Set matrix rows corresponding to interior data. We construct the > < matrix one row at a time. > < */ > < v[0] = sone; > < v[1] = stwo; > < v[2] = sone; > < for (i = mstart; i < mend; i++) > --- > > for (rowcount = rowstart; rowcount < rowend; rowcount++) > 657,661c570,602 > < idx[0] = i - 1; > < idx[1] = i; > < idx[2] = i + 1; > < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); > < CHKERRQ (ierr); > --- > > if (rowcount == 0) //If it is the first row of the whole matrix > > { > > edge[0] = drive + randoms_loc[0]; > > edge[1] = PETSC_i; > > idxedge[0] = 0; > > idxedge[1] = 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > > CHKERRQ (ierr); > > } > > else if (rowcount == m - 1) //If it is the last row of the whole > matrix > > { > > edge[0] = PETSC_i; > > edge[1] = drive + randoms_loc[m - 1]; > > idxedge[0] = m - 2; > > idxedge[1] = m - 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); > > CHKERRQ (ierr); > > } > > else > > { > > middle[0] = PETSC_i; > > middle[1] = drive + randoms_loc[rowcount]; > > middle[2] = PETSC_i; > > idxmiddle[0] = rowcount - 1; > > idxmiddle[1] = rowcount; > > idxmiddle[2] = rowcount + 1; > > ierr = > > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, > > INSERT_VALUES); > > CHKERRQ (ierr); > > } > 683a625,626 > > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); > > CHKERRQ (ierr); > 689,691c632,634 > < #define __FUNCT__ "RHSFunctionHeat" > < PetscErrorCode > < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void > *ctx) > --- > > #define __FUNCT__ "kdel" > > PetscInt > > kdel (PetscInt i, PetscInt j) > 693,704c636,639 > < PetscErrorCode ierr; > < Mat A; > < > < PetscFunctionBeginUser; > < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); > < CHKERRQ (ierr); > < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); > < CHKERRQ (ierr); > < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ > < ierr = MatMult (A, globalin, globalout); > < CHKERRQ (ierr); > < PetscFunctionReturn (0); > --- > > PetscInt value = 0; > > if (i == j) > > value = 1; > > return value; > > > >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > --- > *Analabha Roy* > C.S.I.R Senior Research Associate > > Saha Institute of Nuclear Physics > Section 1, Block AF > Bidhannagar, Calcutta 700064 > India > *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com > *Webpage*: http://www.ph.utexas.edu/~daneel/ > -- --- *Analabha Roy* C.S.I.R Senior Research Associate Saha Institute of Nuclear Physics Section 1, Block AF Bidhannagar, Calcutta 700064 India *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com *Webpage*: http://www.ph.utexas.edu/~daneel/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 16 07:53:59 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 16 Oct 2014 07:53:59 -0500 Subject: [petsc-users] TS example tutorial example 4 adapted - not parallelizing In-Reply-To: References: Message-ID: On Thu, Oct 16, 2014 at 4:08 AM, Analabha Roy wrote: > Hi all, > > Update: Turns out i set my initial condition wrong due to incorrectly > using local indices while setting the value, and running it on multiple > processor causes overstriding, setting all vector elements to zero. > I am glad its working. One way to find out of bounds access is to use valgrind, which works great. Thanks, Matt > I have fixed it. Basically, line 407 changes from the local index > (i-ibase) to the global index (just i). > > for (i = mybase; i < myend; i++) > if (i == floor ((appctx->m) / 2)) > u_localptr[i - mybase] = one; > else > u_localptr[i - mybase] = zero; > > A trivially silly thing I'm afraid. > > Thanks for pointing out my other unrelated error. I think I will compute > the variance using MatMult() instead so as to keep everything collective. > > Basically set a diagonal matrix to X_{ii} = i and Xsq_{ii} = i^2 and > compute expectation values using MatMult() and VecDot() > > Regards, > Analabha > > > > > On Thu, Oct 16, 2014 at 1:18 PM, Analabha Roy > wrote: > >> Hi all, >> >> Thanks for the response. >> >> On Wed, Oct 15, 2014 at 4:15 PM, Matthew Knepley >> wrote: >>> >>> I am not sure what you are doing at line 483. You seem to have all >>> processes indexing a Vec >>> at all entries, which cannot be correct. >>> >>> >> Yes, you're right, of course. I have removed that bit (I'll do it in >> post-processing after I have dumped the full state data to file). >> >> In the meantime, the graphical output is still messed up when the code is >> run with debug flag in parallel. However, parallel graphical output works >> fine in the original ex4.c. I can;t figure out why... >> >> I have updated diff (link to pretty version >> ) attached and posted below. >> >> >> Thanks and Regards, >> Analabha >> >> 0a1,2 >> > //PROBLEMS: >> > //FIX PARALLELIZATION PROBLEM >> 3,8c5,11 >> < "Solves a simple time-dependent linear PDE (the heat equation).\n\ >> < Input parameters include:\n\ >> < -m , where = number of grid points\n\ >> < -time_dependent_rhs : Treat the problem as having a time-dependent >> right-hand side\n\ >> < -debug : Activate debugging printouts\n\ >> < -nox : Deactivate x-window graphics\n\n"; >> --- >> > "Solves a simple time-dependent Schroedinger equation for a TB model >> with periodic drive.\n\ >> > Apart from PetSc's routine options, input parameters include:\n\ >> > -m , where = lattice size\n\ >> > -a , where = drive amplitude\n\ >> > -d , where = disorder strength\n\ >> > -w , where = drive frequency\n\ >> > -debug : Activate debugging printouts\n\n"; >> 11,14c14 >> < Concepts: TS^time-dependent linear problems >> < Concepts: TS^heat equation >> < Concepts: TS^diffusion equation >> < Processors: n >> --- >> > Default problem parameters. These are overwritten by argument vectors >> 15a16,29 >> > #define DEFAULT_SIZE 100 >> > #define DEFAULT_ALPHA 0.3 >> > #define DEFAULT_AMPL 3.0 >> > #define DEFAULT_W 3.0 >> > #define TIME_TOTAL_MAX 100.0 /* default max total time */ >> > #define TIME_STEPS_MAX 1E5 /* default max timesteps */ >> > /* >> > Default problem parameters. These are always hard coded and never >> > overwritten >> > */ >> > #define INIT_STEP 1E-4 >> > #define INIT_TIME 0.0 >> > #define ABSERROR 1E-9 >> > #define RELERROR 1E-9 >> 19,31c33,39 >> < This program solves the one-dimensional heat equation (also called >> the >> < diffusion equation), >> < u_t = u_xx, >> < on the domain 0 <= x <= 1, with the boundary conditions >> < u(t,0) = 0, u(t,1) = 0, >> < and the initial condition >> < u(0,x) = sin(6*pi*x) + 3*sin(2*pi*x). >> < This is a linear, second-order, parabolic equation. >> < >> < We discretize the right-hand side using finite differences with >> < uniform grid spacing h: >> < u_xx = (u_{i+1} - 2u_{i} + u_{i-1})/(h^2) >> < We then demonstrate time evolution using the various TS methods by >> --- >> > This program solves the one-dimensional Schroedinger equation in a >> > tightly bound lattice with a periodic time dependence >> > \partial_t u_m(t) = \sum_m J_{lm}(t) u_m(t), >> > on the domain 0 <= m < L, with the boundary conditions >> > u_m(0) = 0 for all m except m=[L/2], where u_m(0) = 1 >> > This is a set of linear, first-order ODEs >> > The time evolution using the various TS methods can be run by >> 34,46d41 >> < >> < We compare the approximate solution with the exact solution, given by >> < u_exact(x,t) = exp(-36*pi*pi*t) * sin(6*pi*x) + >> < 3*exp(-4*pi*pi*t) * sin(2*pi*x) >> < >> < Notes: >> < This code demonstrates the TS solver interface to two variants of >> < linear problems, u_t = f(u,t), namely >> < - time-dependent f: f(u,t) is a function of t >> < - time-independent f: f(u,t) is simply f(u) >> < >> < The uniprocessor version of this code is ts/examples/tutorials/ex3.c >> < >> 59a55 >> > >> 64d59 >> < >> 75d69 >> < Vec solution; /* global exact solution vector */ >> 77d70 >> < PetscReal h; /* mesh width h = 1/(m-1) */ >> 79,80c72,77 >> < PetscViewer viewer1, viewer2; /* viewers for the solution and error >> */ >> < PetscReal norm_2, norm_max; /* error norms */ >> --- >> > PetscViewer viewer1; /* viewer for the solution */ >> > PetscReal norm_2; /* wavefunction norm */ >> > Vec randoms; /* random numbers between -1 and 1 */ >> > PetscReal alpha; /* disorder strength */ >> > PetscReal h0; /* drive amplitude */ >> > PetscReal w; /* drive frequency */ >> 85a83 >> > extern PetscInt kdel (PetscInt, PetscInt); >> 87,88c85,86 >> < extern PetscErrorCode RHSMatrixHeat (TS, PetscReal, Vec, Mat, Mat, void >> *); >> < extern PetscErrorCode RHSFunctionHeat (TS, PetscReal, Vec, Vec, void *); >> --- >> > extern PetscErrorCode RHSMatrixSchrodinger (TS, PetscReal, Vec, Mat, >> Mat, >> > void *); >> 90d87 >> < extern PetscErrorCode ExactSolution (PetscReal, Vec, AppCtx *); >> 101,102c98,99 >> < PetscReal time_total_max = 1.0; /* default max total time */ >> < PetscInt time_steps_max = 100; /* default max timesteps */ >> --- >> > PetscReal time_total_max = TIME_TOTAL_MAX; /* default max total time >> */ >> > PetscInt time_steps_max = TIME_STEPS_MAX; /* default max timesteps */ >> 108,109c105,107 >> < PetscBool flg; >> < TSProblemType tsproblem = TS_LINEAR; >> --- >> > PetscReal alpha; /* Disorder strength */ >> > PetscReal h0; /* Drive amplitude */ >> > PetscReal w; /* drive frequency */ >> 119c117 >> < m = 60; >> --- >> > m = DEFAULT_SIZE; >> 121a120,132 >> > >> > h0 = DEFAULT_AMPL; >> > ierr = PetscOptionsGetReal (NULL, "-a", &h0, NULL); >> > CHKERRQ (ierr); >> > >> > alpha = DEFAULT_ALPHA; >> > ierr = PetscOptionsGetReal (NULL, "-d", &alpha, NULL); >> > CHKERRQ (ierr); >> > >> > w = DEFAULT_W; >> > ierr = PetscOptionsGetReal (NULL, "-w", &w, NULL); >> > CHKERRQ (ierr); >> > >> 123a135 >> > >> 125,127c137,141 >> < appctx.h = 1.0 / (m - 1.0); >> < appctx.norm_2 = 0.0; >> < appctx.norm_max = 0.0; >> --- >> > appctx.alpha = alpha; >> > appctx.h0 = h0; >> > appctx.w = w; >> > >> > appctx.norm_2 = 1.0; >> 163d176 >> < create global work vector for storing exact solution. >> 167,168d179 >> < ierr = VecDuplicate (u, &appctx.solution); >> < CHKERRQ (ierr); >> 171c182 >> < Set up displays to show graphs of the solution and error >> --- >> > Set up displays to show graphs of the solution >> 173,189c184,195 >> < >> < ierr = >> < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 380, 400, 160, >> < &appctx.viewer1); >> < CHKERRQ (ierr); >> < ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); >> < CHKERRQ (ierr); >> < ierr = PetscDrawSetDoubleBuffer (draw); >> < CHKERRQ (ierr); >> < ierr = >> < PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "", 80, 0, 400, 160, >> < &appctx.viewer2); >> < CHKERRQ (ierr); >> < ierr = PetscViewerDrawGetDraw (appctx.viewer2, 0, &draw); >> < CHKERRQ (ierr); >> < ierr = PetscDrawSetDoubleBuffer (draw); >> < CHKERRQ (ierr); >> --- >> > if (appctx.debug) >> > { >> > ierr = >> > PetscViewerDrawOpen (PETSC_COMM_WORLD, 0, "Probability Distribution", >> > PETSC_DECIDE, PETSC_DECIDE, PETSC_DRAW_HALF_SIZE, >> > PETSC_DRAW_HALF_SIZE, &appctx.viewer1); >> > CHKERRQ (ierr); >> > ierr = PetscViewerDrawGetDraw (appctx.viewer1, 0, &draw); >> > CHKERRQ (ierr); >> > ierr = PetscDrawSetDoubleBuffer (draw); >> > CHKERRQ (ierr); >> > } >> 197,202c203 >> < >> < flg = PETSC_FALSE; >> < ierr = PetscOptionsGetBool (NULL, "-nonlinear", &flg, NULL); >> < CHKERRQ (ierr); >> < ierr = TSSetProblemType (ts, flg ? TS_NONLINEAR : TS_LINEAR); >> < CHKERRQ (ierr); >> --- >> > TSSetProblemType (ts, TS_LINEAR); /* In this case, the dynamics is >> always linear */ >> 205c206 >> < Set optional user-defined monitoring routine >> --- >> > Set user-defined monitoring routine >> 211d211 >> < >> 224,225c224,257 >> < flg = PETSC_FALSE; >> < ierr = PetscOptionsGetBool (NULL, "-time_dependent_rhs", &flg, NULL); >> --- >> > /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> > Create random number context and generate random numbers >> > into application context >> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> - */ >> > PetscRandom rctx; /* random number generator context */ >> > PetscScalar randomscale = 2.0 * PETSC_i * alpha; >> > >> > ierr = PetscRandomCreate (PETSC_COMM_SELF, &rctx); >> > CHKERRQ (ierr); >> > ierr = PetscRandomSetFromOptions (rctx); >> > CHKERRQ (ierr); >> > ierr = PetscRandomSetInterval (rctx, -1.0, 1.0); >> > CHKERRQ (ierr); >> > ierr = VecCreate (PETSC_COMM_SELF, &appctx.randoms); >> > CHKERRQ (ierr); >> > ierr = VecSetSizes (appctx.randoms, PETSC_DECIDE, m); >> > CHKERRQ (ierr); >> > ierr = VecSetFromOptions (appctx.randoms); >> > CHKERRQ (ierr); >> > ierr = VecSetRandom (appctx.randoms, rctx); >> > CHKERRQ (ierr); >> > ierr = VecScale (appctx.randoms, randomscale); >> > CHKERRQ (ierr); >> > ierr = PetscRandomDestroy (&rctx); >> > CHKERRQ (ierr); >> > >> > /* >> > For linear problems with a time-dependent f(u,t) in the equation >> > u_t = f(u,t), the user provides the discretized right-hand-side >> > as a time-dependent matrix. >> > */ >> > ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, >> &appctx); >> > CHKERRQ (ierr); >> > ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixSchrodinger, &appctx); >> 227,254d258 >> < if (flg) >> < { >> < /* >> < For linear problems with a time-dependent f(u,t) in the >> equation >> < u_t = f(u,t), the user provides the discretized right-hand-side >> < as a time-dependent matrix. >> < */ >> < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, >> &appctx); >> < CHKERRQ (ierr); >> < ierr = TSSetRHSJacobian (ts, A, A, RHSMatrixHeat, &appctx); >> < CHKERRQ (ierr); >> < } >> < else >> < { >> < /* >> < For linear problems with a time-independent f(u) in the >> equation >> < u_t = f(u), the user provides the discretized right-hand-side >> < as a matrix only once, and then sets a null matrix evaluation >> < routine. >> < */ >> < ierr = RHSMatrixHeat (ts, 0.0, u, A, A, &appctx); >> < CHKERRQ (ierr); >> < ierr = TSSetRHSFunction (ts, NULL, TSComputeRHSFunctionLinear, >> &appctx); >> < CHKERRQ (ierr); >> < ierr = >> < TSSetRHSJacobian (ts, A, A, TSComputeRHSJacobianConstant, &appctx); >> < CHKERRQ (ierr); >> < } >> 256,266d259 >> < if (tsproblem == TS_NONLINEAR) >> < { >> < SNES snes; >> < ierr = TSSetRHSFunction (ts, NULL, RHSFunctionHeat, &appctx); >> < CHKERRQ (ierr); >> < ierr = TSGetSNES (ts, &snes); >> < CHKERRQ (ierr); >> < ierr = >> < SNESSetJacobian (snes, NULL, NULL, SNESComputeJacobianDefault, NULL); >> < CHKERRQ (ierr); >> < } >> 272,273c265,266 >> < dt = appctx.h * appctx.h / 2.0; >> < ierr = TSSetInitialTimeStep (ts, 0.0, dt); >> --- >> > dt = INIT_STEP; >> > ierr = TSSetInitialTimeStep (ts, INIT_TIME, dt); >> 281a275 >> > - Set default tolerances >> 287a282,285 >> > ierr = TSSetType (ts, TSRK); >> > CHKERRQ (ierr); >> > ierr = TSRKSetType (ts, TSRK5DP); //Default solver is Runge Kutta >> (5) Prince-Dormand (4) >> > CHKERRQ (ierr); >> 289a288,291 >> > >> > ierr = TSSetTolerances (ts, ABSERROR, NULL, RELERROR, NULL); >> > CHKERRQ (ierr); >> > >> 321,324c323,324 >> < PetscPrintf (PETSC_COMM_WORLD, >> < "Avg. error (2 norm) = %g Avg. error (max norm) = %g\n", >> < (double) (appctx.norm_2 / steps), >> < (double) (appctx.norm_max / steps)); >> --- >> > PetscPrintf (PETSC_COMM_WORLD, "Final. (2 norm) = %g \n", >> > (double) (appctx.norm_2)); >> 338,341c338,343 >> < ierr = PetscViewerDestroy (&appctx.viewer1); >> < CHKERRQ (ierr); >> < ierr = PetscViewerDestroy (&appctx.viewer2); >> < CHKERRQ (ierr); >> --- >> > >> > if (appctx.debug) >> > { >> > ierr = PetscViewerDestroy (&appctx.viewer1); >> > CHKERRQ (ierr); >> > } >> 344,345d345 >> < ierr = VecDestroy (&appctx.solution); >> < CHKERRQ (ierr); >> 347a348,349 >> > ierr = VecDestroy (&appctx.randoms); >> > CHKERRQ (ierr); >> 377c379 >> < PetscScalar *u_localptr, h = appctx->h; >> --- >> > PetscScalar *u_localptr; >> 379a382 >> > PetscScalar zero = 0.0, one = 1.0; >> 394,395d396 >> < - Note that the Fortran interface to VecGetArray() differs from the >> < C version. See the users manual for details. >> 402,403c403,404 >> < directly into the array locations. Alternatively, we could use >> < VecSetValues() or VecSetValuesLocal(). >> --- >> > directly into the array locations. >> > This will set the midpoint to unity >> 406,408c407,410 >> < u_localptr[i - mybase] = >> < PetscSinScalar (PETSC_PI * i * 6. * h) + >> < 3. * PetscSinScalar (PETSC_PI * i * 2. * h); >> --- >> > if (i - mybase == floor ((appctx->m) / 2)) >> > u_localptr[i - mybase] = one; >> > else >> > u_localptr[i - mybase] = zero; >> 421c423 >> < ierr = PetscPrintf (appctx->comm, "initial guess vector\n"); >> --- >> > ierr = PetscPrintf (appctx->comm, "initial vector\n"); >> 432,486d433 >> < #define __FUNCT__ "ExactSolution" >> < /* >> < ExactSolution - Computes the exact solution at a given time. >> < >> < Input Parameters: >> < t - current time >> < solution - vector in which exact solution will be computed >> < appctx - user-defined application context >> < >> < Output Parameter: >> < solution - vector with the newly computed exact solution >> < */ >> < PetscErrorCode >> < ExactSolution (PetscReal t, Vec solution, AppCtx * appctx) >> < { >> < PetscScalar *s_localptr, h = appctx->h, ex1, ex2, sc1, sc2; >> < PetscInt i, mybase, myend; >> < PetscErrorCode ierr; >> < >> < /* >> < Determine starting and ending points of each processor's >> < range of grid values >> < */ >> < ierr = VecGetOwnershipRange (solution, &mybase, &myend); >> < CHKERRQ (ierr); >> < >> < /* >> < Get a pointer to vector data. >> < */ >> < ierr = VecGetArray (solution, &s_localptr); >> < CHKERRQ (ierr); >> < >> < /* >> < Simply write the solution directly into the array locations. >> < Alternatively, we culd use VecSetValues() or VecSetValuesLocal(). >> < */ >> < ex1 = PetscExpReal (-36. * PETSC_PI * PETSC_PI * t); >> < ex2 = PetscExpReal (-4. * PETSC_PI * PETSC_PI * t); >> < sc1 = PETSC_PI * 6. * h; >> < sc2 = PETSC_PI * 2. * h; >> < for (i = mybase; i < myend; i++) >> < s_localptr[i - mybase] = >> < PetscSinScalar (sc1 * (PetscReal) i) * ex1 + >> < 3. * PetscSinScalar (sc2 * (PetscReal) i) * ex2; >> < >> < /* >> < Restore vector >> < */ >> < ierr = VecRestoreArray (solution, &s_localptr); >> < CHKERRQ (ierr); >> < return 0; >> < } >> < >> < /* >> --------------------------------------------------------------------- */ >> < #undef __FUNCT__ >> 490,491c437 >> < each timestep. This example plots the solution and computes the >> < error in two different norms. >> --- >> > each timestep. This example plots the solution and returns the norm >> 509,515c455 >> < PetscReal norm_2, norm_max; >> < >> < /* >> < View a graph of the current iterate >> < */ >> < ierr = VecView (u, appctx->viewer2); >> < CHKERRQ (ierr); >> --- >> > PetscReal norm_2; >> 518c458 >> < Compute the exact solution >> --- >> > Compute and store norm of wavefunction in norm_2 >> 520c460 >> < ierr = ExactSolution (time, appctx->solution, appctx); >> --- >> > ierr = VecNorm (u, NORM_2, &norm_2); >> 521a462 >> > appctx->norm_2 = norm_2; >> 523,536d463 >> < /* >> < Print debugging information if desired >> < */ >> < if (appctx->debug) >> < { >> < ierr = PetscPrintf (appctx->comm, "Computed solution vector\n"); >> < CHKERRQ (ierr); >> < ierr = VecView (u, PETSC_VIEWER_STDOUT_WORLD); >> < CHKERRQ (ierr); >> < ierr = PetscPrintf (appctx->comm, "Exact solution vector\n"); >> < CHKERRQ (ierr); >> < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); >> < CHKERRQ (ierr); >> < } >> 538,541c465,467 >> < /* >> < Compute the 2-norm and max-norm of the error >> < */ >> < ierr = VecAXPY (appctx->solution, -1.0, u); >> --- >> > /* Build the lattice sites */ >> > Vec uabs; >> > ierr = VecDuplicate (u, &uabs); >> 543c469,470 >> < ierr = VecNorm (appctx->solution, NORM_2, &norm_2); >> --- >> > >> > ierr = VecCopy (u, uabs); >> 545,546c472 >> < norm_2 = PetscSqrtReal (appctx->h) * norm_2; >> < ierr = VecNorm (appctx->solution, NORM_MAX, &norm_max); >> --- >> > ierr = VecAbs (uabs); >> 549,556c475 >> < /* >> < PetscPrintf() causes only the first processor in this >> < communicator to print the timestep information. >> < */ >> < ierr = >> < PetscPrintf (appctx->comm, >> < "Timestep %D: time = %g 2-norm error = %g max norm error = %g\n", >> < step, (double) time, (double) norm_2, (double) norm_max); >> --- >> > ierr = VecPointwiseMult (uabs, uabs, uabs); >> 558,559d476 >> < appctx->norm_2 += norm_2; >> < appctx->norm_max += norm_max; >> 561,565c478,483 >> < /* >> < View a graph of the error >> < */ >> < ierr = VecView (appctx->solution, appctx->viewer1); >> < CHKERRQ (ierr); >> --- >> > //if (!appctx->debug) >> > //{ >> > /* Create another viewer in application context and dump full output >> to it */ >> > //ierr = VecView (u, appctx->viewer2); >> > //CHKERRQ (ierr); >> > //} >> 572,574c490,497 >> < ierr = PetscPrintf (appctx->comm, "Error vector\n"); >> < CHKERRQ (ierr); >> < ierr = VecView (appctx->solution, PETSC_VIEWER_STDOUT_WORLD); >> --- >> > /* >> > View a graph of the current iterate >> > */ >> > ierr = >> > PetscPrintf (PETSC_COMM_WORLD, "time = %lf\t norm = %lf\n", time, >> > norm_2); >> > >> > ierr = VecView (uabs, appctx->viewer1); >> 576a500,501 >> > ierr = VecDestroy (&uabs); >> > CHKERRQ (ierr); >> 583c508 >> < #define __FUNCT__ "RHSMatrixHeat" >> --- >> > #define __FUNCT__ "RHSMatrixSchrodinger" >> 585c510 >> < RHSMatrixHeat - User-provided routine to compute the right-hand-side >> --- >> > RHSMatrixSchrodinger - User-provided routine to compute the >> right-hand-side >> 599,600c524 >> < Notes: >> < RHSMatrixHeat computes entries for the locally owned part of the >> system. >> --- >> > RHSMatrixSchrodinger computes entries for the locally owned part of >> the system. >> 613c537 >> < RHSMatrixHeat (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void *ctx) >> --- >> > RHSMatrixSchrodinger (TS ts, PetscReal t, Vec X, Mat AA, Mat BB, void >> *ctx) >> 618,619c542,554 >> < PetscInt i, mstart, mend, idx[3]; >> < PetscScalar v[3], stwo = -2. / (appctx->h * appctx->h), sone = -.5 * >> stwo; >> --- >> > PetscInt m = appctx->m; >> > PetscScalar drive; >> > PetscReal h0 = appctx->h0; >> > PetscReal w = appctx->w; >> > PetscScalar *randoms_loc; >> > >> > ierr = VecGetArray (appctx->randoms, &randoms_loc); >> > CHKERRQ (ierr); >> > /*Calculate drive amplitude at this time */ >> > >> > drive = w * t; >> > drive = PetscSinReal (drive); >> > drive = 2.0 * PETSC_i * h0 * drive; >> 623,625c558,566 >> < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> - */ >> < >> < ierr = MatGetOwnershipRange (A, &mstart, &mend); >> --- >> > The Jacobian Matrix in our case is blocked >> > - The values are A_{lm} = >> i\left(\delta_{lm+1}+\delta_{lm-1}\right)+2i \alpha h_m\delta_{lm} + 2i >> h(t) \delta_{lm} >> > - Thus the superdiagonal and subdiagonal elements are just i, and >> diagonal elements are random + drive >> > - The elements are set rowwise. >> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> - */ >> > PetscInt rowstart, rowend, rowcount; >> > PetscScalar edge[2], middle[3]; >> > PetscInt idxedge[2], idxmiddle[3]; >> > ierr = MatGetOwnershipRange (A, &rowstart, &rowend); >> 627,655c568 >> < >> < /* >> < Set matrix rows corresponding to boundary data >> < */ >> < >> < if (mstart == 0) >> < { /* first processor only */ >> < v[0] = 1.0; >> < ierr = MatSetValues (A, 1, &mstart, 1, &mstart, v, INSERT_VALUES); >> < CHKERRQ (ierr); >> < mstart++; >> < } >> < >> < if (mend == appctx->m) >> < { /* last processor only */ >> < mend--; >> < v[0] = 1.0; >> < ierr = MatSetValues (A, 1, &mend, 1, &mend, v, INSERT_VALUES); >> < CHKERRQ (ierr); >> < } >> < >> < /* >> < Set matrix rows corresponding to interior data. We construct the >> < matrix one row at a time. >> < */ >> < v[0] = sone; >> < v[1] = stwo; >> < v[2] = sone; >> < for (i = mstart; i < mend; i++) >> --- >> > for (rowcount = rowstart; rowcount < rowend; rowcount++) >> 657,661c570,602 >> < idx[0] = i - 1; >> < idx[1] = i; >> < idx[2] = i + 1; >> < ierr = MatSetValues (A, 1, &i, 3, idx, v, INSERT_VALUES); >> < CHKERRQ (ierr); >> --- >> > if (rowcount == 0) //If it is the first row of the whole matrix >> > { >> > edge[0] = drive + randoms_loc[0]; >> > edge[1] = PETSC_i; >> > idxedge[0] = 0; >> > idxedge[1] = 1; >> > ierr = >> > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); >> > CHKERRQ (ierr); >> > } >> > else if (rowcount == m - 1) //If it is the last row of the whole >> matrix >> > { >> > edge[0] = PETSC_i; >> > edge[1] = drive + randoms_loc[m - 1]; >> > idxedge[0] = m - 2; >> > idxedge[1] = m - 1; >> > ierr = >> > MatSetValues (A, 1, &rowcount, 2, idxedge, edge, INSERT_VALUES); >> > CHKERRQ (ierr); >> > } >> > else >> > { >> > middle[0] = PETSC_i; >> > middle[1] = drive + randoms_loc[rowcount]; >> > middle[2] = PETSC_i; >> > idxmiddle[0] = rowcount - 1; >> > idxmiddle[1] = rowcount; >> > idxmiddle[2] = rowcount + 1; >> > ierr = >> > MatSetValues (A, 1, &rowcount, 3, idxmiddle, middle, >> > INSERT_VALUES); >> > CHKERRQ (ierr); >> > } >> 683a625,626 >> > ierr = VecRestoreArray (appctx->randoms, &randoms_loc); >> > CHKERRQ (ierr); >> 689,691c632,634 >> < #define __FUNCT__ "RHSFunctionHeat" >> < PetscErrorCode >> < RHSFunctionHeat (TS ts, PetscReal t, Vec globalin, Vec globalout, void >> *ctx) >> --- >> > #define __FUNCT__ "kdel" >> > PetscInt >> > kdel (PetscInt i, PetscInt j) >> 693,704c636,639 >> < PetscErrorCode ierr; >> < Mat A; >> < >> < PetscFunctionBeginUser; >> < ierr = TSGetRHSJacobian (ts, &A, NULL, NULL, &ctx); >> < CHKERRQ (ierr); >> < ierr = RHSMatrixHeat (ts, t, globalin, A, NULL, ctx); >> < CHKERRQ (ierr); >> < /* ierr = MatView(A,PETSC_VIEWER_STDOUT_WORLD);CHKERRQ(ierr); */ >> < ierr = MatMult (A, globalin, globalout); >> < CHKERRQ (ierr); >> < PetscFunctionReturn (0); >> --- >> > PetscInt value = 0; >> > if (i == j) >> > value = 1; >> > return value; >> >> >> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> --- >> *Analabha Roy* >> C.S.I.R Senior Research Associate >> >> Saha Institute of Nuclear Physics >> Section 1, Block AF >> Bidhannagar, Calcutta 700064 >> India >> *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com >> *Webpage*: http://www.ph.utexas.edu/~daneel/ >> > > > > -- > --- > *Analabha Roy* > C.S.I.R Senior Research Associate > > Saha Institute of Nuclear Physics > Section 1, Block AF > Bidhannagar, Calcutta 700064 > India > *Emails*: daneel at physics.utexas.edu, hariseldon99 at gmail.com > *Webpage*: http://www.ph.utexas.edu/~daneel/ > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From sondes.marzouki at gmail.com Thu Oct 16 09:01:34 2014 From: sondes.marzouki at gmail.com (Marzouki Sondos) Date: Thu, 16 Oct 2014 15:01:34 +0100 Subject: [petsc-users] Petsc performance Message-ID: Hello, I still hesitate whether to use Petsc library or not. And to take my decision I have some questions. Please can you help me? Let s suppose that I have two matrixes having the same dimension, A0 and A1 but A0 is sparser than A1. Which means that A0 have less number of non zeroes. My question is: Is Petsc less efficient( time) for dense matrix?? Any suggestions or clarifications are welcome Thanks a lot, -- Sondos Marzouki, Science of Computer Engineer Tel: +216 27 66 32 24 Address: 8300, St-Patrick Street, Suite 300, Montreal, QC, H8N 2H1, CANADA Skype: sondes.marzouki *sondos at emworks.com * EMWorks Inc www.emworks.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 16 09:09:52 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 16 Oct 2014 09:09:52 -0500 Subject: [petsc-users] Petsc performance In-Reply-To: References: Message-ID: On Thu, Oct 16, 2014 at 9:01 AM, Marzouki Sondos wrote: > Hello, > > I still hesitate whether to use Petsc library or not. And to take my > decision I have some questions. Please can you help me? > > Let s suppose that I have two matrixes having the same dimension, A0 and > A1 but A0 is sparser than A1. Which means that A0 have less number of non > zeroes. > > My question is: Is Petsc less efficient( time) for dense matrix?? > You will have to be more specific. For example, in a sparse direct method, the pattern of fill could be different for matrices with different sparsity pattern, and thus from your description it is impossible to predict performance. Likewise, with an iterative method, the spectra could be very different, and thus it is again impossible to predict performance. It is fairly simple, using KSP ex10, to quickly experiment with solvers on you given matrices, and thus determine whether PETSc solvers will work for you. Thanks, Matt > > Any suggestions or clarifications are welcome > Thanks a lot, > -- > Sondos Marzouki, Science of Computer Engineer > > Tel: +216 27 66 32 24 > Address: 8300, St-Patrick Street, Suite 300, Montreal, QC, H8N 2H1, > CANADA > > Skype: sondes.marzouki > *sondos at emworks.com * > EMWorks Inc > www.emworks.com > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jychang48 at gmail.com Thu Oct 16 15:02:29 2014 From: jychang48 at gmail.com (Justin Chang) Date: Thu, 16 Oct 2014 15:02:29 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: Actually, I have some related issues: In a previous thread, I saw that to output as hdf5/mxf you basically need the following two runtime options: -dm_view hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append 1) For the code that I had shown earlier in this thread (test.c), it works serially if I used the SNES solver instead of the TaoSolver. However, when I run it on multiple processors I get the following error: HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 0: #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed major: Object atom minor: Unable to close file #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count major: Object atom minor: Unable to decrement reference count #002: H5F.c line 1838 in H5F_close(): can't close file major: File accessibilty minor: Unable to close file #003: H5F.c line 2000 in H5F_try_close(): problems closing file major: File accessibilty minor: Unable to close file #004: H5F.c line 1145 in H5F_dest(): low level truncate failed major: File accessibilty minor: Write failed #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed major: Virtual File Layer minor: Can't update object #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size failed major: Internal error (too specific to document in detail) minor: Some MPI function failed #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, error stack: MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine major: Internal error (too specific to document in detail) minor: MPI Error String HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 1: #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed major: Object atom minor: Unable to close file #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count major: Object atom minor: Unable to decrement reference count #002: H5F.c line 1838 in H5F_close(): can't close file major: File accessibilty minor: Unable to close file #003: H5F.c line 2000 in H5F_try_close(): problems closing file major: File accessibilty minor: Unable to close file #004: H5F.c line 1145 in H5F_dest(): low level truncate failed major: File accessibilty minor: Write failed #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed major: Virtual File Layer minor: Can't update object #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size failed major: Internal error (too specific to document in detail) minor: Some MPI function failed #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, error stack: MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine major: Internal error (too specific to document in detail) minor: MPI Error String I compared my code with what was in SNES ex12 and I don't seem to understand why I am getting this error. 2) Is there a way to output as HDF5/xmf via the Tao Solver? I tried using -ksp_view_solution hdf5:sol.h5::append but it gave me no solution. The multiple processor issue also applies to this Any help appreciated. Thanks On Tue, Oct 14, 2014 at 11:58 PM, Justin Chang wrote: > Hi Jason, > > Both of those algorithms resolved my problem. Thanks! > > Justin > > On Tue, Oct 14, 2014 at 10:12 PM, Jason Sarich > wrote: > >> Hi Justin, >> >> If the only constraints you have are bounds on the variables, then you >> are much better off using the TAO algorithms tron or blmvm. IPM builds its >> own KKT matrix for a ksp solve that is undoubtably much less efficient than >> what will be used in tron (blmvm avoids ksp solves all together). >> >> That begin said, it looks like there's a bug in IPM and I'll try to track >> it down, thanks for the report. >> >> Jason >> >> >> On Tue, Oct 14, 2014 at 9:34 PM, Justin Chang >> wrote: >> >>> Attached is the source code and the makefile used to compile/run the >>> code. >>> >>> The source code is basically a dumbed down version of SNES ex12 plus >>> necessary TAO routines. >>> >>> Thanks >>> >>> On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley >>> wrote: >>> >>>> On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> So I am writing a non-negative diffusion equation using DMPlex's FEM >>>>> and Tao's SetVariableBounds functions. My code works really perfectly when >>>>> I run it with one processor. However, once I use 2 or more processors, I >>>>> get this error: >>>>> >>>> >>>> It looks like the problem is in the TAO definition, but you can check >>>> by just solving your problem with, for instance, BJacobi-LU in parallel. >>>> >>>> >>>>> [0]PETSC ERROR: --------------------- Error Message >>>>> -------------------------------------------------------------- >>>>> [0]PETSC ERROR: Nonconforming object sizes >>>>> [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse >>>>> and vector to != ctx from size) >>>>> [1]PETSC ERROR: --------------------- Error Message >>>>> -------------------------------------------------------------- >>>>> [1]PETSC ERROR: Nonconforming object sizes >>>>> [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse >>>>> and vector to != ctx from size) >>>>> [1]PETSC ERROR: See >>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>>>> shooting. >>>>> [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>>> [1]PETSC ERROR: [0]PETSC ERROR: See >>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>>>> shooting. >>>>> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco >>>>> by justin Tue Oct 14 19:48:50 2014 >>>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco >>>>> by justin Tue Oct 14 19:48:50 2014 >>>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>>> -with-cmake=cmake >>>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>>> -with-cmake=cmake >>>>> [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in >>>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>>> #1 VecScatterBegin() line 1713 in >>>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>>> [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 >>>>> in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>>> [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >>>>> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>>> [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in >>>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>>> #3 MatMultTranspose() line 2242 in >>>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>>> [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>> [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>> [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>> [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>> [1]PETSC ERROR: #6 TaoSolve() line 190 in >>>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>>> #6 TaoSolve() line 190 in >>>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>>> [0]PETSC ERROR: #7 main() line 341 in >>>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>>> [1]PETSC ERROR: #7 main() line 341 in >>>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>>> [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message >>>>> -------send entire error message to petsc-maint at mcs.anl.gov---------- >>>>> ----------------End of Error Message -------send entire error message >>>>> to petsc-maint at mcs.anl.gov---------- >>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>>> [cli_1]: aborting job: >>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>>> [cli_0]: aborting job: >>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>>> >>>>> >>>>> I have no idea how or why I am getting this error. What does this >>>>> mean? >>>>> >>>> >>>> It looks like one dof is given to proc 0 which should live on proc >>>> 1. We have to look at the divisions >>>> made in the KKT solver. Can you send a small example? >>>> >>>> Matt >>>> >>>> >>>>> My code is essentially built off of SNES ex12.c. The Jacobian >>>>> matrix, residual vector, and solution vector were created using DMPlex and >>>>> the built-in FEM functions within. The Hessian matrix and gradient vector >>>>> were created by simple matmult() functions of the jacobian and residual. >>>>> The lower bounds vector was created by duplicating the solution vector >>>>> (initial guess set to zero). My FormFunctionGradient() is basically the >>>>> same thing as in the maros.c example. can give more information if needed. >>>>> >>>>> Thanks, >>>>> Justin >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 16 15:07:12 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 16 Oct 2014 15:07:12 -0500 Subject: [petsc-users] TaoSolve() not working for multiple processors In-Reply-To: References: Message-ID: On Thu, Oct 16, 2014 at 3:02 PM, Justin Chang wrote: > Actually, I have some related issues: > > In a previous thread, I saw that to output as hdf5/mxf you basically need > the following two runtime options: > > -dm_view hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append > Can you send the code and we will get it to output. Its easiest to have something to work from, and we have not tried this out yet. You can send it to petsc-maint at mcs.anl.gov and its private. Thanks, Matt > 1) For the code that I had shown earlier in this thread (test.c), it works > serially if I used the SNES solver instead of the TaoSolver. However, when > I run it on multiple processors I get the following error: > > HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 0: > #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed > major: Object atom > minor: Unable to close file > #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count > major: Object atom > minor: Unable to decrement reference count > #002: H5F.c line 1838 in H5F_close(): can't close file > major: File accessibilty > minor: Unable to close file > #003: H5F.c line 2000 in H5F_try_close(): problems closing file > major: File accessibilty > minor: Unable to close file > #004: H5F.c line 1145 in H5F_dest(): low level truncate failed > major: File accessibilty > minor: Write failed > #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed > major: Virtual File Layer > minor: Can't update object > #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size > failed > major: Internal error (too specific to document in detail) > minor: Some MPI function failed > #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, > error stack: > MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine > major: Internal error (too specific to document in detail) > minor: MPI Error String > HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 1: > #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed > major: Object atom > minor: Unable to close file > #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count > major: Object atom > minor: Unable to decrement reference count > #002: H5F.c line 1838 in H5F_close(): can't close file > major: File accessibilty > minor: Unable to close file > #003: H5F.c line 2000 in H5F_try_close(): problems closing file > major: File accessibilty > minor: Unable to close file > #004: H5F.c line 1145 in H5F_dest(): low level truncate failed > major: File accessibilty > minor: Write failed > #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed > major: Virtual File Layer > minor: Can't update object > #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size > failed > major: Internal error (too specific to document in detail) > minor: Some MPI function failed > #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, > error stack: > MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine > major: Internal error (too specific to document in detail) > minor: MPI Error String > > > I compared my code with what was in SNES ex12 and I don't seem to > understand why I am getting this error. > > 2) Is there a way to output as HDF5/xmf via the Tao Solver? I tried using > -ksp_view_solution hdf5:sol.h5::append but it gave me no solution. The > multiple processor issue also applies to this > > > Any help appreciated. Thanks > > On Tue, Oct 14, 2014 at 11:58 PM, Justin Chang > wrote: > >> Hi Jason, >> >> Both of those algorithms resolved my problem. Thanks! >> >> Justin >> >> On Tue, Oct 14, 2014 at 10:12 PM, Jason Sarich >> wrote: >> >>> Hi Justin, >>> >>> If the only constraints you have are bounds on the variables, then you >>> are much better off using the TAO algorithms tron or blmvm. IPM builds its >>> own KKT matrix for a ksp solve that is undoubtably much less efficient than >>> what will be used in tron (blmvm avoids ksp solves all together). >>> >>> That begin said, it looks like there's a bug in IPM and I'll try to >>> track it down, thanks for the report. >>> >>> Jason >>> >>> >>> On Tue, Oct 14, 2014 at 9:34 PM, Justin Chang >>> wrote: >>> >>>> Attached is the source code and the makefile used to compile/run the >>>> code. >>>> >>>> The source code is basically a dumbed down version of SNES ex12 plus >>>> necessary TAO routines. >>>> >>>> Thanks >>>> >>>> On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley >>>> wrote: >>>> >>>>> On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> So I am writing a non-negative diffusion equation using DMPlex's FEM >>>>>> and Tao's SetVariableBounds functions. My code works really perfectly when >>>>>> I run it with one processor. However, once I use 2 or more processors, I >>>>>> get this error: >>>>>> >>>>> >>>>> It looks like the problem is in the TAO definition, but you can >>>>> check by just solving your problem with, for instance, BJacobi-LU in >>>>> parallel. >>>>> >>>>> >>>>>> [0]PETSC ERROR: --------------------- Error Message >>>>>> -------------------------------------------------------------- >>>>>> [0]PETSC ERROR: Nonconforming object sizes >>>>>> [0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse >>>>>> and vector to != ctx from size) >>>>>> [1]PETSC ERROR: --------------------- Error Message >>>>>> -------------------------------------------------------------- >>>>>> [1]PETSC ERROR: Nonconforming object sizes >>>>>> [1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse >>>>>> and vector to != ctx from size) >>>>>> [1]PETSC ERROR: See >>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>>>>> shooting. >>>>>> [1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>>>> [1]PETSC ERROR: [0]PETSC ERROR: See >>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >>>>>> shooting. >>>>>> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 >>>>>> GIT Date: 2014-10-04 20:10:35 -0500 >>>>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco >>>>>> by justin Tue Oct 14 19:48:50 2014 >>>>>> [0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco >>>>>> by justin Tue Oct 14 19:48:50 2014 >>>>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>>>> -with-cmake=cmake >>>>>> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >>>>>> --with-fc=gfortran --download-fblaslapack --download-mpich >>>>>> --with-debugging=1 --download-metis --download-parmetis --download-triangle >>>>>> --with-cmake=cmake --download-ctetgen --download-superlu >>>>>> --download-scalapack --download-mumps --download-hdf5 --with-valgrind=1 >>>>>> -with-cmake=cmake >>>>>> [0]PETSC ERROR: #1 VecScatterBegin() line 1713 in >>>>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>>>> #1 VecScatterBegin() line 1713 in >>>>>> /home/justin/petsc-master/src/vec/vec/utils/vscat.c >>>>>> [1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line >>>>>> 1010 in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>>>> [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in >>>>>> /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c >>>>>> [1]PETSC ERROR: #3 MatMultTranspose() line 2242 in >>>>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>>>> #3 MatMultTranspose() line 2242 in >>>>>> /home/justin/petsc-master/src/mat/interface/matrix.c >>>>>> [0]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>>> [1]PETSC ERROR: #4 IPMComputeKKT() line 616 in >>>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>>> [1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>>> [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in >>>>>> /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c >>>>>> [1]PETSC ERROR: #6 TaoSolve() line 190 in >>>>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>>>> #6 TaoSolve() line 190 in >>>>>> /home/justin/petsc-master/src/tao/interface/taosolver.c >>>>>> [0]PETSC ERROR: #7 main() line 341 in >>>>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>>>> [1]PETSC ERROR: #7 main() line 341 in >>>>>> /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c >>>>>> [1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message >>>>>> -------send entire error message to petsc-maint at mcs.anl.gov---------- >>>>>> ----------------End of Error Message -------send entire error message >>>>>> to petsc-maint at mcs.anl.gov---------- >>>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>>>> [cli_1]: aborting job: >>>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1 >>>>>> [cli_0]: aborting job: >>>>>> application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0 >>>>>> >>>>>> >>>>>> I have no idea how or why I am getting this error. What does this >>>>>> mean? >>>>>> >>>>> >>>>> It looks like one dof is given to proc 0 which should live on proc >>>>> 1. We have to look at the divisions >>>>> made in the KKT solver. Can you send a small example? >>>>> >>>>> Matt >>>>> >>>>> >>>>>> My code is essentially built off of SNES ex12.c. The Jacobian >>>>>> matrix, residual vector, and solution vector were created using DMPlex and >>>>>> the built-in FEM functions within. The Hessian matrix and gradient vector >>>>>> were created by simple matmult() functions of the jacobian and residual. >>>>>> The lower bounds vector was created by duplicating the solution vector >>>>>> (initial guess set to zero). My FormFunctionGradient() is basically the >>>>>> same thing as in the maros.c example. can give more information if needed. >>>>>> >>>>>> Thanks, >>>>>> Justin >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>> >>>> >>> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvsang002 at gmail.com Sat Oct 18 06:58:12 2014 From: pvsang002 at gmail.com (Sang pham van) Date: Sat, 18 Oct 2014 07:58:12 -0400 Subject: [petsc-users] Petsc performance In-Reply-To: References: Message-ID: Dear everyone, I want to purchase a high performance workstation which should have from 32 to 64 nodes, price is not so important. The workstation will be used for cfd simulation purpose. Would any one help me to chose one? I would very much appreciate your suggestions. Thank you. Sang On Thu, Oct 16, 2014 at 10:01 AM, Marzouki Sondos wrote: > Hello, > > I still hesitate whether to use Petsc library or not. And to take my > decision I have some questions. Please can you help me? > > Let s suppose that I have two matrixes having the same dimension, A0 and > A1 but A0 is sparser than A1. Which means that A0 have less number of non > zeroes. > > My question is: Is Petsc less efficient( time) for dense matrix?? > > > Any suggestions or clarifications are welcome > Thanks a lot, > -- > Sondos Marzouki, Science of Computer Engineer > > Tel: +216 27 66 32 24 > Address: 8300, St-Patrick Street, Suite 300, Montreal, QC, H8N 2H1, > CANADA > > Skype: sondes.marzouki > *sondos at emworks.com * > EMWorks Inc > www.emworks.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pvsang002 at gmail.com Sat Oct 18 07:00:13 2014 From: pvsang002 at gmail.com (Sang pham van) Date: Sat, 18 Oct 2014 08:00:13 -0400 Subject: [petsc-users] What workstations (32-64 nodes) should I get for cfd simulation purpose? Message-ID: Dear everyone, I want to purchase a high performance workstation which should have from 32 to 64 nodes, price is not so important. The workstation will be used for cfd simulation purpose. Would any one help me to chose one? I would very much appreciate your suggestions. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sat Oct 18 11:03:35 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sat, 18 Oct 2014 11:03:35 -0500 Subject: [petsc-users] Petsc performance In-Reply-To: References: Message-ID: <02098E76-B98C-4AC3-B5EF-A7A98BFD4380@mcs.anl.gov> Sang, The most important single number by far is total ACHIEVABLE memory bandwidth http://en.wikipedia.org/wiki/Memory_bandwidth. The streams benchmark is a good way of measuring (we provide it in src/benchmarks/MPIVersion.c) the total achievable memory bandwidth when you have access to the machine. Of course when you are buying a machine you may not have access. The total PEAK memory bandwidth can be a large amount higher than the achievable but is a way to start looking at machines. Ideally the vendor would give you access to the machine in order to run streams before you purchase. Barry > On Oct 18, 2014, at 6:58 AM, Sang pham van wrote: > > Dear everyone, > > I want to purchase a high performance workstation which should have from 32 to 64 nodes, price is not so important. The workstation will be used for cfd simulation purpose. > Would any one help me to chose one? I would very much appreciate your suggestions. > > Thank you. > > Sang > > On Thu, Oct 16, 2014 at 10:01 AM, Marzouki Sondos wrote: > Hello, > > I still hesitate whether to use Petsc library or not. And to take my decision I have some questions. Please can you help me? > > Let s suppose that I have two matrixes having the same dimension, A0 and A1 but A0 is sparser than A1. Which means that A0 have less number of non zeroes. > > My question is: Is Petsc less efficient( time) for dense matrix?? > > > Any suggestions or clarifications are welcome > Thanks a lot, > -- > Sondos Marzouki, Science of Computer Engineer > > Tel: +216 27 66 32 24 > Address: 8300, St-Patrick Street, Suite 300, Montreal, QC, H8N 2H1, CANADA > Skype: sondes.marzouki > sondos at emworks.com > EMWorks Inc > www.emworks.com > > From salazardetroya at gmail.com Sun Oct 19 12:25:45 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Sun, 19 Oct 2014 12:25:45 -0500 Subject: [petsc-users] Adaptive controllers in TS Message-ID: Hi all I'm trying to find out what is the adaptive controller scheme used in the Runge-Kutta time stepping algorithm. Basically I want to know the function that determines the next time step. I see that the next time step is set with the function TSAdaptChoose(), which calls the pointer function (*choose)() within the structure _TSAdaptOps, but where is this set? I need to know this for an adjoint analysis to calculate gradients. I was wondering if there is an example of such analysis with the TS structure. I've seen that the RK source file includes a funciton to interpolate the solution, something that is necessary in an adjoint analysis, hence my question about the example. Thanks Miguel -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Sun Oct 19 13:41:14 2014 From: jed at jedbrown.org (Jed Brown) Date: Sun, 19 Oct 2014 13:41:14 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: References: Message-ID: <87wq7vrjg5.fsf@jedbrown.org> Miguel Angel Salazar de Troya writes: > Hi all > > I'm trying to find out what is the adaptive controller scheme used in the > Runge-Kutta time stepping algorithm. Basically I want to know the function > that determines the next time step. I see that the next time step is set > with the function TSAdaptChoose(), which calls the pointer function > (*choose)() within the structure _TSAdaptOps, but where is this set? via TSAdaptSetType. The default will be TSAdaptChoose_Basic (adaptbasic.c). You can find this information easily by running in a debugger. > I need to know this for an adjoint analysis to calculate gradients. I was > wondering if there is an example of such analysis with the TS structure. > I've seen that the RK source file includes a funciton to interpolate the > solution, something that is necessary in an adjoint analysis, hence my > question about the example. Interpolation arises for "continuous" adjoints. Be aware that if you want consistent gradients, you'll want a discrete adjoint, which integrates backward in time using the adjoint RK method. The stages of the adjoint RK method coincide with the forward method, so no interpolation is needed. Hong (Cc'd) is developing an example that can integrate discrete adjoint equations and we intend to eventually incorporate that functionality into the library. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From danyang.su at gmail.com Sun Oct 19 18:38:54 2014 From: danyang.su at gmail.com (Danyang Su) Date: Sun, 19 Oct 2014 16:38:54 -0700 Subject: [petsc-users] Check Array and String Bounds in PETSc Message-ID: <54444B8E.4020100@gmail.com> Hi All, How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? Thanks, Danyang From knepley at gmail.com Sun Oct 19 18:48:51 2014 From: knepley at gmail.com (Matthew Knepley) Date: Sun, 19 Oct 2014 18:48:51 -0500 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: <54444B8E.4020100@gmail.com> References: <54444B8E.4020100@gmail.com> Message-ID: On Sun, Oct 19, 2014 at 6:38 PM, Danyang Su wrote: > Hi All, > > How can I avoid "array bounds" error when using PETSc. I have to debug my > codes so I compiled the codes with option "Check Array and String Bounds". > But this results into array bounds error when I need to use ltog, as shown > below. > call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) > call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) > Exactly the aforementioned two lines work fine and the bug is in somewhere > else. But the debug process just stops at the codes when ltog is used. How > can avoid this? > Something else is wrong with the code. Neither of those routines does array lookup. Matt > Thanks, > > Danyang > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From salazardetroya at gmail.com Sun Oct 19 19:06:16 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Sun, 19 Oct 2014 19:06:16 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: <87wq7vrjg5.fsf@jedbrown.org> References: <87wq7vrjg5.fsf@jedbrown.org> Message-ID: Thanks for your response I'm struggling with this problem because the literature is not clear for me on how to calculate the discrete adjoint with adaptive time stepping algorithms. They cover the details when automatic differentiation tools are used. They mention that because the time step depend on the solution, it also depends on the parameter. Hence, there are terms that represent the derivative of the time step w.r.t. the parameters. What it is confusing is that they mention these terms must be removed. I don't understand this. I'm planning to hard-code the discrete adjoint problem (and use the TS if possible). I know this might not be the place to ask this. Maybe Hong can help me here. Miguel On Sun, Oct 19, 2014 at 1:41 PM, Jed Brown wrote: > Miguel Angel Salazar de Troya writes: > > > Hi all > > > > I'm trying to find out what is the adaptive controller scheme used in the > > Runge-Kutta time stepping algorithm. Basically I want to know the > function > > that determines the next time step. I see that the next time step is set > > with the function TSAdaptChoose(), which calls the pointer function > > (*choose)() within the structure _TSAdaptOps, but where is this set? > > via TSAdaptSetType. The default will be TSAdaptChoose_Basic > (adaptbasic.c). You can find this information easily by running in a > debugger. > > > I need to know this for an adjoint analysis to calculate gradients. I was > > wondering if there is an example of such analysis with the TS structure. > > I've seen that the RK source file includes a funciton to interpolate the > > solution, something that is necessary in an adjoint analysis, hence my > > question about the example. > > Interpolation arises for "continuous" adjoints. Be aware that if you > want consistent gradients, you'll want a discrete adjoint, which > integrates backward in time using the adjoint RK method. The stages of > the adjoint RK method coincide with the forward method, so no > interpolation is needed. > > Hong (Cc'd) is developing an example that can integrate discrete adjoint > equations and we intend to eventually incorporate that functionality > into the library. > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Sun Oct 19 19:06:25 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Sun, 19 Oct 2014 19:06:25 -0500 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: <54444B8E.4020100@gmail.com> References: <54444B8E.4020100@gmail.com> Message-ID: <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> Use ISLocalToGlobalMappingGetIndicesF90() > On Oct 19, 2014, at 6:38 PM, Danyang Su wrote: > > Hi All, > > How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. > call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) > call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) > Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? > > Thanks, > > Danyang From filippo.leonardi at sam.math.ethz.ch Mon Oct 20 03:31:15 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Mon, 20 Oct 2014 10:31:15 +0200 Subject: [petsc-users] Multiple solves with PCMG fail Message-ID: <9621705.dtimTkejkQ@besikovitch-ii> Hi, I have a very specific problem that I cannot figure out with PCMG and multiple solves. I got a linear system that I solve many times, with same matrix but different RHS. I can successfully solve the system with standard techniques, such as default solver or LU or PCGAMG. Even MG works if I destroy the ksp each time or I recompute the matrices at each time. But when I try and go to MG and not recomputing the matrices each time the solver fails. Any idea? Here some detail: the setup: ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); Then I solve as usual, for a large number of time steps: ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); ierr = KSPGetSolution(ksp, &phi); The solver converges and does that in a reasonable number of iterations: Linear solve converged due to CONVERGED_RTOL iterations 7 And ksp_view and ksp_monitor do not show any weird stuff. - Weirdly enough using any solver (for instance cg+bjacobi or gamg) everything works (So the matrix and RHS are working fine). - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is not a ComputeMatrix problem). - If I do: ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); between each solve or destroy the ksp entirely each time the solution is also perfect (So is not a boundary scaling or other stuff problem). - If I run MG with only 2 levels (so just coarse) I also get a fine result. Setup RHS is called each time as expected, setup matrix is called just once, also as expected. The only thing I can think is that MG does not update some value that actually needs to be recomputed. Any idea? The solution is not that different from: http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex29.c.html Best, Filippo From bsmith at mcs.anl.gov Mon Oct 20 07:29:04 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 07:29:04 -0500 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <9621705.dtimTkejkQ@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> Message-ID: Can you send us the code: to petsc-maint at mcs.anl.gov or petsc-users at mcs.anl.gov ? Or something that reproduces the problem? Barry > On Oct 20, 2014, at 3:31 AM, Filippo Leonardi wrote: > > Hi, > > I have a very specific problem that I cannot figure out with PCMG and multiple > solves. I got a linear system that I solve many times, with same matrix but > different RHS. I can successfully solve the system with standard techniques, > such as default solver or LU or PCGAMG. Even MG works if I destroy the ksp > each time or I recompute the matrices at each time. But when I try and go to > MG and not recomputing the matrices each time the solver fails. Any idea? > > Here some detail: the setup: > ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > Then I solve as usual, for a large number of time steps: > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > ierr = KSPGetSolution(ksp, &phi); > > The solver converges and does that in a reasonable number of iterations: > Linear solve converged due to CONVERGED_RTOL iterations 7 > And ksp_view and ksp_monitor do not show any weird stuff. > > - Weirdly enough using any solver (for instance cg+bjacobi or gamg) everything > works (So the matrix and RHS are working fine). > - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is not > a ComputeMatrix problem). > - If I do: > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); > between each solve or destroy the ksp entirely each time the solution is also > perfect (So is not a boundary scaling or other stuff problem). > - If I run MG with only 2 levels (so just coarse) I also get a fine result. > > Setup RHS is called each time as expected, setup matrix is called just once, > also as expected. > > The only thing I can think is that MG does not update some value that actually > needs to be recomputed. > > Any idea? > > The solution is not that different from: > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex29.c.html > > Best, > Filippo From danyang.su at gmail.com Mon Oct 20 11:56:39 2014 From: danyang.su at gmail.com (Danyang Su) Date: Mon, 20 Oct 2014 09:56:39 -0700 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> References: <54444B8E.4020100@gmail.com> <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> Message-ID: <54453EC7.4070206@gmail.com> On 19/10/2014 5:06 PM, Barry Smith wrote: > Use ISLocalToGlobalMappingGetIndicesF90() Hi Barry, Would you please give my a brief introduction or example on the parameters for this function? I didn't find it in the PETSc manual page. The following codes will cause stack overflow. Maybe the idx data type is wrong here. Would you please let me know what data type should be used here? PetscInt, pointer :: idx(:) call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) Thanks, Danyang > > >> On Oct 19, 2014, at 6:38 PM, Danyang Su wrote: >> >> Hi All, >> >> How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. >> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) >> Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? >> >> Thanks, >> >> Danyang From bsmith at mcs.anl.gov Mon Oct 20 13:00:10 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 13:00:10 -0500 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: <54453EC7.4070206@gmail.com> References: <54444B8E.4020100@gmail.com> <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> <54453EC7.4070206@gmail.com> Message-ID: > On Oct 20, 2014, at 11:56 AM, Danyang Su wrote: > > On 19/10/2014 5:06 PM, Barry Smith wrote: >> Use ISLocalToGlobalMappingGetIndicesF90() > Hi Barry, > > Would you please give my a brief introduction or example on the parameters for this function? I didn't find it in the PETSc manual page. > The following codes will cause stack overflow. Maybe the idx data type is wrong here. Would you please let me know what data type should be used here? > > PetscInt, pointer :: idx(:) > call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) > call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) Your code above looks ok. You?ll have to be more specific as to what goes wrong. Can you reproduce it in a small example that demonstrates the problem? Barry > > Thanks, > > Danyang >> >> >>> On Oct 19, 2014, at 6:38 PM, Danyang Su wrote: >>> >>> Hi All, >>> >>> How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. >>> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >>> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) >>> Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? >>> >>> Thanks, >>> >>> Danyang > From danyang.su at gmail.com Mon Oct 20 13:44:41 2014 From: danyang.su at gmail.com (Danyang Su) Date: Mon, 20 Oct 2014 11:44:41 -0700 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: References: <54444B8E.4020100@gmail.com> <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> <54453EC7.4070206@gmail.com> Message-ID: <54455819.5010105@gmail.com> On 20/10/2014 11:00 AM, Barry Smith wrote: >> On Oct 20, 2014, at 11:56 AM, Danyang Su wrote: >> >> On 19/10/2014 5:06 PM, Barry Smith wrote: >>> Use ISLocalToGlobalMappingGetIndicesF90() >> Hi Barry, >> >> Would you please give my a brief introduction or example on the parameters for this function? I didn't find it in the PETSc manual page. >> The following codes will cause stack overflow. Maybe the idx data type is wrong here. Would you please let me know what data type should be used here? >> >> PetscInt, pointer :: idx(:) >> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >> call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) > Your code above looks ok. You?ll have to be more specific as to what goes wrong. Can you reproduce it in a small example that demonstrates the problem? > > Barry It seems tricky. I tried different version of PETSc and different method, as shown below. When using PETSc 3.4.4 with the old functions, everything works fine. When using PETSc 3.5.2 method 1, everything works except that I cannot use "check array bounds" option, otherwise, the codes stop at "node_idx_lg2pg(ivol) = ...". This is not a big problem but sometimes I need to use this option to debug. When using PETSc 3.5.2 method 2, the codes stop at "ISLocalToGlobalMappingGetIndicesF90" with stack overflow. All the other codes and compiling options are the same. #ifdef PETSC_V3_5_X ISLocalToGlobalMapping :: ltogm !Method 1 !PetscInt :: ltog(1) !PetscOffset :: idltog !Method 2 PetscInt, pointer :: idx(:) #else PetscInt, pointer :: idx(:) #endif #ifdef PETSC_V3_5_X !Method 1 !call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) !call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) !Method 2 call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) #else call DMDAGetGlobalIndicesF90(dmda_flow%da,PETSC_NULL_INTEGER, & idx,ierr) #endif #ifdef PETSC_V3_5_X !Method 1 !node_idx_lg2pg(ivol) = (ltog(ivol*dof + idltog)+1)/dof !Method 2 node_idx_lg2pg(ivol) = (idx(ivol*dof)+1)/dof #else node_idx_lg2pg(ivol) = (idx(ivol*dof)+1)/dof #endif #ifdef PETSC_V3_5_X !Method 1 !call ISLocalToGlobalMappingRestoreIndices(ltogm,ltog,idltog,ierr) !Method 2 call ISLocalToGlobalMappingRestoreIndicesF90(ltogm,idx,ierr) #endif Thanks, Danyang > >> Thanks, >> >> Danyang >>> >>>> On Oct 19, 2014, at 6:38 PM, Danyang Su wrote: >>>> >>>> Hi All, >>>> >>>> How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. >>>> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >>>> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) >>>> Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? >>>> >>>> Thanks, >>>> >>>> Danyang From bsmith at mcs.anl.gov Mon Oct 20 14:31:02 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 14:31:02 -0500 Subject: [petsc-users] Check Array and String Bounds in PETSc In-Reply-To: <54455819.5010105@gmail.com> References: <54444B8E.4020100@gmail.com> <05D6AD79-D748-4B3E-B2DC-C0E1C4476096@mcs.anl.gov> <54453EC7.4070206@gmail.com> <54455819.5010105@gmail.com> Message-ID: <561204C6-065C-404E-A2CF-904546C04CF8@mcs.anl.gov> > On Oct 20, 2014, at 1:44 PM, Danyang Su wrote: > > On 20/10/2014 11:00 AM, Barry Smith wrote: >>> On Oct 20, 2014, at 11:56 AM, Danyang Su wrote: >>> >>> On 19/10/2014 5:06 PM, Barry Smith wrote: >>>> Use ISLocalToGlobalMappingGetIndicesF90() >>> Hi Barry, >>> >>> Would you please give my a brief introduction or example on the parameters for this function? I didn't find it in the PETSc manual page. >>> The following codes will cause stack overflow. Maybe the idx data type is wrong here. Would you please let me know what data type should be used here? >>> >>> PetscInt, pointer :: idx(:) >>> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >>> call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) >> Your code above looks ok. You?ll have to be more specific as to what goes wrong. Can you reproduce it in a small example that demonstrates the problem? >> >> Barry > It seems tricky. I tried different version of PETSc and different method, as shown below. > When using PETSc 3.4.4 with the old functions, everything works fine. > When using PETSc 3.5.2 method 1, everything works except that I cannot use "check array bounds" option, otherwise, the codes stop at "node_idx_lg2pg(ivol) = ...". This is not a big problem but sometimes I need to use this option to debug. > When using PETSc 3.5.2 method 2, the codes stop at "ISLocalToGlobalMappingGetIndicesF90" with stack overflow. What do you mean by ?stack overflow?? I have no idea what you mean. Please cut and paste the entire error message, runs in debugger etc Barryt > All the other codes and compiling options are the same. > > #ifdef PETSC_V3_5_X > ISLocalToGlobalMapping :: ltogm > > !Method 1 > !PetscInt :: ltog(1) > !PetscOffset :: idltog > > !Method 2 > PetscInt, pointer :: idx(:) > #else > PetscInt, pointer :: idx(:) > #endif > > #ifdef PETSC_V3_5_X > !Method 1 > !call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) > !call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) > > !Method 2 > call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) > call ISLocalToGlobalMappingGetIndicesF90(ltogm,idx,ierr) > #else > call DMDAGetGlobalIndicesF90(dmda_flow%da,PETSC_NULL_INTEGER, & > idx,ierr) > #endif > > #ifdef PETSC_V3_5_X > !Method 1 > !node_idx_lg2pg(ivol) = (ltog(ivol*dof + idltog)+1)/dof > !Method 2 > > node_idx_lg2pg(ivol) = (idx(ivol*dof)+1)/dof > #else > node_idx_lg2pg(ivol) = (idx(ivol*dof)+1)/dof > #endif > > #ifdef PETSC_V3_5_X > !Method 1 > !call ISLocalToGlobalMappingRestoreIndices(ltogm,ltog,idltog,ierr) > > !Method 2 > call ISLocalToGlobalMappingRestoreIndicesF90(ltogm,idx,ierr) > #endif > > Thanks, > > Danyang >> >>> Thanks, >>> >>> Danyang >>>> >>>>> On Oct 19, 2014, at 6:38 PM, Danyang Su wrote: >>>>> >>>>> Hi All, >>>>> >>>>> How can I avoid "array bounds" error when using PETSc. I have to debug my codes so I compiled the codes with option "Check Array and String Bounds". But this results into array bounds error when I need to use ltog, as shown below. >>>>> call DMGetLocalToGlobalMapping(dmda_flow%da,ltogm,ierr) >>>>> call ISLocalToGlobalMappingGetIndices(ltogm,ltog,idltog,ierr) >>>>> Exactly the aforementioned two lines work fine and the bug is in somewhere else. But the debug process just stops at the codes when ltog is used. How can avoid this? >>>>> >>>>> Thanks, >>>>> >>>>> Danyang From hsahasra at purdue.edu Mon Oct 20 15:07:29 2014 From: hsahasra at purdue.edu (Harshad Sahasrabudhe) Date: Mon, 20 Oct 2014 16:07:29 -0400 Subject: [petsc-users] Invalid MAT_CLASSID Message-ID: Hi, I get the following error when I call MatGetType even though I'm calling the function with a Mat object. [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Wrong type of object: Parameter # 1! What could be going wrong? Thanks, Harshad -------------- next part -------------- An HTML attachment was scrubbed... URL: From hsahasra at purdue.edu Mon Oct 20 15:09:31 2014 From: hsahasra at purdue.edu (Harshad Sahasrabudhe) Date: Mon, 20 Oct 2014 16:09:31 -0400 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: Sorry forgot to attach the complete error message: [0]PETSC ERROR: --------------------- Error Message ------------------------------------ [0]PETSC ERROR: Invalid argument! [0]PETSC ERROR: Wrong type of object: Parameter # 1! [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: Petsc Release Version 3.4.3, Oct, 15, 2013 [0]PETSC ERROR: See docs/changes/index.html for recent updates. [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting. [0]PETSC ERROR: See docs/index.html for manual pages. [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: /home/hsahasra/NEMO5/prototype/examples/Macroscopic/FEM/nemo on a linux named carter-fe02.rcac.purdue.edu by hsahasra Mon Oct 20 15:30:43 2014 [0]PETSC ERROR: Libraries linked from /home/hsahasra/NEMO5/libs/petsc/build-real/linux/lib [0]PETSC ERROR: Configure run at Mon Oct 20 11:41:06 2014 [0]PETSC ERROR: Configure options --with-cmake=/apps/rhel5/cmake-2.8.0/bin/cmake --with-x=0 --with-hdf5 --with-hdf5-dir=/apps/rhel6/petsc-3.4.2/impi-4.1.0.030_intel-13.1.1.163/externalpackages/hdf5-1.8.10-patch1/src/ --with-scalar-type=real --with-single-library=0 --with-pic=1 --with-shared-libraries=0 --with-clanguage=C++ --with-fortran --with-debugging=1 --with-cc=/apps/rhel6/intel/impi/4.1.0.030/bin64/mpiicc --with-fc=/apps/rhel6/intel/impi/4.1.0.030/bin64/mpiifort --with-cxx=/apps/rhel6/intel/impi/4.1.0.030/bin64/mpiicpc COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 --download-parmetis=1 --download-metis=1 --download-superlu --download-superlu_dist=1 --download-mumps=1 --download-scalapack=1 --download-blacs=1 [0]PETSC ERROR: ------------------------------------------------------------------------ [0]PETSC ERROR: MatGetType() line 100 in /home/hsahasra/NEMO5/libs/petsc/build-real/src/mat/interface/matreg.c On Mon, Oct 20, 2014 at 4:07 PM, Harshad Sahasrabudhe wrote: > Hi, > > I get the following error when I call MatGetType even though I'm calling > the function with a Mat object. > > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Wrong type of object: Parameter # 1! > > What could be going wrong? > > Thanks, > Harshad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 20 15:12:32 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 20 Oct 2014 15:12:32 -0500 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe wrote: > Hi, > > I get the following error when I call MatGetType even though I'm calling > the function with a Mat object. > > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Wrong type of object: Parameter # 1! > > What could be going wrong? > You are not calling it with a valid Mat object. Did you call MatCreate? Matt > Thanks, > Harshad > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From hsahasra at purdue.edu Mon Oct 20 15:17:01 2014 From: hsahasra at purdue.edu (Harshad Sahasrabudhe) Date: Mon, 20 Oct 2014 16:17:01 -0400 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: I get the Mat object from libMesh::PetscMatrix<>::mat() , and it's been created and assembled before I call MatGetType. On Mon, Oct 20, 2014 at 4:12 PM, Matthew Knepley wrote: > On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe > wrote: > >> Hi, >> >> I get the following error when I call MatGetType even though I'm calling >> the function with a Mat object. >> >> [0]PETSC ERROR: Invalid argument! >> [0]PETSC ERROR: Wrong type of object: Parameter # 1! >> >> What could be going wrong? >> > > You are not calling it with a valid Mat object. Did you call MatCreate? > > Matt > > >> Thanks, >> Harshad >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 20 15:19:48 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 20 Oct 2014 15:19:48 -0500 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: On Mon, Oct 20, 2014 at 3:17 PM, Harshad Sahasrabudhe wrote: > I get the Mat object from libMesh::PetscMatrix<>::mat() > , > and it's been created and assembled before I call MatGetType. > But what you are getting is invalid, so either a) it has not been created and assembled in LibMesh b) the right thing is not getting passed back (LibMesh problem) or c) you are overwriting memory and it manifests here (check with valgrind) It is very easy to look in these structures using the debugger. We are just checking an integer in the header for the object that identifies it as a Mat. Matt > On Mon, Oct 20, 2014 at 4:12 PM, Matthew Knepley > wrote: > >> On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe < >> hsahasra at purdue.edu> wrote: >> >>> Hi, >>> >>> I get the following error when I call MatGetType even though I'm calling >>> the function with a Mat object. >>> >>> [0]PETSC ERROR: Invalid argument! >>> [0]PETSC ERROR: Wrong type of object: Parameter # 1! >>> >>> What could be going wrong? >>> >> >> You are not calling it with a valid Mat object. Did you call MatCreate? >> >> Matt >> >> >>> Thanks, >>> Harshad >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From quecat001 at gmail.com Mon Oct 20 17:20:06 2014 From: quecat001 at gmail.com (Que Cat) Date: Mon, 20 Oct 2014 17:20:06 -0500 Subject: [petsc-users] Nonlinear solver for non-smooth jacobian Message-ID: Hello Petsc-users, I have to solve a problem in which the jacobian is non-smooth ( incoporates the heaviside step function behavior from the derivative of F(x) ). Could you recommend any type of nonlinear solver in Petsc which handles this issue ? Thank you. Que -------------- next part -------------- An HTML attachment was scrubbed... URL: From hsahasra at purdue.edu Mon Oct 20 17:21:06 2014 From: hsahasra at purdue.edu (Harshad Sahasrabudhe) Date: Mon, 20 Oct 2014 18:21:06 -0400 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: Hi Matt, If I have 2 instances of PETSc, one which is initialized by LibMesh and the other which is initialized in my code, then would getting a Mat from one version to the other cause this issue? I know that this is not the right way to do it, I'm just trying to zero in on the problem. a) is not the problem, since I create and assemble the matrix using LibMesh API. b) is not it either, I ran a simple example and it works in that case. For c), I ran my code with Valgrind and didn't get any invalid write errors. Thanks, Harshad On Mon, Oct 20, 2014 at 4:19 PM, Matthew Knepley wrote: > On Mon, Oct 20, 2014 at 3:17 PM, Harshad Sahasrabudhe > wrote: > >> I get the Mat object from libMesh::PetscMatrix<>::mat() >> , >> and it's been created and assembled before I call MatGetType. >> > > But what you are getting is invalid, so either > > a) it has not been created and assembled in LibMesh > > b) the right thing is not getting passed back (LibMesh problem) > > or > > c) you are overwriting memory and it manifests here (check with valgrind) > > It is very easy to look in these structures using the debugger. We are > just checking an integer > in the header for the object that identifies it as a Mat. > > Matt > > >> On Mon, Oct 20, 2014 at 4:12 PM, Matthew Knepley >> wrote: >> >>> On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe < >>> hsahasra at purdue.edu> wrote: >>> >>>> Hi, >>>> >>>> I get the following error when I call MatGetType even though I'm >>>> calling the function with a Mat object. >>>> >>>> [0]PETSC ERROR: Invalid argument! >>>> [0]PETSC ERROR: Wrong type of object: Parameter # 1! >>>> >>>> What could be going wrong? >>>> >>> >>> You are not calling it with a valid Mat object. Did you call MatCreate? >>> >>> Matt >>> >>> >>>> Thanks, >>>> Harshad >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 20 17:29:23 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 20 Oct 2014 17:29:23 -0500 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: On Mon, Oct 20, 2014 at 5:21 PM, Harshad Sahasrabudhe wrote: > Hi Matt, > > If I have 2 instances of PETSc, one which is initialized by LibMesh and > the other which is initialized in my code, then would getting a Mat from > one version to the other cause this issue? I know that this is not the > right way to do it, I'm just trying to zero in on the problem. > No, this is a simple check that is common to any PETSc. Matt > a) is not the problem, since I create and assemble the matrix using > LibMesh API. b) is not it either, I ran a simple example and it works in > that case. For c), I ran my code with Valgrind and didn't get any invalid > write errors. > > Thanks, > Harshad > > On Mon, Oct 20, 2014 at 4:19 PM, Matthew Knepley > wrote: > >> On Mon, Oct 20, 2014 at 3:17 PM, Harshad Sahasrabudhe < >> hsahasra at purdue.edu> wrote: >> >>> I get the Mat object from libMesh::PetscMatrix<>::mat() >>> , >>> and it's been created and assembled before I call MatGetType. >>> >> >> But what you are getting is invalid, so either >> >> a) it has not been created and assembled in LibMesh >> >> b) the right thing is not getting passed back (LibMesh problem) >> >> or >> >> c) you are overwriting memory and it manifests here (check with >> valgrind) >> >> It is very easy to look in these structures using the debugger. We are >> just checking an integer >> in the header for the object that identifies it as a Mat. >> >> Matt >> >> >>> On Mon, Oct 20, 2014 at 4:12 PM, Matthew Knepley >>> wrote: >>> >>>> On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe < >>>> hsahasra at purdue.edu> wrote: >>>> >>>>> Hi, >>>>> >>>>> I get the following error when I call MatGetType even though I'm >>>>> calling the function with a Mat object. >>>>> >>>>> [0]PETSC ERROR: Invalid argument! >>>>> [0]PETSC ERROR: Wrong type of object: Parameter # 1! >>>>> >>>>> What could be going wrong? >>>>> >>>> >>>> You are not calling it with a valid Mat object. Did you call MatCreate? >>>> >>>> Matt >>>> >>>> >>>>> Thanks, >>>>> Harshad >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Oct 20 18:51:08 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 18:51:08 -0500 Subject: [petsc-users] Invalid MAT_CLASSID In-Reply-To: References: Message-ID: <686DD270-3FFE-4D9B-A291-FB488268529D@mcs.anl.gov> ./programname options -on_error_attach_debugger then when it stops look at the object and see what it is and why it isn?t a Mat This is trivial and not worth a bunch of emails back and forth; you need to know the basics of using a debugger. Barry > On Oct 20, 2014, at 5:21 PM, Harshad Sahasrabudhe wrote: > > Hi Matt, > > If I have 2 instances of PETSc, one which is initialized by LibMesh and the other which is initialized in my code, then would getting a Mat from one version to the other cause this issue? I know that this is not the right way to do it, I'm just trying to zero in on the problem. > > a) is not the problem, since I create and assemble the matrix using LibMesh API. b) is not it either, I ran a simple example and it works in that case. For c), I ran my code with Valgrind and didn't get any invalid write errors. > > Thanks, > Harshad > > On Mon, Oct 20, 2014 at 4:19 PM, Matthew Knepley wrote: > On Mon, Oct 20, 2014 at 3:17 PM, Harshad Sahasrabudhe wrote: > I get the Mat object from libMesh::PetscMatrix<>::mat(), and it's been created and assembled before I call MatGetType. > > But what you are getting is invalid, so either > > a) it has not been created and assembled in LibMesh > > b) the right thing is not getting passed back (LibMesh problem) > > or > > c) you are overwriting memory and it manifests here (check with valgrind) > > It is very easy to look in these structures using the debugger. We are just checking an integer > in the header for the object that identifies it as a Mat. > > Matt > > On Mon, Oct 20, 2014 at 4:12 PM, Matthew Knepley wrote: > On Mon, Oct 20, 2014 at 3:07 PM, Harshad Sahasrabudhe wrote: > Hi, > > I get the following error when I call MatGetType even though I'm calling the function with a Mat object. > > [0]PETSC ERROR: Invalid argument! > [0]PETSC ERROR: Wrong type of object: Parameter # 1! > > What could be going wrong? > > You are not calling it with a valid Mat object. Did you call MatCreate? > > Matt > > Thanks, > Harshad > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > From jed at jedbrown.org Mon Oct 20 20:38:51 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 20 Oct 2014 20:38:51 -0500 Subject: [petsc-users] Nonlinear solver for non-smooth jacobian In-Reply-To: References: Message-ID: <87zjcqmcb8.fsf@jedbrown.org> Que Cat writes: > Hello Petsc-users, > > I have to solve a problem in which the jacobian is non-smooth ( incoporates > the heaviside step function behavior from the derivative of F(x) ). Could > you recommend any type of nonlinear solver in Petsc which handles this > issue ? You can reformulate as a variational inequality and use SNESVI. Be warned that it's not for the faint of heart and it's less mature than the "smooth" solvers. If this nonsmoothness is absolutely fundamental to your physics, like contact or condensation, then you should expect to invest significant effort in finding the best formulation. If the nonsmoothness is not essential, you should put that effort into reformulating to avoid or lessen the nonsmoothness. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jychang48 at gmail.com Mon Oct 20 21:48:11 2014 From: jychang48 at gmail.com (Justin Chang) Date: Mon, 20 Oct 2014 21:48:11 -0500 Subject: [petsc-users] Error configuring with Ctetgen Message-ID: Hi all, On my MacOSx I am trying to install the developer's version of petsc with the following options: ./configure --download-exodusii --download-fblaslapack --download-hdf5 --download-metis --download-mpich --download-mumps --download-netcdf --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake --download-ctetgen The error I get is with the --download-ctetgen option ******************************************************************************* UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details): ------------------------------------------------------------------------------- Error running make on Ctetgen: Could not execute "cd /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": make: *** No rule to make target `install-ctetgen'. Stop. ******************************************************************************* Do you guys know what is wrong? Attached are the make and log files. Thanks, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: application/octet-stream Size: 3375277 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: application/octet-stream Size: 94243 bytes Desc: not available URL: From balay at mcs.anl.gov Mon Oct 20 21:57:21 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 20 Oct 2014 21:57:21 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: Try: rm -rf arch-darwin-c-debug and rerun onfigure [basically you have old version of ctetgen - and configure does not have smarts to detect this or update to the appropriate version automatically. And its likely you have old versions of other packages aswell - so its best to do a clean build] BTW - you shouldn't need --download-fblaslapack on OSX [as it provides -llapack -lblas via vecLib] Satish On Mon, 20 Oct 2014, Justin Chang wrote: > Hi all, > > On my MacOSx I am trying to install the developer's version of petsc with > the following options: > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > --download-metis --download-mpich --download-mumps --download-netcdf > --download-parmetis --download-scalapack --download-triangle --with-cc=gcc > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > -with-cmake=cmake --download-ctetgen > > The error I get is with the --download-ctetgen option > > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > ------------------------------------------------------------------------------- > Error running make on Ctetgen: Could not execute "cd > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > make: *** No rule to make target `install-ctetgen'. Stop. > ******************************************************************************* > > Do you guys know what is wrong? Attached are the make and log files. > > Thanks, > Justin > From knepley at gmail.com Mon Oct 20 21:57:21 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 20 Oct 2014 21:57:21 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: On Mon, Oct 20, 2014 at 9:48 PM, Justin Chang wrote: > Hi all, > > On my MacOSx I am trying to install the developer's version of petsc with > the following options: > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > --download-metis --download-mpich --download-mumps --download-netcdf > --download-parmetis --download-scalapack --download-triangle --with-cc=gcc > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > -with-cmake=cmake --download-ctetgen > > The error I get is with the --download-ctetgen option > > > ******************************************************************************* > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > details): > > ------------------------------------------------------------------------------- > Error running make on Ctetgen: Could not execute "cd > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > make: *** No rule to make target `install-ctetgen'. Stop. > > ******************************************************************************* > > Do you guys know what is wrong? Attached are the make and log files. > We have been changing a lot of stuff in order to cleanup configure and make it more uniform. The build was changed to make all GNU make packages look similar. Thus, you need to kill your old copy of ctetgen, and get it to download again. rm -rf $PETSC_ARCH/externalpackages/ctetgen rm -rf $PETSC_ARCH/conf/ctetgen Thanks, Matt > Thanks, > Justin > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 20 22:02:30 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 20 Oct 2014 23:02:30 -0400 Subject: [petsc-users] KSP solver always has Zero pivot in LU Message-ID: Dear All, Last week I raised a question about KSP solver with zero pivot problem. Now I downsized the matrix in my problem and still got the zero pivot problem. I looked at the matrix and computed its eigenvalues in matlab, and it seems the matrix is good (not singular, I mean), but petsc still reported zero pivot in LU factorization. What could possibly cause this? Thank you very much in advance! PS: I attached the matrix output in my simplest test. -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: binaryoutput Type: application/octet-stream Size: 1306144 bytes Desc: not available URL: From bsmith at mcs.anl.gov Mon Oct 20 22:38:01 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 22:38:01 -0500 Subject: [petsc-users] KSP solver always has Zero pivot in LU In-Reply-To: References: Message-ID: <4F59A195-A322-46BD-8B8A-4BFB324AC06D@mcs.anl.gov> What exact options are you running with? I tried ILU and LU and all was fine. $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type lu -ksp_view Mat Object: 1 MPI processes type: seqaij row 0: (0, 1) (1, 0) (3, 0) row 1: (0, 0) (1, 1) (2, 0) (4, 0) row 2: (1, 0) (2, 1) (5, 0) row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, -0.304878) row 5: (2, 0) (4, 0) (5, 1) (8, 0) row 6: (3, 0) (6, 1) (7, 0) row 7: (4, 0) (6, 0) (7, 1) (8, 0) row 8: (5, 0) (7, 0) (8, 1) KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: nd factor fill ratio given 5, needed 1.54545 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=9, cols=9 package used to perform factorization: petsc total: nonzeros=51, allocated nonzeros=51 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=9, cols=9 total: nonzeros=33, allocated nonzeros=33 total number of mallocs used during MatSetValues calls =0 not using I-node routines Number of iterations = 1 Residual norm < 1.e-12 ~/Src/petsc/src/ksp/ksp/examples/tutorials (barry/fix-setters *) $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type ilu -ksp_view Mat Object: 1 MPI processes type: seqaij row 0: (0, 1) (1, 0) (3, 0) row 1: (0, 0) (1, 1) (2, 0) (4, 0) row 2: (1, 0) (2, 1) (5, 0) row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, -0.304878) row 5: (2, 0) (4, 0) (5, 1) (8, 0) row 6: (3, 0) (6, 1) (7, 0) row 7: (4, 0) (6, 0) (7, 1) (8, 0) row 8: (5, 0) (7, 0) (8, 1) KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=9, cols=9 package used to perform factorization: petsc total: nonzeros=33, allocated nonzeros=33 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=9, cols=9 total: nonzeros=33, allocated nonzeros=33 total number of mallocs used during MatSetValues calls =0 not using I-node routines Number of iterations = 2 Residual norm < 1.e-12 > On Oct 20, 2014, at 10:02 PM, Sharp Stone wrote: > > Dear All, > > Last week I raised a question about KSP solver with zero pivot problem. Now I downsized the matrix in my problem and still got the zero pivot problem. I looked at the matrix and computed its eigenvalues in matlab, and it seems the matrix is good (not singular, I mean), but petsc still reported zero pivot in LU factorization. What could possibly cause this? Thank you very much in advance! > > PS: I attached the matrix output in my simplest test. > > -- > Best regards, > > Feng > From jychang48 at gmail.com Mon Oct 20 22:37:55 2014 From: jychang48 at gmail.com (Justin Chang) Date: Mon, 20 Oct 2014 22:37:55 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: I did rm-rf arch-darwin-c-debug and ran config with the following options: ./configure --download-exodusii --download-hdf5 --download-metis --download-mpich --download-mumps --download-netcdf --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake --download-ctetgen And it compiles. However, once I run make all test I get the error "Attempting to use an MPI routine before initializing MPICH" when it attempts to run the test examples. Now I can't run any PETSc code whatsoever. Attached are the new make and configure log files. Thanks, Justin On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > Try: > > rm -rf arch-darwin-c-debug > and rerun onfigure > > [basically you have old version of ctetgen - and configure does not > have smarts to detect this or update to the appropriate version > automatically. And its likely you have old versions of other packages > aswell - so its best to do a clean build] > > BTW - you shouldn't need --download-fblaslapack on OSX [as it provides > -llapack -lblas via vecLib] > > Satish > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > Hi all, > > > > On my MacOSx I am trying to install the developer's version of petsc with > > the following options: > > > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > > --download-metis --download-mpich --download-mumps --download-netcdf > > --download-parmetis --download-scalapack --download-triangle > --with-cc=gcc > > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > > -with-cmake=cmake --download-ctetgen > > > > The error I get is with the --download-ctetgen option > > > > > ******************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > > details): > > > ------------------------------------------------------------------------------- > > Error running make on Ctetgen: Could not execute "cd > > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen > && > > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > > make: *** No rule to make target `install-ctetgen'. Stop. > > > ******************************************************************************* > > > > Do you guys know what is wrong? Attached are the make and log files. > > > > Thanks, > > Justin > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: application/octet-stream Size: 6846130 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log Type: application/octet-stream Size: 94242 bytes Desc: not available URL: From knepley at gmail.com Mon Oct 20 22:47:45 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 20 Oct 2014 22:47:45 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: On Mon, Oct 20, 2014 at 10:37 PM, Justin Chang wrote: > I did rm-rf arch-darwin-c-debug and ran config with the following options: > > ./configure --download-exodusii --download-hdf5 --download-metis > --download-mpich --download-mumps --download-netcdf --download-parmetis > --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ > --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake > --download-ctetgen > > And it compiles. However, once I run make all test I get the error > "Attempting to use an MPI routine before initializing MPICH" when it > attempts to run the test examples. Now I can't run any PETSc code > whatsoever. Attached are the new make and configure log files. > Sounds like you are using the wrong mpiexec. Matt > Thanks, > Justin > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > >> Try: >> >> rm -rf arch-darwin-c-debug >> and rerun onfigure >> >> [basically you have old version of ctetgen - and configure does not >> have smarts to detect this or update to the appropriate version >> automatically. And its likely you have old versions of other packages >> aswell - so its best to do a clean build] >> >> BTW - you shouldn't need --download-fblaslapack on OSX [as it provides >> -llapack -lblas via vecLib] >> >> Satish >> >> On Mon, 20 Oct 2014, Justin Chang wrote: >> >> > Hi all, >> > >> > On my MacOSx I am trying to install the developer's version of petsc >> with >> > the following options: >> > >> > ./configure --download-exodusii --download-fblaslapack --download-hdf5 >> > --download-metis --download-mpich --download-mumps --download-netcdf >> > --download-parmetis --download-scalapack --download-triangle >> --with-cc=gcc >> > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 >> > -with-cmake=cmake --download-ctetgen >> > >> > The error I get is with the --download-ctetgen option >> > >> > >> ******************************************************************************* >> > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log >> for >> > details): >> > >> ------------------------------------------------------------------------------- >> > Error running make on Ctetgen: Could not execute "cd >> > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen >> && >> > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": >> > make: *** No rule to make target `install-ctetgen'. Stop. >> > >> ******************************************************************************* >> > >> > Do you guys know what is wrong? Attached are the make and log files. >> > >> > Thanks, >> > Justin >> > >> >> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Oct 20 22:50:57 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 20 Oct 2014 22:50:57 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: <7CA627B9-96EC-4BCA-A2CD-F3B90E2EC6B3@mcs.anl.gov> cd src/vec/vec/examples/tutorials make ex1 gdb ./ex1 run if it stops type bt carriage return and send all the output > On Oct 20, 2014, at 10:37 PM, Justin Chang wrote: > > I did rm-rf arch-darwin-c-debug and ran config with the following options: > > ./configure --download-exodusii --download-hdf5 --download-metis --download-mpich --download-mumps --download-netcdf --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake --download-ctetgen > > And it compiles. However, once I run make all test I get the error "Attempting to use an MPI routine before initializing MPICH" when it attempts to run the test examples. Now I can't run any PETSc code whatsoever. Attached are the new make and configure log files. > > Thanks, > Justin > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > Try: > > rm -rf arch-darwin-c-debug > and rerun onfigure > > [basically you have old version of ctetgen - and configure does not > have smarts to detect this or update to the appropriate version > automatically. And its likely you have old versions of other packages > aswell - so its best to do a clean build] > > BTW - you shouldn't need --download-fblaslapack on OSX [as it provides > -llapack -lblas via vecLib] > > Satish > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > Hi all, > > > > On my MacOSx I am trying to install the developer's version of petsc with > > the following options: > > > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > > --download-metis --download-mpich --download-mumps --download-netcdf > > --download-parmetis --download-scalapack --download-triangle --with-cc=gcc > > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > > -with-cmake=cmake --download-ctetgen > > > > The error I get is with the --download-ctetgen option > > > > ******************************************************************************* > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for > > details): > > ------------------------------------------------------------------------------- > > Error running make on Ctetgen: Could not execute "cd > > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > > make: *** No rule to make target `install-ctetgen'. Stop. > > ******************************************************************************* > > > > Do you guys know what is wrong? Attached are the make and log files. > > > > Thanks, > > Justin > > > > > From balay at mcs.anl.gov Mon Oct 20 22:59:22 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 20 Oct 2014 22:59:22 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: Message-ID: 'make test' should use the correct mpiexec. >>>>> Trying to download http://www.mpich.org/static/downloads/3.1/mpich-3.1.tar.gz for MPICH #define PETSC_VERSION_DATE_GIT "2014-10-06 13:58:18 -0500" #define PETSC_VERSION_GIT "v3.5.2-529-g545cac9" #define PETSC_VERSION_BRANCH_GIT "master" <<<<< You apear to use a (couple of weeks old) master branch. [latest uses mpich-3.1.3]. But I don't see why that would cause such an error though... Satish On Mon, 20 Oct 2014, Matthew Knepley wrote: > On Mon, Oct 20, 2014 at 10:37 PM, Justin Chang wrote: > > > I did rm-rf arch-darwin-c-debug and ran config with the following options: > > > > ./configure --download-exodusii --download-hdf5 --download-metis > > --download-mpich --download-mumps --download-netcdf --download-parmetis > > --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ > > --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake > > --download-ctetgen > > > > And it compiles. However, once I run make all test I get the error > > "Attempting to use an MPI routine before initializing MPICH" when it > > attempts to run the test examples. Now I can't run any PETSc code > > whatsoever. Attached are the new make and configure log files. > > > > Sounds like you are using the wrong mpiexec. > > Matt > > > > Thanks, > > Justin > > > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > > > >> Try: > >> > >> rm -rf arch-darwin-c-debug > >> and rerun onfigure > >> > >> [basically you have old version of ctetgen - and configure does not > >> have smarts to detect this or update to the appropriate version > >> automatically. And its likely you have old versions of other packages > >> aswell - so its best to do a clean build] > >> > >> BTW - you shouldn't need --download-fblaslapack on OSX [as it provides > >> -llapack -lblas via vecLib] > >> > >> Satish > >> > >> On Mon, 20 Oct 2014, Justin Chang wrote: > >> > >> > Hi all, > >> > > >> > On my MacOSx I am trying to install the developer's version of petsc > >> with > >> > the following options: > >> > > >> > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > >> > --download-metis --download-mpich --download-mumps --download-netcdf > >> > --download-parmetis --download-scalapack --download-triangle > >> --with-cc=gcc > >> > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > >> > -with-cmake=cmake --download-ctetgen > >> > > >> > The error I get is with the --download-ctetgen option > >> > > >> > > >> ******************************************************************************* > >> > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > >> for > >> > details): > >> > > >> ------------------------------------------------------------------------------- > >> > Error running make on Ctetgen: Could not execute "cd > >> > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen > >> && > >> > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > >> > make: *** No rule to make target `install-ctetgen'. Stop. > >> > > >> ******************************************************************************* > >> > > >> > Do you guys know what is wrong? Attached are the make and log files. > >> > > >> > Thanks, > >> > Justin > >> > > >> > >> > > > > > From jychang48 at gmail.com Mon Oct 20 23:14:42 2014 From: jychang48 at gmail.com (Justin Chang) Date: Mon, 20 Oct 2014 23:14:42 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: <7CA627B9-96EC-4BCA-A2CD-F3B90E2EC6B3@mcs.anl.gov> References: <7CA627B9-96EC-4BCA-A2CD-F3B90E2EC6B3@mcs.anl.gov> Message-ID: Barry, I am not sure if this is what you're asking for (I am not too familiar with gdb) but here's what happened after following your instructions: GNU gdb (GDB) 7.8 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-apple-darwin13.4.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./ex1... warning: `/Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1.o': can't open to read symbols: No such file or directory. (no debugging symbols found)...done. (gdb) run Starting program: /Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1 Unable to find Mach task port for process-id 57136: (os/kern) failure (0x5). (please check gdb is codesigned - see taskgated(8)) (gdb) bt carriage No stack. (gdb) On Mon, Oct 20, 2014 at 10:50 PM, Barry Smith wrote: > > > cd src/vec/vec/examples/tutorials > make ex1 > gdb ./ex1 > run > > if it stops type bt carriage return and send all the output > > > > On Oct 20, 2014, at 10:37 PM, Justin Chang wrote: > > > > I did rm-rf arch-darwin-c-debug and ran config with the following > options: > > > > ./configure --download-exodusii --download-hdf5 --download-metis > --download-mpich --download-mumps --download-netcdf --download-parmetis > --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ > --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake > --download-ctetgen > > > > And it compiles. However, once I run make all test I get the error > "Attempting to use an MPI routine before initializing MPICH" when it > attempts to run the test examples. Now I can't run any PETSc code > whatsoever. Attached are the new make and configure log files. > > > > Thanks, > > Justin > > > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > > Try: > > > > rm -rf arch-darwin-c-debug > > and rerun onfigure > > > > [basically you have old version of ctetgen - and configure does not > > have smarts to detect this or update to the appropriate version > > automatically. And its likely you have old versions of other packages > > aswell - so its best to do a clean build] > > > > BTW - you shouldn't need --download-fblaslapack on OSX [as it provides > > -llapack -lblas via vecLib] > > > > Satish > > > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > > > Hi all, > > > > > > On my MacOSx I am trying to install the developer's version of petsc > with > > > the following options: > > > > > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > > > --download-metis --download-mpich --download-mumps --download-netcdf > > > --download-parmetis --download-scalapack --download-triangle > --with-cc=gcc > > > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > > > -with-cmake=cmake --download-ctetgen > > > > > > The error I get is with the --download-ctetgen option > > > > > > > ******************************************************************************* > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > for > > > details): > > > > ------------------------------------------------------------------------------- > > > Error running make on Ctetgen: Could not execute "cd > > > > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > > > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > > > make: *** No rule to make target `install-ctetgen'. Stop. > > > > ******************************************************************************* > > > > > > Do you guys know what is wrong? Attached are the make and log files. > > > > > > Thanks, > > > Justin > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Mon Oct 20 23:24:51 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Mon, 20 Oct 2014 23:24:51 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: <7CA627B9-96EC-4BCA-A2CD-F3B90E2EC6B3@mcs.anl.gov> Message-ID: Looks like this gdb is from homebrew - and has issues. Try using Perhaps you need to use lldb lldb ./ex1 http://stackoverflow.com/questions/18423124/please-check-gdb-is-codesigned-see-taskgated8-how-to-get-gdb-installed-w Satish On Mon, 20 Oct 2014, Justin Chang wrote: > Barry, I am not sure if this is what you're asking for (I am not too > familiar with gdb) but here's what happened after following your > instructions: > > GNU gdb (GDB) 7.8 > Copyright (C) 2014 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-apple-darwin13.4.0". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > . > Find the GDB manual and other documentation resources online at: > . > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from ./ex1... > warning: `/Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1.o': > can't open to read symbols: No such file or directory. > (no debugging symbols found)...done. > (gdb) run > Starting program: > /Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1 > Unable to find Mach task port for process-id 57136: (os/kern) failure (0x5). > (please check gdb is codesigned - see taskgated(8)) > (gdb) bt carriage > No stack. > (gdb) > > > On Mon, Oct 20, 2014 at 10:50 PM, Barry Smith wrote: > > > > > > > cd src/vec/vec/examples/tutorials > > make ex1 > > gdb ./ex1 > > run > > > > if it stops type bt carriage return and send all the output > > > > > > > On Oct 20, 2014, at 10:37 PM, Justin Chang wrote: > > > > > > I did rm-rf arch-darwin-c-debug and ran config with the following > > options: > > > > > > ./configure --download-exodusii --download-hdf5 --download-metis > > --download-mpich --download-mumps --download-netcdf --download-parmetis > > --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ > > --with-debugging=1 --with-fc=gfortran --with-valgrind=1 -with-cmake=cmake > > --download-ctetgen > > > > > > And it compiles. However, once I run make all test I get the error > > "Attempting to use an MPI routine before initializing MPICH" when it > > attempts to run the test examples. Now I can't run any PETSc code > > whatsoever. Attached are the new make and configure log files. > > > > > > Thanks, > > > Justin > > > > > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay wrote: > > > Try: > > > > > > rm -rf arch-darwin-c-debug > > > and rerun onfigure > > > > > > [basically you have old version of ctetgen - and configure does not > > > have smarts to detect this or update to the appropriate version > > > automatically. And its likely you have old versions of other packages > > > aswell - so its best to do a clean build] > > > > > > BTW - you shouldn't need --download-fblaslapack on OSX [as it provides > > > -llapack -lblas via vecLib] > > > > > > Satish > > > > > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > > > > > Hi all, > > > > > > > > On my MacOSx I am trying to install the developer's version of petsc > > with > > > > the following options: > > > > > > > > ./configure --download-exodusii --download-fblaslapack --download-hdf5 > > > > --download-metis --download-mpich --download-mumps --download-netcdf > > > > --download-parmetis --download-scalapack --download-triangle > > --with-cc=gcc > > > > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > > > > -with-cmake=cmake --download-ctetgen > > > > > > > > The error I get is with the --download-ctetgen option > > > > > > > > > > ******************************************************************************* > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log > > for > > > > details): > > > > > > ------------------------------------------------------------------------------- > > > > Error running make on Ctetgen: Could not execute "cd > > > > > > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > > > > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev install-ctetgen": > > > > make: *** No rule to make target `install-ctetgen'. Stop. > > > > > > ******************************************************************************* > > > > > > > > Do you guys know what is wrong? Attached are the make and log files. > > > > > > > > Thanks, > > > > Justin > > > > > > > > > > > > > > > > > > From jed at jedbrown.org Mon Oct 20 23:42:25 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 20 Oct 2014 23:42:25 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: References: <87wq7vrjg5.fsf@jedbrown.org> Message-ID: <87lhoam3ta.fsf@jedbrown.org> Miguel Angel Salazar de Troya writes: > Thanks for your response > > I'm struggling with this problem because the literature is not clear for me > on how to calculate the discrete adjoint with adaptive time stepping > algorithms. They cover the details when automatic differentiation tools are > used. They mention that because the time step depend on the solution, it > also depends on the parameter. Hence, there are terms that represent the > derivative of the time step w.r.t. the parameters. What it is confusing is > that they mention these terms must be removed. I don't understand this. I'm > planning to hard-code the discrete adjoint problem (and use the TS if > possible). Are they suggesting that the time step sizes for a given run should be frozen (at least locally) so that you have consistent gradients for a while? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jychang48 at gmail.com Tue Oct 21 00:18:01 2014 From: jychang48 at gmail.com (Justin Chang) Date: Tue, 21 Oct 2014 00:18:01 -0500 Subject: [petsc-users] Error configuring with Ctetgen In-Reply-To: References: <7CA627B9-96EC-4BCA-A2CD-F3B90E2EC6B3@mcs.anl.gov> Message-ID: I deleted my entire petsc developer directory, redownloaded it via git clone, and configured it with all the same options. That seemed to solve the problem for some reason. Weird On Mon, Oct 20, 2014 at 11:24 PM, Satish Balay wrote: > Looks like this gdb is from homebrew - and has issues. Try using > > Perhaps you need to use lldb > > lldb ./ex1 > > > http://stackoverflow.com/questions/18423124/please-check-gdb-is-codesigned-see-taskgated8-how-to-get-gdb-installed-w > > Satish > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > Barry, I am not sure if this is what you're asking for (I am not too > > familiar with gdb) but here's what happened after following your > > instructions: > > > > GNU gdb (GDB) 7.8 > > Copyright (C) 2014 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later < > http://gnu.org/licenses/gpl.html > > > > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > > and "show warranty" for details. > > This GDB was configured as "x86_64-apple-darwin13.4.0". > > Type "show configuration" for configuration details. > > For bug reporting instructions, please see: > > . > > Find the GDB manual and other documentation resources online at: > > . > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > Reading symbols from ./ex1... > > warning: > `/Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1.o': > > can't open to read symbols: No such file or directory. > > (no debugging symbols found)...done. > > (gdb) run > > Starting program: > > /Users/jychang48/petsc-dev/src/vec/vec/examples/tutorials/ex1 > > Unable to find Mach task port for process-id 57136: (os/kern) failure > (0x5). > > (please check gdb is codesigned - see taskgated(8)) > > (gdb) bt carriage > > No stack. > > (gdb) > > > > > > On Mon, Oct 20, 2014 at 10:50 PM, Barry Smith > wrote: > > > > > > > > > > > cd src/vec/vec/examples/tutorials > > > make ex1 > > > gdb ./ex1 > > > run > > > > > > if it stops type bt carriage return and send all the output > > > > > > > > > > On Oct 20, 2014, at 10:37 PM, Justin Chang > wrote: > > > > > > > > I did rm-rf arch-darwin-c-debug and ran config with the following > > > options: > > > > > > > > ./configure --download-exodusii --download-hdf5 --download-metis > > > --download-mpich --download-mumps --download-netcdf --download-parmetis > > > --download-scalapack --download-triangle --with-cc=gcc --with-cxx=g++ > > > --with-debugging=1 --with-fc=gfortran --with-valgrind=1 > -with-cmake=cmake > > > --download-ctetgen > > > > > > > > And it compiles. However, once I run make all test I get the error > > > "Attempting to use an MPI routine before initializing MPICH" when it > > > attempts to run the test examples. Now I can't run any PETSc code > > > whatsoever. Attached are the new make and configure log files. > > > > > > > > Thanks, > > > > Justin > > > > > > > > On Mon, Oct 20, 2014 at 9:57 PM, Satish Balay > wrote: > > > > Try: > > > > > > > > rm -rf arch-darwin-c-debug > > > > and rerun onfigure > > > > > > > > [basically you have old version of ctetgen - and configure does not > > > > have smarts to detect this or update to the appropriate version > > > > automatically. And its likely you have old versions of other > packages > > > > aswell - so its best to do a clean build] > > > > > > > > BTW - you shouldn't need --download-fblaslapack on OSX [as it > provides > > > > -llapack -lblas via vecLib] > > > > > > > > Satish > > > > > > > > On Mon, 20 Oct 2014, Justin Chang wrote: > > > > > > > > > Hi all, > > > > > > > > > > On my MacOSx I am trying to install the developer's version of > petsc > > > with > > > > > the following options: > > > > > > > > > > ./configure --download-exodusii --download-fblaslapack > --download-hdf5 > > > > > --download-metis --download-mpich --download-mumps > --download-netcdf > > > > > --download-parmetis --download-scalapack --download-triangle > > > --with-cc=gcc > > > > > --with-cxx=g++ --with-debugging=1 --with-fc=gfortran > --with-valgrind=1 > > > > > -with-cmake=cmake --download-ctetgen > > > > > > > > > > The error I get is with the --download-ctetgen option > > > > > > > > > > > > > > ******************************************************************************* > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS (see > configure.log > > > for > > > > > details): > > > > > > > > > ------------------------------------------------------------------------------- > > > > > Error running make on Ctetgen: Could not execute "cd > > > > > > > > > /Users/jychang48/petsc-dev/arch-darwin-c-debug/externalpackages/ctetgen && > > > > > /usr/bin/make PETSC_DIR=/Users/jychang48/petsc-dev > install-ctetgen": > > > > > make: *** No rule to make target `install-ctetgen'. Stop. > > > > > > > > > ******************************************************************************* > > > > > > > > > > Do you guys know what is wrong? Attached are the make and log > files. > > > > > > > > > > Thanks, > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From filippo.leonardi at sam.math.ethz.ch Tue Oct 21 01:50:58 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Tue, 21 Oct 2014 08:50:58 +0200 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> Message-ID: <10715057.mG3fKDZYPb@besikovitch-ii> Here, run this e.g. with mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin In Debug mode it crashes, in standard mode it fails. Using Petsc current. Run without mg and works! Hope it's not me being stupid and doing something terribly wrong! Best, Filippo On Monday 20 October 2014 07:29:04 Barry Smith wrote: > Can you send us the code: to petsc-maint at mcs.anl.gov or > petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > > Barry > > > On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > wrote: > > > > Hi, > > > > I have a very specific problem that I cannot figure out with PCMG and > > multiple solves. I got a linear system that I solve many times, with same > > matrix but different RHS. I can successfully solve the system with > > standard techniques, such as default solver or LU or PCGAMG. Even MG > > works if I destroy the ksp each time or I recompute the matrices at each > > time. But when I try and go to MG and not recomputing the matrices each > > time the solver fails. Any idea? > > > > Here some detail: the setup: > > ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > > ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > > > Then I solve as usual, for a large number of time steps: > > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > ierr = KSPGetSolution(ksp, &phi); > > > > The solver converges and does that in a reasonable number of iterations: > > Linear solve converged due to CONVERGED_RTOL iterations 7 > > And ksp_view and ksp_monitor do not show any weird stuff. > > > > - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > > everything works (So the matrix and RHS are working fine). > > - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is > > not a ComputeMatrix problem). > > - If I do: > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); > > between each solve or destroy the ksp entirely each time the solution is > > also perfect (So is not a boundary scaling or other stuff problem). > > - If I run MG with only 2 levels (so just coarse) I also get a fine > > result. > > > > Setup RHS is called each time as expected, setup matrix is called just > > once, also as expected. > > > > The only thing I can think is that MG does not update some value that > > actually needs to be recomputed. > > > > Any idea? > > > > The solution is not that different from: > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ > > ex29.c.html > > > > Best, > > Filippo -------------- next part -------------- A non-text attachment was scrubbed... Name: test1.cpp Type: text/x-c++src Size: 4959 bytes Desc: not available URL: From thronesf at gmail.com Tue Oct 21 15:10:17 2014 From: thronesf at gmail.com (Sharp Stone) Date: Tue, 21 Oct 2014 16:10:17 -0400 Subject: [petsc-users] KSP solver always has Zero pivot in LU In-Reply-To: <4F59A195-A322-46BD-8B8A-4BFB324AC06D@mcs.anl.gov> References: <4F59A195-A322-46BD-8B8A-4BFB324AC06D@mcs.anl.gov> Message-ID: Hi Barry, Thank you very much for your reply. They are illustrating. I use LU and gmres in the solver. When I use a rather small size of grids, the KSP solver works fine. However, when a larger domain is used (still small in the problem I'm trying to solve), I still got the error "Zero pivot in LU factorization" from Petsc. As you previously suggested, I solved the problem in Matlab, and got the solution, though the matrix is close to singular. Do I have any options to solve such a system with Petsc? For the completeness, I attached the matrix and rhs vector, and look forward to hearing from you. Thank you in advance! On Mon, Oct 20, 2014 at 11:38 PM, Barry Smith wrote: > > What exact options are you running with? I tried ILU and LU and all was > fine. > > $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type lu -ksp_view > Mat Object: 1 MPI processes > type: seqaij > row 0: (0, 1) (1, 0) (3, 0) > row 1: (0, 0) (1, 1) (2, 0) (4, 0) > row 2: (1, 0) (2, 1) (5, 0) > row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) > row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, > -0.304878) > row 5: (2, 0) (4, 0) (5, 1) (8, 0) > row 6: (3, 0) (6, 1) (7, 0) > row 7: (4, 0) (6, 0) (7, 1) (8, 0) > row 8: (5, 0) (7, 0) (8, 1) > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: lu > LU: out-of-place factorization > tolerance for zero pivot 2.22045e-14 > matrix ordering: nd > factor fill ratio given 5, needed 1.54545 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > package used to perform factorization: petsc > total: nonzeros=51, allocated nonzeros=51 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > Number of iterations = 1 > Residual norm < 1.e-12 > ~/Src/petsc/src/ksp/ksp/examples/tutorials (barry/fix-setters *) > $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type ilu -ksp_view > Mat Object: 1 MPI processes > type: seqaij > row 0: (0, 1) (1, 0) (3, 0) > row 1: (0, 0) (1, 1) (2, 0) (4, 0) > row 2: (1, 0) (2, 1) (5, 0) > row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) > row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, > -0.304878) > row 5: (2, 0) (4, 0) (5, 1) (8, 0) > row 6: (3, 0) (6, 1) (7, 0) > row 7: (4, 0) (6, 0) (7, 1) (8, 0) > row 8: (5, 0) (7, 0) (8, 1) > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: ilu > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > package used to perform factorization: petsc > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > Number of iterations = 2 > Residual norm < 1.e-12 > > > > On Oct 20, 2014, at 10:02 PM, Sharp Stone wrote: > > > > Dear All, > > > > Last week I raised a question about KSP solver with zero pivot problem. > Now I downsized the matrix in my problem and still got the zero pivot > problem. I looked at the matrix and computed its eigenvalues in matlab, and > it seems the matrix is good (not singular, I mean), but petsc still > reported zero pivot in LU factorization. What could possibly cause this? > Thank you very much in advance! > > > > PS: I attached the matrix output in my simplest test. > > > > -- > > Best regards, > > > > Feng > > > > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Matrix.dat Type: application/octet-stream Size: 5013424 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: RHSvec.dat Type: application/octet-stream Size: 167240 bytes Desc: not available URL: From knepley at gmail.com Tue Oct 21 17:04:12 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 21 Oct 2014 17:04:12 -0500 Subject: [petsc-users] KSP solver always has Zero pivot in LU In-Reply-To: References: <4F59A195-A322-46BD-8B8A-4BFB324AC06D@mcs.anl.gov> Message-ID: On Tue, Oct 21, 2014 at 3:10 PM, Sharp Stone wrote: > Hi Barry, > > Thank you very much for your reply. They are illustrating. > > I use LU and gmres in the solver. When I use a rather small size of grids, > the KSP solver works fine. However, when a larger domain is used (still > small in the problem I'm trying to solve), I still got the error "Zero > pivot in LU factorization" from Petsc. As you previously suggested, I > solved the problem in Matlab, and got the solution, though the matrix is > close to singular. Do I have any options to solve such a system with Petsc? > For the completeness, I attached the matrix and rhs vector, > You can possibly use an iterative method with a specified null space. Do you know what the null space is? Do you have an idea what preconditioner would be effective for your operator, since block box PCs tend not to work on singular problems? Matt > and look forward to hearing from you. Thank you in advance! > > On Mon, Oct 20, 2014 at 11:38 PM, Barry Smith wrote: > >> >> What exact options are you running with? I tried ILU and LU and all >> was fine. >> >> $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type lu -ksp_view >> Mat Object: 1 MPI processes >> type: seqaij >> row 0: (0, 1) (1, 0) (3, 0) >> row 1: (0, 0) (1, 1) (2, 0) (4, 0) >> row 2: (1, 0) (2, 1) (5, 0) >> row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) >> row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, >> -0.304878) >> row 5: (2, 0) (4, 0) (5, 1) (8, 0) >> row 6: (3, 0) (6, 1) (7, 0) >> row 7: (4, 0) (6, 0) (7, 1) (8, 0) >> row 8: (5, 0) (7, 0) (8, 1) >> KSP Object: 1 MPI processes >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using PRECONDITIONED norm type for convergence test >> PC Object: 1 MPI processes >> type: lu >> LU: out-of-place factorization >> tolerance for zero pivot 2.22045e-14 >> matrix ordering: nd >> factor fill ratio given 5, needed 1.54545 >> Factored matrix follows: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=9, cols=9 >> package used to perform factorization: petsc >> total: nonzeros=51, allocated nonzeros=51 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node routines >> linear system matrix = precond matrix: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=9, cols=9 >> total: nonzeros=33, allocated nonzeros=33 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node routines >> Number of iterations = 1 >> Residual norm < 1.e-12 >> ~/Src/petsc/src/ksp/ksp/examples/tutorials (barry/fix-setters *) >> $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type ilu >> -ksp_view >> Mat Object: 1 MPI processes >> type: seqaij >> row 0: (0, 1) (1, 0) (3, 0) >> row 1: (0, 0) (1, 1) (2, 0) (4, 0) >> row 2: (1, 0) (2, 1) (5, 0) >> row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) >> row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, >> -0.304878) >> row 5: (2, 0) (4, 0) (5, 1) (8, 0) >> row 6: (3, 0) (6, 1) (7, 0) >> row 7: (4, 0) (6, 0) (7, 1) (8, 0) >> row 8: (5, 0) (7, 0) (8, 1) >> KSP Object: 1 MPI processes >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using PRECONDITIONED norm type for convergence test >> PC Object: 1 MPI processes >> type: ilu >> ILU: out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >> matrix ordering: natural >> factor fill ratio given 1, needed 1 >> Factored matrix follows: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=9, cols=9 >> package used to perform factorization: petsc >> total: nonzeros=33, allocated nonzeros=33 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node routines >> linear system matrix = precond matrix: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=9, cols=9 >> total: nonzeros=33, allocated nonzeros=33 >> total number of mallocs used during MatSetValues calls =0 >> not using I-node routines >> Number of iterations = 2 >> Residual norm < 1.e-12 >> >> >> > On Oct 20, 2014, at 10:02 PM, Sharp Stone wrote: >> > >> > Dear All, >> > >> > Last week I raised a question about KSP solver with zero pivot problem. >> Now I downsized the matrix in my problem and still got the zero pivot >> problem. I looked at the matrix and computed its eigenvalues in matlab, and >> it seems the matrix is good (not singular, I mean), but petsc still >> reported zero pivot in LU factorization. What could possibly cause this? >> Thank you very much in advance! >> > >> > PS: I attached the matrix output in my simplest test. >> > >> > -- >> > Best regards, >> > >> > Feng >> > >> >> > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Tue Oct 21 17:29:03 2014 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 21 Oct 2014 18:29:03 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <10715057.mG3fKDZYPb@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> Message-ID: Humm, GAMG works fine form (like you said), but with MG (with or without Galerkin) on 1 or 4 procs I get: [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 Can you tell me exactly what you get with: ./test1 -pc_type mg I get: [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50433 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local And, it seems that the RHS is all zeros on one processor. I am using a debug build. Try to do the same. On Tue, Oct 21, 2014 at 2:50 AM, Filippo Leonardi < filippo.leonardi at sam.math.ethz.ch> wrote: > Here, > > run this e.g. with > mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > In Debug mode it crashes, in standard mode it fails. Using Petsc current. > > Run without mg and works! > > Hope it's not me being stupid and doing something terribly wrong! > > Best, > Filippo > > On Monday 20 October 2014 07:29:04 Barry Smith wrote: > > Can you send us the code: to petsc-maint at mcs.anl.gov or > > petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > > > > Barry > > > > > On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > > wrote: > > > > > > Hi, > > > > > > I have a very specific problem that I cannot figure out with PCMG and > > > multiple solves. I got a linear system that I solve many times, with > same > > > matrix but different RHS. I can successfully solve the system with > > > standard techniques, such as default solver or LU or PCGAMG. Even MG > > > works if I destroy the ksp each time or I recompute the matrices at > each > > > time. But when I try and go to MG and not recomputing the matrices each > > > time the solver fails. Any idea? > > > > > > Here some detail: the setup: > > > ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > > ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > > > ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > > > > > Then I solve as usual, for a large number of time steps: > > > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > ierr = KSPGetSolution(ksp, &phi); > > > > > > The solver converges and does that in a reasonable number of > iterations: > > > Linear solve converged due to CONVERGED_RTOL iterations 7 > > > And ksp_view and ksp_monitor do not show any weird stuff. > > > > > > - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > > > everything works (So the matrix and RHS are working fine). > > > - But the problem persists with Galerkin matrices (-pc_mg_galerkin) > (So is > > > not a ComputeMatrix problem). > > > - If I do: > > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); > > > between each solve or destroy the ksp entirely each time the solution > is > > > also perfect (So is not a boundary scaling or other stuff problem). > > > - If I run MG with only 2 levels (so just coarse) I also get a fine > > > result. > > > > > > Setup RHS is called each time as expected, setup matrix is called just > > > once, also as expected. > > > > > > The only thing I can think is that MG does not update some value that > > > actually needs to be recomputed. > > > > > > Any idea? > > > > > > The solution is not that different from: > > > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ > > > ex29.c.html > > > > > > Best, > > > Filippo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Tue Oct 21 17:53:05 2014 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 21 Oct 2014 18:53:05 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> Message-ID: I am getting this in the output: [0] VecNormalize(): Vector of zero norm can not be normalized; Returning only the zero norm [0] KSPGMRESCycle(): Converged due to zero residual norm on entry [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50632 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local and in gdb I see this (appended). The problem is that KSPComputeEigenvalues is called with NULL outputs. This because KSPChebyshevComputeExtremeEigenvalues_Private get the number of iterations and it is zero, it mallocs zero, and this is an error. Should there be a check like: 326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); if (!n) bail out 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); or is something going wrong upstream? Mark #5 0x000000010fb86057 in PetscAttachDebuggerErrorHandler (comm=1140850689, line=119, fun=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", num=85, p=PETSC_ERROR_INITIAL, mess=0x7fff50102290 "Null Pointer: Parameter # 3", ctx=0x0) at /Users/markadams/Codes/petsc/src/sys/error/adebug.c:462 #6 0x000000010fb88d30 in PetscError (comm=1140850689, line=119, func=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", n=85, p=PETSC_ERROR_INITIAL, mess=0x1115518ca "Null Pointer: Parameter # %d") at /Users/markadams/Codes/petsc/src/sys/error/err.c:378 #7 0x0000000110bd3464 in KSPComputeEigenvalues (ksp=0x7fe92d813e60, n=0, r=0x0, c=0x0, neig=0x7fff50102c14) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:119 #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 #9 0x0000000110b69043 in KSPSolve_Chebyshev (ksp=0x7fe92c869a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:373 #10 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c869a60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 #11 0x0000000110a8e4b8 in PCMGMCycle_Private (pc=0x7fe92c862e60, mglevelsin=0x7fe92c868a60, reason=0x0) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:19 #12 0x0000000110a9e1d5 in PCApply_MG (pc=0x7fe92c862e60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:337 #13 0x0000000110ad6f2c in PCApply (pc=0x7fe92c862e60, x=0x7fe92d002860, y=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/interface/precon.c:440 #14 0x0000000110affbd2 in KSP_PCApply (ksp=0x7fe92c833a60, x=0x7fe92d002860, y=0x7fe92d007c60) at kspimpl.h:230 #15 0x0000000110afcd31 in KSPSolve_CG (ksp=0x7fe92c833a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cg/cg.c:139 #16 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c833a60, b=0x0, x=0x0) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 #17 0x000000010fafea7d in main () (gdb) frame 8 #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); Current language: auto; currently minimal (gdb) p re $1 = (PetscReal *) 0x0 (gdb) list 323 PetscReal *re,*im,min,max; 324 325 PetscFunctionBegin; 326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); 329 min = PETSC_MAX_REAL; 330 max = PETSC_MIN_REAL; 331 for (n=0; n wrote: > Humm, GAMG works fine form (like you said), but with MG (with or without > Galerkin) on 1 or 4 procs I get: > > [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in > /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: > Parameter # 3 > > Can you tell me exactly what you get with: > > ./test1 -pc_type mg > > I get: > > [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in > /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: > Parameter # 3 > [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50433 on > display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine > Macintosh-6.local > > And, it seems that the RHS is all zeros on one processor. > > I am using a debug build. Try to do the same. > > > > > On Tue, Oct 21, 2014 at 2:50 AM, Filippo Leonardi < > filippo.leonardi at sam.math.ethz.ch> wrote: > >> Here, >> >> run this e.g. with >> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin >> >> In Debug mode it crashes, in standard mode it fails. Using Petsc current. >> >> Run without mg and works! >> >> Hope it's not me being stupid and doing something terribly wrong! >> >> Best, >> Filippo >> >> On Monday 20 October 2014 07:29:04 Barry Smith wrote: >> > Can you send us the code: to petsc-maint at mcs.anl.gov or >> > petsc-users at mcs.anl.gov ? Or something that reproduces the problem? >> > >> > Barry >> > >> > > On Oct 20, 2014, at 3:31 AM, Filippo Leonardi >> > > wrote: >> > > >> > > Hi, >> > > >> > > I have a very specific problem that I cannot figure out with PCMG and >> > > multiple solves. I got a linear system that I solve many times, with >> same >> > > matrix but different RHS. I can successfully solve the system with >> > > standard techniques, such as default solver or LU or PCGAMG. Even MG >> > > works if I destroy the ksp each time or I recompute the matrices at >> each >> > > time. But when I try and go to MG and not recomputing the matrices >> each >> > > time the solver fails. Any idea? >> > > >> > > Here some detail: the setup: >> > > ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); >> > > ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); >> > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); >> > > ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); >> > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >> > > >> > > Then I solve as usual, for a large number of time steps: >> > > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); >> > > ierr = KSPGetSolution(ksp, &phi); >> > > >> > > The solver converges and does that in a reasonable number of >> iterations: >> > > Linear solve converged due to CONVERGED_RTOL iterations 7 >> > > And ksp_view and ksp_monitor do not show any weird stuff. >> > > >> > > - Weirdly enough using any solver (for instance cg+bjacobi or gamg) >> > > everything works (So the matrix and RHS are working fine). >> > > - But the problem persists with Galerkin matrices (-pc_mg_galerkin) >> (So is >> > > not a ComputeMatrix problem). >> > > - If I do: >> > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); >> CHKERRQ(ierr); >> > > between each solve or destroy the ksp entirely each time the solution >> is >> > > also perfect (So is not a boundary scaling or other stuff problem). >> > > - If I run MG with only 2 levels (so just coarse) I also get a fine >> > > result. >> > > >> > > Setup RHS is called each time as expected, setup matrix is called just >> > > once, also as expected. >> > > >> > > The only thing I can think is that MG does not update some value that >> > > actually needs to be recomputed. >> > > >> > > Any idea? >> > > >> > > The solution is not that different from: >> > > >> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ >> > > ex29.c.html >> > > >> > > Best, >> > > Filippo >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From salazardetroya at gmail.com Tue Oct 21 18:04:49 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Tue, 21 Oct 2014 18:04:49 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: <87lhoam3ta.fsf@jedbrown.org> References: <87wq7vrjg5.fsf@jedbrown.org> <87lhoam3ta.fsf@jedbrown.org> Message-ID: That might be a reasonable argument, but I'm not sure. This is one of the papers that explains it. I'm re-reading to see if I skipped any details: http://www.sciencedirect.com/science/article/pii/S0377042709006062 Miguel On Mon, Oct 20, 2014 at 11:42 PM, Jed Brown wrote: > Miguel Angel Salazar de Troya writes: > > > Thanks for your response > > > > I'm struggling with this problem because the literature is not clear for > me > > on how to calculate the discrete adjoint with adaptive time stepping > > algorithms. They cover the details when automatic differentiation tools > are > > used. They mention that because the time step depend on the solution, it > > also depends on the parameter. Hence, there are terms that represent the > > derivative of the time step w.r.t. the parameters. What it is confusing > is > > that they mention these terms must be removed. I don't understand this. > I'm > > planning to hard-code the discrete adjoint problem (and use the TS if > > possible). > > Are they suggesting that the time step sizes for a given run should be > frozen (at least locally) so that you have consistent gradients for a > while? > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From salazardetroya at gmail.com Tue Oct 21 18:10:42 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Tue, 21 Oct 2014 18:10:42 -0500 Subject: [petsc-users] Cloning a DMNetwork Message-ID: Hi all I have a DMNetwork and I'm interested in cloning it to have another network with the same components. Can I do this with DMClone()? I have noticed that the components are not copied. Is there any function that takes care of this or it has to be hard coded? Thanks Miguel -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Oct 21 18:47:33 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 21 Oct 2014 18:47:33 -0500 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> Message-ID: Mark, Thanks for figuring it out. It is legitimate to malloc 0 size memory in PETSc and to have arrays of zero length. So 1) the asserts PetscValidScalarPointer(r,3); PetscValidScalarPointer(c,4); are over zealous. They should actually be if (n) PetscValidScalarPointer(r,3); BTW: we should have an additional assert that n >=0 2) that said I do not know if all the later code will function correctly in this case. I see some bad stuff PetscErrorCode KSPComputeEigenvalues_CG(KSP ksp,PetscInt nmax,PetscReal *r,PetscReal *c,PetscInt *neig) { KSP_CG *cgP = (KSP_CG*)ksp->data; PetscScalar *d,*e; PetscReal *ee; PetscErrorCode ierr; PetscInt j,n = ksp->its; PetscFunctionBegin; if (nmax < n) SETERRQ(PetscObjectComm((PetscObject)ksp),PETSC_ERR_ARG_SIZ,"Not enough room in work space r and c for eigenvalues"); *neig = n; ierr = PetscMemzero(c,nmax*sizeof(PetscReal));CHKERRQ(ierr); if (!n) { *r = 0.0; ^^^^ assumes there is space in r for something even if the array dimension is zero which is not good. This likely requires some more work to insure the code can run through properly in this case. We don?t want to simply short circuit the process. Barry > On Oct 21, 2014, at 5:53 PM, Mark Adams wrote: > > I am getting this in the output: > > [0] VecNormalize(): Vector of zero norm can not be normalized; Returning only the zero norm > [0] KSPGMRESCycle(): Converged due to zero residual norm on entry > [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 > [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50632 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local > > and in gdb I see this (appended). The problem is that KSPComputeEigenvalues is called with NULL outputs. This because KSPChebyshevComputeExtremeEigenvalues_Private get the number of iterations and it is zero, it mallocs zero, and this is an error. > > Should there be a check like: > > 326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); > if (!n) bail out > 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); > > or is something going wrong upstream? > > Mark > > #5 0x000000010fb86057 in PetscAttachDebuggerErrorHandler (comm=1140850689, line=119, fun=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", num=85, p=PETSC_ERROR_INITIAL, mess=0x7fff50102290 "Null Pointer: Parameter # 3", ctx=0x0) at /Users/markadams/Codes/petsc/src/sys/error/adebug.c:462 > #6 0x000000010fb88d30 in PetscError (comm=1140850689, line=119, func=0x1115b0a25 "KSPComputeEigenvalues", file=0x1115b09b9 "/Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c", n=85, p=PETSC_ERROR_INITIAL, mess=0x1115518ca "Null Pointer: Parameter # %d") at /Users/markadams/Codes/petsc/src/sys/error/err.c:378 > #7 0x0000000110bd3464 in KSPComputeEigenvalues (ksp=0x7fe92d813e60, n=0, r=0x0, c=0x0, neig=0x7fff50102c14) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:119 > #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 > #9 0x0000000110b69043 in KSPSolve_Chebyshev (ksp=0x7fe92c869a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:373 > #10 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c869a60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 > #11 0x0000000110a8e4b8 in PCMGMCycle_Private (pc=0x7fe92c862e60, mglevelsin=0x7fe92c868a60, reason=0x0) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:19 > #12 0x0000000110a9e1d5 in PCApply_MG (pc=0x7fe92c862e60, b=0x7fe92d002860, x=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/impls/mg/mg.c:337 > #13 0x0000000110ad6f2c in PCApply (pc=0x7fe92c862e60, x=0x7fe92d002860, y=0x7fe92d007c60) at /Users/markadams/Codes/petsc/src/ksp/pc/interface/precon.c:440 > #14 0x0000000110affbd2 in KSP_PCApply (ksp=0x7fe92c833a60, x=0x7fe92d002860, y=0x7fe92d007c60) at kspimpl.h:230 > #15 0x0000000110afcd31 in KSPSolve_CG (ksp=0x7fe92c833a60) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cg/cg.c:139 > #16 0x0000000110bd81ce in KSPSolve (ksp=0x7fe92c833a60, b=0x0, x=0x0) at /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c:460 > #17 0x000000010fafea7d in main () > (gdb) frame 8 > #8 0x0000000110b6a915 in KSPChebyshevComputeExtremeEigenvalues_Private (kspest=0x7fe92d813e60, emin=0x7fff50102ca0, emax=0x7fff50102ca8) at /Users/markadams/Codes/petsc/src/ksp/ksp/impls/cheby/cheby.c:328 > 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); > Current language: auto; currently minimal > (gdb) p re > $1 = (PetscReal *) 0x0 > (gdb) list > 323 PetscReal *re,*im,min,max; > 324 > 325 PetscFunctionBegin; > 326 ierr = KSPGetIterationNumber(kspest,&n);CHKERRQ(ierr); > 327 ierr = PetscMalloc2(n,&re,n,&im);CHKERRQ(ierr); > 328 ierr = KSPComputeEigenvalues(kspest,n,re,im,&neig);CHKERRQ(ierr); > 329 min = PETSC_MAX_REAL; > 330 max = PETSC_MIN_REAL; > 331 for (n=0; n 332 min = PetscMin(min,re[n]); > > > > On Tue, Oct 21, 2014 at 6:29 PM, Mark Adams wrote: > Humm, GAMG works fine form (like you said), but with MG (with or without Galerkin) on 1 or 4 procs I get: > > [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 > > Can you tell me exactly what you get with: > > ./test1 -pc_type mg > > I get: > > [0]PETSC ERROR: KSPComputeEigenvalues() line 119 in /Users/markadams/Codes/petsc/src/ksp/ksp/interface/itfunc.c Null Pointer: Parameter # 3 > [0]PETSC ERROR: PETSC: Attaching /usr/bin/gdb to ./test1 of pid 50433 on display /tmp/launch-ikNsM1/org.macosforge.xquartz:0 on machine Macintosh-6.local > > And, it seems that the RHS is all zeros on one processor. > > I am using a debug build. Try to do the same. > > > > > On Tue, Oct 21, 2014 at 2:50 AM, Filippo Leonardi wrote: > Here, > > run this e.g. with > mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > In Debug mode it crashes, in standard mode it fails. Using Petsc current. > > Run without mg and works! > > Hope it's not me being stupid and doing something terribly wrong! > > Best, > Filippo > > On Monday 20 October 2014 07:29:04 Barry Smith wrote: > > Can you send us the code: to petsc-maint at mcs.anl.gov or > > petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > > > > Barry > > > > > On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > > wrote: > > > > > > Hi, > > > > > > I have a very specific problem that I cannot figure out with PCMG and > > > multiple solves. I got a linear system that I solve many times, with same > > > matrix but different RHS. I can successfully solve the system with > > > standard techniques, such as default solver or LU or PCGAMG. Even MG > > > works if I destroy the ksp each time or I recompute the matrices at each > > > time. But when I try and go to MG and not recomputing the matrices each > > > time the solver fails. Any idea? > > > > > > Here some detail: the setup: > > > ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > > ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > > > ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > > ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > > > > > Then I solve as usual, for a large number of time steps: > > > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > ierr = KSPGetSolution(ksp, &phi); > > > > > > The solver converges and does that in a reasonable number of iterations: > > > Linear solve converged due to CONVERGED_RTOL iterations 7 > > > And ksp_view and ksp_monitor do not show any weird stuff. > > > > > > - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > > > everything works (So the matrix and RHS are working fine). > > > - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is > > > not a ComputeMatrix problem). > > > - If I do: > > > ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); > > > between each solve or destroy the ksp entirely each time the solution is > > > also perfect (So is not a boundary scaling or other stuff problem). > > > - If I run MG with only 2 levels (so just coarse) I also get a fine > > > result. > > > > > > Setup RHS is called each time as expected, setup matrix is called just > > > once, also as expected. > > > > > > The only thing I can think is that MG does not update some value that > > > actually needs to be recomputed. > > > > > > Any idea? > > > > > > The solution is not that different from: > > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ > > > ex29.c.html > > > > > > Best, > > > Filippo > > From bsmith at mcs.anl.gov Tue Oct 21 19:01:18 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 21 Oct 2014 19:01:18 -0500 Subject: [petsc-users] KSP solver always has Zero pivot in LU In-Reply-To: References: <4F59A195-A322-46BD-8B8A-4BFB324AC06D@mcs.anl.gov> Message-ID: <979DE2B9-2F6B-44E3-A276-7D832225E4FC@mcs.anl.gov> Your matrix has very differently scaled values on the diagonal: (18713,1) -1.6000e+11 (18714,1) 2.6550e+13 (18715,1) 5.0000e+11 (18716,1) 5.0001e+11 (18717,1) 1.0000e+00 (18718,1) 1.0000e+00 (18719,1) 1.0000e+00 (18720,1) 1.0000e+00 (18721,1) -2.4000e+11 (18722,1) 4.9999e+11 You do not want this. First I would change the meaning of the variables so all the diagonal entries are positive and then I would change those values of 1.0 on the diagonal to 1.0e12 If you are using MatZeroRowsColumns() or MatZeroRows() you can easily do this. You need to make sure as the matrix gets larger the diagonal entries you put in scale the same way. So for example if the matrix entries involve a 1/h^2 then you need to make sure the diagonal entries you put in scale that same way. Barry > On Oct 21, 2014, at 3:10 PM, Sharp Stone wrote: > > Hi Barry, > > Thank you very much for your reply. They are illustrating. > > I use LU and gmres in the solver. When I use a rather small size of grids, the KSP solver works fine. However, when a larger domain is used (still small in the problem I'm trying to solve), I still got the error "Zero pivot in LU factorization" from Petsc. As you previously suggested, I solved the problem in Matlab, and got the solution, though the matrix is close to singular. Do I have any options to solve such a system with Petsc? For the completeness, I attached the matrix and rhs vector, and look forward to hearing from you. Thank you in advance! > > On Mon, Oct 20, 2014 at 11:38 PM, Barry Smith wrote: > > What exact options are you running with? I tried ILU and LU and all was fine. > > $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type lu -ksp_view > Mat Object: 1 MPI processes > type: seqaij > row 0: (0, 1) (1, 0) (3, 0) > row 1: (0, 0) (1, 1) (2, 0) (4, 0) > row 2: (1, 0) (2, 1) (5, 0) > row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) > row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, -0.304878) > row 5: (2, 0) (4, 0) (5, 1) (8, 0) > row 6: (3, 0) (6, 1) (7, 0) > row 7: (4, 0) (6, 0) (7, 1) (8, 0) > row 8: (5, 0) (7, 0) (8, 1) > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: lu > LU: out-of-place factorization > tolerance for zero pivot 2.22045e-14 > matrix ordering: nd > factor fill ratio given 5, needed 1.54545 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > package used to perform factorization: petsc > total: nonzeros=51, allocated nonzeros=51 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > Number of iterations = 1 > Residual norm < 1.e-12 > ~/Src/petsc/src/ksp/ksp/examples/tutorials (barry/fix-setters *) > $ ./ex10 -f0 ~/Desktop/binaryoutput -rhs 0 -mat_view -pc_type ilu -ksp_view > Mat Object: 1 MPI processes > type: seqaij > row 0: (0, 1) (1, 0) (3, 0) > row 1: (0, 0) (1, 1) (2, 0) (4, 0) > row 2: (1, 0) (2, 1) (5, 0) > row 3: (0, -0.219298) (3, 1) (4, -0.561404) (6, -0.219298) > row 4: (1, -0.304878) (3, -0.097561) (4, 1) (5, -0.292683) (7, -0.304878) > row 5: (2, 0) (4, 0) (5, 1) (8, 0) > row 6: (3, 0) (6, 1) (7, 0) > row 7: (4, 0) (6, 0) (7, 1) (8, 0) > row 8: (5, 0) (7, 0) (8, 1) > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: ilu > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > matrix ordering: natural > factor fill ratio given 1, needed 1 > Factored matrix follows: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > package used to perform factorization: petsc > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=9, cols=9 > total: nonzeros=33, allocated nonzeros=33 > total number of mallocs used during MatSetValues calls =0 > not using I-node routines > Number of iterations = 2 > Residual norm < 1.e-12 > > > > On Oct 20, 2014, at 10:02 PM, Sharp Stone wrote: > > > > Dear All, > > > > Last week I raised a question about KSP solver with zero pivot problem. Now I downsized the matrix in my problem and still got the zero pivot problem. I looked at the matrix and computed its eigenvalues in matlab, and it seems the matrix is good (not singular, I mean), but petsc still reported zero pivot in LU factorization. What could possibly cause this? Thank you very much in advance! > > > > PS: I attached the matrix output in my simplest test. > > > > -- > > Best regards, > > > > Feng > > > > > > > -- > Best regards, > > Feng > From mfadams at lbl.gov Tue Oct 21 20:52:13 2014 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 21 Oct 2014 21:52:13 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> Message-ID: > > > 1) the asserts > PetscValidScalarPointer(r,3); > PetscValidScalarPointer(c,4); > > are over zealous. They should actually be if (n) > PetscValidScalarPointer(r,3); > > That fixed the problem. > BTW: we should have an additional assert that n >=0 > I guess I should have done a if (n<0) SETERRQ(... > > ierr = PetscMemzero(c,nmax*sizeof(PetscReal));CHKERRQ(ierr); > if (!n) { > *r = 0.0; > ^^^^ assumes there is space in r for something even if the array > dimension is zero which is not good. > > I'm not sure how to test this so I just protected this line. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Tue Oct 21 21:27:06 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 21 Oct 2014 21:27:06 -0500 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <10715057.mG3fKDZYPb@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> Message-ID: <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> There are a few issues with your code 1) If you use KSPSetComputeRHS() then the function you provide is called immediately before the solve automatically for every solve. Thus your code with the vector operations here ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); does nothing. And each linear solve ends up being with a zero right hand side. 2) -pc_type mg doesn?t give enough information for the multigrid to know how many levels to use. In fact it is always using one hence it is not multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it to use more than one level. Barry > On Oct 21, 2014, at 1:50 AM, Filippo Leonardi wrote: > > Here, > > run this e.g. with > mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > In Debug mode it crashes, in standard mode it fails. Using Petsc current. > > Run without mg and works! > > Hope it's not me being stupid and doing something terribly wrong! > > Best, > Filippo > > On Monday 20 October 2014 07:29:04 Barry Smith wrote: >> Can you send us the code: to petsc-maint at mcs.anl.gov or >> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? >> >> Barry >> >>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi >>> wrote: >>> >>> Hi, >>> >>> I have a very specific problem that I cannot figure out with PCMG and >>> multiple solves. I got a linear system that I solve many times, with same >>> matrix but different RHS. I can successfully solve the system with >>> standard techniques, such as default solver or LU or PCGAMG. Even MG >>> works if I destroy the ksp each time or I recompute the matrices at each >>> time. But when I try and go to MG and not recomputing the matrices each >>> time the solver fails. Any idea? >>> >>> Here some detail: the setup: >>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); >>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); >>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); >>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); >>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>> >>> Then I solve as usual, for a large number of time steps: >>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); >>> ierr = KSPGetSolution(ksp, &phi); >>> >>> The solver converges and does that in a reasonable number of iterations: >>> Linear solve converged due to CONVERGED_RTOL iterations 7 >>> And ksp_view and ksp_monitor do not show any weird stuff. >>> >>> - Weirdly enough using any solver (for instance cg+bjacobi or gamg) >>> everything works (So the matrix and RHS are working fine). >>> - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So is >>> not a ComputeMatrix problem). >>> - If I do: >>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); >>> between each solve or destroy the ksp entirely each time the solution is >>> also perfect (So is not a boundary scaling or other stuff problem). >>> - If I run MG with only 2 levels (so just coarse) I also get a fine >>> result. >>> >>> Setup RHS is called each time as expected, setup matrix is called just >>> once, also as expected. >>> >>> The only thing I can think is that MG does not update some value that >>> actually needs to be recomputed. >>> >>> Any idea? >>> >>> The solution is not that different from: >>> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ >>> ex29.c.html >>> >>> Best, >>> Filippo > From abhyshr at mcs.anl.gov Tue Oct 21 22:44:35 2014 From: abhyshr at mcs.anl.gov (Abhyankar, Shrirang G.) Date: Wed, 22 Oct 2014 03:44:35 +0000 Subject: [petsc-users] Cloning a DMNetwork In-Reply-To: Message-ID: There is no clone method implemented for DMNetwork yet. I'll work on it tomorrow. Thanks, Shri From: Miguel Angel Salazar de Troya Date: Tue, 21 Oct 2014 18:10:42 -0500 To: "petsc-users at mcs.anl.gov" Subject: [petsc-users] Cloning a DMNetwork >Hi all >I have a DMNetwork and I'm interested in cloning it to have another >network with the same components. Can I do this with DMClone()? I have >noticed that the components are not copied. Is there any function that >takes care of this or it has to be hard coded? > >Thanks >Miguel > >-- >Miguel Angel Salazar de Troya >Graduate Research Assistant >Department of Mechanical Science and Engineering >University of Illinois at Urbana-Champaign >(217) 550-2360 >salaza11 at illinois.edu From filippo.leonardi at sam.math.ethz.ch Wed Oct 22 01:41:27 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Wed, 22 Oct 2014 08:41:27 +0200 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> Message-ID: <1758633.Pqj9JboAtX@besikovitch-ii> Actually sorry, I have figured out the issues. The first iterate in the for cycle (int i = 0) sets the vector to be zero, which, for what I understand from the discussion above, is not handled correctly by PCMG (and only him) (because I guess the answer should still be computable). What I did in my real code was to solve an equation and, strangely enough, for some initial data the first rhs would be zero, exactly as in the code I provided (and I did not thing of that trough). Skipping the first solve of the system I get rid of the problem and things **seems** to b working correctly. What is even weirder is that in debug mode I get the mentioned EigenValue problem, but in Optimized mode the code runs through everything, but at the end the solutions of the whole process is completely wrong. Now things seem to be solved, thanks. On Tuesday 21 October 2014 21:27:06 you wrote: > There are a few issues with your code > > 1) If you use KSPSetComputeRHS() then the function you provide is called > immediately before the solve automatically for every solve. Thus your code > with the vector operations here > > ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); > ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); > ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > does nothing. And each linear solve ends up being with a zero right hand > side. > > 2) -pc_type mg doesn?t give enough information for the multigrid to know how > many levels to use. In fact it is always using one hence it is not > multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it to > use more than one level. > > Barry > > > On Oct 21, 2014, at 1:50 AM, Filippo Leonardi > > wrote: > > > > Here, > > > > run this e.g. with > > mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > > > In Debug mode it crashes, in standard mode it fails. Using Petsc current. > > > > Run without mg and works! > > > > Hope it's not me being stupid and doing something terribly wrong! > > > > Best, > > Filippo > > > > On Monday 20 October 2014 07:29:04 Barry Smith wrote: > >> Can you send us the code: to petsc-maint at mcs.anl.gov or > >> > >> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > >> > >> Barry > >> > >>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > >>> wrote: > >>> > >>> Hi, > >>> > >>> I have a very specific problem that I cannot figure out with PCMG and > >>> multiple solves. I got a linear system that I solve many times, with > >>> same > >>> matrix but different RHS. I can successfully solve the system with > >>> standard techniques, such as default solver or LU or PCGAMG. Even MG > >>> works if I destroy the ksp each time or I recompute the matrices at each > >>> time. But when I try and go to MG and not recomputing the matrices each > >>> time the solver fails. Any idea? > >>> > >>> Here some detail: the setup: > >>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > >>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > >>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > >>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > >>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > >>> > >>> Then I solve as usual, for a large number of time steps: > >>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > >>> ierr = KSPGetSolution(ksp, &phi); > >>> > >>> The solver converges and does that in a reasonable number of iterations: > >>> Linear solve converged due to CONVERGED_RTOL iterations 7 > >>> And ksp_view and ksp_monitor do not show any weird stuff. > >>> > >>> - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > >>> everything works (So the matrix and RHS are working fine). > >>> - But the problem persists with Galerkin matrices (-pc_mg_galerkin) (So > >>> is > >>> not a ComputeMatrix problem). > >>> - If I do: > >>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); CHKERRQ(ierr); > >>> between each solve or destroy the ksp entirely each time the solution is > >>> also perfect (So is not a boundary scaling or other stuff problem). > >>> - If I run MG with only 2 levels (so just coarse) I also get a fine > >>> result. > >>> > >>> Setup RHS is called each time as expected, setup matrix is called just > >>> once, also as expected. > >>> > >>> The only thing I can think is that MG does not update some value that > >>> actually needs to be recomputed. > >>> > >>> Any idea? > >>> > >>> The solution is not that different from: > >>> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorial > >>> s/ > >>> ex29.c.html > >>> > >>> Best, > >>> Filippo > > > > From filippo.leonardi at sam.math.ethz.ch Wed Oct 22 07:36:26 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Wed, 22 Oct 2014 14:36:26 +0200 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> Message-ID: <2815437.UQjMVro9BF@besikovitch-ii> Granted that I solved my problem by **not** solving the system if the Rhs is zero. Here's some result: 1) Using Multigrid with debug crashes (see below) 2) Using MG without debug doesn't crash, but gives wrong solution 3) defaults or gamg or wathever*works both with and without debug Now weirder: 4) changing line 145 from for(int i = 1; i < N; ++i) { to for(int i = 0; i < N; ++i) { i.e. avoiding broken case 1) above (rhs never 0) makes 1), 2) and 3) work perfectly (and any combination thereof) (You can look at the differences in the pictures given by Draw to see that the solution is more or less wrong.) Details: Release build with no debug: mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 -ksp_monitor - ksp_monitor_true_residual 0 KSP Residual norm 0.000000000000e+00 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm 0.000000000000e+00 ||r(i)||/||b|| -nan 0 KSP Residual norm 3.964678029172e+02 0 KSP preconditioned resid norm 3.964678029172e+02 true resid norm 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP Residual norm 6.290716828453e-13 1 KSP preconditioned resid norm 6.290716828453e-13 true resid norm 3.788861141557e+00 ||r(i)||/||b|| 3.388860427931e+00 0 KSP Residual norm 7.929356058344e+02 0 KSP preconditioned resid norm 7.929356058344e+02 true resid norm 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP Residual norm 1.258143365691e-12 1 KSP preconditioned resid norm 1.258143365691e-12 true resid norm 7.577722283114e+00 ||r(i)||/||b|| 3.388860427931e+00 i.e. I don't see the residual to be zero each time. Debug build: mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 -ksp_monitor - ksp_monitor_true_residual [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Null argument, when expecting valid pointer [0]PETSC ERROR: Null Pointer: Parameter # 3 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.0, Jun, 30, 2014 [0]PETSC ERROR: ./test1 on a arch-linux2-cxx-debug named Besikovitch-II by filippo Wed Oct 22 14:20:46 2014 [0]PETSC ERROR: Configure options --with-clanguage=cxx --download-scalapack -- download-mumps --download-parmetis --download-metis --download-fftw [0]PETSC ERROR: #1 KSPComputeEigenvalues() line 119 in /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #2 KSPChebyshevComputeExtremeEigenvalues_Private() line 328 in /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c [0]PETSC ERROR: #3 KSPSolve_Chebyshev() line 373 in /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c [0]PETSC ERROR: #4 KSPSolve() line 459 in /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #5 PCMGMCycle_Private() line 19 in /home/filippo/Workspace/petsc-3.5.0/src/ksp/pc/impls/mg/mg.c Debug build again with defaults: mpirun -np 4 ./test1 -draw_pause -1 -ksp_monitor -ksp_monitor_true_residual 0 KSP Residual norm 0.000000000000e+00 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm 0.000000000000e+00 ||r(i)||/||b|| -nan 0 KSP Residual norm 1.839181443725e+00 0 KSP preconditioned resid norm 1.839181443725e+00 true resid norm 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP Residual norm 1.767908128952e+00 1 KSP preconditioned resid norm 1.767908128952e+00 true resid norm 1.182712814433e+00 ||r(i)||/||b|| 1.057850500373e+00 [...] 82 KSP Residual norm 1.642117972681e-05 82 KSP preconditioned resid norm 1.642117972681e-05 true resid norm 1.556660502826e-05 ||r(i)||/||b|| 1.392319480883e-05 [...] 0 KSP Residual norm 3.678362887450e+00 0 KSP preconditioned resid norm 3.678362887450e+00 true resid norm 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP Residual norm 3.535816257904e+00 1 KSP preconditioned resid norm 3.535816257904e+00 true resid norm 2.365425628867e+00 ||r(i)||/||b|| 1.057850500373e+00 On Wednesday 22 October 2014 07:12:50 you wrote: > > On Oct 22, 2014, at 1:09 AM, Leonardi Filippo > > wrote: > > > > For 1): I must be missing something then: in main i set and scale ctx.rhs, > > then I call KSPSolve that calls ComputeRhs that further scales ctx.rhs > > and copy it to b, which is then used by the solve. To my understanding I > > never override ctx.rhs. > Oh yes. I missed that. > > What I observed was when I ran your code each solve had an initial > residual norm of exactly zero. I?m not sure why that happens. > > For 2): that was a small typo, problem is present no matter what number of > > levels I use. > > > > Thanks, > > Filippo > > > >> On 22-ott-2014, at 04:27, "Barry Smith" wrote: > >> > >> > >> There are a few issues with your code > >> > >> 1) If you use KSPSetComputeRHS() then the function you provide is called > >> immediately before the solve automatically for every solve. Thus your > >> code with the vector operations here > >> > >> ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); > >> > >> ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); > >> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > >> > >> does nothing. And each linear solve ends up being with a zero right hand > >> side. > >> > >> 2) -pc_type mg doesn?t give enough information for the multigrid to know > >> how many levels to use. In fact it is always using one hence it is not > >> multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it to > >> use more than one level. > >> > >> Barry > >> > >>> On Oct 21, 2014, at 1:50 AM, Filippo Leonardi > >>> wrote: > >>> > >>> Here, > >>> > >>> run this e.g. with > >>> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > >>> > >>> In Debug mode it crashes, in standard mode it fails. Using Petsc > >>> current. > >>> > >>> Run without mg and works! > >>> > >>> Hope it's not me being stupid and doing something terribly wrong! > >>> > >>> Best, > >>> Filippo > >>> > >>>> On Monday 20 October 2014 07:29:04 Barry Smith wrote: > >>>> Can you send us the code: to petsc-maint at mcs.anl.gov or > >>>> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > >>>> > >>>> Barry > >>>> > >>>>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > >>>>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I have a very specific problem that I cannot figure out with PCMG and > >>>>> multiple solves. I got a linear system that I solve many times, with > >>>>> same > >>>>> matrix but different RHS. I can successfully solve the system with > >>>>> standard techniques, such as default solver or LU or PCGAMG. Even MG > >>>>> works if I destroy the ksp each time or I recompute the matrices at > >>>>> each > >>>>> time. But when I try and go to MG and not recomputing the matrices > >>>>> each > >>>>> time the solver fails. Any idea? > >>>>> > >>>>> Here some detail: the setup: > >>>>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > >>>>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); CHKERRQ(ierr); > >>>>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > >>>>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > >>>>> > >>>>> Then I solve as usual, for a large number of time steps: > >>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > >>>>> ierr = KSPGetSolution(ksp, &phi); > >>>>> > >>>>> The solver converges and does that in a reasonable number of > >>>>> iterations: > >>>>> Linear solve converged due to CONVERGED_RTOL iterations 7 > >>>>> And ksp_view and ksp_monitor do not show any weird stuff. > >>>>> > >>>>> - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > >>>>> everything works (So the matrix and RHS are working fine). > >>>>> - But the problem persists with Galerkin matrices (-pc_mg_galerkin) > >>>>> (So is > >>>>> not a ComputeMatrix problem). > >>>>> - If I do: > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); > >>>>> CHKERRQ(ierr); > >>>>> between each solve or destroy the ksp entirely each time the solution > >>>>> is > >>>>> also perfect (So is not a boundary scaling or other stuff problem). > >>>>> - If I run MG with only 2 levels (so just coarse) I also get a fine > >>>>> result. > >>>>> > >>>>> Setup RHS is called each time as expected, setup matrix is called just > >>>>> once, also as expected. > >>>>> > >>>>> The only thing I can think is that MG does not update some value that > >>>>> actually needs to be recomputed. > >>>>> > >>>>> Any idea? > >>>>> > >>>>> The solution is not that different from: > >>>>> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutori > >>>>> als/ > >>>>> ex29.c.html > >>>>> > >>>>> Best, > >>>>> Filippo > >>> > >>> -- Filippo Leonardi SAM ETH Z?rich (HG J 45) R?mistrasse 101, Z?rich (CH) -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From arne.morten.kvarving at sintef.no Wed Oct 22 08:21:26 2014 From: arne.morten.kvarving at sintef.no (Arne Morten Kvarving) Date: Wed, 22 Oct 2014 15:21:26 +0200 Subject: [petsc-users] unreliable AMG in PETSc Message-ID: <5447AF56.3000805@sintef.no> hi there. we (sintef ict) use (parts of) PETSc in our (isogeometric) finite element library. sadly i find that the AMG in petsc is somewhat unreliable and it's a pain to deal with users (students) due to this fact. now, we all know that linear algebra, and AMG in particular, is more art than craft at times, certainly to the uneducated, and you can only do this much to automate it. but segfaults are bad. preconditioners which break the symmetry of the systems are bad. preconditioners that breaks the definiteness of systems is also bad. and worse; here it depends on the number of MPI processes. meshes are typically targeted a given world size and if you are unlucky, this number is one that does not work. as an example to illustrate this, consider http://www.math.ntnu.no/~arnemort/petsc-amg-bug.tar.bz2 it's about 30MB due to a somewhat large matrix, hopefully your pipes will survive. this is a (isogeometric) standard Poisson problem using cubic splines, with homogenous dirichlet boundaries and a uniform source (f = 1) on a mesh intended for simulating flow around wings, but i used the Poisson to keep it simple. see http://www.math.ntnu.no/~arnemort/wing.png if you want to get an idea. the tarball has the matrix and vector, as well as a cmake-based buildsystem for a simple test app (hacked ex23.c). it contains a number of tests (64 in total) to illustrate the unreliable behavior. in particular it solves the stored linear system using 1 through 32 processors, with either amg + sor or asm + ilu. the former fails and crashes for several runs, while the latter runs through for all (there to illustrate that the systems are solvable so that it is not drawn into question). i have tried this with v3.4.2, v3.4.4 and v3.5.2 - they all fail. in v3.4.2 one more test fails compared to the others. i have also confirmed that i can produce across systems and across toolchains (gcc 4.8 and icc 14.0). the fails i get are 27:amg_14 39:amg_20 41:amg_21 43:amg_22 45:amg_23 49:amg_25 51:amg_26 53:amg_27 55:amg_28 57:amg_29 61:amg_31 63:amg_32 do not put much thought into the configurations, or the high iteration counts. in the "real" code stronger smoothers etc are employed. but again i wanted to keep it simple. i do suspect that eventually the cause is ML. disclaimer: i have never been able to (find the energy to) decode the "slightly unreadable" code of petsc so i haven't tried to dig. i do realize this "bug report" has a number of holes (as far as the information it contains is entailed), and i can provide whatever you need upon request. but i have already sunk too much time into this and are behind on my projects :) usage instructions below. regards, arnem --- usage: requires petsc >= 3.4, some c++ compiler and cmake. export PETSC_DIR= export PETSC_ARCH= basic usage: tar jxvf petsc-amg-bug.tar.bz2 cd petsc-amg-bug cmake . -DCMAKE_BUILD_TYPE=Release make make test for verbose output of the process - replace make test with 'ctest -V' if your system does not provide the mpi spawner as 'mpirun', you can pass -DMPIRUN_COMMAND= to cmake. DO NOTE: you cannot use petscmpiexec as that does not return the error code to the terminal (probably should report that as a separate bug). or well, you can use it, but if you do, a failed tests with be reported as a success. you can execute particular tests only using ctest -R -V to run only the amg tests do ctest -R amg.* -V to run only a given amg test do ctest -R amg_5 -V From mfadams at lbl.gov Wed Oct 22 08:31:40 2014 From: mfadams at lbl.gov (Mark Adams) Date: Wed, 22 Oct 2014 09:31:40 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <2815437.UQjMVro9BF@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <10715057.mG3fKDZYPb@besikovitch-ii> <2DF46C4D-98E5-4010-955F-9D723A6AA419@mcs.anl.gov> <2815437.UQjMVro9BF@besikovitch-ii> Message-ID: I think we have a fix for this. You can test by using the branch. In PETSC_DIR: $ git checkout mark/ksp-zero-eig $ make Now remake your code and test, On Wed, Oct 22, 2014 at 8:36 AM, Filippo Leonardi < filippo.leonardi at sam.math.ethz.ch> wrote: > Granted that I solved my problem by **not** solving the system if the Rhs > is > zero. Here's some result: > > 1) Using Multigrid with debug crashes (see below) > 2) Using MG without debug doesn't crash, but gives wrong solution > 3) defaults or gamg or wathever*works both with and without debug > > Now weirder: > 4) changing line 145 from > for(int i = 1; i < N; ++i) { > to > for(int i = 0; i < N; ++i) { > i.e. avoiding broken case 1) above (rhs never 0) makes 1), 2) and 3) work > perfectly (and any combination thereof) > > (You can look at the differences in the pictures given by Draw to see that > the > solution is more or less wrong.) > > Details: > > Release build with no debug: > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > -ksp_monitor - > ksp_monitor_true_residual > 0 KSP Residual norm 0.000000000000e+00 > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > 0.000000000000e+00 ||r(i)||/||b|| -nan > 0 KSP Residual norm 3.964678029172e+02 > 0 KSP preconditioned resid norm 3.964678029172e+02 true resid norm > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP Residual norm 6.290716828453e-13 > 1 KSP preconditioned resid norm 6.290716828453e-13 true resid norm > 3.788861141557e+00 ||r(i)||/||b|| 3.388860427931e+00 > 0 KSP Residual norm 7.929356058344e+02 > 0 KSP preconditioned resid norm 7.929356058344e+02 true resid norm > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP Residual norm 1.258143365691e-12 > 1 KSP preconditioned resid norm 1.258143365691e-12 true resid norm > 7.577722283114e+00 ||r(i)||/||b|| 3.388860427931e+00 > > i.e. I don't see the residual to be zero each time. > > Debug build: > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > -ksp_monitor - > ksp_monitor_true_residual > [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: Null argument, when expecting valid pointer > [0]PETSC ERROR: Null Pointer: Parameter # 3 > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for > trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.0, Jun, 30, 2014 > [0]PETSC ERROR: ./test1 on a arch-linux2-cxx-debug named Besikovitch-II by > filippo Wed Oct 22 14:20:46 2014 > [0]PETSC ERROR: Configure options --with-clanguage=cxx > --download-scalapack -- > download-mumps --download-parmetis --download-metis --download-fftw > [0]PETSC ERROR: #1 KSPComputeEigenvalues() line 119 in > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: #2 KSPChebyshevComputeExtremeEigenvalues_Private() line > 328 in > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > [0]PETSC ERROR: #3 KSPSolve_Chebyshev() line 373 in > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > [0]PETSC ERROR: #4 KSPSolve() line 459 in > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > [0]PETSC ERROR: #5 PCMGMCycle_Private() line 19 in > /home/filippo/Workspace/petsc-3.5.0/src/ksp/pc/impls/mg/mg.c > > Debug build again with defaults: > > mpirun -np 4 ./test1 -draw_pause -1 -ksp_monitor -ksp_monitor_true_residual > 0 KSP Residual norm 0.000000000000e+00 > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > 0.000000000000e+00 ||r(i)||/||b|| -nan > 0 KSP Residual norm 1.839181443725e+00 > 0 KSP preconditioned resid norm 1.839181443725e+00 true resid norm > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP Residual norm 1.767908128952e+00 > 1 KSP preconditioned resid norm 1.767908128952e+00 true resid norm > 1.182712814433e+00 ||r(i)||/||b|| 1.057850500373e+00 > [...] > 82 KSP Residual norm 1.642117972681e-05 > 82 KSP preconditioned resid norm 1.642117972681e-05 true resid norm > 1.556660502826e-05 ||r(i)||/||b|| 1.392319480883e-05 > [...] > 0 KSP Residual norm 3.678362887450e+00 > 0 KSP preconditioned resid norm 3.678362887450e+00 true resid norm > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP Residual norm 3.535816257904e+00 > 1 KSP preconditioned resid norm 3.535816257904e+00 true resid norm > 2.365425628867e+00 ||r(i)||/||b|| 1.057850500373e+00 > > On Wednesday 22 October 2014 07:12:50 you wrote: > > > On Oct 22, 2014, at 1:09 AM, Leonardi Filippo > > > wrote: > > > > > > For 1): I must be missing something then: in main i set and scale > ctx.rhs, > > > then I call KSPSolve that calls ComputeRhs that further scales ctx.rhs > > > and copy it to b, which is then used by the solve. To my understanding > I > > > never override ctx.rhs. > > Oh yes. I missed that. > > > > What I observed was when I ran your code each solve had an initial > > residual norm of exactly zero. I?m not sure why that happens. > > > For 2): that was a small typo, problem is present no matter what > number of > > > levels I use. > > > > > > Thanks, > > > Filippo > > > > > >> On 22-ott-2014, at 04:27, "Barry Smith" wrote: > > >> > > >> > > >> There are a few issues with your code > > >> > > >> 1) If you use KSPSetComputeRHS() then the function you provide is > called > > >> immediately before the solve automatically for every solve. Thus your > > >> code with the vector operations here > > >> > > >> ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); > > >> > > >> ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); > > >> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > >> > > >> does nothing. And each linear solve ends up being with a zero right > hand > > >> side. > > >> > > >> 2) -pc_type mg doesn?t give enough information for the multigrid to > know > > >> how many levels to use. In fact it is always using one hence it is not > > >> multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it > to > > >> use more than one level. > > >> > > >> Barry > > >> > > >>> On Oct 21, 2014, at 1:50 AM, Filippo Leonardi > > >>> wrote: > > >>> > > >>> Here, > > >>> > > >>> run this e.g. with > > >>> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > >>> > > >>> In Debug mode it crashes, in standard mode it fails. Using Petsc > > >>> current. > > >>> > > >>> Run without mg and works! > > >>> > > >>> Hope it's not me being stupid and doing something terribly wrong! > > >>> > > >>> Best, > > >>> Filippo > > >>> > > >>>> On Monday 20 October 2014 07:29:04 Barry Smith wrote: > > >>>> Can you send us the code: to petsc-maint at mcs.anl.gov or > > >>>> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > > >>>> > > >>>> Barry > > >>>> > > >>>>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > >>>>> wrote: > > >>>>> > > >>>>> Hi, > > >>>>> > > >>>>> I have a very specific problem that I cannot figure out with PCMG > and > > >>>>> multiple solves. I got a linear system that I solve many times, > with > > >>>>> same > > >>>>> matrix but different RHS. I can successfully solve the system with > > >>>>> standard techniques, such as default solver or LU or PCGAMG. Even > MG > > >>>>> works if I destroy the ksp each time or I recompute the matrices at > > >>>>> each > > >>>>> time. But when I try and go to MG and not recomputing the matrices > > >>>>> each > > >>>>> time the solver fails. Any idea? > > >>>>> > > >>>>> Here some detail: the setup: > > >>>>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > >>>>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); > CHKERRQ(ierr); > > >>>>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > >>>>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > >>>>> > > >>>>> Then I solve as usual, for a large number of time steps: > > >>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > >>>>> ierr = KSPGetSolution(ksp, &phi); > > >>>>> > > >>>>> The solver converges and does that in a reasonable number of > > >>>>> iterations: > > >>>>> Linear solve converged due to CONVERGED_RTOL iterations 7 > > >>>>> And ksp_view and ksp_monitor do not show any weird stuff. > > >>>>> > > >>>>> - Weirdly enough using any solver (for instance cg+bjacobi or gamg) > > >>>>> everything works (So the matrix and RHS are working fine). > > >>>>> - But the problem persists with Galerkin matrices (-pc_mg_galerkin) > > >>>>> (So is > > >>>>> not a ComputeMatrix problem). > > >>>>> - If I do: > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); > > >>>>> CHKERRQ(ierr); > > >>>>> between each solve or destroy the ksp entirely each time the > solution > > >>>>> is > > >>>>> also perfect (So is not a boundary scaling or other stuff problem). > > >>>>> - If I run MG with only 2 levels (so just coarse) I also get a fine > > >>>>> result. > > >>>>> > > >>>>> Setup RHS is called each time as expected, setup matrix is called > just > > >>>>> once, also as expected. > > >>>>> > > >>>>> The only thing I can think is that MG does not update some value > that > > >>>>> actually needs to be recomputed. > > >>>>> > > >>>>> Any idea? > > >>>>> > > >>>>> The solution is not that different from: > > >>>>> > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutori > > >>>>> als/ > > >>>>> ex29.c.html > > >>>>> > > >>>>> Best, > > >>>>> Filippo > > >>> > > >>> > > -- > Filippo Leonardi > SAM ETH Z?rich > (HG J 45) R?mistrasse 101, Z?rich (CH) -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhyshr at mcs.anl.gov Wed Oct 22 10:16:16 2014 From: abhyshr at mcs.anl.gov (Abhyankar, Shrirang G.) Date: Wed, 22 Oct 2014 15:16:16 +0000 Subject: [petsc-users] Cloning a DMNetwork In-Reply-To: Message-ID: Miguel, I've added DMClone_Network to shri/clone-dmnetwork and next branch. Not that DMClone() creates a shallow copy of DMNetwork. If you change the values of the components in the cloned DM then those will also be reflected in the original DM (since they share the address space). Shri -----Original Message----- From: Shri Date: Wed, 22 Oct 2014 03:44:35 +0000 To: "petsc-users at mcs.anl.gov" Subject: Re: [petsc-users] Cloning a DMNetwork >There is no clone method implemented for DMNetwork yet. I'll work on it >tomorrow. > >Thanks, >Shri > >From: Miguel Angel Salazar de Troya >Date: Tue, 21 Oct 2014 18:10:42 -0500 >To: "petsc-users at mcs.anl.gov" >Subject: [petsc-users] Cloning a DMNetwork > > >>Hi all >>I have a DMNetwork and I'm interested in cloning it to have another >>network with the same components. Can I do this with DMClone()? I have >>noticed that the components are not copied. Is there any function that >>takes care of this or it has to be hard coded? >> >>Thanks >>Miguel >> >>-- >>Miguel Angel Salazar de Troya >>Graduate Research Assistant >>Department of Mechanical Science and Engineering >>University of Illinois at Urbana-Champaign >>(217) 550-2360 >>salaza11 at illinois.edu > From salazardetroya at gmail.com Wed Oct 22 12:16:37 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Wed, 22 Oct 2014 12:16:37 -0500 Subject: [petsc-users] Cloning a DMNetwork In-Reply-To: References: Message-ID: Thanks. Miguel On Wed, Oct 22, 2014 at 10:16 AM, Abhyankar, Shrirang G. < abhyshr at mcs.anl.gov> wrote: > Miguel, > I've added DMClone_Network to shri/clone-dmnetwork and next branch. Not > that DMClone() creates a shallow copy of DMNetwork. If you change the > values of the components in the cloned DM then those will also be > reflected in the original DM (since they share the address space). > > Shri > > -----Original Message----- > From: Shri > Date: Wed, 22 Oct 2014 03:44:35 +0000 > To: "petsc-users at mcs.anl.gov" > Subject: Re: [petsc-users] Cloning a DMNetwork > > >There is no clone method implemented for DMNetwork yet. I'll work on it > >tomorrow. > > > >Thanks, > >Shri > > > >From: Miguel Angel Salazar de Troya > >Date: Tue, 21 Oct 2014 18:10:42 -0500 > >To: "petsc-users at mcs.anl.gov" > >Subject: [petsc-users] Cloning a DMNetwork > > > > > >>Hi all > >>I have a DMNetwork and I'm interested in cloning it to have another > >>network with the same components. Can I do this with DMClone()? I have > >>noticed that the components are not copied. Is there any function that > >>takes care of this or it has to be hard coded? > >> > >>Thanks > >>Miguel > >> > >>-- > >>Miguel Angel Salazar de Troya > >>Graduate Research Assistant > >>Department of Mechanical Science and Engineering > >>University of Illinois at Urbana-Champaign > >>(217) 550-2360 > >>salaza11 at illinois.edu > > > > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulhuaizhang at gmail.com Wed Oct 22 14:11:21 2014 From: paulhuaizhang at gmail.com (paul zhang) Date: Wed, 22 Oct 2014 15:11:21 -0400 Subject: [petsc-users] how to compile the example code Message-ID: Hello, I'm not quite sure how to link the petscksp.h correctly so that I can compile the ex1.c file and run it in my computer. Can you shed some light? The PETSc package has been installed to /home/paul/LIB/petsc/. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Oct 22 14:16:15 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 22 Oct 2014 14:16:15 -0500 Subject: [petsc-users] how to compile the example code In-Reply-To: References: Message-ID: Have you tried using the makefile provided along with the example? for eg: cd src/ksp/ksp/examples/tutorials make ex1 PETSC_DIR=/home/paul/LIB/petsc/ Satish On Wed, 22 Oct 2014, paul zhang wrote: > Hello, > > I'm not quite sure how to link the petscksp.h correctly so that I can > compile the ex1.c file and run it in my computer. Can you shed some light? > The PETSc package has been installed to /home/paul/LIB/petsc/. > > > Thanks, > Paul > From paulhuaizhang at gmail.com Wed Oct 22 14:26:04 2014 From: paulhuaizhang at gmail.com (paul zhang) Date: Wed, 22 Oct 2014 15:26:04 -0400 Subject: [petsc-users] how to compile the example code In-Reply-To: References: Message-ID: Satish, It works. Thanks a lot. Well, if I'm going to write my own code, I have to write a new make file, right? Paul On Wed, Oct 22, 2014 at 3:16 PM, Satish Balay wrote: > > Have you tried using the makefile provided along with the example? > > for eg: > > cd src/ksp/ksp/examples/tutorials > make ex1 PETSC_DIR=/home/paul/LIB/petsc/ > > Satish > > On Wed, 22 Oct 2014, paul zhang wrote: > > > Hello, > > > > I'm not quite sure how to link the petscksp.h correctly so that I can > > compile the ex1.c file and run it in my computer. Can you shed some > light? > > The PETSc package has been installed to /home/paul/LIB/petsc/. > > > > > > Thanks, > > Paul > > > > -- Huaibao (Paul) Zhang *Gas Surface Interactions Lab* Department of Mechanical Engineering University of Kentucky, Lexington, KY, 40506-0503 *Office*: 216 Ralph G. Anderson Building *Web*:gsil.engineering.uky.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From balay at mcs.anl.gov Wed Oct 22 14:31:07 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Wed, 22 Oct 2014 14:31:07 -0500 Subject: [petsc-users] how to compile the example code In-Reply-To: References: Message-ID: You can copy the current example makefile (remove unneeded stuff) - and modify as needed. Satish On Wed, 22 Oct 2014, paul zhang wrote: > Satish, > > It works. Thanks a lot. Well, if I'm going to write my own code, I have to > write a new make file, right? > > Paul > > On Wed, Oct 22, 2014 at 3:16 PM, Satish Balay wrote: > > > > > Have you tried using the makefile provided along with the example? > > > > for eg: > > > > cd src/ksp/ksp/examples/tutorials > > make ex1 PETSC_DIR=/home/paul/LIB/petsc/ > > > > Satish > > > > On Wed, 22 Oct 2014, paul zhang wrote: > > > > > Hello, > > > > > > I'm not quite sure how to link the petscksp.h correctly so that I can > > > compile the ex1.c file and run it in my computer. Can you shed some > > light? > > > The PETSc package has been installed to /home/paul/LIB/petsc/. > > > > > > > > > Thanks, > > > Paul > > > > > > > > > > From paulhuaizhang at gmail.com Wed Oct 22 14:32:39 2014 From: paulhuaizhang at gmail.com (paul zhang) Date: Wed, 22 Oct 2014 15:32:39 -0400 Subject: [petsc-users] how to compile the example code In-Reply-To: References: Message-ID: Got that. Thanks again. Paul On Wed, Oct 22, 2014 at 3:31 PM, Satish Balay wrote: > You can copy the current example makefile (remove unneeded stuff) - > and modify as needed. > > Satish > > On Wed, 22 Oct 2014, paul zhang wrote: > > > Satish, > > > > It works. Thanks a lot. Well, if I'm going to write my own code, I have > to > > write a new make file, right? > > > > Paul > > > > On Wed, Oct 22, 2014 at 3:16 PM, Satish Balay wrote: > > > > > > > > Have you tried using the makefile provided along with the example? > > > > > > for eg: > > > > > > cd src/ksp/ksp/examples/tutorials > > > make ex1 PETSC_DIR=/home/paul/LIB/petsc/ > > > > > > Satish > > > > > > On Wed, 22 Oct 2014, paul zhang wrote: > > > > > > > Hello, > > > > > > > > I'm not quite sure how to link the petscksp.h correctly so that I can > > > > compile the ex1.c file and run it in my computer. Can you shed some > > > light? > > > > The PETSc package has been installed to /home/paul/LIB/petsc/. > > > > > > > > > > > > Thanks, > > > > Paul > > > > > > > > > > > > > > > > > > -- Huaibao (Paul) Zhang *Gas Surface Interactions Lab* Department of Mechanical Engineering University of Kentucky, Lexington, KY, 40506-0503 *Office*: 216 Ralph G. Anderson Building *Web*:gsil.engineering.uky.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From baagaard at usgs.gov Wed Oct 22 14:49:13 2014 From: baagaard at usgs.gov (Brad Aagaard) Date: Wed, 22 Oct 2014 12:49:13 -0700 Subject: [petsc-users] -log_view=:FILENAME:ascii_info_detail contains events with count 0 Message-ID: <54480A39.8090200@usgs.gov> I noticed that when I use -log_view=:FILENAME:ascii_info_detail, I get a file that for each stage contains info for all events even when they are not called (count == 0). Is this the intended behavior? If so, is there a way to turn off output of information for events with count == 0? Thanks, Brad From bsmith at mcs.anl.gov Wed Oct 22 14:59:46 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 22 Oct 2014 14:59:46 -0500 Subject: [petsc-users] -log_view=:FILENAME:ascii_info_detail contains events with count 0 In-Reply-To: <54480A39.8090200@usgs.gov> References: <54480A39.8090200@usgs.gov> Message-ID: I think the intention is that you write some simply python that processes the raw data dumped by the -log_view; so for example you could remove the events with count of 0. Would this work for you? Barry > On Oct 22, 2014, at 2:49 PM, Brad Aagaard wrote: > > I noticed that when I use -log_view=:FILENAME:ascii_info_detail, I get a file that for each stage contains info for all events even when they are not called (count == 0). Is this the intended behavior? If so, is there a way to turn off output of information for events with count == 0? > > Thanks, > Brad > From baagaard at usgs.gov Wed Oct 22 15:04:41 2014 From: baagaard at usgs.gov (Brad Aagaard) Date: Wed, 22 Oct 2014 13:04:41 -0700 Subject: [petsc-users] -log_view=:FILENAME:ascii_info_detail contains events with count 0 In-Reply-To: References: <54480A39.8090200@usgs.gov> Message-ID: <54480DD9.8080205@usgs.gov> Barry, Yes, I can process the file and dump the events with a count of 0. It seems odd to me that events with a count of zero were included in the first place and not filtered out when the log was dumped. That would make the log file (even in Python) easy to digest without additional processing. Brad On 10/22/2014 12:59 PM, Barry Smith wrote: > > I think the intention is that you write some simply python that processes the raw data dumped by the -log_view; so for example you could remove the events with count of 0. Would this work for you? > > Barry > >> On Oct 22, 2014, at 2:49 PM, Brad Aagaard wrote: >> >> I noticed that when I use -log_view=:FILENAME:ascii_info_detail, I get a file that for each stage contains info for all events even when they are not called (count == 0). Is this the intended behavior? If so, is there a way to turn off output of information for events with count == 0? >> >> Thanks, >> Brad >> > > > From bsmith at mcs.anl.gov Wed Oct 22 15:23:15 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 22 Oct 2014 15:23:15 -0500 Subject: [petsc-users] -log_view=:FILENAME:ascii_info_detail contains events with count 0 In-Reply-To: <54480DD9.8080205@usgs.gov> References: <54480A39.8090200@usgs.gov> <54480DD9.8080205@usgs.gov> Message-ID: <3B776AFC-2F34-4212-A437-88426E3E7F70@mcs.anl.gov> > On Oct 22, 2014, at 3:04 PM, Brad Aagaard wrote: > > Barry, > > Yes, I can process the file and dump the events with a count of 0. > > It seems odd to me that events with a count of zero were included in the first place and not filtered out when the log was dumped. That would make the log file (even in Python) easy to digest without additional processing. But what if you want to know about events with zero count? Basically I wanted a Python representation that captured essentially all the data inside the running application this would include zero count stuff and my thought is the user is going to process it and get what they want anyways hence I do not want PETSc to do any ?processing?. Barry > > Brad > > > On 10/22/2014 12:59 PM, Barry Smith wrote: >> >> I think the intention is that you write some simply python that processes the raw data dumped by the -log_view; so for example you could remove the events with count of 0. Would this work for you? >> >> Barry >> >>> On Oct 22, 2014, at 2:49 PM, Brad Aagaard wrote: >>> >>> I noticed that when I use -log_view=:FILENAME:ascii_info_detail, I get a file that for each stage contains info for all events even when they are not called (count == 0). Is this the intended behavior? If so, is there a way to turn off output of information for events with count == 0? >>> >>> Thanks, >>> Brad >>> >> >> >> > From evanum at gmail.com Wed Oct 22 15:29:47 2014 From: evanum at gmail.com (Evan Um) Date: Wed, 22 Oct 2014 13:29:47 -0700 Subject: [petsc-users] Calling single-precision MUMPS from PETSC Message-ID: Dear PETSC users, When MUMPS is used inside PETSC, The default MUMPS driver seems to be double-precision MUMPS (i.e. DMUMPS). To reduce memory costs, I want to test a single-precision MUMPS (SMUMPS) from PETSC. Does anyone know how to switch from double to single-precision MUMPS inside PETSC? In advance, thanks for your comments. Regards, Evan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 22 15:36:21 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 22 Oct 2014 15:36:21 -0500 Subject: [petsc-users] Calling single-precision MUMPS from PETSC In-Reply-To: References: Message-ID: There is no support for this. You can only call single precision MUMPS from single precision PETSc and double precision MUMPS from double precision PETSc. You could try to hack the interface that calls MUMPS from PETSc to use the single precision version but we don?t support that. src/mat/impls/aij/mpi/mumps/mumps.c Barry > On Oct 22, 2014, at 3:29 PM, Evan Um wrote: > > Dear PETSC users, > > When MUMPS is used inside PETSC, The default MUMPS driver seems to be double-precision MUMPS (i.e. DMUMPS). To reduce memory costs, I want to test a single-precision MUMPS (SMUMPS) from PETSC. Does anyone know how to switch from double to single-precision MUMPS inside PETSC? In advance, thanks for your comments. > > Regards, > Evan From jed at jedbrown.org Wed Oct 22 15:36:53 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 22 Oct 2014 14:36:53 -0600 Subject: [petsc-users] Calling single-precision MUMPS from PETSC In-Reply-To: References: Message-ID: <87tx2vkfiy.fsf@jedbrown.org> Evan Um writes: > Dear PETSC users, > > When MUMPS is used inside PETSC, The default MUMPS driver seems to be > double-precision MUMPS (i.e. DMUMPS). To reduce memory costs, I want to > test a single-precision MUMPS (SMUMPS) from PETSC. Does anyone know how to > switch from double to single-precision MUMPS inside PETSC? In advance, > thanks for your comments. Configure PETSc --with-precision=single. It is not currently possible to mix precision only when calling MUMPS. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From quecat001 at gmail.com Wed Oct 22 16:03:27 2014 From: quecat001 at gmail.com (Que Cat) Date: Wed, 22 Oct 2014 16:03:27 -0500 Subject: [petsc-users] Nonlinear solver for non-smooth jacobian In-Reply-To: <87zjcqmcb8.fsf@jedbrown.org> References: <87zjcqmcb8.fsf@jedbrown.org> Message-ID: Hello Jed, Thank for the information. I would take your first advise. Does SNESVI have the same functionality as TAO complementary solver? What do you mean by reformulating "You can reformulate as a variational inequality and use SNESVI.?". The Petsc manual does not provide much detail on SNESVI. Do I just set the snes type to SNESVI and call SNESVISetVariableBounds in which the lower and upper bounds on variables are set to PETSC_NINFINITY and INFINITY. Thanks. On Mon, Oct 20, 2014 at 8:38 PM, Jed Brown wrote: > Que Cat writes: > > > Hello Petsc-users, > > > > I have to solve a problem in which the jacobian is non-smooth ( > incoporates > > the heaviside step function behavior from the derivative of F(x) ). Could > > you recommend any type of nonlinear solver in Petsc which handles this > > issue ? > > You can reformulate as a variational inequality and use SNESVI. Be > warned that it's not for the faint of heart and it's less mature than > the "smooth" solvers. > > If this nonsmoothness is absolutely fundamental to your physics, like > contact or condensation, then you should expect to invest significant > effort in finding the best formulation. If the nonsmoothness is not > essential, you should put that effort into reformulating to avoid or > lessen the nonsmoothness. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed Oct 22 16:14:06 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 22 Oct 2014 15:14:06 -0600 Subject: [petsc-users] Nonlinear solver for non-smooth jacobian In-Reply-To: References: <87zjcqmcb8.fsf@jedbrown.org> Message-ID: <87lho7kdsx.fsf@jedbrown.org> Que Cat writes: > Hello Jed, > > Thank for the information. I would take your first advise. Does SNESVI have > the same functionality as TAO complementary solver? They are related, but not the same. SNESVI problems do not have to correspond to energy minimization, but it does need a well-posed variational inequality. Note that you can't simply take PDE + constraints and assume that it corresponds to a well-posed VI. There are examples of SNESVI. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From bsmith at mcs.anl.gov Wed Oct 22 16:17:53 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 22 Oct 2014 16:17:53 -0500 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <5447AF56.3000805@sintef.no> References: <5447AF56.3000805@sintef.no> Message-ID: <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> Arnem, I was able to reproduce your failures. ml is a rather old code from Sandia that is not getting new development (they are working on a complete replacement for several years). I ran you tests using the PETSc AMG solver -pc_type gamg (written largely by Mark Adams) using its default options and it converged for all your cases from 1 to 32 processes with no failure of positive definiteness etc. That said the convergence rate is not super-great, around 70+ iterations and I did not run for timing comparisons. But 75 iterations on 1 processors 76 iterations on 16 processors 78 iterations on 32 processors Do not run it with PETSc 3.4, only with PETSc 3.5 I would suggest you run with gamg and send us reports on problems that come up and drop ml. We are actively interested in improving gamg based on your feedback. Barry > On Oct 22, 2014, at 8:21 AM, Arne Morten Kvarving wrote: > > hi there. > > we (sintef ict) use (parts of) PETSc in our (isogeometric) finite element library. > sadly i find that the AMG in petsc is somewhat unreliable and it's a pain to deal with users (students) due to this fact. > > now, we all know that linear algebra, and AMG in particular, is more art than craft at times, certainly to the uneducated, and you can only do this much to automate it. but segfaults are bad. preconditioners which break the symmetry of the systems are bad. preconditioners that breaks the definiteness of systems is also bad. and worse; here it depends on the number of MPI processes. meshes are typically targeted a given world size and if you are unlucky, this number is one that does not work. > > as an example to illustrate this, consider > > http://www.math.ntnu.no/~arnemort/petsc-amg-bug.tar.bz2 > > it's about 30MB due to a somewhat large matrix, hopefully your pipes will survive. > > this is a (isogeometric) standard Poisson problem using cubic splines, with homogenous dirichlet boundaries and a uniform source (f = 1) on a mesh intended for simulating flow around wings, but i used the Poisson to keep it simple. see > > http://www.math.ntnu.no/~arnemort/wing.png > > if you want to get an idea. > > the tarball has the matrix and vector, as well as a cmake-based buildsystem for a simple test app (hacked ex23.c). it contains a number of tests (64 in total) to illustrate the unreliable behavior. > in particular it solves the stored linear system using 1 through 32 processors, with either amg + sor or asm + ilu. the former fails and crashes for several runs, while the latter runs through for all (there to illustrate that the systems are solvable so that it is not drawn into question). > > i have tried this with v3.4.2, v3.4.4 and v3.5.2 - they all fail. in v3.4.2 one more test fails compared to the others. i have also confirmed that i can produce across systems and across toolchains (gcc 4.8 and icc 14.0). the fails i get are > > 27:amg_14 > 39:amg_20 > 41:amg_21 > 43:amg_22 > 45:amg_23 > 49:amg_25 > 51:amg_26 > 53:amg_27 > 55:amg_28 > 57:amg_29 > 61:amg_31 > 63:amg_32 > > do not put much thought into the configurations, or the high iteration counts. in the "real" code stronger smoothers etc are employed. but again i wanted to keep it simple. > > i do suspect that eventually the cause is ML. disclaimer: i have never been able to (find the energy to) decode the "slightly unreadable" code of petsc so i haven't tried to dig. > > i do realize this "bug report" has a number of holes (as far as the information it contains is entailed), and i can provide whatever you need upon request. but i have already sunk too much time into this and are behind on my projects :) > > usage instructions below. > > regards, > > arnem > > --- > > usage: requires petsc >= 3.4, some c++ compiler and cmake. > > export PETSC_DIR= > export PETSC_ARCH= > > basic usage: > tar jxvf petsc-amg-bug.tar.bz2 > cd petsc-amg-bug > cmake . -DCMAKE_BUILD_TYPE=Release > make > make test > > for verbose output of the process - replace make test with 'ctest -V' > > if your system does not provide the mpi spawner as 'mpirun', you can pass -DMPIRUN_COMMAND= to cmake. DO NOTE: you cannot use petscmpiexec as that does not return the error code to the terminal (probably should report that as a separate bug). or well, you can use it, but if you do, a failed tests with be reported as a success. > > you can execute particular tests only using ctest -R -V > > to run only the amg tests do > > ctest -R amg.* -V > > to run only a given amg test do > > ctest -R amg_5 -V From Buhrkuhn at yale.edu Wed Oct 22 23:07:52 2014 From: Buhrkuhn at yale.edu (Blackboard) Date: Wed, 22 Oct 2014 21:07:52 -0700 Subject: [petsc-users] Email Alert Message-ID: <20141023040749.C3B6713184F@frontend.science.oregonstate.edu> ? New form posted to Blackboard | Technology. Your are to view this updae form regarding your next coming session section. Kindly sing in to view your form online. Sign In Blackboard | Services -------------- next part -------------- An HTML attachment was scrubbed... URL: From filippo.leonardi at sam.math.ethz.ch Thu Oct 23 01:32:14 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Thu, 23 Oct 2014 08:32:14 +0200 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <2815437.UQjMVro9BF@besikovitch-ii> Message-ID: <5202897.4tZeA0US6V@besikovitch-ii> Tested on this branch: the program no longer crashes, but the solution is actually still wrong. To reproduce you can compile a version with the for cycle starting from i = 0 and one from i = 1 and see the difference with: mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 and without mg. On Wednesday 22 October 2014 09:31:40 Mark Adams wrote: > I think we have a fix for this. You can test by using the branch. In > PETSC_DIR: > > $ git checkout mark/ksp-zero-eig > $ make > > Now remake your code and test, > > > On Wed, Oct 22, 2014 at 8:36 AM, Filippo Leonardi < > > filippo.leonardi at sam.math.ethz.ch> wrote: > > Granted that I solved my problem by **not** solving the system if the Rhs > > is > > zero. Here's some result: > > > > 1) Using Multigrid with debug crashes (see below) > > 2) Using MG without debug doesn't crash, but gives wrong solution > > 3) defaults or gamg or wathever*works both with and without debug > > > > Now weirder: > > 4) changing line 145 from > > > > for(int i = 1; i < N; ++i) { > > > > to > > > > for(int i = 0; i < N; ++i) { > > > > i.e. avoiding broken case 1) above (rhs never 0) makes 1), 2) and 3) work > > perfectly (and any combination thereof) > > > > (You can look at the differences in the pictures given by Draw to see that > > the > > solution is more or less wrong.) > > > > Details: > > > > Release build with no debug: > > > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > > -ksp_monitor - > > ksp_monitor_true_residual > > > > 0 KSP Residual norm 0.000000000000e+00 > > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > > > > 0.000000000000e+00 ||r(i)||/||b|| -nan > > > > 0 KSP Residual norm 3.964678029172e+02 > > 0 KSP preconditioned resid norm 3.964678029172e+02 true resid norm > > > > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > 1 KSP Residual norm 6.290716828453e-13 > > 1 KSP preconditioned resid norm 6.290716828453e-13 true resid norm > > > > 3.788861141557e+00 ||r(i)||/||b|| 3.388860427931e+00 > > > > 0 KSP Residual norm 7.929356058344e+02 > > 0 KSP preconditioned resid norm 7.929356058344e+02 true resid norm > > > > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > 1 KSP Residual norm 1.258143365691e-12 > > 1 KSP preconditioned resid norm 1.258143365691e-12 true resid norm > > > > 7.577722283114e+00 ||r(i)||/||b|| 3.388860427931e+00 > > > > i.e. I don't see the residual to be zero each time. > > > > Debug build: > > > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > > -ksp_monitor - > > ksp_monitor_true_residual > > [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message > > -------------------------------------------------------------- > > [0]PETSC ERROR: Null argument, when expecting valid pointer > > [0]PETSC ERROR: Null Pointer: Parameter # 3 > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > > for > > trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.0, Jun, 30, 2014 > > [0]PETSC ERROR: ./test1 on a arch-linux2-cxx-debug named Besikovitch-II by > > filippo Wed Oct 22 14:20:46 2014 > > [0]PETSC ERROR: Configure options --with-clanguage=cxx > > --download-scalapack -- > > download-mumps --download-parmetis --download-metis --download-fftw > > [0]PETSC ERROR: #1 KSPComputeEigenvalues() line 119 in > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #2 KSPChebyshevComputeExtremeEigenvalues_Private() line > > 328 in > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > > [0]PETSC ERROR: #3 KSPSolve_Chebyshev() line 373 in > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > > [0]PETSC ERROR: #4 KSPSolve() line 459 in > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > > [0]PETSC ERROR: #5 PCMGMCycle_Private() line 19 in > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/pc/impls/mg/mg.c > > > > Debug build again with defaults: > > > > mpirun -np 4 ./test1 -draw_pause -1 -ksp_monitor > > -ksp_monitor_true_residual > > > > 0 KSP Residual norm 0.000000000000e+00 > > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > > > > 0.000000000000e+00 ||r(i)||/||b|| -nan > > > > 0 KSP Residual norm 1.839181443725e+00 > > 0 KSP preconditioned resid norm 1.839181443725e+00 true resid norm > > > > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > 1 KSP Residual norm 1.767908128952e+00 > > 1 KSP preconditioned resid norm 1.767908128952e+00 true resid norm > > > > 1.182712814433e+00 ||r(i)||/||b|| 1.057850500373e+00 > > [...] > > > > 82 KSP Residual norm 1.642117972681e-05 > > 82 KSP preconditioned resid norm 1.642117972681e-05 true resid norm > > > > 1.556660502826e-05 ||r(i)||/||b|| 1.392319480883e-05 > > [...] > > 0 KSP Residual norm 3.678362887450e+00 > > > > 0 KSP preconditioned resid norm 3.678362887450e+00 true resid norm > > > > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > 1 KSP Residual norm 3.535816257904e+00 > > 1 KSP preconditioned resid norm 3.535816257904e+00 true resid norm > > > > 2.365425628867e+00 ||r(i)||/||b|| 1.057850500373e+00 > > > > On Wednesday 22 October 2014 07:12:50 you wrote: > > > > On Oct 22, 2014, at 1:09 AM, Leonardi Filippo > > > > wrote: > > > > > > > > For 1): I must be missing something then: in main i set and scale > > > > ctx.rhs, > > > > > > then I call KSPSolve that calls ComputeRhs that further scales ctx.rhs > > > > and copy it to b, which is then used by the solve. To my understanding > > > > I > > > > > > never override ctx.rhs. > > > > > > > Oh yes. I missed that. > > > > > > What I observed was when I ran your code each solve had an initial > > > > > > residual norm of exactly zero. I?m not sure why that happens. > > > > > > > For 2): that was a small typo, problem is present no matter what > > > > number of > > > > > > levels I use. > > > > > > > > Thanks, > > > > Filippo > > > > > > > >> On 22-ott-2014, at 04:27, "Barry Smith" wrote: > > > >> > > > >> > > > >> There are a few issues with your code > > > >> > > > >> 1) If you use KSPSetComputeRHS() then the function you provide is > > > > called > > > > > >> immediately before the solve automatically for every solve. Thus your > > > >> code with the vector operations here > > > >> > > > >> ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); > > > >> > > > >> ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); > > > >> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > >> > > > >> does nothing. And each linear solve ends up being with a zero right > > > > hand > > > > > >> side. > > > >> > > > >> 2) -pc_type mg doesn?t give enough information for the multigrid to > > > > know > > > > > >> how many levels to use. In fact it is always using one hence it is > > > >> not > > > >> multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it > > > > to > > > > > >> use more than one level. > > > >> > > > >> Barry > > > >> > > > >>> On Oct 21, 2014, at 1:50 AM, Filippo Leonardi > > > >>> wrote: > > > >>> > > > >>> Here, > > > >>> > > > >>> run this e.g. with > > > >>> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > > >>> > > > >>> In Debug mode it crashes, in standard mode it fails. Using Petsc > > > >>> current. > > > >>> > > > >>> Run without mg and works! > > > >>> > > > >>> Hope it's not me being stupid and doing something terribly wrong! > > > >>> > > > >>> Best, > > > >>> Filippo > > > >>> > > > >>>> On Monday 20 October 2014 07:29:04 Barry Smith wrote: > > > >>>> Can you send us the code: to petsc-maint at mcs.anl.gov or > > > >>>> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? > > > >>>> > > > >>>> Barry > > > >>>> > > > >>>>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > > >>>>> wrote: > > > >>>>> > > > >>>>> Hi, > > > >>>>> > > > >>>>> I have a very specific problem that I cannot figure out with PCMG > > > > and > > > > > >>>>> multiple solves. I got a linear system that I solve many times, > > > > with > > > > > >>>>> same > > > >>>>> matrix but different RHS. I can successfully solve the system with > > > >>>>> standard techniques, such as default solver or LU or PCGAMG. Even > > > > MG > > > > > >>>>> works if I destroy the ksp each time or I recompute the matrices > > > >>>>> at > > > >>>>> each > > > >>>>> time. But when I try and go to MG and not recomputing the matrices > > > >>>>> each > > > >>>>> time the solver fails. Any idea? > > > >>>>> > > > >>>>> Here some detail: the setup: > > > >>>>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > > >>>>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); > > > > CHKERRQ(ierr); > > > > > >>>>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > > >>>>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > > >>>>> > > > >>>>> Then I solve as usual, for a large number of time steps: > > > >>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > >>>>> ierr = KSPGetSolution(ksp, &phi); > > > >>>>> > > > >>>>> The solver converges and does that in a reasonable number of > > > >>>>> iterations: > > > >>>>> Linear solve converged due to CONVERGED_RTOL iterations 7 > > > >>>>> And ksp_view and ksp_monitor do not show any weird stuff. > > > >>>>> > > > >>>>> - Weirdly enough using any solver (for instance cg+bjacobi or > > > >>>>> gamg) > > > >>>>> everything works (So the matrix and RHS are working fine). > > > >>>>> - But the problem persists with Galerkin matrices > > > >>>>> (-pc_mg_galerkin) > > > >>>>> (So is > > > >>>>> not a ComputeMatrix problem). > > > >>>>> - If I do: > > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); > > > >>>>> CHKERRQ(ierr); > > > >>>>> between each solve or destroy the ksp entirely each time the > > > > solution > > > > > >>>>> is > > > >>>>> also perfect (So is not a boundary scaling or other stuff > > > >>>>> problem). > > > >>>>> - If I run MG with only 2 levels (so just coarse) I also get a > > > >>>>> fine > > > >>>>> result. > > > >>>>> > > > >>>>> Setup RHS is called each time as expected, setup matrix is called > > > > just > > > > > >>>>> once, also as expected. > > > >>>>> > > > >>>>> The only thing I can think is that MG does not update some value > > > > that > > > > > >>>>> actually needs to be recomputed. > > > >>>>> > > > >>>>> Any idea? > > > > > >>>>> The solution is not that different from: > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutori > > > > > >>>>> als/ > > > >>>>> ex29.c.html > > > >>>>> > > > >>>>> Best, > > > >>>>> Filippo > > > >>> > > > >>> > > > > -- > > Filippo Leonardi > > SAM ETH Z?rich > > (HG J 45) R?mistrasse 101, Z?rich (CH) -- Filippo Leonardi SAM ETH Z?rich (HG J 45) R?mistrasse 101, Z?rich (CH) -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From arne.morten.kvarving at sintef.no Thu Oct 23 02:54:41 2014 From: arne.morten.kvarving at sintef.no (Arne Morten Kvarving) Date: Thu, 23 Oct 2014 09:54:41 +0200 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> Message-ID: <5448B441.3090304@sintef.no> On 22/10/14 23:17, Barry Smith wrote: hi barry, thanks for your fast response. > Arnem, > > I was able to reproduce your failures. ml is a rather old code from Sandia that is not getting new development (they are working on a complete replacement for several years). > > I ran you tests using the PETSc AMG solver -pc_type gamg (written largely by Mark Adams) using its default options and it converged for all your cases from 1 to 32 processes with no failure of positive definiteness etc. i can confirm it works fine with the default options. however, i can easily fault gamg as well - turn on richardson ksps and poof. 27 - amg_14 (Failed) 39 - amg_20 (Failed) 41 - amg_21 (Failed) 43 - amg_22 (Failed) 45 - amg_23 (Failed) 49 - amg_25 (Failed) 51 - amg_26 (Failed) 53 - amg_27 (Failed) 55 - amg_28 (Failed) 57 - amg_29 (Failed) 61 - amg_31 (Failed) 63 - amg_32 (Failed) seems the use of chebyshev is a part of the cure, not just the agglomerator. in particular if i run ml with chebyshev the failures are reduced to 43 - amg_22 (Failed) 45 - amg_23 (Failed) 53 - amg_27 (Failed) 61 - amg_31 (Failed) 63 - amg_32 (Failed) unless my memory fails me, chebyshev is used to ensure the linear dependency of the preconditioner on the residual. in particular it's a suitable iterative solver for the coarse solve. so i think the fact that it cures things indicates that something breaks the linear property. hopefully the eigenvalue estimates aren't too expensive and i can run this setup on the real deal. will play. > > That said the convergence rate is not super-great, around 70+ iterations and I did not run for timing comparisons. But > > 75 iterations on 1 processors > 76 iterations on 16 processors > 78 iterations on 32 processors yes, as i said do not pay much attention to this. the real code uses line / plane smoothers due to the large cell aspect ratios of the mesh (10^3-10^4). when it works it uses ~ 15 iterations. > Do not run it with PETSc 3.4, only with PETSc 3.5 i played with gamg a year or so ago and concluded that it was a crashfest, divergencefest etc. i interpret your reponse here to mean that it has gotten quite some attention lately? > > I would suggest you run with gamg and send us reports on problems that come up and drop ml. We are actively interested in improving gamg based on your feedback. good stuff. you have my first report above :) cheers arnem From stali at geology.wisc.edu Thu Oct 23 07:20:00 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Thu, 23 Oct 2014 07:20:00 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? Message-ID: <5448F270.8010902@geology.wisc.edu> Hello I am using the following options (below) for solving linear elasticity/poroelasticity problems involving slip between two surfaces involving non-trivial geometries, i.e., elements with high aspect ratios, large contrasts in material properties etc. The constraints are imposed using Lagrange Multipliers. A picture (shows displacement magnitude) is attached. The boundary nodes, i.e., the base and the four side are pinned. The following options appear to work well for the saddle point problem: -pc_type fieldsplit -pc_fieldsplit_type schur -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower -ksp_monitor However, the number of iterations keep on increasing with the problems size (see attached plot), e.g., 120K Tets *507* Iterations (KSP Residual norm 8.827362494659e-05)in 17 secs on 3 cores 1 Million Tets *1374* Iterations (KSP Residual norm 7.164704416296e-05)in 117 secs on 20 cores 8 Million Tets *2495* Iterations (KSP Residual norm 9.101247550026e-05) in 225 secs on 160 cores So what other options should I try to improve solver performance? Any tips/insights would be appreciated as preconditioning is black magic to me. Thanks in advance. Tabrez -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: interface.png Type: image/png Size: 822600 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: residuals.pdf Type: application/pdf Size: 12162 bytes Desc: not available URL: From bsmith at mcs.anl.gov Thu Oct 23 07:32:56 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 23 Oct 2014 07:32:56 -0500 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <5202897.4tZeA0US6V@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <2815437.UQjMVro9BF@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> Message-ID: > On Oct 23, 2014, at 1:32 AM, Filippo Leonardi wrote: > > Tested on this branch: the program no longer crashes, but the solution is > actually still wrong. Yes, we never said that this fixed the problem. Jed ? > > To reproduce you can compile a version with the for cycle starting from i = 0 > and one from i = 1 and see the difference with: > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > and without mg. > > On Wednesday 22 October 2014 09:31:40 Mark Adams wrote: >> I think we have a fix for this. You can test by using the branch. In >> PETSC_DIR: >> >> $ git checkout mark/ksp-zero-eig >> $ make >> >> Now remake your code and test, >> >> >> On Wed, Oct 22, 2014 at 8:36 AM, Filippo Leonardi < >> >> filippo.leonardi at sam.math.ethz.ch> wrote: >>> Granted that I solved my problem by **not** solving the system if the Rhs >>> is >>> zero. Here's some result: >>> >>> 1) Using Multigrid with debug crashes (see below) >>> 2) Using MG without debug doesn't crash, but gives wrong solution >>> 3) defaults or gamg or wathever*works both with and without debug >>> >>> Now weirder: >>> 4) changing line 145 from >>> >>> for(int i = 1; i < N; ++i) { >>> >>> to >>> >>> for(int i = 0; i < N; ++i) { >>> >>> i.e. avoiding broken case 1) above (rhs never 0) makes 1), 2) and 3) work >>> perfectly (and any combination thereof) >>> >>> (You can look at the differences in the pictures given by Draw to see that >>> the >>> solution is more or less wrong.) >>> >>> Details: >>> >>> Release build with no debug: >>> >>> mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 >>> -ksp_monitor - >>> ksp_monitor_true_residual >>> >>> 0 KSP Residual norm 0.000000000000e+00 >>> 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm >>> >>> 0.000000000000e+00 ||r(i)||/||b|| -nan >>> >>> 0 KSP Residual norm 3.964678029172e+02 >>> 0 KSP preconditioned resid norm 3.964678029172e+02 true resid norm >>> >>> 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 >>> >>> 1 KSP Residual norm 6.290716828453e-13 >>> 1 KSP preconditioned resid norm 6.290716828453e-13 true resid norm >>> >>> 3.788861141557e+00 ||r(i)||/||b|| 3.388860427931e+00 >>> >>> 0 KSP Residual norm 7.929356058344e+02 >>> 0 KSP preconditioned resid norm 7.929356058344e+02 true resid norm >>> >>> 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 >>> >>> 1 KSP Residual norm 1.258143365691e-12 >>> 1 KSP preconditioned resid norm 1.258143365691e-12 true resid norm >>> >>> 7.577722283114e+00 ||r(i)||/||b|| 3.388860427931e+00 >>> >>> i.e. I don't see the residual to be zero each time. >>> >>> Debug build: >>> >>> mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 >>> -ksp_monitor - >>> ksp_monitor_true_residual >>> [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message >>> -------------------------------------------------------------- >>> [0]PETSC ERROR: Null argument, when expecting valid pointer >>> [0]PETSC ERROR: Null Pointer: Parameter # 3 >>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html >>> for >>> trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.5.0, Jun, 30, 2014 >>> [0]PETSC ERROR: ./test1 on a arch-linux2-cxx-debug named Besikovitch-II by >>> filippo Wed Oct 22 14:20:46 2014 >>> [0]PETSC ERROR: Configure options --with-clanguage=cxx >>> --download-scalapack -- >>> download-mumps --download-parmetis --download-metis --download-fftw >>> [0]PETSC ERROR: #1 KSPComputeEigenvalues() line 119 in >>> /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: #2 KSPChebyshevComputeExtremeEigenvalues_Private() line >>> 328 in >>> /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c >>> [0]PETSC ERROR: #3 KSPSolve_Chebyshev() line 373 in >>> /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c >>> [0]PETSC ERROR: #4 KSPSolve() line 459 in >>> /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c >>> [0]PETSC ERROR: #5 PCMGMCycle_Private() line 19 in >>> /home/filippo/Workspace/petsc-3.5.0/src/ksp/pc/impls/mg/mg.c >>> >>> Debug build again with defaults: >>> >>> mpirun -np 4 ./test1 -draw_pause -1 -ksp_monitor >>> -ksp_monitor_true_residual >>> >>> 0 KSP Residual norm 0.000000000000e+00 >>> 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm >>> >>> 0.000000000000e+00 ||r(i)||/||b|| -nan >>> >>> 0 KSP Residual norm 1.839181443725e+00 >>> 0 KSP preconditioned resid norm 1.839181443725e+00 true resid norm >>> >>> 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 >>> >>> 1 KSP Residual norm 1.767908128952e+00 >>> 1 KSP preconditioned resid norm 1.767908128952e+00 true resid norm >>> >>> 1.182712814433e+00 ||r(i)||/||b|| 1.057850500373e+00 >>> [...] >>> >>> 82 KSP Residual norm 1.642117972681e-05 >>> 82 KSP preconditioned resid norm 1.642117972681e-05 true resid norm >>> >>> 1.556660502826e-05 ||r(i)||/||b|| 1.392319480883e-05 >>> [...] >>> 0 KSP Residual norm 3.678362887450e+00 >>> >>> 0 KSP preconditioned resid norm 3.678362887450e+00 true resid norm >>> >>> 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 >>> >>> 1 KSP Residual norm 3.535816257904e+00 >>> 1 KSP preconditioned resid norm 3.535816257904e+00 true resid norm >>> >>> 2.365425628867e+00 ||r(i)||/||b|| 1.057850500373e+00 >>> >>> On Wednesday 22 October 2014 07:12:50 you wrote: >>>>> On Oct 22, 2014, at 1:09 AM, Leonardi Filippo >>>>> wrote: >>>>> >>>>> For 1): I must be missing something then: in main i set and scale >>> >>> ctx.rhs, >>> >>>>> then I call KSPSolve that calls ComputeRhs that further scales ctx.rhs >>>>> and copy it to b, which is then used by the solve. To my understanding >>> >>> I >>> >>>>> never override ctx.rhs. >>>>> >>>> Oh yes. I missed that. >>>> >>>> What I observed was when I ran your code each solve had an initial >>>> >>>> residual norm of exactly zero. I?m not sure why that happens. >>>> >>>>> For 2): that was a small typo, problem is present no matter what >>> >>> number of >>> >>>>> levels I use. >>>>> >>>>> Thanks, >>>>> Filippo >>>>> >>>>>> On 22-ott-2014, at 04:27, "Barry Smith" wrote: >>>>>> >>>>>> >>>>>> There are a few issues with your code >>>>>> >>>>>> 1) If you use KSPSetComputeRHS() then the function you provide is >>> >>> called >>> >>>>>> immediately before the solve automatically for every solve. Thus your >>>>>> code with the vector operations here >>>>>> >>>>>> ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); >>>>>> >>>>>> ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); >>>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); >>>>>> >>>>>> does nothing. And each linear solve ends up being with a zero right >>> >>> hand >>> >>>>>> side. >>>>>> >>>>>> 2) -pc_type mg doesn?t give enough information for the multigrid to >>> >>> know >>> >>>>>> how many levels to use. In fact it is always using one hence it is >>>>>> not >>>>>> multigrid. You need to use -p_mg_levels 2 or 3 or whatever to get it >>> >>> to >>> >>>>>> use more than one level. >>>>>> >>>>>> Barry >>>>>> >>>>>>> On Oct 21, 2014, at 1:50 AM, Filippo Leonardi >>>>>>> wrote: >>>>>>> >>>>>>> Here, >>>>>>> >>>>>>> run this e.g. with >>>>>>> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin >>>>>>> >>>>>>> In Debug mode it crashes, in standard mode it fails. Using Petsc >>>>>>> current. >>>>>>> >>>>>>> Run without mg and works! >>>>>>> >>>>>>> Hope it's not me being stupid and doing something terribly wrong! >>>>>>> >>>>>>> Best, >>>>>>> Filippo >>>>>>> >>>>>>>> On Monday 20 October 2014 07:29:04 Barry Smith wrote: >>>>>>>> Can you send us the code: to petsc-maint at mcs.anl.gov or >>>>>>>> petsc-users at mcs.anl.gov ? Or something that reproduces the problem? >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have a very specific problem that I cannot figure out with PCMG >>> >>> and >>> >>>>>>>>> multiple solves. I got a linear system that I solve many times, >>> >>> with >>> >>>>>>>>> same >>>>>>>>> matrix but different RHS. I can successfully solve the system with >>>>>>>>> standard techniques, such as default solver or LU or PCGAMG. Even >>> >>> MG >>> >>>>>>>>> works if I destroy the ksp each time or I recompute the matrices >>>>>>>>> at >>>>>>>>> each >>>>>>>>> time. But when I try and go to MG and not recomputing the matrices >>>>>>>>> each >>>>>>>>> time the solver fails. Any idea? >>>>>>>>> >>>>>>>>> Here some detail: the setup: >>>>>>>>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); >>>>>>>>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); >>>>>>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); >>> >>> CHKERRQ(ierr); >>> >>>>>>>>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); >>>>>>>>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); >>>>>>>>> >>>>>>>>> Then I solve as usual, for a large number of time steps: >>>>>>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); >>>>>>>>> ierr = KSPGetSolution(ksp, &phi); >>>>>>>>> >>>>>>>>> The solver converges and does that in a reasonable number of >>>>>>>>> iterations: >>>>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 7 >>>>>>>>> And ksp_view and ksp_monitor do not show any weird stuff. >>>>>>>>> >>>>>>>>> - Weirdly enough using any solver (for instance cg+bjacobi or >>>>>>>>> gamg) >>>>>>>>> everything works (So the matrix and RHS are working fine). >>>>>>>>> - But the problem persists with Galerkin matrices >>>>>>>>> (-pc_mg_galerkin) >>>>>>>>> (So is >>>>>>>>> not a ComputeMatrix problem). >>>>>>>>> - If I do: >>>>>>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); >>>>>>>>> CHKERRQ(ierr); >>>>>>>>> between each solve or destroy the ksp entirely each time the >>> >>> solution >>> >>>>>>>>> is >>>>>>>>> also perfect (So is not a boundary scaling or other stuff >>>>>>>>> problem). >>>>>>>>> - If I run MG with only 2 levels (so just coarse) I also get a >>>>>>>>> fine >>>>>>>>> result. >>>>>>>>> >>>>>>>>> Setup RHS is called each time as expected, setup matrix is called >>> >>> just >>> >>>>>>>>> once, also as expected. >>>>>>>>> >>>>>>>>> The only thing I can think is that MG does not update some value >>> >>> that >>> >>>>>>>>> actually needs to be recomputed. >>>>>>>>> >>>>>>>>> Any idea? >>> >>>>>>>>> The solution is not that different from: >>> http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutori >>> >>>>>>>>> als/ >>>>>>>>> ex29.c.html >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Filippo >>>>>>> >>>>>>> >>> >>> -- >>> Filippo Leonardi >>> SAM ETH Z?rich >>> (HG J 45) R?mistrasse 101, Z?rich (CH) > > -- > Filippo Leonardi > SAM ETH Z?rich > (HG J 45) R?mistrasse 101, Z?rich (CH) From knepley at gmail.com Thu Oct 23 07:46:39 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 23 Oct 2014 07:46:39 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <5448F270.8010902@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> Message-ID: On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali wrote: > Hello > > I am using the following options (below) for solving linear > elasticity/poroelasticity problems involving slip between two surfaces > involving non-trivial geometries, i.e., elements with high aspect ratios, > large contrasts in material properties etc. The constraints are imposed > using Lagrange Multipliers. > > A picture (shows displacement magnitude) is attached. The boundary nodes, > i.e., the base and the four side are pinned. > > The following options appear to work well for the saddle point problem: > > -pc_type fieldsplit -pc_fieldsplit_type schur > -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg > -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc > -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower > -ksp_monitor > > However, the number of iterations keep on increasing with the problems > size (see attached plot), e.g., > > 120K Tets *507* Iterations (KSP Residual norm 8.827362494659e-05) > in 17 secs on 3 cores > 1 Million Tets *1374* Iterations (KSP Residual norm 7.164704416296e-05) > in 117 secs on 20 cores > 8 Million Tets *2495* Iterations (KSP Residual norm 9.101247550026e-05) > in 225 secs on 160 cores > > So what other options should I try to improve solver performance? Any > tips/insights would be appreciated as preconditioning is black magic to me. > For reports, always run with -ksp_view -ksp_monitor_true_residual -ksp_converged_reason so that we can see exactly what you used. I believe the default is a diagonal factorization. Since your outer iterates are increasing, I would strengthen this to either upper or full -pc_fieldsplit_schur_factorization_type Thanks, Matt > Thanks in advance. > > Tabrez > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 23 08:04:51 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 23 Oct 2014 08:04:51 -0500 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <5448B441.3090304@sintef.no> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> Message-ID: > On Oct 23, 2014, at 2:54 AM, Arne Morten Kvarving wrote: > >> >> I ran you tests using the PETSc AMG solver -pc_type gamg (written largely by Mark Adams) using its default options and it converged for all your cases from 1 to 32 processes with no failure of positive definiteness etc. > i can confirm it works fine with the default options. however, i can easily fault gamg as well - turn on richardson ksps and poof. > > 27 - amg_14 (Failed) > > > seems the use of chebyshev is a part of the cure, not just the agglomerator. in particular if i run ml with chebyshev the failures are reduced to > > 43 - amg_22 (Failed) > 45 - amg_23 (Failed) > 53 - amg_27 (Failed) > 61 - amg_31 (Failed) > 63 - amg_32 (Failed) > > unless my memory fails me, chebyshev is used to ensure the linear dependency of the preconditioner on the residual. No, both Richardson and Chebyshev are linear operators so this is not the issue. > in particular it's a suitable iterative solver for the coarse solve. > > so i think the fact that it cures things indicates that something breaks the linear property. No. The problem in your cases is that the preconditioner operator (which is a V cycle of multigrid) with Richardson smoothing is an indefinite operator. That is inside the CG algorithm the computation of R? * B *R (where B is an application of the preconditioner) results in a negative number sometimes and sometimes a positive number. The CG algorithm requires that the preconditioner operator be positive definite (or negative definite) but it cannot be indefinite. Now since your original matrix A is positive definite (and hence its inverse is positive definite), one would hope that a significantly close preconditioner would preserve the property of being positive definite (otherwise it is not a close approximation to the inverse). So loosely speaking using Richardson smoothing in this case results in a preconditioner that is not close enough to the inverse of A to preserve the positive definiteness and hence CG breaks down. Of course running with GMRES or any method that allows an indefinite preconditioner is fine. Now I?ve never seen anybody write any analysis for why a preconditioner is not positive definite and how to fix it without just using a ?stronger? preconditioner. That is, I don?t know how to ?fix? an application of a preconditioner that is not positive definite in a general way. Note that for MG there are a variety of ways you can try to make the preconditioner ?stronger? besides changing the smoother to Chebyshev. You could try 1) using more smoothing steps 2) you could try a W cycle instead of V. -pc_mg_cycle_type w 3) you could use 2 V cycles for the preconditioner instead of 1 -pc_mg_multiplicative_cycles 2 4) you could try other GAMG options. BTW: if you get segmentation violation type crashes in GAMG please report them so we can fix the problem. We cannot build a GAMG that will always converge for any problem but we really really really don?t want it crashing due to bugs in the code. Barry > > hopefully the eigenvalue estimates aren't too expensive and i can run this setup on the real deal. will play. > >> >> That said the convergence rate is not super-great, around 70+ iterations and I did not run for timing comparisons. But >> >> 75 iterations on 1 processors >> 76 iterations on 16 processors >> 78 iterations on 32 processors > yes, as i said do not pay much attention to this. the real code uses line / plane smoothers due to the large cell aspect ratios of the mesh (10^3-10^4). when it works it uses ~ 15 iterations. > >> Do not run it with PETSc 3.4, only with PETSc 3.5 > i played with gamg a year or so ago and concluded that it was a crashfest, divergencefest etc. > i interpret your reponse here to mean that it has gotten quite some attention lately? >> >> I would suggest you run with gamg and send us reports on problems that come up and drop ml. We are actively interested in improving gamg based on your feedback. > good stuff. you have my first report above :) > > cheers > > arnem From arne.morten.kvarving at sintef.no Thu Oct 23 08:31:02 2014 From: arne.morten.kvarving at sintef.no (Arne Morten Kvarving) Date: Thu, 23 Oct 2014 15:31:02 +0200 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> Message-ID: <54490316.6010008@sintef.no> On 23/10/14 15:04, Barry Smith wrote: hi barry, thanks for your detailed and patient answer. > The problem in your cases is that the preconditioner operator (which > is a V cycle of multigrid) with Richardson smoothing is an indefinite > operator. That is inside the CG algorithm the computation of R? aha. the devil and his details. it all makes sense now - users of the coupled navier-stokes solver never saw the issues (where a bcgs/gmres drives the loop) and the multigrid is used for the pressure block. only the users of the chorin scheme did, where the pressure update is spd (poisson) and cg is used. i should have picked up on this on my own earlier.. thanks for educating me! i wasn't aware richardson could break definiteness. > * B *R (where B is an application of the preconditioner) results in a > negative number sometimes and sometimes a positive number. The CG > algorithm requires that the preconditioner operator be positive > definite (or negative definite) but it cannot be indefinite. Now since > your original matrix A is positive definite (and hence its inverse is > positive definite), one would hope that a significantly close > preconditioner would preserve the property of being positive definite > (otherwise it is not a close approximation to the inverse). So loosely > speaking using Richardson smoothing in this case results in a > preconditioner that is not close enough to the inverse of A to > preserve the positive definiteness and hence CG breaks down. Of course > running with GMRES or any method that allows an indefinite > preconditioner is fine. Now I?ve never seen anybody write any analysis > for why a preconditioner is not positive definite and how to fix it > without just using a ?stronger? preconditioner. That is, I don?t know > how to ?fix? an application of a preconditioner that is not positive > definite in a general way. Note that indeed. i am aware of this, i tried to be defensive in my original post :) i will have to write a FAQ entry for things to check when INDEFINITE_PC is thrown at users. then i can yell when they come to me before reading it ;) one small thing that could perhaps be added is to warn if the matrix is marked spd and ksp is cg ? i will add this in our code for sure. > for MG there are a variety of ways you can try to make the > preconditioner ?stronger? besides changing the smoother to Chebyshev. > You could try 1) using more smoothing steps 2) you could try a W cycle > instead of V. -pc_mg_cycle_type w 3) you could use 2 V cycles for the > preconditioner instead of 1 -pc_mg_multiplicative_cycles 2 4) you > could try other GAMG options. thanks. i did try all of these (though with ml) before making a fool of myself on the mailing list with my lack of knowledge.. > BTW: if you get segmentation violation type crashes in GAMG please > report them so we can fix the problem. We cannot build a GAMG that > will always converge for any problem but we really really really don?t > want it crashing due to bugs in the code. i will. arnem From jed at jedbrown.org Thu Oct 23 08:53:40 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 23 Oct 2014 07:53:40 -0600 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <54490316.6010008@sintef.no> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> Message-ID: <87bnp2j3iz.fsf@jedbrown.org> Arne Morten Kvarving writes: > thanks for educating me! i wasn't aware richardson could break definiteness. This is usually caused by the Richardson parameter being too aggressive. An analogous breakdown occurs with Chebyshev when the eigenvalue estimate is too small. > one small thing that could perhaps be added is to warn if the matrix is > marked spd and ksp is cg ? i will add this in our code for sure. Do you mean if the matrix is _not_ marked SPD? Interestingly, CG is often tolerant of weak asymmetry, such as you get with nonsymmetric enforcement of Dirichlet boundary conditions. That tolerance can be considered robust when the first application of the preconditioner solves those equations contributing to asymmetry, in which case further iterations take place in a "benign" subspace in which the operator is SPD (so long as the preconditioner does not undo this). We don't have a good system for offering advice or things to check and PETSc's convention is always to make solvers silent unless information is requested. We could have -ksp_view note when a matrix has not been marked SPD. Or we could (with much more effort) create a new feature "-ksp_analyze" that runs some fairly expensive tests and offers advice. Thanks for your feedback and test suite. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From arne.morten.kvarving at sintef.no Thu Oct 23 09:06:10 2014 From: arne.morten.kvarving at sintef.no (Arne Morten Kvarving) Date: Thu, 23 Oct 2014 16:06:10 +0200 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <87bnp2j3iz.fsf@jedbrown.org> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> <87bnp2j3iz.fsf@jedbrown.org> Message-ID: <54490B52.6060004@sintef.no> On 23/10/14 15:53, Jed Brown wrote: hi jed, > Arne Morten Kvarving writes: > Do you mean if the matrix is _not_ marked SPD? Interestingly, CG is no i mean to warn if the matrix is marked SPD, main KSP is CG and Richardson is used on the multigrid ksps. i guess it falls into user app space and is hard to do on a general level since, as you say, cg might survive slight asymmetries (or an user could be filtering out the naughty modes, like the hydrostatic). > often tolerant of weak asymmetry, such as you get with nonsymmetric > enforcement of Dirichlet boundary conditions. That tolerance can be > considered robust when the first application of the preconditioner > solves those equations contributing to asymmetry, in which case further > iterations take place in a "benign" subspace in which the operator is > SPD (so long as the preconditioner does not undo this). > > We don't have a good system for offering advice or things to check and > PETSc's convention is always to make solvers silent unless information > is requested. We could have -ksp_view note when a matrix has not been > marked SPD. Or we could (with much more effort) create a new feature > "-ksp_analyze" that runs some fairly expensive tests and offers advice. thanks. i kinda supected this was the policy. cheers arnem From jed at jedbrown.org Thu Oct 23 09:13:47 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 23 Oct 2014 08:13:47 -0600 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <54490B52.6060004@sintef.no> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> <87bnp2j3iz.fsf@jedbrown.org> <54490B52.6060004@sintef.no> Message-ID: <8738aej2lg.fsf@jedbrown.org> Arne Morten Kvarving writes: > On 23/10/14 15:53, Jed Brown wrote: > > hi jed, >> Arne Morten Kvarving writes: >> Do you mean if the matrix is _not_ marked SPD? Interestingly, CG is > > no i mean to warn if the matrix is marked SPD, main KSP is CG and > Richardson is used on the multigrid ksps. i guess it falls into user > app space and is hard to do on a general level since, as you say, cg > might survive slight asymmetries (or an user could be filtering out > the naughty modes, like the hydrostatic). There are many ways to get a preconditioner that is not SPD. It's no safer if the matrix is marked SPD. And Richardson smoothing is not inherently problematic; more like the relaxation paramater is easy to mess up. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From stali at geology.wisc.edu Thu Oct 23 09:27:14 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Thu, 23 Oct 2014 09:27:14 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> Message-ID: <54491042.8080307@geology.wisc.edu> Matt Sorry about that (I always forget it). The output for the smallest problem is now attached (see log.txt). I am also attaching some results that compare results obtained using FS/LSC and the direct solver (MUMPS), again for the smallest problem. The difference, as you can see is insignificant O(1E-6). Also, I did pass 'upper' and 'full' to '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't improve (in fact, it increases slightly). The attached log is with 'upper'. Regards, Tabrez On 10/23/2014 07:46 AM, Matthew Knepley wrote: > On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali > wrote: > > Hello > > I am using the following options (below) for solving linear > elasticity/poroelasticity problems involving slip between two > surfaces involving non-trivial geometries, i.e., elements with > high aspect ratios, large contrasts in material properties etc. > The constraints are imposed using Lagrange Multipliers. > > A picture (shows displacement magnitude) is attached. The boundary > nodes, i.e., the base and the four side are pinned. > > The following options appear to work well for the saddle point > problem: > > -pc_type fieldsplit -pc_fieldsplit_type schur > -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg > -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc > -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type > lower -ksp_monitor > > However, the number of iterations keep on increasing with the > problems size (see attached plot), e.g., > > 120K Tets *507* Iterations (KSP Residual norm > 8.827362494659e-05)in 17 secs on 3 cores > 1 Million Tets *1374* Iterations (KSP Residual norm > 7.164704416296e-05)in 117 secs on 20 cores > 8 Million Tets *2495* Iterations (KSP Residual norm > 9.101247550026e-05) in 225 secs on 160 cores > > So what other options should I try to improve solver performance? > Any tips/insights would be appreciated as preconditioning is black > magic to me. > > > For reports, always run with > > -ksp_view -ksp_monitor_true_residual -ksp_converged_reason > > so that we can see exactly what you used. > > I believe the default is a diagonal factorization. Since your outer > iterates are increasing, I would strengthen this > to either upper or full > > -pc_fieldsplit_schur_factorization_type > > Thanks, > > Matt > > Thanks in advance. > > Tabrez > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fs.png Type: image/png Size: 67833 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mumps.png Type: image/png Size: 67835 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: diff.png Type: image/png Size: 120316 bytes Desc: not available URL: -------------- next part -------------- Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... 0 KSP preconditioned resid norm 9.320065370611e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 7.598209387365e+05 true resid norm 5.578718188055e+15 ||r(i)||/||b|| 3.176949023291e+13 2 KSP preconditioned resid norm 4.421565330028e+05 true resid norm 1.261511053883e+16 ||r(i)||/||b|| 7.184009256973e+13 3 KSP preconditioned resid norm 3.187985857969e+05 true resid norm 1.576719836079e+16 ||r(i)||/||b|| 8.979049262535e+13 4 KSP preconditioned resid norm 2.009521750645e+05 true resid norm 2.176385620459e+16 ||r(i)||/||b|| 1.239400510681e+14 5 KSP preconditioned resid norm 1.299889680143e+05 true resid norm 2.634009228887e+16 ||r(i)||/||b|| 1.500006410965e+14 6 KSP preconditioned resid norm 1.070383574329e+05 true resid norm 2.973097994788e+16 ||r(i)||/||b|| 1.693109501554e+14 7 KSP preconditioned resid norm 9.206537842646e+04 true resid norm 3.369360846924e+16 ||r(i)||/||b|| 1.918771891842e+14 8 KSP preconditioned resid norm 8.698439688434e+04 true resid norm 3.820339778155e+16 ||r(i)||/||b|| 2.175593804476e+14 9 KSP preconditioned resid norm 8.674241865238e+04 true resid norm 3.980292133280e+16 ||r(i)||/||b|| 2.266682915138e+14 10 KSP preconditioned resid norm 7.632960547410e+04 true resid norm 2.687142759554e+16 ||r(i)||/||b|| 1.530264709138e+14 11 KSP preconditioned resid norm 5.727468611894e+04 true resid norm 1.290872179422e+16 ||r(i)||/||b|| 7.351213973112e+13 12 KSP preconditioned resid norm 3.934457968145e+04 true resid norm 1.087796093873e+16 ||r(i)||/||b|| 6.194743346900e+13 13 KSP preconditioned resid norm 2.986992755659e+04 true resid norm 1.239388097108e+16 ||r(i)||/||b|| 7.058024212471e+13 14 KSP preconditioned resid norm 2.633617864311e+04 true resid norm 1.387818241506e+16 ||r(i)||/||b|| 7.903299034349e+13 15 KSP preconditioned resid norm 2.607755764878e+04 true resid norm 1.450779964395e+16 ||r(i)||/||b|| 8.261851263185e+13 16 KSP preconditioned resid norm 2.488907351246e+04 true resid norm 1.254924272583e+16 ||r(i)||/||b|| 7.146499084009e+13 17 KSP preconditioned resid norm 2.128238344788e+04 true resid norm 9.013455625444e+15 ||r(i)||/||b|| 5.132951348401e+13 18 KSP preconditioned resid norm 1.564570272124e+04 true resid norm 6.297295328303e+15 ||r(i)||/||b|| 3.586161832921e+13 19 KSP preconditioned resid norm 1.136803646612e+04 true resid norm 5.940620356608e+15 ||r(i)||/||b|| 3.383043811045e+13 20 KSP preconditioned resid norm 9.660343914102e+03 true resid norm 6.514920106903e+15 ||r(i)||/||b|| 3.710094034640e+13 21 KSP preconditioned resid norm 9.527340557392e+03 true resid norm 6.873691439010e+15 ||r(i)||/||b|| 3.914405884549e+13 22 KSP preconditioned resid norm 9.297915692556e+03 true resid norm 6.142523083791e+15 ||r(i)||/||b|| 3.498022673627e+13 23 KSP preconditioned resid norm 7.560873143998e+03 true resid norm 4.245832085884e+15 ||r(i)||/||b|| 2.417901683435e+13 24 KSP preconditioned resid norm 5.774744623607e+03 true resid norm 3.220364170377e+15 ||r(i)||/||b|| 1.833921782897e+13 25 KSP preconditioned resid norm 4.673405304521e+03 true resid norm 3.255755445514e+15 ||r(i)||/||b|| 1.854076283123e+13 26 KSP preconditioned resid norm 4.616938981811e+03 true resid norm 3.371375029830e+15 ||r(i)||/||b|| 1.919918921715e+13 27 KSP preconditioned resid norm 4.393230879737e+03 true resid norm 2.975936771275e+15 ||r(i)||/||b|| 1.694726118111e+13 28 KSP preconditioned resid norm 3.661604054289e+03 true resid norm 2.134442107400e+15 ||r(i)||/||b|| 1.215514664802e+13 29 KSP preconditioned resid norm 2.469551153862e+03 true resid norm 1.723844957942e+15 ||r(i)||/||b|| 9.816892287486e+12 30 KSP preconditioned resid norm 1.864676099219e+03 true resid norm 1.854509458670e+15 ||r(i)||/||b|| 1.056099593993e+13 31 KSP preconditioned resid norm 1.851001600800e+03 true resid norm 1.866824774482e+15 ||r(i)||/||b|| 1.063112877192e+13 32 KSP preconditioned resid norm 1.844149161753e+03 true resid norm 1.884418345521e+15 ||r(i)||/||b|| 1.073132002812e+13 33 KSP preconditioned resid norm 1.842973224454e+03 true resid norm 1.870732971026e+15 ||r(i)||/||b|| 1.065338503362e+13 34 KSP preconditioned resid norm 1.818235744524e+03 true resid norm 1.788445711544e+15 ||r(i)||/||b|| 1.018477841140e+13 35 KSP preconditioned resid norm 1.761559804850e+03 true resid norm 1.638218046376e+15 ||r(i)||/||b|| 9.329267130782e+12 36 KSP preconditioned resid norm 1.588153941547e+03 true resid norm 1.424730688802e+15 ||r(i)||/||b|| 8.113506754893e+12 37 KSP preconditioned resid norm 1.424518979913e+03 true resid norm 1.382672406401e+15 ||r(i)||/||b|| 7.873994711636e+12 38 KSP preconditioned resid norm 1.222941115236e+03 true resid norm 1.391813525009e+15 ||r(i)||/||b|| 7.926051235834e+12 39 KSP preconditioned resid norm 1.216304102136e+03 true resid norm 1.414225227134e+15 ||r(i)||/||b|| 8.053680617307e+12 40 KSP preconditioned resid norm 1.210262839729e+03 true resid norm 1.378191008001e+15 ||r(i)||/||b|| 7.848474199957e+12 41 KSP preconditioned resid norm 1.139252076878e+03 true resid norm 1.193955392585e+15 ||r(i)||/||b|| 6.799295627526e+12 42 KSP preconditioned resid norm 9.842399783561e+02 true resid norm 1.005199677586e+15 ||r(i)||/||b|| 5.724376149269e+12 43 KSP preconditioned resid norm 8.134707710669e+02 true resid norm 8.874562331168e+14 ||r(i)||/||b|| 5.053854878439e+12 44 KSP preconditioned resid norm 7.163791581578e+02 true resid norm 8.803736812948e+14 ||r(i)||/||b|| 5.013521408751e+12 45 KSP preconditioned resid norm 6.959740549783e+02 true resid norm 9.128180126435e+14 ||r(i)||/||b|| 5.198284258056e+12 46 KSP preconditioned resid norm 6.925076307503e+02 true resid norm 8.868260369064e+14 ||r(i)||/||b|| 5.050266058986e+12 47 KSP preconditioned resid norm 6.105188458480e+02 true resid norm 7.548178094448e+14 ||r(i)||/||b|| 4.298510198297e+12 48 KSP preconditioned resid norm 4.950270624743e+02 true resid norm 6.166486109354e+14 ||r(i)||/||b|| 3.511669054048e+12 49 KSP preconditioned resid norm 4.124699026969e+02 true resid norm 6.073284333950e+14 ||r(i)||/||b|| 3.458592831274e+12 50 KSP preconditioned resid norm 4.124677149913e+02 true resid norm 6.074020817005e+14 ||r(i)||/||b|| 3.459012241083e+12 51 KSP preconditioned resid norm 3.994519765909e+02 true resid norm 5.804344525896e+14 ||r(i)||/||b|| 3.305437925126e+12 52 KSP preconditioned resid norm 3.402432312122e+02 true resid norm 5.274030089844e+14 ||r(i)||/||b|| 3.003436305245e+12 53 KSP preconditioned resid norm 2.584535957433e+02 true resid norm 4.948782512241e+14 ||r(i)||/||b|| 2.818215446409e+12 54 KSP preconditioned resid norm 1.942850945287e+02 true resid norm 4.921740353134e+14 ||r(i)||/||b|| 2.802815571731e+12 55 KSP preconditioned resid norm 1.779630713101e+02 true resid norm 5.001480907288e+14 ||r(i)||/||b|| 2.848225945064e+12 56 KSP preconditioned resid norm 1.699154044919e+02 true resid norm 5.152269207498e+14 ||r(i)||/||b|| 2.934096341618e+12 57 KSP preconditioned resid norm 1.695301455554e+02 true resid norm 5.102529272569e+14 ||r(i)||/||b|| 2.905770616539e+12 58 KSP preconditioned resid norm 1.506727480233e+02 true resid norm 4.745550585522e+14 ||r(i)||/||b|| 2.702479635901e+12 59 KSP preconditioned resid norm 1.241049799377e+02 true resid norm 4.557815556987e+14 ||r(i)||/||b|| 2.595568945051e+12 60 KSP preconditioned resid norm 1.157452709220e+02 true resid norm 4.599291905487e+14 ||r(i)||/||b|| 2.619188751683e+12 61 KSP preconditioned resid norm 1.157418104701e+02 true resid norm 4.599706702467e+14 ||r(i)||/||b|| 2.619424968824e+12 62 KSP preconditioned resid norm 1.157404801238e+02 true resid norm 4.600541744775e+14 ||r(i)||/||b|| 2.619900505813e+12 63 KSP preconditioned resid norm 1.156832341315e+02 true resid norm 4.591663122947e+14 ||r(i)||/||b|| 2.614844339146e+12 64 KSP preconditioned resid norm 1.145999125999e+02 true resid norm 4.536699590413e+14 ||r(i)||/||b|| 2.583543897877e+12 65 KSP preconditioned resid norm 1.099598785148e+02 true resid norm 4.391221073984e+14 ||r(i)||/||b|| 2.500697298515e+12 66 KSP preconditioned resid norm 1.059603110448e+02 true resid norm 4.326729019487e+14 ||r(i)||/||b|| 2.463970587712e+12 67 KSP preconditioned resid norm 1.029369313601e+02 true resid norm 4.363981976004e+14 ||r(i)||/||b|| 2.485185271773e+12 68 KSP preconditioned resid norm 1.022755200423e+02 true resid norm 4.410096530584e+14 ||r(i)||/||b|| 2.511446427865e+12 69 KSP preconditioned resid norm 1.022460253809e+02 true resid norm 4.393896292629e+14 ||r(i)||/||b|| 2.502220772721e+12 70 KSP preconditioned resid norm 1.016357044175e+02 true resid norm 4.324310211868e+14 ||r(i)||/||b|| 2.462593133565e+12 71 KSP preconditioned resid norm 9.780692861432e+01 true resid norm 4.097305174955e+14 ||r(i)||/||b|| 2.333319094979e+12 72 KSP preconditioned resid norm 9.293865814591e+01 true resid norm 3.901393764692e+14 ||r(i)||/||b|| 2.221752146711e+12 73 KSP preconditioned resid norm 9.019417440931e+01 true resid norm 3.836405236393e+14 ||r(i)||/||b|| 2.184742705736e+12 74 KSP preconditioned resid norm 9.018212875613e+01 true resid norm 3.834107933518e+14 ||r(i)||/||b|| 2.183434445687e+12 75 KSP preconditioned resid norm 8.651130972119e+01 true resid norm 3.642458442900e+14 ||r(i)||/||b|| 2.074294560590e+12 76 KSP preconditioned resid norm 7.823627730236e+01 true resid norm 3.277431679228e+14 ||r(i)||/||b|| 1.866420389278e+12 77 KSP preconditioned resid norm 6.932893314224e+01 true resid norm 2.981192488524e+14 ||r(i)||/||b|| 1.697719125683e+12 78 KSP preconditioned resid norm 6.107570235470e+01 true resid norm 2.806499604039e+14 ||r(i)||/||b|| 1.598235629648e+12 79 KSP preconditioned resid norm 6.048201170682e+01 true resid norm 2.801965706596e+14 ||r(i)||/||b|| 1.595653681508e+12 80 KSP preconditioned resid norm 5.979317931889e+01 true resid norm 2.763780718645e+14 ||r(i)||/||b|| 1.573908227429e+12 81 KSP preconditioned resid norm 5.471353874036e+01 true resid norm 2.520227859501e+14 ||r(i)||/||b|| 1.435210592615e+12 82 KSP preconditioned resid norm 4.706153739565e+01 true resid norm 2.295044563436e+14 ||r(i)||/||b|| 1.306973992669e+12 83 KSP preconditioned resid norm 3.871849733254e+01 true resid norm 2.134141927967e+14 ||r(i)||/||b|| 1.215343719662e+12 84 KSP preconditioned resid norm 3.746281494644e+01 true resid norm 2.143059839597e+14 ||r(i)||/||b|| 1.220422260949e+12 85 KSP preconditioned resid norm 3.746039295275e+01 true resid norm 2.144597193666e+14 ||r(i)||/||b|| 1.221297747995e+12 86 KSP preconditioned resid norm 3.542878682375e+01 true resid norm 2.066216918193e+14 ||r(i)||/||b|| 1.176662021432e+12 87 KSP preconditioned resid norm 3.020504109664e+01 true resid norm 1.951060346604e+14 ||r(i)||/||b|| 1.111083057717e+12 88 KSP preconditioned resid norm 2.595439447791e+01 true resid norm 1.917534876166e+14 ||r(i)||/||b|| 1.091991089460e+12 89 KSP preconditioned resid norm 2.428821413646e+01 true resid norm 1.922628864128e+14 ||r(i)||/||b|| 1.094891996000e+12 90 KSP preconditioned resid norm 2.414955045607e+01 true resid norm 1.934581432787e+14 ||r(i)||/||b|| 1.101698703212e+12 91 KSP preconditioned resid norm 2.414932681083e+01 true resid norm 1.934559392629e+14 ||r(i)||/||b|| 1.101686151860e+12 92 KSP preconditioned resid norm 2.413360156874e+01 true resid norm 1.933297573908e+14 ||r(i)||/||b|| 1.100967575725e+12 93 KSP preconditioned resid norm 2.395512978466e+01 true resid norm 1.922080715819e+14 ||r(i)||/||b|| 1.094579838408e+12 94 KSP preconditioned resid norm 2.343968200082e+01 true resid norm 1.900219483090e+14 ||r(i)||/||b|| 1.082130379657e+12 95 KSP preconditioned resid norm 2.314007567369e+01 true resid norm 1.894438367537e+14 ||r(i)||/||b|| 1.078838170087e+12 96 KSP preconditioned resid norm 2.254710649408e+01 true resid norm 1.891844338215e+14 ||r(i)||/||b|| 1.077360931294e+12 97 KSP preconditioned resid norm 2.212351459471e+01 true resid norm 1.902944578052e+14 ||r(i)||/||b|| 1.083682257254e+12 98 KSP preconditioned resid norm 2.211914933185e+01 true resid norm 1.904979142152e+14 ||r(i)||/||b|| 1.084840893739e+12 99 KSP preconditioned resid norm 2.206491519034e+01 true resid norm 1.894788993438e+14 ||r(i)||/||b|| 1.079037843305e+12 100 KSP preconditioned resid norm 2.200085815041e+01 true resid norm 1.888135932393e+14 ||r(i)||/||b|| 1.075249081250e+12 101 KSP preconditioned resid norm 2.166025138210e+01 true resid norm 1.881678157418e+14 ||r(i)||/||b|| 1.071571530026e+12 102 KSP preconditioned resid norm 2.165971165539e+01 true resid norm 1.881403751427e+14 ||r(i)||/||b|| 1.071415262257e+12 103 KSP preconditioned resid norm 2.154923648675e+01 true resid norm 1.868887015044e+14 ||r(i)||/||b|| 1.064287274772e+12 104 KSP preconditioned resid norm 2.082100706245e+01 true resid norm 1.804746120210e+14 ||r(i)||/||b|| 1.027760541152e+12 105 KSP preconditioned resid norm 1.950332973910e+01 true resid norm 1.686158168048e+14 ||r(i)||/||b|| 9.602274867663e+11 106 KSP preconditioned resid norm 1.803114022004e+01 true resid norm 1.585636396205e+14 ||r(i)||/||b|| 9.029826978900e+11 107 KSP preconditioned resid norm 1.605253698300e+01 true resid norm 1.488398939774e+14 ||r(i)||/||b|| 8.476082495268e+11 108 KSP preconditioned resid norm 1.538632041049e+01 true resid norm 1.488870188431e+14 ||r(i)||/||b|| 8.478766145727e+11 109 KSP preconditioned resid norm 1.535605105198e+01 true resid norm 1.479128152806e+14 ||r(i)||/||b|| 8.423287540207e+11 110 KSP preconditioned resid norm 1.491394079881e+01 true resid norm 1.431147368619e+14 ||r(i)||/||b|| 8.150048239855e+11 111 KSP preconditioned resid norm 1.421217237309e+01 true resid norm 1.373925750821e+14 ||r(i)||/||b|| 7.824184561770e+11 112 KSP preconditioned resid norm 1.379350871871e+01 true resid norm 1.354590413757e+14 ||r(i)||/||b|| 7.714074357009e+11 113 KSP preconditioned resid norm 1.363701465205e+01 true resid norm 1.355728607243e+14 ||r(i)||/||b|| 7.720556101670e+11 114 KSP preconditioned resid norm 1.358432683796e+01 true resid norm 1.344705964971e+14 ||r(i)||/||b|| 7.657784741979e+11 115 KSP preconditioned resid norm 1.281408111240e+01 true resid norm 1.271642070750e+14 ||r(i)||/||b|| 7.241703019333e+11 116 KSP preconditioned resid norm 1.196776579123e+01 true resid norm 1.189771770910e+14 ||r(i)||/||b|| 6.775470884375e+11 117 KSP preconditioned resid norm 1.172460878697e+01 true resid norm 1.163488892520e+14 ||r(i)||/||b|| 6.625796063003e+11 118 KSP preconditioned resid norm 1.160500437672e+01 true resid norm 1.153716120805e+14 ||r(i)||/||b|| 6.570142422670e+11 119 KSP preconditioned resid norm 1.149072500026e+01 true resid norm 1.138952152859e+14 ||r(i)||/||b|| 6.486065091702e+11 120 KSP preconditioned resid norm 1.087946473117e+01 true resid norm 1.072345585908e+14 ||r(i)||/||b|| 6.106756331724e+11 121 KSP preconditioned resid norm 1.074794214866e+01 true resid norm 1.074209221693e+14 ||r(i)||/||b|| 6.117369299946e+11 122 KSP preconditioned resid norm 1.063169350494e+01 true resid norm 1.077263723908e+14 ||r(i)||/||b|| 6.134763972886e+11 123 KSP preconditioned resid norm 1.051477175034e+01 true resid norm 1.083347699574e+14 ||r(i)||/||b|| 6.169410785822e+11 124 KSP preconditioned resid norm 1.051109251924e+01 true resid norm 1.082417770278e+14 ||r(i)||/||b|| 6.164115056826e+11 125 KSP preconditioned resid norm 1.042953620504e+01 true resid norm 1.080065382166e+14 ||r(i)||/||b|| 6.150718758852e+11 126 KSP preconditioned resid norm 1.033579620509e+01 true resid norm 1.078881131768e+14 ||r(i)||/||b|| 6.143974730891e+11 127 KSP preconditioned resid norm 1.017473978142e+01 true resid norm 1.080368281701e+14 ||r(i)||/||b|| 6.152443700585e+11 128 KSP preconditioned resid norm 9.978991301545e+00 true resid norm 1.084748899689e+14 ||r(i)||/||b|| 6.177390291483e+11 129 KSP preconditioned resid norm 9.973075892910e+00 true resid norm 1.086655770735e+14 ||r(i)||/||b|| 6.188249474372e+11 130 KSP preconditioned resid norm 9.925914410792e+00 true resid norm 1.077858192957e+14 ||r(i)||/||b|| 6.138149334544e+11 131 KSP preconditioned resid norm 9.846088812169e+00 true resid norm 1.066953000922e+14 ||r(i)||/||b|| 6.076046826376e+11 132 KSP preconditioned resid norm 9.685628630969e+00 true resid norm 1.059510928093e+14 ||r(i)||/||b|| 6.033665968967e+11 133 KSP preconditioned resid norm 9.649878409604e+00 true resid norm 1.059554270596e+14 ||r(i)||/||b|| 6.033912794341e+11 134 KSP preconditioned resid norm 9.621668875206e+00 true resid norm 1.055880879204e+14 ||r(i)||/||b|| 6.012993692855e+11 135 KSP preconditioned resid norm 9.504075017228e+00 true resid norm 1.041641242658e+14 ||r(i)||/||b|| 5.931902306100e+11 136 KSP preconditioned resid norm 9.301615805096e+00 true resid norm 1.011844456450e+14 ||r(i)||/||b|| 5.762216604747e+11 137 KSP preconditioned resid norm 8.498850795914e+00 true resid norm 9.411390051693e+13 ||r(i)||/||b|| 5.359565660899e+11 138 KSP preconditioned resid norm 7.611153937794e+00 true resid norm 8.664854830227e+13 ||r(i)||/||b|| 4.934431380454e+11 139 KSP preconditioned resid norm 7.367853222946e+00 true resid norm 8.514247469367e+13 ||r(i)||/||b|| 4.848664024610e+11 140 KSP preconditioned resid norm 7.356416093666e+00 true resid norm 8.543337420511e+13 ||r(i)||/||b|| 4.865230068773e+11 141 KSP preconditioned resid norm 7.262323793085e+00 true resid norm 8.350503748081e+13 ||r(i)||/||b|| 4.755415819938e+11 142 KSP preconditioned resid norm 6.818058824121e+00 true resid norm 7.723259625494e+13 ||r(i)||/||b|| 4.398215019423e+11 143 KSP preconditioned resid norm 6.387829564818e+00 true resid norm 7.318686364564e+13 ||r(i)||/||b|| 4.167820046450e+11 144 KSP preconditioned resid norm 6.268529882965e+00 true resid norm 7.288436210329e+13 ||r(i)||/||b|| 4.150593293868e+11 145 KSP preconditioned resid norm 6.219755602331e+00 true resid norm 7.347711605777e+13 ||r(i)||/||b|| 4.184349239827e+11 146 KSP preconditioned resid norm 6.212812254673e+00 true resid norm 7.301084654934e+13 ||r(i)||/||b|| 4.157796286093e+11 147 KSP preconditioned resid norm 5.895208085079e+00 true resid norm 6.729358721321e+13 ||r(i)||/||b|| 3.832211790667e+11 148 KSP preconditioned resid norm 5.543361227376e+00 true resid norm 6.302122535288e+13 ||r(i)||/||b|| 3.588910813959e+11 149 KSP preconditioned resid norm 5.489228291667e+00 true resid norm 6.207043195385e+13 ||r(i)||/||b|| 3.534765362288e+11 150 KSP preconditioned resid norm 5.456424777932e+00 true resid norm 6.203367301621e+13 ||r(i)||/||b|| 3.532672027097e+11 151 KSP preconditioned resid norm 5.456282623298e+00 true resid norm 6.203556060285e+13 ||r(i)||/||b|| 3.532779520724e+11 152 KSP preconditioned resid norm 5.455815290491e+00 true resid norm 6.204881883276e+13 ||r(i)||/||b|| 3.533534545788e+11 153 KSP preconditioned resid norm 5.455300501013e+00 true resid norm 6.202129931661e+13 ||r(i)||/||b|| 3.531967373957e+11 154 KSP preconditioned resid norm 5.435005246052e+00 true resid norm 6.176758564172e+13 ||r(i)||/||b|| 3.517518975876e+11 155 KSP preconditioned resid norm 5.413728418086e+00 true resid norm 6.164123319279e+13 ||r(i)||/||b|| 3.510323500577e+11 156 KSP preconditioned resid norm 5.395209576488e+00 true resid norm 6.161241527836e+13 ||r(i)||/||b|| 3.508682388013e+11 157 KSP preconditioned resid norm 5.380144533844e+00 true resid norm 6.170074608813e+13 ||r(i)||/||b|| 3.513712620234e+11 158 KSP preconditioned resid norm 5.377778467964e+00 true resid norm 6.178232804808e+13 ||r(i)||/||b|| 3.518358521304e+11 159 KSP preconditioned resid norm 5.372262919685e+00 true resid norm 6.159979045041e+13 ||r(i)||/||b|| 3.507963433704e+11 160 KSP preconditioned resid norm 5.340801888965e+00 true resid norm 6.112269046766e+13 ||r(i)||/||b|| 3.480793709887e+11 161 KSP preconditioned resid norm 5.311357296075e+00 true resid norm 6.090585247883e+13 ||r(i)||/||b|| 3.468445295545e+11 162 KSP preconditioned resid norm 5.303655281205e+00 true resid norm 6.089389387132e+13 ||r(i)||/||b|| 3.467764280925e+11 163 KSP preconditioned resid norm 5.300756842884e+00 true resid norm 6.081317522764e+13 ||r(i)||/||b|| 3.463167543690e+11 164 KSP preconditioned resid norm 5.268930132472e+00 true resid norm 6.021677268842e+13 ||r(i)||/||b|| 3.429203819397e+11 165 KSP preconditioned resid norm 5.168338966645e+00 true resid norm 5.870471746945e+13 ||r(i)||/||b|| 3.343095824888e+11 166 KSP preconditioned resid norm 4.952356457960e+00 true resid norm 5.563076070177e+13 ||r(i)||/||b|| 3.168041204426e+11 167 KSP preconditioned resid norm 4.591559968629e+00 true resid norm 5.233720146881e+13 ||r(i)||/||b|| 2.980480739180e+11 168 KSP preconditioned resid norm 4.221828217786e+00 true resid norm 5.009805509813e+13 ||r(i)||/||b|| 2.852966610745e+11 169 KSP preconditioned resid norm 4.208640761234e+00 true resid norm 5.020853311352e+13 ||r(i)||/||b|| 2.859258074326e+11 170 KSP preconditioned resid norm 4.190273969279e+00 true resid norm 4.967977806657e+13 ||r(i)||/||b|| 2.829146715886e+11 171 KSP preconditioned resid norm 4.082670968472e+00 true resid norm 4.756743771668e+13 ||r(i)||/||b|| 2.708853892602e+11 172 KSP preconditioned resid norm 3.997387914753e+00 true resid norm 4.629198001224e+13 ||r(i)||/||b|| 2.636219571030e+11 173 KSP preconditioned resid norm 3.965416363598e+00 true resid norm 4.619872707479e+13 ||r(i)||/||b|| 2.630909035193e+11 174 KSP preconditioned resid norm 3.961972987563e+00 true resid norm 4.628519269197e+13 ||r(i)||/||b|| 2.635833049076e+11 175 KSP preconditioned resid norm 3.921773991599e+00 true resid norm 4.530870796504e+13 ||r(i)||/||b|| 2.580224536602e+11 176 KSP preconditioned resid norm 3.865670603577e+00 true resid norm 4.336427254300e+13 ||r(i)||/||b|| 2.469493504730e+11 177 KSP preconditioned resid norm 3.663287778440e+00 true resid norm 3.731701272473e+13 ||r(i)||/||b|| 2.125116256666e+11 178 KSP preconditioned resid norm 3.620550165055e+00 true resid norm 3.547052820671e+13 ||r(i)||/||b|| 2.019963298795e+11 179 KSP preconditioned resid norm 3.618852985809e+00 true resid norm 3.549804351278e+13 ||r(i)||/||b|| 2.021530230872e+11 180 KSP preconditioned resid norm 3.492841858242e+00 true resid norm 3.389503691139e+13 ||r(i)||/||b|| 1.930242768681e+11 181 KSP preconditioned resid norm 3.483273199644e+00 true resid norm 3.391272603402e+13 ||r(i)||/||b|| 1.931250122682e+11 182 KSP preconditioned resid norm 3.458294068584e+00 true resid norm 3.397869164162e+13 ||r(i)||/||b|| 1.935006709152e+11 183 KSP preconditioned resid norm 3.443080582725e+00 true resid norm 3.408537507902e+13 ||r(i)||/||b|| 1.941082080426e+11 184 KSP preconditioned resid norm 3.439355339706e+00 true resid norm 3.404151708270e+13 ||r(i)||/||b|| 1.938584470511e+11 185 KSP preconditioned resid norm 3.412630761810e+00 true resid norm 3.398223104036e+13 ||r(i)||/||b|| 1.935208269600e+11 186 KSP preconditioned resid norm 3.373644190818e+00 true resid norm 3.396910437454e+13 ||r(i)||/||b|| 1.934460736802e+11 187 KSP preconditioned resid norm 3.341241746072e+00 true resid norm 3.403635561711e+13 ||r(i)||/||b|| 1.938290537164e+11 188 KSP preconditioned resid norm 3.291217089175e+00 true resid norm 3.420350500699e+13 ||r(i)||/||b|| 1.947809302462e+11 189 KSP preconditioned resid norm 3.285720347805e+00 true resid norm 3.431261986340e+13 ||r(i)||/||b|| 1.954023137340e+11 190 KSP preconditioned resid norm 3.280573435762e+00 true resid norm 3.418003609744e+13 ||r(i)||/||b|| 1.946472803167e+11 191 KSP preconditioned resid norm 3.236796260449e+00 true resid norm 3.377333811494e+13 ||r(i)||/||b|| 1.923312307964e+11 192 KSP preconditioned resid norm 3.198631309038e+00 true resid norm 3.354822938074e+13 ||r(i)||/||b|| 1.910492894093e+11 193 KSP preconditioned resid norm 3.182744514614e+00 true resid norm 3.354452935348e+13 ||r(i)||/||b|| 1.910282186228e+11 194 KSP preconditioned resid norm 3.182670403663e+00 true resid norm 3.353478639460e+13 ||r(i)||/||b|| 1.909727347596e+11 195 KSP preconditioned resid norm 3.169124704728e+00 true resid norm 3.325640645399e+13 ||r(i)||/||b|| 1.893874263596e+11 196 KSP preconditioned resid norm 3.106971342375e+00 true resid norm 3.241017978762e+13 ||r(i)||/||b|| 1.845683641834e+11 197 KSP preconditioned resid norm 2.899970196648e+00 true resid norm 3.036352338172e+13 ||r(i)||/||b|| 1.729131364939e+11 198 KSP preconditioned resid norm 2.475278079870e+00 true resid norm 2.715950936084e+13 ||r(i)||/||b|| 1.546670289274e+11 199 KSP preconditioned resid norm 2.381047637607e+00 true resid norm 2.673407165239e+13 ||r(i)||/||b|| 1.522442610679e+11 200 KSP preconditioned resid norm 2.373601090909e+00 true resid norm 2.682709903320e+13 ||r(i)||/||b|| 1.527740301594e+11 201 KSP preconditioned resid norm 2.337491343158e+00 true resid norm 2.619534007312e+13 ||r(i)||/||b|| 1.491763112148e+11 202 KSP preconditioned resid norm 2.267811568073e+00 true resid norm 2.502440072458e+13 ||r(i)||/||b|| 1.425080865540e+11 203 KSP preconditioned resid norm 2.098201534268e+00 true resid norm 2.294623148079e+13 ||r(i)||/||b|| 1.306734006517e+11 204 KSP preconditioned resid norm 2.008406927238e+00 true resid norm 2.234753170652e+13 ||r(i)||/||b|| 1.272639460082e+11 205 KSP preconditioned resid norm 2.008367307117e+00 true resid norm 2.236220589178e+13 ||r(i)||/||b|| 1.273475120479e+11 206 KSP preconditioned resid norm 1.991008441854e+00 true resid norm 2.175441994797e+13 ||r(i)||/||b|| 1.238863138022e+11 207 KSP preconditioned resid norm 1.940230234873e+00 true resid norm 2.011212106819e+13 ||r(i)||/||b|| 1.145338072834e+11 208 KSP preconditioned resid norm 1.881361213794e+00 true resid norm 1.853845416809e+13 ||r(i)||/||b|| 1.055721437744e+11 209 KSP preconditioned resid norm 1.818987851735e+00 true resid norm 1.734967209912e+13 ||r(i)||/||b|| 9.880230901022e+10 210 KSP preconditioned resid norm 1.814368351426e+00 true resid norm 1.725426200733e+13 ||r(i)||/||b|| 9.825897093916e+10 211 KSP preconditioned resid norm 1.814334032599e+00 true resid norm 1.725466394666e+13 ||r(i)||/||b|| 9.826125988926e+10 212 KSP preconditioned resid norm 1.814333658314e+00 true resid norm 1.725449518784e+13 ||r(i)||/||b|| 9.826029884741e+10 213 KSP preconditioned resid norm 1.812072817313e+00 true resid norm 1.722508611451e+13 ||r(i)||/||b|| 9.809282107988e+10 214 KSP preconditioned resid norm 1.787353928634e+00 true resid norm 1.704813258919e+13 ||r(i)||/||b|| 9.708511230077e+10 215 KSP preconditioned resid norm 1.772406155146e+00 true resid norm 1.696458453631e+13 ||r(i)||/||b|| 9.660932575618e+10 216 KSP preconditioned resid norm 1.758905851137e+00 true resid norm 1.694281157824e+13 ||r(i)||/||b|| 9.648533387212e+10 217 KSP preconditioned resid norm 1.748259137178e+00 true resid norm 1.698850218770e+13 ||r(i)||/||b|| 9.674553116511e+10 218 KSP preconditioned resid norm 1.742640027818e+00 true resid norm 1.704790081204e+13 ||r(i)||/||b|| 9.708379238431e+10 219 KSP preconditioned resid norm 1.737415933335e+00 true resid norm 1.697427566935e+13 ||r(i)||/||b|| 9.666451448338e+10 220 KSP preconditioned resid norm 1.699525583191e+00 true resid norm 1.674163519249e+13 ||r(i)||/||b|| 9.533968159024e+10 221 KSP preconditioned resid norm 1.667463334923e+00 true resid norm 1.659971917358e+13 ||r(i)||/||b|| 9.453150318351e+10 222 KSP preconditioned resid norm 1.658273465729e+00 true resid norm 1.658030818430e+13 ||r(i)||/||b|| 9.442096215713e+10 223 KSP preconditioned resid norm 1.657955652749e+00 true resid norm 1.657234282413e+13 ||r(i)||/||b|| 9.437560130119e+10 224 KSP preconditioned resid norm 1.632232987105e+00 true resid norm 1.630247760133e+13 ||r(i)||/||b|| 9.283878222003e+10 225 KSP preconditioned resid norm 1.553189774944e+00 true resid norm 1.571555265526e+13 ||r(i)||/||b|| 8.949638245843e+10 226 KSP preconditioned resid norm 1.472303037963e+00 true resid norm 1.498150510143e+13 ||r(i)||/||b|| 8.531615398912e+10 227 KSP preconditioned resid norm 1.406548006646e+00 true resid norm 1.447030370218e+13 ||r(i)||/||b|| 8.240498204730e+10 228 KSP preconditioned resid norm 1.311700750324e+00 true resid norm 1.404364933200e+13 ||r(i)||/||b|| 7.997528558490e+10 229 KSP preconditioned resid norm 1.306768403545e+00 true resid norm 1.409383532280e+13 ||r(i)||/||b|| 8.026108301912e+10 230 KSP preconditioned resid norm 1.306672220237e+00 true resid norm 1.408440567178e+13 ||r(i)||/||b|| 8.020738337060e+10 231 KSP preconditioned resid norm 1.279457133369e+00 true resid norm 1.359379126564e+13 ||r(i)||/||b|| 7.741344952082e+10 232 KSP preconditioned resid norm 1.250557795254e+00 true resid norm 1.322875871324e+13 ||r(i)||/||b|| 7.533467484228e+10 233 KSP preconditioned resid norm 1.236286846995e+00 true resid norm 1.321879593336e+13 ||r(i)||/||b|| 7.527793915005e+10 234 KSP preconditioned resid norm 1.233526171709e+00 true resid norm 1.328960566546e+13 ||r(i)||/||b|| 7.568118394865e+10 235 KSP preconditioned resid norm 1.225207207311e+00 true resid norm 1.298283046990e+13 ||r(i)||/||b|| 7.393417123888e+10 236 KSP preconditioned resid norm 1.194584730352e+00 true resid norm 1.216602933444e+13 ||r(i)||/||b|| 6.928268055222e+10 237 KSP preconditioned resid norm 1.055836253474e+00 true resid norm 9.652620499673e+12 ||r(i)||/||b|| 5.496940737084e+10 238 KSP preconditioned resid norm 8.950095785846e-01 true resid norm 7.078720720300e+12 ||r(i)||/||b|| 4.031165246285e+10 239 KSP preconditioned resid norm 8.499727448648e-01 true resid norm 6.438925183523e+12 ||r(i)||/||b|| 3.666816710089e+10 240 KSP preconditioned resid norm 8.490505379884e-01 true resid norm 6.374770564367e+12 ||r(i)||/||b|| 3.630282160789e+10 241 KSP preconditioned resid norm 8.490155599911e-01 true resid norm 6.374362814172e+12 ||r(i)||/||b|| 3.630049956627e+10 242 KSP preconditioned resid norm 8.479181382217e-01 true resid norm 6.367617209195e+12 ||r(i)||/||b|| 3.626208492975e+10 243 KSP preconditioned resid norm 8.399683186329e-01 true resid norm 6.335651027393e+12 ||r(i)||/||b|| 3.608004502985e+10 244 KSP preconditioned resid norm 8.201490861141e-01 true resid norm 6.287111239865e+12 ||r(i)||/||b|| 3.580362233672e+10 245 KSP preconditioned resid norm 7.845782237892e-01 true resid norm 6.240387372378e+12 ||r(i)||/||b|| 3.553754088185e+10 246 KSP preconditioned resid norm 7.583413002101e-01 true resid norm 6.229629627639e+12 ||r(i)||/||b|| 3.547627805141e+10 247 KSP preconditioned resid norm 7.426443902980e-01 true resid norm 6.249110858478e+12 ||r(i)||/||b|| 3.558721908697e+10 248 KSP preconditioned resid norm 7.239981008121e-01 true resid norm 6.284768340145e+12 ||r(i)||/||b|| 3.579028007291e+10 249 KSP preconditioned resid norm 7.231356445863e-01 true resid norm 6.267104141391e+12 ||r(i)||/||b|| 3.568968660845e+10 250 KSP preconditioned resid norm 7.123199648891e-01 true resid norm 6.182108768233e+12 ||r(i)||/||b|| 3.520565791469e+10 251 KSP preconditioned resid norm 6.894615267830e-01 true resid norm 6.100314804214e+12 ||r(i)||/||b|| 3.473986049432e+10 252 KSP preconditioned resid norm 6.499416617879e-01 true resid norm 6.057131987298e+12 ||r(i)||/||b|| 3.449394449103e+10 253 KSP preconditioned resid norm 6.318064959724e-01 true resid norm 6.075773341033e+12 ||r(i)||/||b|| 3.460010262368e+10 254 KSP preconditioned resid norm 6.290410832839e-01 true resid norm 6.098647408420e+12 ||r(i)||/||b|| 3.473036506676e+10 255 KSP preconditioned resid norm 6.281199724722e-01 true resid norm 6.071168322234e+12 ||r(i)||/||b|| 3.457387812285e+10 256 KSP preconditioned resid norm 6.260852144562e-01 true resid norm 6.001762104507e+12 ||r(i)||/||b|| 3.417862600904e+10 257 KSP preconditioned resid norm 6.156249484581e-01 true resid norm 5.841183661090e+12 ||r(i)||/||b|| 3.326416947659e+10 258 KSP preconditioned resid norm 6.019606259192e-01 true resid norm 5.740049175207e+12 ||r(i)||/||b|| 3.268823232523e+10 259 KSP preconditioned resid norm 6.003295317810e-01 true resid norm 5.738390815065e+12 ||r(i)||/||b|| 3.267878835360e+10 260 KSP preconditioned resid norm 5.962576894195e-01 true resid norm 5.671200047156e+12 ||r(i)||/||b|| 3.229615270633e+10 261 KSP preconditioned resid norm 5.739594336100e-01 true resid norm 5.386515898251e+12 ||r(i)||/||b|| 3.067494332037e+10 262 KSP preconditioned resid norm 5.573547043016e-01 true resid norm 5.191444121961e+12 ||r(i)||/||b|| 2.956405535602e+10 263 KSP preconditioned resid norm 5.497184302957e-01 true resid norm 5.097024254552e+12 ||r(i)||/||b|| 2.902635638033e+10 264 KSP preconditioned resid norm 5.497169262901e-01 true resid norm 5.096895076884e+12 ||r(i)||/||b|| 2.902562074383e+10 265 KSP preconditioned resid norm 5.398225274240e-01 true resid norm 4.973313754702e+12 ||r(i)||/||b|| 2.832185413013e+10 266 KSP preconditioned resid norm 4.973989554380e-01 true resid norm 4.546698990109e+12 ||r(i)||/||b|| 2.589238321224e+10 267 KSP preconditioned resid norm 4.416719104316e-01 true resid norm 4.154824042541e+12 ||r(i)||/||b|| 2.366074739563e+10 268 KSP preconditioned resid norm 4.113904926086e-01 true resid norm 3.978962447066e+12 ||r(i)||/||b|| 2.265925690059e+10 269 KSP preconditioned resid norm 4.064758686788e-01 true resid norm 3.987822324925e+12 ||r(i)||/||b|| 2.270971182476e+10 270 KSP preconditioned resid norm 3.990693160971e-01 true resid norm 3.843972127868e+12 ||r(i)||/||b|| 2.189051872764e+10 271 KSP preconditioned resid norm 3.987547800016e-01 true resid norm 3.845074987305e+12 ||r(i)||/||b|| 2.189679925320e+10 272 KSP preconditioned resid norm 3.982245631291e-01 true resid norm 3.848886580064e+12 ||r(i)||/||b|| 2.191850537902e+10 273 KSP preconditioned resid norm 3.981178360490e-01 true resid norm 3.845551123024e+12 ||r(i)||/||b|| 2.189951073432e+10 274 KSP preconditioned resid norm 3.960201316015e-01 true resid norm 3.827452546808e+12 ||r(i)||/||b|| 2.179644359220e+10 275 KSP preconditioned resid norm 3.918539460715e-01 true resid norm 3.817887964663e+12 ||r(i)||/||b|| 2.174197554259e+10 276 KSP preconditioned resid norm 3.851497613516e-01 true resid norm 3.818769804850e+12 ||r(i)||/||b|| 2.174699741541e+10 277 KSP preconditioned resid norm 3.769933805891e-01 true resid norm 3.845348865615e+12 ||r(i)||/||b|| 2.189835892586e+10 278 KSP preconditioned resid norm 3.746747135621e-01 true resid norm 3.861211341991e+12 ||r(i)||/||b|| 2.198869200442e+10 279 KSP preconditioned resid norm 3.746514736211e-01 true resid norm 3.863839236316e+12 ||r(i)||/||b|| 2.200365724558e+10 280 KSP preconditioned resid norm 3.745676144761e-01 true resid norm 3.855869072468e+12 ||r(i)||/||b|| 2.195826903381e+10 281 KSP preconditioned resid norm 3.739105027901e-01 true resid norm 3.833114749091e+12 ||r(i)||/||b|| 2.182868850475e+10 282 KSP preconditioned resid norm 3.732053309445e-01 true resid norm 3.820531574269e+12 ||r(i)||/||b|| 2.175703027859e+10 283 KSP preconditioned resid norm 3.731393578798e-01 true resid norm 3.820430278456e+12 ||r(i)||/||b|| 2.175645342272e+10 284 KSP preconditioned resid norm 3.719691878815e-01 true resid norm 3.799470960785e+12 ||r(i)||/||b|| 2.163709502970e+10 285 KSP preconditioned resid norm 3.650402972032e-01 true resid norm 3.718802687331e+12 ||r(i)||/||b|| 2.117770815279e+10 286 KSP preconditioned resid norm 3.536736074421e-01 true resid norm 3.566057064093e+12 ||r(i)||/||b|| 2.030785769216e+10 287 KSP preconditioned resid norm 3.372751368782e-01 true resid norm 3.375927662254e+12 ||r(i)||/||b|| 1.922511538988e+10 288 KSP preconditioned resid norm 3.230172048493e-01 true resid norm 3.241991247938e+12 ||r(i)||/||b|| 1.846237895778e+10 289 KSP preconditioned resid norm 3.222411596715e-01 true resid norm 3.240545803789e+12 ||r(i)||/||b|| 1.845414749274e+10 290 KSP preconditioned resid norm 3.158561073802e-01 true resid norm 3.164282783097e+12 ||r(i)||/||b|| 1.801984749597e+10 291 KSP preconditioned resid norm 2.947057384327e-01 true resid norm 2.935285782195e+12 ||r(i)||/||b|| 1.671576334289e+10 292 KSP preconditioned resid norm 2.734969937071e-01 true resid norm 2.773169304468e+12 ||r(i)||/||b|| 1.579254806617e+10 293 KSP preconditioned resid norm 2.629446898565e-01 true resid norm 2.710328558300e+12 ||r(i)||/||b|| 1.543468477136e+10 294 KSP preconditioned resid norm 2.628264610040e-01 true resid norm 2.704340727815e+12 ||r(i)||/||b|| 1.540058548265e+10 295 KSP preconditioned resid norm 2.620627405448e-01 true resid norm 2.664839154592e+12 ||r(i)||/||b|| 1.517563329787e+10 296 KSP preconditioned resid norm 2.560612716841e-01 true resid norm 2.518533740609e+12 ||r(i)||/||b|| 1.434245831683e+10 297 KSP preconditioned resid norm 2.453412231699e-01 true resid norm 2.334194230408e+12 ||r(i)||/||b|| 1.329268808800e+10 298 KSP preconditioned resid norm 2.349259152821e-01 true resid norm 2.184039546191e+12 ||r(i)||/||b|| 1.243759241676e+10 299 KSP preconditioned resid norm 2.325739406772e-01 true resid norm 2.183091330527e+12 ||r(i)||/||b|| 1.243219255119e+10 300 KSP preconditioned resid norm 2.291034812563e-01 true resid norm 2.088135923521e+12 ||r(i)||/||b|| 1.189144380323e+10 301 KSP preconditioned resid norm 2.289724019368e-01 true resid norm 2.088726131934e+12 ||r(i)||/||b|| 1.189480490156e+10 302 KSP preconditioned resid norm 2.287733145969e-01 true resid norm 2.090621416632e+12 ||r(i)||/||b|| 1.190559810292e+10 303 KSP preconditioned resid norm 2.287724739254e-01 true resid norm 2.090870006058e+12 ||r(i)||/||b|| 1.190701376133e+10 304 KSP preconditioned resid norm 2.278587818883e-01 true resid norm 2.080780944956e+12 ||r(i)||/||b|| 1.184955892720e+10 305 KSP preconditioned resid norm 2.266815948822e-01 true resid norm 2.075322320318e+12 ||r(i)||/||b|| 1.181847334153e+10 306 KSP preconditioned resid norm 2.256604950708e-01 true resid norm 2.074458496400e+12 ||r(i)||/||b|| 1.181355406715e+10 307 KSP preconditioned resid norm 2.239969713158e-01 true resid norm 2.080233402812e+12 ||r(i)||/||b|| 1.184644080325e+10 308 KSP preconditioned resid norm 2.237609152050e-01 true resid norm 2.084727808526e+12 ||r(i)||/||b|| 1.187203538853e+10 309 KSP preconditioned resid norm 2.221927758302e-01 true resid norm 2.068966584455e+12 ||r(i)||/||b|| 1.178227891809e+10 310 KSP preconditioned resid norm 2.165508046426e-01 true resid norm 2.030450736182e+12 ||r(i)||/||b|| 1.156294020545e+10 311 KSP preconditioned resid norm 2.112743051605e-01 true resid norm 2.000408705768e+12 ||r(i)||/||b|| 1.139185789592e+10 312 KSP preconditioned resid norm 2.064851502231e-01 true resid norm 1.987437829130e+12 ||r(i)||/||b|| 1.131799179895e+10 313 KSP preconditioned resid norm 2.060578011406e-01 true resid norm 1.990929196066e+12 ||r(i)||/||b|| 1.133787431390e+10 314 KSP preconditioned resid norm 2.055882045755e-01 true resid norm 1.981172652007e+12 ||r(i)||/||b|| 1.128231308626e+10 315 KSP preconditioned resid norm 2.028523750493e-01 true resid norm 1.947931788744e+12 ||r(i)||/||b|| 1.109301417472e+10 316 KSP preconditioned resid norm 1.957069429061e-01 true resid norm 1.861769885111e+12 ||r(i)||/||b|| 1.060234236380e+10 317 KSP preconditioned resid norm 1.864007939037e-01 true resid norm 1.759741316360e+12 ||r(i)||/||b|| 1.002131362044e+10 318 KSP preconditioned resid norm 1.681828657744e-01 true resid norm 1.648328560641e+12 ||r(i)||/||b|| 9.386844135635e+09 319 KSP preconditioned resid norm 1.625283942858e-01 true resid norm 1.631646648980e+12 ||r(i)||/||b|| 9.291844565535e+09 320 KSP preconditioned resid norm 1.625253419828e-01 true resid norm 1.632330176614e+12 ||r(i)||/||b|| 9.295737094919e+09 321 KSP preconditioned resid norm 1.565154348215e-01 true resid norm 1.539940557784e+12 ||r(i)||/||b|| 8.769599908190e+09 322 KSP preconditioned resid norm 1.495303413399e-01 true resid norm 1.455697666554e+12 ||r(i)||/||b|| 8.289856422338e+09 323 KSP preconditioned resid norm 1.444974066388e-01 true resid norm 1.421272621850e+12 ||r(i)||/||b|| 8.093813875531e+09 324 KSP preconditioned resid norm 1.444499456346e-01 true resid norm 1.423028346899e+12 ||r(i)||/||b|| 8.103812317454e+09 325 KSP preconditioned resid norm 1.443431297858e-01 true resid norm 1.414838877588e+12 ||r(i)||/||b|| 8.057175212565e+09 326 KSP preconditioned resid norm 1.418578453254e-01 true resid norm 1.359425527665e+12 ||r(i)||/||b|| 7.741609195460e+09 327 KSP preconditioned resid norm 1.366226936152e-01 true resid norm 1.296874240743e+12 ||r(i)||/||b|| 7.385394303088e+09 328 KSP preconditioned resid norm 1.343869922669e-01 true resid norm 1.269246613426e+12 ||r(i)||/||b|| 7.228061452312e+09 329 KSP preconditioned resid norm 1.343570681174e-01 true resid norm 1.268946908057e+12 ||r(i)||/||b|| 7.226354700605e+09 330 KSP preconditioned resid norm 1.310684436499e-01 true resid norm 1.228432967689e+12 ||r(i)||/||b|| 6.995637322625e+09 331 KSP preconditioned resid norm 1.309283228701e-01 true resid norm 1.228865427370e+12 ||r(i)||/||b|| 6.998100078972e+09 332 KSP preconditioned resid norm 1.304417134357e-01 true resid norm 1.231196536677e+12 ||r(i)||/||b|| 7.011375199143e+09 333 KSP preconditioned resid norm 1.303184797096e-01 true resid norm 1.233377164660e+12 ||r(i)||/||b|| 7.023793363511e+09 334 KSP preconditioned resid norm 1.302475771702e-01 true resid norm 1.231184222878e+12 ||r(i)||/||b|| 7.011305074948e+09 335 KSP preconditioned resid norm 1.299484489875e-01 true resid norm 1.227800246914e+12 ||r(i)||/||b|| 6.992034126371e+09 336 KSP preconditioned resid norm 1.292404070619e-01 true resid norm 1.225601958757e+12 ||r(i)||/||b|| 6.979515391460e+09 337 KSP preconditioned resid norm 1.282395225865e-01 true resid norm 1.226641565260e+12 ||r(i)||/||b|| 6.985435706403e+09 338 KSP preconditioned resid norm 1.276900923348e-01 true resid norm 1.230586381170e+12 ||r(i)||/||b|| 7.007900506793e+09 339 KSP preconditioned resid norm 1.274836319479e-01 true resid norm 1.226646390544e+12 ||r(i)||/||b|| 6.985463185261e+09 340 KSP preconditioned resid norm 1.261656565290e-01 true resid norm 1.212477403309e+12 ||r(i)||/||b|| 6.904774129749e+09 341 KSP preconditioned resid norm 1.239728309259e-01 true resid norm 1.189991737055e+12 ||r(i)||/||b|| 6.776723539929e+09 342 KSP preconditioned resid norm 1.214572533517e-01 true resid norm 1.170727280155e+12 ||r(i)||/||b|| 6.667016981053e+09 343 KSP preconditioned resid norm 1.209306202662e-01 true resid norm 1.168820899148e+12 ||r(i)||/||b|| 6.656160588828e+09 344 KSP preconditioned resid norm 1.205893423247e-01 true resid norm 1.164236698766e+12 ||r(i)||/||b|| 6.630054643990e+09 345 KSP preconditioned resid norm 1.189186873944e-01 true resid norm 1.146530969947e+12 ||r(i)||/||b|| 6.529224675560e+09 346 KSP preconditioned resid norm 1.125140451701e-01 true resid norm 1.081660405988e+12 ||r(i)||/||b|| 6.159802045017e+09 347 KSP preconditioned resid norm 1.024882580434e-01 true resid norm 1.006164462345e+12 ||r(i)||/||b|| 5.729870371944e+09 348 KSP preconditioned resid norm 9.422132585451e-02 true resid norm 9.769389290720e+11 ||r(i)||/||b|| 5.563437821927e+09 349 KSP preconditioned resid norm 9.138372011350e-02 true resid norm 9.739616927529e+11 ||r(i)||/||b|| 5.546483160126e+09 350 KSP preconditioned resid norm 9.135024327765e-02 true resid norm 9.718116030157e+11 ||r(i)||/||b|| 5.534238903901e+09 351 KSP preconditioned resid norm 9.061386255163e-02 true resid norm 9.570624192117e+11 ||r(i)||/||b|| 5.450245765153e+09 352 KSP preconditioned resid norm 8.837690347022e-02 true resid norm 9.268791945381e+11 ||r(i)||/||b|| 5.278359387468e+09 353 KSP preconditioned resid norm 8.617584200021e-02 true resid norm 9.124042143664e+11 ||r(i)||/||b|| 5.195927774026e+09 354 KSP preconditioned resid norm 8.613272282523e-02 true resid norm 9.141642019657e+11 ||r(i)||/||b|| 5.205950490169e+09 355 KSP preconditioned resid norm 8.613208445007e-02 true resid norm 9.137367197889e+11 ||r(i)||/||b|| 5.203516079543e+09 356 KSP preconditioned resid norm 8.512721276827e-02 true resid norm 8.853575966066e+11 ||r(i)||/||b|| 5.041903636260e+09 357 KSP preconditioned resid norm 8.271004320452e-02 true resid norm 8.238454688088e+11 ||r(i)||/||b|| 4.691606511113e+09 358 KSP preconditioned resid norm 8.194079335891e-02 true resid norm 7.998601475410e+11 ||r(i)||/||b|| 4.555015738096e+09 359 KSP preconditioned resid norm 8.193382956550e-02 true resid norm 7.991570419090e+11 ||r(i)||/||b|| 4.551011716607e+09 360 KSP preconditioned resid norm 8.119172018345e-02 true resid norm 7.862732161050e+11 ||r(i)||/||b|| 4.477641353695e+09 361 KSP preconditioned resid norm 8.117546925443e-02 true resid norm 7.864203017478e+11 ||r(i)||/||b|| 4.478478971896e+09 362 KSP preconditioned resid norm 8.111438804164e-02 true resid norm 7.871982589594e+11 ||r(i)||/||b|| 4.482909255556e+09 363 KSP preconditioned resid norm 8.105884877535e-02 true resid norm 7.886567758525e+11 ||r(i)||/||b|| 4.491215166812e+09 364 KSP preconditioned resid norm 8.101728183962e-02 true resid norm 7.873477564028e+11 ||r(i)||/||b|| 4.483760608395e+09 365 KSP preconditioned resid norm 8.074846662941e-02 true resid norm 7.864323369506e+11 ||r(i)||/||b|| 4.478547509550e+09 366 KSP preconditioned resid norm 8.046924220803e-02 true resid norm 7.864176432426e+11 ||r(i)||/||b|| 4.478463832333e+09 367 KSP preconditioned resid norm 8.022386418935e-02 true resid norm 7.875434564068e+11 ||r(i)||/||b|| 4.484875073969e+09 368 KSP preconditioned resid norm 8.010432821424e-02 true resid norm 7.893295456645e+11 ||r(i)||/||b|| 4.495046432929e+09 369 KSP preconditioned resid norm 8.002735828029e-02 true resid norm 7.870494208944e+11 ||r(i)||/||b|| 4.482061657722e+09 370 KSP preconditioned resid norm 7.917464753128e-02 true resid norm 7.738583141011e+11 ||r(i)||/||b|| 4.406941401724e+09 371 KSP preconditioned resid norm 7.829268076822e-02 true resid norm 7.621979085595e+11 ||r(i)||/||b|| 4.340538129954e+09 372 KSP preconditioned resid norm 7.759852626003e-02 true resid norm 7.578532201493e+11 ||r(i)||/||b|| 4.315796149564e+09 373 KSP preconditioned resid norm 7.741735372797e-02 true resid norm 7.587752925413e+11 ||r(i)||/||b|| 4.321047135340e+09 374 KSP preconditioned resid norm 7.717907674215e-02 true resid norm 7.532898956316e+11 ||r(i)||/||b|| 4.289809087876e+09 375 KSP preconditioned resid norm 7.625985264318e-02 true resid norm 7.391433800450e+11 ||r(i)||/||b|| 4.209248003123e+09 376 KSP preconditioned resid norm 7.427546605166e-02 true resid norm 7.128066404305e+11 ||r(i)||/||b|| 4.059266454720e+09 377 KSP preconditioned resid norm 6.967611991727e-02 true resid norm 6.603591954673e+11 ||r(i)||/||b|| 3.760590569986e+09 378 KSP preconditioned resid norm 6.297767288777e-02 true resid norm 6.151351194298e+11 ||r(i)||/||b|| 3.503050075282e+09 379 KSP preconditioned resid norm 6.262215398206e-02 true resid norm 6.147312288902e+11 ||r(i)||/||b|| 3.500750013490e+09 380 KSP preconditioned resid norm 6.235328899725e-02 true resid norm 6.108575060876e+11 ||r(i)||/||b|| 3.478690071655e+09 381 KSP preconditioned resid norm 5.839109568960e-02 true resid norm 5.664431912112e+11 ||r(i)||/||b|| 3.225760976637e+09 382 KSP preconditioned resid norm 5.377674583117e-02 true resid norm 5.213552704442e+11 ||r(i)||/||b|| 2.968995854229e+09 383 KSP preconditioned resid norm 5.241945056227e-02 true resid norm 5.119854463556e+11 ||r(i)||/||b|| 2.915636906021e+09 384 KSP preconditioned resid norm 5.235419063906e-02 true resid norm 5.101335092560e+11 ||r(i)||/||b|| 2.905090559062e+09 385 KSP preconditioned resid norm 5.222578706724e-02 true resid norm 5.046865970499e+11 ||r(i)||/||b|| 2.874071672949e+09 386 KSP preconditioned resid norm 5.039605441836e-02 true resid norm 4.690823217341e+11 ||r(i)||/||b|| 2.671313684686e+09 387 KSP preconditioned resid norm 4.808565206534e-02 true resid norm 4.267379155536e+11 ||r(i)||/||b|| 2.430172233690e+09 388 KSP preconditioned resid norm 4.666154888309e-02 true resid norm 4.054910047964e+11 ||r(i)||/||b|| 2.309176065569e+09 389 KSP preconditioned resid norm 4.664507930243e-02 true resid norm 4.054219946280e+11 ||r(i)||/||b|| 2.308783068863e+09 390 KSP preconditioned resid norm 4.560435025125e-02 true resid norm 3.835334868800e+11 ||r(i)||/||b|| 2.184133156523e+09 391 KSP preconditioned resid norm 4.555647669030e-02 true resid norm 3.836331322160e+11 ||r(i)||/||b|| 2.184700613315e+09 392 KSP preconditioned resid norm 4.526531135127e-02 true resid norm 3.845467967509e+11 ||r(i)||/||b|| 2.189903718318e+09 393 KSP preconditioned resid norm 4.522490771729e-02 true resid norm 3.852140268748e+11 ||r(i)||/||b|| 2.193703437212e+09 394 KSP preconditioned resid norm 4.492059067286e-02 true resid norm 3.830834655261e+11 ||r(i)||/||b|| 2.181570390574e+09 395 KSP preconditioned resid norm 4.443089499461e-02 true resid norm 3.811197785390e+11 ||r(i)||/||b|| 2.170387654244e+09 396 KSP preconditioned resid norm 4.392623262592e-02 true resid norm 3.803875713569e+11 ||r(i)||/||b|| 2.166217906260e+09 397 KSP preconditioned resid norm 4.308132483382e-02 true resid norm 3.808213983967e+11 ||r(i)||/||b|| 2.168688449392e+09 398 KSP preconditioned resid norm 4.242927987392e-02 true resid norm 3.826706811525e+11 ||r(i)||/||b|| 2.179219680486e+09 399 KSP preconditioned resid norm 4.240182519172e-02 true resid norm 3.819699815377e+11 ||r(i)||/||b|| 2.175229360699e+09 400 KSP preconditioned resid norm 4.189671532041e-02 true resid norm 3.778267330725e+11 ||r(i)||/||b|| 2.151634533499e+09 401 KSP preconditioned resid norm 4.059788723754e-02 true resid norm 3.696714530059e+11 ||r(i)||/||b|| 2.105192128328e+09 402 KSP preconditioned resid norm 3.985154334032e-02 true resid norm 3.660320052239e+11 ||r(i)||/||b|| 2.084466327729e+09 403 KSP preconditioned resid norm 3.980699861193e-02 true resid norm 3.660287796004e+11 ||r(i)||/||b|| 2.084447958561e+09 404 KSP preconditioned resid norm 3.957443613187e-02 true resid norm 3.636434460851e+11 ||r(i)||/||b|| 2.070864044253e+09 405 KSP preconditioned resid norm 3.872420719304e-02 true resid norm 3.565740715742e+11 ||r(i)||/||b|| 2.030605616258e+09 406 KSP preconditioned resid norm 3.615598205882e-02 true resid norm 3.349638497397e+11 ||r(i)||/||b|| 1.907540476855e+09 407 KSP preconditioned resid norm 3.255981497970e-02 true resid norm 3.156535958679e+11 ||r(i)||/||b|| 1.797573114981e+09 408 KSP preconditioned resid norm 3.033952004273e-02 true resid norm 3.092934871523e+11 ||r(i)||/||b|| 1.761353789160e+09 409 KSP preconditioned resid norm 2.956834464235e-02 true resid norm 3.093700056741e+11 ||r(i)||/||b|| 1.761789544176e+09 410 KSP preconditioned resid norm 2.950977134039e-02 true resid norm 3.074077173244e+11 ||r(i)||/||b|| 1.750614772757e+09 411 KSP preconditioned resid norm 2.948462759166e-02 true resid norm 3.062372551691e+11 ||r(i)||/||b|| 1.743949265600e+09 412 KSP preconditioned resid norm 2.896998467535e-02 true resid norm 2.993488520401e+11 ||r(i)||/||b|| 1.704721427134e+09 413 KSP preconditioned resid norm 2.851424512720e-02 true resid norm 2.976176486467e+11 ||r(i)||/||b|| 1.694862630284e+09 414 KSP preconditioned resid norm 2.842666019416e-02 true resid norm 2.992055084233e+11 ||r(i)||/||b|| 1.703905118892e+09 415 KSP preconditioned resid norm 2.839117289498e-02 true resid norm 3.009603829654e+11 ||r(i)||/||b|| 1.713898717376e+09 416 KSP preconditioned resid norm 2.826226641508e-02 true resid norm 2.960777305963e+11 ||r(i)||/||b|| 1.686093158550e+09 417 KSP preconditioned resid norm 2.739279564893e-02 true resid norm 2.765802474280e+11 ||r(i)||/||b|| 1.575059569794e+09 418 KSP preconditioned resid norm 2.720979754887e-02 true resid norm 2.702787317822e+11 ||r(i)||/||b|| 1.539173917747e+09 419 KSP preconditioned resid norm 2.720071145142e-02 true resid norm 2.705214842583e+11 ||r(i)||/||b|| 1.540556336102e+09 420 KSP preconditioned resid norm 2.679219727058e-02 true resid norm 2.646789044532e+11 ||r(i)||/||b|| 1.507284215913e+09 421 KSP preconditioned resid norm 2.677245613117e-02 true resid norm 2.647866160273e+11 ||r(i)||/||b|| 1.507897608037e+09 422 KSP preconditioned resid norm 2.671753481828e-02 true resid norm 2.652052628309e+11 ||r(i)||/||b|| 1.510281703288e+09 423 KSP preconditioned resid norm 2.666450248533e-02 true resid norm 2.660006530176e+11 ||r(i)||/||b|| 1.514811263649e+09 424 KSP preconditioned resid norm 2.665230262710e-02 true resid norm 2.655521805336e+11 ||r(i)||/||b|| 1.512257318151e+09 425 KSP preconditioned resid norm 2.653886771284e-02 true resid norm 2.646828829057e+11 ||r(i)||/||b|| 1.507306872266e+09 426 KSP preconditioned resid norm 2.649465769062e-02 true resid norm 2.646133251578e+11 ||r(i)||/||b|| 1.506910757223e+09 427 KSP preconditioned resid norm 2.643674461804e-02 true resid norm 2.648601960923e+11 ||r(i)||/||b|| 1.508316629232e+09 428 KSP preconditioned resid norm 2.643464570032e-02 true resid norm 2.650348856809e+11 ||r(i)||/||b|| 1.509311445423e+09 429 KSP preconditioned resid norm 2.639071051056e-02 true resid norm 2.637241461722e+11 ||r(i)||/||b|| 1.501847091674e+09 430 KSP preconditioned resid norm 2.625335787177e-02 true resid norm 2.608090518974e+11 ||r(i)||/||b|| 1.485246314225e+09 431 KSP preconditioned resid norm 2.604373005246e-02 true resid norm 2.569425075124e+11 ||r(i)||/||b|| 1.463227251793e+09 432 KSP preconditioned resid norm 2.592947274469e-02 true resid norm 2.555810204825e+11 ||r(i)||/||b|| 1.455473902826e+09 433 KSP preconditioned resid norm 2.590316272926e-02 true resid norm 2.554937533328e+11 ||r(i)||/||b|| 1.454976936898e+09 434 KSP preconditioned resid norm 2.583900454382e-02 true resid norm 2.538850883100e+11 ||r(i)||/||b|| 1.445815967298e+09 435 KSP preconditioned resid norm 2.563637054199e-02 true resid norm 2.498382404589e+11 ||r(i)||/||b|| 1.422770118960e+09 436 KSP preconditioned resid norm 2.509245556191e-02 true resid norm 2.404661691511e+11 ||r(i)||/||b|| 1.369398373366e+09 437 KSP preconditioned resid norm 2.298145659199e-02 true resid norm 2.141147290518e+11 ||r(i)||/||b|| 1.219333109153e+09 438 KSP preconditioned resid norm 2.116678027322e-02 true resid norm 2.009204922250e+11 ||r(i)||/||b|| 1.144195028349e+09 439 KSP preconditioned resid norm 2.076913892396e-02 true resid norm 1.985007726816e+11 ||r(i)||/||b|| 1.130415293684e+09 440 KSP preconditioned resid norm 2.076866846319e-02 true resid norm 1.985439552748e+11 ||r(i)||/||b|| 1.130661208414e+09 441 KSP preconditioned resid norm 1.990345038510e-02 true resid norm 1.880094635249e+11 ||r(i)||/||b|| 1.070669751331e+09 442 KSP preconditioned resid norm 1.860635408680e-02 true resid norm 1.737888454436e+11 ||r(i)||/||b|| 9.896866702695e+08 443 KSP preconditioned resid norm 1.732679091729e-02 true resid norm 1.636479003350e+11 ||r(i)||/||b|| 9.319363689062e+08 444 KSP preconditioned resid norm 1.729564065123e-02 true resid norm 1.637156734927e+11 ||r(i)||/||b|| 9.323223211270e+08 445 KSP preconditioned resid norm 1.724902318940e-02 true resid norm 1.620582858820e+11 ||r(i)||/||b|| 9.228838878283e+08 446 KSP preconditioned resid norm 1.610582639377e-02 true resid norm 1.404940452651e+11 ||r(i)||/||b|| 8.000806006633e+08 447 KSP preconditioned resid norm 1.487044537537e-02 true resid norm 1.173706651617e+11 ||r(i)||/||b|| 6.683983801990e+08 448 KSP preconditioned resid norm 1.362293212605e-02 true resid norm 9.433525576421e+10 ||r(i)||/||b|| 5.372171322503e+08 449 KSP preconditioned resid norm 1.354998986380e-02 true resid norm 9.307930282672e+10 ||r(i)||/||b|| 5.300647751612e+08 450 KSP preconditioned resid norm 1.320310694331e-02 true resid norm 8.831311570404e+10 ||r(i)||/||b|| 5.029224585684e+08 451 KSP preconditioned resid norm 1.319378950685e-02 true resid norm 8.832890761616e+10 ||r(i)||/||b|| 5.030123898001e+08 452 KSP preconditioned resid norm 1.312303974559e-02 true resid norm 8.850817710488e+10 ||r(i)||/||b|| 5.040332874470e+08 453 KSP preconditioned resid norm 1.310261583133e-02 true resid norm 8.870140199595e+10 ||r(i)||/||b|| 5.051336578337e+08 454 KSP preconditioned resid norm 1.296447206314e-02 true resid norm 8.812119603260e+10 ||r(i)||/||b|| 5.018295211011e+08 455 KSP preconditioned resid norm 1.240430361604e-02 true resid norm 8.722744642520e+10 ||r(i)||/||b|| 4.967398269338e+08 456 KSP preconditioned resid norm 1.191354589862e-02 true resid norm 8.704245806869e+10 ||r(i)||/||b|| 4.956863616775e+08 457 KSP preconditioned resid norm 1.125844512425e-02 true resid norm 8.721247012817e+10 ||r(i)||/||b|| 4.966545404385e+08 458 KSP preconditioned resid norm 1.087068081755e-02 true resid norm 8.761726638661e+10 ||r(i)||/||b|| 4.989597600867e+08 459 KSP preconditioned resid norm 1.085533537265e-02 true resid norm 8.743712420236e+10 ||r(i)||/||b|| 4.979338926436e+08 460 KSP preconditioned resid norm 1.066839005792e-02 true resid norm 8.638398863453e+10 ||r(i)||/||b|| 4.919365328545e+08 461 KSP preconditioned resid norm 1.025886136677e-02 true resid norm 8.457994304882e+10 ||r(i)||/||b|| 4.816629168225e+08 462 KSP preconditioned resid norm 9.919337713557e-03 true resid norm 8.397156903652e+10 ||r(i)||/||b|| 4.781983696648e+08 463 KSP preconditioned resid norm 9.860344532387e-03 true resid norm 8.398118650993e+10 ||r(i)||/||b|| 4.782531389178e+08 464 KSP preconditioned resid norm 9.859976788477e-03 true resid norm 8.394195672420e+10 ||r(i)||/||b|| 4.780297344990e+08 465 KSP preconditioned resid norm 9.675049756059e-03 true resid norm 8.236108946379e+10 ||r(i)||/||b|| 4.690270666286e+08 466 KSP preconditioned resid norm 9.170327396442e-03 true resid norm 7.857495153286e+10 ||r(i)||/||b|| 4.474659000735e+08 467 KSP preconditioned resid norm 8.483974906501e-03 true resid norm 7.531157252297e+10 ||r(i)||/||b|| 4.288817228331e+08 468 KSP preconditioned resid norm 8.152302032595e-03 true resid norm 7.439835115997e+10 ||r(i)||/||b|| 4.236811415895e+08 469 KSP preconditioned resid norm 8.093392963250e-03 true resid norm 7.459076324609e+10 ||r(i)||/||b|| 4.247768832427e+08 470 KSP preconditioned resid norm 7.981993224478e-03 true resid norm 7.296712982923e+10 ||r(i)||/||b|| 4.155306721527e+08 471 KSP preconditioned resid norm 7.712571146038e-03 true resid norm 7.162942694814e+10 ||r(i)||/||b|| 4.079127683291e+08 472 KSP preconditioned resid norm 7.325481985277e-03 true resid norm 6.983558709550e+10 ||r(i)||/||b|| 3.976972715507e+08 473 KSP preconditioned resid norm 7.275070772984e-03 true resid norm 6.975438503236e+10 ||r(i)||/||b|| 3.972348448668e+08 474 KSP preconditioned resid norm 7.274707699564e-03 true resid norm 6.971070076010e+10 ||r(i)||/||b|| 3.969860731932e+08 475 KSP preconditioned resid norm 7.268332191455e-03 true resid norm 6.935255928841e+10 ||r(i)||/||b|| 3.949465416012e+08 476 KSP preconditioned resid norm 7.216317523001e-03 true resid norm 6.804985817534e+10 ||r(i)||/||b|| 3.875279646284e+08 477 KSP preconditioned resid norm 6.980123440335e-03 true resid norm 6.429387152029e+10 ||r(i)||/||b|| 3.661385025099e+08 478 KSP preconditioned resid norm 6.762105773451e-03 true resid norm 6.216260461321e+10 ||r(i)||/||b|| 3.540014378823e+08 479 KSP preconditioned resid norm 6.734273234398e-03 true resid norm 6.228320475336e+10 ||r(i)||/||b|| 3.546882273643e+08 480 KSP preconditioned resid norm 6.661150029867e-03 true resid norm 6.087445963201e+10 ||r(i)||/||b|| 3.466657546627e+08 481 KSP preconditioned resid norm 6.658199454094e-03 true resid norm 6.088666952551e+10 ||r(i)||/||b|| 3.467352871394e+08 482 KSP preconditioned resid norm 6.642195017520e-03 true resid norm 6.100067822855e+10 ||r(i)||/||b|| 3.473845399347e+08 483 KSP preconditioned resid norm 6.640965120125e-03 true resid norm 6.094611499210e+10 ||r(i)||/||b|| 3.470738151142e+08 484 KSP preconditioned resid norm 6.578702365638e-03 true resid norm 6.044169453388e+10 ||r(i)||/||b|| 3.442012590394e+08 485 KSP preconditioned resid norm 6.554453847735e-03 true resid norm 6.023303483252e+10 ||r(i)||/||b|| 3.430129910321e+08 486 KSP preconditioned resid norm 6.550275500880e-03 true resid norm 6.022766291392e+10 ||r(i)||/||b|| 3.429823992170e+08 487 KSP preconditioned resid norm 6.548644875395e-03 true resid norm 6.028372073738e+10 ||r(i)||/||b|| 3.433016353596e+08 488 KSP preconditioned resid norm 6.548213437710e-03 true resid norm 6.032404655891e+10 ||r(i)||/||b|| 3.435312814450e+08 489 KSP preconditioned resid norm 6.518662852709e-03 true resid norm 5.968351613026e+10 ||r(i)||/||b|| 3.398836110463e+08 490 KSP preconditioned resid norm 6.351002669544e-03 true resid norm 5.757241494473e+10 ||r(i)||/||b|| 3.278613854680e+08 491 KSP preconditioned resid norm 6.079319974564e-03 true resid norm 5.504366752965e+10 ||r(i)||/||b|| 3.134607626732e+08 492 KSP preconditioned resid norm 5.888870249588e-03 true resid norm 5.380239659526e+10 ||r(i)||/||b|| 3.063920161445e+08 493 KSP preconditioned resid norm 5.783324258671e-03 true resid norm 5.324539765772e+10 ||r(i)||/||b|| 3.032200379750e+08 494 KSP preconditioned resid norm 5.775295181242e-03 true resid norm 5.331438928146e+10 ||r(i)||/||b|| 3.036129290734e+08 495 KSP preconditioned resid norm 5.743289012196e-03 true resid norm 5.265823842070e+10 ||r(i)||/||b|| 2.998763039815e+08 496 KSP preconditioned resid norm 5.461844481659e-03 true resid norm 4.865688953344e+10 ||r(i)||/||b|| 2.770895615602e+08 497 KSP preconditioned resid norm 5.197200716475e-03 true resid norm 4.564156190496e+10 ||r(i)||/||b|| 2.599179786961e+08 498 KSP preconditioned resid norm 4.941064731411e-03 true resid norm 4.407871478233e+10 ||r(i)||/||b|| 2.510179312794e+08 499 KSP preconditioned resid norm 4.822917561764e-03 true resid norm 4.362390678061e+10 ||r(i)||/||b|| 2.484279064957e+08 500 KSP preconditioned resid norm 4.821765547723e-03 true resid norm 4.353698724745e+10 ||r(i)||/||b|| 2.479329201625e+08 501 KSP preconditioned resid norm 4.650512559482e-03 true resid norm 4.037102593246e+10 ||r(i)||/||b|| 2.299035138215e+08 502 KSP preconditioned resid norm 4.393046553933e-03 true resid norm 3.653253661973e+10 ||r(i)||/||b|| 2.080442184387e+08 503 KSP preconditioned resid norm 4.179362199028e-03 true resid norm 3.371113622957e+10 ||r(i)||/||b|| 1.919770056639e+08 504 KSP preconditioned resid norm 4.179013870481e-03 true resid norm 3.368686719967e+10 ||r(i)||/||b|| 1.918387992368e+08 505 KSP preconditioned resid norm 4.152755759095e-03 true resid norm 3.299427940208e+10 ||r(i)||/||b|| 1.878946743448e+08 506 KSP preconditioned resid norm 3.947491870507e-03 true resid norm 2.889734210786e+10 ||r(i)||/||b|| 1.645635783895e+08 507 KSP preconditioned resid norm 3.625400074846e-03 true resid norm 2.335626189630e+10 ||r(i)||/||b|| 1.330084275955e+08 508 KSP preconditioned resid norm 3.553429787828e-03 true resid norm 2.235826192449e+10 ||r(i)||/||b|| 1.273250520802e+08 509 KSP preconditioned resid norm 3.517566521207e-03 true resid norm 2.213699471598e+10 ||r(i)||/||b|| 1.260649872799e+08 510 KSP preconditioned resid norm 3.342996279113e-03 true resid norm 2.044832507521e+10 ||r(i)||/||b|| 1.164484101647e+08 511 KSP preconditioned resid norm 3.328666780498e-03 true resid norm 2.046130379268e+10 ||r(i)||/||b|| 1.165223209134e+08 512 KSP preconditioned resid norm 3.252022274500e-03 true resid norm 2.055203327537e+10 ||r(i)||/||b|| 1.170390040146e+08 513 KSP preconditioned resid norm 3.191741321738e-03 true resid norm 2.069602631008e+10 ||r(i)||/||b|| 1.178590105386e+08 514 KSP preconditioned resid norm 3.184455990627e-03 true resid norm 2.063679516592e+10 ||r(i)||/||b|| 1.175217030797e+08 515 KSP preconditioned resid norm 3.107037383993e-03 true resid norm 2.043805349576e+10 ||r(i)||/||b|| 1.163899159315e+08 516 KSP preconditioned resid norm 2.983266391702e-03 true resid norm 2.023612751558e+10 ||r(i)||/||b|| 1.152399948853e+08 517 KSP preconditioned resid norm 2.760881124202e-03 true resid norm 2.016388969560e+10 ||r(i)||/||b|| 1.148286174615e+08 518 KSP preconditioned resid norm 2.588923414339e-03 true resid norm 2.022532723876e+10 ||r(i)||/||b|| 1.151784898447e+08 519 KSP preconditioned resid norm 2.576347907091e-03 true resid norm 2.029333337944e+10 ||r(i)||/||b|| 1.155657688484e+08 520 KSP preconditioned resid norm 2.565607577158e-03 true resid norm 2.018232988065e+10 ||r(i)||/||b|| 1.149336299857e+08 521 KSP preconditioned resid norm 2.454738547647e-03 true resid norm 1.965786538034e+10 ||r(i)||/||b|| 1.119469277974e+08 522 KSP preconditioned resid norm 2.190817835255e-03 true resid norm 1.862766382686e+10 ||r(i)||/||b|| 1.060801718353e+08 523 KSP preconditioned resid norm 2.056923264246e-03 true resid norm 1.827034711542e+10 ||r(i)||/||b|| 1.040453370593e+08 524 KSP preconditioned resid norm 1.993242791661e-03 true resid norm 1.814670569095e+10 ||r(i)||/||b|| 1.033412281772e+08 525 KSP preconditioned resid norm 1.989053383557e-03 true resid norm 1.818391949160e+10 ||r(i)||/||b|| 1.035531520343e+08 526 KSP preconditioned resid norm 1.987611895736e-03 true resid norm 1.811887212524e+10 ||r(i)||/||b|| 1.031827225556e+08 527 KSP preconditioned resid norm 1.959483745719e-03 true resid norm 1.782009527995e+10 ||r(i)||/||b|| 1.014812585726e+08 528 KSP preconditioned resid norm 1.886276628668e-03 true resid norm 1.747770951882e+10 ||r(i)||/||b|| 9.953145205301e+07 529 KSP preconditioned resid norm 1.860924872867e-03 true resid norm 1.741298143281e+10 ||r(i)||/||b|| 9.916284080096e+07 530 KSP preconditioned resid norm 1.860853183467e-03 true resid norm 1.741777559071e+10 ||r(i)||/||b|| 9.919014240454e+07 531 KSP preconditioned resid norm 1.854254894317e-03 true resid norm 1.731950029390e+10 ||r(i)||/||b|| 9.863048766361e+07 532 KSP preconditioned resid norm 1.816775866931e-03 true resid norm 1.691273957549e+10 ||r(i)||/||b|| 9.631408087718e+07 533 KSP preconditioned resid norm 1.778904550054e-03 true resid norm 1.661282728172e+10 ||r(i)||/||b|| 9.460615078166e+07 534 KSP preconditioned resid norm 1.744559323925e-03 true resid norm 1.653593041182e+10 ||r(i)||/||b|| 9.416824116251e+07 535 KSP preconditioned resid norm 1.740837074189e-03 true resid norm 1.658539607295e+10 ||r(i)||/||b|| 9.444993648839e+07 536 KSP preconditioned resid norm 1.717530724525e-03 true resid norm 1.618091473175e+10 ||r(i)||/||b|| 9.214651021993e+07 537 KSP preconditioned resid norm 1.648889916303e-03 true resid norm 1.541157326939e+10 ||r(i)||/||b|| 8.776529122838e+07 538 KSP preconditioned resid norm 1.541645116608e-03 true resid norm 1.442842568348e+10 ||r(i)||/||b|| 8.216649656347e+07 539 KSP preconditioned resid norm 1.493976764484e-03 true resid norm 1.418123344109e+10 ||r(i)||/||b|| 8.075879478227e+07 540 KSP preconditioned resid norm 1.484966133227e-03 true resid norm 1.403121958994e+10 ||r(i)||/||b|| 7.990450112227e+07 541 KSP preconditioned resid norm 1.484571709259e-03 true resid norm 1.403375806401e+10 ||r(i)||/||b|| 7.991895713611e+07 542 KSP preconditioned resid norm 1.483698010744e-03 true resid norm 1.404612728882e+10 ||r(i)||/||b|| 7.998939696716e+07 543 KSP preconditioned resid norm 1.476030964878e-03 true resid norm 1.399126996005e+10 ||r(i)||/||b|| 7.967699736001e+07 544 KSP preconditioned resid norm 1.452209667395e-03 true resid norm 1.389012749559e+10 ||r(i)||/||b|| 7.910101477252e+07 545 KSP preconditioned resid norm 1.448966866288e-03 true resid norm 1.387862140082e+10 ||r(i)||/||b|| 7.903549026434e+07 546 KSP preconditioned resid norm 1.448451355304e-03 true resid norm 1.388471899546e+10 ||r(i)||/||b|| 7.907021463416e+07 547 KSP preconditioned resid norm 1.448052533242e-03 true resid norm 1.389844864923e+10 ||r(i)||/||b|| 7.914840178871e+07 548 KSP preconditioned resid norm 1.448024524637e-03 true resid norm 1.390384074379e+10 ||r(i)||/||b|| 7.917910850122e+07 549 KSP preconditioned resid norm 1.441912818230e-03 true resid norm 1.376245693865e+10 ||r(i)||/||b|| 7.837396092696e+07 550 KSP preconditioned resid norm 1.413388775949e-03 true resid norm 1.334669201925e+10 ||r(i)||/||b|| 7.600627732998e+07 551 KSP preconditioned resid norm 1.381568200768e-03 true resid norm 1.300769557517e+10 ||r(i)||/||b|| 7.407577217521e+07 552 KSP preconditioned resid norm 1.360525679977e-03 true resid norm 1.287345191709e+10 ||r(i)||/||b|| 7.331128606199e+07 553 KSP preconditioned resid norm 1.359173607598e-03 true resid norm 1.286892163035e+10 ||r(i)||/||b|| 7.328548714270e+07 554 KSP preconditioned resid norm 1.356294712116e-03 true resid norm 1.282004641667e+10 ||r(i)||/||b|| 7.300715427639e+07 555 KSP preconditioned resid norm 1.338919755135e-03 true resid norm 1.249094175065e+10 ||r(i)||/||b|| 7.113298047511e+07 556 KSP preconditioned resid norm 1.273051377080e-03 true resid norm 1.146687543384e+10 ||r(i)||/||b|| 6.530116324523e+07 557 KSP preconditioned resid norm 1.207406350927e-03 true resid norm 1.087429917990e+10 ||r(i)||/||b|| 6.192658061225e+07 558 KSP preconditioned resid norm 1.155009810287e-03 true resid norm 1.059116860199e+10 ||r(i)||/||b|| 6.031421844830e+07 559 KSP preconditioned resid norm 1.154386688843e-03 true resid norm 1.061098370312e+10 ||r(i)||/||b|| 6.042706079681e+07 560 KSP preconditioned resid norm 1.140694530749e-03 true resid norm 1.030974958177e+10 ||r(i)||/||b|| 5.871160320357e+07 561 KSP preconditioned resid norm 1.063523579546e-03 true resid norm 9.228258975279e+09 ||r(i)||/||b|| 5.255276812684e+07 562 KSP preconditioned resid norm 9.498362921280e-04 true resid norm 7.813703917823e+09 ||r(i)||/||b|| 4.449720920329e+07 563 KSP preconditioned resid norm 8.799658277512e-04 true resid norm 7.094432508280e+09 ||r(i)||/||b|| 4.040112740636e+07 KSP Object: 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-09, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, blocksize = 1, factorization UPPER Preconditioner for the Schur complement formed from S itself Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: gamg MG: type is MULTIPLICATIVE, levels=3 cycles=v Cycles per PCApply=1 Using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 1.01207 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 package used to perform factorization: petsc total: nonzeros=503, allocated nonzeros=503 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 18 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 18 nodes, limit used is 5 Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.0821467, max = 1.72508 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=1019, cols=1019 total: nonzeros=34719, allocated nonzeros=34719 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.156841, max = 3.29365 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_1_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_) 4 MPI processes type: lsc KSP Object: (fieldsplit_1_lsc_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 package used to perform factorization: petsc total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=9549, allocated nonzeros=9549 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines linear system matrix = precond matrix: Mat Object: (fieldsplit_1_) 4 MPI processes type: schurcomplement rows=3183, cols=3183 Schur complement A11 - A10 inv(A00) A01 A11 Mat Object: (fieldsplit_1_) 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=3183, allocated nonzeros=3183 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines A10 Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=71946 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines KSP of A00 KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: gamg MG: type is MULTIPLICATIVE, levels=3 cycles=v Cycles per PCApply=1 Using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 1.01207 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 package used to perform factorization: petsc total: nonzeros=503, allocated nonzeros=503 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 18 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 18 nodes, limit used is 5 Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.0821467, max = 1.72508 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=1019, cols=1019 total: nonzeros=34719, allocated nonzeros=34719 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.156841, max = 3.29365 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 A01 Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=3183 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 3757 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=75129, cols=75129 total: nonzeros=3.0016e+06, allocated nonzeros=1.26217e+07 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Recovering stress ... Cleaning up ... Finished ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /home/stali/dev/defmod on a arch-linux2-c-opt named i5 with 4 processors, by stali Thu Oct 23 08:28:33 2014 Using Petsc Development GIT revision: GIT Date: 2014-10-19 14:17:20 -0500 Max Max/Min Avg Total Time (sec): 3.387e+01 1.00102 3.385e+01 Objects: 1.513e+03 1.00132 1.512e+03 Flops: 1.521e+10 1.25426 1.421e+10 5.685e+10 Flops/sec: 4.495e+08 1.25432 4.199e+08 1.680e+09 MPI Messages: 3.319e+04 1.05598 3.217e+04 1.287e+05 MPI Message Lengths: 3.831e+08 2.61394 8.054e+03 1.036e+09 MPI Reductions: 2.203e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 3.3845e+01 100.0% 5.6850e+10 100.0% 1.287e+05 100.0% 8.054e+03 100.0% 2.202e+03 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecMDot 583 1.0 1.7830e+00 5.8 3.27e+08 1.0 0.0e+00 0.0e+00 5.8e+02 2 2 0 0 26 2 2 0 0 26 733 VecNorm 1169 1.0 4.9209e-01 2.2 4.35e+07 1.0 0.0e+00 0.0e+00 1.2e+03 1 0 0 0 53 1 0 0 0 53 353 VecScale 1186 1.0 3.6111e-02 1.1 2.21e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2393 VecCopy 1749 1.0 7.3937e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 9333 1.0 7.3792e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 584 1.0 2.1441e-02 1.1 2.19e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4085 VecAYPX 9876 1.0 3.0367e-01 1.2 1.21e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 1538 VecAXPBYCZ 4656 1.0 1.9113e-01 1.1 2.22e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 4444 VecMAXPY 1149 1.0 6.6691e-01 1.0 6.53e+08 1.0 0.0e+00 0.0e+00 0.0e+00 2 5 0 0 0 2 5 0 0 0 3915 VecAssemblyBegin 16 1.0 6.9537e-03 1.7 0.00e+00 0.0 2.0e+00 1.9e+04 4.5e+01 0 0 0 0 2 0 0 0 0 2 0 VecAssemblyEnd 16 1.0 5.4836e-05 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecPointwiseMult 22 1.0 7.9489e-04 1.2 2.09e+05 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1010 VecScatterBegin 15151 1.0 7.8862e-01 1.4 0.00e+00 0.0 1.3e+05 7.7e+03 0.0e+00 2 0 99 95 0 2 0 99 95 0 0 VecScatterEnd 15151 1.0 3.8561e+00 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 6 0 0 0 0 6 0 0 0 0 0 VecSetRandom 2 1.0 7.1716e-04 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 604 1.0 2.1901e-01 1.5 3.34e+07 1.0 0.0e+00 0.0e+00 6.0e+02 1 0 0 0 27 1 0 0 0 27 610 MatMult 10477 1.0 1.4918e+01 1.1 8.32e+09 1.3 1.1e+05 8.9e+03 0.0e+00 43 54 85 94 0 43 54 85 94 0 2076 MatMultAdd 1164 1.0 2.4949e-01 1.1 7.91e+07 1.3 8.7e+03 4.0e+02 0.0e+00 1 1 7 0 0 1 1 7 0 0 1186 MatMultTranspose 1164 1.0 1.0073e+00 2.2 7.91e+07 1.3 8.7e+03 4.0e+02 0.0e+00 2 1 7 0 0 2 1 7 0 0 294 MatSolve 1164 0.0 9.3598e-02 0.0 1.85e+07 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 204 MatSOR 6984 1.0 1.3192e+01 1.2 5.24e+09 1.3 0.0e+00 0.0e+00 0.0e+00 37 34 0 0 0 37 34 0 0 0 1470 MatLUFactorSym 1 1.0 4.1962e-05 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatLUFactorNum 2 1.0 2.6178e-0468.6 6.89e+03 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 38 MatILUFactorSym 1 1.0 3.0899e-0415.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatConvert 2 1.0 7.5967e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatScale 6 1.0 7.2534e-03 1.2 1.75e+06 1.3 2.4e+01 7.0e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 896 MatResidual 1164 1.0 1.8175e+00 1.3 9.40e+08 1.3 1.4e+04 7.0e+03 0.0e+00 4 6 11 9 0 4 6 11 9 0 1918 MatAssemblyBegin 45 1.0 2.8013e-01 4.6 0.00e+00 0.0 9.6e+01 2.3e+05 4.6e+01 0 0 0 2 2 0 0 0 2 2 0 MatAssemblyEnd 45 1.0 1.4216e-01 1.1 0.00e+00 0.0 3.0e+02 1.6e+03 1.4e+02 0 0 0 0 7 0 0 0 0 7 0 MatGetRow 57132 1.2 1.5872e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetRowIJ 1 0.0 1.0967e-05 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 0.0 1.2803e-04 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatCoarsen 2 1.0 1.0390e-02 1.3 0.00e+00 0.0 1.6e+02 6.3e+03 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 11 1.6 3.7792e-03 1.1 0.00e+00 0.0 6.0e+00 4.0e+00 1.3e+01 0 0 0 0 1 0 0 0 0 1 0 MatAXPY 2 1.0 5.2810e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatMatMult 3 1.0 4.1195e-02 1.0 1.56e+06 1.2 1.6e+02 5.1e+03 4.8e+01 0 0 0 0 2 0 0 0 0 2 142 MatMatMultSym 3 1.0 2.5058e-02 1.0 0.00e+00 0.0 1.3e+02 4.0e+03 4.2e+01 0 0 0 0 2 0 0 0 0 2 0 MatMatMultNum 3 1.0 1.6136e-02 1.0 1.56e+06 1.2 2.7e+01 1.0e+04 6.0e+00 0 0 0 0 0 0 0 0 0 0 363 MatPtAP 2 1.0 1.0186e-01 1.0 7.15e+06 1.3 2.2e+02 6.0e+03 3.4e+01 0 0 0 0 2 0 0 0 0 2 260 MatPtAPSymbolic 2 1.0 6.1335e-02 1.0 0.00e+00 0.0 1.4e+02 8.0e+03 1.4e+01 0 0 0 0 1 0 0 0 0 1 0 MatPtAPNumeric 2 1.0 4.0521e-02 1.0 7.15e+06 1.3 7.2e+01 2.1e+03 2.0e+01 0 0 0 0 1 0 0 0 0 1 653 MatTrnMatMult 2 1.0 9.8435e-01 1.0 6.85e+07 1.4 1.4e+02 1.6e+05 3.8e+01 3 0 0 2 2 3 0 0 2 2 249 MatTrnMatMultSym 2 1.0 5.1378e-01 1.0 0.00e+00 0.0 1.2e+02 6.5e+04 3.4e+01 2 0 0 1 2 2 0 0 1 2 0 MatTrnMatMultNum 2 1.0 4.7064e-01 1.0 6.85e+07 1.4 2.4e+01 6.2e+05 4.0e+00 1 0 0 1 0 1 0 0 1 0 521 MatGetLocalMat 12 1.0 1.6786e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetBrAoCol 8 1.0 9.3710e-03 1.8 0.00e+00 0.0 1.8e+02 1.0e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetSymTrans 4 1.0 7.8607e-04 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 583 1.0 2.0885e+00 3.5 6.54e+08 1.0 0.0e+00 0.0e+00 5.8e+02 3 5 0 0 26 3 5 0 0 26 1252 KSPSetUp 12 1.0 1.5793e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 3.2296e+01 1.0 1.52e+10 1.3 1.3e+05 7.9e+03 2.2e+03 95100100 98 98 95100100 98 98 1760 PCGAMGgraph_AGG 2 1.0 1.4315e-01 1.0 1.62e+06 1.3 7.2e+01 3.5e+03 2.8e+01 0 0 0 0 1 0 0 0 0 1 42 PCGAMGcoarse_AGG 2 1.0 1.0072e+00 1.0 6.85e+07 1.4 4.2e+02 6.0e+04 7.6e+01 3 0 0 2 3 3 0 0 2 3 243 PCGAMGProl_AGG 2 1.0 1.2208e-02 1.1 0.00e+00 0.0 1.5e+02 5.1e+03 4.8e+01 0 0 0 0 2 0 0 0 0 2 0 PCGAMGPOpt_AGG 2 1.0 8.8050e-02 1.0 2.31e+07 1.3 3.8e+02 5.7e+03 9.6e+01 0 0 0 0 4 0 0 0 0 4 980 PCSetUp 6 1.0 1.4989e+00 1.0 9.97e+07 1.4 1.3e+03 2.3e+04 4.6e+02 4 1 1 3 21 4 1 1 3 21 243 PCSetUpOnBlocks 1746 1.0 1.9777e-03 1.5 6.89e+03 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 5 PCApply 582 1.0 2.7544e+01 1.1 1.23e+10 1.3 1.1e+05 6.7e+03 3.7e+02 80 81 89 74 17 80 81 89 74 17 1663 SFSetGraph 2 1.0 5.3620e-04 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFBcastBegin 10 1.0 3.4158e-03 3.3 0.00e+00 0.0 1.6e+02 6.3e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFBcastEnd 10 1.0 1.3132e-03 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 1294 1294 181954152 0 Vector Scatter 24 24 313088 0 Matrix 85 85 129847512 0 Matrix Coarsen 2 2 760 0 Index Set 68 68 267716 0 Krylov Solver 14 14 101980 0 Preconditioner 14 14 8388 0 Viewer 4 3 1380 0 Distributed Mesh 1 1 2316 0 Star Forest Bipartite Graph 4 4 1952 0 Discrete System 1 1 472 0 PetscRandom 2 2 760 0 ======================================================================================================================== Average time to get PetscTime(): 1.19209e-07 Average time for MPI_Barrier(): 7.66277e-05 Average time for zero size MPI_Send(): 5.52535e-05 #PETSc Option Table entries: -f shannon.e.inp -fieldsplit_0_ksp_type preonly -fieldsplit_0_pc_type gamg -fieldsplit_1_ksp_type preonly -fieldsplit_1_pc_type lsc -ksp_monitor_true_residual -ksp_view -log_summary -pc_fieldsplit_detect_saddle_point -pc_fieldsplit_schur_fact_type upper -pc_fieldsplit_type schur -pc_type fieldsplit #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 4 sizeof(void*) 4 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --with-fc=gfortran --with-cc=gcc --download-mpich --with-metis=1 --download-metis=1 --with-ml=1 --download-ml=1 --COPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-shared-libraries --with-debugging=0 ----------------------------------------- Libraries compiled on Mon Oct 20 22:54:18 2014 on i5 Machine characteristics: Linux-3.2.0-4-686-pae-i686-with-debian-7.7 Using PETSc directory: /home/stali/petsc-dev Using PETSc arch: arch-linux2-c-opt ----------------------------------------- Using C compiler: /home/stali/petsc-dev/arch-linux2-c-opt/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O2 ${COPTFLAGS} ${CFLAGS} Using Fortran compiler: /home/stali/petsc-dev/arch-linux2-c-opt/bin/mpif90 -fPIC -Wall -Wno-unused-variable -ffree-line-length-0 -Wno-unused-dummy-argument -O2 ${FOPTFLAGS} ${FFLAGS} ----------------------------------------- Using include paths: -I/home/stali/petsc-dev/arch-linux2-c-opt/include -I/home/stali/petsc-dev/include -I/home/stali/petsc-dev/include -I/home/stali/petsc-dev/arch-linux2-c-opt/include ----------------------------------------- Using C linker: /home/stali/petsc-dev/arch-linux2-c-opt/bin/mpicc Using Fortran linker: /home/stali/petsc-dev/arch-linux2-c-opt/bin/mpif90 Using libraries: -Wl,-rpath,/home/stali/petsc-dev/arch-linux2-c-opt/lib -L/home/stali/petsc-dev/arch-linux2-c-opt/lib -lpetsc -Wl,-rpath,/home/stali/petsc-dev/arch-linux2-c-opt/lib -L/home/stali/petsc-dev/arch-linux2-c-opt/lib -lml -Wl,-rpath,/usr/lib/gcc/i486-linux-gnu/4.7 -L/usr/lib/gcc/i486-linux-gnu/4.7 -Wl,-rpath,/usr/lib/i386-linux-gnu -L/usr/lib/i386-linux-gnu -Wl,-rpath,/lib/i386-linux-gnu -L/lib/i386-linux-gnu -lmpicxx -lstdc++ -llapack -lblas -lmetis -lhwloc -lX11 -lpthread -lssl -lcrypto -lm -lmpifort -lgfortran -lm -lgfortran -lm -lquadmath -lm -lmpicxx -lstdc++ -Wl,-rpath,/home/stali/petsc-dev/arch-linux2-c-opt/lib -L/home/stali/petsc-dev/arch-linux2-c-opt/lib -Wl,-rpath,/usr/lib/gcc/i486-linux-gnu/4.7 -L/usr/lib/gcc/i486-linux-gnu/4.7 -Wl,-rpath,/usr/lib/i386-linux-gnu -L/usr/lib/i386-linux-gnu -Wl,-rpath,/lib/i386-linux-gnu -L/lib/i386-linux-gnu -Wl,-rpath,/usr/lib/i386-linux-gnu -L/usr/lib/i386-linux-gnu -ldl -Wl,-rpath,/home/stali/petsc-dev/arch-linux2-c-opt/lib -lmpi -lgcc_s -ldl ----------------------------------------- From knepley at gmail.com Thu Oct 23 09:54:06 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 23 Oct 2014 09:54:06 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <54491042.8080307@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> Message-ID: On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali wrote: > Matt > > Sorry about that (I always forget it). The output for the smallest problem > is now attached (see log.txt). I am also attaching some results that > compare results obtained using FS/LSC and the direct solver (MUMPS), again > for the smallest problem. The difference, as you can see is insignificant > O(1E-6). > 1) How do you use MUMPS if you have a saddle point 2) You can see from the output that something is seriously wrong with the preconditioner. It looks like it has a null space. Did you add the elastic null modes to GAMG? Without this, it is not going to work. We have helper functions for this: http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html you could just copy that code. And then use http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html I don't see it in the output, so I think this is your problem. In order to test, I would first use MUMPS as the A00 solver and get the Schur stuff worked out. Then I would replace MUMPS with GAMG and tune it until I get back my original convergence. Thanks, Matt > Also, I did pass 'upper' and 'full' to > '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't > improve (in fact, it increases slightly). The attached log is with 'upper'. > > Regards, > > Tabrez > > On 10/23/2014 07:46 AM, Matthew Knepley wrote: > > On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali > wrote: > >> Hello >> >> I am using the following options (below) for solving linear >> elasticity/poroelasticity problems involving slip between two surfaces >> involving non-trivial geometries, i.e., elements with high aspect ratios, >> large contrasts in material properties etc. The constraints are imposed >> using Lagrange Multipliers. >> >> A picture (shows displacement magnitude) is attached. The boundary nodes, >> i.e., the base and the four side are pinned. >> >> The following options appear to work well for the saddle point problem: >> >> -pc_type fieldsplit -pc_fieldsplit_type schur >> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >> -ksp_monitor >> >> However, the number of iterations keep on increasing with the problems >> size (see attached plot), e.g., >> >> 120K Tets *507* Iterations (KSP Residual norm 8.827362494659e-05) >> in 17 secs on 3 cores >> 1 Million Tets *1374* Iterations (KSP Residual norm 7.164704416296e-05) >> in 117 secs on 20 cores >> 8 Million Tets *2495* Iterations (KSP Residual norm 9.101247550026e-05) >> in 225 secs on 160 cores >> >> So what other options should I try to improve solver performance? Any >> tips/insights would be appreciated as preconditioning is black magic to me. >> > > For reports, always run with > > -ksp_view -ksp_monitor_true_residual -ksp_converged_reason > > so that we can see exactly what you used. > > I believe the default is a diagonal factorization. Since your outer > iterates are increasing, I would strengthen this > to either upper or full > > -pc_fieldsplit_schur_factorization_type > > Thanks, > > Matt > > >> Thanks in advance. >> >> Tabrez >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Thu Oct 23 10:06:42 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Thu, 23 Oct 2014 17:06:42 +0200 Subject: [petsc-users] MatCreate() or MatCreateFoobar() Message-ID: <1664541.8U4e9IJJQ8@asaru> Hello, the petsc user doc says: One should use the matrix creation routines for particular data structures, such as MatCreateSeqAIJ() and MatCreateAIJ() for compressed, sparse row formats, instead of the generic MatCreate() routine. (13.4.1, p. 172) However online doc says: It is recommended that one use the MatCreate(), MatSetType() and/or MatSetFromOptions(), MatXXXXSetPreallocation() paradgm instead of this routine directly. (http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSBAIJ.html) Somehow contradicting.... So, what is true? ;-) Best Regards, Florian From knepley at gmail.com Thu Oct 23 10:13:12 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 23 Oct 2014 10:13:12 -0500 Subject: [petsc-users] MatCreate() or MatCreateFoobar() In-Reply-To: <1664541.8U4e9IJJQ8@asaru> References: <1664541.8U4e9IJJQ8@asaru> Message-ID: On Thu, Oct 23, 2014 at 10:06 AM, Florian Lindner wrote: > Hello, > > the petsc user doc says: > One should use the matrix creation routines for particular data > structures, such as MatCreateSeqAIJ() and MatCreateAIJ() for compressed, > sparse row formats, instead of the generic MatCreate() routine. > (13.4.1, p. 172) > > However online doc says: > It is recommended that one use the MatCreate(), MatSetType() and/or > MatSetFromOptions(), MatXXXXSetPreallocation() paradgm instead of this > routine directly. > ( > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSBAIJ.html > ) > > Somehow contradicting.... > So, what is true? ;-) > Online is correct. The manual lags. Thanks, Matt > Best Regards, > Florian > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From salazardetroya at gmail.com Thu Oct 23 10:46:46 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Thu, 23 Oct 2014 10:46:46 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: References: <87wq7vrjg5.fsf@jedbrown.org> <87lhoam3ta.fsf@jedbrown.org> Message-ID: I decided to implement the continuous adjoint because it is clearer to me how to do it and the gradient will converge anyways. I was trying to interpolate the solution, but in the adjoint analysis I need to do it once the simulation has finished. I have saved all the solutions for each time step. My idea was to reset the TS at the time step immediately previous to the time step in which I want to interpolate to: interpolate_timestep : where I want to interpolate to previous_timestep : last time step in our time step history which is smaller that the interpolate_timestep Current_Sol : Solution at the time step previous_timestep TSSetTime(ts,previous_timestep); TSSetSolution(ts,Current_Sol); TSSetRetainStages(ts,PETSC_TRUE); TSInterpolate(ts,interpolate_timestep,X); Vector X should have a close value to Current_Sol, but it's nowhere close. I also compare it to the solution of the next time step after previous_timestep (therefore interpolate_timestep is between these guys) and it's not close either. I've read that TSInterpolate() has to be extended to support continuous adjoints. Thanks Miguel On Tue, Oct 21, 2014 at 6:04 PM, Miguel Angel Salazar de Troya < salazardetroya at gmail.com> wrote: > That might be a reasonable argument, but I'm not sure. This is one of the > papers that explains it. I'm re-reading to see if I skipped any details: > > http://www.sciencedirect.com/science/article/pii/S0377042709006062 > > Miguel > > On Mon, Oct 20, 2014 at 11:42 PM, Jed Brown wrote: > >> Miguel Angel Salazar de Troya writes: >> >> > Thanks for your response >> > >> > I'm struggling with this problem because the literature is not clear >> for me >> > on how to calculate the discrete adjoint with adaptive time stepping >> > algorithms. They cover the details when automatic differentiation tools >> are >> > used. They mention that because the time step depend on the solution, it >> > also depends on the parameter. Hence, there are terms that represent the >> > derivative of the time step w.r.t. the parameters. What it is confusing >> is >> > that they mention these terms must be removed. I don't understand this. >> I'm >> > planning to hard-code the discrete adjoint problem (and use the TS if >> > possible). >> >> Are they suggesting that the time step sizes for a given run should be >> frozen (at least locally) so that you have consistent gradients for a >> while? >> > > > > -- > *Miguel Angel Salazar de Troya* > Graduate Research Assistant > Department of Mechanical Science and Engineering > University of Illinois at Urbana-Champaign > (217) 550-2360 > salaza11 at illinois.edu > > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Thu Oct 23 10:48:39 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Thu, 23 Oct 2014 10:48:39 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> Message-ID: <54492357.4070304@geology.wisc.edu> Matt On 10/23/2014 09:54 AM, Matthew Knepley wrote: > On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali > wrote: > > Matt > > Sorry about that (I always forget it). The output for the smallest > problem is now attached (see log.txt). I am also attaching some > results that compare results obtained using FS/LSC and the direct > solver (MUMPS), again for the smallest problem. The difference, as > you can see is insignificant O(1E-6). > > > 1) How do you use MUMPS if you have a saddle point I simply used -pc_type lu -pc_factor_mat_solver_package mumps. > > 2) You can see from the output that something is seriously wrong with > the preconditioner. It looks like it has a null space. > Did you add the elastic null modes to GAMG? Without this, it is > not going to work. We have helper functions for this: > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html > > you could just copy that code. And then use > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html > > I don't see it in the output, so I think this is your problem. > > In order to test, I would first use MUMPS as the A00 solver and get > the Schur stuff worked out. Then I would > replace MUMPS with GAMG and tune it until I get back my original > convergence. I will try this with MatNullSpaceCreateRigidBody. Btw does it matter if some nodes are pinned? Tabrez > > Thanks, > > Matt > > Also, I did pass 'upper' and 'full' to > '-pc_fieldsplit_schur_factorization_type' but the iteration count > doesn't improve (in fact, it increases slightly). The attached log > is with 'upper'. > > Regards, > > Tabrez > > On 10/23/2014 07:46 AM, Matthew Knepley wrote: >> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >> > wrote: >> >> Hello >> >> I am using the following options (below) for solving linear >> elasticity/poroelasticity problems involving slip between two >> surfaces involving non-trivial geometries, i.e., elements >> with high aspect ratios, large contrasts in material >> properties etc. The constraints are imposed using Lagrange >> Multipliers. >> >> A picture (shows displacement magnitude) is attached. The >> boundary nodes, i.e., the base and the four side are pinned. >> >> The following options appear to work well for the saddle >> point problem: >> >> -pc_type fieldsplit -pc_fieldsplit_type schur >> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type >> lower -ksp_monitor >> >> However, the number of iterations keep on increasing with the >> problems size (see attached plot), e.g., >> >> 120K Tets *507* Iterations (KSP Residual norm >> 8.827362494659e-05)in 17 secs on 3 cores >> 1 Million Tets *1374* Iterations (KSP Residual norm >> 7.164704416296e-05)in 117 secs on 20 cores >> 8 Million Tets *2495* Iterations (KSP Residual norm >> 9.101247550026e-05) in 225 secs on 160 cores >> >> So what other options should I try to improve solver >> performance? Any tips/insights would be appreciated as >> preconditioning is black magic to me. >> >> >> For reports, always run with >> >> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >> >> so that we can see exactly what you used. >> >> I believe the default is a diagonal factorization. Since your >> outer iterates are increasing, I would strengthen this >> to either upper or full >> >> -pc_fieldsplit_schur_factorization_type >> >> Thanks, >> >> Matt >> >> Thanks in advance. >> >> Tabrez >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 23 10:51:15 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 23 Oct 2014 10:51:15 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <54492357.4070304@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> Message-ID: On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali wrote: > Matt > > On 10/23/2014 09:54 AM, Matthew Knepley wrote: > > On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali > wrote: > >> Matt >> >> Sorry about that (I always forget it). The output for the smallest >> problem is now attached (see log.txt). I am also attaching some results >> that compare results obtained using FS/LSC and the direct solver (MUMPS), >> again for the smallest problem. The difference, as you can see is >> insignificant O(1E-6). >> > > 1) How do you use MUMPS if you have a saddle point > > I simply used -pc_type lu -pc_factor_mat_solver_package mumps. > > > 2) You can see from the output that something is seriously wrong with > the preconditioner. It looks like it has a null space. > Did you add the elastic null modes to GAMG? Without this, it is not > going to work. We have helper functions for this: > > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html > > you could just copy that code. And then use > > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html > > I don't see it in the output, so I think this is your problem. > > In order to test, I would first use MUMPS as the A00 solver and get the > Schur stuff worked out. Then I would > replace MUMPS with GAMG and tune it until I get back my original > convergence. > > I will try this with MatNullSpaceCreateRigidBody. Btw does it matter if > some nodes are pinned? > No these are null modes of the operator, not of the particular problem. Matt > Tabrez > > > Thanks, > > Matt > > >> Also, I did pass 'upper' and 'full' to >> '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't >> improve (in fact, it increases slightly). The attached log is with 'upper'. >> >> Regards, >> >> Tabrez >> >> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >> >> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >> wrote: >> >>> Hello >>> >>> I am using the following options (below) for solving linear >>> elasticity/poroelasticity problems involving slip between two surfaces >>> involving non-trivial geometries, i.e., elements with high aspect ratios, >>> large contrasts in material properties etc. The constraints are imposed >>> using Lagrange Multipliers. >>> >>> A picture (shows displacement magnitude) is attached. The boundary >>> nodes, i.e., the base and the four side are pinned. >>> >>> The following options appear to work well for the saddle point problem: >>> >>> -pc_type fieldsplit -pc_fieldsplit_type schur >>> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >>> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >>> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >>> -ksp_monitor >>> >>> However, the number of iterations keep on increasing with the problems >>> size (see attached plot), e.g., >>> >>> 120K Tets *507* Iterations (KSP Residual norm 8.827362494659e-05) >>> in 17 secs on 3 cores >>> 1 Million Tets *1374* Iterations (KSP Residual norm 7.164704416296e-05) >>> in 117 secs on 20 cores >>> 8 Million Tets *2495* Iterations (KSP Residual norm >>> 9.101247550026e-05) in 225 secs on 160 cores >>> >>> So what other options should I try to improve solver performance? Any >>> tips/insights would be appreciated as preconditioning is black magic to me. >>> >> >> For reports, always run with >> >> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >> >> so that we can see exactly what you used. >> >> I believe the default is a diagonal factorization. Since your outer >> iterates are increasing, I would strengthen this >> to either upper or full >> >> -pc_fieldsplit_schur_factorization_type >> >> Thanks, >> >> Matt >> >> >>> Thanks in advance. >>> >>> Tabrez >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 23 12:57:11 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 23 Oct 2014 12:57:11 -0500 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <8738aej2lg.fsf@jedbrown.org> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> <87bnp2j3iz.fsf@jedbrown.org> <54490B52.6060004@sintef.no> <8738aej2lg.fsf@jedbrown.org> Message-ID: <606A00D3-EE0D-47AE-BE67-9653D4093335@mcs.anl.gov> > On Oct 23, 2014, at 9:13 AM, Jed Brown wrote: > > Arne Morten Kvarving writes: > >> > > There are many ways to get a preconditioner that is not SPD. It is any interesting question if there is anything one can ?do? to a preconditioner to insure that it is SPD or detect if it may not be BEFORE actually running the CG with that preconditioned. As I said before I don?t know of any theory or work on this in general. (There are specific cases like when using ICC one can shift the diagonal of the matrix to insure the resulting ?factor? is positive definite (we do this in PETSc) but for something complicated like a multigrid preconditioner I don?t have a clue.) PETSc iterative solvers do not adaptively ?fix themselves? when problems occur; for an industrial situation one might want them to. For example, if CG fails with indefinite matrix or preconditioned it could, in theory, switch itself automatically to GMRES. Or if the MG convergence is slow add more smoothing steps or a different smoother. We did a little research work on this a number of years ago but have never tried to package up making the solvers more robust by reconfiguring themselves in this way. Barry > It's no > safer if the matrix is marked SPD. And Richardson smoothing is not > inherently problematic; more like the relaxation paramater is easy to > mess up. From jed at jedbrown.org Thu Oct 23 13:12:04 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 23 Oct 2014 12:12:04 -0600 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <606A00D3-EE0D-47AE-BE67-9653D4093335@mcs.anl.gov> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> <87bnp2j3iz.fsf@jedbrown.org> <54490B52.6060004@sintef.no> <8738aej2lg.fsf@jedbrown.org> <606A00D3-EE0D-47AE-BE67-9653D4093335@mcs.anl.gov> Message-ID: <87r3xyhczv.fsf@jedbrown.org> Barry Smith writes: > It is any interesting question if there is anything one can ?do? to > a preconditioner to insure that it is SPD or detect if it may not > be BEFORE actually running the CG with that preconditioned. You can use an eigensolver for the extreme eigenvalues of the iteration matrix for the smoother. And if it's unstable, report that. This is one thing that the petsc-eig plugin (which I'll update) can do, but it's too expensive to do by default. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From mailinglists at xgm.de Fri Oct 24 04:23:30 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Fri, 24 Oct 2014 11:23:30 +0200 Subject: [petsc-users] Handling dense matrices in RBF interpolation Message-ID: <1708782.uJSfVK1QMg@asaru> Hello, I am redoing an radial basis function interpolation algorithm using petsc in our software. Using basis function with compact carrier is blazingly fast. These basis function result in sparse matrices. ierr = MatSetType(matrixC, MATSBAIJ); CHKERRV(ierr); // create symmetric, block sparse matrix. ierr = MatSetSizes(matrixC, PETSC_DECIDE, PETSC_DECIDE, n, n); CHKERRV(ierr); ierr = MatSetOption(matrixC, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); CHKERRV(ierr); ierr = MatSetUp(matrixC); CHKERRV(ierr); ierr = MatSetType(matrixA, MATAIJ); CHKERRV(ierr); // create sparse matrix. ierr = MatSetSizes(matrixA, PETSC_DECIDE, PETSC_DECIDE, outputSize, n); CHKERRV(ierr); ierr = MatSetUp(matrixA); CHKERRV(ierr); Benchmark is a direct dense matrix / LU decomposition, a rather naive implementation. The code runs sequentially, using PETSC_COMM_SELF. Using the matrix creation code above it's about 10 or 20 times faster, still without preallocation. During matrix assembly, which involves looping over rows and then over columns I collect all columns in an array and set each row using MatSetValues. Since the user can select the basis function for the RBF at runtime, my code is not aware if it's a compact carrier or not, thus it don't know if it results in sparse or dense matrices. Basis functions that produce dense matrices make the petsc code more then 50 time slower. Running the code with n = 503 (mesh size of 500 + polynom) it takes about 16 seconds, with 32% filling matrix A, 15% filling matrix C and 51% doing KSPSolve of matrix C (A is not KSPSolved). (dense matrices) If I just preallocate a practically dense matrix: ierr = MatSeqAIJSetPreallocation(matrixA, n, NULL); CHKERRV(ierr); runtime changes to 10 seconds, with 0% (zero) percent spent with filling matrix A. (KSPSolve 76%, Matrix C: 22%). For the sparse case it's that way still a lot faster then before, though slower then without preallocation. However, trying to preallocate C, which is of type MATSBAIJ fails for me: ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, n, n, NULL); results in: [0]PETSC ERROR: Arguments are incompatible [0]PETSC ERROR: Cannot change block size 1 to 503 What would you advise me to do here? How can I make the KSPSolve faster when it is a dense matrix? I could guess the preallocation. Create a list of possible basis functions and if they produce sparse or dense matrices. Preallocate full or select MATDENSE type for dense matrices. Any ideas? Thanks a lot, Florian From hzhang at mcs.anl.gov Fri Oct 24 09:42:59 2014 From: hzhang at mcs.anl.gov (Hong) Date: Fri, 24 Oct 2014 09:42:59 -0500 Subject: [petsc-users] Handling dense matrices in RBF interpolation In-Reply-To: <1708782.uJSfVK1QMg@asaru> References: <1708782.uJSfVK1QMg@asaru> Message-ID: Florian: > > > However, trying to preallocate C, which is of type MATSBAIJ fails for me: > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, n, n, NULL); > ^ The 2nd input parameter is block size: MatSeqSBAIJSetPreallocation(Mat B,PetscInt bs,PetscInt nz,const PetscInt nnz[]) You input block size bs=n, which I guess should be 1. Hong > > results in: > > [0]PETSC ERROR: Arguments are incompatible > [0]PETSC ERROR: Cannot change block size 1 to 503 > > What would you advise me to do here? How can I make the KSPSolve faster > when it is a dense matrix? > > I could guess the preallocation. Create a list of possible basis functions > and if they produce sparse or dense matrices. Preallocate full or select > MATDENSE type for dense matrices. > > Any ideas? > > Thanks a lot, > Florian > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Oct 24 09:45:43 2014 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 24 Oct 2014 09:45:43 -0500 Subject: [petsc-users] Handling dense matrices in RBF interpolation In-Reply-To: <1708782.uJSfVK1QMg@asaru> References: <1708782.uJSfVK1QMg@asaru> Message-ID: On Fri, Oct 24, 2014 at 4:23 AM, Florian Lindner wrote: > Hello, > > I am redoing an radial basis function interpolation algorithm using petsc > in our software. > > Using basis function with compact carrier is blazingly fast. These basis > function result in sparse matrices. > > ierr = MatSetType(matrixC, MATSBAIJ); CHKERRV(ierr); // create > symmetric, block sparse matrix. > ierr = MatSetSizes(matrixC, PETSC_DECIDE, PETSC_DECIDE, n, n); > CHKERRV(ierr); > ierr = MatSetOption(matrixC, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > CHKERRV(ierr); > ierr = MatSetUp(matrixC); CHKERRV(ierr); > > ierr = MatSetType(matrixA, MATAIJ); CHKERRV(ierr); // create sparse > matrix. > ierr = MatSetSizes(matrixA, PETSC_DECIDE, PETSC_DECIDE, outputSize, n); > CHKERRV(ierr); > ierr = MatSetUp(matrixA); CHKERRV(ierr); > > Benchmark is a direct dense matrix / LU decomposition, a rather naive > implementation. The code runs sequentially, using PETSC_COMM_SELF. > > Using the matrix creation code above it's about 10 or 20 times faster, > still without preallocation. During matrix assembly, which involves looping > over rows and then over columns I collect all columns in an array and set > each row using MatSetValues. > > Since the user can select the basis function for the RBF at runtime, my > code is not aware if it's a compact carrier or not, thus it don't know if > it results in sparse or dense matrices. Basis functions that produce dense > matrices make the petsc code more then 50 time slower. > > Running the code with n = 503 (mesh size of 500 + polynom) it takes about > 16 seconds, with 32% filling matrix A, 15% filling matrix C and 51% doing > KSPSolve of matrix C (A is not KSPSolved). (dense matrices) > > If I just preallocate a practically dense matrix: > > ierr = MatSeqAIJSetPreallocation(matrixA, n, NULL); CHKERRV(ierr); > > runtime changes to 10 seconds, with 0% (zero) percent spent with filling > matrix A. (KSPSolve 76%, Matrix C: 22%). For the sparse case it's that way > still a lot faster then before, though slower then without preallocation. > > However, trying to preallocate C, which is of type MATSBAIJ fails for me: > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, n, n, NULL); > > results in: > > [0]PETSC ERROR: Arguments are incompatible > [0]PETSC ERROR: Cannot change block size 1 to 503 > > What would you advise me to do here? How can I make the KSPSolve faster > when it is a dense matrix? > > I could guess the preallocation. Create a list of possible basis functions > and if they produce sparse or dense matrices. Preallocate full or select > MATDENSE type for dense matrices. > 1) You should be preallocating all sparse matrices. It is much much much faster. Just run through your loop twice. 2) I would look at your basis and decide whether to use dense or sparse. A densely preallocated sparse matrix is much slower than a dense matrix. 3) You can also look at what we did in http://arxiv.org/abs/0909.5413 Thanks, Matt > Any ideas? > > Thanks a lot, > Florian > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Fri Oct 24 09:59:32 2014 From: jed at jedbrown.org (Jed Brown) Date: Fri, 24 Oct 2014 08:59:32 -0600 Subject: [petsc-users] Handling dense matrices in RBF interpolation In-Reply-To: <1708782.uJSfVK1QMg@asaru> References: <1708782.uJSfVK1QMg@asaru> Message-ID: <87ioj9fr8r.fsf@jedbrown.org> Florian Lindner writes: > Hello, > > I am redoing an radial basis function interpolation algorithm using petsc in our software. > > Using basis function with compact carrier is blazingly fast. These basis function result in sparse matrices. > > ierr = MatSetType(matrixC, MATSBAIJ); CHKERRV(ierr); // create symmetric, block sparse matrix. > ierr = MatSetSizes(matrixC, PETSC_DECIDE, PETSC_DECIDE, n, n); CHKERRV(ierr); > ierr = MatSetOption(matrixC, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); CHKERRV(ierr); > ierr = MatSetUp(matrixC); CHKERRV(ierr); > > ierr = MatSetType(matrixA, MATAIJ); CHKERRV(ierr); // create sparse matrix. > ierr = MatSetSizes(matrixA, PETSC_DECIDE, PETSC_DECIDE, outputSize, n); CHKERRV(ierr); > ierr = MatSetUp(matrixA); CHKERRV(ierr); > > Benchmark is a direct dense matrix / LU decomposition, a rather naive implementation. The code runs sequentially, using PETSC_COMM_SELF. > > Using the matrix creation code above it's about 10 or 20 times faster, still without preallocation. During matrix assembly, which involves looping over rows and then over columns I collect all columns in an array and set each row using MatSetValues. > > Since the user can select the basis function for the RBF at runtime, my code is not aware if it's a compact carrier or not, thus it don't know if it results in sparse or dense matrices. Basis functions that produce dense matrices make the petsc code more then 50 time slower. Why not analyze the basis functions so you can choose an appropriate data structure? > Running the code with n = 503 (mesh size of 500 + polynom) it takes about 16 seconds, with 32% filling matrix A, 15% filling matrix C and 51% doing KSPSolve of matrix C (A is not KSPSolved). (dense matrices) > > If I just preallocate a practically dense matrix: > > ierr = MatSeqAIJSetPreallocation(matrixA, n, NULL); CHKERRV(ierr); > > runtime changes to 10 seconds, with 0% (zero) percent spent with filling matrix A. (KSPSolve 76%, Matrix C: 22%). For the sparse case it's that way still a lot faster then before, though slower then without preallocation. > > However, trying to preallocate C, which is of type MATSBAIJ fails for me: > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, n, n, NULL); > > results in: > > [0]PETSC ERROR: Arguments are incompatible > [0]PETSC ERROR: Cannot change block size 1 to 503 You probably want bs=1 instead of bs=n in the call above. > What would you advise me to do here? How can I make the KSPSolve faster when it is a dense matrix? Uh, find a data-sparse representation or reformulate to expose such sparsity. And use appropriate data structures. > I could guess the preallocation. Yes, you should definitely preallocate when assembling sparse matrices. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From francesco.ballarin at polimi.it Sun Oct 26 04:32:34 2014 From: francesco.ballarin at polimi.it (Francesco Ballarin) Date: Sun, 26 Oct 2014 10:32:34 +0100 Subject: [petsc-users] MatZeroRows and MATELEMENTAL Message-ID: Dear PETSc users, I have a code for PETSc 3.3 in which the PLAPACK interface, now discontinued, was used. I am trying to update it to PETSc 3.5.2, using the Elemental interface. I see that MatZeroRows is not implemented for MATELEMENTAL. What function should I use instead? Does it also support the version with optional arguments x and b? Thanks, best regards, Francesco -------------- next part -------------- An HTML attachment was scrubbed... URL: From jollage at gmail.com Sun Oct 26 11:10:07 2014 From: jollage at gmail.com (Mengqi Zhang) Date: Sun, 26 Oct 2014 14:10:07 -0200 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel Message-ID: Dear all PETSc users, I'm new to PETSc/SLEPc. And I'm currently trying to use Python/PETSc4py/SLEPc4py to solve a big generalized eigenvalue problem, A x = B lambda x. After several days' digging and trying, I can write a script for solving a smaller eigenvalue problem locally, but when the problem size increases, I realize I should go parallel. My question is how to do it in parallel? How could I preallocate the matrices among the processors? I presume there should be a way to do that, but I couldn't find a clear guide on the Internet. Currently, the matrices are implemented locally, and then I use E = SLEPc.EPS(); E.create() E.setOperators(A,B) E.setProblemType(SLEPc.EPS.ProblemType.GNHEP) E.setFromOptions() to locally solve the eigenvalue problem. Could you help me and guide me to set the code up? Any suggestion is welcomed and appreciated. Thank you very much. Best regards, Mengqi -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Sun Oct 26 12:21:59 2014 From: jroman at dsic.upv.es (Jose E. Roman) Date: Sun, 26 Oct 2014 18:21:59 +0100 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: References: Message-ID: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> El 26/10/2014, a las 17:10, Mengqi Zhang escribi?: > Dear all PETSc users, > > I'm new to PETSc/SLEPc. And I'm currently trying to use Python/PETSc4py/SLEPc4py to solve a big generalized eigenvalue problem, A x = B lambda x. > > After several days' digging and trying, I can write a script for solving a smaller eigenvalue problem locally, but when the problem size increases, I realize I should go parallel. My question is how to do it in parallel? How could I preallocate the matrices among the processors? I presume there should be a way to do that, but I couldn't find a clear guide on the Internet. > > Currently, the matrices are implemented locally, and then I use > > E = SLEPc.EPS(); E.create() > > E.setOperators(A,B) > > E.setProblemType(SLEPc.EPS.ProblemType.GNHEP) > > E.setFromOptions() > > to locally solve the eigenvalue problem. > > > > Could you help me and guide me to set the code up? Any suggestion is welcomed and appreciated. Thank you very much. > > > > Best regards, > > Mengqi > You may want to do A.setPreallocationNNZ(5) or something like that. Also, take into account that in generalized eigenproblems you will probably want to use a parallel linear solver, see discussion in section 3.4.1 of SLEPc's users guide. Jose From jianjun.xiao at kit.edu Sun Oct 26 12:45:27 2014 From: jianjun.xiao at kit.edu (Xiao) Date: Sun, 26 Oct 2014 18:45:27 +0100 Subject: [petsc-users] Cannot use PESSL instead of ESSL! Message-ID: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu> Hello, I tried to compile PETSc 3.5.1 with Intel Compiler XE 2015. Somehow I got the error below during the configuration. Cannot use PESSL instead of ESSL! When GNU compiler was used, there was no such an error. Could anybody tell me what I should do? Thank you. JJ From balay at mcs.anl.gov Sun Oct 26 12:50:27 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Sun, 26 Oct 2014 12:50:27 -0500 Subject: [petsc-users] Cannot use PESSL instead of ESSL! In-Reply-To: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu> References: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu> Message-ID: you have to send in more details than this. [configure.log?] Are you using intel-mpi-5.0? Satish On Sun, 26 Oct 2014, Xiao wrote: > Hello, > > I tried to compile PETSc 3.5.1 with Intel Compiler XE 2015. Somehow I got the > error below during the configuration. > > Cannot use PESSL instead of ESSL! > > When GNU compiler was used, there was no such an error. Could anybody tell me > what I should do? Thank you. > > JJ > From jianjun.xiao at kit.edu Sun Oct 26 12:57:26 2014 From: jianjun.xiao at kit.edu (Xiao, Jianjun (IKET)) Date: Sun, 26 Oct 2014 18:57:26 +0100 Subject: [petsc-users] Cannot use PESSL instead of ESSL! In-Reply-To: References: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu>, Message-ID: <56D054AF2E93E044AC1D2685709D2868017344DA00EB@KIT-MSX-07.kit.edu> Dear Satish, Please find the configuration.log in the attachment. Yes, I am using intel-mpi-5.0.1. Thank you. JJ ________________________________________ From: Satish Balay [balay at mcs.anl.gov] Sent: Sunday, October 26, 2014 6:50 PM To: Xiao, Jianjun (IKET) Cc: petsc-users at mcs.anl.gov Subject: Re: [petsc-users] Cannot use PESSL instead of ESSL! you have to send in more details than this. [configure.log?] Are you using intel-mpi-5.0? Satish On Sun, 26 Oct 2014, Xiao wrote: > Hello, > > I tried to compile PETSc 3.5.1 with Intel Compiler XE 2015. Somehow I got the > error below during the configuration. > > Cannot use PESSL instead of ESSL! > > When GNU compiler was used, there was no such an error. Could anybody tell me > what I should do? Thank you. > > JJ > -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.log Type: text/x-log Size: 1865362 bytes Desc: configure.log URL: From balay at mcs.anl.gov Sun Oct 26 13:24:21 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Sun, 26 Oct 2014 13:24:21 -0500 Subject: [petsc-users] Cannot use PESSL instead of ESSL! In-Reply-To: <56D054AF2E93E044AC1D2685709D2868017344DA00EB@KIT-MSX-07.kit.edu> References: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu>, <56D054AF2E93E044AC1D2685709D2868017344DA00EB@KIT-MSX-07.kit.edu> Message-ID: mpicc in intel-mpi-5.0.1 is broken. [it does not return proper error codes]. I believe they have a fix that will be released in intel-mpi-5.0.2 [in the near future]. Your alternatives: 1. Use this MPI - but not mpicc ./configure --with-cc=icc --with-fc=ifort --with-cxx=0 --with-mpi-include=/home/xiao/intel/impi/5.0.1.035/intel64/include --with-mpi-lib="-L/home/xiao/intel/impi/5.0.1.035/intel64/lib/release -L/home/xiao/intel/impi/5.0.1.035/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread" --with-debugging=0 COPTFLAGS=-O3 FOPTFLAGS=-O3 CXXOPTFLAGS=-O3 --with-blas-lapack-dir=/home/xiao/intel/mkl 2. Use impi-4 not impi-5 3. Wait for Intel to release the bugfix in IMPI-5.0.2 4. Use MPICH or OpenMPI [not impi] BTW: you appear to use gcc from mpicc - not icc. >>>>> Executing: /home/xiao/intel/impi/5.0.1.035/intel64/bin/mpicc -show stdout: gcc -I/home/xiao/intel/impi/5.0.1.035/intel64/include -L/home/xiao/intel/impi/5.0.1.035/intel64/lib/release -L/home/xiao/intel/impi/5.0.1.035/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /home/xiao/intel/impi/5.0.1.035/intel64/lib/release -Xlinker -rpath -Xlinker /home/xiao/intel/impi/5.0.1.035/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib/release -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread <<<<<<< Satish On Sun, 26 Oct 2014, Xiao, Jianjun (IKET) wrote: > Dear Satish, > > Please find the configuration.log in the attachment. > > Yes, I am using intel-mpi-5.0.1. > > Thank you. > > JJ > ________________________________________ > From: Satish Balay [balay at mcs.anl.gov] > Sent: Sunday, October 26, 2014 6:50 PM > To: Xiao, Jianjun (IKET) > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Cannot use PESSL instead of ESSL! > > you have to send in more details than this. [configure.log?] > > Are you using intel-mpi-5.0? > > Satish > > On Sun, 26 Oct 2014, Xiao wrote: > > > Hello, > > > > I tried to compile PETSc 3.5.1 with Intel Compiler XE 2015. Somehow I got the > > error below during the configuration. > > > > Cannot use PESSL instead of ESSL! > > > > When GNU compiler was used, there was no such an error. Could anybody tell me > > what I should do? Thank you. > > > > JJ > > > > From jianjun.xiao at kit.edu Sun Oct 26 13:33:51 2014 From: jianjun.xiao at kit.edu (Xiao, Jianjun (IKET)) Date: Sun, 26 Oct 2014 19:33:51 +0100 Subject: [petsc-users] Cannot use PESSL instead of ESSL! In-Reply-To: References: <1538529.VqR4sb0HMV@iketgfltt.iket.kit.edu>, <56D054AF2E93E044AC1D2685709D2868017344DA00EB@KIT-MSX-07.kit.edu>, Message-ID: <56D054AF2E93E044AC1D2685709D2868017344DA00EC@KIT-MSX-07.kit.edu> Option 1 works fine for me. Thanks. ________________________________________ From: Satish Balay [balay at mcs.anl.gov] Sent: Sunday, October 26, 2014 7:24 PM To: Xiao, Jianjun (IKET) Cc: petsc-users Subject: Re: [petsc-users] Cannot use PESSL instead of ESSL! mpicc in intel-mpi-5.0.1 is broken. [it does not return proper error codes]. I believe they have a fix that will be released in intel-mpi-5.0.2 [in the near future]. Your alternatives: 1. Use this MPI - but not mpicc ./configure --with-cc=icc --with-fc=ifort --with-cxx=0 --with-mpi-include=/home/xiao/intel/impi/5.0.1.035/intel64/include --with-mpi-lib="-L/home/xiao/intel/impi/5.0.1.035/intel64/lib/release -L/home/xiao/intel/impi/5.0.1.035/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread" --with-debugging=0 COPTFLAGS=-O3 FOPTFLAGS=-O3 CXXOPTFLAGS=-O3 --with-blas-lapack-dir=/home/xiao/intel/mkl 2. Use impi-4 not impi-5 3. Wait for Intel to release the bugfix in IMPI-5.0.2 4. Use MPICH or OpenMPI [not impi] BTW: you appear to use gcc from mpicc - not icc. >>>>> Executing: /home/xiao/intel/impi/5.0.1.035/intel64/bin/mpicc -show stdout: gcc -I/home/xiao/intel/impi/5.0.1.035/intel64/include -L/home/xiao/intel/impi/5.0.1.035/intel64/lib/release -L/home/xiao/intel/impi/5.0.1.035/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /home/xiao/intel/impi/5.0.1.035/intel64/lib/release -Xlinker -rpath -Xlinker /home/xiao/intel/impi/5.0.1.035/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib/release -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread <<<<<<< Satish On Sun, 26 Oct 2014, Xiao, Jianjun (IKET) wrote: > Dear Satish, > > Please find the configuration.log in the attachment. > > Yes, I am using intel-mpi-5.0.1. > > Thank you. > > JJ > ________________________________________ > From: Satish Balay [balay at mcs.anl.gov] > Sent: Sunday, October 26, 2014 6:50 PM > To: Xiao, Jianjun (IKET) > Cc: petsc-users at mcs.anl.gov > Subject: Re: [petsc-users] Cannot use PESSL instead of ESSL! > > you have to send in more details than this. [configure.log?] > > Are you using intel-mpi-5.0? > > Satish > > On Sun, 26 Oct 2014, Xiao wrote: > > > Hello, > > > > I tried to compile PETSc 3.5.1 with Intel Compiler XE 2015. Somehow I got the > > error below during the configuration. > > > > Cannot use PESSL instead of ESSL! > > > > When GNU compiler was used, there was no such an error. Could anybody tell me > > what I should do? Thank you. > > > > JJ > > > > From thronesf at gmail.com Mon Oct 27 12:52:58 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 13:52:58 -0400 Subject: [petsc-users] TSSetIJacobian is never called Message-ID: Dear All, I'm using the TS theta solver for backward Euler method. But I found that the TSSetIJacobian() routine has never been called. I did the code just following what the example does: src/ts/examples/tutorials/ex17. I can run the example without errors. But when I run my code, it won't be as good as the example as stated above. I also attached the piece of my setups to this email. How could I fix this problem? Thank you in advance! TSCreate(PETSC_COMM_WORLD, &ts); TSSetProblemType(ts, TS_NONLINEAR); TSSetType(ts, TSTHETA); TSThetaSetTheta(ts, 1.0); TSSetIFunction(ts, NULL, FormIFunction, &appctx); DMSetMatType(da, MATAIJ); DMCreateMatrix(da, &J); TSSetDM(ts, da); TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); TSSetDuration(ts, timeStepsMax, Tend); // Evaluate initial conditions FormInitialSolution(&appctx); TSSetSolution(ts, solution); dt = 1.0; TSSetInitialTimeStep(ts, 0.0, dt); TSSolve(ts, solution); -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 12:57:49 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 12:57:49 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone wrote: > Dear All, > > I'm using the TS theta solver for backward Euler method. But I found that > the TSSetIJacobian() routine has never been called. I did the code just > following what the example does: src/ts/examples/tutorials/ex17. I can run > the example without errors. But when I run my code, it won't be as good as > the example as stated above. I also attached the piece of my setups to this > email. > > How could I fix this problem? Thank you in advance! > I cannot understand the paragraph above. 1) TS ex17 does call TSSetIJacobian(), unless you choose a matrix-free version. You seem to imply above that it does not. 2) Do you mean that you do not call it? You do not have too, since you can use the finite difference approximations offered by SNES Thanks, Matt > > TSCreate(PETSC_COMM_WORLD, &ts); > TSSetProblemType(ts, TS_NONLINEAR); > TSSetType(ts, TSTHETA); > TSThetaSetTheta(ts, 1.0); > TSSetIFunction(ts, NULL, FormIFunction, &appctx); > DMSetMatType(da, MATAIJ); > DMCreateMatrix(da, &J); > TSSetDM(ts, da); > TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); > TSSetDuration(ts, timeStepsMax, Tend); > // Evaluate initial conditions > FormInitialSolution(&appctx); > TSSetSolution(ts, solution); > dt = 1.0; > TSSetInitialTimeStep(ts, 0.0, dt); > TSSolve(ts, solution); > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 27 13:02:32 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 14:02:32 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: Hi Matt, Thank you for your reply and sorry for the confusion. No. What I was trying to say is my code has never called the TSSetIJacobian() routine. I don't know why this happens, and how to resolve this either. Thank you in advance! On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone wrote: > >> Dear All, >> >> I'm using the TS theta solver for backward Euler method. But I found that >> the TSSetIJacobian() routine has never been called. I did the code just >> following what the example does: src/ts/examples/tutorials/ex17. I can run >> the example without errors. But when I run my code, it won't be as good as >> the example as stated above. I also attached the piece of my setups to this >> email. >> >> How could I fix this problem? Thank you in advance! >> > > I cannot understand the paragraph above. > > 1) TS ex17 does call TSSetIJacobian(), unless you choose a matrix-free > version. You seem to imply above that it does not. > > 2) Do you mean that you do not call it? You do not have too, since you > can use the finite difference approximations offered by SNES > > Thanks, > > Matt > > >> >> TSCreate(PETSC_COMM_WORLD, &ts); >> TSSetProblemType(ts, TS_NONLINEAR); >> TSSetType(ts, TSTHETA); >> TSThetaSetTheta(ts, 1.0); >> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >> DMSetMatType(da, MATAIJ); >> DMCreateMatrix(da, &J); >> TSSetDM(ts, da); >> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >> TSSetDuration(ts, timeStepsMax, Tend); >> // Evaluate initial conditions >> FormInitialSolution(&appctx); >> TSSetSolution(ts, solution); >> dt = 1.0; >> TSSetInitialTimeStep(ts, 0.0, dt); >> TSSolve(ts, solution); >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 13:05:29 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 13:05:29 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: On Mon, Oct 27, 2014 at 1:02 PM, Sharp Stone wrote: > Hi Matt, > > Thank you for your reply and sorry for the confusion. > > No. What I was trying to say is my code has never called the > TSSetIJacobian() routine. I don't know why this happens, and how to resolve > this either. > If your code has not called TSSetIJacobian(), I can only assume it is because you did not put it in. However, in the code fragment you sent, it was definitely there. Thanks, Matt > Thank you in advance! > > On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley > wrote: > >> On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone wrote: >> >>> Dear All, >>> >>> I'm using the TS theta solver for backward Euler method. But I found >>> that the TSSetIJacobian() routine has never been called. I did the code >>> just following what the example does: src/ts/examples/tutorials/ex17. I can >>> run the example without errors. But when I run my code, it won't be as good >>> as the example as stated above. I also attached the piece of my setups to >>> this email. >>> >>> How could I fix this problem? Thank you in advance! >>> >> >> I cannot understand the paragraph above. >> >> 1) TS ex17 does call TSSetIJacobian(), unless you choose a matrix-free >> version. You seem to imply above that it does not. >> >> 2) Do you mean that you do not call it? You do not have too, since you >> can use the finite difference approximations offered by SNES >> >> Thanks, >> >> Matt >> >> >>> >>> TSCreate(PETSC_COMM_WORLD, &ts); >>> TSSetProblemType(ts, TS_NONLINEAR); >>> TSSetType(ts, TSTHETA); >>> TSThetaSetTheta(ts, 1.0); >>> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >>> DMSetMatType(da, MATAIJ); >>> DMCreateMatrix(da, &J); >>> TSSetDM(ts, da); >>> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >>> TSSetDuration(ts, timeStepsMax, Tend); >>> // Evaluate initial conditions >>> FormInitialSolution(&appctx); >>> TSSetSolution(ts, solution); >>> dt = 1.0; >>> TSSetInitialTimeStep(ts, 0.0, dt); >>> TSSolve(ts, solution); >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon Oct 27 13:09:24 2014 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 27 Oct 2014 14:09:24 -0400 Subject: [petsc-users] unreliable AMG in PETSc In-Reply-To: <87r3xyhczv.fsf@jedbrown.org> References: <5447AF56.3000805@sintef.no> <50EB5E6E-DA3B-432C-B6F2-0DBE3E5277D1@mcs.anl.gov> <5448B441.3090304@sintef.no> <54490316.6010008@sintef.no> <87bnp2j3iz.fsf@jedbrown.org> <54490B52.6060004@sintef.no> <8738aej2lg.fsf@jedbrown.org> <606A00D3-EE0D-47AE-BE67-9653D4093335@mcs.anl.gov> <87r3xyhczv.fsf@jedbrown.org> Message-ID: Just to reiterate,add, emphasize what what was said. AMG solvers are very tricky to make robust to all -- even reasonable -- inputs. To say nothing of all inputs, but there are a finite number of ways the code can segv and we want fix them -- one by one. This code is in fact sable, for 2-3 year now but users find valid ways to create a segv or mysterious failure and we fix these. One at a time, but there is a finite number. Many PC can be unsymmetric (ILU, MG with F-cycles) and in the case of ILU indefinite to boot (thus the fix that Barry talked about). And as Jed says CG seems to be pretty robust to this. I've seen PETSc CG return negative eigenvalues for large deformation elasticity that do in fact go indefinite (buckling) but it worked fine and Newton eventually converged to an SPD system. AMG (MG in general) should not be indefinite with valid inputs, for scalar Laplacian at least. So a V-cycle preconditioner for an SPD matrix, with an SPD smoother should stay SPD. It might die but it should not be indefinite. Mark On Thu, Oct 23, 2014 at 2:12 PM, Jed Brown wrote: > Barry Smith writes: > > It is any interesting question if there is anything one can ?do? to > > a preconditioner to insure that it is SPD or detect if it may not > > be BEFORE actually running the CG with that preconditioned. > > You can use an eigensolver for the extreme eigenvalues of the iteration > matrix for the smoother. And if it's unstable, report that. This is > one thing that the petsc-eig plugin (which I'll update) can do, but it's > too expensive to do by default. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 27 13:11:05 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 14:11:05 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: Hi Matt, Thank you for your reply. That's why I am confused and need help. I set printf routines as I run my code, but it never output it on the screen. Thanks! On Mon, Oct 27, 2014 at 2:05 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 1:02 PM, Sharp Stone wrote: > >> Hi Matt, >> >> Thank you for your reply and sorry for the confusion. >> >> No. What I was trying to say is my code has never called the >> TSSetIJacobian() routine. I don't know why this happens, and how to resolve >> this either. >> > > If your code has not called TSSetIJacobian(), I can only assume it is > because you did not put it in. However, in > the code fragment you sent, it was definitely there. > > Thanks, > > Matt > > >> Thank you in advance! >> >> On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley >> wrote: >> >>> On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone >>> wrote: >>> >>>> Dear All, >>>> >>>> I'm using the TS theta solver for backward Euler method. But I found >>>> that the TSSetIJacobian() routine has never been called. I did the code >>>> just following what the example does: src/ts/examples/tutorials/ex17. I can >>>> run the example without errors. But when I run my code, it won't be as good >>>> as the example as stated above. I also attached the piece of my setups to >>>> this email. >>>> >>>> How could I fix this problem? Thank you in advance! >>>> >>> >>> I cannot understand the paragraph above. >>> >>> 1) TS ex17 does call TSSetIJacobian(), unless you choose a matrix-free >>> version. You seem to imply above that it does not. >>> >>> 2) Do you mean that you do not call it? You do not have too, since you >>> can use the finite difference approximations offered by SNES >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> >>>> TSCreate(PETSC_COMM_WORLD, &ts); >>>> TSSetProblemType(ts, TS_NONLINEAR); >>>> TSSetType(ts, TSTHETA); >>>> TSThetaSetTheta(ts, 1.0); >>>> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >>>> DMSetMatType(da, MATAIJ); >>>> DMCreateMatrix(da, &J); >>>> TSSetDM(ts, da); >>>> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >>>> TSSetDuration(ts, timeStepsMax, Tend); >>>> // Evaluate initial conditions >>>> FormInitialSolution(&appctx); >>>> TSSetSolution(ts, solution); >>>> dt = 1.0; >>>> TSSetInitialTimeStep(ts, 0.0, dt); >>>> TSSolve(ts, solution); >>>> >>>> -- >>>> Best regards, >>>> >>>> Feng >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 13:13:59 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 13:13:59 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: On Mon, Oct 27, 2014 at 1:11 PM, Sharp Stone wrote: > Hi Matt, > > Thank you for your reply. > > That's why I am confused and need help. I set printf routines as I run my > code, but it never output it on the screen. > Here is one point of confusion: 1) TSSetIJacobian() sets a callback routine for creating the Jacobian. This was definitely called 2) FormIJacobian() is your function which I assume creates the Jacobian. This is what I think you are saying is not called. Not all TS solvers use a linear solve. Your first step is to run with -ts_view so that we can see what solver is actually being used. Matt > Thanks! > > On Mon, Oct 27, 2014 at 2:05 PM, Matthew Knepley > wrote: > >> On Mon, Oct 27, 2014 at 1:02 PM, Sharp Stone wrote: >> >>> Hi Matt, >>> >>> Thank you for your reply and sorry for the confusion. >>> >>> No. What I was trying to say is my code has never called the >>> TSSetIJacobian() routine. I don't know why this happens, and how to resolve >>> this either. >>> >> >> If your code has not called TSSetIJacobian(), I can only assume it is >> because you did not put it in. However, in >> the code fragment you sent, it was definitely there. >> >> Thanks, >> >> Matt >> >> >>> Thank you in advance! >>> >>> On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley >>> wrote: >>> >>>> On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone >>>> wrote: >>>> >>>>> Dear All, >>>>> >>>>> I'm using the TS theta solver for backward Euler method. But I found >>>>> that the TSSetIJacobian() routine has never been called. I did the code >>>>> just following what the example does: src/ts/examples/tutorials/ex17. I can >>>>> run the example without errors. But when I run my code, it won't be as good >>>>> as the example as stated above. I also attached the piece of my setups to >>>>> this email. >>>>> >>>>> How could I fix this problem? Thank you in advance! >>>>> >>>> >>>> I cannot understand the paragraph above. >>>> >>>> 1) TS ex17 does call TSSetIJacobian(), unless you choose a >>>> matrix-free version. You seem to imply above that it does not. >>>> >>>> 2) Do you mean that you do not call it? You do not have too, since >>>> you can use the finite difference approximations offered by SNES >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> >>>>> TSCreate(PETSC_COMM_WORLD, &ts); >>>>> TSSetProblemType(ts, TS_NONLINEAR); >>>>> TSSetType(ts, TSTHETA); >>>>> TSThetaSetTheta(ts, 1.0); >>>>> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >>>>> DMSetMatType(da, MATAIJ); >>>>> DMCreateMatrix(da, &J); >>>>> TSSetDM(ts, da); >>>>> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >>>>> TSSetDuration(ts, timeStepsMax, Tend); >>>>> // Evaluate initial conditions >>>>> FormInitialSolution(&appctx); >>>>> TSSetSolution(ts, solution); >>>>> dt = 1.0; >>>>> TSSetInitialTimeStep(ts, 0.0, dt); >>>>> TSSolve(ts, solution); >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Feng >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon Oct 27 13:17:56 2014 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 27 Oct 2014 14:17:56 -0400 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> Message-ID: The null space for GAMG is not critical but useful for elasticity. If you in fact have an indefinite operator (eg, not "pinned) the you need to use an iterative coarse grid solver. You are using '-fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_type preonly'. And you have a hard elasticity problem. You are going to want to start with a stronger solver. Use cg instead of preonly. As Matt said start with MUMPS, then go to CG/GAMG, then you can see how far you can cut the _0_ solver down. Mark On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley wrote: > On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali > wrote: > >> Matt >> >> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >> >> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >> wrote: >> >>> Matt >>> >>> Sorry about that (I always forget it). The output for the smallest >>> problem is now attached (see log.txt). I am also attaching some results >>> that compare results obtained using FS/LSC and the direct solver (MUMPS), >>> again for the smallest problem. The difference, as you can see is >>> insignificant O(1E-6). >>> >> >> 1) How do you use MUMPS if you have a saddle point >> >> I simply used -pc_type lu -pc_factor_mat_solver_package mumps. >> >> >> 2) You can see from the output that something is seriously wrong with >> the preconditioner. It looks like it has a null space. >> Did you add the elastic null modes to GAMG? Without this, it is not >> going to work. We have helper functions for this: >> >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >> >> you could just copy that code. And then use >> >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >> >> I don't see it in the output, so I think this is your problem. >> >> In order to test, I would first use MUMPS as the A00 solver and get the >> Schur stuff worked out. Then I would >> replace MUMPS with GAMG and tune it until I get back my original >> convergence. >> >> I will try this with MatNullSpaceCreateRigidBody. Btw does it matter if >> some nodes are pinned? >> > > No these are null modes of the operator, not of the particular problem. > > Matt > > >> Tabrez >> >> >> Thanks, >> >> Matt >> >> >>> Also, I did pass 'upper' and 'full' to >>> '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't >>> improve (in fact, it increases slightly). The attached log is with 'upper'. >>> >>> Regards, >>> >>> Tabrez >>> >>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>> >>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>> wrote: >>> >>>> Hello >>>> >>>> I am using the following options (below) for solving linear >>>> elasticity/poroelasticity problems involving slip between two surfaces >>>> involving non-trivial geometries, i.e., elements with high aspect ratios, >>>> large contrasts in material properties etc. The constraints are imposed >>>> using Lagrange Multipliers. >>>> >>>> A picture (shows displacement magnitude) is attached. The boundary >>>> nodes, i.e., the base and the four side are pinned. >>>> >>>> The following options appear to work well for the saddle point problem: >>>> >>>> -pc_type fieldsplit -pc_fieldsplit_type schur >>>> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >>>> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >>>> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >>>> -ksp_monitor >>>> >>>> However, the number of iterations keep on increasing with the problems >>>> size (see attached plot), e.g., >>>> >>>> 120K Tets *507* Iterations (KSP Residual norm >>>> 8.827362494659e-05) in 17 secs on 3 cores >>>> 1 Million Tets *1374* Iterations (KSP Residual norm >>>> 7.164704416296e-05) in 117 secs on 20 cores >>>> 8 Million Tets *2495* Iterations (KSP Residual norm >>>> 9.101247550026e-05) in 225 secs on 160 cores >>>> >>>> So what other options should I try to improve solver performance? Any >>>> tips/insights would be appreciated as preconditioning is black magic to me. >>>> >>> >>> For reports, always run with >>> >>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >>> >>> so that we can see exactly what you used. >>> >>> I believe the default is a diagonal factorization. Since your outer >>> iterates are increasing, I would strengthen this >>> to either upper or full >>> >>> -pc_fieldsplit_schur_factorization_type >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thanks in advance. >>>> >>>> Tabrez >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfadams at lbl.gov Mon Oct 27 13:31:45 2014 From: mfadams at lbl.gov (Mark Adams) Date: Mon, 27 Oct 2014 14:31:45 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <5202897.4tZeA0US6V@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <2815437.UQjMVro9BF@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> Message-ID: On Thu, Oct 23, 2014 at 2:32 AM, Filippo Leonardi < filippo.leonardi at sam.math.ethz.ch> wrote: > Tested on this branch: the program no longer crashes, but the solution is > actually still wrong. > > To reproduce you can compile a version with the for cycle starting from i > = 0 > and one from i = 1 and see the difference with: mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > and without mg. > > I'm not sure what you are asking. starting i=0 or i=1, and with and without mg. So four possibilities? The solutions will be different for i=0 or 1 in this code: ierr = VecSet(ctx.rhs, 1); CHKERRQ(ierr); ierr = setup(&ctx); CHKERRQ(ierr); PetscViewer viewer; PetscViewerDrawOpen(PETSC_COMM_WORLD,NULL,NULL,0,0,300,300,&viewer); for(int i = 0; i < N; ++i) { ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); ierr = KSPGetSolution(ksp, &phi); CHKERRQ(ierr); ierr = VecView(ctx.rhs, viewer); CHKERRQ(ierr); ierr = VecView(phi, viewer); CHKERRQ(ierr); } > On Wednesday 22 October 2014 09:31:40 Mark Adams wrote: > > I think we have a fix for this. You can test by using the branch. In > > PETSC_DIR: > > > > $ git checkout mark/ksp-zero-eig > > $ make > > > > Now remake your code and test, > > > > > > On Wed, Oct 22, 2014 at 8:36 AM, Filippo Leonardi < > > > > filippo.leonardi at sam.math.ethz.ch> wrote: > > > Granted that I solved my problem by **not** solving the system if the > Rhs > > > is > > > zero. Here's some result: > > > > > > 1) Using Multigrid with debug crashes (see below) > > > 2) Using MG without debug doesn't crash, but gives wrong solution > > > 3) defaults or gamg or wathever*works both with and without debug > > > > > > Now weirder: > > > 4) changing line 145 from > > > > > > for(int i = 1; i < N; ++i) { > > > > > > to > > > > > > for(int i = 0; i < N; ++i) { > > > > > > i.e. avoiding broken case 1) above (rhs never 0) makes 1), 2) and 3) > work > > > perfectly (and any combination thereof) > > > > > > (You can look at the differences in the pictures given by Draw to see > that > > > the > > > solution is more or less wrong.) > > > > > > Details: > > > > > > Release build with no debug: > > > > > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > > > -ksp_monitor - > > > ksp_monitor_true_residual > > > > > > 0 KSP Residual norm 0.000000000000e+00 > > > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > > > > > > 0.000000000000e+00 ||r(i)||/||b|| -nan > > > > > > 0 KSP Residual norm 3.964678029172e+02 > > > 0 KSP preconditioned resid norm 3.964678029172e+02 true resid norm > > > > > > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > > > 1 KSP Residual norm 6.290716828453e-13 > > > 1 KSP preconditioned resid norm 6.290716828453e-13 true resid norm > > > > > > 3.788861141557e+00 ||r(i)||/||b|| 3.388860427931e+00 > > > > > > 0 KSP Residual norm 7.929356058344e+02 > > > 0 KSP preconditioned resid norm 7.929356058344e+02 true resid norm > > > > > > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > > > 1 KSP Residual norm 1.258143365691e-12 > > > 1 KSP preconditioned resid norm 1.258143365691e-12 true resid norm > > > > > > 7.577722283114e+00 ||r(i)||/||b|| 3.388860427931e+00 > > > > > > i.e. I don't see the residual to be zero each time. > > > > > > Debug build: > > > > > > mpirun -np 4 ./test1 -pc_type mg -pc_mg_levels 5 -draw_pause -1 > > > -ksp_monitor - > > > ksp_monitor_true_residual > > > [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message > > > -------------------------------------------------------------- > > > [0]PETSC ERROR: Null argument, when expecting valid pointer > > > [0]PETSC ERROR: Null Pointer: Parameter # 3 > > > [0]PETSC ERROR: See > http://www.mcs.anl.gov/petsc/documentation/faq.html > > > for > > > trouble shooting. > > > [0]PETSC ERROR: Petsc Release Version 3.5.0, Jun, 30, 2014 > > > [0]PETSC ERROR: ./test1 on a arch-linux2-cxx-debug named > Besikovitch-II by > > > filippo Wed Oct 22 14:20:46 2014 > > > [0]PETSC ERROR: Configure options --with-clanguage=cxx > > > --download-scalapack -- > > > download-mumps --download-parmetis --download-metis --download-fftw > > > [0]PETSC ERROR: #1 KSPComputeEigenvalues() line 119 in > > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > > > [0]PETSC ERROR: #2 KSPChebyshevComputeExtremeEigenvalues_Private() line > > > 328 in > > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > > > [0]PETSC ERROR: #3 KSPSolve_Chebyshev() line 373 in > > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/impls/cheby/cheby.c > > > [0]PETSC ERROR: #4 KSPSolve() line 459 in > > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/ksp/interface/itfunc.c > > > [0]PETSC ERROR: #5 PCMGMCycle_Private() line 19 in > > > /home/filippo/Workspace/petsc-3.5.0/src/ksp/pc/impls/mg/mg.c > > > > > > Debug build again with defaults: > > > > > > mpirun -np 4 ./test1 -draw_pause -1 -ksp_monitor > > > -ksp_monitor_true_residual > > > > > > 0 KSP Residual norm 0.000000000000e+00 > > > 0 KSP preconditioned resid norm 0.000000000000e+00 true resid norm > > > > > > 0.000000000000e+00 ||r(i)||/||b|| -nan > > > > > > 0 KSP Residual norm 1.839181443725e+00 > > > 0 KSP preconditioned resid norm 1.839181443725e+00 true resid norm > > > > > > 1.118033988750e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > > > 1 KSP Residual norm 1.767908128952e+00 > > > 1 KSP preconditioned resid norm 1.767908128952e+00 true resid norm > > > > > > 1.182712814433e+00 ||r(i)||/||b|| 1.057850500373e+00 > > > [...] > > > > > > 82 KSP Residual norm 1.642117972681e-05 > > > 82 KSP preconditioned resid norm 1.642117972681e-05 true resid norm > > > > > > 1.556660502826e-05 ||r(i)||/||b|| 1.392319480883e-05 > > > [...] > > > 0 KSP Residual norm 3.678362887450e+00 > > > > > > 0 KSP preconditioned resid norm 3.678362887450e+00 true resid norm > > > > > > 2.236067977500e+00 ||r(i)||/||b|| 1.000000000000e+00 > > > > > > 1 KSP Residual norm 3.535816257904e+00 > > > 1 KSP preconditioned resid norm 3.535816257904e+00 true resid norm > > > > > > 2.365425628867e+00 ||r(i)||/||b|| 1.057850500373e+00 > > > > > > On Wednesday 22 October 2014 07:12:50 you wrote: > > > > > On Oct 22, 2014, at 1:09 AM, Leonardi Filippo > > > > > wrote: > > > > > > > > > > For 1): I must be missing something then: in main i set and scale > > > > > > ctx.rhs, > > > > > > > > then I call KSPSolve that calls ComputeRhs that further scales > ctx.rhs > > > > > and copy it to b, which is then used by the solve. To my > understanding > > > > > > I > > > > > > > > never override ctx.rhs. > > > > > > > > > Oh yes. I missed that. > > > > > > > > What I observed was when I ran your code each solve had an initial > > > > > > > > residual norm of exactly zero. I?m not sure why that happens. > > > > > > > > > For 2): that was a small typo, problem is present no matter what > > > > > > number of > > > > > > > > levels I use. > > > > > > > > > > Thanks, > > > > > Filippo > > > > > > > > > >> On 22-ott-2014, at 04:27, "Barry Smith" > wrote: > > > > >> > > > > >> > > > > >> There are a few issues with your code > > > > >> > > > > >> 1) If you use KSPSetComputeRHS() then the function you provide is > > > > > > called > > > > > > > >> immediately before the solve automatically for every solve. Thus > your > > > > >> code with the vector operations here > > > > >> > > > > >> ierr = VecSet(ctx.rhs, (float) i * mx); CHKERRQ(ierr); > > > > >> > > > > >> ierr = VecScale(ctx.rhs, rank); CHKERRQ(ierr); > > > > >> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > > >> > > > > >> does nothing. And each linear solve ends up being with a zero > right > > > > > > hand > > > > > > > >> side. > > > > >> > > > > >> 2) -pc_type mg doesn?t give enough information for the multigrid > to > > > > > > know > > > > > > > >> how many levels to use. In fact it is always using one hence it is > > > > >> not > > > > >> multigrid. You need to use -p_mg_levels 2 or 3 or whatever to > get it > > > > > > to > > > > > > > >> use more than one level. > > > > >> > > > > >> Barry > > > > >> > > > > >>> On Oct 21, 2014, at 1:50 AM, Filippo Leonardi > > > > >>> wrote: > > > > >>> > > > > >>> Here, > > > > >>> > > > > >>> run this e.g. with > > > > >>> mpirun -np 4 ./test1 -draw_pause -1 -pc_type mg -pc_mg_galerkin > > > > >>> > > > > >>> In Debug mode it crashes, in standard mode it fails. Using Petsc > > > > >>> current. > > > > >>> > > > > >>> Run without mg and works! > > > > >>> > > > > >>> Hope it's not me being stupid and doing something terribly wrong! > > > > >>> > > > > >>> Best, > > > > >>> Filippo > > > > >>> > > > > >>>> On Monday 20 October 2014 07:29:04 Barry Smith wrote: > > > > >>>> Can you send us the code: to petsc-maint at mcs.anl.gov or > > > > >>>> petsc-users at mcs.anl.gov ? Or something that reproduces the > problem? > > > > >>>> > > > > >>>> Barry > > > > >>>> > > > > >>>>> On Oct 20, 2014, at 3:31 AM, Filippo Leonardi > > > > >>>>> wrote: > > > > >>>>> > > > > >>>>> Hi, > > > > >>>>> > > > > >>>>> I have a very specific problem that I cannot figure out with > PCMG > > > > > > and > > > > > > > >>>>> multiple solves. I got a linear system that I solve many times, > > > > > > with > > > > > > > >>>>> same > > > > >>>>> matrix but different RHS. I can successfully solve the system > with > > > > >>>>> standard techniques, such as default solver or LU or PCGAMG. > Even > > > > > > MG > > > > > > > >>>>> works if I destroy the ksp each time or I recompute the > matrices > > > > >>>>> at > > > > >>>>> each > > > > >>>>> time. But when I try and go to MG and not recomputing the > matrices > > > > >>>>> each > > > > >>>>> time the solver fails. Any idea? > > > > >>>>> > > > > >>>>> Here some detail: the setup: > > > > >>>>> ierr = KSPSetDMActive(ksp, PETSC_FALSE); CHKERRQ(ierr); > > > > >>>>> ierr = KSPSetDM(ksp, da); CHKERRQ(ierr); > > > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, ctx); > > > > > > CHKERRQ(ierr); > > > > > > > >>>>> ierr = KSPSetComputeRHS(ksp, ComputeRHS, ctx); CHKERRQ(ierr); > > > > >>>>> ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr); > > > > >>>>> > > > > >>>>> Then I solve as usual, for a large number of time steps: > > > > >>>>> ierr = KSPSolve(ksp, NULL, NULL); CHKERRQ(ierr); > > > > >>>>> ierr = KSPGetSolution(ksp, &phi); > > > > >>>>> > > > > >>>>> The solver converges and does that in a reasonable number of > > > > >>>>> iterations: > > > > >>>>> Linear solve converged due to CONVERGED_RTOL iterations 7 > > > > >>>>> And ksp_view and ksp_monitor do not show any weird stuff. > > > > >>>>> > > > > >>>>> - Weirdly enough using any solver (for instance cg+bjacobi or > > > > >>>>> gamg) > > > > >>>>> everything works (So the matrix and RHS are working fine). > > > > >>>>> - But the problem persists with Galerkin matrices > > > > >>>>> (-pc_mg_galerkin) > > > > >>>>> (So is > > > > >>>>> not a ComputeMatrix problem). > > > > >>>>> - If I do: > > > > >>>>> ierr = KSPSetComputeOperators(ksp, ComputeMatrix, this); > > > > >>>>> CHKERRQ(ierr); > > > > >>>>> between each solve or destroy the ksp entirely each time the > > > > > > solution > > > > > > > >>>>> is > > > > >>>>> also perfect (So is not a boundary scaling or other stuff > > > > >>>>> problem). > > > > >>>>> - If I run MG with only 2 levels (so just coarse) I also get a > > > > >>>>> fine > > > > >>>>> result. > > > > >>>>> > > > > >>>>> Setup RHS is called each time as expected, setup matrix is > called > > > > > > just > > > > > > > >>>>> once, also as expected. > > > > >>>>> > > > > >>>>> The only thing I can think is that MG does not update some > value > > > > > > that > > > > > > > >>>>> actually needs to be recomputed. > > > > >>>>> > > > > >>>>> Any idea? > > > > > > > >>>>> The solution is not that different from: > > > http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutori > > > > > > > >>>>> als/ > > > > >>>>> ex29.c.html > > > > >>>>> > > > > >>>>> Best, > > > > >>>>> Filippo > > > > >>> > > > > >>> > > > > > > -- > > > Filippo Leonardi > > > SAM ETH Z?rich > > > (HG J 45) R?mistrasse 101, Z?rich (CH) > > -- > Filippo Leonardi > SAM ETH Z?rich > (HG J 45) R?mistrasse 101, Z?rich (CH) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 27 13:40:59 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 14:40:59 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: Hi Matt, Thank you very much for your reply. The ts_view output is attached below. I found my code results say "SNES has not been set up", and PC "has not been set up". Does this cause the problem? If so I do not see example ex17 explicitly set up the snes object? PS: I don't know why the -ts_view option does not work in my command line, so I use TSView() routine just before the TSSolve() routine. TS output: TS Object: 1 MPI processes type: theta maximum steps=100000000 maximum time=5e-06 total number of nonlinear solver iterations=0 total number of nonlinear solve failures=0 total number of linear solver iterations=0 total number of rejected steps=0 Theta=1 Extrapolation=no SNES Object: 1 MPI processes type: newtonls SNES has not been set up so information may be incomplete maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=0 total number of function evaluations=0 SNESLineSearch Object: 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using DEFAULT norm type for convergence test PC Object: 1 MPI processes type: ilu PC has not been set up so information may be incomplete ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=20904, cols=20904, bs=4 total: nonzeros=410816, allocated nonzeros=410816 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 5226 nodes, limit used is 5 On Mon, Oct 27, 2014 at 2:13 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 1:11 PM, Sharp Stone wrote: > >> Hi Matt, >> >> Thank you for your reply. >> >> That's why I am confused and need help. I set printf routines as I run my >> code, but it never output it on the screen. >> > > Here is one point of confusion: > > 1) TSSetIJacobian() sets a callback routine for creating the Jacobian. > This was definitely called > > 2) FormIJacobian() is your function which I assume creates the Jacobian. > This is what I think you are saying is not called. > > Not all TS solvers use a linear solve. Your first step is to run with > -ts_view so that we can see what solver is actually being used. > > Matt > > >> Thanks! >> >> On Mon, Oct 27, 2014 at 2:05 PM, Matthew Knepley >> wrote: >> >>> On Mon, Oct 27, 2014 at 1:02 PM, Sharp Stone wrote: >>> >>>> Hi Matt, >>>> >>>> Thank you for your reply and sorry for the confusion. >>>> >>>> No. What I was trying to say is my code has never called the >>>> TSSetIJacobian() routine. I don't know why this happens, and how to resolve >>>> this either. >>>> >>> >>> If your code has not called TSSetIJacobian(), I can only assume it is >>> because you did not put it in. However, in >>> the code fragment you sent, it was definitely there. >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Thank you in advance! >>>> >>>> On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley >>>> wrote: >>>> >>>>> On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone >>>>> wrote: >>>>> >>>>>> Dear All, >>>>>> >>>>>> I'm using the TS theta solver for backward Euler method. But I found >>>>>> that the TSSetIJacobian() routine has never been called. I did the code >>>>>> just following what the example does: src/ts/examples/tutorials/ex17. I can >>>>>> run the example without errors. But when I run my code, it won't be as good >>>>>> as the example as stated above. I also attached the piece of my setups to >>>>>> this email. >>>>>> >>>>>> How could I fix this problem? Thank you in advance! >>>>>> >>>>> >>>>> I cannot understand the paragraph above. >>>>> >>>>> 1) TS ex17 does call TSSetIJacobian(), unless you choose a >>>>> matrix-free version. You seem to imply above that it does not. >>>>> >>>>> 2) Do you mean that you do not call it? You do not have too, since >>>>> you can use the finite difference approximations offered by SNES >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> >>>>>> TSCreate(PETSC_COMM_WORLD, &ts); >>>>>> TSSetProblemType(ts, TS_NONLINEAR); >>>>>> TSSetType(ts, TSTHETA); >>>>>> TSThetaSetTheta(ts, 1.0); >>>>>> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >>>>>> DMSetMatType(da, MATAIJ); >>>>>> DMCreateMatrix(da, &J); >>>>>> TSSetDM(ts, da); >>>>>> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >>>>>> TSSetDuration(ts, timeStepsMax, Tend); >>>>>> // Evaluate initial conditions >>>>>> FormInitialSolution(&appctx); >>>>>> TSSetSolution(ts, solution); >>>>>> dt = 1.0; >>>>>> TSSetInitialTimeStep(ts, 0.0, dt); >>>>>> TSSolve(ts, solution); >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> >>>>>> Feng >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> Feng >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 13:43:14 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 13:43:14 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: On Mon, Oct 27, 2014 at 1:40 PM, Sharp Stone wrote: > Hi Matt, > > Thank you very much for your reply. > > The ts_view output is attached below. I found my code results say "SNES > has not been set up", and PC "has not been set up". Does this cause the > problem? If so I do not see example ex17 explicitly set up the snes object? > Never ever ever send part of an error message. Send the entire thing copied from the terminal. > PS: I don't know why the -ts_view option does not work in my command line, > so I use TSView() routine just before the TSSolve() routine. > Always check the return code from all calls. One of your calls is failing and you do not notice. Matt > > > > TS output: > > TS Object: 1 MPI processes > type: theta > maximum steps=100000000 > maximum time=5e-06 > total number of nonlinear solver iterations=0 > total number of nonlinear solve failures=0 > total number of linear solver iterations=0 > total number of rejected steps=0 > Theta=1 > Extrapolation=no > SNES Object: 1 MPI processes > type: newtonls > SNES has not been set up so information may be incomplete > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=0 > SNESLineSearch Object: 1 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 1 MPI processes > type: ilu > PC has not been set up so information may be incomplete > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > matrix ordering: natural > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=20904, cols=20904, bs=4 > total: nonzeros=410816, allocated nonzeros=410816 > total number of mallocs used during MatSetValues calls =0 > using I-node routines: found 5226 nodes, limit used is 5 > > > > On Mon, Oct 27, 2014 at 2:13 PM, Matthew Knepley > wrote: > >> On Mon, Oct 27, 2014 at 1:11 PM, Sharp Stone wrote: >> >>> Hi Matt, >>> >>> Thank you for your reply. >>> >>> That's why I am confused and need help. I set printf routines as I run >>> my code, but it never output it on the screen. >>> >> >> Here is one point of confusion: >> >> 1) TSSetIJacobian() sets a callback routine for creating the Jacobian. >> This was definitely called >> >> 2) FormIJacobian() is your function which I assume creates the >> Jacobian. This is what I think you are saying is not called. >> >> Not all TS solvers use a linear solve. Your first step is to run with >> -ts_view so that we can see what solver is actually being used. >> >> Matt >> >> >>> Thanks! >>> >>> On Mon, Oct 27, 2014 at 2:05 PM, Matthew Knepley >>> wrote: >>> >>>> On Mon, Oct 27, 2014 at 1:02 PM, Sharp Stone >>>> wrote: >>>> >>>>> Hi Matt, >>>>> >>>>> Thank you for your reply and sorry for the confusion. >>>>> >>>>> No. What I was trying to say is my code has never called the >>>>> TSSetIJacobian() routine. I don't know why this happens, and how to resolve >>>>> this either. >>>>> >>>> >>>> If your code has not called TSSetIJacobian(), I can only assume it is >>>> because you did not put it in. However, in >>>> the code fragment you sent, it was definitely there. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Thank you in advance! >>>>> >>>>> On Mon, Oct 27, 2014 at 1:57 PM, Matthew Knepley >>>>> wrote: >>>>> >>>>>> On Mon, Oct 27, 2014 at 12:52 PM, Sharp Stone >>>>>> wrote: >>>>>> >>>>>>> Dear All, >>>>>>> >>>>>>> I'm using the TS theta solver for backward Euler method. But I found >>>>>>> that the TSSetIJacobian() routine has never been called. I did the code >>>>>>> just following what the example does: src/ts/examples/tutorials/ex17. I can >>>>>>> run the example without errors. But when I run my code, it won't be as good >>>>>>> as the example as stated above. I also attached the piece of my setups to >>>>>>> this email. >>>>>>> >>>>>>> How could I fix this problem? Thank you in advance! >>>>>>> >>>>>> >>>>>> I cannot understand the paragraph above. >>>>>> >>>>>> 1) TS ex17 does call TSSetIJacobian(), unless you choose a >>>>>> matrix-free version. You seem to imply above that it does not. >>>>>> >>>>>> 2) Do you mean that you do not call it? You do not have too, since >>>>>> you can use the finite difference approximations offered by SNES >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> >>>>>>> TSCreate(PETSC_COMM_WORLD, &ts); >>>>>>> TSSetProblemType(ts, TS_NONLINEAR); >>>>>>> TSSetType(ts, TSTHETA); >>>>>>> TSThetaSetTheta(ts, 1.0); >>>>>>> TSSetIFunction(ts, NULL, FormIFunction, &appctx); >>>>>>> DMSetMatType(da, MATAIJ); >>>>>>> DMCreateMatrix(da, &J); >>>>>>> TSSetDM(ts, da); >>>>>>> TSSetIJacobian(ts, J, J, FormIJacobian, &appctx); >>>>>>> TSSetDuration(ts, timeStepsMax, Tend); >>>>>>> // Evaluate initial conditions >>>>>>> FormInitialSolution(&appctx); >>>>>>> TSSetSolution(ts, solution); >>>>>>> dt = 1.0; >>>>>>> TSSetInitialTimeStep(ts, 0.0, dt); >>>>>>> TSSolve(ts, solution); >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> >>>>>>> Feng >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Feng >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Mon Oct 27 13:46:40 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 27 Oct 2014 12:46:40 -0600 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: Message-ID: <8738a92vvz.fsf@jedbrown.org> Sharp Stone writes: > Hi Matt, > > Thank you very much for your reply. > > The ts_view output is attached below. I found my code results say "SNES has > not been set up", and PC "has not been set up". Does this cause the > problem? If so I do not see example ex17 explicitly set up the snes object? Those objects are set up on the first time step (inside TSSolve). > PS: I don't know why the -ts_view option does not work in my command line, Are you sure it doesn't output after the TSSolve? The example does. > so I use TSView() routine just before the TSSolve() routine. You should call it *after* the solve so we can see what happened during the solve. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Mon Oct 27 13:52:18 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 27 Oct 2014 12:52:18 -0600 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <2815437.UQjMVro9BF@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> Message-ID: <87zjch1h25.fsf@jedbrown.org> Barry Smith writes: >> On Oct 23, 2014, at 1:32 AM, Filippo Leonardi wrote: >> >> Tested on this branch: the program no longer crashes, but the solution is >> actually still wrong. > > Yes, we never said that this fixed the problem. > > Jed ? Oops, one solution discussed on the petsc-dev fork of this thread is -ksp_chebyshev_estimate_eigenvalues_random, which should provide reasonable estimates independent of right-hand side. I'm not sure whether this should be made default since it makes some debugging tasks more difficult. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From mpovolot at purdue.edu Mon Oct 27 13:51:43 2014 From: mpovolot at purdue.edu (Michael Povolotskyi) Date: Mon, 27 Oct 2014 14:51:43 -0400 Subject: [petsc-users] Orthogonality of eigenvectors in SLEPC Message-ID: <544E943F.5070007@purdue.edu> Hello, will the eigenvectors that correspond to degenerate eigenvalues of a Hermitian matrix be orthogonal? I'm using Krylov Schur method. Thank you, Michael. -- Michael Povolotskyi, PhD Research Assistant Professor Network for Computational Nanotechnology 207 S Martin Jischke Drive Purdue University, DLR, room 441-10 West Lafayette, Indiana 47907 phone: +1-765-494-9396 fax: +1-765-496-6026 From filippo.leonardi at sam.math.ethz.ch Mon Oct 27 14:02:43 2014 From: filippo.leonardi at sam.math.ethz.ch (Filippo Leonardi) Date: Mon, 27 Oct 2014 20:02:43 +0100 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> Message-ID: <7911368.Xkc5LuFApY@besikovitch-ii> On Monday 27 October 2014 14:31:45 Mark Adams wrote: > > I'm not sure what you are asking. starting i=0 or i=1, and with and > without mg. So four possibilities? The solutions will be different for > i=0 or 1 in this code: > Yes, 4 cases: i = 0, MG: wrong solution *(for *any* solution, even afterwards, i.e. i >= 0)* (and error, if debug) i = 1, MG: ok i = 0, CG: (or anything): ok i = 1, CG (or anything): ok > -ksp_chebyshev_estimate_eigenvalues_random Can try this but the problem, to me, seems deeper (because inherited from each solve afterwards). -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 593 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ETHZ.vcf Type: text/vcard Size: 594 bytes Desc: not available URL: From thronesf at gmail.com Mon Oct 27 14:05:43 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 15:05:43 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: <8738a92vvz.fsf@jedbrown.org> References: <8738a92vvz.fsf@jedbrown.org> Message-ID: Hi Jed and Matt, Thank you very much for your replies. This time I copied the whole output from my screen, from compile to execution. This time I put the TSView() routine just after the TSSolve() routine to make the tsview output. And I intentionally to do only 5 time steps to make the test. Thank you in advance! host206-47:streamer_Implicit_SG GPL$ make pStreamer /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O0 -fPIC -I/Users/GPL/local/PETSc/petsc-3.5.2/include -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 -lpthread -lssl -lcrypto -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin -lmpichf90 -lgfortran -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm -lclang_rt.osx -lmpichcxx -lc++ -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin -lclang_rt.osx -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin -lclang_rt.osx -ldl host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer time=0.000000 step=0 dt = 0.001000 time=0.002000 step=1 dt = 0.002000 time=0.005000 step=2 dt = 0.003000 time=0.009000 step=3 dt = 0.004000 time=0.014000 step=4 dt = 0.005000 time=0.020000 step=5 dt = 0.006000 Step time elapse = 0.854144 s Total time elapse = 5.276643 s TS Object: 1 MPI processes type: theta maximum steps=10 maximum time=2e-11 total number of nonlinear solver iterations=0 total number of nonlinear solve failures=0 total number of linear solver iterations=0 total number of rejected steps=5 Theta=1 Extrapolation=no SNES Object: 1 MPI processes type: newtonls maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=0 total number of function evaluations=1 SNESLineSearch Object: 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using DEFAULT norm type for convergence test PC Object: 1 MPI processes type: ilu PC has not been set up so information may be incomplete ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=20904, cols=20904, bs=4 total: nonzeros=410816, allocated nonzeros=410816 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 5226 nodes, limit used is 5 On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: > Sharp Stone writes: > > > Hi Matt, > > > > Thank you very much for your reply. > > > > The ts_view output is attached below. I found my code results say "SNES > has > > not been set up", and PC "has not been set up". Does this cause the > > problem? If so I do not see example ex17 explicitly set up the snes > object? > > Those objects are set up on the first time step (inside TSSolve). > > > PS: I don't know why the -ts_view option does not work in my command > line, > > Are you sure it doesn't output after the TSSolve? The example does. > > > so I use TSView() routine just before the TSSolve() routine. > > You should call it *after* the solve so we can see what happened during > the solve. > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Mon Oct 27 14:09:18 2014 From: jroman at dsic.upv.es (Jose E. Roman) Date: Mon, 27 Oct 2014 20:09:18 +0100 Subject: [petsc-users] Orthogonality of eigenvectors in SLEPC In-Reply-To: <544E943F.5070007@purdue.edu> References: <544E943F.5070007@purdue.edu> Message-ID: El 27/10/2014, a las 19:51, Michael Povolotskyi escribi?: > Hello, > will the eigenvectors that correspond to degenerate eigenvalues of a Hermitian matrix be orthogonal? > I'm using Krylov Schur method. > Thank you, > Michael. > > -- > Michael Povolotskyi, PhD > Research Assistant Professor > Network for Computational Nanotechnology > 207 S Martin Jischke Drive > Purdue University, DLR, room 441-10 > West Lafayette, Indiana 47907 > > phone: +1-765-494-9396 > fax: +1-765-496-6026 > Yes, they should. Let me know if that is not the case. Jose From knepley at gmail.com Mon Oct 27 14:09:28 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 14:09:28 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone wrote: > Hi Jed and Matt, > > Thank you very much for your replies. > > This time I copied the whole output from my screen, from compile to > execution. This time I put the TSView() routine just after the TSSolve() > routine to make the tsview output. And I intentionally to do only 5 time > steps to make the test. Thank you in advance! > You have to make the command line arguments work. Quit calling TSView() in the code. We need many more options for debugging. First track down why you are not processing options. Do the examples take options for you? Once that works, run with -ts_view -ts_monitor -snes_monitor -ksp_monitor_true_residual Matt > host206-47:streamer_Implicit_SG GPL$ make pStreamer > /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o > pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing > -Wno-unknown-pragmas -g -O0 -fPIC > -I/Users/GPL/local/PETSc/petsc-3.5.2/include > -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include > -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc > /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc > -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first > -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall > -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 > -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o > lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o > -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib > -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc > -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 > -lpthread -lssl -lcrypto > -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin > -lmpichf90 -lgfortran > -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 > -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm > -lclang_rt.osx -lmpichcxx -lc++ > -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin > -lclang_rt.osx > -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl > -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem > -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin > -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin > -lclang_rt.osx -ldl > host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer > > time=0.000000 step=0 > dt = 0.001000 > > time=0.002000 step=1 > dt = 0.002000 > > time=0.005000 step=2 > dt = 0.003000 > > time=0.009000 step=3 > dt = 0.004000 > > time=0.014000 step=4 > dt = 0.005000 > > time=0.020000 step=5 > dt = 0.006000 > > Step time elapse = 0.854144 s Total time elapse = 5.276643 s > TS Object: 1 MPI processes > type: theta > maximum steps=10 > maximum time=2e-11 > total number of nonlinear solver iterations=0 > total number of nonlinear solve failures=0 > total number of linear solver iterations=0 > total number of rejected steps=5 > Theta=1 > Extrapolation=no > SNES Object: 1 MPI processes > type: newtonls > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=1 > SNESLineSearch Object: 1 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 1 MPI processes > type: ilu > PC has not been set up so information may be incomplete > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > matrix ordering: natural > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=20904, cols=20904, bs=4 > total: nonzeros=410816, allocated nonzeros=410816 > total number of mallocs used during MatSetValues calls =0 > using I-node routines: found 5226 nodes, limit used is 5 > > > On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: > >> Sharp Stone writes: >> >> > Hi Matt, >> > >> > Thank you very much for your reply. >> > >> > The ts_view output is attached below. I found my code results say "SNES >> has >> > not been set up", and PC "has not been set up". Does this cause the >> > problem? If so I do not see example ex17 explicitly set up the snes >> object? >> >> Those objects are set up on the first time step (inside TSSolve). >> >> > PS: I don't know why the -ts_view option does not work in my command >> line, >> >> Are you sure it doesn't output after the TSSolve? The example does. >> >> > so I use TSView() routine just before the TSSolve() routine. >> >> You should call it *after* the solve so we can see what happened during >> the solve. >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 27 14:31:34 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 15:31:34 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: Hi Matt, Thank you for your reply and suggestion. I use the command line options to make the output on the screen. It was very long and attached below. Thanks a million in advance! 0 KSP preconditioned resid norm 5.607581107219e-11 true resid norm 9.787719465322e-12 ||r(i)||/||b|| 6.796750851600e-16 0 TS dt 2e-12 time 0 0 KSP preconditioned resid norm 3.622586114352e+17 true resid norm 5.225681998280e+16 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 3.704153274287e+17 true resid norm 1.035782762954e+17 ||r(i)||/||b|| 1.982100639294e+00 2 KSP preconditioned resid norm 2.842418683024e+17 true resid norm 7.903974218109e+16 ||r(i)||/||b|| 1.512524914587e+00 3 KSP preconditioned resid norm 2.393970839759e+17 true resid norm 4.943510895366e+16 ||r(i)||/||b|| 9.460030091754e-01 4 KSP preconditioned resid norm 2.271787411557e+17 true resid norm 4.669216610635e+16 ||r(i)||/||b|| 8.935133466162e-01 5 KSP preconditioned resid norm 2.360179849286e+17 true resid norm 4.108243474851e+16 ||r(i)||/||b|| 7.861640789094e-01 6 KSP preconditioned resid norm 2.619090922116e+17 true resid norm 4.955060344596e+16 ||r(i)||/||b|| 9.482131416009e-01 7 KSP preconditioned resid norm 2.333689240438e+17 true resid norm 4.336784470641e+16 ||r(i)||/||b|| 8.298982739609e-01 8 KSP preconditioned resid norm 3.244607818183e+17 true resid norm 5.407474341938e+16 ||r(i)||/||b|| 1.034788252274e+00 9 KSP preconditioned resid norm 3.024748487424e+17 true resid norm 6.388129145873e+16 ||r(i)||/||b|| 1.222448887624e+00 10 KSP preconditioned resid norm 2.127860348138e+17 true resid norm 3.704786520226e+16 ||r(i)||/||b|| 7.089575143389e-01 11 KSP preconditioned resid norm 4.652609350461e+16 true resid norm 8.497318285406e+15 ||r(i)||/||b|| 1.626068767331e-01 12 KSP preconditioned resid norm 2.920260091494e+16 true resid norm 4.651858168992e+15 ||r(i)||/||b|| 8.901915904036e-02 13 KSP preconditioned resid norm 2.527529714994e+16 true resid norm 3.813962069025e+15 ||r(i)||/||b|| 7.298496292505e-02 14 KSP preconditioned resid norm 2.224122069397e+16 true resid norm 4.355442499437e+15 ||r(i)||/||b|| 8.334687225268e-02 15 KSP preconditioned resid norm 2.040909225547e+16 true resid norm 5.536568280983e+15 ||r(i)||/||b|| 1.059492001772e-01 16 KSP preconditioned resid norm 1.834652616545e+16 true resid norm 3.131287565910e+15 ||r(i)||/||b|| 5.992112736559e-02 17 KSP preconditioned resid norm 1.760978175265e+16 true resid norm 3.245011634605e+15 ||r(i)||/||b|| 6.209738050790e-02 18 KSP preconditioned resid norm 1.385338263500e+16 true resid norm 3.596318213647e+15 ||r(i)||/||b|| 6.882007391247e-02 19 KSP preconditioned resid norm 1.488783377120e+16 true resid norm 2.508274919240e+15 ||r(i)||/||b|| 4.799899649588e-02 20 KSP preconditioned resid norm 1.320707251811e+16 true resid norm 2.042806338080e+15 ||r(i)||/||b|| 3.909166954193e-02 21 KSP preconditioned resid norm 7.616453043037e+15 true resid norm 1.348500002677e+15 ||r(i)||/||b|| 2.580524423647e-02 22 KSP preconditioned resid norm 8.879333117289e+15 true resid norm 1.392661876801e+15 ||r(i)||/||b|| 2.665033726238e-02 23 KSP preconditioned resid norm 8.820610261864e+15 true resid norm 1.365093600480e+15 ||r(i)||/||b|| 2.612278360851e-02 24 KSP preconditioned resid norm 7.993346710634e+15 true resid norm 1.790642691323e+15 ||r(i)||/||b|| 3.426620088847e-02 25 KSP preconditioned resid norm 7.525623400757e+15 true resid norm 1.148549131012e+15 ||r(i)||/||b|| 2.197893272859e-02 26 KSP preconditioned resid norm 7.109494224141e+15 true resid norm 1.253098862904e+15 ||r(i)||/||b|| 2.397962339302e-02 27 KSP preconditioned resid norm 9.199926655972e+15 true resid norm 1.867276980934e+15 ||r(i)||/||b|| 3.573269444158e-02 28 KSP preconditioned resid norm 1.121259577312e+16 true resid norm 3.202544820818e+15 ||r(i)||/||b|| 6.128472459426e-02 29 KSP preconditioned resid norm 4.340241766754e+15 true resid norm 6.879443404198e+14 ||r(i)||/||b|| 1.316468052680e-02 30 KSP preconditioned resid norm 4.050053526588e+15 true resid norm 6.673524276162e+14 ||r(i)||/||b|| 1.277062836652e-02 31 KSP preconditioned resid norm 3.675002682668e+15 true resid norm 5.703629309888e+14 ||r(i)||/||b|| 1.091461231618e-02 32 KSP preconditioned resid norm 3.175854297440e+15 true resid norm 4.918363244864e+14 ||r(i)||/||b|| 9.411906898435e-03 33 KSP preconditioned resid norm 2.534108954658e+15 true resid norm 3.888660709280e+14 ||r(i)||/||b|| 7.441441539229e-03 34 KSP preconditioned resid norm 1.517593193539e+15 true resid norm 2.347716358056e+14 ||r(i)||/||b|| 4.492650641255e-03 35 KSP preconditioned resid norm 1.238737221149e+15 true resid norm 4.944967490035e+14 ||r(i)||/||b|| 9.462817468922e-03 36 KSP preconditioned resid norm 8.673662184926e+14 true resid norm 1.322920275286e+14 ||r(i)||/||b|| 2.531574396837e-03 37 KSP preconditioned resid norm 7.206551748402e+14 true resid norm 1.263301396525e+14 ||r(i)||/||b|| 2.417486171069e-03 38 KSP preconditioned resid norm 5.143409845488e+14 true resid norm 7.953540049469e+13 ||r(i)||/||b|| 1.522009960056e-03 39 KSP preconditioned resid norm 4.261910201431e+14 true resid norm 7.194412501613e+13 ||r(i)||/||b|| 1.376741352417e-03 40 KSP preconditioned resid norm 4.660682952219e+14 true resid norm 7.611539723637e+13 ||r(i)||/||b|| 1.456563894654e-03 41 KSP preconditioned resid norm 3.209169783039e+14 true resid norm 4.903825175831e+13 ||r(i)||/||b|| 9.384086474158e-04 0 KSP preconditioned resid norm 5.647841057638e+16 true resid norm 8.145493103696e+15 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 5.871590984109e+16 true resid norm 1.629681152051e+16 ||r(i)||/||b|| 2.000715157823e+00 2 KSP preconditioned resid norm 4.554798588549e+16 true resid norm 1.256325647853e+16 ||r(i)||/||b|| 1.542356775532e+00 3 KSP preconditioned resid norm 3.879559549735e+16 true resid norm 7.930640596410e+15 ||r(i)||/||b|| 9.736231429392e-01 4 KSP preconditioned resid norm 3.752982759961e+16 true resid norm 7.738765648717e+15 ||r(i)||/||b|| 9.500671782787e-01 5 KSP preconditioned resid norm 3.847367502586e+16 true resid norm 6.717373650268e+15 ||r(i)||/||b|| 8.246736649031e-01 6 KSP preconditioned resid norm 4.072409022879e+16 true resid norm 7.643006165053e+15 ||r(i)||/||b|| 9.383110473184e-01 7 KSP preconditioned resid norm 3.699615223300e+16 true resid norm 6.864744274739e+15 ||r(i)||/||b|| 8.427659550315e-01 8 KSP preconditioned resid norm 4.452919179646e+16 true resid norm 7.438964756180e+15 ||r(i)||/||b|| 9.132614393603e-01 9 KSP preconditioned resid norm 4.218921641123e+16 true resid norm 8.817640722426e+15 ||r(i)||/||b|| 1.082517732220e+00 10 KSP preconditioned resid norm 2.723066166891e+18 true resid norm 4.777495288747e+17 ||r(i)||/||b|| 5.865200826920e+01 11 KSP preconditioned resid norm 1.543090703715e+16 true resid norm 2.978277930467e+15 ||r(i)||/||b|| 3.656350686879e-01 12 KSP preconditioned resid norm 5.543831189057e+15 true resid norm 9.036431670746e+14 ||r(i)||/||b|| 1.109378100958e-01 13 KSP preconditioned resid norm 4.835578439031e+15 true resid norm 7.302084941558e+14 ||r(i)||/||b|| 8.964570773800e-02 14 KSP preconditioned resid norm 4.632268883924e+15 true resid norm 7.764942849443e+14 ||r(i)||/||b|| 9.532808819050e-02 15 KSP preconditioned resid norm 4.279662320804e+15 true resid norm 7.279053691611e+14 ||r(i)||/||b|| 8.936295935612e-02 16 KSP preconditioned resid norm 4.003909782590e+15 true resid norm 6.398763789786e+14 ||r(i)||/||b|| 7.855588002257e-02 17 KSP preconditioned resid norm 3.778065018292e+15 true resid norm 9.619695351776e+14 ||r(i)||/||b|| 1.180983794267e-01 18 KSP preconditioned resid norm 4.432967840881e+15 true resid norm 1.765362859847e+15 ||r(i)||/||b|| 2.167287894512e-01 19 KSP preconditioned resid norm 3.349713073090e+15 true resid norm 5.343310168427e+14 ||r(i)||/||b|| 6.559836341894e-02 20 KSP preconditioned resid norm 3.170973083244e+15 true resid norm 4.905381423867e+14 ||r(i)||/||b|| 6.022203151385e-02 21 KSP preconditioned resid norm 7.415620721710e+15 true resid norm 1.207250421775e+15 ||r(i)||/||b|| 1.482108457286e-01 22 KSP preconditioned resid norm 1.972990201442e+15 true resid norm 3.077655556372e+14 ||r(i)||/||b|| 3.778353891154e-02 23 KSP preconditioned resid norm 2.209676808077e+15 true resid norm 3.428403970507e+14 ||r(i)||/||b|| 4.208958164793e-02 24 KSP preconditioned resid norm 2.003292171306e+15 true resid norm 4.370066270665e+14 ||r(i)||/||b|| 5.365011319796e-02 25 KSP preconditioned resid norm 1.896876965875e+15 true resid norm 2.911542588080e+14 ||r(i)||/||b|| 3.574421524903e-02 26 KSP preconditioned resid norm 1.776666236555e+15 true resid norm 2.752957912495e+14 ||r(i)||/||b|| 3.379731438537e-02 27 KSP preconditioned resid norm 2.106240629900e+15 true resid norm 3.329282684307e+14 ||r(i)||/||b|| 4.087269661791e-02 28 KSP preconditioned resid norm 1.993718847257e+15 true resid norm 3.265440954005e+14 ||r(i)||/||b|| 4.008892908550e-02 29 KSP preconditioned resid norm 1.305019272390e+15 true resid norm 2.016059917794e+14 ||r(i)||/||b|| 2.475061843560e-02 30 KSP preconditioned resid norm 1.188165021558e+15 true resid norm 2.349019593212e+14 ||r(i)||/||b|| 2.883827367242e-02 31 KSP preconditioned resid norm 1.072198412006e+15 true resid norm 1.660526922017e+14 ||r(i)||/||b|| 2.038583669371e-02 32 KSP preconditioned resid norm 9.531727749289e+14 true resid norm 1.481127635008e+14 ||r(i)||/||b|| 1.818340051551e-02 33 KSP preconditioned resid norm 8.113851256730e+14 true resid norm 1.246949236714e+14 ||r(i)||/||b|| 1.530845610990e-02 34 KSP preconditioned resid norm 1.978771889204e+15 true resid norm 3.123099443158e+14 ||r(i)||/||b|| 3.834144113069e-02 35 KSP preconditioned resid norm 5.701229171299e+14 true resid norm 9.665965877556e+13 ||r(i)||/||b|| 1.186664300676e-02 36 KSP preconditioned resid norm 4.923209980376e+14 true resid norm 8.358449899120e+13 ||r(i)||/||b|| 1.026144125679e-02 37 KSP preconditioned resid norm 4.337271696165e+14 true resid norm 7.056948273999e+13 ||r(i)||/||b|| 8.663623164566e-03 38 KSP preconditioned resid norm 3.355561922811e+14 true resid norm 5.206223795577e+13 ||r(i)||/||b|| 6.391539136181e-03 39 KSP preconditioned resid norm 2.948796430248e+14 true resid norm 4.783720648479e+13 ||r(i)||/||b|| 5.872843531485e-03 40 KSP preconditioned resid norm 2.903844355011e+14 true resid norm 4.553096332971e+13 ||r(i)||/||b|| 5.589712341546e-03 41 KSP preconditioned resid norm 2.470740134237e+14 true resid norm 3.845088511805e+13 ||r(i)||/||b|| 4.720510425650e-03 42 KSP preconditioned resid norm 2.144333147632e+14 true resid norm 4.199300915233e+13 ||r(i)||/||b|| 5.155367344584e-03 43 KSP preconditioned resid norm 1.666875797521e+14 true resid norm 2.786670988964e+13 ||r(i)||/||b|| 3.421120064173e-03 44 KSP preconditioned resid norm 1.097596670806e+14 true resid norm 1.683182082477e+13 ||r(i)||/||b|| 2.066396792741e-03 45 KSP preconditioned resid norm 1.086486049833e+14 true resid norm 1.868163126673e+13 ||r(i)||/||b|| 2.293492981813e-03 46 KSP preconditioned resid norm 9.986155841385e+13 true resid norm 1.544797785039e+13 ||r(i)||/||b|| 1.896506160367e-03 47 KSP preconditioned resid norm 9.193832565781e+13 true resid norm 1.424417809082e+13 ||r(i)||/||b|| 1.748718943038e-03 48 KSP preconditioned resid norm 7.868838175665e+13 true resid norm 1.234477959732e+13 ||r(i)||/||b|| 1.515534963957e-03 49 KSP preconditioned resid norm 6.573261298706e+13 true resid norm 1.037919549254e+13 ||r(i)||/||b|| 1.274225557668e-03 50 KSP preconditioned resid norm 6.394833480259e+13 true resid norm 1.032066032237e+13 ||r(i)||/||b|| 1.267039354277e-03 51 KSP preconditioned resid norm 6.650072664422e+13 true resid norm 2.391163155719e+13 ||r(i)||/||b|| 2.935565870940e-03 52 KSP preconditioned resid norm 6.239223696051e+13 true resid norm 9.858797480463e+12 ||r(i)||/||b|| 1.210337711291e-03 53 KSP preconditioned resid norm 6.247432830217e+13 true resid norm 9.562224327829e+12 ||r(i)||/||b|| 1.173928233208e-03 54 KSP preconditioned resid norm 6.218810094746e+13 true resid norm 9.709963371483e+12 ||r(i)||/||b|| 1.192065753156e-03 55 KSP preconditioned resid norm 6.285033387840e+13 true resid norm 9.623710764220e+12 ||r(i)||/||b|| 1.181476755515e-03 56 KSP preconditioned resid norm 5.407058856328e+13 true resid norm 9.490503427247e+12 ||r(i)||/||b|| 1.165123253611e-03 0 KSP preconditioned resid norm 9.075431914728e+17 true resid norm 1.310345788054e+17 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 8.641655528143e+17 true resid norm 2.471138002109e+17 ||r(i)||/||b|| 1.885867092974e+00 2 KSP preconditioned resid norm 6.384433252903e+17 true resid norm 1.816534419632e+17 ||r(i)||/||b|| 1.386301567259e+00 3 KSP preconditioned resid norm 5.179227150533e+17 true resid norm 1.107056703740e+17 ||r(i)||/||b|| 8.448584441089e-01 4 KSP preconditioned resid norm 4.625114083211e+17 true resid norm 9.438923102711e+16 ||r(i)||/||b|| 7.203383403647e-01 5 KSP preconditioned resid norm 4.977065165555e+17 true resid norm 8.614996614554e+16 ||r(i)||/||b|| 6.574597860420e-01 6 KSP preconditioned resid norm 6.630931668382e+17 true resid norm 1.310543864565e+17 ||r(i)||/||b|| 1.000151163542e+00 7 KSP preconditioned resid norm 5.452278853431e+17 true resid norm 1.019521297074e+17 ||r(i)||/||b|| 7.780551564087e-01 8 KSP preconditioned resid norm 1.958476842674e+18 true resid norm 3.244825770314e+17 ||r(i)||/||b|| 2.476312588551e+00 9 KSP preconditioned resid norm 1.577830260132e+18 true resid norm 3.469004540765e+17 ||r(i)||/||b|| 2.647396261651e+00 10 KSP preconditioned resid norm 1.081897998863e+17 true resid norm 1.828781621979e+16 ||r(i)||/||b|| 1.395648109568e-01 11 KSP preconditioned resid norm 5.614465792481e+16 true resid norm 9.930812946447e+15 ||r(i)||/||b|| 7.578772746079e-02 12 KSP preconditioned resid norm 4.960081715579e+16 true resid norm 8.035214375965e+15 ||r(i)||/||b|| 6.132132792135e-02 13 KSP preconditioned resid norm 3.329600867897e+16 true resid norm 6.602562228940e+15 ||r(i)||/||b|| 5.038793797127e-02 14 KSP preconditioned resid norm 3.130960732609e+16 true resid norm 4.730473520325e+15 ||r(i)||/||b|| 3.610095566719e-02 15 KSP preconditioned resid norm 2.560518921527e+16 true resid norm 3.945110867578e+15 ||r(i)||/||b|| 3.010740297365e-02 16 KSP preconditioned resid norm 1.584886017619e+16 true resid norm 3.416780637867e+15 ||r(i)||/||b|| 2.607541207073e-02 17 KSP preconditioned resid norm 1.443824757417e+16 true resid norm 2.286337927483e+15 ||r(i)||/||b|| 1.744835560449e-02 18 KSP preconditioned resid norm 1.039145582094e+16 true resid norm 1.711526931765e+15 ||r(i)||/||b|| 1.306164332627e-02 19 KSP preconditioned resid norm 1.018242603191e+16 true resid norm 1.657426513724e+15 ||r(i)||/||b|| 1.264877201754e-02 20 KSP preconditioned resid norm 8.198983478967e+15 true resid norm 1.311419876212e+15 ||r(i)||/||b|| 1.000819698256e-02 21 KSP preconditioned resid norm 6.481494503642e+15 true resid norm 1.039872641976e+15 ||r(i)||/||b|| 7.935864345546e-03 22 KSP preconditioned resid norm 6.325821399407e+15 true resid norm 1.165377948147e+15 ||r(i)||/||b|| 8.893667295849e-03 23 KSP preconditioned resid norm 5.405294478066e+15 true resid norm 1.922243319095e+15 ||r(i)||/||b|| 1.466974089297e-02 24 KSP preconditioned resid norm 4.960676496180e+15 true resid norm 8.018184223519e+14 ||r(i)||/||b|| 6.119136106374e-03 25 KSP preconditioned resid norm 4.419138742847e+15 true resid norm 7.148418830282e+14 ||r(i)||/||b|| 5.455368266491e-03 26 KSP preconditioned resid norm 2.795870478230e+15 true resid norm 4.843283048525e+14 ||r(i)||/||b|| 3.696186985663e-03 27 KSP preconditioned resid norm 2.911201681995e+15 true resid norm 4.441742607418e+14 ||r(i)||/||b|| 3.389748452592e-03 28 KSP preconditioned resid norm 2.776697567476e+15 true resid norm 5.074732182061e+14 ||r(i)||/||b|| 3.872819089681e-03 29 KSP preconditioned resid norm 1.941501506934e+15 true resid norm 2.950418833153e+14 ||r(i)||/||b|| 2.251633774878e-03 30 KSP preconditioned resid norm 8.413150178496e+14 true resid norm 1.695476752862e+14 ||r(i)||/||b|| 1.293915520864e-03 0 KSP preconditioned resid norm 1.415973479575e+17 true resid norm 2.042710742436e+16 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.440919303250e+17 true resid norm 4.045863378014e+16 ||r(i)||/||b|| 1.980634503928e+00 2 KSP preconditioned resid norm 1.100352174605e+17 true resid norm 3.072773288453e+16 ||r(i)||/||b|| 1.504262558872e+00 3 KSP preconditioned resid norm 9.218323125716e+16 true resid norm 1.914173874994e+16 ||r(i)||/||b|| 9.370753456316e-01 4 KSP preconditioned resid norm 8.672799672395e+16 true resid norm 1.779286318286e+16 ||r(i)||/||b|| 8.710417394506e-01 5 KSP preconditioned resid norm 9.059066321416e+16 true resid norm 1.574500663517e+16 ||r(i)||/||b|| 7.707898288327e-01 6 KSP preconditioned resid norm 1.027848583976e+17 true resid norm 1.954996111922e+16 ||r(i)||/||b|| 9.570596909823e-01 7 KSP preconditioned resid norm 9.069106612200e+16 true resid norm 1.686428629204e+16 ||r(i)||/||b|| 8.255836688812e-01 8 KSP preconditioned resid norm 1.359707392060e+17 true resid norm 2.263432282464e+16 ||r(i)||/||b|| 1.108053252691e+00 9 KSP preconditioned resid norm 1.256447896822e+17 true resid norm 2.668588264948e+16 ||r(i)||/||b|| 1.306395570116e+00 10 KSP preconditioned resid norm 5.661761773340e+16 true resid norm 9.807578010522e+15 ||r(i)||/||b|| 4.801256392682e-01 11 KSP preconditioned resid norm 1.597811133832e+16 true resid norm 2.854210532053e+15 ||r(i)||/||b|| 1.397266129148e-01 12 KSP preconditioned resid norm 1.141170314391e+16 true resid norm 1.817769027418e+15 ||r(i)||/||b|| 8.898807793264e-02 13 KSP preconditioned resid norm 9.557798166843e+15 true resid norm 1.455009507586e+15 ||r(i)||/||b|| 7.122934624852e-02 14 KSP preconditioned resid norm 7.997842098400e+15 true resid norm 2.021581420109e+15 ||r(i)||/||b|| 9.896562338034e-02 15 KSP preconditioned resid norm 7.518666379332e+15 true resid norm 2.232979183117e+15 ||r(i)||/||b|| 1.093145072735e-01 16 KSP preconditioned resid norm 6.614897634500e+15 true resid norm 1.222952133628e+15 ||r(i)||/||b|| 5.986908024822e-02 17 KSP preconditioned resid norm 6.199437182741e+15 true resid norm 1.163257046730e+15 ||r(i)||/||b|| 5.694673369870e-02 18 KSP preconditioned resid norm 4.584922270319e+15 true resid norm 9.962391827866e+14 ||r(i)||/||b|| 4.877044811535e-02 19 KSP preconditioned resid norm 5.012002077991e+15 true resid norm 8.755757394511e+14 ||r(i)||/||b|| 4.286342266977e-02 20 KSP preconditioned resid norm 4.312819580189e+15 true resid norm 6.706322150678e+14 ||r(i)||/||b|| 3.283050317090e-02 21 KSP preconditioned resid norm 2.607093537931e+15 true resid norm 5.192589254045e+14 ||r(i)||/||b|| 2.542009079490e-02 22 KSP preconditioned resid norm 2.989363098445e+15 true resid norm 4.636989016003e+14 ||r(i)||/||b|| 2.270017442838e-02 23 KSP preconditioned resid norm 3.016276187245e+15 true resid norm 4.674022912685e+14 ||r(i)||/||b|| 2.288147222994e-02 24 KSP preconditioned resid norm 2.901859180715e+15 true resid norm 4.492447164798e+14 ||r(i)||/||b|| 2.199257619530e-02 25 KSP preconditioned resid norm 2.843000955863e+15 true resid norm 5.109023428494e+14 ||r(i)||/||b|| 2.501099799574e-02 26 KSP preconditioned resid norm 2.741940909323e+15 true resid norm 4.263494245202e+14 ||r(i)||/||b|| 2.087174731415e-02 27 KSP preconditioned resid norm 1.296293793548e+15 true resid norm 2.124064840417e+14 ||r(i)||/||b|| 1.039826538477e-02 28 KSP preconditioned resid norm 9.720512789714e+14 true resid norm 2.091841490290e+14 ||r(i)||/||b|| 1.024051740089e-02 29 KSP preconditioned resid norm 1.459179957917e+15 true resid norm 2.244723575034e+14 ||r(i)||/||b|| 1.098894487800e-02 30 KSP preconditioned resid norm 1.339547884067e+15 true resid norm 2.746667338979e+14 ||r(i)||/||b|| 1.344618835119e-02 31 KSP preconditioned resid norm 1.186062213325e+15 true resid norm 1.831856176134e+14 ||r(i)||/||b|| 8.967770806105e-03 32 KSP preconditioned resid norm 9.909356538944e+14 true resid norm 1.788908155328e+14 ||r(i)||/||b|| 8.757520671747e-03 33 KSP preconditioned resid norm 7.609550150745e+14 true resid norm 1.203572113192e+14 ||r(i)||/||b|| 5.892033992812e-03 34 KSP preconditioned resid norm 5.051711143811e+14 true resid norm 7.920683224412e+13 ||r(i)||/||b|| 3.877535404237e-03 35 KSP preconditioned resid norm 5.443461822227e+14 true resid norm 8.557198028123e+13 ||r(i)||/||b|| 4.189138408269e-03 36 KSP preconditioned resid norm 4.648220720251e+14 true resid norm 7.162139608842e+13 ||r(i)||/||b|| 3.506193735634e-03 37 KSP preconditioned resid norm 3.842876907480e+14 true resid norm 6.998069618798e+13 ||r(i)||/||b|| 3.425873998417e-03 38 KSP preconditioned resid norm 2.822934553902e+14 true resid norm 5.594039593962e+13 ||r(i)||/||b|| 2.738537316003e-03 39 KSP preconditioned resid norm 2.288268021627e+14 true resid norm 3.624972614121e+13 ||r(i)||/||b|| 1.774589293929e-03 40 KSP preconditioned resid norm 3.336880274482e+14 true resid norm 5.633095049288e+13 ||r(i)||/||b|| 2.757656741242e-03 41 KSP preconditioned resid norm 1.655073694079e+14 true resid norm 2.539321805860e+13 ||r(i)||/||b|| 1.243113747388e-03 42 KSP preconditioned resid norm 1.317678812236e+14 true resid norm 2.429513710603e+13 ||r(i)||/||b|| 1.189357680523e-03 0 KSP preconditioned resid norm 4.735355646927e+18 true resid norm 6.977368406398e+17 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 1.655056377336e+18 true resid norm 5.433322725028e+17 ||r(i)||/||b|| 7.787065851426e-01 2 KSP preconditioned resid norm 8.847192945348e+17 true resid norm 2.725627800798e+17 ||r(i)||/||b|| 3.906383670810e-01 3 KSP preconditioned resid norm 5.344946494044e+17 true resid norm 1.588736353868e+17 ||r(i)||/||b|| 2.276985048419e-01 4 KSP preconditioned resid norm 3.198139750974e+17 true resid norm 6.624530455539e+16 ||r(i)||/||b|| 9.494310848578e-02 5 KSP preconditioned resid norm 3.286368493485e+17 true resid norm 6.812427209313e+16 ||r(i)||/||b|| 9.763605434775e-02 6 KSP preconditioned resid norm 4.284030910438e+17 true resid norm 1.696553569581e+17 ||r(i)||/||b|| 2.431509232084e-01 7 KSP preconditioned resid norm 1.587968810131e+18 true resid norm 3.732963000458e+17 ||r(i)||/||b|| 5.350101618592e-01 8 KSP preconditioned resid norm 6.621891209840e+16 true resid norm 1.229792367414e+16 ||r(i)||/||b|| 1.762544695628e-02 9 KSP preconditioned resid norm 5.638286198157e+16 true resid norm 2.091986582514e+16 ||r(i)||/||b|| 2.998245843800e-02 10 KSP preconditioned resid norm 1.309067125882e+16 true resid norm 2.099826104432e+15 ||r(i)||/||b|| 3.009481486611e-03 11 KSP preconditioned resid norm 1.075925809079e+16 true resid norm 2.578010577039e+15 ||r(i)||/||b|| 3.694817912546e-03 12 KSP preconditioned resid norm 9.319606428298e+15 true resid norm 2.026298982436e+15 ||r(i)||/||b|| 2.904102040216e-03 13 KSP preconditioned resid norm 4.070270220193e+15 true resid norm 2.216532493559e+15 ||r(i)||/||b|| 3.176745678968e-03 14 KSP preconditioned resid norm 3.400982828163e+15 true resid norm 9.161069365806e+14 ||r(i)||/||b|| 1.312969135671e-03 15 KSP preconditioned resid norm 4.173720697476e+14 true resid norm 7.019272258636e+13 ||r(i)||/||b|| 1.006005681483e-04 16 KSP preconditioned resid norm 2.871344308441e+14 true resid norm 8.503915375563e+13 ||r(i)||/||b|| 1.218785490496e-04 17 KSP preconditioned resid norm 8.731930916187e+13 true resid norm 2.939915802842e+13 ||r(i)||/||b|| 4.213502328681e-05 18 KSP preconditioned resid norm 7.273170647297e+13 true resid norm 2.554194915512e+13 ||r(i)||/||b|| 3.660685184933e-05 19 KSP preconditioned resid norm 5.198384943683e+13 true resid norm 8.213409786802e+12 ||r(i)||/||b|| 1.177150081293e-05 20 KSP preconditioned resid norm 2.826075121950e+13 true resid norm 1.225537290577e+13 ||r(i)||/||b|| 1.756446297795e-05 0 KSP preconditioned resid norm 7.535755573391e+17 true resid norm 1.091530949677e+17 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 5.749658535033e+17 true resid norm 1.739986551510e+17 ||r(i)||/||b|| 1.594078987889e+00 2 KSP preconditioned resid norm 3.832130567981e+17 true resid norm 1.138657236506e+17 ||r(i)||/||b|| 1.043174485197e+00 3 KSP preconditioned resid norm 2.820261830557e+17 true resid norm 6.703201541200e+16 ||r(i)||/||b|| 6.141100756863e-01 4 KSP preconditioned resid norm 2.196333397214e+17 true resid norm 4.464718002673e+16 ||r(i)||/||b|| 4.090326530819e-01 5 KSP preconditioned resid norm 2.438749906836e+17 true resid norm 4.259961207836e+16 ||r(i)||/||b|| 3.902739733672e-01 6 KSP preconditioned resid norm 7.293863369143e+17 true resid norm 1.861518427200e+17 ||r(i)||/||b|| 1.705419738901e+00 7 KSP preconditioned resid norm 3.735172838302e+17 true resid norm 7.178953918634e+16 ||r(i)||/||b|| 6.576958647631e-01 8 KSP preconditioned resid norm 2.306462817079e+17 true resid norm 3.812340554474e+16 ||r(i)||/||b|| 3.492654565226e-01 9 KSP preconditioned resid norm 2.324630150277e+17 true resid norm 5.879635462988e+16 ||r(i)||/||b|| 5.386595281360e-01 10 KSP preconditioned resid norm 2.486084300490e+16 true resid norm 3.943582532938e+15 ||r(i)||/||b|| 3.612891172811e-02 11 KSP preconditioned resid norm 2.044336976942e+16 true resid norm 4.427406712026e+15 ||r(i)||/||b|| 4.056143999707e-02 12 KSP preconditioned resid norm 1.903964203761e+16 true resid norm 3.227987898262e+15 ||r(i)||/||b|| 2.957303133931e-02 13 KSP preconditioned resid norm 6.823460749064e+15 true resid norm 1.876444880951e+15 ||r(i)||/||b|| 1.719094526368e-02 14 KSP preconditioned resid norm 7.090194359812e+15 true resid norm 1.450790953432e+15 ||r(i)||/||b|| 1.329134051455e-02 15 KSP preconditioned resid norm 4.884314111756e+15 true resid norm 8.008003608330e+14 ||r(i)||/||b|| 7.336487903253e-03 16 KSP preconditioned resid norm 1.603299777176e+15 true resid norm 2.941609026573e+14 ||r(i)||/||b|| 2.694938725689e-03 17 KSP preconditioned resid norm 1.548805654760e+15 true resid norm 2.838534773138e+14 ||r(i)||/||b|| 2.600507822503e-03 18 KSP preconditioned resid norm 1.022454633110e+15 true resid norm 2.215058790848e+14 ||r(i)||/||b|| 2.029313773928e-03 19 KSP preconditioned resid norm 9.587995256078e+14 true resid norm 2.226912518453e+14 ||r(i)||/||b|| 2.040173500451e-03 20 KSP preconditioned resid norm 7.709554287637e+14 true resid norm 1.253194818485e+14 ||r(i)||/||b|| 1.148107452983e-03 21 KSP preconditioned resid norm 3.692994290799e+14 true resid norm 6.513442758709e+13 ||r(i)||/||b|| 5.967254305192e-04 22 KSP preconditioned resid norm 3.577186864519e+14 true resid norm 5.884014914924e+13 ||r(i)||/||b|| 5.390607491859e-04 23 KSP preconditioned resid norm 1.723423887037e+14 true resid norm 4.313917181929e+13 ||r(i)||/||b|| 3.952171198816e-04 24 KSP preconditioned resid norm 1.656260736170e+14 true resid norm 3.420915571267e+13 ||r(i)||/||b|| 3.134052746997e-04 25 KSP preconditioned resid norm 1.188400863617e+14 true resid norm 1.877160245085e+13 ||r(i)||/||b|| 1.719749903236e-04 26 KSP preconditioned resid norm 8.416275062676e+13 true resid norm 1.813244070153e+13 ||r(i)||/||b|| 1.661193455568e-04 27 KSP preconditioned resid norm 5.224466611453e+13 true resid norm 8.293327735079e+12 ||r(i)||/||b|| 7.597886012791e-05 28 KSP preconditioned resid norm 4.471999764526e+13 true resid norm 9.742703301619e+12 ||r(i)||/||b|| 8.925723365429e-05 29 KSP preconditioned resid norm 9.410327220658e+14 true resid norm 1.612776349391e+14 ||r(i)||/||b|| 1.477536069746e-03 30 KSP preconditioned resid norm 5.783713583723e+14 true resid norm 1.102180319983e+14 ||r(i)||/||b|| 1.009756361291e-03 31 KSP preconditioned resid norm 1.833775181615e+13 true resid norm 4.198845362007e+12 ||r(i)||/||b|| 3.846748791915e-05 32 KSP preconditioned resid norm 1.462922686654e+13 true resid norm 3.704838629537e+12 ||r(i)||/||b|| 3.394167275452e-05 33 KSP preconditioned resid norm 1.053043620710e+13 true resid norm 1.904310347574e+12 ||r(i)||/||b|| 1.744623318411e-05 34 KSP preconditioned resid norm 8.014167149326e+13 true resid norm 1.458222541169e+13 ||r(i)||/||b|| 1.335942459168e-04 35 KSP preconditioned resid norm 5.245679087504e+12 true resid norm 1.495201096266e+12 ||r(i)||/||b|| 1.369820156459e-05 0 KSP preconditioned resid norm 1.124208361458e+16 true resid norm 6.487794900041e+15 ||r(i)||/||b|| 1.232063762752e-01 1 KSP preconditioned resid norm 5.136632948471e+15 true resid norm 1.030125368299e+15 ||r(i)||/||b|| 1.956258107611e-02 2 KSP preconditioned resid norm 4.225773164973e+15 true resid norm 7.107211019673e+14 ||r(i)||/||b|| 1.349693892375e-02 3 KSP preconditioned resid norm 4.144329493920e+15 true resid norm 7.618007419548e+14 ||r(i)||/||b|| 1.446696609651e-02 4 KSP preconditioned resid norm 4.333716562490e+15 true resid norm 8.456727780045e+14 ||r(i)||/||b|| 1.605973679776e-02 5 KSP preconditioned resid norm 4.885745055980e+15 true resid norm 7.972847407816e+14 ||r(i)||/||b|| 1.514082446881e-02 6 KSP preconditioned resid norm 6.011621624545e+15 true resid norm 9.523510473075e+14 ||r(i)||/||b|| 1.808560894548e-02 7 KSP preconditioned resid norm 7.637951309150e+15 true resid norm 1.487731882292e+15 ||r(i)||/||b|| 2.825275103643e-02 8 KSP preconditioned resid norm 8.460249904060e+15 true resid norm 1.336944180115e+15 ||r(i)||/||b|| 2.538921933447e-02 9 KSP preconditioned resid norm 8.381050292195e+15 true resid norm 1.752911449852e+15 ||r(i)||/||b|| 3.328863982216e-02 10 KSP preconditioned resid norm 7.434169027728e+15 true resid norm 1.180498665014e+15 ||r(i)||/||b|| 2.241824301708e-02 11 KSP preconditioned resid norm 5.682629267055e+15 true resid norm 9.983854763086e+14 ||r(i)||/||b|| 1.895982511114e-02 12 KSP preconditioned resid norm 3.541898147790e+15 true resid norm 5.480576756264e+14 ||r(i)||/||b|| 1.040788145188e-02 13 KSP preconditioned resid norm 2.510644084392e+15 true resid norm 4.650369826671e+14 ||r(i)||/||b|| 8.831278169411e-03 14 KSP preconditioned resid norm 2.282038802861e+15 true resid norm 3.635360161467e+14 ||r(i)||/||b|| 6.903725516148e-03 15 KSP preconditioned resid norm 2.130130680848e+15 true resid norm 3.489368266499e+14 ||r(i)||/||b|| 6.626479816775e-03 16 KSP preconditioned resid norm 1.929984942209e+15 true resid norm 3.145683546766e+14 ||r(i)||/||b|| 5.973805841227e-03 17 KSP preconditioned resid norm 1.692158321133e+15 true resid norm 2.756020615583e+14 ||r(i)||/||b|| 5.233817008974e-03 18 KSP preconditioned resid norm 1.473781775606e+15 true resid norm 2.951884895649e+14 ||r(i)||/||b|| 5.605772789952e-03 19 KSP preconditioned resid norm 1.275239549902e+15 true resid norm 1.993708462138e+14 ||r(i)||/||b|| 3.786149204063e-03 20 KSP preconditioned resid norm 1.159341081209e+15 true resid norm 2.011812256624e+14 ||r(i)||/||b|| 3.820529189093e-03 21 KSP preconditioned resid norm 1.026465449352e+15 true resid norm 1.599378003014e+14 ||r(i)||/||b|| 3.037296509548e-03 22 KSP preconditioned resid norm 9.313333291328e+14 true resid norm 1.626450968383e+14 ||r(i)||/||b|| 3.088709385718e-03 23 KSP preconditioned resid norm 8.475207473455e+14 true resid norm 1.311587742521e+14 ||r(i)||/||b|| 2.490768826892e-03 24 KSP preconditioned resid norm 7.894756750239e+14 true resid norm 1.443612018407e+14 ||r(i)||/||b|| 2.741489339222e-03 25 KSP preconditioned resid norm 7.370222420808e+14 true resid norm 1.135397001576e+14 ||r(i)||/||b|| 2.156174052249e-03 26 KSP preconditioned resid norm 6.874978587995e+14 true resid norm 1.225152011526e+14 ||r(i)||/||b|| 2.326623175549e-03 27 KSP preconditioned resid norm 6.203646083349e+14 true resid norm 9.547951255722e+13 ||r(i)||/||b|| 1.813202317882e-03 28 KSP preconditioned resid norm 5.655995299941e+14 true resid norm 9.635247011837e+13 ||r(i)||/||b|| 1.829780205963e-03 29 KSP preconditioned resid norm 5.170668664072e+14 true resid norm 7.945307015395e+13 ||r(i)||/||b|| 1.508852392597e-03 30 KSP preconditioned resid norm 4.771681972381e+14 true resid norm 7.711681382438e+13 ||r(i)||/||b|| 1.464485750178e-03 31 KSP preconditioned resid norm 4.347723667483e+14 true resid norm 6.696963877348e+13 ||r(i)||/||b|| 1.271785967477e-03 32 KSP preconditioned resid norm 4.053435837983e+14 true resid norm 6.425971653178e+13 ||r(i)||/||b|| 1.220323227897e-03 33 KSP preconditioned resid norm 3.774514402447e+14 true resid norm 5.808746381526e+13 ||r(i)||/||b|| 1.103109150945e-03 34 KSP preconditioned resid norm 3.533225997981e+14 true resid norm 5.617172311710e+13 ||r(i)||/||b|| 1.066728304612e-03 0 KSP preconditioned resid norm 1.117659190837e+16 true resid norm 6.478857863821e+15 ||r(i)||/||b|| 6.224768870229e-01 1 KSP preconditioned resid norm 5.124485271748e+15 true resid norm 1.017923791746e+15 ||r(i)||/||b|| 9.780026764454e-02 2 KSP preconditioned resid norm 4.246328499709e+15 true resid norm 7.102918780385e+14 ||r(i)||/||b|| 6.824355255393e-02 3 KSP preconditioned resid norm 4.210000472667e+15 true resid norm 7.697163273299e+14 ||r(i)||/||b|| 7.395294562682e-02 4 KSP preconditioned resid norm 4.471596919519e+15 true resid norm 8.475637573508e+14 ||r(i)||/||b|| 8.143238520100e-02 5 KSP preconditioned resid norm 5.135403291038e+15 true resid norm 8.255731593574e+14 ||r(i)||/||b|| 7.931956851781e-02 6 KSP preconditioned resid norm 6.474268664840e+15 true resid norm 1.007942498042e+15 ||r(i)||/||b|| 9.684128308834e-02 7 KSP preconditioned resid norm 8.529281532903e+15 true resid norm 1.642696055823e+15 ||r(i)||/||b|| 1.578272511369e-01 8 KSP preconditioned resid norm 9.784577605090e+15 true resid norm 1.538078623395e+15 ||r(i)||/||b|| 1.477757983909e-01 9 KSP preconditioned resid norm 1.010006302597e+16 true resid norm 2.046930405602e+15 ||r(i)||/||b|| 1.966653526923e-01 10 KSP preconditioned resid norm 9.531884877717e+15 true resid norm 1.482620346274e+15 ||r(i)||/||b|| 1.424474679309e-01 11 KSP preconditioned resid norm 7.931210548314e+15 true resid norm 1.282982557872e+15 ||r(i)||/||b|| 1.232666321002e-01 12 KSP preconditioned resid norm 5.236526900965e+15 true resid norm 7.915538690853e+14 ||r(i)||/||b|| 7.605105694487e-02 13 KSP preconditioned resid norm 3.885416517845e+15 true resid norm 6.196979959619e+14 ||r(i)||/||b|| 5.953945703529e-02 14 KSP preconditioned resid norm 3.337626627490e+15 true resid norm 7.828597862542e+14 ||r(i)||/||b|| 7.521574526958e-02 15 KSP preconditioned resid norm 3.054301414200e+15 true resid norm 4.972852337202e+14 ||r(i)||/||b|| 4.777826134714e-02 16 KSP preconditioned resid norm 2.784835626497e+15 true resid norm 4.900149140724e+14 ||r(i)||/||b|| 4.707974225054e-02 17 KSP preconditioned resid norm 2.551999131175e+15 true resid norm 4.223064940199e+14 ||r(i)||/||b|| 4.057444032459e-02 18 KSP preconditioned resid norm 2.358735848485e+15 true resid norm 3.867160573888e+14 ||r(i)||/||b|| 3.715497586533e-02 19 KSP preconditioned resid norm 2.111367549832e+15 true resid norm 3.463614183500e+14 ||r(i)||/||b|| 3.327777549857e-02 20 KSP preconditioned resid norm 1.933287196882e+15 true resid norm 3.127201079650e+14 ||r(i)||/||b|| 3.004557954614e-02 21 KSP preconditioned resid norm 1.712007284199e+15 true resid norm 2.787684847304e+14 ||r(i)||/||b|| 2.678356929917e-02 22 KSP preconditioned resid norm 1.556420935486e+15 true resid norm 2.516757515483e+14 ||r(i)||/||b|| 2.418054874112e-02 23 KSP preconditioned resid norm 1.424455851932e+15 true resid norm 2.267048417314e+14 ||r(i)||/||b|| 2.178138911521e-02 24 KSP preconditioned resid norm 1.341093049266e+15 true resid norm 2.200423231461e+14 ||r(i)||/||b|| 2.114126643990e-02 25 KSP preconditioned resid norm 1.265242012282e+15 true resid norm 1.979405460182e+14 ||r(i)||/||b|| 1.901776786755e-02 26 KSP preconditioned resid norm 1.190343462827e+15 true resid norm 1.929502180424e+14 ||r(i)||/||b|| 1.853830622649e-02 27 KSP preconditioned resid norm 1.080874612432e+15 true resid norm 1.688981747210e+14 ||r(i)||/||b|| 1.622742962325e-02 28 KSP preconditioned resid norm 9.890419339482e+14 true resid norm 1.571112460256e+14 ||r(i)||/||b|| 1.509496293914e-02 29 KSP preconditioned resid norm 9.054720544549e+14 true resid norm 1.411421952126e+14 ||r(i)||/||b|| 1.356068556375e-02 30 KSP preconditioned resid norm 8.347895469778e+14 true resid norm 1.302020126144e+14 ||r(i)||/||b|| 1.250957270554e-02 31 KSP preconditioned resid norm 7.581633555103e+14 true resid norm 1.198729260222e+14 ||r(i)||/||b|| 1.151717284080e-02 32 KSP preconditioned resid norm 7.073149016328e+14 true resid norm 1.097657041094e+14 ||r(i)||/||b|| 1.054608933119e-02 33 KSP preconditioned resid norm 6.626892210136e+14 true resid norm 1.045304064903e+14 ||r(i)||/||b|| 1.004309145207e-02 34 KSP preconditioned resid norm 6.277118615935e+14 true resid norm 9.735143814138e+13 ||r(i)||/||b|| 9.353349222222e-03 35 KSP preconditioned resid norm 6.007031548284e+14 true resid norm 9.325545854852e+13 ||r(i)||/||b|| 8.959814948147e-03 36 KSP preconditioned resid norm 5.720600717487e+14 true resid norm 8.974565981239e+13 ||r(i)||/||b|| 8.622599865294e-03 37 KSP preconditioned resid norm 5.429834406497e+14 true resid norm 8.373407335142e+13 ||r(i)||/||b|| 8.045017565303e-03 38 KSP preconditioned resid norm 5.122271874419e+14 true resid norm 8.063599233502e+13 ||r(i)||/||b|| 7.747359572588e-03 39 KSP preconditioned resid norm 4.877390474820e+14 true resid norm 7.540978455988e+13 ||r(i)||/||b|| 7.245235029160e-03 40 KSP preconditioned resid norm 4.664511103016e+14 true resid norm 7.249582547116e+13 ||r(i)||/||b|| 6.965267136580e-03 41 KSP preconditioned resid norm 4.430412921029e+14 true resid norm 6.907345898754e+13 ||r(i)||/||b|| 6.636452385624e-03 42 KSP preconditioned resid norm 4.184290025581e+14 true resid norm 6.506826627971e+13 ||r(i)||/||b|| 6.251640750440e-03 43 KSP preconditioned resid norm 3.961557540211e+14 true resid norm 6.169515842630e+13 ||r(i)||/||b|| 5.927558679138e-03 44 KSP preconditioned resid norm 3.628856632080e+14 true resid norm 5.666822624927e+13 ||r(i)||/||b|| 5.444580172956e-03 45 KSP preconditioned resid norm 3.439726178254e+14 true resid norm 5.290740915351e+13 ||r(i)||/||b|| 5.083247702381e-03 46 KSP preconditioned resid norm 3.808130490503e+14 true resid norm 7.046303156753e+13 ||r(i)||/||b|| 6.769959993302e-03 47 KSP preconditioned resid norm 2.548925306353e+14 true resid norm 3.911087819219e+13 ||r(i)||/||b|| 3.757702085387e-03 48 KSP preconditioned resid norm 1.722230117103e+14 true resid norm 2.660198748286e+13 ||r(i)||/||b|| 2.555870603278e-03 49 KSP preconditioned resid norm 8.498499404656e+13 true resid norm 1.314954336818e+13 ||r(i)||/||b|| 1.263384225066e-03 50 KSP preconditioned resid norm 1.168755498979e+14 true resid norm 2.563042385986e+13 ||r(i)||/||b|| 2.462524536378e-03 51 KSP preconditioned resid norm 5.069372366763e+13 true resid norm 7.752336990767e+12 ||r(i)||/||b|| 7.448304467540e-04 0 KSP preconditioned resid norm 2.551555721110e+16 true resid norm 1.503348678660e+16 ||r(i)||/||b|| 1.139781099215e-01 1 KSP preconditioned resid norm 1.135435821514e+16 true resid norm 2.150528625386e+15 ||r(i)||/||b|| 1.630448022690e-02 2 KSP preconditioned resid norm 8.921910642422e+15 true resid norm 1.462019059252e+15 ||r(i)||/||b|| 1.108446572695e-02 3 KSP preconditioned resid norm 8.401630743802e+15 true resid norm 1.508644008325e+15 ||r(i)||/||b|| 1.143795814332e-02 4 KSP preconditioned resid norm 8.507938797573e+15 true resid norm 1.437822857483e+15 ||r(i)||/||b|| 1.090101943907e-02 5 KSP preconditioned resid norm 9.195685552195e+15 true resid norm 1.421948919345e+15 ||r(i)||/||b|| 1.078066935052e-02 6 KSP preconditioned resid norm 1.067998630800e+16 true resid norm 1.627316531548e+15 ||r(i)||/||b|| 1.233768753335e-02 7 KSP preconditioned resid norm 1.295276293823e+16 true resid norm 2.221334900964e+15 ||r(i)||/||b|| 1.684130615262e-02 8 KSP preconditioned resid norm 1.417997817727e+16 true resid norm 2.186215590047e+15 ||r(i)||/||b|| 1.657504505585e-02 9 KSP preconditioned resid norm 1.359372910192e+16 true resid norm 2.107067442870e+15 ||r(i)||/||b|| 1.597497427073e-02 10 KSP preconditioned resid norm 1.179811021994e+16 true resid norm 3.232410469947e+15 ||r(i)||/||b|| 2.450689191967e-02 11 KSP preconditioned resid norm 7.523875759744e+15 true resid norm 1.204652282825e+15 ||r(i)||/||b|| 9.133209897209e-03 12 KSP preconditioned resid norm 3.961884677960e+15 true resid norm 7.586571607759e+14 ||r(i)||/||b|| 5.751846560354e-03 13 KSP preconditioned resid norm 2.175818220311e+15 true resid norm 3.487257134817e+14 ||r(i)||/||b|| 2.643904123366e-03 14 KSP preconditioned resid norm 1.788448374434e+15 true resid norm 3.516091940967e+14 ||r(i)||/||b|| 2.665765563440e-03 15 KSP preconditioned resid norm 1.604495215225e+15 true resid norm 2.568394922627e+14 ||r(i)||/||b|| 1.947258164179e-03 16 KSP preconditioned resid norm 1.391089499920e+15 true resid norm 2.519771358465e+14 ||r(i)||/||b|| 1.910393649516e-03 17 KSP preconditioned resid norm 1.231055803595e+15 true resid norm 2.082047844556e+14 ||r(i)||/||b|| 1.578528530720e-03 18 KSP preconditioned resid norm 1.101068654831e+15 true resid norm 1.838589386072e+14 ||r(i)||/||b|| 1.393947699032e-03 19 KSP preconditioned resid norm 9.356667448723e+14 true resid norm 1.565965285410e+14 ||r(i)||/||b|| 1.187254600129e-03 20 KSP preconditioned resid norm 8.181513038456e+14 true resid norm 1.328185221533e+14 ||r(i)||/||b|| 1.006978908651e-03 0 KSP preconditioned resid norm 2.272810589988e+16 true resid norm 1.323636987884e+16 ||r(i)||/||b|| 5.438005326146e-01 1 KSP preconditioned resid norm 1.026969580669e+16 true resid norm 1.999385093393e+15 ||r(i)||/||b|| 8.214236143606e-02 2 KSP preconditioned resid norm 8.322628960001e+15 true resid norm 1.379596337850e+15 ||r(i)||/||b|| 5.667907667911e-02 3 KSP preconditioned resid norm 8.076035080201e+15 true resid norm 1.472961452514e+15 ||r(i)||/||b|| 6.051487150403e-02 4 KSP preconditioned resid norm 8.405382352119e+15 true resid norm 1.516410203314e+15 ||r(i)||/||b|| 6.229991181666e-02 5 KSP preconditioned resid norm 9.416373856303e+15 true resid norm 1.470293081560e+15 ||r(i)||/||b|| 6.040524465323e-02 6 KSP preconditioned resid norm 1.145894310132e+16 true resid norm 1.749278072065e+15 ||r(i)||/||b|| 7.186701157399e-02 7 KSP preconditioned resid norm 1.439168534642e+16 true resid norm 2.685145033101e+15 ||r(i)||/||b|| 1.103159939254e-01 8 KSP preconditioned resid norm 1.556059742979e+16 true resid norm 2.450465189043e+15 ||r(i)||/||b|| 1.006744513151e-01 9 KSP preconditioned resid norm 1.521075410830e+16 true resid norm 2.889962061237e+15 ||r(i)||/||b|| 1.187306582184e-01 10 KSP preconditioned resid norm 1.335253374250e+16 true resid norm 2.051263914753e+15 ||r(i)||/||b|| 8.427374118333e-02 11 KSP preconditioned resid norm 9.966633192923e+15 true resid norm 1.537352185287e+15 ||r(i)||/||b|| 6.316028826844e-02 12 KSP preconditioned resid norm 5.467457389741e+15 true resid norm 8.244964108686e+14 ||r(i)||/||b|| 3.387345559797e-02 13 KSP preconditioned resid norm 7.465607339270e+15 true resid norm 1.255988023601e+15 ||r(i)||/||b|| 5.160077592599e-02 14 KSP preconditioned resid norm 3.395725692367e+15 true resid norm 8.163584192182e+14 ||r(i)||/||b|| 3.353911587836e-02 15 KSP preconditioned resid norm 2.979451058224e+15 true resid norm 4.736799198293e+14 ||r(i)||/||b|| 1.946057680844e-02 16 KSP preconditioned resid norm 2.589762095203e+15 true resid norm 5.000061827618e+14 ||r(i)||/||b|| 2.054216004731e-02 17 KSP preconditioned resid norm 2.297387399767e+15 true resid norm 3.733072000155e+14 ||r(i)||/||b|| 1.533688285047e-02 18 KSP preconditioned resid norm 2.076822069911e+15 true resid norm 3.563941616114e+14 ||r(i)||/||b|| 1.464203075912e-02 19 KSP preconditioned resid norm 1.810897092306e+15 true resid norm 2.945299484347e+14 ||r(i)||/||b|| 1.210041305100e-02 20 KSP preconditioned resid norm 1.630399807803e+15 true resid norm 2.670493324597e+14 ||r(i)||/||b|| 1.097140458866e-02 21 KSP preconditioned resid norm 1.416015877821e+15 true resid norm 2.289562555093e+14 ||r(i)||/||b|| 9.406395773998e-03 22 KSP preconditioned resid norm 1.270953330458e+15 true resid norm 2.078621859272e+14 ||r(i)||/||b|| 8.539770983461e-03 23 KSP preconditioned resid norm 1.149572826465e+15 true resid norm 1.837793084226e+14 ||r(i)||/||b|| 7.550354569913e-03 24 KSP preconditioned resid norm 1.075403245776e+15 true resid norm 1.776412893147e+14 ||r(i)||/||b|| 7.298181346390e-03 25 KSP preconditioned resid norm 1.008390645992e+15 true resid norm 1.577927704554e+14 ||r(i)||/||b|| 6.482728527673e-03 26 KSP preconditioned resid norm 9.445856732256e+14 true resid norm 1.515432067895e+14 ||r(i)||/||b|| 6.225972628492e-03 27 KSP preconditioned resid norm 8.526344464253e+14 true resid norm 1.348203382746e+14 ||r(i)||/||b|| 5.538933441126e-03 28 KSP preconditioned resid norm 7.744588594030e+14 true resid norm 1.215509014949e+14 ||r(i)||/||b|| 4.993774394172e-03 29 KSP preconditioned resid norm 7.032530926217e+14 true resid norm 1.102818922582e+14 ||r(i)||/||b|| 4.530800536453e-03 30 KSP preconditioned resid norm 6.388922318141e+14 true resid norm 9.838037383987e+13 ||r(i)||/||b|| 4.041840790388e-03 31 KSP preconditioned resid norm 5.692116056101e+14 true resid norm 9.317411265002e+13 ||r(i)||/||b|| 3.827947734067e-03 32 KSP preconditioned resid norm 5.208032879647e+14 true resid norm 8.007380640801e+13 ||r(i)||/||b|| 3.289737214338e-03 33 KSP preconditioned resid norm 4.765676071483e+14 true resid norm 7.802986158610e+13 ||r(i)||/||b|| 3.205764169389e-03 34 KSP preconditioned resid norm 4.409793613234e+14 true resid norm 6.791885197478e+13 ||r(i)||/||b|| 2.790365350662e-03 35 KSP preconditioned resid norm 4.123618028942e+14 true resid norm 6.437750648670e+13 ||r(i)||/||b|| 2.644873378149e-03 36 KSP preconditioned resid norm 3.827034276493e+14 true resid norm 5.974308940262e+13 ||r(i)||/||b|| 2.454473857604e-03 37 KSP preconditioned resid norm 3.509107846575e+14 true resid norm 5.419166518671e+13 ||r(i)||/||b|| 2.226400188387e-03 38 KSP preconditioned resid norm 3.170601404687e+14 true resid norm 4.972345072887e+13 ||r(i)||/||b|| 2.042828905305e-03 39 KSP preconditioned resid norm 2.901553052614e+14 true resid norm 4.518683026559e+13 ||r(i)||/||b|| 1.856447242750e-03 40 KSP preconditioned resid norm 2.667015211684e+14 true resid norm 4.122270495295e+13 ||r(i)||/||b|| 1.693585863377e-03 41 KSP preconditioned resid norm 2.422095249996e+14 true resid norm 3.839959453856e+13 ||r(i)||/||b|| 1.577601725654e-03 42 KSP preconditioned resid norm 2.195894402502e+14 true resid norm 3.399029120498e+13 ||r(i)||/||b|| 1.396450736130e-03 43 KSP preconditioned resid norm 1.983452570770e+14 true resid norm 3.143816040870e+13 ||r(i)||/||b|| 1.291599474113e-03 44 KSP preconditioned resid norm 1.779873290535e+14 true resid norm 2.755766172822e+13 ||r(i)||/||b|| 1.132173795580e-03 45 KSP preconditioned resid norm 1.542400485092e+14 true resid norm 2.400274515345e+13 ||r(i)||/||b|| 9.861242710910e-04 46 KSP preconditioned resid norm 1.223281336233e+14 true resid norm 1.907355666953e+13 ||r(i)||/||b|| 7.836144177513e-04 0 KSP preconditioned resid norm 7.586062313136e+16 true resid norm 4.608637352265e+16 ||r(i)||/||b|| 6.590757061047e-02 1 KSP preconditioned resid norm 2.763422737801e+16 true resid norm 5.227955984017e+15 ||r(i)||/||b|| 7.476437216213e-03 2 KSP preconditioned resid norm 1.553456028419e+16 true resid norm 2.906104653716e+15 ||r(i)||/||b|| 4.155985447023e-03 3 KSP preconditioned resid norm 1.040767126705e+16 true resid norm 2.091654777018e+15 ||r(i)||/||b|| 2.991250436342e-03 4 KSP preconditioned resid norm 7.531366829970e+15 true resid norm 1.318758653617e+15 ||r(i)||/||b|| 1.885940950392e-03 5 KSP preconditioned resid norm 5.773017012836e+15 true resid norm 9.450385953724e+14 ||r(i)||/||b|| 1.351488372664e-03 6 KSP preconditioned resid norm 4.731507925306e+15 true resid norm 7.935497854840e+14 ||r(i)||/||b|| 1.134846040642e-03 7 KSP preconditioned resid norm 3.788639277852e+15 true resid norm 7.524409398539e+14 ||r(i)||/||b|| 1.076056773034e-03 8 KSP preconditioned resid norm 2.738698664678e+15 true resid norm 4.915430614067e+14 ||r(i)||/||b|| 7.029498429037e-04 9 KSP preconditioned resid norm 1.837368414949e+15 true resid norm 3.357340764188e+14 ||r(i)||/||b|| 4.801292802316e-04 10 KSP preconditioned resid norm 1.059331899595e+15 true resid norm 1.989071439556e+14 ||r(i)||/||b|| 2.844547234496e-04 11 KSP preconditioned resid norm 4.945892712172e+14 true resid norm 8.452757650210e+13 ||r(i)||/||b|| 1.208818744245e-04 12 KSP preconditioned resid norm 1.987888303244e+14 true resid norm 3.594134462521e+13 ||r(i)||/||b|| 5.139928633259e-05 13 KSP preconditioned resid norm 1.163784016975e+14 true resid norm 2.571324386412e+13 ||r(i)||/||b|| 3.677220197779e-05 14 KSP preconditioned resid norm 8.494315446492e+13 true resid norm 1.429434206797e+13 ||r(i)||/||b|| 2.044216732982e-05 15 KSP preconditioned resid norm 8.642186180523e+13 true resid norm 1.607306715480e+13 ||r(i)||/||b|| 2.298590076546e-05 16 KSP preconditioned resid norm 3.925619743667e+13 true resid norm 1.371348923807e+13 ||r(i)||/||b|| 1.961149665703e-05 0 KSP preconditioned resid norm 3.953744769406e+16 true resid norm 2.171194598493e+16 ||r(i)||/||b|| 1.950905346919e-01 1 KSP preconditioned resid norm 1.491502757726e+16 true resid norm 3.091776185716e+15 ||r(i)||/||b|| 2.778084790914e-02 2 KSP preconditioned resid norm 1.008617531922e+16 true resid norm 1.691017660922e+15 ||r(i)||/||b|| 1.519447127732e-02 3 KSP preconditioned resid norm 8.326895039450e+15 true resid norm 1.670298498749e+15 ||r(i)||/||b|| 1.500830130298e-02 4 KSP preconditioned resid norm 7.300566603499e+15 true resid norm 1.589496633697e+15 ||r(i)||/||b|| 1.428226416804e-02 5 KSP preconditioned resid norm 6.872352176548e+15 true resid norm 1.096739362492e+15 ||r(i)||/||b|| 9.854642637499e-03 6 KSP preconditioned resid norm 6.961475959227e+15 true resid norm 1.092150928708e+15 ||r(i)||/||b|| 9.813413721358e-03 7 KSP preconditioned resid norm 7.025042419735e+15 true resid norm 1.296288429690e+15 ||r(i)||/||b|| 1.164767096595e-02 8 KSP preconditioned resid norm 6.408702665765e+15 true resid norm 1.056644614043e+15 ||r(i)||/||b|| 9.494375256645e-03 9 KSP preconditioned resid norm 5.399106179995e+15 true resid norm 1.078818901202e+15 ||r(i)||/||b|| 9.693620112044e-03 10 KSP preconditioned resid norm 4.030707639439e+15 true resid norm 7.107969460630e+14 ||r(i)||/||b|| 6.386795378039e-03 11 KSP preconditioned resid norm 2.682586389004e+15 true resid norm 4.368532286577e+14 ||r(i)||/||b|| 3.925301307393e-03 12 KSP preconditioned resid norm 1.621400972048e+15 true resid norm 2.499302020485e+14 ||r(i)||/||b|| 2.245723012904e-03 13 KSP preconditioned resid norm 1.118646687359e+15 true resid norm 1.825993809337e+14 ||r(i)||/||b|| 1.640728605602e-03 14 KSP preconditioned resid norm 8.665732278929e+14 true resid norm 1.337846645968e+14 ||r(i)||/||b|| 1.202108819167e-03 15 KSP preconditioned resid norm 1.790412083411e+15 true resid norm 2.690613625509e+14 ||r(i)||/||b|| 2.417624155910e-03 16 KSP preconditioned resid norm 4.902192144952e+14 true resid norm 8.615757491331e+13 ||r(i)||/||b|| 7.741603340972e-04 17 KSP preconditioned resid norm 3.873085562421e+14 true resid norm 6.966874756362e+13 ||r(i)||/||b|| 6.260016132563e-04 18 KSP preconditioned resid norm 3.069894649217e+14 true resid norm 4.954148501038e+13 ||r(i)||/||b|| 4.451500941838e-04 19 KSP preconditioned resid norm 2.366855243734e+14 true resid norm 4.444373941350e+13 ||r(i)||/||b|| 3.993448073197e-04 20 KSP preconditioned resid norm 1.853280795186e+14 true resid norm 2.839442050460e+13 ||r(i)||/||b|| 2.551352459311e-04 21 KSP preconditioned resid norm 1.419103820891e+14 true resid norm 2.714177608773e+13 ||r(i)||/||b|| 2.438797339086e-04 22 KSP preconditioned resid norm 1.116662401807e+14 true resid norm 1.710034207306e+13 ||r(i)||/||b|| 1.536534256654e-04 23 KSP preconditioned resid norm 8.768706858701e+13 true resid norm 1.892684059360e+13 ||r(i)||/||b|| 1.700652467538e-04 24 KSP preconditioned resid norm 7.191985915577e+13 true resid norm 1.104970014738e+13 ||r(i)||/||b|| 9.928598345972e-05 25 KSP preconditioned resid norm 5.804472664630e+13 true resid norm 1.353159760302e+13 ||r(i)||/||b|| 1.215868265996e-04 26 KSP preconditioned resid norm 4.574262022619e+13 true resid norm 6.915191476157e+12 ||r(i)||/||b|| 6.213576634340e-05 27 KSP preconditioned resid norm 3.470073834736e+13 true resid norm 8.786634213858e+12 ||r(i)||/||b|| 7.895142923224e-05 28 KSP preconditioned resid norm 2.674822501624e+13 true resid norm 4.041336605553e+12 ||r(i)||/||b|| 3.631302877201e-05 29 KSP preconditioned resid norm 2.088626286399e+13 true resid norm 5.261273540428e+12 ||r(i)||/||b|| 4.727465096287e-05 30 KSP preconditioned resid norm 1.567496199145e+13 true resid norm 2.371012817670e+12 ||r(i)||/||b|| 2.130450023603e-05 31 KSP preconditioned resid norm 1.119841843422e+13 true resid norm 3.387071792996e+12 ||r(i)||/||b|| 3.043419726606e-05 32 KSP preconditioned resid norm 7.994377199528e+12 true resid norm 1.204577998308e+12 ||r(i)||/||b|| 1.082361599145e-05 33 KSP preconditioned resid norm 6.174157756589e+12 true resid norm 1.362745552188e+12 ||r(i)||/||b|| 1.224481484110e-05 0 SNES Function norm 0.000000000000e+00 1 TS dt 3e-12 time 2e-12 0 KSP preconditioned resid norm 2.220368634013e+19 true resid norm 1.204306153911e+19 ||r(i)||/||b|| 2.302397943981e+02 1 KSP preconditioned resid norm 5.390402695434e+18 true resid norm 1.775960718595e+18 ||r(i)||/||b|| 3.395289722473e+01 2 KSP preconditioned resid norm 2.218493119258e+18 true resid norm 5.320083037595e+17 ||r(i)||/||b|| 1.017095877804e+01 3 KSP preconditioned resid norm 1.401798813920e+18 true resid norm 2.963744523802e+17 ||r(i)||/||b|| 5.666100165582e+00 4 KSP preconditioned resid norm 1.004143331118e+18 true resid norm 1.656359344976e+17 ||r(i)||/||b|| 3.166635276239e+00 5 KSP preconditioned resid norm 8.673495646713e+17 true resid norm 1.510463721187e+17 ||r(i)||/||b|| 2.887711363779e+00 6 KSP preconditioned resid norm 7.389956702949e+17 true resid norm 1.526304376340e+17 ||r(i)||/||b|| 2.917995599842e+00 7 KSP preconditioned resid norm 5.566613599927e+17 true resid norm 1.017416269196e+17 ||r(i)||/||b|| 1.945101018344e+00 8 KSP preconditioned resid norm 3.566897773081e+17 true resid norm 6.307244640523e+16 ||r(i)||/||b|| 1.205821878877e+00 9 KSP preconditioned resid norm 1.893453815442e+17 true resid norm 3.268203516935e+16 ||r(i)||/||b|| 6.248166243663e-01 10 KSP preconditioned resid norm 6.894933140555e+16 true resid norm 1.152954614680e+16 ||r(i)||/||b|| 2.204223839364e-01 11 KSP preconditioned resid norm 5.103537427509e+16 true resid norm 9.272910080166e+15 ||r(i)||/||b|| 1.772799136993e-01 12 KSP preconditioned resid norm 4.991235858989e+16 true resid norm 8.396555957121e+15 ||r(i)||/||b|| 1.605257359967e-01 13 KSP preconditioned resid norm 4.918689912532e+16 true resid norm 8.423517310572e+15 ||r(i)||/||b|| 1.610411843696e-01 14 KSP preconditioned resid norm 5.686949456502e+16 true resid norm 9.761258523981e+15 ||r(i)||/||b|| 1.866161813032e-01 15 KSP preconditioned resid norm 1.616570395103e+17 true resid norm 4.351134639515e+16 ||r(i)||/||b|| 8.318518854587e-01 16 KSP preconditioned resid norm 4.128330778786e+16 true resid norm 7.104207079385e+15 ||r(i)||/||b|| 1.358185517866e-01 17 KSP preconditioned resid norm 2.161181135112e+16 true resid norm 5.859480309696e+15 ||r(i)||/||b|| 1.120218091889e-01 18 KSP preconditioned resid norm 1.410375919962e+16 true resid norm 2.425065017616e+15 ||r(i)||/||b|| 4.636250252850e-02 19 KSP preconditioned resid norm 1.082544604615e+16 true resid norm 2.485306543509e+15 ||r(i)||/||b|| 4.751420274116e-02 20 KSP preconditioned resid norm 8.996941383859e+15 true resid norm 1.577914541048e+15 ||r(i)||/||b|| 3.016664145813e-02 21 KSP preconditioned resid norm 7.748060569447e+15 true resid norm 1.336297549587e+15 ||r(i)||/||b|| 2.554739690340e-02 22 KSP preconditioned resid norm 6.873596381580e+15 true resid norm 1.173018262854e+15 ||r(i)||/||b|| 2.242581612555e-02 23 KSP preconditioned resid norm 6.175955090548e+15 true resid norm 1.012331331012e+15 ||r(i)||/||b|| 1.935379610558e-02 24 KSP preconditioned resid norm 5.614288705672e+15 true resid norm 9.369191861544e+14 ||r(i)||/||b|| 1.791206331440e-02 25 KSP preconditioned resid norm 5.220540757763e+15 true resid norm 8.444260437985e+14 ||r(i)||/||b|| 1.614377524163e-02 26 KSP preconditioned resid norm 5.059721722058e+15 true resid norm 8.452379209101e+14 ||r(i)||/||b|| 1.615929674492e-02 27 KSP preconditioned resid norm 4.893746226621e+15 true resid norm 9.389168158486e+14 ||r(i)||/||b|| 1.795025408912e-02 28 KSP preconditioned resid norm 4.794914716345e+15 true resid norm 7.493396637157e+14 ||r(i)||/||b|| 1.432590953288e-02 29 KSP preconditioned resid norm 4.860853428688e+15 true resid norm 7.693465766642e+14 ||r(i)||/||b|| 1.470840259819e-02 30 KSP preconditioned resid norm 5.112203028306e+15 true resid norm 8.019578406020e+14 ||r(i)||/||b|| 1.533186621496e-02 31 KSP preconditioned resid norm 5.649006086453e+15 true resid norm 8.938106702489e+14 ||r(i)||/||b|| 1.708791276044e-02 32 KSP preconditioned resid norm 8.383819984522e+15 true resid norm 1.347106201218e+15 ||r(i)||/||b|| 2.575403719341e-02 33 KSP preconditioned resid norm 1.617520068743e+16 true resid norm 2.603058585140e+15 ||r(i)||/||b|| 4.976539159104e-02 34 KSP preconditioned resid norm 3.383001122808e+16 true resid norm 5.541435636245e+15 ||r(i)||/||b|| 1.059414167582e-01 35 KSP preconditioned resid norm 6.256166371964e+15 true resid norm 1.029049029941e+15 ||r(i)||/||b|| 1.967340582872e-02 36 KSP preconditioned resid norm 3.290396673886e+15 true resid norm 5.409340387326e+14 ||r(i)||/||b|| 1.034160138236e-02 37 KSP preconditioned resid norm 2.075346751579e+15 true resid norm 3.809761571158e+14 ||r(i)||/||b|| 7.283519377531e-03 38 KSP preconditioned resid norm 1.750045982315e+15 true resid norm 2.837816436037e+14 ||r(i)||/||b|| 5.425350278670e-03 39 KSP preconditioned resid norm 1.596755323762e+15 true resid norm 2.766061504317e+14 ||r(i)||/||b|| 5.288168876146e-03 40 KSP preconditioned resid norm 1.469812007135e+15 true resid norm 2.359521384401e+14 ||r(i)||/||b|| 4.510943638858e-03 41 KSP preconditioned resid norm 1.314668041488e+15 true resid norm 2.158813968118e+14 ||r(i)||/||b|| 4.127230293966e-03 42 KSP preconditioned resid norm 1.190446489712e+15 true resid norm 1.941922420158e+14 ||r(i)||/||b|| 3.712576053042e-03 43 KSP preconditioned resid norm 1.100371263474e+15 true resid norm 1.837040226442e+14 ||r(i)||/||b|| 3.512061801423e-03 44 KSP preconditioned resid norm 1.032027622009e+15 true resid norm 1.624908335172e+14 ||r(i)||/||b|| 3.106507093655e-03 45 KSP preconditioned resid norm 8.066010465859e+14 true resid norm 2.499636130742e+14 ||r(i)||/||b|| 4.778815643705e-03 46 KSP preconditioned resid norm 6.546087590187e+14 true resid norm 1.014535455422e+14 ||r(i)||/||b|| 1.939593465560e-03 47 KSP preconditioned resid norm 6.561404017041e+14 true resid norm 1.083911025925e+14 ||r(i)||/||b|| 2.072225994564e-03 48 KSP preconditioned resid norm 6.834623592895e+14 true resid norm 1.073483632090e+14 ||r(i)||/||b|| 2.052290855937e-03 49 KSP preconditioned resid norm 8.808387787841e+14 true resid norm 1.369903526339e+14 ||r(i)||/||b|| 2.618987748466e-03 50 KSP preconditioned resid norm 3.995486126369e+14 true resid norm 1.237415046233e+14 ||r(i)||/||b|| 2.365695673849e-03 51 KSP preconditioned resid norm 3.217227010099e+14 true resid norm 4.935649374236e+13 ||r(i)||/||b|| 9.435996764234e-04 0 KSP preconditioned resid norm 6.543260687832e+18 true resid norm 3.546137973950e+18 ||r(i)||/||b|| 4.156987252753e+02 1 KSP preconditioned resid norm 1.591265056482e+18 true resid norm 5.223494588784e+17 ||r(i)||/||b|| 6.123281321796e+01 2 KSP preconditioned resid norm 6.570987793466e+17 true resid norm 1.552598280892e+17 ||r(i)||/||b|| 1.820045161730e+01 3 KSP preconditioned resid norm 4.166120135488e+17 true resid norm 8.739095834512e+16 ||r(i)||/||b|| 1.024447166228e+01 4 KSP preconditioned resid norm 2.995238174583e+17 true resid norm 4.935051588719e+16 ||r(i)||/||b|| 5.785151817748e+00 5 KSP preconditioned resid norm 2.594691834001e+17 true resid norm 4.470128066210e+16 ||r(i)||/||b|| 5.240141676921e+00 6 KSP preconditioned resid norm 2.219773011977e+17 true resid norm 4.652509796902e+16 ||r(i)||/||b|| 5.453940050023e+00 7 KSP preconditioned resid norm 1.675026903695e+17 true resid norm 2.988993956885e+16 ||r(i)||/||b|| 3.503870934691e+00 8 KSP preconditioned resid norm 1.081971008730e+17 true resid norm 1.908275119338e+16 ||r(i)||/||b|| 2.236990044975e+00 9 KSP preconditioned resid norm 5.710427760412e+16 true resid norm 9.635681698385e+15 ||r(i)||/||b|| 1.129550127097e+00 10 KSP preconditioned resid norm 1.779335357179e+16 true resid norm 2.952424081259e+15 ||r(i)||/||b|| 3.461001619416e-01 11 KSP preconditioned resid norm 1.125492898955e+16 true resid norm 2.030196257790e+15 ||r(i)||/||b|| 2.379913028263e-01 12 KSP preconditioned resid norm 1.086499382749e+16 true resid norm 1.877721745252e+15 ||r(i)||/||b|| 2.201173619461e-01 13 KSP preconditioned resid norm 1.075258231290e+16 true resid norm 1.822442508498e+15 ||r(i)||/||b|| 2.136372113085e-01 14 KSP preconditioned resid norm 1.173739259962e+16 true resid norm 2.130298543993e+15 ||r(i)||/||b|| 2.497258695795e-01 15 KSP preconditioned resid norm 4.943363278431e+16 true resid norm 1.361259414632e+16 ||r(i)||/||b|| 1.595746718229e+00 16 KSP preconditioned resid norm 1.782643390767e+17 true resid norm 3.375538781586e+16 ||r(i)||/||b|| 3.957001049961e+00 17 KSP preconditioned resid norm 3.253381322946e+16 true resid norm 8.720742251900e+15 ||r(i)||/||b|| 1.022295653525e+00 18 KSP preconditioned resid norm 2.077141437527e+16 true resid norm 3.948414579285e+15 ||r(i)||/||b|| 4.628559067708e-01 19 KSP preconditioned resid norm 1.484650052668e+16 true resid norm 3.391990202104e+15 ||r(i)||/||b|| 3.976286352982e-01 20 KSP preconditioned resid norm 1.135224344133e+16 true resid norm 2.492450462789e+15 ||r(i)||/||b|| 2.921794041305e-01 21 KSP preconditioned resid norm 8.560056764067e+15 true resid norm 1.658566536392e+15 ||r(i)||/||b|| 1.944267256456e-01 22 KSP preconditioned resid norm 6.591497757947e+15 true resid norm 1.287448297256e+15 ||r(i)||/||b|| 1.509221073626e-01 23 KSP preconditioned resid norm 4.783676751639e+15 true resid norm 9.069671731055e+14 ||r(i)||/||b|| 1.063199177517e-01 24 KSP preconditioned resid norm 3.029245298062e+15 true resid norm 6.184462851213e+14 ||r(i)||/||b|| 7.249783687627e-02 25 KSP preconditioned resid norm 1.567581368038e+15 true resid norm 2.751547089998e+14 ||r(i)||/||b|| 3.225522036225e-02 26 KSP preconditioned resid norm 1.174427256669e+15 true resid norm 2.270458347820e+14 ||r(i)||/||b|| 2.661562093503e-02 27 KSP preconditioned resid norm 1.068689317098e+15 true resid norm 2.026010487900e+14 ||r(i)||/||b|| 2.375006227624e-02 28 KSP preconditioned resid norm 1.006302654001e+15 true resid norm 1.659657859887e+14 ||r(i)||/||b|| 1.945546568735e-02 29 KSP preconditioned resid norm 9.398219810967e+14 true resid norm 1.788496139471e+14 ||r(i)||/||b|| 2.096578223407e-02 30 KSP preconditioned resid norm 8.922669862958e+14 true resid norm 1.448103190511e+14 ||r(i)||/||b|| 1.697549996037e-02 31 KSP preconditioned resid norm 8.611553511040e+14 true resid norm 1.569494235094e+14 ||r(i)||/||b|| 1.839851572749e-02 32 KSP preconditioned resid norm 8.275858647114e+14 true resid norm 1.327021986732e+14 ||r(i)||/||b|| 1.555611632568e-02 33 KSP preconditioned resid norm 7.965262562283e+14 true resid norm 1.578752878362e+14 ||r(i)||/||b|| 1.850705087847e-02 34 KSP preconditioned resid norm 7.584298711071e+14 true resid norm 1.400338111929e+14 ||r(i)||/||b|| 1.641557018817e-02 35 KSP preconditioned resid norm 7.453216416858e+14 true resid norm 1.168697043482e+14 ||r(i)||/||b|| 1.370014011799e-02 36 KSP preconditioned resid norm 7.552391370873e+14 true resid norm 1.200616262811e+14 ||r(i)||/||b|| 1.407431559803e-02 37 KSP preconditioned resid norm 7.900785942536e+14 true resid norm 1.248526226554e+14 ||r(i)||/||b|| 1.463594379756e-02 38 KSP preconditioned resid norm 8.292853068973e+14 true resid norm 1.319608976445e+14 ||r(i)||/||b|| 1.546921674790e-02 39 KSP preconditioned resid norm 8.694401463969e+14 true resid norm 1.375708051207e+14 ||r(i)||/||b|| 1.612684242516e-02 40 KSP preconditioned resid norm 9.345453545993e+14 true resid norm 1.486706547594e+14 ||r(i)||/||b|| 1.742803075440e-02 41 KSP preconditioned resid norm 1.186246552213e+15 true resid norm 1.878863391090e+14 ||r(i)||/||b|| 2.202511922492e-02 42 KSP preconditioned resid norm 2.140906233315e+15 true resid norm 3.526342198473e+14 ||r(i)||/||b|| 4.133781504155e-02 43 KSP preconditioned resid norm 1.599648547908e+16 true resid norm 2.542842342413e+15 ||r(i)||/||b|| 2.980866306056e-01 44 KSP preconditioned resid norm 1.201331553169e+15 true resid norm 1.958292468332e+14 ||r(i)||/||b|| 2.295623263342e-02 45 KSP preconditioned resid norm 6.227596295745e+14 true resid norm 9.990152835185e+13 ||r(i)||/||b|| 1.171103276127e-02 46 KSP preconditioned resid norm 4.228157632423e+14 true resid norm 6.914757907633e+13 ||r(i)||/||b|| 8.105877630555e-03 47 KSP preconditioned resid norm 2.986102023248e+14 true resid norm 4.782856914894e+13 ||r(i)||/||b|| 5.606740452011e-03 48 KSP preconditioned resid norm 1.831310965020e+14 true resid norm 3.038059690813e+13 ||r(i)||/||b|| 3.561388615047e-03 49 KSP preconditioned resid norm 1.519167951348e+14 true resid norm 2.657532215634e+13 ||r(i)||/||b|| 3.115312383591e-03 50 KSP preconditioned resid norm 1.784468495322e+14 true resid norm 2.872094801120e+13 ||r(i)||/||b|| 3.366835008863e-03 51 KSP preconditioned resid norm 1.082269784640e+14 true resid norm 1.734196241756e+13 ||r(i)||/||b|| 2.032924754679e-03 52 KSP preconditioned resid norm 7.941561433572e+13 true resid norm 2.262252591710e+13 ||r(i)||/||b|| 2.651942833395e-03 53 KSP preconditioned resid norm 1.279466501710e+14 true resid norm 2.100188573520e+13 ||r(i)||/||b|| 2.461962053545e-03 54 KSP preconditioned resid norm 1.148448766150e+14 true resid norm 1.892568589516e+13 ||r(i)||/||b|| 2.218577945746e-03 55 KSP preconditioned resid norm 4.111681106774e+13 true resid norm 1.052257112253e+13 ||r(i)||/||b|| 1.233516415432e-03 0 KSP preconditioned resid norm 1.892564880723e+19 true resid norm 1.028289393047e+19 ||r(i)||/||b|| 7.840877673877e+01 1 KSP preconditioned resid norm 4.581010831586e+18 true resid norm 1.519789346059e+18 ||r(i)||/||b|| 1.158864657468e+01 2 KSP preconditioned resid norm 1.874273717290e+18 true resid norm 4.647134076709e+17 ||r(i)||/||b|| 3.543517036739e+00 3 KSP preconditioned resid norm 1.173044525535e+18 true resid norm 2.550993039028e+17 ||r(i)||/||b|| 1.945174627025e+00 4 KSP preconditioned resid norm 8.299292116891e+17 true resid norm 1.381054316961e+17 ||r(i)||/||b|| 1.053076889978e+00 5 KSP preconditioned resid norm 7.051404070011e+17 true resid norm 1.287518923939e+17 ||r(i)||/||b|| 9.817545968746e-01 6 KSP preconditioned resid norm 5.863125538322e+17 true resid norm 1.166548434193e+17 ||r(i)||/||b|| 8.895125861467e-01 7 KSP preconditioned resid norm 4.330808264262e+17 true resid norm 8.612437830885e+16 ||r(i)||/||b|| 6.567127110566e-01 8 KSP preconditioned resid norm 2.702964509584e+17 true resid norm 4.804107972053e+16 ||r(i)||/||b|| 3.663212243137e-01 9 KSP preconditioned resid norm 1.547354820777e+17 true resid norm 2.879372380790e+16 ||r(i)||/||b|| 2.195569337580e-01 10 KSP preconditioned resid norm 9.913511217220e+16 true resid norm 1.649311378149e+16 ||r(i)||/||b|| 1.257627361485e-01 11 KSP preconditioned resid norm 9.246165206306e+16 true resid norm 1.559859930326e+16 ||r(i)||/||b|| 1.189419144530e-01 12 KSP preconditioned resid norm 9.248856361042e+16 true resid norm 1.507587985844e+16 ||r(i)||/||b|| 1.149560917339e-01 13 KSP preconditioned resid norm 7.088634726331e+16 true resid norm 1.150049603396e+16 ||r(i)||/||b|| 8.769319532120e-02 14 KSP preconditioned resid norm 2.014973749241e+17 true resid norm 3.263663179758e+16 ||r(i)||/||b|| 2.488597464319e-01 15 KSP preconditioned resid norm 1.687600147354e+16 true resid norm 4.438944439784e+15 ||r(i)||/||b|| 3.384768975431e-02 16 KSP preconditioned resid norm 1.189486995696e+16 true resid norm 1.895011071195e+15 ||r(i)||/||b|| 1.444977464550e-02 17 KSP preconditioned resid norm 1.038277428537e+16 true resid norm 2.030560404119e+15 ||r(i)||/||b|| 1.548336085714e-02 18 KSP preconditioned resid norm 9.585935517274e+15 true resid norm 1.570427582757e+15 ||r(i)||/||b|| 1.197477155297e-02 19 KSP preconditioned resid norm 8.986106113745e+15 true resid norm 1.453955802798e+15 ||r(i)||/||b|| 1.108665485617e-02 20 KSP preconditioned resid norm 8.480772996688e+15 true resid norm 1.519414795393e+15 ||r(i)||/||b|| 1.158579056355e-02 21 KSP preconditioned resid norm 8.009707085363e+15 true resid norm 1.276353172356e+15 ||r(i)||/||b|| 9.732405255546e-03 22 KSP preconditioned resid norm 7.603487959173e+15 true resid norm 1.348218761629e+15 ||r(i)||/||b|| 1.028039232831e-02 23 KSP preconditioned resid norm 7.264676993739e+15 true resid norm 1.145141030012e+15 ||r(i)||/||b|| 8.731890843545e-03 24 KSP preconditioned resid norm 6.442617339194e+15 true resid norm 1.846348609557e+15 ||r(i)||/||b|| 1.407871528071e-02 25 KSP preconditioned resid norm 5.874716908283e+15 true resid norm 9.094105988409e+14 ||r(i)||/||b|| 6.934407093038e-03 26 KSP preconditioned resid norm 5.974445043424e+15 true resid norm 9.497988242860e+14 ||r(i)||/||b|| 7.242374030480e-03 27 KSP preconditioned resid norm 6.328543343408e+15 true resid norm 9.841709349231e+14 ||r(i)||/||b|| 7.504467091753e-03 28 KSP preconditioned resid norm 1.045930080864e+16 true resid norm 1.637386301334e+15 ||r(i)||/||b|| 1.248534292045e-02 29 KSP preconditioned resid norm 1.763968686067e+16 true resid norm 3.035143897342e+15 ||r(i)||/||b|| 2.314347710148e-02 30 KSP preconditioned resid norm 3.994992144337e+15 true resid norm 6.366779249631e+14 ||r(i)||/||b|| 4.854775086713e-03 31 KSP preconditioned resid norm 2.497471912676e+15 true resid norm 4.171284268487e+14 ||r(i)||/||b|| 3.180673642395e-03 32 KSP preconditioned resid norm 1.774822233295e+15 true resid norm 3.030553568855e+14 ||r(i)||/||b|| 2.310847508319e-03 33 KSP preconditioned resid norm 1.554413264158e+15 true resid norm 2.490797343960e+14 ||r(i)||/||b|| 1.899274408204e-03 34 KSP preconditioned resid norm 1.363683273075e+15 true resid norm 2.337415683510e+14 ||r(i)||/||b|| 1.782318340667e-03 35 KSP preconditioned resid norm 1.216181955878e+15 true resid norm 1.941708948493e+14 ||r(i)||/||b|| 1.480585372791e-03 36 KSP preconditioned resid norm 1.106171392348e+15 true resid norm 1.781889599099e+14 ||r(i)||/||b|| 1.358720460346e-03 37 KSP preconditioned resid norm 9.985678909493e+14 true resid norm 1.843602345870e+14 ||r(i)||/||b|| 1.405777456326e-03 38 KSP preconditioned resid norm 9.306373857300e+14 true resid norm 1.483324768155e+14 ||r(i)||/||b|| 1.131059810243e-03 39 KSP preconditioned resid norm 8.621292024153e+14 true resid norm 1.776747195898e+14 ||r(i)||/||b|| 1.354799292363e-03 0 KSP preconditioned resid norm 6.470804796325e+18 true resid norm 3.511545401133e+18 ||r(i)||/||b|| 1.662319590124e+02 1 KSP preconditioned resid norm 1.571603694611e+18 true resid norm 5.187449890798e+17 ||r(i)||/||b|| 2.455670820454e+01 2 KSP preconditioned resid norm 6.468415902839e+17 true resid norm 1.566954416728e+17 ||r(i)||/||b|| 7.417756930949e+00 3 KSP preconditioned resid norm 4.084120869852e+17 true resid norm 8.688409741290e+16 ||r(i)||/||b|| 4.112979349580e+00 4 KSP preconditioned resid norm 2.920961241699e+17 true resid norm 4.832079671037e+16 ||r(i)||/||b|| 2.287443213923e+00 5 KSP preconditioned resid norm 2.515909976499e+17 true resid norm 4.435653262803e+16 ||r(i)||/||b|| 2.099780145624e+00 6 KSP preconditioned resid norm 2.133092401708e+17 true resid norm 4.355815215083e+16 ||r(i)||/||b|| 2.061985859746e+00 7 KSP preconditioned resid norm 1.601275153782e+17 true resid norm 2.999237788966e+16 ||r(i)||/||b|| 1.419799877977e+00 8 KSP preconditioned resid norm 1.019875018318e+17 true resid norm 1.811096161004e+16 ||r(i)||/||b|| 8.573491964719e-01 9 KSP preconditioned resid norm 5.552203589730e+16 true resid norm 9.849208071572e+15 ||r(i)||/||b|| 4.662486072173e-01 10 KSP preconditioned resid norm 2.614195079413e+16 true resid norm 4.400774101946e+15 ||r(i)||/||b|| 2.083268807806e-01 11 KSP preconditioned resid norm 2.230316715231e+16 true resid norm 4.001821966241e+15 ||r(i)||/||b|| 1.894410093210e-01 12 KSP preconditioned resid norm 2.202763899809e+16 true resid norm 3.668851069689e+15 ||r(i)||/||b|| 1.736786032846e-01 13 KSP preconditioned resid norm 2.171352861704e+16 true resid norm 3.746836567903e+15 ||r(i)||/||b|| 1.773703346056e-01 14 KSP preconditioned resid norm 2.756407833586e+16 true resid norm 4.737724354756e+15 ||r(i)||/||b|| 2.242776643292e-01 15 KSP preconditioned resid norm 1.909305469788e+16 true resid norm 7.268311827741e+15 ||r(i)||/||b|| 3.440723601207e-01 16 KSP preconditioned resid norm 8.249997128004e+15 true resid norm 1.447767843213e+15 ||r(i)||/||b|| 6.853543305884e-02 17 KSP preconditioned resid norm 5.434797146758e+15 true resid norm 1.776552830818e+15 ||r(i)||/||b|| 8.409968364943e-02 18 KSP preconditioned resid norm 4.166874856572e+15 true resid norm 7.323760066865e+14 ||r(i)||/||b|| 3.466972071212e-02 19 KSP preconditioned resid norm 3.463504540881e+15 true resid norm 7.653814460137e+14 ||r(i)||/||b|| 3.623215497131e-02 20 KSP preconditioned resid norm 3.008102754263e+15 true resid norm 5.633011056242e+14 ||r(i)||/||b|| 2.666593639130e-02 21 KSP preconditioned resid norm 2.629337883031e+15 true resid norm 5.301916607139e+14 ||r(i)||/||b|| 2.509857864406e-02 22 KSP preconditioned resid norm 2.387457188115e+15 true resid norm 3.879435274146e+14 ||r(i)||/||b|| 1.836473836491e-02 23 KSP preconditioned resid norm 2.223797660329e+15 true resid norm 3.481194862204e+14 ||r(i)||/||b|| 1.647952042601e-02 24 KSP preconditioned resid norm 2.082583007435e+15 true resid norm 3.393016280495e+14 ||r(i)||/||b|| 1.606209457197e-02 25 KSP preconditioned resid norm 1.971105202361e+15 true resid norm 3.085385809365e+14 ||r(i)||/||b|| 1.460581222257e-02 26 KSP preconditioned resid norm 1.916167824316e+15 true resid norm 3.157123008884e+14 ||r(i)||/||b|| 1.494540672721e-02 27 KSP preconditioned resid norm 1.825444848282e+15 true resid norm 4.173235963436e+14 ||r(i)||/||b|| 1.975555233884e-02 28 KSP preconditioned resid norm 1.787001647353e+15 true resid norm 2.787303202952e+14 ||r(i)||/||b|| 1.319472821393e-02 29 KSP preconditioned resid norm 1.896717764390e+15 true resid norm 3.068021792990e+14 ||r(i)||/||b|| 1.452361324382e-02 30 KSP preconditioned resid norm 2.229137880818e+15 true resid norm 3.494609542895e+14 ||r(i)||/||b|| 1.654302376702e-02 31 KSP preconditioned resid norm 3.112887805330e+15 true resid norm 5.056849908847e+14 ||r(i)||/||b|| 2.393846499915e-02 32 KSP preconditioned resid norm 2.024054955579e+16 true resid norm 3.265471021005e+15 ||r(i)||/||b|| 1.545831202253e-01 33 KSP preconditioned resid norm 2.565517746075e+15 true resid norm 4.297071200158e+14 ||r(i)||/||b|| 2.034177212653e-02 34 KSP preconditioned resid norm 1.151189963639e+15 true resid norm 1.893961732619e+14 ||r(i)||/||b|| 8.965766724994e-03 35 KSP preconditioned resid norm 7.095661584181e+14 true resid norm 1.215463627012e+14 ||r(i)||/||b|| 5.753845579254e-03 36 KSP preconditioned resid norm 5.356379112526e+14 true resid norm 8.697359204859e+13 ||r(i)||/||b|| 4.117215908392e-03 37 KSP preconditioned resid norm 4.260660574229e+14 true resid norm 9.044225863386e+13 ||r(i)||/||b|| 4.281418040435e-03 38 KSP preconditioned resid norm 3.822193347318e+14 true resid norm 6.067358042990e+13 ||r(i)||/||b|| 2.872207812522e-03 39 KSP preconditioned resid norm 3.556139762092e+14 true resid norm 7.037316076443e+13 ||r(i)||/||b|| 3.331373238687e-03 40 KSP preconditioned resid norm 3.319733183305e+14 true resid norm 5.228450610578e+13 ||r(i)||/||b|| 2.475080024071e-03 41 KSP preconditioned resid norm 3.093370284781e+14 true resid norm 5.468964654818e+13 ||r(i)||/||b|| 2.588936221775e-03 42 KSP preconditioned resid norm 2.908716641400e+14 true resid norm 4.719126609844e+13 ||r(i)||/||b|| 2.233972714489e-03 43 KSP preconditioned resid norm 2.664499581099e+14 true resid norm 6.079882213095e+13 ||r(i)||/||b|| 2.878136590578e-03 44 KSP preconditioned resid norm 2.513459163383e+14 true resid norm 3.896503433878e+13 ||r(i)||/||b|| 1.844553679708e-03 45 KSP preconditioned resid norm 2.506735910619e+14 true resid norm 4.872960867423e+13 ||r(i)||/||b|| 2.306795836731e-03 46 KSP preconditioned resid norm 2.610309833791e+14 true resid norm 4.046617714344e+13 ||r(i)||/||b|| 1.915615813519e-03 47 KSP preconditioned resid norm 3.265892246735e+14 true resid norm 6.092635527969e+13 ||r(i)||/||b|| 2.884173842765e-03 48 KSP preconditioned resid norm 6.619675585826e+14 true resid norm 1.029872445074e+14 ||r(i)||/||b|| 4.875281237212e-03 49 KSP preconditioned resid norm 6.614927581836e+14 true resid norm 1.173864813817e+14 ||r(i)||/||b|| 5.556922247214e-03 50 KSP preconditioned resid norm 2.572515469657e+14 true resid norm 4.031945474821e+13 ||r(i)||/||b|| 1.908670167541e-03 51 KSP preconditioned resid norm 1.432897312628e+14 true resid norm 2.304000171834e+13 ||r(i)||/||b|| 1.090683497942e-03 52 KSP preconditioned resid norm 8.898633954299e+13 true resid norm 1.420071536381e+13 ||r(i)||/||b|| 6.722432617682e-04 0 KSP preconditioned resid norm 6.694445614266e+18 true resid norm 3.650633549648e+18 ||r(i)||/||b|| 5.229628447406e+00 1 KSP preconditioned resid norm 1.597624793291e+18 true resid norm 5.366629107368e+17 ||r(i)||/||b|| 7.687837156178e-01 2 KSP preconditioned resid norm 6.300333922334e+17 true resid norm 1.705382687722e+17 ||r(i)||/||b|| 2.443005493741e-01 3 KSP preconditioned resid norm 3.642099405484e+17 true resid norm 9.482515314454e+16 ||r(i)||/||b|| 1.358395225569e-01 4 KSP preconditioned resid norm 2.247296120507e+17 true resid norm 3.802694326323e+16 ||r(i)||/||b|| 5.447459504021e-02 5 KSP preconditioned resid norm 1.557575416788e+17 true resid norm 3.828791753195e+16 ||r(i)||/||b|| 5.484844753491e-02 6 KSP preconditioned resid norm 1.004780833173e+17 true resid norm 1.699813376802e+16 ||r(i)||/||b|| 2.435027309565e-02 7 KSP preconditioned resid norm 7.104370054381e+16 true resid norm 1.472894912543e+16 ||r(i)||/||b|| 2.109960649273e-02 8 KSP preconditioned resid norm 5.767747590200e+16 true resid norm 9.399551072817e+15 ||r(i)||/||b|| 1.346510380041e-02 9 KSP preconditioned resid norm 4.524165642090e+16 true resid norm 1.246569325329e+16 ||r(i)||/||b|| 1.785743300921e-02 10 KSP preconditioned resid norm 3.195588487377e+16 true resid norm 5.031148957130e+15 ||r(i)||/||b|| 7.207253029241e-03 11 KSP preconditioned resid norm 2.437445149875e+16 true resid norm 4.973530315557e+15 ||r(i)||/||b|| 7.124712811776e-03 12 KSP preconditioned resid norm 1.257635618330e+16 true resid norm 1.885290715052e+15 ||r(i)||/||b|| 2.700728468356e-03 13 KSP preconditioned resid norm 2.495249888250e+15 true resid norm 4.460762730080e+14 ||r(i)||/||b|| 6.390159777228e-04 14 KSP preconditioned resid norm 7.692041830184e+14 true resid norm 1.319158639688e+14 ||r(i)||/||b|| 1.889729400372e-04 15 KSP preconditioned resid norm 5.963386353631e+14 true resid norm 1.441107752347e+14 ||r(i)||/||b|| 2.064424707371e-04 16 KSP preconditioned resid norm 3.269005832710e+14 true resid norm 5.055770257989e+13 ||r(i)||/||b|| 7.242523689424e-05 17 KSP preconditioned resid norm 1.442136448583e+14 true resid norm 7.615238793008e+13 ||r(i)||/||b|| 1.090902959284e-04 18 KSP preconditioned resid norm 5.418830608399e+13 true resid norm 8.650648142713e+12 ||r(i)||/||b|| 1.239228068235e-05 19 KSP preconditioned resid norm 3.534476706711e+13 true resid norm 1.235947718863e+13 ||r(i)||/||b|| 1.770527570673e-05 0 KSP preconditioned resid norm 4.827689913975e+18 true resid norm 2.628955418091e+18 ||r(i)||/||b|| 2.387179080674e+01 1 KSP preconditioned resid norm 1.161014015330e+18 true resid norm 3.885001078629e+17 ||r(i)||/||b|| 3.527710374804e+00 2 KSP preconditioned resid norm 4.696713579932e+17 true resid norm 1.228966476828e+17 ||r(i)||/||b|| 1.115942493412e+00 3 KSP preconditioned resid norm 2.869928353163e+17 true resid norm 6.759586363313e+16 ||r(i)||/||b|| 6.137929555390e-01 4 KSP preconditioned resid norm 1.966510821692e+17 true resid norm 3.310379360335e+16 ||r(i)||/||b|| 3.005934716010e-01 5 KSP preconditioned resid norm 1.592497528073e+17 true resid norm 3.239157026660e+16 ||r(i)||/||b|| 2.941262464874e-01 6 KSP preconditioned resid norm 1.230094082012e+17 true resid norm 2.280559390645e+16 ||r(i)||/||b|| 2.070823883934e-01 7 KSP preconditioned resid norm 8.598887604154e+16 true resid norm 1.938013722659e+16 ||r(i)||/||b|| 1.759781008439e-01 8 KSP preconditioned resid norm 5.549487092651e+16 true resid norm 9.386421691835e+15 ||r(i)||/||b|| 8.523183524123e-02 9 KSP preconditioned resid norm 4.475548308156e+16 true resid norm 7.579602548804e+15 ||r(i)||/||b|| 6.882531563606e-02 10 KSP preconditioned resid norm 4.379582896462e+16 true resid norm 6.844431748768e+15 ||r(i)||/||b|| 6.214971991278e-02 11 KSP preconditioned resid norm 4.081432560058e+16 true resid norm 7.719907534397e+15 ||r(i)||/||b|| 7.009933163578e-02 12 KSP preconditioned resid norm 4.084955665921e+16 true resid norm 6.376903215279e+15 ||r(i)||/||b|| 5.790440511177e-02 13 KSP preconditioned resid norm 7.513638228384e+16 true resid norm 1.130789675242e+16 ||r(i)||/||b|| 1.026794687656e-01 14 KSP preconditioned resid norm 2.195838265499e+15 true resid norm 3.518796225338e+14 ||r(i)||/||b|| 3.195184171050e-03 15 KSP preconditioned resid norm 2.237408970122e+15 true resid norm 3.676139977919e+14 ||r(i)||/||b|| 3.338057538948e-03 16 KSP preconditioned resid norm 1.675719433917e+15 true resid norm 3.658174858611e+14 ||r(i)||/||b|| 3.321744612262e-03 17 KSP preconditioned resid norm 1.403876583239e+15 true resid norm 2.164126267214e+14 ||r(i)||/||b|| 1.965098729891e-03 18 KSP preconditioned resid norm 1.133854982860e+15 true resid norm 2.692061619571e+14 ||r(i)||/||b|| 2.444481613457e-03 19 KSP preconditioned resid norm 9.034784290513e+14 true resid norm 1.408322491509e+14 ||r(i)||/||b|| 1.278803728446e-03 20 KSP preconditioned resid norm 7.964711449547e+14 true resid norm 1.641000281547e+14 ||r(i)||/||b|| 1.490082911460e-03 21 KSP preconditioned resid norm 7.094597664828e+14 true resid norm 1.088610845746e+14 ||r(i)||/||b|| 9.884949056480e-04 22 KSP preconditioned resid norm 6.287860765855e+14 true resid norm 1.439087926971e+14 ||r(i)||/||b|| 1.306739768531e-03 23 KSP preconditioned resid norm 5.617871594119e+14 true resid norm 8.640965180691e+13 ||r(i)||/||b|| 7.846284183530e-04 24 KSP preconditioned resid norm 4.992037484390e+14 true resid norm 1.232499062518e+14 ||r(i)||/||b|| 1.119150198876e-03 25 KSP preconditioned resid norm 3.174300847299e+14 true resid norm 1.199443314096e+14 ||r(i)||/||b|| 1.089134478340e-03 26 KSP preconditioned resid norm 2.311461716948e+14 true resid norm 3.806900580053e+13 ||r(i)||/||b|| 3.456792520850e-04 27 KSP preconditioned resid norm 2.486080516907e+14 true resid norm 3.795889860412e+13 ||r(i)||/||b|| 3.446794420688e-04 28 KSP preconditioned resid norm 3.709940932117e+14 true resid norm 6.158317563060e+13 ||r(i)||/||b|| 5.591957458661e-04 29 KSP preconditioned resid norm 5.526902041486e+14 true resid norm 8.543527855752e+13 ||r(i)||/||b|| 7.757807847199e-04 30 KSP preconditioned resid norm 1.980272137271e+15 true resid norm 3.202004161054e+14 ||r(i)||/||b|| 2.907526425475e-03 31 KSP preconditioned resid norm 4.796148387368e+14 true resid norm 7.312671161483e+13 ||r(i)||/||b|| 6.640148973394e-04 32 KSP preconditioned resid norm 1.482366487345e+14 true resid norm 3.206675205939e+13 ||r(i)||/||b|| 2.911767889807e-04 33 KSP preconditioned resid norm 7.403721045502e+13 true resid norm 1.122274917558e+13 ||r(i)||/||b|| 1.019063004083e-04 34 KSP preconditioned resid norm 3.421203410880e+13 true resid norm 9.812523407473e+12 ||r(i)||/||b|| 8.910098073838e-05 35 KSP preconditioned resid norm 1.809542403951e+13 true resid norm 2.774316021276e+12 ||r(i)||/||b|| 2.519171349804e-05 36 KSP preconditioned resid norm 1.070863017107e+13 true resid norm 1.674563212179e+12 ||r(i)||/||b|| 1.520559170335e-05 37 KSP preconditioned resid norm 6.984609262190e+12 true resid norm 1.277736482286e+12 ||r(i)||/||b|| 1.160227282721e-05 0 KSP preconditioned resid norm 9.569153877844e+15 true resid norm 5.893805322157e+15 ||r(i)||/||b|| 1.119437717455e-01 1 KSP preconditioned resid norm 4.594033327917e+15 true resid norm 8.149731925277e+14 ||r(i)||/||b|| 1.547916296116e-02 2 KSP preconditioned resid norm 3.788286152976e+15 true resid norm 6.151807578016e+14 ||r(i)||/||b|| 1.168441279773e-02 3 KSP preconditioned resid norm 3.712195492568e+15 true resid norm 6.206672982073e+14 ||r(i)||/||b|| 1.178862119847e-02 4 KSP preconditioned resid norm 3.886667577513e+15 true resid norm 6.330116967950e+14 ||r(i)||/||b|| 1.202308407301e-02 5 KSP preconditioned resid norm 4.317240607103e+15 true resid norm 6.816590875948e+14 ||r(i)||/||b|| 1.294706647725e-02 6 KSP preconditioned resid norm 5.249548366505e+15 true resid norm 7.833242603614e+14 ||r(i)||/||b|| 1.487804014750e-02 7 KSP preconditioned resid norm 6.521511569741e+15 true resid norm 1.182301409963e+15 ||r(i)||/||b|| 2.245599777001e-02 8 KSP preconditioned resid norm 7.063327681244e+15 true resid norm 1.110968116855e+15 ||r(i)||/||b|| 2.110113152571e-02 9 KSP preconditioned resid norm 7.059112088082e+15 true resid norm 1.341815038025e+15 ||r(i)||/||b|| 2.548571392011e-02 10 KSP preconditioned resid norm 6.417041101993e+15 true resid norm 1.020408664201e+15 ||r(i)||/||b|| 1.938109393654e-02 11 KSP preconditioned resid norm 4.850416656929e+15 true resid norm 8.501719778107e+14 ||r(i)||/||b|| 1.614770977770e-02 12 KSP preconditioned resid norm 2.420686956668e+15 true resid norm 3.735196627993e+14 ||r(i)||/||b|| 7.094431795646e-03 13 KSP preconditioned resid norm 1.348084459782e+15 true resid norm 2.753333760274e+14 ||r(i)||/||b|| 5.229534216894e-03 14 KSP preconditioned resid norm 1.225904445555e+15 true resid norm 2.239730314894e+14 ||r(i)||/||b|| 4.254023426925e-03 15 KSP preconditioned resid norm 1.156415256406e+15 true resid norm 2.064224387096e+14 ||r(i)||/||b|| 3.920676897010e-03 16 KSP preconditioned resid norm 1.062658328422e+15 true resid norm 1.696405409630e+14 ||r(i)||/||b|| 3.222061292889e-03 17 KSP preconditioned resid norm 9.163863908341e+14 true resid norm 1.701529770139e+14 ||r(i)||/||b|| 3.231794227925e-03 18 KSP preconditioned resid norm 8.234055963962e+14 true resid norm 1.322040551676e+14 ||r(i)||/||b|| 2.511012795057e-03 19 KSP preconditioned resid norm 7.097124944641e+14 true resid norm 1.155726966735e+14 ||r(i)||/||b|| 2.195125707289e-03 20 KSP preconditioned resid norm 6.309033027960e+14 true resid norm 1.023450925395e+14 ||r(i)||/||b|| 1.943887701115e-03 21 KSP preconditioned resid norm 5.667708572251e+14 true resid norm 8.941024380489e+13 ||r(i)||/||b|| 1.698210133710e-03 22 KSP preconditioned resid norm 5.205793964231e+14 true resid norm 8.689007901528e+13 ||r(i)||/||b|| 1.650343477697e-03 23 KSP preconditioned resid norm 4.890134322830e+14 true resid norm 7.566844928998e+13 ||r(i)||/||b|| 1.437205871699e-03 24 KSP preconditioned resid norm 4.617102016960e+14 true resid norm 8.638231658778e+13 ||r(i)||/||b|| 1.640699310952e-03 25 KSP preconditioned resid norm 4.386224186730e+14 true resid norm 6.736465167645e+13 ||r(i)||/||b|| 1.279488001179e-03 26 KSP preconditioned resid norm 4.119635069909e+14 true resid norm 8.360600093542e+13 ||r(i)||/||b|| 1.587967463073e-03 27 KSP preconditioned resid norm 3.764202708342e+14 true resid norm 5.766864517117e+13 ||r(i)||/||b|| 1.095327262956e-03 28 KSP preconditioned resid norm 3.345193309034e+14 true resid norm 7.248849449141e+13 ||r(i)||/||b|| 1.376807518738e-03 0 KSP preconditioned resid norm 9.592548277384e+15 true resid norm 5.933564471549e+15 ||r(i)||/||b|| 5.704186299543e-01 1 KSP preconditioned resid norm 4.627892277467e+15 true resid norm 8.123960687021e+14 ||r(i)||/||b|| 7.809906755228e-02 2 KSP preconditioned resid norm 3.843955835291e+15 true resid norm 6.217930929753e+14 ||r(i)||/||b|| 5.977559794129e-02 3 KSP preconditioned resid norm 3.802651796154e+15 true resid norm 6.334637368548e+14 ||r(i)||/||b|| 6.089754626161e-02 4 KSP preconditioned resid norm 4.025480937197e+15 true resid norm 6.491593110279e+14 ||r(i)||/||b|| 6.240642814182e-02 5 KSP preconditioned resid norm 4.519579789539e+15 true resid norm 7.102235297384e+14 ||r(i)||/||b|| 6.827678956506e-02 6 KSP preconditioned resid norm 5.567308133052e+15 true resid norm 8.274501588234e+14 ||r(i)||/||b|| 7.954628085944e-02 7 KSP preconditioned resid norm 7.053789984861e+15 true resid norm 1.260366188593e+15 ||r(i)||/||b|| 1.211643284547e-01 8 KSP preconditioned resid norm 7.835759896085e+15 true resid norm 1.231567591968e+15 ||r(i)||/||b|| 1.183957976482e-01 9 KSP preconditioned resid norm 8.126559485840e+15 true resid norm 1.514279927927e+15 ||r(i)||/||b|| 1.455741293445e-01 10 KSP preconditioned resid norm 7.865908148994e+15 true resid norm 1.247217430777e+15 ||r(i)||/||b|| 1.199002827947e-01 11 KSP preconditioned resid norm 6.705237228445e+15 true resid norm 1.156150868849e+15 ||r(i)||/||b|| 1.111456693176e-01 12 KSP preconditioned resid norm 3.837356917105e+15 true resid norm 5.892678306289e+14 ||r(i)||/||b|| 5.664880700887e-02 13 KSP preconditioned resid norm 1.588477046031e+15 true resid norm 3.087098247775e+14 ||r(i)||/||b|| 2.967758017080e-02 14 KSP preconditioned resid norm 1.304101915863e+15 true resid norm 2.634312731694e+14 ||r(i)||/||b|| 2.532476164182e-02 15 KSP preconditioned resid norm 1.188980488344e+15 true resid norm 2.212236776838e+14 ||r(i)||/||b|| 2.126716710384e-02 16 KSP preconditioned resid norm 1.053945473260e+15 true resid norm 1.687417173497e+14 ||r(i)||/||b|| 1.622185444993e-02 17 KSP preconditioned resid norm 8.630322699779e+14 true resid norm 2.032342536888e+14 ||r(i)||/||b|| 1.953776774564e-02 18 KSP preconditioned resid norm 7.497367164260e+14 true resid norm 1.258314731452e+14 ||r(i)||/||b|| 1.209671132095e-02 19 KSP preconditioned resid norm 6.482413928213e+14 true resid norm 1.007668116028e+14 ||r(i)||/||b|| 9.687139474922e-03 20 KSP preconditioned resid norm 5.423124085977e+14 true resid norm 1.262959277696e+14 ||r(i)||/||b|| 1.214136130695e-02 21 KSP preconditioned resid norm 4.512195524384e+14 true resid norm 7.029203398299e+13 ||r(i)||/||b|| 6.757470305334e-03 22 KSP preconditioned resid norm 3.576286880164e+14 true resid norm 9.097241124393e+13 ||r(i)||/||b|| 8.745562373884e-03 23 KSP preconditioned resid norm 2.908403868181e+14 true resid norm 4.482717971847e+13 ||r(i)||/||b|| 4.309426241567e-03 24 KSP preconditioned resid norm 2.299942905294e+14 true resid norm 6.485244464292e+13 ||r(i)||/||b|| 6.234539592478e-03 25 KSP preconditioned resid norm 1.957293334591e+14 true resid norm 3.000864324033e+13 ||r(i)||/||b|| 2.884857701642e-03 26 KSP preconditioned resid norm 1.685971736537e+14 true resid norm 4.568513443333e+13 ||r(i)||/||b|| 4.391905054321e-03 27 KSP preconditioned resid norm 1.478986486404e+14 true resid norm 2.270701271232e+13 ||r(i)||/||b|| 2.182921099758e-03 28 KSP preconditioned resid norm 1.388836935911e+14 true resid norm 2.435562734780e+13 ||r(i)||/||b|| 2.341409392284e-03 29 KSP preconditioned resid norm 1.300875297573e+14 true resid norm 2.023811842122e+13 ||r(i)||/||b|| 1.945575857149e-03 30 KSP preconditioned resid norm 1.230463362777e+14 true resid norm 2.013991197226e+13 ||r(i)||/||b|| 1.936134856156e-03 31 KSP preconditioned resid norm 1.148174878708e+14 true resid norm 1.790715664590e+13 ||r(i)||/||b|| 1.721490650234e-03 32 KSP preconditioned resid norm 1.085020096185e+14 true resid norm 1.697737976905e+13 ||r(i)||/||b|| 1.632107269502e-03 33 KSP preconditioned resid norm 1.008662380835e+14 true resid norm 1.575408064147e+13 ||r(i)||/||b|| 1.514506354280e-03 34 KSP preconditioned resid norm 9.448474542578e+13 true resid norm 1.450932025652e+13 ||r(i)||/||b|| 1.394842277686e-03 35 KSP preconditioned resid norm 8.955256800693e+13 true resid norm 1.427784032077e+13 ||r(i)||/||b|| 1.372589133148e-03 36 KSP preconditioned resid norm 8.409700445745e+13 true resid norm 1.287414873178e+13 ||r(i)||/||b|| 1.237646328210e-03 37 KSP preconditioned resid norm 7.999039854664e+13 true resid norm 1.307106382082e+13 ||r(i)||/||b|| 1.256576607951e-03 38 KSP preconditioned resid norm 7.508174679474e+13 true resid norm 1.150183263556e+13 ||r(i)||/||b|| 1.105719781996e-03 39 KSP preconditioned resid norm 7.047760131088e+13 true resid norm 1.131577846513e+13 ||r(i)||/||b|| 1.087833608263e-03 40 KSP preconditioned resid norm 6.612188631103e+13 true resid norm 1.012435424497e+13 ||r(i)||/||b|| 9.732969626060e-04 41 KSP preconditioned resid norm 6.096165352930e+13 true resid norm 1.005634567468e+13 ||r(i)||/||b|| 9.667590113164e-04 42 KSP preconditioned resid norm 5.668115744306e+13 true resid norm 8.662975702181e+12 ||r(i)||/||b|| 8.328084669947e-04 43 KSP preconditioned resid norm 5.243169528258e+13 true resid norm 8.972837494107e+12 ||r(i)||/||b|| 8.625967906361e-04 0 KSP preconditioned resid norm 2.201027933913e+16 true resid norm 1.374916449250e+16 ||r(i)||/||b|| 1.042532971848e-01 1 KSP preconditioned resid norm 1.033786235817e+16 true resid norm 1.769244185634e+15 ||r(i)||/||b|| 1.341532716246e-02 2 KSP preconditioned resid norm 8.200152533928e+15 true resid norm 1.329548327593e+15 ||r(i)||/||b|| 1.008132508660e-02 3 KSP preconditioned resid norm 7.725254307721e+15 true resid norm 1.292264285665e+15 ||r(i)||/||b|| 9.798618140627e-03 4 KSP preconditioned resid norm 7.783851928031e+15 true resid norm 1.241739377824e+15 ||r(i)||/||b|| 9.415512080968e-03 5 KSP preconditioned resid norm 8.247256237278e+15 true resid norm 1.289875346607e+15 ||r(i)||/||b|| 9.780503965494e-03 6 KSP preconditioned resid norm 9.468674539247e+15 true resid norm 1.409079612219e+15 ||r(i)||/||b|| 1.068437254131e-02 7 KSP preconditioned resid norm 1.106031993220e+16 true resid norm 1.997396506013e+15 ||r(i)||/||b|| 1.514529640334e-02 8 KSP preconditioned resid norm 1.133839709036e+16 true resid norm 1.772431554854e+15 ||r(i)||/||b|| 1.343949544924e-02 9 KSP preconditioned resid norm 1.055805824819e+16 true resid norm 2.045498232098e+15 ||r(i)||/||b|| 1.551002864197e-02 10 KSP preconditioned resid norm 8.628048224174e+15 true resid norm 1.368695682952e+15 ||r(i)||/||b|| 1.037816064155e-02 11 KSP preconditioned resid norm 5.442091241360e+15 true resid norm 9.810024540837e+14 ||r(i)||/||b|| 7.438469475027e-03 12 KSP preconditioned resid norm 2.000681531075e+15 true resid norm 3.042659606098e+14 ||r(i)||/||b|| 2.307102342980e-03 13 KSP preconditioned resid norm 8.105673271529e+14 true resid norm 1.475101242619e+14 ||r(i)||/||b|| 1.118498278992e-03 0 KSP preconditioned resid norm 1.928660996668e+16 true resid norm 1.203367101178e+16 ||r(i)||/||b|| 4.946564650623e-01 1 KSP preconditioned resid norm 9.233394049572e+15 true resid norm 1.583660413052e+15 ||r(i)||/||b|| 6.509799553373e-02 2 KSP preconditioned resid norm 7.543213441610e+15 true resid norm 1.216909563864e+15 ||r(i)||/||b|| 5.002232341004e-02 3 KSP preconditioned resid norm 7.341955893763e+15 true resid norm 1.220114169499e+15 ||r(i)||/||b|| 5.015405203166e-02 4 KSP preconditioned resid norm 7.657405519141e+15 true resid norm 1.212628294074e+15 ||r(i)||/||b|| 4.984633739730e-02 5 KSP preconditioned resid norm 8.417297612797e+15 true resid norm 1.311888034770e+15 ||r(i)||/||b|| 5.392651146949e-02 6 KSP preconditioned resid norm 1.008979814671e+16 true resid norm 1.496346817141e+15 ||r(i)||/||b|| 6.150888007071e-02 7 KSP preconditioned resid norm 1.235951387667e+16 true resid norm 2.203847820374e+15 ||r(i)||/||b|| 9.059143891287e-02 8 KSP preconditioned resid norm 1.311708980126e+16 true resid norm 2.061617788942e+15 ||r(i)||/||b|| 8.474492669688e-02 9 KSP preconditioned resid norm 1.289258338081e+16 true resid norm 2.353735302483e+15 ||r(i)||/||b|| 9.675271854106e-02 10 KSP preconditioned resid norm 1.151089236912e+16 true resid norm 1.820534806530e+15 ||r(i)||/||b|| 7.483496191971e-02 11 KSP preconditioned resid norm 8.373719062509e+15 true resid norm 1.433022884373e+15 ||r(i)||/||b|| 5.890588446727e-02 12 KSP preconditioned resid norm 3.564533969408e+15 true resid norm 5.439956922043e+14 ||r(i)||/||b|| 2.236150430334e-02 13 KSP preconditioned resid norm 1.257109874849e+15 true resid norm 2.286041152149e+14 ||r(i)||/||b|| 9.397008063476e-03 14 KSP preconditioned resid norm 1.068785016666e+15 true resid norm 1.971356784282e+14 ||r(i)||/||b|| 8.103465495567e-03 15 KSP preconditioned resid norm 9.963029863724e+14 true resid norm 1.558476632901e+14 ||r(i)||/||b|| 6.406279026229e-03 16 KSP preconditioned resid norm 8.667425302785e+14 true resid norm 2.560422780209e+14 ||r(i)||/||b|| 1.052488206037e-02 17 KSP preconditioned resid norm 7.240699142388e+14 true resid norm 1.131314963782e+14 ||r(i)||/||b|| 4.650386904448e-03 18 KSP preconditioned resid norm 6.352047441583e+14 true resid norm 1.637714957863e+14 ||r(i)||/||b|| 6.731996337968e-03 19 KSP preconditioned resid norm 5.532686184638e+14 true resid norm 8.631039548890e+13 ||r(i)||/||b|| 3.547877874413e-03 20 KSP preconditioned resid norm 4.676701949378e+14 true resid norm 1.246119427256e+14 ||r(i)||/||b|| 5.122302498784e-03 21 KSP preconditioned resid norm 4.001088470241e+14 true resid norm 6.181513171397e+13 ||r(i)||/||b|| 2.540974779106e-03 22 KSP preconditioned resid norm 3.482451502111e+14 true resid norm 6.477073783496e+13 ||r(i)||/||b|| 2.662468018741e-03 23 KSP preconditioned resid norm 3.041601141164e+14 true resid norm 4.679581096972e+13 ||r(i)||/||b|| 1.923590100755e-03 24 KSP preconditioned resid norm 2.690997224286e+14 true resid norm 4.294556821793e+13 ||r(i)||/||b|| 1.765321899192e-03 25 KSP preconditioned resid norm 2.482501027210e+14 true resid norm 3.840071688811e+13 ||r(i)||/||b|| 1.578501095230e-03 26 KSP preconditioned resid norm 2.328482920528e+14 true resid norm 3.868960801558e+13 ||r(i)||/||b|| 1.590376263145e-03 27 KSP preconditioned resid norm 2.171581409229e+14 true resid norm 3.334869397249e+13 ||r(i)||/||b|| 1.370832479858e-03 28 KSP preconditioned resid norm 2.002313534549e+14 true resid norm 3.534153915591e+13 ||r(i)||/||b|| 1.452750437635e-03 29 KSP preconditioned resid norm 1.813916789125e+14 true resid norm 2.780025874135e+13 ||r(i)||/||b|| 1.142758323985e-03 30 KSP preconditioned resid norm 1.655865002619e+14 true resid norm 2.941107681765e+13 ||r(i)||/||b|| 1.208972663292e-03 31 KSP preconditioned resid norm 1.477358437937e+14 true resid norm 2.263549873151e+13 ||r(i)||/||b|| 9.304555340168e-04 32 KSP preconditioned resid norm 1.346388933582e+14 true resid norm 2.267328899678e+13 ||r(i)||/||b|| 9.320089418685e-04 0 KSP preconditioned resid norm 6.130047183925e+16 true resid norm 4.019546453762e+16 ||r(i)||/||b|| 5.748521837148e-02 1 KSP preconditioned resid norm 2.383257388870e+16 true resid norm 4.333010621750e+15 ||r(i)||/||b|| 6.196820080636e-03 2 KSP preconditioned resid norm 1.380338515654e+16 true resid norm 2.902899468751e+15 ||r(i)||/||b|| 4.151558186756e-03 3 KSP preconditioned resid norm 9.355621435823e+15 true resid norm 1.914392486860e+15 ||r(i)||/||b|| 2.737852924995e-03 4 KSP preconditioned resid norm 6.905538449399e+15 true resid norm 1.160515328667e+15 ||r(i)||/||b|| 1.659701607116e-03 5 KSP preconditioned resid norm 5.388961375964e+15 true resid norm 9.343353155170e+14 ||r(i)||/||b|| 1.336232091419e-03 6 KSP preconditioned resid norm 4.457024362952e+15 true resid norm 7.443375401358e+14 ||r(i)||/||b|| 1.064508310303e-03 7 KSP preconditioned resid norm 3.604925656449e+15 true resid norm 7.177624609068e+14 ||r(i)||/||b|| 1.026502175773e-03 8 KSP preconditioned resid norm 2.616540301450e+15 true resid norm 4.566819450076e+14 ||r(i)||/||b|| 6.531199884630e-04 9 KSP preconditioned resid norm 1.730734442406e+15 true resid norm 3.416454932875e+14 ||r(i)||/||b|| 4.886015378397e-04 10 KSP preconditioned resid norm 9.605665765368e+14 true resid norm 1.715807715453e+14 ||r(i)||/||b|| 2.453848520993e-04 11 KSP preconditioned resid norm 3.885455022454e+14 true resid norm 7.526175493349e+13 ||r(i)||/||b|| 1.076349898463e-04 12 KSP preconditioned resid norm 1.085553513563e+14 true resid norm 1.877760955419e+13 ||r(i)||/||b|| 2.685464636705e-05 13 KSP preconditioned resid norm 5.523836175310e+13 true resid norm 1.483397872543e+13 ||r(i)||/||b|| 2.121469464673e-05 14 KSP preconditioned resid norm 4.329267915925e+13 true resid norm 6.950523101530e+12 ||r(i)||/||b|| 9.940234374291e-06 0 KSP preconditioned resid norm 2.469234905564e+16 true resid norm 1.581399294540e+16 ||r(i)||/||b|| 1.421323674244e-01 1 KSP preconditioned resid norm 1.094883401656e+16 true resid norm 1.827560974935e+15 ||r(i)||/||b|| 1.642567875658e-02 2 KSP preconditioned resid norm 7.902898352613e+15 true resid norm 1.331982197511e+15 ||r(i)||/||b|| 1.197153582609e-02 3 KSP preconditioned resid norm 6.757313480243e+15 true resid norm 1.177585457449e+15 ||r(i)||/||b|| 1.058385503836e-02 4 KSP preconditioned resid norm 6.237102240639e+15 true resid norm 9.859442147838e+14 ||r(i)||/||b|| 8.861429613598e-03 5 KSP preconditioned resid norm 6.056820105725e+15 true resid norm 9.499166634556e+14 ||r(i)||/||b|| 8.537622642110e-03 6 KSP preconditioned resid norm 6.282037371920e+15 true resid norm 9.440405134922e+14 ||r(i)||/||b|| 8.484809218675e-03 7 KSP preconditioned resid norm 6.512961686254e+15 true resid norm 1.175538616906e+15 ||r(i)||/||b|| 1.056545852755e-02 8 KSP preconditioned resid norm 5.899315381141e+15 true resid norm 9.360504236018e+14 ||r(i)||/||b|| 8.412996211297e-03 9 KSP preconditioned resid norm 4.860546577526e+15 true resid norm 9.070630816401e+14 ||r(i)||/||b|| 8.152464949358e-03 10 KSP preconditioned resid norm 3.431569954082e+15 true resid norm 5.519072654101e+14 ||r(i)||/||b|| 4.960409840975e-03 11 KSP preconditioned resid norm 1.788273694951e+15 true resid norm 3.211455497861e+14 ||r(i)||/||b|| 2.886379008546e-03 12 KSP preconditioned resid norm 6.021242921522e+14 true resid norm 9.258232372098e+13 ||r(i)||/||b|| 8.321076718289e-04 13 KSP preconditioned resid norm 3.678076314547e+14 true resid norm 7.160952479278e+13 ||r(i)||/||b|| 6.436091962400e-04 14 KSP preconditioned resid norm 3.596149520932e+14 true resid norm 5.543217762956e+13 ||r(i)||/||b|| 4.982110884443e-04 15 KSP preconditioned resid norm 1.443083471369e+14 true resid norm 3.905973798110e+13 ||r(i)||/||b|| 3.510595362853e-04 16 KSP preconditioned resid norm 1.158730309561e+14 true resid norm 1.821149714099e+13 ||r(i)||/||b|| 1.636805588525e-04 17 KSP preconditioned resid norm 9.042472136695e+13 true resid norm 2.428568415416e+13 ||r(i)||/||b|| 2.182738916901e-04 18 KSP preconditioned resid norm 7.667332005406e+13 true resid norm 1.178343148104e+13 ||r(i)||/||b|| 1.059066497984e-04 19 KSP preconditioned resid norm 6.435860750477e+13 true resid norm 1.303667702134e+13 ||r(i)||/||b|| 1.171705194752e-04 20 KSP preconditioned resid norm 5.245023814970e+13 true resid norm 8.751976411105e+12 ||r(i)||/||b|| 7.866066029293e-05 21 KSP preconditioned resid norm 4.161239918955e+13 true resid norm 6.811593020631e+12 ||r(i)||/||b|| 6.122096078432e-05 22 KSP preconditioned resid norm 3.107777724805e+13 true resid norm 6.092648679105e+12 ||r(i)||/||b|| 5.475926185348e-05 23 KSP preconditioned resid norm 2.312582950537e+13 true resid norm 3.536350440396e+12 ||r(i)||/||b|| 3.178386773481e-05 24 KSP preconditioned resid norm 1.680324737468e+13 true resid norm 4.383700522921e+12 ||r(i)||/||b|| 3.939964660118e-05 25 KSP preconditioned resid norm 1.300784122000e+13 true resid norm 1.953230538144e+12 ||r(i)||/||b|| 1.755516658383e-05 26 KSP preconditioned resid norm 9.922666654442e+12 true resid norm 3.054726768240e+12 ||r(i)||/||b|| 2.745514993611e-05 27 KSP preconditioned resid norm 7.226267347787e+12 true resid norm 1.085334598547e+12 ||r(i)||/||b|| 9.754726492653e-06 0 SNES Function norm 0.000000000000e+00 2 TS dt 4e-12 time 5e-12 0 KSP preconditioned resid norm 2.220258026867e+19 true resid norm 1.204245784292e+19 ||r(i)||/||b|| 2.302282866910e+02 1 KSP preconditioned resid norm 5.390122646802e+18 true resid norm 1.775868988684e+18 ||r(i)||/||b|| 3.395114851018e+01 2 KSP preconditioned resid norm 2.218369826326e+18 true resid norm 5.319781926844e+17 ||r(i)||/||b|| 1.017038460556e+01 3 KSP preconditioned resid norm 1.401713665358e+18 true resid norm 2.963575488339e+17 ||r(i)||/||b|| 5.665777834223e+00 4 KSP preconditioned resid norm 1.004075064815e+18 true resid norm 1.656255011022e+17 ||r(i)||/||b|| 3.166436274762e+00 5 KSP preconditioned resid norm 8.672862633339e+17 true resid norm 1.510357773334e+17 ||r(i)||/||b|| 2.887509235913e+00 6 KSP preconditioned resid norm 7.389427240230e+17 true resid norm 1.526197236053e+17 ||r(i)||/||b|| 2.917791196717e+00 7 KSP preconditioned resid norm 5.566238898025e+17 true resid norm 1.017349876189e+17 ||r(i)||/||b|| 1.944974373301e+00 8 KSP preconditioned resid norm 3.566703993070e+17 true resid norm 6.306898440820e+16 ||r(i)||/||b|| 1.205755869196e+00 9 KSP preconditioned resid norm 1.893399021800e+17 true resid norm 3.268112455206e+16 ||r(i)||/||b|| 6.247993068277e-01 10 KSP preconditioned resid norm 6.895422631218e+16 true resid norm 1.153042493210e+16 ||r(i)||/||b|| 2.204392169408e-01 11 KSP preconditioned resid norm 5.104209415340e+16 true resid norm 9.274112790250e+15 ||r(i)||/||b|| 1.773029331827e-01 12 KSP preconditioned resid norm 4.991932910946e+16 true resid norm 8.397724827599e+15 ||r(i)||/||b|| 1.605481060744e-01 13 KSP preconditioned resid norm 4.919289629176e+16 true resid norm 8.425037065615e+15 ||r(i)||/||b|| 1.610702627509e-01 14 KSP preconditioned resid norm 5.687556145039e+16 true resid norm 9.761935947260e+15 ||r(i)||/||b|| 1.866291596982e-01 15 KSP preconditioned resid norm 1.615122498031e+17 true resid norm 4.345955527842e+16 ||r(i)||/||b|| 8.308618624718e-01 16 KSP preconditioned resid norm 4.126010760136e+16 true resid norm 7.099883585094e+15 ||r(i)||/||b|| 1.357359149456e-01 17 KSP preconditioned resid norm 2.161057719247e+16 true resid norm 5.853862804145e+15 ||r(i)||/||b|| 1.119144298865e-01 18 KSP preconditioned resid norm 1.411351808998e+16 true resid norm 2.426558230375e+15 ||r(i)||/||b|| 4.639105664496e-02 19 KSP preconditioned resid norm 1.083945264815e+16 true resid norm 2.485250997240e+15 ||r(i)||/||b|| 4.751314777723e-02 20 KSP preconditioned resid norm 9.014213050640e+15 true resid norm 1.580604299892e+15 ||r(i)||/||b|| 3.021806882344e-02 21 KSP preconditioned resid norm 7.767372671831e+15 true resid norm 1.339058517057e+15 ||r(i)||/||b|| 2.560018496078e-02 22 KSP preconditioned resid norm 6.894209717493e+15 true resid norm 1.176401403286e+15 ||r(i)||/||b|| 2.249049845740e-02 23 KSP preconditioned resid norm 6.197326579110e+15 true resid norm 1.015741229888e+15 ||r(i)||/||b|| 1.941898955585e-02 24 KSP preconditioned resid norm 5.635415722302e+15 true resid norm 9.407688590092e+14 ||r(i)||/||b|| 1.798566417314e-02 25 KSP preconditioned resid norm 5.240066189102e+15 true resid norm 8.479712706847e+14 ||r(i)||/||b|| 1.621155542826e-02 26 KSP preconditioned resid norm 5.077381341025e+15 true resid norm 8.489262350542e+14 ||r(i)||/||b|| 1.622981248288e-02 27 KSP preconditioned resid norm 4.908170126730e+15 true resid norm 9.447288544322e+14 ||r(i)||/||b|| 1.806137155559e-02 28 KSP preconditioned resid norm 4.807291179172e+15 true resid norm 7.514342780137e+14 ||r(i)||/||b|| 1.436595657171e-02 29 KSP preconditioned resid norm 4.874566711284e+15 true resid norm 7.721244730622e+14 ||r(i)||/||b|| 1.476151271311e-02 30 KSP preconditioned resid norm 5.131077173970e+15 true resid norm 8.050523689285e+14 ||r(i)||/||b|| 1.539102980576e-02 31 KSP preconditioned resid norm 5.679198106771e+15 true resid norm 8.990076632730e+14 ||r(i)||/||b|| 1.718727162987e-02 32 KSP preconditioned resid norm 8.479698883566e+15 true resid norm 1.362468331537e+15 ||r(i)||/||b|| 2.604773491692e-02 33 KSP preconditioned resid norm 1.652298710742e+16 true resid norm 2.658951324175e+15 ||r(i)||/||b|| 5.083395895960e-02 34 KSP preconditioned resid norm 3.352309602032e+16 true resid norm 5.485872403557e+15 ||r(i)||/||b|| 1.048791717564e-01 35 KSP preconditioned resid norm 6.378357733800e+15 true resid norm 1.047516366293e+15 ||r(i)||/||b|| 2.002646813784e-02 36 KSP preconditioned resid norm 3.388128874134e+15 true resid norm 5.554504439336e+14 ||r(i)||/||b|| 1.061912823085e-02 37 KSP preconditioned resid norm 2.152326148598e+15 true resid norm 3.939853481438e+14 ||r(i)||/||b|| 7.532230784419e-03 38 KSP preconditioned resid norm 1.818288964313e+15 true resid norm 2.940442100180e+14 ||r(i)||/||b|| 5.621551311776e-03 39 KSP preconditioned resid norm 1.660791920226e+15 true resid norm 2.869955671543e+14 ||r(i)||/||b|| 5.486795019399e-03 40 KSP preconditioned resid norm 1.530575944029e+15 true resid norm 2.452126997689e+14 ||r(i)||/||b|| 4.687988156493e-03 41 KSP preconditioned resid norm 1.370742276038e+15 true resid norm 2.241019638220e+14 ||r(i)||/||b|| 4.284392094026e-03 42 KSP preconditioned resid norm 1.242467362496e+15 true resid norm 2.048557595739e+14 ||r(i)||/||b|| 3.916442238011e-03 43 KSP preconditioned resid norm 1.151462192104e+15 true resid norm 1.847175725318e+14 ||r(i)||/||b|| 3.531439412157e-03 44 KSP preconditioned resid norm 1.085430472672e+15 true resid norm 1.860586924987e+14 ||r(i)||/||b|| 3.557079008015e-03 45 KSP preconditioned resid norm 1.036632239565e+15 true resid norm 1.638613644087e+14 ||r(i)||/||b|| 3.132709424833e-03 46 KSP preconditioned resid norm 1.028748911897e+15 true resid norm 1.653234224458e+14 ||r(i)||/||b|| 3.160661120519e-03 47 KSP preconditioned resid norm 9.769896259678e+14 true resid norm 1.886232302190e+14 ||r(i)||/||b|| 3.606107963166e-03 48 KSP preconditioned resid norm 1.056096824670e+15 true resid norm 1.655410058571e+14 ||r(i)||/||b|| 3.164820890613e-03 49 KSP preconditioned resid norm 7.777695958066e+14 true resid norm 2.880991932726e+14 ||r(i)||/||b|| 5.507894196468e-03 50 KSP preconditioned resid norm 1.009825961040e+15 true resid norm 1.550789761056e+14 ||r(i)||/||b|| 2.964807303982e-03 51 KSP preconditioned resid norm 9.471596692833e+14 true resid norm 2.053798093618e+14 ||r(i)||/||b|| 3.926461046995e-03 52 KSP preconditioned resid norm 8.991518749617e+14 true resid norm 1.386233583178e+14 ||r(i)||/||b|| 2.650208013776e-03 53 KSP preconditioned resid norm 7.186213371144e+13 true resid norm 1.103090149144e+13 ||r(i)||/||b|| 2.108893038414e-04 0 KSP preconditioned resid norm 6.542826642292e+18 true resid norm 3.545903145424e+18 ||r(i)||/||b|| 4.156717341790e+02 1 KSP preconditioned resid norm 1.591157510957e+18 true resid norm 5.223152980213e+17 ||r(i)||/||b|| 6.122888776501e+01 2 KSP preconditioned resid norm 6.570525108060e+17 true resid norm 1.552502546069e+17 ||r(i)||/||b|| 1.819935286374e+01 3 KSP preconditioned resid norm 4.165816250413e+17 true resid norm 8.738489300540e+16 ||r(i)||/||b|| 1.024377387845e+01 4 KSP preconditioned resid norm 2.995010683652e+17 true resid norm 4.934680552583e+16 ||r(i)||/||b|| 5.784724339013e+00 5 KSP preconditioned resid norm 2.594493782706e+17 true resid norm 4.469794627497e+16 ||r(i)||/||b|| 5.239757568206e+00 6 KSP preconditioned resid norm 2.219609972421e+17 true resid norm 4.652118621437e+16 ||r(i)||/||b|| 5.453488535001e+00 7 KSP preconditioned resid norm 1.674912012123e+17 true resid norm 2.988815541157e+16 ||r(i)||/||b|| 3.503666310619e+00 8 KSP preconditioned resid norm 1.081901407775e+17 true resid norm 1.908145564246e+16 ||r(i)||/||b|| 2.236841062001e+00 9 KSP preconditioned resid norm 5.710123690945e+16 true resid norm 9.635193037737e+15 ||r(i)||/||b|| 1.129494302267e+00 10 KSP preconditioned resid norm 1.779373863118e+16 true resid norm 2.952490629534e+15 ||r(i)||/||b|| 3.461084101266e-01 11 KSP preconditioned resid norm 1.125622359514e+16 true resid norm 2.030418337832e+15 ||r(i)||/||b|| 2.380176437376e-01 12 KSP preconditioned resid norm 1.086650353983e+16 true resid norm 1.877894898988e+15 ||r(i)||/||b|| 2.201379443417e-01 13 KSP preconditioned resid norm 1.075411372217e+16 true resid norm 1.822725565845e+15 ||r(i)||/||b|| 2.136706688859e-01 14 KSP preconditioned resid norm 1.173963298627e+16 true resid norm 2.130483772307e+15 ||r(i)||/||b|| 2.497479056692e-01 15 KSP preconditioned resid norm 4.943816039086e+16 true resid norm 1.362098103563e+16 ||r(i)||/||b|| 1.596731939959e+00 16 KSP preconditioned resid norm 1.779827523151e+17 true resid norm 3.369509176778e+16 ||r(i)||/||b|| 3.949937901296e+00 17 KSP preconditioned resid norm 3.250308535899e+16 true resid norm 8.719651405683e+15 ||r(i)||/||b|| 1.022169098418e+00 18 KSP preconditioned resid norm 2.074788387365e+16 true resid norm 3.942900424212e+15 ||r(i)||/||b|| 4.622101027047e-01 19 KSP preconditioned resid norm 1.482962263858e+16 true resid norm 3.390403569469e+15 ||r(i)||/||b|| 3.974431543926e-01 20 KSP preconditioned resid norm 1.133931148892e+16 true resid norm 2.488262609273e+15 ||r(i)||/||b|| 2.916888565398e-01 21 KSP preconditioned resid norm 8.550489811072e+15 true resid norm 1.657275963132e+15 ||r(i)||/||b|| 1.942756881268e-01 22 KSP preconditioned resid norm 6.584107021814e+15 true resid norm 1.285719379545e+15 ||r(i)||/||b|| 1.507196283273e-01 23 KSP preconditioned resid norm 4.778317630297e+15 true resid norm 9.061620975056e+14 ||r(i)||/||b|| 1.062256793459e-01 24 KSP preconditioned resid norm 3.026263700342e+15 true resid norm 6.175894175717e+14 ||r(i)||/||b|| 7.239748342928e-02 25 KSP preconditioned resid norm 1.568620488361e+15 true resid norm 2.753144451691e+14 ||r(i)||/||b|| 3.227398723952e-02 26 KSP preconditioned resid norm 1.178330886619e+15 true resid norm 2.275547590347e+14 ||r(i)||/||b|| 2.667531442045e-02 27 KSP preconditioned resid norm 1.073508363176e+15 true resid norm 2.033900922703e+14 ||r(i)||/||b|| 2.384258929293e-02 28 KSP preconditioned resid norm 1.011358091649e+15 true resid norm 1.668017917552e+14 ||r(i)||/||b|| 1.955349235427e-02 29 KSP preconditioned resid norm 9.448460925412e+14 true resid norm 1.798337339042e+14 ||r(i)||/||b|| 2.108117367287e-02 30 KSP preconditioned resid norm 8.970299972493e+14 true resid norm 1.456018479256e+14 ||r(i)||/||b|| 1.706830958004e-02 31 KSP preconditioned resid norm 8.656072021210e+14 true resid norm 1.578084000747e+14 ||r(i)||/||b|| 1.849923380218e-02 32 KSP preconditioned resid norm 8.314911059315e+14 true resid norm 1.333543211106e+14 ||r(i)||/||b|| 1.563258206526e-02 33 KSP preconditioned resid norm 8.004072124456e+14 true resid norm 1.582598207144e+14 ||r(i)||/||b|| 1.855215199888e-02 34 KSP preconditioned resid norm 7.705686923615e+14 true resid norm 1.334906781329e+14 ||r(i)||/||b|| 1.564856664172e-02 35 KSP preconditioned resid norm 7.648238175354e+14 true resid norm 1.202157111000e+14 ||r(i)||/||b|| 1.409239650919e-02 36 KSP preconditioned resid norm 7.709241111814e+14 true resid norm 1.292601566237e+14 ||r(i)||/||b|| 1.515263989468e-02 37 KSP preconditioned resid norm 8.026855071425e+14 true resid norm 1.264132558814e+14 ||r(i)||/||b|| 1.481890935550e-02 38 KSP preconditioned resid norm 8.397026301556e+14 true resid norm 1.379015425365e+14 ||r(i)||/||b|| 1.616563424923e-02 39 KSP preconditioned resid norm 8.772641473437e+14 true resid norm 1.384855714837e+14 ||r(i)||/||b|| 1.623409757587e-02 40 KSP preconditioned resid norm 9.403663540144e+14 true resid norm 1.523096876425e+14 ||r(i)||/||b|| 1.785464221614e-02 41 KSP preconditioned resid norm 1.191025293024e+15 true resid norm 1.883988208054e+14 ||r(i)||/||b|| 2.208522380610e-02 42 KSP preconditioned resid norm 2.105592809750e+15 true resid norm 3.674437948704e+14 ||r(i)||/||b|| 4.307393438656e-02 43 KSP preconditioned resid norm 1.653069974154e+16 true resid norm 2.622043044873e+15 ||r(i)||/||b|| 3.073713902652e-01 44 KSP preconditioned resid norm 1.224677806349e+15 true resid norm 2.078786938958e+14 ||r(i)||/||b|| 2.436876971726e-02 45 KSP preconditioned resid norm 6.142260967855e+14 true resid norm 9.787693095074e+13 ||r(i)||/||b|| 1.147371260744e-02 46 KSP preconditioned resid norm 4.190317366481e+14 true resid norm 7.181248026289e+13 ||r(i)||/||b|| 8.418283574692e-03 47 KSP preconditioned resid norm 2.851377922408e+14 true resid norm 4.551212829362e+13 ||r(i)||/||b|| 5.335200798815e-03 48 KSP preconditioned resid norm 1.874725496828e+14 true resid norm 3.515118139860e+13 ||r(i)||/||b|| 4.120629337903e-03 49 KSP preconditioned resid norm 1.260137969667e+14 true resid norm 2.018341951104e+13 ||r(i)||/||b|| 2.366019782758e-03 50 KSP preconditioned resid norm 8.902880553848e+13 true resid norm 2.353973309483e+13 ||r(i)||/||b|| 2.759466707450e-03 51 KSP preconditioned resid norm 6.273228119323e+13 true resid norm 1.015825242680e+13 ||r(i)||/||b|| 1.190810416783e-03 52 KSP preconditioned resid norm 4.428322011916e+13 true resid norm 1.394037459429e+13 ||r(i)||/||b|| 1.634173141528e-03 0 KSP preconditioned resid norm 1.892236961136e+19 true resid norm 1.028105836629e+19 ||r(i)||/||b|| 7.839481546009e+01 1 KSP preconditioned resid norm 4.580244542528e+18 true resid norm 1.519483928522e+18 ||r(i)||/||b|| 1.158632291804e+01 2 KSP preconditioned resid norm 1.874052673028e+18 true resid norm 4.645668462025e+17 ||r(i)||/||b|| 3.542401071890e+00 3 KSP preconditioned resid norm 1.172977406681e+18 true resid norm 2.550460309107e+17 ||r(i)||/||b|| 1.944769284904e+00 4 KSP preconditioned resid norm 8.299170411979e+17 true resid norm 1.381052822672e+17 ||r(i)||/||b|| 1.053076223446e+00 5 KSP preconditioned resid norm 7.051011041666e+17 true resid norm 1.287511951247e+17 ||r(i)||/||b|| 9.817497209388e-01 6 KSP preconditioned resid norm 5.862228682149e+17 true resid norm 1.166604371743e+17 ||r(i)||/||b|| 8.895556389170e-01 7 KSP preconditioned resid norm 4.329574557115e+17 true resid norm 8.609505694316e+16 ||r(i)||/||b|| 6.564894255647e-01 8 KSP preconditioned resid norm 2.701974155126e+17 true resid norm 4.802624326608e+16 ||r(i)||/||b|| 3.662082583277e-01 9 KSP preconditioned resid norm 1.546976756797e+17 true resid norm 2.878555741954e+16 ||r(i)||/||b|| 2.194947622532e-01 10 KSP preconditioned resid norm 9.916116392638e+16 true resid norm 1.649766880508e+16 ||r(i)||/||b|| 1.257975254509e-01 11 KSP preconditioned resid norm 9.248895040376e+16 true resid norm 1.560136782751e+16 ||r(i)||/||b|| 1.189630783317e-01 12 KSP preconditioned resid norm 9.251831545179e+16 true resid norm 1.508056895396e+16 ||r(i)||/||b|| 1.149918985048e-01 13 KSP preconditioned resid norm 7.096157383949e+16 true resid norm 1.151221892162e+16 ||r(i)||/||b|| 8.778262370879e-02 14 KSP preconditioned resid norm 2.017629866031e+17 true resid norm 3.268014415733e+16 ||r(i)||/||b|| 2.491916473135e-01 15 KSP preconditioned resid norm 1.669814416925e+16 true resid norm 4.397468530146e+15 ||r(i)||/||b|| 3.353144410138e-02 16 KSP preconditioned resid norm 1.171914014315e+16 true resid norm 1.865370183430e+15 ||r(i)||/||b|| 1.422376433288e-02 17 KSP preconditioned resid norm 1.021030261775e+16 true resid norm 2.000923353135e+15 ||r(i)||/||b|| 1.525738026477e-02 18 KSP preconditioned resid norm 9.416598060128e+15 true resid norm 1.541246281727e+15 ||r(i)||/||b|| 1.175226455582e-02 19 KSP preconditioned resid norm 8.821999311745e+15 true resid norm 1.426601243635e+15 ||r(i)||/||b|| 1.087807667707e-02 20 KSP preconditioned resid norm 8.323362047556e+15 true resid norm 1.490210405686e+15 ||r(i)||/||b|| 1.136310733664e-02 21 KSP preconditioned resid norm 7.861641988008e+15 true resid norm 1.251748347072e+15 ||r(i)||/||b|| 9.544793655962e-03 22 KSP preconditioned resid norm 7.464952525350e+15 true resid norm 1.325467789570e+15 ||r(i)||/||b|| 1.010691692037e-02 23 KSP preconditioned resid norm 7.139096535480e+15 true resid norm 1.125905927178e+15 ||r(i)||/||b|| 8.585223839986e-03 24 KSP preconditioned resid norm 5.858113828393e+15 true resid norm 2.042448137714e+15 ||r(i)||/||b|| 1.557401379686e-02 25 KSP preconditioned resid norm 5.027356938946e+15 true resid norm 7.946522117539e+14 ||r(i)||/||b|| 6.059358022871e-03 26 KSP preconditioned resid norm 5.122614017475e+15 true resid norm 7.976827488389e+14 ||r(i)||/||b|| 6.082466382639e-03 27 KSP preconditioned resid norm 5.425317067876e+15 true resid norm 8.948524578548e+14 ||r(i)||/||b|| 6.823401910404e-03 28 KSP preconditioned resid norm 9.316529191423e+15 true resid norm 1.453760069771e+15 ||r(i)||/||b|| 1.108516733711e-02 29 KSP preconditioned resid norm 9.523628080058e+15 true resid norm 2.109739324167e+15 ||r(i)||/||b|| 1.608712051759e-02 30 KSP preconditioned resid norm 2.300173794637e+15 true resid norm 3.594261469430e+14 ||r(i)||/||b|| 2.740685390281e-03 31 KSP preconditioned resid norm 1.385027169120e+15 true resid norm 2.935564708865e+14 ||r(i)||/||b|| 2.238417927644e-03 32 KSP preconditioned resid norm 9.719758142764e+14 true resid norm 1.523176597454e+14 ||r(i)||/||b|| 1.161448014555e-03 33 KSP preconditioned resid norm 8.580268959656e+14 true resid norm 1.580637131624e+14 ||r(i)||/||b|| 1.205262647368e-03 0 KSP preconditioned resid norm 6.470797769938e+18 true resid norm 3.511548722307e+18 ||r(i)||/||b|| 1.662246831388e+02 1 KSP preconditioned resid norm 1.571597715354e+18 true resid norm 5.187477038218e+17 ||r(i)||/||b|| 2.455573865428e+01 2 KSP preconditioned resid norm 6.468337546861e+17 true resid norm 1.566996108585e+17 ||r(i)||/||b|| 7.417622599812e+00 3 KSP preconditioned resid norm 4.084044703454e+17 true resid norm 8.688412887362e+16 ||r(i)||/||b|| 4.112796926342e+00 4 KSP preconditioned resid norm 2.920894102259e+17 true resid norm 4.831956260441e+16 ||r(i)||/||b|| 2.287282512214e+00 5 KSP preconditioned resid norm 2.515869150351e+17 true resid norm 4.435569840236e+16 ||r(i)||/||b|| 2.099646764259e+00 6 KSP preconditioned resid norm 2.133085363553e+17 true resid norm 4.355632900946e+16 ||r(i)||/||b|| 2.061807356479e+00 7 KSP preconditioned resid norm 1.601295664581e+17 true resid norm 2.999352167191e+16 ||r(i)||/||b|| 1.419790534148e+00 8 KSP preconditioned resid norm 1.019894513782e+17 true resid norm 1.811122934468e+16 ||r(i)||/||b|| 8.573235336161e-01 9 KSP preconditioned resid norm 5.552366492146e+16 true resid norm 9.849650352808e+15 ||r(i)||/||b|| 4.662486949199e-01 10 KSP preconditioned resid norm 2.614329828139e+16 true resid norm 4.400937150634e+15 ||r(i)||/||b|| 2.083252835795e-01 11 KSP preconditioned resid norm 2.230447954838e+16 true resid norm 4.002038016135e+15 ||r(i)||/||b|| 1.894427654999e-01 12 KSP preconditioned resid norm 2.202888172936e+16 true resid norm 3.669015023581e+15 ||r(i)||/||b|| 1.736785982356e-01 13 KSP preconditioned resid norm 2.171444770780e+16 true resid norm 3.746639147837e+15 ||r(i)||/||b|| 1.773530582755e-01 14 KSP preconditioned resid norm 2.756620760322e+16 true resid norm 4.738417417687e+15 ||r(i)||/||b|| 2.243004429444e-01 15 KSP preconditioned resid norm 1.911455835947e+16 true resid norm 7.274462982878e+15 ||r(i)||/||b|| 3.443481494796e-01 16 KSP preconditioned resid norm 8.270124664006e+15 true resid norm 1.450957167614e+15 ||r(i)||/||b|| 6.868334017480e-02 17 KSP preconditioned resid norm 5.454864800054e+15 true resid norm 1.779587764442e+15 ||r(i)||/||b|| 8.423958647728e-02 18 KSP preconditioned resid norm 4.188178169772e+15 true resid norm 7.356207570842e+14 ||r(i)||/||b|| 3.482176581514e-02 19 KSP preconditioned resid norm 3.485885419242e+15 true resid norm 7.684000525717e+14 ||r(i)||/||b|| 3.637342533543e-02 20 KSP preconditioned resid norm 3.031526098420e+15 true resid norm 5.668894180734e+14 ||r(i)||/||b|| 2.683460243492e-02 21 KSP preconditioned resid norm 2.653824581081e+15 true resid norm 5.337896513987e+14 ||r(i)||/||b|| 2.526777290683e-02 22 KSP preconditioned resid norm 2.412624108418e+15 true resid norm 3.917881686680e+14 ||r(i)||/||b|| 1.854590932504e-02 23 KSP preconditioned resid norm 2.249293018506e+15 true resid norm 3.520070992955e+14 ||r(i)||/||b|| 1.666280982271e-02 24 KSP preconditioned resid norm 2.107969657718e+15 true resid norm 3.433991961929e+14 ||r(i)||/||b|| 1.625534118740e-02 25 KSP preconditioned resid norm 1.995899579616e+15 true resid norm 3.124097002184e+14 ||r(i)||/||b|| 1.478840464277e-02 26 KSP preconditioned resid norm 1.940326192847e+15 true resid norm 3.197343858652e+14 ||r(i)||/||b|| 1.513513016106e-02 27 KSP preconditioned resid norm 1.843618399864e+15 true resid norm 4.284525431489e+14 ||r(i)||/||b|| 2.028147517148e-02 28 KSP preconditioned resid norm 1.800569430365e+15 true resid norm 2.810328443449e+14 ||r(i)||/||b|| 1.330313180793e-02 29 KSP preconditioned resid norm 1.911265522964e+15 true resid norm 3.078500227429e+14 ||r(i)||/||b|| 1.457256513619e-02 30 KSP preconditioned resid norm 2.245578740852e+15 true resid norm 3.522723048093e+14 ||r(i)||/||b|| 1.667536374293e-02 31 KSP preconditioned resid norm 3.129444954320e+15 true resid norm 5.056459987900e+14 ||r(i)||/||b|| 2.393554883500e-02 32 KSP preconditioned resid norm 2.268235755551e+16 true resid norm 3.663405821329e+15 ||r(i)||/||b|| 1.734130778226e-01 33 KSP preconditioned resid norm 2.666354866266e+15 true resid norm 4.420681584183e+14 ||r(i)||/||b|| 2.092599174035e-02 34 KSP preconditioned resid norm 1.195729215971e+15 true resid norm 1.967322086000e+14 ||r(i)||/||b|| 9.312628593187e-03 35 KSP preconditioned resid norm 7.390918117697e+14 true resid norm 1.247185996576e+14 ||r(i)||/||b|| 5.903751122089e-03 36 KSP preconditioned resid norm 5.589675490221e+14 true resid norm 9.067132579829e+13 ||r(i)||/||b|| 4.292069850790e-03 37 KSP preconditioned resid norm 4.470335591004e+14 true resid norm 9.109636100840e+13 ||r(i)||/||b|| 4.312189561126e-03 38 KSP preconditioned resid norm 4.031608678043e+14 true resid norm 6.399106058589e+13 ||r(i)||/||b|| 3.029117523568e-03 39 KSP preconditioned resid norm 3.769205171045e+14 true resid norm 7.200688789282e+13 ||r(i)||/||b|| 3.408559319641e-03 40 KSP preconditioned resid norm 3.535103364248e+14 true resid norm 5.569365215364e+13 ||r(i)||/||b|| 2.636346641945e-03 41 KSP preconditioned resid norm 3.299087996116e+14 true resid norm 5.691184302265e+13 ||r(i)||/||b|| 2.694011623187e-03 42 KSP preconditioned resid norm 3.102396074559e+14 true resid norm 5.072539063269e+13 ||r(i)||/||b|| 2.401166166782e-03 43 KSP preconditioned resid norm 2.923631153501e+14 true resid norm 5.690604160219e+13 ||r(i)||/||b|| 2.693737003823e-03 44 KSP preconditioned resid norm 2.831787236990e+14 true resid norm 4.398532156025e+13 ||r(i)||/||b|| 2.082114393761e-03 45 KSP preconditioned resid norm 2.880585984465e+14 true resid norm 4.666639157225e+13 ||r(i)||/||b|| 2.209027060639e-03 46 KSP preconditioned resid norm 3.054400711070e+14 true resid norm 4.771426500640e+13 ||r(i)||/||b|| 2.258629798159e-03 47 KSP preconditioned resid norm 3.761477931303e+14 true resid norm 5.957625439087e+13 ||r(i)||/||b|| 2.820135726954e-03 48 KSP preconditioned resid norm 9.608784364697e+14 true resid norm 1.580857575603e+14 ||r(i)||/||b|| 7.483238034628e-03 49 KSP preconditioned resid norm 1.902148260101e+15 true resid norm 2.982038509627e+14 ||r(i)||/||b|| 1.411594841961e-02 50 KSP preconditioned resid norm 2.165055327372e+14 true resid norm 4.058692191960e+13 ||r(i)||/||b|| 1.921245800409e-03 51 KSP preconditioned resid norm 2.195213865618e+14 true resid norm 3.429224338721e+13 ||r(i)||/||b|| 1.623277289290e-03 52 KSP preconditioned resid norm 1.864119738188e+14 true resid norm 5.012858400476e+13 ||r(i)||/||b|| 2.372915386152e-03 53 KSP preconditioned resid norm 2.971924491921e+13 true resid norm 4.598158953060e+12 ||r(i)||/||b|| 2.176610878666e-04 0 KSP preconditioned resid norm 6.694681472101e+18 true resid norm 3.650754008704e+18 ||r(i)||/||b|| 5.229798534711e+00 1 KSP preconditioned resid norm 1.597667311920e+18 true resid norm 5.366749653247e+17 ||r(i)||/||b|| 7.688006205237e-01 2 KSP preconditioned resid norm 6.300446691769e+17 true resid norm 1.705328302298e+17 ||r(i)||/||b|| 2.442926429799e-01 3 KSP preconditioned resid norm 3.642174434692e+17 true resid norm 9.481577406412e+16 ||r(i)||/||b|| 1.358260225383e-01 4 KSP preconditioned resid norm 2.247332932327e+17 true resid norm 3.802800659066e+16 ||r(i)||/||b|| 5.447609251997e-02 5 KSP preconditioned resid norm 1.557523729879e+17 true resid norm 3.829141784861e+16 ||r(i)||/||b|| 5.485343588727e-02 6 KSP preconditioned resid norm 1.004692897232e+17 true resid norm 1.699728149080e+16 ||r(i)||/||b|| 2.434904067015e-02 7 KSP preconditioned resid norm 7.103483293598e+16 true resid norm 1.472858902835e+16 ||r(i)||/||b|| 2.109908066531e-02 8 KSP preconditioned resid norm 5.766848376574e+16 true resid norm 9.397954931890e+15 ||r(i)||/||b|| 1.346281091931e-02 9 KSP preconditioned resid norm 4.523472389397e+16 true resid norm 1.246375028587e+16 ||r(i)||/||b|| 1.785464121292e-02 10 KSP preconditioned resid norm 3.195342596604e+16 true resid norm 5.030521805863e+15 ||r(i)||/||b|| 7.206351210301e-03 11 KSP preconditioned resid norm 2.437460193025e+16 true resid norm 4.974645265358e+15 ||r(i)||/||b|| 7.126306636231e-03 12 KSP preconditioned resid norm 1.258045317564e+16 true resid norm 1.885909944349e+15 ||r(i)||/||b|| 2.701614252848e-03 13 KSP preconditioned resid norm 2.495567071066e+15 true resid norm 4.460991862355e+14 ||r(i)||/||b|| 6.390484992820e-04 14 KSP preconditioned resid norm 7.678105595258e+14 true resid norm 1.316278119117e+14 ||r(i)||/||b|| 1.885602087190e-04 15 KSP preconditioned resid norm 5.950875767888e+14 true resid norm 1.439411571316e+14 ||r(i)||/||b|| 2.061993908263e-04 16 KSP preconditioned resid norm 3.263039815738e+14 true resid norm 5.046549667518e+13 ||r(i)||/||b|| 7.229311532250e-05 17 KSP preconditioned resid norm 1.436785883111e+14 true resid norm 7.588987254543e+13 ||r(i)||/||b|| 1.087141843277e-04 18 KSP preconditioned resid norm 5.401544347363e+13 true resid norm 8.625334293000e+12 ||r(i)||/||b|| 1.235601208391e-05 19 KSP preconditioned resid norm 3.528863285477e+13 true resid norm 1.233589079533e+13 ||r(i)||/||b|| 1.767147922099e-05 0 KSP preconditioned resid norm 4.827871909881e+18 true resid norm 2.629048338874e+18 ||r(i)||/||b|| 2.387243403092e+01 1 KSP preconditioned resid norm 1.161059375885e+18 true resid norm 3.885115498370e+17 ||r(i)||/||b|| 3.527784638492e+00 2 KSP preconditioned resid norm 4.696929326234e+17 true resid norm 1.228954029189e+17 ||r(i)||/||b|| 1.115921816843e+00 3 KSP preconditioned resid norm 2.870100194440e+17 true resid norm 6.759377089469e+16 ||r(i)||/||b|| 6.137687971441e-01 4 KSP preconditioned resid norm 1.966638027618e+17 true resid norm 3.310588396768e+16 ||r(i)||/||b|| 3.006099276942e-01 5 KSP preconditioned resid norm 1.592573327953e+17 true resid norm 3.239413757211e+16 ||r(i)||/||b|| 2.941470876529e-01 6 KSP preconditioned resid norm 1.230114084345e+17 true resid norm 2.280735920899e+16 ||r(i)||/||b|| 2.070966783247e-01 7 KSP preconditioned resid norm 8.598639005636e+16 true resid norm 1.938027655308e+16 ||r(i)||/||b|| 1.759778877677e-01 8 KSP preconditioned resid norm 5.549090128976e+16 true resid norm 9.386098364471e+15 ||r(i)||/||b|| 8.522818340779e-02 9 KSP preconditioned resid norm 4.474765035679e+16 true resid norm 7.579175521646e+15 ||r(i)||/||b|| 6.882085999480e-02 10 KSP preconditioned resid norm 4.378403606928e+16 true resid norm 6.842614495131e+15 ||r(i)||/||b|| 6.213269673237e-02 11 KSP preconditioned resid norm 4.079517449398e+16 true resid norm 7.704845808673e+15 ||r(i)||/||b|| 6.996197847191e-02 12 KSP preconditioned resid norm 4.084405622719e+16 true resid norm 6.377952548774e+15 ||r(i)||/||b|| 5.791344693880e-02 13 KSP preconditioned resid norm 7.523868588173e+16 true resid norm 1.132249240798e+16 ||r(i)||/||b|| 1.028111385699e-01 14 KSP preconditioned resid norm 2.184581375390e+15 true resid norm 3.502138477296e+14 ||r(i)||/||b|| 3.180031668881e-03 15 KSP preconditioned resid norm 2.222269400035e+15 true resid norm 3.654767263686e+14 ||r(i)||/||b|| 3.318622526282e-03 16 KSP preconditioned resid norm 1.669508240008e+15 true resid norm 3.629680148217e+14 ||r(i)||/||b|| 3.295842781223e-03 17 KSP preconditioned resid norm 1.401913603747e+15 true resid norm 2.161155966391e+14 ||r(i)||/||b|| 1.962385113858e-03 18 KSP preconditioned resid norm 1.134372518022e+15 true resid norm 2.686701598571e+14 ||r(i)||/||b|| 2.439594043376e-03 19 KSP preconditioned resid norm 9.057728167563e+14 true resid norm 1.409896456102e+14 ||r(i)||/||b|| 1.280222186904e-03 20 KSP preconditioned resid norm 7.989861774441e+14 true resid norm 1.659027150307e+14 ||r(i)||/||b|| 1.506439254676e-03 21 KSP preconditioned resid norm 7.119014585200e+14 true resid norm 1.092105820565e+14 ||r(i)||/||b|| 9.916601292842e-04 22 KSP preconditioned resid norm 6.306860453827e+14 true resid norm 1.460692513610e+14 ||r(i)||/||b|| 1.326346311515e-03 23 KSP preconditioned resid norm 5.637172379886e+14 true resid norm 8.672554380666e+13 ||r(i)||/||b|| 7.874902080372e-04 24 KSP preconditioned resid norm 4.961067656606e+14 true resid norm 1.292132364829e+14 ||r(i)||/||b|| 1.173289367962e-03 25 KSP preconditioned resid norm 4.386188245672e+14 true resid norm 8.044020700160e+13 ||r(i)||/||b|| 7.304177358342e-04 26 KSP preconditioned resid norm 4.419078090293e+14 true resid norm 6.703649620362e+13 ||r(i)||/||b|| 6.087085998464e-04 27 KSP preconditioned resid norm 4.436677280477e+14 true resid norm 1.368871842229e+14 ||r(i)||/||b|| 1.242970784036e-03 28 KSP preconditioned resid norm 7.847692003240e+14 true resid norm 1.196853324186e+14 ||r(i)||/||b|| 1.086773552386e-03 29 KSP preconditioned resid norm 1.771704167839e+15 true resid norm 6.548256652554e+14 ||r(i)||/||b|| 5.945985193355e-03 30 KSP preconditioned resid norm 1.053899282998e+15 true resid norm 1.880673778730e+14 ||r(i)||/||b|| 1.707700084953e-03 31 KSP preconditioned resid norm 2.462670432282e+14 true resid norm 3.919805233311e+13 ||r(i)||/||b|| 3.559283808617e-04 32 KSP preconditioned resid norm 1.093017734594e+14 true resid norm 1.903326631551e+13 ||r(i)||/||b|| 1.728269457018e-04 33 KSP preconditioned resid norm 6.432042300545e+13 true resid norm 1.002471721316e+13 ||r(i)||/||b|| 9.102700654495e-05 34 KSP preconditioned resid norm 3.805689715853e+13 true resid norm 7.047382673791e+12 ||r(i)||/||b|| 6.399204437707e-05 35 KSP preconditioned resid norm 2.527168724251e+13 true resid norm 4.009865388649e+12 ||r(i)||/||b|| 3.641060742320e-05 36 KSP preconditioned resid norm 1.580947387777e+13 true resid norm 2.531811988645e+12 ||r(i)||/||b|| 2.298950300149e-05 37 KSP preconditioned resid norm 1.075548264218e+13 true resid norm 1.788042513750e+12 ||r(i)||/||b|| 1.623588517670e-05 38 KSP preconditioned resid norm 6.581274433602e+12 true resid norm 1.035380330637e+12 ||r(i)||/||b|| 9.401519277742e-06 0 KSP preconditioned resid norm 9.555736051823e+15 true resid norm 5.892741751019e+15 ||r(i)||/||b|| 1.119235791581e-01 1 KSP preconditioned resid norm 4.575036500991e+15 true resid norm 8.101073993664e+14 ||r(i)||/||b|| 1.538674584270e-02 2 KSP preconditioned resid norm 3.765161451471e+15 true resid norm 6.105904357791e+14 ||r(i)||/||b|| 1.159722754867e-02 3 KSP preconditioned resid norm 3.685525899039e+15 true resid norm 6.160361102521e+14 ||r(i)||/||b|| 1.170065977151e-02 4 KSP preconditioned resid norm 3.856425370708e+15 true resid norm 6.278594964367e+14 ||r(i)||/||b|| 1.192522683307e-02 5 KSP preconditioned resid norm 4.281981871434e+15 true resid norm 6.756473177150e+14 ||r(i)||/||b|| 1.283288310305e-02 6 KSP preconditioned resid norm 5.205822014832e+15 true resid norm 7.760311477213e+14 ||r(i)||/||b|| 1.473951977891e-02 7 KSP preconditioned resid norm 6.471690051175e+15 true resid norm 1.171341036231e+15 ||r(i)||/||b|| 2.224782397211e-02 8 KSP preconditioned resid norm 7.016914241690e+15 true resid norm 1.103022939349e+15 ||r(i)||/||b|| 2.095022664859e-02 9 KSP preconditioned resid norm 7.025590013206e+15 true resid norm 1.330551872513e+15 ||r(i)||/||b|| 2.527178928238e-02 10 KSP preconditioned resid norm 6.405136813848e+15 true resid norm 1.017867183616e+15 ||r(i)||/||b|| 1.933282385541e-02 11 KSP preconditioned resid norm 4.854020251396e+15 true resid norm 8.473369029366e+14 ||r(i)||/||b|| 1.609386308386e-02 12 KSP preconditioned resid norm 2.353499572340e+15 true resid norm 3.616493792413e+14 ||r(i)||/||b|| 6.868974517339e-03 13 KSP preconditioned resid norm 1.176204335431e+15 true resid norm 2.390307465685e+14 ||r(i)||/||b|| 4.540021914275e-03 14 KSP preconditioned resid norm 1.050288991204e+15 true resid norm 1.923001174300e+14 ||r(i)||/||b|| 3.652445385304e-03 15 KSP preconditioned resid norm 9.850224343838e+14 true resid norm 1.772358652449e+14 ||r(i)||/||b|| 3.366323051568e-03 16 KSP preconditioned resid norm 8.835883676210e+14 true resid norm 1.426145250921e+14 ||r(i)||/||b|| 2.708743868757e-03 17 KSP preconditioned resid norm 7.483591829964e+14 true resid norm 1.680673516819e+14 ||r(i)||/||b|| 3.192181217957e-03 18 KSP preconditioned resid norm 6.652269160454e+14 true resid norm 1.088301440117e+14 ||r(i)||/||b|| 2.067061438079e-03 19 KSP preconditioned resid norm 5.777832230782e+14 true resid norm 9.220881620683e+13 ||r(i)||/||b|| 1.751364844390e-03 20 KSP preconditioned resid norm 5.145663259773e+14 true resid norm 8.255284700142e+13 ||r(i)||/||b|| 1.567964539511e-03 21 KSP preconditioned resid norm 4.536494497263e+14 true resid norm 7.162151119367e+13 ||r(i)||/||b|| 1.360340604800e-03 22 KSP preconditioned resid norm 4.039727797390e+14 true resid norm 6.757078541538e+13 ||r(i)||/||b|| 1.283403289972e-03 23 KSP preconditioned resid norm 3.668438626905e+14 true resid norm 5.700238490686e+13 ||r(i)||/||b|| 1.082672753854e-03 24 KSP preconditioned resid norm 3.368796010179e+14 true resid norm 6.388426371627e+13 ||r(i)||/||b|| 1.213383472264e-03 0 KSP preconditioned resid norm 9.591788600089e+15 true resid norm 5.932754803820e+15 ||r(i)||/||b|| 5.703607413375e-01 1 KSP preconditioned resid norm 4.627577110935e+15 true resid norm 8.119149369849e+14 ||r(i)||/||b|| 7.805554429176e-02 2 KSP preconditioned resid norm 3.843546862524e+15 true resid norm 6.215691855461e+14 ||r(i)||/||b|| 5.975616272433e-02 3 KSP preconditioned resid norm 3.802220665831e+15 true resid norm 6.332285028379e+14 ||r(i)||/||b|| 6.087706137495e-02 4 KSP preconditioned resid norm 4.025305304206e+15 true resid norm 6.486950645643e+14 ||r(i)||/||b|| 6.236397932519e-02 5 KSP preconditioned resid norm 4.519185013022e+15 true resid norm 7.100155628952e+14 ||r(i)||/||b|| 6.825918417416e-02 6 KSP preconditioned resid norm 5.566187938033e+15 true resid norm 8.272878114077e+14 ||r(i)||/||b|| 7.953345537054e-02 7 KSP preconditioned resid norm 7.046115999320e+15 true resid norm 1.260178524183e+15 ||r(i)||/||b|| 1.211505246783e-01 8 KSP preconditioned resid norm 7.802424274220e+15 true resid norm 1.227649475790e+15 ||r(i)||/||b|| 1.180232604022e-01 9 KSP preconditioned resid norm 8.076065684234e+15 true resid norm 1.496448814554e+15 ||r(i)||/||b|| 1.438649806820e-01 10 KSP preconditioned resid norm 7.812303545308e+15 true resid norm 1.236539634992e+15 ||r(i)||/||b|| 1.188779388714e-01 11 KSP preconditioned resid norm 6.692698687119e+15 true resid norm 1.122795851888e+15 ||r(i)||/||b|| 1.079428858312e-01 12 KSP preconditioned resid norm 4.016544769049e+15 true resid norm 6.096078457742e+14 ||r(i)||/||b|| 5.860622836074e-02 13 KSP preconditioned resid norm 1.711147574309e+15 true resid norm 2.966258404639e+14 ||r(i)||/||b|| 2.851689305581e-02 14 KSP preconditioned resid norm 1.304839987953e+15 true resid norm 2.700754536307e+14 ||r(i)||/||b|| 2.596440288593e-02 15 KSP preconditioned resid norm 1.171070442303e+15 true resid norm 2.199042966295e+14 ||r(i)||/||b|| 2.114106882828e-02 16 KSP preconditioned resid norm 1.057836577493e+15 true resid norm 1.660135728859e+14 ||r(i)||/||b|| 1.596014459292e-02 17 KSP preconditioned resid norm 8.905346433993e+14 true resid norm 1.613310037432e+14 ||r(i)||/||b|| 1.550997368650e-02 18 KSP preconditioned resid norm 7.864866777768e+14 true resid norm 1.268576094498e+14 ||r(i)||/||b|| 1.219578468395e-02 19 KSP preconditioned resid norm 6.743846682412e+14 true resid norm 1.066594941052e+14 ||r(i)||/||b|| 1.025398657793e-02 20 KSP preconditioned resid norm 5.775523980161e+14 true resid norm 1.005326356082e+14 ||r(i)||/||b|| 9.664965175573e-03 21 KSP preconditioned resid norm 4.897344092103e+14 true resid norm 7.691922473584e+13 ||r(i)||/||b|| 7.394828792721e-03 22 KSP preconditioned resid norm 4.026909639422e+14 true resid norm 7.643768412999e+13 ||r(i)||/||b|| 7.348534640001e-03 23 KSP preconditioned resid norm 3.340361168756e+14 true resid norm 5.217079213762e+13 ||r(i)||/||b|| 5.015574157998e-03 24 KSP preconditioned resid norm 2.772587883161e+14 true resid norm 6.149190087025e+13 ||r(i)||/||b|| 5.911683075799e-03 25 KSP preconditioned resid norm 2.446809593449e+14 true resid norm 3.772191549633e+13 ||r(i)||/||b|| 3.626493997916e-03 26 KSP preconditioned resid norm 2.130893101456e+14 true resid norm 5.402695231969e+13 ||r(i)||/||b|| 5.194020922190e-03 27 KSP preconditioned resid norm 1.886821885223e+14 true resid norm 2.887742158299e+13 ||r(i)||/||b|| 2.776205679591e-03 28 KSP preconditioned resid norm 1.654479962436e+14 true resid norm 4.199650238007e+13 ||r(i)||/||b|| 4.037442473714e-03 29 KSP preconditioned resid norm 1.453563006355e+14 true resid norm 2.222396036725e+13 ||r(i)||/||b|| 2.136557961634e-03 30 KSP preconditioned resid norm 1.311692747504e+14 true resid norm 2.933890714503e+13 ||r(i)||/||b|| 2.820571788757e-03 31 KSP preconditioned resid norm 1.173199137783e+14 true resid norm 1.793346595553e+13 ||r(i)||/||b|| 1.724080174451e-03 32 KSP preconditioned resid norm 1.086053135569e+14 true resid norm 2.009773461598e+13 ||r(i)||/||b|| 1.932147744820e-03 33 KSP preconditioned resid norm 9.929347161574e+13 true resid norm 1.516318812509e+13 ||r(i)||/||b|| 1.457752343733e-03 34 KSP preconditioned resid norm 9.235712158960e+13 true resid norm 1.538423395802e+13 ||r(i)||/||b|| 1.479003157109e-03 35 KSP preconditioned resid norm 8.718922541063e+13 true resid norm 1.331044977413e+13 ||r(i)||/||b|| 1.279634546134e-03 36 KSP preconditioned resid norm 8.163106494702e+13 true resid norm 1.369060193385e+13 ||r(i)||/||b|| 1.316181458118e-03 37 KSP preconditioned resid norm 7.751439049037e+13 true resid norm 1.182966275510e+13 ||r(i)||/||b|| 1.137275252708e-03 38 KSP preconditioned resid norm 7.269591005999e+13 true resid norm 1.225102361382e+13 ||r(i)||/||b|| 1.177783869649e-03 39 KSP preconditioned resid norm 6.809139852739e+13 true resid norm 1.039056641301e+13 ||r(i)||/||b|| 9.989240004362e-04 40 KSP preconditioned resid norm 6.390023598760e+13 true resid norm 1.042637257280e+13 ||r(i)||/||b|| 1.002366318301e-03 41 KSP preconditioned resid norm 5.919860950635e+13 true resid norm 9.048648521202e+12 ||r(i)||/||b|| 8.699152500509e-04 42 KSP preconditioned resid norm 5.538975217584e+13 true resid norm 8.775858022438e+12 ||r(i)||/||b|| 8.436898292725e-04 0 KSP preconditioned resid norm 2.200871087933e+16 true resid norm 1.375121498750e+16 ||r(i)||/||b|| 1.042688751858e-01 1 KSP preconditioned resid norm 1.033852082172e+16 true resid norm 1.769345381246e+15 ||r(i)||/||b|| 1.341609835098e-02 2 KSP preconditioned resid norm 8.201114088620e+15 true resid norm 1.330146487456e+15 ||r(i)||/||b|| 1.008586355500e-02 3 KSP preconditioned resid norm 7.727276364323e+15 true resid norm 1.292967289192e+15 ||r(i)||/||b|| 9.803951506731e-03 4 KSP preconditioned resid norm 7.788308934018e+15 true resid norm 1.241309862269e+15 ||r(i)||/||b|| 9.412257986909e-03 5 KSP preconditioned resid norm 8.253685478294e+15 true resid norm 1.290166199902e+15 ||r(i)||/||b|| 9.782712188615e-03 6 KSP preconditioned resid norm 9.476714658211e+15 true resid norm 1.410015893632e+15 ||r(i)||/||b|| 1.069147499743e-02 7 KSP preconditioned resid norm 1.110541369388e+16 true resid norm 1.990482855418e+15 ||r(i)||/||b|| 1.509287787295e-02 8 KSP preconditioned resid norm 1.149682905149e+16 true resid norm 1.789313061240e+15 ||r(i)||/||b|| 1.356750370206e-02 9 KSP preconditioned resid norm 1.081618830942e+16 true resid norm 1.972959789140e+15 ||r(i)||/||b|| 1.496000885648e-02 10 KSP preconditioned resid norm 8.800226542664e+15 true resid norm 1.343393349540e+15 ||r(i)||/||b|| 1.018630816375e-02 11 KSP preconditioned resid norm 4.142701353678e+15 true resid norm 6.194874024708e+14 ||r(i)||/||b|| 4.697276182950e-03 12 KSP preconditioned resid norm 3.300703256438e+15 true resid norm 1.133601440349e+15 ||r(i)||/||b|| 8.595556625482e-03 13 KSP preconditioned resid norm 7.882205618989e+14 true resid norm 1.186911081255e+14 ||r(i)||/||b|| 8.999778092374e-04 0 KSP preconditioned resid norm 1.928554431739e+16 true resid norm 1.203381747345e+16 ||r(i)||/||b|| 4.946634802067e-01 1 KSP preconditioned resid norm 9.232243648078e+15 true resid norm 1.583125971956e+15 ||r(i)||/||b|| 6.507615763833e-02 2 KSP preconditioned resid norm 7.541746545280e+15 true resid norm 1.216565269124e+15 ||r(i)||/||b|| 5.000827137780e-02 3 KSP preconditioned resid norm 7.340199256082e+15 true resid norm 1.219811880454e+15 ||r(i)||/||b|| 5.014172695523e-02 4 KSP preconditioned resid norm 7.655426776880e+15 true resid norm 1.212245393966e+15 ||r(i)||/||b|| 4.983069809450e-02 5 KSP preconditioned resid norm 8.414958315915e+15 true resid norm 1.311448634719e+15 ||r(i)||/||b|| 5.390855787813e-02 6 KSP preconditioned resid norm 1.008678778264e+16 true resid norm 1.495852838926e+15 ||r(i)||/||b|| 6.148869823002e-02 7 KSP preconditioned resid norm 1.235685231672e+16 true resid norm 2.203374571492e+15 ||r(i)||/||b|| 9.057216765485e-02 8 KSP preconditioned resid norm 1.311846317918e+16 true resid norm 2.061677565336e+15 ||r(i)||/||b|| 8.474755428051e-02 9 KSP preconditioned resid norm 1.289737803434e+16 true resid norm 2.355893652982e+15 ||r(i)||/||b|| 9.684163449812e-02 10 KSP preconditioned resid norm 1.151803850475e+16 true resid norm 1.821777204240e+15 ||r(i)||/||b|| 7.488618254339e-02 11 KSP preconditioned resid norm 8.380110732685e+15 true resid norm 1.434954901150e+15 ||r(i)||/||b|| 5.898542062058e-02 12 KSP preconditioned resid norm 3.553409074636e+15 true resid norm 5.423523740165e+14 ||r(i)||/||b|| 2.229399884295e-02 13 KSP preconditioned resid norm 1.190458477552e+15 true resid norm 2.162492452839e+14 ||r(i)||/||b|| 8.889166260017e-03 14 KSP preconditioned resid norm 9.902014249371e+14 true resid norm 1.847474743657e+14 ||r(i)||/||b|| 7.594250854373e-03 15 KSP preconditioned resid norm 9.211725470007e+14 true resid norm 1.442747306714e+14 ||r(i)||/||b|| 5.930573613672e-03 16 KSP preconditioned resid norm 8.405580663519e+14 true resid norm 2.201247255509e+14 ||r(i)||/||b|| 9.048472196027e-03 17 KSP preconditioned resid norm 7.117085749555e+14 true resid norm 1.108024781635e+14 ||r(i)||/||b|| 4.554659365975e-03 18 KSP preconditioned resid norm 6.347627750827e+14 true resid norm 1.457120949776e+14 ||r(i)||/||b|| 5.989658075575e-03 19 KSP preconditioned resid norm 5.626851770343e+14 true resid norm 8.750418282909e+13 ||r(i)||/||b|| 3.596956967843e-03 20 KSP preconditioned resid norm 4.865367652981e+14 true resid norm 1.207991751614e+14 ||r(i)||/||b|| 4.965584738448e-03 21 KSP preconditioned resid norm 4.252070053539e+14 true resid norm 6.577223220744e+13 ||r(i)||/||b|| 2.703640914986e-03 22 KSP preconditioned resid norm 3.618870286814e+14 true resid norm 8.993615915309e+13 ||r(i)||/||b|| 3.696926065336e-03 23 KSP preconditioned resid norm 3.105522471772e+14 true resid norm 4.771648977942e+13 ||r(i)||/||b|| 1.961439497451e-03 24 KSP preconditioned resid norm 2.735005931489e+14 true resid norm 4.708110884633e+13 ||r(i)||/||b|| 1.935321455997e-03 25 KSP preconditioned resid norm 2.496548247596e+14 true resid norm 3.887274137414e+13 ||r(i)||/||b|| 1.597907361960e-03 26 KSP preconditioned resid norm 2.329765393701e+14 true resid norm 3.903456593754e+13 ||r(i)||/||b|| 1.604559341009e-03 27 KSP preconditioned resid norm 2.169265400605e+14 true resid norm 3.334703074182e+13 ||r(i)||/||b|| 1.370766867430e-03 28 KSP preconditioned resid norm 1.998370208104e+14 true resid norm 3.474062054769e+13 ||r(i)||/||b|| 1.428051929704e-03 29 KSP preconditioned resid norm 1.805920498739e+14 true resid norm 2.773007978134e+13 ||r(i)||/||b|| 1.139875837515e-03 30 KSP preconditioned resid norm 1.643922523062e+14 true resid norm 2.817409671160e+13 ||r(i)||/||b|| 1.158127648337e-03 31 KSP preconditioned resid norm 1.456868079593e+14 true resid norm 2.236751337771e+13 ||r(i)||/||b|| 9.194415683468e-04 32 KSP preconditioned resid norm 1.315816507773e+14 true resid norm 2.157979387266e+13 ||r(i)||/||b|| 8.870614800950e-04 0 KSP preconditioned resid norm 6.130010040591e+16 true resid norm 4.019535303993e+16 ||r(i)||/||b|| 5.748506002867e-02 1 KSP preconditioned resid norm 2.383231267167e+16 true resid norm 4.333028232365e+15 ||r(i)||/||b|| 6.196845386481e-03 2 KSP preconditioned resid norm 1.380314332389e+16 true resid norm 2.902938687246e+15 ||r(i)||/||b|| 4.151614355275e-03 3 KSP preconditioned resid norm 9.355423030603e+15 true resid norm 1.914374573200e+15 ||r(i)||/||b|| 2.737827359010e-03 4 KSP preconditioned resid norm 6.905389736952e+15 true resid norm 1.160495154173e+15 ||r(i)||/||b|| 1.659672786910e-03 5 KSP preconditioned resid norm 5.388850338707e+15 true resid norm 9.343171842551e+14 ||r(i)||/||b|| 1.336206187053e-03 6 KSP preconditioned resid norm 4.456941671648e+15 true resid norm 7.443205431054e+14 ||r(i)||/||b|| 1.064484022780e-03 7 KSP preconditioned resid norm 3.604897929141e+15 true resid norm 7.177355376747e+14 ||r(i)||/||b|| 1.026463691635e-03 8 KSP preconditioned resid norm 2.616552572944e+15 true resid norm 4.566784838372e+14 ||r(i)||/||b|| 6.531150511630e-04 9 KSP preconditioned resid norm 1.730753896972e+15 true resid norm 3.416577764278e+14 ||r(i)||/||b|| 4.886191139484e-04 10 KSP preconditioned resid norm 9.605912905151e+14 true resid norm 1.715828170206e+14 ||r(i)||/||b|| 2.453877821777e-04 11 KSP preconditioned resid norm 3.885469679759e+14 true resid norm 7.526615556743e+13 ||r(i)||/||b|| 1.076412854646e-04 12 KSP preconditioned resid norm 1.085402187481e+14 true resid norm 1.877491024324e+13 ||r(i)||/||b|| 2.685078649001e-05 13 KSP preconditioned resid norm 5.521537327924e+13 true resid norm 1.482832280216e+13 ||r(i)||/||b|| 2.120660628506e-05 14 KSP preconditioned resid norm 4.327780981098e+13 true resid norm 6.949078996691e+12 ||r(i)||/||b|| 9.938169292156e-06 0 KSP preconditioned resid norm 2.469228993807e+16 true resid norm 1.581397706747e+16 ||r(i)||/||b|| 1.421322431696e-01 1 KSP preconditioned resid norm 1.094866250702e+16 true resid norm 1.827538407052e+15 ||r(i)||/||b|| 1.642547805430e-02 2 KSP preconditioned resid norm 7.902646639796e+15 true resid norm 1.331972302022e+15 ||r(i)||/||b|| 1.197144844200e-02 3 KSP preconditioned resid norm 6.757113896005e+15 true resid norm 1.177404713092e+15 ||r(i)||/||b|| 1.058223192536e-02 4 KSP preconditioned resid norm 6.236819639281e+15 true resid norm 9.858933018833e+14 ||r(i)||/||b|| 8.860973171059e-03 5 KSP preconditioned resid norm 6.056207345085e+15 true resid norm 9.498733496025e+14 ||r(i)||/||b|| 8.537234455953e-03 6 KSP preconditioned resid norm 6.281149615757e+15 true resid norm 9.439016000829e+14 ||r(i)||/||b|| 8.483561799717e-03 7 KSP preconditioned resid norm 6.511636840454e+15 true resid norm 1.175604965975e+15 ||r(i)||/||b|| 1.056605622877e-02 8 KSP preconditioned resid norm 5.897282381771e+15 true resid norm 9.358325816295e+14 ||r(i)||/||b|| 8.411039391972e-03 9 KSP preconditioned resid norm 4.858643828149e+15 true resid norm 9.074074922507e+14 ||r(i)||/||b|| 8.155561487934e-03 10 KSP preconditioned resid norm 3.431226959566e+15 true resid norm 5.523076022464e+14 ||r(i)||/||b|| 4.964008616682e-03 11 KSP preconditioned resid norm 1.790601246330e+15 true resid norm 3.233761759516e+14 ||r(i)||/||b|| 2.906427717678e-03 12 KSP preconditioned resid norm 6.061504971272e+14 true resid norm 9.361691978333e+13 ||r(i)||/||b|| 8.414064817893e-04 13 KSP preconditioned resid norm 3.669164519745e+14 true resid norm 7.433339410578e+13 ||r(i)||/||b|| 6.680907656304e-04 14 KSP preconditioned resid norm 1.028448839513e+15 true resid norm 1.739193394480e+14 ||r(i)||/||b|| 1.563145421348e-03 15 KSP preconditioned resid norm 1.340577188948e+14 true resid norm 2.477633902186e+13 ||r(i)||/||b|| 2.226838086134e-04 16 KSP preconditioned resid norm 1.124181660974e+14 true resid norm 1.917872742131e+13 ||r(i)||/||b|| 1.723738144997e-04 17 KSP preconditioned resid norm 9.394070158671e+13 true resid norm 1.682505899954e+13 ||r(i)||/||b|| 1.512196057236e-04 18 KSP preconditioned resid norm 9.034650415597e+13 true resid norm 1.892962426081e+13 ||r(i)||/||b|| 1.701349348787e-04 19 KSP preconditioned resid norm 8.058877605551e+13 true resid norm 1.275341142002e+13 ||r(i)||/||b|| 1.146246112195e-04 20 KSP preconditioned resid norm 6.303186171389e+13 true resid norm 1.900109222650e+13 ||r(i)||/||b|| 1.707772718592e-04 21 KSP preconditioned resid norm 4.601610061053e+13 true resid norm 6.959914424356e+12 ||r(i)||/||b|| 6.255404603042e-05 22 KSP preconditioned resid norm 2.822582332621e+13 true resid norm 1.013252988889e+13 ||r(i)||/||b|| 9.106875493415e-05 23 KSP preconditioned resid norm 1.858033653578e+13 true resid norm 2.785463510953e+12 ||r(i)||/||b|| 2.503507975190e-05 24 KSP preconditioned resid norm 9.505670647827e+12 true resid norm 4.268589348463e+12 ||r(i)||/||b|| 3.836506001486e-05 25 KSP preconditioned resid norm 5.844140386668e+12 true resid norm 8.921656809030e+11 ||r(i)||/||b|| 8.018571733391e-06 0 SNES Function norm 0.000000000000e+00 3 TS dt 5e-12 time 9e-12 0 KSP preconditioned resid norm 2.220122692212e+19 true resid norm 1.204173138214e+19 ||r(i)||/||b|| 2.302144004449e+02 1 KSP preconditioned resid norm 5.389791318038e+18 true resid norm 1.775764523088e+18 ||r(i)||/||b|| 3.394915166605e+01 2 KSP preconditioned resid norm 2.218229032455e+18 true resid norm 5.319501602511e+17 ||r(i)||/||b|| 1.016984878025e+01 3 KSP preconditioned resid norm 1.401621992174e+18 true resid norm 2.963396938661e+17 ||r(i)||/||b|| 5.665436537857e+00 4 KSP preconditioned resid norm 1.004007060031e+18 true resid norm 1.656144361093e+17 ||r(i)||/||b|| 3.166224764861e+00 5 KSP preconditioned resid norm 8.672273682794e+17 true resid norm 1.510259905938e+17 ||r(i)||/||b|| 2.887322160975e+00 6 KSP preconditioned resid norm 7.388939161376e+17 true resid norm 1.526077665500e+17 ||r(i)||/||b|| 2.917562629878e+00 7 KSP preconditioned resid norm 5.565893055963e+17 true resid norm 1.017298978392e+17 ||r(i)||/||b|| 1.944877085792e+00 8 KSP preconditioned resid norm 3.566491384538e+17 true resid norm 6.306517676931e+16 ||r(i)||/||b|| 1.205683086440e+00 9 KSP preconditioned resid norm 1.893313432840e+17 true resid norm 3.267997700517e+16 ||r(i)||/||b|| 6.247773741206e-01 10 KSP preconditioned resid norm 6.895898463878e+16 true resid norm 1.153123828824e+16 ||r(i)||/||b|| 2.204547688925e-01 11 KSP preconditioned resid norm 5.104999855255e+16 true resid norm 9.275473058095e+15 ||r(i)||/||b|| 1.773289405940e-01 12 KSP preconditioned resid norm 4.992747088606e+16 true resid norm 8.399049844320e+15 ||r(i)||/||b|| 1.605734393881e-01 13 KSP preconditioned resid norm 4.920015136673e+16 true resid norm 8.426577623649e+15 ||r(i)||/||b|| 1.610997167989e-01 14 KSP preconditioned resid norm 5.688370526613e+16 true resid norm 9.763030126011e+15 ||r(i)||/||b|| 1.866500800972e-01 15 KSP preconditioned resid norm 1.613946025228e+17 true resid norm 4.341380499600e+16 ||r(i)||/||b|| 8.299872145473e-01 16 KSP preconditioned resid norm 4.125620543313e+16 true resid norm 7.099475531667e+15 ||r(i)||/||b|| 1.357281150965e-01 17 KSP preconditioned resid norm 2.163115342752e+16 true resid norm 5.850497008723e+15 ||r(i)||/||b|| 1.118500835491e-01 18 KSP preconditioned resid norm 1.414953599369e+16 true resid norm 2.432122296311e+15 ||r(i)||/||b|| 4.649743118207e-02 19 KSP preconditioned resid norm 1.088056550362e+16 true resid norm 2.488538030816e+15 ||r(i)||/||b|| 4.757598991109e-02 20 KSP preconditioned resid norm 9.058853458567e+15 true resid norm 1.587978910975e+15 ||r(i)||/||b|| 3.035905729067e-02 21 KSP preconditioned resid norm 7.813748136791e+15 true resid norm 1.345999130102e+15 ||r(i)||/||b|| 2.573287618717e-02 22 KSP preconditioned resid norm 6.941551759120e+15 true resid norm 1.184314230023e+15 ||r(i)||/||b|| 2.264177648138e-02 23 KSP preconditioned resid norm 6.244923673699e+15 true resid norm 1.023439246806e+15 ||r(i)||/||b|| 1.956616080515e-02 24 KSP preconditioned resid norm 5.681880462174e+15 true resid norm 9.491910634069e+14 ||r(i)||/||b|| 1.814668045943e-02 25 KSP preconditioned resid norm 5.283519017995e+15 true resid norm 8.557005279630e+14 ||r(i)||/||b|| 1.635932390068e-02 26 KSP preconditioned resid norm 5.117675200632e+15 true resid norm 8.569254569155e+14 ||r(i)||/||b|| 1.638274215138e-02 27 KSP preconditioned resid norm 4.944059018035e+15 true resid norm 9.547778379458e+14 ||r(i)||/||b|| 1.825348868410e-02 28 KSP preconditioned resid norm 4.840428844487e+15 true resid norm 7.569152804275e+14 ||r(i)||/||b|| 1.447074278120e-02 29 KSP preconditioned resid norm 4.909458338739e+15 true resid norm 7.780937746322e+14 ||r(i)||/||b|| 1.487563425328e-02 30 KSP preconditioned resid norm 5.172548892535e+15 true resid norm 8.119476371329e+14 ||r(i)||/||b|| 1.552285402683e-02 31 KSP preconditioned resid norm 5.734458283292e+15 true resid norm 9.082595641858e+14 ||r(i)||/||b|| 1.736415008622e-02 32 KSP preconditioned resid norm 8.602942487649e+15 true resid norm 1.383459272467e+15 ||r(i)||/||b|| 2.644904099284e-02 33 KSP preconditioned resid norm 1.681001901654e+16 true resid norm 2.705594423024e+15 ||r(i)||/||b|| 5.172568446988e-02 34 KSP preconditioned resid norm 3.498037913905e+16 true resid norm 5.724738344713e+15 ||r(i)||/||b|| 1.094458233545e-01 35 KSP preconditioned resid norm 6.660621055336e+15 true resid norm 1.092449655644e+15 ||r(i)||/||b|| 2.088550512457e-02 36 KSP preconditioned resid norm 3.564205941873e+15 true resid norm 5.832825274169e+14 ||r(i)||/||b|| 1.115122344769e-02 37 KSP preconditioned resid norm 2.275128811690e+15 true resid norm 4.150874600928e+14 ||r(i)||/||b|| 7.935662050992e-03 38 KSP preconditioned resid norm 1.923021972098e+15 true resid norm 3.105449109783e+14 ||r(i)||/||b|| 5.937012562674e-03 39 KSP preconditioned resid norm 1.756400018558e+15 true resid norm 3.028037938881e+14 ||r(i)||/||b|| 5.789017513361e-03 40 KSP preconditioned resid norm 1.618977002834e+15 true resid norm 2.591473194248e+14 ||r(i)||/||b|| 4.954390932253e-03 41 KSP preconditioned resid norm 1.449378745707e+15 true resid norm 2.366294470672e+14 ||r(i)||/||b|| 4.523893164149e-03 42 KSP preconditioned resid norm 1.311727507569e+15 true resid norm 2.162961405943e+14 ||r(i)||/||b|| 4.135160031831e-03 43 KSP preconditioned resid norm 1.220101213287e+15 true resid norm 1.960093358179e+14 ||r(i)||/||b|| 3.747315921186e-03 44 KSP preconditioned resid norm 1.141924730669e+15 true resid norm 1.884563341884e+14 ||r(i)||/||b|| 3.602917272310e-03 45 KSP preconditioned resid norm 1.062154013672e+15 true resid norm 2.268601642641e+14 ||r(i)||/||b|| 4.337123545072e-03 46 KSP preconditioned resid norm 1.050847341809e+15 true resid norm 1.644035317108e+14 ||r(i)||/||b|| 3.143074636258e-03 47 KSP preconditioned resid norm 8.268920862881e+14 true resid norm 2.491484388741e+14 ||r(i)||/||b|| 4.763231852378e-03 48 KSP preconditioned resid norm 6.970056044801e+14 true resid norm 1.086553600705e+14 ||r(i)||/||b|| 2.077278406232e-03 49 KSP preconditioned resid norm 8.789823309588e+14 true resid norm 1.565701303869e+14 ||r(i)||/||b|| 2.993315292522e-03 50 KSP preconditioned resid norm 9.889574125697e+14 true resid norm 1.557528361421e+14 ||r(i)||/||b|| 2.977690221792e-03 51 KSP preconditioned resid norm 3.685341349551e+14 true resid norm 5.749337875960e+13 ||r(i)||/||b|| 1.099161183775e-03 52 KSP preconditioned resid norm 2.728364862859e+15 true resid norm 4.745949443528e+14 ||r(i)||/||b|| 9.073329000715e-03 53 KSP preconditioned resid norm 8.484691724076e+15 true resid norm 1.333040899485e+15 ||r(i)||/||b|| 2.548514011022e-02 54 KSP preconditioned resid norm 8.341541248747e+13 true resid norm 1.303089350822e+13 ||r(i)||/||b|| 2.491252496055e-04 0 KSP preconditioned resid norm 6.542890621126e+18 true resid norm 3.545938224251e+18 ||r(i)||/||b|| 4.156801777589e+02 1 KSP preconditioned resid norm 1.591173149842e+18 true resid norm 5.223204285160e+17 ||r(i)||/||b|| 6.123012721646e+01 2 KSP preconditioned resid norm 6.570590492388e+17 true resid norm 1.552517142001e+17 ||r(i)||/||b|| 1.819971360886e+01 3 KSP preconditioned resid norm 4.165858181866e+17 true resid norm 8.738575659546e+16 ||r(i)||/||b|| 1.024398185698e+01 4 KSP preconditioned resid norm 2.995041215234e+17 true resid norm 4.934730887237e+16 ||r(i)||/||b|| 5.784843622966e+00 5 KSP preconditioned resid norm 2.594520327186e+17 true resid norm 4.469839192099e+16 ||r(i)||/||b|| 5.239864409419e+00 6 KSP preconditioned resid norm 2.219632225075e+17 true resid norm 4.652171577038e+16 ||r(i)||/||b|| 5.453607439866e+00 7 KSP preconditioned resid norm 1.674928029034e+17 true resid norm 2.988841295071e+16 ||r(i)||/||b|| 3.503733010156e+00 8 KSP preconditioned resid norm 1.081911414020e+17 true resid norm 1.908165723430e+16 ||r(i)||/||b|| 2.236888002402e+00 9 KSP preconditioned resid norm 5.710171826154e+16 true resid norm 9.635278906551e+15 ||r(i)||/||b|| 1.129516137997e+00 10 KSP preconditioned resid norm 1.779377501866e+16 true resid norm 2.952494296810e+15 ||r(i)||/||b|| 3.461124465555e-01 11 KSP preconditioned resid norm 1.125612460139e+16 true resid norm 2.030403352863e+15 ||r(i)||/||b|| 2.380183672880e-01 12 KSP preconditioned resid norm 1.086634567103e+16 true resid norm 1.877885131752e+15 ||r(i)||/||b|| 2.201390932416e-01 13 KSP preconditioned resid norm 1.075395224737e+16 true resid norm 1.822688133743e+15 ||r(i)||/||b|| 2.136685073225e-01 14 KSP preconditioned resid norm 1.173925328196e+16 true resid norm 2.130469004546e+15 ||r(i)||/||b|| 2.497487769141e-01 15 KSP preconditioned resid norm 4.944016549258e+16 true resid norm 1.361939461929e+16 ||r(i)||/||b|| 1.596562607209e+00 16 KSP preconditioned resid norm 1.780247590037e+17 true resid norm 3.370503336823e+16 ||r(i)||/||b|| 3.951144485840e+00 17 KSP preconditioned resid norm 3.251104571145e+16 true resid norm 8.719659549123e+15 ||r(i)||/||b|| 1.022180704274e+00 18 KSP preconditioned resid norm 2.075437272594e+16 true resid norm 3.944434594682e+15 ||r(i)||/||b|| 4.623947654424e-01 19 KSP preconditioned resid norm 1.483444135710e+16 true resid norm 3.390829411015e+15 ||r(i)||/||b|| 3.974972160206e-01 20 KSP preconditioned resid norm 1.134313319987e+16 true resid norm 2.489467119154e+15 ||r(i)||/||b|| 2.918330972428e-01 21 KSP preconditioned resid norm 8.553477032620e+15 true resid norm 1.657674168535e+15 ||r(i)||/||b|| 1.943243930000e-01 22 KSP preconditioned resid norm 6.586595023874e+15 true resid norm 1.286251502369e+15 ||r(i)||/||b|| 1.507835780925e-01 23 KSP preconditioned resid norm 4.780400299659e+15 true resid norm 9.064518348736e+14 ||r(i)||/||b|| 1.062607513219e-01 24 KSP preconditioned resid norm 3.028073960021e+15 true resid norm 6.179142836679e+14 ||r(i)||/||b|| 7.243632094834e-02 25 KSP preconditioned resid norm 1.570529076731e+15 true resid norm 2.755631911412e+14 ||r(i)||/||b|| 3.230348331903e-02 26 KSP preconditioned resid norm 1.180384548694e+15 true resid norm 2.278461878994e+14 ||r(i)||/||b|| 2.670975575377e-02 27 KSP preconditioned resid norm 1.075603583388e+15 true resid norm 2.036512758236e+14 ||r(i)||/||b|| 2.387345553745e-02 28 KSP preconditioned resid norm 1.013485206919e+15 true resid norm 1.671016982388e+14 ||r(i)||/||b|| 1.958885328364e-02 29 KSP preconditioned resid norm 9.469966879519e+14 true resid norm 1.801244790801e+14 ||r(i)||/||b|| 2.111547656715e-02 30 KSP preconditioned resid norm 8.991813841494e+14 true resid norm 1.459100485336e+14 ||r(i)||/||b|| 1.710461690970e-02 31 KSP preconditioned resid norm 8.677534755734e+14 true resid norm 1.581142867341e+14 ||r(i)||/||b|| 1.853528478482e-02 32 KSP preconditioned resid norm 8.336211910215e+14 true resid norm 1.336646371267e+14 ||r(i)||/||b|| 1.566912241756e-02 33 KSP preconditioned resid norm 8.025169227655e+14 true resid norm 1.586001394574e+14 ||r(i)||/||b|| 1.859223990744e-02 34 KSP preconditioned resid norm 7.724033526395e+14 true resid norm 1.340849232530e+14 ||r(i)||/||b|| 1.571839135589e-02 35 KSP preconditioned resid norm 7.664262315912e+14 true resid norm 1.204381005271e+14 ||r(i)||/||b|| 1.411861343033e-02 36 KSP preconditioned resid norm 7.727266103384e+14 true resid norm 1.292430217432e+14 ||r(i)||/||b|| 1.515078911553e-02 37 KSP preconditioned resid norm 8.047214262262e+14 true resid norm 1.267167533285e+14 ||r(i)||/||b|| 1.485464190786e-02 38 KSP preconditioned resid norm 8.419506152984e+14 true resid norm 1.380565821207e+14 ||r(i)||/||b|| 1.618397754485e-02 39 KSP preconditioned resid norm 8.796981754885e+14 true resid norm 1.388509364066e+14 ||r(i)||/||b|| 1.627709742171e-02 40 KSP preconditioned resid norm 9.432799067464e+14 true resid norm 1.525727137977e+14 ||r(i)||/||b|| 1.788566206789e-02 41 KSP preconditioned resid norm 1.193594717717e+15 true resid norm 1.887759239390e+14 ||r(i)||/||b|| 2.212966065874e-02 42 KSP preconditioned resid norm 2.117323854280e+15 true resid norm 3.659661973675e+14 ||r(i)||/||b|| 4.290116870483e-02 43 KSP preconditioned resid norm 1.749515464589e+16 true resid norm 2.774904343458e+15 ||r(i)||/||b|| 3.252940851772e-01 44 KSP preconditioned resid norm 1.219803820486e+15 true resid norm 2.038867414461e+14 ||r(i)||/||b|| 2.390105849768e-02 45 KSP preconditioned resid norm 6.218668840796e+14 true resid norm 9.918031040094e+13 ||r(i)||/||b|| 1.162662360435e-02 46 KSP preconditioned resid norm 4.213016096566e+14 true resid norm 7.032265378826e+13 ||r(i)||/||b|| 8.243723206248e-03 47 KSP preconditioned resid norm 2.932264460599e+14 true resid norm 4.681093096736e+13 ||r(i)||/||b|| 5.487511308712e-03 48 KSP preconditioned resid norm 1.890454719223e+14 true resid norm 3.330632990756e+13 ||r(i)||/||b|| 3.904405621560e-03 49 KSP preconditioned resid norm 1.361587907879e+14 true resid norm 2.180429132446e+13 ||r(i)||/||b|| 2.556054595557e-03 50 KSP preconditioned resid norm 1.041049830937e+14 true resid norm 1.811134568183e+13 ||r(i)||/||b|| 2.123141159366e-03 51 KSP preconditioned resid norm 9.375298557199e+13 true resid norm 1.575768352161e+13 ||r(i)||/||b|| 1.847228088333e-03 52 KSP preconditioned resid norm 1.220175150775e+14 true resid norm 1.950548748269e+13 ||r(i)||/||b|| 2.286572408009e-03 53 KSP preconditioned resid norm 7.051895161295e+13 true resid norm 1.295324819768e+13 ||r(i)||/||b|| 1.518472170932e-03 54 KSP preconditioned resid norm 5.292501274648e+13 true resid norm 8.611444708372e+12 ||r(i)||/||b|| 1.009494988563e-03 0 KSP preconditioned resid norm 1.892610202067e+19 true resid norm 1.028308040523e+19 ||r(i)||/||b|| 7.841022917690e+01 1 KSP preconditioned resid norm 4.581169591238e+18 true resid norm 1.519782710159e+18 ||r(i)||/||b|| 1.158860048804e+01 2 KSP preconditioned resid norm 1.874457569158e+18 true resid norm 4.646597685037e+17 ||r(i)||/||b|| 3.543109409036e+00 3 KSP preconditioned resid norm 1.173257076173e+18 true resid norm 2.550995626245e+17 ||r(i)||/||b|| 1.945177357373e+00 4 KSP preconditioned resid norm 8.301427099842e+17 true resid norm 1.381385637907e+17 ||r(i)||/||b|| 1.053329937932e+00 5 KSP preconditioned resid norm 7.053087339556e+17 true resid norm 1.287851939525e+17 ||r(i)||/||b|| 9.820089092432e-01 6 KSP preconditioned resid norm 5.863912202870e+17 true resid norm 1.166954217660e+17 ||r(i)||/||b|| 8.898223493329e-01 7 KSP preconditioned resid norm 4.330693111087e+17 true resid norm 8.611440299479e+16 ||r(i)||/||b|| 6.566369033559e-01 8 KSP preconditioned resid norm 2.702458300388e+17 true resid norm 4.803465439349e+16 ||r(i)||/||b|| 3.662723727716e-01 9 KSP preconditioned resid norm 1.546904751775e+17 true resid norm 2.878502712633e+16 ||r(i)||/||b|| 2.194907055953e-01 10 KSP preconditioned resid norm 9.910545254967e+16 true resid norm 1.648892832929e+16 ||r(i)||/||b|| 1.257308703453e-01 11 KSP preconditioned resid norm 9.242232753221e+16 true resid norm 1.559261342084e+16 ||r(i)||/||b|| 1.188963174082e-01 12 KSP preconditioned resid norm 9.244969774564e+16 true resid norm 1.506990436290e+16 ||r(i)||/||b|| 1.149105723385e-01 13 KSP preconditioned resid norm 7.091380549056e+16 true resid norm 1.150476534699e+16 ||r(i)||/||b|| 8.772578370826e-02 14 KSP preconditioned resid norm 2.015813147523e+17 true resid norm 3.265267485752e+16 ||r(i)||/||b|| 2.489821743992e-01 15 KSP preconditioned resid norm 1.668693825394e+16 true resid norm 4.392881515438e+15 ||r(i)||/||b|| 3.349646533904e-02 16 KSP preconditioned resid norm 1.171106201192e+16 true resid norm 1.864120995445e+15 ||r(i)||/||b|| 1.421423821522e-02 17 KSP preconditioned resid norm 1.020170882678e+16 true resid norm 1.999245817269e+15 ||r(i)||/||b|| 1.524458786039e-02 18 KSP preconditioned resid norm 9.407404703690e+15 true resid norm 1.539877936168e+15 ||r(i)||/||b|| 1.174182998880e-02 19 KSP preconditioned resid norm 8.812299187434e+15 true resid norm 1.424937011002e+15 ||r(i)||/||b|| 1.086538597310e-02 20 KSP preconditioned resid norm 8.313368595823e+15 true resid norm 1.488773269648e+15 ||r(i)||/||b|| 1.135214825376e-02 21 KSP preconditioned resid norm 7.851626197782e+15 true resid norm 1.250085440746e+15 ||r(i)||/||b|| 9.532113144783e-03 22 KSP preconditioned resid norm 7.455044400782e+15 true resid norm 1.324254107888e+15 ||r(i)||/||b|| 1.009766178966e-02 23 KSP preconditioned resid norm 7.129687083603e+15 true resid norm 1.124533100660e+15 ||r(i)||/||b|| 8.574755293646e-03 24 KSP preconditioned resid norm 5.784281419699e+15 true resid norm 2.057123935555e+15 ||r(i)||/||b|| 1.568591831199e-02 25 KSP preconditioned resid norm 4.924869719264e+15 true resid norm 7.825081154476e+14 ||r(i)||/||b|| 5.966756871199e-03 26 KSP preconditioned resid norm 5.016674985414e+15 true resid norm 7.817733101871e+14 ||r(i)||/||b|| 5.961153856674e-03 27 KSP preconditioned resid norm 5.308634596254e+15 true resid norm 8.859615986698e+14 ||r(i)||/||b|| 6.755607197068e-03 28 KSP preconditioned resid norm 9.149615612203e+15 true resid norm 1.429126026168e+15 ||r(i)||/||b|| 1.089732792301e-02 29 KSP preconditioned resid norm 8.897313413659e+15 true resid norm 2.002400213933e+15 ||r(i)||/||b|| 1.526864066904e-02 30 KSP preconditioned resid norm 2.145204463192e+15 true resid norm 3.349465461999e+14 ||r(i)||/||b|| 2.554024126484e-03 31 KSP preconditioned resid norm 1.279996253934e+15 true resid norm 2.794251271078e+14 ||r(i)||/||b|| 2.130663905259e-03 32 KSP preconditioned resid norm 8.936440443654e+14 true resid norm 1.396095375555e+14 ||r(i)||/||b|| 1.064546361948e-03 0 KSP preconditioned resid norm 6.470811776559e+18 true resid norm 3.511555308471e+18 ||r(i)||/||b|| 1.662220839803e+02 1 KSP preconditioned resid norm 1.571601370240e+18 true resid norm 5.187483735196e+17 ||r(i)||/||b|| 2.455534033589e+01 2 KSP preconditioned resid norm 6.468356854753e+17 true resid norm 1.566993981909e+17 ||r(i)||/||b|| 7.417482635946e+00 3 KSP preconditioned resid norm 4.084058718904e+17 true resid norm 8.688424998267e+16 ||r(i)||/||b|| 4.112730636007e+00 4 KSP preconditioned resid norm 2.920904676538e+17 true resid norm 4.831974981420e+16 ||r(i)||/||b|| 2.287251319137e+00 5 KSP preconditioned resid norm 2.515876340477e+17 true resid norm 4.435582123557e+16 ||r(i)||/||b|| 2.099615809737e+00 6 KSP preconditioned resid norm 2.133089016156e+17 true resid norm 4.355659888860e+16 ||r(i)||/||b|| 2.061784025127e+00 7 KSP preconditioned resid norm 1.601295948028e+17 true resid norm 2.999342671961e+16 ||r(i)||/||b|| 1.419761176199e+00 8 KSP preconditioned resid norm 1.019894482623e+17 true resid norm 1.811123232414e+16 ||r(i)||/||b|| 8.573086612382e-01 9 KSP preconditioned resid norm 5.552358385594e+16 true resid norm 9.849609129144e+15 ||r(i)||/||b|| 4.662385786400e-01 10 KSP preconditioned resid norm 2.614302078211e+16 true resid norm 4.400897857400e+15 ||r(i)||/||b|| 2.083197754216e-01 11 KSP preconditioned resid norm 2.230412831734e+16 true resid norm 4.001989581448e+15 ||r(i)||/||b|| 1.894371552944e-01 12 KSP preconditioned resid norm 2.202852939300e+16 true resid norm 3.668964514289e+15 ||r(i)||/||b|| 1.736731658885e-01 13 KSP preconditioned resid norm 2.171413942501e+16 true resid norm 3.746648405904e+15 ||r(i)||/||b|| 1.773503907139e-01 14 KSP preconditioned resid norm 2.756569557917e+16 true resid norm 4.738274109991e+15 ||r(i)||/||b|| 2.242897314282e-01 15 KSP preconditioned resid norm 1.911125733606e+16 true resid norm 7.273426207009e+15 ||r(i)||/||b|| 3.442930427122e-01 16 KSP preconditioned resid norm 8.267301781208e+15 true resid norm 1.450502353903e+15 ||r(i)||/||b|| 6.866060844958e-02 17 KSP preconditioned resid norm 5.452175853074e+15 true resid norm 1.779140334910e+15 ||r(i)||/||b|| 8.421693186737e-02 18 KSP preconditioned resid norm 4.185422899260e+15 true resid norm 7.352001585133e+14 ||r(i)||/||b|| 3.480124667150e-02 19 KSP preconditioned resid norm 3.483100692842e+15 true resid norm 7.680069742951e+14 ||r(i)||/||b|| 3.635418171280e-02 20 KSP preconditioned resid norm 3.028709630445e+15 true resid norm 5.664408475792e+14 ||r(i)||/||b|| 2.681289909034e-02 21 KSP preconditioned resid norm 2.650986791756e+15 true resid norm 5.333477625647e+14 ||r(i)||/||b|| 2.524641328184e-02 22 KSP preconditioned resid norm 2.409785359425e+15 true resid norm 3.913611824173e+14 ||r(i)||/||b|| 1.852537283791e-02 23 KSP preconditioned resid norm 2.246473514462e+15 true resid norm 3.515886291592e+14 ||r(i)||/||b|| 1.664270942896e-02 24 KSP preconditioned resid norm 2.105216345845e+15 true resid norm 3.429537728013e+14 ||r(i)||/||b|| 1.623397207682e-02 25 KSP preconditioned resid norm 1.993254024352e+15 true resid norm 3.120073169264e+14 ||r(i)||/||b|| 1.476909855627e-02 26 KSP preconditioned resid norm 1.937764630771e+15 true resid norm 3.193060027359e+14 ||r(i)||/||b|| 1.511458728106e-02 27 KSP preconditioned resid norm 1.841409046328e+15 true resid norm 4.276359096161e+14 ||r(i)||/||b|| 2.024246404712e-02 28 KSP preconditioned resid norm 1.798600034298e+15 true resid norm 2.807262398909e+14 ||r(i)||/||b|| 1.328838549404e-02 29 KSP preconditioned resid norm 1.909156742948e+15 true resid norm 3.075705616539e+14 ||r(i)||/||b|| 1.455908144342e-02 30 KSP preconditioned resid norm 2.243056909373e+15 true resid norm 3.518787704137e+14 ||r(i)||/||b|| 1.665644348118e-02 31 KSP preconditioned resid norm 3.125791809917e+15 true resid norm 5.051797337710e+14 ||r(i)||/||b|| 2.391305867501e-02 32 KSP preconditioned resid norm 2.268406906040e+16 true resid norm 3.663605847107e+15 ||r(i)||/||b|| 1.734195093893e-01 33 KSP preconditioned resid norm 2.663631019961e+15 true resid norm 4.418031676456e+14 ||r(i)||/||b|| 2.091308174984e-02 34 KSP preconditioned resid norm 1.194136462714e+15 true resid norm 1.964734637439e+14 ||r(i)||/||b|| 9.300217630505e-03 35 KSP preconditioned resid norm 7.378290473645e+14 true resid norm 1.245828993721e+14 ||r(i)||/||b|| 5.897224261846e-03 36 KSP preconditioned resid norm 5.578220476287e+14 true resid norm 9.049566410806e+13 ||r(i)||/||b|| 4.283679611405e-03 37 KSP preconditioned resid norm 4.459209275003e+14 true resid norm 9.101333159944e+13 ||r(i)||/||b|| 4.308183787380e-03 38 KSP preconditioned resid norm 4.020379023591e+14 true resid norm 6.381912771158e+13 ||r(i)||/||b|| 3.020925907227e-03 39 KSP preconditioned resid norm 3.758207934261e+14 true resid norm 7.185355109637e+13 ||r(i)||/||b|| 3.401241317717e-03 40 KSP preconditioned resid norm 3.524000257362e+14 true resid norm 5.552480372659e+13 ||r(i)||/||b|| 2.628307908397e-03 41 KSP preconditioned resid norm 3.288683013967e+14 true resid norm 5.672396056313e+13 ||r(i)||/||b|| 2.685070889720e-03 42 KSP preconditioned resid norm 3.091159619430e+14 true resid norm 5.054650403677e+13 ||r(i)||/||b|| 2.392656387510e-03 43 KSP preconditioned resid norm 2.914764729338e+14 true resid norm 5.652869757159e+13 ||r(i)||/||b|| 2.675827970692e-03 44 KSP preconditioned resid norm 2.823730992718e+14 true resid norm 4.387571428023e+13 ||r(i)||/||b|| 2.076889589689e-03 45 KSP preconditioned resid norm 2.874969520743e+14 true resid norm 4.617969867525e+13 ||r(i)||/||b|| 2.185950405754e-03 46 KSP preconditioned resid norm 3.038236713170e+14 true resid norm 4.761130615995e+13 ||r(i)||/||b|| 2.253716611508e-03 47 KSP preconditioned resid norm 3.972150696557e+14 true resid norm 6.226543710285e+13 ||r(i)||/||b|| 2.947380805938e-03 48 KSP preconditioned resid norm 7.035007479461e+14 true resid norm 1.474495546291e+14 ||r(i)||/||b|| 6.979634406808e-03 49 KSP preconditioned resid norm 2.896572855920e+13 true resid norm 4.634266191547e+12 ||r(i)||/||b|| 2.193664392014e-04 0 KSP preconditioned resid norm 6.694681597421e+18 true resid norm 3.650754072308e+18 ||r(i)||/||b|| 5.229798618159e+00 1 KSP preconditioned resid norm 1.597667334547e+18 true resid norm 5.366749713724e+17 ||r(i)||/||b|| 7.688006280602e-01 2 KSP preconditioned resid norm 6.300446759218e+17 true resid norm 1.705328267289e+17 ||r(i)||/||b|| 2.442926376067e-01 3 KSP preconditioned resid norm 3.642174486076e+17 true resid norm 9.481576843120e+16 ||r(i)||/||b|| 1.358260142699e-01 4 KSP preconditioned resid norm 2.247332959633e+17 true resid norm 3.802800729913e+16 ||r(i)||/||b|| 5.447609345502e-02 5 KSP preconditioned resid norm 1.557523701694e+17 true resid norm 3.829141999854e+16 ||r(i)||/||b|| 5.485343888670e-02 6 KSP preconditioned resid norm 1.004692846458e+17 true resid norm 1.699728099753e+16 ||r(i)||/||b|| 2.434903992784e-02 7 KSP preconditioned resid norm 7.103482800944e+16 true resid norm 1.472858884839e+16 ||r(i)||/||b|| 2.109908037658e-02 8 KSP preconditioned resid norm 5.766847884006e+16 true resid norm 9.397954060610e+15 ||r(i)||/||b|| 1.346280965144e-02 9 KSP preconditioned resid norm 4.523472015836e+16 true resid norm 1.246374917128e+16 ||r(i)||/||b|| 1.785463959006e-02 10 KSP preconditioned resid norm 3.195342481549e+16 true resid norm 5.030521483345e+15 ||r(i)||/||b|| 7.206350737723e-03 11 KSP preconditioned resid norm 2.437460220100e+16 true resid norm 4.974645944970e+15 ||r(i)||/||b|| 7.126307599347e-03 12 KSP preconditioned resid norm 1.258045555912e+16 true resid norm 1.885910304740e+15 ||r(i)||/||b|| 2.701614765157e-03 13 KSP preconditioned resid norm 2.495567167662e+15 true resid norm 4.460991814847e+14 ||r(i)||/||b|| 6.390484915397e-04 14 KSP preconditioned resid norm 7.678096336284e+14 true resid norm 1.316276217008e+14 ||r(i)||/||b|| 1.885599359606e-04 15 KSP preconditioned resid norm 5.950867488217e+14 true resid norm 1.439410415068e+14 ||r(i)||/||b|| 2.061992248885e-04 16 KSP preconditioned resid norm 3.263035982098e+14 true resid norm 5.046543709851e+13 ||r(i)||/||b|| 7.229302987142e-05 17 KSP preconditioned resid norm 1.436782483869e+14 true resid norm 7.588970331259e+13 ||r(i)||/||b|| 1.087139417380e-04 18 KSP preconditioned resid norm 5.401533723431e+13 true resid norm 8.625318760812e+12 ||r(i)||/||b|| 1.235598981554e-05 19 KSP preconditioned resid norm 3.528859982338e+13 true resid norm 1.233587619696e+13 ||r(i)||/||b|| 1.767145828255e-05 0 KSP preconditioned resid norm 4.827874033276e+18 true resid norm 2.629049503582e+18 ||r(i)||/||b|| 2.387244449971e+01 1 KSP preconditioned resid norm 1.161059891938e+18 true resid norm 3.885117259710e+17 ||r(i)||/||b|| 3.527786222014e+00 2 KSP preconditioned resid norm 4.696931551640e+17 true resid norm 1.228954734511e+17 ||r(i)||/||b|| 1.115922452289e+00 3 KSP preconditioned resid norm 2.870101730651e+17 true resid norm 6.759381584873e+16 ||r(i)||/||b|| 6.137692025860e-01 4 KSP preconditioned resid norm 1.966639305477e+17 true resid norm 3.310590025777e+16 ||r(i)||/||b|| 3.006100742644e-01 5 KSP preconditioned resid norm 1.592574496560e+17 true resid norm 3.239415772489e+16 ||r(i)||/||b|| 2.941472693263e-01 6 KSP preconditioned resid norm 1.230114823548e+17 true resid norm 2.280737281236e+16 ||r(i)||/||b|| 2.070968009182e-01 7 KSP preconditioned resid norm 8.598640814452e+16 true resid norm 1.938029193987e+16 ||r(i)||/||b|| 1.759780266946e-01 8 KSP preconditioned resid norm 5.549085767183e+16 true resid norm 9.386094645880e+15 ||r(i)||/||b|| 8.522814925985e-02 9 KSP preconditioned resid norm 4.474754629313e+16 true resid norm 7.579167805290e+15 ||r(i)||/||b|| 6.882078961970e-02 10 KSP preconditioned resid norm 4.378389532275e+16 true resid norm 6.842593279419e+15 ||r(i)||/||b|| 6.213250380964e-02 11 KSP preconditioned resid norm 4.079498017991e+16 true resid norm 7.704710537522e+15 ||r(i)||/||b|| 6.996074986140e-02 12 KSP preconditioned resid norm 4.084398127844e+16 true resid norm 6.377957981626e+15 ||r(i)||/||b|| 5.791349601078e-02 13 KSP preconditioned resid norm 7.523960004025e+16 true resid norm 1.132262464871e+16 ||r(i)||/||b|| 1.028123388886e-01 14 KSP preconditioned resid norm 2.184546709977e+15 true resid norm 3.502101186681e+14 ||r(i)||/||b|| 3.179997793785e-03 15 KSP preconditioned resid norm 2.222188254453e+15 true resid norm 3.654675413503e+14 ||r(i)||/||b|| 3.318539109075e-03 16 KSP preconditioned resid norm 1.669487552266e+15 true resid norm 3.629534589137e+14 ||r(i)||/||b|| 3.295710595061e-03 17 KSP preconditioned resid norm 1.401917499993e+15 true resid norm 2.161165604483e+14 ||r(i)||/||b|| 1.962393856693e-03 18 KSP preconditioned resid norm 1.134393254100e+15 true resid norm 2.686682335159e+14 ||r(i)||/||b|| 2.439576540763e-03 19 KSP preconditioned resid norm 9.058009036885e+14 true resid norm 1.409928994873e+14 ||r(i)||/||b|| 1.280251727203e-03 20 KSP preconditioned resid norm 7.990098022926e+14 true resid norm 1.659150954933e+14 ||r(i)||/||b|| 1.506551665700e-03 21 KSP preconditioned resid norm 7.119179981764e+14 true resid norm 1.092130722300e+14 ||r(i)||/||b|| 9.916827362515e-04 22 KSP preconditioned resid norm 6.306959704581e+14 true resid norm 1.460783700778e+14 ||r(i)||/||b|| 1.326429105857e-03 23 KSP preconditioned resid norm 5.637228678118e+14 true resid norm 8.672640313846e+13 ||r(i)||/||b|| 7.874980074592e-04 24 KSP preconditioned resid norm 4.960976390096e+14 true resid norm 1.292271182132e+14 ||r(i)||/||b|| 1.173415412376e-03 25 KSP preconditioned resid norm 4.386404742743e+14 true resid norm 8.037964656859e+13 ||r(i)||/||b|| 7.298678282782e-04 26 KSP preconditioned resid norm 4.419824802921e+14 true resid norm 6.704823034514e+13 ||r(i)||/||b|| 6.088151461346e-04 27 KSP preconditioned resid norm 4.437076279615e+14 true resid norm 1.370394110315e+14 ||r(i)||/||b|| 1.244353036968e-03 28 KSP preconditioned resid norm 7.848825686766e+14 true resid norm 1.196984074237e+14 ||r(i)||/||b|| 1.086892271915e-03 29 KSP preconditioned resid norm 1.771839364121e+15 true resid norm 6.554952272410e+14 ||r(i)||/||b|| 5.952064961431e-03 30 KSP preconditioned resid norm 1.052090569223e+15 true resid norm 1.877636113350e+14 ||r(i)||/||b|| 1.704941799139e-03 31 KSP preconditioned resid norm 2.459236824172e+14 true resid norm 3.913333475140e+13 ||r(i)||/||b|| 3.553407270074e-04 32 KSP preconditioned resid norm 1.091606883884e+14 true resid norm 1.900051073024e+13 ||r(i)||/||b|| 1.725295158025e-04 33 KSP preconditioned resid norm 6.424114279947e+13 true resid norm 1.001416869814e+13 ||r(i)||/||b|| 9.093122291205e-05 34 KSP preconditioned resid norm 3.801391191576e+13 true resid norm 7.035993054761e+12 ||r(i)||/||b|| 6.388862342504e-05 35 KSP preconditioned resid norm 2.524482106623e+13 true resid norm 4.006294246762e+12 ||r(i)||/||b|| 3.637818037470e-05 36 KSP preconditioned resid norm 1.579134058219e+13 true resid norm 2.528643754315e+12 ||r(i)||/||b|| 2.296073451724e-05 37 KSP preconditioned resid norm 1.074218958170e+13 true resid norm 1.786180378482e+12 ||r(i)||/||b|| 1.621897643756e-05 38 KSP preconditioned resid norm 6.572401461924e+12 true resid norm 1.033900044668e+12 ||r(i)||/||b|| 9.388077858923e-06 0 KSP preconditioned resid norm 9.558506771869e+15 true resid norm 5.892974320929e+15 ||r(i)||/||b|| 1.119279921775e-01 1 KSP preconditioned resid norm 4.577878026258e+15 true resid norm 8.113551231445e+14 ||r(i)||/||b|| 1.541044384903e-02 2 KSP preconditioned resid norm 3.768474917795e+15 true resid norm 6.114238708763e+14 ||r(i)||/||b|| 1.161305692331e-02 3 KSP preconditioned resid norm 3.689006206393e+15 true resid norm 6.168086911615e+14 ||r(i)||/||b|| 1.171533331040e-02 4 KSP preconditioned resid norm 3.859762637542e+15 true resid norm 6.287723791867e+14 ||r(i)||/||b|| 1.194256518123e-02 5 KSP preconditioned resid norm 4.285185348381e+15 true resid norm 6.764146832146e+14 ||r(i)||/||b|| 1.284745753985e-02 6 KSP preconditioned resid norm 5.208557345650e+15 true resid norm 7.765898979336e+14 ||r(i)||/||b|| 1.475013181583e-02 7 KSP preconditioned resid norm 6.471756089495e+15 true resid norm 1.171552566673e+15 ||r(i)||/||b|| 2.225184081532e-02 8 KSP preconditioned resid norm 7.012039525053e+15 true resid norm 1.102268075574e+15 ||r(i)||/||b|| 2.093588836833e-02 9 KSP preconditioned resid norm 7.014970322733e+15 true resid norm 1.328280835862e+15 ||r(i)||/||b|| 2.522865346245e-02 10 KSP preconditioned resid norm 6.390016130531e+15 true resid norm 1.015385597333e+15 ||r(i)||/||b|| 1.928568919633e-02 11 KSP preconditioned resid norm 4.845316666400e+15 true resid norm 8.456481888760e+14 ||r(i)||/||b|| 1.606178793844e-02 12 KSP preconditioned resid norm 2.410560854094e+15 true resid norm 3.709640804249e+14 ||r(i)||/||b|| 7.045892690295e-03 13 KSP preconditioned resid norm 1.334157374345e+15 true resid norm 2.713196530598e+14 ||r(i)||/||b|| 5.153299904502e-03 14 KSP preconditioned resid norm 1.217530036393e+15 true resid norm 2.232136761716e+14 ||r(i)||/||b|| 4.239600792372e-03 15 KSP preconditioned resid norm 1.145671346103e+15 true resid norm 2.123551517116e+14 ||r(i)||/||b|| 4.033359805286e-03 16 KSP preconditioned resid norm 1.048828510236e+15 true resid norm 1.668978069044e+14 ||r(i)||/||b|| 3.169967389692e-03 17 KSP preconditioned resid norm 9.144204694073e+14 true resid norm 1.578839852075e+14 ||r(i)||/||b|| 2.998763697051e-03 18 KSP preconditioned resid norm 8.235850213860e+14 true resid norm 1.321988157805e+14 ||r(i)||/||b|| 2.510913371198e-03 19 KSP preconditioned resid norm 6.978607225935e+14 true resid norm 1.125789224246e+14 ||r(i)||/||b|| 2.138263644512e-03 20 KSP preconditioned resid norm 6.009340490988e+14 true resid norm 1.003011158758e+14 ||r(i)||/||b|| 1.905065574997e-03 21 KSP preconditioned resid norm 5.123703002158e+14 true resid norm 8.114653676231e+13 ||r(i)||/||b|| 1.541253777351e-03 22 KSP preconditioned resid norm 4.394549158605e+14 true resid norm 8.003106905102e+13 ||r(i)||/||b|| 1.520067182184e-03 23 KSP preconditioned resid norm 3.852402878098e+14 true resid norm 5.993269339949e+13 ||r(i)||/||b|| 1.138329419521e-03 24 KSP preconditioned resid norm 3.390788847804e+14 true resid norm 7.376626581336e+13 ||r(i)||/||b|| 1.401076871080e-03 0 KSP preconditioned resid norm 9.592017367891e+15 true resid norm 5.933298102242e+15 ||r(i)||/||b|| 5.704123038824e-01 1 KSP preconditioned resid norm 4.627662883687e+15 true resid norm 8.121175703537e+14 ||r(i)||/||b|| 7.807493342594e-02 2 KSP preconditioned resid norm 3.843678575472e+15 true resid norm 6.216594738645e+14 ||r(i)||/||b|| 5.976477274644e-02 3 KSP preconditioned resid norm 3.802359495548e+15 true resid norm 6.332926539714e+14 ||r(i)||/||b|| 6.088315731972e-02 4 KSP preconditioned resid norm 4.025235313693e+15 true resid norm 6.488898363947e+14 ||r(i)||/||b|| 6.238263107055e-02 5 KSP preconditioned resid norm 4.519136017232e+15 true resid norm 7.100489487455e+14 ||r(i)||/||b|| 6.826231376612e-02 6 KSP preconditioned resid norm 5.566459161126e+15 true resid norm 8.272746471009e+14 ||r(i)||/||b|| 7.953209652789e-02 7 KSP preconditioned resid norm 7.051037635374e+15 true resid norm 1.259719337020e+15 ||r(i)||/||b|| 1.211062375246e-01 8 KSP preconditioned resid norm 7.825859719364e+15 true resid norm 1.230378249193e+15 ||r(i)||/||b|| 1.182854593980e-01 9 KSP preconditioned resid norm 8.112136766042e+15 true resid norm 1.509007778424e+15 ||r(i)||/||b|| 1.450721990763e-01 10 KSP preconditioned resid norm 7.849971536540e+15 true resid norm 1.244897342905e+15 ||r(i)||/||b|| 1.196812884212e-01 11 KSP preconditioned resid norm 6.691547560102e+15 true resid norm 1.152431148742e+15 ||r(i)||/||b|| 1.107918218994e-01 12 KSP preconditioned resid norm 3.838210718255e+15 true resid norm 5.898239658450e+14 ||r(i)||/||b|| 5.670418735838e-02 13 KSP preconditioned resid norm 1.585805121666e+15 true resid norm 3.093588848689e+14 ||r(i)||/||b|| 2.974098236828e-02 14 KSP preconditioned resid norm 1.284246821472e+15 true resid norm 2.667115224243e+14 ||r(i)||/||b|| 2.564097258496e-02 15 KSP preconditioned resid norm 1.153005952878e+15 true resid norm 2.281947914341e+14 ||r(i)||/||b|| 2.193807128394e-02 16 KSP preconditioned resid norm 1.029063568485e+15 true resid norm 1.605944355705e+14 ||r(i)||/||b|| 1.543914369478e-02 17 KSP preconditioned resid norm 8.755986525312e+14 true resid norm 1.474488326268e+14 ||r(i)||/||b|| 1.417535860732e-02 18 KSP preconditioned resid norm 7.769999155370e+14 true resid norm 1.263847845681e+14 ||r(i)||/||b|| 1.215031419270e-02 19 KSP preconditioned resid norm 6.633003954457e+14 true resid norm 1.037940616116e+14 ||r(i)||/||b|| 9.978499106727e-03 20 KSP preconditioned resid norm 5.634039280371e+14 true resid norm 1.015458370156e+14 ||r(i)||/||b|| 9.762360468599e-03 21 KSP preconditioned resid norm 4.739327677772e+14 true resid norm 7.392362794765e+13 ||r(i)||/||b|| 7.106831007366e-03 22 KSP preconditioned resid norm 3.847430187788e+14 true resid norm 7.626771651358e+13 ||r(i)||/||b|| 7.332185765606e-03 23 KSP preconditioned resid norm 3.166693300565e+14 true resid norm 4.910897850051e+13 ||r(i)||/||b|| 4.721213241789e-03 24 KSP preconditioned resid norm 2.603961739838e+14 true resid norm 5.903512955155e+13 ||r(i)||/||b|| 5.675488350193e-03 25 KSP preconditioned resid norm 2.300385519514e+14 true resid norm 3.524302891352e+13 ||r(i)||/||b|| 3.388175846206e-03 26 KSP preconditioned resid norm 1.995854205790e+14 true resid norm 5.125398452570e+13 ||r(i)||/||b|| 4.927428707047e-03 27 KSP preconditioned resid norm 1.777736693158e+14 true resid norm 2.713391594494e+13 ||r(i)||/||b|| 2.608586192838e-03 28 KSP preconditioned resid norm 1.569155636768e+14 true resid norm 4.045925214031e+13 ||r(i)||/||b|| 3.889650381462e-03 29 KSP preconditioned resid norm 1.392286125087e+14 true resid norm 2.127117420025e+13 ||r(i)||/||b|| 2.044956999086e-03 30 KSP preconditioned resid norm 1.268134787155e+14 true resid norm 2.836210701297e+13 ||r(i)||/||b|| 2.726661382160e-03 31 KSP preconditioned resid norm 1.148332814766e+14 true resid norm 1.759485016641e+13 ||r(i)||/||b|| 1.691524485529e-03 32 KSP preconditioned resid norm 1.072826684457e+14 true resid norm 1.915166944365e+13 ||r(i)||/||b|| 1.841193161425e-03 33 KSP preconditioned resid norm 9.907876839152e+13 true resid norm 1.514435361562e+13 ||r(i)||/||b|| 1.455939932199e-03 34 KSP preconditioned resid norm 9.246074413874e+13 true resid norm 1.530899883812e+13 ||r(i)||/||b|| 1.471768508325e-03 35 KSP preconditioned resid norm 8.716833484735e+13 true resid norm 1.331822154534e+13 ||r(i)||/||b|| 1.280380204127e-03 36 KSP preconditioned resid norm 8.124134503515e+13 true resid norm 1.344545385639e+13 ||r(i)||/||b|| 1.292611997376e-03 37 KSP preconditioned resid norm 7.675303961825e+13 true resid norm 1.172301521125e+13 ||r(i)||/||b|| 1.127021093474e-03 38 KSP preconditioned resid norm 7.135475879451e+13 true resid norm 1.180708573448e+13 ||r(i)||/||b|| 1.135103421383e-03 39 KSP preconditioned resid norm 6.615399842578e+13 true resid norm 1.009923425569e+13 ||r(i)||/||b|| 9.709148908363e-04 40 KSP preconditioned resid norm 6.137620595303e+13 true resid norm 9.912104999293e+12 ||r(i)||/||b|| 9.529247564410e-04 41 KSP preconditioned resid norm 5.598390411354e+13 true resid norm 8.555770285650e+12 ||r(i)||/||b|| 8.225301604653e-04 0 KSP preconditioned resid norm 2.200654561303e+16 true resid norm 1.374535903551e+16 ||r(i)||/||b|| 1.042244800141e-01 1 KSP preconditioned resid norm 1.033788570768e+16 true resid norm 1.766933748516e+15 ||r(i)||/||b|| 1.339781308605e-02 2 KSP preconditioned resid norm 8.199712159807e+15 true resid norm 1.328504928257e+15 ||r(i)||/||b|| 1.007341714291e-02 3 KSP preconditioned resid norm 7.724963557325e+15 true resid norm 1.290091137776e+15 ||r(i)||/||b|| 9.782143751798e-03 4 KSP preconditioned resid norm 7.785648142256e+15 true resid norm 1.238722001104e+15 ||r(i)||/||b|| 9.392636170038e-03 5 KSP preconditioned resid norm 8.249045346662e+15 true resid norm 1.288778250144e+15 ||r(i)||/||b|| 9.772188753144e-03 6 KSP preconditioned resid norm 9.469275305646e+15 true resid norm 1.408770770519e+15 ||r(i)||/||b|| 1.068203461525e-02 7 KSP preconditioned resid norm 1.105611185924e+16 true resid norm 1.994387714295e+15 ||r(i)||/||b|| 1.512248766524e-02 8 KSP preconditioned resid norm 1.130059557946e+16 true resid norm 1.768360656538e+15 ||r(i)||/||b|| 1.340863264676e-02 9 KSP preconditioned resid norm 1.052660615325e+16 true resid norm 1.953565171874e+15 ||r(i)||/||b|| 1.481294986084e-02 10 KSP preconditioned resid norm 8.559164814525e+15 true resid norm 1.322544665666e+15 ||r(i)||/||b|| 1.002822332384e-02 11 KSP preconditioned resid norm 4.990037328467e+15 true resid norm 7.740616289621e+14 ||r(i)||/||b|| 5.869338921523e-03 12 KSP preconditioned resid norm 1.334591152980e+17 true resid norm 2.007497042766e+16 ||r(i)||/||b|| 1.522188942984e-01 13 KSP preconditioned resid norm 7.977709053830e+14 true resid norm 2.666709181112e+14 ||r(i)||/||b|| 2.022037962283e-03 0 KSP preconditioned resid norm 1.928493598618e+16 true resid norm 1.203377031534e+16 ||r(i)||/||b|| 4.946657154689e-01 1 KSP preconditioned resid norm 9.231841001478e+15 true resid norm 1.582820058439e+15 ||r(i)||/||b|| 6.506413170177e-02 2 KSP preconditioned resid norm 7.541228065487e+15 true resid norm 1.216411600127e+15 ||r(i)||/||b|| 5.000237653817e-02 3 KSP preconditioned resid norm 7.339567418801e+15 true resid norm 1.219667619599e+15 ||r(i)||/||b|| 5.013621997708e-02 4 KSP preconditioned resid norm 7.654746286546e+15 true resid norm 1.212045913869e+15 ||r(i)||/||b|| 4.982291862433e-02 5 KSP preconditioned resid norm 8.414088698130e+15 true resid norm 1.311254037213e+15 ||r(i)||/||b|| 5.390101352132e-02 6 KSP preconditioned resid norm 1.008553581054e+16 true resid norm 1.495637339777e+15 ||r(i)||/||b|| 6.148035863870e-02 7 KSP preconditioned resid norm 1.235521461592e+16 true resid norm 2.202862093934e+15 ||r(i)||/||b|| 9.055186572627e-02 8 KSP preconditioned resid norm 1.311621757723e+16 true resid norm 2.061404522413e+15 ||r(i)||/||b|| 8.473704551686e-02 9 KSP preconditioned resid norm 1.289521641859e+16 true resid norm 2.354903652303e+15 ||r(i)||/||b|| 9.680175618294e-02 10 KSP preconditioned resid norm 1.151646040420e+16 true resid norm 1.821568826477e+15 ||r(i)||/||b|| 7.487824872947e-02 11 KSP preconditioned resid norm 8.378553284195e+15 true resid norm 1.434374156491e+15 ||r(i)||/||b|| 5.896204595714e-02 12 KSP preconditioned resid norm 3.545538676243e+15 true resid norm 5.411687842502e+14 ||r(i)||/||b|| 2.224553376337e-02 13 KSP preconditioned resid norm 1.159722122499e+15 true resid norm 2.107714204908e+14 ||r(i)||/||b|| 8.664067269472e-03 14 KSP preconditioned resid norm 9.539010029354e+14 true resid norm 1.788314612223e+14 ||r(i)||/||b|| 7.351128565343e-03 15 KSP preconditioned resid norm 8.844744179759e+14 true resid norm 1.382762666459e+14 ||r(i)||/||b|| 5.684048023218e-03 16 KSP preconditioned resid norm 7.998051624629e+14 true resid norm 2.151800146054e+14 ||r(i)||/||b|| 8.845288973459e-03 17 KSP preconditioned resid norm 6.751752084065e+14 true resid norm 1.049375541763e+14 ||r(i)||/||b|| 4.313611524562e-03 18 KSP preconditioned resid norm 6.016391198259e+14 true resid norm 1.405509222696e+14 ||r(i)||/||b|| 5.777551066910e-03 19 KSP preconditioned resid norm 5.356608668579e+14 true resid norm 8.319469793388e+13 ||r(i)||/||b|| 3.419839642795e-03 20 KSP preconditioned resid norm 4.666955314009e+14 true resid norm 1.159814975417e+14 ||r(i)||/||b|| 4.767588956680e-03 21 KSP preconditioned resid norm 4.123539296814e+14 true resid norm 6.367161120725e+13 ||r(i)||/||b|| 2.617314631039e-03 22 KSP preconditioned resid norm 3.572486233332e+14 true resid norm 8.643344455842e+13 ||r(i)||/||b|| 3.552973056037e-03 23 KSP preconditioned resid norm 3.110551508962e+14 true resid norm 4.773889823616e+13 ||r(i)||/||b|| 1.962377179627e-03 24 KSP preconditioned resid norm 2.787205893436e+14 true resid norm 4.531540777818e+13 ||r(i)||/||b|| 1.862756062561e-03 25 KSP preconditioned resid norm 2.574052602586e+14 true resid norm 4.020202768069e+13 ||r(i)||/||b|| 1.652563100745e-03 26 KSP preconditioned resid norm 2.417321052344e+14 true resid norm 3.935243057179e+13 ||r(i)||/||b|| 1.617639169947e-03 27 KSP preconditioned resid norm 2.257728086711e+14 true resid norm 3.470313603079e+13 ||r(i)||/||b|| 1.426523123165e-03 28 KSP preconditioned resid norm 2.082687407590e+14 true resid norm 3.533906053355e+13 ||r(i)||/||b|| 1.452663729217e-03 29 KSP preconditioned resid norm 1.877437540892e+14 true resid norm 2.879571658728e+13 ||r(i)||/||b|| 1.183690013589e-03 30 KSP preconditioned resid norm 1.698389677822e+14 true resid norm 2.905672512335e+13 ||r(i)||/||b|| 1.194419150913e-03 31 KSP preconditioned resid norm 1.483411237433e+14 true resid norm 2.274318996948e+13 ||r(i)||/||b|| 9.348920615478e-04 32 KSP preconditioned resid norm 1.319650244589e+14 true resid norm 2.174098043497e+13 ||r(i)||/||b|| 8.936947739606e-04 0 KSP preconditioned resid norm 6.130010037542e+16 true resid norm 4.019535297204e+16 ||r(i)||/||b|| 5.748505993040e-02 1 KSP preconditioned resid norm 2.383231272294e+16 true resid norm 4.333028202793e+15 ||r(i)||/||b|| 6.196845344062e-03 2 KSP preconditioned resid norm 1.380314308424e+16 true resid norm 2.902938638811e+15 ||r(i)||/||b|| 4.151614285921e-03 3 KSP preconditioned resid norm 9.355422663861e+15 true resid norm 1.914374541722e+15 ||r(i)||/||b|| 2.737827313936e-03 4 KSP preconditioned resid norm 6.905389449720e+15 true resid norm 1.160495113499e+15 ||r(i)||/||b|| 1.659672728707e-03 5 KSP preconditioned resid norm 5.388850196885e+15 true resid norm 9.343171628690e+14 ||r(i)||/||b|| 1.336206156440e-03 6 KSP preconditioned resid norm 4.456941607393e+15 true resid norm 7.443205302890e+14 ||r(i)||/||b|| 1.064484004428e-03 7 KSP preconditioned resid norm 3.604897945956e+15 true resid norm 7.177354673261e+14 ||r(i)||/||b|| 1.026463591006e-03 8 KSP preconditioned resid norm 2.616552593649e+15 true resid norm 4.566784798390e+14 ||r(i)||/||b|| 6.531150454316e-04 9 KSP preconditioned resid norm 1.730753896393e+15 true resid norm 3.416577992882e+14 ||r(i)||/||b|| 4.886191466320e-04 10 KSP preconditioned resid norm 9.605913238033e+14 true resid norm 1.715828091641e+14 ||r(i)||/||b|| 2.453877709368e-04 11 KSP preconditioned resid norm 3.885469473149e+14 true resid norm 7.526615944264e+13 ||r(i)||/||b|| 1.076412910045e-04 12 KSP preconditioned resid norm 1.085402135701e+14 true resid norm 1.877490961949e+13 ||r(i)||/||b|| 2.685078559740e-05 13 KSP preconditioned resid norm 5.521535097548e+13 true resid norm 1.482831670997e+13 ||r(i)||/||b|| 2.120659757192e-05 14 KSP preconditioned resid norm 4.327778991064e+13 true resid norm 6.949075957635e+12 ||r(i)||/||b|| 9.938164945670e-06 0 KSP preconditioned resid norm 2.469229041488e+16 true resid norm 1.581397714283e+16 ||r(i)||/||b|| 1.421322439001e-01 1 KSP preconditioned resid norm 1.094866313372e+16 true resid norm 1.827538507852e+15 ||r(i)||/||b|| 1.642547896642e-02 2 KSP preconditioned resid norm 7.902647446780e+15 true resid norm 1.331972422469e+15 ||r(i)||/||b|| 1.197144952903e-02 3 KSP preconditioned resid norm 6.757114900348e+15 true resid norm 1.177404752019e+15 ||r(i)||/||b|| 1.058223227919e-02 4 KSP preconditioned resid norm 6.236820681429e+15 true resid norm 9.858934558838e+14 ||r(i)||/||b|| 8.860974558495e-03 5 KSP preconditioned resid norm 6.056208231119e+15 true resid norm 9.498735329089e+14 ||r(i)||/||b|| 8.537236106662e-03 6 KSP preconditioned resid norm 6.281150442631e+15 true resid norm 9.439017368810e+14 ||r(i)||/||b|| 8.483563032401e-03 7 KSP preconditioned resid norm 6.511637444968e+15 true resid norm 1.175605283055e+15 ||r(i)||/||b|| 1.056605908257e-02 8 KSP preconditioned resid norm 5.897282453520e+15 true resid norm 9.358326231502e+14 ||r(i)||/||b|| 8.411039768298e-03 9 KSP preconditioned resid norm 4.858643646570e+15 true resid norm 9.074077133036e+14 ||r(i)||/||b|| 8.155563477757e-03 10 KSP preconditioned resid norm 3.431227043473e+15 true resid norm 5.523077341885e+14 ||r(i)||/||b|| 4.964009804403e-03 11 KSP preconditioned resid norm 1.790602152143e+15 true resid norm 3.233768236148e+14 ||r(i)||/||b|| 2.906433539808e-03 12 KSP preconditioned resid norm 6.061523931075e+14 true resid norm 9.361733206500e+13 ||r(i)||/||b|| 8.414101875932e-04 13 KSP preconditioned resid norm 3.669173259935e+14 true resid norm 7.433445474494e+13 ||r(i)||/||b|| 6.681002986515e-04 14 KSP preconditioned resid norm 1.034316883992e+15 true resid norm 1.749783469552e+14 ||r(i)||/||b|| 1.572663528097e-03 15 KSP preconditioned resid norm 1.340579303848e+14 true resid norm 2.477734260421e+13 ||r(i)||/||b|| 2.226928286548e-04 16 KSP preconditioned resid norm 1.124182191319e+14 true resid norm 1.917839477947e+13 ||r(i)||/||b|| 1.723708248589e-04 17 KSP preconditioned resid norm 9.394060484065e+13 true resid norm 1.682558945581e+13 ||r(i)||/||b|| 1.512243733944e-04 18 KSP preconditioned resid norm 9.034672558986e+13 true resid norm 1.892870450718e+13 ||r(i)||/||b|| 1.701266684168e-04 19 KSP preconditioned resid norm 8.058964131802e+13 true resid norm 1.275370577304e+13 ||r(i)||/||b|| 1.146272568369e-04 20 KSP preconditioned resid norm 6.303390378547e+13 true resid norm 1.900069972515e+13 ||r(i)||/||b|| 1.707737442148e-04 21 KSP preconditioned resid norm 4.601826552473e+13 true resid norm 6.960266416937e+12 ||r(i)||/||b|| 6.255720967895e-05 22 KSP preconditioned resid norm 2.822788042823e+13 true resid norm 1.013300658091e+13 ||r(i)||/||b|| 9.107303936204e-05 23 KSP preconditioned resid norm 1.858182222289e+13 true resid norm 2.785683196819e+12 ||r(i)||/||b|| 2.503705424529e-05 24 KSP preconditioned resid norm 9.506926663598e+12 true resid norm 4.268961211449e+12 ||r(i)||/||b|| 3.836840224479e-05 25 KSP preconditioned resid norm 5.845171932356e+12 true resid norm 8.923162702109e+11 ||r(i)||/||b|| 8.019925197071e-06 0 SNES Function norm 0.000000000000e+00 4 TS dt 6e-12 time 1.4e-11 0 KSP preconditioned resid norm 2.220105437401e+19 true resid norm 1.204162959566e+19 ||r(i)||/||b|| 2.302124615573e+02 1 KSP preconditioned resid norm 5.389751984623e+18 true resid norm 1.775745946542e+18 ||r(i)||/||b|| 3.394879756150e+01 2 KSP preconditioned resid norm 2.218217192238e+18 true resid norm 5.319400119211e+17 ||r(i)||/||b|| 1.016965507636e+01 3 KSP preconditioned resid norm 1.401615472906e+18 true resid norm 2.963370402601e+17 ||r(i)||/||b|| 5.665385980104e+00 4 KSP preconditioned resid norm 1.004001983833e+18 true resid norm 1.656139793050e+17 ||r(i)||/||b|| 3.166216128906e+00 5 KSP preconditioned resid norm 8.672200498511e+17 true resid norm 1.510248645324e+17 ||r(i)||/||b|| 2.887300721564e+00 6 KSP preconditioned resid norm 7.388847075384e+17 true resid norm 1.526081348396e+17 ||r(i)||/||b|| 2.917569760470e+00 7 KSP preconditioned resid norm 5.565797182786e+17 true resid norm 1.017271957075e+17 ||r(i)||/||b|| 1.944825486043e+00 8 KSP preconditioned resid norm 3.566434137336e+17 true resid norm 6.306431985913e+16 ||r(i)||/||b|| 1.205666741023e+00 9 KSP preconditioned resid norm 1.893287605930e+17 true resid norm 3.267942353405e+16 ||r(i)||/||b|| 6.247668120231e-01 10 KSP preconditioned resid norm 6.895870882324e+16 true resid norm 1.153127225742e+16 ||r(i)||/||b|| 2.204554250883e-01 11 KSP preconditioned resid norm 5.104972946485e+16 true resid norm 9.275497869859e+15 ||r(i)||/||b|| 1.773294203933e-01 12 KSP preconditioned resid norm 4.992720890366e+16 true resid norm 8.399052910493e+15 ||r(i)||/||b|| 1.605735029394e-01 13 KSP preconditioned resid norm 4.919979261885e+16 true resid norm 8.426637216788e+15 ||r(i)||/||b|| 1.611008610517e-01 14 KSP preconditioned resid norm 5.688292138515e+16 true resid norm 9.762950550877e+15 ||r(i)||/||b|| 1.866485645090e-01 15 KSP preconditioned resid norm 1.614157649132e+17 true resid norm 4.341833601566e+16 ||r(i)||/||b|| 8.300738642954e-01 16 KSP preconditioned resid norm 4.125579354715e+16 true resid norm 7.099585962609e+15 ||r(i)||/||b|| 1.357302304896e-01 17 KSP preconditioned resid norm 2.163051916179e+16 true resid norm 5.849969022355e+15 ||r(i)||/||b|| 1.118399929155e-01 18 KSP preconditioned resid norm 1.415003279507e+16 true resid norm 2.432246275858e+15 ||r(i)||/||b|| 4.649980285729e-02 19 KSP preconditioned resid norm 1.088163683743e+16 true resid norm 2.488576155885e+15 ||r(i)||/||b|| 4.757672024934e-02 20 KSP preconditioned resid norm 9.060374162401e+15 true resid norm 1.588297599957e+15 ||r(i)||/||b|| 3.036515093466e-02 21 KSP preconditioned resid norm 7.815461281280e+15 true resid norm 1.346385634634e+15 ||r(i)||/||b|| 2.574026619006e-02 22 KSP preconditioned resid norm 6.943207518072e+15 true resid norm 1.184751633436e+15 ||r(i)||/||b|| 2.265013947659e-02 23 KSP preconditioned resid norm 6.246190157097e+15 true resid norm 1.023840274541e+15 ||r(i)||/||b|| 1.957382827391e-02 24 KSP preconditioned resid norm 5.682318425500e+15 true resid norm 9.496257176169e+14 ||r(i)||/||b|| 1.815499075718e-02 25 KSP preconditioned resid norm 5.282641218542e+15 true resid norm 8.559439950771e+14 ||r(i)||/||b|| 1.636397901932e-02 26 KSP preconditioned resid norm 5.115699254805e+15 true resid norm 8.571023693113e+14 ||r(i)||/||b|| 1.638612487439e-02 27 KSP preconditioned resid norm 4.941038732000e+15 true resid norm 9.550184315260e+14 ||r(i)||/||b|| 1.825808892456e-02 28 KSP preconditioned resid norm 4.836641976772e+15 true resid norm 7.564719174843e+14 ||r(i)||/||b|| 1.446226699132e-02 29 KSP preconditioned resid norm 4.905885514019e+15 true resid norm 7.773698572205e+14 ||r(i)||/||b|| 1.486179482182e-02 30 KSP preconditioned resid norm 5.169672483375e+15 true resid norm 8.117514654202e+14 ||r(i)||/||b|| 1.551910408326e-02 31 KSP preconditioned resid norm 5.732174203621e+15 true resid norm 9.079138157636e+14 ||r(i)||/||b|| 1.735754058438e-02 32 KSP preconditioned resid norm 8.584579912384e+15 true resid norm 1.381429969496e+15 ||r(i)||/||b|| 2.641024549212e-02 33 KSP preconditioned resid norm 1.658456221337e+16 true resid norm 2.668310391151e+15 ||r(i)||/||b|| 5.101288811997e-02 34 KSP preconditioned resid norm 3.797117274875e+16 true resid norm 6.214594485387e+15 ||r(i)||/||b|| 1.188109202908e-01 35 KSP preconditioned resid norm 6.896188371971e+15 true resid norm 1.129650731508e+15 ||r(i)||/||b|| 2.159671774774e-02 36 KSP preconditioned resid norm 3.678389972715e+15 true resid norm 6.014447539523e+14 ||r(i)||/||b|| 1.149845012239e-02 37 KSP preconditioned resid norm 2.344506104501e+15 true resid norm 4.262436512363e+14 ||r(i)||/||b|| 8.148946900809e-03 38 KSP preconditioned resid norm 1.979277723710e+15 true resid norm 3.194432632084e+14 ||r(i)||/||b|| 6.107131876701e-03 39 KSP preconditioned resid norm 1.806670892810e+15 true resid norm 3.108273060962e+14 ||r(i)||/||b|| 5.942411588660e-03 40 KSP preconditioned resid norm 1.663957842585e+15 true resid norm 2.662101226861e+14 ||r(i)||/||b|| 5.089418101442e-03 41 KSP preconditioned resid norm 1.486329834699e+15 true resid norm 2.427881601889e+14 ||r(i)||/||b|| 4.641635880760e-03 42 KSP preconditioned resid norm 1.346929765546e+15 true resid norm 2.207126725819e+14 ||r(i)||/||b|| 4.219595632660e-03 43 KSP preconditioned resid norm 1.242596801597e+15 true resid norm 2.017748115787e+14 ||r(i)||/||b|| 3.857540682909e-03 44 KSP preconditioned resid norm 1.178815061343e+15 true resid norm 1.881589524072e+14 ||r(i)||/||b|| 3.597232023587e-03 45 KSP preconditioned resid norm 9.347996687830e+14 true resid norm 2.754924905110e+14 ||r(i)||/||b|| 5.266878861971e-03 46 KSP preconditioned resid norm 7.515414326072e+14 true resid norm 1.170313953112e+14 ||r(i)||/||b|| 2.237411920043e-03 47 KSP preconditioned resid norm 7.344982573734e+14 true resid norm 1.334969192887e+14 ||r(i)||/||b|| 2.552200610027e-03 48 KSP preconditioned resid norm 8.030126805909e+14 true resid norm 1.311086080593e+14 ||r(i)||/||b|| 2.506540759528e-03 49 KSP preconditioned resid norm 9.282223496248e+14 true resid norm 1.446635532628e+14 ||r(i)||/||b|| 2.765684862641e-03 50 KSP preconditioned resid norm 1.074003818336e+15 true resid norm 3.137568207714e+14 ||r(i)||/||b|| 5.998418193015e-03 51 KSP preconditioned resid norm 1.638671425800e+15 true resid norm 2.543484445494e+14 ||r(i)||/||b|| 4.862645960648e-03 52 KSP preconditioned resid norm 2.418883444028e+15 true resid norm 6.581384678327e+14 ||r(i)||/||b|| 1.258232330779e-02 53 KSP preconditioned resid norm 8.572181890774e+14 true resid norm 1.327918910006e+14 ||r(i)||/||b|| 2.538721844848e-03 54 KSP preconditioned resid norm 2.577275091696e+14 true resid norm 8.010767963629e+13 ||r(i)||/||b|| 1.531502523990e-03 0 KSP preconditioned resid norm 6.542967041071e+18 true resid norm 3.545980595607e+18 ||r(i)||/||b|| 4.156826068038e+02 1 KSP preconditioned resid norm 1.591191551470e+18 true resid norm 5.223269779130e+17 ||r(i)||/||b|| 6.123052112913e+01 2 KSP preconditioned resid norm 6.570663371928e+17 true resid norm 1.552539857829e+17 ||r(i)||/||b|| 1.819986877731e+01 3 KSP preconditioned resid norm 4.165904471738e+17 true resid norm 8.738677344351e+16 ||r(i)||/||b|| 1.024403851227e+01 4 KSP preconditioned resid norm 2.995075360323e+17 true resid norm 4.934782956985e+16 ||r(i)||/||b|| 5.784869342240e+00 5 KSP preconditioned resid norm 2.594552595600e+17 true resid norm 4.469892332113e+16 ||r(i)||/||b|| 5.239894710780e+00 6 KSP preconditioned resid norm 2.219662091794e+17 true resid norm 4.652217437246e+16 ||r(i)||/||b|| 5.453627902330e+00 7 KSP preconditioned resid norm 1.674952385698e+17 true resid norm 2.988890119169e+16 ||r(i)||/||b|| 3.503768852333e+00 8 KSP preconditioned resid norm 1.081926357303e+17 true resid norm 1.908191297028e+16 ||r(i)||/||b|| 2.236904323761e+00 9 KSP preconditioned resid norm 5.710239913499e+16 true resid norm 9.635393149266e+15 ||r(i)||/||b|| 1.129522633832e+00 10 KSP preconditioned resid norm 1.779371916701e+16 true resid norm 2.952475144259e+15 ||r(i)||/||b|| 3.461080881292e-01 11 KSP preconditioned resid norm 1.125590077314e+16 true resid norm 2.030360591090e+15 ||r(i)||/||b|| 2.380119012218e-01 12 KSP preconditioned resid norm 1.086608349874e+16 true resid norm 1.877856566597e+15 ||r(i)||/||b|| 2.201344005587e-01 13 KSP preconditioned resid norm 1.075368648574e+16 true resid norm 1.822636849246e+15 ||r(i)||/||b|| 2.136611908395e-01 14 KSP preconditioned resid norm 1.173878698877e+16 true resid norm 2.130421173676e+15 ||r(i)||/||b|| 2.497416449940e-01 15 KSP preconditioned resid norm 4.944017897832e+16 true resid norm 1.361808358525e+16 ||r(i)||/||b|| 1.596399171332e+00 16 KSP preconditioned resid norm 1.780572750257e+17 true resid norm 3.371295639234e+16 ||r(i)||/||b|| 3.952049149280e+00 17 KSP preconditioned resid norm 3.251503916379e+16 true resid norm 8.719191629936e+15 ||r(i)||/||b|| 1.022119610706e+00 18 KSP preconditioned resid norm 2.075735168556e+16 true resid norm 3.945307122742e+15 ||r(i)||/||b|| 4.624942255619e-01 19 KSP preconditioned resid norm 1.483613863843e+16 true resid norm 3.390657011871e+15 ||r(i)||/||b|| 3.974745793076e-01 20 KSP preconditioned resid norm 1.134405248775e+16 true resid norm 2.490065225313e+15 ||r(i)||/||b|| 2.919014292554e-01 21 KSP preconditioned resid norm 8.553736210329e+15 true resid norm 1.657655420050e+15 ||r(i)||/||b|| 1.943210087055e-01 22 KSP preconditioned resid norm 6.586515952486e+15 true resid norm 1.286392357530e+15 ||r(i)||/||b|| 1.507991694069e-01 23 KSP preconditioned resid norm 4.780127454522e+15 true resid norm 9.064248964802e+14 ||r(i)||/||b|| 1.062569446397e-01 24 KSP preconditioned resid norm 3.027986246405e+15 true resid norm 6.179735775190e+14 ||r(i)||/||b|| 7.244282948339e-02 25 KSP preconditioned resid norm 1.571830581724e+15 true resid norm 2.758055375192e+14 ||r(i)||/||b|| 3.233169548331e-02 26 KSP preconditioned resid norm 1.182949949046e+15 true resid norm 2.283703518838e+14 ||r(i)||/||b|| 2.677103854019e-02 27 KSP preconditioned resid norm 1.078564628165e+15 true resid norm 2.041848112402e+14 ||r(i)||/||b|| 2.393585422076e-02 28 KSP preconditioned resid norm 1.016575362098e+15 true resid norm 1.676746397025e+14 ||r(i)||/||b|| 1.965589755702e-02 29 KSP preconditioned resid norm 9.501065512919e+14 true resid norm 1.807838083536e+14 ||r(i)||/||b|| 2.119263845308e-02 30 KSP preconditioned resid norm 9.022105299639e+14 true resid norm 1.464542351381e+14 ||r(i)||/||b|| 1.716830552177e-02 31 KSP preconditioned resid norm 8.706508374697e+14 true resid norm 1.587763993315e+14 ||r(i)||/||b|| 1.861278870358e-02 32 KSP preconditioned resid norm 8.363545887412e+14 true resid norm 1.341498283380e+14 ||r(i)||/||b|| 1.572590397559e-02 33 KSP preconditioned resid norm 8.049250107078e+14 true resid norm 1.594310708223e+14 ||r(i)||/||b|| 1.868953349802e-02 34 KSP preconditioned resid norm 7.730163706333e+14 true resid norm 1.360746921537e+14 ||r(i)||/||b|| 1.595154886763e-02 35 KSP preconditioned resid norm 7.656443951451e+14 true resid norm 1.202971650940e+14 ||r(i)||/||b|| 1.410200587090e-02 36 KSP preconditioned resid norm 7.731881795958e+14 true resid norm 1.274146022068e+14 ||r(i)||/||b|| 1.493635753557e-02 37 KSP preconditioned resid norm 8.066446225342e+14 true resid norm 1.271087970298e+14 ||r(i)||/||b|| 1.490050908979e-02 38 KSP preconditioned resid norm 8.449932808607e+14 true resid norm 1.374808484319e+14 ||r(i)||/||b|| 1.611638753257e-02 39 KSP preconditioned resid norm 8.841110986186e+14 true resid norm 1.396540752943e+14 ||r(i)||/||b|| 1.637114713515e-02 40 KSP preconditioned resid norm 9.488359303223e+14 true resid norm 1.529468173159e+14 ||r(i)||/||b|| 1.792940768006e-02 41 KSP preconditioned resid norm 1.207535627829e+15 true resid norm 1.911287075526e+14 ||r(i)||/||b|| 2.240533394033e-02 42 KSP preconditioned resid norm 2.145532208476e+15 true resid norm 3.678432914805e+14 ||r(i)||/||b|| 4.312095178618e-02 43 KSP preconditioned resid norm 1.285566323841e+16 true resid norm 2.041241798720e+15 ||r(i)||/||b|| 2.392874662258e-01 44 KSP preconditioned resid norm 1.228721458245e+15 true resid norm 2.037912017178e+14 ||r(i)||/||b|| 2.388971278599e-02 45 KSP preconditioned resid norm 6.120797893644e+14 true resid norm 9.792977962736e+13 ||r(i)||/||b|| 1.147995737192e-02 46 KSP preconditioned resid norm 4.263304414021e+14 true resid norm 6.948352885735e+13 ||r(i)||/||b|| 8.145305262283e-03 47 KSP preconditioned resid norm 2.943667166575e+14 true resid norm 4.740665276317e+13 ||r(i)||/||b|| 5.557312136691e-03 48 KSP preconditioned resid norm 1.946102625201e+14 true resid norm 3.133411275125e+13 ||r(i)||/||b|| 3.673185828050e-03 49 KSP preconditioned resid norm 1.369383238938e+14 true resid norm 2.605829767015e+13 ||r(i)||/||b|| 3.054720919178e-03 50 KSP preconditioned resid norm 9.297342410318e+13 true resid norm 1.484605459745e+13 ||r(i)||/||b|| 1.740349815638e-03 51 KSP preconditioned resid norm 6.674823642018e+13 true resid norm 2.111621172189e+13 ||r(i)||/||b|| 2.475377881439e-03 52 KSP preconditioned resid norm 5.794642992605e+13 true resid norm 9.469052681739e+12 ||r(i)||/||b|| 1.110023136501e-03 53 KSP preconditioned resid norm 4.582540149696e+13 true resid norm 7.443523694856e+12 ||r(i)||/||b|| 8.725776269382e-04 0 KSP preconditioned resid norm 1.892653228182e+19 true resid norm 1.028330526045e+19 ||r(i)||/||b|| 7.841194129911e+01 1 KSP preconditioned resid norm 4.581281818866e+18 true resid norm 1.519814562161e+18 ||r(i)||/||b|| 1.158884300480e+01 2 KSP preconditioned resid norm 1.874494262231e+18 true resid norm 4.646649902787e+17 ||r(i)||/||b|| 3.543149115843e+00 3 KSP preconditioned resid norm 1.173263764111e+18 true resid norm 2.551017670341e+17 ||r(i)||/||b|| 1.945194105918e+00 4 KSP preconditioned resid norm 8.301325588811e+17 true resid norm 1.381381450838e+17 ||r(i)||/||b|| 1.053326712486e+00 5 KSP preconditioned resid norm 7.052947600755e+17 true resid norm 1.287824897888e+17 ||r(i)||/||b|| 9.819882590194e-01 6 KSP preconditioned resid norm 5.863848966069e+17 true resid norm 1.166961566059e+17 ||r(i)||/||b|| 8.898279249584e-01 7 KSP preconditioned resid norm 4.330736686179e+17 true resid norm 8.611330077675e+16 ||r(i)||/||b|| 6.566284783504e-01 8 KSP preconditioned resid norm 2.702589924096e+17 true resid norm 4.803726753137e+16 ||r(i)||/||b|| 3.662922870069e-01 9 KSP preconditioned resid norm 1.546977321412e+17 true resid norm 2.878601764016e+16 ||r(i)||/||b|| 2.194982516096e-01 10 KSP preconditioned resid norm 9.910027101694e+16 true resid norm 1.648820746608e+16 ||r(i)||/||b|| 1.257253697341e-01 11 KSP preconditioned resid norm 9.241668505040e+16 true resid norm 1.559203946128e+16 ||r(i)||/||b|| 1.188919371745e-01 12 KSP preconditioned resid norm 9.244329678182e+16 true resid norm 1.506886748242e+16 ||r(i)||/||b|| 1.149026623784e-01 13 KSP preconditioned resid norm 7.090985777634e+16 true resid norm 1.150411231671e+16 ||r(i)||/||b|| 8.772080151562e-02 14 KSP preconditioned resid norm 2.014608975967e+17 true resid norm 3.263298275594e+16 ||r(i)||/||b|| 2.488320110574e-01 15 KSP preconditioned resid norm 1.667109940317e+16 true resid norm 4.390284971711e+15 ||r(i)||/||b|| 3.347666521311e-02 16 KSP preconditioned resid norm 1.169352181037e+16 true resid norm 1.861078956987e+15 ||r(i)||/||b|| 1.419104171589e-02 17 KSP preconditioned resid norm 1.018452985290e+16 true resid norm 1.996695440591e+15 ||r(i)||/||b|| 1.522514033324e-02 18 KSP preconditioned resid norm 9.390559252345e+15 true resid norm 1.536845912378e+15 ||r(i)||/||b|| 1.171870992984e-02 19 KSP preconditioned resid norm 8.796086996183e+15 true resid norm 1.422370571244e+15 ||r(i)||/||b|| 1.084581609835e-02 20 KSP preconditioned resid norm 8.297954532675e+15 true resid norm 1.485654737712e+15 ||r(i)||/||b|| 1.132836856767e-02 21 KSP preconditioned resid norm 7.837153561450e+15 true resid norm 1.247795221846e+15 ||r(i)||/||b|| 9.514649542211e-03 22 KSP preconditioned resid norm 7.441474574169e+15 true resid norm 1.321880544976e+15 ||r(i)||/||b|| 1.007956265733e-02 23 KSP preconditioned resid norm 7.117292465449e+15 true resid norm 1.122736430687e+15 ||r(i)||/||b|| 8.561055114844e-03 24 KSP preconditioned resid norm 5.694316647476e+15 true resid norm 2.071287152890e+15 ||r(i)||/||b|| 1.579391475139e-02 25 KSP preconditioned resid norm 4.803477075252e+15 true resid norm 7.688470974380e+14 ||r(i)||/||b|| 5.862589113657e-03 26 KSP preconditioned resid norm 4.890644639484e+15 true resid norm 7.629702578172e+14 ||r(i)||/||b|| 5.817777217899e-03 27 KSP preconditioned resid norm 5.168669214975e+15 true resid norm 8.754612707062e+14 ||r(i)||/||b|| 6.675540211016e-03 28 KSP preconditioned resid norm 8.939764002038e+15 true resid norm 1.397979783339e+15 ||r(i)||/||b|| 1.065983221661e-02 29 KSP preconditioned resid norm 8.205557149295e+15 true resid norm 1.863983796228e+15 ||r(i)||/||b|| 1.421319160626e-02 30 KSP preconditioned resid norm 1.967760781914e+15 true resid norm 3.069470515034e+14 ||r(i)||/||b|| 2.340523165932e-03 31 KSP preconditioned resid norm 1.158003829729e+15 true resid norm 2.612136672864e+14 ||r(i)||/||b|| 1.991798378735e-03 32 KSP preconditioned resid norm 8.020817038350e+14 true resid norm 1.249397423467e+14 ||r(i)||/||b|| 9.526866600465e-04 0 KSP preconditioned resid norm 6.470845423033e+18 true resid norm 3.511573094792e+18 ||r(i)||/||b|| 1.662215647083e+02 1 KSP preconditioned resid norm 1.571609661901e+18 true resid norm 5.187508694693e+17 ||r(i)||/||b|| 2.455525739870e+01 2 KSP preconditioned resid norm 6.468393005498e+17 true resid norm 1.566999779328e+17 ||r(i)||/||b|| 7.417449336415e+00 3 KSP preconditioned resid norm 4.084082755984e+17 true resid norm 8.688467105425e+16 ||r(i)||/||b|| 4.112716888400e+00 4 KSP preconditioned resid norm 2.920922708944e+17 true resid norm 4.832004038605e+16 ||r(i)||/||b|| 2.287246343141e+00 5 KSP preconditioned resid norm 2.515891595070e+17 true resid norm 4.435607154638e+16 ||r(i)||/||b|| 2.099610464520e+00 6 KSP preconditioned resid norm 2.133101049372e+17 true resid norm 4.355693207474e+16 ||r(i)||/||b|| 2.061782912647e+00 7 KSP preconditioned resid norm 1.601303798951e+17 true resid norm 2.999351550015e+16 ||r(i)||/||b|| 1.419753752223e+00 8 KSP preconditioned resid norm 1.019899000725e+17 true resid norm 1.811131044506e+16 ||r(i)||/||b|| 8.573053386130e-01 9 KSP preconditioned resid norm 5.552370825926e+16 true resid norm 9.849613114257e+15 ||r(i)||/||b|| 4.662349492457e-01 10 KSP preconditioned resid norm 2.614277580792e+16 true resid norm 4.400858558792e+15 ||r(i)||/||b|| 2.083162092759e-01 11 KSP preconditioned resid norm 2.230379890809e+16 true resid norm 4.001941486401e+15 ||r(i)||/||b|| 1.894333273960e-01 12 KSP preconditioned resid norm 2.202819377737e+16 true resid norm 3.668912744052e+15 ||r(i)||/||b|| 1.736692931151e-01 13 KSP preconditioned resid norm 2.171381974288e+16 true resid norm 3.746629440349e+15 ||r(i)||/||b|| 1.773480406489e-01 14 KSP preconditioned resid norm 2.756519085776e+16 true resid norm 4.738155898553e+15 ||r(i)||/||b|| 2.242822991374e-01 15 KSP preconditioned resid norm 1.910971370209e+16 true resid norm 7.272906924885e+15 ||r(i)||/||b|| 3.442656428895e-01 16 KSP preconditioned resid norm 8.265967788035e+15 true resid norm 1.450280377529e+15 ||r(i)||/||b|| 6.864953885656e-02 17 KSP preconditioned resid norm 5.450920356042e+15 true resid norm 1.778917951827e+15 ||r(i)||/||b|| 8.420571563182e-02 18 KSP preconditioned resid norm 4.184159413509e+15 true resid norm 7.350039137637e+14 ||r(i)||/||b|| 3.479167236863e-02 19 KSP preconditioned resid norm 3.481860538622e+15 true resid norm 7.678205813365e+14 ||r(i)||/||b|| 3.634506103098e-02 20 KSP preconditioned resid norm 3.027493421078e+15 true resid norm 5.662352682787e+14 ||r(i)||/||b|| 2.680294834981e-02 21 KSP preconditioned resid norm 2.649808113090e+15 true resid norm 5.331458151920e+14 ||r(i)||/||b|| 2.523664728788e-02 22 KSP preconditioned resid norm 2.408644968014e+15 true resid norm 3.911896234018e+14 ||r(i)||/||b|| 1.851710032633e-02 23 KSP preconditioned resid norm 2.245371706567e+15 true resid norm 3.514281238053e+14 ||r(i)||/||b|| 1.663497556353e-02 24 KSP preconditioned resid norm 2.104176299070e+15 true resid norm 3.427795705053e+14 ||r(i)||/||b|| 1.622559320890e-02 25 KSP preconditioned resid norm 1.992290690493e+15 true resid norm 3.118626867030e+14 ||r(i)||/||b|| 1.476213148881e-02 26 KSP preconditioned resid norm 1.936852873636e+15 true resid norm 3.191431362940e+14 ||r(i)||/||b|| 1.510675416649e-02 27 KSP preconditioned resid norm 1.840320660654e+15 true resid norm 4.277077423449e+14 ||r(i)||/||b|| 2.024569850927e-02 28 KSP preconditioned resid norm 1.797325493063e+15 true resid norm 2.805438081480e+14 ||r(i)||/||b|| 1.327964120375e-02 29 KSP preconditioned resid norm 1.907733126356e+15 true resid norm 3.072709450643e+14 ||r(i)||/||b|| 1.454477976088e-02 30 KSP preconditioned resid norm 2.241086833192e+15 true resid norm 3.515931723170e+14 ||r(i)||/||b|| 1.664278819369e-02 31 KSP preconditioned resid norm 3.121906541214e+15 true resid norm 5.044161632328e+14 ||r(i)||/||b|| 2.387671896708e-02 32 KSP preconditioned resid norm 2.288516501159e+16 true resid norm 3.696560518057e+15 ||r(i)||/||b|| 1.749780103571e-01 33 KSP preconditioned resid norm 2.665344016377e+15 true resid norm 4.419048360378e+14 ||r(i)||/||b|| 2.091772300206e-02 34 KSP preconditioned resid norm 1.193724857597e+15 true resid norm 1.964465911853e+14 ||r(i)||/||b|| 9.298869448811e-03 35 KSP preconditioned resid norm 7.370892713374e+14 true resid norm 1.243994812053e+14 ||r(i)||/||b|| 5.888493805102e-03 36 KSP preconditioned resid norm 5.570336976172e+14 true resid norm 9.038862530299e+13 ||r(i)||/||b|| 4.278577812313e-03 37 KSP preconditioned resid norm 4.452876529911e+14 true resid norm 9.072905275923e+13 ||r(i)||/||b|| 4.294692067355e-03 38 KSP preconditioned resid norm 4.015373409323e+14 true resid norm 6.374984875718e+13 ||r(i)||/||b|| 3.017621824831e-03 39 KSP preconditioned resid norm 3.753907061691e+14 true resid norm 7.170117945509e+13 ||r(i)||/||b|| 3.394000898950e-03 40 KSP preconditioned resid norm 3.520150126535e+14 true resid norm 5.546554172401e+13 ||r(i)||/||b|| 2.625481197140e-03 41 KSP preconditioned resid norm 3.283953946659e+14 true resid norm 5.672854000420e+13 ||r(i)||/||b|| 2.685265671132e-03 42 KSP preconditioned resid norm 3.086422594501e+14 true resid norm 5.033158713486e+13 ||r(i)||/||b|| 2.382463625836e-03 43 KSP preconditioned resid norm 2.903330837015e+14 true resid norm 5.711996083788e+13 ||r(i)||/||b|| 2.703793715879e-03 44 KSP preconditioned resid norm 2.805776139313e+14 true resid norm 4.359341431583e+13 ||r(i)||/||b|| 2.063509812540e-03 45 KSP preconditioned resid norm 2.850165946773e+14 true resid norm 4.589953632859e+13 ||r(i)||/||b|| 2.172670920403e-03 46 KSP preconditioned resid norm 3.036441891069e+14 true resid norm 4.745822530443e+13 ||r(i)||/||b|| 2.246452018920e-03 47 KSP preconditioned resid norm 3.630944657131e+14 true resid norm 5.763700439451e+13 ||r(i)||/||b|| 2.728268157016e-03 48 KSP preconditioned resid norm 1.221109219167e+15 true resid norm 1.980487895228e+14 ||r(i)||/||b|| 9.374710078478e-03 49 KSP preconditioned resid norm 3.188684128549e+15 true resid norm 5.020159499531e+14 ||r(i)||/||b|| 2.376310401554e-02 50 KSP preconditioned resid norm 3.895126724245e+13 true resid norm 6.299322456844e+12 ||r(i)||/||b|| 2.981806749036e-04 0 KSP preconditioned resid norm 6.694681597053e+18 true resid norm 3.650754072115e+18 ||r(i)||/||b|| 5.229798617888e+00 1 KSP preconditioned resid norm 1.597667334483e+18 true resid norm 5.366749713509e+17 ||r(i)||/||b|| 7.688006280303e-01 2 KSP preconditioned resid norm 6.300446759111e+17 true resid norm 1.705328267334e+17 ||r(i)||/||b|| 2.442926376135e-01 3 KSP preconditioned resid norm 3.642174486049e+17 true resid norm 9.481576844129e+16 ||r(i)||/||b|| 1.358260142845e-01 4 KSP preconditioned resid norm 2.247332959635e+17 true resid norm 3.802800729856e+16 ||r(i)||/||b|| 5.447609345427e-02 5 KSP preconditioned resid norm 1.557523701769e+17 true resid norm 3.829141999541e+16 ||r(i)||/||b|| 5.485343888228e-02 6 KSP preconditioned resid norm 1.004692846569e+17 true resid norm 1.699728099873e+16 ||r(i)||/||b|| 2.434903992959e-02 7 KSP preconditioned resid norm 7.103482802189e+16 true resid norm 1.472858884963e+16 ||r(i)||/||b|| 2.109908037838e-02 8 KSP preconditioned resid norm 5.766847885215e+16 true resid norm 9.397954062732e+15 ||r(i)||/||b|| 1.346280965449e-02 9 KSP preconditioned resid norm 4.523472016725e+16 true resid norm 1.246374917365e+16 ||r(i)||/||b|| 1.785463959348e-02 10 KSP preconditioned resid norm 3.195342481960e+16 true resid norm 5.030521484214e+15 ||r(i)||/||b|| 7.206350738975e-03 11 KSP preconditioned resid norm 2.437460220226e+16 true resid norm 4.974645944208e+15 ||r(i)||/||b|| 7.126307598263e-03 12 KSP preconditioned resid norm 1.258045555577e+16 true resid norm 1.885910304237e+15 ||r(i)||/||b|| 2.701614764439e-03 13 KSP preconditioned resid norm 2.495567167302e+15 true resid norm 4.460991814541e+14 ||r(i)||/||b|| 6.390484914966e-04 14 KSP preconditioned resid norm 7.678096351052e+14 true resid norm 1.316276220022e+14 ||r(i)||/||b|| 1.885599363926e-04 15 KSP preconditioned resid norm 5.950867501843e+14 true resid norm 1.439410416931e+14 ||r(i)||/||b|| 2.061992251557e-04 16 KSP preconditioned resid norm 3.263035989760e+14 true resid norm 5.046543721443e+13 ||r(i)||/||b|| 7.229303003757e-05 17 KSP preconditioned resid norm 1.436782488882e+14 true resid norm 7.588970356348e+13 ||r(i)||/||b|| 1.087139420975e-04 18 KSP preconditioned resid norm 5.401533739797e+13 true resid norm 8.625318785849e+12 ||r(i)||/||b|| 1.235598985142e-05 19 KSP preconditioned resid norm 3.528859989845e+13 true resid norm 1.233587622450e+13 ||r(i)||/||b|| 1.767145832202e-05 0 KSP preconditioned resid norm 4.827874033841e+18 true resid norm 2.629049503905e+18 ||r(i)||/||b|| 2.387244450275e+01 1 KSP preconditioned resid norm 1.161059892070e+18 true resid norm 3.885117260269e+17 ||r(i)||/||b|| 3.527786222537e+00 2 KSP preconditioned resid norm 4.696931552127e+17 true resid norm 1.228954734891e+17 ||r(i)||/||b|| 1.115922452639e+00 3 KSP preconditioned resid norm 2.870101730937e+17 true resid norm 6.759381587642e+16 ||r(i)||/||b|| 6.137692028400e-01 4 KSP preconditioned resid norm 1.966639305793e+17 true resid norm 3.310590026030e+16 ||r(i)||/||b|| 3.006100742887e-01 5 KSP preconditioned resid norm 1.592574497021e+17 true resid norm 3.239415772705e+16 ||r(i)||/||b|| 2.941472693472e-01 6 KSP preconditioned resid norm 1.230114824013e+17 true resid norm 2.280737281616e+16 ||r(i)||/||b|| 2.070968009535e-01 7 KSP preconditioned resid norm 8.598640818077e+16 true resid norm 1.938029194893e+16 ||r(i)||/||b|| 1.759780267776e-01 8 KSP preconditioned resid norm 5.549085768793e+16 true resid norm 9.386094648379e+15 ||r(i)||/||b|| 8.522814928290e-02 9 KSP preconditioned resid norm 4.474754629124e+16 true resid norm 7.579167807628e+15 ||r(i)||/||b|| 6.882078964122e-02 10 KSP preconditioned resid norm 4.378389530908e+16 true resid norm 6.842593278076e+15 ||r(i)||/||b|| 6.213250379770e-02 11 KSP preconditioned resid norm 4.079498014536e+16 true resid norm 7.704710526729e+15 ||r(i)||/||b|| 6.996074976369e-02 12 KSP preconditioned resid norm 4.084398125729e+16 true resid norm 6.377957983596e+15 ||r(i)||/||b|| 5.791349602893e-02 13 KSP preconditioned resid norm 7.523960017073e+16 true resid norm 1.132262466824e+16 ||r(i)||/||b|| 1.028123390664e-01 14 KSP preconditioned resid norm 2.184546755774e+15 true resid norm 3.502101254985e+14 ||r(i)||/||b|| 3.179997855820e-03 15 KSP preconditioned resid norm 2.222188316359e+15 true resid norm 3.654675514798e+14 ||r(i)||/||b|| 3.318539201068e-03 16 KSP preconditioned resid norm 1.669487598659e+15 true resid norm 3.629534674524e+14 ||r(i)||/||b|| 3.295710672608e-03 17 KSP preconditioned resid norm 1.401917540926e+15 true resid norm 2.161165666331e+14 ||r(i)||/||b|| 1.962393912861e-03 18 KSP preconditioned resid norm 1.134393288674e+15 true resid norm 2.686682412462e+14 ||r(i)||/||b|| 2.439576610967e-03 19 KSP preconditioned resid norm 9.058009317761e+14 true resid norm 1.409929037370e+14 ||r(i)||/||b|| 1.280251765797e-03 20 KSP preconditioned resid norm 7.990098269893e+14 true resid norm 1.659151006791e+14 ||r(i)||/||b|| 1.506551712794e-03 21 KSP preconditioned resid norm 7.119180202197e+14 true resid norm 1.092130755589e+14 ||r(i)||/||b|| 9.916827664832e-04 22 KSP preconditioned resid norm 6.306959900843e+14 true resid norm 1.460783745609e+14 ||r(i)||/||b|| 1.326429146571e-03 23 KSP preconditioned resid norm 5.637228851054e+14 true resid norm 8.672640572399e+13 ||r(i)||/||b|| 7.874980309399e-04 24 KSP preconditioned resid norm 4.960976586203e+14 true resid norm 1.292271181476e+14 ||r(i)||/||b|| 1.173415411786e-03 25 KSP preconditioned resid norm 4.386404759546e+14 true resid norm 8.037967670881e+13 ||r(i)||/||b|| 7.298681019622e-04 26 KSP preconditioned resid norm 4.419824575339e+14 true resid norm 6.704822670651e+13 ||r(i)||/||b|| 6.088151130974e-04 27 KSP preconditioned resid norm 4.437076214979e+14 true resid norm 1.370393448160e+14 ||r(i)||/||b|| 1.244352435720e-03 28 KSP preconditioned resid norm 7.848825938920e+14 true resid norm 1.196984132854e+14 ||r(i)||/||b|| 1.086892325146e-03 29 KSP preconditioned resid norm 1.771840029177e+15 true resid norm 6.554951997284e+14 ||r(i)||/||b|| 5.952064711635e-03 30 KSP preconditioned resid norm 1.052090974187e+15 true resid norm 1.877636782214e+14 ||r(i)||/||b|| 1.704942406491e-03 31 KSP preconditioned resid norm 2.459238084389e+14 true resid norm 3.913335964196e+13 ||r(i)||/||b|| 3.553409530216e-04 32 KSP preconditioned resid norm 1.091607449298e+14 true resid norm 1.900052367477e+13 ||r(i)||/||b|| 1.725296333429e-04 33 KSP preconditioned resid norm 6.424117612547e+13 true resid norm 1.001417309383e+13 ||r(i)||/||b|| 9.093126282646e-05 34 KSP preconditioned resid norm 3.801393062657e+13 true resid norm 7.035998067262e+12 ||r(i)||/||b|| 6.388866894011e-05 35 KSP preconditioned resid norm 2.524483314829e+13 true resid norm 4.006295820236e+12 ||r(i)||/||b|| 3.637819466241e-05 36 KSP preconditioned resid norm 1.579134906712e+13 true resid norm 2.528645237998e+12 ||r(i)||/||b|| 2.296074798956e-05 37 KSP preconditioned resid norm 1.074219589287e+13 true resid norm 1.786181249150e+12 ||r(i)||/||b|| 1.621898434352e-05 38 KSP preconditioned resid norm 6.572405677048e+12 true resid norm 1.033900747383e+12 ||r(i)||/||b|| 9.388084239790e-06 0 KSP preconditioned resid norm 9.565416792628e+15 true resid norm 5.898678788482e+15 ||r(i)||/||b|| 1.120363504643e-01 1 KSP preconditioned resid norm 4.584205668003e+15 true resid norm 8.149165580746e+14 ||r(i)||/||b|| 1.547808931009e-02 2 KSP preconditioned resid norm 3.775899089492e+15 true resid norm 6.134705686008e+14 ||r(i)||/||b|| 1.165193191355e-02 3 KSP preconditioned resid norm 3.695846031998e+15 true resid norm 6.195110682482e+14 ||r(i)||/||b|| 1.176666193357e-02 4 KSP preconditioned resid norm 3.863984235758e+15 true resid norm 6.333337603322e+14 ||r(i)||/||b|| 1.202920275504e-02 5 KSP preconditioned resid norm 4.289553507880e+15 true resid norm 6.781260017663e+14 ||r(i)||/||b|| 1.287996263523e-02 6 KSP preconditioned resid norm 5.214022047398e+15 true resid norm 7.772905386519e+14 ||r(i)||/||b|| 1.476344081849e-02 7 KSP preconditioned resid norm 6.520856882448e+15 true resid norm 1.156631144536e+15 ||r(i)||/||b|| 2.196843342618e-02 8 KSP preconditioned resid norm 7.245755589579e+15 true resid norm 1.125518734778e+15 ||r(i)||/||b|| 2.137750095326e-02 9 KSP preconditioned resid norm 7.402293382659e+15 true resid norm 1.342363548018e+15 ||r(i)||/||b|| 2.549613537355e-02 10 KSP preconditioned resid norm 6.814780966449e+15 true resid norm 1.040341685629e+15 ||r(i)||/||b|| 1.975969363197e-02 11 KSP preconditioned resid norm 4.531806636058e+15 true resid norm 6.813426758243e+14 ||r(i)||/||b|| 1.294105842210e-02 12 KSP preconditioned resid norm 4.019547715129e+15 true resid norm 1.067806433320e+15 ||r(i)||/||b|| 2.028134436224e-02 13 KSP preconditioned resid norm 1.578855518519e+15 true resid norm 2.462140978868e+14 ||r(i)||/||b|| 4.676458907031e-03 14 KSP preconditioned resid norm 1.216192317102e+15 true resid norm 3.547355821423e+14 ||r(i)||/||b|| 6.737657944804e-03 15 KSP preconditioned resid norm 1.068138625918e+15 true resid norm 1.711364338225e+14 ||r(i)||/||b|| 3.250473905172e-03 16 KSP preconditioned resid norm 1.009888628733e+15 true resid norm 1.725105447584e+14 ||r(i)||/||b|| 3.276573033454e-03 17 KSP preconditioned resid norm 9.012147556868e+14 true resid norm 1.461953215911e+14 ||r(i)||/||b|| 2.776755757239e-03 18 KSP preconditioned resid norm 8.184497465452e+14 true resid norm 1.394097869305e+14 ||r(i)||/||b|| 2.647874940607e-03 19 KSP preconditioned resid norm 6.946418330368e+14 true resid norm 1.094539907094e+14 ||r(i)||/||b|| 2.078910566681e-03 20 KSP preconditioned resid norm 5.822189800664e+14 true resid norm 1.086460879389e+14 ||r(i)||/||b|| 2.063565693502e-03 21 KSP preconditioned resid norm 4.765744627590e+14 true resid norm 7.425312666441e+13 ||r(i)||/||b|| 1.410324179423e-03 22 KSP preconditioned resid norm 3.785099184439e+14 true resid norm 8.229511757789e+13 ||r(i)||/||b|| 1.563069454208e-03 23 KSP preconditioned resid norm 3.069654165781e+14 true resid norm 4.744586000483e+13 ||r(i)||/||b|| 9.011612922478e-04 0 KSP preconditioned resid norm 9.591954950201e+15 true resid norm 5.933293318674e+15 ||r(i)||/||b|| 5.704140552403e-01 1 KSP preconditioned resid norm 4.627616930759e+15 true resid norm 8.120897483902e+14 ||r(i)||/||b|| 7.807256134471e-02 2 KSP preconditioned resid norm 3.843607704549e+15 true resid norm 6.216441009903e+14 ||r(i)||/||b|| 5.976352651335e-02 3 KSP preconditioned resid norm 3.802257051246e+15 true resid norm 6.332758127660e+14 ||r(i)||/||b|| 6.088177425993e-02 4 KSP preconditioned resid norm 4.025094330467e+15 true resid norm 6.488587775849e+14 ||r(i)||/||b|| 6.237988697367e-02 5 KSP preconditioned resid norm 4.518903996358e+15 true resid norm 7.100173385586e+14 ||r(i)||/||b|| 6.825953945399e-02 6 KSP preconditioned resid norm 5.566041493723e+15 true resid norm 8.272295550982e+14 ||r(i)||/||b|| 7.952806979104e-02 7 KSP preconditioned resid norm 7.050235529471e+15 true resid norm 1.259531120704e+15 ||r(i)||/||b|| 1.210886122891e-01 8 KSP preconditioned resid norm 7.824426472314e+15 true resid norm 1.230246429464e+15 ||r(i)||/||b|| 1.182732450740e-01 9 KSP preconditioned resid norm 8.110342869221e+15 true resid norm 1.508507345577e+15 ||r(i)||/||b|| 1.450246509206e-01 10 KSP preconditioned resid norm 7.848071091100e+15 true resid norm 1.244716713436e+15 ||r(i)||/||b|| 1.196643870448e-01 11 KSP preconditioned resid norm 6.689698623315e+15 true resid norm 1.152050117302e+15 ||r(i)||/||b|| 1.107556198481e-01 12 KSP preconditioned resid norm 3.836259156428e+15 true resid norm 5.895445738700e+14 ||r(i)||/||b|| 5.667754703239e-02 13 KSP preconditioned resid norm 1.583619332521e+15 true resid norm 3.090404276347e+14 ||r(i)||/||b|| 2.971048186772e-02 14 KSP preconditioned resid norm 1.282447478166e+15 true resid norm 2.666239295029e+14 ||r(i)||/||b|| 2.563265098882e-02 15 KSP preconditioned resid norm 1.151484666969e+15 true resid norm 2.280596910606e+14 ||r(i)||/||b|| 2.192516806902e-02 16 KSP preconditioned resid norm 1.027737067851e+15 true resid norm 1.605668418840e+14 ||r(i)||/||b|| 1.543655074795e-02 17 KSP preconditioned resid norm 8.747586168174e+14 true resid norm 1.470663481317e+14 ||r(i)||/||b|| 1.413864232250e-02 18 KSP preconditioned resid norm 7.765674155007e+14 true resid norm 1.265290838281e+14 ||r(i)||/||b|| 1.216423391459e-02 19 KSP preconditioned resid norm 6.633620818064e+14 true resid norm 1.039325479930e+14 ||r(i)||/||b|| 9.991851571798e-03 20 KSP preconditioned resid norm 5.640296981466e+14 true resid norm 1.018948133357e+14 ||r(i)||/||b|| 9.795948145668e-03 21 KSP preconditioned resid norm 4.751732697839e+14 true resid norm 7.431313741469e+13 ||r(i)||/||b|| 7.144305159654e-03 22 KSP preconditioned resid norm 3.865078030089e+14 true resid norm 7.686807394274e+13 ||r(i)||/||b|| 7.389931260973e-03 23 KSP preconditioned resid norm 3.183297004119e+14 true resid norm 4.953673129366e+13 ||r(i)||/||b|| 4.762354777175e-03 24 KSP preconditioned resid norm 2.612111917568e+14 true resid norm 5.971714747402e+13 ||r(i)||/||b|| 5.741078087414e-03 25 KSP preconditioned resid norm 2.296313138344e+14 true resid norm 3.527848689641e+13 ||r(i)||/||b|| 3.391597834880e-03 26 KSP preconditioned resid norm 1.981337219705e+14 true resid norm 5.164126794406e+13 ||r(i)||/||b|| 4.964680403211e-03 27 KSP preconditioned resid norm 1.751049134665e+14 true resid norm 2.675806111216e+13 ||r(i)||/||b|| 2.572462430151e-03 28 KSP preconditioned resid norm 1.533798667157e+14 true resid norm 3.995390488198e+13 ||r(i)||/||b|| 3.841082461690e-03 29 KSP preconditioned resid norm 1.350471089065e+14 true resid norm 2.063214947275e+13 ||r(i)||/||b|| 1.983530463939e-03 30 KSP preconditioned resid norm 1.221880384902e+14 true resid norm 2.772903253208e+13 ||r(i)||/||b|| 2.665809533591e-03 31 KSP preconditioned resid norm 1.099123503540e+14 true resid norm 1.682484274214e+13 ||r(i)||/||b|| 1.617504185596e-03 32 KSP preconditioned resid norm 1.022845787224e+14 true resid norm 1.853354426028e+13 ||r(i)||/||b|| 1.781775073585e-03 33 KSP preconditioned resid norm 9.416346598487e+13 true resid norm 1.438140346990e+13 ||r(i)||/||b|| 1.382597190584e-03 34 KSP preconditioned resid norm 8.782252733286e+13 true resid norm 1.465353724876e+13 ||r(i)||/||b|| 1.408759546637e-03 35 KSP preconditioned resid norm 8.286435025538e+13 true resid norm 1.264742128603e+13 ||r(i)||/||b|| 1.215895873779e-03 36 KSP preconditioned resid norm 7.744963280138e+13 true resid norm 1.290903292754e+13 ||r(i)||/||b|| 1.241046654183e-03 37 KSP preconditioned resid norm 7.340074137876e+13 true resid norm 1.120214736846e+13 ||r(i)||/||b|| 1.076950348592e-03 38 KSP preconditioned resid norm 6.860608702874e+13 true resid norm 1.142914172047e+13 ||r(i)||/||b|| 1.098773097257e-03 39 KSP preconditioned resid norm 6.401516623396e+13 true resid norm 9.766911914727e+12 ||r(i)||/||b|| 9.389698997216e-04 40 KSP preconditioned resid norm 5.980495528832e+13 true resid norm 9.707430083091e+12 ||r(i)||/||b|| 9.332514443927e-04 41 KSP preconditioned resid norm 5.502664117143e+13 true resid norm 8.404607061129e+12 ||r(i)||/||b|| 8.080008418514e-04 0 KSP preconditioned resid norm 2.200733964807e+16 true resid norm 1.374508262416e+16 ||r(i)||/||b|| 1.042223674371e-01 1 KSP preconditioned resid norm 1.033612993855e+16 true resid norm 1.767505005693e+15 ||r(i)||/||b|| 1.340214251070e-02 2 KSP preconditioned resid norm 8.198008013445e+15 true resid norm 1.328272543525e+15 ||r(i)||/||b|| 1.007165346861e-02 3 KSP preconditioned resid norm 7.723889011566e+15 true resid norm 1.289831076731e+15 ||r(i)||/||b|| 9.780170267923e-03 4 KSP preconditioned resid norm 7.785469249486e+15 true resid norm 1.238697760343e+15 ||r(i)||/||b|| 9.392450860579e-03 5 KSP preconditioned resid norm 8.250177532689e+15 true resid norm 1.288999707195e+15 ||r(i)||/||b|| 9.773866391570e-03 6 KSP preconditioned resid norm 9.471871112821e+15 true resid norm 1.409248720218e+15 ||r(i)||/||b|| 1.068565696875e-02 7 KSP preconditioned resid norm 1.106232045691e+16 true resid norm 1.993194297637e+15 ||r(i)||/||b|| 1.511343613874e-02 8 KSP preconditioned resid norm 1.131471113452e+16 true resid norm 1.768387012694e+15 ||r(i)||/||b|| 1.340883034665e-02 9 KSP preconditioned resid norm 1.055003108779e+16 true resid norm 1.935924228946e+15 ||r(i)||/||b|| 1.467918468275e-02 10 KSP preconditioned resid norm 8.569038225337e+15 true resid norm 1.316932619063e+15 ||r(i)||/||b|| 9.985668261662e-03 11 KSP preconditioned resid norm 4.806783660704e+15 true resid norm 7.346602562794e+14 ||r(i)||/||b|| 5.570576275535e-03 12 KSP preconditioned resid norm 6.311127886872e+15 true resid norm 1.031465284608e+15 ||r(i)||/||b|| 7.821106415332e-03 13 KSP preconditioned resid norm 9.706706695861e+14 true resid norm 2.189061895566e+14 ||r(i)||/||b|| 1.659860616781e-03 14 KSP preconditioned resid norm 6.844878378158e+14 true resid norm 1.339168617782e+14 ||r(i)||/||b|| 1.015427317239e-03 0 KSP preconditioned resid norm 1.928571496492e+16 true resid norm 1.203387628158e+16 ||r(i)||/||b|| 4.946666172277e-01 1 KSP preconditioned resid norm 9.232456268915e+15 true resid norm 1.583223656595e+15 ||r(i)||/||b|| 6.508026775390e-02 2 KSP preconditioned resid norm 7.541947576336e+15 true resid norm 1.216650042172e+15 ||r(i)||/||b|| 5.001182882627e-02 3 KSP preconditioned resid norm 7.340367763150e+15 true resid norm 1.219887358936e+15 ||r(i)||/||b|| 5.014490253381e-02 4 KSP preconditioned resid norm 7.655525114233e+15 true resid norm 1.212298323935e+15 ||r(i)||/||b|| 4.983294633744e-02 5 KSP preconditioned resid norm 8.414914513850e+15 true resid norm 1.311466433986e+15 ||r(i)||/||b|| 5.390936796484e-02 6 KSP preconditioned resid norm 1.008643582787e+16 true resid norm 1.495807999597e+15 ||r(i)||/||b|| 6.148694451139e-02 7 KSP preconditioned resid norm 1.235608743666e+16 true resid norm 2.203234082057e+15 ||r(i)||/||b|| 9.056652443733e-02 8 KSP preconditioned resid norm 1.311751365507e+16 true resid norm 2.061480991281e+15 ||r(i)||/||b|| 8.473959716509e-02 9 KSP preconditioned resid norm 1.289621791383e+16 true resid norm 2.355792563000e+15 ||r(i)||/||b|| 9.683761996231e-02 10 KSP preconditioned resid norm 1.151686987354e+16 true resid norm 1.821530468335e+15 ||r(i)||/||b|| 7.487614911976e-02 11 KSP preconditioned resid norm 8.379707704664e+15 true resid norm 1.435040037498e+15 ||r(i)||/||b|| 5.898900606299e-02 12 KSP preconditioned resid norm 3.557464484347e+15 true resid norm 5.430083136124e+14 ||r(i)||/||b|| 2.232099444401e-02 13 KSP preconditioned resid norm 1.204545626126e+15 true resid norm 2.194281091480e+14 ||r(i)||/||b|| 9.019850124522e-03 14 KSP preconditioned resid norm 1.005673666245e+15 true resid norm 1.883222474780e+14 ||r(i)||/||b|| 7.741207149622e-03 15 KSP preconditioned resid norm 9.379382960505e+14 true resid norm 1.480571711312e+14 ||r(i)||/||b|| 6.086063898783e-03 16 KSP preconditioned resid norm 8.825746357103e+14 true resid norm 2.025244331207e+14 ||r(i)||/||b|| 8.325004669617e-03 17 KSP preconditioned resid norm 7.524485997458e+14 true resid norm 1.181787653170e+14 ||r(i)||/||b|| 4.857876938371e-03 18 KSP preconditioned resid norm 6.768878630597e+14 true resid norm 1.383948785570e+14 ||r(i)||/||b|| 5.688884015051e-03 19 KSP preconditioned resid norm 6.018014871695e+14 true resid norm 9.381957375927e+13 ||r(i)||/||b|| 3.856563761775e-03 20 KSP preconditioned resid norm 5.246326908653e+14 true resid norm 1.170575496120e+14 ||r(i)||/||b|| 4.811788050053e-03 21 KSP preconditioned resid norm 4.606732721936e+14 true resid norm 7.159556750064e+13 ||r(i)||/||b|| 2.943019884477e-03 22 KSP preconditioned resid norm 3.919505243136e+14 true resid norm 9.177422576855e+13 ||r(i)||/||b|| 3.772487330545e-03 23 KSP preconditioned resid norm 3.382362447667e+14 true resid norm 5.210255290014e+13 ||r(i)||/||b|| 2.141736626583e-03 24 KSP preconditioned resid norm 2.828412329842e+14 true resid norm 7.512251290427e+13 ||r(i)||/||b|| 3.087999117364e-03 25 KSP preconditioned resid norm 2.453088437412e+14 true resid norm 3.770320821115e+13 ||r(i)||/||b|| 1.549834652446e-03 26 KSP preconditioned resid norm 2.155681340535e+14 true resid norm 5.419527743350e+13 ||r(i)||/||b|| 2.227760526238e-03 27 KSP preconditioned resid norm 1.897133167426e+14 true resid norm 2.950727994670e+13 ||r(i)||/||b|| 1.212931395777e-03 28 KSP preconditioned resid norm 1.747505144272e+14 true resid norm 2.784171913979e+13 ||r(i)||/||b|| 1.144466562762e-03 29 KSP preconditioned resid norm 1.591031918648e+14 true resid norm 2.517709500033e+13 ||r(i)||/||b|| 1.034934058155e-03 30 KSP preconditioned resid norm 1.457115463937e+14 true resid norm 2.260424017892e+13 ||r(i)||/||b|| 9.291738391412e-04 31 KSP preconditioned resid norm 1.296458013341e+14 true resid norm 2.025545529149e+13 ||r(i)||/||b|| 8.326242779131e-04 0 KSP preconditioned resid norm 6.130010037612e+16 true resid norm 4.019535297216e+16 ||r(i)||/||b|| 5.748505993058e-02 1 KSP preconditioned resid norm 2.383231272358e+16 true resid norm 4.333028202730e+15 ||r(i)||/||b|| 6.196845343971e-03 2 KSP preconditioned resid norm 1.380314308485e+16 true resid norm 2.902938638700e+15 ||r(i)||/||b|| 4.151614285761e-03 3 KSP preconditioned resid norm 9.355422664357e+15 true resid norm 1.914374541766e+15 ||r(i)||/||b|| 2.737827313999e-03 4 KSP preconditioned resid norm 6.905389450100e+15 true resid norm 1.160495113546e+15 ||r(i)||/||b|| 1.659672728773e-03 5 KSP preconditioned resid norm 5.388850197186e+15 true resid norm 9.343171629150e+14 ||r(i)||/||b|| 1.336206156506e-03 6 KSP preconditioned resid norm 4.456941607624e+15 true resid norm 7.443205303349e+14 ||r(i)||/||b|| 1.064484004494e-03 7 KSP preconditioned resid norm 3.604897946036e+15 true resid norm 7.177354673946e+14 ||r(i)||/||b|| 1.026463591104e-03 8 KSP preconditioned resid norm 2.616552593607e+15 true resid norm 4.566784798486e+14 ||r(i)||/||b|| 6.531150454453e-04 9 KSP preconditioned resid norm 1.730753896325e+15 true resid norm 3.416577992536e+14 ||r(i)||/||b|| 4.886191465826e-04 10 KSP preconditioned resid norm 9.605913237294e+14 true resid norm 1.715828091570e+14 ||r(i)||/||b|| 2.453877709266e-04 11 KSP preconditioned resid norm 3.885469473062e+14 true resid norm 7.526615943221e+13 ||r(i)||/||b|| 1.076412909896e-04 12 KSP preconditioned resid norm 1.085402136107e+14 true resid norm 1.877490962768e+13 ||r(i)||/||b|| 2.685078560912e-05 13 KSP preconditioned resid norm 5.521535103061e+13 true resid norm 1.482831672696e+13 ||r(i)||/||b|| 2.120659759623e-05 14 KSP preconditioned resid norm 4.327778995114e+13 true resid norm 6.949075963503e+12 ||r(i)||/||b|| 9.938164954062e-06 0 KSP preconditioned resid norm 2.469229041475e+16 true resid norm 1.581397714280e+16 ||r(i)||/||b|| 1.421322438999e-01 1 KSP preconditioned resid norm 1.094866313351e+16 true resid norm 1.827538507817e+15 ||r(i)||/||b|| 1.642547896610e-02 2 KSP preconditioned resid norm 7.902647446500e+15 true resid norm 1.331972422409e+15 ||r(i)||/||b|| 1.197144952849e-02 3 KSP preconditioned resid norm 6.757114899989e+15 true resid norm 1.177404752006e+15 ||r(i)||/||b|| 1.058223227906e-02 4 KSP preconditioned resid norm 6.236820681061e+15 true resid norm 9.858934558297e+14 ||r(i)||/||b|| 8.860974558006e-03 5 KSP preconditioned resid norm 6.056208230839e+15 true resid norm 9.498735328449e+14 ||r(i)||/||b|| 8.537236106085e-03 6 KSP preconditioned resid norm 6.281150442401e+15 true resid norm 9.439017368432e+14 ||r(i)||/||b|| 8.483563032058e-03 7 KSP preconditioned resid norm 6.511637444792e+15 true resid norm 1.175605282966e+15 ||r(i)||/||b|| 1.056605908176e-02 8 KSP preconditioned resid norm 5.897282453446e+15 true resid norm 9.358326231383e+14 ||r(i)||/||b|| 8.411039768188e-03 9 KSP preconditioned resid norm 4.858643646559e+15 true resid norm 9.074077132105e+14 ||r(i)||/||b|| 8.155563476918e-03 10 KSP preconditioned resid norm 3.431227043388e+15 true resid norm 5.523077341519e+14 ||r(i)||/||b|| 4.964009804073e-03 11 KSP preconditioned resid norm 1.790602151882e+15 true resid norm 3.233768234561e+14 ||r(i)||/||b|| 2.906433538380e-03 12 KSP preconditioned resid norm 6.061523925825e+14 true resid norm 9.361733195934e+13 ||r(i)||/||b|| 8.414101866433e-04 13 KSP preconditioned resid norm 3.669173256365e+14 true resid norm 7.433445448820e+13 ||r(i)||/||b|| 6.681002963438e-04 14 KSP preconditioned resid norm 1.034315631221e+15 true resid norm 1.749781208578e+14 ||r(i)||/||b|| 1.572661495987e-03 15 KSP preconditioned resid norm 1.340579302998e+14 true resid norm 2.477734238014e+13 ||r(i)||/||b|| 2.226928266408e-04 16 KSP preconditioned resid norm 1.124182190968e+14 true resid norm 1.917839484668e+13 ||r(i)||/||b|| 1.723708254629e-04 17 KSP preconditioned resid norm 9.394060483803e+13 true resid norm 1.682558933028e+13 ||r(i)||/||b|| 1.512243722661e-04 18 KSP preconditioned resid norm 9.034672550576e+13 true resid norm 1.892870471110e+13 ||r(i)||/||b|| 1.701266702495e-04 19 KSP preconditioned resid norm 8.058964108831e+13 true resid norm 1.275370569496e+13 ||r(i)||/||b|| 1.146272561351e-04 20 KSP preconditioned resid norm 6.303390314758e+13 true resid norm 1.900069985546e+13 ||r(i)||/||b|| 1.707737453860e-04 21 KSP preconditioned resid norm 4.601826493253e+13 true resid norm 6.960266318594e+12 ||r(i)||/||b|| 6.255720879505e-05 22 KSP preconditioned resid norm 2.822787946131e+13 true resid norm 1.013300639920e+13 ||r(i)||/||b|| 9.107303772884e-05 23 KSP preconditioned resid norm 1.858182164082e+13 true resid norm 2.785683110677e+12 ||r(i)||/||b|| 2.503705347106e-05 24 KSP preconditioned resid norm 9.506925688468e+12 true resid norm 4.268960894748e+12 ||r(i)||/||b|| 3.836839939834e-05 25 KSP preconditioned resid norm 5.845171242291e+12 true resid norm 8.923161711832e+11 ||r(i)||/||b|| 8.019924307031e-06 0 SNES Function norm 0.000000000000e+00 5 TS dt 7e-12 time 2e-11 TS Object: 1 MPI processes type: theta maximum steps=10 maximum time=2e-11 total number of nonlinear solver iterations=0 total number of nonlinear solve failures=0 total number of linear solver iterations=0 total number of rejected steps=5 Theta=1 Extrapolation=no SNES Object: 1 MPI processes type: newtonls maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=0 total number of function evaluations=1 SNESLineSearch Object: 1 MPI processes type: bt interpolation: cubic alpha=1.000000e-04 maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=40 KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using DEFAULT norm type for convergence test PC Object: 1 MPI processes type: ilu PC has not been set up so information may be incomplete ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=20904, cols=20904, bs=4 total: nonzeros=410816, allocated nonzeros=410816 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 5226 nodes, limit used is 5 On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone wrote: > >> Hi Jed and Matt, >> >> Thank you very much for your replies. >> >> This time I copied the whole output from my screen, from compile to >> execution. This time I put the TSView() routine just after the TSSolve() >> routine to make the tsview output. And I intentionally to do only 5 time >> steps to make the test. Thank you in advance! >> > > You have to make the command line arguments work. Quit calling TSView() in > the code. We need many more options > for debugging. First track down why you are not processing options. Do the > examples take options for you? > > Once that works, run with -ts_view -ts_monitor -snes_monitor > -ksp_monitor_true_residual > > Matt > > >> host206-47:streamer_Implicit_SG GPL$ make pStreamer >> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o >> pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >> -Wno-unknown-pragmas -g -O0 -fPIC >> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >> -Wl,-commons,use_dylibs -Wl,-search_paths_first >> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >> -lpthread -lssl -lcrypto >> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >> -lmpichf90 -lgfortran >> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >> -lclang_rt.osx -lmpichcxx -lc++ >> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >> -lclang_rt.osx >> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >> -lclang_rt.osx -ldl >> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >> >> time=0.000000 step=0 >> dt = 0.001000 >> >> time=0.002000 step=1 >> dt = 0.002000 >> >> time=0.005000 step=2 >> dt = 0.003000 >> >> time=0.009000 step=3 >> dt = 0.004000 >> >> time=0.014000 step=4 >> dt = 0.005000 >> >> time=0.020000 step=5 >> dt = 0.006000 >> >> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >> TS Object: 1 MPI processes >> type: theta >> maximum steps=10 >> maximum time=2e-11 >> total number of nonlinear solver iterations=0 >> total number of nonlinear solve failures=0 >> total number of linear solver iterations=0 >> total number of rejected steps=5 >> Theta=1 >> Extrapolation=no >> SNES Object: 1 MPI processes >> type: newtonls >> maximum iterations=50, maximum function evaluations=10000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of linear solver iterations=0 >> total number of function evaluations=1 >> SNESLineSearch Object: 1 MPI processes >> type: bt >> interpolation: cubic >> alpha=1.000000e-04 >> maxstep=1.000000e+08, minlambda=1.000000e-12 >> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >> lambda=1.000000e-08 >> maximum iterations=40 >> KSP Object: 1 MPI processes >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: 1 MPI processes >> type: ilu >> PC has not been set up so information may be incomplete >> ILU: out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >> matrix ordering: natural >> linear system matrix = precond matrix: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=20904, cols=20904, bs=4 >> total: nonzeros=410816, allocated nonzeros=410816 >> total number of mallocs used during MatSetValues calls =0 >> using I-node routines: found 5226 nodes, limit used is 5 >> >> >> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >> >>> Sharp Stone writes: >>> >>> > Hi Matt, >>> > >>> > Thank you very much for your reply. >>> > >>> > The ts_view output is attached below. I found my code results say >>> "SNES has >>> > not been set up", and PC "has not been set up". Does this cause the >>> > problem? If so I do not see example ex17 explicitly set up the snes >>> object? >>> >>> Those objects are set up on the first time step (inside TSSolve). >>> >>> > PS: I don't know why the -ts_view option does not work in my command >>> line, >>> >>> Are you sure it doesn't output after the TSSolve? The example does. >>> >>> > so I use TSView() routine just before the TSSolve() routine. >>> >>> You should call it *after* the solve so we can see what happened during >>> the solve. >>> >> >> >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 14:39:28 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 14:39:28 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: On Mon, Oct 27, 2014 at 2:31 PM, Sharp Stone wrote: > Hi Matt, > > Thank you for your reply and suggestion. I use the command line options to > make the output on the screen. It was very long and attached below. Thanks > a million in advance! > Notice that you never actually solve anything. There is something wrong with your formulation. Try something where you know the exact solution first. Also run with -snes_converged_reason -ksp_converged_reason Matt > > > 0 KSP preconditioned resid norm 5.607581107219e-11 true resid norm > 9.787719465322e-12 ||r(i)||/||b|| 6.796750851600e-16 > 0 TS dt 2e-12 time 0 > 0 KSP preconditioned resid norm 3.622586114352e+17 true resid norm > 5.225681998280e+16 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 3.704153274287e+17 true resid norm > 1.035782762954e+17 ||r(i)||/||b|| 1.982100639294e+00 > 2 KSP preconditioned resid norm 2.842418683024e+17 true resid norm > 7.903974218109e+16 ||r(i)||/||b|| 1.512524914587e+00 > 3 KSP preconditioned resid norm 2.393970839759e+17 true resid norm > 4.943510895366e+16 ||r(i)||/||b|| 9.460030091754e-01 > 4 KSP preconditioned resid norm 2.271787411557e+17 true resid norm > 4.669216610635e+16 ||r(i)||/||b|| 8.935133466162e-01 > 5 KSP preconditioned resid norm 2.360179849286e+17 true resid norm > 4.108243474851e+16 ||r(i)||/||b|| 7.861640789094e-01 > 6 KSP preconditioned resid norm 2.619090922116e+17 true resid norm > 4.955060344596e+16 ||r(i)||/||b|| 9.482131416009e-01 > 7 KSP preconditioned resid norm 2.333689240438e+17 true resid norm > 4.336784470641e+16 ||r(i)||/||b|| 8.298982739609e-01 > 8 KSP preconditioned resid norm 3.244607818183e+17 true resid norm > 5.407474341938e+16 ||r(i)||/||b|| 1.034788252274e+00 > 9 KSP preconditioned resid norm 3.024748487424e+17 true resid norm > 6.388129145873e+16 ||r(i)||/||b|| 1.222448887624e+00 > 10 KSP preconditioned resid norm 2.127860348138e+17 true resid norm > 3.704786520226e+16 ||r(i)||/||b|| 7.089575143389e-01 > 11 KSP preconditioned resid norm 4.652609350461e+16 true resid norm > 8.497318285406e+15 ||r(i)||/||b|| 1.626068767331e-01 > 12 KSP preconditioned resid norm 2.920260091494e+16 true resid norm > 4.651858168992e+15 ||r(i)||/||b|| 8.901915904036e-02 > 13 KSP preconditioned resid norm 2.527529714994e+16 true resid norm > 3.813962069025e+15 ||r(i)||/||b|| 7.298496292505e-02 > 14 KSP preconditioned resid norm 2.224122069397e+16 true resid norm > 4.355442499437e+15 ||r(i)||/||b|| 8.334687225268e-02 > 15 KSP preconditioned resid norm 2.040909225547e+16 true resid norm > 5.536568280983e+15 ||r(i)||/||b|| 1.059492001772e-01 > 16 KSP preconditioned resid norm 1.834652616545e+16 true resid norm > 3.131287565910e+15 ||r(i)||/||b|| 5.992112736559e-02 > 17 KSP preconditioned resid norm 1.760978175265e+16 true resid norm > 3.245011634605e+15 ||r(i)||/||b|| 6.209738050790e-02 > 18 KSP preconditioned resid norm 1.385338263500e+16 true resid norm > 3.596318213647e+15 ||r(i)||/||b|| 6.882007391247e-02 > 19 KSP preconditioned resid norm 1.488783377120e+16 true resid norm > 2.508274919240e+15 ||r(i)||/||b|| 4.799899649588e-02 > 20 KSP preconditioned resid norm 1.320707251811e+16 true resid norm > 2.042806338080e+15 ||r(i)||/||b|| 3.909166954193e-02 > 21 KSP preconditioned resid norm 7.616453043037e+15 true resid norm > 1.348500002677e+15 ||r(i)||/||b|| 2.580524423647e-02 > 22 KSP preconditioned resid norm 8.879333117289e+15 true resid norm > 1.392661876801e+15 ||r(i)||/||b|| 2.665033726238e-02 > 23 KSP preconditioned resid norm 8.820610261864e+15 true resid norm > 1.365093600480e+15 ||r(i)||/||b|| 2.612278360851e-02 > 24 KSP preconditioned resid norm 7.993346710634e+15 true resid norm > 1.790642691323e+15 ||r(i)||/||b|| 3.426620088847e-02 > 25 KSP preconditioned resid norm 7.525623400757e+15 true resid norm > 1.148549131012e+15 ||r(i)||/||b|| 2.197893272859e-02 > 26 KSP preconditioned resid norm 7.109494224141e+15 true resid norm > 1.253098862904e+15 ||r(i)||/||b|| 2.397962339302e-02 > 27 KSP preconditioned resid norm 9.199926655972e+15 true resid norm > 1.867276980934e+15 ||r(i)||/||b|| 3.573269444158e-02 > 28 KSP preconditioned resid norm 1.121259577312e+16 true resid norm > 3.202544820818e+15 ||r(i)||/||b|| 6.128472459426e-02 > 29 KSP preconditioned resid norm 4.340241766754e+15 true resid norm > 6.879443404198e+14 ||r(i)||/||b|| 1.316468052680e-02 > 30 KSP preconditioned resid norm 4.050053526588e+15 true resid norm > 6.673524276162e+14 ||r(i)||/||b|| 1.277062836652e-02 > 31 KSP preconditioned resid norm 3.675002682668e+15 true resid norm > 5.703629309888e+14 ||r(i)||/||b|| 1.091461231618e-02 > 32 KSP preconditioned resid norm 3.175854297440e+15 true resid norm > 4.918363244864e+14 ||r(i)||/||b|| 9.411906898435e-03 > 33 KSP preconditioned resid norm 2.534108954658e+15 true resid norm > 3.888660709280e+14 ||r(i)||/||b|| 7.441441539229e-03 > 34 KSP preconditioned resid norm 1.517593193539e+15 true resid norm > 2.347716358056e+14 ||r(i)||/||b|| 4.492650641255e-03 > 35 KSP preconditioned resid norm 1.238737221149e+15 true resid norm > 4.944967490035e+14 ||r(i)||/||b|| 9.462817468922e-03 > 36 KSP preconditioned resid norm 8.673662184926e+14 true resid norm > 1.322920275286e+14 ||r(i)||/||b|| 2.531574396837e-03 > 37 KSP preconditioned resid norm 7.206551748402e+14 true resid norm > 1.263301396525e+14 ||r(i)||/||b|| 2.417486171069e-03 > 38 KSP preconditioned resid norm 5.143409845488e+14 true resid norm > 7.953540049469e+13 ||r(i)||/||b|| 1.522009960056e-03 > 39 KSP preconditioned resid norm 4.261910201431e+14 true resid norm > 7.194412501613e+13 ||r(i)||/||b|| 1.376741352417e-03 > 40 KSP preconditioned resid norm 4.660682952219e+14 true resid norm > 7.611539723637e+13 ||r(i)||/||b|| 1.456563894654e-03 > 41 KSP preconditioned resid norm 3.209169783039e+14 true resid norm > 4.903825175831e+13 ||r(i)||/||b|| 9.384086474158e-04 > 0 KSP preconditioned resid norm 5.647841057638e+16 true resid norm > 8.145493103696e+15 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 5.871590984109e+16 true resid norm > 1.629681152051e+16 ||r(i)||/||b|| 2.000715157823e+00 > 2 KSP preconditioned resid norm 4.554798588549e+16 true resid norm > 1.256325647853e+16 ||r(i)||/||b|| 1.542356775532e+00 > 3 KSP preconditioned resid norm 3.879559549735e+16 true resid norm > 7.930640596410e+15 ||r(i)||/||b|| 9.736231429392e-01 > 4 KSP preconditioned resid norm 3.752982759961e+16 true resid norm > 7.738765648717e+15 ||r(i)||/||b|| 9.500671782787e-01 > 5 KSP preconditioned resid norm 3.847367502586e+16 true resid norm > 6.717373650268e+15 ||r(i)||/||b|| 8.246736649031e-01 > 6 KSP preconditioned resid norm 4.072409022879e+16 true resid norm > 7.643006165053e+15 ||r(i)||/||b|| 9.383110473184e-01 > 7 KSP preconditioned resid norm 3.699615223300e+16 true resid norm > 6.864744274739e+15 ||r(i)||/||b|| 8.427659550315e-01 > 8 KSP preconditioned resid norm 4.452919179646e+16 true resid norm > 7.438964756180e+15 ||r(i)||/||b|| 9.132614393603e-01 > 9 KSP preconditioned resid norm 4.218921641123e+16 true resid norm > 8.817640722426e+15 ||r(i)||/||b|| 1.082517732220e+00 > 10 KSP preconditioned resid norm 2.723066166891e+18 true resid norm > 4.777495288747e+17 ||r(i)||/||b|| 5.865200826920e+01 > 11 KSP preconditioned resid norm 1.543090703715e+16 true resid norm > 2.978277930467e+15 ||r(i)||/||b|| 3.656350686879e-01 > 12 KSP preconditioned resid norm 5.543831189057e+15 true resid norm > 9.036431670746e+14 ||r(i)||/||b|| 1.109378100958e-01 > 13 KSP preconditioned resid norm 4.835578439031e+15 true resid norm > 7.302084941558e+14 ||r(i)||/||b|| 8.964570773800e-02 > 14 KSP preconditioned resid norm 4.632268883924e+15 true resid norm > 7.764942849443e+14 ||r(i)||/||b|| 9.532808819050e-02 > 15 KSP preconditioned resid norm 4.279662320804e+15 true resid norm > 7.279053691611e+14 ||r(i)||/||b|| 8.936295935612e-02 > 16 KSP preconditioned resid norm 4.003909782590e+15 true resid norm > 6.398763789786e+14 ||r(i)||/||b|| 7.855588002257e-02 > 17 KSP preconditioned resid norm 3.778065018292e+15 true resid norm > 9.619695351776e+14 ||r(i)||/||b|| 1.180983794267e-01 > 18 KSP preconditioned resid norm 4.432967840881e+15 true resid norm > 1.765362859847e+15 ||r(i)||/||b|| 2.167287894512e-01 > 19 KSP preconditioned resid norm 3.349713073090e+15 true resid norm > 5.343310168427e+14 ||r(i)||/||b|| 6.559836341894e-02 > 20 KSP preconditioned resid norm 3.170973083244e+15 true resid norm > 4.905381423867e+14 ||r(i)||/||b|| 6.022203151385e-02 > 21 KSP preconditioned resid norm 7.415620721710e+15 true resid norm > 1.207250421775e+15 ||r(i)||/||b|| 1.482108457286e-01 > 22 KSP preconditioned resid norm 1.972990201442e+15 true resid norm > 3.077655556372e+14 ||r(i)||/||b|| 3.778353891154e-02 > 23 KSP preconditioned resid norm 2.209676808077e+15 true resid norm > 3.428403970507e+14 ||r(i)||/||b|| 4.208958164793e-02 > 24 KSP preconditioned resid norm 2.003292171306e+15 true resid norm > 4.370066270665e+14 ||r(i)||/||b|| 5.365011319796e-02 > 25 KSP preconditioned resid norm 1.896876965875e+15 true resid norm > 2.911542588080e+14 ||r(i)||/||b|| 3.574421524903e-02 > 26 KSP preconditioned resid norm 1.776666236555e+15 true resid norm > 2.752957912495e+14 ||r(i)||/||b|| 3.379731438537e-02 > 27 KSP preconditioned resid norm 2.106240629900e+15 true resid norm > 3.329282684307e+14 ||r(i)||/||b|| 4.087269661791e-02 > 28 KSP preconditioned resid norm 1.993718847257e+15 true resid norm > 3.265440954005e+14 ||r(i)||/||b|| 4.008892908550e-02 > 29 KSP preconditioned resid norm 1.305019272390e+15 true resid norm > 2.016059917794e+14 ||r(i)||/||b|| 2.475061843560e-02 > 30 KSP preconditioned resid norm 1.188165021558e+15 true resid norm > 2.349019593212e+14 ||r(i)||/||b|| 2.883827367242e-02 > 31 KSP preconditioned resid norm 1.072198412006e+15 true resid norm > 1.660526922017e+14 ||r(i)||/||b|| 2.038583669371e-02 > 32 KSP preconditioned resid norm 9.531727749289e+14 true resid norm > 1.481127635008e+14 ||r(i)||/||b|| 1.818340051551e-02 > 33 KSP preconditioned resid norm 8.113851256730e+14 true resid norm > 1.246949236714e+14 ||r(i)||/||b|| 1.530845610990e-02 > 34 KSP preconditioned resid norm 1.978771889204e+15 true resid norm > 3.123099443158e+14 ||r(i)||/||b|| 3.834144113069e-02 > 35 KSP preconditioned resid norm 5.701229171299e+14 true resid norm > 9.665965877556e+13 ||r(i)||/||b|| 1.186664300676e-02 > 36 KSP preconditioned resid norm 4.923209980376e+14 true resid norm > 8.358449899120e+13 ||r(i)||/||b|| 1.026144125679e-02 > 37 KSP preconditioned resid norm 4.337271696165e+14 true resid norm > 7.056948273999e+13 ||r(i)||/||b|| 8.663623164566e-03 > 38 KSP preconditioned resid norm 3.355561922811e+14 true resid norm > 5.206223795577e+13 ||r(i)||/||b|| 6.391539136181e-03 > 39 KSP preconditioned resid norm 2.948796430248e+14 true resid norm > 4.783720648479e+13 ||r(i)||/||b|| 5.872843531485e-03 > 40 KSP preconditioned resid norm 2.903844355011e+14 true resid norm > 4.553096332971e+13 ||r(i)||/||b|| 5.589712341546e-03 > 41 KSP preconditioned resid norm 2.470740134237e+14 true resid norm > 3.845088511805e+13 ||r(i)||/||b|| 4.720510425650e-03 > 42 KSP preconditioned resid norm 2.144333147632e+14 true resid norm > 4.199300915233e+13 ||r(i)||/||b|| 5.155367344584e-03 > 43 KSP preconditioned resid norm 1.666875797521e+14 true resid norm > 2.786670988964e+13 ||r(i)||/||b|| 3.421120064173e-03 > 44 KSP preconditioned resid norm 1.097596670806e+14 true resid norm > 1.683182082477e+13 ||r(i)||/||b|| 2.066396792741e-03 > 45 KSP preconditioned resid norm 1.086486049833e+14 true resid norm > 1.868163126673e+13 ||r(i)||/||b|| 2.293492981813e-03 > 46 KSP preconditioned resid norm 9.986155841385e+13 true resid norm > 1.544797785039e+13 ||r(i)||/||b|| 1.896506160367e-03 > 47 KSP preconditioned resid norm 9.193832565781e+13 true resid norm > 1.424417809082e+13 ||r(i)||/||b|| 1.748718943038e-03 > 48 KSP preconditioned resid norm 7.868838175665e+13 true resid norm > 1.234477959732e+13 ||r(i)||/||b|| 1.515534963957e-03 > 49 KSP preconditioned resid norm 6.573261298706e+13 true resid norm > 1.037919549254e+13 ||r(i)||/||b|| 1.274225557668e-03 > 50 KSP preconditioned resid norm 6.394833480259e+13 true resid norm > 1.032066032237e+13 ||r(i)||/||b|| 1.267039354277e-03 > 51 KSP preconditioned resid norm 6.650072664422e+13 true resid norm > 2.391163155719e+13 ||r(i)||/||b|| 2.935565870940e-03 > 52 KSP preconditioned resid norm 6.239223696051e+13 true resid norm > 9.858797480463e+12 ||r(i)||/||b|| 1.210337711291e-03 > 53 KSP preconditioned resid norm 6.247432830217e+13 true resid norm > 9.562224327829e+12 ||r(i)||/||b|| 1.173928233208e-03 > 54 KSP preconditioned resid norm 6.218810094746e+13 true resid norm > 9.709963371483e+12 ||r(i)||/||b|| 1.192065753156e-03 > 55 KSP preconditioned resid norm 6.285033387840e+13 true resid norm > 9.623710764220e+12 ||r(i)||/||b|| 1.181476755515e-03 > 56 KSP preconditioned resid norm 5.407058856328e+13 true resid norm > 9.490503427247e+12 ||r(i)||/||b|| 1.165123253611e-03 > 0 KSP preconditioned resid norm 9.075431914728e+17 true resid norm > 1.310345788054e+17 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 8.641655528143e+17 true resid norm > 2.471138002109e+17 ||r(i)||/||b|| 1.885867092974e+00 > 2 KSP preconditioned resid norm 6.384433252903e+17 true resid norm > 1.816534419632e+17 ||r(i)||/||b|| 1.386301567259e+00 > 3 KSP preconditioned resid norm 5.179227150533e+17 true resid norm > 1.107056703740e+17 ||r(i)||/||b|| 8.448584441089e-01 > 4 KSP preconditioned resid norm 4.625114083211e+17 true resid norm > 9.438923102711e+16 ||r(i)||/||b|| 7.203383403647e-01 > 5 KSP preconditioned resid norm 4.977065165555e+17 true resid norm > 8.614996614554e+16 ||r(i)||/||b|| 6.574597860420e-01 > 6 KSP preconditioned resid norm 6.630931668382e+17 true resid norm > 1.310543864565e+17 ||r(i)||/||b|| 1.000151163542e+00 > 7 KSP preconditioned resid norm 5.452278853431e+17 true resid norm > 1.019521297074e+17 ||r(i)||/||b|| 7.780551564087e-01 > 8 KSP preconditioned resid norm 1.958476842674e+18 true resid norm > 3.244825770314e+17 ||r(i)||/||b|| 2.476312588551e+00 > 9 KSP preconditioned resid norm 1.577830260132e+18 true resid norm > 3.469004540765e+17 ||r(i)||/||b|| 2.647396261651e+00 > 10 KSP preconditioned resid norm 1.081897998863e+17 true resid norm > 1.828781621979e+16 ||r(i)||/||b|| 1.395648109568e-01 > 11 KSP preconditioned resid norm 5.614465792481e+16 true resid norm > 9.930812946447e+15 ||r(i)||/||b|| 7.578772746079e-02 > 12 KSP preconditioned resid norm 4.960081715579e+16 true resid norm > 8.035214375965e+15 ||r(i)||/||b|| 6.132132792135e-02 > 13 KSP preconditioned resid norm 3.329600867897e+16 true resid norm > 6.602562228940e+15 ||r(i)||/||b|| 5.038793797127e-02 > 14 KSP preconditioned resid norm 3.130960732609e+16 true resid norm > 4.730473520325e+15 ||r(i)||/||b|| 3.610095566719e-02 > 15 KSP preconditioned resid norm 2.560518921527e+16 true resid norm > 3.945110867578e+15 ||r(i)||/||b|| 3.010740297365e-02 > 16 KSP preconditioned resid norm 1.584886017619e+16 true resid norm > 3.416780637867e+15 ||r(i)||/||b|| 2.607541207073e-02 > 17 KSP preconditioned resid norm 1.443824757417e+16 true resid norm > 2.286337927483e+15 ||r(i)||/||b|| 1.744835560449e-02 > 18 KSP preconditioned resid norm 1.039145582094e+16 true resid norm > 1.711526931765e+15 ||r(i)||/||b|| 1.306164332627e-02 > 19 KSP preconditioned resid norm 1.018242603191e+16 true resid norm > 1.657426513724e+15 ||r(i)||/||b|| 1.264877201754e-02 > 20 KSP preconditioned resid norm 8.198983478967e+15 true resid norm > 1.311419876212e+15 ||r(i)||/||b|| 1.000819698256e-02 > 21 KSP preconditioned resid norm 6.481494503642e+15 true resid norm > 1.039872641976e+15 ||r(i)||/||b|| 7.935864345546e-03 > 22 KSP preconditioned resid norm 6.325821399407e+15 true resid norm > 1.165377948147e+15 ||r(i)||/||b|| 8.893667295849e-03 > 23 KSP preconditioned resid norm 5.405294478066e+15 true resid norm > 1.922243319095e+15 ||r(i)||/||b|| 1.466974089297e-02 > 24 KSP preconditioned resid norm 4.960676496180e+15 true resid norm > 8.018184223519e+14 ||r(i)||/||b|| 6.119136106374e-03 > 25 KSP preconditioned resid norm 4.419138742847e+15 true resid norm > 7.148418830282e+14 ||r(i)||/||b|| 5.455368266491e-03 > 26 KSP preconditioned resid norm 2.795870478230e+15 true resid norm > 4.843283048525e+14 ||r(i)||/||b|| 3.696186985663e-03 > 27 KSP preconditioned resid norm 2.911201681995e+15 true resid norm > 4.441742607418e+14 ||r(i)||/||b|| 3.389748452592e-03 > 28 KSP preconditioned resid norm 2.776697567476e+15 true resid norm > 5.074732182061e+14 ||r(i)||/||b|| 3.872819089681e-03 > 29 KSP preconditioned resid norm 1.941501506934e+15 true resid norm > 2.950418833153e+14 ||r(i)||/||b|| 2.251633774878e-03 > 30 KSP preconditioned resid norm 8.413150178496e+14 true resid norm > 1.695476752862e+14 ||r(i)||/||b|| 1.293915520864e-03 > 0 KSP preconditioned resid norm 1.415973479575e+17 true resid norm > 2.042710742436e+16 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 1.440919303250e+17 true resid norm > 4.045863378014e+16 ||r(i)||/||b|| 1.980634503928e+00 > 2 KSP preconditioned resid norm 1.100352174605e+17 true resid norm > 3.072773288453e+16 ||r(i)||/||b|| 1.504262558872e+00 > 3 KSP preconditioned resid norm 9.218323125716e+16 true resid norm > 1.914173874994e+16 ||r(i)||/||b|| 9.370753456316e-01 > 4 KSP preconditioned resid norm 8.672799672395e+16 true resid norm > 1.779286318286e+16 ||r(i)||/||b|| 8.710417394506e-01 > 5 KSP preconditioned resid norm 9.059066321416e+16 true resid norm > 1.574500663517e+16 ||r(i)||/||b|| 7.707898288327e-01 > 6 KSP preconditioned resid norm 1.027848583976e+17 true resid norm > 1.954996111922e+16 ||r(i)||/||b|| 9.570596909823e-01 > 7 KSP preconditioned resid norm 9.069106612200e+16 true resid norm > 1.686428629204e+16 ||r(i)||/||b|| 8.255836688812e-01 > 8 KSP preconditioned resid norm 1.359707392060e+17 true resid norm > 2.263432282464e+16 ||r(i)||/||b|| 1.108053252691e+00 > 9 KSP preconditioned resid norm 1.256447896822e+17 true resid norm > 2.668588264948e+16 ||r(i)||/||b|| 1.306395570116e+00 > 10 KSP preconditioned resid norm 5.661761773340e+16 true resid norm > 9.807578010522e+15 ||r(i)||/||b|| 4.801256392682e-01 > 11 KSP preconditioned resid norm 1.597811133832e+16 true resid norm > 2.854210532053e+15 ||r(i)||/||b|| 1.397266129148e-01 > 12 KSP preconditioned resid norm 1.141170314391e+16 true resid norm > 1.817769027418e+15 ||r(i)||/||b|| 8.898807793264e-02 > 13 KSP preconditioned resid norm 9.557798166843e+15 true resid norm > 1.455009507586e+15 ||r(i)||/||b|| 7.122934624852e-02 > 14 KSP preconditioned resid norm 7.997842098400e+15 true resid norm > 2.021581420109e+15 ||r(i)||/||b|| 9.896562338034e-02 > 15 KSP preconditioned resid norm 7.518666379332e+15 true resid norm > 2.232979183117e+15 ||r(i)||/||b|| 1.093145072735e-01 > 16 KSP preconditioned resid norm 6.614897634500e+15 true resid norm > 1.222952133628e+15 ||r(i)||/||b|| 5.986908024822e-02 > 17 KSP preconditioned resid norm 6.199437182741e+15 true resid norm > 1.163257046730e+15 ||r(i)||/||b|| 5.694673369870e-02 > 18 KSP preconditioned resid norm 4.584922270319e+15 true resid norm > 9.962391827866e+14 ||r(i)||/||b|| 4.877044811535e-02 > 19 KSP preconditioned resid norm 5.012002077991e+15 true resid norm > 8.755757394511e+14 ||r(i)||/||b|| 4.286342266977e-02 > 20 KSP preconditioned resid norm 4.312819580189e+15 true resid norm > 6.706322150678e+14 ||r(i)||/||b|| 3.283050317090e-02 > 21 KSP preconditioned resid norm 2.607093537931e+15 true resid norm > 5.192589254045e+14 ||r(i)||/||b|| 2.542009079490e-02 > 22 KSP preconditioned resid norm 2.989363098445e+15 true resid norm > 4.636989016003e+14 ||r(i)||/||b|| 2.270017442838e-02 > 23 KSP preconditioned resid norm 3.016276187245e+15 true resid norm > 4.674022912685e+14 ||r(i)||/||b|| 2.288147222994e-02 > 24 KSP preconditioned resid norm 2.901859180715e+15 true resid norm > 4.492447164798e+14 ||r(i)||/||b|| 2.199257619530e-02 > 25 KSP preconditioned resid norm 2.843000955863e+15 true resid norm > 5.109023428494e+14 ||r(i)||/||b|| 2.501099799574e-02 > 26 KSP preconditioned resid norm 2.741940909323e+15 true resid norm > 4.263494245202e+14 ||r(i)||/||b|| 2.087174731415e-02 > 27 KSP preconditioned resid norm 1.296293793548e+15 true resid norm > 2.124064840417e+14 ||r(i)||/||b|| 1.039826538477e-02 > 28 KSP preconditioned resid norm 9.720512789714e+14 true resid norm > 2.091841490290e+14 ||r(i)||/||b|| 1.024051740089e-02 > 29 KSP preconditioned resid norm 1.459179957917e+15 true resid norm > 2.244723575034e+14 ||r(i)||/||b|| 1.098894487800e-02 > 30 KSP preconditioned resid norm 1.339547884067e+15 true resid norm > 2.746667338979e+14 ||r(i)||/||b|| 1.344618835119e-02 > 31 KSP preconditioned resid norm 1.186062213325e+15 true resid norm > 1.831856176134e+14 ||r(i)||/||b|| 8.967770806105e-03 > 32 KSP preconditioned resid norm 9.909356538944e+14 true resid norm > 1.788908155328e+14 ||r(i)||/||b|| 8.757520671747e-03 > 33 KSP preconditioned resid norm 7.609550150745e+14 true resid norm > 1.203572113192e+14 ||r(i)||/||b|| 5.892033992812e-03 > 34 KSP preconditioned resid norm 5.051711143811e+14 true resid norm > 7.920683224412e+13 ||r(i)||/||b|| 3.877535404237e-03 > 35 KSP preconditioned resid norm 5.443461822227e+14 true resid norm > 8.557198028123e+13 ||r(i)||/||b|| 4.189138408269e-03 > 36 KSP preconditioned resid norm 4.648220720251e+14 true resid norm > 7.162139608842e+13 ||r(i)||/||b|| 3.506193735634e-03 > 37 KSP preconditioned resid norm 3.842876907480e+14 true resid norm > 6.998069618798e+13 ||r(i)||/||b|| 3.425873998417e-03 > 38 KSP preconditioned resid norm 2.822934553902e+14 true resid norm > 5.594039593962e+13 ||r(i)||/||b|| 2.738537316003e-03 > 39 KSP preconditioned resid norm 2.288268021627e+14 true resid norm > 3.624972614121e+13 ||r(i)||/||b|| 1.774589293929e-03 > 40 KSP preconditioned resid norm 3.336880274482e+14 true resid norm > 5.633095049288e+13 ||r(i)||/||b|| 2.757656741242e-03 > 41 KSP preconditioned resid norm 1.655073694079e+14 true resid norm > 2.539321805860e+13 ||r(i)||/||b|| 1.243113747388e-03 > 42 KSP preconditioned resid norm 1.317678812236e+14 true resid norm > 2.429513710603e+13 ||r(i)||/||b|| 1.189357680523e-03 > 0 KSP preconditioned resid norm 4.735355646927e+18 true resid norm > 6.977368406398e+17 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 1.655056377336e+18 true resid norm > 5.433322725028e+17 ||r(i)||/||b|| 7.787065851426e-01 > 2 KSP preconditioned resid norm 8.847192945348e+17 true resid norm > 2.725627800798e+17 ||r(i)||/||b|| 3.906383670810e-01 > 3 KSP preconditioned resid norm 5.344946494044e+17 true resid norm > 1.588736353868e+17 ||r(i)||/||b|| 2.276985048419e-01 > 4 KSP preconditioned resid norm 3.198139750974e+17 true resid norm > 6.624530455539e+16 ||r(i)||/||b|| 9.494310848578e-02 > 5 KSP preconditioned resid norm 3.286368493485e+17 true resid norm > 6.812427209313e+16 ||r(i)||/||b|| 9.763605434775e-02 > 6 KSP preconditioned resid norm 4.284030910438e+17 true resid norm > 1.696553569581e+17 ||r(i)||/||b|| 2.431509232084e-01 > 7 KSP preconditioned resid norm 1.587968810131e+18 true resid norm > 3.732963000458e+17 ||r(i)||/||b|| 5.350101618592e-01 > 8 KSP preconditioned resid norm 6.621891209840e+16 true resid norm > 1.229792367414e+16 ||r(i)||/||b|| 1.762544695628e-02 > 9 KSP preconditioned resid norm 5.638286198157e+16 true resid norm > 2.091986582514e+16 ||r(i)||/||b|| 2.998245843800e-02 > 10 KSP preconditioned resid norm 1.309067125882e+16 true resid norm > 2.099826104432e+15 ||r(i)||/||b|| 3.009481486611e-03 > 11 KSP preconditioned resid norm 1.075925809079e+16 true resid norm > 2.578010577039e+15 ||r(i)||/||b|| 3.694817912546e-03 > 12 KSP preconditioned resid norm 9.319606428298e+15 true resid norm > 2.026298982436e+15 ||r(i)||/||b|| 2.904102040216e-03 > 13 KSP preconditioned resid norm 4.070270220193e+15 true resid norm > 2.216532493559e+15 ||r(i)||/||b|| 3.176745678968e-03 > 14 KSP preconditioned resid norm 3.400982828163e+15 true resid norm > 9.161069365806e+14 ||r(i)||/||b|| 1.312969135671e-03 > 15 KSP preconditioned resid norm 4.173720697476e+14 true resid norm > 7.019272258636e+13 ||r(i)||/||b|| 1.006005681483e-04 > 16 KSP preconditioned resid norm 2.871344308441e+14 true resid norm > 8.503915375563e+13 ||r(i)||/||b|| 1.218785490496e-04 > 17 KSP preconditioned resid norm 8.731930916187e+13 true resid norm > 2.939915802842e+13 ||r(i)||/||b|| 4.213502328681e-05 > 18 KSP preconditioned resid norm 7.273170647297e+13 true resid norm > 2.554194915512e+13 ||r(i)||/||b|| 3.660685184933e-05 > 19 KSP preconditioned resid norm 5.198384943683e+13 true resid norm > 8.213409786802e+12 ||r(i)||/||b|| 1.177150081293e-05 > 20 KSP preconditioned resid norm 2.826075121950e+13 true resid norm > 1.225537290577e+13 ||r(i)||/||b|| 1.756446297795e-05 > 0 KSP preconditioned resid norm 7.535755573391e+17 true resid norm > 1.091530949677e+17 ||r(i)||/||b|| 1.000000000000e+00 > 1 KSP preconditioned resid norm 5.749658535033e+17 true resid norm > 1.739986551510e+17 ||r(i)||/||b|| 1.594078987889e+00 > 2 KSP preconditioned resid norm 3.832130567981e+17 true resid norm > 1.138657236506e+17 ||r(i)||/||b|| 1.043174485197e+00 > 3 KSP preconditioned resid norm 2.820261830557e+17 true resid norm > 6.703201541200e+16 ||r(i)||/||b|| 6.141100756863e-01 > 4 KSP preconditioned resid norm 2.196333397214e+17 true resid norm > 4.464718002673e+16 ||r(i)||/||b|| 4.090326530819e-01 > 5 KSP preconditioned resid norm 2.438749906836e+17 true resid norm > 4.259961207836e+16 ||r(i)||/||b|| 3.902739733672e-01 > 6 KSP preconditioned resid norm 7.293863369143e+17 true resid norm > 1.861518427200e+17 ||r(i)||/||b|| 1.705419738901e+00 > 7 KSP preconditioned resid norm 3.735172838302e+17 true resid norm > 7.178953918634e+16 ||r(i)||/||b|| 6.576958647631e-01 > 8 KSP preconditioned resid norm 2.306462817079e+17 true resid norm > 3.812340554474e+16 ||r(i)||/||b|| 3.492654565226e-01 > 9 KSP preconditioned resid norm 2.324630150277e+17 true resid norm > 5.879635462988e+16 ||r(i)||/||b|| 5.386595281360e-01 > 10 KSP preconditioned resid norm 2.486084300490e+16 true resid norm > 3.943582532938e+15 ||r(i)||/||b|| 3.612891172811e-02 > 11 KSP preconditioned resid norm 2.044336976942e+16 true resid norm > 4.427406712026e+15 ||r(i)||/||b|| 4.056143999707e-02 > 12 KSP preconditioned resid norm 1.903964203761e+16 true resid norm > 3.227987898262e+15 ||r(i)||/||b|| 2.957303133931e-02 > 13 KSP preconditioned resid norm 6.823460749064e+15 true resid norm > 1.876444880951e+15 ||r(i)||/||b|| 1.719094526368e-02 > 14 KSP preconditioned resid norm 7.090194359812e+15 true resid norm > 1.450790953432e+15 ||r(i)||/||b|| 1.329134051455e-02 > 15 KSP preconditioned resid norm 4.884314111756e+15 true resid norm > 8.008003608330e+14 ||r(i)||/||b|| 7.336487903253e-03 > 16 KSP preconditioned resid norm 1.603299777176e+15 true resid norm > 2.941609026573e+14 ||r(i)||/||b|| 2.694938725689e-03 > 17 KSP preconditioned resid norm 1.548805654760e+15 true resid norm > 2.838534773138e+14 ||r(i)||/||b|| 2.600507822503e-03 > 18 KSP preconditioned resid norm 1.022454633110e+15 true resid norm > 2.215058790848e+14 ||r(i)||/||b|| 2.029313773928e-03 > 19 KSP preconditioned resid norm 9.587995256078e+14 true resid norm > 2.226912518453e+14 ||r(i)||/||b|| 2.040173500451e-03 > 20 KSP preconditioned resid norm 7.709554287637e+14 true resid norm > 1.253194818485e+14 ||r(i)||/||b|| 1.148107452983e-03 > 21 KSP preconditioned resid norm 3.692994290799e+14 true resid norm > 6.513442758709e+13 ||r(i)||/||b|| 5.967254305192e-04 > 22 KSP preconditioned resid norm 3.577186864519e+14 true resid norm > 5.884014914924e+13 ||r(i)||/||b|| 5.390607491859e-04 > 23 KSP preconditioned resid norm 1.723423887037e+14 true resid norm > 4.313917181929e+13 ||r(i)||/||b|| 3.952171198816e-04 > 24 KSP preconditioned resid norm 1.656260736170e+14 true resid norm > 3.420915571267e+13 ||r(i)||/||b|| 3.134052746997e-04 > 25 KSP preconditioned resid norm 1.188400863617e+14 true resid norm > 1.877160245085e+13 ||r(i)||/||b|| 1.719749903236e-04 > 26 KSP preconditioned resid norm 8.416275062676e+13 true resid norm > 1.813244070153e+13 ||r(i)||/||b|| 1.661193455568e-04 > 27 KSP preconditioned resid norm 5.224466611453e+13 true resid norm > 8.293327735079e+12 ||r(i)||/||b|| 7.597886012791e-05 > 28 KSP preconditioned resid norm 4.471999764526e+13 true resid norm > 9.742703301619e+12 ||r(i)||/||b|| 8.925723365429e-05 > 29 KSP preconditioned resid norm 9.410327220658e+14 true resid norm > 1.612776349391e+14 ||r(i)||/||b|| 1.477536069746e-03 > 30 KSP preconditioned resid norm 5.783713583723e+14 true resid norm > 1.102180319983e+14 ||r(i)||/||b|| 1.009756361291e-03 > 31 KSP preconditioned resid norm 1.833775181615e+13 true resid norm > 4.198845362007e+12 ||r(i)||/||b|| 3.846748791915e-05 > 32 KSP preconditioned resid norm 1.462922686654e+13 true resid norm > 3.704838629537e+12 ||r(i)||/||b|| 3.394167275452e-05 > 33 KSP preconditioned resid norm 1.053043620710e+13 true resid norm > 1.904310347574e+12 ||r(i)||/||b|| 1.744623318411e-05 > 34 KSP preconditioned resid norm 8.014167149326e+13 true resid norm > 1.458222541169e+13 ||r(i)||/||b|| 1.335942459168e-04 > 35 KSP preconditioned resid norm 5.245679087504e+12 true resid norm > 1.495201096266e+12 ||r(i)||/||b|| 1.369820156459e-05 > 0 KSP preconditioned resid norm 1.124208361458e+16 true resid norm > 6.487794900041e+15 ||r(i)||/||b|| 1.232063762752e-01 > 1 KSP preconditioned resid norm 5.136632948471e+15 true resid norm > 1.030125368299e+15 ||r(i)||/||b|| 1.956258107611e-02 > 2 KSP preconditioned resid norm 4.225773164973e+15 true resid norm > 7.107211019673e+14 ||r(i)||/||b|| 1.349693892375e-02 > 3 KSP preconditioned resid norm 4.144329493920e+15 true resid norm > 7.618007419548e+14 ||r(i)||/||b|| 1.446696609651e-02 > 4 KSP preconditioned resid norm 4.333716562490e+15 true resid norm > 8.456727780045e+14 ||r(i)||/||b|| 1.605973679776e-02 > 5 KSP preconditioned resid norm 4.885745055980e+15 true resid norm > 7.972847407816e+14 ||r(i)||/||b|| 1.514082446881e-02 > 6 KSP preconditioned resid norm 6.011621624545e+15 true resid norm > 9.523510473075e+14 ||r(i)||/||b|| 1.808560894548e-02 > 7 KSP preconditioned resid norm 7.637951309150e+15 true resid norm > 1.487731882292e+15 ||r(i)||/||b|| 2.825275103643e-02 > 8 KSP preconditioned resid norm 8.460249904060e+15 true resid norm > 1.336944180115e+15 ||r(i)||/||b|| 2.538921933447e-02 > 9 KSP preconditioned resid norm 8.381050292195e+15 true resid norm > 1.752911449852e+15 ||r(i)||/||b|| 3.328863982216e-02 > 10 KSP preconditioned resid norm 7.434169027728e+15 true resid norm > 1.180498665014e+15 ||r(i)||/||b|| 2.241824301708e-02 > 11 KSP preconditioned resid norm 5.682629267055e+15 true resid norm > 9.983854763086e+14 ||r(i)||/||b|| 1.895982511114e-02 > 12 KSP preconditioned resid norm 3.541898147790e+15 true resid norm > 5.480576756264e+14 ||r(i)||/||b|| 1.040788145188e-02 > 13 KSP preconditioned resid norm 2.510644084392e+15 true resid norm > 4.650369826671e+14 ||r(i)||/||b|| 8.831278169411e-03 > 14 KSP preconditioned resid norm 2.282038802861e+15 true resid norm > 3.635360161467e+14 ||r(i)||/||b|| 6.903725516148e-03 > 15 KSP preconditioned resid norm 2.130130680848e+15 true resid norm > 3.489368266499e+14 ||r(i)||/||b|| 6.626479816775e-03 > 16 KSP preconditioned resid norm 1.929984942209e+15 true resid norm > 3.145683546766e+14 ||r(i)||/||b|| 5.973805841227e-03 > 17 KSP preconditioned resid norm 1.692158321133e+15 true resid norm > 2.756020615583e+14 ||r(i)||/||b|| 5.233817008974e-03 > 18 KSP preconditioned resid norm 1.473781775606e+15 true resid norm > 2.951884895649e+14 ||r(i)||/||b|| 5.605772789952e-03 > 19 KSP preconditioned resid norm 1.275239549902e+15 true resid norm > 1.993708462138e+14 ||r(i)||/||b|| 3.786149204063e-03 > 20 KSP preconditioned resid norm 1.159341081209e+15 true resid norm > 2.011812256624e+14 ||r(i)||/||b|| 3.820529189093e-03 > 21 KSP preconditioned resid norm 1.026465449352e+15 true resid norm > 1.599378003014e+14 ||r(i)||/||b|| 3.037296509548e-03 > 22 KSP preconditioned resid norm 9.313333291328e+14 true resid norm > 1.626450968383e+14 ||r(i)||/||b|| 3.088709385718e-03 > 23 KSP preconditioned resid norm 8.475207473455e+14 true resid norm > 1.311587742521e+14 ||r(i)||/||b|| 2.490768826892e-03 > 24 KSP preconditioned resid norm 7.894756750239e+14 true resid norm > 1.443612018407e+14 ||r(i)||/||b|| 2.741489339222e-03 > 25 KSP preconditioned resid norm 7.370222420808e+14 true resid norm > 1.135397001576e+14 ||r(i)||/||b|| 2.156174052249e-03 > 26 KSP preconditioned resid norm 6.874978587995e+14 true resid norm > 1.225152011526e+14 ||r(i)||/||b|| 2.326623175549e-03 > 27 KSP preconditioned resid norm 6.203646083349e+14 true resid norm > 9.547951255722e+13 ||r(i)||/||b|| 1.813202317882e-03 > 28 KSP preconditioned resid norm 5.655995299941e+14 true resid norm > 9.635247011837e+13 ||r(i)||/||b|| 1.829780205963e-03 > 29 KSP preconditioned resid norm 5.170668664072e+14 true resid norm > 7.945307015395e+13 ||r(i)||/||b|| 1.508852392597e-03 > 30 KSP preconditioned resid norm 4.771681972381e+14 true resid norm > 7.711681382438e+13 ||r(i)||/||b|| 1.464485750178e-03 > 31 KSP preconditioned resid norm 4.347723667483e+14 true resid norm > 6.696963877348e+13 ||r(i)||/||b|| 1.271785967477e-03 > 32 KSP preconditioned resid norm 4.053435837983e+14 true resid norm > 6.425971653178e+13 ||r(i)||/||b|| 1.220323227897e-03 > 33 KSP preconditioned resid norm 3.774514402447e+14 true resid norm > 5.808746381526e+13 ||r(i)||/||b|| 1.103109150945e-03 > 34 KSP preconditioned resid norm 3.533225997981e+14 true resid norm > 5.617172311710e+13 ||r(i)||/||b|| 1.066728304612e-03 > 0 KSP preconditioned resid norm 1.117659190837e+16 true resid norm > 6.478857863821e+15 ||r(i)||/||b|| 6.224768870229e-01 > 1 KSP preconditioned resid norm 5.124485271748e+15 true resid norm > 1.017923791746e+15 ||r(i)||/||b|| 9.780026764454e-02 > 2 KSP preconditioned resid norm 4.246328499709e+15 true resid norm > 7.102918780385e+14 ||r(i)||/||b|| 6.824355255393e-02 > 3 KSP preconditioned resid norm 4.210000472667e+15 true resid norm > 7.697163273299e+14 ||r(i)||/||b|| 7.395294562682e-02 > 4 KSP preconditioned resid norm 4.471596919519e+15 true resid norm > 8.475637573508e+14 ||r(i)||/||b|| 8.143238520100e-02 > 5 KSP preconditioned resid norm 5.135403291038e+15 true resid norm > 8.255731593574e+14 ||r(i)||/||b|| 7.931956851781e-02 > 6 KSP preconditioned resid norm 6.474268664840e+15 true resid norm > 1.007942498042e+15 ||r(i)||/||b|| 9.684128308834e-02 > 7 KSP preconditioned resid norm 8.529281532903e+15 true resid norm > 1.642696055823e+15 ||r(i)||/||b|| 1.578272511369e-01 > 8 KSP preconditioned resid norm 9.784577605090e+15 true resid norm > 1.538078623395e+15 ||r(i)||/||b|| 1.477757983909e-01 > 9 KSP preconditioned resid norm 1.010006302597e+16 true resid norm > 2.046930405602e+15 ||r(i)||/||b|| 1.966653526923e-01 > 10 KSP preconditioned resid norm 9.531884877717e+15 true resid norm > 1.482620346274e+15 ||r(i)||/||b|| 1.424474679309e-01 > 11 KSP preconditioned resid norm 7.931210548314e+15 true resid norm > 1.282982557872e+15 ||r(i)||/||b|| 1.232666321002e-01 > 12 KSP preconditioned resid norm 5.236526900965e+15 true resid norm > 7.915538690853e+14 ||r(i)||/||b|| 7.605105694487e-02 > 13 KSP preconditioned resid norm 3.885416517845e+15 true resid norm > 6.196979959619e+14 ||r(i)||/||b|| 5.953945703529e-02 > 14 KSP preconditioned resid norm 3.337626627490e+15 true resid norm > 7.828597862542e+14 ||r(i)||/||b|| 7.521574526958e-02 > 15 KSP preconditioned resid norm 3.054301414200e+15 true resid norm > 4.972852337202e+14 ||r(i)||/||b|| 4.777826134714e-02 > 16 KSP preconditioned resid norm 2.784835626497e+15 true resid norm > 4.900149140724e+14 ||r(i)||/||b|| 4.707974225054e-02 > 17 KSP preconditioned resid norm 2.551999131175e+15 true resid norm > 4.223064940199e+14 ||r(i)||/||b|| 4.057444032459e-02 > 18 KSP preconditioned resid norm 2.358735848485e+15 true resid norm > 3.867160573888e+14 ||r(i)||/||b|| 3.715497586533e-02 > 19 KSP preconditioned resid norm 2.111367549832e+15 true resid norm > 3.463614183500e+14 ||r(i)||/||b|| 3.327777549857e-02 > 20 KSP preconditioned resid norm 1.933287196882e+15 true resid norm > 3.127201079650e+14 ||r(i)||/||b|| 3.004557954614e-02 > 21 KSP preconditioned resid norm 1.712007284199e+15 true resid norm > 2.787684847304e+14 ||r(i)||/||b|| 2.678356929917e-02 > 22 KSP preconditioned resid norm 1.556420935486e+15 true resid norm > 2.516757515483e+14 ||r(i)||/||b|| 2.418054874112e-02 > 23 KSP preconditioned resid norm 1.424455851932e+15 true resid norm > 2.267048417314e+14 ||r(i)||/||b|| 2.178138911521e-02 > 24 KSP preconditioned resid norm 1.341093049266e+15 true resid norm > 2.200423231461e+14 ||r(i)||/||b|| 2.114126643990e-02 > 25 KSP preconditioned resid norm 1.265242012282e+15 true resid norm > 1.979405460182e+14 ||r(i)||/||b|| 1.901776786755e-02 > 26 KSP preconditioned resid norm 1.190343462827e+15 true resid norm > 1.929502180424e+14 ||r(i)||/||b|| 1.853830622649e-02 > 27 KSP preconditioned resid norm 1.080874612432e+15 true resid norm > 1.688981747210e+14 ||r(i)||/||b|| 1.622742962325e-02 > 28 KSP preconditioned resid norm 9.890419339482e+14 true resid norm > 1.571112460256e+14 ||r(i)||/||b|| 1.509496293914e-02 > 29 KSP preconditioned resid norm 9.054720544549e+14 true resid norm > 1.411421952126e+14 ||r(i)||/||b|| 1.356068556375e-02 > 30 KSP preconditioned resid norm 8.347895469778e+14 true resid norm > 1.302020126144e+14 ||r(i)||/||b|| 1.250957270554e-02 > 31 KSP preconditioned resid norm 7.581633555103e+14 true resid norm > 1.198729260222e+14 ||r(i)||/||b|| 1.151717284080e-02 > 32 KSP preconditioned resid norm 7.073149016328e+14 true resid norm > 1.097657041094e+14 ||r(i)||/||b|| 1.054608933119e-02 > 33 KSP preconditioned resid norm 6.626892210136e+14 true resid norm > 1.045304064903e+14 ||r(i)||/||b|| 1.004309145207e-02 > 34 KSP preconditioned resid norm 6.277118615935e+14 true resid norm > 9.735143814138e+13 ||r(i)||/||b|| 9.353349222222e-03 > 35 KSP preconditioned resid norm 6.007031548284e+14 true resid norm > 9.325545854852e+13 ||r(i)||/||b|| 8.959814948147e-03 > 36 KSP preconditioned resid norm 5.720600717487e+14 true resid norm > 8.974565981239e+13 ||r(i)||/||b|| 8.622599865294e-03 > 37 KSP preconditioned resid norm 5.429834406497e+14 true resid norm > 8.373407335142e+13 ||r(i)||/||b|| 8.045017565303e-03 > 38 KSP preconditioned resid norm 5.122271874419e+14 true resid norm > 8.063599233502e+13 ||r(i)||/||b|| 7.747359572588e-03 > 39 KSP preconditioned resid norm 4.877390474820e+14 true resid norm > 7.540978455988e+13 ||r(i)||/||b|| 7.245235029160e-03 > 40 KSP preconditioned resid norm 4.664511103016e+14 true resid norm > 7.249582547116e+13 ||r(i)||/||b|| 6.965267136580e-03 > 41 KSP preconditioned resid norm 4.430412921029e+14 true resid norm > 6.907345898754e+13 ||r(i)||/||b|| 6.636452385624e-03 > 42 KSP preconditioned resid norm 4.184290025581e+14 true resid norm > 6.506826627971e+13 ||r(i)||/||b|| 6.251640750440e-03 > 43 KSP preconditioned resid norm 3.961557540211e+14 true resid norm > 6.169515842630e+13 ||r(i)||/||b|| 5.927558679138e-03 > 44 KSP preconditioned resid norm 3.628856632080e+14 true resid norm > 5.666822624927e+13 ||r(i)||/||b|| 5.444580172956e-03 > 45 KSP preconditioned resid norm 3.439726178254e+14 true resid norm > 5.290740915351e+13 ||r(i)||/||b|| 5.083247702381e-03 > 46 KSP preconditioned resid norm 3.808130490503e+14 true resid norm > 7.046303156753e+13 ||r(i)||/||b|| 6.769959993302e-03 > 47 KSP preconditioned resid norm 2.548925306353e+14 true resid norm > 3.911087819219e+13 ||r(i)||/||b|| 3.757702085387e-03 > 48 KSP preconditioned resid norm 1.722230117103e+14 true resid norm > 2.660198748286e+13 ||r(i)||/||b|| 2.555870603278e-03 > 49 KSP preconditioned resid norm 8.498499404656e+13 true resid norm > 1.314954336818e+13 ||r(i)||/||b|| 1.263384225066e-03 > 50 KSP preconditioned resid norm 1.168755498979e+14 true resid norm > 2.563042385986e+13 ||r(i)||/||b|| 2.462524536378e-03 > 51 KSP preconditioned resid norm 5.069372366763e+13 true resid norm > 7.752336990767e+12 ||r(i)||/||b|| 7.448304467540e-04 > 0 KSP preconditioned resid norm 2.551555721110e+16 true resid norm > 1.503348678660e+16 ||r(i)||/||b|| 1.139781099215e-01 > 1 KSP preconditioned resid norm 1.135435821514e+16 true resid norm > 2.150528625386e+15 ||r(i)||/||b|| 1.630448022690e-02 > 2 KSP preconditioned resid norm 8.921910642422e+15 true resid norm > 1.462019059252e+15 ||r(i)||/||b|| 1.108446572695e-02 > 3 KSP preconditioned resid norm 8.401630743802e+15 true resid norm > 1.508644008325e+15 ||r(i)||/||b|| 1.143795814332e-02 > 4 KSP preconditioned resid norm 8.507938797573e+15 true resid norm > 1.437822857483e+15 ||r(i)||/||b|| 1.090101943907e-02 > 5 KSP preconditioned resid norm 9.195685552195e+15 true resid norm > 1.421948919345e+15 ||r(i)||/||b|| 1.078066935052e-02 > 6 KSP preconditioned resid norm 1.067998630800e+16 true resid norm > 1.627316531548e+15 ||r(i)||/||b|| 1.233768753335e-02 > 7 KSP preconditioned resid norm 1.295276293823e+16 true resid norm > 2.221334900964e+15 ||r(i)||/||b|| 1.684130615262e-02 > 8 KSP preconditioned resid norm 1.417997817727e+16 true resid norm > 2.186215590047e+15 ||r(i)||/||b|| 1.657504505585e-02 > 9 KSP preconditioned resid norm 1.359372910192e+16 true resid norm > 2.107067442870e+15 ||r(i)||/||b|| 1.597497427073e-02 > 10 KSP preconditioned resid norm 1.179811021994e+16 true resid norm > 3.232410469947e+15 ||r(i)||/||b|| 2.450689191967e-02 > 11 KSP preconditioned resid norm 7.523875759744e+15 true resid norm > 1.204652282825e+15 ||r(i)||/||b|| 9.133209897209e-03 > 12 KSP preconditioned resid norm 3.961884677960e+15 true resid norm > 7.586571607759e+14 ||r(i)||/||b|| 5.751846560354e-03 > 13 KSP preconditioned resid norm 2.175818220311e+15 true resid norm > 3.487257134817e+14 ||r(i)||/||b|| 2.643904123366e-03 > 14 KSP preconditioned resid norm 1.788448374434e+15 true resid norm > 3.516091940967e+14 ||r(i)||/||b|| 2.665765563440e-03 > 15 KSP preconditioned resid norm 1.604495215225e+15 true resid norm > 2.568394922627e+14 ||r(i)||/||b|| 1.947258164179e-03 > 16 KSP preconditioned resid norm 1.391089499920e+15 true resid norm > 2.519771358465e+14 ||r(i)||/||b|| 1.910393649516e-03 > 17 KSP preconditioned resid norm 1.231055803595e+15 true resid norm > 2.082047844556e+14 ||r(i)||/||b|| 1.578528530720e-03 > 18 KSP preconditioned resid norm 1.101068654831e+15 true resid norm > 1.838589386072e+14 ||r(i)||/||b|| 1.393947699032e-03 > 19 KSP preconditioned resid norm 9.356667448723e+14 true resid norm > 1.565965285410e+14 ||r(i)||/||b|| 1.187254600129e-03 > 20 KSP preconditioned resid norm 8.181513038456e+14 true resid norm > 1.328185221533e+14 ||r(i)||/||b|| 1.006978908651e-03 > 0 KSP preconditioned resid norm 2.272810589988e+16 true resid norm > 1.323636987884e+16 ||r(i)||/||b|| 5.438005326146e-01 > 1 KSP preconditioned resid norm 1.026969580669e+16 true resid norm > 1.999385093393e+15 ||r(i)||/||b|| 8.214236143606e-02 > 2 KSP preconditioned resid norm 8.322628960001e+15 true resid norm > 1.379596337850e+15 ||r(i)||/||b|| 5.667907667911e-02 > 3 KSP preconditioned resid norm 8.076035080201e+15 true resid norm > 1.472961452514e+15 ||r(i)||/||b|| 6.051487150403e-02 > 4 KSP preconditioned resid norm 8.405382352119e+15 true resid norm > 1.516410203314e+15 ||r(i)||/||b|| 6.229991181666e-02 > 5 KSP preconditioned resid norm 9.416373856303e+15 true resid norm > 1.470293081560e+15 ||r(i)||/||b|| 6.040524465323e-02 > 6 KSP preconditioned resid norm 1.145894310132e+16 true resid norm > 1.749278072065e+15 ||r(i)||/||b|| 7.186701157399e-02 > 7 KSP preconditioned resid norm 1.439168534642e+16 true resid norm > 2.685145033101e+15 ||r(i)||/||b|| 1.103159939254e-01 > 8 KSP preconditioned resid norm 1.556059742979e+16 true resid norm > 2.450465189043e+15 ||r(i)||/||b|| 1.006744513151e-01 > 9 KSP preconditioned resid norm 1.521075410830e+16 true resid norm > 2.889962061237e+15 ||r(i)||/||b|| 1.187306582184e-01 > 10 KSP preconditioned resid norm 1.335253374250e+16 true resid norm > 2.051263914753e+15 ||r(i)||/||b|| 8.427374118333e-02 > 11 KSP preconditioned resid norm 9.966633192923e+15 true resid norm > 1.537352185287e+15 ||r(i)||/||b|| 6.316028826844e-02 > 12 KSP preconditioned resid norm 5.467457389741e+15 true resid norm > 8.244964108686e+14 ||r(i)||/||b|| 3.387345559797e-02 > 13 KSP preconditioned resid norm 7.465607339270e+15 true resid norm > 1.255988023601e+15 ||r(i)||/||b|| 5.160077592599e-02 > 14 KSP preconditioned resid norm 3.395725692367e+15 true resid norm > 8.163584192182e+14 ||r(i)||/||b|| 3.353911587836e-02 > 15 KSP preconditioned resid norm 2.979451058224e+15 true resid norm > 4.736799198293e+14 ||r(i)||/||b|| 1.946057680844e-02 > 16 KSP preconditioned resid norm 2.589762095203e+15 true resid norm > 5.000061827618e+14 ||r(i)||/||b|| 2.054216004731e-02 > 17 KSP preconditioned resid norm 2.297387399767e+15 true resid norm > 3.733072000155e+14 ||r(i)||/||b|| 1.533688285047e-02 > 18 KSP preconditioned resid norm 2.076822069911e+15 true resid norm > 3.563941616114e+14 ||r(i)||/||b|| 1.464203075912e-02 > 19 KSP preconditioned resid norm 1.810897092306e+15 true resid norm > 2.945299484347e+14 ||r(i)||/||b|| 1.210041305100e-02 > 20 KSP preconditioned resid norm 1.630399807803e+15 true resid norm > 2.670493324597e+14 ||r(i)||/||b|| 1.097140458866e-02 > 21 KSP preconditioned resid norm 1.416015877821e+15 true resid norm > 2.289562555093e+14 ||r(i)||/||b|| 9.406395773998e-03 > 22 KSP preconditioned resid norm 1.270953330458e+15 true resid norm > 2.078621859272e+14 ||r(i)||/||b|| 8.539770983461e-03 > 23 KSP preconditioned resid norm 1.149572826465e+15 true resid norm > 1.837793084226e+14 ||r(i)||/||b|| 7.550354569913e-03 > 24 KSP preconditioned resid norm 1.075403245776e+15 true resid norm > 1.776412893147e+14 ||r(i)||/||b|| 7.298181346390e-03 > 25 KSP preconditioned resid norm 1.008390645992e+15 true resid norm > 1.577927704554e+14 ||r(i)||/||b|| 6.482728527673e-03 > 26 KSP preconditioned resid norm 9.445856732256e+14 true resid norm > 1.515432067895e+14 ||r(i)||/||b|| 6.225972628492e-03 > 27 KSP preconditioned resid norm 8.526344464253e+14 true resid norm > 1.348203382746e+14 ||r(i)||/||b|| 5.538933441126e-03 > 28 KSP preconditioned resid norm 7.744588594030e+14 true resid norm > 1.215509014949e+14 ||r(i)||/||b|| 4.993774394172e-03 > 29 KSP preconditioned resid norm 7.032530926217e+14 true resid norm > 1.102818922582e+14 ||r(i)||/||b|| 4.530800536453e-03 > 30 KSP preconditioned resid norm 6.388922318141e+14 true resid norm > 9.838037383987e+13 ||r(i)||/||b|| 4.041840790388e-03 > 31 KSP preconditioned resid norm 5.692116056101e+14 true resid norm > 9.317411265002e+13 ||r(i)||/||b|| 3.827947734067e-03 > 32 KSP preconditioned resid norm 5.208032879647e+14 true resid norm > 8.007380640801e+13 ||r(i)||/||b|| 3.289737214338e-03 > 33 KSP preconditioned resid norm 4.765676071483e+14 true resid norm > 7.802986158610e+13 ||r(i)||/||b|| 3.205764169389e-03 > 34 KSP preconditioned resid norm 4.409793613234e+14 true resid norm > 6.791885197478e+13 ||r(i)||/||b|| 2.790365350662e-03 > 35 KSP preconditioned resid norm 4.123618028942e+14 true resid norm > 6.437750648670e+13 ||r(i)||/||b|| 2.644873378149e-03 > 36 KSP preconditioned resid norm 3.827034276493e+14 true resid norm > 5.974308940262e+13 ||r(i)||/||b|| 2.454473857604e-03 > 37 KSP preconditioned resid norm 3.509107846575e+14 true resid norm > 5.419166518671e+13 ||r(i)||/||b|| 2.226400188387e-03 > 38 KSP preconditioned resid norm 3.170601404687e+14 true resid norm > 4.972345072887e+13 ||r(i)||/||b|| 2.042828905305e-03 > 39 KSP preconditioned resid norm 2.901553052614e+14 true resid norm > 4.518683026559e+13 ||r(i)||/||b|| 1.856447242750e-03 > 40 KSP preconditioned resid norm 2.667015211684e+14 true resid norm > 4.122270495295e+13 ||r(i)||/||b|| 1.693585863377e-03 > 41 KSP preconditioned resid norm 2.422095249996e+14 true resid norm > 3.839959453856e+13 ||r(i)||/||b|| 1.577601725654e-03 > 42 KSP preconditioned resid norm 2.195894402502e+14 true resid norm > 3.399029120498e+13 ||r(i)||/||b|| 1.396450736130e-03 > 43 KSP preconditioned resid norm 1.983452570770e+14 true resid norm > 3.143816040870e+13 ||r(i)||/||b|| 1.291599474113e-03 > 44 KSP preconditioned resid norm 1.779873290535e+14 true resid norm > 2.755766172822e+13 ||r(i)||/||b|| 1.132173795580e-03 > 45 KSP preconditioned resid norm 1.542400485092e+14 true resid norm > 2.400274515345e+13 ||r(i)||/||b|| 9.861242710910e-04 > 46 KSP preconditioned resid norm 1.223281336233e+14 true resid norm > 1.907355666953e+13 ||r(i)||/||b|| 7.836144177513e-04 > 0 KSP preconditioned resid norm 7.586062313136e+16 true resid norm > 4.608637352265e+16 ||r(i)||/||b|| 6.590757061047e-02 > 1 KSP preconditioned resid norm 2.763422737801e+16 true resid norm > 5.227955984017e+15 ||r(i)||/||b|| 7.476437216213e-03 > 2 KSP preconditioned resid norm 1.553456028419e+16 true resid norm > 2.906104653716e+15 ||r(i)||/||b|| 4.155985447023e-03 > 3 KSP preconditioned resid norm 1.040767126705e+16 true resid norm > 2.091654777018e+15 ||r(i)||/||b|| 2.991250436342e-03 > 4 KSP preconditioned resid norm 7.531366829970e+15 true resid norm > 1.318758653617e+15 ||r(i)||/||b|| 1.885940950392e-03 > 5 KSP preconditioned resid norm 5.773017012836e+15 true resid norm > 9.450385953724e+14 ||r(i)||/||b|| 1.351488372664e-03 > 6 KSP preconditioned resid norm 4.731507925306e+15 true resid norm > 7.935497854840e+14 ||r(i)||/||b|| 1.134846040642e-03 > 7 KSP preconditioned resid norm 3.788639277852e+15 true resid norm > 7.524409398539e+14 ||r(i)||/||b|| 1.076056773034e-03 > 8 KSP preconditioned resid norm 2.738698664678e+15 true resid norm > 4.915430614067e+14 ||r(i)||/||b|| 7.029498429037e-04 > 9 KSP preconditioned resid norm 1.837368414949e+15 true resid norm > 3.357340764188e+14 ||r(i)||/||b|| 4.801292802316e-04 > 10 KSP preconditioned resid norm 1.059331899595e+15 true resid norm > 1.989071439556e+14 ||r(i)||/||b|| 2.844547234496e-04 > 11 KSP preconditioned resid norm 4.945892712172e+14 true resid norm > 8.452757650210e+13 ||r(i)||/||b|| 1.208818744245e-04 > 12 KSP preconditioned resid norm 1.987888303244e+14 true resid norm > 3.594134462521e+13 ||r(i)||/||b|| 5.139928633259e-05 > 13 KSP preconditioned resid norm 1.163784016975e+14 true resid norm > 2.571324386412e+13 ||r(i)||/||b|| 3.677220197779e-05 > 14 KSP preconditioned resid norm 8.494315446492e+13 true resid norm > 1.429434206797e+13 ||r(i)||/||b|| 2.044216732982e-05 > 15 KSP preconditioned resid norm 8.642186180523e+13 true resid norm > 1.607306715480e+13 ||r(i)||/||b|| 2.298590076546e-05 > 16 KSP preconditioned resid norm 3.925619743667e+13 true resid norm > 1.371348923807e+13 ||r(i)||/||b|| 1.961149665703e-05 > 0 KSP preconditioned resid norm 3.953744769406e+16 true resid norm > 2.171194598493e+16 ||r(i)||/||b|| 1.950905346919e-01 > 1 KSP preconditioned resid norm 1.491502757726e+16 true resid norm > 3.091776185716e+15 ||r(i)||/||b|| 2.778084790914e-02 > 2 KSP preconditioned resid norm 1.008617531922e+16 true resid norm > 1.691017660922e+15 ||r(i)||/||b|| 1.519447127732e-02 > 3 KSP preconditioned resid norm 8.326895039450e+15 true resid norm > 1.670298498749e+15 ||r(i)||/||b|| 1.500830130298e-02 > 4 KSP preconditioned resid norm 7.300566603499e+15 true resid norm > 1.589496633697e+15 ||r(i)||/||b|| 1.428226416804e-02 > 5 KSP preconditioned resid norm 6.872352176548e+15 true resid norm > 1.096739362492e+15 ||r(i)||/||b|| 9.854642637499e-03 > 6 KSP preconditioned resid norm 6.961475959227e+15 true resid norm > 1.092150928708e+15 ||r(i)||/||b|| 9.813413721358e-03 > 7 KSP preconditioned resid norm 7.025042419735e+15 true resid norm > 1.296288429690e+15 ||r(i)||/||b|| 1.164767096595e-02 > 8 KSP preconditioned resid norm 6.408702665765e+15 true resid norm > 1.056644614043e+15 ||r(i)||/||b|| 9.494375256645e-03 > 9 KSP preconditioned resid norm 5.399106179995e+15 true resid norm > 1.078818901202e+15 ||r(i)||/||b|| 9.693620112044e-03 > 10 KSP preconditioned resid norm 4.030707639439e+15 true resid norm > 7.107969460630e+14 ||r(i)||/||b|| 6.386795378039e-03 > 11 KSP preconditioned resid norm 2.682586389004e+15 true resid norm > 4.368532286577e+14 ||r(i)||/||b|| 3.925301307393e-03 > 12 KSP preconditioned resid norm 1.621400972048e+15 true resid norm > 2.499302020485e+14 ||r(i)||/||b|| 2.245723012904e-03 > 13 KSP preconditioned resid norm 1.118646687359e+15 true resid norm > 1.825993809337e+14 ||r(i)||/||b|| 1.640728605602e-03 > 14 KSP preconditioned resid norm 8.665732278929e+14 true resid norm > 1.337846645968e+14 ||r(i)||/||b|| 1.202108819167e-03 > 15 KSP preconditioned resid norm 1.790412083411e+15 true resid norm > 2.690613625509e+14 ||r(i)||/||b|| 2.417624155910e-03 > 16 KSP preconditioned resid norm 4.902192144952e+14 true resid norm > 8.615757491331e+13 ||r(i)||/||b|| 7.741603340972e-04 > 17 KSP preconditioned resid norm 3.873085562421e+14 true resid norm > 6.966874756362e+13 ||r(i)||/||b|| 6.260016132563e-04 > 18 KSP preconditioned resid norm 3.069894649217e+14 true resid norm > 4.954148501038e+13 ||r(i)||/||b|| 4.451500941838e-04 > 19 KSP preconditioned resid norm 2.366855243734e+14 true resid norm > 4.444373941350e+13 ||r(i)||/||b|| 3.993448073197e-04 > 20 KSP preconditioned resid norm 1.853280795186e+14 true resid norm > 2.839442050460e+13 ||r(i)||/||b|| 2.551352459311e-04 > 21 KSP preconditioned resid norm 1.419103820891e+14 true resid norm > 2.714177608773e+13 ||r(i)||/||b|| 2.438797339086e-04 > 22 KSP preconditioned resid norm 1.116662401807e+14 true resid norm > 1.710034207306e+13 ||r(i)||/||b|| 1.536534256654e-04 > 23 KSP preconditioned resid norm 8.768706858701e+13 true resid norm > 1.892684059360e+13 ||r(i)||/||b|| 1.700652467538e-04 > 24 KSP preconditioned resid norm 7.191985915577e+13 true resid norm > 1.104970014738e+13 ||r(i)||/||b|| 9.928598345972e-05 > 25 KSP preconditioned resid norm 5.804472664630e+13 true resid norm > 1.353159760302e+13 ||r(i)||/||b|| 1.215868265996e-04 > 26 KSP preconditioned resid norm 4.574262022619e+13 true resid norm > 6.915191476157e+12 ||r(i)||/||b|| 6.213576634340e-05 > 27 KSP preconditioned resid norm 3.470073834736e+13 true resid norm > 8.786634213858e+12 ||r(i)||/||b|| 7.895142923224e-05 > 28 KSP preconditioned resid norm 2.674822501624e+13 true resid norm > 4.041336605553e+12 ||r(i)||/||b|| 3.631302877201e-05 > 29 KSP preconditioned resid norm 2.088626286399e+13 true resid norm > 5.261273540428e+12 ||r(i)||/||b|| 4.727465096287e-05 > 30 KSP preconditioned resid norm 1.567496199145e+13 true resid norm > 2.371012817670e+12 ||r(i)||/||b|| 2.130450023603e-05 > 31 KSP preconditioned resid norm 1.119841843422e+13 true resid norm > 3.387071792996e+12 ||r(i)||/||b|| 3.043419726606e-05 > 32 KSP preconditioned resid norm 7.994377199528e+12 true resid norm > 1.204577998308e+12 ||r(i)||/||b|| 1.082361599145e-05 > 33 KSP preconditioned resid norm 6.174157756589e+12 true resid norm > 1.362745552188e+12 ||r(i)||/||b|| 1.224481484110e-05 > 0 SNES Function norm 0.000000000000e+00 > 1 TS dt 3e-12 time 2e-12 > 0 KSP preconditioned resid norm 2.220368634013e+19 true resid norm > 1.204306153911e+19 ||r(i)||/||b|| 2.302397943981e+02 > 1 KSP preconditioned resid norm 5.390402695434e+18 true resid norm > 1.775960718595e+18 ||r(i)||/||b|| 3.395289722473e+01 > 2 KSP preconditioned resid norm 2.218493119258e+18 true resid norm > 5.320083037595e+17 ||r(i)||/||b|| 1.017095877804e+01 > 3 KSP preconditioned resid norm 1.401798813920e+18 true resid norm > 2.963744523802e+17 ||r(i)||/||b|| 5.666100165582e+00 > 4 KSP preconditioned resid norm 1.004143331118e+18 true resid norm > 1.656359344976e+17 ||r(i)||/||b|| 3.166635276239e+00 > 5 KSP preconditioned resid norm 8.673495646713e+17 true resid norm > 1.510463721187e+17 ||r(i)||/||b|| 2.887711363779e+00 > 6 KSP preconditioned resid norm 7.389956702949e+17 true resid norm > 1.526304376340e+17 ||r(i)||/||b|| 2.917995599842e+00 > 7 KSP preconditioned resid norm 5.566613599927e+17 true resid norm > 1.017416269196e+17 ||r(i)||/||b|| 1.945101018344e+00 > 8 KSP preconditioned resid norm 3.566897773081e+17 true resid norm > 6.307244640523e+16 ||r(i)||/||b|| 1.205821878877e+00 > 9 KSP preconditioned resid norm 1.893453815442e+17 true resid norm > 3.268203516935e+16 ||r(i)||/||b|| 6.248166243663e-01 > 10 KSP preconditioned resid norm 6.894933140555e+16 true resid norm > 1.152954614680e+16 ||r(i)||/||b|| 2.204223839364e-01 > 11 KSP preconditioned resid norm 5.103537427509e+16 true resid norm > 9.272910080166e+15 ||r(i)||/||b|| 1.772799136993e-01 > 12 KSP preconditioned resid norm 4.991235858989e+16 true resid norm > 8.396555957121e+15 ||r(i)||/||b|| 1.605257359967e-01 > 13 KSP preconditioned resid norm 4.918689912532e+16 true resid norm > 8.423517310572e+15 ||r(i)||/||b|| 1.610411843696e-01 > 14 KSP preconditioned resid norm 5.686949456502e+16 true resid norm > 9.761258523981e+15 ||r(i)||/||b|| 1.866161813032e-01 > 15 KSP preconditioned resid norm 1.616570395103e+17 true resid norm > 4.351134639515e+16 ||r(i)||/||b|| 8.318518854587e-01 > 16 KSP preconditioned resid norm 4.128330778786e+16 true resid norm > 7.104207079385e+15 ||r(i)||/||b|| 1.358185517866e-01 > 17 KSP preconditioned resid norm 2.161181135112e+16 true resid norm > 5.859480309696e+15 ||r(i)||/||b|| 1.120218091889e-01 > 18 KSP preconditioned resid norm 1.410375919962e+16 true resid norm > 2.425065017616e+15 ||r(i)||/||b|| 4.636250252850e-02 > 19 KSP preconditioned resid norm 1.082544604615e+16 true resid norm > 2.485306543509e+15 ||r(i)||/||b|| 4.751420274116e-02 > 20 KSP preconditioned resid norm 8.996941383859e+15 true resid norm > 1.577914541048e+15 ||r(i)||/||b|| 3.016664145813e-02 > 21 KSP preconditioned resid norm 7.748060569447e+15 true resid norm > 1.336297549587e+15 ||r(i)||/||b|| 2.554739690340e-02 > 22 KSP preconditioned resid norm 6.873596381580e+15 true resid norm > 1.173018262854e+15 ||r(i)||/||b|| 2.242581612555e-02 > 23 KSP preconditioned resid norm 6.175955090548e+15 true resid norm > 1.012331331012e+15 ||r(i)||/||b|| 1.935379610558e-02 > 24 KSP preconditioned resid norm 5.614288705672e+15 true resid norm > 9.369191861544e+14 ||r(i)||/||b|| 1.791206331440e-02 > 25 KSP preconditioned resid norm 5.220540757763e+15 true resid norm > 8.444260437985e+14 ||r(i)||/||b|| 1.614377524163e-02 > 26 KSP preconditioned resid norm 5.059721722058e+15 true resid norm > 8.452379209101e+14 ||r(i)||/||b|| 1.615929674492e-02 > 27 KSP preconditioned resid norm 4.893746226621e+15 true resid norm > 9.389168158486e+14 ||r(i)||/||b|| 1.795025408912e-02 > 28 KSP preconditioned resid norm 4.794914716345e+15 true resid norm > 7.493396637157e+14 ||r(i)||/||b|| 1.432590953288e-02 > 29 KSP preconditioned resid norm 4.860853428688e+15 true resid norm > 7.693465766642e+14 ||r(i)||/||b|| 1.470840259819e-02 > 30 KSP preconditioned resid norm 5.112203028306e+15 true resid norm > 8.019578406020e+14 ||r(i)||/||b|| 1.533186621496e-02 > 31 KSP preconditioned resid norm 5.649006086453e+15 true resid norm > 8.938106702489e+14 ||r(i)||/||b|| 1.708791276044e-02 > 32 KSP preconditioned resid norm 8.383819984522e+15 true resid norm > 1.347106201218e+15 ||r(i)||/||b|| 2.575403719341e-02 > 33 KSP preconditioned resid norm 1.617520068743e+16 true resid norm > 2.603058585140e+15 ||r(i)||/||b|| 4.976539159104e-02 > 34 KSP preconditioned resid norm 3.383001122808e+16 true resid norm > 5.541435636245e+15 ||r(i)||/||b|| 1.059414167582e-01 > 35 KSP preconditioned resid norm 6.256166371964e+15 true resid norm > 1.029049029941e+15 ||r(i)||/||b|| 1.967340582872e-02 > 36 KSP preconditioned resid norm 3.290396673886e+15 true resid norm > 5.409340387326e+14 ||r(i)||/||b|| 1.034160138236e-02 > 37 KSP preconditioned resid norm 2.075346751579e+15 true resid norm > 3.809761571158e+14 ||r(i)||/||b|| 7.283519377531e-03 > 38 KSP preconditioned resid norm 1.750045982315e+15 true resid norm > 2.837816436037e+14 ||r(i)||/||b|| 5.425350278670e-03 > 39 KSP preconditioned resid norm 1.596755323762e+15 true resid norm > 2.766061504317e+14 ||r(i)||/||b|| 5.288168876146e-03 > 40 KSP preconditioned resid norm 1.469812007135e+15 true resid norm > 2.359521384401e+14 ||r(i)||/||b|| 4.510943638858e-03 > 41 KSP preconditioned resid norm 1.314668041488e+15 true resid norm > 2.158813968118e+14 ||r(i)||/||b|| 4.127230293966e-03 > 42 KSP preconditioned resid norm 1.190446489712e+15 true resid norm > 1.941922420158e+14 ||r(i)||/||b|| 3.712576053042e-03 > 43 KSP preconditioned resid norm 1.100371263474e+15 true resid norm > 1.837040226442e+14 ||r(i)||/||b|| 3.512061801423e-03 > 44 KSP preconditioned resid norm 1.032027622009e+15 true resid norm > 1.624908335172e+14 ||r(i)||/||b|| 3.106507093655e-03 > 45 KSP preconditioned resid norm 8.066010465859e+14 true resid norm > 2.499636130742e+14 ||r(i)||/||b|| 4.778815643705e-03 > 46 KSP preconditioned resid norm 6.546087590187e+14 true resid norm > 1.014535455422e+14 ||r(i)||/||b|| 1.939593465560e-03 > 47 KSP preconditioned resid norm 6.561404017041e+14 true resid norm > 1.083911025925e+14 ||r(i)||/||b|| 2.072225994564e-03 > 48 KSP preconditioned resid norm 6.834623592895e+14 true resid norm > 1.073483632090e+14 ||r(i)||/||b|| 2.052290855937e-03 > 49 KSP preconditioned resid norm 8.808387787841e+14 true resid norm > 1.369903526339e+14 ||r(i)||/||b|| 2.618987748466e-03 > 50 KSP preconditioned resid norm 3.995486126369e+14 true resid norm > 1.237415046233e+14 ||r(i)||/||b|| 2.365695673849e-03 > 51 KSP preconditioned resid norm 3.217227010099e+14 true resid norm > 4.935649374236e+13 ||r(i)||/||b|| 9.435996764234e-04 > 0 KSP preconditioned resid norm 6.543260687832e+18 true resid norm > 3.546137973950e+18 ||r(i)||/||b|| 4.156987252753e+02 > 1 KSP preconditioned resid norm 1.591265056482e+18 true resid norm > 5.223494588784e+17 ||r(i)||/||b|| 6.123281321796e+01 > 2 KSP preconditioned resid norm 6.570987793466e+17 true resid norm > 1.552598280892e+17 ||r(i)||/||b|| 1.820045161730e+01 > 3 KSP preconditioned resid norm 4.166120135488e+17 true resid norm > 8.739095834512e+16 ||r(i)||/||b|| 1.024447166228e+01 > 4 KSP preconditioned resid norm 2.995238174583e+17 true resid norm > 4.935051588719e+16 ||r(i)||/||b|| 5.785151817748e+00 > 5 KSP preconditioned resid norm 2.594691834001e+17 true resid norm > 4.470128066210e+16 ||r(i)||/||b|| 5.240141676921e+00 > 6 KSP preconditioned resid norm 2.219773011977e+17 true resid norm > 4.652509796902e+16 ||r(i)||/||b|| 5.453940050023e+00 > 7 KSP preconditioned resid norm 1.675026903695e+17 true resid norm > 2.988993956885e+16 ||r(i)||/||b|| 3.503870934691e+00 > 8 KSP preconditioned resid norm 1.081971008730e+17 true resid norm > 1.908275119338e+16 ||r(i)||/||b|| 2.236990044975e+00 > 9 KSP preconditioned resid norm 5.710427760412e+16 true resid norm > 9.635681698385e+15 ||r(i)||/||b|| 1.129550127097e+00 > 10 KSP preconditioned resid norm 1.779335357179e+16 true resid norm > 2.952424081259e+15 ||r(i)||/||b|| 3.461001619416e-01 > 11 KSP preconditioned resid norm 1.125492898955e+16 true resid norm > 2.030196257790e+15 ||r(i)||/||b|| 2.379913028263e-01 > 12 KSP preconditioned resid norm 1.086499382749e+16 true resid norm > 1.877721745252e+15 ||r(i)||/||b|| 2.201173619461e-01 > 13 KSP preconditioned resid norm 1.075258231290e+16 true resid norm > 1.822442508498e+15 ||r(i)||/||b|| 2.136372113085e-01 > 14 KSP preconditioned resid norm 1.173739259962e+16 true resid norm > 2.130298543993e+15 ||r(i)||/||b|| 2.497258695795e-01 > 15 KSP preconditioned resid norm 4.943363278431e+16 true resid norm > 1.361259414632e+16 ||r(i)||/||b|| 1.595746718229e+00 > 16 KSP preconditioned resid norm 1.782643390767e+17 true resid norm > 3.375538781586e+16 ||r(i)||/||b|| 3.957001049961e+00 > 17 KSP preconditioned resid norm 3.253381322946e+16 true resid norm > 8.720742251900e+15 ||r(i)||/||b|| 1.022295653525e+00 > 18 KSP preconditioned resid norm 2.077141437527e+16 true resid norm > 3.948414579285e+15 ||r(i)||/||b|| 4.628559067708e-01 > 19 KSP preconditioned resid norm 1.484650052668e+16 true resid norm > 3.391990202104e+15 ||r(i)||/||b|| 3.976286352982e-01 > 20 KSP preconditioned resid norm 1.135224344133e+16 true resid norm > 2.492450462789e+15 ||r(i)||/||b|| 2.921794041305e-01 > 21 KSP preconditioned resid norm 8.560056764067e+15 true resid norm > 1.658566536392e+15 ||r(i)||/||b|| 1.944267256456e-01 > 22 KSP preconditioned resid norm 6.591497757947e+15 true resid norm > 1.287448297256e+15 ||r(i)||/||b|| 1.509221073626e-01 > 23 KSP preconditioned resid norm 4.783676751639e+15 true resid norm > 9.069671731055e+14 ||r(i)||/||b|| 1.063199177517e-01 > 24 KSP preconditioned resid norm 3.029245298062e+15 true resid norm > 6.184462851213e+14 ||r(i)||/||b|| 7.249783687627e-02 > 25 KSP preconditioned resid norm 1.567581368038e+15 true resid norm > 2.751547089998e+14 ||r(i)||/||b|| 3.225522036225e-02 > 26 KSP preconditioned resid norm 1.174427256669e+15 true resid norm > 2.270458347820e+14 ||r(i)||/||b|| 2.661562093503e-02 > 27 KSP preconditioned resid norm 1.068689317098e+15 true resid norm > 2.026010487900e+14 ||r(i)||/||b|| 2.375006227624e-02 > 28 KSP preconditioned resid norm 1.006302654001e+15 true resid norm > 1.659657859887e+14 ||r(i)||/||b|| 1.945546568735e-02 > 29 KSP preconditioned resid norm 9.398219810967e+14 true resid norm > 1.788496139471e+14 ||r(i)||/||b|| 2.096578223407e-02 > 30 KSP preconditioned resid norm 8.922669862958e+14 true resid norm > 1.448103190511e+14 ||r(i)||/||b|| 1.697549996037e-02 > 31 KSP preconditioned resid norm 8.611553511040e+14 true resid norm > 1.569494235094e+14 ||r(i)||/||b|| 1.839851572749e-02 > 32 KSP preconditioned resid norm 8.275858647114e+14 true resid norm > 1.327021986732e+14 ||r(i)||/||b|| 1.555611632568e-02 > 33 KSP preconditioned resid norm 7.965262562283e+14 true resid norm > 1.578752878362e+14 ||r(i)||/||b|| 1.850705087847e-02 > 34 KSP preconditioned resid norm 7.584298711071e+14 true resid norm > 1.400338111929e+14 ||r(i)||/||b|| 1.641557018817e-02 > 35 KSP preconditioned resid norm 7.453216416858e+14 true resid norm > 1.168697043482e+14 ||r(i)||/||b|| 1.370014011799e-02 > 36 KSP preconditioned resid norm 7.552391370873e+14 true resid norm > 1.200616262811e+14 ||r(i)||/||b|| 1.407431559803e-02 > 37 KSP preconditioned resid norm 7.900785942536e+14 true resid norm > 1.248526226554e+14 ||r(i)||/||b|| 1.463594379756e-02 > 38 KSP preconditioned resid norm 8.292853068973e+14 true resid norm > 1.319608976445e+14 ||r(i)||/||b|| 1.546921674790e-02 > 39 KSP preconditioned resid norm 8.694401463969e+14 true resid norm > 1.375708051207e+14 ||r(i)||/||b|| 1.612684242516e-02 > 40 KSP preconditioned resid norm 9.345453545993e+14 true resid norm > 1.486706547594e+14 ||r(i)||/||b|| 1.742803075440e-02 > 41 KSP preconditioned resid norm 1.186246552213e+15 true resid norm > 1.878863391090e+14 ||r(i)||/||b|| 2.202511922492e-02 > 42 KSP preconditioned resid norm 2.140906233315e+15 true resid norm > 3.526342198473e+14 ||r(i)||/||b|| 4.133781504155e-02 > 43 KSP preconditioned resid norm 1.599648547908e+16 true resid norm > 2.542842342413e+15 ||r(i)||/||b|| 2.980866306056e-01 > 44 KSP preconditioned resid norm 1.201331553169e+15 true resid norm > 1.958292468332e+14 ||r(i)||/||b|| 2.295623263342e-02 > 45 KSP preconditioned resid norm 6.227596295745e+14 true resid norm > 9.990152835185e+13 ||r(i)||/||b|| 1.171103276127e-02 > 46 KSP preconditioned resid norm 4.228157632423e+14 true resid norm > 6.914757907633e+13 ||r(i)||/||b|| 8.105877630555e-03 > 47 KSP preconditioned resid norm 2.986102023248e+14 true resid norm > 4.782856914894e+13 ||r(i)||/||b|| 5.606740452011e-03 > 48 KSP preconditioned resid norm 1.831310965020e+14 true resid norm > 3.038059690813e+13 ||r(i)||/||b|| 3.561388615047e-03 > 49 KSP preconditioned resid norm 1.519167951348e+14 true resid norm > 2.657532215634e+13 ||r(i)||/||b|| 3.115312383591e-03 > 50 KSP preconditioned resid norm 1.784468495322e+14 true resid norm > 2.872094801120e+13 ||r(i)||/||b|| 3.366835008863e-03 > 51 KSP preconditioned resid norm 1.082269784640e+14 true resid norm > 1.734196241756e+13 ||r(i)||/||b|| 2.032924754679e-03 > 52 KSP preconditioned resid norm 7.941561433572e+13 true resid norm > 2.262252591710e+13 ||r(i)||/||b|| 2.651942833395e-03 > 53 KSP preconditioned resid norm 1.279466501710e+14 true resid norm > 2.100188573520e+13 ||r(i)||/||b|| 2.461962053545e-03 > 54 KSP preconditioned resid norm 1.148448766150e+14 true resid norm > 1.892568589516e+13 ||r(i)||/||b|| 2.218577945746e-03 > 55 KSP preconditioned resid norm 4.111681106774e+13 true resid norm > 1.052257112253e+13 ||r(i)||/||b|| 1.233516415432e-03 > 0 KSP preconditioned resid norm 1.892564880723e+19 true resid norm > 1.028289393047e+19 ||r(i)||/||b|| 7.840877673877e+01 > 1 KSP preconditioned resid norm 4.581010831586e+18 true resid norm > 1.519789346059e+18 ||r(i)||/||b|| 1.158864657468e+01 > 2 KSP preconditioned resid norm 1.874273717290e+18 true resid norm > 4.647134076709e+17 ||r(i)||/||b|| 3.543517036739e+00 > 3 KSP preconditioned resid norm 1.173044525535e+18 true resid norm > 2.550993039028e+17 ||r(i)||/||b|| 1.945174627025e+00 > 4 KSP preconditioned resid norm 8.299292116891e+17 true resid norm > 1.381054316961e+17 ||r(i)||/||b|| 1.053076889978e+00 > 5 KSP preconditioned resid norm 7.051404070011e+17 true resid norm > 1.287518923939e+17 ||r(i)||/||b|| 9.817545968746e-01 > 6 KSP preconditioned resid norm 5.863125538322e+17 true resid norm > 1.166548434193e+17 ||r(i)||/||b|| 8.895125861467e-01 > 7 KSP preconditioned resid norm 4.330808264262e+17 true resid norm > 8.612437830885e+16 ||r(i)||/||b|| 6.567127110566e-01 > 8 KSP preconditioned resid norm 2.702964509584e+17 true resid norm > 4.804107972053e+16 ||r(i)||/||b|| 3.663212243137e-01 > 9 KSP preconditioned resid norm 1.547354820777e+17 true resid norm > 2.879372380790e+16 ||r(i)||/||b|| 2.195569337580e-01 > 10 KSP preconditioned resid norm 9.913511217220e+16 true resid norm > 1.649311378149e+16 ||r(i)||/||b|| 1.257627361485e-01 > 11 KSP preconditioned resid norm 9.246165206306e+16 true resid norm > 1.559859930326e+16 ||r(i)||/||b|| 1.189419144530e-01 > 12 KSP preconditioned resid norm 9.248856361042e+16 true resid norm > 1.507587985844e+16 ||r(i)||/||b|| 1.149560917339e-01 > 13 KSP preconditioned resid norm 7.088634726331e+16 true resid norm > 1.150049603396e+16 ||r(i)||/||b|| 8.769319532120e-02 > 14 KSP preconditioned resid norm 2.014973749241e+17 true resid norm > 3.263663179758e+16 ||r(i)||/||b|| 2.488597464319e-01 > 15 KSP preconditioned resid norm 1.687600147354e+16 true resid norm > 4.438944439784e+15 ||r(i)||/||b|| 3.384768975431e-02 > 16 KSP preconditioned resid norm 1.189486995696e+16 true resid norm > 1.895011071195e+15 ||r(i)||/||b|| 1.444977464550e-02 > 17 KSP preconditioned resid norm 1.038277428537e+16 true resid norm > 2.030560404119e+15 ||r(i)||/||b|| 1.548336085714e-02 > 18 KSP preconditioned resid norm 9.585935517274e+15 true resid norm > 1.570427582757e+15 ||r(i)||/||b|| 1.197477155297e-02 > 19 KSP preconditioned resid norm 8.986106113745e+15 true resid norm > 1.453955802798e+15 ||r(i)||/||b|| 1.108665485617e-02 > 20 KSP preconditioned resid norm 8.480772996688e+15 true resid norm > 1.519414795393e+15 ||r(i)||/||b|| 1.158579056355e-02 > 21 KSP preconditioned resid norm 8.009707085363e+15 true resid norm > 1.276353172356e+15 ||r(i)||/||b|| 9.732405255546e-03 > 22 KSP preconditioned resid norm 7.603487959173e+15 true resid norm > 1.348218761629e+15 ||r(i)||/||b|| 1.028039232831e-02 > 23 KSP preconditioned resid norm 7.264676993739e+15 true resid norm > 1.145141030012e+15 ||r(i)||/||b|| 8.731890843545e-03 > 24 KSP preconditioned resid norm 6.442617339194e+15 true resid norm > 1.846348609557e+15 ||r(i)||/||b|| 1.407871528071e-02 > 25 KSP preconditioned resid norm 5.874716908283e+15 true resid norm > 9.094105988409e+14 ||r(i)||/||b|| 6.934407093038e-03 > 26 KSP preconditioned resid norm 5.974445043424e+15 true resid norm > 9.497988242860e+14 ||r(i)||/||b|| 7.242374030480e-03 > 27 KSP preconditioned resid norm 6.328543343408e+15 true resid norm > 9.841709349231e+14 ||r(i)||/||b|| 7.504467091753e-03 > 28 KSP preconditioned resid norm 1.045930080864e+16 true resid norm > 1.637386301334e+15 ||r(i)||/||b|| 1.248534292045e-02 > 29 KSP preconditioned resid norm 1.763968686067e+16 true resid norm > 3.035143897342e+15 ||r(i)||/||b|| 2.314347710148e-02 > 30 KSP preconditioned resid norm 3.994992144337e+15 true resid norm > 6.366779249631e+14 ||r(i)||/||b|| 4.854775086713e-03 > 31 KSP preconditioned resid norm 2.497471912676e+15 true resid norm > 4.171284268487e+14 ||r(i)||/||b|| 3.180673642395e-03 > 32 KSP preconditioned resid norm 1.774822233295e+15 true resid norm > 3.030553568855e+14 ||r(i)||/||b|| 2.310847508319e-03 > 33 KSP preconditioned resid norm 1.554413264158e+15 true resid norm > 2.490797343960e+14 ||r(i)||/||b|| 1.899274408204e-03 > 34 KSP preconditioned resid norm 1.363683273075e+15 true resid norm > 2.337415683510e+14 ||r(i)||/||b|| 1.782318340667e-03 > 35 KSP preconditioned resid norm 1.216181955878e+15 true resid norm > 1.941708948493e+14 ||r(i)||/||b|| 1.480585372791e-03 > 36 KSP preconditioned resid norm 1.106171392348e+15 true resid norm > 1.781889599099e+14 ||r(i)||/||b|| 1.358720460346e-03 > 37 KSP preconditioned resid norm 9.985678909493e+14 true resid norm > 1.843602345870e+14 ||r(i)||/||b|| 1.405777456326e-03 > 38 KSP preconditioned resid norm 9.306373857300e+14 true resid norm > 1.483324768155e+14 ||r(i)||/||b|| 1.131059810243e-03 > 39 KSP preconditioned resid norm 8.621292024153e+14 true resid norm > 1.776747195898e+14 ||r(i)||/||b|| 1.354799292363e-03 > 0 KSP preconditioned resid norm 6.470804796325e+18 true resid norm > 3.511545401133e+18 ||r(i)||/||b|| 1.662319590124e+02 > 1 KSP preconditioned resid norm 1.571603694611e+18 true resid norm > 5.187449890798e+17 ||r(i)||/||b|| 2.455670820454e+01 > 2 KSP preconditioned resid norm 6.468415902839e+17 true resid norm > 1.566954416728e+17 ||r(i)||/||b|| 7.417756930949e+00 > 3 KSP preconditioned resid norm 4.084120869852e+17 true resid norm > 8.688409741290e+16 ||r(i)||/||b|| 4.112979349580e+00 > 4 KSP preconditioned resid norm 2.920961241699e+17 true resid norm > 4.832079671037e+16 ||r(i)||/||b|| 2.287443213923e+00 > 5 KSP preconditioned resid norm 2.515909976499e+17 true resid norm > 4.435653262803e+16 ||r(i)||/||b|| 2.099780145624e+00 > 6 KSP preconditioned resid norm 2.133092401708e+17 true resid norm > 4.355815215083e+16 ||r(i)||/||b|| 2.061985859746e+00 > 7 KSP preconditioned resid norm 1.601275153782e+17 true resid norm > 2.999237788966e+16 ||r(i)||/||b|| 1.419799877977e+00 > 8 KSP preconditioned resid norm 1.019875018318e+17 true resid norm > 1.811096161004e+16 ||r(i)||/||b|| 8.573491964719e-01 > 9 KSP preconditioned resid norm 5.552203589730e+16 true resid norm > 9.849208071572e+15 ||r(i)||/||b|| 4.662486072173e-01 > 10 KSP preconditioned resid norm 2.614195079413e+16 true resid norm > 4.400774101946e+15 ||r(i)||/||b|| 2.083268807806e-01 > 11 KSP preconditioned resid norm 2.230316715231e+16 true resid norm > 4.001821966241e+15 ||r(i)||/||b|| 1.894410093210e-01 > 12 KSP preconditioned resid norm 2.202763899809e+16 true resid norm > 3.668851069689e+15 ||r(i)||/||b|| 1.736786032846e-01 > 13 KSP preconditioned resid norm 2.171352861704e+16 true resid norm > 3.746836567903e+15 ||r(i)||/||b|| 1.773703346056e-01 > 14 KSP preconditioned resid norm 2.756407833586e+16 true resid norm > 4.737724354756e+15 ||r(i)||/||b|| 2.242776643292e-01 > 15 KSP preconditioned resid norm 1.909305469788e+16 true resid norm > 7.268311827741e+15 ||r(i)||/||b|| 3.440723601207e-01 > 16 KSP preconditioned resid norm 8.249997128004e+15 true resid norm > 1.447767843213e+15 ||r(i)||/||b|| 6.853543305884e-02 > 17 KSP preconditioned resid norm 5.434797146758e+15 true resid norm > 1.776552830818e+15 ||r(i)||/||b|| 8.409968364943e-02 > 18 KSP preconditioned resid norm 4.166874856572e+15 true resid norm > 7.323760066865e+14 ||r(i)||/||b|| 3.466972071212e-02 > 19 KSP preconditioned resid norm 3.463504540881e+15 true resid norm > 7.653814460137e+14 ||r(i)||/||b|| 3.623215497131e-02 > 20 KSP preconditioned resid norm 3.008102754263e+15 true resid norm > 5.633011056242e+14 ||r(i)||/||b|| 2.666593639130e-02 > 21 KSP preconditioned resid norm 2.629337883031e+15 true resid norm > 5.301916607139e+14 ||r(i)||/||b|| 2.509857864406e-02 > 22 KSP preconditioned resid norm 2.387457188115e+15 true resid norm > 3.879435274146e+14 ||r(i)||/||b|| 1.836473836491e-02 > 23 KSP preconditioned resid norm 2.223797660329e+15 true resid norm > 3.481194862204e+14 ||r(i)||/||b|| 1.647952042601e-02 > 24 KSP preconditioned resid norm 2.082583007435e+15 true resid norm > 3.393016280495e+14 ||r(i)||/||b|| 1.606209457197e-02 > 25 KSP preconditioned resid norm 1.971105202361e+15 true resid norm > 3.085385809365e+14 ||r(i)||/||b|| 1.460581222257e-02 > 26 KSP preconditioned resid norm 1.916167824316e+15 true resid norm > 3.157123008884e+14 ||r(i)||/||b|| 1.494540672721e-02 > 27 KSP preconditioned resid norm 1.825444848282e+15 true resid norm > 4.173235963436e+14 ||r(i)||/||b|| 1.975555233884e-02 > 28 KSP preconditioned resid norm 1.787001647353e+15 true resid norm > 2.787303202952e+14 ||r(i)||/||b|| 1.319472821393e-02 > 29 KSP preconditioned resid norm 1.896717764390e+15 true resid norm > 3.068021792990e+14 ||r(i)||/||b|| 1.452361324382e-02 > 30 KSP preconditioned resid norm 2.229137880818e+15 true resid norm > 3.494609542895e+14 ||r(i)||/||b|| 1.654302376702e-02 > 31 KSP preconditioned resid norm 3.112887805330e+15 true resid norm > 5.056849908847e+14 ||r(i)||/||b|| 2.393846499915e-02 > 32 KSP preconditioned resid norm 2.024054955579e+16 true resid norm > 3.265471021005e+15 ||r(i)||/||b|| 1.545831202253e-01 > 33 KSP preconditioned resid norm 2.565517746075e+15 true resid norm > 4.297071200158e+14 ||r(i)||/||b|| 2.034177212653e-02 > 34 KSP preconditioned resid norm 1.151189963639e+15 true resid norm > 1.893961732619e+14 ||r(i)||/||b|| 8.965766724994e-03 > 35 KSP preconditioned resid norm 7.095661584181e+14 true resid norm > 1.215463627012e+14 ||r(i)||/||b|| 5.753845579254e-03 > 36 KSP preconditioned resid norm 5.356379112526e+14 true resid norm > 8.697359204859e+13 ||r(i)||/||b|| 4.117215908392e-03 > 37 KSP preconditioned resid norm 4.260660574229e+14 true resid norm > 9.044225863386e+13 ||r(i)||/||b|| 4.281418040435e-03 > 38 KSP preconditioned resid norm 3.822193347318e+14 true resid norm > 6.067358042990e+13 ||r(i)||/||b|| 2.872207812522e-03 > 39 KSP preconditioned resid norm 3.556139762092e+14 true resid norm > 7.037316076443e+13 ||r(i)||/||b|| 3.331373238687e-03 > 40 KSP preconditioned resid norm 3.319733183305e+14 true resid norm > 5.228450610578e+13 ||r(i)||/||b|| 2.475080024071e-03 > 41 KSP preconditioned resid norm 3.093370284781e+14 true resid norm > 5.468964654818e+13 ||r(i)||/||b|| 2.588936221775e-03 > 42 KSP preconditioned resid norm 2.908716641400e+14 true resid norm > 4.719126609844e+13 ||r(i)||/||b|| 2.233972714489e-03 > 43 KSP preconditioned resid norm 2.664499581099e+14 true resid norm > 6.079882213095e+13 ||r(i)||/||b|| 2.878136590578e-03 > 44 KSP preconditioned resid norm 2.513459163383e+14 true resid norm > 3.896503433878e+13 ||r(i)||/||b|| 1.844553679708e-03 > 45 KSP preconditioned resid norm 2.506735910619e+14 true resid norm > 4.872960867423e+13 ||r(i)||/||b|| 2.306795836731e-03 > 46 KSP preconditioned resid norm 2.610309833791e+14 true resid norm > 4.046617714344e+13 ||r(i)||/||b|| 1.915615813519e-03 > 47 KSP preconditioned resid norm 3.265892246735e+14 true resid norm > 6.092635527969e+13 ||r(i)||/||b|| 2.884173842765e-03 > 48 KSP preconditioned resid norm 6.619675585826e+14 true resid norm > 1.029872445074e+14 ||r(i)||/||b|| 4.875281237212e-03 > 49 KSP preconditioned resid norm 6.614927581836e+14 true resid norm > 1.173864813817e+14 ||r(i)||/||b|| 5.556922247214e-03 > 50 KSP preconditioned resid norm 2.572515469657e+14 true resid norm > 4.031945474821e+13 ||r(i)||/||b|| 1.908670167541e-03 > 51 KSP preconditioned resid norm 1.432897312628e+14 true resid norm > 2.304000171834e+13 ||r(i)||/||b|| 1.090683497942e-03 > 52 KSP preconditioned resid norm 8.898633954299e+13 true resid norm > 1.420071536381e+13 ||r(i)||/||b|| 6.722432617682e-04 > 0 KSP preconditioned resid norm 6.694445614266e+18 true resid norm > 3.650633549648e+18 ||r(i)||/||b|| 5.229628447406e+00 > 1 KSP preconditioned resid norm 1.597624793291e+18 true resid norm > 5.366629107368e+17 ||r(i)||/||b|| 7.687837156178e-01 > 2 KSP preconditioned resid norm 6.300333922334e+17 true resid norm > 1.705382687722e+17 ||r(i)||/||b|| 2.443005493741e-01 > 3 KSP preconditioned resid norm 3.642099405484e+17 true resid norm > 9.482515314454e+16 ||r(i)||/||b|| 1.358395225569e-01 > 4 KSP preconditioned resid norm 2.247296120507e+17 true resid norm > 3.802694326323e+16 ||r(i)||/||b|| 5.447459504021e-02 > 5 KSP preconditioned resid norm 1.557575416788e+17 true resid norm > 3.828791753195e+16 ||r(i)||/||b|| 5.484844753491e-02 > 6 KSP preconditioned resid norm 1.004780833173e+17 true resid norm > 1.699813376802e+16 ||r(i)||/||b|| 2.435027309565e-02 > 7 KSP preconditioned resid norm 7.104370054381e+16 true resid norm > 1.472894912543e+16 ||r(i)||/||b|| 2.109960649273e-02 > 8 KSP preconditioned resid norm 5.767747590200e+16 true resid norm > 9.399551072817e+15 ||r(i)||/||b|| 1.346510380041e-02 > 9 KSP preconditioned resid norm 4.524165642090e+16 true resid norm > 1.246569325329e+16 ||r(i)||/||b|| 1.785743300921e-02 > 10 KSP preconditioned resid norm 3.195588487377e+16 true resid norm > 5.031148957130e+15 ||r(i)||/||b|| 7.207253029241e-03 > 11 KSP preconditioned resid norm 2.437445149875e+16 true resid norm > 4.973530315557e+15 ||r(i)||/||b|| 7.124712811776e-03 > 12 KSP preconditioned resid norm 1.257635618330e+16 true resid norm > 1.885290715052e+15 ||r(i)||/||b|| 2.700728468356e-03 > 13 KSP preconditioned resid norm 2.495249888250e+15 true resid norm > 4.460762730080e+14 ||r(i)||/||b|| 6.390159777228e-04 > 14 KSP preconditioned resid norm 7.692041830184e+14 true resid norm > 1.319158639688e+14 ||r(i)||/||b|| 1.889729400372e-04 > 15 KSP preconditioned resid norm 5.963386353631e+14 true resid norm > 1.441107752347e+14 ||r(i)||/||b|| 2.064424707371e-04 > 16 KSP preconditioned resid norm 3.269005832710e+14 true resid norm > 5.055770257989e+13 ||r(i)||/||b|| 7.242523689424e-05 > 17 KSP preconditioned resid norm 1.442136448583e+14 true resid norm > 7.615238793008e+13 ||r(i)||/||b|| 1.090902959284e-04 > 18 KSP preconditioned resid norm 5.418830608399e+13 true resid norm > 8.650648142713e+12 ||r(i)||/||b|| 1.239228068235e-05 > 19 KSP preconditioned resid norm 3.534476706711e+13 true resid norm > 1.235947718863e+13 ||r(i)||/||b|| 1.770527570673e-05 > 0 KSP preconditioned resid norm 4.827689913975e+18 true resid norm > 2.628955418091e+18 ||r(i)||/||b|| 2.387179080674e+01 > 1 KSP preconditioned resid norm 1.161014015330e+18 true resid norm > 3.885001078629e+17 ||r(i)||/||b|| 3.527710374804e+00 > 2 KSP preconditioned resid norm 4.696713579932e+17 true resid norm > 1.228966476828e+17 ||r(i)||/||b|| 1.115942493412e+00 > 3 KSP preconditioned resid norm 2.869928353163e+17 true resid norm > 6.759586363313e+16 ||r(i)||/||b|| 6.137929555390e-01 > 4 KSP preconditioned resid norm 1.966510821692e+17 true resid norm > 3.310379360335e+16 ||r(i)||/||b|| 3.005934716010e-01 > 5 KSP preconditioned resid norm 1.592497528073e+17 true resid norm > 3.239157026660e+16 ||r(i)||/||b|| 2.941262464874e-01 > 6 KSP preconditioned resid norm 1.230094082012e+17 true resid norm > 2.280559390645e+16 ||r(i)||/||b|| 2.070823883934e-01 > 7 KSP preconditioned resid norm 8.598887604154e+16 true resid norm > 1.938013722659e+16 ||r(i)||/||b|| 1.759781008439e-01 > 8 KSP preconditioned resid norm 5.549487092651e+16 true resid norm > 9.386421691835e+15 ||r(i)||/||b|| 8.523183524123e-02 > 9 KSP preconditioned resid norm 4.475548308156e+16 true resid norm > 7.579602548804e+15 ||r(i)||/||b|| 6.882531563606e-02 > 10 KSP preconditioned resid norm 4.379582896462e+16 true resid norm > 6.844431748768e+15 ||r(i)||/||b|| 6.214971991278e-02 > 11 KSP preconditioned resid norm 4.081432560058e+16 true resid norm > 7.719907534397e+15 ||r(i)||/||b|| 7.009933163578e-02 > 12 KSP preconditioned resid norm 4.084955665921e+16 true resid norm > 6.376903215279e+15 ||r(i)||/||b|| 5.790440511177e-02 > 13 KSP preconditioned resid norm 7.513638228384e+16 true resid norm > 1.130789675242e+16 ||r(i)||/||b|| 1.026794687656e-01 > 14 KSP preconditioned resid norm 2.195838265499e+15 true resid norm > 3.518796225338e+14 ||r(i)||/||b|| 3.195184171050e-03 > 15 KSP preconditioned resid norm 2.237408970122e+15 true resid norm > 3.676139977919e+14 ||r(i)||/||b|| 3.338057538948e-03 > 16 KSP preconditioned resid norm 1.675719433917e+15 true resid norm > 3.658174858611e+14 ||r(i)||/||b|| 3.321744612262e-03 > 17 KSP preconditioned resid norm 1.403876583239e+15 true resid norm > 2.164126267214e+14 ||r(i)||/||b|| 1.965098729891e-03 > 18 KSP preconditioned resid norm 1.133854982860e+15 true resid norm > 2.692061619571e+14 ||r(i)||/||b|| 2.444481613457e-03 > 19 KSP preconditioned resid norm 9.034784290513e+14 true resid norm > 1.408322491509e+14 ||r(i)||/||b|| 1.278803728446e-03 > 20 KSP preconditioned resid norm 7.964711449547e+14 true resid norm > 1.641000281547e+14 ||r(i)||/||b|| 1.490082911460e-03 > 21 KSP preconditioned resid norm 7.094597664828e+14 true resid norm > 1.088610845746e+14 ||r(i)||/||b|| 9.884949056480e-04 > 22 KSP preconditioned resid norm 6.287860765855e+14 true resid norm > 1.439087926971e+14 ||r(i)||/||b|| 1.306739768531e-03 > 23 KSP preconditioned resid norm 5.617871594119e+14 true resid norm > 8.640965180691e+13 ||r(i)||/||b|| 7.846284183530e-04 > 24 KSP preconditioned resid norm 4.992037484390e+14 true resid norm > 1.232499062518e+14 ||r(i)||/||b|| 1.119150198876e-03 > 25 KSP preconditioned resid norm 3.174300847299e+14 true resid norm > 1.199443314096e+14 ||r(i)||/||b|| 1.089134478340e-03 > 26 KSP preconditioned resid norm 2.311461716948e+14 true resid norm > 3.806900580053e+13 ||r(i)||/||b|| 3.456792520850e-04 > 27 KSP preconditioned resid norm 2.486080516907e+14 true resid norm > 3.795889860412e+13 ||r(i)||/||b|| 3.446794420688e-04 > 28 KSP preconditioned resid norm 3.709940932117e+14 true resid norm > 6.158317563060e+13 ||r(i)||/||b|| 5.591957458661e-04 > 29 KSP preconditioned resid norm 5.526902041486e+14 true resid norm > 8.543527855752e+13 ||r(i)||/||b|| 7.757807847199e-04 > 30 KSP preconditioned resid norm 1.980272137271e+15 true resid norm > 3.202004161054e+14 ||r(i)||/||b|| 2.907526425475e-03 > 31 KSP preconditioned resid norm 4.796148387368e+14 true resid norm > 7.312671161483e+13 ||r(i)||/||b|| 6.640148973394e-04 > 32 KSP preconditioned resid norm 1.482366487345e+14 true resid norm > 3.206675205939e+13 ||r(i)||/||b|| 2.911767889807e-04 > 33 KSP preconditioned resid norm 7.403721045502e+13 true resid norm > 1.122274917558e+13 ||r(i)||/||b|| 1.019063004083e-04 > 34 KSP preconditioned resid norm 3.421203410880e+13 true resid norm > 9.812523407473e+12 ||r(i)||/||b|| 8.910098073838e-05 > 35 KSP preconditioned resid norm 1.809542403951e+13 true resid norm > 2.774316021276e+12 ||r(i)||/||b|| 2.519171349804e-05 > 36 KSP preconditioned resid norm 1.070863017107e+13 true resid norm > 1.674563212179e+12 ||r(i)||/||b|| 1.520559170335e-05 > 37 KSP preconditioned resid norm 6.984609262190e+12 true resid norm > 1.277736482286e+12 ||r(i)||/||b|| 1.160227282721e-05 > 0 KSP preconditioned resid norm 9.569153877844e+15 true resid norm > 5.893805322157e+15 ||r(i)||/||b|| 1.119437717455e-01 > 1 KSP preconditioned resid norm 4.594033327917e+15 true resid norm > 8.149731925277e+14 ||r(i)||/||b|| 1.547916296116e-02 > 2 KSP preconditioned resid norm 3.788286152976e+15 true resid norm > 6.151807578016e+14 ||r(i)||/||b|| 1.168441279773e-02 > 3 KSP preconditioned resid norm 3.712195492568e+15 true resid norm > 6.206672982073e+14 ||r(i)||/||b|| 1.178862119847e-02 > 4 KSP preconditioned resid norm 3.886667577513e+15 true resid norm > 6.330116967950e+14 ||r(i)||/||b|| 1.202308407301e-02 > 5 KSP preconditioned resid norm 4.317240607103e+15 true resid norm > 6.816590875948e+14 ||r(i)||/||b|| 1.294706647725e-02 > 6 KSP preconditioned resid norm 5.249548366505e+15 true resid norm > 7.833242603614e+14 ||r(i)||/||b|| 1.487804014750e-02 > 7 KSP preconditioned resid norm 6.521511569741e+15 true resid norm > 1.182301409963e+15 ||r(i)||/||b|| 2.245599777001e-02 > 8 KSP preconditioned resid norm 7.063327681244e+15 true resid norm > 1.110968116855e+15 ||r(i)||/||b|| 2.110113152571e-02 > 9 KSP preconditioned resid norm 7.059112088082e+15 true resid norm > 1.341815038025e+15 ||r(i)||/||b|| 2.548571392011e-02 > 10 KSP preconditioned resid norm 6.417041101993e+15 true resid norm > 1.020408664201e+15 ||r(i)||/||b|| 1.938109393654e-02 > 11 KSP preconditioned resid norm 4.850416656929e+15 true resid norm > 8.501719778107e+14 ||r(i)||/||b|| 1.614770977770e-02 > 12 KSP preconditioned resid norm 2.420686956668e+15 true resid norm > 3.735196627993e+14 ||r(i)||/||b|| 7.094431795646e-03 > 13 KSP preconditioned resid norm 1.348084459782e+15 true resid norm > 2.753333760274e+14 ||r(i)||/||b|| 5.229534216894e-03 > 14 KSP preconditioned resid norm 1.225904445555e+15 true resid norm > 2.239730314894e+14 ||r(i)||/||b|| 4.254023426925e-03 > 15 KSP preconditioned resid norm 1.156415256406e+15 true resid norm > 2.064224387096e+14 ||r(i)||/||b|| 3.920676897010e-03 > 16 KSP preconditioned resid norm 1.062658328422e+15 true resid norm > 1.696405409630e+14 ||r(i)||/||b|| 3.222061292889e-03 > 17 KSP preconditioned resid norm 9.163863908341e+14 true resid norm > 1.701529770139e+14 ||r(i)||/||b|| 3.231794227925e-03 > 18 KSP preconditioned resid norm 8.234055963962e+14 true resid norm > 1.322040551676e+14 ||r(i)||/||b|| 2.511012795057e-03 > 19 KSP preconditioned resid norm 7.097124944641e+14 true resid norm > 1.155726966735e+14 ||r(i)||/||b|| 2.195125707289e-03 > 20 KSP preconditioned resid norm 6.309033027960e+14 true resid norm > 1.023450925395e+14 ||r(i)||/||b|| 1.943887701115e-03 > 21 KSP preconditioned resid norm 5.667708572251e+14 true resid norm > 8.941024380489e+13 ||r(i)||/||b|| 1.698210133710e-03 > 22 KSP preconditioned resid norm 5.205793964231e+14 true resid norm > 8.689007901528e+13 ||r(i)||/||b|| 1.650343477697e-03 > 23 KSP preconditioned resid norm 4.890134322830e+14 true resid norm > 7.566844928998e+13 ||r(i)||/||b|| 1.437205871699e-03 > 24 KSP preconditioned resid norm 4.617102016960e+14 true resid norm > 8.638231658778e+13 ||r(i)||/||b|| 1.640699310952e-03 > 25 KSP preconditioned resid norm 4.386224186730e+14 true resid norm > 6.736465167645e+13 ||r(i)||/||b|| 1.279488001179e-03 > 26 KSP preconditioned resid norm 4.119635069909e+14 true resid norm > 8.360600093542e+13 ||r(i)||/||b|| 1.587967463073e-03 > 27 KSP preconditioned resid norm 3.764202708342e+14 true resid norm > 5.766864517117e+13 ||r(i)||/||b|| 1.095327262956e-03 > 28 KSP preconditioned resid norm 3.345193309034e+14 true resid norm > 7.248849449141e+13 ||r(i)||/||b|| 1.376807518738e-03 > 0 KSP preconditioned resid norm 9.592548277384e+15 true resid norm > 5.933564471549e+15 ||r(i)||/||b|| 5.704186299543e-01 > 1 KSP preconditioned resid norm 4.627892277467e+15 true resid norm > 8.123960687021e+14 ||r(i)||/||b|| 7.809906755228e-02 > 2 KSP preconditioned resid norm 3.843955835291e+15 true resid norm > 6.217930929753e+14 ||r(i)||/||b|| 5.977559794129e-02 > 3 KSP preconditioned resid norm 3.802651796154e+15 true resid norm > 6.334637368548e+14 ||r(i)||/||b|| 6.089754626161e-02 > 4 KSP preconditioned resid norm 4.025480937197e+15 true resid norm > 6.491593110279e+14 ||r(i)||/||b|| 6.240642814182e-02 > 5 KSP preconditioned resid norm 4.519579789539e+15 true resid norm > 7.102235297384e+14 ||r(i)||/||b|| 6.827678956506e-02 > 6 KSP preconditioned resid norm 5.567308133052e+15 true resid norm > 8.274501588234e+14 ||r(i)||/||b|| 7.954628085944e-02 > 7 KSP preconditioned resid norm 7.053789984861e+15 true resid norm > 1.260366188593e+15 ||r(i)||/||b|| 1.211643284547e-01 > 8 KSP preconditioned resid norm 7.835759896085e+15 true resid norm > 1.231567591968e+15 ||r(i)||/||b|| 1.183957976482e-01 > 9 KSP preconditioned resid norm 8.126559485840e+15 true resid norm > 1.514279927927e+15 ||r(i)||/||b|| 1.455741293445e-01 > 10 KSP preconditioned resid norm 7.865908148994e+15 true resid norm > 1.247217430777e+15 ||r(i)||/||b|| 1.199002827947e-01 > 11 KSP preconditioned resid norm 6.705237228445e+15 true resid norm > 1.156150868849e+15 ||r(i)||/||b|| 1.111456693176e-01 > 12 KSP preconditioned resid norm 3.837356917105e+15 true resid norm > 5.892678306289e+14 ||r(i)||/||b|| 5.664880700887e-02 > 13 KSP preconditioned resid norm 1.588477046031e+15 true resid norm > 3.087098247775e+14 ||r(i)||/||b|| 2.967758017080e-02 > 14 KSP preconditioned resid norm 1.304101915863e+15 true resid norm > 2.634312731694e+14 ||r(i)||/||b|| 2.532476164182e-02 > 15 KSP preconditioned resid norm 1.188980488344e+15 true resid norm > 2.212236776838e+14 ||r(i)||/||b|| 2.126716710384e-02 > 16 KSP preconditioned resid norm 1.053945473260e+15 true resid norm > 1.687417173497e+14 ||r(i)||/||b|| 1.622185444993e-02 > 17 KSP preconditioned resid norm 8.630322699779e+14 true resid norm > 2.032342536888e+14 ||r(i)||/||b|| 1.953776774564e-02 > 18 KSP preconditioned resid norm 7.497367164260e+14 true resid norm > 1.258314731452e+14 ||r(i)||/||b|| 1.209671132095e-02 > 19 KSP preconditioned resid norm 6.482413928213e+14 true resid norm > 1.007668116028e+14 ||r(i)||/||b|| 9.687139474922e-03 > 20 KSP preconditioned resid norm 5.423124085977e+14 true resid norm > 1.262959277696e+14 ||r(i)||/||b|| 1.214136130695e-02 > 21 KSP preconditioned resid norm 4.512195524384e+14 true resid norm > 7.029203398299e+13 ||r(i)||/||b|| 6.757470305334e-03 > 22 KSP preconditioned resid norm 3.576286880164e+14 true resid norm > 9.097241124393e+13 ||r(i)||/||b|| 8.745562373884e-03 > 23 KSP preconditioned resid norm 2.908403868181e+14 true resid norm > 4.482717971847e+13 ||r(i)||/||b|| 4.309426241567e-03 > 24 KSP preconditioned resid norm 2.299942905294e+14 true resid norm > 6.485244464292e+13 ||r(i)||/||b|| 6.234539592478e-03 > 25 KSP preconditioned resid norm 1.957293334591e+14 true resid norm > 3.000864324033e+13 ||r(i)||/||b|| 2.884857701642e-03 > 26 KSP preconditioned resid norm 1.685971736537e+14 true resid norm > 4.568513443333e+13 ||r(i)||/||b|| 4.391905054321e-03 > 27 KSP preconditioned resid norm 1.478986486404e+14 true resid norm > 2.270701271232e+13 ||r(i)||/||b|| 2.182921099758e-03 > 28 KSP preconditioned resid norm 1.388836935911e+14 true resid norm > 2.435562734780e+13 ||r(i)||/||b|| 2.341409392284e-03 > 29 KSP preconditioned resid norm 1.300875297573e+14 true resid norm > 2.023811842122e+13 ||r(i)||/||b|| 1.945575857149e-03 > 30 KSP preconditioned resid norm 1.230463362777e+14 true resid norm > 2.013991197226e+13 ||r(i)||/||b|| 1.936134856156e-03 > 31 KSP preconditioned resid norm 1.148174878708e+14 true resid norm > 1.790715664590e+13 ||r(i)||/||b|| 1.721490650234e-03 > 32 KSP preconditioned resid norm 1.085020096185e+14 true resid norm > 1.697737976905e+13 ||r(i)||/||b|| 1.632107269502e-03 > 33 KSP preconditioned resid norm 1.008662380835e+14 true resid norm > 1.575408064147e+13 ||r(i)||/||b|| 1.514506354280e-03 > 34 KSP preconditioned resid norm 9.448474542578e+13 true resid norm > 1.450932025652e+13 ||r(i)||/||b|| 1.394842277686e-03 > 35 KSP preconditioned resid norm 8.955256800693e+13 true resid norm > 1.427784032077e+13 ||r(i)||/||b|| 1.372589133148e-03 > 36 KSP preconditioned resid norm 8.409700445745e+13 true resid norm > 1.287414873178e+13 ||r(i)||/||b|| 1.237646328210e-03 > 37 KSP preconditioned resid norm 7.999039854664e+13 true resid norm > 1.307106382082e+13 ||r(i)||/||b|| 1.256576607951e-03 > 38 KSP preconditioned resid norm 7.508174679474e+13 true resid norm > 1.150183263556e+13 ||r(i)||/||b|| 1.105719781996e-03 > 39 KSP preconditioned resid norm 7.047760131088e+13 true resid norm > 1.131577846513e+13 ||r(i)||/||b|| 1.087833608263e-03 > 40 KSP preconditioned resid norm 6.612188631103e+13 true resid norm > 1.012435424497e+13 ||r(i)||/||b|| 9.732969626060e-04 > 41 KSP preconditioned resid norm 6.096165352930e+13 true resid norm > 1.005634567468e+13 ||r(i)||/||b|| 9.667590113164e-04 > 42 KSP preconditioned resid norm 5.668115744306e+13 true resid norm > 8.662975702181e+12 ||r(i)||/||b|| 8.328084669947e-04 > 43 KSP preconditioned resid norm 5.243169528258e+13 true resid norm > 8.972837494107e+12 ||r(i)||/||b|| 8.625967906361e-04 > 0 KSP preconditioned resid norm 2.201027933913e+16 true resid norm > 1.374916449250e+16 ||r(i)||/||b|| 1.042532971848e-01 > 1 KSP preconditioned resid norm 1.033786235817e+16 true resid norm > 1.769244185634e+15 ||r(i)||/||b|| 1.341532716246e-02 > 2 KSP preconditioned resid norm 8.200152533928e+15 true resid norm > 1.329548327593e+15 ||r(i)||/||b|| 1.008132508660e-02 > 3 KSP preconditioned resid norm 7.725254307721e+15 true resid norm > 1.292264285665e+15 ||r(i)||/||b|| 9.798618140627e-03 > 4 KSP preconditioned resid norm 7.783851928031e+15 true resid norm > 1.241739377824e+15 ||r(i)||/||b|| 9.415512080968e-03 > 5 KSP preconditioned resid norm 8.247256237278e+15 true resid norm > 1.289875346607e+15 ||r(i)||/||b|| 9.780503965494e-03 > 6 KSP preconditioned resid norm 9.468674539247e+15 true resid norm > 1.409079612219e+15 ||r(i)||/||b|| 1.068437254131e-02 > 7 KSP preconditioned resid norm 1.106031993220e+16 true resid norm > 1.997396506013e+15 ||r(i)||/||b|| 1.514529640334e-02 > 8 KSP preconditioned resid norm 1.133839709036e+16 true resid norm > 1.772431554854e+15 ||r(i)||/||b|| 1.343949544924e-02 > 9 KSP preconditioned resid norm 1.055805824819e+16 true resid norm > 2.045498232098e+15 ||r(i)||/||b|| 1.551002864197e-02 > 10 KSP preconditioned resid norm 8.628048224174e+15 true resid norm > 1.368695682952e+15 ||r(i)||/||b|| 1.037816064155e-02 > 11 KSP preconditioned resid norm 5.442091241360e+15 true resid norm > 9.810024540837e+14 ||r(i)||/||b|| 7.438469475027e-03 > 12 KSP preconditioned resid norm 2.000681531075e+15 true resid norm > 3.042659606098e+14 ||r(i)||/||b|| 2.307102342980e-03 > 13 KSP preconditioned resid norm 8.105673271529e+14 true resid norm > 1.475101242619e+14 ||r(i)||/||b|| 1.118498278992e-03 > 0 KSP preconditioned resid norm 1.928660996668e+16 true resid norm > 1.203367101178e+16 ||r(i)||/||b|| 4.946564650623e-01 > 1 KSP preconditioned resid norm 9.233394049572e+15 true resid norm > 1.583660413052e+15 ||r(i)||/||b|| 6.509799553373e-02 > 2 KSP preconditioned resid norm 7.543213441610e+15 true resid norm > 1.216909563864e+15 ||r(i)||/||b|| 5.002232341004e-02 > 3 KSP preconditioned resid norm 7.341955893763e+15 true resid norm > 1.220114169499e+15 ||r(i)||/||b|| 5.015405203166e-02 > 4 KSP preconditioned resid norm 7.657405519141e+15 true resid norm > 1.212628294074e+15 ||r(i)||/||b|| 4.984633739730e-02 > 5 KSP preconditioned resid norm 8.417297612797e+15 true resid norm > 1.311888034770e+15 ||r(i)||/||b|| 5.392651146949e-02 > 6 KSP preconditioned resid norm 1.008979814671e+16 true resid norm > 1.496346817141e+15 ||r(i)||/||b|| 6.150888007071e-02 > 7 KSP preconditioned resid norm 1.235951387667e+16 true resid norm > 2.203847820374e+15 ||r(i)||/||b|| 9.059143891287e-02 > 8 KSP preconditioned resid norm 1.311708980126e+16 true resid norm > 2.061617788942e+15 ||r(i)||/||b|| 8.474492669688e-02 > 9 KSP preconditioned resid norm 1.289258338081e+16 true resid norm > 2.353735302483e+15 ||r(i)||/||b|| 9.675271854106e-02 > 10 KSP preconditioned resid norm 1.151089236912e+16 true resid norm > 1.820534806530e+15 ||r(i)||/||b|| 7.483496191971e-02 > 11 KSP preconditioned resid norm 8.373719062509e+15 true resid norm > 1.433022884373e+15 ||r(i)||/||b|| 5.890588446727e-02 > 12 KSP preconditioned resid norm 3.564533969408e+15 true resid norm > 5.439956922043e+14 ||r(i)||/||b|| 2.236150430334e-02 > 13 KSP preconditioned resid norm 1.257109874849e+15 true resid norm > 2.286041152149e+14 ||r(i)||/||b|| 9.397008063476e-03 > 14 KSP preconditioned resid norm 1.068785016666e+15 true resid norm > 1.971356784282e+14 ||r(i)||/||b|| 8.103465495567e-03 > 15 KSP preconditioned resid norm 9.963029863724e+14 true resid norm > 1.558476632901e+14 ||r(i)||/||b|| 6.406279026229e-03 > 16 KSP preconditioned resid norm 8.667425302785e+14 true resid norm > 2.560422780209e+14 ||r(i)||/||b|| 1.052488206037e-02 > 17 KSP preconditioned resid norm 7.240699142388e+14 true resid norm > 1.131314963782e+14 ||r(i)||/||b|| 4.650386904448e-03 > 18 KSP preconditioned resid norm 6.352047441583e+14 true resid norm > 1.637714957863e+14 ||r(i)||/||b|| 6.731996337968e-03 > 19 KSP preconditioned resid norm 5.532686184638e+14 true resid norm > 8.631039548890e+13 ||r(i)||/||b|| 3.547877874413e-03 > 20 KSP preconditioned resid norm 4.676701949378e+14 true resid norm > 1.246119427256e+14 ||r(i)||/||b|| 5.122302498784e-03 > 21 KSP preconditioned resid norm 4.001088470241e+14 true resid norm > 6.181513171397e+13 ||r(i)||/||b|| 2.540974779106e-03 > 22 KSP preconditioned resid norm 3.482451502111e+14 true resid norm > 6.477073783496e+13 ||r(i)||/||b|| 2.662468018741e-03 > 23 KSP preconditioned resid norm 3.041601141164e+14 true resid norm > 4.679581096972e+13 ||r(i)||/||b|| 1.923590100755e-03 > 24 KSP preconditioned resid norm 2.690997224286e+14 true resid norm > 4.294556821793e+13 ||r(i)||/||b|| 1.765321899192e-03 > 25 KSP preconditioned resid norm 2.482501027210e+14 true resid norm > 3.840071688811e+13 ||r(i)||/||b|| 1.578501095230e-03 > 26 KSP preconditioned resid norm 2.328482920528e+14 true resid norm > 3.868960801558e+13 ||r(i)||/||b|| 1.590376263145e-03 > 27 KSP preconditioned resid norm 2.171581409229e+14 true resid norm > 3.334869397249e+13 ||r(i)||/||b|| 1.370832479858e-03 > 28 KSP preconditioned resid norm 2.002313534549e+14 true resid norm > 3.534153915591e+13 ||r(i)||/||b|| 1.452750437635e-03 > 29 KSP preconditioned resid norm 1.813916789125e+14 true resid norm > 2.780025874135e+13 ||r(i)||/||b|| 1.142758323985e-03 > 30 KSP preconditioned resid norm 1.655865002619e+14 true resid norm > 2.941107681765e+13 ||r(i)||/||b|| 1.208972663292e-03 > 31 KSP preconditioned resid norm 1.477358437937e+14 true resid norm > 2.263549873151e+13 ||r(i)||/||b|| 9.304555340168e-04 > 32 KSP preconditioned resid norm 1.346388933582e+14 true resid norm > 2.267328899678e+13 ||r(i)||/||b|| 9.320089418685e-04 > 0 KSP preconditioned resid norm 6.130047183925e+16 true resid norm > 4.019546453762e+16 ||r(i)||/||b|| 5.748521837148e-02 > 1 KSP preconditioned resid norm 2.383257388870e+16 true resid norm > 4.333010621750e+15 ||r(i)||/||b|| 6.196820080636e-03 > 2 KSP preconditioned resid norm 1.380338515654e+16 true resid norm > 2.902899468751e+15 ||r(i)||/||b|| 4.151558186756e-03 > 3 KSP preconditioned resid norm 9.355621435823e+15 true resid norm > 1.914392486860e+15 ||r(i)||/||b|| 2.737852924995e-03 > 4 KSP preconditioned resid norm 6.905538449399e+15 true resid norm > 1.160515328667e+15 ||r(i)||/||b|| 1.659701607116e-03 > 5 KSP preconditioned resid norm 5.388961375964e+15 true resid norm > 9.343353155170e+14 ||r(i)||/||b|| 1.336232091419e-03 > 6 KSP preconditioned resid norm 4.457024362952e+15 true resid norm > 7.443375401358e+14 ||r(i)||/||b|| 1.064508310303e-03 > 7 KSP preconditioned resid norm 3.604925656449e+15 true resid norm > 7.177624609068e+14 ||r(i)||/||b|| 1.026502175773e-03 > 8 KSP preconditioned resid norm 2.616540301450e+15 true resid norm > 4.566819450076e+14 ||r(i)||/||b|| 6.531199884630e-04 > 9 KSP preconditioned resid norm 1.730734442406e+15 true resid norm > 3.416454932875e+14 ||r(i)||/||b|| 4.886015378397e-04 > 10 KSP preconditioned resid norm 9.605665765368e+14 true resid norm > 1.715807715453e+14 ||r(i)||/||b|| 2.453848520993e-04 > 11 KSP preconditioned resid norm 3.885455022454e+14 true resid norm > 7.526175493349e+13 ||r(i)||/||b|| 1.076349898463e-04 > 12 KSP preconditioned resid norm 1.085553513563e+14 true resid norm > 1.877760955419e+13 ||r(i)||/||b|| 2.685464636705e-05 > 13 KSP preconditioned resid norm 5.523836175310e+13 true resid norm > 1.483397872543e+13 ||r(i)||/||b|| 2.121469464673e-05 > 14 KSP preconditioned resid norm 4.329267915925e+13 true resid norm > 6.950523101530e+12 ||r(i)||/||b|| 9.940234374291e-06 > 0 KSP preconditioned resid norm 2.469234905564e+16 true resid norm > 1.581399294540e+16 ||r(i)||/||b|| 1.421323674244e-01 > 1 KSP preconditioned resid norm 1.094883401656e+16 true resid norm > 1.827560974935e+15 ||r(i)||/||b|| 1.642567875658e-02 > 2 KSP preconditioned resid norm 7.902898352613e+15 true resid norm > 1.331982197511e+15 ||r(i)||/||b|| 1.197153582609e-02 > 3 KSP preconditioned resid norm 6.757313480243e+15 true resid norm > 1.177585457449e+15 ||r(i)||/||b|| 1.058385503836e-02 > 4 KSP preconditioned resid norm 6.237102240639e+15 true resid norm > 9.859442147838e+14 ||r(i)||/||b|| 8.861429613598e-03 > 5 KSP preconditioned resid norm 6.056820105725e+15 true resid norm > 9.499166634556e+14 ||r(i)||/||b|| 8.537622642110e-03 > 6 KSP preconditioned resid norm 6.282037371920e+15 true resid norm > 9.440405134922e+14 ||r(i)||/||b|| 8.484809218675e-03 > 7 KSP preconditioned resid norm 6.512961686254e+15 true resid norm > 1.175538616906e+15 ||r(i)||/||b|| 1.056545852755e-02 > 8 KSP preconditioned resid norm 5.899315381141e+15 true resid norm > 9.360504236018e+14 ||r(i)||/||b|| 8.412996211297e-03 > 9 KSP preconditioned resid norm 4.860546577526e+15 true resid norm > 9.070630816401e+14 ||r(i)||/||b|| 8.152464949358e-03 > 10 KSP preconditioned resid norm 3.431569954082e+15 true resid norm > 5.519072654101e+14 ||r(i)||/||b|| 4.960409840975e-03 > 11 KSP preconditioned resid norm 1.788273694951e+15 true resid norm > 3.211455497861e+14 ||r(i)||/||b|| 2.886379008546e-03 > 12 KSP preconditioned resid norm 6.021242921522e+14 true resid norm > 9.258232372098e+13 ||r(i)||/||b|| 8.321076718289e-04 > 13 KSP preconditioned resid norm 3.678076314547e+14 true resid norm > 7.160952479278e+13 ||r(i)||/||b|| 6.436091962400e-04 > 14 KSP preconditioned resid norm 3.596149520932e+14 true resid norm > 5.543217762956e+13 ||r(i)||/||b|| 4.982110884443e-04 > 15 KSP preconditioned resid norm 1.443083471369e+14 true resid norm > 3.905973798110e+13 ||r(i)||/||b|| 3.510595362853e-04 > 16 KSP preconditioned resid norm 1.158730309561e+14 true resid norm > 1.821149714099e+13 ||r(i)||/||b|| 1.636805588525e-04 > 17 KSP preconditioned resid norm 9.042472136695e+13 true resid norm > 2.428568415416e+13 ||r(i)||/||b|| 2.182738916901e-04 > 18 KSP preconditioned resid norm 7.667332005406e+13 true resid norm > 1.178343148104e+13 ||r(i)||/||b|| 1.059066497984e-04 > 19 KSP preconditioned resid norm 6.435860750477e+13 true resid norm > 1.303667702134e+13 ||r(i)||/||b|| 1.171705194752e-04 > 20 KSP preconditioned resid norm 5.245023814970e+13 true resid norm > 8.751976411105e+12 ||r(i)||/||b|| 7.866066029293e-05 > 21 KSP preconditioned resid norm 4.161239918955e+13 true resid norm > 6.811593020631e+12 ||r(i)||/||b|| 6.122096078432e-05 > 22 KSP preconditioned resid norm 3.107777724805e+13 true resid norm > 6.092648679105e+12 ||r(i)||/||b|| 5.475926185348e-05 > 23 KSP preconditioned resid norm 2.312582950537e+13 true resid norm > 3.536350440396e+12 ||r(i)||/||b|| 3.178386773481e-05 > 24 KSP preconditioned resid norm 1.680324737468e+13 true resid norm > 4.383700522921e+12 ||r(i)||/||b|| 3.939964660118e-05 > 25 KSP preconditioned resid norm 1.300784122000e+13 true resid norm > 1.953230538144e+12 ||r(i)||/||b|| 1.755516658383e-05 > 26 KSP preconditioned resid norm 9.922666654442e+12 true resid norm > 3.054726768240e+12 ||r(i)||/||b|| 2.745514993611e-05 > 27 KSP preconditioned resid norm 7.226267347787e+12 true resid norm > 1.085334598547e+12 ||r(i)||/||b|| 9.754726492653e-06 > 0 SNES Function norm 0.000000000000e+00 > 2 TS dt 4e-12 time 5e-12 > 0 KSP preconditioned resid norm 2.220258026867e+19 true resid norm > 1.204245784292e+19 ||r(i)||/||b|| 2.302282866910e+02 > 1 KSP preconditioned resid norm 5.390122646802e+18 true resid norm > 1.775868988684e+18 ||r(i)||/||b|| 3.395114851018e+01 > 2 KSP preconditioned resid norm 2.218369826326e+18 true resid norm > 5.319781926844e+17 ||r(i)||/||b|| 1.017038460556e+01 > 3 KSP preconditioned resid norm 1.401713665358e+18 true resid norm > 2.963575488339e+17 ||r(i)||/||b|| 5.665777834223e+00 > 4 KSP preconditioned resid norm 1.004075064815e+18 true resid norm > 1.656255011022e+17 ||r(i)||/||b|| 3.166436274762e+00 > 5 KSP preconditioned resid norm 8.672862633339e+17 true resid norm > 1.510357773334e+17 ||r(i)||/||b|| 2.887509235913e+00 > 6 KSP preconditioned resid norm 7.389427240230e+17 true resid norm > 1.526197236053e+17 ||r(i)||/||b|| 2.917791196717e+00 > 7 KSP preconditioned resid norm 5.566238898025e+17 true resid norm > 1.017349876189e+17 ||r(i)||/||b|| 1.944974373301e+00 > 8 KSP preconditioned resid norm 3.566703993070e+17 true resid norm > 6.306898440820e+16 ||r(i)||/||b|| 1.205755869196e+00 > 9 KSP preconditioned resid norm 1.893399021800e+17 true resid norm > 3.268112455206e+16 ||r(i)||/||b|| 6.247993068277e-01 > 10 KSP preconditioned resid norm 6.895422631218e+16 true resid norm > 1.153042493210e+16 ||r(i)||/||b|| 2.204392169408e-01 > 11 KSP preconditioned resid norm 5.104209415340e+16 true resid norm > 9.274112790250e+15 ||r(i)||/||b|| 1.773029331827e-01 > 12 KSP preconditioned resid norm 4.991932910946e+16 true resid norm > 8.397724827599e+15 ||r(i)||/||b|| 1.605481060744e-01 > 13 KSP preconditioned resid norm 4.919289629176e+16 true resid norm > 8.425037065615e+15 ||r(i)||/||b|| 1.610702627509e-01 > 14 KSP preconditioned resid norm 5.687556145039e+16 true resid norm > 9.761935947260e+15 ||r(i)||/||b|| 1.866291596982e-01 > 15 KSP preconditioned resid norm 1.615122498031e+17 true resid norm > 4.345955527842e+16 ||r(i)||/||b|| 8.308618624718e-01 > 16 KSP preconditioned resid norm 4.126010760136e+16 true resid norm > 7.099883585094e+15 ||r(i)||/||b|| 1.357359149456e-01 > 17 KSP preconditioned resid norm 2.161057719247e+16 true resid norm > 5.853862804145e+15 ||r(i)||/||b|| 1.119144298865e-01 > 18 KSP preconditioned resid norm 1.411351808998e+16 true resid norm > 2.426558230375e+15 ||r(i)||/||b|| 4.639105664496e-02 > 19 KSP preconditioned resid norm 1.083945264815e+16 true resid norm > 2.485250997240e+15 ||r(i)||/||b|| 4.751314777723e-02 > 20 KSP preconditioned resid norm 9.014213050640e+15 true resid norm > 1.580604299892e+15 ||r(i)||/||b|| 3.021806882344e-02 > 21 KSP preconditioned resid norm 7.767372671831e+15 true resid norm > 1.339058517057e+15 ||r(i)||/||b|| 2.560018496078e-02 > 22 KSP preconditioned resid norm 6.894209717493e+15 true resid norm > 1.176401403286e+15 ||r(i)||/||b|| 2.249049845740e-02 > 23 KSP preconditioned resid norm 6.197326579110e+15 true resid norm > 1.015741229888e+15 ||r(i)||/||b|| 1.941898955585e-02 > 24 KSP preconditioned resid norm 5.635415722302e+15 true resid norm > 9.407688590092e+14 ||r(i)||/||b|| 1.798566417314e-02 > 25 KSP preconditioned resid norm 5.240066189102e+15 true resid norm > 8.479712706847e+14 ||r(i)||/||b|| 1.621155542826e-02 > 26 KSP preconditioned resid norm 5.077381341025e+15 true resid norm > 8.489262350542e+14 ||r(i)||/||b|| 1.622981248288e-02 > 27 KSP preconditioned resid norm 4.908170126730e+15 true resid norm > 9.447288544322e+14 ||r(i)||/||b|| 1.806137155559e-02 > 28 KSP preconditioned resid norm 4.807291179172e+15 true resid norm > 7.514342780137e+14 ||r(i)||/||b|| 1.436595657171e-02 > 29 KSP preconditioned resid norm 4.874566711284e+15 true resid norm > 7.721244730622e+14 ||r(i)||/||b|| 1.476151271311e-02 > 30 KSP preconditioned resid norm 5.131077173970e+15 true resid norm > 8.050523689285e+14 ||r(i)||/||b|| 1.539102980576e-02 > 31 KSP preconditioned resid norm 5.679198106771e+15 true resid norm > 8.990076632730e+14 ||r(i)||/||b|| 1.718727162987e-02 > 32 KSP preconditioned resid norm 8.479698883566e+15 true resid norm > 1.362468331537e+15 ||r(i)||/||b|| 2.604773491692e-02 > 33 KSP preconditioned resid norm 1.652298710742e+16 true resid norm > 2.658951324175e+15 ||r(i)||/||b|| 5.083395895960e-02 > 34 KSP preconditioned resid norm 3.352309602032e+16 true resid norm > 5.485872403557e+15 ||r(i)||/||b|| 1.048791717564e-01 > 35 KSP preconditioned resid norm 6.378357733800e+15 true resid norm > 1.047516366293e+15 ||r(i)||/||b|| 2.002646813784e-02 > 36 KSP preconditioned resid norm 3.388128874134e+15 true resid norm > 5.554504439336e+14 ||r(i)||/||b|| 1.061912823085e-02 > 37 KSP preconditioned resid norm 2.152326148598e+15 true resid norm > 3.939853481438e+14 ||r(i)||/||b|| 7.532230784419e-03 > 38 KSP preconditioned resid norm 1.818288964313e+15 true resid norm > 2.940442100180e+14 ||r(i)||/||b|| 5.621551311776e-03 > 39 KSP preconditioned resid norm 1.660791920226e+15 true resid norm > 2.869955671543e+14 ||r(i)||/||b|| 5.486795019399e-03 > 40 KSP preconditioned resid norm 1.530575944029e+15 true resid norm > 2.452126997689e+14 ||r(i)||/||b|| 4.687988156493e-03 > 41 KSP preconditioned resid norm 1.370742276038e+15 true resid norm > 2.241019638220e+14 ||r(i)||/||b|| 4.284392094026e-03 > 42 KSP preconditioned resid norm 1.242467362496e+15 true resid norm > 2.048557595739e+14 ||r(i)||/||b|| 3.916442238011e-03 > 43 KSP preconditioned resid norm 1.151462192104e+15 true resid norm > 1.847175725318e+14 ||r(i)||/||b|| 3.531439412157e-03 > 44 KSP preconditioned resid norm 1.085430472672e+15 true resid norm > 1.860586924987e+14 ||r(i)||/||b|| 3.557079008015e-03 > 45 KSP preconditioned resid norm 1.036632239565e+15 true resid norm > 1.638613644087e+14 ||r(i)||/||b|| 3.132709424833e-03 > 46 KSP preconditioned resid norm 1.028748911897e+15 true resid norm > 1.653234224458e+14 ||r(i)||/||b|| 3.160661120519e-03 > 47 KSP preconditioned resid norm 9.769896259678e+14 true resid norm > 1.886232302190e+14 ||r(i)||/||b|| 3.606107963166e-03 > 48 KSP preconditioned resid norm 1.056096824670e+15 true resid norm > 1.655410058571e+14 ||r(i)||/||b|| 3.164820890613e-03 > 49 KSP preconditioned resid norm 7.777695958066e+14 true resid norm > 2.880991932726e+14 ||r(i)||/||b|| 5.507894196468e-03 > 50 KSP preconditioned resid norm 1.009825961040e+15 true resid norm > 1.550789761056e+14 ||r(i)||/||b|| 2.964807303982e-03 > 51 KSP preconditioned resid norm 9.471596692833e+14 true resid norm > 2.053798093618e+14 ||r(i)||/||b|| 3.926461046995e-03 > 52 KSP preconditioned resid norm 8.991518749617e+14 true resid norm > 1.386233583178e+14 ||r(i)||/||b|| 2.650208013776e-03 > 53 KSP preconditioned resid norm 7.186213371144e+13 true resid norm > 1.103090149144e+13 ||r(i)||/||b|| 2.108893038414e-04 > 0 KSP preconditioned resid norm 6.542826642292e+18 true resid norm > 3.545903145424e+18 ||r(i)||/||b|| 4.156717341790e+02 > 1 KSP preconditioned resid norm 1.591157510957e+18 true resid norm > 5.223152980213e+17 ||r(i)||/||b|| 6.122888776501e+01 > 2 KSP preconditioned resid norm 6.570525108060e+17 true resid norm > 1.552502546069e+17 ||r(i)||/||b|| 1.819935286374e+01 > 3 KSP preconditioned resid norm 4.165816250413e+17 true resid norm > 8.738489300540e+16 ||r(i)||/||b|| 1.024377387845e+01 > 4 KSP preconditioned resid norm 2.995010683652e+17 true resid norm > 4.934680552583e+16 ||r(i)||/||b|| 5.784724339013e+00 > 5 KSP preconditioned resid norm 2.594493782706e+17 true resid norm > 4.469794627497e+16 ||r(i)||/||b|| 5.239757568206e+00 > 6 KSP preconditioned resid norm 2.219609972421e+17 true resid norm > 4.652118621437e+16 ||r(i)||/||b|| 5.453488535001e+00 > 7 KSP preconditioned resid norm 1.674912012123e+17 true resid norm > 2.988815541157e+16 ||r(i)||/||b|| 3.503666310619e+00 > 8 KSP preconditioned resid norm 1.081901407775e+17 true resid norm > 1.908145564246e+16 ||r(i)||/||b|| 2.236841062001e+00 > 9 KSP preconditioned resid norm 5.710123690945e+16 true resid norm > 9.635193037737e+15 ||r(i)||/||b|| 1.129494302267e+00 > 10 KSP preconditioned resid norm 1.779373863118e+16 true resid norm > 2.952490629534e+15 ||r(i)||/||b|| 3.461084101266e-01 > 11 KSP preconditioned resid norm 1.125622359514e+16 true resid norm > 2.030418337832e+15 ||r(i)||/||b|| 2.380176437376e-01 > 12 KSP preconditioned resid norm 1.086650353983e+16 true resid norm > 1.877894898988e+15 ||r(i)||/||b|| 2.201379443417e-01 > 13 KSP preconditioned resid norm 1.075411372217e+16 true resid norm > 1.822725565845e+15 ||r(i)||/||b|| 2.136706688859e-01 > 14 KSP preconditioned resid norm 1.173963298627e+16 true resid norm > 2.130483772307e+15 ||r(i)||/||b|| 2.497479056692e-01 > 15 KSP preconditioned resid norm 4.943816039086e+16 true resid norm > 1.362098103563e+16 ||r(i)||/||b|| 1.596731939959e+00 > 16 KSP preconditioned resid norm 1.779827523151e+17 true resid norm > 3.369509176778e+16 ||r(i)||/||b|| 3.949937901296e+00 > 17 KSP preconditioned resid norm 3.250308535899e+16 true resid norm > 8.719651405683e+15 ||r(i)||/||b|| 1.022169098418e+00 > 18 KSP preconditioned resid norm 2.074788387365e+16 true resid norm > 3.942900424212e+15 ||r(i)||/||b|| 4.622101027047e-01 > 19 KSP preconditioned resid norm 1.482962263858e+16 true resid norm > 3.390403569469e+15 ||r(i)||/||b|| 3.974431543926e-01 > 20 KSP preconditioned resid norm 1.133931148892e+16 true resid norm > 2.488262609273e+15 ||r(i)||/||b|| 2.916888565398e-01 > 21 KSP preconditioned resid norm 8.550489811072e+15 true resid norm > 1.657275963132e+15 ||r(i)||/||b|| 1.942756881268e-01 > 22 KSP preconditioned resid norm 6.584107021814e+15 true resid norm > 1.285719379545e+15 ||r(i)||/||b|| 1.507196283273e-01 > 23 KSP preconditioned resid norm 4.778317630297e+15 true resid norm > 9.061620975056e+14 ||r(i)||/||b|| 1.062256793459e-01 > 24 KSP preconditioned resid norm 3.026263700342e+15 true resid norm > 6.175894175717e+14 ||r(i)||/||b|| 7.239748342928e-02 > 25 KSP preconditioned resid norm 1.568620488361e+15 true resid norm > 2.753144451691e+14 ||r(i)||/||b|| 3.227398723952e-02 > 26 KSP preconditioned resid norm 1.178330886619e+15 true resid norm > 2.275547590347e+14 ||r(i)||/||b|| 2.667531442045e-02 > 27 KSP preconditioned resid norm 1.073508363176e+15 true resid norm > 2.033900922703e+14 ||r(i)||/||b|| 2.384258929293e-02 > 28 KSP preconditioned resid norm 1.011358091649e+15 true resid norm > 1.668017917552e+14 ||r(i)||/||b|| 1.955349235427e-02 > 29 KSP preconditioned resid norm 9.448460925412e+14 true resid norm > 1.798337339042e+14 ||r(i)||/||b|| 2.108117367287e-02 > 30 KSP preconditioned resid norm 8.970299972493e+14 true resid norm > 1.456018479256e+14 ||r(i)||/||b|| 1.706830958004e-02 > 31 KSP preconditioned resid norm 8.656072021210e+14 true resid norm > 1.578084000747e+14 ||r(i)||/||b|| 1.849923380218e-02 > 32 KSP preconditioned resid norm 8.314911059315e+14 true resid norm > 1.333543211106e+14 ||r(i)||/||b|| 1.563258206526e-02 > 33 KSP preconditioned resid norm 8.004072124456e+14 true resid norm > 1.582598207144e+14 ||r(i)||/||b|| 1.855215199888e-02 > 34 KSP preconditioned resid norm 7.705686923615e+14 true resid norm > 1.334906781329e+14 ||r(i)||/||b|| 1.564856664172e-02 > 35 KSP preconditioned resid norm 7.648238175354e+14 true resid norm > 1.202157111000e+14 ||r(i)||/||b|| 1.409239650919e-02 > 36 KSP preconditioned resid norm 7.709241111814e+14 true resid norm > 1.292601566237e+14 ||r(i)||/||b|| 1.515263989468e-02 > 37 KSP preconditioned resid norm 8.026855071425e+14 true resid norm > 1.264132558814e+14 ||r(i)||/||b|| 1.481890935550e-02 > 38 KSP preconditioned resid norm 8.397026301556e+14 true resid norm > 1.379015425365e+14 ||r(i)||/||b|| 1.616563424923e-02 > 39 KSP preconditioned resid norm 8.772641473437e+14 true resid norm > 1.384855714837e+14 ||r(i)||/||b|| 1.623409757587e-02 > 40 KSP preconditioned resid norm 9.403663540144e+14 true resid norm > 1.523096876425e+14 ||r(i)||/||b|| 1.785464221614e-02 > 41 KSP preconditioned resid norm 1.191025293024e+15 true resid norm > 1.883988208054e+14 ||r(i)||/||b|| 2.208522380610e-02 > 42 KSP preconditioned resid norm 2.105592809750e+15 true resid norm > 3.674437948704e+14 ||r(i)||/||b|| 4.307393438656e-02 > 43 KSP preconditioned resid norm 1.653069974154e+16 true resid norm > 2.622043044873e+15 ||r(i)||/||b|| 3.073713902652e-01 > 44 KSP preconditioned resid norm 1.224677806349e+15 true resid norm > 2.078786938958e+14 ||r(i)||/||b|| 2.436876971726e-02 > 45 KSP preconditioned resid norm 6.142260967855e+14 true resid norm > 9.787693095074e+13 ||r(i)||/||b|| 1.147371260744e-02 > 46 KSP preconditioned resid norm 4.190317366481e+14 true resid norm > 7.181248026289e+13 ||r(i)||/||b|| 8.418283574692e-03 > 47 KSP preconditioned resid norm 2.851377922408e+14 true resid norm > 4.551212829362e+13 ||r(i)||/||b|| 5.335200798815e-03 > 48 KSP preconditioned resid norm 1.874725496828e+14 true resid norm > 3.515118139860e+13 ||r(i)||/||b|| 4.120629337903e-03 > 49 KSP preconditioned resid norm 1.260137969667e+14 true resid norm > 2.018341951104e+13 ||r(i)||/||b|| 2.366019782758e-03 > 50 KSP preconditioned resid norm 8.902880553848e+13 true resid norm > 2.353973309483e+13 ||r(i)||/||b|| 2.759466707450e-03 > 51 KSP preconditioned resid norm 6.273228119323e+13 true resid norm > 1.015825242680e+13 ||r(i)||/||b|| 1.190810416783e-03 > 52 KSP preconditioned resid norm 4.428322011916e+13 true resid norm > 1.394037459429e+13 ||r(i)||/||b|| 1.634173141528e-03 > 0 KSP preconditioned resid norm 1.892236961136e+19 true resid norm > 1.028105836629e+19 ||r(i)||/||b|| 7.839481546009e+01 > 1 KSP preconditioned resid norm 4.580244542528e+18 true resid norm > 1.519483928522e+18 ||r(i)||/||b|| 1.158632291804e+01 > 2 KSP preconditioned resid norm 1.874052673028e+18 true resid norm > 4.645668462025e+17 ||r(i)||/||b|| 3.542401071890e+00 > 3 KSP preconditioned resid norm 1.172977406681e+18 true resid norm > 2.550460309107e+17 ||r(i)||/||b|| 1.944769284904e+00 > 4 KSP preconditioned resid norm 8.299170411979e+17 true resid norm > 1.381052822672e+17 ||r(i)||/||b|| 1.053076223446e+00 > 5 KSP preconditioned resid norm 7.051011041666e+17 true resid norm > 1.287511951247e+17 ||r(i)||/||b|| 9.817497209388e-01 > 6 KSP preconditioned resid norm 5.862228682149e+17 true resid norm > 1.166604371743e+17 ||r(i)||/||b|| 8.895556389170e-01 > 7 KSP preconditioned resid norm 4.329574557115e+17 true resid norm > 8.609505694316e+16 ||r(i)||/||b|| 6.564894255647e-01 > 8 KSP preconditioned resid norm 2.701974155126e+17 true resid norm > 4.802624326608e+16 ||r(i)||/||b|| 3.662082583277e-01 > 9 KSP preconditioned resid norm 1.546976756797e+17 true resid norm > 2.878555741954e+16 ||r(i)||/||b|| 2.194947622532e-01 > 10 KSP preconditioned resid norm 9.916116392638e+16 true resid norm > 1.649766880508e+16 ||r(i)||/||b|| 1.257975254509e-01 > 11 KSP preconditioned resid norm 9.248895040376e+16 true resid norm > 1.560136782751e+16 ||r(i)||/||b|| 1.189630783317e-01 > 12 KSP preconditioned resid norm 9.251831545179e+16 true resid norm > 1.508056895396e+16 ||r(i)||/||b|| 1.149918985048e-01 > 13 KSP preconditioned resid norm 7.096157383949e+16 true resid norm > 1.151221892162e+16 ||r(i)||/||b|| 8.778262370879e-02 > 14 KSP preconditioned resid norm 2.017629866031e+17 true resid norm > 3.268014415733e+16 ||r(i)||/||b|| 2.491916473135e-01 > 15 KSP preconditioned resid norm 1.669814416925e+16 true resid norm > 4.397468530146e+15 ||r(i)||/||b|| 3.353144410138e-02 > 16 KSP preconditioned resid norm 1.171914014315e+16 true resid norm > 1.865370183430e+15 ||r(i)||/||b|| 1.422376433288e-02 > 17 KSP preconditioned resid norm 1.021030261775e+16 true resid norm > 2.000923353135e+15 ||r(i)||/||b|| 1.525738026477e-02 > 18 KSP preconditioned resid norm 9.416598060128e+15 true resid norm > 1.541246281727e+15 ||r(i)||/||b|| 1.175226455582e-02 > 19 KSP preconditioned resid norm 8.821999311745e+15 true resid norm > 1.426601243635e+15 ||r(i)||/||b|| 1.087807667707e-02 > 20 KSP preconditioned resid norm 8.323362047556e+15 true resid norm > 1.490210405686e+15 ||r(i)||/||b|| 1.136310733664e-02 > 21 KSP preconditioned resid norm 7.861641988008e+15 true resid norm > 1.251748347072e+15 ||r(i)||/||b|| 9.544793655962e-03 > 22 KSP preconditioned resid norm 7.464952525350e+15 true resid norm > 1.325467789570e+15 ||r(i)||/||b|| 1.010691692037e-02 > 23 KSP preconditioned resid norm 7.139096535480e+15 true resid norm > 1.125905927178e+15 ||r(i)||/||b|| 8.585223839986e-03 > 24 KSP preconditioned resid norm 5.858113828393e+15 true resid norm > 2.042448137714e+15 ||r(i)||/||b|| 1.557401379686e-02 > 25 KSP preconditioned resid norm 5.027356938946e+15 true resid norm > 7.946522117539e+14 ||r(i)||/||b|| 6.059358022871e-03 > 26 KSP preconditioned resid norm 5.122614017475e+15 true resid norm > 7.976827488389e+14 ||r(i)||/||b|| 6.082466382639e-03 > 27 KSP preconditioned resid norm 5.425317067876e+15 true resid norm > 8.948524578548e+14 ||r(i)||/||b|| 6.823401910404e-03 > 28 KSP preconditioned resid norm 9.316529191423e+15 true resid norm > 1.453760069771e+15 ||r(i)||/||b|| 1.108516733711e-02 > 29 KSP preconditioned resid norm 9.523628080058e+15 true resid norm > 2.109739324167e+15 ||r(i)||/||b|| 1.608712051759e-02 > 30 KSP preconditioned resid norm 2.300173794637e+15 true resid norm > 3.594261469430e+14 ||r(i)||/||b|| 2.740685390281e-03 > 31 KSP preconditioned resid norm 1.385027169120e+15 true resid norm > 2.935564708865e+14 ||r(i)||/||b|| 2.238417927644e-03 > 32 KSP preconditioned resid norm 9.719758142764e+14 true resid norm > 1.523176597454e+14 ||r(i)||/||b|| 1.161448014555e-03 > 33 KSP preconditioned resid norm 8.580268959656e+14 true resid norm > 1.580637131624e+14 ||r(i)||/||b|| 1.205262647368e-03 > 0 KSP preconditioned resid norm 6.470797769938e+18 true resid norm > 3.511548722307e+18 ||r(i)||/||b|| 1.662246831388e+02 > 1 KSP preconditioned resid norm 1.571597715354e+18 true resid norm > 5.187477038218e+17 ||r(i)||/||b|| 2.455573865428e+01 > 2 KSP preconditioned resid norm 6.468337546861e+17 true resid norm > 1.566996108585e+17 ||r(i)||/||b|| 7.417622599812e+00 > 3 KSP preconditioned resid norm 4.084044703454e+17 true resid norm > 8.688412887362e+16 ||r(i)||/||b|| 4.112796926342e+00 > 4 KSP preconditioned resid norm 2.920894102259e+17 true resid norm > 4.831956260441e+16 ||r(i)||/||b|| 2.287282512214e+00 > 5 KSP preconditioned resid norm 2.515869150351e+17 true resid norm > 4.435569840236e+16 ||r(i)||/||b|| 2.099646764259e+00 > 6 KSP preconditioned resid norm 2.133085363553e+17 true resid norm > 4.355632900946e+16 ||r(i)||/||b|| 2.061807356479e+00 > 7 KSP preconditioned resid norm 1.601295664581e+17 true resid norm > 2.999352167191e+16 ||r(i)||/||b|| 1.419790534148e+00 > 8 KSP preconditioned resid norm 1.019894513782e+17 true resid norm > 1.811122934468e+16 ||r(i)||/||b|| 8.573235336161e-01 > 9 KSP preconditioned resid norm 5.552366492146e+16 true resid norm > 9.849650352808e+15 ||r(i)||/||b|| 4.662486949199e-01 > 10 KSP preconditioned resid norm 2.614329828139e+16 true resid norm > 4.400937150634e+15 ||r(i)||/||b|| 2.083252835795e-01 > 11 KSP preconditioned resid norm 2.230447954838e+16 true resid norm > 4.002038016135e+15 ||r(i)||/||b|| 1.894427654999e-01 > 12 KSP preconditioned resid norm 2.202888172936e+16 true resid norm > 3.669015023581e+15 ||r(i)||/||b|| 1.736785982356e-01 > 13 KSP preconditioned resid norm 2.171444770780e+16 true resid norm > 3.746639147837e+15 ||r(i)||/||b|| 1.773530582755e-01 > 14 KSP preconditioned resid norm 2.756620760322e+16 true resid norm > 4.738417417687e+15 ||r(i)||/||b|| 2.243004429444e-01 > 15 KSP preconditioned resid norm 1.911455835947e+16 true resid norm > 7.274462982878e+15 ||r(i)||/||b|| 3.443481494796e-01 > 16 KSP preconditioned resid norm 8.270124664006e+15 true resid norm > 1.450957167614e+15 ||r(i)||/||b|| 6.868334017480e-02 > 17 KSP preconditioned resid norm 5.454864800054e+15 true resid norm > 1.779587764442e+15 ||r(i)||/||b|| 8.423958647728e-02 > 18 KSP preconditioned resid norm 4.188178169772e+15 true resid norm > 7.356207570842e+14 ||r(i)||/||b|| 3.482176581514e-02 > 19 KSP preconditioned resid norm 3.485885419242e+15 true resid norm > 7.684000525717e+14 ||r(i)||/||b|| 3.637342533543e-02 > 20 KSP preconditioned resid norm 3.031526098420e+15 true resid norm > 5.668894180734e+14 ||r(i)||/||b|| 2.683460243492e-02 > 21 KSP preconditioned resid norm 2.653824581081e+15 true resid norm > 5.337896513987e+14 ||r(i)||/||b|| 2.526777290683e-02 > 22 KSP preconditioned resid norm 2.412624108418e+15 true resid norm > 3.917881686680e+14 ||r(i)||/||b|| 1.854590932504e-02 > 23 KSP preconditioned resid norm 2.249293018506e+15 true resid norm > 3.520070992955e+14 ||r(i)||/||b|| 1.666280982271e-02 > 24 KSP preconditioned resid norm 2.107969657718e+15 true resid norm > 3.433991961929e+14 ||r(i)||/||b|| 1.625534118740e-02 > 25 KSP preconditioned resid norm 1.995899579616e+15 true resid norm > 3.124097002184e+14 ||r(i)||/||b|| 1.478840464277e-02 > 26 KSP preconditioned resid norm 1.940326192847e+15 true resid norm > 3.197343858652e+14 ||r(i)||/||b|| 1.513513016106e-02 > 27 KSP preconditioned resid norm 1.843618399864e+15 true resid norm > 4.284525431489e+14 ||r(i)||/||b|| 2.028147517148e-02 > 28 KSP preconditioned resid norm 1.800569430365e+15 true resid norm > 2.810328443449e+14 ||r(i)||/||b|| 1.330313180793e-02 > 29 KSP preconditioned resid norm 1.911265522964e+15 true resid norm > 3.078500227429e+14 ||r(i)||/||b|| 1.457256513619e-02 > 30 KSP preconditioned resid norm 2.245578740852e+15 true resid norm > 3.522723048093e+14 ||r(i)||/||b|| 1.667536374293e-02 > 31 KSP preconditioned resid norm 3.129444954320e+15 true resid norm > 5.056459987900e+14 ||r(i)||/||b|| 2.393554883500e-02 > 32 KSP preconditioned resid norm 2.268235755551e+16 true resid norm > 3.663405821329e+15 ||r(i)||/||b|| 1.734130778226e-01 > 33 KSP preconditioned resid norm 2.666354866266e+15 true resid norm > 4.420681584183e+14 ||r(i)||/||b|| 2.092599174035e-02 > 34 KSP preconditioned resid norm 1.195729215971e+15 true resid norm > 1.967322086000e+14 ||r(i)||/||b|| 9.312628593187e-03 > 35 KSP preconditioned resid norm 7.390918117697e+14 true resid norm > 1.247185996576e+14 ||r(i)||/||b|| 5.903751122089e-03 > 36 KSP preconditioned resid norm 5.589675490221e+14 true resid norm > 9.067132579829e+13 ||r(i)||/||b|| 4.292069850790e-03 > 37 KSP preconditioned resid norm 4.470335591004e+14 true resid norm > 9.109636100840e+13 ||r(i)||/||b|| 4.312189561126e-03 > 38 KSP preconditioned resid norm 4.031608678043e+14 true resid norm > 6.399106058589e+13 ||r(i)||/||b|| 3.029117523568e-03 > 39 KSP preconditioned resid norm 3.769205171045e+14 true resid norm > 7.200688789282e+13 ||r(i)||/||b|| 3.408559319641e-03 > 40 KSP preconditioned resid norm 3.535103364248e+14 true resid norm > 5.569365215364e+13 ||r(i)||/||b|| 2.636346641945e-03 > 41 KSP preconditioned resid norm 3.299087996116e+14 true resid norm > 5.691184302265e+13 ||r(i)||/||b|| 2.694011623187e-03 > 42 KSP preconditioned resid norm 3.102396074559e+14 true resid norm > 5.072539063269e+13 ||r(i)||/||b|| 2.401166166782e-03 > 43 KSP preconditioned resid norm 2.923631153501e+14 true resid norm > 5.690604160219e+13 ||r(i)||/||b|| 2.693737003823e-03 > 44 KSP preconditioned resid norm 2.831787236990e+14 true resid norm > 4.398532156025e+13 ||r(i)||/||b|| 2.082114393761e-03 > 45 KSP preconditioned resid norm 2.880585984465e+14 true resid norm > 4.666639157225e+13 ||r(i)||/||b|| 2.209027060639e-03 > 46 KSP preconditioned resid norm 3.054400711070e+14 true resid norm > 4.771426500640e+13 ||r(i)||/||b|| 2.258629798159e-03 > 47 KSP preconditioned resid norm 3.761477931303e+14 true resid norm > 5.957625439087e+13 ||r(i)||/||b|| 2.820135726954e-03 > 48 KSP preconditioned resid norm 9.608784364697e+14 true resid norm > 1.580857575603e+14 ||r(i)||/||b|| 7.483238034628e-03 > 49 KSP preconditioned resid norm 1.902148260101e+15 true resid norm > 2.982038509627e+14 ||r(i)||/||b|| 1.411594841961e-02 > 50 KSP preconditioned resid norm 2.165055327372e+14 true resid norm > 4.058692191960e+13 ||r(i)||/||b|| 1.921245800409e-03 > 51 KSP preconditioned resid norm 2.195213865618e+14 true resid norm > 3.429224338721e+13 ||r(i)||/||b|| 1.623277289290e-03 > 52 KSP preconditioned resid norm 1.864119738188e+14 true resid norm > 5.012858400476e+13 ||r(i)||/||b|| 2.372915386152e-03 > 53 KSP preconditioned resid norm 2.971924491921e+13 true resid norm > 4.598158953060e+12 ||r(i)||/||b|| 2.176610878666e-04 > 0 KSP preconditioned resid norm 6.694681472101e+18 true resid norm > 3.650754008704e+18 ||r(i)||/||b|| 5.229798534711e+00 > 1 KSP preconditioned resid norm 1.597667311920e+18 true resid norm > 5.366749653247e+17 ||r(i)||/||b|| 7.688006205237e-01 > 2 KSP preconditioned resid norm 6.300446691769e+17 true resid norm > 1.705328302298e+17 ||r(i)||/||b|| 2.442926429799e-01 > 3 KSP preconditioned resid norm 3.642174434692e+17 true resid norm > 9.481577406412e+16 ||r(i)||/||b|| 1.358260225383e-01 > 4 KSP preconditioned resid norm 2.247332932327e+17 true resid norm > 3.802800659066e+16 ||r(i)||/||b|| 5.447609251997e-02 > 5 KSP preconditioned resid norm 1.557523729879e+17 true resid norm > 3.829141784861e+16 ||r(i)||/||b|| 5.485343588727e-02 > 6 KSP preconditioned resid norm 1.004692897232e+17 true resid norm > 1.699728149080e+16 ||r(i)||/||b|| 2.434904067015e-02 > 7 KSP preconditioned resid norm 7.103483293598e+16 true resid norm > 1.472858902835e+16 ||r(i)||/||b|| 2.109908066531e-02 > 8 KSP preconditioned resid norm 5.766848376574e+16 true resid norm > 9.397954931890e+15 ||r(i)||/||b|| 1.346281091931e-02 > 9 KSP preconditioned resid norm 4.523472389397e+16 true resid norm > 1.246375028587e+16 ||r(i)||/||b|| 1.785464121292e-02 > 10 KSP preconditioned resid norm 3.195342596604e+16 true resid norm > 5.030521805863e+15 ||r(i)||/||b|| 7.206351210301e-03 > 11 KSP preconditioned resid norm 2.437460193025e+16 true resid norm > 4.974645265358e+15 ||r(i)||/||b|| 7.126306636231e-03 > 12 KSP preconditioned resid norm 1.258045317564e+16 true resid norm > 1.885909944349e+15 ||r(i)||/||b|| 2.701614252848e-03 > 13 KSP preconditioned resid norm 2.495567071066e+15 true resid norm > 4.460991862355e+14 ||r(i)||/||b|| 6.390484992820e-04 > 14 KSP preconditioned resid norm 7.678105595258e+14 true resid norm > 1.316278119117e+14 ||r(i)||/||b|| 1.885602087190e-04 > 15 KSP preconditioned resid norm 5.950875767888e+14 true resid norm > 1.439411571316e+14 ||r(i)||/||b|| 2.061993908263e-04 > 16 KSP preconditioned resid norm 3.263039815738e+14 true resid norm > 5.046549667518e+13 ||r(i)||/||b|| 7.229311532250e-05 > 17 KSP preconditioned resid norm 1.436785883111e+14 true resid norm > 7.588987254543e+13 ||r(i)||/||b|| 1.087141843277e-04 > 18 KSP preconditioned resid norm 5.401544347363e+13 true resid norm > 8.625334293000e+12 ||r(i)||/||b|| 1.235601208391e-05 > 19 KSP preconditioned resid norm 3.528863285477e+13 true resid norm > 1.233589079533e+13 ||r(i)||/||b|| 1.767147922099e-05 > 0 KSP preconditioned resid norm 4.827871909881e+18 true resid norm > 2.629048338874e+18 ||r(i)||/||b|| 2.387243403092e+01 > 1 KSP preconditioned resid norm 1.161059375885e+18 true resid norm > 3.885115498370e+17 ||r(i)||/||b|| 3.527784638492e+00 > 2 KSP preconditioned resid norm 4.696929326234e+17 true resid norm > 1.228954029189e+17 ||r(i)||/||b|| 1.115921816843e+00 > 3 KSP preconditioned resid norm 2.870100194440e+17 true resid norm > 6.759377089469e+16 ||r(i)||/||b|| 6.137687971441e-01 > 4 KSP preconditioned resid norm 1.966638027618e+17 true resid norm > 3.310588396768e+16 ||r(i)||/||b|| 3.006099276942e-01 > 5 KSP preconditioned resid norm 1.592573327953e+17 true resid norm > 3.239413757211e+16 ||r(i)||/||b|| 2.941470876529e-01 > 6 KSP preconditioned resid norm 1.230114084345e+17 true resid norm > 2.280735920899e+16 ||r(i)||/||b|| 2.070966783247e-01 > 7 KSP preconditioned resid norm 8.598639005636e+16 true resid norm > 1.938027655308e+16 ||r(i)||/||b|| 1.759778877677e-01 > 8 KSP preconditioned resid norm 5.549090128976e+16 true resid norm > 9.386098364471e+15 ||r(i)||/||b|| 8.522818340779e-02 > 9 KSP preconditioned resid norm 4.474765035679e+16 true resid norm > 7.579175521646e+15 ||r(i)||/||b|| 6.882085999480e-02 > 10 KSP preconditioned resid norm 4.378403606928e+16 true resid norm > 6.842614495131e+15 ||r(i)||/||b|| 6.213269673237e-02 > 11 KSP preconditioned resid norm 4.079517449398e+16 true resid norm > 7.704845808673e+15 ||r(i)||/||b|| 6.996197847191e-02 > 12 KSP preconditioned resid norm 4.084405622719e+16 true resid norm > 6.377952548774e+15 ||r(i)||/||b|| 5.791344693880e-02 > 13 KSP preconditioned resid norm 7.523868588173e+16 true resid norm > 1.132249240798e+16 ||r(i)||/||b|| 1.028111385699e-01 > 14 KSP preconditioned resid norm 2.184581375390e+15 true resid norm > 3.502138477296e+14 ||r(i)||/||b|| 3.180031668881e-03 > 15 KSP preconditioned resid norm 2.222269400035e+15 true resid norm > 3.654767263686e+14 ||r(i)||/||b|| 3.318622526282e-03 > 16 KSP preconditioned resid norm 1.669508240008e+15 true resid norm > 3.629680148217e+14 ||r(i)||/||b|| 3.295842781223e-03 > 17 KSP preconditioned resid norm 1.401913603747e+15 true resid norm > 2.161155966391e+14 ||r(i)||/||b|| 1.962385113858e-03 > 18 KSP preconditioned resid norm 1.134372518022e+15 true resid norm > 2.686701598571e+14 ||r(i)||/||b|| 2.439594043376e-03 > 19 KSP preconditioned resid norm 9.057728167563e+14 true resid norm > 1.409896456102e+14 ||r(i)||/||b|| 1.280222186904e-03 > 20 KSP preconditioned resid norm 7.989861774441e+14 true resid norm > 1.659027150307e+14 ||r(i)||/||b|| 1.506439254676e-03 > 21 KSP preconditioned resid norm 7.119014585200e+14 true resid norm > 1.092105820565e+14 ||r(i)||/||b|| 9.916601292842e-04 > 22 KSP preconditioned resid norm 6.306860453827e+14 true resid norm > 1.460692513610e+14 ||r(i)||/||b|| 1.326346311515e-03 > 23 KSP preconditioned resid norm 5.637172379886e+14 true resid norm > 8.672554380666e+13 ||r(i)||/||b|| 7.874902080372e-04 > 24 KSP preconditioned resid norm 4.961067656606e+14 true resid norm > 1.292132364829e+14 ||r(i)||/||b|| 1.173289367962e-03 > 25 KSP preconditioned resid norm 4.386188245672e+14 true resid norm > 8.044020700160e+13 ||r(i)||/||b|| 7.304177358342e-04 > 26 KSP preconditioned resid norm 4.419078090293e+14 true resid norm > 6.703649620362e+13 ||r(i)||/||b|| 6.087085998464e-04 > 27 KSP preconditioned resid norm 4.436677280477e+14 true resid norm > 1.368871842229e+14 ||r(i)||/||b|| 1.242970784036e-03 > 28 KSP preconditioned resid norm 7.847692003240e+14 true resid norm > 1.196853324186e+14 ||r(i)||/||b|| 1.086773552386e-03 > 29 KSP preconditioned resid norm 1.771704167839e+15 true resid norm > 6.548256652554e+14 ||r(i)||/||b|| 5.945985193355e-03 > 30 KSP preconditioned resid norm 1.053899282998e+15 true resid norm > 1.880673778730e+14 ||r(i)||/||b|| 1.707700084953e-03 > 31 KSP preconditioned resid norm 2.462670432282e+14 true resid norm > 3.919805233311e+13 ||r(i)||/||b|| 3.559283808617e-04 > 32 KSP preconditioned resid norm 1.093017734594e+14 true resid norm > 1.903326631551e+13 ||r(i)||/||b|| 1.728269457018e-04 > 33 KSP preconditioned resid norm 6.432042300545e+13 true resid norm > 1.002471721316e+13 ||r(i)||/||b|| 9.102700654495e-05 > 34 KSP preconditioned resid norm 3.805689715853e+13 true resid norm > 7.047382673791e+12 ||r(i)||/||b|| 6.399204437707e-05 > 35 KSP preconditioned resid norm 2.527168724251e+13 true resid norm > 4.009865388649e+12 ||r(i)||/||b|| 3.641060742320e-05 > 36 KSP preconditioned resid norm 1.580947387777e+13 true resid norm > 2.531811988645e+12 ||r(i)||/||b|| 2.298950300149e-05 > 37 KSP preconditioned resid norm 1.075548264218e+13 true resid norm > 1.788042513750e+12 ||r(i)||/||b|| 1.623588517670e-05 > 38 KSP preconditioned resid norm 6.581274433602e+12 true resid norm > 1.035380330637e+12 ||r(i)||/||b|| 9.401519277742e-06 > 0 KSP preconditioned resid norm 9.555736051823e+15 true resid norm > 5.892741751019e+15 ||r(i)||/||b|| 1.119235791581e-01 > 1 KSP preconditioned resid norm 4.575036500991e+15 true resid norm > 8.101073993664e+14 ||r(i)||/||b|| 1.538674584270e-02 > 2 KSP preconditioned resid norm 3.765161451471e+15 true resid norm > 6.105904357791e+14 ||r(i)||/||b|| 1.159722754867e-02 > 3 KSP preconditioned resid norm 3.685525899039e+15 true resid norm > 6.160361102521e+14 ||r(i)||/||b|| 1.170065977151e-02 > 4 KSP preconditioned resid norm 3.856425370708e+15 true resid norm > 6.278594964367e+14 ||r(i)||/||b|| 1.192522683307e-02 > 5 KSP preconditioned resid norm 4.281981871434e+15 true resid norm > 6.756473177150e+14 ||r(i)||/||b|| 1.283288310305e-02 > 6 KSP preconditioned resid norm 5.205822014832e+15 true resid norm > 7.760311477213e+14 ||r(i)||/||b|| 1.473951977891e-02 > 7 KSP preconditioned resid norm 6.471690051175e+15 true resid norm > 1.171341036231e+15 ||r(i)||/||b|| 2.224782397211e-02 > 8 KSP preconditioned resid norm 7.016914241690e+15 true resid norm > 1.103022939349e+15 ||r(i)||/||b|| 2.095022664859e-02 > 9 KSP preconditioned resid norm 7.025590013206e+15 true resid norm > 1.330551872513e+15 ||r(i)||/||b|| 2.527178928238e-02 > 10 KSP preconditioned resid norm 6.405136813848e+15 true resid norm > 1.017867183616e+15 ||r(i)||/||b|| 1.933282385541e-02 > 11 KSP preconditioned resid norm 4.854020251396e+15 true resid norm > 8.473369029366e+14 ||r(i)||/||b|| 1.609386308386e-02 > 12 KSP preconditioned resid norm 2.353499572340e+15 true resid norm > 3.616493792413e+14 ||r(i)||/||b|| 6.868974517339e-03 > 13 KSP preconditioned resid norm 1.176204335431e+15 true resid norm > 2.390307465685e+14 ||r(i)||/||b|| 4.540021914275e-03 > 14 KSP preconditioned resid norm 1.050288991204e+15 true resid norm > 1.923001174300e+14 ||r(i)||/||b|| 3.652445385304e-03 > 15 KSP preconditioned resid norm 9.850224343838e+14 true resid norm > 1.772358652449e+14 ||r(i)||/||b|| 3.366323051568e-03 > 16 KSP preconditioned resid norm 8.835883676210e+14 true resid norm > 1.426145250921e+14 ||r(i)||/||b|| 2.708743868757e-03 > 17 KSP preconditioned resid norm 7.483591829964e+14 true resid norm > 1.680673516819e+14 ||r(i)||/||b|| 3.192181217957e-03 > 18 KSP preconditioned resid norm 6.652269160454e+14 true resid norm > 1.088301440117e+14 ||r(i)||/||b|| 2.067061438079e-03 > 19 KSP preconditioned resid norm 5.777832230782e+14 true resid norm > 9.220881620683e+13 ||r(i)||/||b|| 1.751364844390e-03 > 20 KSP preconditioned resid norm 5.145663259773e+14 true resid norm > 8.255284700142e+13 ||r(i)||/||b|| 1.567964539511e-03 > 21 KSP preconditioned resid norm 4.536494497263e+14 true resid norm > 7.162151119367e+13 ||r(i)||/||b|| 1.360340604800e-03 > 22 KSP preconditioned resid norm 4.039727797390e+14 true resid norm > 6.757078541538e+13 ||r(i)||/||b|| 1.283403289972e-03 > 23 KSP preconditioned resid norm 3.668438626905e+14 true resid norm > 5.700238490686e+13 ||r(i)||/||b|| 1.082672753854e-03 > 24 KSP preconditioned resid norm 3.368796010179e+14 true resid norm > 6.388426371627e+13 ||r(i)||/||b|| 1.213383472264e-03 > 0 KSP preconditioned resid norm 9.591788600089e+15 true resid norm > 5.932754803820e+15 ||r(i)||/||b|| 5.703607413375e-01 > 1 KSP preconditioned resid norm 4.627577110935e+15 true resid norm > 8.119149369849e+14 ||r(i)||/||b|| 7.805554429176e-02 > 2 KSP preconditioned resid norm 3.843546862524e+15 true resid norm > 6.215691855461e+14 ||r(i)||/||b|| 5.975616272433e-02 > 3 KSP preconditioned resid norm 3.802220665831e+15 true resid norm > 6.332285028379e+14 ||r(i)||/||b|| 6.087706137495e-02 > 4 KSP preconditioned resid norm 4.025305304206e+15 true resid norm > 6.486950645643e+14 ||r(i)||/||b|| 6.236397932519e-02 > 5 KSP preconditioned resid norm 4.519185013022e+15 true resid norm > 7.100155628952e+14 ||r(i)||/||b|| 6.825918417416e-02 > 6 KSP preconditioned resid norm 5.566187938033e+15 true resid norm > 8.272878114077e+14 ||r(i)||/||b|| 7.953345537054e-02 > 7 KSP preconditioned resid norm 7.046115999320e+15 true resid norm > 1.260178524183e+15 ||r(i)||/||b|| 1.211505246783e-01 > 8 KSP preconditioned resid norm 7.802424274220e+15 true resid norm > 1.227649475790e+15 ||r(i)||/||b|| 1.180232604022e-01 > 9 KSP preconditioned resid norm 8.076065684234e+15 true resid norm > 1.496448814554e+15 ||r(i)||/||b|| 1.438649806820e-01 > 10 KSP preconditioned resid norm 7.812303545308e+15 true resid norm > 1.236539634992e+15 ||r(i)||/||b|| 1.188779388714e-01 > 11 KSP preconditioned resid norm 6.692698687119e+15 true resid norm > 1.122795851888e+15 ||r(i)||/||b|| 1.079428858312e-01 > 12 KSP preconditioned resid norm 4.016544769049e+15 true resid norm > 6.096078457742e+14 ||r(i)||/||b|| 5.860622836074e-02 > 13 KSP preconditioned resid norm 1.711147574309e+15 true resid norm > 2.966258404639e+14 ||r(i)||/||b|| 2.851689305581e-02 > 14 KSP preconditioned resid norm 1.304839987953e+15 true resid norm > 2.700754536307e+14 ||r(i)||/||b|| 2.596440288593e-02 > 15 KSP preconditioned resid norm 1.171070442303e+15 true resid norm > 2.199042966295e+14 ||r(i)||/||b|| 2.114106882828e-02 > 16 KSP preconditioned resid norm 1.057836577493e+15 true resid norm > 1.660135728859e+14 ||r(i)||/||b|| 1.596014459292e-02 > 17 KSP preconditioned resid norm 8.905346433993e+14 true resid norm > 1.613310037432e+14 ||r(i)||/||b|| 1.550997368650e-02 > 18 KSP preconditioned resid norm 7.864866777768e+14 true resid norm > 1.268576094498e+14 ||r(i)||/||b|| 1.219578468395e-02 > 19 KSP preconditioned resid norm 6.743846682412e+14 true resid norm > 1.066594941052e+14 ||r(i)||/||b|| 1.025398657793e-02 > 20 KSP preconditioned resid norm 5.775523980161e+14 true resid norm > 1.005326356082e+14 ||r(i)||/||b|| 9.664965175573e-03 > 21 KSP preconditioned resid norm 4.897344092103e+14 true resid norm > 7.691922473584e+13 ||r(i)||/||b|| 7.394828792721e-03 > 22 KSP preconditioned resid norm 4.026909639422e+14 true resid norm > 7.643768412999e+13 ||r(i)||/||b|| 7.348534640001e-03 > 23 KSP preconditioned resid norm 3.340361168756e+14 true resid norm > 5.217079213762e+13 ||r(i)||/||b|| 5.015574157998e-03 > 24 KSP preconditioned resid norm 2.772587883161e+14 true resid norm > 6.149190087025e+13 ||r(i)||/||b|| 5.911683075799e-03 > 25 KSP preconditioned resid norm 2.446809593449e+14 true resid norm > 3.772191549633e+13 ||r(i)||/||b|| 3.626493997916e-03 > 26 KSP preconditioned resid norm 2.130893101456e+14 true resid norm > 5.402695231969e+13 ||r(i)||/||b|| 5.194020922190e-03 > 27 KSP preconditioned resid norm 1.886821885223e+14 true resid norm > 2.887742158299e+13 ||r(i)||/||b|| 2.776205679591e-03 > 28 KSP preconditioned resid norm 1.654479962436e+14 true resid norm > 4.199650238007e+13 ||r(i)||/||b|| 4.037442473714e-03 > 29 KSP preconditioned resid norm 1.453563006355e+14 true resid norm > 2.222396036725e+13 ||r(i)||/||b|| 2.136557961634e-03 > 30 KSP preconditioned resid norm 1.311692747504e+14 true resid norm > 2.933890714503e+13 ||r(i)||/||b|| 2.820571788757e-03 > 31 KSP preconditioned resid norm 1.173199137783e+14 true resid norm > 1.793346595553e+13 ||r(i)||/||b|| 1.724080174451e-03 > 32 KSP preconditioned resid norm 1.086053135569e+14 true resid norm > 2.009773461598e+13 ||r(i)||/||b|| 1.932147744820e-03 > 33 KSP preconditioned resid norm 9.929347161574e+13 true resid norm > 1.516318812509e+13 ||r(i)||/||b|| 1.457752343733e-03 > 34 KSP preconditioned resid norm 9.235712158960e+13 true resid norm > 1.538423395802e+13 ||r(i)||/||b|| 1.479003157109e-03 > 35 KSP preconditioned resid norm 8.718922541063e+13 true resid norm > 1.331044977413e+13 ||r(i)||/||b|| 1.279634546134e-03 > 36 KSP preconditioned resid norm 8.163106494702e+13 true resid norm > 1.369060193385e+13 ||r(i)||/||b|| 1.316181458118e-03 > 37 KSP preconditioned resid norm 7.751439049037e+13 true resid norm > 1.182966275510e+13 ||r(i)||/||b|| 1.137275252708e-03 > 38 KSP preconditioned resid norm 7.269591005999e+13 true resid norm > 1.225102361382e+13 ||r(i)||/||b|| 1.177783869649e-03 > 39 KSP preconditioned resid norm 6.809139852739e+13 true resid norm > 1.039056641301e+13 ||r(i)||/||b|| 9.989240004362e-04 > 40 KSP preconditioned resid norm 6.390023598760e+13 true resid norm > 1.042637257280e+13 ||r(i)||/||b|| 1.002366318301e-03 > 41 KSP preconditioned resid norm 5.919860950635e+13 true resid norm > 9.048648521202e+12 ||r(i)||/||b|| 8.699152500509e-04 > 42 KSP preconditioned resid norm 5.538975217584e+13 true resid norm > 8.775858022438e+12 ||r(i)||/||b|| 8.436898292725e-04 > 0 KSP preconditioned resid norm 2.200871087933e+16 true resid norm > 1.375121498750e+16 ||r(i)||/||b|| 1.042688751858e-01 > 1 KSP preconditioned resid norm 1.033852082172e+16 true resid norm > 1.769345381246e+15 ||r(i)||/||b|| 1.341609835098e-02 > 2 KSP preconditioned resid norm 8.201114088620e+15 true resid norm > 1.330146487456e+15 ||r(i)||/||b|| 1.008586355500e-02 > 3 KSP preconditioned resid norm 7.727276364323e+15 true resid norm > 1.292967289192e+15 ||r(i)||/||b|| 9.803951506731e-03 > 4 KSP preconditioned resid norm 7.788308934018e+15 true resid norm > 1.241309862269e+15 ||r(i)||/||b|| 9.412257986909e-03 > 5 KSP preconditioned resid norm 8.253685478294e+15 true resid norm > 1.290166199902e+15 ||r(i)||/||b|| 9.782712188615e-03 > 6 KSP preconditioned resid norm 9.476714658211e+15 true resid norm > 1.410015893632e+15 ||r(i)||/||b|| 1.069147499743e-02 > 7 KSP preconditioned resid norm 1.110541369388e+16 true resid norm > 1.990482855418e+15 ||r(i)||/||b|| 1.509287787295e-02 > 8 KSP preconditioned resid norm 1.149682905149e+16 true resid norm > 1.789313061240e+15 ||r(i)||/||b|| 1.356750370206e-02 > 9 KSP preconditioned resid norm 1.081618830942e+16 true resid norm > 1.972959789140e+15 ||r(i)||/||b|| 1.496000885648e-02 > 10 KSP preconditioned resid norm 8.800226542664e+15 true resid norm > 1.343393349540e+15 ||r(i)||/||b|| 1.018630816375e-02 > 11 KSP preconditioned resid norm 4.142701353678e+15 true resid norm > 6.194874024708e+14 ||r(i)||/||b|| 4.697276182950e-03 > 12 KSP preconditioned resid norm 3.300703256438e+15 true resid norm > 1.133601440349e+15 ||r(i)||/||b|| 8.595556625482e-03 > 13 KSP preconditioned resid norm 7.882205618989e+14 true resid norm > 1.186911081255e+14 ||r(i)||/||b|| 8.999778092374e-04 > 0 KSP preconditioned resid norm 1.928554431739e+16 true resid norm > 1.203381747345e+16 ||r(i)||/||b|| 4.946634802067e-01 > 1 KSP preconditioned resid norm 9.232243648078e+15 true resid norm > 1.583125971956e+15 ||r(i)||/||b|| 6.507615763833e-02 > 2 KSP preconditioned resid norm 7.541746545280e+15 true resid norm > 1.216565269124e+15 ||r(i)||/||b|| 5.000827137780e-02 > 3 KSP preconditioned resid norm 7.340199256082e+15 true resid norm > 1.219811880454e+15 ||r(i)||/||b|| 5.014172695523e-02 > 4 KSP preconditioned resid norm 7.655426776880e+15 true resid norm > 1.212245393966e+15 ||r(i)||/||b|| 4.983069809450e-02 > 5 KSP preconditioned resid norm 8.414958315915e+15 true resid norm > 1.311448634719e+15 ||r(i)||/||b|| 5.390855787813e-02 > 6 KSP preconditioned resid norm 1.008678778264e+16 true resid norm > 1.495852838926e+15 ||r(i)||/||b|| 6.148869823002e-02 > 7 KSP preconditioned resid norm 1.235685231672e+16 true resid norm > 2.203374571492e+15 ||r(i)||/||b|| 9.057216765485e-02 > 8 KSP preconditioned resid norm 1.311846317918e+16 true resid norm > 2.061677565336e+15 ||r(i)||/||b|| 8.474755428051e-02 > 9 KSP preconditioned resid norm 1.289737803434e+16 true resid norm > 2.355893652982e+15 ||r(i)||/||b|| 9.684163449812e-02 > 10 KSP preconditioned resid norm 1.151803850475e+16 true resid norm > 1.821777204240e+15 ||r(i)||/||b|| 7.488618254339e-02 > 11 KSP preconditioned resid norm 8.380110732685e+15 true resid norm > 1.434954901150e+15 ||r(i)||/||b|| 5.898542062058e-02 > 12 KSP preconditioned resid norm 3.553409074636e+15 true resid norm > 5.423523740165e+14 ||r(i)||/||b|| 2.229399884295e-02 > 13 KSP preconditioned resid norm 1.190458477552e+15 true resid norm > 2.162492452839e+14 ||r(i)||/||b|| 8.889166260017e-03 > 14 KSP preconditioned resid norm 9.902014249371e+14 true resid norm > 1.847474743657e+14 ||r(i)||/||b|| 7.594250854373e-03 > 15 KSP preconditioned resid norm 9.211725470007e+14 true resid norm > 1.442747306714e+14 ||r(i)||/||b|| 5.930573613672e-03 > 16 KSP preconditioned resid norm 8.405580663519e+14 true resid norm > 2.201247255509e+14 ||r(i)||/||b|| 9.048472196027e-03 > 17 KSP preconditioned resid norm 7.117085749555e+14 true resid norm > 1.108024781635e+14 ||r(i)||/||b|| 4.554659365975e-03 > 18 KSP preconditioned resid norm 6.347627750827e+14 true resid norm > 1.457120949776e+14 ||r(i)||/||b|| 5.989658075575e-03 > 19 KSP preconditioned resid norm 5.626851770343e+14 true resid norm > 8.750418282909e+13 ||r(i)||/||b|| 3.596956967843e-03 > 20 KSP preconditioned resid norm 4.865367652981e+14 true resid norm > 1.207991751614e+14 ||r(i)||/||b|| 4.965584738448e-03 > 21 KSP preconditioned resid norm 4.252070053539e+14 true resid norm > 6.577223220744e+13 ||r(i)||/||b|| 2.703640914986e-03 > 22 KSP preconditioned resid norm 3.618870286814e+14 true resid norm > 8.993615915309e+13 ||r(i)||/||b|| 3.696926065336e-03 > 23 KSP preconditioned resid norm 3.105522471772e+14 true resid norm > 4.771648977942e+13 ||r(i)||/||b|| 1.961439497451e-03 > 24 KSP preconditioned resid norm 2.735005931489e+14 true resid norm > 4.708110884633e+13 ||r(i)||/||b|| 1.935321455997e-03 > 25 KSP preconditioned resid norm 2.496548247596e+14 true resid norm > 3.887274137414e+13 ||r(i)||/||b|| 1.597907361960e-03 > 26 KSP preconditioned resid norm 2.329765393701e+14 true resid norm > 3.903456593754e+13 ||r(i)||/||b|| 1.604559341009e-03 > 27 KSP preconditioned resid norm 2.169265400605e+14 true resid norm > 3.334703074182e+13 ||r(i)||/||b|| 1.370766867430e-03 > 28 KSP preconditioned resid norm 1.998370208104e+14 true resid norm > 3.474062054769e+13 ||r(i)||/||b|| 1.428051929704e-03 > 29 KSP preconditioned resid norm 1.805920498739e+14 true resid norm > 2.773007978134e+13 ||r(i)||/||b|| 1.139875837515e-03 > 30 KSP preconditioned resid norm 1.643922523062e+14 true resid norm > 2.817409671160e+13 ||r(i)||/||b|| 1.158127648337e-03 > 31 KSP preconditioned resid norm 1.456868079593e+14 true resid norm > 2.236751337771e+13 ||r(i)||/||b|| 9.194415683468e-04 > 32 KSP preconditioned resid norm 1.315816507773e+14 true resid norm > 2.157979387266e+13 ||r(i)||/||b|| 8.870614800950e-04 > 0 KSP preconditioned resid norm 6.130010040591e+16 true resid norm > 4.019535303993e+16 ||r(i)||/||b|| 5.748506002867e-02 > 1 KSP preconditioned resid norm 2.383231267167e+16 true resid norm > 4.333028232365e+15 ||r(i)||/||b|| 6.196845386481e-03 > 2 KSP preconditioned resid norm 1.380314332389e+16 true resid norm > 2.902938687246e+15 ||r(i)||/||b|| 4.151614355275e-03 > 3 KSP preconditioned resid norm 9.355423030603e+15 true resid norm > 1.914374573200e+15 ||r(i)||/||b|| 2.737827359010e-03 > 4 KSP preconditioned resid norm 6.905389736952e+15 true resid norm > 1.160495154173e+15 ||r(i)||/||b|| 1.659672786910e-03 > 5 KSP preconditioned resid norm 5.388850338707e+15 true resid norm > 9.343171842551e+14 ||r(i)||/||b|| 1.336206187053e-03 > 6 KSP preconditioned resid norm 4.456941671648e+15 true resid norm > 7.443205431054e+14 ||r(i)||/||b|| 1.064484022780e-03 > 7 KSP preconditioned resid norm 3.604897929141e+15 true resid norm > 7.177355376747e+14 ||r(i)||/||b|| 1.026463691635e-03 > 8 KSP preconditioned resid norm 2.616552572944e+15 true resid norm > 4.566784838372e+14 ||r(i)||/||b|| 6.531150511630e-04 > 9 KSP preconditioned resid norm 1.730753896972e+15 true resid norm > 3.416577764278e+14 ||r(i)||/||b|| 4.886191139484e-04 > 10 KSP preconditioned resid norm 9.605912905151e+14 true resid norm > 1.715828170206e+14 ||r(i)||/||b|| 2.453877821777e-04 > 11 KSP preconditioned resid norm 3.885469679759e+14 true resid norm > 7.526615556743e+13 ||r(i)||/||b|| 1.076412854646e-04 > 12 KSP preconditioned resid norm 1.085402187481e+14 true resid norm > 1.877491024324e+13 ||r(i)||/||b|| 2.685078649001e-05 > 13 KSP preconditioned resid norm 5.521537327924e+13 true resid norm > 1.482832280216e+13 ||r(i)||/||b|| 2.120660628506e-05 > 14 KSP preconditioned resid norm 4.327780981098e+13 true resid norm > 6.949078996691e+12 ||r(i)||/||b|| 9.938169292156e-06 > 0 KSP preconditioned resid norm 2.469228993807e+16 true resid norm > 1.581397706747e+16 ||r(i)||/||b|| 1.421322431696e-01 > 1 KSP preconditioned resid norm 1.094866250702e+16 true resid norm > 1.827538407052e+15 ||r(i)||/||b|| 1.642547805430e-02 > 2 KSP preconditioned resid norm 7.902646639796e+15 true resid norm > 1.331972302022e+15 ||r(i)||/||b|| 1.197144844200e-02 > 3 KSP preconditioned resid norm 6.757113896005e+15 true resid norm > 1.177404713092e+15 ||r(i)||/||b|| 1.058223192536e-02 > 4 KSP preconditioned resid norm 6.236819639281e+15 true resid norm > 9.858933018833e+14 ||r(i)||/||b|| 8.860973171059e-03 > 5 KSP preconditioned resid norm 6.056207345085e+15 true resid norm > 9.498733496025e+14 ||r(i)||/||b|| 8.537234455953e-03 > 6 KSP preconditioned resid norm 6.281149615757e+15 true resid norm > 9.439016000829e+14 ||r(i)||/||b|| 8.483561799717e-03 > 7 KSP preconditioned resid norm 6.511636840454e+15 true resid norm > 1.175604965975e+15 ||r(i)||/||b|| 1.056605622877e-02 > 8 KSP preconditioned resid norm 5.897282381771e+15 true resid norm > 9.358325816295e+14 ||r(i)||/||b|| 8.411039391972e-03 > 9 KSP preconditioned resid norm 4.858643828149e+15 true resid norm > 9.074074922507e+14 ||r(i)||/||b|| 8.155561487934e-03 > 10 KSP preconditioned resid norm 3.431226959566e+15 true resid norm > 5.523076022464e+14 ||r(i)||/||b|| 4.964008616682e-03 > 11 KSP preconditioned resid norm 1.790601246330e+15 true resid norm > 3.233761759516e+14 ||r(i)||/||b|| 2.906427717678e-03 > 12 KSP preconditioned resid norm 6.061504971272e+14 true resid norm > 9.361691978333e+13 ||r(i)||/||b|| 8.414064817893e-04 > 13 KSP preconditioned resid norm 3.669164519745e+14 true resid norm > 7.433339410578e+13 ||r(i)||/||b|| 6.680907656304e-04 > 14 KSP preconditioned resid norm 1.028448839513e+15 true resid norm > 1.739193394480e+14 ||r(i)||/||b|| 1.563145421348e-03 > 15 KSP preconditioned resid norm 1.340577188948e+14 true resid norm > 2.477633902186e+13 ||r(i)||/||b|| 2.226838086134e-04 > 16 KSP preconditioned resid norm 1.124181660974e+14 true resid norm > 1.917872742131e+13 ||r(i)||/||b|| 1.723738144997e-04 > 17 KSP preconditioned resid norm 9.394070158671e+13 true resid norm > 1.682505899954e+13 ||r(i)||/||b|| 1.512196057236e-04 > 18 KSP preconditioned resid norm 9.034650415597e+13 true resid norm > 1.892962426081e+13 ||r(i)||/||b|| 1.701349348787e-04 > 19 KSP preconditioned resid norm 8.058877605551e+13 true resid norm > 1.275341142002e+13 ||r(i)||/||b|| 1.146246112195e-04 > 20 KSP preconditioned resid norm 6.303186171389e+13 true resid norm > 1.900109222650e+13 ||r(i)||/||b|| 1.707772718592e-04 > 21 KSP preconditioned resid norm 4.601610061053e+13 true resid norm > 6.959914424356e+12 ||r(i)||/||b|| 6.255404603042e-05 > 22 KSP preconditioned resid norm 2.822582332621e+13 true resid norm > 1.013252988889e+13 ||r(i)||/||b|| 9.106875493415e-05 > 23 KSP preconditioned resid norm 1.858033653578e+13 true resid norm > 2.785463510953e+12 ||r(i)||/||b|| 2.503507975190e-05 > 24 KSP preconditioned resid norm 9.505670647827e+12 true resid norm > 4.268589348463e+12 ||r(i)||/||b|| 3.836506001486e-05 > 25 KSP preconditioned resid norm 5.844140386668e+12 true resid norm > 8.921656809030e+11 ||r(i)||/||b|| 8.018571733391e-06 > 0 SNES Function norm 0.000000000000e+00 > 3 TS dt 5e-12 time 9e-12 > 0 KSP preconditioned resid norm 2.220122692212e+19 true resid norm > 1.204173138214e+19 ||r(i)||/||b|| 2.302144004449e+02 > 1 KSP preconditioned resid norm 5.389791318038e+18 true resid norm > 1.775764523088e+18 ||r(i)||/||b|| 3.394915166605e+01 > 2 KSP preconditioned resid norm 2.218229032455e+18 true resid norm > 5.319501602511e+17 ||r(i)||/||b|| 1.016984878025e+01 > 3 KSP preconditioned resid norm 1.401621992174e+18 true resid norm > 2.963396938661e+17 ||r(i)||/||b|| 5.665436537857e+00 > 4 KSP preconditioned resid norm 1.004007060031e+18 true resid norm > 1.656144361093e+17 ||r(i)||/||b|| 3.166224764861e+00 > 5 KSP preconditioned resid norm 8.672273682794e+17 true resid norm > 1.510259905938e+17 ||r(i)||/||b|| 2.887322160975e+00 > 6 KSP preconditioned resid norm 7.388939161376e+17 true resid norm > 1.526077665500e+17 ||r(i)||/||b|| 2.917562629878e+00 > 7 KSP preconditioned resid norm 5.565893055963e+17 true resid norm > 1.017298978392e+17 ||r(i)||/||b|| 1.944877085792e+00 > 8 KSP preconditioned resid norm 3.566491384538e+17 true resid norm > 6.306517676931e+16 ||r(i)||/||b|| 1.205683086440e+00 > 9 KSP preconditioned resid norm 1.893313432840e+17 true resid norm > 3.267997700517e+16 ||r(i)||/||b|| 6.247773741206e-01 > 10 KSP preconditioned resid norm 6.895898463878e+16 true resid norm > 1.153123828824e+16 ||r(i)||/||b|| 2.204547688925e-01 > 11 KSP preconditioned resid norm 5.104999855255e+16 true resid norm > 9.275473058095e+15 ||r(i)||/||b|| 1.773289405940e-01 > 12 KSP preconditioned resid norm 4.992747088606e+16 true resid norm > 8.399049844320e+15 ||r(i)||/||b|| 1.605734393881e-01 > 13 KSP preconditioned resid norm 4.920015136673e+16 true resid norm > 8.426577623649e+15 ||r(i)||/||b|| 1.610997167989e-01 > 14 KSP preconditioned resid norm 5.688370526613e+16 true resid norm > 9.763030126011e+15 ||r(i)||/||b|| 1.866500800972e-01 > 15 KSP preconditioned resid norm 1.613946025228e+17 true resid norm > 4.341380499600e+16 ||r(i)||/||b|| 8.299872145473e-01 > 16 KSP preconditioned resid norm 4.125620543313e+16 true resid norm > 7.099475531667e+15 ||r(i)||/||b|| 1.357281150965e-01 > 17 KSP preconditioned resid norm 2.163115342752e+16 true resid norm > 5.850497008723e+15 ||r(i)||/||b|| 1.118500835491e-01 > 18 KSP preconditioned resid norm 1.414953599369e+16 true resid norm > 2.432122296311e+15 ||r(i)||/||b|| 4.649743118207e-02 > 19 KSP preconditioned resid norm 1.088056550362e+16 true resid norm > 2.488538030816e+15 ||r(i)||/||b|| 4.757598991109e-02 > 20 KSP preconditioned resid norm 9.058853458567e+15 true resid norm > 1.587978910975e+15 ||r(i)||/||b|| 3.035905729067e-02 > 21 KSP preconditioned resid norm 7.813748136791e+15 true resid norm > 1.345999130102e+15 ||r(i)||/||b|| 2.573287618717e-02 > 22 KSP preconditioned resid norm 6.941551759120e+15 true resid norm > 1.184314230023e+15 ||r(i)||/||b|| 2.264177648138e-02 > 23 KSP preconditioned resid norm 6.244923673699e+15 true resid norm > 1.023439246806e+15 ||r(i)||/||b|| 1.956616080515e-02 > 24 KSP preconditioned resid norm 5.681880462174e+15 true resid norm > 9.491910634069e+14 ||r(i)||/||b|| 1.814668045943e-02 > 25 KSP preconditioned resid norm 5.283519017995e+15 true resid norm > 8.557005279630e+14 ||r(i)||/||b|| 1.635932390068e-02 > 26 KSP preconditioned resid norm 5.117675200632e+15 true resid norm > 8.569254569155e+14 ||r(i)||/||b|| 1.638274215138e-02 > 27 KSP preconditioned resid norm 4.944059018035e+15 true resid norm > 9.547778379458e+14 ||r(i)||/||b|| 1.825348868410e-02 > 28 KSP preconditioned resid norm 4.840428844487e+15 true resid norm > 7.569152804275e+14 ||r(i)||/||b|| 1.447074278120e-02 > 29 KSP preconditioned resid norm 4.909458338739e+15 true resid norm > 7.780937746322e+14 ||r(i)||/||b|| 1.487563425328e-02 > 30 KSP preconditioned resid norm 5.172548892535e+15 true resid norm > 8.119476371329e+14 ||r(i)||/||b|| 1.552285402683e-02 > 31 KSP preconditioned resid norm 5.734458283292e+15 true resid norm > 9.082595641858e+14 ||r(i)||/||b|| 1.736415008622e-02 > 32 KSP preconditioned resid norm 8.602942487649e+15 true resid norm > 1.383459272467e+15 ||r(i)||/||b|| 2.644904099284e-02 > 33 KSP preconditioned resid norm 1.681001901654e+16 true resid norm > 2.705594423024e+15 ||r(i)||/||b|| 5.172568446988e-02 > 34 KSP preconditioned resid norm 3.498037913905e+16 true resid norm > 5.724738344713e+15 ||r(i)||/||b|| 1.094458233545e-01 > 35 KSP preconditioned resid norm 6.660621055336e+15 true resid norm > 1.092449655644e+15 ||r(i)||/||b|| 2.088550512457e-02 > 36 KSP preconditioned resid norm 3.564205941873e+15 true resid norm > 5.832825274169e+14 ||r(i)||/||b|| 1.115122344769e-02 > 37 KSP preconditioned resid norm 2.275128811690e+15 true resid norm > 4.150874600928e+14 ||r(i)||/||b|| 7.935662050992e-03 > 38 KSP preconditioned resid norm 1.923021972098e+15 true resid norm > 3.105449109783e+14 ||r(i)||/||b|| 5.937012562674e-03 > 39 KSP preconditioned resid norm 1.756400018558e+15 true resid norm > 3.028037938881e+14 ||r(i)||/||b|| 5.789017513361e-03 > 40 KSP preconditioned resid norm 1.618977002834e+15 true resid norm > 2.591473194248e+14 ||r(i)||/||b|| 4.954390932253e-03 > 41 KSP preconditioned resid norm 1.449378745707e+15 true resid norm > 2.366294470672e+14 ||r(i)||/||b|| 4.523893164149e-03 > 42 KSP preconditioned resid norm 1.311727507569e+15 true resid norm > 2.162961405943e+14 ||r(i)||/||b|| 4.135160031831e-03 > 43 KSP preconditioned resid norm 1.220101213287e+15 true resid norm > 1.960093358179e+14 ||r(i)||/||b|| 3.747315921186e-03 > 44 KSP preconditioned resid norm 1.141924730669e+15 true resid norm > 1.884563341884e+14 ||r(i)||/||b|| 3.602917272310e-03 > 45 KSP preconditioned resid norm 1.062154013672e+15 true resid norm > 2.268601642641e+14 ||r(i)||/||b|| 4.337123545072e-03 > 46 KSP preconditioned resid norm 1.050847341809e+15 true resid norm > 1.644035317108e+14 ||r(i)||/||b|| 3.143074636258e-03 > 47 KSP preconditioned resid norm 8.268920862881e+14 true resid norm > 2.491484388741e+14 ||r(i)||/||b|| 4.763231852378e-03 > 48 KSP preconditioned resid norm 6.970056044801e+14 true resid norm > 1.086553600705e+14 ||r(i)||/||b|| 2.077278406232e-03 > 49 KSP preconditioned resid norm 8.789823309588e+14 true resid norm > 1.565701303869e+14 ||r(i)||/||b|| 2.993315292522e-03 > 50 KSP preconditioned resid norm 9.889574125697e+14 true resid norm > 1.557528361421e+14 ||r(i)||/||b|| 2.977690221792e-03 > 51 KSP preconditioned resid norm 3.685341349551e+14 true resid norm > 5.749337875960e+13 ||r(i)||/||b|| 1.099161183775e-03 > 52 KSP preconditioned resid norm 2.728364862859e+15 true resid norm > 4.745949443528e+14 ||r(i)||/||b|| 9.073329000715e-03 > 53 KSP preconditioned resid norm 8.484691724076e+15 true resid norm > 1.333040899485e+15 ||r(i)||/||b|| 2.548514011022e-02 > 54 KSP preconditioned resid norm 8.341541248747e+13 true resid norm > 1.303089350822e+13 ||r(i)||/||b|| 2.491252496055e-04 > 0 KSP preconditioned resid norm 6.542890621126e+18 true resid norm > 3.545938224251e+18 ||r(i)||/||b|| 4.156801777589e+02 > 1 KSP preconditioned resid norm 1.591173149842e+18 true resid norm > 5.223204285160e+17 ||r(i)||/||b|| 6.123012721646e+01 > 2 KSP preconditioned resid norm 6.570590492388e+17 true resid norm > 1.552517142001e+17 ||r(i)||/||b|| 1.819971360886e+01 > 3 KSP preconditioned resid norm 4.165858181866e+17 true resid norm > 8.738575659546e+16 ||r(i)||/||b|| 1.024398185698e+01 > 4 KSP preconditioned resid norm 2.995041215234e+17 true resid norm > 4.934730887237e+16 ||r(i)||/||b|| 5.784843622966e+00 > 5 KSP preconditioned resid norm 2.594520327186e+17 true resid norm > 4.469839192099e+16 ||r(i)||/||b|| 5.239864409419e+00 > 6 KSP preconditioned resid norm 2.219632225075e+17 true resid norm > 4.652171577038e+16 ||r(i)||/||b|| 5.453607439866e+00 > 7 KSP preconditioned resid norm 1.674928029034e+17 true resid norm > 2.988841295071e+16 ||r(i)||/||b|| 3.503733010156e+00 > 8 KSP preconditioned resid norm 1.081911414020e+17 true resid norm > 1.908165723430e+16 ||r(i)||/||b|| 2.236888002402e+00 > 9 KSP preconditioned resid norm 5.710171826154e+16 true resid norm > 9.635278906551e+15 ||r(i)||/||b|| 1.129516137997e+00 > 10 KSP preconditioned resid norm 1.779377501866e+16 true resid norm > 2.952494296810e+15 ||r(i)||/||b|| 3.461124465555e-01 > 11 KSP preconditioned resid norm 1.125612460139e+16 true resid norm > 2.030403352863e+15 ||r(i)||/||b|| 2.380183672880e-01 > 12 KSP preconditioned resid norm 1.086634567103e+16 true resid norm > 1.877885131752e+15 ||r(i)||/||b|| 2.201390932416e-01 > 13 KSP preconditioned resid norm 1.075395224737e+16 true resid norm > 1.822688133743e+15 ||r(i)||/||b|| 2.136685073225e-01 > 14 KSP preconditioned resid norm 1.173925328196e+16 true resid norm > 2.130469004546e+15 ||r(i)||/||b|| 2.497487769141e-01 > 15 KSP preconditioned resid norm 4.944016549258e+16 true resid norm > 1.361939461929e+16 ||r(i)||/||b|| 1.596562607209e+00 > 16 KSP preconditioned resid norm 1.780247590037e+17 true resid norm > 3.370503336823e+16 ||r(i)||/||b|| 3.951144485840e+00 > 17 KSP preconditioned resid norm 3.251104571145e+16 true resid norm > 8.719659549123e+15 ||r(i)||/||b|| 1.022180704274e+00 > 18 KSP preconditioned resid norm 2.075437272594e+16 true resid norm > 3.944434594682e+15 ||r(i)||/||b|| 4.623947654424e-01 > 19 KSP preconditioned resid norm 1.483444135710e+16 true resid norm > 3.390829411015e+15 ||r(i)||/||b|| 3.974972160206e-01 > 20 KSP preconditioned resid norm 1.134313319987e+16 true resid norm > 2.489467119154e+15 ||r(i)||/||b|| 2.918330972428e-01 > 21 KSP preconditioned resid norm 8.553477032620e+15 true resid norm > 1.657674168535e+15 ||r(i)||/||b|| 1.943243930000e-01 > 22 KSP preconditioned resid norm 6.586595023874e+15 true resid norm > 1.286251502369e+15 ||r(i)||/||b|| 1.507835780925e-01 > 23 KSP preconditioned resid norm 4.780400299659e+15 true resid norm > 9.064518348736e+14 ||r(i)||/||b|| 1.062607513219e-01 > 24 KSP preconditioned resid norm 3.028073960021e+15 true resid norm > 6.179142836679e+14 ||r(i)||/||b|| 7.243632094834e-02 > 25 KSP preconditioned resid norm 1.570529076731e+15 true resid norm > 2.755631911412e+14 ||r(i)||/||b|| 3.230348331903e-02 > 26 KSP preconditioned resid norm 1.180384548694e+15 true resid norm > 2.278461878994e+14 ||r(i)||/||b|| 2.670975575377e-02 > 27 KSP preconditioned resid norm 1.075603583388e+15 true resid norm > 2.036512758236e+14 ||r(i)||/||b|| 2.387345553745e-02 > 28 KSP preconditioned resid norm 1.013485206919e+15 true resid norm > 1.671016982388e+14 ||r(i)||/||b|| 1.958885328364e-02 > 29 KSP preconditioned resid norm 9.469966879519e+14 true resid norm > 1.801244790801e+14 ||r(i)||/||b|| 2.111547656715e-02 > 30 KSP preconditioned resid norm 8.991813841494e+14 true resid norm > 1.459100485336e+14 ||r(i)||/||b|| 1.710461690970e-02 > 31 KSP preconditioned resid norm 8.677534755734e+14 true resid norm > 1.581142867341e+14 ||r(i)||/||b|| 1.853528478482e-02 > 32 KSP preconditioned resid norm 8.336211910215e+14 true resid norm > 1.336646371267e+14 ||r(i)||/||b|| 1.566912241756e-02 > 33 KSP preconditioned resid norm 8.025169227655e+14 true resid norm > 1.586001394574e+14 ||r(i)||/||b|| 1.859223990744e-02 > 34 KSP preconditioned resid norm 7.724033526395e+14 true resid norm > 1.340849232530e+14 ||r(i)||/||b|| 1.571839135589e-02 > 35 KSP preconditioned resid norm 7.664262315912e+14 true resid norm > 1.204381005271e+14 ||r(i)||/||b|| 1.411861343033e-02 > 36 KSP preconditioned resid norm 7.727266103384e+14 true resid norm > 1.292430217432e+14 ||r(i)||/||b|| 1.515078911553e-02 > 37 KSP preconditioned resid norm 8.047214262262e+14 true resid norm > 1.267167533285e+14 ||r(i)||/||b|| 1.485464190786e-02 > 38 KSP preconditioned resid norm 8.419506152984e+14 true resid norm > 1.380565821207e+14 ||r(i)||/||b|| 1.618397754485e-02 > 39 KSP preconditioned resid norm 8.796981754885e+14 true resid norm > 1.388509364066e+14 ||r(i)||/||b|| 1.627709742171e-02 > 40 KSP preconditioned resid norm 9.432799067464e+14 true resid norm > 1.525727137977e+14 ||r(i)||/||b|| 1.788566206789e-02 > 41 KSP preconditioned resid norm 1.193594717717e+15 true resid norm > 1.887759239390e+14 ||r(i)||/||b|| 2.212966065874e-02 > 42 KSP preconditioned resid norm 2.117323854280e+15 true resid norm > 3.659661973675e+14 ||r(i)||/||b|| 4.290116870483e-02 > 43 KSP preconditioned resid norm 1.749515464589e+16 true resid norm > 2.774904343458e+15 ||r(i)||/||b|| 3.252940851772e-01 > 44 KSP preconditioned resid norm 1.219803820486e+15 true resid norm > 2.038867414461e+14 ||r(i)||/||b|| 2.390105849768e-02 > 45 KSP preconditioned resid norm 6.218668840796e+14 true resid norm > 9.918031040094e+13 ||r(i)||/||b|| 1.162662360435e-02 > 46 KSP preconditioned resid norm 4.213016096566e+14 true resid norm > 7.032265378826e+13 ||r(i)||/||b|| 8.243723206248e-03 > 47 KSP preconditioned resid norm 2.932264460599e+14 true resid norm > 4.681093096736e+13 ||r(i)||/||b|| 5.487511308712e-03 > 48 KSP preconditioned resid norm 1.890454719223e+14 true resid norm > 3.330632990756e+13 ||r(i)||/||b|| 3.904405621560e-03 > 49 KSP preconditioned resid norm 1.361587907879e+14 true resid norm > 2.180429132446e+13 ||r(i)||/||b|| 2.556054595557e-03 > 50 KSP preconditioned resid norm 1.041049830937e+14 true resid norm > 1.811134568183e+13 ||r(i)||/||b|| 2.123141159366e-03 > 51 KSP preconditioned resid norm 9.375298557199e+13 true resid norm > 1.575768352161e+13 ||r(i)||/||b|| 1.847228088333e-03 > 52 KSP preconditioned resid norm 1.220175150775e+14 true resid norm > 1.950548748269e+13 ||r(i)||/||b|| 2.286572408009e-03 > 53 KSP preconditioned resid norm 7.051895161295e+13 true resid norm > 1.295324819768e+13 ||r(i)||/||b|| 1.518472170932e-03 > 54 KSP preconditioned resid norm 5.292501274648e+13 true resid norm > 8.611444708372e+12 ||r(i)||/||b|| 1.009494988563e-03 > 0 KSP preconditioned resid norm 1.892610202067e+19 true resid norm > 1.028308040523e+19 ||r(i)||/||b|| 7.841022917690e+01 > 1 KSP preconditioned resid norm 4.581169591238e+18 true resid norm > 1.519782710159e+18 ||r(i)||/||b|| 1.158860048804e+01 > 2 KSP preconditioned resid norm 1.874457569158e+18 true resid norm > 4.646597685037e+17 ||r(i)||/||b|| 3.543109409036e+00 > 3 KSP preconditioned resid norm 1.173257076173e+18 true resid norm > 2.550995626245e+17 ||r(i)||/||b|| 1.945177357373e+00 > 4 KSP preconditioned resid norm 8.301427099842e+17 true resid norm > 1.381385637907e+17 ||r(i)||/||b|| 1.053329937932e+00 > 5 KSP preconditioned resid norm 7.053087339556e+17 true resid norm > 1.287851939525e+17 ||r(i)||/||b|| 9.820089092432e-01 > 6 KSP preconditioned resid norm 5.863912202870e+17 true resid norm > 1.166954217660e+17 ||r(i)||/||b|| 8.898223493329e-01 > 7 KSP preconditioned resid norm 4.330693111087e+17 true resid norm > 8.611440299479e+16 ||r(i)||/||b|| 6.566369033559e-01 > 8 KSP preconditioned resid norm 2.702458300388e+17 true resid norm > 4.803465439349e+16 ||r(i)||/||b|| 3.662723727716e-01 > 9 KSP preconditioned resid norm 1.546904751775e+17 true resid norm > 2.878502712633e+16 ||r(i)||/||b|| 2.194907055953e-01 > 10 KSP preconditioned resid norm 9.910545254967e+16 true resid norm > 1.648892832929e+16 ||r(i)||/||b|| 1.257308703453e-01 > 11 KSP preconditioned resid norm 9.242232753221e+16 true resid norm > 1.559261342084e+16 ||r(i)||/||b|| 1.188963174082e-01 > 12 KSP preconditioned resid norm 9.244969774564e+16 true resid norm > 1.506990436290e+16 ||r(i)||/||b|| 1.149105723385e-01 > 13 KSP preconditioned resid norm 7.091380549056e+16 true resid norm > 1.150476534699e+16 ||r(i)||/||b|| 8.772578370826e-02 > 14 KSP preconditioned resid norm 2.015813147523e+17 true resid norm > 3.265267485752e+16 ||r(i)||/||b|| 2.489821743992e-01 > 15 KSP preconditioned resid norm 1.668693825394e+16 true resid norm > 4.392881515438e+15 ||r(i)||/||b|| 3.349646533904e-02 > 16 KSP preconditioned resid norm 1.171106201192e+16 true resid norm > 1.864120995445e+15 ||r(i)||/||b|| 1.421423821522e-02 > 17 KSP preconditioned resid norm 1.020170882678e+16 true resid norm > 1.999245817269e+15 ||r(i)||/||b|| 1.524458786039e-02 > 18 KSP preconditioned resid norm 9.407404703690e+15 true resid norm > 1.539877936168e+15 ||r(i)||/||b|| 1.174182998880e-02 > 19 KSP preconditioned resid norm 8.812299187434e+15 true resid norm > 1.424937011002e+15 ||r(i)||/||b|| 1.086538597310e-02 > 20 KSP preconditioned resid norm 8.313368595823e+15 true resid norm > 1.488773269648e+15 ||r(i)||/||b|| 1.135214825376e-02 > 21 KSP preconditioned resid norm 7.851626197782e+15 true resid norm > 1.250085440746e+15 ||r(i)||/||b|| 9.532113144783e-03 > 22 KSP preconditioned resid norm 7.455044400782e+15 true resid norm > 1.324254107888e+15 ||r(i)||/||b|| 1.009766178966e-02 > 23 KSP preconditioned resid norm 7.129687083603e+15 true resid norm > 1.124533100660e+15 ||r(i)||/||b|| 8.574755293646e-03 > 24 KSP preconditioned resid norm 5.784281419699e+15 true resid norm > 2.057123935555e+15 ||r(i)||/||b|| 1.568591831199e-02 > 25 KSP preconditioned resid norm 4.924869719264e+15 true resid norm > 7.825081154476e+14 ||r(i)||/||b|| 5.966756871199e-03 > 26 KSP preconditioned resid norm 5.016674985414e+15 true resid norm > 7.817733101871e+14 ||r(i)||/||b|| 5.961153856674e-03 > 27 KSP preconditioned resid norm 5.308634596254e+15 true resid norm > 8.859615986698e+14 ||r(i)||/||b|| 6.755607197068e-03 > 28 KSP preconditioned resid norm 9.149615612203e+15 true resid norm > 1.429126026168e+15 ||r(i)||/||b|| 1.089732792301e-02 > 29 KSP preconditioned resid norm 8.897313413659e+15 true resid norm > 2.002400213933e+15 ||r(i)||/||b|| 1.526864066904e-02 > 30 KSP preconditioned resid norm 2.145204463192e+15 true resid norm > 3.349465461999e+14 ||r(i)||/||b|| 2.554024126484e-03 > 31 KSP preconditioned resid norm 1.279996253934e+15 true resid norm > 2.794251271078e+14 ||r(i)||/||b|| 2.130663905259e-03 > 32 KSP preconditioned resid norm 8.936440443654e+14 true resid norm > 1.396095375555e+14 ||r(i)||/||b|| 1.064546361948e-03 > 0 KSP preconditioned resid norm 6.470811776559e+18 true resid norm > 3.511555308471e+18 ||r(i)||/||b|| 1.662220839803e+02 > 1 KSP preconditioned resid norm 1.571601370240e+18 true resid norm > 5.187483735196e+17 ||r(i)||/||b|| 2.455534033589e+01 > 2 KSP preconditioned resid norm 6.468356854753e+17 true resid norm > 1.566993981909e+17 ||r(i)||/||b|| 7.417482635946e+00 > 3 KSP preconditioned resid norm 4.084058718904e+17 true resid norm > 8.688424998267e+16 ||r(i)||/||b|| 4.112730636007e+00 > 4 KSP preconditioned resid norm 2.920904676538e+17 true resid norm > 4.831974981420e+16 ||r(i)||/||b|| 2.287251319137e+00 > 5 KSP preconditioned resid norm 2.515876340477e+17 true resid norm > 4.435582123557e+16 ||r(i)||/||b|| 2.099615809737e+00 > 6 KSP preconditioned resid norm 2.133089016156e+17 true resid norm > 4.355659888860e+16 ||r(i)||/||b|| 2.061784025127e+00 > 7 KSP preconditioned resid norm 1.601295948028e+17 true resid norm > 2.999342671961e+16 ||r(i)||/||b|| 1.419761176199e+00 > 8 KSP preconditioned resid norm 1.019894482623e+17 true resid norm > 1.811123232414e+16 ||r(i)||/||b|| 8.573086612382e-01 > 9 KSP preconditioned resid norm 5.552358385594e+16 true resid norm > 9.849609129144e+15 ||r(i)||/||b|| 4.662385786400e-01 > 10 KSP preconditioned resid norm 2.614302078211e+16 true resid norm > 4.400897857400e+15 ||r(i)||/||b|| 2.083197754216e-01 > 11 KSP preconditioned resid norm 2.230412831734e+16 true resid norm > 4.001989581448e+15 ||r(i)||/||b|| 1.894371552944e-01 > 12 KSP preconditioned resid norm 2.202852939300e+16 true resid norm > 3.668964514289e+15 ||r(i)||/||b|| 1.736731658885e-01 > 13 KSP preconditioned resid norm 2.171413942501e+16 true resid norm > 3.746648405904e+15 ||r(i)||/||b|| 1.773503907139e-01 > 14 KSP preconditioned resid norm 2.756569557917e+16 true resid norm > 4.738274109991e+15 ||r(i)||/||b|| 2.242897314282e-01 > 15 KSP preconditioned resid norm 1.911125733606e+16 true resid norm > 7.273426207009e+15 ||r(i)||/||b|| 3.442930427122e-01 > 16 KSP preconditioned resid norm 8.267301781208e+15 true resid norm > 1.450502353903e+15 ||r(i)||/||b|| 6.866060844958e-02 > 17 KSP preconditioned resid norm 5.452175853074e+15 true resid norm > 1.779140334910e+15 ||r(i)||/||b|| 8.421693186737e-02 > 18 KSP preconditioned resid norm 4.185422899260e+15 true resid norm > 7.352001585133e+14 ||r(i)||/||b|| 3.480124667150e-02 > 19 KSP preconditioned resid norm 3.483100692842e+15 true resid norm > 7.680069742951e+14 ||r(i)||/||b|| 3.635418171280e-02 > 20 KSP preconditioned resid norm 3.028709630445e+15 true resid norm > 5.664408475792e+14 ||r(i)||/||b|| 2.681289909034e-02 > 21 KSP preconditioned resid norm 2.650986791756e+15 true resid norm > 5.333477625647e+14 ||r(i)||/||b|| 2.524641328184e-02 > 22 KSP preconditioned resid norm 2.409785359425e+15 true resid norm > 3.913611824173e+14 ||r(i)||/||b|| 1.852537283791e-02 > 23 KSP preconditioned resid norm 2.246473514462e+15 true resid norm > 3.515886291592e+14 ||r(i)||/||b|| 1.664270942896e-02 > 24 KSP preconditioned resid norm 2.105216345845e+15 true resid norm > 3.429537728013e+14 ||r(i)||/||b|| 1.623397207682e-02 > 25 KSP preconditioned resid norm 1.993254024352e+15 true resid norm > 3.120073169264e+14 ||r(i)||/||b|| 1.476909855627e-02 > 26 KSP preconditioned resid norm 1.937764630771e+15 true resid norm > 3.193060027359e+14 ||r(i)||/||b|| 1.511458728106e-02 > 27 KSP preconditioned resid norm 1.841409046328e+15 true resid norm > 4.276359096161e+14 ||r(i)||/||b|| 2.024246404712e-02 > 28 KSP preconditioned resid norm 1.798600034298e+15 true resid norm > 2.807262398909e+14 ||r(i)||/||b|| 1.328838549404e-02 > 29 KSP preconditioned resid norm 1.909156742948e+15 true resid norm > 3.075705616539e+14 ||r(i)||/||b|| 1.455908144342e-02 > 30 KSP preconditioned resid norm 2.243056909373e+15 true resid norm > 3.518787704137e+14 ||r(i)||/||b|| 1.665644348118e-02 > 31 KSP preconditioned resid norm 3.125791809917e+15 true resid norm > 5.051797337710e+14 ||r(i)||/||b|| 2.391305867501e-02 > 32 KSP preconditioned resid norm 2.268406906040e+16 true resid norm > 3.663605847107e+15 ||r(i)||/||b|| 1.734195093893e-01 > 33 KSP preconditioned resid norm 2.663631019961e+15 true resid norm > 4.418031676456e+14 ||r(i)||/||b|| 2.091308174984e-02 > 34 KSP preconditioned resid norm 1.194136462714e+15 true resid norm > 1.964734637439e+14 ||r(i)||/||b|| 9.300217630505e-03 > 35 KSP preconditioned resid norm 7.378290473645e+14 true resid norm > 1.245828993721e+14 ||r(i)||/||b|| 5.897224261846e-03 > 36 KSP preconditioned resid norm 5.578220476287e+14 true resid norm > 9.049566410806e+13 ||r(i)||/||b|| 4.283679611405e-03 > 37 KSP preconditioned resid norm 4.459209275003e+14 true resid norm > 9.101333159944e+13 ||r(i)||/||b|| 4.308183787380e-03 > 38 KSP preconditioned resid norm 4.020379023591e+14 true resid norm > 6.381912771158e+13 ||r(i)||/||b|| 3.020925907227e-03 > 39 KSP preconditioned resid norm 3.758207934261e+14 true resid norm > 7.185355109637e+13 ||r(i)||/||b|| 3.401241317717e-03 > 40 KSP preconditioned resid norm 3.524000257362e+14 true resid norm > 5.552480372659e+13 ||r(i)||/||b|| 2.628307908397e-03 > 41 KSP preconditioned resid norm 3.288683013967e+14 true resid norm > 5.672396056313e+13 ||r(i)||/||b|| 2.685070889720e-03 > 42 KSP preconditioned resid norm 3.091159619430e+14 true resid norm > 5.054650403677e+13 ||r(i)||/||b|| 2.392656387510e-03 > 43 KSP preconditioned resid norm 2.914764729338e+14 true resid norm > 5.652869757159e+13 ||r(i)||/||b|| 2.675827970692e-03 > 44 KSP preconditioned resid norm 2.823730992718e+14 true resid norm > 4.387571428023e+13 ||r(i)||/||b|| 2.076889589689e-03 > 45 KSP preconditioned resid norm 2.874969520743e+14 true resid norm > 4.617969867525e+13 ||r(i)||/||b|| 2.185950405754e-03 > 46 KSP preconditioned resid norm 3.038236713170e+14 true resid norm > 4.761130615995e+13 ||r(i)||/||b|| 2.253716611508e-03 > 47 KSP preconditioned resid norm 3.972150696557e+14 true resid norm > 6.226543710285e+13 ||r(i)||/||b|| 2.947380805938e-03 > 48 KSP preconditioned resid norm 7.035007479461e+14 true resid norm > 1.474495546291e+14 ||r(i)||/||b|| 6.979634406808e-03 > 49 KSP preconditioned resid norm 2.896572855920e+13 true resid norm > 4.634266191547e+12 ||r(i)||/||b|| 2.193664392014e-04 > 0 KSP preconditioned resid norm 6.694681597421e+18 true resid norm > 3.650754072308e+18 ||r(i)||/||b|| 5.229798618159e+00 > 1 KSP preconditioned resid norm 1.597667334547e+18 true resid norm > 5.366749713724e+17 ||r(i)||/||b|| 7.688006280602e-01 > 2 KSP preconditioned resid norm 6.300446759218e+17 true resid norm > 1.705328267289e+17 ||r(i)||/||b|| 2.442926376067e-01 > 3 KSP preconditioned resid norm 3.642174486076e+17 true resid norm > 9.481576843120e+16 ||r(i)||/||b|| 1.358260142699e-01 > 4 KSP preconditioned resid norm 2.247332959633e+17 true resid norm > 3.802800729913e+16 ||r(i)||/||b|| 5.447609345502e-02 > 5 KSP preconditioned resid norm 1.557523701694e+17 true resid norm > 3.829141999854e+16 ||r(i)||/||b|| 5.485343888670e-02 > 6 KSP preconditioned resid norm 1.004692846458e+17 true resid norm > 1.699728099753e+16 ||r(i)||/||b|| 2.434903992784e-02 > 7 KSP preconditioned resid norm 7.103482800944e+16 true resid norm > 1.472858884839e+16 ||r(i)||/||b|| 2.109908037658e-02 > 8 KSP preconditioned resid norm 5.766847884006e+16 true resid norm > 9.397954060610e+15 ||r(i)||/||b|| 1.346280965144e-02 > 9 KSP preconditioned resid norm 4.523472015836e+16 true resid norm > 1.246374917128e+16 ||r(i)||/||b|| 1.785463959006e-02 > 10 KSP preconditioned resid norm 3.195342481549e+16 true resid norm > 5.030521483345e+15 ||r(i)||/||b|| 7.206350737723e-03 > 11 KSP preconditioned resid norm 2.437460220100e+16 true resid norm > 4.974645944970e+15 ||r(i)||/||b|| 7.126307599347e-03 > 12 KSP preconditioned resid norm 1.258045555912e+16 true resid norm > 1.885910304740e+15 ||r(i)||/||b|| 2.701614765157e-03 > 13 KSP preconditioned resid norm 2.495567167662e+15 true resid norm > 4.460991814847e+14 ||r(i)||/||b|| 6.390484915397e-04 > 14 KSP preconditioned resid norm 7.678096336284e+14 true resid norm > 1.316276217008e+14 ||r(i)||/||b|| 1.885599359606e-04 > 15 KSP preconditioned resid norm 5.950867488217e+14 true resid norm > 1.439410415068e+14 ||r(i)||/||b|| 2.061992248885e-04 > 16 KSP preconditioned resid norm 3.263035982098e+14 true resid norm > 5.046543709851e+13 ||r(i)||/||b|| 7.229302987142e-05 > 17 KSP preconditioned resid norm 1.436782483869e+14 true resid norm > 7.588970331259e+13 ||r(i)||/||b|| 1.087139417380e-04 > 18 KSP preconditioned resid norm 5.401533723431e+13 true resid norm > 8.625318760812e+12 ||r(i)||/||b|| 1.235598981554e-05 > 19 KSP preconditioned resid norm 3.528859982338e+13 true resid norm > 1.233587619696e+13 ||r(i)||/||b|| 1.767145828255e-05 > 0 KSP preconditioned resid norm 4.827874033276e+18 true resid norm > 2.629049503582e+18 ||r(i)||/||b|| 2.387244449971e+01 > 1 KSP preconditioned resid norm 1.161059891938e+18 true resid norm > 3.885117259710e+17 ||r(i)||/||b|| 3.527786222014e+00 > 2 KSP preconditioned resid norm 4.696931551640e+17 true resid norm > 1.228954734511e+17 ||r(i)||/||b|| 1.115922452289e+00 > 3 KSP preconditioned resid norm 2.870101730651e+17 true resid norm > 6.759381584873e+16 ||r(i)||/||b|| 6.137692025860e-01 > 4 KSP preconditioned resid norm 1.966639305477e+17 true resid norm > 3.310590025777e+16 ||r(i)||/||b|| 3.006100742644e-01 > 5 KSP preconditioned resid norm 1.592574496560e+17 true resid norm > 3.239415772489e+16 ||r(i)||/||b|| 2.941472693263e-01 > 6 KSP preconditioned resid norm 1.230114823548e+17 true resid norm > 2.280737281236e+16 ||r(i)||/||b|| 2.070968009182e-01 > 7 KSP preconditioned resid norm 8.598640814452e+16 true resid norm > 1.938029193987e+16 ||r(i)||/||b|| 1.759780266946e-01 > 8 KSP preconditioned resid norm 5.549085767183e+16 true resid norm > 9.386094645880e+15 ||r(i)||/||b|| 8.522814925985e-02 > 9 KSP preconditioned resid norm 4.474754629313e+16 true resid norm > 7.579167805290e+15 ||r(i)||/||b|| 6.882078961970e-02 > 10 KSP preconditioned resid norm 4.378389532275e+16 true resid norm > 6.842593279419e+15 ||r(i)||/||b|| 6.213250380964e-02 > 11 KSP preconditioned resid norm 4.079498017991e+16 true resid norm > 7.704710537522e+15 ||r(i)||/||b|| 6.996074986140e-02 > 12 KSP preconditioned resid norm 4.084398127844e+16 true resid norm > 6.377957981626e+15 ||r(i)||/||b|| 5.791349601078e-02 > 13 KSP preconditioned resid norm 7.523960004025e+16 true resid norm > 1.132262464871e+16 ||r(i)||/||b|| 1.028123388886e-01 > 14 KSP preconditioned resid norm 2.184546709977e+15 true resid norm > 3.502101186681e+14 ||r(i)||/||b|| 3.179997793785e-03 > 15 KSP preconditioned resid norm 2.222188254453e+15 true resid norm > 3.654675413503e+14 ||r(i)||/||b|| 3.318539109075e-03 > 16 KSP preconditioned resid norm 1.669487552266e+15 true resid norm > 3.629534589137e+14 ||r(i)||/||b|| 3.295710595061e-03 > 17 KSP preconditioned resid norm 1.401917499993e+15 true resid norm > 2.161165604483e+14 ||r(i)||/||b|| 1.962393856693e-03 > 18 KSP preconditioned resid norm 1.134393254100e+15 true resid norm > 2.686682335159e+14 ||r(i)||/||b|| 2.439576540763e-03 > 19 KSP preconditioned resid norm 9.058009036885e+14 true resid norm > 1.409928994873e+14 ||r(i)||/||b|| 1.280251727203e-03 > 20 KSP preconditioned resid norm 7.990098022926e+14 true resid norm > 1.659150954933e+14 ||r(i)||/||b|| 1.506551665700e-03 > 21 KSP preconditioned resid norm 7.119179981764e+14 true resid norm > 1.092130722300e+14 ||r(i)||/||b|| 9.916827362515e-04 > 22 KSP preconditioned resid norm 6.306959704581e+14 true resid norm > 1.460783700778e+14 ||r(i)||/||b|| 1.326429105857e-03 > 23 KSP preconditioned resid norm 5.637228678118e+14 true resid norm > 8.672640313846e+13 ||r(i)||/||b|| 7.874980074592e-04 > 24 KSP preconditioned resid norm 4.960976390096e+14 true resid norm > 1.292271182132e+14 ||r(i)||/||b|| 1.173415412376e-03 > 25 KSP preconditioned resid norm 4.386404742743e+14 true resid norm > 8.037964656859e+13 ||r(i)||/||b|| 7.298678282782e-04 > 26 KSP preconditioned resid norm 4.419824802921e+14 true resid norm > 6.704823034514e+13 ||r(i)||/||b|| 6.088151461346e-04 > 27 KSP preconditioned resid norm 4.437076279615e+14 true resid norm > 1.370394110315e+14 ||r(i)||/||b|| 1.244353036968e-03 > 28 KSP preconditioned resid norm 7.848825686766e+14 true resid norm > 1.196984074237e+14 ||r(i)||/||b|| 1.086892271915e-03 > 29 KSP preconditioned resid norm 1.771839364121e+15 true resid norm > 6.554952272410e+14 ||r(i)||/||b|| 5.952064961431e-03 > 30 KSP preconditioned resid norm 1.052090569223e+15 true resid norm > 1.877636113350e+14 ||r(i)||/||b|| 1.704941799139e-03 > 31 KSP preconditioned resid norm 2.459236824172e+14 true resid norm > 3.913333475140e+13 ||r(i)||/||b|| 3.553407270074e-04 > 32 KSP preconditioned resid norm 1.091606883884e+14 true resid norm > 1.900051073024e+13 ||r(i)||/||b|| 1.725295158025e-04 > 33 KSP preconditioned resid norm 6.424114279947e+13 true resid norm > 1.001416869814e+13 ||r(i)||/||b|| 9.093122291205e-05 > 34 KSP preconditioned resid norm 3.801391191576e+13 true resid norm > 7.035993054761e+12 ||r(i)||/||b|| 6.388862342504e-05 > 35 KSP preconditioned resid norm 2.524482106623e+13 true resid norm > 4.006294246762e+12 ||r(i)||/||b|| 3.637818037470e-05 > 36 KSP preconditioned resid norm 1.579134058219e+13 true resid norm > 2.528643754315e+12 ||r(i)||/||b|| 2.296073451724e-05 > 37 KSP preconditioned resid norm 1.074218958170e+13 true resid norm > 1.786180378482e+12 ||r(i)||/||b|| 1.621897643756e-05 > 38 KSP preconditioned resid norm 6.572401461924e+12 true resid norm > 1.033900044668e+12 ||r(i)||/||b|| 9.388077858923e-06 > 0 KSP preconditioned resid norm 9.558506771869e+15 true resid norm > 5.892974320929e+15 ||r(i)||/||b|| 1.119279921775e-01 > 1 KSP preconditioned resid norm 4.577878026258e+15 true resid norm > 8.113551231445e+14 ||r(i)||/||b|| 1.541044384903e-02 > 2 KSP preconditioned resid norm 3.768474917795e+15 true resid norm > 6.114238708763e+14 ||r(i)||/||b|| 1.161305692331e-02 > 3 KSP preconditioned resid norm 3.689006206393e+15 true resid norm > 6.168086911615e+14 ||r(i)||/||b|| 1.171533331040e-02 > 4 KSP preconditioned resid norm 3.859762637542e+15 true resid norm > 6.287723791867e+14 ||r(i)||/||b|| 1.194256518123e-02 > 5 KSP preconditioned resid norm 4.285185348381e+15 true resid norm > 6.764146832146e+14 ||r(i)||/||b|| 1.284745753985e-02 > 6 KSP preconditioned resid norm 5.208557345650e+15 true resid norm > 7.765898979336e+14 ||r(i)||/||b|| 1.475013181583e-02 > 7 KSP preconditioned resid norm 6.471756089495e+15 true resid norm > 1.171552566673e+15 ||r(i)||/||b|| 2.225184081532e-02 > 8 KSP preconditioned resid norm 7.012039525053e+15 true resid norm > 1.102268075574e+15 ||r(i)||/||b|| 2.093588836833e-02 > 9 KSP preconditioned resid norm 7.014970322733e+15 true resid norm > 1.328280835862e+15 ||r(i)||/||b|| 2.522865346245e-02 > 10 KSP preconditioned resid norm 6.390016130531e+15 true resid norm > 1.015385597333e+15 ||r(i)||/||b|| 1.928568919633e-02 > 11 KSP preconditioned resid norm 4.845316666400e+15 true resid norm > 8.456481888760e+14 ||r(i)||/||b|| 1.606178793844e-02 > 12 KSP preconditioned resid norm 2.410560854094e+15 true resid norm > 3.709640804249e+14 ||r(i)||/||b|| 7.045892690295e-03 > 13 KSP preconditioned resid norm 1.334157374345e+15 true resid norm > 2.713196530598e+14 ||r(i)||/||b|| 5.153299904502e-03 > 14 KSP preconditioned resid norm 1.217530036393e+15 true resid norm > 2.232136761716e+14 ||r(i)||/||b|| 4.239600792372e-03 > 15 KSP preconditioned resid norm 1.145671346103e+15 true resid norm > 2.123551517116e+14 ||r(i)||/||b|| 4.033359805286e-03 > 16 KSP preconditioned resid norm 1.048828510236e+15 true resid norm > 1.668978069044e+14 ||r(i)||/||b|| 3.169967389692e-03 > 17 KSP preconditioned resid norm 9.144204694073e+14 true resid norm > 1.578839852075e+14 ||r(i)||/||b|| 2.998763697051e-03 > 18 KSP preconditioned resid norm 8.235850213860e+14 true resid norm > 1.321988157805e+14 ||r(i)||/||b|| 2.510913371198e-03 > 19 KSP preconditioned resid norm 6.978607225935e+14 true resid norm > 1.125789224246e+14 ||r(i)||/||b|| 2.138263644512e-03 > 20 KSP preconditioned resid norm 6.009340490988e+14 true resid norm > 1.003011158758e+14 ||r(i)||/||b|| 1.905065574997e-03 > 21 KSP preconditioned resid norm 5.123703002158e+14 true resid norm > 8.114653676231e+13 ||r(i)||/||b|| 1.541253777351e-03 > 22 KSP preconditioned resid norm 4.394549158605e+14 true resid norm > 8.003106905102e+13 ||r(i)||/||b|| 1.520067182184e-03 > 23 KSP preconditioned resid norm 3.852402878098e+14 true resid norm > 5.993269339949e+13 ||r(i)||/||b|| 1.138329419521e-03 > 24 KSP preconditioned resid norm 3.390788847804e+14 true resid norm > 7.376626581336e+13 ||r(i)||/||b|| 1.401076871080e-03 > 0 KSP preconditioned resid norm 9.592017367891e+15 true resid norm > 5.933298102242e+15 ||r(i)||/||b|| 5.704123038824e-01 > 1 KSP preconditioned resid norm 4.627662883687e+15 true resid norm > 8.121175703537e+14 ||r(i)||/||b|| 7.807493342594e-02 > 2 KSP preconditioned resid norm 3.843678575472e+15 true resid norm > 6.216594738645e+14 ||r(i)||/||b|| 5.976477274644e-02 > 3 KSP preconditioned resid norm 3.802359495548e+15 true resid norm > 6.332926539714e+14 ||r(i)||/||b|| 6.088315731972e-02 > 4 KSP preconditioned resid norm 4.025235313693e+15 true resid norm > 6.488898363947e+14 ||r(i)||/||b|| 6.238263107055e-02 > 5 KSP preconditioned resid norm 4.519136017232e+15 true resid norm > 7.100489487455e+14 ||r(i)||/||b|| 6.826231376612e-02 > 6 KSP preconditioned resid norm 5.566459161126e+15 true resid norm > 8.272746471009e+14 ||r(i)||/||b|| 7.953209652789e-02 > 7 KSP preconditioned resid norm 7.051037635374e+15 true resid norm > 1.259719337020e+15 ||r(i)||/||b|| 1.211062375246e-01 > 8 KSP preconditioned resid norm 7.825859719364e+15 true resid norm > 1.230378249193e+15 ||r(i)||/||b|| 1.182854593980e-01 > 9 KSP preconditioned resid norm 8.112136766042e+15 true resid norm > 1.509007778424e+15 ||r(i)||/||b|| 1.450721990763e-01 > 10 KSP preconditioned resid norm 7.849971536540e+15 true resid norm > 1.244897342905e+15 ||r(i)||/||b|| 1.196812884212e-01 > 11 KSP preconditioned resid norm 6.691547560102e+15 true resid norm > 1.152431148742e+15 ||r(i)||/||b|| 1.107918218994e-01 > 12 KSP preconditioned resid norm 3.838210718255e+15 true resid norm > 5.898239658450e+14 ||r(i)||/||b|| 5.670418735838e-02 > 13 KSP preconditioned resid norm 1.585805121666e+15 true resid norm > 3.093588848689e+14 ||r(i)||/||b|| 2.974098236828e-02 > 14 KSP preconditioned resid norm 1.284246821472e+15 true resid norm > 2.667115224243e+14 ||r(i)||/||b|| 2.564097258496e-02 > 15 KSP preconditioned resid norm 1.153005952878e+15 true resid norm > 2.281947914341e+14 ||r(i)||/||b|| 2.193807128394e-02 > 16 KSP preconditioned resid norm 1.029063568485e+15 true resid norm > 1.605944355705e+14 ||r(i)||/||b|| 1.543914369478e-02 > 17 KSP preconditioned resid norm 8.755986525312e+14 true resid norm > 1.474488326268e+14 ||r(i)||/||b|| 1.417535860732e-02 > 18 KSP preconditioned resid norm 7.769999155370e+14 true resid norm > 1.263847845681e+14 ||r(i)||/||b|| 1.215031419270e-02 > 19 KSP preconditioned resid norm 6.633003954457e+14 true resid norm > 1.037940616116e+14 ||r(i)||/||b|| 9.978499106727e-03 > 20 KSP preconditioned resid norm 5.634039280371e+14 true resid norm > 1.015458370156e+14 ||r(i)||/||b|| 9.762360468599e-03 > 21 KSP preconditioned resid norm 4.739327677772e+14 true resid norm > 7.392362794765e+13 ||r(i)||/||b|| 7.106831007366e-03 > 22 KSP preconditioned resid norm 3.847430187788e+14 true resid norm > 7.626771651358e+13 ||r(i)||/||b|| 7.332185765606e-03 > 23 KSP preconditioned resid norm 3.166693300565e+14 true resid norm > 4.910897850051e+13 ||r(i)||/||b|| 4.721213241789e-03 > 24 KSP preconditioned resid norm 2.603961739838e+14 true resid norm > 5.903512955155e+13 ||r(i)||/||b|| 5.675488350193e-03 > 25 KSP preconditioned resid norm 2.300385519514e+14 true resid norm > 3.524302891352e+13 ||r(i)||/||b|| 3.388175846206e-03 > 26 KSP preconditioned resid norm 1.995854205790e+14 true resid norm > 5.125398452570e+13 ||r(i)||/||b|| 4.927428707047e-03 > 27 KSP preconditioned resid norm 1.777736693158e+14 true resid norm > 2.713391594494e+13 ||r(i)||/||b|| 2.608586192838e-03 > 28 KSP preconditioned resid norm 1.569155636768e+14 true resid norm > 4.045925214031e+13 ||r(i)||/||b|| 3.889650381462e-03 > 29 KSP preconditioned resid norm 1.392286125087e+14 true resid norm > 2.127117420025e+13 ||r(i)||/||b|| 2.044956999086e-03 > 30 KSP preconditioned resid norm 1.268134787155e+14 true resid norm > 2.836210701297e+13 ||r(i)||/||b|| 2.726661382160e-03 > 31 KSP preconditioned resid norm 1.148332814766e+14 true resid norm > 1.759485016641e+13 ||r(i)||/||b|| 1.691524485529e-03 > 32 KSP preconditioned resid norm 1.072826684457e+14 true resid norm > 1.915166944365e+13 ||r(i)||/||b|| 1.841193161425e-03 > 33 KSP preconditioned resid norm 9.907876839152e+13 true resid norm > 1.514435361562e+13 ||r(i)||/||b|| 1.455939932199e-03 > 34 KSP preconditioned resid norm 9.246074413874e+13 true resid norm > 1.530899883812e+13 ||r(i)||/||b|| 1.471768508325e-03 > 35 KSP preconditioned resid norm 8.716833484735e+13 true resid norm > 1.331822154534e+13 ||r(i)||/||b|| 1.280380204127e-03 > 36 KSP preconditioned resid norm 8.124134503515e+13 true resid norm > 1.344545385639e+13 ||r(i)||/||b|| 1.292611997376e-03 > 37 KSP preconditioned resid norm 7.675303961825e+13 true resid norm > 1.172301521125e+13 ||r(i)||/||b|| 1.127021093474e-03 > 38 KSP preconditioned resid norm 7.135475879451e+13 true resid norm > 1.180708573448e+13 ||r(i)||/||b|| 1.135103421383e-03 > 39 KSP preconditioned resid norm 6.615399842578e+13 true resid norm > 1.009923425569e+13 ||r(i)||/||b|| 9.709148908363e-04 > 40 KSP preconditioned resid norm 6.137620595303e+13 true resid norm > 9.912104999293e+12 ||r(i)||/||b|| 9.529247564410e-04 > 41 KSP preconditioned resid norm 5.598390411354e+13 true resid norm > 8.555770285650e+12 ||r(i)||/||b|| 8.225301604653e-04 > 0 KSP preconditioned resid norm 2.200654561303e+16 true resid norm > 1.374535903551e+16 ||r(i)||/||b|| 1.042244800141e-01 > 1 KSP preconditioned resid norm 1.033788570768e+16 true resid norm > 1.766933748516e+15 ||r(i)||/||b|| 1.339781308605e-02 > 2 KSP preconditioned resid norm 8.199712159807e+15 true resid norm > 1.328504928257e+15 ||r(i)||/||b|| 1.007341714291e-02 > 3 KSP preconditioned resid norm 7.724963557325e+15 true resid norm > 1.290091137776e+15 ||r(i)||/||b|| 9.782143751798e-03 > 4 KSP preconditioned resid norm 7.785648142256e+15 true resid norm > 1.238722001104e+15 ||r(i)||/||b|| 9.392636170038e-03 > 5 KSP preconditioned resid norm 8.249045346662e+15 true resid norm > 1.288778250144e+15 ||r(i)||/||b|| 9.772188753144e-03 > 6 KSP preconditioned resid norm 9.469275305646e+15 true resid norm > 1.408770770519e+15 ||r(i)||/||b|| 1.068203461525e-02 > 7 KSP preconditioned resid norm 1.105611185924e+16 true resid norm > 1.994387714295e+15 ||r(i)||/||b|| 1.512248766524e-02 > 8 KSP preconditioned resid norm 1.130059557946e+16 true resid norm > 1.768360656538e+15 ||r(i)||/||b|| 1.340863264676e-02 > 9 KSP preconditioned resid norm 1.052660615325e+16 true resid norm > 1.953565171874e+15 ||r(i)||/||b|| 1.481294986084e-02 > 10 KSP preconditioned resid norm 8.559164814525e+15 true resid norm > 1.322544665666e+15 ||r(i)||/||b|| 1.002822332384e-02 > 11 KSP preconditioned resid norm 4.990037328467e+15 true resid norm > 7.740616289621e+14 ||r(i)||/||b|| 5.869338921523e-03 > 12 KSP preconditioned resid norm 1.334591152980e+17 true resid norm > 2.007497042766e+16 ||r(i)||/||b|| 1.522188942984e-01 > 13 KSP preconditioned resid norm 7.977709053830e+14 true resid norm > 2.666709181112e+14 ||r(i)||/||b|| 2.022037962283e-03 > 0 KSP preconditioned resid norm 1.928493598618e+16 true resid norm > 1.203377031534e+16 ||r(i)||/||b|| 4.946657154689e-01 > 1 KSP preconditioned resid norm 9.231841001478e+15 true resid norm > 1.582820058439e+15 ||r(i)||/||b|| 6.506413170177e-02 > 2 KSP preconditioned resid norm 7.541228065487e+15 true resid norm > 1.216411600127e+15 ||r(i)||/||b|| 5.000237653817e-02 > 3 KSP preconditioned resid norm 7.339567418801e+15 true resid norm > 1.219667619599e+15 ||r(i)||/||b|| 5.013621997708e-02 > 4 KSP preconditioned resid norm 7.654746286546e+15 true resid norm > 1.212045913869e+15 ||r(i)||/||b|| 4.982291862433e-02 > 5 KSP preconditioned resid norm 8.414088698130e+15 true resid norm > 1.311254037213e+15 ||r(i)||/||b|| 5.390101352132e-02 > 6 KSP preconditioned resid norm 1.008553581054e+16 true resid norm > 1.495637339777e+15 ||r(i)||/||b|| 6.148035863870e-02 > 7 KSP preconditioned resid norm 1.235521461592e+16 true resid norm > 2.202862093934e+15 ||r(i)||/||b|| 9.055186572627e-02 > 8 KSP preconditioned resid norm 1.311621757723e+16 true resid norm > 2.061404522413e+15 ||r(i)||/||b|| 8.473704551686e-02 > 9 KSP preconditioned resid norm 1.289521641859e+16 true resid norm > 2.354903652303e+15 ||r(i)||/||b|| 9.680175618294e-02 > 10 KSP preconditioned resid norm 1.151646040420e+16 true resid norm > 1.821568826477e+15 ||r(i)||/||b|| 7.487824872947e-02 > 11 KSP preconditioned resid norm 8.378553284195e+15 true resid norm > 1.434374156491e+15 ||r(i)||/||b|| 5.896204595714e-02 > 12 KSP preconditioned resid norm 3.545538676243e+15 true resid norm > 5.411687842502e+14 ||r(i)||/||b|| 2.224553376337e-02 > 13 KSP preconditioned resid norm 1.159722122499e+15 true resid norm > 2.107714204908e+14 ||r(i)||/||b|| 8.664067269472e-03 > 14 KSP preconditioned resid norm 9.539010029354e+14 true resid norm > 1.788314612223e+14 ||r(i)||/||b|| 7.351128565343e-03 > 15 KSP preconditioned resid norm 8.844744179759e+14 true resid norm > 1.382762666459e+14 ||r(i)||/||b|| 5.684048023218e-03 > 16 KSP preconditioned resid norm 7.998051624629e+14 true resid norm > 2.151800146054e+14 ||r(i)||/||b|| 8.845288973459e-03 > 17 KSP preconditioned resid norm 6.751752084065e+14 true resid norm > 1.049375541763e+14 ||r(i)||/||b|| 4.313611524562e-03 > 18 KSP preconditioned resid norm 6.016391198259e+14 true resid norm > 1.405509222696e+14 ||r(i)||/||b|| 5.777551066910e-03 > 19 KSP preconditioned resid norm 5.356608668579e+14 true resid norm > 8.319469793388e+13 ||r(i)||/||b|| 3.419839642795e-03 > 20 KSP preconditioned resid norm 4.666955314009e+14 true resid norm > 1.159814975417e+14 ||r(i)||/||b|| 4.767588956680e-03 > 21 KSP preconditioned resid norm 4.123539296814e+14 true resid norm > 6.367161120725e+13 ||r(i)||/||b|| 2.617314631039e-03 > 22 KSP preconditioned resid norm 3.572486233332e+14 true resid norm > 8.643344455842e+13 ||r(i)||/||b|| 3.552973056037e-03 > 23 KSP preconditioned resid norm 3.110551508962e+14 true resid norm > 4.773889823616e+13 ||r(i)||/||b|| 1.962377179627e-03 > 24 KSP preconditioned resid norm 2.787205893436e+14 true resid norm > 4.531540777818e+13 ||r(i)||/||b|| 1.862756062561e-03 > 25 KSP preconditioned resid norm 2.574052602586e+14 true resid norm > 4.020202768069e+13 ||r(i)||/||b|| 1.652563100745e-03 > 26 KSP preconditioned resid norm 2.417321052344e+14 true resid norm > 3.935243057179e+13 ||r(i)||/||b|| 1.617639169947e-03 > 27 KSP preconditioned resid norm 2.257728086711e+14 true resid norm > 3.470313603079e+13 ||r(i)||/||b|| 1.426523123165e-03 > 28 KSP preconditioned resid norm 2.082687407590e+14 true resid norm > 3.533906053355e+13 ||r(i)||/||b|| 1.452663729217e-03 > 29 KSP preconditioned resid norm 1.877437540892e+14 true resid norm > 2.879571658728e+13 ||r(i)||/||b|| 1.183690013589e-03 > 30 KSP preconditioned resid norm 1.698389677822e+14 true resid norm > 2.905672512335e+13 ||r(i)||/||b|| 1.194419150913e-03 > 31 KSP preconditioned resid norm 1.483411237433e+14 true resid norm > 2.274318996948e+13 ||r(i)||/||b|| 9.348920615478e-04 > 32 KSP preconditioned resid norm 1.319650244589e+14 true resid norm > 2.174098043497e+13 ||r(i)||/||b|| 8.936947739606e-04 > 0 KSP preconditioned resid norm 6.130010037542e+16 true resid norm > 4.019535297204e+16 ||r(i)||/||b|| 5.748505993040e-02 > 1 KSP preconditioned resid norm 2.383231272294e+16 true resid norm > 4.333028202793e+15 ||r(i)||/||b|| 6.196845344062e-03 > 2 KSP preconditioned resid norm 1.380314308424e+16 true resid norm > 2.902938638811e+15 ||r(i)||/||b|| 4.151614285921e-03 > 3 KSP preconditioned resid norm 9.355422663861e+15 true resid norm > 1.914374541722e+15 ||r(i)||/||b|| 2.737827313936e-03 > 4 KSP preconditioned resid norm 6.905389449720e+15 true resid norm > 1.160495113499e+15 ||r(i)||/||b|| 1.659672728707e-03 > 5 KSP preconditioned resid norm 5.388850196885e+15 true resid norm > 9.343171628690e+14 ||r(i)||/||b|| 1.336206156440e-03 > 6 KSP preconditioned resid norm 4.456941607393e+15 true resid norm > 7.443205302890e+14 ||r(i)||/||b|| 1.064484004428e-03 > 7 KSP preconditioned resid norm 3.604897945956e+15 true resid norm > 7.177354673261e+14 ||r(i)||/||b|| 1.026463591006e-03 > 8 KSP preconditioned resid norm 2.616552593649e+15 true resid norm > 4.566784798390e+14 ||r(i)||/||b|| 6.531150454316e-04 > 9 KSP preconditioned resid norm 1.730753896393e+15 true resid norm > 3.416577992882e+14 ||r(i)||/||b|| 4.886191466320e-04 > 10 KSP preconditioned resid norm 9.605913238033e+14 true resid norm > 1.715828091641e+14 ||r(i)||/||b|| 2.453877709368e-04 > 11 KSP preconditioned resid norm 3.885469473149e+14 true resid norm > 7.526615944264e+13 ||r(i)||/||b|| 1.076412910045e-04 > 12 KSP preconditioned resid norm 1.085402135701e+14 true resid norm > 1.877490961949e+13 ||r(i)||/||b|| 2.685078559740e-05 > 13 KSP preconditioned resid norm 5.521535097548e+13 true resid norm > 1.482831670997e+13 ||r(i)||/||b|| 2.120659757192e-05 > 14 KSP preconditioned resid norm 4.327778991064e+13 true resid norm > 6.949075957635e+12 ||r(i)||/||b|| 9.938164945670e-06 > 0 KSP preconditioned resid norm 2.469229041488e+16 true resid norm > 1.581397714283e+16 ||r(i)||/||b|| 1.421322439001e-01 > 1 KSP preconditioned resid norm 1.094866313372e+16 true resid norm > 1.827538507852e+15 ||r(i)||/||b|| 1.642547896642e-02 > 2 KSP preconditioned resid norm 7.902647446780e+15 true resid norm > 1.331972422469e+15 ||r(i)||/||b|| 1.197144952903e-02 > 3 KSP preconditioned resid norm 6.757114900348e+15 true resid norm > 1.177404752019e+15 ||r(i)||/||b|| 1.058223227919e-02 > 4 KSP preconditioned resid norm 6.236820681429e+15 true resid norm > 9.858934558838e+14 ||r(i)||/||b|| 8.860974558495e-03 > 5 KSP preconditioned resid norm 6.056208231119e+15 true resid norm > 9.498735329089e+14 ||r(i)||/||b|| 8.537236106662e-03 > 6 KSP preconditioned resid norm 6.281150442631e+15 true resid norm > 9.439017368810e+14 ||r(i)||/||b|| 8.483563032401e-03 > 7 KSP preconditioned resid norm 6.511637444968e+15 true resid norm > 1.175605283055e+15 ||r(i)||/||b|| 1.056605908257e-02 > 8 KSP preconditioned resid norm 5.897282453520e+15 true resid norm > 9.358326231502e+14 ||r(i)||/||b|| 8.411039768298e-03 > 9 KSP preconditioned resid norm 4.858643646570e+15 true resid norm > 9.074077133036e+14 ||r(i)||/||b|| 8.155563477757e-03 > 10 KSP preconditioned resid norm 3.431227043473e+15 true resid norm > 5.523077341885e+14 ||r(i)||/||b|| 4.964009804403e-03 > 11 KSP preconditioned resid norm 1.790602152143e+15 true resid norm > 3.233768236148e+14 ||r(i)||/||b|| 2.906433539808e-03 > 12 KSP preconditioned resid norm 6.061523931075e+14 true resid norm > 9.361733206500e+13 ||r(i)||/||b|| 8.414101875932e-04 > 13 KSP preconditioned resid norm 3.669173259935e+14 true resid norm > 7.433445474494e+13 ||r(i)||/||b|| 6.681002986515e-04 > 14 KSP preconditioned resid norm 1.034316883992e+15 true resid norm > 1.749783469552e+14 ||r(i)||/||b|| 1.572663528097e-03 > 15 KSP preconditioned resid norm 1.340579303848e+14 true resid norm > 2.477734260421e+13 ||r(i)||/||b|| 2.226928286548e-04 > 16 KSP preconditioned resid norm 1.124182191319e+14 true resid norm > 1.917839477947e+13 ||r(i)||/||b|| 1.723708248589e-04 > 17 KSP preconditioned resid norm 9.394060484065e+13 true resid norm > 1.682558945581e+13 ||r(i)||/||b|| 1.512243733944e-04 > 18 KSP preconditioned resid norm 9.034672558986e+13 true resid norm > 1.892870450718e+13 ||r(i)||/||b|| 1.701266684168e-04 > 19 KSP preconditioned resid norm 8.058964131802e+13 true resid norm > 1.275370577304e+13 ||r(i)||/||b|| 1.146272568369e-04 > 20 KSP preconditioned resid norm 6.303390378547e+13 true resid norm > 1.900069972515e+13 ||r(i)||/||b|| 1.707737442148e-04 > 21 KSP preconditioned resid norm 4.601826552473e+13 true resid norm > 6.960266416937e+12 ||r(i)||/||b|| 6.255720967895e-05 > 22 KSP preconditioned resid norm 2.822788042823e+13 true resid norm > 1.013300658091e+13 ||r(i)||/||b|| 9.107303936204e-05 > 23 KSP preconditioned resid norm 1.858182222289e+13 true resid norm > 2.785683196819e+12 ||r(i)||/||b|| 2.503705424529e-05 > 24 KSP preconditioned resid norm 9.506926663598e+12 true resid norm > 4.268961211449e+12 ||r(i)||/||b|| 3.836840224479e-05 > 25 KSP preconditioned resid norm 5.845171932356e+12 true resid norm > 8.923162702109e+11 ||r(i)||/||b|| 8.019925197071e-06 > 0 SNES Function norm 0.000000000000e+00 > 4 TS dt 6e-12 time 1.4e-11 > 0 KSP preconditioned resid norm 2.220105437401e+19 true resid norm > 1.204162959566e+19 ||r(i)||/||b|| 2.302124615573e+02 > 1 KSP preconditioned resid norm 5.389751984623e+18 true resid norm > 1.775745946542e+18 ||r(i)||/||b|| 3.394879756150e+01 > 2 KSP preconditioned resid norm 2.218217192238e+18 true resid norm > 5.319400119211e+17 ||r(i)||/||b|| 1.016965507636e+01 > 3 KSP preconditioned resid norm 1.401615472906e+18 true resid norm > 2.963370402601e+17 ||r(i)||/||b|| 5.665385980104e+00 > 4 KSP preconditioned resid norm 1.004001983833e+18 true resid norm > 1.656139793050e+17 ||r(i)||/||b|| 3.166216128906e+00 > 5 KSP preconditioned resid norm 8.672200498511e+17 true resid norm > 1.510248645324e+17 ||r(i)||/||b|| 2.887300721564e+00 > 6 KSP preconditioned resid norm 7.388847075384e+17 true resid norm > 1.526081348396e+17 ||r(i)||/||b|| 2.917569760470e+00 > 7 KSP preconditioned resid norm 5.565797182786e+17 true resid norm > 1.017271957075e+17 ||r(i)||/||b|| 1.944825486043e+00 > 8 KSP preconditioned resid norm 3.566434137336e+17 true resid norm > 6.306431985913e+16 ||r(i)||/||b|| 1.205666741023e+00 > 9 KSP preconditioned resid norm 1.893287605930e+17 true resid norm > 3.267942353405e+16 ||r(i)||/||b|| 6.247668120231e-01 > 10 KSP preconditioned resid norm 6.895870882324e+16 true resid norm > 1.153127225742e+16 ||r(i)||/||b|| 2.204554250883e-01 > 11 KSP preconditioned resid norm 5.104972946485e+16 true resid norm > 9.275497869859e+15 ||r(i)||/||b|| 1.773294203933e-01 > 12 KSP preconditioned resid norm 4.992720890366e+16 true resid norm > 8.399052910493e+15 ||r(i)||/||b|| 1.605735029394e-01 > 13 KSP preconditioned resid norm 4.919979261885e+16 true resid norm > 8.426637216788e+15 ||r(i)||/||b|| 1.611008610517e-01 > 14 KSP preconditioned resid norm 5.688292138515e+16 true resid norm > 9.762950550877e+15 ||r(i)||/||b|| 1.866485645090e-01 > 15 KSP preconditioned resid norm 1.614157649132e+17 true resid norm > 4.341833601566e+16 ||r(i)||/||b|| 8.300738642954e-01 > 16 KSP preconditioned resid norm 4.125579354715e+16 true resid norm > 7.099585962609e+15 ||r(i)||/||b|| 1.357302304896e-01 > 17 KSP preconditioned resid norm 2.163051916179e+16 true resid norm > 5.849969022355e+15 ||r(i)||/||b|| 1.118399929155e-01 > 18 KSP preconditioned resid norm 1.415003279507e+16 true resid norm > 2.432246275858e+15 ||r(i)||/||b|| 4.649980285729e-02 > 19 KSP preconditioned resid norm 1.088163683743e+16 true resid norm > 2.488576155885e+15 ||r(i)||/||b|| 4.757672024934e-02 > 20 KSP preconditioned resid norm 9.060374162401e+15 true resid norm > 1.588297599957e+15 ||r(i)||/||b|| 3.036515093466e-02 > 21 KSP preconditioned resid norm 7.815461281280e+15 true resid norm > 1.346385634634e+15 ||r(i)||/||b|| 2.574026619006e-02 > 22 KSP preconditioned resid norm 6.943207518072e+15 true resid norm > 1.184751633436e+15 ||r(i)||/||b|| 2.265013947659e-02 > 23 KSP preconditioned resid norm 6.246190157097e+15 true resid norm > 1.023840274541e+15 ||r(i)||/||b|| 1.957382827391e-02 > 24 KSP preconditioned resid norm 5.682318425500e+15 true resid norm > 9.496257176169e+14 ||r(i)||/||b|| 1.815499075718e-02 > 25 KSP preconditioned resid norm 5.282641218542e+15 true resid norm > 8.559439950771e+14 ||r(i)||/||b|| 1.636397901932e-02 > 26 KSP preconditioned resid norm 5.115699254805e+15 true resid norm > 8.571023693113e+14 ||r(i)||/||b|| 1.638612487439e-02 > 27 KSP preconditioned resid norm 4.941038732000e+15 true resid norm > 9.550184315260e+14 ||r(i)||/||b|| 1.825808892456e-02 > 28 KSP preconditioned resid norm 4.836641976772e+15 true resid norm > 7.564719174843e+14 ||r(i)||/||b|| 1.446226699132e-02 > 29 KSP preconditioned resid norm 4.905885514019e+15 true resid norm > 7.773698572205e+14 ||r(i)||/||b|| 1.486179482182e-02 > 30 KSP preconditioned resid norm 5.169672483375e+15 true resid norm > 8.117514654202e+14 ||r(i)||/||b|| 1.551910408326e-02 > 31 KSP preconditioned resid norm 5.732174203621e+15 true resid norm > 9.079138157636e+14 ||r(i)||/||b|| 1.735754058438e-02 > 32 KSP preconditioned resid norm 8.584579912384e+15 true resid norm > 1.381429969496e+15 ||r(i)||/||b|| 2.641024549212e-02 > 33 KSP preconditioned resid norm 1.658456221337e+16 true resid norm > 2.668310391151e+15 ||r(i)||/||b|| 5.101288811997e-02 > 34 KSP preconditioned resid norm 3.797117274875e+16 true resid norm > 6.214594485387e+15 ||r(i)||/||b|| 1.188109202908e-01 > 35 KSP preconditioned resid norm 6.896188371971e+15 true resid norm > 1.129650731508e+15 ||r(i)||/||b|| 2.159671774774e-02 > 36 KSP preconditioned resid norm 3.678389972715e+15 true resid norm > 6.014447539523e+14 ||r(i)||/||b|| 1.149845012239e-02 > 37 KSP preconditioned resid norm 2.344506104501e+15 true resid norm > 4.262436512363e+14 ||r(i)||/||b|| 8.148946900809e-03 > 38 KSP preconditioned resid norm 1.979277723710e+15 true resid norm > 3.194432632084e+14 ||r(i)||/||b|| 6.107131876701e-03 > 39 KSP preconditioned resid norm 1.806670892810e+15 true resid norm > 3.108273060962e+14 ||r(i)||/||b|| 5.942411588660e-03 > 40 KSP preconditioned resid norm 1.663957842585e+15 true resid norm > 2.662101226861e+14 ||r(i)||/||b|| 5.089418101442e-03 > 41 KSP preconditioned resid norm 1.486329834699e+15 true resid norm > 2.427881601889e+14 ||r(i)||/||b|| 4.641635880760e-03 > 42 KSP preconditioned resid norm 1.346929765546e+15 true resid norm > 2.207126725819e+14 ||r(i)||/||b|| 4.219595632660e-03 > 43 KSP preconditioned resid norm 1.242596801597e+15 true resid norm > 2.017748115787e+14 ||r(i)||/||b|| 3.857540682909e-03 > 44 KSP preconditioned resid norm 1.178815061343e+15 true resid norm > 1.881589524072e+14 ||r(i)||/||b|| 3.597232023587e-03 > 45 KSP preconditioned resid norm 9.347996687830e+14 true resid norm > 2.754924905110e+14 ||r(i)||/||b|| 5.266878861971e-03 > 46 KSP preconditioned resid norm 7.515414326072e+14 true resid norm > 1.170313953112e+14 ||r(i)||/||b|| 2.237411920043e-03 > 47 KSP preconditioned resid norm 7.344982573734e+14 true resid norm > 1.334969192887e+14 ||r(i)||/||b|| 2.552200610027e-03 > 48 KSP preconditioned resid norm 8.030126805909e+14 true resid norm > 1.311086080593e+14 ||r(i)||/||b|| 2.506540759528e-03 > 49 KSP preconditioned resid norm 9.282223496248e+14 true resid norm > 1.446635532628e+14 ||r(i)||/||b|| 2.765684862641e-03 > 50 KSP preconditioned resid norm 1.074003818336e+15 true resid norm > 3.137568207714e+14 ||r(i)||/||b|| 5.998418193015e-03 > 51 KSP preconditioned resid norm 1.638671425800e+15 true resid norm > 2.543484445494e+14 ||r(i)||/||b|| 4.862645960648e-03 > 52 KSP preconditioned resid norm 2.418883444028e+15 true resid norm > 6.581384678327e+14 ||r(i)||/||b|| 1.258232330779e-02 > 53 KSP preconditioned resid norm 8.572181890774e+14 true resid norm > 1.327918910006e+14 ||r(i)||/||b|| 2.538721844848e-03 > 54 KSP preconditioned resid norm 2.577275091696e+14 true resid norm > 8.010767963629e+13 ||r(i)||/||b|| 1.531502523990e-03 > 0 KSP preconditioned resid norm 6.542967041071e+18 true resid norm > 3.545980595607e+18 ||r(i)||/||b|| 4.156826068038e+02 > 1 KSP preconditioned resid norm 1.591191551470e+18 true resid norm > 5.223269779130e+17 ||r(i)||/||b|| 6.123052112913e+01 > 2 KSP preconditioned resid norm 6.570663371928e+17 true resid norm > 1.552539857829e+17 ||r(i)||/||b|| 1.819986877731e+01 > 3 KSP preconditioned resid norm 4.165904471738e+17 true resid norm > 8.738677344351e+16 ||r(i)||/||b|| 1.024403851227e+01 > 4 KSP preconditioned resid norm 2.995075360323e+17 true resid norm > 4.934782956985e+16 ||r(i)||/||b|| 5.784869342240e+00 > 5 KSP preconditioned resid norm 2.594552595600e+17 true resid norm > 4.469892332113e+16 ||r(i)||/||b|| 5.239894710780e+00 > 6 KSP preconditioned resid norm 2.219662091794e+17 true resid norm > 4.652217437246e+16 ||r(i)||/||b|| 5.453627902330e+00 > 7 KSP preconditioned resid norm 1.674952385698e+17 true resid norm > 2.988890119169e+16 ||r(i)||/||b|| 3.503768852333e+00 > 8 KSP preconditioned resid norm 1.081926357303e+17 true resid norm > 1.908191297028e+16 ||r(i)||/||b|| 2.236904323761e+00 > 9 KSP preconditioned resid norm 5.710239913499e+16 true resid norm > 9.635393149266e+15 ||r(i)||/||b|| 1.129522633832e+00 > 10 KSP preconditioned resid norm 1.779371916701e+16 true resid norm > 2.952475144259e+15 ||r(i)||/||b|| 3.461080881292e-01 > 11 KSP preconditioned resid norm 1.125590077314e+16 true resid norm > 2.030360591090e+15 ||r(i)||/||b|| 2.380119012218e-01 > 12 KSP preconditioned resid norm 1.086608349874e+16 true resid norm > 1.877856566597e+15 ||r(i)||/||b|| 2.201344005587e-01 > 13 KSP preconditioned resid norm 1.075368648574e+16 true resid norm > 1.822636849246e+15 ||r(i)||/||b|| 2.136611908395e-01 > 14 KSP preconditioned resid norm 1.173878698877e+16 true resid norm > 2.130421173676e+15 ||r(i)||/||b|| 2.497416449940e-01 > 15 KSP preconditioned resid norm 4.944017897832e+16 true resid norm > 1.361808358525e+16 ||r(i)||/||b|| 1.596399171332e+00 > 16 KSP preconditioned resid norm 1.780572750257e+17 true resid norm > 3.371295639234e+16 ||r(i)||/||b|| 3.952049149280e+00 > 17 KSP preconditioned resid norm 3.251503916379e+16 true resid norm > 8.719191629936e+15 ||r(i)||/||b|| 1.022119610706e+00 > 18 KSP preconditioned resid norm 2.075735168556e+16 true resid norm > 3.945307122742e+15 ||r(i)||/||b|| 4.624942255619e-01 > 19 KSP preconditioned resid norm 1.483613863843e+16 true resid norm > 3.390657011871e+15 ||r(i)||/||b|| 3.974745793076e-01 > 20 KSP preconditioned resid norm 1.134405248775e+16 true resid norm > 2.490065225313e+15 ||r(i)||/||b|| 2.919014292554e-01 > 21 KSP preconditioned resid norm 8.553736210329e+15 true resid norm > 1.657655420050e+15 ||r(i)||/||b|| 1.943210087055e-01 > 22 KSP preconditioned resid norm 6.586515952486e+15 true resid norm > 1.286392357530e+15 ||r(i)||/||b|| 1.507991694069e-01 > 23 KSP preconditioned resid norm 4.780127454522e+15 true resid norm > 9.064248964802e+14 ||r(i)||/||b|| 1.062569446397e-01 > 24 KSP preconditioned resid norm 3.027986246405e+15 true resid norm > 6.179735775190e+14 ||r(i)||/||b|| 7.244282948339e-02 > 25 KSP preconditioned resid norm 1.571830581724e+15 true resid norm > 2.758055375192e+14 ||r(i)||/||b|| 3.233169548331e-02 > 26 KSP preconditioned resid norm 1.182949949046e+15 true resid norm > 2.283703518838e+14 ||r(i)||/||b|| 2.677103854019e-02 > 27 KSP preconditioned resid norm 1.078564628165e+15 true resid norm > 2.041848112402e+14 ||r(i)||/||b|| 2.393585422076e-02 > 28 KSP preconditioned resid norm 1.016575362098e+15 true resid norm > 1.676746397025e+14 ||r(i)||/||b|| 1.965589755702e-02 > 29 KSP preconditioned resid norm 9.501065512919e+14 true resid norm > 1.807838083536e+14 ||r(i)||/||b|| 2.119263845308e-02 > 30 KSP preconditioned resid norm 9.022105299639e+14 true resid norm > 1.464542351381e+14 ||r(i)||/||b|| 1.716830552177e-02 > 31 KSP preconditioned resid norm 8.706508374697e+14 true resid norm > 1.587763993315e+14 ||r(i)||/||b|| 1.861278870358e-02 > 32 KSP preconditioned resid norm 8.363545887412e+14 true resid norm > 1.341498283380e+14 ||r(i)||/||b|| 1.572590397559e-02 > 33 KSP preconditioned resid norm 8.049250107078e+14 true resid norm > 1.594310708223e+14 ||r(i)||/||b|| 1.868953349802e-02 > 34 KSP preconditioned resid norm 7.730163706333e+14 true resid norm > 1.360746921537e+14 ||r(i)||/||b|| 1.595154886763e-02 > 35 KSP preconditioned resid norm 7.656443951451e+14 true resid norm > 1.202971650940e+14 ||r(i)||/||b|| 1.410200587090e-02 > 36 KSP preconditioned resid norm 7.731881795958e+14 true resid norm > 1.274146022068e+14 ||r(i)||/||b|| 1.493635753557e-02 > 37 KSP preconditioned resid norm 8.066446225342e+14 true resid norm > 1.271087970298e+14 ||r(i)||/||b|| 1.490050908979e-02 > 38 KSP preconditioned resid norm 8.449932808607e+14 true resid norm > 1.374808484319e+14 ||r(i)||/||b|| 1.611638753257e-02 > 39 KSP preconditioned resid norm 8.841110986186e+14 true resid norm > 1.396540752943e+14 ||r(i)||/||b|| 1.637114713515e-02 > 40 KSP preconditioned resid norm 9.488359303223e+14 true resid norm > 1.529468173159e+14 ||r(i)||/||b|| 1.792940768006e-02 > 41 KSP preconditioned resid norm 1.207535627829e+15 true resid norm > 1.911287075526e+14 ||r(i)||/||b|| 2.240533394033e-02 > 42 KSP preconditioned resid norm 2.145532208476e+15 true resid norm > 3.678432914805e+14 ||r(i)||/||b|| 4.312095178618e-02 > 43 KSP preconditioned resid norm 1.285566323841e+16 true resid norm > 2.041241798720e+15 ||r(i)||/||b|| 2.392874662258e-01 > 44 KSP preconditioned resid norm 1.228721458245e+15 true resid norm > 2.037912017178e+14 ||r(i)||/||b|| 2.388971278599e-02 > 45 KSP preconditioned resid norm 6.120797893644e+14 true resid norm > 9.792977962736e+13 ||r(i)||/||b|| 1.147995737192e-02 > 46 KSP preconditioned resid norm 4.263304414021e+14 true resid norm > 6.948352885735e+13 ||r(i)||/||b|| 8.145305262283e-03 > 47 KSP preconditioned resid norm 2.943667166575e+14 true resid norm > 4.740665276317e+13 ||r(i)||/||b|| 5.557312136691e-03 > 48 KSP preconditioned resid norm 1.946102625201e+14 true resid norm > 3.133411275125e+13 ||r(i)||/||b|| 3.673185828050e-03 > 49 KSP preconditioned resid norm 1.369383238938e+14 true resid norm > 2.605829767015e+13 ||r(i)||/||b|| 3.054720919178e-03 > 50 KSP preconditioned resid norm 9.297342410318e+13 true resid norm > 1.484605459745e+13 ||r(i)||/||b|| 1.740349815638e-03 > 51 KSP preconditioned resid norm 6.674823642018e+13 true resid norm > 2.111621172189e+13 ||r(i)||/||b|| 2.475377881439e-03 > 52 KSP preconditioned resid norm 5.794642992605e+13 true resid norm > 9.469052681739e+12 ||r(i)||/||b|| 1.110023136501e-03 > 53 KSP preconditioned resid norm 4.582540149696e+13 true resid norm > 7.443523694856e+12 ||r(i)||/||b|| 8.725776269382e-04 > 0 KSP preconditioned resid norm 1.892653228182e+19 true resid norm > 1.028330526045e+19 ||r(i)||/||b|| 7.841194129911e+01 > 1 KSP preconditioned resid norm 4.581281818866e+18 true resid norm > 1.519814562161e+18 ||r(i)||/||b|| 1.158884300480e+01 > 2 KSP preconditioned resid norm 1.874494262231e+18 true resid norm > 4.646649902787e+17 ||r(i)||/||b|| 3.543149115843e+00 > 3 KSP preconditioned resid norm 1.173263764111e+18 true resid norm > 2.551017670341e+17 ||r(i)||/||b|| 1.945194105918e+00 > 4 KSP preconditioned resid norm 8.301325588811e+17 true resid norm > 1.381381450838e+17 ||r(i)||/||b|| 1.053326712486e+00 > 5 KSP preconditioned resid norm 7.052947600755e+17 true resid norm > 1.287824897888e+17 ||r(i)||/||b|| 9.819882590194e-01 > 6 KSP preconditioned resid norm 5.863848966069e+17 true resid norm > 1.166961566059e+17 ||r(i)||/||b|| 8.898279249584e-01 > 7 KSP preconditioned resid norm 4.330736686179e+17 true resid norm > 8.611330077675e+16 ||r(i)||/||b|| 6.566284783504e-01 > 8 KSP preconditioned resid norm 2.702589924096e+17 true resid norm > 4.803726753137e+16 ||r(i)||/||b|| 3.662922870069e-01 > 9 KSP preconditioned resid norm 1.546977321412e+17 true resid norm > 2.878601764016e+16 ||r(i)||/||b|| 2.194982516096e-01 > 10 KSP preconditioned resid norm 9.910027101694e+16 true resid norm > 1.648820746608e+16 ||r(i)||/||b|| 1.257253697341e-01 > 11 KSP preconditioned resid norm 9.241668505040e+16 true resid norm > 1.559203946128e+16 ||r(i)||/||b|| 1.188919371745e-01 > 12 KSP preconditioned resid norm 9.244329678182e+16 true resid norm > 1.506886748242e+16 ||r(i)||/||b|| 1.149026623784e-01 > 13 KSP preconditioned resid norm 7.090985777634e+16 true resid norm > 1.150411231671e+16 ||r(i)||/||b|| 8.772080151562e-02 > 14 KSP preconditioned resid norm 2.014608975967e+17 true resid norm > 3.263298275594e+16 ||r(i)||/||b|| 2.488320110574e-01 > 15 KSP preconditioned resid norm 1.667109940317e+16 true resid norm > 4.390284971711e+15 ||r(i)||/||b|| 3.347666521311e-02 > 16 KSP preconditioned resid norm 1.169352181037e+16 true resid norm > 1.861078956987e+15 ||r(i)||/||b|| 1.419104171589e-02 > 17 KSP preconditioned resid norm 1.018452985290e+16 true resid norm > 1.996695440591e+15 ||r(i)||/||b|| 1.522514033324e-02 > 18 KSP preconditioned resid norm 9.390559252345e+15 true resid norm > 1.536845912378e+15 ||r(i)||/||b|| 1.171870992984e-02 > 19 KSP preconditioned resid norm 8.796086996183e+15 true resid norm > 1.422370571244e+15 ||r(i)||/||b|| 1.084581609835e-02 > 20 KSP preconditioned resid norm 8.297954532675e+15 true resid norm > 1.485654737712e+15 ||r(i)||/||b|| 1.132836856767e-02 > 21 KSP preconditioned resid norm 7.837153561450e+15 true resid norm > 1.247795221846e+15 ||r(i)||/||b|| 9.514649542211e-03 > 22 KSP preconditioned resid norm 7.441474574169e+15 true resid norm > 1.321880544976e+15 ||r(i)||/||b|| 1.007956265733e-02 > 23 KSP preconditioned resid norm 7.117292465449e+15 true resid norm > 1.122736430687e+15 ||r(i)||/||b|| 8.561055114844e-03 > 24 KSP preconditioned resid norm 5.694316647476e+15 true resid norm > 2.071287152890e+15 ||r(i)||/||b|| 1.579391475139e-02 > 25 KSP preconditioned resid norm 4.803477075252e+15 true resid norm > 7.688470974380e+14 ||r(i)||/||b|| 5.862589113657e-03 > 26 KSP preconditioned resid norm 4.890644639484e+15 true resid norm > 7.629702578172e+14 ||r(i)||/||b|| 5.817777217899e-03 > 27 KSP preconditioned resid norm 5.168669214975e+15 true resid norm > 8.754612707062e+14 ||r(i)||/||b|| 6.675540211016e-03 > 28 KSP preconditioned resid norm 8.939764002038e+15 true resid norm > 1.397979783339e+15 ||r(i)||/||b|| 1.065983221661e-02 > 29 KSP preconditioned resid norm 8.205557149295e+15 true resid norm > 1.863983796228e+15 ||r(i)||/||b|| 1.421319160626e-02 > 30 KSP preconditioned resid norm 1.967760781914e+15 true resid norm > 3.069470515034e+14 ||r(i)||/||b|| 2.340523165932e-03 > 31 KSP preconditioned resid norm 1.158003829729e+15 true resid norm > 2.612136672864e+14 ||r(i)||/||b|| 1.991798378735e-03 > 32 KSP preconditioned resid norm 8.020817038350e+14 true resid norm > 1.249397423467e+14 ||r(i)||/||b|| 9.526866600465e-04 > 0 KSP preconditioned resid norm 6.470845423033e+18 true resid norm > 3.511573094792e+18 ||r(i)||/||b|| 1.662215647083e+02 > 1 KSP preconditioned resid norm 1.571609661901e+18 true resid norm > 5.187508694693e+17 ||r(i)||/||b|| 2.455525739870e+01 > 2 KSP preconditioned resid norm 6.468393005498e+17 true resid norm > 1.566999779328e+17 ||r(i)||/||b|| 7.417449336415e+00 > 3 KSP preconditioned resid norm 4.084082755984e+17 true resid norm > 8.688467105425e+16 ||r(i)||/||b|| 4.112716888400e+00 > 4 KSP preconditioned resid norm 2.920922708944e+17 true resid norm > 4.832004038605e+16 ||r(i)||/||b|| 2.287246343141e+00 > 5 KSP preconditioned resid norm 2.515891595070e+17 true resid norm > 4.435607154638e+16 ||r(i)||/||b|| 2.099610464520e+00 > 6 KSP preconditioned resid norm 2.133101049372e+17 true resid norm > 4.355693207474e+16 ||r(i)||/||b|| 2.061782912647e+00 > 7 KSP preconditioned resid norm 1.601303798951e+17 true resid norm > 2.999351550015e+16 ||r(i)||/||b|| 1.419753752223e+00 > 8 KSP preconditioned resid norm 1.019899000725e+17 true resid norm > 1.811131044506e+16 ||r(i)||/||b|| 8.573053386130e-01 > 9 KSP preconditioned resid norm 5.552370825926e+16 true resid norm > 9.849613114257e+15 ||r(i)||/||b|| 4.662349492457e-01 > 10 KSP preconditioned resid norm 2.614277580792e+16 true resid norm > 4.400858558792e+15 ||r(i)||/||b|| 2.083162092759e-01 > 11 KSP preconditioned resid norm 2.230379890809e+16 true resid norm > 4.001941486401e+15 ||r(i)||/||b|| 1.894333273960e-01 > 12 KSP preconditioned resid norm 2.202819377737e+16 true resid norm > 3.668912744052e+15 ||r(i)||/||b|| 1.736692931151e-01 > 13 KSP preconditioned resid norm 2.171381974288e+16 true resid norm > 3.746629440349e+15 ||r(i)||/||b|| 1.773480406489e-01 > 14 KSP preconditioned resid norm 2.756519085776e+16 true resid norm > 4.738155898553e+15 ||r(i)||/||b|| 2.242822991374e-01 > 15 KSP preconditioned resid norm 1.910971370209e+16 true resid norm > 7.272906924885e+15 ||r(i)||/||b|| 3.442656428895e-01 > 16 KSP preconditioned resid norm 8.265967788035e+15 true resid norm > 1.450280377529e+15 ||r(i)||/||b|| 6.864953885656e-02 > 17 KSP preconditioned resid norm 5.450920356042e+15 true resid norm > 1.778917951827e+15 ||r(i)||/||b|| 8.420571563182e-02 > 18 KSP preconditioned resid norm 4.184159413509e+15 true resid norm > 7.350039137637e+14 ||r(i)||/||b|| 3.479167236863e-02 > 19 KSP preconditioned resid norm 3.481860538622e+15 true resid norm > 7.678205813365e+14 ||r(i)||/||b|| 3.634506103098e-02 > 20 KSP preconditioned resid norm 3.027493421078e+15 true resid norm > 5.662352682787e+14 ||r(i)||/||b|| 2.680294834981e-02 > 21 KSP preconditioned resid norm 2.649808113090e+15 true resid norm > 5.331458151920e+14 ||r(i)||/||b|| 2.523664728788e-02 > 22 KSP preconditioned resid norm 2.408644968014e+15 true resid norm > 3.911896234018e+14 ||r(i)||/||b|| 1.851710032633e-02 > 23 KSP preconditioned resid norm 2.245371706567e+15 true resid norm > 3.514281238053e+14 ||r(i)||/||b|| 1.663497556353e-02 > 24 KSP preconditioned resid norm 2.104176299070e+15 true resid norm > 3.427795705053e+14 ||r(i)||/||b|| 1.622559320890e-02 > 25 KSP preconditioned resid norm 1.992290690493e+15 true resid norm > 3.118626867030e+14 ||r(i)||/||b|| 1.476213148881e-02 > 26 KSP preconditioned resid norm 1.936852873636e+15 true resid norm > 3.191431362940e+14 ||r(i)||/||b|| 1.510675416649e-02 > 27 KSP preconditioned resid norm 1.840320660654e+15 true resid norm > 4.277077423449e+14 ||r(i)||/||b|| 2.024569850927e-02 > 28 KSP preconditioned resid norm 1.797325493063e+15 true resid norm > 2.805438081480e+14 ||r(i)||/||b|| 1.327964120375e-02 > 29 KSP preconditioned resid norm 1.907733126356e+15 true resid norm > 3.072709450643e+14 ||r(i)||/||b|| 1.454477976088e-02 > 30 KSP preconditioned resid norm 2.241086833192e+15 true resid norm > 3.515931723170e+14 ||r(i)||/||b|| 1.664278819369e-02 > 31 KSP preconditioned resid norm 3.121906541214e+15 true resid norm > 5.044161632328e+14 ||r(i)||/||b|| 2.387671896708e-02 > 32 KSP preconditioned resid norm 2.288516501159e+16 true resid norm > 3.696560518057e+15 ||r(i)||/||b|| 1.749780103571e-01 > 33 KSP preconditioned resid norm 2.665344016377e+15 true resid norm > 4.419048360378e+14 ||r(i)||/||b|| 2.091772300206e-02 > 34 KSP preconditioned resid norm 1.193724857597e+15 true resid norm > 1.964465911853e+14 ||r(i)||/||b|| 9.298869448811e-03 > 35 KSP preconditioned resid norm 7.370892713374e+14 true resid norm > 1.243994812053e+14 ||r(i)||/||b|| 5.888493805102e-03 > 36 KSP preconditioned resid norm 5.570336976172e+14 true resid norm > 9.038862530299e+13 ||r(i)||/||b|| 4.278577812313e-03 > 37 KSP preconditioned resid norm 4.452876529911e+14 true resid norm > 9.072905275923e+13 ||r(i)||/||b|| 4.294692067355e-03 > 38 KSP preconditioned resid norm 4.015373409323e+14 true resid norm > 6.374984875718e+13 ||r(i)||/||b|| 3.017621824831e-03 > 39 KSP preconditioned resid norm 3.753907061691e+14 true resid norm > 7.170117945509e+13 ||r(i)||/||b|| 3.394000898950e-03 > 40 KSP preconditioned resid norm 3.520150126535e+14 true resid norm > 5.546554172401e+13 ||r(i)||/||b|| 2.625481197140e-03 > 41 KSP preconditioned resid norm 3.283953946659e+14 true resid norm > 5.672854000420e+13 ||r(i)||/||b|| 2.685265671132e-03 > 42 KSP preconditioned resid norm 3.086422594501e+14 true resid norm > 5.033158713486e+13 ||r(i)||/||b|| 2.382463625836e-03 > 43 KSP preconditioned resid norm 2.903330837015e+14 true resid norm > 5.711996083788e+13 ||r(i)||/||b|| 2.703793715879e-03 > 44 KSP preconditioned resid norm 2.805776139313e+14 true resid norm > 4.359341431583e+13 ||r(i)||/||b|| 2.063509812540e-03 > 45 KSP preconditioned resid norm 2.850165946773e+14 true resid norm > 4.589953632859e+13 ||r(i)||/||b|| 2.172670920403e-03 > 46 KSP preconditioned resid norm 3.036441891069e+14 true resid norm > 4.745822530443e+13 ||r(i)||/||b|| 2.246452018920e-03 > 47 KSP preconditioned resid norm 3.630944657131e+14 true resid norm > 5.763700439451e+13 ||r(i)||/||b|| 2.728268157016e-03 > 48 KSP preconditioned resid norm 1.221109219167e+15 true resid norm > 1.980487895228e+14 ||r(i)||/||b|| 9.374710078478e-03 > 49 KSP preconditioned resid norm 3.188684128549e+15 true resid norm > 5.020159499531e+14 ||r(i)||/||b|| 2.376310401554e-02 > 50 KSP preconditioned resid norm 3.895126724245e+13 true resid norm > 6.299322456844e+12 ||r(i)||/||b|| 2.981806749036e-04 > 0 KSP preconditioned resid norm 6.694681597053e+18 true resid norm > 3.650754072115e+18 ||r(i)||/||b|| 5.229798617888e+00 > 1 KSP preconditioned resid norm 1.597667334483e+18 true resid norm > 5.366749713509e+17 ||r(i)||/||b|| 7.688006280303e-01 > 2 KSP preconditioned resid norm 6.300446759111e+17 true resid norm > 1.705328267334e+17 ||r(i)||/||b|| 2.442926376135e-01 > 3 KSP preconditioned resid norm 3.642174486049e+17 true resid norm > 9.481576844129e+16 ||r(i)||/||b|| 1.358260142845e-01 > 4 KSP preconditioned resid norm 2.247332959635e+17 true resid norm > 3.802800729856e+16 ||r(i)||/||b|| 5.447609345427e-02 > 5 KSP preconditioned resid norm 1.557523701769e+17 true resid norm > 3.829141999541e+16 ||r(i)||/||b|| 5.485343888228e-02 > 6 KSP preconditioned resid norm 1.004692846569e+17 true resid norm > 1.699728099873e+16 ||r(i)||/||b|| 2.434903992959e-02 > 7 KSP preconditioned resid norm 7.103482802189e+16 true resid norm > 1.472858884963e+16 ||r(i)||/||b|| 2.109908037838e-02 > 8 KSP preconditioned resid norm 5.766847885215e+16 true resid norm > 9.397954062732e+15 ||r(i)||/||b|| 1.346280965449e-02 > 9 KSP preconditioned resid norm 4.523472016725e+16 true resid norm > 1.246374917365e+16 ||r(i)||/||b|| 1.785463959348e-02 > 10 KSP preconditioned resid norm 3.195342481960e+16 true resid norm > 5.030521484214e+15 ||r(i)||/||b|| 7.206350738975e-03 > 11 KSP preconditioned resid norm 2.437460220226e+16 true resid norm > 4.974645944208e+15 ||r(i)||/||b|| 7.126307598263e-03 > 12 KSP preconditioned resid norm 1.258045555577e+16 true resid norm > 1.885910304237e+15 ||r(i)||/||b|| 2.701614764439e-03 > 13 KSP preconditioned resid norm 2.495567167302e+15 true resid norm > 4.460991814541e+14 ||r(i)||/||b|| 6.390484914966e-04 > 14 KSP preconditioned resid norm 7.678096351052e+14 true resid norm > 1.316276220022e+14 ||r(i)||/||b|| 1.885599363926e-04 > 15 KSP preconditioned resid norm 5.950867501843e+14 true resid norm > 1.439410416931e+14 ||r(i)||/||b|| 2.061992251557e-04 > 16 KSP preconditioned resid norm 3.263035989760e+14 true resid norm > 5.046543721443e+13 ||r(i)||/||b|| 7.229303003757e-05 > 17 KSP preconditioned resid norm 1.436782488882e+14 true resid norm > 7.588970356348e+13 ||r(i)||/||b|| 1.087139420975e-04 > 18 KSP preconditioned resid norm 5.401533739797e+13 true resid norm > 8.625318785849e+12 ||r(i)||/||b|| 1.235598985142e-05 > 19 KSP preconditioned resid norm 3.528859989845e+13 true resid norm > 1.233587622450e+13 ||r(i)||/||b|| 1.767145832202e-05 > 0 KSP preconditioned resid norm 4.827874033841e+18 true resid norm > 2.629049503905e+18 ||r(i)||/||b|| 2.387244450275e+01 > 1 KSP preconditioned resid norm 1.161059892070e+18 true resid norm > 3.885117260269e+17 ||r(i)||/||b|| 3.527786222537e+00 > 2 KSP preconditioned resid norm 4.696931552127e+17 true resid norm > 1.228954734891e+17 ||r(i)||/||b|| 1.115922452639e+00 > 3 KSP preconditioned resid norm 2.870101730937e+17 true resid norm > 6.759381587642e+16 ||r(i)||/||b|| 6.137692028400e-01 > 4 KSP preconditioned resid norm 1.966639305793e+17 true resid norm > 3.310590026030e+16 ||r(i)||/||b|| 3.006100742887e-01 > 5 KSP preconditioned resid norm 1.592574497021e+17 true resid norm > 3.239415772705e+16 ||r(i)||/||b|| 2.941472693472e-01 > 6 KSP preconditioned resid norm 1.230114824013e+17 true resid norm > 2.280737281616e+16 ||r(i)||/||b|| 2.070968009535e-01 > 7 KSP preconditioned resid norm 8.598640818077e+16 true resid norm > 1.938029194893e+16 ||r(i)||/||b|| 1.759780267776e-01 > 8 KSP preconditioned resid norm 5.549085768793e+16 true resid norm > 9.386094648379e+15 ||r(i)||/||b|| 8.522814928290e-02 > 9 KSP preconditioned resid norm 4.474754629124e+16 true resid norm > 7.579167807628e+15 ||r(i)||/||b|| 6.882078964122e-02 > 10 KSP preconditioned resid norm 4.378389530908e+16 true resid norm > 6.842593278076e+15 ||r(i)||/||b|| 6.213250379770e-02 > 11 KSP preconditioned resid norm 4.079498014536e+16 true resid norm > 7.704710526729e+15 ||r(i)||/||b|| 6.996074976369e-02 > 12 KSP preconditioned resid norm 4.084398125729e+16 true resid norm > 6.377957983596e+15 ||r(i)||/||b|| 5.791349602893e-02 > 13 KSP preconditioned resid norm 7.523960017073e+16 true resid norm > 1.132262466824e+16 ||r(i)||/||b|| 1.028123390664e-01 > 14 KSP preconditioned resid norm 2.184546755774e+15 true resid norm > 3.502101254985e+14 ||r(i)||/||b|| 3.179997855820e-03 > 15 KSP preconditioned resid norm 2.222188316359e+15 true resid norm > 3.654675514798e+14 ||r(i)||/||b|| 3.318539201068e-03 > 16 KSP preconditioned resid norm 1.669487598659e+15 true resid norm > 3.629534674524e+14 ||r(i)||/||b|| 3.295710672608e-03 > 17 KSP preconditioned resid norm 1.401917540926e+15 true resid norm > 2.161165666331e+14 ||r(i)||/||b|| 1.962393912861e-03 > 18 KSP preconditioned resid norm 1.134393288674e+15 true resid norm > 2.686682412462e+14 ||r(i)||/||b|| 2.439576610967e-03 > 19 KSP preconditioned resid norm 9.058009317761e+14 true resid norm > 1.409929037370e+14 ||r(i)||/||b|| 1.280251765797e-03 > 20 KSP preconditioned resid norm 7.990098269893e+14 true resid norm > 1.659151006791e+14 ||r(i)||/||b|| 1.506551712794e-03 > 21 KSP preconditioned resid norm 7.119180202197e+14 true resid norm > 1.092130755589e+14 ||r(i)||/||b|| 9.916827664832e-04 > 22 KSP preconditioned resid norm 6.306959900843e+14 true resid norm > 1.460783745609e+14 ||r(i)||/||b|| 1.326429146571e-03 > 23 KSP preconditioned resid norm 5.637228851054e+14 true resid norm > 8.672640572399e+13 ||r(i)||/||b|| 7.874980309399e-04 > 24 KSP preconditioned resid norm 4.960976586203e+14 true resid norm > 1.292271181476e+14 ||r(i)||/||b|| 1.173415411786e-03 > 25 KSP preconditioned resid norm 4.386404759546e+14 true resid norm > 8.037967670881e+13 ||r(i)||/||b|| 7.298681019622e-04 > 26 KSP preconditioned resid norm 4.419824575339e+14 true resid norm > 6.704822670651e+13 ||r(i)||/||b|| 6.088151130974e-04 > 27 KSP preconditioned resid norm 4.437076214979e+14 true resid norm > 1.370393448160e+14 ||r(i)||/||b|| 1.244352435720e-03 > 28 KSP preconditioned resid norm 7.848825938920e+14 true resid norm > 1.196984132854e+14 ||r(i)||/||b|| 1.086892325146e-03 > 29 KSP preconditioned resid norm 1.771840029177e+15 true resid norm > 6.554951997284e+14 ||r(i)||/||b|| 5.952064711635e-03 > 30 KSP preconditioned resid norm 1.052090974187e+15 true resid norm > 1.877636782214e+14 ||r(i)||/||b|| 1.704942406491e-03 > 31 KSP preconditioned resid norm 2.459238084389e+14 true resid norm > 3.913335964196e+13 ||r(i)||/||b|| 3.553409530216e-04 > 32 KSP preconditioned resid norm 1.091607449298e+14 true resid norm > 1.900052367477e+13 ||r(i)||/||b|| 1.725296333429e-04 > 33 KSP preconditioned resid norm 6.424117612547e+13 true resid norm > 1.001417309383e+13 ||r(i)||/||b|| 9.093126282646e-05 > 34 KSP preconditioned resid norm 3.801393062657e+13 true resid norm > 7.035998067262e+12 ||r(i)||/||b|| 6.388866894011e-05 > 35 KSP preconditioned resid norm 2.524483314829e+13 true resid norm > 4.006295820236e+12 ||r(i)||/||b|| 3.637819466241e-05 > 36 KSP preconditioned resid norm 1.579134906712e+13 true resid norm > 2.528645237998e+12 ||r(i)||/||b|| 2.296074798956e-05 > 37 KSP preconditioned resid norm 1.074219589287e+13 true resid norm > 1.786181249150e+12 ||r(i)||/||b|| 1.621898434352e-05 > 38 KSP preconditioned resid norm 6.572405677048e+12 true resid norm > 1.033900747383e+12 ||r(i)||/||b|| 9.388084239790e-06 > 0 KSP preconditioned resid norm 9.565416792628e+15 true resid norm > 5.898678788482e+15 ||r(i)||/||b|| 1.120363504643e-01 > 1 KSP preconditioned resid norm 4.584205668003e+15 true resid norm > 8.149165580746e+14 ||r(i)||/||b|| 1.547808931009e-02 > 2 KSP preconditioned resid norm 3.775899089492e+15 true resid norm > 6.134705686008e+14 ||r(i)||/||b|| 1.165193191355e-02 > 3 KSP preconditioned resid norm 3.695846031998e+15 true resid norm > 6.195110682482e+14 ||r(i)||/||b|| 1.176666193357e-02 > 4 KSP preconditioned resid norm 3.863984235758e+15 true resid norm > 6.333337603322e+14 ||r(i)||/||b|| 1.202920275504e-02 > 5 KSP preconditioned resid norm 4.289553507880e+15 true resid norm > 6.781260017663e+14 ||r(i)||/||b|| 1.287996263523e-02 > 6 KSP preconditioned resid norm 5.214022047398e+15 true resid norm > 7.772905386519e+14 ||r(i)||/||b|| 1.476344081849e-02 > 7 KSP preconditioned resid norm 6.520856882448e+15 true resid norm > 1.156631144536e+15 ||r(i)||/||b|| 2.196843342618e-02 > 8 KSP preconditioned resid norm 7.245755589579e+15 true resid norm > 1.125518734778e+15 ||r(i)||/||b|| 2.137750095326e-02 > 9 KSP preconditioned resid norm 7.402293382659e+15 true resid norm > 1.342363548018e+15 ||r(i)||/||b|| 2.549613537355e-02 > 10 KSP preconditioned resid norm 6.814780966449e+15 true resid norm > 1.040341685629e+15 ||r(i)||/||b|| 1.975969363197e-02 > 11 KSP preconditioned resid norm 4.531806636058e+15 true resid norm > 6.813426758243e+14 ||r(i)||/||b|| 1.294105842210e-02 > 12 KSP preconditioned resid norm 4.019547715129e+15 true resid norm > 1.067806433320e+15 ||r(i)||/||b|| 2.028134436224e-02 > 13 KSP preconditioned resid norm 1.578855518519e+15 true resid norm > 2.462140978868e+14 ||r(i)||/||b|| 4.676458907031e-03 > 14 KSP preconditioned resid norm 1.216192317102e+15 true resid norm > 3.547355821423e+14 ||r(i)||/||b|| 6.737657944804e-03 > 15 KSP preconditioned resid norm 1.068138625918e+15 true resid norm > 1.711364338225e+14 ||r(i)||/||b|| 3.250473905172e-03 > 16 KSP preconditioned resid norm 1.009888628733e+15 true resid norm > 1.725105447584e+14 ||r(i)||/||b|| 3.276573033454e-03 > 17 KSP preconditioned resid norm 9.012147556868e+14 true resid norm > 1.461953215911e+14 ||r(i)||/||b|| 2.776755757239e-03 > 18 KSP preconditioned resid norm 8.184497465452e+14 true resid norm > 1.394097869305e+14 ||r(i)||/||b|| 2.647874940607e-03 > 19 KSP preconditioned resid norm 6.946418330368e+14 true resid norm > 1.094539907094e+14 ||r(i)||/||b|| 2.078910566681e-03 > 20 KSP preconditioned resid norm 5.822189800664e+14 true resid norm > 1.086460879389e+14 ||r(i)||/||b|| 2.063565693502e-03 > 21 KSP preconditioned resid norm 4.765744627590e+14 true resid norm > 7.425312666441e+13 ||r(i)||/||b|| 1.410324179423e-03 > 22 KSP preconditioned resid norm 3.785099184439e+14 true resid norm > 8.229511757789e+13 ||r(i)||/||b|| 1.563069454208e-03 > 23 KSP preconditioned resid norm 3.069654165781e+14 true resid norm > 4.744586000483e+13 ||r(i)||/||b|| 9.011612922478e-04 > 0 KSP preconditioned resid norm 9.591954950201e+15 true resid norm > 5.933293318674e+15 ||r(i)||/||b|| 5.704140552403e-01 > 1 KSP preconditioned resid norm 4.627616930759e+15 true resid norm > 8.120897483902e+14 ||r(i)||/||b|| 7.807256134471e-02 > 2 KSP preconditioned resid norm 3.843607704549e+15 true resid norm > 6.216441009903e+14 ||r(i)||/||b|| 5.976352651335e-02 > 3 KSP preconditioned resid norm 3.802257051246e+15 true resid norm > 6.332758127660e+14 ||r(i)||/||b|| 6.088177425993e-02 > 4 KSP preconditioned resid norm 4.025094330467e+15 true resid norm > 6.488587775849e+14 ||r(i)||/||b|| 6.237988697367e-02 > 5 KSP preconditioned resid norm 4.518903996358e+15 true resid norm > 7.100173385586e+14 ||r(i)||/||b|| 6.825953945399e-02 > 6 KSP preconditioned resid norm 5.566041493723e+15 true resid norm > 8.272295550982e+14 ||r(i)||/||b|| 7.952806979104e-02 > 7 KSP preconditioned resid norm 7.050235529471e+15 true resid norm > 1.259531120704e+15 ||r(i)||/||b|| 1.210886122891e-01 > 8 KSP preconditioned resid norm 7.824426472314e+15 true resid norm > 1.230246429464e+15 ||r(i)||/||b|| 1.182732450740e-01 > 9 KSP preconditioned resid norm 8.110342869221e+15 true resid norm > 1.508507345577e+15 ||r(i)||/||b|| 1.450246509206e-01 > 10 KSP preconditioned resid norm 7.848071091100e+15 true resid norm > 1.244716713436e+15 ||r(i)||/||b|| 1.196643870448e-01 > 11 KSP preconditioned resid norm 6.689698623315e+15 true resid norm > 1.152050117302e+15 ||r(i)||/||b|| 1.107556198481e-01 > 12 KSP preconditioned resid norm 3.836259156428e+15 true resid norm > 5.895445738700e+14 ||r(i)||/||b|| 5.667754703239e-02 > 13 KSP preconditioned resid norm 1.583619332521e+15 true resid norm > 3.090404276347e+14 ||r(i)||/||b|| 2.971048186772e-02 > 14 KSP preconditioned resid norm 1.282447478166e+15 true resid norm > 2.666239295029e+14 ||r(i)||/||b|| 2.563265098882e-02 > 15 KSP preconditioned resid norm 1.151484666969e+15 true resid norm > 2.280596910606e+14 ||r(i)||/||b|| 2.192516806902e-02 > 16 KSP preconditioned resid norm 1.027737067851e+15 true resid norm > 1.605668418840e+14 ||r(i)||/||b|| 1.543655074795e-02 > 17 KSP preconditioned resid norm 8.747586168174e+14 true resid norm > 1.470663481317e+14 ||r(i)||/||b|| 1.413864232250e-02 > 18 KSP preconditioned resid norm 7.765674155007e+14 true resid norm > 1.265290838281e+14 ||r(i)||/||b|| 1.216423391459e-02 > 19 KSP preconditioned resid norm 6.633620818064e+14 true resid norm > 1.039325479930e+14 ||r(i)||/||b|| 9.991851571798e-03 > 20 KSP preconditioned resid norm 5.640296981466e+14 true resid norm > 1.018948133357e+14 ||r(i)||/||b|| 9.795948145668e-03 > 21 KSP preconditioned resid norm 4.751732697839e+14 true resid norm > 7.431313741469e+13 ||r(i)||/||b|| 7.144305159654e-03 > 22 KSP preconditioned resid norm 3.865078030089e+14 true resid norm > 7.686807394274e+13 ||r(i)||/||b|| 7.389931260973e-03 > 23 KSP preconditioned resid norm 3.183297004119e+14 true resid norm > 4.953673129366e+13 ||r(i)||/||b|| 4.762354777175e-03 > 24 KSP preconditioned resid norm 2.612111917568e+14 true resid norm > 5.971714747402e+13 ||r(i)||/||b|| 5.741078087414e-03 > 25 KSP preconditioned resid norm 2.296313138344e+14 true resid norm > 3.527848689641e+13 ||r(i)||/||b|| 3.391597834880e-03 > 26 KSP preconditioned resid norm 1.981337219705e+14 true resid norm > 5.164126794406e+13 ||r(i)||/||b|| 4.964680403211e-03 > 27 KSP preconditioned resid norm 1.751049134665e+14 true resid norm > 2.675806111216e+13 ||r(i)||/||b|| 2.572462430151e-03 > 28 KSP preconditioned resid norm 1.533798667157e+14 true resid norm > 3.995390488198e+13 ||r(i)||/||b|| 3.841082461690e-03 > 29 KSP preconditioned resid norm 1.350471089065e+14 true resid norm > 2.063214947275e+13 ||r(i)||/||b|| 1.983530463939e-03 > 30 KSP preconditioned resid norm 1.221880384902e+14 true resid norm > 2.772903253208e+13 ||r(i)||/||b|| 2.665809533591e-03 > 31 KSP preconditioned resid norm 1.099123503540e+14 true resid norm > 1.682484274214e+13 ||r(i)||/||b|| 1.617504185596e-03 > 32 KSP preconditioned resid norm 1.022845787224e+14 true resid norm > 1.853354426028e+13 ||r(i)||/||b|| 1.781775073585e-03 > 33 KSP preconditioned resid norm 9.416346598487e+13 true resid norm > 1.438140346990e+13 ||r(i)||/||b|| 1.382597190584e-03 > 34 KSP preconditioned resid norm 8.782252733286e+13 true resid norm > 1.465353724876e+13 ||r(i)||/||b|| 1.408759546637e-03 > 35 KSP preconditioned resid norm 8.286435025538e+13 true resid norm > 1.264742128603e+13 ||r(i)||/||b|| 1.215895873779e-03 > 36 KSP preconditioned resid norm 7.744963280138e+13 true resid norm > 1.290903292754e+13 ||r(i)||/||b|| 1.241046654183e-03 > 37 KSP preconditioned resid norm 7.340074137876e+13 true resid norm > 1.120214736846e+13 ||r(i)||/||b|| 1.076950348592e-03 > 38 KSP preconditioned resid norm 6.860608702874e+13 true resid norm > 1.142914172047e+13 ||r(i)||/||b|| 1.098773097257e-03 > 39 KSP preconditioned resid norm 6.401516623396e+13 true resid norm > 9.766911914727e+12 ||r(i)||/||b|| 9.389698997216e-04 > 40 KSP preconditioned resid norm 5.980495528832e+13 true resid norm > 9.707430083091e+12 ||r(i)||/||b|| 9.332514443927e-04 > 41 KSP preconditioned resid norm 5.502664117143e+13 true resid norm > 8.404607061129e+12 ||r(i)||/||b|| 8.080008418514e-04 > 0 KSP preconditioned resid norm 2.200733964807e+16 true resid norm > 1.374508262416e+16 ||r(i)||/||b|| 1.042223674371e-01 > 1 KSP preconditioned resid norm 1.033612993855e+16 true resid norm > 1.767505005693e+15 ||r(i)||/||b|| 1.340214251070e-02 > 2 KSP preconditioned resid norm 8.198008013445e+15 true resid norm > 1.328272543525e+15 ||r(i)||/||b|| 1.007165346861e-02 > 3 KSP preconditioned resid norm 7.723889011566e+15 true resid norm > 1.289831076731e+15 ||r(i)||/||b|| 9.780170267923e-03 > 4 KSP preconditioned resid norm 7.785469249486e+15 true resid norm > 1.238697760343e+15 ||r(i)||/||b|| 9.392450860579e-03 > 5 KSP preconditioned resid norm 8.250177532689e+15 true resid norm > 1.288999707195e+15 ||r(i)||/||b|| 9.773866391570e-03 > 6 KSP preconditioned resid norm 9.471871112821e+15 true resid norm > 1.409248720218e+15 ||r(i)||/||b|| 1.068565696875e-02 > 7 KSP preconditioned resid norm 1.106232045691e+16 true resid norm > 1.993194297637e+15 ||r(i)||/||b|| 1.511343613874e-02 > 8 KSP preconditioned resid norm 1.131471113452e+16 true resid norm > 1.768387012694e+15 ||r(i)||/||b|| 1.340883034665e-02 > 9 KSP preconditioned resid norm 1.055003108779e+16 true resid norm > 1.935924228946e+15 ||r(i)||/||b|| 1.467918468275e-02 > 10 KSP preconditioned resid norm 8.569038225337e+15 true resid norm > 1.316932619063e+15 ||r(i)||/||b|| 9.985668261662e-03 > 11 KSP preconditioned resid norm 4.806783660704e+15 true resid norm > 7.346602562794e+14 ||r(i)||/||b|| 5.570576275535e-03 > 12 KSP preconditioned resid norm 6.311127886872e+15 true resid norm > 1.031465284608e+15 ||r(i)||/||b|| 7.821106415332e-03 > 13 KSP preconditioned resid norm 9.706706695861e+14 true resid norm > 2.189061895566e+14 ||r(i)||/||b|| 1.659860616781e-03 > 14 KSP preconditioned resid norm 6.844878378158e+14 true resid norm > 1.339168617782e+14 ||r(i)||/||b|| 1.015427317239e-03 > 0 KSP preconditioned resid norm 1.928571496492e+16 true resid norm > 1.203387628158e+16 ||r(i)||/||b|| 4.946666172277e-01 > 1 KSP preconditioned resid norm 9.232456268915e+15 true resid norm > 1.583223656595e+15 ||r(i)||/||b|| 6.508026775390e-02 > 2 KSP preconditioned resid norm 7.541947576336e+15 true resid norm > 1.216650042172e+15 ||r(i)||/||b|| 5.001182882627e-02 > 3 KSP preconditioned resid norm 7.340367763150e+15 true resid norm > 1.219887358936e+15 ||r(i)||/||b|| 5.014490253381e-02 > 4 KSP preconditioned resid norm 7.655525114233e+15 true resid norm > 1.212298323935e+15 ||r(i)||/||b|| 4.983294633744e-02 > 5 KSP preconditioned resid norm 8.414914513850e+15 true resid norm > 1.311466433986e+15 ||r(i)||/||b|| 5.390936796484e-02 > 6 KSP preconditioned resid norm 1.008643582787e+16 true resid norm > 1.495807999597e+15 ||r(i)||/||b|| 6.148694451139e-02 > 7 KSP preconditioned resid norm 1.235608743666e+16 true resid norm > 2.203234082057e+15 ||r(i)||/||b|| 9.056652443733e-02 > 8 KSP preconditioned resid norm 1.311751365507e+16 true resid norm > 2.061480991281e+15 ||r(i)||/||b|| 8.473959716509e-02 > 9 KSP preconditioned resid norm 1.289621791383e+16 true resid norm > 2.355792563000e+15 ||r(i)||/||b|| 9.683761996231e-02 > 10 KSP preconditioned resid norm 1.151686987354e+16 true resid norm > 1.821530468335e+15 ||r(i)||/||b|| 7.487614911976e-02 > 11 KSP preconditioned resid norm 8.379707704664e+15 true resid norm > 1.435040037498e+15 ||r(i)||/||b|| 5.898900606299e-02 > 12 KSP preconditioned resid norm 3.557464484347e+15 true resid norm > 5.430083136124e+14 ||r(i)||/||b|| 2.232099444401e-02 > 13 KSP preconditioned resid norm 1.204545626126e+15 true resid norm > 2.194281091480e+14 ||r(i)||/||b|| 9.019850124522e-03 > 14 KSP preconditioned resid norm 1.005673666245e+15 true resid norm > 1.883222474780e+14 ||r(i)||/||b|| 7.741207149622e-03 > 15 KSP preconditioned resid norm 9.379382960505e+14 true resid norm > 1.480571711312e+14 ||r(i)||/||b|| 6.086063898783e-03 > 16 KSP preconditioned resid norm 8.825746357103e+14 true resid norm > 2.025244331207e+14 ||r(i)||/||b|| 8.325004669617e-03 > 17 KSP preconditioned resid norm 7.524485997458e+14 true resid norm > 1.181787653170e+14 ||r(i)||/||b|| 4.857876938371e-03 > 18 KSP preconditioned resid norm 6.768878630597e+14 true resid norm > 1.383948785570e+14 ||r(i)||/||b|| 5.688884015051e-03 > 19 KSP preconditioned resid norm 6.018014871695e+14 true resid norm > 9.381957375927e+13 ||r(i)||/||b|| 3.856563761775e-03 > 20 KSP preconditioned resid norm 5.246326908653e+14 true resid norm > 1.170575496120e+14 ||r(i)||/||b|| 4.811788050053e-03 > 21 KSP preconditioned resid norm 4.606732721936e+14 true resid norm > 7.159556750064e+13 ||r(i)||/||b|| 2.943019884477e-03 > 22 KSP preconditioned resid norm 3.919505243136e+14 true resid norm > 9.177422576855e+13 ||r(i)||/||b|| 3.772487330545e-03 > 23 KSP preconditioned resid norm 3.382362447667e+14 true resid norm > 5.210255290014e+13 ||r(i)||/||b|| 2.141736626583e-03 > 24 KSP preconditioned resid norm 2.828412329842e+14 true resid norm > 7.512251290427e+13 ||r(i)||/||b|| 3.087999117364e-03 > 25 KSP preconditioned resid norm 2.453088437412e+14 true resid norm > 3.770320821115e+13 ||r(i)||/||b|| 1.549834652446e-03 > 26 KSP preconditioned resid norm 2.155681340535e+14 true resid norm > 5.419527743350e+13 ||r(i)||/||b|| 2.227760526238e-03 > 27 KSP preconditioned resid norm 1.897133167426e+14 true resid norm > 2.950727994670e+13 ||r(i)||/||b|| 1.212931395777e-03 > 28 KSP preconditioned resid norm 1.747505144272e+14 true resid norm > 2.784171913979e+13 ||r(i)||/||b|| 1.144466562762e-03 > 29 KSP preconditioned resid norm 1.591031918648e+14 true resid norm > 2.517709500033e+13 ||r(i)||/||b|| 1.034934058155e-03 > 30 KSP preconditioned resid norm 1.457115463937e+14 true resid norm > 2.260424017892e+13 ||r(i)||/||b|| 9.291738391412e-04 > 31 KSP preconditioned resid norm 1.296458013341e+14 true resid norm > 2.025545529149e+13 ||r(i)||/||b|| 8.326242779131e-04 > 0 KSP preconditioned resid norm 6.130010037612e+16 true resid norm > 4.019535297216e+16 ||r(i)||/||b|| 5.748505993058e-02 > 1 KSP preconditioned resid norm 2.383231272358e+16 true resid norm > 4.333028202730e+15 ||r(i)||/||b|| 6.196845343971e-03 > 2 KSP preconditioned resid norm 1.380314308485e+16 true resid norm > 2.902938638700e+15 ||r(i)||/||b|| 4.151614285761e-03 > 3 KSP preconditioned resid norm 9.355422664357e+15 true resid norm > 1.914374541766e+15 ||r(i)||/||b|| 2.737827313999e-03 > 4 KSP preconditioned resid norm 6.905389450100e+15 true resid norm > 1.160495113546e+15 ||r(i)||/||b|| 1.659672728773e-03 > 5 KSP preconditioned resid norm 5.388850197186e+15 true resid norm > 9.343171629150e+14 ||r(i)||/||b|| 1.336206156506e-03 > 6 KSP preconditioned resid norm 4.456941607624e+15 true resid norm > 7.443205303349e+14 ||r(i)||/||b|| 1.064484004494e-03 > 7 KSP preconditioned resid norm 3.604897946036e+15 true resid norm > 7.177354673946e+14 ||r(i)||/||b|| 1.026463591104e-03 > 8 KSP preconditioned resid norm 2.616552593607e+15 true resid norm > 4.566784798486e+14 ||r(i)||/||b|| 6.531150454453e-04 > 9 KSP preconditioned resid norm 1.730753896325e+15 true resid norm > 3.416577992536e+14 ||r(i)||/||b|| 4.886191465826e-04 > 10 KSP preconditioned resid norm 9.605913237294e+14 true resid norm > 1.715828091570e+14 ||r(i)||/||b|| 2.453877709266e-04 > 11 KSP preconditioned resid norm 3.885469473062e+14 true resid norm > 7.526615943221e+13 ||r(i)||/||b|| 1.076412909896e-04 > 12 KSP preconditioned resid norm 1.085402136107e+14 true resid norm > 1.877490962768e+13 ||r(i)||/||b|| 2.685078560912e-05 > 13 KSP preconditioned resid norm 5.521535103061e+13 true resid norm > 1.482831672696e+13 ||r(i)||/||b|| 2.120659759623e-05 > 14 KSP preconditioned resid norm 4.327778995114e+13 true resid norm > 6.949075963503e+12 ||r(i)||/||b|| 9.938164954062e-06 > 0 KSP preconditioned resid norm 2.469229041475e+16 true resid norm > 1.581397714280e+16 ||r(i)||/||b|| 1.421322438999e-01 > 1 KSP preconditioned resid norm 1.094866313351e+16 true resid norm > 1.827538507817e+15 ||r(i)||/||b|| 1.642547896610e-02 > 2 KSP preconditioned resid norm 7.902647446500e+15 true resid norm > 1.331972422409e+15 ||r(i)||/||b|| 1.197144952849e-02 > 3 KSP preconditioned resid norm 6.757114899989e+15 true resid norm > 1.177404752006e+15 ||r(i)||/||b|| 1.058223227906e-02 > 4 KSP preconditioned resid norm 6.236820681061e+15 true resid norm > 9.858934558297e+14 ||r(i)||/||b|| 8.860974558006e-03 > 5 KSP preconditioned resid norm 6.056208230839e+15 true resid norm > 9.498735328449e+14 ||r(i)||/||b|| 8.537236106085e-03 > 6 KSP preconditioned resid norm 6.281150442401e+15 true resid norm > 9.439017368432e+14 ||r(i)||/||b|| 8.483563032058e-03 > 7 KSP preconditioned resid norm 6.511637444792e+15 true resid norm > 1.175605282966e+15 ||r(i)||/||b|| 1.056605908176e-02 > 8 KSP preconditioned resid norm 5.897282453446e+15 true resid norm > 9.358326231383e+14 ||r(i)||/||b|| 8.411039768188e-03 > 9 KSP preconditioned resid norm 4.858643646559e+15 true resid norm > 9.074077132105e+14 ||r(i)||/||b|| 8.155563476918e-03 > 10 KSP preconditioned resid norm 3.431227043388e+15 true resid norm > 5.523077341519e+14 ||r(i)||/||b|| 4.964009804073e-03 > 11 KSP preconditioned resid norm 1.790602151882e+15 true resid norm > 3.233768234561e+14 ||r(i)||/||b|| 2.906433538380e-03 > 12 KSP preconditioned resid norm 6.061523925825e+14 true resid norm > 9.361733195934e+13 ||r(i)||/||b|| 8.414101866433e-04 > 13 KSP preconditioned resid norm 3.669173256365e+14 true resid norm > 7.433445448820e+13 ||r(i)||/||b|| 6.681002963438e-04 > 14 KSP preconditioned resid norm 1.034315631221e+15 true resid norm > 1.749781208578e+14 ||r(i)||/||b|| 1.572661495987e-03 > 15 KSP preconditioned resid norm 1.340579302998e+14 true resid norm > 2.477734238014e+13 ||r(i)||/||b|| 2.226928266408e-04 > 16 KSP preconditioned resid norm 1.124182190968e+14 true resid norm > 1.917839484668e+13 ||r(i)||/||b|| 1.723708254629e-04 > 17 KSP preconditioned resid norm 9.394060483803e+13 true resid norm > 1.682558933028e+13 ||r(i)||/||b|| 1.512243722661e-04 > 18 KSP preconditioned resid norm 9.034672550576e+13 true resid norm > 1.892870471110e+13 ||r(i)||/||b|| 1.701266702495e-04 > 19 KSP preconditioned resid norm 8.058964108831e+13 true resid norm > 1.275370569496e+13 ||r(i)||/||b|| 1.146272561351e-04 > 20 KSP preconditioned resid norm 6.303390314758e+13 true resid norm > 1.900069985546e+13 ||r(i)||/||b|| 1.707737453860e-04 > 21 KSP preconditioned resid norm 4.601826493253e+13 true resid norm > 6.960266318594e+12 ||r(i)||/||b|| 6.255720879505e-05 > 22 KSP preconditioned resid norm 2.822787946131e+13 true resid norm > 1.013300639920e+13 ||r(i)||/||b|| 9.107303772884e-05 > 23 KSP preconditioned resid norm 1.858182164082e+13 true resid norm > 2.785683110677e+12 ||r(i)||/||b|| 2.503705347106e-05 > 24 KSP preconditioned resid norm 9.506925688468e+12 true resid norm > 4.268960894748e+12 ||r(i)||/||b|| 3.836839939834e-05 > 25 KSP preconditioned resid norm 5.845171242291e+12 true resid norm > 8.923161711832e+11 ||r(i)||/||b|| 8.019924307031e-06 > 0 SNES Function norm 0.000000000000e+00 > 5 TS dt 7e-12 time 2e-11 > TS Object: 1 MPI processes > type: theta > maximum steps=10 > maximum time=2e-11 > total number of nonlinear solver iterations=0 > total number of nonlinear solve failures=0 > total number of linear solver iterations=0 > total number of rejected steps=5 > Theta=1 > Extrapolation=no > SNES Object: 1 MPI processes > type: newtonls > maximum iterations=50, maximum function evaluations=10000 > tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 > total number of linear solver iterations=0 > total number of function evaluations=1 > SNESLineSearch Object: 1 MPI processes > type: bt > interpolation: cubic > alpha=1.000000e-04 > maxstep=1.000000e+08, minlambda=1.000000e-12 > tolerances: relative=1.000000e-08, absolute=1.000000e-15, > lambda=1.000000e-08 > maximum iterations=40 > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using DEFAULT norm type for convergence test > PC Object: 1 MPI processes > type: ilu > PC has not been set up so information may be incomplete > ILU: out-of-place factorization > 0 levels of fill > tolerance for zero pivot 2.22045e-14 > using diagonal shift on blocks to prevent zero pivot [INBLOCKS] > matrix ordering: natural > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: seqaij > rows=20904, cols=20904, bs=4 > total: nonzeros=410816, allocated nonzeros=410816 > total number of mallocs used during MatSetValues calls =0 > using I-node routines: found 5226 nodes, limit used is 5 > > On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley > wrote: > >> On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone wrote: >> >>> Hi Jed and Matt, >>> >>> Thank you very much for your replies. >>> >>> This time I copied the whole output from my screen, from compile to >>> execution. This time I put the TSView() routine just after the TSSolve() >>> routine to make the tsview output. And I intentionally to do only 5 time >>> steps to make the test. Thank you in advance! >>> >> >> You have to make the command line arguments work. Quit calling TSView() >> in the code. We need many more options >> for debugging. First track down why you are not processing options. Do >> the examples take options for you? >> >> Once that works, run with -ts_view -ts_monitor -snes_monitor >> -ksp_monitor_true_residual >> >> Matt >> >> >>> host206-47:streamer_Implicit_SG GPL$ make pStreamer >>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o >>> pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>> -Wno-unknown-pragmas -g -O0 -fPIC >>> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >>> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >>> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>> -Wl,-commons,use_dylibs -Wl,-search_paths_first >>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >>> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >>> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >>> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >>> -lpthread -lssl -lcrypto >>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >>> -lmpichf90 -lgfortran >>> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >>> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >>> -lclang_rt.osx -lmpichcxx -lc++ >>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>> -lclang_rt.osx >>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >>> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>> -lclang_rt.osx -ldl >>> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >>> >>> time=0.000000 step=0 >>> dt = 0.001000 >>> >>> time=0.002000 step=1 >>> dt = 0.002000 >>> >>> time=0.005000 step=2 >>> dt = 0.003000 >>> >>> time=0.009000 step=3 >>> dt = 0.004000 >>> >>> time=0.014000 step=4 >>> dt = 0.005000 >>> >>> time=0.020000 step=5 >>> dt = 0.006000 >>> >>> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >>> TS Object: 1 MPI processes >>> type: theta >>> maximum steps=10 >>> maximum time=2e-11 >>> total number of nonlinear solver iterations=0 >>> total number of nonlinear solve failures=0 >>> total number of linear solver iterations=0 >>> total number of rejected steps=5 >>> Theta=1 >>> Extrapolation=no >>> SNES Object: 1 MPI processes >>> type: newtonls >>> maximum iterations=50, maximum function evaluations=10000 >>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>> total number of linear solver iterations=0 >>> total number of function evaluations=1 >>> SNESLineSearch Object: 1 MPI processes >>> type: bt >>> interpolation: cubic >>> alpha=1.000000e-04 >>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>> lambda=1.000000e-08 >>> maximum iterations=40 >>> KSP Object: 1 MPI processes >>> type: gmres >>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>> Orthogonalization with no iterative refinement >>> GMRES: happy breakdown tolerance 1e-30 >>> maximum iterations=10000, initial guess is zero >>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using DEFAULT norm type for convergence test >>> PC Object: 1 MPI processes >>> type: ilu >>> PC has not been set up so information may be incomplete >>> ILU: out-of-place factorization >>> 0 levels of fill >>> tolerance for zero pivot 2.22045e-14 >>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>> matrix ordering: natural >>> linear system matrix = precond matrix: >>> Mat Object: 1 MPI processes >>> type: seqaij >>> rows=20904, cols=20904, bs=4 >>> total: nonzeros=410816, allocated nonzeros=410816 >>> total number of mallocs used during MatSetValues calls =0 >>> using I-node routines: found 5226 nodes, limit used is 5 >>> >>> >>> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >>> >>>> Sharp Stone writes: >>>> >>>> > Hi Matt, >>>> > >>>> > Thank you very much for your reply. >>>> > >>>> > The ts_view output is attached below. I found my code results say >>>> "SNES has >>>> > not been set up", and PC "has not been set up". Does this cause the >>>> > problem? If so I do not see example ex17 explicitly set up the snes >>>> object? >>>> >>>> Those objects are set up on the first time step (inside TSSolve). >>>> >>>> > PS: I don't know why the -ts_view option does not work in my command >>>> line, >>>> >>>> Are you sure it doesn't output after the TSSolve? The example does. >>>> >>>> > so I use TSView() routine just before the TSSolve() routine. >>>> >>>> You should call it *after* the solve so we can see what happened during >>>> the solve. >>>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From thronesf at gmail.com Mon Oct 27 14:58:19 2014 From: thronesf at gmail.com (Sharp Stone) Date: Mon, 27 Oct 2014 15:58:19 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: Hi Matt, That is why we asked this. We do not think the calculation of DAE F-functions are wrong. But even if the F-functions are not accurate, we expect the TS solver would still call TSSetIJacobian(), but actually it does not call TSSetIJacobian() routine at all... Thank you in advance! On Mon, Oct 27, 2014 at 3:39 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 2:31 PM, Sharp Stone wrote: > >> Hi Matt, >> >> Thank you for your reply and suggestion. I use the command line options >> to make the output on the screen. It was very long and attached below. >> Thanks a million in advance! >> > > Notice that you never actually solve anything. There is something wrong > with your formulation. Try something where you > know the exact solution first. Also run with > > -snes_converged_reason -ksp_converged_reason > > Matt > > >> >> >> 0 KSP preconditioned resid norm 5.607581107219e-11 true resid norm >> 9.787719465322e-12 ||r(i)||/||b|| 6.796750851600e-16 >> 0 TS dt 2e-12 time 0 >> 0 KSP preconditioned resid norm 3.622586114352e+17 true resid norm >> 5.225681998280e+16 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 3.704153274287e+17 true resid norm >> 1.035782762954e+17 ||r(i)||/||b|| 1.982100639294e+00 >> 2 KSP preconditioned resid norm 2.842418683024e+17 true resid norm >> 7.903974218109e+16 ||r(i)||/||b|| 1.512524914587e+00 >> 3 KSP preconditioned resid norm 2.393970839759e+17 true resid norm >> 4.943510895366e+16 ||r(i)||/||b|| 9.460030091754e-01 >> 4 KSP preconditioned resid norm 2.271787411557e+17 true resid norm >> 4.669216610635e+16 ||r(i)||/||b|| 8.935133466162e-01 >> 5 KSP preconditioned resid norm 2.360179849286e+17 true resid norm >> 4.108243474851e+16 ||r(i)||/||b|| 7.861640789094e-01 >> 6 KSP preconditioned resid norm 2.619090922116e+17 true resid norm >> 4.955060344596e+16 ||r(i)||/||b|| 9.482131416009e-01 >> 7 KSP preconditioned resid norm 2.333689240438e+17 true resid norm >> 4.336784470641e+16 ||r(i)||/||b|| 8.298982739609e-01 >> 8 KSP preconditioned resid norm 3.244607818183e+17 true resid norm >> 5.407474341938e+16 ||r(i)||/||b|| 1.034788252274e+00 >> 9 KSP preconditioned resid norm 3.024748487424e+17 true resid norm >> 6.388129145873e+16 ||r(i)||/||b|| 1.222448887624e+00 >> 10 KSP preconditioned resid norm 2.127860348138e+17 true resid norm >> 3.704786520226e+16 ||r(i)||/||b|| 7.089575143389e-01 >> 11 KSP preconditioned resid norm 4.652609350461e+16 true resid norm >> 8.497318285406e+15 ||r(i)||/||b|| 1.626068767331e-01 >> 12 KSP preconditioned resid norm 2.920260091494e+16 true resid norm >> 4.651858168992e+15 ||r(i)||/||b|| 8.901915904036e-02 >> 13 KSP preconditioned resid norm 2.527529714994e+16 true resid norm >> 3.813962069025e+15 ||r(i)||/||b|| 7.298496292505e-02 >> 14 KSP preconditioned resid norm 2.224122069397e+16 true resid norm >> 4.355442499437e+15 ||r(i)||/||b|| 8.334687225268e-02 >> 15 KSP preconditioned resid norm 2.040909225547e+16 true resid norm >> 5.536568280983e+15 ||r(i)||/||b|| 1.059492001772e-01 >> 16 KSP preconditioned resid norm 1.834652616545e+16 true resid norm >> 3.131287565910e+15 ||r(i)||/||b|| 5.992112736559e-02 >> 17 KSP preconditioned resid norm 1.760978175265e+16 true resid norm >> 3.245011634605e+15 ||r(i)||/||b|| 6.209738050790e-02 >> 18 KSP preconditioned resid norm 1.385338263500e+16 true resid norm >> 3.596318213647e+15 ||r(i)||/||b|| 6.882007391247e-02 >> 19 KSP preconditioned resid norm 1.488783377120e+16 true resid norm >> 2.508274919240e+15 ||r(i)||/||b|| 4.799899649588e-02 >> 20 KSP preconditioned resid norm 1.320707251811e+16 true resid norm >> 2.042806338080e+15 ||r(i)||/||b|| 3.909166954193e-02 >> 21 KSP preconditioned resid norm 7.616453043037e+15 true resid norm >> 1.348500002677e+15 ||r(i)||/||b|| 2.580524423647e-02 >> 22 KSP preconditioned resid norm 8.879333117289e+15 true resid norm >> 1.392661876801e+15 ||r(i)||/||b|| 2.665033726238e-02 >> 23 KSP preconditioned resid norm 8.820610261864e+15 true resid norm >> 1.365093600480e+15 ||r(i)||/||b|| 2.612278360851e-02 >> 24 KSP preconditioned resid norm 7.993346710634e+15 true resid norm >> 1.790642691323e+15 ||r(i)||/||b|| 3.426620088847e-02 >> 25 KSP preconditioned resid norm 7.525623400757e+15 true resid norm >> 1.148549131012e+15 ||r(i)||/||b|| 2.197893272859e-02 >> 26 KSP preconditioned resid norm 7.109494224141e+15 true resid norm >> 1.253098862904e+15 ||r(i)||/||b|| 2.397962339302e-02 >> 27 KSP preconditioned resid norm 9.199926655972e+15 true resid norm >> 1.867276980934e+15 ||r(i)||/||b|| 3.573269444158e-02 >> 28 KSP preconditioned resid norm 1.121259577312e+16 true resid norm >> 3.202544820818e+15 ||r(i)||/||b|| 6.128472459426e-02 >> 29 KSP preconditioned resid norm 4.340241766754e+15 true resid norm >> 6.879443404198e+14 ||r(i)||/||b|| 1.316468052680e-02 >> 30 KSP preconditioned resid norm 4.050053526588e+15 true resid norm >> 6.673524276162e+14 ||r(i)||/||b|| 1.277062836652e-02 >> 31 KSP preconditioned resid norm 3.675002682668e+15 true resid norm >> 5.703629309888e+14 ||r(i)||/||b|| 1.091461231618e-02 >> 32 KSP preconditioned resid norm 3.175854297440e+15 true resid norm >> 4.918363244864e+14 ||r(i)||/||b|| 9.411906898435e-03 >> 33 KSP preconditioned resid norm 2.534108954658e+15 true resid norm >> 3.888660709280e+14 ||r(i)||/||b|| 7.441441539229e-03 >> 34 KSP preconditioned resid norm 1.517593193539e+15 true resid norm >> 2.347716358056e+14 ||r(i)||/||b|| 4.492650641255e-03 >> 35 KSP preconditioned resid norm 1.238737221149e+15 true resid norm >> 4.944967490035e+14 ||r(i)||/||b|| 9.462817468922e-03 >> 36 KSP preconditioned resid norm 8.673662184926e+14 true resid norm >> 1.322920275286e+14 ||r(i)||/||b|| 2.531574396837e-03 >> 37 KSP preconditioned resid norm 7.206551748402e+14 true resid norm >> 1.263301396525e+14 ||r(i)||/||b|| 2.417486171069e-03 >> 38 KSP preconditioned resid norm 5.143409845488e+14 true resid norm >> 7.953540049469e+13 ||r(i)||/||b|| 1.522009960056e-03 >> 39 KSP preconditioned resid norm 4.261910201431e+14 true resid norm >> 7.194412501613e+13 ||r(i)||/||b|| 1.376741352417e-03 >> 40 KSP preconditioned resid norm 4.660682952219e+14 true resid norm >> 7.611539723637e+13 ||r(i)||/||b|| 1.456563894654e-03 >> 41 KSP preconditioned resid norm 3.209169783039e+14 true resid norm >> 4.903825175831e+13 ||r(i)||/||b|| 9.384086474158e-04 >> 0 KSP preconditioned resid norm 5.647841057638e+16 true resid norm >> 8.145493103696e+15 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 5.871590984109e+16 true resid norm >> 1.629681152051e+16 ||r(i)||/||b|| 2.000715157823e+00 >> 2 KSP preconditioned resid norm 4.554798588549e+16 true resid norm >> 1.256325647853e+16 ||r(i)||/||b|| 1.542356775532e+00 >> 3 KSP preconditioned resid norm 3.879559549735e+16 true resid norm >> 7.930640596410e+15 ||r(i)||/||b|| 9.736231429392e-01 >> 4 KSP preconditioned resid norm 3.752982759961e+16 true resid norm >> 7.738765648717e+15 ||r(i)||/||b|| 9.500671782787e-01 >> 5 KSP preconditioned resid norm 3.847367502586e+16 true resid norm >> 6.717373650268e+15 ||r(i)||/||b|| 8.246736649031e-01 >> 6 KSP preconditioned resid norm 4.072409022879e+16 true resid norm >> 7.643006165053e+15 ||r(i)||/||b|| 9.383110473184e-01 >> 7 KSP preconditioned resid norm 3.699615223300e+16 true resid norm >> 6.864744274739e+15 ||r(i)||/||b|| 8.427659550315e-01 >> 8 KSP preconditioned resid norm 4.452919179646e+16 true resid norm >> 7.438964756180e+15 ||r(i)||/||b|| 9.132614393603e-01 >> 9 KSP preconditioned resid norm 4.218921641123e+16 true resid norm >> 8.817640722426e+15 ||r(i)||/||b|| 1.082517732220e+00 >> 10 KSP preconditioned resid norm 2.723066166891e+18 true resid norm >> 4.777495288747e+17 ||r(i)||/||b|| 5.865200826920e+01 >> 11 KSP preconditioned resid norm 1.543090703715e+16 true resid norm >> 2.978277930467e+15 ||r(i)||/||b|| 3.656350686879e-01 >> 12 KSP preconditioned resid norm 5.543831189057e+15 true resid norm >> 9.036431670746e+14 ||r(i)||/||b|| 1.109378100958e-01 >> 13 KSP preconditioned resid norm 4.835578439031e+15 true resid norm >> 7.302084941558e+14 ||r(i)||/||b|| 8.964570773800e-02 >> 14 KSP preconditioned resid norm 4.632268883924e+15 true resid norm >> 7.764942849443e+14 ||r(i)||/||b|| 9.532808819050e-02 >> 15 KSP preconditioned resid norm 4.279662320804e+15 true resid norm >> 7.279053691611e+14 ||r(i)||/||b|| 8.936295935612e-02 >> 16 KSP preconditioned resid norm 4.003909782590e+15 true resid norm >> 6.398763789786e+14 ||r(i)||/||b|| 7.855588002257e-02 >> 17 KSP preconditioned resid norm 3.778065018292e+15 true resid norm >> 9.619695351776e+14 ||r(i)||/||b|| 1.180983794267e-01 >> 18 KSP preconditioned resid norm 4.432967840881e+15 true resid norm >> 1.765362859847e+15 ||r(i)||/||b|| 2.167287894512e-01 >> 19 KSP preconditioned resid norm 3.349713073090e+15 true resid norm >> 5.343310168427e+14 ||r(i)||/||b|| 6.559836341894e-02 >> 20 KSP preconditioned resid norm 3.170973083244e+15 true resid norm >> 4.905381423867e+14 ||r(i)||/||b|| 6.022203151385e-02 >> 21 KSP preconditioned resid norm 7.415620721710e+15 true resid norm >> 1.207250421775e+15 ||r(i)||/||b|| 1.482108457286e-01 >> 22 KSP preconditioned resid norm 1.972990201442e+15 true resid norm >> 3.077655556372e+14 ||r(i)||/||b|| 3.778353891154e-02 >> 23 KSP preconditioned resid norm 2.209676808077e+15 true resid norm >> 3.428403970507e+14 ||r(i)||/||b|| 4.208958164793e-02 >> 24 KSP preconditioned resid norm 2.003292171306e+15 true resid norm >> 4.370066270665e+14 ||r(i)||/||b|| 5.365011319796e-02 >> 25 KSP preconditioned resid norm 1.896876965875e+15 true resid norm >> 2.911542588080e+14 ||r(i)||/||b|| 3.574421524903e-02 >> 26 KSP preconditioned resid norm 1.776666236555e+15 true resid norm >> 2.752957912495e+14 ||r(i)||/||b|| 3.379731438537e-02 >> 27 KSP preconditioned resid norm 2.106240629900e+15 true resid norm >> 3.329282684307e+14 ||r(i)||/||b|| 4.087269661791e-02 >> 28 KSP preconditioned resid norm 1.993718847257e+15 true resid norm >> 3.265440954005e+14 ||r(i)||/||b|| 4.008892908550e-02 >> 29 KSP preconditioned resid norm 1.305019272390e+15 true resid norm >> 2.016059917794e+14 ||r(i)||/||b|| 2.475061843560e-02 >> 30 KSP preconditioned resid norm 1.188165021558e+15 true resid norm >> 2.349019593212e+14 ||r(i)||/||b|| 2.883827367242e-02 >> 31 KSP preconditioned resid norm 1.072198412006e+15 true resid norm >> 1.660526922017e+14 ||r(i)||/||b|| 2.038583669371e-02 >> 32 KSP preconditioned resid norm 9.531727749289e+14 true resid norm >> 1.481127635008e+14 ||r(i)||/||b|| 1.818340051551e-02 >> 33 KSP preconditioned resid norm 8.113851256730e+14 true resid norm >> 1.246949236714e+14 ||r(i)||/||b|| 1.530845610990e-02 >> 34 KSP preconditioned resid norm 1.978771889204e+15 true resid norm >> 3.123099443158e+14 ||r(i)||/||b|| 3.834144113069e-02 >> 35 KSP preconditioned resid norm 5.701229171299e+14 true resid norm >> 9.665965877556e+13 ||r(i)||/||b|| 1.186664300676e-02 >> 36 KSP preconditioned resid norm 4.923209980376e+14 true resid norm >> 8.358449899120e+13 ||r(i)||/||b|| 1.026144125679e-02 >> 37 KSP preconditioned resid norm 4.337271696165e+14 true resid norm >> 7.056948273999e+13 ||r(i)||/||b|| 8.663623164566e-03 >> 38 KSP preconditioned resid norm 3.355561922811e+14 true resid norm >> 5.206223795577e+13 ||r(i)||/||b|| 6.391539136181e-03 >> 39 KSP preconditioned resid norm 2.948796430248e+14 true resid norm >> 4.783720648479e+13 ||r(i)||/||b|| 5.872843531485e-03 >> 40 KSP preconditioned resid norm 2.903844355011e+14 true resid norm >> 4.553096332971e+13 ||r(i)||/||b|| 5.589712341546e-03 >> 41 KSP preconditioned resid norm 2.470740134237e+14 true resid norm >> 3.845088511805e+13 ||r(i)||/||b|| 4.720510425650e-03 >> 42 KSP preconditioned resid norm 2.144333147632e+14 true resid norm >> 4.199300915233e+13 ||r(i)||/||b|| 5.155367344584e-03 >> 43 KSP preconditioned resid norm 1.666875797521e+14 true resid norm >> 2.786670988964e+13 ||r(i)||/||b|| 3.421120064173e-03 >> 44 KSP preconditioned resid norm 1.097596670806e+14 true resid norm >> 1.683182082477e+13 ||r(i)||/||b|| 2.066396792741e-03 >> 45 KSP preconditioned resid norm 1.086486049833e+14 true resid norm >> 1.868163126673e+13 ||r(i)||/||b|| 2.293492981813e-03 >> 46 KSP preconditioned resid norm 9.986155841385e+13 true resid norm >> 1.544797785039e+13 ||r(i)||/||b|| 1.896506160367e-03 >> 47 KSP preconditioned resid norm 9.193832565781e+13 true resid norm >> 1.424417809082e+13 ||r(i)||/||b|| 1.748718943038e-03 >> 48 KSP preconditioned resid norm 7.868838175665e+13 true resid norm >> 1.234477959732e+13 ||r(i)||/||b|| 1.515534963957e-03 >> 49 KSP preconditioned resid norm 6.573261298706e+13 true resid norm >> 1.037919549254e+13 ||r(i)||/||b|| 1.274225557668e-03 >> 50 KSP preconditioned resid norm 6.394833480259e+13 true resid norm >> 1.032066032237e+13 ||r(i)||/||b|| 1.267039354277e-03 >> 51 KSP preconditioned resid norm 6.650072664422e+13 true resid norm >> 2.391163155719e+13 ||r(i)||/||b|| 2.935565870940e-03 >> 52 KSP preconditioned resid norm 6.239223696051e+13 true resid norm >> 9.858797480463e+12 ||r(i)||/||b|| 1.210337711291e-03 >> 53 KSP preconditioned resid norm 6.247432830217e+13 true resid norm >> 9.562224327829e+12 ||r(i)||/||b|| 1.173928233208e-03 >> 54 KSP preconditioned resid norm 6.218810094746e+13 true resid norm >> 9.709963371483e+12 ||r(i)||/||b|| 1.192065753156e-03 >> 55 KSP preconditioned resid norm 6.285033387840e+13 true resid norm >> 9.623710764220e+12 ||r(i)||/||b|| 1.181476755515e-03 >> 56 KSP preconditioned resid norm 5.407058856328e+13 true resid norm >> 9.490503427247e+12 ||r(i)||/||b|| 1.165123253611e-03 >> 0 KSP preconditioned resid norm 9.075431914728e+17 true resid norm >> 1.310345788054e+17 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 8.641655528143e+17 true resid norm >> 2.471138002109e+17 ||r(i)||/||b|| 1.885867092974e+00 >> 2 KSP preconditioned resid norm 6.384433252903e+17 true resid norm >> 1.816534419632e+17 ||r(i)||/||b|| 1.386301567259e+00 >> 3 KSP preconditioned resid norm 5.179227150533e+17 true resid norm >> 1.107056703740e+17 ||r(i)||/||b|| 8.448584441089e-01 >> 4 KSP preconditioned resid norm 4.625114083211e+17 true resid norm >> 9.438923102711e+16 ||r(i)||/||b|| 7.203383403647e-01 >> 5 KSP preconditioned resid norm 4.977065165555e+17 true resid norm >> 8.614996614554e+16 ||r(i)||/||b|| 6.574597860420e-01 >> 6 KSP preconditioned resid norm 6.630931668382e+17 true resid norm >> 1.310543864565e+17 ||r(i)||/||b|| 1.000151163542e+00 >> 7 KSP preconditioned resid norm 5.452278853431e+17 true resid norm >> 1.019521297074e+17 ||r(i)||/||b|| 7.780551564087e-01 >> 8 KSP preconditioned resid norm 1.958476842674e+18 true resid norm >> 3.244825770314e+17 ||r(i)||/||b|| 2.476312588551e+00 >> 9 KSP preconditioned resid norm 1.577830260132e+18 true resid norm >> 3.469004540765e+17 ||r(i)||/||b|| 2.647396261651e+00 >> 10 KSP preconditioned resid norm 1.081897998863e+17 true resid norm >> 1.828781621979e+16 ||r(i)||/||b|| 1.395648109568e-01 >> 11 KSP preconditioned resid norm 5.614465792481e+16 true resid norm >> 9.930812946447e+15 ||r(i)||/||b|| 7.578772746079e-02 >> 12 KSP preconditioned resid norm 4.960081715579e+16 true resid norm >> 8.035214375965e+15 ||r(i)||/||b|| 6.132132792135e-02 >> 13 KSP preconditioned resid norm 3.329600867897e+16 true resid norm >> 6.602562228940e+15 ||r(i)||/||b|| 5.038793797127e-02 >> 14 KSP preconditioned resid norm 3.130960732609e+16 true resid norm >> 4.730473520325e+15 ||r(i)||/||b|| 3.610095566719e-02 >> 15 KSP preconditioned resid norm 2.560518921527e+16 true resid norm >> 3.945110867578e+15 ||r(i)||/||b|| 3.010740297365e-02 >> 16 KSP preconditioned resid norm 1.584886017619e+16 true resid norm >> 3.416780637867e+15 ||r(i)||/||b|| 2.607541207073e-02 >> 17 KSP preconditioned resid norm 1.443824757417e+16 true resid norm >> 2.286337927483e+15 ||r(i)||/||b|| 1.744835560449e-02 >> 18 KSP preconditioned resid norm 1.039145582094e+16 true resid norm >> 1.711526931765e+15 ||r(i)||/||b|| 1.306164332627e-02 >> 19 KSP preconditioned resid norm 1.018242603191e+16 true resid norm >> 1.657426513724e+15 ||r(i)||/||b|| 1.264877201754e-02 >> 20 KSP preconditioned resid norm 8.198983478967e+15 true resid norm >> 1.311419876212e+15 ||r(i)||/||b|| 1.000819698256e-02 >> 21 KSP preconditioned resid norm 6.481494503642e+15 true resid norm >> 1.039872641976e+15 ||r(i)||/||b|| 7.935864345546e-03 >> 22 KSP preconditioned resid norm 6.325821399407e+15 true resid norm >> 1.165377948147e+15 ||r(i)||/||b|| 8.893667295849e-03 >> 23 KSP preconditioned resid norm 5.405294478066e+15 true resid norm >> 1.922243319095e+15 ||r(i)||/||b|| 1.466974089297e-02 >> 24 KSP preconditioned resid norm 4.960676496180e+15 true resid norm >> 8.018184223519e+14 ||r(i)||/||b|| 6.119136106374e-03 >> 25 KSP preconditioned resid norm 4.419138742847e+15 true resid norm >> 7.148418830282e+14 ||r(i)||/||b|| 5.455368266491e-03 >> 26 KSP preconditioned resid norm 2.795870478230e+15 true resid norm >> 4.843283048525e+14 ||r(i)||/||b|| 3.696186985663e-03 >> 27 KSP preconditioned resid norm 2.911201681995e+15 true resid norm >> 4.441742607418e+14 ||r(i)||/||b|| 3.389748452592e-03 >> 28 KSP preconditioned resid norm 2.776697567476e+15 true resid norm >> 5.074732182061e+14 ||r(i)||/||b|| 3.872819089681e-03 >> 29 KSP preconditioned resid norm 1.941501506934e+15 true resid norm >> 2.950418833153e+14 ||r(i)||/||b|| 2.251633774878e-03 >> 30 KSP preconditioned resid norm 8.413150178496e+14 true resid norm >> 1.695476752862e+14 ||r(i)||/||b|| 1.293915520864e-03 >> 0 KSP preconditioned resid norm 1.415973479575e+17 true resid norm >> 2.042710742436e+16 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 1.440919303250e+17 true resid norm >> 4.045863378014e+16 ||r(i)||/||b|| 1.980634503928e+00 >> 2 KSP preconditioned resid norm 1.100352174605e+17 true resid norm >> 3.072773288453e+16 ||r(i)||/||b|| 1.504262558872e+00 >> 3 KSP preconditioned resid norm 9.218323125716e+16 true resid norm >> 1.914173874994e+16 ||r(i)||/||b|| 9.370753456316e-01 >> 4 KSP preconditioned resid norm 8.672799672395e+16 true resid norm >> 1.779286318286e+16 ||r(i)||/||b|| 8.710417394506e-01 >> 5 KSP preconditioned resid norm 9.059066321416e+16 true resid norm >> 1.574500663517e+16 ||r(i)||/||b|| 7.707898288327e-01 >> 6 KSP preconditioned resid norm 1.027848583976e+17 true resid norm >> 1.954996111922e+16 ||r(i)||/||b|| 9.570596909823e-01 >> 7 KSP preconditioned resid norm 9.069106612200e+16 true resid norm >> 1.686428629204e+16 ||r(i)||/||b|| 8.255836688812e-01 >> 8 KSP preconditioned resid norm 1.359707392060e+17 true resid norm >> 2.263432282464e+16 ||r(i)||/||b|| 1.108053252691e+00 >> 9 KSP preconditioned resid norm 1.256447896822e+17 true resid norm >> 2.668588264948e+16 ||r(i)||/||b|| 1.306395570116e+00 >> 10 KSP preconditioned resid norm 5.661761773340e+16 true resid norm >> 9.807578010522e+15 ||r(i)||/||b|| 4.801256392682e-01 >> 11 KSP preconditioned resid norm 1.597811133832e+16 true resid norm >> 2.854210532053e+15 ||r(i)||/||b|| 1.397266129148e-01 >> 12 KSP preconditioned resid norm 1.141170314391e+16 true resid norm >> 1.817769027418e+15 ||r(i)||/||b|| 8.898807793264e-02 >> 13 KSP preconditioned resid norm 9.557798166843e+15 true resid norm >> 1.455009507586e+15 ||r(i)||/||b|| 7.122934624852e-02 >> 14 KSP preconditioned resid norm 7.997842098400e+15 true resid norm >> 2.021581420109e+15 ||r(i)||/||b|| 9.896562338034e-02 >> 15 KSP preconditioned resid norm 7.518666379332e+15 true resid norm >> 2.232979183117e+15 ||r(i)||/||b|| 1.093145072735e-01 >> 16 KSP preconditioned resid norm 6.614897634500e+15 true resid norm >> 1.222952133628e+15 ||r(i)||/||b|| 5.986908024822e-02 >> 17 KSP preconditioned resid norm 6.199437182741e+15 true resid norm >> 1.163257046730e+15 ||r(i)||/||b|| 5.694673369870e-02 >> 18 KSP preconditioned resid norm 4.584922270319e+15 true resid norm >> 9.962391827866e+14 ||r(i)||/||b|| 4.877044811535e-02 >> 19 KSP preconditioned resid norm 5.012002077991e+15 true resid norm >> 8.755757394511e+14 ||r(i)||/||b|| 4.286342266977e-02 >> 20 KSP preconditioned resid norm 4.312819580189e+15 true resid norm >> 6.706322150678e+14 ||r(i)||/||b|| 3.283050317090e-02 >> 21 KSP preconditioned resid norm 2.607093537931e+15 true resid norm >> 5.192589254045e+14 ||r(i)||/||b|| 2.542009079490e-02 >> 22 KSP preconditioned resid norm 2.989363098445e+15 true resid norm >> 4.636989016003e+14 ||r(i)||/||b|| 2.270017442838e-02 >> 23 KSP preconditioned resid norm 3.016276187245e+15 true resid norm >> 4.674022912685e+14 ||r(i)||/||b|| 2.288147222994e-02 >> 24 KSP preconditioned resid norm 2.901859180715e+15 true resid norm >> 4.492447164798e+14 ||r(i)||/||b|| 2.199257619530e-02 >> 25 KSP preconditioned resid norm 2.843000955863e+15 true resid norm >> 5.109023428494e+14 ||r(i)||/||b|| 2.501099799574e-02 >> 26 KSP preconditioned resid norm 2.741940909323e+15 true resid norm >> 4.263494245202e+14 ||r(i)||/||b|| 2.087174731415e-02 >> 27 KSP preconditioned resid norm 1.296293793548e+15 true resid norm >> 2.124064840417e+14 ||r(i)||/||b|| 1.039826538477e-02 >> 28 KSP preconditioned resid norm 9.720512789714e+14 true resid norm >> 2.091841490290e+14 ||r(i)||/||b|| 1.024051740089e-02 >> 29 KSP preconditioned resid norm 1.459179957917e+15 true resid norm >> 2.244723575034e+14 ||r(i)||/||b|| 1.098894487800e-02 >> 30 KSP preconditioned resid norm 1.339547884067e+15 true resid norm >> 2.746667338979e+14 ||r(i)||/||b|| 1.344618835119e-02 >> 31 KSP preconditioned resid norm 1.186062213325e+15 true resid norm >> 1.831856176134e+14 ||r(i)||/||b|| 8.967770806105e-03 >> 32 KSP preconditioned resid norm 9.909356538944e+14 true resid norm >> 1.788908155328e+14 ||r(i)||/||b|| 8.757520671747e-03 >> 33 KSP preconditioned resid norm 7.609550150745e+14 true resid norm >> 1.203572113192e+14 ||r(i)||/||b|| 5.892033992812e-03 >> 34 KSP preconditioned resid norm 5.051711143811e+14 true resid norm >> 7.920683224412e+13 ||r(i)||/||b|| 3.877535404237e-03 >> 35 KSP preconditioned resid norm 5.443461822227e+14 true resid norm >> 8.557198028123e+13 ||r(i)||/||b|| 4.189138408269e-03 >> 36 KSP preconditioned resid norm 4.648220720251e+14 true resid norm >> 7.162139608842e+13 ||r(i)||/||b|| 3.506193735634e-03 >> 37 KSP preconditioned resid norm 3.842876907480e+14 true resid norm >> 6.998069618798e+13 ||r(i)||/||b|| 3.425873998417e-03 >> 38 KSP preconditioned resid norm 2.822934553902e+14 true resid norm >> 5.594039593962e+13 ||r(i)||/||b|| 2.738537316003e-03 >> 39 KSP preconditioned resid norm 2.288268021627e+14 true resid norm >> 3.624972614121e+13 ||r(i)||/||b|| 1.774589293929e-03 >> 40 KSP preconditioned resid norm 3.336880274482e+14 true resid norm >> 5.633095049288e+13 ||r(i)||/||b|| 2.757656741242e-03 >> 41 KSP preconditioned resid norm 1.655073694079e+14 true resid norm >> 2.539321805860e+13 ||r(i)||/||b|| 1.243113747388e-03 >> 42 KSP preconditioned resid norm 1.317678812236e+14 true resid norm >> 2.429513710603e+13 ||r(i)||/||b|| 1.189357680523e-03 >> 0 KSP preconditioned resid norm 4.735355646927e+18 true resid norm >> 6.977368406398e+17 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 1.655056377336e+18 true resid norm >> 5.433322725028e+17 ||r(i)||/||b|| 7.787065851426e-01 >> 2 KSP preconditioned resid norm 8.847192945348e+17 true resid norm >> 2.725627800798e+17 ||r(i)||/||b|| 3.906383670810e-01 >> 3 KSP preconditioned resid norm 5.344946494044e+17 true resid norm >> 1.588736353868e+17 ||r(i)||/||b|| 2.276985048419e-01 >> 4 KSP preconditioned resid norm 3.198139750974e+17 true resid norm >> 6.624530455539e+16 ||r(i)||/||b|| 9.494310848578e-02 >> 5 KSP preconditioned resid norm 3.286368493485e+17 true resid norm >> 6.812427209313e+16 ||r(i)||/||b|| 9.763605434775e-02 >> 6 KSP preconditioned resid norm 4.284030910438e+17 true resid norm >> 1.696553569581e+17 ||r(i)||/||b|| 2.431509232084e-01 >> 7 KSP preconditioned resid norm 1.587968810131e+18 true resid norm >> 3.732963000458e+17 ||r(i)||/||b|| 5.350101618592e-01 >> 8 KSP preconditioned resid norm 6.621891209840e+16 true resid norm >> 1.229792367414e+16 ||r(i)||/||b|| 1.762544695628e-02 >> 9 KSP preconditioned resid norm 5.638286198157e+16 true resid norm >> 2.091986582514e+16 ||r(i)||/||b|| 2.998245843800e-02 >> 10 KSP preconditioned resid norm 1.309067125882e+16 true resid norm >> 2.099826104432e+15 ||r(i)||/||b|| 3.009481486611e-03 >> 11 KSP preconditioned resid norm 1.075925809079e+16 true resid norm >> 2.578010577039e+15 ||r(i)||/||b|| 3.694817912546e-03 >> 12 KSP preconditioned resid norm 9.319606428298e+15 true resid norm >> 2.026298982436e+15 ||r(i)||/||b|| 2.904102040216e-03 >> 13 KSP preconditioned resid norm 4.070270220193e+15 true resid norm >> 2.216532493559e+15 ||r(i)||/||b|| 3.176745678968e-03 >> 14 KSP preconditioned resid norm 3.400982828163e+15 true resid norm >> 9.161069365806e+14 ||r(i)||/||b|| 1.312969135671e-03 >> 15 KSP preconditioned resid norm 4.173720697476e+14 true resid norm >> 7.019272258636e+13 ||r(i)||/||b|| 1.006005681483e-04 >> 16 KSP preconditioned resid norm 2.871344308441e+14 true resid norm >> 8.503915375563e+13 ||r(i)||/||b|| 1.218785490496e-04 >> 17 KSP preconditioned resid norm 8.731930916187e+13 true resid norm >> 2.939915802842e+13 ||r(i)||/||b|| 4.213502328681e-05 >> 18 KSP preconditioned resid norm 7.273170647297e+13 true resid norm >> 2.554194915512e+13 ||r(i)||/||b|| 3.660685184933e-05 >> 19 KSP preconditioned resid norm 5.198384943683e+13 true resid norm >> 8.213409786802e+12 ||r(i)||/||b|| 1.177150081293e-05 >> 20 KSP preconditioned resid norm 2.826075121950e+13 true resid norm >> 1.225537290577e+13 ||r(i)||/||b|| 1.756446297795e-05 >> 0 KSP preconditioned resid norm 7.535755573391e+17 true resid norm >> 1.091530949677e+17 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP preconditioned resid norm 5.749658535033e+17 true resid norm >> 1.739986551510e+17 ||r(i)||/||b|| 1.594078987889e+00 >> 2 KSP preconditioned resid norm 3.832130567981e+17 true resid norm >> 1.138657236506e+17 ||r(i)||/||b|| 1.043174485197e+00 >> 3 KSP preconditioned resid norm 2.820261830557e+17 true resid norm >> 6.703201541200e+16 ||r(i)||/||b|| 6.141100756863e-01 >> 4 KSP preconditioned resid norm 2.196333397214e+17 true resid norm >> 4.464718002673e+16 ||r(i)||/||b|| 4.090326530819e-01 >> 5 KSP preconditioned resid norm 2.438749906836e+17 true resid norm >> 4.259961207836e+16 ||r(i)||/||b|| 3.902739733672e-01 >> 6 KSP preconditioned resid norm 7.293863369143e+17 true resid norm >> 1.861518427200e+17 ||r(i)||/||b|| 1.705419738901e+00 >> 7 KSP preconditioned resid norm 3.735172838302e+17 true resid norm >> 7.178953918634e+16 ||r(i)||/||b|| 6.576958647631e-01 >> 8 KSP preconditioned resid norm 2.306462817079e+17 true resid norm >> 3.812340554474e+16 ||r(i)||/||b|| 3.492654565226e-01 >> 9 KSP preconditioned resid norm 2.324630150277e+17 true resid norm >> 5.879635462988e+16 ||r(i)||/||b|| 5.386595281360e-01 >> 10 KSP preconditioned resid norm 2.486084300490e+16 true resid norm >> 3.943582532938e+15 ||r(i)||/||b|| 3.612891172811e-02 >> 11 KSP preconditioned resid norm 2.044336976942e+16 true resid norm >> 4.427406712026e+15 ||r(i)||/||b|| 4.056143999707e-02 >> 12 KSP preconditioned resid norm 1.903964203761e+16 true resid norm >> 3.227987898262e+15 ||r(i)||/||b|| 2.957303133931e-02 >> 13 KSP preconditioned resid norm 6.823460749064e+15 true resid norm >> 1.876444880951e+15 ||r(i)||/||b|| 1.719094526368e-02 >> 14 KSP preconditioned resid norm 7.090194359812e+15 true resid norm >> 1.450790953432e+15 ||r(i)||/||b|| 1.329134051455e-02 >> 15 KSP preconditioned resid norm 4.884314111756e+15 true resid norm >> 8.008003608330e+14 ||r(i)||/||b|| 7.336487903253e-03 >> 16 KSP preconditioned resid norm 1.603299777176e+15 true resid norm >> 2.941609026573e+14 ||r(i)||/||b|| 2.694938725689e-03 >> 17 KSP preconditioned resid norm 1.548805654760e+15 true resid norm >> 2.838534773138e+14 ||r(i)||/||b|| 2.600507822503e-03 >> 18 KSP preconditioned resid norm 1.022454633110e+15 true resid norm >> 2.215058790848e+14 ||r(i)||/||b|| 2.029313773928e-03 >> 19 KSP preconditioned resid norm 9.587995256078e+14 true resid norm >> 2.226912518453e+14 ||r(i)||/||b|| 2.040173500451e-03 >> 20 KSP preconditioned resid norm 7.709554287637e+14 true resid norm >> 1.253194818485e+14 ||r(i)||/||b|| 1.148107452983e-03 >> 21 KSP preconditioned resid norm 3.692994290799e+14 true resid norm >> 6.513442758709e+13 ||r(i)||/||b|| 5.967254305192e-04 >> 22 KSP preconditioned resid norm 3.577186864519e+14 true resid norm >> 5.884014914924e+13 ||r(i)||/||b|| 5.390607491859e-04 >> 23 KSP preconditioned resid norm 1.723423887037e+14 true resid norm >> 4.313917181929e+13 ||r(i)||/||b|| 3.952171198816e-04 >> 24 KSP preconditioned resid norm 1.656260736170e+14 true resid norm >> 3.420915571267e+13 ||r(i)||/||b|| 3.134052746997e-04 >> 25 KSP preconditioned resid norm 1.188400863617e+14 true resid norm >> 1.877160245085e+13 ||r(i)||/||b|| 1.719749903236e-04 >> 26 KSP preconditioned resid norm 8.416275062676e+13 true resid norm >> 1.813244070153e+13 ||r(i)||/||b|| 1.661193455568e-04 >> 27 KSP preconditioned resid norm 5.224466611453e+13 true resid norm >> 8.293327735079e+12 ||r(i)||/||b|| 7.597886012791e-05 >> 28 KSP preconditioned resid norm 4.471999764526e+13 true resid norm >> 9.742703301619e+12 ||r(i)||/||b|| 8.925723365429e-05 >> 29 KSP preconditioned resid norm 9.410327220658e+14 true resid norm >> 1.612776349391e+14 ||r(i)||/||b|| 1.477536069746e-03 >> 30 KSP preconditioned resid norm 5.783713583723e+14 true resid norm >> 1.102180319983e+14 ||r(i)||/||b|| 1.009756361291e-03 >> 31 KSP preconditioned resid norm 1.833775181615e+13 true resid norm >> 4.198845362007e+12 ||r(i)||/||b|| 3.846748791915e-05 >> 32 KSP preconditioned resid norm 1.462922686654e+13 true resid norm >> 3.704838629537e+12 ||r(i)||/||b|| 3.394167275452e-05 >> 33 KSP preconditioned resid norm 1.053043620710e+13 true resid norm >> 1.904310347574e+12 ||r(i)||/||b|| 1.744623318411e-05 >> 34 KSP preconditioned resid norm 8.014167149326e+13 true resid norm >> 1.458222541169e+13 ||r(i)||/||b|| 1.335942459168e-04 >> 35 KSP preconditioned resid norm 5.245679087504e+12 true resid norm >> 1.495201096266e+12 ||r(i)||/||b|| 1.369820156459e-05 >> 0 KSP preconditioned resid norm 1.124208361458e+16 true resid norm >> 6.487794900041e+15 ||r(i)||/||b|| 1.232063762752e-01 >> 1 KSP preconditioned resid norm 5.136632948471e+15 true resid norm >> 1.030125368299e+15 ||r(i)||/||b|| 1.956258107611e-02 >> 2 KSP preconditioned resid norm 4.225773164973e+15 true resid norm >> 7.107211019673e+14 ||r(i)||/||b|| 1.349693892375e-02 >> 3 KSP preconditioned resid norm 4.144329493920e+15 true resid norm >> 7.618007419548e+14 ||r(i)||/||b|| 1.446696609651e-02 >> 4 KSP preconditioned resid norm 4.333716562490e+15 true resid norm >> 8.456727780045e+14 ||r(i)||/||b|| 1.605973679776e-02 >> 5 KSP preconditioned resid norm 4.885745055980e+15 true resid norm >> 7.972847407816e+14 ||r(i)||/||b|| 1.514082446881e-02 >> 6 KSP preconditioned resid norm 6.011621624545e+15 true resid norm >> 9.523510473075e+14 ||r(i)||/||b|| 1.808560894548e-02 >> 7 KSP preconditioned resid norm 7.637951309150e+15 true resid norm >> 1.487731882292e+15 ||r(i)||/||b|| 2.825275103643e-02 >> 8 KSP preconditioned resid norm 8.460249904060e+15 true resid norm >> 1.336944180115e+15 ||r(i)||/||b|| 2.538921933447e-02 >> 9 KSP preconditioned resid norm 8.381050292195e+15 true resid norm >> 1.752911449852e+15 ||r(i)||/||b|| 3.328863982216e-02 >> 10 KSP preconditioned resid norm 7.434169027728e+15 true resid norm >> 1.180498665014e+15 ||r(i)||/||b|| 2.241824301708e-02 >> 11 KSP preconditioned resid norm 5.682629267055e+15 true resid norm >> 9.983854763086e+14 ||r(i)||/||b|| 1.895982511114e-02 >> 12 KSP preconditioned resid norm 3.541898147790e+15 true resid norm >> 5.480576756264e+14 ||r(i)||/||b|| 1.040788145188e-02 >> 13 KSP preconditioned resid norm 2.510644084392e+15 true resid norm >> 4.650369826671e+14 ||r(i)||/||b|| 8.831278169411e-03 >> 14 KSP preconditioned resid norm 2.282038802861e+15 true resid norm >> 3.635360161467e+14 ||r(i)||/||b|| 6.903725516148e-03 >> 15 KSP preconditioned resid norm 2.130130680848e+15 true resid norm >> 3.489368266499e+14 ||r(i)||/||b|| 6.626479816775e-03 >> 16 KSP preconditioned resid norm 1.929984942209e+15 true resid norm >> 3.145683546766e+14 ||r(i)||/||b|| 5.973805841227e-03 >> 17 KSP preconditioned resid norm 1.692158321133e+15 true resid norm >> 2.756020615583e+14 ||r(i)||/||b|| 5.233817008974e-03 >> 18 KSP preconditioned resid norm 1.473781775606e+15 true resid norm >> 2.951884895649e+14 ||r(i)||/||b|| 5.605772789952e-03 >> 19 KSP preconditioned resid norm 1.275239549902e+15 true resid norm >> 1.993708462138e+14 ||r(i)||/||b|| 3.786149204063e-03 >> 20 KSP preconditioned resid norm 1.159341081209e+15 true resid norm >> 2.011812256624e+14 ||r(i)||/||b|| 3.820529189093e-03 >> 21 KSP preconditioned resid norm 1.026465449352e+15 true resid norm >> 1.599378003014e+14 ||r(i)||/||b|| 3.037296509548e-03 >> 22 KSP preconditioned resid norm 9.313333291328e+14 true resid norm >> 1.626450968383e+14 ||r(i)||/||b|| 3.088709385718e-03 >> 23 KSP preconditioned resid norm 8.475207473455e+14 true resid norm >> 1.311587742521e+14 ||r(i)||/||b|| 2.490768826892e-03 >> 24 KSP preconditioned resid norm 7.894756750239e+14 true resid norm >> 1.443612018407e+14 ||r(i)||/||b|| 2.741489339222e-03 >> 25 KSP preconditioned resid norm 7.370222420808e+14 true resid norm >> 1.135397001576e+14 ||r(i)||/||b|| 2.156174052249e-03 >> 26 KSP preconditioned resid norm 6.874978587995e+14 true resid norm >> 1.225152011526e+14 ||r(i)||/||b|| 2.326623175549e-03 >> 27 KSP preconditioned resid norm 6.203646083349e+14 true resid norm >> 9.547951255722e+13 ||r(i)||/||b|| 1.813202317882e-03 >> 28 KSP preconditioned resid norm 5.655995299941e+14 true resid norm >> 9.635247011837e+13 ||r(i)||/||b|| 1.829780205963e-03 >> 29 KSP preconditioned resid norm 5.170668664072e+14 true resid norm >> 7.945307015395e+13 ||r(i)||/||b|| 1.508852392597e-03 >> 30 KSP preconditioned resid norm 4.771681972381e+14 true resid norm >> 7.711681382438e+13 ||r(i)||/||b|| 1.464485750178e-03 >> 31 KSP preconditioned resid norm 4.347723667483e+14 true resid norm >> 6.696963877348e+13 ||r(i)||/||b|| 1.271785967477e-03 >> 32 KSP preconditioned resid norm 4.053435837983e+14 true resid norm >> 6.425971653178e+13 ||r(i)||/||b|| 1.220323227897e-03 >> 33 KSP preconditioned resid norm 3.774514402447e+14 true resid norm >> 5.808746381526e+13 ||r(i)||/||b|| 1.103109150945e-03 >> 34 KSP preconditioned resid norm 3.533225997981e+14 true resid norm >> 5.617172311710e+13 ||r(i)||/||b|| 1.066728304612e-03 >> 0 KSP preconditioned resid norm 1.117659190837e+16 true resid norm >> 6.478857863821e+15 ||r(i)||/||b|| 6.224768870229e-01 >> 1 KSP preconditioned resid norm 5.124485271748e+15 true resid norm >> 1.017923791746e+15 ||r(i)||/||b|| 9.780026764454e-02 >> 2 KSP preconditioned resid norm 4.246328499709e+15 true resid norm >> 7.102918780385e+14 ||r(i)||/||b|| 6.824355255393e-02 >> 3 KSP preconditioned resid norm 4.210000472667e+15 true resid norm >> 7.697163273299e+14 ||r(i)||/||b|| 7.395294562682e-02 >> 4 KSP preconditioned resid norm 4.471596919519e+15 true resid norm >> 8.475637573508e+14 ||r(i)||/||b|| 8.143238520100e-02 >> 5 KSP preconditioned resid norm 5.135403291038e+15 true resid norm >> 8.255731593574e+14 ||r(i)||/||b|| 7.931956851781e-02 >> 6 KSP preconditioned resid norm 6.474268664840e+15 true resid norm >> 1.007942498042e+15 ||r(i)||/||b|| 9.684128308834e-02 >> 7 KSP preconditioned resid norm 8.529281532903e+15 true resid norm >> 1.642696055823e+15 ||r(i)||/||b|| 1.578272511369e-01 >> 8 KSP preconditioned resid norm 9.784577605090e+15 true resid norm >> 1.538078623395e+15 ||r(i)||/||b|| 1.477757983909e-01 >> 9 KSP preconditioned resid norm 1.010006302597e+16 true resid norm >> 2.046930405602e+15 ||r(i)||/||b|| 1.966653526923e-01 >> 10 KSP preconditioned resid norm 9.531884877717e+15 true resid norm >> 1.482620346274e+15 ||r(i)||/||b|| 1.424474679309e-01 >> 11 KSP preconditioned resid norm 7.931210548314e+15 true resid norm >> 1.282982557872e+15 ||r(i)||/||b|| 1.232666321002e-01 >> 12 KSP preconditioned resid norm 5.236526900965e+15 true resid norm >> 7.915538690853e+14 ||r(i)||/||b|| 7.605105694487e-02 >> 13 KSP preconditioned resid norm 3.885416517845e+15 true resid norm >> 6.196979959619e+14 ||r(i)||/||b|| 5.953945703529e-02 >> 14 KSP preconditioned resid norm 3.337626627490e+15 true resid norm >> 7.828597862542e+14 ||r(i)||/||b|| 7.521574526958e-02 >> 15 KSP preconditioned resid norm 3.054301414200e+15 true resid norm >> 4.972852337202e+14 ||r(i)||/||b|| 4.777826134714e-02 >> 16 KSP preconditioned resid norm 2.784835626497e+15 true resid norm >> 4.900149140724e+14 ||r(i)||/||b|| 4.707974225054e-02 >> 17 KSP preconditioned resid norm 2.551999131175e+15 true resid norm >> 4.223064940199e+14 ||r(i)||/||b|| 4.057444032459e-02 >> 18 KSP preconditioned resid norm 2.358735848485e+15 true resid norm >> 3.867160573888e+14 ||r(i)||/||b|| 3.715497586533e-02 >> 19 KSP preconditioned resid norm 2.111367549832e+15 true resid norm >> 3.463614183500e+14 ||r(i)||/||b|| 3.327777549857e-02 >> 20 KSP preconditioned resid norm 1.933287196882e+15 true resid norm >> 3.127201079650e+14 ||r(i)||/||b|| 3.004557954614e-02 >> 21 KSP preconditioned resid norm 1.712007284199e+15 true resid norm >> 2.787684847304e+14 ||r(i)||/||b|| 2.678356929917e-02 >> 22 KSP preconditioned resid norm 1.556420935486e+15 true resid norm >> 2.516757515483e+14 ||r(i)||/||b|| 2.418054874112e-02 >> 23 KSP preconditioned resid norm 1.424455851932e+15 true resid norm >> 2.267048417314e+14 ||r(i)||/||b|| 2.178138911521e-02 >> 24 KSP preconditioned resid norm 1.341093049266e+15 true resid norm >> 2.200423231461e+14 ||r(i)||/||b|| 2.114126643990e-02 >> 25 KSP preconditioned resid norm 1.265242012282e+15 true resid norm >> 1.979405460182e+14 ||r(i)||/||b|| 1.901776786755e-02 >> 26 KSP preconditioned resid norm 1.190343462827e+15 true resid norm >> 1.929502180424e+14 ||r(i)||/||b|| 1.853830622649e-02 >> 27 KSP preconditioned resid norm 1.080874612432e+15 true resid norm >> 1.688981747210e+14 ||r(i)||/||b|| 1.622742962325e-02 >> 28 KSP preconditioned resid norm 9.890419339482e+14 true resid norm >> 1.571112460256e+14 ||r(i)||/||b|| 1.509496293914e-02 >> 29 KSP preconditioned resid norm 9.054720544549e+14 true resid norm >> 1.411421952126e+14 ||r(i)||/||b|| 1.356068556375e-02 >> 30 KSP preconditioned resid norm 8.347895469778e+14 true resid norm >> 1.302020126144e+14 ||r(i)||/||b|| 1.250957270554e-02 >> 31 KSP preconditioned resid norm 7.581633555103e+14 true resid norm >> 1.198729260222e+14 ||r(i)||/||b|| 1.151717284080e-02 >> 32 KSP preconditioned resid norm 7.073149016328e+14 true resid norm >> 1.097657041094e+14 ||r(i)||/||b|| 1.054608933119e-02 >> 33 KSP preconditioned resid norm 6.626892210136e+14 true resid norm >> 1.045304064903e+14 ||r(i)||/||b|| 1.004309145207e-02 >> 34 KSP preconditioned resid norm 6.277118615935e+14 true resid norm >> 9.735143814138e+13 ||r(i)||/||b|| 9.353349222222e-03 >> 35 KSP preconditioned resid norm 6.007031548284e+14 true resid norm >> 9.325545854852e+13 ||r(i)||/||b|| 8.959814948147e-03 >> 36 KSP preconditioned resid norm 5.720600717487e+14 true resid norm >> 8.974565981239e+13 ||r(i)||/||b|| 8.622599865294e-03 >> 37 KSP preconditioned resid norm 5.429834406497e+14 true resid norm >> 8.373407335142e+13 ||r(i)||/||b|| 8.045017565303e-03 >> 38 KSP preconditioned resid norm 5.122271874419e+14 true resid norm >> 8.063599233502e+13 ||r(i)||/||b|| 7.747359572588e-03 >> 39 KSP preconditioned resid norm 4.877390474820e+14 true resid norm >> 7.540978455988e+13 ||r(i)||/||b|| 7.245235029160e-03 >> 40 KSP preconditioned resid norm 4.664511103016e+14 true resid norm >> 7.249582547116e+13 ||r(i)||/||b|| 6.965267136580e-03 >> 41 KSP preconditioned resid norm 4.430412921029e+14 true resid norm >> 6.907345898754e+13 ||r(i)||/||b|| 6.636452385624e-03 >> 42 KSP preconditioned resid norm 4.184290025581e+14 true resid norm >> 6.506826627971e+13 ||r(i)||/||b|| 6.251640750440e-03 >> 43 KSP preconditioned resid norm 3.961557540211e+14 true resid norm >> 6.169515842630e+13 ||r(i)||/||b|| 5.927558679138e-03 >> 44 KSP preconditioned resid norm 3.628856632080e+14 true resid norm >> 5.666822624927e+13 ||r(i)||/||b|| 5.444580172956e-03 >> 45 KSP preconditioned resid norm 3.439726178254e+14 true resid norm >> 5.290740915351e+13 ||r(i)||/||b|| 5.083247702381e-03 >> 46 KSP preconditioned resid norm 3.808130490503e+14 true resid norm >> 7.046303156753e+13 ||r(i)||/||b|| 6.769959993302e-03 >> 47 KSP preconditioned resid norm 2.548925306353e+14 true resid norm >> 3.911087819219e+13 ||r(i)||/||b|| 3.757702085387e-03 >> 48 KSP preconditioned resid norm 1.722230117103e+14 true resid norm >> 2.660198748286e+13 ||r(i)||/||b|| 2.555870603278e-03 >> 49 KSP preconditioned resid norm 8.498499404656e+13 true resid norm >> 1.314954336818e+13 ||r(i)||/||b|| 1.263384225066e-03 >> 50 KSP preconditioned resid norm 1.168755498979e+14 true resid norm >> 2.563042385986e+13 ||r(i)||/||b|| 2.462524536378e-03 >> 51 KSP preconditioned resid norm 5.069372366763e+13 true resid norm >> 7.752336990767e+12 ||r(i)||/||b|| 7.448304467540e-04 >> 0 KSP preconditioned resid norm 2.551555721110e+16 true resid norm >> 1.503348678660e+16 ||r(i)||/||b|| 1.139781099215e-01 >> 1 KSP preconditioned resid norm 1.135435821514e+16 true resid norm >> 2.150528625386e+15 ||r(i)||/||b|| 1.630448022690e-02 >> 2 KSP preconditioned resid norm 8.921910642422e+15 true resid norm >> 1.462019059252e+15 ||r(i)||/||b|| 1.108446572695e-02 >> 3 KSP preconditioned resid norm 8.401630743802e+15 true resid norm >> 1.508644008325e+15 ||r(i)||/||b|| 1.143795814332e-02 >> 4 KSP preconditioned resid norm 8.507938797573e+15 true resid norm >> 1.437822857483e+15 ||r(i)||/||b|| 1.090101943907e-02 >> 5 KSP preconditioned resid norm 9.195685552195e+15 true resid norm >> 1.421948919345e+15 ||r(i)||/||b|| 1.078066935052e-02 >> 6 KSP preconditioned resid norm 1.067998630800e+16 true resid norm >> 1.627316531548e+15 ||r(i)||/||b|| 1.233768753335e-02 >> 7 KSP preconditioned resid norm 1.295276293823e+16 true resid norm >> 2.221334900964e+15 ||r(i)||/||b|| 1.684130615262e-02 >> 8 KSP preconditioned resid norm 1.417997817727e+16 true resid norm >> 2.186215590047e+15 ||r(i)||/||b|| 1.657504505585e-02 >> 9 KSP preconditioned resid norm 1.359372910192e+16 true resid norm >> 2.107067442870e+15 ||r(i)||/||b|| 1.597497427073e-02 >> 10 KSP preconditioned resid norm 1.179811021994e+16 true resid norm >> 3.232410469947e+15 ||r(i)||/||b|| 2.450689191967e-02 >> 11 KSP preconditioned resid norm 7.523875759744e+15 true resid norm >> 1.204652282825e+15 ||r(i)||/||b|| 9.133209897209e-03 >> 12 KSP preconditioned resid norm 3.961884677960e+15 true resid norm >> 7.586571607759e+14 ||r(i)||/||b|| 5.751846560354e-03 >> 13 KSP preconditioned resid norm 2.175818220311e+15 true resid norm >> 3.487257134817e+14 ||r(i)||/||b|| 2.643904123366e-03 >> 14 KSP preconditioned resid norm 1.788448374434e+15 true resid norm >> 3.516091940967e+14 ||r(i)||/||b|| 2.665765563440e-03 >> 15 KSP preconditioned resid norm 1.604495215225e+15 true resid norm >> 2.568394922627e+14 ||r(i)||/||b|| 1.947258164179e-03 >> 16 KSP preconditioned resid norm 1.391089499920e+15 true resid norm >> 2.519771358465e+14 ||r(i)||/||b|| 1.910393649516e-03 >> 17 KSP preconditioned resid norm 1.231055803595e+15 true resid norm >> 2.082047844556e+14 ||r(i)||/||b|| 1.578528530720e-03 >> 18 KSP preconditioned resid norm 1.101068654831e+15 true resid norm >> 1.838589386072e+14 ||r(i)||/||b|| 1.393947699032e-03 >> 19 KSP preconditioned resid norm 9.356667448723e+14 true resid norm >> 1.565965285410e+14 ||r(i)||/||b|| 1.187254600129e-03 >> 20 KSP preconditioned resid norm 8.181513038456e+14 true resid norm >> 1.328185221533e+14 ||r(i)||/||b|| 1.006978908651e-03 >> 0 KSP preconditioned resid norm 2.272810589988e+16 true resid norm >> 1.323636987884e+16 ||r(i)||/||b|| 5.438005326146e-01 >> 1 KSP preconditioned resid norm 1.026969580669e+16 true resid norm >> 1.999385093393e+15 ||r(i)||/||b|| 8.214236143606e-02 >> 2 KSP preconditioned resid norm 8.322628960001e+15 true resid norm >> 1.379596337850e+15 ||r(i)||/||b|| 5.667907667911e-02 >> 3 KSP preconditioned resid norm 8.076035080201e+15 true resid norm >> 1.472961452514e+15 ||r(i)||/||b|| 6.051487150403e-02 >> 4 KSP preconditioned resid norm 8.405382352119e+15 true resid norm >> 1.516410203314e+15 ||r(i)||/||b|| 6.229991181666e-02 >> 5 KSP preconditioned resid norm 9.416373856303e+15 true resid norm >> 1.470293081560e+15 ||r(i)||/||b|| 6.040524465323e-02 >> 6 KSP preconditioned resid norm 1.145894310132e+16 true resid norm >> 1.749278072065e+15 ||r(i)||/||b|| 7.186701157399e-02 >> 7 KSP preconditioned resid norm 1.439168534642e+16 true resid norm >> 2.685145033101e+15 ||r(i)||/||b|| 1.103159939254e-01 >> 8 KSP preconditioned resid norm 1.556059742979e+16 true resid norm >> 2.450465189043e+15 ||r(i)||/||b|| 1.006744513151e-01 >> 9 KSP preconditioned resid norm 1.521075410830e+16 true resid norm >> 2.889962061237e+15 ||r(i)||/||b|| 1.187306582184e-01 >> 10 KSP preconditioned resid norm 1.335253374250e+16 true resid norm >> 2.051263914753e+15 ||r(i)||/||b|| 8.427374118333e-02 >> 11 KSP preconditioned resid norm 9.966633192923e+15 true resid norm >> 1.537352185287e+15 ||r(i)||/||b|| 6.316028826844e-02 >> 12 KSP preconditioned resid norm 5.467457389741e+15 true resid norm >> 8.244964108686e+14 ||r(i)||/||b|| 3.387345559797e-02 >> 13 KSP preconditioned resid norm 7.465607339270e+15 true resid norm >> 1.255988023601e+15 ||r(i)||/||b|| 5.160077592599e-02 >> 14 KSP preconditioned resid norm 3.395725692367e+15 true resid norm >> 8.163584192182e+14 ||r(i)||/||b|| 3.353911587836e-02 >> 15 KSP preconditioned resid norm 2.979451058224e+15 true resid norm >> 4.736799198293e+14 ||r(i)||/||b|| 1.946057680844e-02 >> 16 KSP preconditioned resid norm 2.589762095203e+15 true resid norm >> 5.000061827618e+14 ||r(i)||/||b|| 2.054216004731e-02 >> 17 KSP preconditioned resid norm 2.297387399767e+15 true resid norm >> 3.733072000155e+14 ||r(i)||/||b|| 1.533688285047e-02 >> 18 KSP preconditioned resid norm 2.076822069911e+15 true resid norm >> 3.563941616114e+14 ||r(i)||/||b|| 1.464203075912e-02 >> 19 KSP preconditioned resid norm 1.810897092306e+15 true resid norm >> 2.945299484347e+14 ||r(i)||/||b|| 1.210041305100e-02 >> 20 KSP preconditioned resid norm 1.630399807803e+15 true resid norm >> 2.670493324597e+14 ||r(i)||/||b|| 1.097140458866e-02 >> 21 KSP preconditioned resid norm 1.416015877821e+15 true resid norm >> 2.289562555093e+14 ||r(i)||/||b|| 9.406395773998e-03 >> 22 KSP preconditioned resid norm 1.270953330458e+15 true resid norm >> 2.078621859272e+14 ||r(i)||/||b|| 8.539770983461e-03 >> 23 KSP preconditioned resid norm 1.149572826465e+15 true resid norm >> 1.837793084226e+14 ||r(i)||/||b|| 7.550354569913e-03 >> 24 KSP preconditioned resid norm 1.075403245776e+15 true resid norm >> 1.776412893147e+14 ||r(i)||/||b|| 7.298181346390e-03 >> 25 KSP preconditioned resid norm 1.008390645992e+15 true resid norm >> 1.577927704554e+14 ||r(i)||/||b|| 6.482728527673e-03 >> 26 KSP preconditioned resid norm 9.445856732256e+14 true resid norm >> 1.515432067895e+14 ||r(i)||/||b|| 6.225972628492e-03 >> 27 KSP preconditioned resid norm 8.526344464253e+14 true resid norm >> 1.348203382746e+14 ||r(i)||/||b|| 5.538933441126e-03 >> 28 KSP preconditioned resid norm 7.744588594030e+14 true resid norm >> 1.215509014949e+14 ||r(i)||/||b|| 4.993774394172e-03 >> 29 KSP preconditioned resid norm 7.032530926217e+14 true resid norm >> 1.102818922582e+14 ||r(i)||/||b|| 4.530800536453e-03 >> 30 KSP preconditioned resid norm 6.388922318141e+14 true resid norm >> 9.838037383987e+13 ||r(i)||/||b|| 4.041840790388e-03 >> 31 KSP preconditioned resid norm 5.692116056101e+14 true resid norm >> 9.317411265002e+13 ||r(i)||/||b|| 3.827947734067e-03 >> 32 KSP preconditioned resid norm 5.208032879647e+14 true resid norm >> 8.007380640801e+13 ||r(i)||/||b|| 3.289737214338e-03 >> 33 KSP preconditioned resid norm 4.765676071483e+14 true resid norm >> 7.802986158610e+13 ||r(i)||/||b|| 3.205764169389e-03 >> 34 KSP preconditioned resid norm 4.409793613234e+14 true resid norm >> 6.791885197478e+13 ||r(i)||/||b|| 2.790365350662e-03 >> 35 KSP preconditioned resid norm 4.123618028942e+14 true resid norm >> 6.437750648670e+13 ||r(i)||/||b|| 2.644873378149e-03 >> 36 KSP preconditioned resid norm 3.827034276493e+14 true resid norm >> 5.974308940262e+13 ||r(i)||/||b|| 2.454473857604e-03 >> 37 KSP preconditioned resid norm 3.509107846575e+14 true resid norm >> 5.419166518671e+13 ||r(i)||/||b|| 2.226400188387e-03 >> 38 KSP preconditioned resid norm 3.170601404687e+14 true resid norm >> 4.972345072887e+13 ||r(i)||/||b|| 2.042828905305e-03 >> 39 KSP preconditioned resid norm 2.901553052614e+14 true resid norm >> 4.518683026559e+13 ||r(i)||/||b|| 1.856447242750e-03 >> 40 KSP preconditioned resid norm 2.667015211684e+14 true resid norm >> 4.122270495295e+13 ||r(i)||/||b|| 1.693585863377e-03 >> 41 KSP preconditioned resid norm 2.422095249996e+14 true resid norm >> 3.839959453856e+13 ||r(i)||/||b|| 1.577601725654e-03 >> 42 KSP preconditioned resid norm 2.195894402502e+14 true resid norm >> 3.399029120498e+13 ||r(i)||/||b|| 1.396450736130e-03 >> 43 KSP preconditioned resid norm 1.983452570770e+14 true resid norm >> 3.143816040870e+13 ||r(i)||/||b|| 1.291599474113e-03 >> 44 KSP preconditioned resid norm 1.779873290535e+14 true resid norm >> 2.755766172822e+13 ||r(i)||/||b|| 1.132173795580e-03 >> 45 KSP preconditioned resid norm 1.542400485092e+14 true resid norm >> 2.400274515345e+13 ||r(i)||/||b|| 9.861242710910e-04 >> 46 KSP preconditioned resid norm 1.223281336233e+14 true resid norm >> 1.907355666953e+13 ||r(i)||/||b|| 7.836144177513e-04 >> 0 KSP preconditioned resid norm 7.586062313136e+16 true resid norm >> 4.608637352265e+16 ||r(i)||/||b|| 6.590757061047e-02 >> 1 KSP preconditioned resid norm 2.763422737801e+16 true resid norm >> 5.227955984017e+15 ||r(i)||/||b|| 7.476437216213e-03 >> 2 KSP preconditioned resid norm 1.553456028419e+16 true resid norm >> 2.906104653716e+15 ||r(i)||/||b|| 4.155985447023e-03 >> 3 KSP preconditioned resid norm 1.040767126705e+16 true resid norm >> 2.091654777018e+15 ||r(i)||/||b|| 2.991250436342e-03 >> 4 KSP preconditioned resid norm 7.531366829970e+15 true resid norm >> 1.318758653617e+15 ||r(i)||/||b|| 1.885940950392e-03 >> 5 KSP preconditioned resid norm 5.773017012836e+15 true resid norm >> 9.450385953724e+14 ||r(i)||/||b|| 1.351488372664e-03 >> 6 KSP preconditioned resid norm 4.731507925306e+15 true resid norm >> 7.935497854840e+14 ||r(i)||/||b|| 1.134846040642e-03 >> 7 KSP preconditioned resid norm 3.788639277852e+15 true resid norm >> 7.524409398539e+14 ||r(i)||/||b|| 1.076056773034e-03 >> 8 KSP preconditioned resid norm 2.738698664678e+15 true resid norm >> 4.915430614067e+14 ||r(i)||/||b|| 7.029498429037e-04 >> 9 KSP preconditioned resid norm 1.837368414949e+15 true resid norm >> 3.357340764188e+14 ||r(i)||/||b|| 4.801292802316e-04 >> 10 KSP preconditioned resid norm 1.059331899595e+15 true resid norm >> 1.989071439556e+14 ||r(i)||/||b|| 2.844547234496e-04 >> 11 KSP preconditioned resid norm 4.945892712172e+14 true resid norm >> 8.452757650210e+13 ||r(i)||/||b|| 1.208818744245e-04 >> 12 KSP preconditioned resid norm 1.987888303244e+14 true resid norm >> 3.594134462521e+13 ||r(i)||/||b|| 5.139928633259e-05 >> 13 KSP preconditioned resid norm 1.163784016975e+14 true resid norm >> 2.571324386412e+13 ||r(i)||/||b|| 3.677220197779e-05 >> 14 KSP preconditioned resid norm 8.494315446492e+13 true resid norm >> 1.429434206797e+13 ||r(i)||/||b|| 2.044216732982e-05 >> 15 KSP preconditioned resid norm 8.642186180523e+13 true resid norm >> 1.607306715480e+13 ||r(i)||/||b|| 2.298590076546e-05 >> 16 KSP preconditioned resid norm 3.925619743667e+13 true resid norm >> 1.371348923807e+13 ||r(i)||/||b|| 1.961149665703e-05 >> 0 KSP preconditioned resid norm 3.953744769406e+16 true resid norm >> 2.171194598493e+16 ||r(i)||/||b|| 1.950905346919e-01 >> 1 KSP preconditioned resid norm 1.491502757726e+16 true resid norm >> 3.091776185716e+15 ||r(i)||/||b|| 2.778084790914e-02 >> 2 KSP preconditioned resid norm 1.008617531922e+16 true resid norm >> 1.691017660922e+15 ||r(i)||/||b|| 1.519447127732e-02 >> 3 KSP preconditioned resid norm 8.326895039450e+15 true resid norm >> 1.670298498749e+15 ||r(i)||/||b|| 1.500830130298e-02 >> 4 KSP preconditioned resid norm 7.300566603499e+15 true resid norm >> 1.589496633697e+15 ||r(i)||/||b|| 1.428226416804e-02 >> 5 KSP preconditioned resid norm 6.872352176548e+15 true resid norm >> 1.096739362492e+15 ||r(i)||/||b|| 9.854642637499e-03 >> 6 KSP preconditioned resid norm 6.961475959227e+15 true resid norm >> 1.092150928708e+15 ||r(i)||/||b|| 9.813413721358e-03 >> 7 KSP preconditioned resid norm 7.025042419735e+15 true resid norm >> 1.296288429690e+15 ||r(i)||/||b|| 1.164767096595e-02 >> 8 KSP preconditioned resid norm 6.408702665765e+15 true resid norm >> 1.056644614043e+15 ||r(i)||/||b|| 9.494375256645e-03 >> 9 KSP preconditioned resid norm 5.399106179995e+15 true resid norm >> 1.078818901202e+15 ||r(i)||/||b|| 9.693620112044e-03 >> 10 KSP preconditioned resid norm 4.030707639439e+15 true resid norm >> 7.107969460630e+14 ||r(i)||/||b|| 6.386795378039e-03 >> 11 KSP preconditioned resid norm 2.682586389004e+15 true resid norm >> 4.368532286577e+14 ||r(i)||/||b|| 3.925301307393e-03 >> 12 KSP preconditioned resid norm 1.621400972048e+15 true resid norm >> 2.499302020485e+14 ||r(i)||/||b|| 2.245723012904e-03 >> 13 KSP preconditioned resid norm 1.118646687359e+15 true resid norm >> 1.825993809337e+14 ||r(i)||/||b|| 1.640728605602e-03 >> 14 KSP preconditioned resid norm 8.665732278929e+14 true resid norm >> 1.337846645968e+14 ||r(i)||/||b|| 1.202108819167e-03 >> 15 KSP preconditioned resid norm 1.790412083411e+15 true resid norm >> 2.690613625509e+14 ||r(i)||/||b|| 2.417624155910e-03 >> 16 KSP preconditioned resid norm 4.902192144952e+14 true resid norm >> 8.615757491331e+13 ||r(i)||/||b|| 7.741603340972e-04 >> 17 KSP preconditioned resid norm 3.873085562421e+14 true resid norm >> 6.966874756362e+13 ||r(i)||/||b|| 6.260016132563e-04 >> 18 KSP preconditioned resid norm 3.069894649217e+14 true resid norm >> 4.954148501038e+13 ||r(i)||/||b|| 4.451500941838e-04 >> 19 KSP preconditioned resid norm 2.366855243734e+14 true resid norm >> 4.444373941350e+13 ||r(i)||/||b|| 3.993448073197e-04 >> 20 KSP preconditioned resid norm 1.853280795186e+14 true resid norm >> 2.839442050460e+13 ||r(i)||/||b|| 2.551352459311e-04 >> 21 KSP preconditioned resid norm 1.419103820891e+14 true resid norm >> 2.714177608773e+13 ||r(i)||/||b|| 2.438797339086e-04 >> 22 KSP preconditioned resid norm 1.116662401807e+14 true resid norm >> 1.710034207306e+13 ||r(i)||/||b|| 1.536534256654e-04 >> 23 KSP preconditioned resid norm 8.768706858701e+13 true resid norm >> 1.892684059360e+13 ||r(i)||/||b|| 1.700652467538e-04 >> 24 KSP preconditioned resid norm 7.191985915577e+13 true resid norm >> 1.104970014738e+13 ||r(i)||/||b|| 9.928598345972e-05 >> 25 KSP preconditioned resid norm 5.804472664630e+13 true resid norm >> 1.353159760302e+13 ||r(i)||/||b|| 1.215868265996e-04 >> 26 KSP preconditioned resid norm 4.574262022619e+13 true resid norm >> 6.915191476157e+12 ||r(i)||/||b|| 6.213576634340e-05 >> 27 KSP preconditioned resid norm 3.470073834736e+13 true resid norm >> 8.786634213858e+12 ||r(i)||/||b|| 7.895142923224e-05 >> 28 KSP preconditioned resid norm 2.674822501624e+13 true resid norm >> 4.041336605553e+12 ||r(i)||/||b|| 3.631302877201e-05 >> 29 KSP preconditioned resid norm 2.088626286399e+13 true resid norm >> 5.261273540428e+12 ||r(i)||/||b|| 4.727465096287e-05 >> 30 KSP preconditioned resid norm 1.567496199145e+13 true resid norm >> 2.371012817670e+12 ||r(i)||/||b|| 2.130450023603e-05 >> 31 KSP preconditioned resid norm 1.119841843422e+13 true resid norm >> 3.387071792996e+12 ||r(i)||/||b|| 3.043419726606e-05 >> 32 KSP preconditioned resid norm 7.994377199528e+12 true resid norm >> 1.204577998308e+12 ||r(i)||/||b|| 1.082361599145e-05 >> 33 KSP preconditioned resid norm 6.174157756589e+12 true resid norm >> 1.362745552188e+12 ||r(i)||/||b|| 1.224481484110e-05 >> 0 SNES Function norm 0.000000000000e+00 >> 1 TS dt 3e-12 time 2e-12 >> 0 KSP preconditioned resid norm 2.220368634013e+19 true resid norm >> 1.204306153911e+19 ||r(i)||/||b|| 2.302397943981e+02 >> 1 KSP preconditioned resid norm 5.390402695434e+18 true resid norm >> 1.775960718595e+18 ||r(i)||/||b|| 3.395289722473e+01 >> 2 KSP preconditioned resid norm 2.218493119258e+18 true resid norm >> 5.320083037595e+17 ||r(i)||/||b|| 1.017095877804e+01 >> 3 KSP preconditioned resid norm 1.401798813920e+18 true resid norm >> 2.963744523802e+17 ||r(i)||/||b|| 5.666100165582e+00 >> 4 KSP preconditioned resid norm 1.004143331118e+18 true resid norm >> 1.656359344976e+17 ||r(i)||/||b|| 3.166635276239e+00 >> 5 KSP preconditioned resid norm 8.673495646713e+17 true resid norm >> 1.510463721187e+17 ||r(i)||/||b|| 2.887711363779e+00 >> 6 KSP preconditioned resid norm 7.389956702949e+17 true resid norm >> 1.526304376340e+17 ||r(i)||/||b|| 2.917995599842e+00 >> 7 KSP preconditioned resid norm 5.566613599927e+17 true resid norm >> 1.017416269196e+17 ||r(i)||/||b|| 1.945101018344e+00 >> 8 KSP preconditioned resid norm 3.566897773081e+17 true resid norm >> 6.307244640523e+16 ||r(i)||/||b|| 1.205821878877e+00 >> 9 KSP preconditioned resid norm 1.893453815442e+17 true resid norm >> 3.268203516935e+16 ||r(i)||/||b|| 6.248166243663e-01 >> 10 KSP preconditioned resid norm 6.894933140555e+16 true resid norm >> 1.152954614680e+16 ||r(i)||/||b|| 2.204223839364e-01 >> 11 KSP preconditioned resid norm 5.103537427509e+16 true resid norm >> 9.272910080166e+15 ||r(i)||/||b|| 1.772799136993e-01 >> 12 KSP preconditioned resid norm 4.991235858989e+16 true resid norm >> 8.396555957121e+15 ||r(i)||/||b|| 1.605257359967e-01 >> 13 KSP preconditioned resid norm 4.918689912532e+16 true resid norm >> 8.423517310572e+15 ||r(i)||/||b|| 1.610411843696e-01 >> 14 KSP preconditioned resid norm 5.686949456502e+16 true resid norm >> 9.761258523981e+15 ||r(i)||/||b|| 1.866161813032e-01 >> 15 KSP preconditioned resid norm 1.616570395103e+17 true resid norm >> 4.351134639515e+16 ||r(i)||/||b|| 8.318518854587e-01 >> 16 KSP preconditioned resid norm 4.128330778786e+16 true resid norm >> 7.104207079385e+15 ||r(i)||/||b|| 1.358185517866e-01 >> 17 KSP preconditioned resid norm 2.161181135112e+16 true resid norm >> 5.859480309696e+15 ||r(i)||/||b|| 1.120218091889e-01 >> 18 KSP preconditioned resid norm 1.410375919962e+16 true resid norm >> 2.425065017616e+15 ||r(i)||/||b|| 4.636250252850e-02 >> 19 KSP preconditioned resid norm 1.082544604615e+16 true resid norm >> 2.485306543509e+15 ||r(i)||/||b|| 4.751420274116e-02 >> 20 KSP preconditioned resid norm 8.996941383859e+15 true resid norm >> 1.577914541048e+15 ||r(i)||/||b|| 3.016664145813e-02 >> 21 KSP preconditioned resid norm 7.748060569447e+15 true resid norm >> 1.336297549587e+15 ||r(i)||/||b|| 2.554739690340e-02 >> 22 KSP preconditioned resid norm 6.873596381580e+15 true resid norm >> 1.173018262854e+15 ||r(i)||/||b|| 2.242581612555e-02 >> 23 KSP preconditioned resid norm 6.175955090548e+15 true resid norm >> 1.012331331012e+15 ||r(i)||/||b|| 1.935379610558e-02 >> 24 KSP preconditioned resid norm 5.614288705672e+15 true resid norm >> 9.369191861544e+14 ||r(i)||/||b|| 1.791206331440e-02 >> 25 KSP preconditioned resid norm 5.220540757763e+15 true resid norm >> 8.444260437985e+14 ||r(i)||/||b|| 1.614377524163e-02 >> 26 KSP preconditioned resid norm 5.059721722058e+15 true resid norm >> 8.452379209101e+14 ||r(i)||/||b|| 1.615929674492e-02 >> 27 KSP preconditioned resid norm 4.893746226621e+15 true resid norm >> 9.389168158486e+14 ||r(i)||/||b|| 1.795025408912e-02 >> 28 KSP preconditioned resid norm 4.794914716345e+15 true resid norm >> 7.493396637157e+14 ||r(i)||/||b|| 1.432590953288e-02 >> 29 KSP preconditioned resid norm 4.860853428688e+15 true resid norm >> 7.693465766642e+14 ||r(i)||/||b|| 1.470840259819e-02 >> 30 KSP preconditioned resid norm 5.112203028306e+15 true resid norm >> 8.019578406020e+14 ||r(i)||/||b|| 1.533186621496e-02 >> 31 KSP preconditioned resid norm 5.649006086453e+15 true resid norm >> 8.938106702489e+14 ||r(i)||/||b|| 1.708791276044e-02 >> 32 KSP preconditioned resid norm 8.383819984522e+15 true resid norm >> 1.347106201218e+15 ||r(i)||/||b|| 2.575403719341e-02 >> 33 KSP preconditioned resid norm 1.617520068743e+16 true resid norm >> 2.603058585140e+15 ||r(i)||/||b|| 4.976539159104e-02 >> 34 KSP preconditioned resid norm 3.383001122808e+16 true resid norm >> 5.541435636245e+15 ||r(i)||/||b|| 1.059414167582e-01 >> 35 KSP preconditioned resid norm 6.256166371964e+15 true resid norm >> 1.029049029941e+15 ||r(i)||/||b|| 1.967340582872e-02 >> 36 KSP preconditioned resid norm 3.290396673886e+15 true resid norm >> 5.409340387326e+14 ||r(i)||/||b|| 1.034160138236e-02 >> 37 KSP preconditioned resid norm 2.075346751579e+15 true resid norm >> 3.809761571158e+14 ||r(i)||/||b|| 7.283519377531e-03 >> 38 KSP preconditioned resid norm 1.750045982315e+15 true resid norm >> 2.837816436037e+14 ||r(i)||/||b|| 5.425350278670e-03 >> 39 KSP preconditioned resid norm 1.596755323762e+15 true resid norm >> 2.766061504317e+14 ||r(i)||/||b|| 5.288168876146e-03 >> 40 KSP preconditioned resid norm 1.469812007135e+15 true resid norm >> 2.359521384401e+14 ||r(i)||/||b|| 4.510943638858e-03 >> 41 KSP preconditioned resid norm 1.314668041488e+15 true resid norm >> 2.158813968118e+14 ||r(i)||/||b|| 4.127230293966e-03 >> 42 KSP preconditioned resid norm 1.190446489712e+15 true resid norm >> 1.941922420158e+14 ||r(i)||/||b|| 3.712576053042e-03 >> 43 KSP preconditioned resid norm 1.100371263474e+15 true resid norm >> 1.837040226442e+14 ||r(i)||/||b|| 3.512061801423e-03 >> 44 KSP preconditioned resid norm 1.032027622009e+15 true resid norm >> 1.624908335172e+14 ||r(i)||/||b|| 3.106507093655e-03 >> 45 KSP preconditioned resid norm 8.066010465859e+14 true resid norm >> 2.499636130742e+14 ||r(i)||/||b|| 4.778815643705e-03 >> 46 KSP preconditioned resid norm 6.546087590187e+14 true resid norm >> 1.014535455422e+14 ||r(i)||/||b|| 1.939593465560e-03 >> 47 KSP preconditioned resid norm 6.561404017041e+14 true resid norm >> 1.083911025925e+14 ||r(i)||/||b|| 2.072225994564e-03 >> 48 KSP preconditioned resid norm 6.834623592895e+14 true resid norm >> 1.073483632090e+14 ||r(i)||/||b|| 2.052290855937e-03 >> 49 KSP preconditioned resid norm 8.808387787841e+14 true resid norm >> 1.369903526339e+14 ||r(i)||/||b|| 2.618987748466e-03 >> 50 KSP preconditioned resid norm 3.995486126369e+14 true resid norm >> 1.237415046233e+14 ||r(i)||/||b|| 2.365695673849e-03 >> 51 KSP preconditioned resid norm 3.217227010099e+14 true resid norm >> 4.935649374236e+13 ||r(i)||/||b|| 9.435996764234e-04 >> 0 KSP preconditioned resid norm 6.543260687832e+18 true resid norm >> 3.546137973950e+18 ||r(i)||/||b|| 4.156987252753e+02 >> 1 KSP preconditioned resid norm 1.591265056482e+18 true resid norm >> 5.223494588784e+17 ||r(i)||/||b|| 6.123281321796e+01 >> 2 KSP preconditioned resid norm 6.570987793466e+17 true resid norm >> 1.552598280892e+17 ||r(i)||/||b|| 1.820045161730e+01 >> 3 KSP preconditioned resid norm 4.166120135488e+17 true resid norm >> 8.739095834512e+16 ||r(i)||/||b|| 1.024447166228e+01 >> 4 KSP preconditioned resid norm 2.995238174583e+17 true resid norm >> 4.935051588719e+16 ||r(i)||/||b|| 5.785151817748e+00 >> 5 KSP preconditioned resid norm 2.594691834001e+17 true resid norm >> 4.470128066210e+16 ||r(i)||/||b|| 5.240141676921e+00 >> 6 KSP preconditioned resid norm 2.219773011977e+17 true resid norm >> 4.652509796902e+16 ||r(i)||/||b|| 5.453940050023e+00 >> 7 KSP preconditioned resid norm 1.675026903695e+17 true resid norm >> 2.988993956885e+16 ||r(i)||/||b|| 3.503870934691e+00 >> 8 KSP preconditioned resid norm 1.081971008730e+17 true resid norm >> 1.908275119338e+16 ||r(i)||/||b|| 2.236990044975e+00 >> 9 KSP preconditioned resid norm 5.710427760412e+16 true resid norm >> 9.635681698385e+15 ||r(i)||/||b|| 1.129550127097e+00 >> 10 KSP preconditioned resid norm 1.779335357179e+16 true resid norm >> 2.952424081259e+15 ||r(i)||/||b|| 3.461001619416e-01 >> 11 KSP preconditioned resid norm 1.125492898955e+16 true resid norm >> 2.030196257790e+15 ||r(i)||/||b|| 2.379913028263e-01 >> 12 KSP preconditioned resid norm 1.086499382749e+16 true resid norm >> 1.877721745252e+15 ||r(i)||/||b|| 2.201173619461e-01 >> 13 KSP preconditioned resid norm 1.075258231290e+16 true resid norm >> 1.822442508498e+15 ||r(i)||/||b|| 2.136372113085e-01 >> 14 KSP preconditioned resid norm 1.173739259962e+16 true resid norm >> 2.130298543993e+15 ||r(i)||/||b|| 2.497258695795e-01 >> 15 KSP preconditioned resid norm 4.943363278431e+16 true resid norm >> 1.361259414632e+16 ||r(i)||/||b|| 1.595746718229e+00 >> 16 KSP preconditioned resid norm 1.782643390767e+17 true resid norm >> 3.375538781586e+16 ||r(i)||/||b|| 3.957001049961e+00 >> 17 KSP preconditioned resid norm 3.253381322946e+16 true resid norm >> 8.720742251900e+15 ||r(i)||/||b|| 1.022295653525e+00 >> 18 KSP preconditioned resid norm 2.077141437527e+16 true resid norm >> 3.948414579285e+15 ||r(i)||/||b|| 4.628559067708e-01 >> 19 KSP preconditioned resid norm 1.484650052668e+16 true resid norm >> 3.391990202104e+15 ||r(i)||/||b|| 3.976286352982e-01 >> 20 KSP preconditioned resid norm 1.135224344133e+16 true resid norm >> 2.492450462789e+15 ||r(i)||/||b|| 2.921794041305e-01 >> 21 KSP preconditioned resid norm 8.560056764067e+15 true resid norm >> 1.658566536392e+15 ||r(i)||/||b|| 1.944267256456e-01 >> 22 KSP preconditioned resid norm 6.591497757947e+15 true resid norm >> 1.287448297256e+15 ||r(i)||/||b|| 1.509221073626e-01 >> 23 KSP preconditioned resid norm 4.783676751639e+15 true resid norm >> 9.069671731055e+14 ||r(i)||/||b|| 1.063199177517e-01 >> 24 KSP preconditioned resid norm 3.029245298062e+15 true resid norm >> 6.184462851213e+14 ||r(i)||/||b|| 7.249783687627e-02 >> 25 KSP preconditioned resid norm 1.567581368038e+15 true resid norm >> 2.751547089998e+14 ||r(i)||/||b|| 3.225522036225e-02 >> 26 KSP preconditioned resid norm 1.174427256669e+15 true resid norm >> 2.270458347820e+14 ||r(i)||/||b|| 2.661562093503e-02 >> 27 KSP preconditioned resid norm 1.068689317098e+15 true resid norm >> 2.026010487900e+14 ||r(i)||/||b|| 2.375006227624e-02 >> 28 KSP preconditioned resid norm 1.006302654001e+15 true resid norm >> 1.659657859887e+14 ||r(i)||/||b|| 1.945546568735e-02 >> 29 KSP preconditioned resid norm 9.398219810967e+14 true resid norm >> 1.788496139471e+14 ||r(i)||/||b|| 2.096578223407e-02 >> 30 KSP preconditioned resid norm 8.922669862958e+14 true resid norm >> 1.448103190511e+14 ||r(i)||/||b|| 1.697549996037e-02 >> 31 KSP preconditioned resid norm 8.611553511040e+14 true resid norm >> 1.569494235094e+14 ||r(i)||/||b|| 1.839851572749e-02 >> 32 KSP preconditioned resid norm 8.275858647114e+14 true resid norm >> 1.327021986732e+14 ||r(i)||/||b|| 1.555611632568e-02 >> 33 KSP preconditioned resid norm 7.965262562283e+14 true resid norm >> 1.578752878362e+14 ||r(i)||/||b|| 1.850705087847e-02 >> 34 KSP preconditioned resid norm 7.584298711071e+14 true resid norm >> 1.400338111929e+14 ||r(i)||/||b|| 1.641557018817e-02 >> 35 KSP preconditioned resid norm 7.453216416858e+14 true resid norm >> 1.168697043482e+14 ||r(i)||/||b|| 1.370014011799e-02 >> 36 KSP preconditioned resid norm 7.552391370873e+14 true resid norm >> 1.200616262811e+14 ||r(i)||/||b|| 1.407431559803e-02 >> 37 KSP preconditioned resid norm 7.900785942536e+14 true resid norm >> 1.248526226554e+14 ||r(i)||/||b|| 1.463594379756e-02 >> 38 KSP preconditioned resid norm 8.292853068973e+14 true resid norm >> 1.319608976445e+14 ||r(i)||/||b|| 1.546921674790e-02 >> 39 KSP preconditioned resid norm 8.694401463969e+14 true resid norm >> 1.375708051207e+14 ||r(i)||/||b|| 1.612684242516e-02 >> 40 KSP preconditioned resid norm 9.345453545993e+14 true resid norm >> 1.486706547594e+14 ||r(i)||/||b|| 1.742803075440e-02 >> 41 KSP preconditioned resid norm 1.186246552213e+15 true resid norm >> 1.878863391090e+14 ||r(i)||/||b|| 2.202511922492e-02 >> 42 KSP preconditioned resid norm 2.140906233315e+15 true resid norm >> 3.526342198473e+14 ||r(i)||/||b|| 4.133781504155e-02 >> 43 KSP preconditioned resid norm 1.599648547908e+16 true resid norm >> 2.542842342413e+15 ||r(i)||/||b|| 2.980866306056e-01 >> 44 KSP preconditioned resid norm 1.201331553169e+15 true resid norm >> 1.958292468332e+14 ||r(i)||/||b|| 2.295623263342e-02 >> 45 KSP preconditioned resid norm 6.227596295745e+14 true resid norm >> 9.990152835185e+13 ||r(i)||/||b|| 1.171103276127e-02 >> 46 KSP preconditioned resid norm 4.228157632423e+14 true resid norm >> 6.914757907633e+13 ||r(i)||/||b|| 8.105877630555e-03 >> 47 KSP preconditioned resid norm 2.986102023248e+14 true resid norm >> 4.782856914894e+13 ||r(i)||/||b|| 5.606740452011e-03 >> 48 KSP preconditioned resid norm 1.831310965020e+14 true resid norm >> 3.038059690813e+13 ||r(i)||/||b|| 3.561388615047e-03 >> 49 KSP preconditioned resid norm 1.519167951348e+14 true resid norm >> 2.657532215634e+13 ||r(i)||/||b|| 3.115312383591e-03 >> 50 KSP preconditioned resid norm 1.784468495322e+14 true resid norm >> 2.872094801120e+13 ||r(i)||/||b|| 3.366835008863e-03 >> 51 KSP preconditioned resid norm 1.082269784640e+14 true resid norm >> 1.734196241756e+13 ||r(i)||/||b|| 2.032924754679e-03 >> 52 KSP preconditioned resid norm 7.941561433572e+13 true resid norm >> 2.262252591710e+13 ||r(i)||/||b|| 2.651942833395e-03 >> 53 KSP preconditioned resid norm 1.279466501710e+14 true resid norm >> 2.100188573520e+13 ||r(i)||/||b|| 2.461962053545e-03 >> 54 KSP preconditioned resid norm 1.148448766150e+14 true resid norm >> 1.892568589516e+13 ||r(i)||/||b|| 2.218577945746e-03 >> 55 KSP preconditioned resid norm 4.111681106774e+13 true resid norm >> 1.052257112253e+13 ||r(i)||/||b|| 1.233516415432e-03 >> 0 KSP preconditioned resid norm 1.892564880723e+19 true resid norm >> 1.028289393047e+19 ||r(i)||/||b|| 7.840877673877e+01 >> 1 KSP preconditioned resid norm 4.581010831586e+18 true resid norm >> 1.519789346059e+18 ||r(i)||/||b|| 1.158864657468e+01 >> 2 KSP preconditioned resid norm 1.874273717290e+18 true resid norm >> 4.647134076709e+17 ||r(i)||/||b|| 3.543517036739e+00 >> 3 KSP preconditioned resid norm 1.173044525535e+18 true resid norm >> 2.550993039028e+17 ||r(i)||/||b|| 1.945174627025e+00 >> 4 KSP preconditioned resid norm 8.299292116891e+17 true resid norm >> 1.381054316961e+17 ||r(i)||/||b|| 1.053076889978e+00 >> 5 KSP preconditioned resid norm 7.051404070011e+17 true resid norm >> 1.287518923939e+17 ||r(i)||/||b|| 9.817545968746e-01 >> 6 KSP preconditioned resid norm 5.863125538322e+17 true resid norm >> 1.166548434193e+17 ||r(i)||/||b|| 8.895125861467e-01 >> 7 KSP preconditioned resid norm 4.330808264262e+17 true resid norm >> 8.612437830885e+16 ||r(i)||/||b|| 6.567127110566e-01 >> 8 KSP preconditioned resid norm 2.702964509584e+17 true resid norm >> 4.804107972053e+16 ||r(i)||/||b|| 3.663212243137e-01 >> 9 KSP preconditioned resid norm 1.547354820777e+17 true resid norm >> 2.879372380790e+16 ||r(i)||/||b|| 2.195569337580e-01 >> 10 KSP preconditioned resid norm 9.913511217220e+16 true resid norm >> 1.649311378149e+16 ||r(i)||/||b|| 1.257627361485e-01 >> 11 KSP preconditioned resid norm 9.246165206306e+16 true resid norm >> 1.559859930326e+16 ||r(i)||/||b|| 1.189419144530e-01 >> 12 KSP preconditioned resid norm 9.248856361042e+16 true resid norm >> 1.507587985844e+16 ||r(i)||/||b|| 1.149560917339e-01 >> 13 KSP preconditioned resid norm 7.088634726331e+16 true resid norm >> 1.150049603396e+16 ||r(i)||/||b|| 8.769319532120e-02 >> 14 KSP preconditioned resid norm 2.014973749241e+17 true resid norm >> 3.263663179758e+16 ||r(i)||/||b|| 2.488597464319e-01 >> 15 KSP preconditioned resid norm 1.687600147354e+16 true resid norm >> 4.438944439784e+15 ||r(i)||/||b|| 3.384768975431e-02 >> 16 KSP preconditioned resid norm 1.189486995696e+16 true resid norm >> 1.895011071195e+15 ||r(i)||/||b|| 1.444977464550e-02 >> 17 KSP preconditioned resid norm 1.038277428537e+16 true resid norm >> 2.030560404119e+15 ||r(i)||/||b|| 1.548336085714e-02 >> 18 KSP preconditioned resid norm 9.585935517274e+15 true resid norm >> 1.570427582757e+15 ||r(i)||/||b|| 1.197477155297e-02 >> 19 KSP preconditioned resid norm 8.986106113745e+15 true resid norm >> 1.453955802798e+15 ||r(i)||/||b|| 1.108665485617e-02 >> 20 KSP preconditioned resid norm 8.480772996688e+15 true resid norm >> 1.519414795393e+15 ||r(i)||/||b|| 1.158579056355e-02 >> 21 KSP preconditioned resid norm 8.009707085363e+15 true resid norm >> 1.276353172356e+15 ||r(i)||/||b|| 9.732405255546e-03 >> 22 KSP preconditioned resid norm 7.603487959173e+15 true resid norm >> 1.348218761629e+15 ||r(i)||/||b|| 1.028039232831e-02 >> 23 KSP preconditioned resid norm 7.264676993739e+15 true resid norm >> 1.145141030012e+15 ||r(i)||/||b|| 8.731890843545e-03 >> 24 KSP preconditioned resid norm 6.442617339194e+15 true resid norm >> 1.846348609557e+15 ||r(i)||/||b|| 1.407871528071e-02 >> 25 KSP preconditioned resid norm 5.874716908283e+15 true resid norm >> 9.094105988409e+14 ||r(i)||/||b|| 6.934407093038e-03 >> 26 KSP preconditioned resid norm 5.974445043424e+15 true resid norm >> 9.497988242860e+14 ||r(i)||/||b|| 7.242374030480e-03 >> 27 KSP preconditioned resid norm 6.328543343408e+15 true resid norm >> 9.841709349231e+14 ||r(i)||/||b|| 7.504467091753e-03 >> 28 KSP preconditioned resid norm 1.045930080864e+16 true resid norm >> 1.637386301334e+15 ||r(i)||/||b|| 1.248534292045e-02 >> 29 KSP preconditioned resid norm 1.763968686067e+16 true resid norm >> 3.035143897342e+15 ||r(i)||/||b|| 2.314347710148e-02 >> 30 KSP preconditioned resid norm 3.994992144337e+15 true resid norm >> 6.366779249631e+14 ||r(i)||/||b|| 4.854775086713e-03 >> 31 KSP preconditioned resid norm 2.497471912676e+15 true resid norm >> 4.171284268487e+14 ||r(i)||/||b|| 3.180673642395e-03 >> 32 KSP preconditioned resid norm 1.774822233295e+15 true resid norm >> 3.030553568855e+14 ||r(i)||/||b|| 2.310847508319e-03 >> 33 KSP preconditioned resid norm 1.554413264158e+15 true resid norm >> 2.490797343960e+14 ||r(i)||/||b|| 1.899274408204e-03 >> 34 KSP preconditioned resid norm 1.363683273075e+15 true resid norm >> 2.337415683510e+14 ||r(i)||/||b|| 1.782318340667e-03 >> 35 KSP preconditioned resid norm 1.216181955878e+15 true resid norm >> 1.941708948493e+14 ||r(i)||/||b|| 1.480585372791e-03 >> 36 KSP preconditioned resid norm 1.106171392348e+15 true resid norm >> 1.781889599099e+14 ||r(i)||/||b|| 1.358720460346e-03 >> 37 KSP preconditioned resid norm 9.985678909493e+14 true resid norm >> 1.843602345870e+14 ||r(i)||/||b|| 1.405777456326e-03 >> 38 KSP preconditioned resid norm 9.306373857300e+14 true resid norm >> 1.483324768155e+14 ||r(i)||/||b|| 1.131059810243e-03 >> 39 KSP preconditioned resid norm 8.621292024153e+14 true resid norm >> 1.776747195898e+14 ||r(i)||/||b|| 1.354799292363e-03 >> 0 KSP preconditioned resid norm 6.470804796325e+18 true resid norm >> 3.511545401133e+18 ||r(i)||/||b|| 1.662319590124e+02 >> 1 KSP preconditioned resid norm 1.571603694611e+18 true resid norm >> 5.187449890798e+17 ||r(i)||/||b|| 2.455670820454e+01 >> 2 KSP preconditioned resid norm 6.468415902839e+17 true resid norm >> 1.566954416728e+17 ||r(i)||/||b|| 7.417756930949e+00 >> 3 KSP preconditioned resid norm 4.084120869852e+17 true resid norm >> 8.688409741290e+16 ||r(i)||/||b|| 4.112979349580e+00 >> 4 KSP preconditioned resid norm 2.920961241699e+17 true resid norm >> 4.832079671037e+16 ||r(i)||/||b|| 2.287443213923e+00 >> 5 KSP preconditioned resid norm 2.515909976499e+17 true resid norm >> 4.435653262803e+16 ||r(i)||/||b|| 2.099780145624e+00 >> 6 KSP preconditioned resid norm 2.133092401708e+17 true resid norm >> 4.355815215083e+16 ||r(i)||/||b|| 2.061985859746e+00 >> 7 KSP preconditioned resid norm 1.601275153782e+17 true resid norm >> 2.999237788966e+16 ||r(i)||/||b|| 1.419799877977e+00 >> 8 KSP preconditioned resid norm 1.019875018318e+17 true resid norm >> 1.811096161004e+16 ||r(i)||/||b|| 8.573491964719e-01 >> 9 KSP preconditioned resid norm 5.552203589730e+16 true resid norm >> 9.849208071572e+15 ||r(i)||/||b|| 4.662486072173e-01 >> 10 KSP preconditioned resid norm 2.614195079413e+16 true resid norm >> 4.400774101946e+15 ||r(i)||/||b|| 2.083268807806e-01 >> 11 KSP preconditioned resid norm 2.230316715231e+16 true resid norm >> 4.001821966241e+15 ||r(i)||/||b|| 1.894410093210e-01 >> 12 KSP preconditioned resid norm 2.202763899809e+16 true resid norm >> 3.668851069689e+15 ||r(i)||/||b|| 1.736786032846e-01 >> 13 KSP preconditioned resid norm 2.171352861704e+16 true resid norm >> 3.746836567903e+15 ||r(i)||/||b|| 1.773703346056e-01 >> 14 KSP preconditioned resid norm 2.756407833586e+16 true resid norm >> 4.737724354756e+15 ||r(i)||/||b|| 2.242776643292e-01 >> 15 KSP preconditioned resid norm 1.909305469788e+16 true resid norm >> 7.268311827741e+15 ||r(i)||/||b|| 3.440723601207e-01 >> 16 KSP preconditioned resid norm 8.249997128004e+15 true resid norm >> 1.447767843213e+15 ||r(i)||/||b|| 6.853543305884e-02 >> 17 KSP preconditioned resid norm 5.434797146758e+15 true resid norm >> 1.776552830818e+15 ||r(i)||/||b|| 8.409968364943e-02 >> 18 KSP preconditioned resid norm 4.166874856572e+15 true resid norm >> 7.323760066865e+14 ||r(i)||/||b|| 3.466972071212e-02 >> 19 KSP preconditioned resid norm 3.463504540881e+15 true resid norm >> 7.653814460137e+14 ||r(i)||/||b|| 3.623215497131e-02 >> 20 KSP preconditioned resid norm 3.008102754263e+15 true resid norm >> 5.633011056242e+14 ||r(i)||/||b|| 2.666593639130e-02 >> 21 KSP preconditioned resid norm 2.629337883031e+15 true resid norm >> 5.301916607139e+14 ||r(i)||/||b|| 2.509857864406e-02 >> 22 KSP preconditioned resid norm 2.387457188115e+15 true resid norm >> 3.879435274146e+14 ||r(i)||/||b|| 1.836473836491e-02 >> 23 KSP preconditioned resid norm 2.223797660329e+15 true resid norm >> 3.481194862204e+14 ||r(i)||/||b|| 1.647952042601e-02 >> 24 KSP preconditioned resid norm 2.082583007435e+15 true resid norm >> 3.393016280495e+14 ||r(i)||/||b|| 1.606209457197e-02 >> 25 KSP preconditioned resid norm 1.971105202361e+15 true resid norm >> 3.085385809365e+14 ||r(i)||/||b|| 1.460581222257e-02 >> 26 KSP preconditioned resid norm 1.916167824316e+15 true resid norm >> 3.157123008884e+14 ||r(i)||/||b|| 1.494540672721e-02 >> 27 KSP preconditioned resid norm 1.825444848282e+15 true resid norm >> 4.173235963436e+14 ||r(i)||/||b|| 1.975555233884e-02 >> 28 KSP preconditioned resid norm 1.787001647353e+15 true resid norm >> 2.787303202952e+14 ||r(i)||/||b|| 1.319472821393e-02 >> 29 KSP preconditioned resid norm 1.896717764390e+15 true resid norm >> 3.068021792990e+14 ||r(i)||/||b|| 1.452361324382e-02 >> 30 KSP preconditioned resid norm 2.229137880818e+15 true resid norm >> 3.494609542895e+14 ||r(i)||/||b|| 1.654302376702e-02 >> 31 KSP preconditioned resid norm 3.112887805330e+15 true resid norm >> 5.056849908847e+14 ||r(i)||/||b|| 2.393846499915e-02 >> 32 KSP preconditioned resid norm 2.024054955579e+16 true resid norm >> 3.265471021005e+15 ||r(i)||/||b|| 1.545831202253e-01 >> 33 KSP preconditioned resid norm 2.565517746075e+15 true resid norm >> 4.297071200158e+14 ||r(i)||/||b|| 2.034177212653e-02 >> 34 KSP preconditioned resid norm 1.151189963639e+15 true resid norm >> 1.893961732619e+14 ||r(i)||/||b|| 8.965766724994e-03 >> 35 KSP preconditioned resid norm 7.095661584181e+14 true resid norm >> 1.215463627012e+14 ||r(i)||/||b|| 5.753845579254e-03 >> 36 KSP preconditioned resid norm 5.356379112526e+14 true resid norm >> 8.697359204859e+13 ||r(i)||/||b|| 4.117215908392e-03 >> 37 KSP preconditioned resid norm 4.260660574229e+14 true resid norm >> 9.044225863386e+13 ||r(i)||/||b|| 4.281418040435e-03 >> 38 KSP preconditioned resid norm 3.822193347318e+14 true resid norm >> 6.067358042990e+13 ||r(i)||/||b|| 2.872207812522e-03 >> 39 KSP preconditioned resid norm 3.556139762092e+14 true resid norm >> 7.037316076443e+13 ||r(i)||/||b|| 3.331373238687e-03 >> 40 KSP preconditioned resid norm 3.319733183305e+14 true resid norm >> 5.228450610578e+13 ||r(i)||/||b|| 2.475080024071e-03 >> 41 KSP preconditioned resid norm 3.093370284781e+14 true resid norm >> 5.468964654818e+13 ||r(i)||/||b|| 2.588936221775e-03 >> 42 KSP preconditioned resid norm 2.908716641400e+14 true resid norm >> 4.719126609844e+13 ||r(i)||/||b|| 2.233972714489e-03 >> 43 KSP preconditioned resid norm 2.664499581099e+14 true resid norm >> 6.079882213095e+13 ||r(i)||/||b|| 2.878136590578e-03 >> 44 KSP preconditioned resid norm 2.513459163383e+14 true resid norm >> 3.896503433878e+13 ||r(i)||/||b|| 1.844553679708e-03 >> 45 KSP preconditioned resid norm 2.506735910619e+14 true resid norm >> 4.872960867423e+13 ||r(i)||/||b|| 2.306795836731e-03 >> 46 KSP preconditioned resid norm 2.610309833791e+14 true resid norm >> 4.046617714344e+13 ||r(i)||/||b|| 1.915615813519e-03 >> 47 KSP preconditioned resid norm 3.265892246735e+14 true resid norm >> 6.092635527969e+13 ||r(i)||/||b|| 2.884173842765e-03 >> 48 KSP preconditioned resid norm 6.619675585826e+14 true resid norm >> 1.029872445074e+14 ||r(i)||/||b|| 4.875281237212e-03 >> 49 KSP preconditioned resid norm 6.614927581836e+14 true resid norm >> 1.173864813817e+14 ||r(i)||/||b|| 5.556922247214e-03 >> 50 KSP preconditioned resid norm 2.572515469657e+14 true resid norm >> 4.031945474821e+13 ||r(i)||/||b|| 1.908670167541e-03 >> 51 KSP preconditioned resid norm 1.432897312628e+14 true resid norm >> 2.304000171834e+13 ||r(i)||/||b|| 1.090683497942e-03 >> 52 KSP preconditioned resid norm 8.898633954299e+13 true resid norm >> 1.420071536381e+13 ||r(i)||/||b|| 6.722432617682e-04 >> 0 KSP preconditioned resid norm 6.694445614266e+18 true resid norm >> 3.650633549648e+18 ||r(i)||/||b|| 5.229628447406e+00 >> 1 KSP preconditioned resid norm 1.597624793291e+18 true resid norm >> 5.366629107368e+17 ||r(i)||/||b|| 7.687837156178e-01 >> 2 KSP preconditioned resid norm 6.300333922334e+17 true resid norm >> 1.705382687722e+17 ||r(i)||/||b|| 2.443005493741e-01 >> 3 KSP preconditioned resid norm 3.642099405484e+17 true resid norm >> 9.482515314454e+16 ||r(i)||/||b|| 1.358395225569e-01 >> 4 KSP preconditioned resid norm 2.247296120507e+17 true resid norm >> 3.802694326323e+16 ||r(i)||/||b|| 5.447459504021e-02 >> 5 KSP preconditioned resid norm 1.557575416788e+17 true resid norm >> 3.828791753195e+16 ||r(i)||/||b|| 5.484844753491e-02 >> 6 KSP preconditioned resid norm 1.004780833173e+17 true resid norm >> 1.699813376802e+16 ||r(i)||/||b|| 2.435027309565e-02 >> 7 KSP preconditioned resid norm 7.104370054381e+16 true resid norm >> 1.472894912543e+16 ||r(i)||/||b|| 2.109960649273e-02 >> 8 KSP preconditioned resid norm 5.767747590200e+16 true resid norm >> 9.399551072817e+15 ||r(i)||/||b|| 1.346510380041e-02 >> 9 KSP preconditioned resid norm 4.524165642090e+16 true resid norm >> 1.246569325329e+16 ||r(i)||/||b|| 1.785743300921e-02 >> 10 KSP preconditioned resid norm 3.195588487377e+16 true resid norm >> 5.031148957130e+15 ||r(i)||/||b|| 7.207253029241e-03 >> 11 KSP preconditioned resid norm 2.437445149875e+16 true resid norm >> 4.973530315557e+15 ||r(i)||/||b|| 7.124712811776e-03 >> 12 KSP preconditioned resid norm 1.257635618330e+16 true resid norm >> 1.885290715052e+15 ||r(i)||/||b|| 2.700728468356e-03 >> 13 KSP preconditioned resid norm 2.495249888250e+15 true resid norm >> 4.460762730080e+14 ||r(i)||/||b|| 6.390159777228e-04 >> 14 KSP preconditioned resid norm 7.692041830184e+14 true resid norm >> 1.319158639688e+14 ||r(i)||/||b|| 1.889729400372e-04 >> 15 KSP preconditioned resid norm 5.963386353631e+14 true resid norm >> 1.441107752347e+14 ||r(i)||/||b|| 2.064424707371e-04 >> 16 KSP preconditioned resid norm 3.269005832710e+14 true resid norm >> 5.055770257989e+13 ||r(i)||/||b|| 7.242523689424e-05 >> 17 KSP preconditioned resid norm 1.442136448583e+14 true resid norm >> 7.615238793008e+13 ||r(i)||/||b|| 1.090902959284e-04 >> 18 KSP preconditioned resid norm 5.418830608399e+13 true resid norm >> 8.650648142713e+12 ||r(i)||/||b|| 1.239228068235e-05 >> 19 KSP preconditioned resid norm 3.534476706711e+13 true resid norm >> 1.235947718863e+13 ||r(i)||/||b|| 1.770527570673e-05 >> 0 KSP preconditioned resid norm 4.827689913975e+18 true resid norm >> 2.628955418091e+18 ||r(i)||/||b|| 2.387179080674e+01 >> 1 KSP preconditioned resid norm 1.161014015330e+18 true resid norm >> 3.885001078629e+17 ||r(i)||/||b|| 3.527710374804e+00 >> 2 KSP preconditioned resid norm 4.696713579932e+17 true resid norm >> 1.228966476828e+17 ||r(i)||/||b|| 1.115942493412e+00 >> 3 KSP preconditioned resid norm 2.869928353163e+17 true resid norm >> 6.759586363313e+16 ||r(i)||/||b|| 6.137929555390e-01 >> 4 KSP preconditioned resid norm 1.966510821692e+17 true resid norm >> 3.310379360335e+16 ||r(i)||/||b|| 3.005934716010e-01 >> 5 KSP preconditioned resid norm 1.592497528073e+17 true resid norm >> 3.239157026660e+16 ||r(i)||/||b|| 2.941262464874e-01 >> 6 KSP preconditioned resid norm 1.230094082012e+17 true resid norm >> 2.280559390645e+16 ||r(i)||/||b|| 2.070823883934e-01 >> 7 KSP preconditioned resid norm 8.598887604154e+16 true resid norm >> 1.938013722659e+16 ||r(i)||/||b|| 1.759781008439e-01 >> 8 KSP preconditioned resid norm 5.549487092651e+16 true resid norm >> 9.386421691835e+15 ||r(i)||/||b|| 8.523183524123e-02 >> 9 KSP preconditioned resid norm 4.475548308156e+16 true resid norm >> 7.579602548804e+15 ||r(i)||/||b|| 6.882531563606e-02 >> 10 KSP preconditioned resid norm 4.379582896462e+16 true resid norm >> 6.844431748768e+15 ||r(i)||/||b|| 6.214971991278e-02 >> 11 KSP preconditioned resid norm 4.081432560058e+16 true resid norm >> 7.719907534397e+15 ||r(i)||/||b|| 7.009933163578e-02 >> 12 KSP preconditioned resid norm 4.084955665921e+16 true resid norm >> 6.376903215279e+15 ||r(i)||/||b|| 5.790440511177e-02 >> 13 KSP preconditioned resid norm 7.513638228384e+16 true resid norm >> 1.130789675242e+16 ||r(i)||/||b|| 1.026794687656e-01 >> 14 KSP preconditioned resid norm 2.195838265499e+15 true resid norm >> 3.518796225338e+14 ||r(i)||/||b|| 3.195184171050e-03 >> 15 KSP preconditioned resid norm 2.237408970122e+15 true resid norm >> 3.676139977919e+14 ||r(i)||/||b|| 3.338057538948e-03 >> 16 KSP preconditioned resid norm 1.675719433917e+15 true resid norm >> 3.658174858611e+14 ||r(i)||/||b|| 3.321744612262e-03 >> 17 KSP preconditioned resid norm 1.403876583239e+15 true resid norm >> 2.164126267214e+14 ||r(i)||/||b|| 1.965098729891e-03 >> 18 KSP preconditioned resid norm 1.133854982860e+15 true resid norm >> 2.692061619571e+14 ||r(i)||/||b|| 2.444481613457e-03 >> 19 KSP preconditioned resid norm 9.034784290513e+14 true resid norm >> 1.408322491509e+14 ||r(i)||/||b|| 1.278803728446e-03 >> 20 KSP preconditioned resid norm 7.964711449547e+14 true resid norm >> 1.641000281547e+14 ||r(i)||/||b|| 1.490082911460e-03 >> 21 KSP preconditioned resid norm 7.094597664828e+14 true resid norm >> 1.088610845746e+14 ||r(i)||/||b|| 9.884949056480e-04 >> 22 KSP preconditioned resid norm 6.287860765855e+14 true resid norm >> 1.439087926971e+14 ||r(i)||/||b|| 1.306739768531e-03 >> 23 KSP preconditioned resid norm 5.617871594119e+14 true resid norm >> 8.640965180691e+13 ||r(i)||/||b|| 7.846284183530e-04 >> 24 KSP preconditioned resid norm 4.992037484390e+14 true resid norm >> 1.232499062518e+14 ||r(i)||/||b|| 1.119150198876e-03 >> 25 KSP preconditioned resid norm 3.174300847299e+14 true resid norm >> 1.199443314096e+14 ||r(i)||/||b|| 1.089134478340e-03 >> 26 KSP preconditioned resid norm 2.311461716948e+14 true resid norm >> 3.806900580053e+13 ||r(i)||/||b|| 3.456792520850e-04 >> 27 KSP preconditioned resid norm 2.486080516907e+14 true resid norm >> 3.795889860412e+13 ||r(i)||/||b|| 3.446794420688e-04 >> 28 KSP preconditioned resid norm 3.709940932117e+14 true resid norm >> 6.158317563060e+13 ||r(i)||/||b|| 5.591957458661e-04 >> 29 KSP preconditioned resid norm 5.526902041486e+14 true resid norm >> 8.543527855752e+13 ||r(i)||/||b|| 7.757807847199e-04 >> 30 KSP preconditioned resid norm 1.980272137271e+15 true resid norm >> 3.202004161054e+14 ||r(i)||/||b|| 2.907526425475e-03 >> 31 KSP preconditioned resid norm 4.796148387368e+14 true resid norm >> 7.312671161483e+13 ||r(i)||/||b|| 6.640148973394e-04 >> 32 KSP preconditioned resid norm 1.482366487345e+14 true resid norm >> 3.206675205939e+13 ||r(i)||/||b|| 2.911767889807e-04 >> 33 KSP preconditioned resid norm 7.403721045502e+13 true resid norm >> 1.122274917558e+13 ||r(i)||/||b|| 1.019063004083e-04 >> 34 KSP preconditioned resid norm 3.421203410880e+13 true resid norm >> 9.812523407473e+12 ||r(i)||/||b|| 8.910098073838e-05 >> 35 KSP preconditioned resid norm 1.809542403951e+13 true resid norm >> 2.774316021276e+12 ||r(i)||/||b|| 2.519171349804e-05 >> 36 KSP preconditioned resid norm 1.070863017107e+13 true resid norm >> 1.674563212179e+12 ||r(i)||/||b|| 1.520559170335e-05 >> 37 KSP preconditioned resid norm 6.984609262190e+12 true resid norm >> 1.277736482286e+12 ||r(i)||/||b|| 1.160227282721e-05 >> 0 KSP preconditioned resid norm 9.569153877844e+15 true resid norm >> 5.893805322157e+15 ||r(i)||/||b|| 1.119437717455e-01 >> 1 KSP preconditioned resid norm 4.594033327917e+15 true resid norm >> 8.149731925277e+14 ||r(i)||/||b|| 1.547916296116e-02 >> 2 KSP preconditioned resid norm 3.788286152976e+15 true resid norm >> 6.151807578016e+14 ||r(i)||/||b|| 1.168441279773e-02 >> 3 KSP preconditioned resid norm 3.712195492568e+15 true resid norm >> 6.206672982073e+14 ||r(i)||/||b|| 1.178862119847e-02 >> 4 KSP preconditioned resid norm 3.886667577513e+15 true resid norm >> 6.330116967950e+14 ||r(i)||/||b|| 1.202308407301e-02 >> 5 KSP preconditioned resid norm 4.317240607103e+15 true resid norm >> 6.816590875948e+14 ||r(i)||/||b|| 1.294706647725e-02 >> 6 KSP preconditioned resid norm 5.249548366505e+15 true resid norm >> 7.833242603614e+14 ||r(i)||/||b|| 1.487804014750e-02 >> 7 KSP preconditioned resid norm 6.521511569741e+15 true resid norm >> 1.182301409963e+15 ||r(i)||/||b|| 2.245599777001e-02 >> 8 KSP preconditioned resid norm 7.063327681244e+15 true resid norm >> 1.110968116855e+15 ||r(i)||/||b|| 2.110113152571e-02 >> 9 KSP preconditioned resid norm 7.059112088082e+15 true resid norm >> 1.341815038025e+15 ||r(i)||/||b|| 2.548571392011e-02 >> 10 KSP preconditioned resid norm 6.417041101993e+15 true resid norm >> 1.020408664201e+15 ||r(i)||/||b|| 1.938109393654e-02 >> 11 KSP preconditioned resid norm 4.850416656929e+15 true resid norm >> 8.501719778107e+14 ||r(i)||/||b|| 1.614770977770e-02 >> 12 KSP preconditioned resid norm 2.420686956668e+15 true resid norm >> 3.735196627993e+14 ||r(i)||/||b|| 7.094431795646e-03 >> 13 KSP preconditioned resid norm 1.348084459782e+15 true resid norm >> 2.753333760274e+14 ||r(i)||/||b|| 5.229534216894e-03 >> 14 KSP preconditioned resid norm 1.225904445555e+15 true resid norm >> 2.239730314894e+14 ||r(i)||/||b|| 4.254023426925e-03 >> 15 KSP preconditioned resid norm 1.156415256406e+15 true resid norm >> 2.064224387096e+14 ||r(i)||/||b|| 3.920676897010e-03 >> 16 KSP preconditioned resid norm 1.062658328422e+15 true resid norm >> 1.696405409630e+14 ||r(i)||/||b|| 3.222061292889e-03 >> 17 KSP preconditioned resid norm 9.163863908341e+14 true resid norm >> 1.701529770139e+14 ||r(i)||/||b|| 3.231794227925e-03 >> 18 KSP preconditioned resid norm 8.234055963962e+14 true resid norm >> 1.322040551676e+14 ||r(i)||/||b|| 2.511012795057e-03 >> 19 KSP preconditioned resid norm 7.097124944641e+14 true resid norm >> 1.155726966735e+14 ||r(i)||/||b|| 2.195125707289e-03 >> 20 KSP preconditioned resid norm 6.309033027960e+14 true resid norm >> 1.023450925395e+14 ||r(i)||/||b|| 1.943887701115e-03 >> 21 KSP preconditioned resid norm 5.667708572251e+14 true resid norm >> 8.941024380489e+13 ||r(i)||/||b|| 1.698210133710e-03 >> 22 KSP preconditioned resid norm 5.205793964231e+14 true resid norm >> 8.689007901528e+13 ||r(i)||/||b|| 1.650343477697e-03 >> 23 KSP preconditioned resid norm 4.890134322830e+14 true resid norm >> 7.566844928998e+13 ||r(i)||/||b|| 1.437205871699e-03 >> 24 KSP preconditioned resid norm 4.617102016960e+14 true resid norm >> 8.638231658778e+13 ||r(i)||/||b|| 1.640699310952e-03 >> 25 KSP preconditioned resid norm 4.386224186730e+14 true resid norm >> 6.736465167645e+13 ||r(i)||/||b|| 1.279488001179e-03 >> 26 KSP preconditioned resid norm 4.119635069909e+14 true resid norm >> 8.360600093542e+13 ||r(i)||/||b|| 1.587967463073e-03 >> 27 KSP preconditioned resid norm 3.764202708342e+14 true resid norm >> 5.766864517117e+13 ||r(i)||/||b|| 1.095327262956e-03 >> 28 KSP preconditioned resid norm 3.345193309034e+14 true resid norm >> 7.248849449141e+13 ||r(i)||/||b|| 1.376807518738e-03 >> 0 KSP preconditioned resid norm 9.592548277384e+15 true resid norm >> 5.933564471549e+15 ||r(i)||/||b|| 5.704186299543e-01 >> 1 KSP preconditioned resid norm 4.627892277467e+15 true resid norm >> 8.123960687021e+14 ||r(i)||/||b|| 7.809906755228e-02 >> 2 KSP preconditioned resid norm 3.843955835291e+15 true resid norm >> 6.217930929753e+14 ||r(i)||/||b|| 5.977559794129e-02 >> 3 KSP preconditioned resid norm 3.802651796154e+15 true resid norm >> 6.334637368548e+14 ||r(i)||/||b|| 6.089754626161e-02 >> 4 KSP preconditioned resid norm 4.025480937197e+15 true resid norm >> 6.491593110279e+14 ||r(i)||/||b|| 6.240642814182e-02 >> 5 KSP preconditioned resid norm 4.519579789539e+15 true resid norm >> 7.102235297384e+14 ||r(i)||/||b|| 6.827678956506e-02 >> 6 KSP preconditioned resid norm 5.567308133052e+15 true resid norm >> 8.274501588234e+14 ||r(i)||/||b|| 7.954628085944e-02 >> 7 KSP preconditioned resid norm 7.053789984861e+15 true resid norm >> 1.260366188593e+15 ||r(i)||/||b|| 1.211643284547e-01 >> 8 KSP preconditioned resid norm 7.835759896085e+15 true resid norm >> 1.231567591968e+15 ||r(i)||/||b|| 1.183957976482e-01 >> 9 KSP preconditioned resid norm 8.126559485840e+15 true resid norm >> 1.514279927927e+15 ||r(i)||/||b|| 1.455741293445e-01 >> 10 KSP preconditioned resid norm 7.865908148994e+15 true resid norm >> 1.247217430777e+15 ||r(i)||/||b|| 1.199002827947e-01 >> 11 KSP preconditioned resid norm 6.705237228445e+15 true resid norm >> 1.156150868849e+15 ||r(i)||/||b|| 1.111456693176e-01 >> 12 KSP preconditioned resid norm 3.837356917105e+15 true resid norm >> 5.892678306289e+14 ||r(i)||/||b|| 5.664880700887e-02 >> 13 KSP preconditioned resid norm 1.588477046031e+15 true resid norm >> 3.087098247775e+14 ||r(i)||/||b|| 2.967758017080e-02 >> 14 KSP preconditioned resid norm 1.304101915863e+15 true resid norm >> 2.634312731694e+14 ||r(i)||/||b|| 2.532476164182e-02 >> 15 KSP preconditioned resid norm 1.188980488344e+15 true resid norm >> 2.212236776838e+14 ||r(i)||/||b|| 2.126716710384e-02 >> 16 KSP preconditioned resid norm 1.053945473260e+15 true resid norm >> 1.687417173497e+14 ||r(i)||/||b|| 1.622185444993e-02 >> 17 KSP preconditioned resid norm 8.630322699779e+14 true resid norm >> 2.032342536888e+14 ||r(i)||/||b|| 1.953776774564e-02 >> 18 KSP preconditioned resid norm 7.497367164260e+14 true resid norm >> 1.258314731452e+14 ||r(i)||/||b|| 1.209671132095e-02 >> 19 KSP preconditioned resid norm 6.482413928213e+14 true resid norm >> 1.007668116028e+14 ||r(i)||/||b|| 9.687139474922e-03 >> 20 KSP preconditioned resid norm 5.423124085977e+14 true resid norm >> 1.262959277696e+14 ||r(i)||/||b|| 1.214136130695e-02 >> 21 KSP preconditioned resid norm 4.512195524384e+14 true resid norm >> 7.029203398299e+13 ||r(i)||/||b|| 6.757470305334e-03 >> 22 KSP preconditioned resid norm 3.576286880164e+14 true resid norm >> 9.097241124393e+13 ||r(i)||/||b|| 8.745562373884e-03 >> 23 KSP preconditioned resid norm 2.908403868181e+14 true resid norm >> 4.482717971847e+13 ||r(i)||/||b|| 4.309426241567e-03 >> 24 KSP preconditioned resid norm 2.299942905294e+14 true resid norm >> 6.485244464292e+13 ||r(i)||/||b|| 6.234539592478e-03 >> 25 KSP preconditioned resid norm 1.957293334591e+14 true resid norm >> 3.000864324033e+13 ||r(i)||/||b|| 2.884857701642e-03 >> 26 KSP preconditioned resid norm 1.685971736537e+14 true resid norm >> 4.568513443333e+13 ||r(i)||/||b|| 4.391905054321e-03 >> 27 KSP preconditioned resid norm 1.478986486404e+14 true resid norm >> 2.270701271232e+13 ||r(i)||/||b|| 2.182921099758e-03 >> 28 KSP preconditioned resid norm 1.388836935911e+14 true resid norm >> 2.435562734780e+13 ||r(i)||/||b|| 2.341409392284e-03 >> 29 KSP preconditioned resid norm 1.300875297573e+14 true resid norm >> 2.023811842122e+13 ||r(i)||/||b|| 1.945575857149e-03 >> 30 KSP preconditioned resid norm 1.230463362777e+14 true resid norm >> 2.013991197226e+13 ||r(i)||/||b|| 1.936134856156e-03 >> 31 KSP preconditioned resid norm 1.148174878708e+14 true resid norm >> 1.790715664590e+13 ||r(i)||/||b|| 1.721490650234e-03 >> 32 KSP preconditioned resid norm 1.085020096185e+14 true resid norm >> 1.697737976905e+13 ||r(i)||/||b|| 1.632107269502e-03 >> 33 KSP preconditioned resid norm 1.008662380835e+14 true resid norm >> 1.575408064147e+13 ||r(i)||/||b|| 1.514506354280e-03 >> 34 KSP preconditioned resid norm 9.448474542578e+13 true resid norm >> 1.450932025652e+13 ||r(i)||/||b|| 1.394842277686e-03 >> 35 KSP preconditioned resid norm 8.955256800693e+13 true resid norm >> 1.427784032077e+13 ||r(i)||/||b|| 1.372589133148e-03 >> 36 KSP preconditioned resid norm 8.409700445745e+13 true resid norm >> 1.287414873178e+13 ||r(i)||/||b|| 1.237646328210e-03 >> 37 KSP preconditioned resid norm 7.999039854664e+13 true resid norm >> 1.307106382082e+13 ||r(i)||/||b|| 1.256576607951e-03 >> 38 KSP preconditioned resid norm 7.508174679474e+13 true resid norm >> 1.150183263556e+13 ||r(i)||/||b|| 1.105719781996e-03 >> 39 KSP preconditioned resid norm 7.047760131088e+13 true resid norm >> 1.131577846513e+13 ||r(i)||/||b|| 1.087833608263e-03 >> 40 KSP preconditioned resid norm 6.612188631103e+13 true resid norm >> 1.012435424497e+13 ||r(i)||/||b|| 9.732969626060e-04 >> 41 KSP preconditioned resid norm 6.096165352930e+13 true resid norm >> 1.005634567468e+13 ||r(i)||/||b|| 9.667590113164e-04 >> 42 KSP preconditioned resid norm 5.668115744306e+13 true resid norm >> 8.662975702181e+12 ||r(i)||/||b|| 8.328084669947e-04 >> 43 KSP preconditioned resid norm 5.243169528258e+13 true resid norm >> 8.972837494107e+12 ||r(i)||/||b|| 8.625967906361e-04 >> 0 KSP preconditioned resid norm 2.201027933913e+16 true resid norm >> 1.374916449250e+16 ||r(i)||/||b|| 1.042532971848e-01 >> 1 KSP preconditioned resid norm 1.033786235817e+16 true resid norm >> 1.769244185634e+15 ||r(i)||/||b|| 1.341532716246e-02 >> 2 KSP preconditioned resid norm 8.200152533928e+15 true resid norm >> 1.329548327593e+15 ||r(i)||/||b|| 1.008132508660e-02 >> 3 KSP preconditioned resid norm 7.725254307721e+15 true resid norm >> 1.292264285665e+15 ||r(i)||/||b|| 9.798618140627e-03 >> 4 KSP preconditioned resid norm 7.783851928031e+15 true resid norm >> 1.241739377824e+15 ||r(i)||/||b|| 9.415512080968e-03 >> 5 KSP preconditioned resid norm 8.247256237278e+15 true resid norm >> 1.289875346607e+15 ||r(i)||/||b|| 9.780503965494e-03 >> 6 KSP preconditioned resid norm 9.468674539247e+15 true resid norm >> 1.409079612219e+15 ||r(i)||/||b|| 1.068437254131e-02 >> 7 KSP preconditioned resid norm 1.106031993220e+16 true resid norm >> 1.997396506013e+15 ||r(i)||/||b|| 1.514529640334e-02 >> 8 KSP preconditioned resid norm 1.133839709036e+16 true resid norm >> 1.772431554854e+15 ||r(i)||/||b|| 1.343949544924e-02 >> 9 KSP preconditioned resid norm 1.055805824819e+16 true resid norm >> 2.045498232098e+15 ||r(i)||/||b|| 1.551002864197e-02 >> 10 KSP preconditioned resid norm 8.628048224174e+15 true resid norm >> 1.368695682952e+15 ||r(i)||/||b|| 1.037816064155e-02 >> 11 KSP preconditioned resid norm 5.442091241360e+15 true resid norm >> 9.810024540837e+14 ||r(i)||/||b|| 7.438469475027e-03 >> 12 KSP preconditioned resid norm 2.000681531075e+15 true resid norm >> 3.042659606098e+14 ||r(i)||/||b|| 2.307102342980e-03 >> 13 KSP preconditioned resid norm 8.105673271529e+14 true resid norm >> 1.475101242619e+14 ||r(i)||/||b|| 1.118498278992e-03 >> 0 KSP preconditioned resid norm 1.928660996668e+16 true resid norm >> 1.203367101178e+16 ||r(i)||/||b|| 4.946564650623e-01 >> 1 KSP preconditioned resid norm 9.233394049572e+15 true resid norm >> 1.583660413052e+15 ||r(i)||/||b|| 6.509799553373e-02 >> 2 KSP preconditioned resid norm 7.543213441610e+15 true resid norm >> 1.216909563864e+15 ||r(i)||/||b|| 5.002232341004e-02 >> 3 KSP preconditioned resid norm 7.341955893763e+15 true resid norm >> 1.220114169499e+15 ||r(i)||/||b|| 5.015405203166e-02 >> 4 KSP preconditioned resid norm 7.657405519141e+15 true resid norm >> 1.212628294074e+15 ||r(i)||/||b|| 4.984633739730e-02 >> 5 KSP preconditioned resid norm 8.417297612797e+15 true resid norm >> 1.311888034770e+15 ||r(i)||/||b|| 5.392651146949e-02 >> 6 KSP preconditioned resid norm 1.008979814671e+16 true resid norm >> 1.496346817141e+15 ||r(i)||/||b|| 6.150888007071e-02 >> 7 KSP preconditioned resid norm 1.235951387667e+16 true resid norm >> 2.203847820374e+15 ||r(i)||/||b|| 9.059143891287e-02 >> 8 KSP preconditioned resid norm 1.311708980126e+16 true resid norm >> 2.061617788942e+15 ||r(i)||/||b|| 8.474492669688e-02 >> 9 KSP preconditioned resid norm 1.289258338081e+16 true resid norm >> 2.353735302483e+15 ||r(i)||/||b|| 9.675271854106e-02 >> 10 KSP preconditioned resid norm 1.151089236912e+16 true resid norm >> 1.820534806530e+15 ||r(i)||/||b|| 7.483496191971e-02 >> 11 KSP preconditioned resid norm 8.373719062509e+15 true resid norm >> 1.433022884373e+15 ||r(i)||/||b|| 5.890588446727e-02 >> 12 KSP preconditioned resid norm 3.564533969408e+15 true resid norm >> 5.439956922043e+14 ||r(i)||/||b|| 2.236150430334e-02 >> 13 KSP preconditioned resid norm 1.257109874849e+15 true resid norm >> 2.286041152149e+14 ||r(i)||/||b|| 9.397008063476e-03 >> 14 KSP preconditioned resid norm 1.068785016666e+15 true resid norm >> 1.971356784282e+14 ||r(i)||/||b|| 8.103465495567e-03 >> 15 KSP preconditioned resid norm 9.963029863724e+14 true resid norm >> 1.558476632901e+14 ||r(i)||/||b|| 6.406279026229e-03 >> 16 KSP preconditioned resid norm 8.667425302785e+14 true resid norm >> 2.560422780209e+14 ||r(i)||/||b|| 1.052488206037e-02 >> 17 KSP preconditioned resid norm 7.240699142388e+14 true resid norm >> 1.131314963782e+14 ||r(i)||/||b|| 4.650386904448e-03 >> 18 KSP preconditioned resid norm 6.352047441583e+14 true resid norm >> 1.637714957863e+14 ||r(i)||/||b|| 6.731996337968e-03 >> 19 KSP preconditioned resid norm 5.532686184638e+14 true resid norm >> 8.631039548890e+13 ||r(i)||/||b|| 3.547877874413e-03 >> 20 KSP preconditioned resid norm 4.676701949378e+14 true resid norm >> 1.246119427256e+14 ||r(i)||/||b|| 5.122302498784e-03 >> 21 KSP preconditioned resid norm 4.001088470241e+14 true resid norm >> 6.181513171397e+13 ||r(i)||/||b|| 2.540974779106e-03 >> 22 KSP preconditioned resid norm 3.482451502111e+14 true resid norm >> 6.477073783496e+13 ||r(i)||/||b|| 2.662468018741e-03 >> 23 KSP preconditioned resid norm 3.041601141164e+14 true resid norm >> 4.679581096972e+13 ||r(i)||/||b|| 1.923590100755e-03 >> 24 KSP preconditioned resid norm 2.690997224286e+14 true resid norm >> 4.294556821793e+13 ||r(i)||/||b|| 1.765321899192e-03 >> 25 KSP preconditioned resid norm 2.482501027210e+14 true resid norm >> 3.840071688811e+13 ||r(i)||/||b|| 1.578501095230e-03 >> 26 KSP preconditioned resid norm 2.328482920528e+14 true resid norm >> 3.868960801558e+13 ||r(i)||/||b|| 1.590376263145e-03 >> 27 KSP preconditioned resid norm 2.171581409229e+14 true resid norm >> 3.334869397249e+13 ||r(i)||/||b|| 1.370832479858e-03 >> 28 KSP preconditioned resid norm 2.002313534549e+14 true resid norm >> 3.534153915591e+13 ||r(i)||/||b|| 1.452750437635e-03 >> 29 KSP preconditioned resid norm 1.813916789125e+14 true resid norm >> 2.780025874135e+13 ||r(i)||/||b|| 1.142758323985e-03 >> 30 KSP preconditioned resid norm 1.655865002619e+14 true resid norm >> 2.941107681765e+13 ||r(i)||/||b|| 1.208972663292e-03 >> 31 KSP preconditioned resid norm 1.477358437937e+14 true resid norm >> 2.263549873151e+13 ||r(i)||/||b|| 9.304555340168e-04 >> 32 KSP preconditioned resid norm 1.346388933582e+14 true resid norm >> 2.267328899678e+13 ||r(i)||/||b|| 9.320089418685e-04 >> 0 KSP preconditioned resid norm 6.130047183925e+16 true resid norm >> 4.019546453762e+16 ||r(i)||/||b|| 5.748521837148e-02 >> 1 KSP preconditioned resid norm 2.383257388870e+16 true resid norm >> 4.333010621750e+15 ||r(i)||/||b|| 6.196820080636e-03 >> 2 KSP preconditioned resid norm 1.380338515654e+16 true resid norm >> 2.902899468751e+15 ||r(i)||/||b|| 4.151558186756e-03 >> 3 KSP preconditioned resid norm 9.355621435823e+15 true resid norm >> 1.914392486860e+15 ||r(i)||/||b|| 2.737852924995e-03 >> 4 KSP preconditioned resid norm 6.905538449399e+15 true resid norm >> 1.160515328667e+15 ||r(i)||/||b|| 1.659701607116e-03 >> 5 KSP preconditioned resid norm 5.388961375964e+15 true resid norm >> 9.343353155170e+14 ||r(i)||/||b|| 1.336232091419e-03 >> 6 KSP preconditioned resid norm 4.457024362952e+15 true resid norm >> 7.443375401358e+14 ||r(i)||/||b|| 1.064508310303e-03 >> 7 KSP preconditioned resid norm 3.604925656449e+15 true resid norm >> 7.177624609068e+14 ||r(i)||/||b|| 1.026502175773e-03 >> 8 KSP preconditioned resid norm 2.616540301450e+15 true resid norm >> 4.566819450076e+14 ||r(i)||/||b|| 6.531199884630e-04 >> 9 KSP preconditioned resid norm 1.730734442406e+15 true resid norm >> 3.416454932875e+14 ||r(i)||/||b|| 4.886015378397e-04 >> 10 KSP preconditioned resid norm 9.605665765368e+14 true resid norm >> 1.715807715453e+14 ||r(i)||/||b|| 2.453848520993e-04 >> 11 KSP preconditioned resid norm 3.885455022454e+14 true resid norm >> 7.526175493349e+13 ||r(i)||/||b|| 1.076349898463e-04 >> 12 KSP preconditioned resid norm 1.085553513563e+14 true resid norm >> 1.877760955419e+13 ||r(i)||/||b|| 2.685464636705e-05 >> 13 KSP preconditioned resid norm 5.523836175310e+13 true resid norm >> 1.483397872543e+13 ||r(i)||/||b|| 2.121469464673e-05 >> 14 KSP preconditioned resid norm 4.329267915925e+13 true resid norm >> 6.950523101530e+12 ||r(i)||/||b|| 9.940234374291e-06 >> 0 KSP preconditioned resid norm 2.469234905564e+16 true resid norm >> 1.581399294540e+16 ||r(i)||/||b|| 1.421323674244e-01 >> 1 KSP preconditioned resid norm 1.094883401656e+16 true resid norm >> 1.827560974935e+15 ||r(i)||/||b|| 1.642567875658e-02 >> 2 KSP preconditioned resid norm 7.902898352613e+15 true resid norm >> 1.331982197511e+15 ||r(i)||/||b|| 1.197153582609e-02 >> 3 KSP preconditioned resid norm 6.757313480243e+15 true resid norm >> 1.177585457449e+15 ||r(i)||/||b|| 1.058385503836e-02 >> 4 KSP preconditioned resid norm 6.237102240639e+15 true resid norm >> 9.859442147838e+14 ||r(i)||/||b|| 8.861429613598e-03 >> 5 KSP preconditioned resid norm 6.056820105725e+15 true resid norm >> 9.499166634556e+14 ||r(i)||/||b|| 8.537622642110e-03 >> 6 KSP preconditioned resid norm 6.282037371920e+15 true resid norm >> 9.440405134922e+14 ||r(i)||/||b|| 8.484809218675e-03 >> 7 KSP preconditioned resid norm 6.512961686254e+15 true resid norm >> 1.175538616906e+15 ||r(i)||/||b|| 1.056545852755e-02 >> 8 KSP preconditioned resid norm 5.899315381141e+15 true resid norm >> 9.360504236018e+14 ||r(i)||/||b|| 8.412996211297e-03 >> 9 KSP preconditioned resid norm 4.860546577526e+15 true resid norm >> 9.070630816401e+14 ||r(i)||/||b|| 8.152464949358e-03 >> 10 KSP preconditioned resid norm 3.431569954082e+15 true resid norm >> 5.519072654101e+14 ||r(i)||/||b|| 4.960409840975e-03 >> 11 KSP preconditioned resid norm 1.788273694951e+15 true resid norm >> 3.211455497861e+14 ||r(i)||/||b|| 2.886379008546e-03 >> 12 KSP preconditioned resid norm 6.021242921522e+14 true resid norm >> 9.258232372098e+13 ||r(i)||/||b|| 8.321076718289e-04 >> 13 KSP preconditioned resid norm 3.678076314547e+14 true resid norm >> 7.160952479278e+13 ||r(i)||/||b|| 6.436091962400e-04 >> 14 KSP preconditioned resid norm 3.596149520932e+14 true resid norm >> 5.543217762956e+13 ||r(i)||/||b|| 4.982110884443e-04 >> 15 KSP preconditioned resid norm 1.443083471369e+14 true resid norm >> 3.905973798110e+13 ||r(i)||/||b|| 3.510595362853e-04 >> 16 KSP preconditioned resid norm 1.158730309561e+14 true resid norm >> 1.821149714099e+13 ||r(i)||/||b|| 1.636805588525e-04 >> 17 KSP preconditioned resid norm 9.042472136695e+13 true resid norm >> 2.428568415416e+13 ||r(i)||/||b|| 2.182738916901e-04 >> 18 KSP preconditioned resid norm 7.667332005406e+13 true resid norm >> 1.178343148104e+13 ||r(i)||/||b|| 1.059066497984e-04 >> 19 KSP preconditioned resid norm 6.435860750477e+13 true resid norm >> 1.303667702134e+13 ||r(i)||/||b|| 1.171705194752e-04 >> 20 KSP preconditioned resid norm 5.245023814970e+13 true resid norm >> 8.751976411105e+12 ||r(i)||/||b|| 7.866066029293e-05 >> 21 KSP preconditioned resid norm 4.161239918955e+13 true resid norm >> 6.811593020631e+12 ||r(i)||/||b|| 6.122096078432e-05 >> 22 KSP preconditioned resid norm 3.107777724805e+13 true resid norm >> 6.092648679105e+12 ||r(i)||/||b|| 5.475926185348e-05 >> 23 KSP preconditioned resid norm 2.312582950537e+13 true resid norm >> 3.536350440396e+12 ||r(i)||/||b|| 3.178386773481e-05 >> 24 KSP preconditioned resid norm 1.680324737468e+13 true resid norm >> 4.383700522921e+12 ||r(i)||/||b|| 3.939964660118e-05 >> 25 KSP preconditioned resid norm 1.300784122000e+13 true resid norm >> 1.953230538144e+12 ||r(i)||/||b|| 1.755516658383e-05 >> 26 KSP preconditioned resid norm 9.922666654442e+12 true resid norm >> 3.054726768240e+12 ||r(i)||/||b|| 2.745514993611e-05 >> 27 KSP preconditioned resid norm 7.226267347787e+12 true resid norm >> 1.085334598547e+12 ||r(i)||/||b|| 9.754726492653e-06 >> 0 SNES Function norm 0.000000000000e+00 >> 2 TS dt 4e-12 time 5e-12 >> 0 KSP preconditioned resid norm 2.220258026867e+19 true resid norm >> 1.204245784292e+19 ||r(i)||/||b|| 2.302282866910e+02 >> 1 KSP preconditioned resid norm 5.390122646802e+18 true resid norm >> 1.775868988684e+18 ||r(i)||/||b|| 3.395114851018e+01 >> 2 KSP preconditioned resid norm 2.218369826326e+18 true resid norm >> 5.319781926844e+17 ||r(i)||/||b|| 1.017038460556e+01 >> 3 KSP preconditioned resid norm 1.401713665358e+18 true resid norm >> 2.963575488339e+17 ||r(i)||/||b|| 5.665777834223e+00 >> 4 KSP preconditioned resid norm 1.004075064815e+18 true resid norm >> 1.656255011022e+17 ||r(i)||/||b|| 3.166436274762e+00 >> 5 KSP preconditioned resid norm 8.672862633339e+17 true resid norm >> 1.510357773334e+17 ||r(i)||/||b|| 2.887509235913e+00 >> 6 KSP preconditioned resid norm 7.389427240230e+17 true resid norm >> 1.526197236053e+17 ||r(i)||/||b|| 2.917791196717e+00 >> 7 KSP preconditioned resid norm 5.566238898025e+17 true resid norm >> 1.017349876189e+17 ||r(i)||/||b|| 1.944974373301e+00 >> 8 KSP preconditioned resid norm 3.566703993070e+17 true resid norm >> 6.306898440820e+16 ||r(i)||/||b|| 1.205755869196e+00 >> 9 KSP preconditioned resid norm 1.893399021800e+17 true resid norm >> 3.268112455206e+16 ||r(i)||/||b|| 6.247993068277e-01 >> 10 KSP preconditioned resid norm 6.895422631218e+16 true resid norm >> 1.153042493210e+16 ||r(i)||/||b|| 2.204392169408e-01 >> 11 KSP preconditioned resid norm 5.104209415340e+16 true resid norm >> 9.274112790250e+15 ||r(i)||/||b|| 1.773029331827e-01 >> 12 KSP preconditioned resid norm 4.991932910946e+16 true resid norm >> 8.397724827599e+15 ||r(i)||/||b|| 1.605481060744e-01 >> 13 KSP preconditioned resid norm 4.919289629176e+16 true resid norm >> 8.425037065615e+15 ||r(i)||/||b|| 1.610702627509e-01 >> 14 KSP preconditioned resid norm 5.687556145039e+16 true resid norm >> 9.761935947260e+15 ||r(i)||/||b|| 1.866291596982e-01 >> 15 KSP preconditioned resid norm 1.615122498031e+17 true resid norm >> 4.345955527842e+16 ||r(i)||/||b|| 8.308618624718e-01 >> 16 KSP preconditioned resid norm 4.126010760136e+16 true resid norm >> 7.099883585094e+15 ||r(i)||/||b|| 1.357359149456e-01 >> 17 KSP preconditioned resid norm 2.161057719247e+16 true resid norm >> 5.853862804145e+15 ||r(i)||/||b|| 1.119144298865e-01 >> 18 KSP preconditioned resid norm 1.411351808998e+16 true resid norm >> 2.426558230375e+15 ||r(i)||/||b|| 4.639105664496e-02 >> 19 KSP preconditioned resid norm 1.083945264815e+16 true resid norm >> 2.485250997240e+15 ||r(i)||/||b|| 4.751314777723e-02 >> 20 KSP preconditioned resid norm 9.014213050640e+15 true resid norm >> 1.580604299892e+15 ||r(i)||/||b|| 3.021806882344e-02 >> 21 KSP preconditioned resid norm 7.767372671831e+15 true resid norm >> 1.339058517057e+15 ||r(i)||/||b|| 2.560018496078e-02 >> 22 KSP preconditioned resid norm 6.894209717493e+15 true resid norm >> 1.176401403286e+15 ||r(i)||/||b|| 2.249049845740e-02 >> 23 KSP preconditioned resid norm 6.197326579110e+15 true resid norm >> 1.015741229888e+15 ||r(i)||/||b|| 1.941898955585e-02 >> 24 KSP preconditioned resid norm 5.635415722302e+15 true resid norm >> 9.407688590092e+14 ||r(i)||/||b|| 1.798566417314e-02 >> 25 KSP preconditioned resid norm 5.240066189102e+15 true resid norm >> 8.479712706847e+14 ||r(i)||/||b|| 1.621155542826e-02 >> 26 KSP preconditioned resid norm 5.077381341025e+15 true resid norm >> 8.489262350542e+14 ||r(i)||/||b|| 1.622981248288e-02 >> 27 KSP preconditioned resid norm 4.908170126730e+15 true resid norm >> 9.447288544322e+14 ||r(i)||/||b|| 1.806137155559e-02 >> 28 KSP preconditioned resid norm 4.807291179172e+15 true resid norm >> 7.514342780137e+14 ||r(i)||/||b|| 1.436595657171e-02 >> 29 KSP preconditioned resid norm 4.874566711284e+15 true resid norm >> 7.721244730622e+14 ||r(i)||/||b|| 1.476151271311e-02 >> 30 KSP preconditioned resid norm 5.131077173970e+15 true resid norm >> 8.050523689285e+14 ||r(i)||/||b|| 1.539102980576e-02 >> 31 KSP preconditioned resid norm 5.679198106771e+15 true resid norm >> 8.990076632730e+14 ||r(i)||/||b|| 1.718727162987e-02 >> 32 KSP preconditioned resid norm 8.479698883566e+15 true resid norm >> 1.362468331537e+15 ||r(i)||/||b|| 2.604773491692e-02 >> 33 KSP preconditioned resid norm 1.652298710742e+16 true resid norm >> 2.658951324175e+15 ||r(i)||/||b|| 5.083395895960e-02 >> 34 KSP preconditioned resid norm 3.352309602032e+16 true resid norm >> 5.485872403557e+15 ||r(i)||/||b|| 1.048791717564e-01 >> 35 KSP preconditioned resid norm 6.378357733800e+15 true resid norm >> 1.047516366293e+15 ||r(i)||/||b|| 2.002646813784e-02 >> 36 KSP preconditioned resid norm 3.388128874134e+15 true resid norm >> 5.554504439336e+14 ||r(i)||/||b|| 1.061912823085e-02 >> 37 KSP preconditioned resid norm 2.152326148598e+15 true resid norm >> 3.939853481438e+14 ||r(i)||/||b|| 7.532230784419e-03 >> 38 KSP preconditioned resid norm 1.818288964313e+15 true resid norm >> 2.940442100180e+14 ||r(i)||/||b|| 5.621551311776e-03 >> 39 KSP preconditioned resid norm 1.660791920226e+15 true resid norm >> 2.869955671543e+14 ||r(i)||/||b|| 5.486795019399e-03 >> 40 KSP preconditioned resid norm 1.530575944029e+15 true resid norm >> 2.452126997689e+14 ||r(i)||/||b|| 4.687988156493e-03 >> 41 KSP preconditioned resid norm 1.370742276038e+15 true resid norm >> 2.241019638220e+14 ||r(i)||/||b|| 4.284392094026e-03 >> 42 KSP preconditioned resid norm 1.242467362496e+15 true resid norm >> 2.048557595739e+14 ||r(i)||/||b|| 3.916442238011e-03 >> 43 KSP preconditioned resid norm 1.151462192104e+15 true resid norm >> 1.847175725318e+14 ||r(i)||/||b|| 3.531439412157e-03 >> 44 KSP preconditioned resid norm 1.085430472672e+15 true resid norm >> 1.860586924987e+14 ||r(i)||/||b|| 3.557079008015e-03 >> 45 KSP preconditioned resid norm 1.036632239565e+15 true resid norm >> 1.638613644087e+14 ||r(i)||/||b|| 3.132709424833e-03 >> 46 KSP preconditioned resid norm 1.028748911897e+15 true resid norm >> 1.653234224458e+14 ||r(i)||/||b|| 3.160661120519e-03 >> 47 KSP preconditioned resid norm 9.769896259678e+14 true resid norm >> 1.886232302190e+14 ||r(i)||/||b|| 3.606107963166e-03 >> 48 KSP preconditioned resid norm 1.056096824670e+15 true resid norm >> 1.655410058571e+14 ||r(i)||/||b|| 3.164820890613e-03 >> 49 KSP preconditioned resid norm 7.777695958066e+14 true resid norm >> 2.880991932726e+14 ||r(i)||/||b|| 5.507894196468e-03 >> 50 KSP preconditioned resid norm 1.009825961040e+15 true resid norm >> 1.550789761056e+14 ||r(i)||/||b|| 2.964807303982e-03 >> 51 KSP preconditioned resid norm 9.471596692833e+14 true resid norm >> 2.053798093618e+14 ||r(i)||/||b|| 3.926461046995e-03 >> 52 KSP preconditioned resid norm 8.991518749617e+14 true resid norm >> 1.386233583178e+14 ||r(i)||/||b|| 2.650208013776e-03 >> 53 KSP preconditioned resid norm 7.186213371144e+13 true resid norm >> 1.103090149144e+13 ||r(i)||/||b|| 2.108893038414e-04 >> 0 KSP preconditioned resid norm 6.542826642292e+18 true resid norm >> 3.545903145424e+18 ||r(i)||/||b|| 4.156717341790e+02 >> 1 KSP preconditioned resid norm 1.591157510957e+18 true resid norm >> 5.223152980213e+17 ||r(i)||/||b|| 6.122888776501e+01 >> 2 KSP preconditioned resid norm 6.570525108060e+17 true resid norm >> 1.552502546069e+17 ||r(i)||/||b|| 1.819935286374e+01 >> 3 KSP preconditioned resid norm 4.165816250413e+17 true resid norm >> 8.738489300540e+16 ||r(i)||/||b|| 1.024377387845e+01 >> 4 KSP preconditioned resid norm 2.995010683652e+17 true resid norm >> 4.934680552583e+16 ||r(i)||/||b|| 5.784724339013e+00 >> 5 KSP preconditioned resid norm 2.594493782706e+17 true resid norm >> 4.469794627497e+16 ||r(i)||/||b|| 5.239757568206e+00 >> 6 KSP preconditioned resid norm 2.219609972421e+17 true resid norm >> 4.652118621437e+16 ||r(i)||/||b|| 5.453488535001e+00 >> 7 KSP preconditioned resid norm 1.674912012123e+17 true resid norm >> 2.988815541157e+16 ||r(i)||/||b|| 3.503666310619e+00 >> 8 KSP preconditioned resid norm 1.081901407775e+17 true resid norm >> 1.908145564246e+16 ||r(i)||/||b|| 2.236841062001e+00 >> 9 KSP preconditioned resid norm 5.710123690945e+16 true resid norm >> 9.635193037737e+15 ||r(i)||/||b|| 1.129494302267e+00 >> 10 KSP preconditioned resid norm 1.779373863118e+16 true resid norm >> 2.952490629534e+15 ||r(i)||/||b|| 3.461084101266e-01 >> 11 KSP preconditioned resid norm 1.125622359514e+16 true resid norm >> 2.030418337832e+15 ||r(i)||/||b|| 2.380176437376e-01 >> 12 KSP preconditioned resid norm 1.086650353983e+16 true resid norm >> 1.877894898988e+15 ||r(i)||/||b|| 2.201379443417e-01 >> 13 KSP preconditioned resid norm 1.075411372217e+16 true resid norm >> 1.822725565845e+15 ||r(i)||/||b|| 2.136706688859e-01 >> 14 KSP preconditioned resid norm 1.173963298627e+16 true resid norm >> 2.130483772307e+15 ||r(i)||/||b|| 2.497479056692e-01 >> 15 KSP preconditioned resid norm 4.943816039086e+16 true resid norm >> 1.362098103563e+16 ||r(i)||/||b|| 1.596731939959e+00 >> 16 KSP preconditioned resid norm 1.779827523151e+17 true resid norm >> 3.369509176778e+16 ||r(i)||/||b|| 3.949937901296e+00 >> 17 KSP preconditioned resid norm 3.250308535899e+16 true resid norm >> 8.719651405683e+15 ||r(i)||/||b|| 1.022169098418e+00 >> 18 KSP preconditioned resid norm 2.074788387365e+16 true resid norm >> 3.942900424212e+15 ||r(i)||/||b|| 4.622101027047e-01 >> 19 KSP preconditioned resid norm 1.482962263858e+16 true resid norm >> 3.390403569469e+15 ||r(i)||/||b|| 3.974431543926e-01 >> 20 KSP preconditioned resid norm 1.133931148892e+16 true resid norm >> 2.488262609273e+15 ||r(i)||/||b|| 2.916888565398e-01 >> 21 KSP preconditioned resid norm 8.550489811072e+15 true resid norm >> 1.657275963132e+15 ||r(i)||/||b|| 1.942756881268e-01 >> 22 KSP preconditioned resid norm 6.584107021814e+15 true resid norm >> 1.285719379545e+15 ||r(i)||/||b|| 1.507196283273e-01 >> 23 KSP preconditioned resid norm 4.778317630297e+15 true resid norm >> 9.061620975056e+14 ||r(i)||/||b|| 1.062256793459e-01 >> 24 KSP preconditioned resid norm 3.026263700342e+15 true resid norm >> 6.175894175717e+14 ||r(i)||/||b|| 7.239748342928e-02 >> 25 KSP preconditioned resid norm 1.568620488361e+15 true resid norm >> 2.753144451691e+14 ||r(i)||/||b|| 3.227398723952e-02 >> 26 KSP preconditioned resid norm 1.178330886619e+15 true resid norm >> 2.275547590347e+14 ||r(i)||/||b|| 2.667531442045e-02 >> 27 KSP preconditioned resid norm 1.073508363176e+15 true resid norm >> 2.033900922703e+14 ||r(i)||/||b|| 2.384258929293e-02 >> 28 KSP preconditioned resid norm 1.011358091649e+15 true resid norm >> 1.668017917552e+14 ||r(i)||/||b|| 1.955349235427e-02 >> 29 KSP preconditioned resid norm 9.448460925412e+14 true resid norm >> 1.798337339042e+14 ||r(i)||/||b|| 2.108117367287e-02 >> 30 KSP preconditioned resid norm 8.970299972493e+14 true resid norm >> 1.456018479256e+14 ||r(i)||/||b|| 1.706830958004e-02 >> 31 KSP preconditioned resid norm 8.656072021210e+14 true resid norm >> 1.578084000747e+14 ||r(i)||/||b|| 1.849923380218e-02 >> 32 KSP preconditioned resid norm 8.314911059315e+14 true resid norm >> 1.333543211106e+14 ||r(i)||/||b|| 1.563258206526e-02 >> 33 KSP preconditioned resid norm 8.004072124456e+14 true resid norm >> 1.582598207144e+14 ||r(i)||/||b|| 1.855215199888e-02 >> 34 KSP preconditioned resid norm 7.705686923615e+14 true resid norm >> 1.334906781329e+14 ||r(i)||/||b|| 1.564856664172e-02 >> 35 KSP preconditioned resid norm 7.648238175354e+14 true resid norm >> 1.202157111000e+14 ||r(i)||/||b|| 1.409239650919e-02 >> 36 KSP preconditioned resid norm 7.709241111814e+14 true resid norm >> 1.292601566237e+14 ||r(i)||/||b|| 1.515263989468e-02 >> 37 KSP preconditioned resid norm 8.026855071425e+14 true resid norm >> 1.264132558814e+14 ||r(i)||/||b|| 1.481890935550e-02 >> 38 KSP preconditioned resid norm 8.397026301556e+14 true resid norm >> 1.379015425365e+14 ||r(i)||/||b|| 1.616563424923e-02 >> 39 KSP preconditioned resid norm 8.772641473437e+14 true resid norm >> 1.384855714837e+14 ||r(i)||/||b|| 1.623409757587e-02 >> 40 KSP preconditioned resid norm 9.403663540144e+14 true resid norm >> 1.523096876425e+14 ||r(i)||/||b|| 1.785464221614e-02 >> 41 KSP preconditioned resid norm 1.191025293024e+15 true resid norm >> 1.883988208054e+14 ||r(i)||/||b|| 2.208522380610e-02 >> 42 KSP preconditioned resid norm 2.105592809750e+15 true resid norm >> 3.674437948704e+14 ||r(i)||/||b|| 4.307393438656e-02 >> 43 KSP preconditioned resid norm 1.653069974154e+16 true resid norm >> 2.622043044873e+15 ||r(i)||/||b|| 3.073713902652e-01 >> 44 KSP preconditioned resid norm 1.224677806349e+15 true resid norm >> 2.078786938958e+14 ||r(i)||/||b|| 2.436876971726e-02 >> 45 KSP preconditioned resid norm 6.142260967855e+14 true resid norm >> 9.787693095074e+13 ||r(i)||/||b|| 1.147371260744e-02 >> 46 KSP preconditioned resid norm 4.190317366481e+14 true resid norm >> 7.181248026289e+13 ||r(i)||/||b|| 8.418283574692e-03 >> 47 KSP preconditioned resid norm 2.851377922408e+14 true resid norm >> 4.551212829362e+13 ||r(i)||/||b|| 5.335200798815e-03 >> 48 KSP preconditioned resid norm 1.874725496828e+14 true resid norm >> 3.515118139860e+13 ||r(i)||/||b|| 4.120629337903e-03 >> 49 KSP preconditioned resid norm 1.260137969667e+14 true resid norm >> 2.018341951104e+13 ||r(i)||/||b|| 2.366019782758e-03 >> 50 KSP preconditioned resid norm 8.902880553848e+13 true resid norm >> 2.353973309483e+13 ||r(i)||/||b|| 2.759466707450e-03 >> 51 KSP preconditioned resid norm 6.273228119323e+13 true resid norm >> 1.015825242680e+13 ||r(i)||/||b|| 1.190810416783e-03 >> 52 KSP preconditioned resid norm 4.428322011916e+13 true resid norm >> 1.394037459429e+13 ||r(i)||/||b|| 1.634173141528e-03 >> 0 KSP preconditioned resid norm 1.892236961136e+19 true resid norm >> 1.028105836629e+19 ||r(i)||/||b|| 7.839481546009e+01 >> 1 KSP preconditioned resid norm 4.580244542528e+18 true resid norm >> 1.519483928522e+18 ||r(i)||/||b|| 1.158632291804e+01 >> 2 KSP preconditioned resid norm 1.874052673028e+18 true resid norm >> 4.645668462025e+17 ||r(i)||/||b|| 3.542401071890e+00 >> 3 KSP preconditioned resid norm 1.172977406681e+18 true resid norm >> 2.550460309107e+17 ||r(i)||/||b|| 1.944769284904e+00 >> 4 KSP preconditioned resid norm 8.299170411979e+17 true resid norm >> 1.381052822672e+17 ||r(i)||/||b|| 1.053076223446e+00 >> 5 KSP preconditioned resid norm 7.051011041666e+17 true resid norm >> 1.287511951247e+17 ||r(i)||/||b|| 9.817497209388e-01 >> 6 KSP preconditioned resid norm 5.862228682149e+17 true resid norm >> 1.166604371743e+17 ||r(i)||/||b|| 8.895556389170e-01 >> 7 KSP preconditioned resid norm 4.329574557115e+17 true resid norm >> 8.609505694316e+16 ||r(i)||/||b|| 6.564894255647e-01 >> 8 KSP preconditioned resid norm 2.701974155126e+17 true resid norm >> 4.802624326608e+16 ||r(i)||/||b|| 3.662082583277e-01 >> 9 KSP preconditioned resid norm 1.546976756797e+17 true resid norm >> 2.878555741954e+16 ||r(i)||/||b|| 2.194947622532e-01 >> 10 KSP preconditioned resid norm 9.916116392638e+16 true resid norm >> 1.649766880508e+16 ||r(i)||/||b|| 1.257975254509e-01 >> 11 KSP preconditioned resid norm 9.248895040376e+16 true resid norm >> 1.560136782751e+16 ||r(i)||/||b|| 1.189630783317e-01 >> 12 KSP preconditioned resid norm 9.251831545179e+16 true resid norm >> 1.508056895396e+16 ||r(i)||/||b|| 1.149918985048e-01 >> 13 KSP preconditioned resid norm 7.096157383949e+16 true resid norm >> 1.151221892162e+16 ||r(i)||/||b|| 8.778262370879e-02 >> 14 KSP preconditioned resid norm 2.017629866031e+17 true resid norm >> 3.268014415733e+16 ||r(i)||/||b|| 2.491916473135e-01 >> 15 KSP preconditioned resid norm 1.669814416925e+16 true resid norm >> 4.397468530146e+15 ||r(i)||/||b|| 3.353144410138e-02 >> 16 KSP preconditioned resid norm 1.171914014315e+16 true resid norm >> 1.865370183430e+15 ||r(i)||/||b|| 1.422376433288e-02 >> 17 KSP preconditioned resid norm 1.021030261775e+16 true resid norm >> 2.000923353135e+15 ||r(i)||/||b|| 1.525738026477e-02 >> 18 KSP preconditioned resid norm 9.416598060128e+15 true resid norm >> 1.541246281727e+15 ||r(i)||/||b|| 1.175226455582e-02 >> 19 KSP preconditioned resid norm 8.821999311745e+15 true resid norm >> 1.426601243635e+15 ||r(i)||/||b|| 1.087807667707e-02 >> 20 KSP preconditioned resid norm 8.323362047556e+15 true resid norm >> 1.490210405686e+15 ||r(i)||/||b|| 1.136310733664e-02 >> 21 KSP preconditioned resid norm 7.861641988008e+15 true resid norm >> 1.251748347072e+15 ||r(i)||/||b|| 9.544793655962e-03 >> 22 KSP preconditioned resid norm 7.464952525350e+15 true resid norm >> 1.325467789570e+15 ||r(i)||/||b|| 1.010691692037e-02 >> 23 KSP preconditioned resid norm 7.139096535480e+15 true resid norm >> 1.125905927178e+15 ||r(i)||/||b|| 8.585223839986e-03 >> 24 KSP preconditioned resid norm 5.858113828393e+15 true resid norm >> 2.042448137714e+15 ||r(i)||/||b|| 1.557401379686e-02 >> 25 KSP preconditioned resid norm 5.027356938946e+15 true resid norm >> 7.946522117539e+14 ||r(i)||/||b|| 6.059358022871e-03 >> 26 KSP preconditioned resid norm 5.122614017475e+15 true resid norm >> 7.976827488389e+14 ||r(i)||/||b|| 6.082466382639e-03 >> 27 KSP preconditioned resid norm 5.425317067876e+15 true resid norm >> 8.948524578548e+14 ||r(i)||/||b|| 6.823401910404e-03 >> 28 KSP preconditioned resid norm 9.316529191423e+15 true resid norm >> 1.453760069771e+15 ||r(i)||/||b|| 1.108516733711e-02 >> 29 KSP preconditioned resid norm 9.523628080058e+15 true resid norm >> 2.109739324167e+15 ||r(i)||/||b|| 1.608712051759e-02 >> 30 KSP preconditioned resid norm 2.300173794637e+15 true resid norm >> 3.594261469430e+14 ||r(i)||/||b|| 2.740685390281e-03 >> 31 KSP preconditioned resid norm 1.385027169120e+15 true resid norm >> 2.935564708865e+14 ||r(i)||/||b|| 2.238417927644e-03 >> 32 KSP preconditioned resid norm 9.719758142764e+14 true resid norm >> 1.523176597454e+14 ||r(i)||/||b|| 1.161448014555e-03 >> 33 KSP preconditioned resid norm 8.580268959656e+14 true resid norm >> 1.580637131624e+14 ||r(i)||/||b|| 1.205262647368e-03 >> 0 KSP preconditioned resid norm 6.470797769938e+18 true resid norm >> 3.511548722307e+18 ||r(i)||/||b|| 1.662246831388e+02 >> 1 KSP preconditioned resid norm 1.571597715354e+18 true resid norm >> 5.187477038218e+17 ||r(i)||/||b|| 2.455573865428e+01 >> 2 KSP preconditioned resid norm 6.468337546861e+17 true resid norm >> 1.566996108585e+17 ||r(i)||/||b|| 7.417622599812e+00 >> 3 KSP preconditioned resid norm 4.084044703454e+17 true resid norm >> 8.688412887362e+16 ||r(i)||/||b|| 4.112796926342e+00 >> 4 KSP preconditioned resid norm 2.920894102259e+17 true resid norm >> 4.831956260441e+16 ||r(i)||/||b|| 2.287282512214e+00 >> 5 KSP preconditioned resid norm 2.515869150351e+17 true resid norm >> 4.435569840236e+16 ||r(i)||/||b|| 2.099646764259e+00 >> 6 KSP preconditioned resid norm 2.133085363553e+17 true resid norm >> 4.355632900946e+16 ||r(i)||/||b|| 2.061807356479e+00 >> 7 KSP preconditioned resid norm 1.601295664581e+17 true resid norm >> 2.999352167191e+16 ||r(i)||/||b|| 1.419790534148e+00 >> 8 KSP preconditioned resid norm 1.019894513782e+17 true resid norm >> 1.811122934468e+16 ||r(i)||/||b|| 8.573235336161e-01 >> 9 KSP preconditioned resid norm 5.552366492146e+16 true resid norm >> 9.849650352808e+15 ||r(i)||/||b|| 4.662486949199e-01 >> 10 KSP preconditioned resid norm 2.614329828139e+16 true resid norm >> 4.400937150634e+15 ||r(i)||/||b|| 2.083252835795e-01 >> 11 KSP preconditioned resid norm 2.230447954838e+16 true resid norm >> 4.002038016135e+15 ||r(i)||/||b|| 1.894427654999e-01 >> 12 KSP preconditioned resid norm 2.202888172936e+16 true resid norm >> 3.669015023581e+15 ||r(i)||/||b|| 1.736785982356e-01 >> 13 KSP preconditioned resid norm 2.171444770780e+16 true resid norm >> 3.746639147837e+15 ||r(i)||/||b|| 1.773530582755e-01 >> 14 KSP preconditioned resid norm 2.756620760322e+16 true resid norm >> 4.738417417687e+15 ||r(i)||/||b|| 2.243004429444e-01 >> 15 KSP preconditioned resid norm 1.911455835947e+16 true resid norm >> 7.274462982878e+15 ||r(i)||/||b|| 3.443481494796e-01 >> 16 KSP preconditioned resid norm 8.270124664006e+15 true resid norm >> 1.450957167614e+15 ||r(i)||/||b|| 6.868334017480e-02 >> 17 KSP preconditioned resid norm 5.454864800054e+15 true resid norm >> 1.779587764442e+15 ||r(i)||/||b|| 8.423958647728e-02 >> 18 KSP preconditioned resid norm 4.188178169772e+15 true resid norm >> 7.356207570842e+14 ||r(i)||/||b|| 3.482176581514e-02 >> 19 KSP preconditioned resid norm 3.485885419242e+15 true resid norm >> 7.684000525717e+14 ||r(i)||/||b|| 3.637342533543e-02 >> 20 KSP preconditioned resid norm 3.031526098420e+15 true resid norm >> 5.668894180734e+14 ||r(i)||/||b|| 2.683460243492e-02 >> 21 KSP preconditioned resid norm 2.653824581081e+15 true resid norm >> 5.337896513987e+14 ||r(i)||/||b|| 2.526777290683e-02 >> 22 KSP preconditioned resid norm 2.412624108418e+15 true resid norm >> 3.917881686680e+14 ||r(i)||/||b|| 1.854590932504e-02 >> 23 KSP preconditioned resid norm 2.249293018506e+15 true resid norm >> 3.520070992955e+14 ||r(i)||/||b|| 1.666280982271e-02 >> 24 KSP preconditioned resid norm 2.107969657718e+15 true resid norm >> 3.433991961929e+14 ||r(i)||/||b|| 1.625534118740e-02 >> 25 KSP preconditioned resid norm 1.995899579616e+15 true resid norm >> 3.124097002184e+14 ||r(i)||/||b|| 1.478840464277e-02 >> 26 KSP preconditioned resid norm 1.940326192847e+15 true resid norm >> 3.197343858652e+14 ||r(i)||/||b|| 1.513513016106e-02 >> 27 KSP preconditioned resid norm 1.843618399864e+15 true resid norm >> 4.284525431489e+14 ||r(i)||/||b|| 2.028147517148e-02 >> 28 KSP preconditioned resid norm 1.800569430365e+15 true resid norm >> 2.810328443449e+14 ||r(i)||/||b|| 1.330313180793e-02 >> 29 KSP preconditioned resid norm 1.911265522964e+15 true resid norm >> 3.078500227429e+14 ||r(i)||/||b|| 1.457256513619e-02 >> 30 KSP preconditioned resid norm 2.245578740852e+15 true resid norm >> 3.522723048093e+14 ||r(i)||/||b|| 1.667536374293e-02 >> 31 KSP preconditioned resid norm 3.129444954320e+15 true resid norm >> 5.056459987900e+14 ||r(i)||/||b|| 2.393554883500e-02 >> 32 KSP preconditioned resid norm 2.268235755551e+16 true resid norm >> 3.663405821329e+15 ||r(i)||/||b|| 1.734130778226e-01 >> 33 KSP preconditioned resid norm 2.666354866266e+15 true resid norm >> 4.420681584183e+14 ||r(i)||/||b|| 2.092599174035e-02 >> 34 KSP preconditioned resid norm 1.195729215971e+15 true resid norm >> 1.967322086000e+14 ||r(i)||/||b|| 9.312628593187e-03 >> 35 KSP preconditioned resid norm 7.390918117697e+14 true resid norm >> 1.247185996576e+14 ||r(i)||/||b|| 5.903751122089e-03 >> 36 KSP preconditioned resid norm 5.589675490221e+14 true resid norm >> 9.067132579829e+13 ||r(i)||/||b|| 4.292069850790e-03 >> 37 KSP preconditioned resid norm 4.470335591004e+14 true resid norm >> 9.109636100840e+13 ||r(i)||/||b|| 4.312189561126e-03 >> 38 KSP preconditioned resid norm 4.031608678043e+14 true resid norm >> 6.399106058589e+13 ||r(i)||/||b|| 3.029117523568e-03 >> 39 KSP preconditioned resid norm 3.769205171045e+14 true resid norm >> 7.200688789282e+13 ||r(i)||/||b|| 3.408559319641e-03 >> 40 KSP preconditioned resid norm 3.535103364248e+14 true resid norm >> 5.569365215364e+13 ||r(i)||/||b|| 2.636346641945e-03 >> 41 KSP preconditioned resid norm 3.299087996116e+14 true resid norm >> 5.691184302265e+13 ||r(i)||/||b|| 2.694011623187e-03 >> 42 KSP preconditioned resid norm 3.102396074559e+14 true resid norm >> 5.072539063269e+13 ||r(i)||/||b|| 2.401166166782e-03 >> 43 KSP preconditioned resid norm 2.923631153501e+14 true resid norm >> 5.690604160219e+13 ||r(i)||/||b|| 2.693737003823e-03 >> 44 KSP preconditioned resid norm 2.831787236990e+14 true resid norm >> 4.398532156025e+13 ||r(i)||/||b|| 2.082114393761e-03 >> 45 KSP preconditioned resid norm 2.880585984465e+14 true resid norm >> 4.666639157225e+13 ||r(i)||/||b|| 2.209027060639e-03 >> 46 KSP preconditioned resid norm 3.054400711070e+14 true resid norm >> 4.771426500640e+13 ||r(i)||/||b|| 2.258629798159e-03 >> 47 KSP preconditioned resid norm 3.761477931303e+14 true resid norm >> 5.957625439087e+13 ||r(i)||/||b|| 2.820135726954e-03 >> 48 KSP preconditioned resid norm 9.608784364697e+14 true resid norm >> 1.580857575603e+14 ||r(i)||/||b|| 7.483238034628e-03 >> 49 KSP preconditioned resid norm 1.902148260101e+15 true resid norm >> 2.982038509627e+14 ||r(i)||/||b|| 1.411594841961e-02 >> 50 KSP preconditioned resid norm 2.165055327372e+14 true resid norm >> 4.058692191960e+13 ||r(i)||/||b|| 1.921245800409e-03 >> 51 KSP preconditioned resid norm 2.195213865618e+14 true resid norm >> 3.429224338721e+13 ||r(i)||/||b|| 1.623277289290e-03 >> 52 KSP preconditioned resid norm 1.864119738188e+14 true resid norm >> 5.012858400476e+13 ||r(i)||/||b|| 2.372915386152e-03 >> 53 KSP preconditioned resid norm 2.971924491921e+13 true resid norm >> 4.598158953060e+12 ||r(i)||/||b|| 2.176610878666e-04 >> 0 KSP preconditioned resid norm 6.694681472101e+18 true resid norm >> 3.650754008704e+18 ||r(i)||/||b|| 5.229798534711e+00 >> 1 KSP preconditioned resid norm 1.597667311920e+18 true resid norm >> 5.366749653247e+17 ||r(i)||/||b|| 7.688006205237e-01 >> 2 KSP preconditioned resid norm 6.300446691769e+17 true resid norm >> 1.705328302298e+17 ||r(i)||/||b|| 2.442926429799e-01 >> 3 KSP preconditioned resid norm 3.642174434692e+17 true resid norm >> 9.481577406412e+16 ||r(i)||/||b|| 1.358260225383e-01 >> 4 KSP preconditioned resid norm 2.247332932327e+17 true resid norm >> 3.802800659066e+16 ||r(i)||/||b|| 5.447609251997e-02 >> 5 KSP preconditioned resid norm 1.557523729879e+17 true resid norm >> 3.829141784861e+16 ||r(i)||/||b|| 5.485343588727e-02 >> 6 KSP preconditioned resid norm 1.004692897232e+17 true resid norm >> 1.699728149080e+16 ||r(i)||/||b|| 2.434904067015e-02 >> 7 KSP preconditioned resid norm 7.103483293598e+16 true resid norm >> 1.472858902835e+16 ||r(i)||/||b|| 2.109908066531e-02 >> 8 KSP preconditioned resid norm 5.766848376574e+16 true resid norm >> 9.397954931890e+15 ||r(i)||/||b|| 1.346281091931e-02 >> 9 KSP preconditioned resid norm 4.523472389397e+16 true resid norm >> 1.246375028587e+16 ||r(i)||/||b|| 1.785464121292e-02 >> 10 KSP preconditioned resid norm 3.195342596604e+16 true resid norm >> 5.030521805863e+15 ||r(i)||/||b|| 7.206351210301e-03 >> 11 KSP preconditioned resid norm 2.437460193025e+16 true resid norm >> 4.974645265358e+15 ||r(i)||/||b|| 7.126306636231e-03 >> 12 KSP preconditioned resid norm 1.258045317564e+16 true resid norm >> 1.885909944349e+15 ||r(i)||/||b|| 2.701614252848e-03 >> 13 KSP preconditioned resid norm 2.495567071066e+15 true resid norm >> 4.460991862355e+14 ||r(i)||/||b|| 6.390484992820e-04 >> 14 KSP preconditioned resid norm 7.678105595258e+14 true resid norm >> 1.316278119117e+14 ||r(i)||/||b|| 1.885602087190e-04 >> 15 KSP preconditioned resid norm 5.950875767888e+14 true resid norm >> 1.439411571316e+14 ||r(i)||/||b|| 2.061993908263e-04 >> 16 KSP preconditioned resid norm 3.263039815738e+14 true resid norm >> 5.046549667518e+13 ||r(i)||/||b|| 7.229311532250e-05 >> 17 KSP preconditioned resid norm 1.436785883111e+14 true resid norm >> 7.588987254543e+13 ||r(i)||/||b|| 1.087141843277e-04 >> 18 KSP preconditioned resid norm 5.401544347363e+13 true resid norm >> 8.625334293000e+12 ||r(i)||/||b|| 1.235601208391e-05 >> 19 KSP preconditioned resid norm 3.528863285477e+13 true resid norm >> 1.233589079533e+13 ||r(i)||/||b|| 1.767147922099e-05 >> 0 KSP preconditioned resid norm 4.827871909881e+18 true resid norm >> 2.629048338874e+18 ||r(i)||/||b|| 2.387243403092e+01 >> 1 KSP preconditioned resid norm 1.161059375885e+18 true resid norm >> 3.885115498370e+17 ||r(i)||/||b|| 3.527784638492e+00 >> 2 KSP preconditioned resid norm 4.696929326234e+17 true resid norm >> 1.228954029189e+17 ||r(i)||/||b|| 1.115921816843e+00 >> 3 KSP preconditioned resid norm 2.870100194440e+17 true resid norm >> 6.759377089469e+16 ||r(i)||/||b|| 6.137687971441e-01 >> 4 KSP preconditioned resid norm 1.966638027618e+17 true resid norm >> 3.310588396768e+16 ||r(i)||/||b|| 3.006099276942e-01 >> 5 KSP preconditioned resid norm 1.592573327953e+17 true resid norm >> 3.239413757211e+16 ||r(i)||/||b|| 2.941470876529e-01 >> 6 KSP preconditioned resid norm 1.230114084345e+17 true resid norm >> 2.280735920899e+16 ||r(i)||/||b|| 2.070966783247e-01 >> 7 KSP preconditioned resid norm 8.598639005636e+16 true resid norm >> 1.938027655308e+16 ||r(i)||/||b|| 1.759778877677e-01 >> 8 KSP preconditioned resid norm 5.549090128976e+16 true resid norm >> 9.386098364471e+15 ||r(i)||/||b|| 8.522818340779e-02 >> 9 KSP preconditioned resid norm 4.474765035679e+16 true resid norm >> 7.579175521646e+15 ||r(i)||/||b|| 6.882085999480e-02 >> 10 KSP preconditioned resid norm 4.378403606928e+16 true resid norm >> 6.842614495131e+15 ||r(i)||/||b|| 6.213269673237e-02 >> 11 KSP preconditioned resid norm 4.079517449398e+16 true resid norm >> 7.704845808673e+15 ||r(i)||/||b|| 6.996197847191e-02 >> 12 KSP preconditioned resid norm 4.084405622719e+16 true resid norm >> 6.377952548774e+15 ||r(i)||/||b|| 5.791344693880e-02 >> 13 KSP preconditioned resid norm 7.523868588173e+16 true resid norm >> 1.132249240798e+16 ||r(i)||/||b|| 1.028111385699e-01 >> 14 KSP preconditioned resid norm 2.184581375390e+15 true resid norm >> 3.502138477296e+14 ||r(i)||/||b|| 3.180031668881e-03 >> 15 KSP preconditioned resid norm 2.222269400035e+15 true resid norm >> 3.654767263686e+14 ||r(i)||/||b|| 3.318622526282e-03 >> 16 KSP preconditioned resid norm 1.669508240008e+15 true resid norm >> 3.629680148217e+14 ||r(i)||/||b|| 3.295842781223e-03 >> 17 KSP preconditioned resid norm 1.401913603747e+15 true resid norm >> 2.161155966391e+14 ||r(i)||/||b|| 1.962385113858e-03 >> 18 KSP preconditioned resid norm 1.134372518022e+15 true resid norm >> 2.686701598571e+14 ||r(i)||/||b|| 2.439594043376e-03 >> 19 KSP preconditioned resid norm 9.057728167563e+14 true resid norm >> 1.409896456102e+14 ||r(i)||/||b|| 1.280222186904e-03 >> 20 KSP preconditioned resid norm 7.989861774441e+14 true resid norm >> 1.659027150307e+14 ||r(i)||/||b|| 1.506439254676e-03 >> 21 KSP preconditioned resid norm 7.119014585200e+14 true resid norm >> 1.092105820565e+14 ||r(i)||/||b|| 9.916601292842e-04 >> 22 KSP preconditioned resid norm 6.306860453827e+14 true resid norm >> 1.460692513610e+14 ||r(i)||/||b|| 1.326346311515e-03 >> 23 KSP preconditioned resid norm 5.637172379886e+14 true resid norm >> 8.672554380666e+13 ||r(i)||/||b|| 7.874902080372e-04 >> 24 KSP preconditioned resid norm 4.961067656606e+14 true resid norm >> 1.292132364829e+14 ||r(i)||/||b|| 1.173289367962e-03 >> 25 KSP preconditioned resid norm 4.386188245672e+14 true resid norm >> 8.044020700160e+13 ||r(i)||/||b|| 7.304177358342e-04 >> 26 KSP preconditioned resid norm 4.419078090293e+14 true resid norm >> 6.703649620362e+13 ||r(i)||/||b|| 6.087085998464e-04 >> 27 KSP preconditioned resid norm 4.436677280477e+14 true resid norm >> 1.368871842229e+14 ||r(i)||/||b|| 1.242970784036e-03 >> 28 KSP preconditioned resid norm 7.847692003240e+14 true resid norm >> 1.196853324186e+14 ||r(i)||/||b|| 1.086773552386e-03 >> 29 KSP preconditioned resid norm 1.771704167839e+15 true resid norm >> 6.548256652554e+14 ||r(i)||/||b|| 5.945985193355e-03 >> 30 KSP preconditioned resid norm 1.053899282998e+15 true resid norm >> 1.880673778730e+14 ||r(i)||/||b|| 1.707700084953e-03 >> 31 KSP preconditioned resid norm 2.462670432282e+14 true resid norm >> 3.919805233311e+13 ||r(i)||/||b|| 3.559283808617e-04 >> 32 KSP preconditioned resid norm 1.093017734594e+14 true resid norm >> 1.903326631551e+13 ||r(i)||/||b|| 1.728269457018e-04 >> 33 KSP preconditioned resid norm 6.432042300545e+13 true resid norm >> 1.002471721316e+13 ||r(i)||/||b|| 9.102700654495e-05 >> 34 KSP preconditioned resid norm 3.805689715853e+13 true resid norm >> 7.047382673791e+12 ||r(i)||/||b|| 6.399204437707e-05 >> 35 KSP preconditioned resid norm 2.527168724251e+13 true resid norm >> 4.009865388649e+12 ||r(i)||/||b|| 3.641060742320e-05 >> 36 KSP preconditioned resid norm 1.580947387777e+13 true resid norm >> 2.531811988645e+12 ||r(i)||/||b|| 2.298950300149e-05 >> 37 KSP preconditioned resid norm 1.075548264218e+13 true resid norm >> 1.788042513750e+12 ||r(i)||/||b|| 1.623588517670e-05 >> 38 KSP preconditioned resid norm 6.581274433602e+12 true resid norm >> 1.035380330637e+12 ||r(i)||/||b|| 9.401519277742e-06 >> 0 KSP preconditioned resid norm 9.555736051823e+15 true resid norm >> 5.892741751019e+15 ||r(i)||/||b|| 1.119235791581e-01 >> 1 KSP preconditioned resid norm 4.575036500991e+15 true resid norm >> 8.101073993664e+14 ||r(i)||/||b|| 1.538674584270e-02 >> 2 KSP preconditioned resid norm 3.765161451471e+15 true resid norm >> 6.105904357791e+14 ||r(i)||/||b|| 1.159722754867e-02 >> 3 KSP preconditioned resid norm 3.685525899039e+15 true resid norm >> 6.160361102521e+14 ||r(i)||/||b|| 1.170065977151e-02 >> 4 KSP preconditioned resid norm 3.856425370708e+15 true resid norm >> 6.278594964367e+14 ||r(i)||/||b|| 1.192522683307e-02 >> 5 KSP preconditioned resid norm 4.281981871434e+15 true resid norm >> 6.756473177150e+14 ||r(i)||/||b|| 1.283288310305e-02 >> 6 KSP preconditioned resid norm 5.205822014832e+15 true resid norm >> 7.760311477213e+14 ||r(i)||/||b|| 1.473951977891e-02 >> 7 KSP preconditioned resid norm 6.471690051175e+15 true resid norm >> 1.171341036231e+15 ||r(i)||/||b|| 2.224782397211e-02 >> 8 KSP preconditioned resid norm 7.016914241690e+15 true resid norm >> 1.103022939349e+15 ||r(i)||/||b|| 2.095022664859e-02 >> 9 KSP preconditioned resid norm 7.025590013206e+15 true resid norm >> 1.330551872513e+15 ||r(i)||/||b|| 2.527178928238e-02 >> 10 KSP preconditioned resid norm 6.405136813848e+15 true resid norm >> 1.017867183616e+15 ||r(i)||/||b|| 1.933282385541e-02 >> 11 KSP preconditioned resid norm 4.854020251396e+15 true resid norm >> 8.473369029366e+14 ||r(i)||/||b|| 1.609386308386e-02 >> 12 KSP preconditioned resid norm 2.353499572340e+15 true resid norm >> 3.616493792413e+14 ||r(i)||/||b|| 6.868974517339e-03 >> 13 KSP preconditioned resid norm 1.176204335431e+15 true resid norm >> 2.390307465685e+14 ||r(i)||/||b|| 4.540021914275e-03 >> 14 KSP preconditioned resid norm 1.050288991204e+15 true resid norm >> 1.923001174300e+14 ||r(i)||/||b|| 3.652445385304e-03 >> 15 KSP preconditioned resid norm 9.850224343838e+14 true resid norm >> 1.772358652449e+14 ||r(i)||/||b|| 3.366323051568e-03 >> 16 KSP preconditioned resid norm 8.835883676210e+14 true resid norm >> 1.426145250921e+14 ||r(i)||/||b|| 2.708743868757e-03 >> 17 KSP preconditioned resid norm 7.483591829964e+14 true resid norm >> 1.680673516819e+14 ||r(i)||/||b|| 3.192181217957e-03 >> 18 KSP preconditioned resid norm 6.652269160454e+14 true resid norm >> 1.088301440117e+14 ||r(i)||/||b|| 2.067061438079e-03 >> 19 KSP preconditioned resid norm 5.777832230782e+14 true resid norm >> 9.220881620683e+13 ||r(i)||/||b|| 1.751364844390e-03 >> 20 KSP preconditioned resid norm 5.145663259773e+14 true resid norm >> 8.255284700142e+13 ||r(i)||/||b|| 1.567964539511e-03 >> 21 KSP preconditioned resid norm 4.536494497263e+14 true resid norm >> 7.162151119367e+13 ||r(i)||/||b|| 1.360340604800e-03 >> 22 KSP preconditioned resid norm 4.039727797390e+14 true resid norm >> 6.757078541538e+13 ||r(i)||/||b|| 1.283403289972e-03 >> 23 KSP preconditioned resid norm 3.668438626905e+14 true resid norm >> 5.700238490686e+13 ||r(i)||/||b|| 1.082672753854e-03 >> 24 KSP preconditioned resid norm 3.368796010179e+14 true resid norm >> 6.388426371627e+13 ||r(i)||/||b|| 1.213383472264e-03 >> 0 KSP preconditioned resid norm 9.591788600089e+15 true resid norm >> 5.932754803820e+15 ||r(i)||/||b|| 5.703607413375e-01 >> 1 KSP preconditioned resid norm 4.627577110935e+15 true resid norm >> 8.119149369849e+14 ||r(i)||/||b|| 7.805554429176e-02 >> 2 KSP preconditioned resid norm 3.843546862524e+15 true resid norm >> 6.215691855461e+14 ||r(i)||/||b|| 5.975616272433e-02 >> 3 KSP preconditioned resid norm 3.802220665831e+15 true resid norm >> 6.332285028379e+14 ||r(i)||/||b|| 6.087706137495e-02 >> 4 KSP preconditioned resid norm 4.025305304206e+15 true resid norm >> 6.486950645643e+14 ||r(i)||/||b|| 6.236397932519e-02 >> 5 KSP preconditioned resid norm 4.519185013022e+15 true resid norm >> 7.100155628952e+14 ||r(i)||/||b|| 6.825918417416e-02 >> 6 KSP preconditioned resid norm 5.566187938033e+15 true resid norm >> 8.272878114077e+14 ||r(i)||/||b|| 7.953345537054e-02 >> 7 KSP preconditioned resid norm 7.046115999320e+15 true resid norm >> 1.260178524183e+15 ||r(i)||/||b|| 1.211505246783e-01 >> 8 KSP preconditioned resid norm 7.802424274220e+15 true resid norm >> 1.227649475790e+15 ||r(i)||/||b|| 1.180232604022e-01 >> 9 KSP preconditioned resid norm 8.076065684234e+15 true resid norm >> 1.496448814554e+15 ||r(i)||/||b|| 1.438649806820e-01 >> 10 KSP preconditioned resid norm 7.812303545308e+15 true resid norm >> 1.236539634992e+15 ||r(i)||/||b|| 1.188779388714e-01 >> 11 KSP preconditioned resid norm 6.692698687119e+15 true resid norm >> 1.122795851888e+15 ||r(i)||/||b|| 1.079428858312e-01 >> 12 KSP preconditioned resid norm 4.016544769049e+15 true resid norm >> 6.096078457742e+14 ||r(i)||/||b|| 5.860622836074e-02 >> 13 KSP preconditioned resid norm 1.711147574309e+15 true resid norm >> 2.966258404639e+14 ||r(i)||/||b|| 2.851689305581e-02 >> 14 KSP preconditioned resid norm 1.304839987953e+15 true resid norm >> 2.700754536307e+14 ||r(i)||/||b|| 2.596440288593e-02 >> 15 KSP preconditioned resid norm 1.171070442303e+15 true resid norm >> 2.199042966295e+14 ||r(i)||/||b|| 2.114106882828e-02 >> 16 KSP preconditioned resid norm 1.057836577493e+15 true resid norm >> 1.660135728859e+14 ||r(i)||/||b|| 1.596014459292e-02 >> 17 KSP preconditioned resid norm 8.905346433993e+14 true resid norm >> 1.613310037432e+14 ||r(i)||/||b|| 1.550997368650e-02 >> 18 KSP preconditioned resid norm 7.864866777768e+14 true resid norm >> 1.268576094498e+14 ||r(i)||/||b|| 1.219578468395e-02 >> 19 KSP preconditioned resid norm 6.743846682412e+14 true resid norm >> 1.066594941052e+14 ||r(i)||/||b|| 1.025398657793e-02 >> 20 KSP preconditioned resid norm 5.775523980161e+14 true resid norm >> 1.005326356082e+14 ||r(i)||/||b|| 9.664965175573e-03 >> 21 KSP preconditioned resid norm 4.897344092103e+14 true resid norm >> 7.691922473584e+13 ||r(i)||/||b|| 7.394828792721e-03 >> 22 KSP preconditioned resid norm 4.026909639422e+14 true resid norm >> 7.643768412999e+13 ||r(i)||/||b|| 7.348534640001e-03 >> 23 KSP preconditioned resid norm 3.340361168756e+14 true resid norm >> 5.217079213762e+13 ||r(i)||/||b|| 5.015574157998e-03 >> 24 KSP preconditioned resid norm 2.772587883161e+14 true resid norm >> 6.149190087025e+13 ||r(i)||/||b|| 5.911683075799e-03 >> 25 KSP preconditioned resid norm 2.446809593449e+14 true resid norm >> 3.772191549633e+13 ||r(i)||/||b|| 3.626493997916e-03 >> 26 KSP preconditioned resid norm 2.130893101456e+14 true resid norm >> 5.402695231969e+13 ||r(i)||/||b|| 5.194020922190e-03 >> 27 KSP preconditioned resid norm 1.886821885223e+14 true resid norm >> 2.887742158299e+13 ||r(i)||/||b|| 2.776205679591e-03 >> 28 KSP preconditioned resid norm 1.654479962436e+14 true resid norm >> 4.199650238007e+13 ||r(i)||/||b|| 4.037442473714e-03 >> 29 KSP preconditioned resid norm 1.453563006355e+14 true resid norm >> 2.222396036725e+13 ||r(i)||/||b|| 2.136557961634e-03 >> 30 KSP preconditioned resid norm 1.311692747504e+14 true resid norm >> 2.933890714503e+13 ||r(i)||/||b|| 2.820571788757e-03 >> 31 KSP preconditioned resid norm 1.173199137783e+14 true resid norm >> 1.793346595553e+13 ||r(i)||/||b|| 1.724080174451e-03 >> 32 KSP preconditioned resid norm 1.086053135569e+14 true resid norm >> 2.009773461598e+13 ||r(i)||/||b|| 1.932147744820e-03 >> 33 KSP preconditioned resid norm 9.929347161574e+13 true resid norm >> 1.516318812509e+13 ||r(i)||/||b|| 1.457752343733e-03 >> 34 KSP preconditioned resid norm 9.235712158960e+13 true resid norm >> 1.538423395802e+13 ||r(i)||/||b|| 1.479003157109e-03 >> 35 KSP preconditioned resid norm 8.718922541063e+13 true resid norm >> 1.331044977413e+13 ||r(i)||/||b|| 1.279634546134e-03 >> 36 KSP preconditioned resid norm 8.163106494702e+13 true resid norm >> 1.369060193385e+13 ||r(i)||/||b|| 1.316181458118e-03 >> 37 KSP preconditioned resid norm 7.751439049037e+13 true resid norm >> 1.182966275510e+13 ||r(i)||/||b|| 1.137275252708e-03 >> 38 KSP preconditioned resid norm 7.269591005999e+13 true resid norm >> 1.225102361382e+13 ||r(i)||/||b|| 1.177783869649e-03 >> 39 KSP preconditioned resid norm 6.809139852739e+13 true resid norm >> 1.039056641301e+13 ||r(i)||/||b|| 9.989240004362e-04 >> 40 KSP preconditioned resid norm 6.390023598760e+13 true resid norm >> 1.042637257280e+13 ||r(i)||/||b|| 1.002366318301e-03 >> 41 KSP preconditioned resid norm 5.919860950635e+13 true resid norm >> 9.048648521202e+12 ||r(i)||/||b|| 8.699152500509e-04 >> 42 KSP preconditioned resid norm 5.538975217584e+13 true resid norm >> 8.775858022438e+12 ||r(i)||/||b|| 8.436898292725e-04 >> 0 KSP preconditioned resid norm 2.200871087933e+16 true resid norm >> 1.375121498750e+16 ||r(i)||/||b|| 1.042688751858e-01 >> 1 KSP preconditioned resid norm 1.033852082172e+16 true resid norm >> 1.769345381246e+15 ||r(i)||/||b|| 1.341609835098e-02 >> 2 KSP preconditioned resid norm 8.201114088620e+15 true resid norm >> 1.330146487456e+15 ||r(i)||/||b|| 1.008586355500e-02 >> 3 KSP preconditioned resid norm 7.727276364323e+15 true resid norm >> 1.292967289192e+15 ||r(i)||/||b|| 9.803951506731e-03 >> 4 KSP preconditioned resid norm 7.788308934018e+15 true resid norm >> 1.241309862269e+15 ||r(i)||/||b|| 9.412257986909e-03 >> 5 KSP preconditioned resid norm 8.253685478294e+15 true resid norm >> 1.290166199902e+15 ||r(i)||/||b|| 9.782712188615e-03 >> 6 KSP preconditioned resid norm 9.476714658211e+15 true resid norm >> 1.410015893632e+15 ||r(i)||/||b|| 1.069147499743e-02 >> 7 KSP preconditioned resid norm 1.110541369388e+16 true resid norm >> 1.990482855418e+15 ||r(i)||/||b|| 1.509287787295e-02 >> 8 KSP preconditioned resid norm 1.149682905149e+16 true resid norm >> 1.789313061240e+15 ||r(i)||/||b|| 1.356750370206e-02 >> 9 KSP preconditioned resid norm 1.081618830942e+16 true resid norm >> 1.972959789140e+15 ||r(i)||/||b|| 1.496000885648e-02 >> 10 KSP preconditioned resid norm 8.800226542664e+15 true resid norm >> 1.343393349540e+15 ||r(i)||/||b|| 1.018630816375e-02 >> 11 KSP preconditioned resid norm 4.142701353678e+15 true resid norm >> 6.194874024708e+14 ||r(i)||/||b|| 4.697276182950e-03 >> 12 KSP preconditioned resid norm 3.300703256438e+15 true resid norm >> 1.133601440349e+15 ||r(i)||/||b|| 8.595556625482e-03 >> 13 KSP preconditioned resid norm 7.882205618989e+14 true resid norm >> 1.186911081255e+14 ||r(i)||/||b|| 8.999778092374e-04 >> 0 KSP preconditioned resid norm 1.928554431739e+16 true resid norm >> 1.203381747345e+16 ||r(i)||/||b|| 4.946634802067e-01 >> 1 KSP preconditioned resid norm 9.232243648078e+15 true resid norm >> 1.583125971956e+15 ||r(i)||/||b|| 6.507615763833e-02 >> 2 KSP preconditioned resid norm 7.541746545280e+15 true resid norm >> 1.216565269124e+15 ||r(i)||/||b|| 5.000827137780e-02 >> 3 KSP preconditioned resid norm 7.340199256082e+15 true resid norm >> 1.219811880454e+15 ||r(i)||/||b|| 5.014172695523e-02 >> 4 KSP preconditioned resid norm 7.655426776880e+15 true resid norm >> 1.212245393966e+15 ||r(i)||/||b|| 4.983069809450e-02 >> 5 KSP preconditioned resid norm 8.414958315915e+15 true resid norm >> 1.311448634719e+15 ||r(i)||/||b|| 5.390855787813e-02 >> 6 KSP preconditioned resid norm 1.008678778264e+16 true resid norm >> 1.495852838926e+15 ||r(i)||/||b|| 6.148869823002e-02 >> 7 KSP preconditioned resid norm 1.235685231672e+16 true resid norm >> 2.203374571492e+15 ||r(i)||/||b|| 9.057216765485e-02 >> 8 KSP preconditioned resid norm 1.311846317918e+16 true resid norm >> 2.061677565336e+15 ||r(i)||/||b|| 8.474755428051e-02 >> 9 KSP preconditioned resid norm 1.289737803434e+16 true resid norm >> 2.355893652982e+15 ||r(i)||/||b|| 9.684163449812e-02 >> 10 KSP preconditioned resid norm 1.151803850475e+16 true resid norm >> 1.821777204240e+15 ||r(i)||/||b|| 7.488618254339e-02 >> 11 KSP preconditioned resid norm 8.380110732685e+15 true resid norm >> 1.434954901150e+15 ||r(i)||/||b|| 5.898542062058e-02 >> 12 KSP preconditioned resid norm 3.553409074636e+15 true resid norm >> 5.423523740165e+14 ||r(i)||/||b|| 2.229399884295e-02 >> 13 KSP preconditioned resid norm 1.190458477552e+15 true resid norm >> 2.162492452839e+14 ||r(i)||/||b|| 8.889166260017e-03 >> 14 KSP preconditioned resid norm 9.902014249371e+14 true resid norm >> 1.847474743657e+14 ||r(i)||/||b|| 7.594250854373e-03 >> 15 KSP preconditioned resid norm 9.211725470007e+14 true resid norm >> 1.442747306714e+14 ||r(i)||/||b|| 5.930573613672e-03 >> 16 KSP preconditioned resid norm 8.405580663519e+14 true resid norm >> 2.201247255509e+14 ||r(i)||/||b|| 9.048472196027e-03 >> 17 KSP preconditioned resid norm 7.117085749555e+14 true resid norm >> 1.108024781635e+14 ||r(i)||/||b|| 4.554659365975e-03 >> 18 KSP preconditioned resid norm 6.347627750827e+14 true resid norm >> 1.457120949776e+14 ||r(i)||/||b|| 5.989658075575e-03 >> 19 KSP preconditioned resid norm 5.626851770343e+14 true resid norm >> 8.750418282909e+13 ||r(i)||/||b|| 3.596956967843e-03 >> 20 KSP preconditioned resid norm 4.865367652981e+14 true resid norm >> 1.207991751614e+14 ||r(i)||/||b|| 4.965584738448e-03 >> 21 KSP preconditioned resid norm 4.252070053539e+14 true resid norm >> 6.577223220744e+13 ||r(i)||/||b|| 2.703640914986e-03 >> 22 KSP preconditioned resid norm 3.618870286814e+14 true resid norm >> 8.993615915309e+13 ||r(i)||/||b|| 3.696926065336e-03 >> 23 KSP preconditioned resid norm 3.105522471772e+14 true resid norm >> 4.771648977942e+13 ||r(i)||/||b|| 1.961439497451e-03 >> 24 KSP preconditioned resid norm 2.735005931489e+14 true resid norm >> 4.708110884633e+13 ||r(i)||/||b|| 1.935321455997e-03 >> 25 KSP preconditioned resid norm 2.496548247596e+14 true resid norm >> 3.887274137414e+13 ||r(i)||/||b|| 1.597907361960e-03 >> 26 KSP preconditioned resid norm 2.329765393701e+14 true resid norm >> 3.903456593754e+13 ||r(i)||/||b|| 1.604559341009e-03 >> 27 KSP preconditioned resid norm 2.169265400605e+14 true resid norm >> 3.334703074182e+13 ||r(i)||/||b|| 1.370766867430e-03 >> 28 KSP preconditioned resid norm 1.998370208104e+14 true resid norm >> 3.474062054769e+13 ||r(i)||/||b|| 1.428051929704e-03 >> 29 KSP preconditioned resid norm 1.805920498739e+14 true resid norm >> 2.773007978134e+13 ||r(i)||/||b|| 1.139875837515e-03 >> 30 KSP preconditioned resid norm 1.643922523062e+14 true resid norm >> 2.817409671160e+13 ||r(i)||/||b|| 1.158127648337e-03 >> 31 KSP preconditioned resid norm 1.456868079593e+14 true resid norm >> 2.236751337771e+13 ||r(i)||/||b|| 9.194415683468e-04 >> 32 KSP preconditioned resid norm 1.315816507773e+14 true resid norm >> 2.157979387266e+13 ||r(i)||/||b|| 8.870614800950e-04 >> 0 KSP preconditioned resid norm 6.130010040591e+16 true resid norm >> 4.019535303993e+16 ||r(i)||/||b|| 5.748506002867e-02 >> 1 KSP preconditioned resid norm 2.383231267167e+16 true resid norm >> 4.333028232365e+15 ||r(i)||/||b|| 6.196845386481e-03 >> 2 KSP preconditioned resid norm 1.380314332389e+16 true resid norm >> 2.902938687246e+15 ||r(i)||/||b|| 4.151614355275e-03 >> 3 KSP preconditioned resid norm 9.355423030603e+15 true resid norm >> 1.914374573200e+15 ||r(i)||/||b|| 2.737827359010e-03 >> 4 KSP preconditioned resid norm 6.905389736952e+15 true resid norm >> 1.160495154173e+15 ||r(i)||/||b|| 1.659672786910e-03 >> 5 KSP preconditioned resid norm 5.388850338707e+15 true resid norm >> 9.343171842551e+14 ||r(i)||/||b|| 1.336206187053e-03 >> 6 KSP preconditioned resid norm 4.456941671648e+15 true resid norm >> 7.443205431054e+14 ||r(i)||/||b|| 1.064484022780e-03 >> 7 KSP preconditioned resid norm 3.604897929141e+15 true resid norm >> 7.177355376747e+14 ||r(i)||/||b|| 1.026463691635e-03 >> 8 KSP preconditioned resid norm 2.616552572944e+15 true resid norm >> 4.566784838372e+14 ||r(i)||/||b|| 6.531150511630e-04 >> 9 KSP preconditioned resid norm 1.730753896972e+15 true resid norm >> 3.416577764278e+14 ||r(i)||/||b|| 4.886191139484e-04 >> 10 KSP preconditioned resid norm 9.605912905151e+14 true resid norm >> 1.715828170206e+14 ||r(i)||/||b|| 2.453877821777e-04 >> 11 KSP preconditioned resid norm 3.885469679759e+14 true resid norm >> 7.526615556743e+13 ||r(i)||/||b|| 1.076412854646e-04 >> 12 KSP preconditioned resid norm 1.085402187481e+14 true resid norm >> 1.877491024324e+13 ||r(i)||/||b|| 2.685078649001e-05 >> 13 KSP preconditioned resid norm 5.521537327924e+13 true resid norm >> 1.482832280216e+13 ||r(i)||/||b|| 2.120660628506e-05 >> 14 KSP preconditioned resid norm 4.327780981098e+13 true resid norm >> 6.949078996691e+12 ||r(i)||/||b|| 9.938169292156e-06 >> 0 KSP preconditioned resid norm 2.469228993807e+16 true resid norm >> 1.581397706747e+16 ||r(i)||/||b|| 1.421322431696e-01 >> 1 KSP preconditioned resid norm 1.094866250702e+16 true resid norm >> 1.827538407052e+15 ||r(i)||/||b|| 1.642547805430e-02 >> 2 KSP preconditioned resid norm 7.902646639796e+15 true resid norm >> 1.331972302022e+15 ||r(i)||/||b|| 1.197144844200e-02 >> 3 KSP preconditioned resid norm 6.757113896005e+15 true resid norm >> 1.177404713092e+15 ||r(i)||/||b|| 1.058223192536e-02 >> 4 KSP preconditioned resid norm 6.236819639281e+15 true resid norm >> 9.858933018833e+14 ||r(i)||/||b|| 8.860973171059e-03 >> 5 KSP preconditioned resid norm 6.056207345085e+15 true resid norm >> 9.498733496025e+14 ||r(i)||/||b|| 8.537234455953e-03 >> 6 KSP preconditioned resid norm 6.281149615757e+15 true resid norm >> 9.439016000829e+14 ||r(i)||/||b|| 8.483561799717e-03 >> 7 KSP preconditioned resid norm 6.511636840454e+15 true resid norm >> 1.175604965975e+15 ||r(i)||/||b|| 1.056605622877e-02 >> 8 KSP preconditioned resid norm 5.897282381771e+15 true resid norm >> 9.358325816295e+14 ||r(i)||/||b|| 8.411039391972e-03 >> 9 KSP preconditioned resid norm 4.858643828149e+15 true resid norm >> 9.074074922507e+14 ||r(i)||/||b|| 8.155561487934e-03 >> 10 KSP preconditioned resid norm 3.431226959566e+15 true resid norm >> 5.523076022464e+14 ||r(i)||/||b|| 4.964008616682e-03 >> 11 KSP preconditioned resid norm 1.790601246330e+15 true resid norm >> 3.233761759516e+14 ||r(i)||/||b|| 2.906427717678e-03 >> 12 KSP preconditioned resid norm 6.061504971272e+14 true resid norm >> 9.361691978333e+13 ||r(i)||/||b|| 8.414064817893e-04 >> 13 KSP preconditioned resid norm 3.669164519745e+14 true resid norm >> 7.433339410578e+13 ||r(i)||/||b|| 6.680907656304e-04 >> 14 KSP preconditioned resid norm 1.028448839513e+15 true resid norm >> 1.739193394480e+14 ||r(i)||/||b|| 1.563145421348e-03 >> 15 KSP preconditioned resid norm 1.340577188948e+14 true resid norm >> 2.477633902186e+13 ||r(i)||/||b|| 2.226838086134e-04 >> 16 KSP preconditioned resid norm 1.124181660974e+14 true resid norm >> 1.917872742131e+13 ||r(i)||/||b|| 1.723738144997e-04 >> 17 KSP preconditioned resid norm 9.394070158671e+13 true resid norm >> 1.682505899954e+13 ||r(i)||/||b|| 1.512196057236e-04 >> 18 KSP preconditioned resid norm 9.034650415597e+13 true resid norm >> 1.892962426081e+13 ||r(i)||/||b|| 1.701349348787e-04 >> 19 KSP preconditioned resid norm 8.058877605551e+13 true resid norm >> 1.275341142002e+13 ||r(i)||/||b|| 1.146246112195e-04 >> 20 KSP preconditioned resid norm 6.303186171389e+13 true resid norm >> 1.900109222650e+13 ||r(i)||/||b|| 1.707772718592e-04 >> 21 KSP preconditioned resid norm 4.601610061053e+13 true resid norm >> 6.959914424356e+12 ||r(i)||/||b|| 6.255404603042e-05 >> 22 KSP preconditioned resid norm 2.822582332621e+13 true resid norm >> 1.013252988889e+13 ||r(i)||/||b|| 9.106875493415e-05 >> 23 KSP preconditioned resid norm 1.858033653578e+13 true resid norm >> 2.785463510953e+12 ||r(i)||/||b|| 2.503507975190e-05 >> 24 KSP preconditioned resid norm 9.505670647827e+12 true resid norm >> 4.268589348463e+12 ||r(i)||/||b|| 3.836506001486e-05 >> 25 KSP preconditioned resid norm 5.844140386668e+12 true resid norm >> 8.921656809030e+11 ||r(i)||/||b|| 8.018571733391e-06 >> 0 SNES Function norm 0.000000000000e+00 >> 3 TS dt 5e-12 time 9e-12 >> 0 KSP preconditioned resid norm 2.220122692212e+19 true resid norm >> 1.204173138214e+19 ||r(i)||/||b|| 2.302144004449e+02 >> 1 KSP preconditioned resid norm 5.389791318038e+18 true resid norm >> 1.775764523088e+18 ||r(i)||/||b|| 3.394915166605e+01 >> 2 KSP preconditioned resid norm 2.218229032455e+18 true resid norm >> 5.319501602511e+17 ||r(i)||/||b|| 1.016984878025e+01 >> 3 KSP preconditioned resid norm 1.401621992174e+18 true resid norm >> 2.963396938661e+17 ||r(i)||/||b|| 5.665436537857e+00 >> 4 KSP preconditioned resid norm 1.004007060031e+18 true resid norm >> 1.656144361093e+17 ||r(i)||/||b|| 3.166224764861e+00 >> 5 KSP preconditioned resid norm 8.672273682794e+17 true resid norm >> 1.510259905938e+17 ||r(i)||/||b|| 2.887322160975e+00 >> 6 KSP preconditioned resid norm 7.388939161376e+17 true resid norm >> 1.526077665500e+17 ||r(i)||/||b|| 2.917562629878e+00 >> 7 KSP preconditioned resid norm 5.565893055963e+17 true resid norm >> 1.017298978392e+17 ||r(i)||/||b|| 1.944877085792e+00 >> 8 KSP preconditioned resid norm 3.566491384538e+17 true resid norm >> 6.306517676931e+16 ||r(i)||/||b|| 1.205683086440e+00 >> 9 KSP preconditioned resid norm 1.893313432840e+17 true resid norm >> 3.267997700517e+16 ||r(i)||/||b|| 6.247773741206e-01 >> 10 KSP preconditioned resid norm 6.895898463878e+16 true resid norm >> 1.153123828824e+16 ||r(i)||/||b|| 2.204547688925e-01 >> 11 KSP preconditioned resid norm 5.104999855255e+16 true resid norm >> 9.275473058095e+15 ||r(i)||/||b|| 1.773289405940e-01 >> 12 KSP preconditioned resid norm 4.992747088606e+16 true resid norm >> 8.399049844320e+15 ||r(i)||/||b|| 1.605734393881e-01 >> 13 KSP preconditioned resid norm 4.920015136673e+16 true resid norm >> 8.426577623649e+15 ||r(i)||/||b|| 1.610997167989e-01 >> 14 KSP preconditioned resid norm 5.688370526613e+16 true resid norm >> 9.763030126011e+15 ||r(i)||/||b|| 1.866500800972e-01 >> 15 KSP preconditioned resid norm 1.613946025228e+17 true resid norm >> 4.341380499600e+16 ||r(i)||/||b|| 8.299872145473e-01 >> 16 KSP preconditioned resid norm 4.125620543313e+16 true resid norm >> 7.099475531667e+15 ||r(i)||/||b|| 1.357281150965e-01 >> 17 KSP preconditioned resid norm 2.163115342752e+16 true resid norm >> 5.850497008723e+15 ||r(i)||/||b|| 1.118500835491e-01 >> 18 KSP preconditioned resid norm 1.414953599369e+16 true resid norm >> 2.432122296311e+15 ||r(i)||/||b|| 4.649743118207e-02 >> 19 KSP preconditioned resid norm 1.088056550362e+16 true resid norm >> 2.488538030816e+15 ||r(i)||/||b|| 4.757598991109e-02 >> 20 KSP preconditioned resid norm 9.058853458567e+15 true resid norm >> 1.587978910975e+15 ||r(i)||/||b|| 3.035905729067e-02 >> 21 KSP preconditioned resid norm 7.813748136791e+15 true resid norm >> 1.345999130102e+15 ||r(i)||/||b|| 2.573287618717e-02 >> 22 KSP preconditioned resid norm 6.941551759120e+15 true resid norm >> 1.184314230023e+15 ||r(i)||/||b|| 2.264177648138e-02 >> 23 KSP preconditioned resid norm 6.244923673699e+15 true resid norm >> 1.023439246806e+15 ||r(i)||/||b|| 1.956616080515e-02 >> 24 KSP preconditioned resid norm 5.681880462174e+15 true resid norm >> 9.491910634069e+14 ||r(i)||/||b|| 1.814668045943e-02 >> 25 KSP preconditioned resid norm 5.283519017995e+15 true resid norm >> 8.557005279630e+14 ||r(i)||/||b|| 1.635932390068e-02 >> 26 KSP preconditioned resid norm 5.117675200632e+15 true resid norm >> 8.569254569155e+14 ||r(i)||/||b|| 1.638274215138e-02 >> 27 KSP preconditioned resid norm 4.944059018035e+15 true resid norm >> 9.547778379458e+14 ||r(i)||/||b|| 1.825348868410e-02 >> 28 KSP preconditioned resid norm 4.840428844487e+15 true resid norm >> 7.569152804275e+14 ||r(i)||/||b|| 1.447074278120e-02 >> 29 KSP preconditioned resid norm 4.909458338739e+15 true resid norm >> 7.780937746322e+14 ||r(i)||/||b|| 1.487563425328e-02 >> 30 KSP preconditioned resid norm 5.172548892535e+15 true resid norm >> 8.119476371329e+14 ||r(i)||/||b|| 1.552285402683e-02 >> 31 KSP preconditioned resid norm 5.734458283292e+15 true resid norm >> 9.082595641858e+14 ||r(i)||/||b|| 1.736415008622e-02 >> 32 KSP preconditioned resid norm 8.602942487649e+15 true resid norm >> 1.383459272467e+15 ||r(i)||/||b|| 2.644904099284e-02 >> 33 KSP preconditioned resid norm 1.681001901654e+16 true resid norm >> 2.705594423024e+15 ||r(i)||/||b|| 5.172568446988e-02 >> 34 KSP preconditioned resid norm 3.498037913905e+16 true resid norm >> 5.724738344713e+15 ||r(i)||/||b|| 1.094458233545e-01 >> 35 KSP preconditioned resid norm 6.660621055336e+15 true resid norm >> 1.092449655644e+15 ||r(i)||/||b|| 2.088550512457e-02 >> 36 KSP preconditioned resid norm 3.564205941873e+15 true resid norm >> 5.832825274169e+14 ||r(i)||/||b|| 1.115122344769e-02 >> 37 KSP preconditioned resid norm 2.275128811690e+15 true resid norm >> 4.150874600928e+14 ||r(i)||/||b|| 7.935662050992e-03 >> 38 KSP preconditioned resid norm 1.923021972098e+15 true resid norm >> 3.105449109783e+14 ||r(i)||/||b|| 5.937012562674e-03 >> 39 KSP preconditioned resid norm 1.756400018558e+15 true resid norm >> 3.028037938881e+14 ||r(i)||/||b|| 5.789017513361e-03 >> 40 KSP preconditioned resid norm 1.618977002834e+15 true resid norm >> 2.591473194248e+14 ||r(i)||/||b|| 4.954390932253e-03 >> 41 KSP preconditioned resid norm 1.449378745707e+15 true resid norm >> 2.366294470672e+14 ||r(i)||/||b|| 4.523893164149e-03 >> 42 KSP preconditioned resid norm 1.311727507569e+15 true resid norm >> 2.162961405943e+14 ||r(i)||/||b|| 4.135160031831e-03 >> 43 KSP preconditioned resid norm 1.220101213287e+15 true resid norm >> 1.960093358179e+14 ||r(i)||/||b|| 3.747315921186e-03 >> 44 KSP preconditioned resid norm 1.141924730669e+15 true resid norm >> 1.884563341884e+14 ||r(i)||/||b|| 3.602917272310e-03 >> 45 KSP preconditioned resid norm 1.062154013672e+15 true resid norm >> 2.268601642641e+14 ||r(i)||/||b|| 4.337123545072e-03 >> 46 KSP preconditioned resid norm 1.050847341809e+15 true resid norm >> 1.644035317108e+14 ||r(i)||/||b|| 3.143074636258e-03 >> 47 KSP preconditioned resid norm 8.268920862881e+14 true resid norm >> 2.491484388741e+14 ||r(i)||/||b|| 4.763231852378e-03 >> 48 KSP preconditioned resid norm 6.970056044801e+14 true resid norm >> 1.086553600705e+14 ||r(i)||/||b|| 2.077278406232e-03 >> 49 KSP preconditioned resid norm 8.789823309588e+14 true resid norm >> 1.565701303869e+14 ||r(i)||/||b|| 2.993315292522e-03 >> 50 KSP preconditioned resid norm 9.889574125697e+14 true resid norm >> 1.557528361421e+14 ||r(i)||/||b|| 2.977690221792e-03 >> 51 KSP preconditioned resid norm 3.685341349551e+14 true resid norm >> 5.749337875960e+13 ||r(i)||/||b|| 1.099161183775e-03 >> 52 KSP preconditioned resid norm 2.728364862859e+15 true resid norm >> 4.745949443528e+14 ||r(i)||/||b|| 9.073329000715e-03 >> 53 KSP preconditioned resid norm 8.484691724076e+15 true resid norm >> 1.333040899485e+15 ||r(i)||/||b|| 2.548514011022e-02 >> 54 KSP preconditioned resid norm 8.341541248747e+13 true resid norm >> 1.303089350822e+13 ||r(i)||/||b|| 2.491252496055e-04 >> 0 KSP preconditioned resid norm 6.542890621126e+18 true resid norm >> 3.545938224251e+18 ||r(i)||/||b|| 4.156801777589e+02 >> 1 KSP preconditioned resid norm 1.591173149842e+18 true resid norm >> 5.223204285160e+17 ||r(i)||/||b|| 6.123012721646e+01 >> 2 KSP preconditioned resid norm 6.570590492388e+17 true resid norm >> 1.552517142001e+17 ||r(i)||/||b|| 1.819971360886e+01 >> 3 KSP preconditioned resid norm 4.165858181866e+17 true resid norm >> 8.738575659546e+16 ||r(i)||/||b|| 1.024398185698e+01 >> 4 KSP preconditioned resid norm 2.995041215234e+17 true resid norm >> 4.934730887237e+16 ||r(i)||/||b|| 5.784843622966e+00 >> 5 KSP preconditioned resid norm 2.594520327186e+17 true resid norm >> 4.469839192099e+16 ||r(i)||/||b|| 5.239864409419e+00 >> 6 KSP preconditioned resid norm 2.219632225075e+17 true resid norm >> 4.652171577038e+16 ||r(i)||/||b|| 5.453607439866e+00 >> 7 KSP preconditioned resid norm 1.674928029034e+17 true resid norm >> 2.988841295071e+16 ||r(i)||/||b|| 3.503733010156e+00 >> 8 KSP preconditioned resid norm 1.081911414020e+17 true resid norm >> 1.908165723430e+16 ||r(i)||/||b|| 2.236888002402e+00 >> 9 KSP preconditioned resid norm 5.710171826154e+16 true resid norm >> 9.635278906551e+15 ||r(i)||/||b|| 1.129516137997e+00 >> 10 KSP preconditioned resid norm 1.779377501866e+16 true resid norm >> 2.952494296810e+15 ||r(i)||/||b|| 3.461124465555e-01 >> 11 KSP preconditioned resid norm 1.125612460139e+16 true resid norm >> 2.030403352863e+15 ||r(i)||/||b|| 2.380183672880e-01 >> 12 KSP preconditioned resid norm 1.086634567103e+16 true resid norm >> 1.877885131752e+15 ||r(i)||/||b|| 2.201390932416e-01 >> 13 KSP preconditioned resid norm 1.075395224737e+16 true resid norm >> 1.822688133743e+15 ||r(i)||/||b|| 2.136685073225e-01 >> 14 KSP preconditioned resid norm 1.173925328196e+16 true resid norm >> 2.130469004546e+15 ||r(i)||/||b|| 2.497487769141e-01 >> 15 KSP preconditioned resid norm 4.944016549258e+16 true resid norm >> 1.361939461929e+16 ||r(i)||/||b|| 1.596562607209e+00 >> 16 KSP preconditioned resid norm 1.780247590037e+17 true resid norm >> 3.370503336823e+16 ||r(i)||/||b|| 3.951144485840e+00 >> 17 KSP preconditioned resid norm 3.251104571145e+16 true resid norm >> 8.719659549123e+15 ||r(i)||/||b|| 1.022180704274e+00 >> 18 KSP preconditioned resid norm 2.075437272594e+16 true resid norm >> 3.944434594682e+15 ||r(i)||/||b|| 4.623947654424e-01 >> 19 KSP preconditioned resid norm 1.483444135710e+16 true resid norm >> 3.390829411015e+15 ||r(i)||/||b|| 3.974972160206e-01 >> 20 KSP preconditioned resid norm 1.134313319987e+16 true resid norm >> 2.489467119154e+15 ||r(i)||/||b|| 2.918330972428e-01 >> 21 KSP preconditioned resid norm 8.553477032620e+15 true resid norm >> 1.657674168535e+15 ||r(i)||/||b|| 1.943243930000e-01 >> 22 KSP preconditioned resid norm 6.586595023874e+15 true resid norm >> 1.286251502369e+15 ||r(i)||/||b|| 1.507835780925e-01 >> 23 KSP preconditioned resid norm 4.780400299659e+15 true resid norm >> 9.064518348736e+14 ||r(i)||/||b|| 1.062607513219e-01 >> 24 KSP preconditioned resid norm 3.028073960021e+15 true resid norm >> 6.179142836679e+14 ||r(i)||/||b|| 7.243632094834e-02 >> 25 KSP preconditioned resid norm 1.570529076731e+15 true resid norm >> 2.755631911412e+14 ||r(i)||/||b|| 3.230348331903e-02 >> 26 KSP preconditioned resid norm 1.180384548694e+15 true resid norm >> 2.278461878994e+14 ||r(i)||/||b|| 2.670975575377e-02 >> 27 KSP preconditioned resid norm 1.075603583388e+15 true resid norm >> 2.036512758236e+14 ||r(i)||/||b|| 2.387345553745e-02 >> 28 KSP preconditioned resid norm 1.013485206919e+15 true resid norm >> 1.671016982388e+14 ||r(i)||/||b|| 1.958885328364e-02 >> 29 KSP preconditioned resid norm 9.469966879519e+14 true resid norm >> 1.801244790801e+14 ||r(i)||/||b|| 2.111547656715e-02 >> 30 KSP preconditioned resid norm 8.991813841494e+14 true resid norm >> 1.459100485336e+14 ||r(i)||/||b|| 1.710461690970e-02 >> 31 KSP preconditioned resid norm 8.677534755734e+14 true resid norm >> 1.581142867341e+14 ||r(i)||/||b|| 1.853528478482e-02 >> 32 KSP preconditioned resid norm 8.336211910215e+14 true resid norm >> 1.336646371267e+14 ||r(i)||/||b|| 1.566912241756e-02 >> 33 KSP preconditioned resid norm 8.025169227655e+14 true resid norm >> 1.586001394574e+14 ||r(i)||/||b|| 1.859223990744e-02 >> 34 KSP preconditioned resid norm 7.724033526395e+14 true resid norm >> 1.340849232530e+14 ||r(i)||/||b|| 1.571839135589e-02 >> 35 KSP preconditioned resid norm 7.664262315912e+14 true resid norm >> 1.204381005271e+14 ||r(i)||/||b|| 1.411861343033e-02 >> 36 KSP preconditioned resid norm 7.727266103384e+14 true resid norm >> 1.292430217432e+14 ||r(i)||/||b|| 1.515078911553e-02 >> 37 KSP preconditioned resid norm 8.047214262262e+14 true resid norm >> 1.267167533285e+14 ||r(i)||/||b|| 1.485464190786e-02 >> 38 KSP preconditioned resid norm 8.419506152984e+14 true resid norm >> 1.380565821207e+14 ||r(i)||/||b|| 1.618397754485e-02 >> 39 KSP preconditioned resid norm 8.796981754885e+14 true resid norm >> 1.388509364066e+14 ||r(i)||/||b|| 1.627709742171e-02 >> 40 KSP preconditioned resid norm 9.432799067464e+14 true resid norm >> 1.525727137977e+14 ||r(i)||/||b|| 1.788566206789e-02 >> 41 KSP preconditioned resid norm 1.193594717717e+15 true resid norm >> 1.887759239390e+14 ||r(i)||/||b|| 2.212966065874e-02 >> 42 KSP preconditioned resid norm 2.117323854280e+15 true resid norm >> 3.659661973675e+14 ||r(i)||/||b|| 4.290116870483e-02 >> 43 KSP preconditioned resid norm 1.749515464589e+16 true resid norm >> 2.774904343458e+15 ||r(i)||/||b|| 3.252940851772e-01 >> 44 KSP preconditioned resid norm 1.219803820486e+15 true resid norm >> 2.038867414461e+14 ||r(i)||/||b|| 2.390105849768e-02 >> 45 KSP preconditioned resid norm 6.218668840796e+14 true resid norm >> 9.918031040094e+13 ||r(i)||/||b|| 1.162662360435e-02 >> 46 KSP preconditioned resid norm 4.213016096566e+14 true resid norm >> 7.032265378826e+13 ||r(i)||/||b|| 8.243723206248e-03 >> 47 KSP preconditioned resid norm 2.932264460599e+14 true resid norm >> 4.681093096736e+13 ||r(i)||/||b|| 5.487511308712e-03 >> 48 KSP preconditioned resid norm 1.890454719223e+14 true resid norm >> 3.330632990756e+13 ||r(i)||/||b|| 3.904405621560e-03 >> 49 KSP preconditioned resid norm 1.361587907879e+14 true resid norm >> 2.180429132446e+13 ||r(i)||/||b|| 2.556054595557e-03 >> 50 KSP preconditioned resid norm 1.041049830937e+14 true resid norm >> 1.811134568183e+13 ||r(i)||/||b|| 2.123141159366e-03 >> 51 KSP preconditioned resid norm 9.375298557199e+13 true resid norm >> 1.575768352161e+13 ||r(i)||/||b|| 1.847228088333e-03 >> 52 KSP preconditioned resid norm 1.220175150775e+14 true resid norm >> 1.950548748269e+13 ||r(i)||/||b|| 2.286572408009e-03 >> 53 KSP preconditioned resid norm 7.051895161295e+13 true resid norm >> 1.295324819768e+13 ||r(i)||/||b|| 1.518472170932e-03 >> 54 KSP preconditioned resid norm 5.292501274648e+13 true resid norm >> 8.611444708372e+12 ||r(i)||/||b|| 1.009494988563e-03 >> 0 KSP preconditioned resid norm 1.892610202067e+19 true resid norm >> 1.028308040523e+19 ||r(i)||/||b|| 7.841022917690e+01 >> 1 KSP preconditioned resid norm 4.581169591238e+18 true resid norm >> 1.519782710159e+18 ||r(i)||/||b|| 1.158860048804e+01 >> 2 KSP preconditioned resid norm 1.874457569158e+18 true resid norm >> 4.646597685037e+17 ||r(i)||/||b|| 3.543109409036e+00 >> 3 KSP preconditioned resid norm 1.173257076173e+18 true resid norm >> 2.550995626245e+17 ||r(i)||/||b|| 1.945177357373e+00 >> 4 KSP preconditioned resid norm 8.301427099842e+17 true resid norm >> 1.381385637907e+17 ||r(i)||/||b|| 1.053329937932e+00 >> 5 KSP preconditioned resid norm 7.053087339556e+17 true resid norm >> 1.287851939525e+17 ||r(i)||/||b|| 9.820089092432e-01 >> 6 KSP preconditioned resid norm 5.863912202870e+17 true resid norm >> 1.166954217660e+17 ||r(i)||/||b|| 8.898223493329e-01 >> 7 KSP preconditioned resid norm 4.330693111087e+17 true resid norm >> 8.611440299479e+16 ||r(i)||/||b|| 6.566369033559e-01 >> 8 KSP preconditioned resid norm 2.702458300388e+17 true resid norm >> 4.803465439349e+16 ||r(i)||/||b|| 3.662723727716e-01 >> 9 KSP preconditioned resid norm 1.546904751775e+17 true resid norm >> 2.878502712633e+16 ||r(i)||/||b|| 2.194907055953e-01 >> 10 KSP preconditioned resid norm 9.910545254967e+16 true resid norm >> 1.648892832929e+16 ||r(i)||/||b|| 1.257308703453e-01 >> 11 KSP preconditioned resid norm 9.242232753221e+16 true resid norm >> 1.559261342084e+16 ||r(i)||/||b|| 1.188963174082e-01 >> 12 KSP preconditioned resid norm 9.244969774564e+16 true resid norm >> 1.506990436290e+16 ||r(i)||/||b|| 1.149105723385e-01 >> 13 KSP preconditioned resid norm 7.091380549056e+16 true resid norm >> 1.150476534699e+16 ||r(i)||/||b|| 8.772578370826e-02 >> 14 KSP preconditioned resid norm 2.015813147523e+17 true resid norm >> 3.265267485752e+16 ||r(i)||/||b|| 2.489821743992e-01 >> 15 KSP preconditioned resid norm 1.668693825394e+16 true resid norm >> 4.392881515438e+15 ||r(i)||/||b|| 3.349646533904e-02 >> 16 KSP preconditioned resid norm 1.171106201192e+16 true resid norm >> 1.864120995445e+15 ||r(i)||/||b|| 1.421423821522e-02 >> 17 KSP preconditioned resid norm 1.020170882678e+16 true resid norm >> 1.999245817269e+15 ||r(i)||/||b|| 1.524458786039e-02 >> 18 KSP preconditioned resid norm 9.407404703690e+15 true resid norm >> 1.539877936168e+15 ||r(i)||/||b|| 1.174182998880e-02 >> 19 KSP preconditioned resid norm 8.812299187434e+15 true resid norm >> 1.424937011002e+15 ||r(i)||/||b|| 1.086538597310e-02 >> 20 KSP preconditioned resid norm 8.313368595823e+15 true resid norm >> 1.488773269648e+15 ||r(i)||/||b|| 1.135214825376e-02 >> 21 KSP preconditioned resid norm 7.851626197782e+15 true resid norm >> 1.250085440746e+15 ||r(i)||/||b|| 9.532113144783e-03 >> 22 KSP preconditioned resid norm 7.455044400782e+15 true resid norm >> 1.324254107888e+15 ||r(i)||/||b|| 1.009766178966e-02 >> 23 KSP preconditioned resid norm 7.129687083603e+15 true resid norm >> 1.124533100660e+15 ||r(i)||/||b|| 8.574755293646e-03 >> 24 KSP preconditioned resid norm 5.784281419699e+15 true resid norm >> 2.057123935555e+15 ||r(i)||/||b|| 1.568591831199e-02 >> 25 KSP preconditioned resid norm 4.924869719264e+15 true resid norm >> 7.825081154476e+14 ||r(i)||/||b|| 5.966756871199e-03 >> 26 KSP preconditioned resid norm 5.016674985414e+15 true resid norm >> 7.817733101871e+14 ||r(i)||/||b|| 5.961153856674e-03 >> 27 KSP preconditioned resid norm 5.308634596254e+15 true resid norm >> 8.859615986698e+14 ||r(i)||/||b|| 6.755607197068e-03 >> 28 KSP preconditioned resid norm 9.149615612203e+15 true resid norm >> 1.429126026168e+15 ||r(i)||/||b|| 1.089732792301e-02 >> 29 KSP preconditioned resid norm 8.897313413659e+15 true resid norm >> 2.002400213933e+15 ||r(i)||/||b|| 1.526864066904e-02 >> 30 KSP preconditioned resid norm 2.145204463192e+15 true resid norm >> 3.349465461999e+14 ||r(i)||/||b|| 2.554024126484e-03 >> 31 KSP preconditioned resid norm 1.279996253934e+15 true resid norm >> 2.794251271078e+14 ||r(i)||/||b|| 2.130663905259e-03 >> 32 KSP preconditioned resid norm 8.936440443654e+14 true resid norm >> 1.396095375555e+14 ||r(i)||/||b|| 1.064546361948e-03 >> 0 KSP preconditioned resid norm 6.470811776559e+18 true resid norm >> 3.511555308471e+18 ||r(i)||/||b|| 1.662220839803e+02 >> 1 KSP preconditioned resid norm 1.571601370240e+18 true resid norm >> 5.187483735196e+17 ||r(i)||/||b|| 2.455534033589e+01 >> 2 KSP preconditioned resid norm 6.468356854753e+17 true resid norm >> 1.566993981909e+17 ||r(i)||/||b|| 7.417482635946e+00 >> 3 KSP preconditioned resid norm 4.084058718904e+17 true resid norm >> 8.688424998267e+16 ||r(i)||/||b|| 4.112730636007e+00 >> 4 KSP preconditioned resid norm 2.920904676538e+17 true resid norm >> 4.831974981420e+16 ||r(i)||/||b|| 2.287251319137e+00 >> 5 KSP preconditioned resid norm 2.515876340477e+17 true resid norm >> 4.435582123557e+16 ||r(i)||/||b|| 2.099615809737e+00 >> 6 KSP preconditioned resid norm 2.133089016156e+17 true resid norm >> 4.355659888860e+16 ||r(i)||/||b|| 2.061784025127e+00 >> 7 KSP preconditioned resid norm 1.601295948028e+17 true resid norm >> 2.999342671961e+16 ||r(i)||/||b|| 1.419761176199e+00 >> 8 KSP preconditioned resid norm 1.019894482623e+17 true resid norm >> 1.811123232414e+16 ||r(i)||/||b|| 8.573086612382e-01 >> 9 KSP preconditioned resid norm 5.552358385594e+16 true resid norm >> 9.849609129144e+15 ||r(i)||/||b|| 4.662385786400e-01 >> 10 KSP preconditioned resid norm 2.614302078211e+16 true resid norm >> 4.400897857400e+15 ||r(i)||/||b|| 2.083197754216e-01 >> 11 KSP preconditioned resid norm 2.230412831734e+16 true resid norm >> 4.001989581448e+15 ||r(i)||/||b|| 1.894371552944e-01 >> 12 KSP preconditioned resid norm 2.202852939300e+16 true resid norm >> 3.668964514289e+15 ||r(i)||/||b|| 1.736731658885e-01 >> 13 KSP preconditioned resid norm 2.171413942501e+16 true resid norm >> 3.746648405904e+15 ||r(i)||/||b|| 1.773503907139e-01 >> 14 KSP preconditioned resid norm 2.756569557917e+16 true resid norm >> 4.738274109991e+15 ||r(i)||/||b|| 2.242897314282e-01 >> 15 KSP preconditioned resid norm 1.911125733606e+16 true resid norm >> 7.273426207009e+15 ||r(i)||/||b|| 3.442930427122e-01 >> 16 KSP preconditioned resid norm 8.267301781208e+15 true resid norm >> 1.450502353903e+15 ||r(i)||/||b|| 6.866060844958e-02 >> 17 KSP preconditioned resid norm 5.452175853074e+15 true resid norm >> 1.779140334910e+15 ||r(i)||/||b|| 8.421693186737e-02 >> 18 KSP preconditioned resid norm 4.185422899260e+15 true resid norm >> 7.352001585133e+14 ||r(i)||/||b|| 3.480124667150e-02 >> 19 KSP preconditioned resid norm 3.483100692842e+15 true resid norm >> 7.680069742951e+14 ||r(i)||/||b|| 3.635418171280e-02 >> 20 KSP preconditioned resid norm 3.028709630445e+15 true resid norm >> 5.664408475792e+14 ||r(i)||/||b|| 2.681289909034e-02 >> 21 KSP preconditioned resid norm 2.650986791756e+15 true resid norm >> 5.333477625647e+14 ||r(i)||/||b|| 2.524641328184e-02 >> 22 KSP preconditioned resid norm 2.409785359425e+15 true resid norm >> 3.913611824173e+14 ||r(i)||/||b|| 1.852537283791e-02 >> 23 KSP preconditioned resid norm 2.246473514462e+15 true resid norm >> 3.515886291592e+14 ||r(i)||/||b|| 1.664270942896e-02 >> 24 KSP preconditioned resid norm 2.105216345845e+15 true resid norm >> 3.429537728013e+14 ||r(i)||/||b|| 1.623397207682e-02 >> 25 KSP preconditioned resid norm 1.993254024352e+15 true resid norm >> 3.120073169264e+14 ||r(i)||/||b|| 1.476909855627e-02 >> 26 KSP preconditioned resid norm 1.937764630771e+15 true resid norm >> 3.193060027359e+14 ||r(i)||/||b|| 1.511458728106e-02 >> 27 KSP preconditioned resid norm 1.841409046328e+15 true resid norm >> 4.276359096161e+14 ||r(i)||/||b|| 2.024246404712e-02 >> 28 KSP preconditioned resid norm 1.798600034298e+15 true resid norm >> 2.807262398909e+14 ||r(i)||/||b|| 1.328838549404e-02 >> 29 KSP preconditioned resid norm 1.909156742948e+15 true resid norm >> 3.075705616539e+14 ||r(i)||/||b|| 1.455908144342e-02 >> 30 KSP preconditioned resid norm 2.243056909373e+15 true resid norm >> 3.518787704137e+14 ||r(i)||/||b|| 1.665644348118e-02 >> 31 KSP preconditioned resid norm 3.125791809917e+15 true resid norm >> 5.051797337710e+14 ||r(i)||/||b|| 2.391305867501e-02 >> 32 KSP preconditioned resid norm 2.268406906040e+16 true resid norm >> 3.663605847107e+15 ||r(i)||/||b|| 1.734195093893e-01 >> 33 KSP preconditioned resid norm 2.663631019961e+15 true resid norm >> 4.418031676456e+14 ||r(i)||/||b|| 2.091308174984e-02 >> 34 KSP preconditioned resid norm 1.194136462714e+15 true resid norm >> 1.964734637439e+14 ||r(i)||/||b|| 9.300217630505e-03 >> 35 KSP preconditioned resid norm 7.378290473645e+14 true resid norm >> 1.245828993721e+14 ||r(i)||/||b|| 5.897224261846e-03 >> 36 KSP preconditioned resid norm 5.578220476287e+14 true resid norm >> 9.049566410806e+13 ||r(i)||/||b|| 4.283679611405e-03 >> 37 KSP preconditioned resid norm 4.459209275003e+14 true resid norm >> 9.101333159944e+13 ||r(i)||/||b|| 4.308183787380e-03 >> 38 KSP preconditioned resid norm 4.020379023591e+14 true resid norm >> 6.381912771158e+13 ||r(i)||/||b|| 3.020925907227e-03 >> 39 KSP preconditioned resid norm 3.758207934261e+14 true resid norm >> 7.185355109637e+13 ||r(i)||/||b|| 3.401241317717e-03 >> 40 KSP preconditioned resid norm 3.524000257362e+14 true resid norm >> 5.552480372659e+13 ||r(i)||/||b|| 2.628307908397e-03 >> 41 KSP preconditioned resid norm 3.288683013967e+14 true resid norm >> 5.672396056313e+13 ||r(i)||/||b|| 2.685070889720e-03 >> 42 KSP preconditioned resid norm 3.091159619430e+14 true resid norm >> 5.054650403677e+13 ||r(i)||/||b|| 2.392656387510e-03 >> 43 KSP preconditioned resid norm 2.914764729338e+14 true resid norm >> 5.652869757159e+13 ||r(i)||/||b|| 2.675827970692e-03 >> 44 KSP preconditioned resid norm 2.823730992718e+14 true resid norm >> 4.387571428023e+13 ||r(i)||/||b|| 2.076889589689e-03 >> 45 KSP preconditioned resid norm 2.874969520743e+14 true resid norm >> 4.617969867525e+13 ||r(i)||/||b|| 2.185950405754e-03 >> 46 KSP preconditioned resid norm 3.038236713170e+14 true resid norm >> 4.761130615995e+13 ||r(i)||/||b|| 2.253716611508e-03 >> 47 KSP preconditioned resid norm 3.972150696557e+14 true resid norm >> 6.226543710285e+13 ||r(i)||/||b|| 2.947380805938e-03 >> 48 KSP preconditioned resid norm 7.035007479461e+14 true resid norm >> 1.474495546291e+14 ||r(i)||/||b|| 6.979634406808e-03 >> 49 KSP preconditioned resid norm 2.896572855920e+13 true resid norm >> 4.634266191547e+12 ||r(i)||/||b|| 2.193664392014e-04 >> 0 KSP preconditioned resid norm 6.694681597421e+18 true resid norm >> 3.650754072308e+18 ||r(i)||/||b|| 5.229798618159e+00 >> 1 KSP preconditioned resid norm 1.597667334547e+18 true resid norm >> 5.366749713724e+17 ||r(i)||/||b|| 7.688006280602e-01 >> 2 KSP preconditioned resid norm 6.300446759218e+17 true resid norm >> 1.705328267289e+17 ||r(i)||/||b|| 2.442926376067e-01 >> 3 KSP preconditioned resid norm 3.642174486076e+17 true resid norm >> 9.481576843120e+16 ||r(i)||/||b|| 1.358260142699e-01 >> 4 KSP preconditioned resid norm 2.247332959633e+17 true resid norm >> 3.802800729913e+16 ||r(i)||/||b|| 5.447609345502e-02 >> 5 KSP preconditioned resid norm 1.557523701694e+17 true resid norm >> 3.829141999854e+16 ||r(i)||/||b|| 5.485343888670e-02 >> 6 KSP preconditioned resid norm 1.004692846458e+17 true resid norm >> 1.699728099753e+16 ||r(i)||/||b|| 2.434903992784e-02 >> 7 KSP preconditioned resid norm 7.103482800944e+16 true resid norm >> 1.472858884839e+16 ||r(i)||/||b|| 2.109908037658e-02 >> 8 KSP preconditioned resid norm 5.766847884006e+16 true resid norm >> 9.397954060610e+15 ||r(i)||/||b|| 1.346280965144e-02 >> 9 KSP preconditioned resid norm 4.523472015836e+16 true resid norm >> 1.246374917128e+16 ||r(i)||/||b|| 1.785463959006e-02 >> 10 KSP preconditioned resid norm 3.195342481549e+16 true resid norm >> 5.030521483345e+15 ||r(i)||/||b|| 7.206350737723e-03 >> 11 KSP preconditioned resid norm 2.437460220100e+16 true resid norm >> 4.974645944970e+15 ||r(i)||/||b|| 7.126307599347e-03 >> 12 KSP preconditioned resid norm 1.258045555912e+16 true resid norm >> 1.885910304740e+15 ||r(i)||/||b|| 2.701614765157e-03 >> 13 KSP preconditioned resid norm 2.495567167662e+15 true resid norm >> 4.460991814847e+14 ||r(i)||/||b|| 6.390484915397e-04 >> 14 KSP preconditioned resid norm 7.678096336284e+14 true resid norm >> 1.316276217008e+14 ||r(i)||/||b|| 1.885599359606e-04 >> 15 KSP preconditioned resid norm 5.950867488217e+14 true resid norm >> 1.439410415068e+14 ||r(i)||/||b|| 2.061992248885e-04 >> 16 KSP preconditioned resid norm 3.263035982098e+14 true resid norm >> 5.046543709851e+13 ||r(i)||/||b|| 7.229302987142e-05 >> 17 KSP preconditioned resid norm 1.436782483869e+14 true resid norm >> 7.588970331259e+13 ||r(i)||/||b|| 1.087139417380e-04 >> 18 KSP preconditioned resid norm 5.401533723431e+13 true resid norm >> 8.625318760812e+12 ||r(i)||/||b|| 1.235598981554e-05 >> 19 KSP preconditioned resid norm 3.528859982338e+13 true resid norm >> 1.233587619696e+13 ||r(i)||/||b|| 1.767145828255e-05 >> 0 KSP preconditioned resid norm 4.827874033276e+18 true resid norm >> 2.629049503582e+18 ||r(i)||/||b|| 2.387244449971e+01 >> 1 KSP preconditioned resid norm 1.161059891938e+18 true resid norm >> 3.885117259710e+17 ||r(i)||/||b|| 3.527786222014e+00 >> 2 KSP preconditioned resid norm 4.696931551640e+17 true resid norm >> 1.228954734511e+17 ||r(i)||/||b|| 1.115922452289e+00 >> 3 KSP preconditioned resid norm 2.870101730651e+17 true resid norm >> 6.759381584873e+16 ||r(i)||/||b|| 6.137692025860e-01 >> 4 KSP preconditioned resid norm 1.966639305477e+17 true resid norm >> 3.310590025777e+16 ||r(i)||/||b|| 3.006100742644e-01 >> 5 KSP preconditioned resid norm 1.592574496560e+17 true resid norm >> 3.239415772489e+16 ||r(i)||/||b|| 2.941472693263e-01 >> 6 KSP preconditioned resid norm 1.230114823548e+17 true resid norm >> 2.280737281236e+16 ||r(i)||/||b|| 2.070968009182e-01 >> 7 KSP preconditioned resid norm 8.598640814452e+16 true resid norm >> 1.938029193987e+16 ||r(i)||/||b|| 1.759780266946e-01 >> 8 KSP preconditioned resid norm 5.549085767183e+16 true resid norm >> 9.386094645880e+15 ||r(i)||/||b|| 8.522814925985e-02 >> 9 KSP preconditioned resid norm 4.474754629313e+16 true resid norm >> 7.579167805290e+15 ||r(i)||/||b|| 6.882078961970e-02 >> 10 KSP preconditioned resid norm 4.378389532275e+16 true resid norm >> 6.842593279419e+15 ||r(i)||/||b|| 6.213250380964e-02 >> 11 KSP preconditioned resid norm 4.079498017991e+16 true resid norm >> 7.704710537522e+15 ||r(i)||/||b|| 6.996074986140e-02 >> 12 KSP preconditioned resid norm 4.084398127844e+16 true resid norm >> 6.377957981626e+15 ||r(i)||/||b|| 5.791349601078e-02 >> 13 KSP preconditioned resid norm 7.523960004025e+16 true resid norm >> 1.132262464871e+16 ||r(i)||/||b|| 1.028123388886e-01 >> 14 KSP preconditioned resid norm 2.184546709977e+15 true resid norm >> 3.502101186681e+14 ||r(i)||/||b|| 3.179997793785e-03 >> 15 KSP preconditioned resid norm 2.222188254453e+15 true resid norm >> 3.654675413503e+14 ||r(i)||/||b|| 3.318539109075e-03 >> 16 KSP preconditioned resid norm 1.669487552266e+15 true resid norm >> 3.629534589137e+14 ||r(i)||/||b|| 3.295710595061e-03 >> 17 KSP preconditioned resid norm 1.401917499993e+15 true resid norm >> 2.161165604483e+14 ||r(i)||/||b|| 1.962393856693e-03 >> 18 KSP preconditioned resid norm 1.134393254100e+15 true resid norm >> 2.686682335159e+14 ||r(i)||/||b|| 2.439576540763e-03 >> 19 KSP preconditioned resid norm 9.058009036885e+14 true resid norm >> 1.409928994873e+14 ||r(i)||/||b|| 1.280251727203e-03 >> 20 KSP preconditioned resid norm 7.990098022926e+14 true resid norm >> 1.659150954933e+14 ||r(i)||/||b|| 1.506551665700e-03 >> 21 KSP preconditioned resid norm 7.119179981764e+14 true resid norm >> 1.092130722300e+14 ||r(i)||/||b|| 9.916827362515e-04 >> 22 KSP preconditioned resid norm 6.306959704581e+14 true resid norm >> 1.460783700778e+14 ||r(i)||/||b|| 1.326429105857e-03 >> 23 KSP preconditioned resid norm 5.637228678118e+14 true resid norm >> 8.672640313846e+13 ||r(i)||/||b|| 7.874980074592e-04 >> 24 KSP preconditioned resid norm 4.960976390096e+14 true resid norm >> 1.292271182132e+14 ||r(i)||/||b|| 1.173415412376e-03 >> 25 KSP preconditioned resid norm 4.386404742743e+14 true resid norm >> 8.037964656859e+13 ||r(i)||/||b|| 7.298678282782e-04 >> 26 KSP preconditioned resid norm 4.419824802921e+14 true resid norm >> 6.704823034514e+13 ||r(i)||/||b|| 6.088151461346e-04 >> 27 KSP preconditioned resid norm 4.437076279615e+14 true resid norm >> 1.370394110315e+14 ||r(i)||/||b|| 1.244353036968e-03 >> 28 KSP preconditioned resid norm 7.848825686766e+14 true resid norm >> 1.196984074237e+14 ||r(i)||/||b|| 1.086892271915e-03 >> 29 KSP preconditioned resid norm 1.771839364121e+15 true resid norm >> 6.554952272410e+14 ||r(i)||/||b|| 5.952064961431e-03 >> 30 KSP preconditioned resid norm 1.052090569223e+15 true resid norm >> 1.877636113350e+14 ||r(i)||/||b|| 1.704941799139e-03 >> 31 KSP preconditioned resid norm 2.459236824172e+14 true resid norm >> 3.913333475140e+13 ||r(i)||/||b|| 3.553407270074e-04 >> 32 KSP preconditioned resid norm 1.091606883884e+14 true resid norm >> 1.900051073024e+13 ||r(i)||/||b|| 1.725295158025e-04 >> 33 KSP preconditioned resid norm 6.424114279947e+13 true resid norm >> 1.001416869814e+13 ||r(i)||/||b|| 9.093122291205e-05 >> 34 KSP preconditioned resid norm 3.801391191576e+13 true resid norm >> 7.035993054761e+12 ||r(i)||/||b|| 6.388862342504e-05 >> 35 KSP preconditioned resid norm 2.524482106623e+13 true resid norm >> 4.006294246762e+12 ||r(i)||/||b|| 3.637818037470e-05 >> 36 KSP preconditioned resid norm 1.579134058219e+13 true resid norm >> 2.528643754315e+12 ||r(i)||/||b|| 2.296073451724e-05 >> 37 KSP preconditioned resid norm 1.074218958170e+13 true resid norm >> 1.786180378482e+12 ||r(i)||/||b|| 1.621897643756e-05 >> 38 KSP preconditioned resid norm 6.572401461924e+12 true resid norm >> 1.033900044668e+12 ||r(i)||/||b|| 9.388077858923e-06 >> 0 KSP preconditioned resid norm 9.558506771869e+15 true resid norm >> 5.892974320929e+15 ||r(i)||/||b|| 1.119279921775e-01 >> 1 KSP preconditioned resid norm 4.577878026258e+15 true resid norm >> 8.113551231445e+14 ||r(i)||/||b|| 1.541044384903e-02 >> 2 KSP preconditioned resid norm 3.768474917795e+15 true resid norm >> 6.114238708763e+14 ||r(i)||/||b|| 1.161305692331e-02 >> 3 KSP preconditioned resid norm 3.689006206393e+15 true resid norm >> 6.168086911615e+14 ||r(i)||/||b|| 1.171533331040e-02 >> 4 KSP preconditioned resid norm 3.859762637542e+15 true resid norm >> 6.287723791867e+14 ||r(i)||/||b|| 1.194256518123e-02 >> 5 KSP preconditioned resid norm 4.285185348381e+15 true resid norm >> 6.764146832146e+14 ||r(i)||/||b|| 1.284745753985e-02 >> 6 KSP preconditioned resid norm 5.208557345650e+15 true resid norm >> 7.765898979336e+14 ||r(i)||/||b|| 1.475013181583e-02 >> 7 KSP preconditioned resid norm 6.471756089495e+15 true resid norm >> 1.171552566673e+15 ||r(i)||/||b|| 2.225184081532e-02 >> 8 KSP preconditioned resid norm 7.012039525053e+15 true resid norm >> 1.102268075574e+15 ||r(i)||/||b|| 2.093588836833e-02 >> 9 KSP preconditioned resid norm 7.014970322733e+15 true resid norm >> 1.328280835862e+15 ||r(i)||/||b|| 2.522865346245e-02 >> 10 KSP preconditioned resid norm 6.390016130531e+15 true resid norm >> 1.015385597333e+15 ||r(i)||/||b|| 1.928568919633e-02 >> 11 KSP preconditioned resid norm 4.845316666400e+15 true resid norm >> 8.456481888760e+14 ||r(i)||/||b|| 1.606178793844e-02 >> 12 KSP preconditioned resid norm 2.410560854094e+15 true resid norm >> 3.709640804249e+14 ||r(i)||/||b|| 7.045892690295e-03 >> 13 KSP preconditioned resid norm 1.334157374345e+15 true resid norm >> 2.713196530598e+14 ||r(i)||/||b|| 5.153299904502e-03 >> 14 KSP preconditioned resid norm 1.217530036393e+15 true resid norm >> 2.232136761716e+14 ||r(i)||/||b|| 4.239600792372e-03 >> 15 KSP preconditioned resid norm 1.145671346103e+15 true resid norm >> 2.123551517116e+14 ||r(i)||/||b|| 4.033359805286e-03 >> 16 KSP preconditioned resid norm 1.048828510236e+15 true resid norm >> 1.668978069044e+14 ||r(i)||/||b|| 3.169967389692e-03 >> 17 KSP preconditioned resid norm 9.144204694073e+14 true resid norm >> 1.578839852075e+14 ||r(i)||/||b|| 2.998763697051e-03 >> 18 KSP preconditioned resid norm 8.235850213860e+14 true resid norm >> 1.321988157805e+14 ||r(i)||/||b|| 2.510913371198e-03 >> 19 KSP preconditioned resid norm 6.978607225935e+14 true resid norm >> 1.125789224246e+14 ||r(i)||/||b|| 2.138263644512e-03 >> 20 KSP preconditioned resid norm 6.009340490988e+14 true resid norm >> 1.003011158758e+14 ||r(i)||/||b|| 1.905065574997e-03 >> 21 KSP preconditioned resid norm 5.123703002158e+14 true resid norm >> 8.114653676231e+13 ||r(i)||/||b|| 1.541253777351e-03 >> 22 KSP preconditioned resid norm 4.394549158605e+14 true resid norm >> 8.003106905102e+13 ||r(i)||/||b|| 1.520067182184e-03 >> 23 KSP preconditioned resid norm 3.852402878098e+14 true resid norm >> 5.993269339949e+13 ||r(i)||/||b|| 1.138329419521e-03 >> 24 KSP preconditioned resid norm 3.390788847804e+14 true resid norm >> 7.376626581336e+13 ||r(i)||/||b|| 1.401076871080e-03 >> 0 KSP preconditioned resid norm 9.592017367891e+15 true resid norm >> 5.933298102242e+15 ||r(i)||/||b|| 5.704123038824e-01 >> 1 KSP preconditioned resid norm 4.627662883687e+15 true resid norm >> 8.121175703537e+14 ||r(i)||/||b|| 7.807493342594e-02 >> 2 KSP preconditioned resid norm 3.843678575472e+15 true resid norm >> 6.216594738645e+14 ||r(i)||/||b|| 5.976477274644e-02 >> 3 KSP preconditioned resid norm 3.802359495548e+15 true resid norm >> 6.332926539714e+14 ||r(i)||/||b|| 6.088315731972e-02 >> 4 KSP preconditioned resid norm 4.025235313693e+15 true resid norm >> 6.488898363947e+14 ||r(i)||/||b|| 6.238263107055e-02 >> 5 KSP preconditioned resid norm 4.519136017232e+15 true resid norm >> 7.100489487455e+14 ||r(i)||/||b|| 6.826231376612e-02 >> 6 KSP preconditioned resid norm 5.566459161126e+15 true resid norm >> 8.272746471009e+14 ||r(i)||/||b|| 7.953209652789e-02 >> 7 KSP preconditioned resid norm 7.051037635374e+15 true resid norm >> 1.259719337020e+15 ||r(i)||/||b|| 1.211062375246e-01 >> 8 KSP preconditioned resid norm 7.825859719364e+15 true resid norm >> 1.230378249193e+15 ||r(i)||/||b|| 1.182854593980e-01 >> 9 KSP preconditioned resid norm 8.112136766042e+15 true resid norm >> 1.509007778424e+15 ||r(i)||/||b|| 1.450721990763e-01 >> 10 KSP preconditioned resid norm 7.849971536540e+15 true resid norm >> 1.244897342905e+15 ||r(i)||/||b|| 1.196812884212e-01 >> 11 KSP preconditioned resid norm 6.691547560102e+15 true resid norm >> 1.152431148742e+15 ||r(i)||/||b|| 1.107918218994e-01 >> 12 KSP preconditioned resid norm 3.838210718255e+15 true resid norm >> 5.898239658450e+14 ||r(i)||/||b|| 5.670418735838e-02 >> 13 KSP preconditioned resid norm 1.585805121666e+15 true resid norm >> 3.093588848689e+14 ||r(i)||/||b|| 2.974098236828e-02 >> 14 KSP preconditioned resid norm 1.284246821472e+15 true resid norm >> 2.667115224243e+14 ||r(i)||/||b|| 2.564097258496e-02 >> 15 KSP preconditioned resid norm 1.153005952878e+15 true resid norm >> 2.281947914341e+14 ||r(i)||/||b|| 2.193807128394e-02 >> 16 KSP preconditioned resid norm 1.029063568485e+15 true resid norm >> 1.605944355705e+14 ||r(i)||/||b|| 1.543914369478e-02 >> 17 KSP preconditioned resid norm 8.755986525312e+14 true resid norm >> 1.474488326268e+14 ||r(i)||/||b|| 1.417535860732e-02 >> 18 KSP preconditioned resid norm 7.769999155370e+14 true resid norm >> 1.263847845681e+14 ||r(i)||/||b|| 1.215031419270e-02 >> 19 KSP preconditioned resid norm 6.633003954457e+14 true resid norm >> 1.037940616116e+14 ||r(i)||/||b|| 9.978499106727e-03 >> 20 KSP preconditioned resid norm 5.634039280371e+14 true resid norm >> 1.015458370156e+14 ||r(i)||/||b|| 9.762360468599e-03 >> 21 KSP preconditioned resid norm 4.739327677772e+14 true resid norm >> 7.392362794765e+13 ||r(i)||/||b|| 7.106831007366e-03 >> 22 KSP preconditioned resid norm 3.847430187788e+14 true resid norm >> 7.626771651358e+13 ||r(i)||/||b|| 7.332185765606e-03 >> 23 KSP preconditioned resid norm 3.166693300565e+14 true resid norm >> 4.910897850051e+13 ||r(i)||/||b|| 4.721213241789e-03 >> 24 KSP preconditioned resid norm 2.603961739838e+14 true resid norm >> 5.903512955155e+13 ||r(i)||/||b|| 5.675488350193e-03 >> 25 KSP preconditioned resid norm 2.300385519514e+14 true resid norm >> 3.524302891352e+13 ||r(i)||/||b|| 3.388175846206e-03 >> 26 KSP preconditioned resid norm 1.995854205790e+14 true resid norm >> 5.125398452570e+13 ||r(i)||/||b|| 4.927428707047e-03 >> 27 KSP preconditioned resid norm 1.777736693158e+14 true resid norm >> 2.713391594494e+13 ||r(i)||/||b|| 2.608586192838e-03 >> 28 KSP preconditioned resid norm 1.569155636768e+14 true resid norm >> 4.045925214031e+13 ||r(i)||/||b|| 3.889650381462e-03 >> 29 KSP preconditioned resid norm 1.392286125087e+14 true resid norm >> 2.127117420025e+13 ||r(i)||/||b|| 2.044956999086e-03 >> 30 KSP preconditioned resid norm 1.268134787155e+14 true resid norm >> 2.836210701297e+13 ||r(i)||/||b|| 2.726661382160e-03 >> 31 KSP preconditioned resid norm 1.148332814766e+14 true resid norm >> 1.759485016641e+13 ||r(i)||/||b|| 1.691524485529e-03 >> 32 KSP preconditioned resid norm 1.072826684457e+14 true resid norm >> 1.915166944365e+13 ||r(i)||/||b|| 1.841193161425e-03 >> 33 KSP preconditioned resid norm 9.907876839152e+13 true resid norm >> 1.514435361562e+13 ||r(i)||/||b|| 1.455939932199e-03 >> 34 KSP preconditioned resid norm 9.246074413874e+13 true resid norm >> 1.530899883812e+13 ||r(i)||/||b|| 1.471768508325e-03 >> 35 KSP preconditioned resid norm 8.716833484735e+13 true resid norm >> 1.331822154534e+13 ||r(i)||/||b|| 1.280380204127e-03 >> 36 KSP preconditioned resid norm 8.124134503515e+13 true resid norm >> 1.344545385639e+13 ||r(i)||/||b|| 1.292611997376e-03 >> 37 KSP preconditioned resid norm 7.675303961825e+13 true resid norm >> 1.172301521125e+13 ||r(i)||/||b|| 1.127021093474e-03 >> 38 KSP preconditioned resid norm 7.135475879451e+13 true resid norm >> 1.180708573448e+13 ||r(i)||/||b|| 1.135103421383e-03 >> 39 KSP preconditioned resid norm 6.615399842578e+13 true resid norm >> 1.009923425569e+13 ||r(i)||/||b|| 9.709148908363e-04 >> 40 KSP preconditioned resid norm 6.137620595303e+13 true resid norm >> 9.912104999293e+12 ||r(i)||/||b|| 9.529247564410e-04 >> 41 KSP preconditioned resid norm 5.598390411354e+13 true resid norm >> 8.555770285650e+12 ||r(i)||/||b|| 8.225301604653e-04 >> 0 KSP preconditioned resid norm 2.200654561303e+16 true resid norm >> 1.374535903551e+16 ||r(i)||/||b|| 1.042244800141e-01 >> 1 KSP preconditioned resid norm 1.033788570768e+16 true resid norm >> 1.766933748516e+15 ||r(i)||/||b|| 1.339781308605e-02 >> 2 KSP preconditioned resid norm 8.199712159807e+15 true resid norm >> 1.328504928257e+15 ||r(i)||/||b|| 1.007341714291e-02 >> 3 KSP preconditioned resid norm 7.724963557325e+15 true resid norm >> 1.290091137776e+15 ||r(i)||/||b|| 9.782143751798e-03 >> 4 KSP preconditioned resid norm 7.785648142256e+15 true resid norm >> 1.238722001104e+15 ||r(i)||/||b|| 9.392636170038e-03 >> 5 KSP preconditioned resid norm 8.249045346662e+15 true resid norm >> 1.288778250144e+15 ||r(i)||/||b|| 9.772188753144e-03 >> 6 KSP preconditioned resid norm 9.469275305646e+15 true resid norm >> 1.408770770519e+15 ||r(i)||/||b|| 1.068203461525e-02 >> 7 KSP preconditioned resid norm 1.105611185924e+16 true resid norm >> 1.994387714295e+15 ||r(i)||/||b|| 1.512248766524e-02 >> 8 KSP preconditioned resid norm 1.130059557946e+16 true resid norm >> 1.768360656538e+15 ||r(i)||/||b|| 1.340863264676e-02 >> 9 KSP preconditioned resid norm 1.052660615325e+16 true resid norm >> 1.953565171874e+15 ||r(i)||/||b|| 1.481294986084e-02 >> 10 KSP preconditioned resid norm 8.559164814525e+15 true resid norm >> 1.322544665666e+15 ||r(i)||/||b|| 1.002822332384e-02 >> 11 KSP preconditioned resid norm 4.990037328467e+15 true resid norm >> 7.740616289621e+14 ||r(i)||/||b|| 5.869338921523e-03 >> 12 KSP preconditioned resid norm 1.334591152980e+17 true resid norm >> 2.007497042766e+16 ||r(i)||/||b|| 1.522188942984e-01 >> 13 KSP preconditioned resid norm 7.977709053830e+14 true resid norm >> 2.666709181112e+14 ||r(i)||/||b|| 2.022037962283e-03 >> 0 KSP preconditioned resid norm 1.928493598618e+16 true resid norm >> 1.203377031534e+16 ||r(i)||/||b|| 4.946657154689e-01 >> 1 KSP preconditioned resid norm 9.231841001478e+15 true resid norm >> 1.582820058439e+15 ||r(i)||/||b|| 6.506413170177e-02 >> 2 KSP preconditioned resid norm 7.541228065487e+15 true resid norm >> 1.216411600127e+15 ||r(i)||/||b|| 5.000237653817e-02 >> 3 KSP preconditioned resid norm 7.339567418801e+15 true resid norm >> 1.219667619599e+15 ||r(i)||/||b|| 5.013621997708e-02 >> 4 KSP preconditioned resid norm 7.654746286546e+15 true resid norm >> 1.212045913869e+15 ||r(i)||/||b|| 4.982291862433e-02 >> 5 KSP preconditioned resid norm 8.414088698130e+15 true resid norm >> 1.311254037213e+15 ||r(i)||/||b|| 5.390101352132e-02 >> 6 KSP preconditioned resid norm 1.008553581054e+16 true resid norm >> 1.495637339777e+15 ||r(i)||/||b|| 6.148035863870e-02 >> 7 KSP preconditioned resid norm 1.235521461592e+16 true resid norm >> 2.202862093934e+15 ||r(i)||/||b|| 9.055186572627e-02 >> 8 KSP preconditioned resid norm 1.311621757723e+16 true resid norm >> 2.061404522413e+15 ||r(i)||/||b|| 8.473704551686e-02 >> 9 KSP preconditioned resid norm 1.289521641859e+16 true resid norm >> 2.354903652303e+15 ||r(i)||/||b|| 9.680175618294e-02 >> 10 KSP preconditioned resid norm 1.151646040420e+16 true resid norm >> 1.821568826477e+15 ||r(i)||/||b|| 7.487824872947e-02 >> 11 KSP preconditioned resid norm 8.378553284195e+15 true resid norm >> 1.434374156491e+15 ||r(i)||/||b|| 5.896204595714e-02 >> 12 KSP preconditioned resid norm 3.545538676243e+15 true resid norm >> 5.411687842502e+14 ||r(i)||/||b|| 2.224553376337e-02 >> 13 KSP preconditioned resid norm 1.159722122499e+15 true resid norm >> 2.107714204908e+14 ||r(i)||/||b|| 8.664067269472e-03 >> 14 KSP preconditioned resid norm 9.539010029354e+14 true resid norm >> 1.788314612223e+14 ||r(i)||/||b|| 7.351128565343e-03 >> 15 KSP preconditioned resid norm 8.844744179759e+14 true resid norm >> 1.382762666459e+14 ||r(i)||/||b|| 5.684048023218e-03 >> 16 KSP preconditioned resid norm 7.998051624629e+14 true resid norm >> 2.151800146054e+14 ||r(i)||/||b|| 8.845288973459e-03 >> 17 KSP preconditioned resid norm 6.751752084065e+14 true resid norm >> 1.049375541763e+14 ||r(i)||/||b|| 4.313611524562e-03 >> 18 KSP preconditioned resid norm 6.016391198259e+14 true resid norm >> 1.405509222696e+14 ||r(i)||/||b|| 5.777551066910e-03 >> 19 KSP preconditioned resid norm 5.356608668579e+14 true resid norm >> 8.319469793388e+13 ||r(i)||/||b|| 3.419839642795e-03 >> 20 KSP preconditioned resid norm 4.666955314009e+14 true resid norm >> 1.159814975417e+14 ||r(i)||/||b|| 4.767588956680e-03 >> 21 KSP preconditioned resid norm 4.123539296814e+14 true resid norm >> 6.367161120725e+13 ||r(i)||/||b|| 2.617314631039e-03 >> 22 KSP preconditioned resid norm 3.572486233332e+14 true resid norm >> 8.643344455842e+13 ||r(i)||/||b|| 3.552973056037e-03 >> 23 KSP preconditioned resid norm 3.110551508962e+14 true resid norm >> 4.773889823616e+13 ||r(i)||/||b|| 1.962377179627e-03 >> 24 KSP preconditioned resid norm 2.787205893436e+14 true resid norm >> 4.531540777818e+13 ||r(i)||/||b|| 1.862756062561e-03 >> 25 KSP preconditioned resid norm 2.574052602586e+14 true resid norm >> 4.020202768069e+13 ||r(i)||/||b|| 1.652563100745e-03 >> 26 KSP preconditioned resid norm 2.417321052344e+14 true resid norm >> 3.935243057179e+13 ||r(i)||/||b|| 1.617639169947e-03 >> 27 KSP preconditioned resid norm 2.257728086711e+14 true resid norm >> 3.470313603079e+13 ||r(i)||/||b|| 1.426523123165e-03 >> 28 KSP preconditioned resid norm 2.082687407590e+14 true resid norm >> 3.533906053355e+13 ||r(i)||/||b|| 1.452663729217e-03 >> 29 KSP preconditioned resid norm 1.877437540892e+14 true resid norm >> 2.879571658728e+13 ||r(i)||/||b|| 1.183690013589e-03 >> 30 KSP preconditioned resid norm 1.698389677822e+14 true resid norm >> 2.905672512335e+13 ||r(i)||/||b|| 1.194419150913e-03 >> 31 KSP preconditioned resid norm 1.483411237433e+14 true resid norm >> 2.274318996948e+13 ||r(i)||/||b|| 9.348920615478e-04 >> 32 KSP preconditioned resid norm 1.319650244589e+14 true resid norm >> 2.174098043497e+13 ||r(i)||/||b|| 8.936947739606e-04 >> 0 KSP preconditioned resid norm 6.130010037542e+16 true resid norm >> 4.019535297204e+16 ||r(i)||/||b|| 5.748505993040e-02 >> 1 KSP preconditioned resid norm 2.383231272294e+16 true resid norm >> 4.333028202793e+15 ||r(i)||/||b|| 6.196845344062e-03 >> 2 KSP preconditioned resid norm 1.380314308424e+16 true resid norm >> 2.902938638811e+15 ||r(i)||/||b|| 4.151614285921e-03 >> 3 KSP preconditioned resid norm 9.355422663861e+15 true resid norm >> 1.914374541722e+15 ||r(i)||/||b|| 2.737827313936e-03 >> 4 KSP preconditioned resid norm 6.905389449720e+15 true resid norm >> 1.160495113499e+15 ||r(i)||/||b|| 1.659672728707e-03 >> 5 KSP preconditioned resid norm 5.388850196885e+15 true resid norm >> 9.343171628690e+14 ||r(i)||/||b|| 1.336206156440e-03 >> 6 KSP preconditioned resid norm 4.456941607393e+15 true resid norm >> 7.443205302890e+14 ||r(i)||/||b|| 1.064484004428e-03 >> 7 KSP preconditioned resid norm 3.604897945956e+15 true resid norm >> 7.177354673261e+14 ||r(i)||/||b|| 1.026463591006e-03 >> 8 KSP preconditioned resid norm 2.616552593649e+15 true resid norm >> 4.566784798390e+14 ||r(i)||/||b|| 6.531150454316e-04 >> 9 KSP preconditioned resid norm 1.730753896393e+15 true resid norm >> 3.416577992882e+14 ||r(i)||/||b|| 4.886191466320e-04 >> 10 KSP preconditioned resid norm 9.605913238033e+14 true resid norm >> 1.715828091641e+14 ||r(i)||/||b|| 2.453877709368e-04 >> 11 KSP preconditioned resid norm 3.885469473149e+14 true resid norm >> 7.526615944264e+13 ||r(i)||/||b|| 1.076412910045e-04 >> 12 KSP preconditioned resid norm 1.085402135701e+14 true resid norm >> 1.877490961949e+13 ||r(i)||/||b|| 2.685078559740e-05 >> 13 KSP preconditioned resid norm 5.521535097548e+13 true resid norm >> 1.482831670997e+13 ||r(i)||/||b|| 2.120659757192e-05 >> 14 KSP preconditioned resid norm 4.327778991064e+13 true resid norm >> 6.949075957635e+12 ||r(i)||/||b|| 9.938164945670e-06 >> 0 KSP preconditioned resid norm 2.469229041488e+16 true resid norm >> 1.581397714283e+16 ||r(i)||/||b|| 1.421322439001e-01 >> 1 KSP preconditioned resid norm 1.094866313372e+16 true resid norm >> 1.827538507852e+15 ||r(i)||/||b|| 1.642547896642e-02 >> 2 KSP preconditioned resid norm 7.902647446780e+15 true resid norm >> 1.331972422469e+15 ||r(i)||/||b|| 1.197144952903e-02 >> 3 KSP preconditioned resid norm 6.757114900348e+15 true resid norm >> 1.177404752019e+15 ||r(i)||/||b|| 1.058223227919e-02 >> 4 KSP preconditioned resid norm 6.236820681429e+15 true resid norm >> 9.858934558838e+14 ||r(i)||/||b|| 8.860974558495e-03 >> 5 KSP preconditioned resid norm 6.056208231119e+15 true resid norm >> 9.498735329089e+14 ||r(i)||/||b|| 8.537236106662e-03 >> 6 KSP preconditioned resid norm 6.281150442631e+15 true resid norm >> 9.439017368810e+14 ||r(i)||/||b|| 8.483563032401e-03 >> 7 KSP preconditioned resid norm 6.511637444968e+15 true resid norm >> 1.175605283055e+15 ||r(i)||/||b|| 1.056605908257e-02 >> 8 KSP preconditioned resid norm 5.897282453520e+15 true resid norm >> 9.358326231502e+14 ||r(i)||/||b|| 8.411039768298e-03 >> 9 KSP preconditioned resid norm 4.858643646570e+15 true resid norm >> 9.074077133036e+14 ||r(i)||/||b|| 8.155563477757e-03 >> 10 KSP preconditioned resid norm 3.431227043473e+15 true resid norm >> 5.523077341885e+14 ||r(i)||/||b|| 4.964009804403e-03 >> 11 KSP preconditioned resid norm 1.790602152143e+15 true resid norm >> 3.233768236148e+14 ||r(i)||/||b|| 2.906433539808e-03 >> 12 KSP preconditioned resid norm 6.061523931075e+14 true resid norm >> 9.361733206500e+13 ||r(i)||/||b|| 8.414101875932e-04 >> 13 KSP preconditioned resid norm 3.669173259935e+14 true resid norm >> 7.433445474494e+13 ||r(i)||/||b|| 6.681002986515e-04 >> 14 KSP preconditioned resid norm 1.034316883992e+15 true resid norm >> 1.749783469552e+14 ||r(i)||/||b|| 1.572663528097e-03 >> 15 KSP preconditioned resid norm 1.340579303848e+14 true resid norm >> 2.477734260421e+13 ||r(i)||/||b|| 2.226928286548e-04 >> 16 KSP preconditioned resid norm 1.124182191319e+14 true resid norm >> 1.917839477947e+13 ||r(i)||/||b|| 1.723708248589e-04 >> 17 KSP preconditioned resid norm 9.394060484065e+13 true resid norm >> 1.682558945581e+13 ||r(i)||/||b|| 1.512243733944e-04 >> 18 KSP preconditioned resid norm 9.034672558986e+13 true resid norm >> 1.892870450718e+13 ||r(i)||/||b|| 1.701266684168e-04 >> 19 KSP preconditioned resid norm 8.058964131802e+13 true resid norm >> 1.275370577304e+13 ||r(i)||/||b|| 1.146272568369e-04 >> 20 KSP preconditioned resid norm 6.303390378547e+13 true resid norm >> 1.900069972515e+13 ||r(i)||/||b|| 1.707737442148e-04 >> 21 KSP preconditioned resid norm 4.601826552473e+13 true resid norm >> 6.960266416937e+12 ||r(i)||/||b|| 6.255720967895e-05 >> 22 KSP preconditioned resid norm 2.822788042823e+13 true resid norm >> 1.013300658091e+13 ||r(i)||/||b|| 9.107303936204e-05 >> 23 KSP preconditioned resid norm 1.858182222289e+13 true resid norm >> 2.785683196819e+12 ||r(i)||/||b|| 2.503705424529e-05 >> 24 KSP preconditioned resid norm 9.506926663598e+12 true resid norm >> 4.268961211449e+12 ||r(i)||/||b|| 3.836840224479e-05 >> 25 KSP preconditioned resid norm 5.845171932356e+12 true resid norm >> 8.923162702109e+11 ||r(i)||/||b|| 8.019925197071e-06 >> 0 SNES Function norm 0.000000000000e+00 >> 4 TS dt 6e-12 time 1.4e-11 >> 0 KSP preconditioned resid norm 2.220105437401e+19 true resid norm >> 1.204162959566e+19 ||r(i)||/||b|| 2.302124615573e+02 >> 1 KSP preconditioned resid norm 5.389751984623e+18 true resid norm >> 1.775745946542e+18 ||r(i)||/||b|| 3.394879756150e+01 >> 2 KSP preconditioned resid norm 2.218217192238e+18 true resid norm >> 5.319400119211e+17 ||r(i)||/||b|| 1.016965507636e+01 >> 3 KSP preconditioned resid norm 1.401615472906e+18 true resid norm >> 2.963370402601e+17 ||r(i)||/||b|| 5.665385980104e+00 >> 4 KSP preconditioned resid norm 1.004001983833e+18 true resid norm >> 1.656139793050e+17 ||r(i)||/||b|| 3.166216128906e+00 >> 5 KSP preconditioned resid norm 8.672200498511e+17 true resid norm >> 1.510248645324e+17 ||r(i)||/||b|| 2.887300721564e+00 >> 6 KSP preconditioned resid norm 7.388847075384e+17 true resid norm >> 1.526081348396e+17 ||r(i)||/||b|| 2.917569760470e+00 >> 7 KSP preconditioned resid norm 5.565797182786e+17 true resid norm >> 1.017271957075e+17 ||r(i)||/||b|| 1.944825486043e+00 >> 8 KSP preconditioned resid norm 3.566434137336e+17 true resid norm >> 6.306431985913e+16 ||r(i)||/||b|| 1.205666741023e+00 >> 9 KSP preconditioned resid norm 1.893287605930e+17 true resid norm >> 3.267942353405e+16 ||r(i)||/||b|| 6.247668120231e-01 >> 10 KSP preconditioned resid norm 6.895870882324e+16 true resid norm >> 1.153127225742e+16 ||r(i)||/||b|| 2.204554250883e-01 >> 11 KSP preconditioned resid norm 5.104972946485e+16 true resid norm >> 9.275497869859e+15 ||r(i)||/||b|| 1.773294203933e-01 >> 12 KSP preconditioned resid norm 4.992720890366e+16 true resid norm >> 8.399052910493e+15 ||r(i)||/||b|| 1.605735029394e-01 >> 13 KSP preconditioned resid norm 4.919979261885e+16 true resid norm >> 8.426637216788e+15 ||r(i)||/||b|| 1.611008610517e-01 >> 14 KSP preconditioned resid norm 5.688292138515e+16 true resid norm >> 9.762950550877e+15 ||r(i)||/||b|| 1.866485645090e-01 >> 15 KSP preconditioned resid norm 1.614157649132e+17 true resid norm >> 4.341833601566e+16 ||r(i)||/||b|| 8.300738642954e-01 >> 16 KSP preconditioned resid norm 4.125579354715e+16 true resid norm >> 7.099585962609e+15 ||r(i)||/||b|| 1.357302304896e-01 >> 17 KSP preconditioned resid norm 2.163051916179e+16 true resid norm >> 5.849969022355e+15 ||r(i)||/||b|| 1.118399929155e-01 >> 18 KSP preconditioned resid norm 1.415003279507e+16 true resid norm >> 2.432246275858e+15 ||r(i)||/||b|| 4.649980285729e-02 >> 19 KSP preconditioned resid norm 1.088163683743e+16 true resid norm >> 2.488576155885e+15 ||r(i)||/||b|| 4.757672024934e-02 >> 20 KSP preconditioned resid norm 9.060374162401e+15 true resid norm >> 1.588297599957e+15 ||r(i)||/||b|| 3.036515093466e-02 >> 21 KSP preconditioned resid norm 7.815461281280e+15 true resid norm >> 1.346385634634e+15 ||r(i)||/||b|| 2.574026619006e-02 >> 22 KSP preconditioned resid norm 6.943207518072e+15 true resid norm >> 1.184751633436e+15 ||r(i)||/||b|| 2.265013947659e-02 >> 23 KSP preconditioned resid norm 6.246190157097e+15 true resid norm >> 1.023840274541e+15 ||r(i)||/||b|| 1.957382827391e-02 >> 24 KSP preconditioned resid norm 5.682318425500e+15 true resid norm >> 9.496257176169e+14 ||r(i)||/||b|| 1.815499075718e-02 >> 25 KSP preconditioned resid norm 5.282641218542e+15 true resid norm >> 8.559439950771e+14 ||r(i)||/||b|| 1.636397901932e-02 >> 26 KSP preconditioned resid norm 5.115699254805e+15 true resid norm >> 8.571023693113e+14 ||r(i)||/||b|| 1.638612487439e-02 >> 27 KSP preconditioned resid norm 4.941038732000e+15 true resid norm >> 9.550184315260e+14 ||r(i)||/||b|| 1.825808892456e-02 >> 28 KSP preconditioned resid norm 4.836641976772e+15 true resid norm >> 7.564719174843e+14 ||r(i)||/||b|| 1.446226699132e-02 >> 29 KSP preconditioned resid norm 4.905885514019e+15 true resid norm >> 7.773698572205e+14 ||r(i)||/||b|| 1.486179482182e-02 >> 30 KSP preconditioned resid norm 5.169672483375e+15 true resid norm >> 8.117514654202e+14 ||r(i)||/||b|| 1.551910408326e-02 >> 31 KSP preconditioned resid norm 5.732174203621e+15 true resid norm >> 9.079138157636e+14 ||r(i)||/||b|| 1.735754058438e-02 >> 32 KSP preconditioned resid norm 8.584579912384e+15 true resid norm >> 1.381429969496e+15 ||r(i)||/||b|| 2.641024549212e-02 >> 33 KSP preconditioned resid norm 1.658456221337e+16 true resid norm >> 2.668310391151e+15 ||r(i)||/||b|| 5.101288811997e-02 >> 34 KSP preconditioned resid norm 3.797117274875e+16 true resid norm >> 6.214594485387e+15 ||r(i)||/||b|| 1.188109202908e-01 >> 35 KSP preconditioned resid norm 6.896188371971e+15 true resid norm >> 1.129650731508e+15 ||r(i)||/||b|| 2.159671774774e-02 >> 36 KSP preconditioned resid norm 3.678389972715e+15 true resid norm >> 6.014447539523e+14 ||r(i)||/||b|| 1.149845012239e-02 >> 37 KSP preconditioned resid norm 2.344506104501e+15 true resid norm >> 4.262436512363e+14 ||r(i)||/||b|| 8.148946900809e-03 >> 38 KSP preconditioned resid norm 1.979277723710e+15 true resid norm >> 3.194432632084e+14 ||r(i)||/||b|| 6.107131876701e-03 >> 39 KSP preconditioned resid norm 1.806670892810e+15 true resid norm >> 3.108273060962e+14 ||r(i)||/||b|| 5.942411588660e-03 >> 40 KSP preconditioned resid norm 1.663957842585e+15 true resid norm >> 2.662101226861e+14 ||r(i)||/||b|| 5.089418101442e-03 >> 41 KSP preconditioned resid norm 1.486329834699e+15 true resid norm >> 2.427881601889e+14 ||r(i)||/||b|| 4.641635880760e-03 >> 42 KSP preconditioned resid norm 1.346929765546e+15 true resid norm >> 2.207126725819e+14 ||r(i)||/||b|| 4.219595632660e-03 >> 43 KSP preconditioned resid norm 1.242596801597e+15 true resid norm >> 2.017748115787e+14 ||r(i)||/||b|| 3.857540682909e-03 >> 44 KSP preconditioned resid norm 1.178815061343e+15 true resid norm >> 1.881589524072e+14 ||r(i)||/||b|| 3.597232023587e-03 >> 45 KSP preconditioned resid norm 9.347996687830e+14 true resid norm >> 2.754924905110e+14 ||r(i)||/||b|| 5.266878861971e-03 >> 46 KSP preconditioned resid norm 7.515414326072e+14 true resid norm >> 1.170313953112e+14 ||r(i)||/||b|| 2.237411920043e-03 >> 47 KSP preconditioned resid norm 7.344982573734e+14 true resid norm >> 1.334969192887e+14 ||r(i)||/||b|| 2.552200610027e-03 >> 48 KSP preconditioned resid norm 8.030126805909e+14 true resid norm >> 1.311086080593e+14 ||r(i)||/||b|| 2.506540759528e-03 >> 49 KSP preconditioned resid norm 9.282223496248e+14 true resid norm >> 1.446635532628e+14 ||r(i)||/||b|| 2.765684862641e-03 >> 50 KSP preconditioned resid norm 1.074003818336e+15 true resid norm >> 3.137568207714e+14 ||r(i)||/||b|| 5.998418193015e-03 >> 51 KSP preconditioned resid norm 1.638671425800e+15 true resid norm >> 2.543484445494e+14 ||r(i)||/||b|| 4.862645960648e-03 >> 52 KSP preconditioned resid norm 2.418883444028e+15 true resid norm >> 6.581384678327e+14 ||r(i)||/||b|| 1.258232330779e-02 >> 53 KSP preconditioned resid norm 8.572181890774e+14 true resid norm >> 1.327918910006e+14 ||r(i)||/||b|| 2.538721844848e-03 >> 54 KSP preconditioned resid norm 2.577275091696e+14 true resid norm >> 8.010767963629e+13 ||r(i)||/||b|| 1.531502523990e-03 >> 0 KSP preconditioned resid norm 6.542967041071e+18 true resid norm >> 3.545980595607e+18 ||r(i)||/||b|| 4.156826068038e+02 >> 1 KSP preconditioned resid norm 1.591191551470e+18 true resid norm >> 5.223269779130e+17 ||r(i)||/||b|| 6.123052112913e+01 >> 2 KSP preconditioned resid norm 6.570663371928e+17 true resid norm >> 1.552539857829e+17 ||r(i)||/||b|| 1.819986877731e+01 >> 3 KSP preconditioned resid norm 4.165904471738e+17 true resid norm >> 8.738677344351e+16 ||r(i)||/||b|| 1.024403851227e+01 >> 4 KSP preconditioned resid norm 2.995075360323e+17 true resid norm >> 4.934782956985e+16 ||r(i)||/||b|| 5.784869342240e+00 >> 5 KSP preconditioned resid norm 2.594552595600e+17 true resid norm >> 4.469892332113e+16 ||r(i)||/||b|| 5.239894710780e+00 >> 6 KSP preconditioned resid norm 2.219662091794e+17 true resid norm >> 4.652217437246e+16 ||r(i)||/||b|| 5.453627902330e+00 >> 7 KSP preconditioned resid norm 1.674952385698e+17 true resid norm >> 2.988890119169e+16 ||r(i)||/||b|| 3.503768852333e+00 >> 8 KSP preconditioned resid norm 1.081926357303e+17 true resid norm >> 1.908191297028e+16 ||r(i)||/||b|| 2.236904323761e+00 >> 9 KSP preconditioned resid norm 5.710239913499e+16 true resid norm >> 9.635393149266e+15 ||r(i)||/||b|| 1.129522633832e+00 >> 10 KSP preconditioned resid norm 1.779371916701e+16 true resid norm >> 2.952475144259e+15 ||r(i)||/||b|| 3.461080881292e-01 >> 11 KSP preconditioned resid norm 1.125590077314e+16 true resid norm >> 2.030360591090e+15 ||r(i)||/||b|| 2.380119012218e-01 >> 12 KSP preconditioned resid norm 1.086608349874e+16 true resid norm >> 1.877856566597e+15 ||r(i)||/||b|| 2.201344005587e-01 >> 13 KSP preconditioned resid norm 1.075368648574e+16 true resid norm >> 1.822636849246e+15 ||r(i)||/||b|| 2.136611908395e-01 >> 14 KSP preconditioned resid norm 1.173878698877e+16 true resid norm >> 2.130421173676e+15 ||r(i)||/||b|| 2.497416449940e-01 >> 15 KSP preconditioned resid norm 4.944017897832e+16 true resid norm >> 1.361808358525e+16 ||r(i)||/||b|| 1.596399171332e+00 >> 16 KSP preconditioned resid norm 1.780572750257e+17 true resid norm >> 3.371295639234e+16 ||r(i)||/||b|| 3.952049149280e+00 >> 17 KSP preconditioned resid norm 3.251503916379e+16 true resid norm >> 8.719191629936e+15 ||r(i)||/||b|| 1.022119610706e+00 >> 18 KSP preconditioned resid norm 2.075735168556e+16 true resid norm >> 3.945307122742e+15 ||r(i)||/||b|| 4.624942255619e-01 >> 19 KSP preconditioned resid norm 1.483613863843e+16 true resid norm >> 3.390657011871e+15 ||r(i)||/||b|| 3.974745793076e-01 >> 20 KSP preconditioned resid norm 1.134405248775e+16 true resid norm >> 2.490065225313e+15 ||r(i)||/||b|| 2.919014292554e-01 >> 21 KSP preconditioned resid norm 8.553736210329e+15 true resid norm >> 1.657655420050e+15 ||r(i)||/||b|| 1.943210087055e-01 >> 22 KSP preconditioned resid norm 6.586515952486e+15 true resid norm >> 1.286392357530e+15 ||r(i)||/||b|| 1.507991694069e-01 >> 23 KSP preconditioned resid norm 4.780127454522e+15 true resid norm >> 9.064248964802e+14 ||r(i)||/||b|| 1.062569446397e-01 >> 24 KSP preconditioned resid norm 3.027986246405e+15 true resid norm >> 6.179735775190e+14 ||r(i)||/||b|| 7.244282948339e-02 >> 25 KSP preconditioned resid norm 1.571830581724e+15 true resid norm >> 2.758055375192e+14 ||r(i)||/||b|| 3.233169548331e-02 >> 26 KSP preconditioned resid norm 1.182949949046e+15 true resid norm >> 2.283703518838e+14 ||r(i)||/||b|| 2.677103854019e-02 >> 27 KSP preconditioned resid norm 1.078564628165e+15 true resid norm >> 2.041848112402e+14 ||r(i)||/||b|| 2.393585422076e-02 >> 28 KSP preconditioned resid norm 1.016575362098e+15 true resid norm >> 1.676746397025e+14 ||r(i)||/||b|| 1.965589755702e-02 >> 29 KSP preconditioned resid norm 9.501065512919e+14 true resid norm >> 1.807838083536e+14 ||r(i)||/||b|| 2.119263845308e-02 >> 30 KSP preconditioned resid norm 9.022105299639e+14 true resid norm >> 1.464542351381e+14 ||r(i)||/||b|| 1.716830552177e-02 >> 31 KSP preconditioned resid norm 8.706508374697e+14 true resid norm >> 1.587763993315e+14 ||r(i)||/||b|| 1.861278870358e-02 >> 32 KSP preconditioned resid norm 8.363545887412e+14 true resid norm >> 1.341498283380e+14 ||r(i)||/||b|| 1.572590397559e-02 >> 33 KSP preconditioned resid norm 8.049250107078e+14 true resid norm >> 1.594310708223e+14 ||r(i)||/||b|| 1.868953349802e-02 >> 34 KSP preconditioned resid norm 7.730163706333e+14 true resid norm >> 1.360746921537e+14 ||r(i)||/||b|| 1.595154886763e-02 >> 35 KSP preconditioned resid norm 7.656443951451e+14 true resid norm >> 1.202971650940e+14 ||r(i)||/||b|| 1.410200587090e-02 >> 36 KSP preconditioned resid norm 7.731881795958e+14 true resid norm >> 1.274146022068e+14 ||r(i)||/||b|| 1.493635753557e-02 >> 37 KSP preconditioned resid norm 8.066446225342e+14 true resid norm >> 1.271087970298e+14 ||r(i)||/||b|| 1.490050908979e-02 >> 38 KSP preconditioned resid norm 8.449932808607e+14 true resid norm >> 1.374808484319e+14 ||r(i)||/||b|| 1.611638753257e-02 >> 39 KSP preconditioned resid norm 8.841110986186e+14 true resid norm >> 1.396540752943e+14 ||r(i)||/||b|| 1.637114713515e-02 >> 40 KSP preconditioned resid norm 9.488359303223e+14 true resid norm >> 1.529468173159e+14 ||r(i)||/||b|| 1.792940768006e-02 >> 41 KSP preconditioned resid norm 1.207535627829e+15 true resid norm >> 1.911287075526e+14 ||r(i)||/||b|| 2.240533394033e-02 >> 42 KSP preconditioned resid norm 2.145532208476e+15 true resid norm >> 3.678432914805e+14 ||r(i)||/||b|| 4.312095178618e-02 >> 43 KSP preconditioned resid norm 1.285566323841e+16 true resid norm >> 2.041241798720e+15 ||r(i)||/||b|| 2.392874662258e-01 >> 44 KSP preconditioned resid norm 1.228721458245e+15 true resid norm >> 2.037912017178e+14 ||r(i)||/||b|| 2.388971278599e-02 >> 45 KSP preconditioned resid norm 6.120797893644e+14 true resid norm >> 9.792977962736e+13 ||r(i)||/||b|| 1.147995737192e-02 >> 46 KSP preconditioned resid norm 4.263304414021e+14 true resid norm >> 6.948352885735e+13 ||r(i)||/||b|| 8.145305262283e-03 >> 47 KSP preconditioned resid norm 2.943667166575e+14 true resid norm >> 4.740665276317e+13 ||r(i)||/||b|| 5.557312136691e-03 >> 48 KSP preconditioned resid norm 1.946102625201e+14 true resid norm >> 3.133411275125e+13 ||r(i)||/||b|| 3.673185828050e-03 >> 49 KSP preconditioned resid norm 1.369383238938e+14 true resid norm >> 2.605829767015e+13 ||r(i)||/||b|| 3.054720919178e-03 >> 50 KSP preconditioned resid norm 9.297342410318e+13 true resid norm >> 1.484605459745e+13 ||r(i)||/||b|| 1.740349815638e-03 >> 51 KSP preconditioned resid norm 6.674823642018e+13 true resid norm >> 2.111621172189e+13 ||r(i)||/||b|| 2.475377881439e-03 >> 52 KSP preconditioned resid norm 5.794642992605e+13 true resid norm >> 9.469052681739e+12 ||r(i)||/||b|| 1.110023136501e-03 >> 53 KSP preconditioned resid norm 4.582540149696e+13 true resid norm >> 7.443523694856e+12 ||r(i)||/||b|| 8.725776269382e-04 >> 0 KSP preconditioned resid norm 1.892653228182e+19 true resid norm >> 1.028330526045e+19 ||r(i)||/||b|| 7.841194129911e+01 >> 1 KSP preconditioned resid norm 4.581281818866e+18 true resid norm >> 1.519814562161e+18 ||r(i)||/||b|| 1.158884300480e+01 >> 2 KSP preconditioned resid norm 1.874494262231e+18 true resid norm >> 4.646649902787e+17 ||r(i)||/||b|| 3.543149115843e+00 >> 3 KSP preconditioned resid norm 1.173263764111e+18 true resid norm >> 2.551017670341e+17 ||r(i)||/||b|| 1.945194105918e+00 >> 4 KSP preconditioned resid norm 8.301325588811e+17 true resid norm >> 1.381381450838e+17 ||r(i)||/||b|| 1.053326712486e+00 >> 5 KSP preconditioned resid norm 7.052947600755e+17 true resid norm >> 1.287824897888e+17 ||r(i)||/||b|| 9.819882590194e-01 >> 6 KSP preconditioned resid norm 5.863848966069e+17 true resid norm >> 1.166961566059e+17 ||r(i)||/||b|| 8.898279249584e-01 >> 7 KSP preconditioned resid norm 4.330736686179e+17 true resid norm >> 8.611330077675e+16 ||r(i)||/||b|| 6.566284783504e-01 >> 8 KSP preconditioned resid norm 2.702589924096e+17 true resid norm >> 4.803726753137e+16 ||r(i)||/||b|| 3.662922870069e-01 >> 9 KSP preconditioned resid norm 1.546977321412e+17 true resid norm >> 2.878601764016e+16 ||r(i)||/||b|| 2.194982516096e-01 >> 10 KSP preconditioned resid norm 9.910027101694e+16 true resid norm >> 1.648820746608e+16 ||r(i)||/||b|| 1.257253697341e-01 >> 11 KSP preconditioned resid norm 9.241668505040e+16 true resid norm >> 1.559203946128e+16 ||r(i)||/||b|| 1.188919371745e-01 >> 12 KSP preconditioned resid norm 9.244329678182e+16 true resid norm >> 1.506886748242e+16 ||r(i)||/||b|| 1.149026623784e-01 >> 13 KSP preconditioned resid norm 7.090985777634e+16 true resid norm >> 1.150411231671e+16 ||r(i)||/||b|| 8.772080151562e-02 >> 14 KSP preconditioned resid norm 2.014608975967e+17 true resid norm >> 3.263298275594e+16 ||r(i)||/||b|| 2.488320110574e-01 >> 15 KSP preconditioned resid norm 1.667109940317e+16 true resid norm >> 4.390284971711e+15 ||r(i)||/||b|| 3.347666521311e-02 >> 16 KSP preconditioned resid norm 1.169352181037e+16 true resid norm >> 1.861078956987e+15 ||r(i)||/||b|| 1.419104171589e-02 >> 17 KSP preconditioned resid norm 1.018452985290e+16 true resid norm >> 1.996695440591e+15 ||r(i)||/||b|| 1.522514033324e-02 >> 18 KSP preconditioned resid norm 9.390559252345e+15 true resid norm >> 1.536845912378e+15 ||r(i)||/||b|| 1.171870992984e-02 >> 19 KSP preconditioned resid norm 8.796086996183e+15 true resid norm >> 1.422370571244e+15 ||r(i)||/||b|| 1.084581609835e-02 >> 20 KSP preconditioned resid norm 8.297954532675e+15 true resid norm >> 1.485654737712e+15 ||r(i)||/||b|| 1.132836856767e-02 >> 21 KSP preconditioned resid norm 7.837153561450e+15 true resid norm >> 1.247795221846e+15 ||r(i)||/||b|| 9.514649542211e-03 >> 22 KSP preconditioned resid norm 7.441474574169e+15 true resid norm >> 1.321880544976e+15 ||r(i)||/||b|| 1.007956265733e-02 >> 23 KSP preconditioned resid norm 7.117292465449e+15 true resid norm >> 1.122736430687e+15 ||r(i)||/||b|| 8.561055114844e-03 >> 24 KSP preconditioned resid norm 5.694316647476e+15 true resid norm >> 2.071287152890e+15 ||r(i)||/||b|| 1.579391475139e-02 >> 25 KSP preconditioned resid norm 4.803477075252e+15 true resid norm >> 7.688470974380e+14 ||r(i)||/||b|| 5.862589113657e-03 >> 26 KSP preconditioned resid norm 4.890644639484e+15 true resid norm >> 7.629702578172e+14 ||r(i)||/||b|| 5.817777217899e-03 >> 27 KSP preconditioned resid norm 5.168669214975e+15 true resid norm >> 8.754612707062e+14 ||r(i)||/||b|| 6.675540211016e-03 >> 28 KSP preconditioned resid norm 8.939764002038e+15 true resid norm >> 1.397979783339e+15 ||r(i)||/||b|| 1.065983221661e-02 >> 29 KSP preconditioned resid norm 8.205557149295e+15 true resid norm >> 1.863983796228e+15 ||r(i)||/||b|| 1.421319160626e-02 >> 30 KSP preconditioned resid norm 1.967760781914e+15 true resid norm >> 3.069470515034e+14 ||r(i)||/||b|| 2.340523165932e-03 >> 31 KSP preconditioned resid norm 1.158003829729e+15 true resid norm >> 2.612136672864e+14 ||r(i)||/||b|| 1.991798378735e-03 >> 32 KSP preconditioned resid norm 8.020817038350e+14 true resid norm >> 1.249397423467e+14 ||r(i)||/||b|| 9.526866600465e-04 >> 0 KSP preconditioned resid norm 6.470845423033e+18 true resid norm >> 3.511573094792e+18 ||r(i)||/||b|| 1.662215647083e+02 >> 1 KSP preconditioned resid norm 1.571609661901e+18 true resid norm >> 5.187508694693e+17 ||r(i)||/||b|| 2.455525739870e+01 >> 2 KSP preconditioned resid norm 6.468393005498e+17 true resid norm >> 1.566999779328e+17 ||r(i)||/||b|| 7.417449336415e+00 >> 3 KSP preconditioned resid norm 4.084082755984e+17 true resid norm >> 8.688467105425e+16 ||r(i)||/||b|| 4.112716888400e+00 >> 4 KSP preconditioned resid norm 2.920922708944e+17 true resid norm >> 4.832004038605e+16 ||r(i)||/||b|| 2.287246343141e+00 >> 5 KSP preconditioned resid norm 2.515891595070e+17 true resid norm >> 4.435607154638e+16 ||r(i)||/||b|| 2.099610464520e+00 >> 6 KSP preconditioned resid norm 2.133101049372e+17 true resid norm >> 4.355693207474e+16 ||r(i)||/||b|| 2.061782912647e+00 >> 7 KSP preconditioned resid norm 1.601303798951e+17 true resid norm >> 2.999351550015e+16 ||r(i)||/||b|| 1.419753752223e+00 >> 8 KSP preconditioned resid norm 1.019899000725e+17 true resid norm >> 1.811131044506e+16 ||r(i)||/||b|| 8.573053386130e-01 >> 9 KSP preconditioned resid norm 5.552370825926e+16 true resid norm >> 9.849613114257e+15 ||r(i)||/||b|| 4.662349492457e-01 >> 10 KSP preconditioned resid norm 2.614277580792e+16 true resid norm >> 4.400858558792e+15 ||r(i)||/||b|| 2.083162092759e-01 >> 11 KSP preconditioned resid norm 2.230379890809e+16 true resid norm >> 4.001941486401e+15 ||r(i)||/||b|| 1.894333273960e-01 >> 12 KSP preconditioned resid norm 2.202819377737e+16 true resid norm >> 3.668912744052e+15 ||r(i)||/||b|| 1.736692931151e-01 >> 13 KSP preconditioned resid norm 2.171381974288e+16 true resid norm >> 3.746629440349e+15 ||r(i)||/||b|| 1.773480406489e-01 >> 14 KSP preconditioned resid norm 2.756519085776e+16 true resid norm >> 4.738155898553e+15 ||r(i)||/||b|| 2.242822991374e-01 >> 15 KSP preconditioned resid norm 1.910971370209e+16 true resid norm >> 7.272906924885e+15 ||r(i)||/||b|| 3.442656428895e-01 >> 16 KSP preconditioned resid norm 8.265967788035e+15 true resid norm >> 1.450280377529e+15 ||r(i)||/||b|| 6.864953885656e-02 >> 17 KSP preconditioned resid norm 5.450920356042e+15 true resid norm >> 1.778917951827e+15 ||r(i)||/||b|| 8.420571563182e-02 >> 18 KSP preconditioned resid norm 4.184159413509e+15 true resid norm >> 7.350039137637e+14 ||r(i)||/||b|| 3.479167236863e-02 >> 19 KSP preconditioned resid norm 3.481860538622e+15 true resid norm >> 7.678205813365e+14 ||r(i)||/||b|| 3.634506103098e-02 >> 20 KSP preconditioned resid norm 3.027493421078e+15 true resid norm >> 5.662352682787e+14 ||r(i)||/||b|| 2.680294834981e-02 >> 21 KSP preconditioned resid norm 2.649808113090e+15 true resid norm >> 5.331458151920e+14 ||r(i)||/||b|| 2.523664728788e-02 >> 22 KSP preconditioned resid norm 2.408644968014e+15 true resid norm >> 3.911896234018e+14 ||r(i)||/||b|| 1.851710032633e-02 >> 23 KSP preconditioned resid norm 2.245371706567e+15 true resid norm >> 3.514281238053e+14 ||r(i)||/||b|| 1.663497556353e-02 >> 24 KSP preconditioned resid norm 2.104176299070e+15 true resid norm >> 3.427795705053e+14 ||r(i)||/||b|| 1.622559320890e-02 >> 25 KSP preconditioned resid norm 1.992290690493e+15 true resid norm >> 3.118626867030e+14 ||r(i)||/||b|| 1.476213148881e-02 >> 26 KSP preconditioned resid norm 1.936852873636e+15 true resid norm >> 3.191431362940e+14 ||r(i)||/||b|| 1.510675416649e-02 >> 27 KSP preconditioned resid norm 1.840320660654e+15 true resid norm >> 4.277077423449e+14 ||r(i)||/||b|| 2.024569850927e-02 >> 28 KSP preconditioned resid norm 1.797325493063e+15 true resid norm >> 2.805438081480e+14 ||r(i)||/||b|| 1.327964120375e-02 >> 29 KSP preconditioned resid norm 1.907733126356e+15 true resid norm >> 3.072709450643e+14 ||r(i)||/||b|| 1.454477976088e-02 >> 30 KSP preconditioned resid norm 2.241086833192e+15 true resid norm >> 3.515931723170e+14 ||r(i)||/||b|| 1.664278819369e-02 >> 31 KSP preconditioned resid norm 3.121906541214e+15 true resid norm >> 5.044161632328e+14 ||r(i)||/||b|| 2.387671896708e-02 >> 32 KSP preconditioned resid norm 2.288516501159e+16 true resid norm >> 3.696560518057e+15 ||r(i)||/||b|| 1.749780103571e-01 >> 33 KSP preconditioned resid norm 2.665344016377e+15 true resid norm >> 4.419048360378e+14 ||r(i)||/||b|| 2.091772300206e-02 >> 34 KSP preconditioned resid norm 1.193724857597e+15 true resid norm >> 1.964465911853e+14 ||r(i)||/||b|| 9.298869448811e-03 >> 35 KSP preconditioned resid norm 7.370892713374e+14 true resid norm >> 1.243994812053e+14 ||r(i)||/||b|| 5.888493805102e-03 >> 36 KSP preconditioned resid norm 5.570336976172e+14 true resid norm >> 9.038862530299e+13 ||r(i)||/||b|| 4.278577812313e-03 >> 37 KSP preconditioned resid norm 4.452876529911e+14 true resid norm >> 9.072905275923e+13 ||r(i)||/||b|| 4.294692067355e-03 >> 38 KSP preconditioned resid norm 4.015373409323e+14 true resid norm >> 6.374984875718e+13 ||r(i)||/||b|| 3.017621824831e-03 >> 39 KSP preconditioned resid norm 3.753907061691e+14 true resid norm >> 7.170117945509e+13 ||r(i)||/||b|| 3.394000898950e-03 >> 40 KSP preconditioned resid norm 3.520150126535e+14 true resid norm >> 5.546554172401e+13 ||r(i)||/||b|| 2.625481197140e-03 >> 41 KSP preconditioned resid norm 3.283953946659e+14 true resid norm >> 5.672854000420e+13 ||r(i)||/||b|| 2.685265671132e-03 >> 42 KSP preconditioned resid norm 3.086422594501e+14 true resid norm >> 5.033158713486e+13 ||r(i)||/||b|| 2.382463625836e-03 >> 43 KSP preconditioned resid norm 2.903330837015e+14 true resid norm >> 5.711996083788e+13 ||r(i)||/||b|| 2.703793715879e-03 >> 44 KSP preconditioned resid norm 2.805776139313e+14 true resid norm >> 4.359341431583e+13 ||r(i)||/||b|| 2.063509812540e-03 >> 45 KSP preconditioned resid norm 2.850165946773e+14 true resid norm >> 4.589953632859e+13 ||r(i)||/||b|| 2.172670920403e-03 >> 46 KSP preconditioned resid norm 3.036441891069e+14 true resid norm >> 4.745822530443e+13 ||r(i)||/||b|| 2.246452018920e-03 >> 47 KSP preconditioned resid norm 3.630944657131e+14 true resid norm >> 5.763700439451e+13 ||r(i)||/||b|| 2.728268157016e-03 >> 48 KSP preconditioned resid norm 1.221109219167e+15 true resid norm >> 1.980487895228e+14 ||r(i)||/||b|| 9.374710078478e-03 >> 49 KSP preconditioned resid norm 3.188684128549e+15 true resid norm >> 5.020159499531e+14 ||r(i)||/||b|| 2.376310401554e-02 >> 50 KSP preconditioned resid norm 3.895126724245e+13 true resid norm >> 6.299322456844e+12 ||r(i)||/||b|| 2.981806749036e-04 >> 0 KSP preconditioned resid norm 6.694681597053e+18 true resid norm >> 3.650754072115e+18 ||r(i)||/||b|| 5.229798617888e+00 >> 1 KSP preconditioned resid norm 1.597667334483e+18 true resid norm >> 5.366749713509e+17 ||r(i)||/||b|| 7.688006280303e-01 >> 2 KSP preconditioned resid norm 6.300446759111e+17 true resid norm >> 1.705328267334e+17 ||r(i)||/||b|| 2.442926376135e-01 >> 3 KSP preconditioned resid norm 3.642174486049e+17 true resid norm >> 9.481576844129e+16 ||r(i)||/||b|| 1.358260142845e-01 >> 4 KSP preconditioned resid norm 2.247332959635e+17 true resid norm >> 3.802800729856e+16 ||r(i)||/||b|| 5.447609345427e-02 >> 5 KSP preconditioned resid norm 1.557523701769e+17 true resid norm >> 3.829141999541e+16 ||r(i)||/||b|| 5.485343888228e-02 >> 6 KSP preconditioned resid norm 1.004692846569e+17 true resid norm >> 1.699728099873e+16 ||r(i)||/||b|| 2.434903992959e-02 >> 7 KSP preconditioned resid norm 7.103482802189e+16 true resid norm >> 1.472858884963e+16 ||r(i)||/||b|| 2.109908037838e-02 >> 8 KSP preconditioned resid norm 5.766847885215e+16 true resid norm >> 9.397954062732e+15 ||r(i)||/||b|| 1.346280965449e-02 >> 9 KSP preconditioned resid norm 4.523472016725e+16 true resid norm >> 1.246374917365e+16 ||r(i)||/||b|| 1.785463959348e-02 >> 10 KSP preconditioned resid norm 3.195342481960e+16 true resid norm >> 5.030521484214e+15 ||r(i)||/||b|| 7.206350738975e-03 >> 11 KSP preconditioned resid norm 2.437460220226e+16 true resid norm >> 4.974645944208e+15 ||r(i)||/||b|| 7.126307598263e-03 >> 12 KSP preconditioned resid norm 1.258045555577e+16 true resid norm >> 1.885910304237e+15 ||r(i)||/||b|| 2.701614764439e-03 >> 13 KSP preconditioned resid norm 2.495567167302e+15 true resid norm >> 4.460991814541e+14 ||r(i)||/||b|| 6.390484914966e-04 >> 14 KSP preconditioned resid norm 7.678096351052e+14 true resid norm >> 1.316276220022e+14 ||r(i)||/||b|| 1.885599363926e-04 >> 15 KSP preconditioned resid norm 5.950867501843e+14 true resid norm >> 1.439410416931e+14 ||r(i)||/||b|| 2.061992251557e-04 >> 16 KSP preconditioned resid norm 3.263035989760e+14 true resid norm >> 5.046543721443e+13 ||r(i)||/||b|| 7.229303003757e-05 >> 17 KSP preconditioned resid norm 1.436782488882e+14 true resid norm >> 7.588970356348e+13 ||r(i)||/||b|| 1.087139420975e-04 >> 18 KSP preconditioned resid norm 5.401533739797e+13 true resid norm >> 8.625318785849e+12 ||r(i)||/||b|| 1.235598985142e-05 >> 19 KSP preconditioned resid norm 3.528859989845e+13 true resid norm >> 1.233587622450e+13 ||r(i)||/||b|| 1.767145832202e-05 >> 0 KSP preconditioned resid norm 4.827874033841e+18 true resid norm >> 2.629049503905e+18 ||r(i)||/||b|| 2.387244450275e+01 >> 1 KSP preconditioned resid norm 1.161059892070e+18 true resid norm >> 3.885117260269e+17 ||r(i)||/||b|| 3.527786222537e+00 >> 2 KSP preconditioned resid norm 4.696931552127e+17 true resid norm >> 1.228954734891e+17 ||r(i)||/||b|| 1.115922452639e+00 >> 3 KSP preconditioned resid norm 2.870101730937e+17 true resid norm >> 6.759381587642e+16 ||r(i)||/||b|| 6.137692028400e-01 >> 4 KSP preconditioned resid norm 1.966639305793e+17 true resid norm >> 3.310590026030e+16 ||r(i)||/||b|| 3.006100742887e-01 >> 5 KSP preconditioned resid norm 1.592574497021e+17 true resid norm >> 3.239415772705e+16 ||r(i)||/||b|| 2.941472693472e-01 >> 6 KSP preconditioned resid norm 1.230114824013e+17 true resid norm >> 2.280737281616e+16 ||r(i)||/||b|| 2.070968009535e-01 >> 7 KSP preconditioned resid norm 8.598640818077e+16 true resid norm >> 1.938029194893e+16 ||r(i)||/||b|| 1.759780267776e-01 >> 8 KSP preconditioned resid norm 5.549085768793e+16 true resid norm >> 9.386094648379e+15 ||r(i)||/||b|| 8.522814928290e-02 >> 9 KSP preconditioned resid norm 4.474754629124e+16 true resid norm >> 7.579167807628e+15 ||r(i)||/||b|| 6.882078964122e-02 >> 10 KSP preconditioned resid norm 4.378389530908e+16 true resid norm >> 6.842593278076e+15 ||r(i)||/||b|| 6.213250379770e-02 >> 11 KSP preconditioned resid norm 4.079498014536e+16 true resid norm >> 7.704710526729e+15 ||r(i)||/||b|| 6.996074976369e-02 >> 12 KSP preconditioned resid norm 4.084398125729e+16 true resid norm >> 6.377957983596e+15 ||r(i)||/||b|| 5.791349602893e-02 >> 13 KSP preconditioned resid norm 7.523960017073e+16 true resid norm >> 1.132262466824e+16 ||r(i)||/||b|| 1.028123390664e-01 >> 14 KSP preconditioned resid norm 2.184546755774e+15 true resid norm >> 3.502101254985e+14 ||r(i)||/||b|| 3.179997855820e-03 >> 15 KSP preconditioned resid norm 2.222188316359e+15 true resid norm >> 3.654675514798e+14 ||r(i)||/||b|| 3.318539201068e-03 >> 16 KSP preconditioned resid norm 1.669487598659e+15 true resid norm >> 3.629534674524e+14 ||r(i)||/||b|| 3.295710672608e-03 >> 17 KSP preconditioned resid norm 1.401917540926e+15 true resid norm >> 2.161165666331e+14 ||r(i)||/||b|| 1.962393912861e-03 >> 18 KSP preconditioned resid norm 1.134393288674e+15 true resid norm >> 2.686682412462e+14 ||r(i)||/||b|| 2.439576610967e-03 >> 19 KSP preconditioned resid norm 9.058009317761e+14 true resid norm >> 1.409929037370e+14 ||r(i)||/||b|| 1.280251765797e-03 >> 20 KSP preconditioned resid norm 7.990098269893e+14 true resid norm >> 1.659151006791e+14 ||r(i)||/||b|| 1.506551712794e-03 >> 21 KSP preconditioned resid norm 7.119180202197e+14 true resid norm >> 1.092130755589e+14 ||r(i)||/||b|| 9.916827664832e-04 >> 22 KSP preconditioned resid norm 6.306959900843e+14 true resid norm >> 1.460783745609e+14 ||r(i)||/||b|| 1.326429146571e-03 >> 23 KSP preconditioned resid norm 5.637228851054e+14 true resid norm >> 8.672640572399e+13 ||r(i)||/||b|| 7.874980309399e-04 >> 24 KSP preconditioned resid norm 4.960976586203e+14 true resid norm >> 1.292271181476e+14 ||r(i)||/||b|| 1.173415411786e-03 >> 25 KSP preconditioned resid norm 4.386404759546e+14 true resid norm >> 8.037967670881e+13 ||r(i)||/||b|| 7.298681019622e-04 >> 26 KSP preconditioned resid norm 4.419824575339e+14 true resid norm >> 6.704822670651e+13 ||r(i)||/||b|| 6.088151130974e-04 >> 27 KSP preconditioned resid norm 4.437076214979e+14 true resid norm >> 1.370393448160e+14 ||r(i)||/||b|| 1.244352435720e-03 >> 28 KSP preconditioned resid norm 7.848825938920e+14 true resid norm >> 1.196984132854e+14 ||r(i)||/||b|| 1.086892325146e-03 >> 29 KSP preconditioned resid norm 1.771840029177e+15 true resid norm >> 6.554951997284e+14 ||r(i)||/||b|| 5.952064711635e-03 >> 30 KSP preconditioned resid norm 1.052090974187e+15 true resid norm >> 1.877636782214e+14 ||r(i)||/||b|| 1.704942406491e-03 >> 31 KSP preconditioned resid norm 2.459238084389e+14 true resid norm >> 3.913335964196e+13 ||r(i)||/||b|| 3.553409530216e-04 >> 32 KSP preconditioned resid norm 1.091607449298e+14 true resid norm >> 1.900052367477e+13 ||r(i)||/||b|| 1.725296333429e-04 >> 33 KSP preconditioned resid norm 6.424117612547e+13 true resid norm >> 1.001417309383e+13 ||r(i)||/||b|| 9.093126282646e-05 >> 34 KSP preconditioned resid norm 3.801393062657e+13 true resid norm >> 7.035998067262e+12 ||r(i)||/||b|| 6.388866894011e-05 >> 35 KSP preconditioned resid norm 2.524483314829e+13 true resid norm >> 4.006295820236e+12 ||r(i)||/||b|| 3.637819466241e-05 >> 36 KSP preconditioned resid norm 1.579134906712e+13 true resid norm >> 2.528645237998e+12 ||r(i)||/||b|| 2.296074798956e-05 >> 37 KSP preconditioned resid norm 1.074219589287e+13 true resid norm >> 1.786181249150e+12 ||r(i)||/||b|| 1.621898434352e-05 >> 38 KSP preconditioned resid norm 6.572405677048e+12 true resid norm >> 1.033900747383e+12 ||r(i)||/||b|| 9.388084239790e-06 >> 0 KSP preconditioned resid norm 9.565416792628e+15 true resid norm >> 5.898678788482e+15 ||r(i)||/||b|| 1.120363504643e-01 >> 1 KSP preconditioned resid norm 4.584205668003e+15 true resid norm >> 8.149165580746e+14 ||r(i)||/||b|| 1.547808931009e-02 >> 2 KSP preconditioned resid norm 3.775899089492e+15 true resid norm >> 6.134705686008e+14 ||r(i)||/||b|| 1.165193191355e-02 >> 3 KSP preconditioned resid norm 3.695846031998e+15 true resid norm >> 6.195110682482e+14 ||r(i)||/||b|| 1.176666193357e-02 >> 4 KSP preconditioned resid norm 3.863984235758e+15 true resid norm >> 6.333337603322e+14 ||r(i)||/||b|| 1.202920275504e-02 >> 5 KSP preconditioned resid norm 4.289553507880e+15 true resid norm >> 6.781260017663e+14 ||r(i)||/||b|| 1.287996263523e-02 >> 6 KSP preconditioned resid norm 5.214022047398e+15 true resid norm >> 7.772905386519e+14 ||r(i)||/||b|| 1.476344081849e-02 >> 7 KSP preconditioned resid norm 6.520856882448e+15 true resid norm >> 1.156631144536e+15 ||r(i)||/||b|| 2.196843342618e-02 >> 8 KSP preconditioned resid norm 7.245755589579e+15 true resid norm >> 1.125518734778e+15 ||r(i)||/||b|| 2.137750095326e-02 >> 9 KSP preconditioned resid norm 7.402293382659e+15 true resid norm >> 1.342363548018e+15 ||r(i)||/||b|| 2.549613537355e-02 >> 10 KSP preconditioned resid norm 6.814780966449e+15 true resid norm >> 1.040341685629e+15 ||r(i)||/||b|| 1.975969363197e-02 >> 11 KSP preconditioned resid norm 4.531806636058e+15 true resid norm >> 6.813426758243e+14 ||r(i)||/||b|| 1.294105842210e-02 >> 12 KSP preconditioned resid norm 4.019547715129e+15 true resid norm >> 1.067806433320e+15 ||r(i)||/||b|| 2.028134436224e-02 >> 13 KSP preconditioned resid norm 1.578855518519e+15 true resid norm >> 2.462140978868e+14 ||r(i)||/||b|| 4.676458907031e-03 >> 14 KSP preconditioned resid norm 1.216192317102e+15 true resid norm >> 3.547355821423e+14 ||r(i)||/||b|| 6.737657944804e-03 >> 15 KSP preconditioned resid norm 1.068138625918e+15 true resid norm >> 1.711364338225e+14 ||r(i)||/||b|| 3.250473905172e-03 >> 16 KSP preconditioned resid norm 1.009888628733e+15 true resid norm >> 1.725105447584e+14 ||r(i)||/||b|| 3.276573033454e-03 >> 17 KSP preconditioned resid norm 9.012147556868e+14 true resid norm >> 1.461953215911e+14 ||r(i)||/||b|| 2.776755757239e-03 >> 18 KSP preconditioned resid norm 8.184497465452e+14 true resid norm >> 1.394097869305e+14 ||r(i)||/||b|| 2.647874940607e-03 >> 19 KSP preconditioned resid norm 6.946418330368e+14 true resid norm >> 1.094539907094e+14 ||r(i)||/||b|| 2.078910566681e-03 >> 20 KSP preconditioned resid norm 5.822189800664e+14 true resid norm >> 1.086460879389e+14 ||r(i)||/||b|| 2.063565693502e-03 >> 21 KSP preconditioned resid norm 4.765744627590e+14 true resid norm >> 7.425312666441e+13 ||r(i)||/||b|| 1.410324179423e-03 >> 22 KSP preconditioned resid norm 3.785099184439e+14 true resid norm >> 8.229511757789e+13 ||r(i)||/||b|| 1.563069454208e-03 >> 23 KSP preconditioned resid norm 3.069654165781e+14 true resid norm >> 4.744586000483e+13 ||r(i)||/||b|| 9.011612922478e-04 >> 0 KSP preconditioned resid norm 9.591954950201e+15 true resid norm >> 5.933293318674e+15 ||r(i)||/||b|| 5.704140552403e-01 >> 1 KSP preconditioned resid norm 4.627616930759e+15 true resid norm >> 8.120897483902e+14 ||r(i)||/||b|| 7.807256134471e-02 >> 2 KSP preconditioned resid norm 3.843607704549e+15 true resid norm >> 6.216441009903e+14 ||r(i)||/||b|| 5.976352651335e-02 >> 3 KSP preconditioned resid norm 3.802257051246e+15 true resid norm >> 6.332758127660e+14 ||r(i)||/||b|| 6.088177425993e-02 >> 4 KSP preconditioned resid norm 4.025094330467e+15 true resid norm >> 6.488587775849e+14 ||r(i)||/||b|| 6.237988697367e-02 >> 5 KSP preconditioned resid norm 4.518903996358e+15 true resid norm >> 7.100173385586e+14 ||r(i)||/||b|| 6.825953945399e-02 >> 6 KSP preconditioned resid norm 5.566041493723e+15 true resid norm >> 8.272295550982e+14 ||r(i)||/||b|| 7.952806979104e-02 >> 7 KSP preconditioned resid norm 7.050235529471e+15 true resid norm >> 1.259531120704e+15 ||r(i)||/||b|| 1.210886122891e-01 >> 8 KSP preconditioned resid norm 7.824426472314e+15 true resid norm >> 1.230246429464e+15 ||r(i)||/||b|| 1.182732450740e-01 >> 9 KSP preconditioned resid norm 8.110342869221e+15 true resid norm >> 1.508507345577e+15 ||r(i)||/||b|| 1.450246509206e-01 >> 10 KSP preconditioned resid norm 7.848071091100e+15 true resid norm >> 1.244716713436e+15 ||r(i)||/||b|| 1.196643870448e-01 >> 11 KSP preconditioned resid norm 6.689698623315e+15 true resid norm >> 1.152050117302e+15 ||r(i)||/||b|| 1.107556198481e-01 >> 12 KSP preconditioned resid norm 3.836259156428e+15 true resid norm >> 5.895445738700e+14 ||r(i)||/||b|| 5.667754703239e-02 >> 13 KSP preconditioned resid norm 1.583619332521e+15 true resid norm >> 3.090404276347e+14 ||r(i)||/||b|| 2.971048186772e-02 >> 14 KSP preconditioned resid norm 1.282447478166e+15 true resid norm >> 2.666239295029e+14 ||r(i)||/||b|| 2.563265098882e-02 >> 15 KSP preconditioned resid norm 1.151484666969e+15 true resid norm >> 2.280596910606e+14 ||r(i)||/||b|| 2.192516806902e-02 >> 16 KSP preconditioned resid norm 1.027737067851e+15 true resid norm >> 1.605668418840e+14 ||r(i)||/||b|| 1.543655074795e-02 >> 17 KSP preconditioned resid norm 8.747586168174e+14 true resid norm >> 1.470663481317e+14 ||r(i)||/||b|| 1.413864232250e-02 >> 18 KSP preconditioned resid norm 7.765674155007e+14 true resid norm >> 1.265290838281e+14 ||r(i)||/||b|| 1.216423391459e-02 >> 19 KSP preconditioned resid norm 6.633620818064e+14 true resid norm >> 1.039325479930e+14 ||r(i)||/||b|| 9.991851571798e-03 >> 20 KSP preconditioned resid norm 5.640296981466e+14 true resid norm >> 1.018948133357e+14 ||r(i)||/||b|| 9.795948145668e-03 >> 21 KSP preconditioned resid norm 4.751732697839e+14 true resid norm >> 7.431313741469e+13 ||r(i)||/||b|| 7.144305159654e-03 >> 22 KSP preconditioned resid norm 3.865078030089e+14 true resid norm >> 7.686807394274e+13 ||r(i)||/||b|| 7.389931260973e-03 >> 23 KSP preconditioned resid norm 3.183297004119e+14 true resid norm >> 4.953673129366e+13 ||r(i)||/||b|| 4.762354777175e-03 >> 24 KSP preconditioned resid norm 2.612111917568e+14 true resid norm >> 5.971714747402e+13 ||r(i)||/||b|| 5.741078087414e-03 >> 25 KSP preconditioned resid norm 2.296313138344e+14 true resid norm >> 3.527848689641e+13 ||r(i)||/||b|| 3.391597834880e-03 >> 26 KSP preconditioned resid norm 1.981337219705e+14 true resid norm >> 5.164126794406e+13 ||r(i)||/||b|| 4.964680403211e-03 >> 27 KSP preconditioned resid norm 1.751049134665e+14 true resid norm >> 2.675806111216e+13 ||r(i)||/||b|| 2.572462430151e-03 >> 28 KSP preconditioned resid norm 1.533798667157e+14 true resid norm >> 3.995390488198e+13 ||r(i)||/||b|| 3.841082461690e-03 >> 29 KSP preconditioned resid norm 1.350471089065e+14 true resid norm >> 2.063214947275e+13 ||r(i)||/||b|| 1.983530463939e-03 >> 30 KSP preconditioned resid norm 1.221880384902e+14 true resid norm >> 2.772903253208e+13 ||r(i)||/||b|| 2.665809533591e-03 >> 31 KSP preconditioned resid norm 1.099123503540e+14 true resid norm >> 1.682484274214e+13 ||r(i)||/||b|| 1.617504185596e-03 >> 32 KSP preconditioned resid norm 1.022845787224e+14 true resid norm >> 1.853354426028e+13 ||r(i)||/||b|| 1.781775073585e-03 >> 33 KSP preconditioned resid norm 9.416346598487e+13 true resid norm >> 1.438140346990e+13 ||r(i)||/||b|| 1.382597190584e-03 >> 34 KSP preconditioned resid norm 8.782252733286e+13 true resid norm >> 1.465353724876e+13 ||r(i)||/||b|| 1.408759546637e-03 >> 35 KSP preconditioned resid norm 8.286435025538e+13 true resid norm >> 1.264742128603e+13 ||r(i)||/||b|| 1.215895873779e-03 >> 36 KSP preconditioned resid norm 7.744963280138e+13 true resid norm >> 1.290903292754e+13 ||r(i)||/||b|| 1.241046654183e-03 >> 37 KSP preconditioned resid norm 7.340074137876e+13 true resid norm >> 1.120214736846e+13 ||r(i)||/||b|| 1.076950348592e-03 >> 38 KSP preconditioned resid norm 6.860608702874e+13 true resid norm >> 1.142914172047e+13 ||r(i)||/||b|| 1.098773097257e-03 >> 39 KSP preconditioned resid norm 6.401516623396e+13 true resid norm >> 9.766911914727e+12 ||r(i)||/||b|| 9.389698997216e-04 >> 40 KSP preconditioned resid norm 5.980495528832e+13 true resid norm >> 9.707430083091e+12 ||r(i)||/||b|| 9.332514443927e-04 >> 41 KSP preconditioned resid norm 5.502664117143e+13 true resid norm >> 8.404607061129e+12 ||r(i)||/||b|| 8.080008418514e-04 >> 0 KSP preconditioned resid norm 2.200733964807e+16 true resid norm >> 1.374508262416e+16 ||r(i)||/||b|| 1.042223674371e-01 >> 1 KSP preconditioned resid norm 1.033612993855e+16 true resid norm >> 1.767505005693e+15 ||r(i)||/||b|| 1.340214251070e-02 >> 2 KSP preconditioned resid norm 8.198008013445e+15 true resid norm >> 1.328272543525e+15 ||r(i)||/||b|| 1.007165346861e-02 >> 3 KSP preconditioned resid norm 7.723889011566e+15 true resid norm >> 1.289831076731e+15 ||r(i)||/||b|| 9.780170267923e-03 >> 4 KSP preconditioned resid norm 7.785469249486e+15 true resid norm >> 1.238697760343e+15 ||r(i)||/||b|| 9.392450860579e-03 >> 5 KSP preconditioned resid norm 8.250177532689e+15 true resid norm >> 1.288999707195e+15 ||r(i)||/||b|| 9.773866391570e-03 >> 6 KSP preconditioned resid norm 9.471871112821e+15 true resid norm >> 1.409248720218e+15 ||r(i)||/||b|| 1.068565696875e-02 >> 7 KSP preconditioned resid norm 1.106232045691e+16 true resid norm >> 1.993194297637e+15 ||r(i)||/||b|| 1.511343613874e-02 >> 8 KSP preconditioned resid norm 1.131471113452e+16 true resid norm >> 1.768387012694e+15 ||r(i)||/||b|| 1.340883034665e-02 >> 9 KSP preconditioned resid norm 1.055003108779e+16 true resid norm >> 1.935924228946e+15 ||r(i)||/||b|| 1.467918468275e-02 >> 10 KSP preconditioned resid norm 8.569038225337e+15 true resid norm >> 1.316932619063e+15 ||r(i)||/||b|| 9.985668261662e-03 >> 11 KSP preconditioned resid norm 4.806783660704e+15 true resid norm >> 7.346602562794e+14 ||r(i)||/||b|| 5.570576275535e-03 >> 12 KSP preconditioned resid norm 6.311127886872e+15 true resid norm >> 1.031465284608e+15 ||r(i)||/||b|| 7.821106415332e-03 >> 13 KSP preconditioned resid norm 9.706706695861e+14 true resid norm >> 2.189061895566e+14 ||r(i)||/||b|| 1.659860616781e-03 >> 14 KSP preconditioned resid norm 6.844878378158e+14 true resid norm >> 1.339168617782e+14 ||r(i)||/||b|| 1.015427317239e-03 >> 0 KSP preconditioned resid norm 1.928571496492e+16 true resid norm >> 1.203387628158e+16 ||r(i)||/||b|| 4.946666172277e-01 >> 1 KSP preconditioned resid norm 9.232456268915e+15 true resid norm >> 1.583223656595e+15 ||r(i)||/||b|| 6.508026775390e-02 >> 2 KSP preconditioned resid norm 7.541947576336e+15 true resid norm >> 1.216650042172e+15 ||r(i)||/||b|| 5.001182882627e-02 >> 3 KSP preconditioned resid norm 7.340367763150e+15 true resid norm >> 1.219887358936e+15 ||r(i)||/||b|| 5.014490253381e-02 >> 4 KSP preconditioned resid norm 7.655525114233e+15 true resid norm >> 1.212298323935e+15 ||r(i)||/||b|| 4.983294633744e-02 >> 5 KSP preconditioned resid norm 8.414914513850e+15 true resid norm >> 1.311466433986e+15 ||r(i)||/||b|| 5.390936796484e-02 >> 6 KSP preconditioned resid norm 1.008643582787e+16 true resid norm >> 1.495807999597e+15 ||r(i)||/||b|| 6.148694451139e-02 >> 7 KSP preconditioned resid norm 1.235608743666e+16 true resid norm >> 2.203234082057e+15 ||r(i)||/||b|| 9.056652443733e-02 >> 8 KSP preconditioned resid norm 1.311751365507e+16 true resid norm >> 2.061480991281e+15 ||r(i)||/||b|| 8.473959716509e-02 >> 9 KSP preconditioned resid norm 1.289621791383e+16 true resid norm >> 2.355792563000e+15 ||r(i)||/||b|| 9.683761996231e-02 >> 10 KSP preconditioned resid norm 1.151686987354e+16 true resid norm >> 1.821530468335e+15 ||r(i)||/||b|| 7.487614911976e-02 >> 11 KSP preconditioned resid norm 8.379707704664e+15 true resid norm >> 1.435040037498e+15 ||r(i)||/||b|| 5.898900606299e-02 >> 12 KSP preconditioned resid norm 3.557464484347e+15 true resid norm >> 5.430083136124e+14 ||r(i)||/||b|| 2.232099444401e-02 >> 13 KSP preconditioned resid norm 1.204545626126e+15 true resid norm >> 2.194281091480e+14 ||r(i)||/||b|| 9.019850124522e-03 >> 14 KSP preconditioned resid norm 1.005673666245e+15 true resid norm >> 1.883222474780e+14 ||r(i)||/||b|| 7.741207149622e-03 >> 15 KSP preconditioned resid norm 9.379382960505e+14 true resid norm >> 1.480571711312e+14 ||r(i)||/||b|| 6.086063898783e-03 >> 16 KSP preconditioned resid norm 8.825746357103e+14 true resid norm >> 2.025244331207e+14 ||r(i)||/||b|| 8.325004669617e-03 >> 17 KSP preconditioned resid norm 7.524485997458e+14 true resid norm >> 1.181787653170e+14 ||r(i)||/||b|| 4.857876938371e-03 >> 18 KSP preconditioned resid norm 6.768878630597e+14 true resid norm >> 1.383948785570e+14 ||r(i)||/||b|| 5.688884015051e-03 >> 19 KSP preconditioned resid norm 6.018014871695e+14 true resid norm >> 9.381957375927e+13 ||r(i)||/||b|| 3.856563761775e-03 >> 20 KSP preconditioned resid norm 5.246326908653e+14 true resid norm >> 1.170575496120e+14 ||r(i)||/||b|| 4.811788050053e-03 >> 21 KSP preconditioned resid norm 4.606732721936e+14 true resid norm >> 7.159556750064e+13 ||r(i)||/||b|| 2.943019884477e-03 >> 22 KSP preconditioned resid norm 3.919505243136e+14 true resid norm >> 9.177422576855e+13 ||r(i)||/||b|| 3.772487330545e-03 >> 23 KSP preconditioned resid norm 3.382362447667e+14 true resid norm >> 5.210255290014e+13 ||r(i)||/||b|| 2.141736626583e-03 >> 24 KSP preconditioned resid norm 2.828412329842e+14 true resid norm >> 7.512251290427e+13 ||r(i)||/||b|| 3.087999117364e-03 >> 25 KSP preconditioned resid norm 2.453088437412e+14 true resid norm >> 3.770320821115e+13 ||r(i)||/||b|| 1.549834652446e-03 >> 26 KSP preconditioned resid norm 2.155681340535e+14 true resid norm >> 5.419527743350e+13 ||r(i)||/||b|| 2.227760526238e-03 >> 27 KSP preconditioned resid norm 1.897133167426e+14 true resid norm >> 2.950727994670e+13 ||r(i)||/||b|| 1.212931395777e-03 >> 28 KSP preconditioned resid norm 1.747505144272e+14 true resid norm >> 2.784171913979e+13 ||r(i)||/||b|| 1.144466562762e-03 >> 29 KSP preconditioned resid norm 1.591031918648e+14 true resid norm >> 2.517709500033e+13 ||r(i)||/||b|| 1.034934058155e-03 >> 30 KSP preconditioned resid norm 1.457115463937e+14 true resid norm >> 2.260424017892e+13 ||r(i)||/||b|| 9.291738391412e-04 >> 31 KSP preconditioned resid norm 1.296458013341e+14 true resid norm >> 2.025545529149e+13 ||r(i)||/||b|| 8.326242779131e-04 >> 0 KSP preconditioned resid norm 6.130010037612e+16 true resid norm >> 4.019535297216e+16 ||r(i)||/||b|| 5.748505993058e-02 >> 1 KSP preconditioned resid norm 2.383231272358e+16 true resid norm >> 4.333028202730e+15 ||r(i)||/||b|| 6.196845343971e-03 >> 2 KSP preconditioned resid norm 1.380314308485e+16 true resid norm >> 2.902938638700e+15 ||r(i)||/||b|| 4.151614285761e-03 >> 3 KSP preconditioned resid norm 9.355422664357e+15 true resid norm >> 1.914374541766e+15 ||r(i)||/||b|| 2.737827313999e-03 >> 4 KSP preconditioned resid norm 6.905389450100e+15 true resid norm >> 1.160495113546e+15 ||r(i)||/||b|| 1.659672728773e-03 >> 5 KSP preconditioned resid norm 5.388850197186e+15 true resid norm >> 9.343171629150e+14 ||r(i)||/||b|| 1.336206156506e-03 >> 6 KSP preconditioned resid norm 4.456941607624e+15 true resid norm >> 7.443205303349e+14 ||r(i)||/||b|| 1.064484004494e-03 >> 7 KSP preconditioned resid norm 3.604897946036e+15 true resid norm >> 7.177354673946e+14 ||r(i)||/||b|| 1.026463591104e-03 >> 8 KSP preconditioned resid norm 2.616552593607e+15 true resid norm >> 4.566784798486e+14 ||r(i)||/||b|| 6.531150454453e-04 >> 9 KSP preconditioned resid norm 1.730753896325e+15 true resid norm >> 3.416577992536e+14 ||r(i)||/||b|| 4.886191465826e-04 >> 10 KSP preconditioned resid norm 9.605913237294e+14 true resid norm >> 1.715828091570e+14 ||r(i)||/||b|| 2.453877709266e-04 >> 11 KSP preconditioned resid norm 3.885469473062e+14 true resid norm >> 7.526615943221e+13 ||r(i)||/||b|| 1.076412909896e-04 >> 12 KSP preconditioned resid norm 1.085402136107e+14 true resid norm >> 1.877490962768e+13 ||r(i)||/||b|| 2.685078560912e-05 >> 13 KSP preconditioned resid norm 5.521535103061e+13 true resid norm >> 1.482831672696e+13 ||r(i)||/||b|| 2.120659759623e-05 >> 14 KSP preconditioned resid norm 4.327778995114e+13 true resid norm >> 6.949075963503e+12 ||r(i)||/||b|| 9.938164954062e-06 >> 0 KSP preconditioned resid norm 2.469229041475e+16 true resid norm >> 1.581397714280e+16 ||r(i)||/||b|| 1.421322438999e-01 >> 1 KSP preconditioned resid norm 1.094866313351e+16 true resid norm >> 1.827538507817e+15 ||r(i)||/||b|| 1.642547896610e-02 >> 2 KSP preconditioned resid norm 7.902647446500e+15 true resid norm >> 1.331972422409e+15 ||r(i)||/||b|| 1.197144952849e-02 >> 3 KSP preconditioned resid norm 6.757114899989e+15 true resid norm >> 1.177404752006e+15 ||r(i)||/||b|| 1.058223227906e-02 >> 4 KSP preconditioned resid norm 6.236820681061e+15 true resid norm >> 9.858934558297e+14 ||r(i)||/||b|| 8.860974558006e-03 >> 5 KSP preconditioned resid norm 6.056208230839e+15 true resid norm >> 9.498735328449e+14 ||r(i)||/||b|| 8.537236106085e-03 >> 6 KSP preconditioned resid norm 6.281150442401e+15 true resid norm >> 9.439017368432e+14 ||r(i)||/||b|| 8.483563032058e-03 >> 7 KSP preconditioned resid norm 6.511637444792e+15 true resid norm >> 1.175605282966e+15 ||r(i)||/||b|| 1.056605908176e-02 >> 8 KSP preconditioned resid norm 5.897282453446e+15 true resid norm >> 9.358326231383e+14 ||r(i)||/||b|| 8.411039768188e-03 >> 9 KSP preconditioned resid norm 4.858643646559e+15 true resid norm >> 9.074077132105e+14 ||r(i)||/||b|| 8.155563476918e-03 >> 10 KSP preconditioned resid norm 3.431227043388e+15 true resid norm >> 5.523077341519e+14 ||r(i)||/||b|| 4.964009804073e-03 >> 11 KSP preconditioned resid norm 1.790602151882e+15 true resid norm >> 3.233768234561e+14 ||r(i)||/||b|| 2.906433538380e-03 >> 12 KSP preconditioned resid norm 6.061523925825e+14 true resid norm >> 9.361733195934e+13 ||r(i)||/||b|| 8.414101866433e-04 >> 13 KSP preconditioned resid norm 3.669173256365e+14 true resid norm >> 7.433445448820e+13 ||r(i)||/||b|| 6.681002963438e-04 >> 14 KSP preconditioned resid norm 1.034315631221e+15 true resid norm >> 1.749781208578e+14 ||r(i)||/||b|| 1.572661495987e-03 >> 15 KSP preconditioned resid norm 1.340579302998e+14 true resid norm >> 2.477734238014e+13 ||r(i)||/||b|| 2.226928266408e-04 >> 16 KSP preconditioned resid norm 1.124182190968e+14 true resid norm >> 1.917839484668e+13 ||r(i)||/||b|| 1.723708254629e-04 >> 17 KSP preconditioned resid norm 9.394060483803e+13 true resid norm >> 1.682558933028e+13 ||r(i)||/||b|| 1.512243722661e-04 >> 18 KSP preconditioned resid norm 9.034672550576e+13 true resid norm >> 1.892870471110e+13 ||r(i)||/||b|| 1.701266702495e-04 >> 19 KSP preconditioned resid norm 8.058964108831e+13 true resid norm >> 1.275370569496e+13 ||r(i)||/||b|| 1.146272561351e-04 >> 20 KSP preconditioned resid norm 6.303390314758e+13 true resid norm >> 1.900069985546e+13 ||r(i)||/||b|| 1.707737453860e-04 >> 21 KSP preconditioned resid norm 4.601826493253e+13 true resid norm >> 6.960266318594e+12 ||r(i)||/||b|| 6.255720879505e-05 >> 22 KSP preconditioned resid norm 2.822787946131e+13 true resid norm >> 1.013300639920e+13 ||r(i)||/||b|| 9.107303772884e-05 >> 23 KSP preconditioned resid norm 1.858182164082e+13 true resid norm >> 2.785683110677e+12 ||r(i)||/||b|| 2.503705347106e-05 >> 24 KSP preconditioned resid norm 9.506925688468e+12 true resid norm >> 4.268960894748e+12 ||r(i)||/||b|| 3.836839939834e-05 >> 25 KSP preconditioned resid norm 5.845171242291e+12 true resid norm >> 8.923161711832e+11 ||r(i)||/||b|| 8.019924307031e-06 >> 0 SNES Function norm 0.000000000000e+00 >> 5 TS dt 7e-12 time 2e-11 >> TS Object: 1 MPI processes >> type: theta >> maximum steps=10 >> maximum time=2e-11 >> total number of nonlinear solver iterations=0 >> total number of nonlinear solve failures=0 >> total number of linear solver iterations=0 >> total number of rejected steps=5 >> Theta=1 >> Extrapolation=no >> SNES Object: 1 MPI processes >> type: newtonls >> maximum iterations=50, maximum function evaluations=10000 >> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >> total number of linear solver iterations=0 >> total number of function evaluations=1 >> SNESLineSearch Object: 1 MPI processes >> type: bt >> interpolation: cubic >> alpha=1.000000e-04 >> maxstep=1.000000e+08, minlambda=1.000000e-12 >> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >> lambda=1.000000e-08 >> maximum iterations=40 >> KSP Object: 1 MPI processes >> type: gmres >> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >> Orthogonalization with no iterative refinement >> GMRES: happy breakdown tolerance 1e-30 >> maximum iterations=10000, initial guess is zero >> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >> left preconditioning >> using DEFAULT norm type for convergence test >> PC Object: 1 MPI processes >> type: ilu >> PC has not been set up so information may be incomplete >> ILU: out-of-place factorization >> 0 levels of fill >> tolerance for zero pivot 2.22045e-14 >> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >> matrix ordering: natural >> linear system matrix = precond matrix: >> Mat Object: 1 MPI processes >> type: seqaij >> rows=20904, cols=20904, bs=4 >> total: nonzeros=410816, allocated nonzeros=410816 >> total number of mallocs used during MatSetValues calls =0 >> using I-node routines: found 5226 nodes, limit used is 5 >> >> On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley >> wrote: >> >>> On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone wrote: >>> >>>> Hi Jed and Matt, >>>> >>>> Thank you very much for your replies. >>>> >>>> This time I copied the whole output from my screen, from compile to >>>> execution. This time I put the TSView() routine just after the TSSolve() >>>> routine to make the tsview output. And I intentionally to do only 5 time >>>> steps to make the test. Thank you in advance! >>>> >>> >>> You have to make the command line arguments work. Quit calling TSView() >>> in the code. We need many more options >>> for debugging. First track down why you are not processing options. Do >>> the examples take options for you? >>> >>> Once that works, run with -ts_view -ts_monitor -snes_monitor >>> -ksp_monitor_true_residual >>> >>> Matt >>> >>> >>>> host206-47:streamer_Implicit_SG GPL$ make pStreamer >>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o >>>> pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>>> -Wno-unknown-pragmas -g -O0 -fPIC >>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >>>> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first >>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >>>> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >>>> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >>>> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >>>> -lpthread -lssl -lcrypto >>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >>>> -lmpichf90 -lgfortran >>>> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >>>> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >>>> -lclang_rt.osx -lmpichcxx -lc++ >>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>> -lclang_rt.osx >>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >>>> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>> -lclang_rt.osx -ldl >>>> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >>>> >>>> time=0.000000 step=0 >>>> dt = 0.001000 >>>> >>>> time=0.002000 step=1 >>>> dt = 0.002000 >>>> >>>> time=0.005000 step=2 >>>> dt = 0.003000 >>>> >>>> time=0.009000 step=3 >>>> dt = 0.004000 >>>> >>>> time=0.014000 step=4 >>>> dt = 0.005000 >>>> >>>> time=0.020000 step=5 >>>> dt = 0.006000 >>>> >>>> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >>>> TS Object: 1 MPI processes >>>> type: theta >>>> maximum steps=10 >>>> maximum time=2e-11 >>>> total number of nonlinear solver iterations=0 >>>> total number of nonlinear solve failures=0 >>>> total number of linear solver iterations=0 >>>> total number of rejected steps=5 >>>> Theta=1 >>>> Extrapolation=no >>>> SNES Object: 1 MPI processes >>>> type: newtonls >>>> maximum iterations=50, maximum function evaluations=10000 >>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>> total number of linear solver iterations=0 >>>> total number of function evaluations=1 >>>> SNESLineSearch Object: 1 MPI processes >>>> type: bt >>>> interpolation: cubic >>>> alpha=1.000000e-04 >>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>> lambda=1.000000e-08 >>>> maximum iterations=40 >>>> KSP Object: 1 MPI processes >>>> type: gmres >>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>> Orthogonalization with no iterative refinement >>>> GMRES: happy breakdown tolerance 1e-30 >>>> maximum iterations=10000, initial guess is zero >>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>> left preconditioning >>>> using DEFAULT norm type for convergence test >>>> PC Object: 1 MPI processes >>>> type: ilu >>>> PC has not been set up so information may be incomplete >>>> ILU: out-of-place factorization >>>> 0 levels of fill >>>> tolerance for zero pivot 2.22045e-14 >>>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>>> matrix ordering: natural >>>> linear system matrix = precond matrix: >>>> Mat Object: 1 MPI processes >>>> type: seqaij >>>> rows=20904, cols=20904, bs=4 >>>> total: nonzeros=410816, allocated nonzeros=410816 >>>> total number of mallocs used during MatSetValues calls =0 >>>> using I-node routines: found 5226 nodes, limit used is 5 >>>> >>>> >>>> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >>>> >>>>> Sharp Stone writes: >>>>> >>>>> > Hi Matt, >>>>> > >>>>> > Thank you very much for your reply. >>>>> > >>>>> > The ts_view output is attached below. I found my code results say >>>>> "SNES has >>>>> > not been set up", and PC "has not been set up". Does this cause the >>>>> > problem? If so I do not see example ex17 explicitly set up the snes >>>>> object? >>>>> >>>>> Those objects are set up on the first time step (inside TSSolve). >>>>> >>>>> > PS: I don't know why the -ts_view option does not work in my command >>>>> line, >>>>> >>>>> Are you sure it doesn't output after the TSSolve? The example does. >>>>> >>>>> > so I use TSView() routine just before the TSSolve() routine. >>>>> >>>>> You should call it *after* the solve so we can see what happened during >>>>> the solve. >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> Feng >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Mon Oct 27 15:09:06 2014 From: knepley at gmail.com (Matthew Knepley) Date: Mon, 27 Oct 2014 15:09:06 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: On Mon, Oct 27, 2014 at 2:58 PM, Sharp Stone wrote: > Hi Matt, > > That is why we asked this. We do not think the calculation of DAE > F-functions are wrong. But even if the F-functions are not accurate, we > expect the TS solver would still call TSSetIJacobian(), but actually it > does not call TSSetIJacobian() routine at all... > SNES is not solving anything. It gets an initial residual of 0.0 so it never iterates and never forms a Jacobian. Something is wrong with your residual calculation. Also, you seem to be solving a linear equations somewhere. Give that solve a prefix so that it does not interfere with the other output. Matt > Thank you in advance! > > > On Mon, Oct 27, 2014 at 3:39 PM, Matthew Knepley > wrote: > >> On Mon, Oct 27, 2014 at 2:31 PM, Sharp Stone wrote: >> >>> Hi Matt, >>> >>> Thank you for your reply and suggestion. I use the command line options >>> to make the output on the screen. It was very long and attached below. >>> Thanks a million in advance! >>> >> >> Notice that you never actually solve anything. There is something wrong >> with your formulation. Try something where you >> know the exact solution first. Also run with >> >> -snes_converged_reason -ksp_converged_reason >> >> Matt >> >> >>> >>> >>> 0 KSP preconditioned resid norm 5.607581107219e-11 true resid norm >>> 9.787719465322e-12 ||r(i)||/||b|| 6.796750851600e-16 >>> 0 TS dt 2e-12 time 0 >>> 0 KSP preconditioned resid norm 3.622586114352e+17 true resid norm >>> 5.225681998280e+16 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 3.704153274287e+17 true resid norm >>> 1.035782762954e+17 ||r(i)||/||b|| 1.982100639294e+00 >>> 2 KSP preconditioned resid norm 2.842418683024e+17 true resid norm >>> 7.903974218109e+16 ||r(i)||/||b|| 1.512524914587e+00 >>> 3 KSP preconditioned resid norm 2.393970839759e+17 true resid norm >>> 4.943510895366e+16 ||r(i)||/||b|| 9.460030091754e-01 >>> 4 KSP preconditioned resid norm 2.271787411557e+17 true resid norm >>> 4.669216610635e+16 ||r(i)||/||b|| 8.935133466162e-01 >>> 5 KSP preconditioned resid norm 2.360179849286e+17 true resid norm >>> 4.108243474851e+16 ||r(i)||/||b|| 7.861640789094e-01 >>> 6 KSP preconditioned resid norm 2.619090922116e+17 true resid norm >>> 4.955060344596e+16 ||r(i)||/||b|| 9.482131416009e-01 >>> 7 KSP preconditioned resid norm 2.333689240438e+17 true resid norm >>> 4.336784470641e+16 ||r(i)||/||b|| 8.298982739609e-01 >>> 8 KSP preconditioned resid norm 3.244607818183e+17 true resid norm >>> 5.407474341938e+16 ||r(i)||/||b|| 1.034788252274e+00 >>> 9 KSP preconditioned resid norm 3.024748487424e+17 true resid norm >>> 6.388129145873e+16 ||r(i)||/||b|| 1.222448887624e+00 >>> 10 KSP preconditioned resid norm 2.127860348138e+17 true resid norm >>> 3.704786520226e+16 ||r(i)||/||b|| 7.089575143389e-01 >>> 11 KSP preconditioned resid norm 4.652609350461e+16 true resid norm >>> 8.497318285406e+15 ||r(i)||/||b|| 1.626068767331e-01 >>> 12 KSP preconditioned resid norm 2.920260091494e+16 true resid norm >>> 4.651858168992e+15 ||r(i)||/||b|| 8.901915904036e-02 >>> 13 KSP preconditioned resid norm 2.527529714994e+16 true resid norm >>> 3.813962069025e+15 ||r(i)||/||b|| 7.298496292505e-02 >>> 14 KSP preconditioned resid norm 2.224122069397e+16 true resid norm >>> 4.355442499437e+15 ||r(i)||/||b|| 8.334687225268e-02 >>> 15 KSP preconditioned resid norm 2.040909225547e+16 true resid norm >>> 5.536568280983e+15 ||r(i)||/||b|| 1.059492001772e-01 >>> 16 KSP preconditioned resid norm 1.834652616545e+16 true resid norm >>> 3.131287565910e+15 ||r(i)||/||b|| 5.992112736559e-02 >>> 17 KSP preconditioned resid norm 1.760978175265e+16 true resid norm >>> 3.245011634605e+15 ||r(i)||/||b|| 6.209738050790e-02 >>> 18 KSP preconditioned resid norm 1.385338263500e+16 true resid norm >>> 3.596318213647e+15 ||r(i)||/||b|| 6.882007391247e-02 >>> 19 KSP preconditioned resid norm 1.488783377120e+16 true resid norm >>> 2.508274919240e+15 ||r(i)||/||b|| 4.799899649588e-02 >>> 20 KSP preconditioned resid norm 1.320707251811e+16 true resid norm >>> 2.042806338080e+15 ||r(i)||/||b|| 3.909166954193e-02 >>> 21 KSP preconditioned resid norm 7.616453043037e+15 true resid norm >>> 1.348500002677e+15 ||r(i)||/||b|| 2.580524423647e-02 >>> 22 KSP preconditioned resid norm 8.879333117289e+15 true resid norm >>> 1.392661876801e+15 ||r(i)||/||b|| 2.665033726238e-02 >>> 23 KSP preconditioned resid norm 8.820610261864e+15 true resid norm >>> 1.365093600480e+15 ||r(i)||/||b|| 2.612278360851e-02 >>> 24 KSP preconditioned resid norm 7.993346710634e+15 true resid norm >>> 1.790642691323e+15 ||r(i)||/||b|| 3.426620088847e-02 >>> 25 KSP preconditioned resid norm 7.525623400757e+15 true resid norm >>> 1.148549131012e+15 ||r(i)||/||b|| 2.197893272859e-02 >>> 26 KSP preconditioned resid norm 7.109494224141e+15 true resid norm >>> 1.253098862904e+15 ||r(i)||/||b|| 2.397962339302e-02 >>> 27 KSP preconditioned resid norm 9.199926655972e+15 true resid norm >>> 1.867276980934e+15 ||r(i)||/||b|| 3.573269444158e-02 >>> 28 KSP preconditioned resid norm 1.121259577312e+16 true resid norm >>> 3.202544820818e+15 ||r(i)||/||b|| 6.128472459426e-02 >>> 29 KSP preconditioned resid norm 4.340241766754e+15 true resid norm >>> 6.879443404198e+14 ||r(i)||/||b|| 1.316468052680e-02 >>> 30 KSP preconditioned resid norm 4.050053526588e+15 true resid norm >>> 6.673524276162e+14 ||r(i)||/||b|| 1.277062836652e-02 >>> 31 KSP preconditioned resid norm 3.675002682668e+15 true resid norm >>> 5.703629309888e+14 ||r(i)||/||b|| 1.091461231618e-02 >>> 32 KSP preconditioned resid norm 3.175854297440e+15 true resid norm >>> 4.918363244864e+14 ||r(i)||/||b|| 9.411906898435e-03 >>> 33 KSP preconditioned resid norm 2.534108954658e+15 true resid norm >>> 3.888660709280e+14 ||r(i)||/||b|| 7.441441539229e-03 >>> 34 KSP preconditioned resid norm 1.517593193539e+15 true resid norm >>> 2.347716358056e+14 ||r(i)||/||b|| 4.492650641255e-03 >>> 35 KSP preconditioned resid norm 1.238737221149e+15 true resid norm >>> 4.944967490035e+14 ||r(i)||/||b|| 9.462817468922e-03 >>> 36 KSP preconditioned resid norm 8.673662184926e+14 true resid norm >>> 1.322920275286e+14 ||r(i)||/||b|| 2.531574396837e-03 >>> 37 KSP preconditioned resid norm 7.206551748402e+14 true resid norm >>> 1.263301396525e+14 ||r(i)||/||b|| 2.417486171069e-03 >>> 38 KSP preconditioned resid norm 5.143409845488e+14 true resid norm >>> 7.953540049469e+13 ||r(i)||/||b|| 1.522009960056e-03 >>> 39 KSP preconditioned resid norm 4.261910201431e+14 true resid norm >>> 7.194412501613e+13 ||r(i)||/||b|| 1.376741352417e-03 >>> 40 KSP preconditioned resid norm 4.660682952219e+14 true resid norm >>> 7.611539723637e+13 ||r(i)||/||b|| 1.456563894654e-03 >>> 41 KSP preconditioned resid norm 3.209169783039e+14 true resid norm >>> 4.903825175831e+13 ||r(i)||/||b|| 9.384086474158e-04 >>> 0 KSP preconditioned resid norm 5.647841057638e+16 true resid norm >>> 8.145493103696e+15 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 5.871590984109e+16 true resid norm >>> 1.629681152051e+16 ||r(i)||/||b|| 2.000715157823e+00 >>> 2 KSP preconditioned resid norm 4.554798588549e+16 true resid norm >>> 1.256325647853e+16 ||r(i)||/||b|| 1.542356775532e+00 >>> 3 KSP preconditioned resid norm 3.879559549735e+16 true resid norm >>> 7.930640596410e+15 ||r(i)||/||b|| 9.736231429392e-01 >>> 4 KSP preconditioned resid norm 3.752982759961e+16 true resid norm >>> 7.738765648717e+15 ||r(i)||/||b|| 9.500671782787e-01 >>> 5 KSP preconditioned resid norm 3.847367502586e+16 true resid norm >>> 6.717373650268e+15 ||r(i)||/||b|| 8.246736649031e-01 >>> 6 KSP preconditioned resid norm 4.072409022879e+16 true resid norm >>> 7.643006165053e+15 ||r(i)||/||b|| 9.383110473184e-01 >>> 7 KSP preconditioned resid norm 3.699615223300e+16 true resid norm >>> 6.864744274739e+15 ||r(i)||/||b|| 8.427659550315e-01 >>> 8 KSP preconditioned resid norm 4.452919179646e+16 true resid norm >>> 7.438964756180e+15 ||r(i)||/||b|| 9.132614393603e-01 >>> 9 KSP preconditioned resid norm 4.218921641123e+16 true resid norm >>> 8.817640722426e+15 ||r(i)||/||b|| 1.082517732220e+00 >>> 10 KSP preconditioned resid norm 2.723066166891e+18 true resid norm >>> 4.777495288747e+17 ||r(i)||/||b|| 5.865200826920e+01 >>> 11 KSP preconditioned resid norm 1.543090703715e+16 true resid norm >>> 2.978277930467e+15 ||r(i)||/||b|| 3.656350686879e-01 >>> 12 KSP preconditioned resid norm 5.543831189057e+15 true resid norm >>> 9.036431670746e+14 ||r(i)||/||b|| 1.109378100958e-01 >>> 13 KSP preconditioned resid norm 4.835578439031e+15 true resid norm >>> 7.302084941558e+14 ||r(i)||/||b|| 8.964570773800e-02 >>> 14 KSP preconditioned resid norm 4.632268883924e+15 true resid norm >>> 7.764942849443e+14 ||r(i)||/||b|| 9.532808819050e-02 >>> 15 KSP preconditioned resid norm 4.279662320804e+15 true resid norm >>> 7.279053691611e+14 ||r(i)||/||b|| 8.936295935612e-02 >>> 16 KSP preconditioned resid norm 4.003909782590e+15 true resid norm >>> 6.398763789786e+14 ||r(i)||/||b|| 7.855588002257e-02 >>> 17 KSP preconditioned resid norm 3.778065018292e+15 true resid norm >>> 9.619695351776e+14 ||r(i)||/||b|| 1.180983794267e-01 >>> 18 KSP preconditioned resid norm 4.432967840881e+15 true resid norm >>> 1.765362859847e+15 ||r(i)||/||b|| 2.167287894512e-01 >>> 19 KSP preconditioned resid norm 3.349713073090e+15 true resid norm >>> 5.343310168427e+14 ||r(i)||/||b|| 6.559836341894e-02 >>> 20 KSP preconditioned resid norm 3.170973083244e+15 true resid norm >>> 4.905381423867e+14 ||r(i)||/||b|| 6.022203151385e-02 >>> 21 KSP preconditioned resid norm 7.415620721710e+15 true resid norm >>> 1.207250421775e+15 ||r(i)||/||b|| 1.482108457286e-01 >>> 22 KSP preconditioned resid norm 1.972990201442e+15 true resid norm >>> 3.077655556372e+14 ||r(i)||/||b|| 3.778353891154e-02 >>> 23 KSP preconditioned resid norm 2.209676808077e+15 true resid norm >>> 3.428403970507e+14 ||r(i)||/||b|| 4.208958164793e-02 >>> 24 KSP preconditioned resid norm 2.003292171306e+15 true resid norm >>> 4.370066270665e+14 ||r(i)||/||b|| 5.365011319796e-02 >>> 25 KSP preconditioned resid norm 1.896876965875e+15 true resid norm >>> 2.911542588080e+14 ||r(i)||/||b|| 3.574421524903e-02 >>> 26 KSP preconditioned resid norm 1.776666236555e+15 true resid norm >>> 2.752957912495e+14 ||r(i)||/||b|| 3.379731438537e-02 >>> 27 KSP preconditioned resid norm 2.106240629900e+15 true resid norm >>> 3.329282684307e+14 ||r(i)||/||b|| 4.087269661791e-02 >>> 28 KSP preconditioned resid norm 1.993718847257e+15 true resid norm >>> 3.265440954005e+14 ||r(i)||/||b|| 4.008892908550e-02 >>> 29 KSP preconditioned resid norm 1.305019272390e+15 true resid norm >>> 2.016059917794e+14 ||r(i)||/||b|| 2.475061843560e-02 >>> 30 KSP preconditioned resid norm 1.188165021558e+15 true resid norm >>> 2.349019593212e+14 ||r(i)||/||b|| 2.883827367242e-02 >>> 31 KSP preconditioned resid norm 1.072198412006e+15 true resid norm >>> 1.660526922017e+14 ||r(i)||/||b|| 2.038583669371e-02 >>> 32 KSP preconditioned resid norm 9.531727749289e+14 true resid norm >>> 1.481127635008e+14 ||r(i)||/||b|| 1.818340051551e-02 >>> 33 KSP preconditioned resid norm 8.113851256730e+14 true resid norm >>> 1.246949236714e+14 ||r(i)||/||b|| 1.530845610990e-02 >>> 34 KSP preconditioned resid norm 1.978771889204e+15 true resid norm >>> 3.123099443158e+14 ||r(i)||/||b|| 3.834144113069e-02 >>> 35 KSP preconditioned resid norm 5.701229171299e+14 true resid norm >>> 9.665965877556e+13 ||r(i)||/||b|| 1.186664300676e-02 >>> 36 KSP preconditioned resid norm 4.923209980376e+14 true resid norm >>> 8.358449899120e+13 ||r(i)||/||b|| 1.026144125679e-02 >>> 37 KSP preconditioned resid norm 4.337271696165e+14 true resid norm >>> 7.056948273999e+13 ||r(i)||/||b|| 8.663623164566e-03 >>> 38 KSP preconditioned resid norm 3.355561922811e+14 true resid norm >>> 5.206223795577e+13 ||r(i)||/||b|| 6.391539136181e-03 >>> 39 KSP preconditioned resid norm 2.948796430248e+14 true resid norm >>> 4.783720648479e+13 ||r(i)||/||b|| 5.872843531485e-03 >>> 40 KSP preconditioned resid norm 2.903844355011e+14 true resid norm >>> 4.553096332971e+13 ||r(i)||/||b|| 5.589712341546e-03 >>> 41 KSP preconditioned resid norm 2.470740134237e+14 true resid norm >>> 3.845088511805e+13 ||r(i)||/||b|| 4.720510425650e-03 >>> 42 KSP preconditioned resid norm 2.144333147632e+14 true resid norm >>> 4.199300915233e+13 ||r(i)||/||b|| 5.155367344584e-03 >>> 43 KSP preconditioned resid norm 1.666875797521e+14 true resid norm >>> 2.786670988964e+13 ||r(i)||/||b|| 3.421120064173e-03 >>> 44 KSP preconditioned resid norm 1.097596670806e+14 true resid norm >>> 1.683182082477e+13 ||r(i)||/||b|| 2.066396792741e-03 >>> 45 KSP preconditioned resid norm 1.086486049833e+14 true resid norm >>> 1.868163126673e+13 ||r(i)||/||b|| 2.293492981813e-03 >>> 46 KSP preconditioned resid norm 9.986155841385e+13 true resid norm >>> 1.544797785039e+13 ||r(i)||/||b|| 1.896506160367e-03 >>> 47 KSP preconditioned resid norm 9.193832565781e+13 true resid norm >>> 1.424417809082e+13 ||r(i)||/||b|| 1.748718943038e-03 >>> 48 KSP preconditioned resid norm 7.868838175665e+13 true resid norm >>> 1.234477959732e+13 ||r(i)||/||b|| 1.515534963957e-03 >>> 49 KSP preconditioned resid norm 6.573261298706e+13 true resid norm >>> 1.037919549254e+13 ||r(i)||/||b|| 1.274225557668e-03 >>> 50 KSP preconditioned resid norm 6.394833480259e+13 true resid norm >>> 1.032066032237e+13 ||r(i)||/||b|| 1.267039354277e-03 >>> 51 KSP preconditioned resid norm 6.650072664422e+13 true resid norm >>> 2.391163155719e+13 ||r(i)||/||b|| 2.935565870940e-03 >>> 52 KSP preconditioned resid norm 6.239223696051e+13 true resid norm >>> 9.858797480463e+12 ||r(i)||/||b|| 1.210337711291e-03 >>> 53 KSP preconditioned resid norm 6.247432830217e+13 true resid norm >>> 9.562224327829e+12 ||r(i)||/||b|| 1.173928233208e-03 >>> 54 KSP preconditioned resid norm 6.218810094746e+13 true resid norm >>> 9.709963371483e+12 ||r(i)||/||b|| 1.192065753156e-03 >>> 55 KSP preconditioned resid norm 6.285033387840e+13 true resid norm >>> 9.623710764220e+12 ||r(i)||/||b|| 1.181476755515e-03 >>> 56 KSP preconditioned resid norm 5.407058856328e+13 true resid norm >>> 9.490503427247e+12 ||r(i)||/||b|| 1.165123253611e-03 >>> 0 KSP preconditioned resid norm 9.075431914728e+17 true resid norm >>> 1.310345788054e+17 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 8.641655528143e+17 true resid norm >>> 2.471138002109e+17 ||r(i)||/||b|| 1.885867092974e+00 >>> 2 KSP preconditioned resid norm 6.384433252903e+17 true resid norm >>> 1.816534419632e+17 ||r(i)||/||b|| 1.386301567259e+00 >>> 3 KSP preconditioned resid norm 5.179227150533e+17 true resid norm >>> 1.107056703740e+17 ||r(i)||/||b|| 8.448584441089e-01 >>> 4 KSP preconditioned resid norm 4.625114083211e+17 true resid norm >>> 9.438923102711e+16 ||r(i)||/||b|| 7.203383403647e-01 >>> 5 KSP preconditioned resid norm 4.977065165555e+17 true resid norm >>> 8.614996614554e+16 ||r(i)||/||b|| 6.574597860420e-01 >>> 6 KSP preconditioned resid norm 6.630931668382e+17 true resid norm >>> 1.310543864565e+17 ||r(i)||/||b|| 1.000151163542e+00 >>> 7 KSP preconditioned resid norm 5.452278853431e+17 true resid norm >>> 1.019521297074e+17 ||r(i)||/||b|| 7.780551564087e-01 >>> 8 KSP preconditioned resid norm 1.958476842674e+18 true resid norm >>> 3.244825770314e+17 ||r(i)||/||b|| 2.476312588551e+00 >>> 9 KSP preconditioned resid norm 1.577830260132e+18 true resid norm >>> 3.469004540765e+17 ||r(i)||/||b|| 2.647396261651e+00 >>> 10 KSP preconditioned resid norm 1.081897998863e+17 true resid norm >>> 1.828781621979e+16 ||r(i)||/||b|| 1.395648109568e-01 >>> 11 KSP preconditioned resid norm 5.614465792481e+16 true resid norm >>> 9.930812946447e+15 ||r(i)||/||b|| 7.578772746079e-02 >>> 12 KSP preconditioned resid norm 4.960081715579e+16 true resid norm >>> 8.035214375965e+15 ||r(i)||/||b|| 6.132132792135e-02 >>> 13 KSP preconditioned resid norm 3.329600867897e+16 true resid norm >>> 6.602562228940e+15 ||r(i)||/||b|| 5.038793797127e-02 >>> 14 KSP preconditioned resid norm 3.130960732609e+16 true resid norm >>> 4.730473520325e+15 ||r(i)||/||b|| 3.610095566719e-02 >>> 15 KSP preconditioned resid norm 2.560518921527e+16 true resid norm >>> 3.945110867578e+15 ||r(i)||/||b|| 3.010740297365e-02 >>> 16 KSP preconditioned resid norm 1.584886017619e+16 true resid norm >>> 3.416780637867e+15 ||r(i)||/||b|| 2.607541207073e-02 >>> 17 KSP preconditioned resid norm 1.443824757417e+16 true resid norm >>> 2.286337927483e+15 ||r(i)||/||b|| 1.744835560449e-02 >>> 18 KSP preconditioned resid norm 1.039145582094e+16 true resid norm >>> 1.711526931765e+15 ||r(i)||/||b|| 1.306164332627e-02 >>> 19 KSP preconditioned resid norm 1.018242603191e+16 true resid norm >>> 1.657426513724e+15 ||r(i)||/||b|| 1.264877201754e-02 >>> 20 KSP preconditioned resid norm 8.198983478967e+15 true resid norm >>> 1.311419876212e+15 ||r(i)||/||b|| 1.000819698256e-02 >>> 21 KSP preconditioned resid norm 6.481494503642e+15 true resid norm >>> 1.039872641976e+15 ||r(i)||/||b|| 7.935864345546e-03 >>> 22 KSP preconditioned resid norm 6.325821399407e+15 true resid norm >>> 1.165377948147e+15 ||r(i)||/||b|| 8.893667295849e-03 >>> 23 KSP preconditioned resid norm 5.405294478066e+15 true resid norm >>> 1.922243319095e+15 ||r(i)||/||b|| 1.466974089297e-02 >>> 24 KSP preconditioned resid norm 4.960676496180e+15 true resid norm >>> 8.018184223519e+14 ||r(i)||/||b|| 6.119136106374e-03 >>> 25 KSP preconditioned resid norm 4.419138742847e+15 true resid norm >>> 7.148418830282e+14 ||r(i)||/||b|| 5.455368266491e-03 >>> 26 KSP preconditioned resid norm 2.795870478230e+15 true resid norm >>> 4.843283048525e+14 ||r(i)||/||b|| 3.696186985663e-03 >>> 27 KSP preconditioned resid norm 2.911201681995e+15 true resid norm >>> 4.441742607418e+14 ||r(i)||/||b|| 3.389748452592e-03 >>> 28 KSP preconditioned resid norm 2.776697567476e+15 true resid norm >>> 5.074732182061e+14 ||r(i)||/||b|| 3.872819089681e-03 >>> 29 KSP preconditioned resid norm 1.941501506934e+15 true resid norm >>> 2.950418833153e+14 ||r(i)||/||b|| 2.251633774878e-03 >>> 30 KSP preconditioned resid norm 8.413150178496e+14 true resid norm >>> 1.695476752862e+14 ||r(i)||/||b|| 1.293915520864e-03 >>> 0 KSP preconditioned resid norm 1.415973479575e+17 true resid norm >>> 2.042710742436e+16 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 1.440919303250e+17 true resid norm >>> 4.045863378014e+16 ||r(i)||/||b|| 1.980634503928e+00 >>> 2 KSP preconditioned resid norm 1.100352174605e+17 true resid norm >>> 3.072773288453e+16 ||r(i)||/||b|| 1.504262558872e+00 >>> 3 KSP preconditioned resid norm 9.218323125716e+16 true resid norm >>> 1.914173874994e+16 ||r(i)||/||b|| 9.370753456316e-01 >>> 4 KSP preconditioned resid norm 8.672799672395e+16 true resid norm >>> 1.779286318286e+16 ||r(i)||/||b|| 8.710417394506e-01 >>> 5 KSP preconditioned resid norm 9.059066321416e+16 true resid norm >>> 1.574500663517e+16 ||r(i)||/||b|| 7.707898288327e-01 >>> 6 KSP preconditioned resid norm 1.027848583976e+17 true resid norm >>> 1.954996111922e+16 ||r(i)||/||b|| 9.570596909823e-01 >>> 7 KSP preconditioned resid norm 9.069106612200e+16 true resid norm >>> 1.686428629204e+16 ||r(i)||/||b|| 8.255836688812e-01 >>> 8 KSP preconditioned resid norm 1.359707392060e+17 true resid norm >>> 2.263432282464e+16 ||r(i)||/||b|| 1.108053252691e+00 >>> 9 KSP preconditioned resid norm 1.256447896822e+17 true resid norm >>> 2.668588264948e+16 ||r(i)||/||b|| 1.306395570116e+00 >>> 10 KSP preconditioned resid norm 5.661761773340e+16 true resid norm >>> 9.807578010522e+15 ||r(i)||/||b|| 4.801256392682e-01 >>> 11 KSP preconditioned resid norm 1.597811133832e+16 true resid norm >>> 2.854210532053e+15 ||r(i)||/||b|| 1.397266129148e-01 >>> 12 KSP preconditioned resid norm 1.141170314391e+16 true resid norm >>> 1.817769027418e+15 ||r(i)||/||b|| 8.898807793264e-02 >>> 13 KSP preconditioned resid norm 9.557798166843e+15 true resid norm >>> 1.455009507586e+15 ||r(i)||/||b|| 7.122934624852e-02 >>> 14 KSP preconditioned resid norm 7.997842098400e+15 true resid norm >>> 2.021581420109e+15 ||r(i)||/||b|| 9.896562338034e-02 >>> 15 KSP preconditioned resid norm 7.518666379332e+15 true resid norm >>> 2.232979183117e+15 ||r(i)||/||b|| 1.093145072735e-01 >>> 16 KSP preconditioned resid norm 6.614897634500e+15 true resid norm >>> 1.222952133628e+15 ||r(i)||/||b|| 5.986908024822e-02 >>> 17 KSP preconditioned resid norm 6.199437182741e+15 true resid norm >>> 1.163257046730e+15 ||r(i)||/||b|| 5.694673369870e-02 >>> 18 KSP preconditioned resid norm 4.584922270319e+15 true resid norm >>> 9.962391827866e+14 ||r(i)||/||b|| 4.877044811535e-02 >>> 19 KSP preconditioned resid norm 5.012002077991e+15 true resid norm >>> 8.755757394511e+14 ||r(i)||/||b|| 4.286342266977e-02 >>> 20 KSP preconditioned resid norm 4.312819580189e+15 true resid norm >>> 6.706322150678e+14 ||r(i)||/||b|| 3.283050317090e-02 >>> 21 KSP preconditioned resid norm 2.607093537931e+15 true resid norm >>> 5.192589254045e+14 ||r(i)||/||b|| 2.542009079490e-02 >>> 22 KSP preconditioned resid norm 2.989363098445e+15 true resid norm >>> 4.636989016003e+14 ||r(i)||/||b|| 2.270017442838e-02 >>> 23 KSP preconditioned resid norm 3.016276187245e+15 true resid norm >>> 4.674022912685e+14 ||r(i)||/||b|| 2.288147222994e-02 >>> 24 KSP preconditioned resid norm 2.901859180715e+15 true resid norm >>> 4.492447164798e+14 ||r(i)||/||b|| 2.199257619530e-02 >>> 25 KSP preconditioned resid norm 2.843000955863e+15 true resid norm >>> 5.109023428494e+14 ||r(i)||/||b|| 2.501099799574e-02 >>> 26 KSP preconditioned resid norm 2.741940909323e+15 true resid norm >>> 4.263494245202e+14 ||r(i)||/||b|| 2.087174731415e-02 >>> 27 KSP preconditioned resid norm 1.296293793548e+15 true resid norm >>> 2.124064840417e+14 ||r(i)||/||b|| 1.039826538477e-02 >>> 28 KSP preconditioned resid norm 9.720512789714e+14 true resid norm >>> 2.091841490290e+14 ||r(i)||/||b|| 1.024051740089e-02 >>> 29 KSP preconditioned resid norm 1.459179957917e+15 true resid norm >>> 2.244723575034e+14 ||r(i)||/||b|| 1.098894487800e-02 >>> 30 KSP preconditioned resid norm 1.339547884067e+15 true resid norm >>> 2.746667338979e+14 ||r(i)||/||b|| 1.344618835119e-02 >>> 31 KSP preconditioned resid norm 1.186062213325e+15 true resid norm >>> 1.831856176134e+14 ||r(i)||/||b|| 8.967770806105e-03 >>> 32 KSP preconditioned resid norm 9.909356538944e+14 true resid norm >>> 1.788908155328e+14 ||r(i)||/||b|| 8.757520671747e-03 >>> 33 KSP preconditioned resid norm 7.609550150745e+14 true resid norm >>> 1.203572113192e+14 ||r(i)||/||b|| 5.892033992812e-03 >>> 34 KSP preconditioned resid norm 5.051711143811e+14 true resid norm >>> 7.920683224412e+13 ||r(i)||/||b|| 3.877535404237e-03 >>> 35 KSP preconditioned resid norm 5.443461822227e+14 true resid norm >>> 8.557198028123e+13 ||r(i)||/||b|| 4.189138408269e-03 >>> 36 KSP preconditioned resid norm 4.648220720251e+14 true resid norm >>> 7.162139608842e+13 ||r(i)||/||b|| 3.506193735634e-03 >>> 37 KSP preconditioned resid norm 3.842876907480e+14 true resid norm >>> 6.998069618798e+13 ||r(i)||/||b|| 3.425873998417e-03 >>> 38 KSP preconditioned resid norm 2.822934553902e+14 true resid norm >>> 5.594039593962e+13 ||r(i)||/||b|| 2.738537316003e-03 >>> 39 KSP preconditioned resid norm 2.288268021627e+14 true resid norm >>> 3.624972614121e+13 ||r(i)||/||b|| 1.774589293929e-03 >>> 40 KSP preconditioned resid norm 3.336880274482e+14 true resid norm >>> 5.633095049288e+13 ||r(i)||/||b|| 2.757656741242e-03 >>> 41 KSP preconditioned resid norm 1.655073694079e+14 true resid norm >>> 2.539321805860e+13 ||r(i)||/||b|| 1.243113747388e-03 >>> 42 KSP preconditioned resid norm 1.317678812236e+14 true resid norm >>> 2.429513710603e+13 ||r(i)||/||b|| 1.189357680523e-03 >>> 0 KSP preconditioned resid norm 4.735355646927e+18 true resid norm >>> 6.977368406398e+17 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 1.655056377336e+18 true resid norm >>> 5.433322725028e+17 ||r(i)||/||b|| 7.787065851426e-01 >>> 2 KSP preconditioned resid norm 8.847192945348e+17 true resid norm >>> 2.725627800798e+17 ||r(i)||/||b|| 3.906383670810e-01 >>> 3 KSP preconditioned resid norm 5.344946494044e+17 true resid norm >>> 1.588736353868e+17 ||r(i)||/||b|| 2.276985048419e-01 >>> 4 KSP preconditioned resid norm 3.198139750974e+17 true resid norm >>> 6.624530455539e+16 ||r(i)||/||b|| 9.494310848578e-02 >>> 5 KSP preconditioned resid norm 3.286368493485e+17 true resid norm >>> 6.812427209313e+16 ||r(i)||/||b|| 9.763605434775e-02 >>> 6 KSP preconditioned resid norm 4.284030910438e+17 true resid norm >>> 1.696553569581e+17 ||r(i)||/||b|| 2.431509232084e-01 >>> 7 KSP preconditioned resid norm 1.587968810131e+18 true resid norm >>> 3.732963000458e+17 ||r(i)||/||b|| 5.350101618592e-01 >>> 8 KSP preconditioned resid norm 6.621891209840e+16 true resid norm >>> 1.229792367414e+16 ||r(i)||/||b|| 1.762544695628e-02 >>> 9 KSP preconditioned resid norm 5.638286198157e+16 true resid norm >>> 2.091986582514e+16 ||r(i)||/||b|| 2.998245843800e-02 >>> 10 KSP preconditioned resid norm 1.309067125882e+16 true resid norm >>> 2.099826104432e+15 ||r(i)||/||b|| 3.009481486611e-03 >>> 11 KSP preconditioned resid norm 1.075925809079e+16 true resid norm >>> 2.578010577039e+15 ||r(i)||/||b|| 3.694817912546e-03 >>> 12 KSP preconditioned resid norm 9.319606428298e+15 true resid norm >>> 2.026298982436e+15 ||r(i)||/||b|| 2.904102040216e-03 >>> 13 KSP preconditioned resid norm 4.070270220193e+15 true resid norm >>> 2.216532493559e+15 ||r(i)||/||b|| 3.176745678968e-03 >>> 14 KSP preconditioned resid norm 3.400982828163e+15 true resid norm >>> 9.161069365806e+14 ||r(i)||/||b|| 1.312969135671e-03 >>> 15 KSP preconditioned resid norm 4.173720697476e+14 true resid norm >>> 7.019272258636e+13 ||r(i)||/||b|| 1.006005681483e-04 >>> 16 KSP preconditioned resid norm 2.871344308441e+14 true resid norm >>> 8.503915375563e+13 ||r(i)||/||b|| 1.218785490496e-04 >>> 17 KSP preconditioned resid norm 8.731930916187e+13 true resid norm >>> 2.939915802842e+13 ||r(i)||/||b|| 4.213502328681e-05 >>> 18 KSP preconditioned resid norm 7.273170647297e+13 true resid norm >>> 2.554194915512e+13 ||r(i)||/||b|| 3.660685184933e-05 >>> 19 KSP preconditioned resid norm 5.198384943683e+13 true resid norm >>> 8.213409786802e+12 ||r(i)||/||b|| 1.177150081293e-05 >>> 20 KSP preconditioned resid norm 2.826075121950e+13 true resid norm >>> 1.225537290577e+13 ||r(i)||/||b|| 1.756446297795e-05 >>> 0 KSP preconditioned resid norm 7.535755573391e+17 true resid norm >>> 1.091530949677e+17 ||r(i)||/||b|| 1.000000000000e+00 >>> 1 KSP preconditioned resid norm 5.749658535033e+17 true resid norm >>> 1.739986551510e+17 ||r(i)||/||b|| 1.594078987889e+00 >>> 2 KSP preconditioned resid norm 3.832130567981e+17 true resid norm >>> 1.138657236506e+17 ||r(i)||/||b|| 1.043174485197e+00 >>> 3 KSP preconditioned resid norm 2.820261830557e+17 true resid norm >>> 6.703201541200e+16 ||r(i)||/||b|| 6.141100756863e-01 >>> 4 KSP preconditioned resid norm 2.196333397214e+17 true resid norm >>> 4.464718002673e+16 ||r(i)||/||b|| 4.090326530819e-01 >>> 5 KSP preconditioned resid norm 2.438749906836e+17 true resid norm >>> 4.259961207836e+16 ||r(i)||/||b|| 3.902739733672e-01 >>> 6 KSP preconditioned resid norm 7.293863369143e+17 true resid norm >>> 1.861518427200e+17 ||r(i)||/||b|| 1.705419738901e+00 >>> 7 KSP preconditioned resid norm 3.735172838302e+17 true resid norm >>> 7.178953918634e+16 ||r(i)||/||b|| 6.576958647631e-01 >>> 8 KSP preconditioned resid norm 2.306462817079e+17 true resid norm >>> 3.812340554474e+16 ||r(i)||/||b|| 3.492654565226e-01 >>> 9 KSP preconditioned resid norm 2.324630150277e+17 true resid norm >>> 5.879635462988e+16 ||r(i)||/||b|| 5.386595281360e-01 >>> 10 KSP preconditioned resid norm 2.486084300490e+16 true resid norm >>> 3.943582532938e+15 ||r(i)||/||b|| 3.612891172811e-02 >>> 11 KSP preconditioned resid norm 2.044336976942e+16 true resid norm >>> 4.427406712026e+15 ||r(i)||/||b|| 4.056143999707e-02 >>> 12 KSP preconditioned resid norm 1.903964203761e+16 true resid norm >>> 3.227987898262e+15 ||r(i)||/||b|| 2.957303133931e-02 >>> 13 KSP preconditioned resid norm 6.823460749064e+15 true resid norm >>> 1.876444880951e+15 ||r(i)||/||b|| 1.719094526368e-02 >>> 14 KSP preconditioned resid norm 7.090194359812e+15 true resid norm >>> 1.450790953432e+15 ||r(i)||/||b|| 1.329134051455e-02 >>> 15 KSP preconditioned resid norm 4.884314111756e+15 true resid norm >>> 8.008003608330e+14 ||r(i)||/||b|| 7.336487903253e-03 >>> 16 KSP preconditioned resid norm 1.603299777176e+15 true resid norm >>> 2.941609026573e+14 ||r(i)||/||b|| 2.694938725689e-03 >>> 17 KSP preconditioned resid norm 1.548805654760e+15 true resid norm >>> 2.838534773138e+14 ||r(i)||/||b|| 2.600507822503e-03 >>> 18 KSP preconditioned resid norm 1.022454633110e+15 true resid norm >>> 2.215058790848e+14 ||r(i)||/||b|| 2.029313773928e-03 >>> 19 KSP preconditioned resid norm 9.587995256078e+14 true resid norm >>> 2.226912518453e+14 ||r(i)||/||b|| 2.040173500451e-03 >>> 20 KSP preconditioned resid norm 7.709554287637e+14 true resid norm >>> 1.253194818485e+14 ||r(i)||/||b|| 1.148107452983e-03 >>> 21 KSP preconditioned resid norm 3.692994290799e+14 true resid norm >>> 6.513442758709e+13 ||r(i)||/||b|| 5.967254305192e-04 >>> 22 KSP preconditioned resid norm 3.577186864519e+14 true resid norm >>> 5.884014914924e+13 ||r(i)||/||b|| 5.390607491859e-04 >>> 23 KSP preconditioned resid norm 1.723423887037e+14 true resid norm >>> 4.313917181929e+13 ||r(i)||/||b|| 3.952171198816e-04 >>> 24 KSP preconditioned resid norm 1.656260736170e+14 true resid norm >>> 3.420915571267e+13 ||r(i)||/||b|| 3.134052746997e-04 >>> 25 KSP preconditioned resid norm 1.188400863617e+14 true resid norm >>> 1.877160245085e+13 ||r(i)||/||b|| 1.719749903236e-04 >>> 26 KSP preconditioned resid norm 8.416275062676e+13 true resid norm >>> 1.813244070153e+13 ||r(i)||/||b|| 1.661193455568e-04 >>> 27 KSP preconditioned resid norm 5.224466611453e+13 true resid norm >>> 8.293327735079e+12 ||r(i)||/||b|| 7.597886012791e-05 >>> 28 KSP preconditioned resid norm 4.471999764526e+13 true resid norm >>> 9.742703301619e+12 ||r(i)||/||b|| 8.925723365429e-05 >>> 29 KSP preconditioned resid norm 9.410327220658e+14 true resid norm >>> 1.612776349391e+14 ||r(i)||/||b|| 1.477536069746e-03 >>> 30 KSP preconditioned resid norm 5.783713583723e+14 true resid norm >>> 1.102180319983e+14 ||r(i)||/||b|| 1.009756361291e-03 >>> 31 KSP preconditioned resid norm 1.833775181615e+13 true resid norm >>> 4.198845362007e+12 ||r(i)||/||b|| 3.846748791915e-05 >>> 32 KSP preconditioned resid norm 1.462922686654e+13 true resid norm >>> 3.704838629537e+12 ||r(i)||/||b|| 3.394167275452e-05 >>> 33 KSP preconditioned resid norm 1.053043620710e+13 true resid norm >>> 1.904310347574e+12 ||r(i)||/||b|| 1.744623318411e-05 >>> 34 KSP preconditioned resid norm 8.014167149326e+13 true resid norm >>> 1.458222541169e+13 ||r(i)||/||b|| 1.335942459168e-04 >>> 35 KSP preconditioned resid norm 5.245679087504e+12 true resid norm >>> 1.495201096266e+12 ||r(i)||/||b|| 1.369820156459e-05 >>> 0 KSP preconditioned resid norm 1.124208361458e+16 true resid norm >>> 6.487794900041e+15 ||r(i)||/||b|| 1.232063762752e-01 >>> 1 KSP preconditioned resid norm 5.136632948471e+15 true resid norm >>> 1.030125368299e+15 ||r(i)||/||b|| 1.956258107611e-02 >>> 2 KSP preconditioned resid norm 4.225773164973e+15 true resid norm >>> 7.107211019673e+14 ||r(i)||/||b|| 1.349693892375e-02 >>> 3 KSP preconditioned resid norm 4.144329493920e+15 true resid norm >>> 7.618007419548e+14 ||r(i)||/||b|| 1.446696609651e-02 >>> 4 KSP preconditioned resid norm 4.333716562490e+15 true resid norm >>> 8.456727780045e+14 ||r(i)||/||b|| 1.605973679776e-02 >>> 5 KSP preconditioned resid norm 4.885745055980e+15 true resid norm >>> 7.972847407816e+14 ||r(i)||/||b|| 1.514082446881e-02 >>> 6 KSP preconditioned resid norm 6.011621624545e+15 true resid norm >>> 9.523510473075e+14 ||r(i)||/||b|| 1.808560894548e-02 >>> 7 KSP preconditioned resid norm 7.637951309150e+15 true resid norm >>> 1.487731882292e+15 ||r(i)||/||b|| 2.825275103643e-02 >>> 8 KSP preconditioned resid norm 8.460249904060e+15 true resid norm >>> 1.336944180115e+15 ||r(i)||/||b|| 2.538921933447e-02 >>> 9 KSP preconditioned resid norm 8.381050292195e+15 true resid norm >>> 1.752911449852e+15 ||r(i)||/||b|| 3.328863982216e-02 >>> 10 KSP preconditioned resid norm 7.434169027728e+15 true resid norm >>> 1.180498665014e+15 ||r(i)||/||b|| 2.241824301708e-02 >>> 11 KSP preconditioned resid norm 5.682629267055e+15 true resid norm >>> 9.983854763086e+14 ||r(i)||/||b|| 1.895982511114e-02 >>> 12 KSP preconditioned resid norm 3.541898147790e+15 true resid norm >>> 5.480576756264e+14 ||r(i)||/||b|| 1.040788145188e-02 >>> 13 KSP preconditioned resid norm 2.510644084392e+15 true resid norm >>> 4.650369826671e+14 ||r(i)||/||b|| 8.831278169411e-03 >>> 14 KSP preconditioned resid norm 2.282038802861e+15 true resid norm >>> 3.635360161467e+14 ||r(i)||/||b|| 6.903725516148e-03 >>> 15 KSP preconditioned resid norm 2.130130680848e+15 true resid norm >>> 3.489368266499e+14 ||r(i)||/||b|| 6.626479816775e-03 >>> 16 KSP preconditioned resid norm 1.929984942209e+15 true resid norm >>> 3.145683546766e+14 ||r(i)||/||b|| 5.973805841227e-03 >>> 17 KSP preconditioned resid norm 1.692158321133e+15 true resid norm >>> 2.756020615583e+14 ||r(i)||/||b|| 5.233817008974e-03 >>> 18 KSP preconditioned resid norm 1.473781775606e+15 true resid norm >>> 2.951884895649e+14 ||r(i)||/||b|| 5.605772789952e-03 >>> 19 KSP preconditioned resid norm 1.275239549902e+15 true resid norm >>> 1.993708462138e+14 ||r(i)||/||b|| 3.786149204063e-03 >>> 20 KSP preconditioned resid norm 1.159341081209e+15 true resid norm >>> 2.011812256624e+14 ||r(i)||/||b|| 3.820529189093e-03 >>> 21 KSP preconditioned resid norm 1.026465449352e+15 true resid norm >>> 1.599378003014e+14 ||r(i)||/||b|| 3.037296509548e-03 >>> 22 KSP preconditioned resid norm 9.313333291328e+14 true resid norm >>> 1.626450968383e+14 ||r(i)||/||b|| 3.088709385718e-03 >>> 23 KSP preconditioned resid norm 8.475207473455e+14 true resid norm >>> 1.311587742521e+14 ||r(i)||/||b|| 2.490768826892e-03 >>> 24 KSP preconditioned resid norm 7.894756750239e+14 true resid norm >>> 1.443612018407e+14 ||r(i)||/||b|| 2.741489339222e-03 >>> 25 KSP preconditioned resid norm 7.370222420808e+14 true resid norm >>> 1.135397001576e+14 ||r(i)||/||b|| 2.156174052249e-03 >>> 26 KSP preconditioned resid norm 6.874978587995e+14 true resid norm >>> 1.225152011526e+14 ||r(i)||/||b|| 2.326623175549e-03 >>> 27 KSP preconditioned resid norm 6.203646083349e+14 true resid norm >>> 9.547951255722e+13 ||r(i)||/||b|| 1.813202317882e-03 >>> 28 KSP preconditioned resid norm 5.655995299941e+14 true resid norm >>> 9.635247011837e+13 ||r(i)||/||b|| 1.829780205963e-03 >>> 29 KSP preconditioned resid norm 5.170668664072e+14 true resid norm >>> 7.945307015395e+13 ||r(i)||/||b|| 1.508852392597e-03 >>> 30 KSP preconditioned resid norm 4.771681972381e+14 true resid norm >>> 7.711681382438e+13 ||r(i)||/||b|| 1.464485750178e-03 >>> 31 KSP preconditioned resid norm 4.347723667483e+14 true resid norm >>> 6.696963877348e+13 ||r(i)||/||b|| 1.271785967477e-03 >>> 32 KSP preconditioned resid norm 4.053435837983e+14 true resid norm >>> 6.425971653178e+13 ||r(i)||/||b|| 1.220323227897e-03 >>> 33 KSP preconditioned resid norm 3.774514402447e+14 true resid norm >>> 5.808746381526e+13 ||r(i)||/||b|| 1.103109150945e-03 >>> 34 KSP preconditioned resid norm 3.533225997981e+14 true resid norm >>> 5.617172311710e+13 ||r(i)||/||b|| 1.066728304612e-03 >>> 0 KSP preconditioned resid norm 1.117659190837e+16 true resid norm >>> 6.478857863821e+15 ||r(i)||/||b|| 6.224768870229e-01 >>> 1 KSP preconditioned resid norm 5.124485271748e+15 true resid norm >>> 1.017923791746e+15 ||r(i)||/||b|| 9.780026764454e-02 >>> 2 KSP preconditioned resid norm 4.246328499709e+15 true resid norm >>> 7.102918780385e+14 ||r(i)||/||b|| 6.824355255393e-02 >>> 3 KSP preconditioned resid norm 4.210000472667e+15 true resid norm >>> 7.697163273299e+14 ||r(i)||/||b|| 7.395294562682e-02 >>> 4 KSP preconditioned resid norm 4.471596919519e+15 true resid norm >>> 8.475637573508e+14 ||r(i)||/||b|| 8.143238520100e-02 >>> 5 KSP preconditioned resid norm 5.135403291038e+15 true resid norm >>> 8.255731593574e+14 ||r(i)||/||b|| 7.931956851781e-02 >>> 6 KSP preconditioned resid norm 6.474268664840e+15 true resid norm >>> 1.007942498042e+15 ||r(i)||/||b|| 9.684128308834e-02 >>> 7 KSP preconditioned resid norm 8.529281532903e+15 true resid norm >>> 1.642696055823e+15 ||r(i)||/||b|| 1.578272511369e-01 >>> 8 KSP preconditioned resid norm 9.784577605090e+15 true resid norm >>> 1.538078623395e+15 ||r(i)||/||b|| 1.477757983909e-01 >>> 9 KSP preconditioned resid norm 1.010006302597e+16 true resid norm >>> 2.046930405602e+15 ||r(i)||/||b|| 1.966653526923e-01 >>> 10 KSP preconditioned resid norm 9.531884877717e+15 true resid norm >>> 1.482620346274e+15 ||r(i)||/||b|| 1.424474679309e-01 >>> 11 KSP preconditioned resid norm 7.931210548314e+15 true resid norm >>> 1.282982557872e+15 ||r(i)||/||b|| 1.232666321002e-01 >>> 12 KSP preconditioned resid norm 5.236526900965e+15 true resid norm >>> 7.915538690853e+14 ||r(i)||/||b|| 7.605105694487e-02 >>> 13 KSP preconditioned resid norm 3.885416517845e+15 true resid norm >>> 6.196979959619e+14 ||r(i)||/||b|| 5.953945703529e-02 >>> 14 KSP preconditioned resid norm 3.337626627490e+15 true resid norm >>> 7.828597862542e+14 ||r(i)||/||b|| 7.521574526958e-02 >>> 15 KSP preconditioned resid norm 3.054301414200e+15 true resid norm >>> 4.972852337202e+14 ||r(i)||/||b|| 4.777826134714e-02 >>> 16 KSP preconditioned resid norm 2.784835626497e+15 true resid norm >>> 4.900149140724e+14 ||r(i)||/||b|| 4.707974225054e-02 >>> 17 KSP preconditioned resid norm 2.551999131175e+15 true resid norm >>> 4.223064940199e+14 ||r(i)||/||b|| 4.057444032459e-02 >>> 18 KSP preconditioned resid norm 2.358735848485e+15 true resid norm >>> 3.867160573888e+14 ||r(i)||/||b|| 3.715497586533e-02 >>> 19 KSP preconditioned resid norm 2.111367549832e+15 true resid norm >>> 3.463614183500e+14 ||r(i)||/||b|| 3.327777549857e-02 >>> 20 KSP preconditioned resid norm 1.933287196882e+15 true resid norm >>> 3.127201079650e+14 ||r(i)||/||b|| 3.004557954614e-02 >>> 21 KSP preconditioned resid norm 1.712007284199e+15 true resid norm >>> 2.787684847304e+14 ||r(i)||/||b|| 2.678356929917e-02 >>> 22 KSP preconditioned resid norm 1.556420935486e+15 true resid norm >>> 2.516757515483e+14 ||r(i)||/||b|| 2.418054874112e-02 >>> 23 KSP preconditioned resid norm 1.424455851932e+15 true resid norm >>> 2.267048417314e+14 ||r(i)||/||b|| 2.178138911521e-02 >>> 24 KSP preconditioned resid norm 1.341093049266e+15 true resid norm >>> 2.200423231461e+14 ||r(i)||/||b|| 2.114126643990e-02 >>> 25 KSP preconditioned resid norm 1.265242012282e+15 true resid norm >>> 1.979405460182e+14 ||r(i)||/||b|| 1.901776786755e-02 >>> 26 KSP preconditioned resid norm 1.190343462827e+15 true resid norm >>> 1.929502180424e+14 ||r(i)||/||b|| 1.853830622649e-02 >>> 27 KSP preconditioned resid norm 1.080874612432e+15 true resid norm >>> 1.688981747210e+14 ||r(i)||/||b|| 1.622742962325e-02 >>> 28 KSP preconditioned resid norm 9.890419339482e+14 true resid norm >>> 1.571112460256e+14 ||r(i)||/||b|| 1.509496293914e-02 >>> 29 KSP preconditioned resid norm 9.054720544549e+14 true resid norm >>> 1.411421952126e+14 ||r(i)||/||b|| 1.356068556375e-02 >>> 30 KSP preconditioned resid norm 8.347895469778e+14 true resid norm >>> 1.302020126144e+14 ||r(i)||/||b|| 1.250957270554e-02 >>> 31 KSP preconditioned resid norm 7.581633555103e+14 true resid norm >>> 1.198729260222e+14 ||r(i)||/||b|| 1.151717284080e-02 >>> 32 KSP preconditioned resid norm 7.073149016328e+14 true resid norm >>> 1.097657041094e+14 ||r(i)||/||b|| 1.054608933119e-02 >>> 33 KSP preconditioned resid norm 6.626892210136e+14 true resid norm >>> 1.045304064903e+14 ||r(i)||/||b|| 1.004309145207e-02 >>> 34 KSP preconditioned resid norm 6.277118615935e+14 true resid norm >>> 9.735143814138e+13 ||r(i)||/||b|| 9.353349222222e-03 >>> 35 KSP preconditioned resid norm 6.007031548284e+14 true resid norm >>> 9.325545854852e+13 ||r(i)||/||b|| 8.959814948147e-03 >>> 36 KSP preconditioned resid norm 5.720600717487e+14 true resid norm >>> 8.974565981239e+13 ||r(i)||/||b|| 8.622599865294e-03 >>> 37 KSP preconditioned resid norm 5.429834406497e+14 true resid norm >>> 8.373407335142e+13 ||r(i)||/||b|| 8.045017565303e-03 >>> 38 KSP preconditioned resid norm 5.122271874419e+14 true resid norm >>> 8.063599233502e+13 ||r(i)||/||b|| 7.747359572588e-03 >>> 39 KSP preconditioned resid norm 4.877390474820e+14 true resid norm >>> 7.540978455988e+13 ||r(i)||/||b|| 7.245235029160e-03 >>> 40 KSP preconditioned resid norm 4.664511103016e+14 true resid norm >>> 7.249582547116e+13 ||r(i)||/||b|| 6.965267136580e-03 >>> 41 KSP preconditioned resid norm 4.430412921029e+14 true resid norm >>> 6.907345898754e+13 ||r(i)||/||b|| 6.636452385624e-03 >>> 42 KSP preconditioned resid norm 4.184290025581e+14 true resid norm >>> 6.506826627971e+13 ||r(i)||/||b|| 6.251640750440e-03 >>> 43 KSP preconditioned resid norm 3.961557540211e+14 true resid norm >>> 6.169515842630e+13 ||r(i)||/||b|| 5.927558679138e-03 >>> 44 KSP preconditioned resid norm 3.628856632080e+14 true resid norm >>> 5.666822624927e+13 ||r(i)||/||b|| 5.444580172956e-03 >>> 45 KSP preconditioned resid norm 3.439726178254e+14 true resid norm >>> 5.290740915351e+13 ||r(i)||/||b|| 5.083247702381e-03 >>> 46 KSP preconditioned resid norm 3.808130490503e+14 true resid norm >>> 7.046303156753e+13 ||r(i)||/||b|| 6.769959993302e-03 >>> 47 KSP preconditioned resid norm 2.548925306353e+14 true resid norm >>> 3.911087819219e+13 ||r(i)||/||b|| 3.757702085387e-03 >>> 48 KSP preconditioned resid norm 1.722230117103e+14 true resid norm >>> 2.660198748286e+13 ||r(i)||/||b|| 2.555870603278e-03 >>> 49 KSP preconditioned resid norm 8.498499404656e+13 true resid norm >>> 1.314954336818e+13 ||r(i)||/||b|| 1.263384225066e-03 >>> 50 KSP preconditioned resid norm 1.168755498979e+14 true resid norm >>> 2.563042385986e+13 ||r(i)||/||b|| 2.462524536378e-03 >>> 51 KSP preconditioned resid norm 5.069372366763e+13 true resid norm >>> 7.752336990767e+12 ||r(i)||/||b|| 7.448304467540e-04 >>> 0 KSP preconditioned resid norm 2.551555721110e+16 true resid norm >>> 1.503348678660e+16 ||r(i)||/||b|| 1.139781099215e-01 >>> 1 KSP preconditioned resid norm 1.135435821514e+16 true resid norm >>> 2.150528625386e+15 ||r(i)||/||b|| 1.630448022690e-02 >>> 2 KSP preconditioned resid norm 8.921910642422e+15 true resid norm >>> 1.462019059252e+15 ||r(i)||/||b|| 1.108446572695e-02 >>> 3 KSP preconditioned resid norm 8.401630743802e+15 true resid norm >>> 1.508644008325e+15 ||r(i)||/||b|| 1.143795814332e-02 >>> 4 KSP preconditioned resid norm 8.507938797573e+15 true resid norm >>> 1.437822857483e+15 ||r(i)||/||b|| 1.090101943907e-02 >>> 5 KSP preconditioned resid norm 9.195685552195e+15 true resid norm >>> 1.421948919345e+15 ||r(i)||/||b|| 1.078066935052e-02 >>> 6 KSP preconditioned resid norm 1.067998630800e+16 true resid norm >>> 1.627316531548e+15 ||r(i)||/||b|| 1.233768753335e-02 >>> 7 KSP preconditioned resid norm 1.295276293823e+16 true resid norm >>> 2.221334900964e+15 ||r(i)||/||b|| 1.684130615262e-02 >>> 8 KSP preconditioned resid norm 1.417997817727e+16 true resid norm >>> 2.186215590047e+15 ||r(i)||/||b|| 1.657504505585e-02 >>> 9 KSP preconditioned resid norm 1.359372910192e+16 true resid norm >>> 2.107067442870e+15 ||r(i)||/||b|| 1.597497427073e-02 >>> 10 KSP preconditioned resid norm 1.179811021994e+16 true resid norm >>> 3.232410469947e+15 ||r(i)||/||b|| 2.450689191967e-02 >>> 11 KSP preconditioned resid norm 7.523875759744e+15 true resid norm >>> 1.204652282825e+15 ||r(i)||/||b|| 9.133209897209e-03 >>> 12 KSP preconditioned resid norm 3.961884677960e+15 true resid norm >>> 7.586571607759e+14 ||r(i)||/||b|| 5.751846560354e-03 >>> 13 KSP preconditioned resid norm 2.175818220311e+15 true resid norm >>> 3.487257134817e+14 ||r(i)||/||b|| 2.643904123366e-03 >>> 14 KSP preconditioned resid norm 1.788448374434e+15 true resid norm >>> 3.516091940967e+14 ||r(i)||/||b|| 2.665765563440e-03 >>> 15 KSP preconditioned resid norm 1.604495215225e+15 true resid norm >>> 2.568394922627e+14 ||r(i)||/||b|| 1.947258164179e-03 >>> 16 KSP preconditioned resid norm 1.391089499920e+15 true resid norm >>> 2.519771358465e+14 ||r(i)||/||b|| 1.910393649516e-03 >>> 17 KSP preconditioned resid norm 1.231055803595e+15 true resid norm >>> 2.082047844556e+14 ||r(i)||/||b|| 1.578528530720e-03 >>> 18 KSP preconditioned resid norm 1.101068654831e+15 true resid norm >>> 1.838589386072e+14 ||r(i)||/||b|| 1.393947699032e-03 >>> 19 KSP preconditioned resid norm 9.356667448723e+14 true resid norm >>> 1.565965285410e+14 ||r(i)||/||b|| 1.187254600129e-03 >>> 20 KSP preconditioned resid norm 8.181513038456e+14 true resid norm >>> 1.328185221533e+14 ||r(i)||/||b|| 1.006978908651e-03 >>> 0 KSP preconditioned resid norm 2.272810589988e+16 true resid norm >>> 1.323636987884e+16 ||r(i)||/||b|| 5.438005326146e-01 >>> 1 KSP preconditioned resid norm 1.026969580669e+16 true resid norm >>> 1.999385093393e+15 ||r(i)||/||b|| 8.214236143606e-02 >>> 2 KSP preconditioned resid norm 8.322628960001e+15 true resid norm >>> 1.379596337850e+15 ||r(i)||/||b|| 5.667907667911e-02 >>> 3 KSP preconditioned resid norm 8.076035080201e+15 true resid norm >>> 1.472961452514e+15 ||r(i)||/||b|| 6.051487150403e-02 >>> 4 KSP preconditioned resid norm 8.405382352119e+15 true resid norm >>> 1.516410203314e+15 ||r(i)||/||b|| 6.229991181666e-02 >>> 5 KSP preconditioned resid norm 9.416373856303e+15 true resid norm >>> 1.470293081560e+15 ||r(i)||/||b|| 6.040524465323e-02 >>> 6 KSP preconditioned resid norm 1.145894310132e+16 true resid norm >>> 1.749278072065e+15 ||r(i)||/||b|| 7.186701157399e-02 >>> 7 KSP preconditioned resid norm 1.439168534642e+16 true resid norm >>> 2.685145033101e+15 ||r(i)||/||b|| 1.103159939254e-01 >>> 8 KSP preconditioned resid norm 1.556059742979e+16 true resid norm >>> 2.450465189043e+15 ||r(i)||/||b|| 1.006744513151e-01 >>> 9 KSP preconditioned resid norm 1.521075410830e+16 true resid norm >>> 2.889962061237e+15 ||r(i)||/||b|| 1.187306582184e-01 >>> 10 KSP preconditioned resid norm 1.335253374250e+16 true resid norm >>> 2.051263914753e+15 ||r(i)||/||b|| 8.427374118333e-02 >>> 11 KSP preconditioned resid norm 9.966633192923e+15 true resid norm >>> 1.537352185287e+15 ||r(i)||/||b|| 6.316028826844e-02 >>> 12 KSP preconditioned resid norm 5.467457389741e+15 true resid norm >>> 8.244964108686e+14 ||r(i)||/||b|| 3.387345559797e-02 >>> 13 KSP preconditioned resid norm 7.465607339270e+15 true resid norm >>> 1.255988023601e+15 ||r(i)||/||b|| 5.160077592599e-02 >>> 14 KSP preconditioned resid norm 3.395725692367e+15 true resid norm >>> 8.163584192182e+14 ||r(i)||/||b|| 3.353911587836e-02 >>> 15 KSP preconditioned resid norm 2.979451058224e+15 true resid norm >>> 4.736799198293e+14 ||r(i)||/||b|| 1.946057680844e-02 >>> 16 KSP preconditioned resid norm 2.589762095203e+15 true resid norm >>> 5.000061827618e+14 ||r(i)||/||b|| 2.054216004731e-02 >>> 17 KSP preconditioned resid norm 2.297387399767e+15 true resid norm >>> 3.733072000155e+14 ||r(i)||/||b|| 1.533688285047e-02 >>> 18 KSP preconditioned resid norm 2.076822069911e+15 true resid norm >>> 3.563941616114e+14 ||r(i)||/||b|| 1.464203075912e-02 >>> 19 KSP preconditioned resid norm 1.810897092306e+15 true resid norm >>> 2.945299484347e+14 ||r(i)||/||b|| 1.210041305100e-02 >>> 20 KSP preconditioned resid norm 1.630399807803e+15 true resid norm >>> 2.670493324597e+14 ||r(i)||/||b|| 1.097140458866e-02 >>> 21 KSP preconditioned resid norm 1.416015877821e+15 true resid norm >>> 2.289562555093e+14 ||r(i)||/||b|| 9.406395773998e-03 >>> 22 KSP preconditioned resid norm 1.270953330458e+15 true resid norm >>> 2.078621859272e+14 ||r(i)||/||b|| 8.539770983461e-03 >>> 23 KSP preconditioned resid norm 1.149572826465e+15 true resid norm >>> 1.837793084226e+14 ||r(i)||/||b|| 7.550354569913e-03 >>> 24 KSP preconditioned resid norm 1.075403245776e+15 true resid norm >>> 1.776412893147e+14 ||r(i)||/||b|| 7.298181346390e-03 >>> 25 KSP preconditioned resid norm 1.008390645992e+15 true resid norm >>> 1.577927704554e+14 ||r(i)||/||b|| 6.482728527673e-03 >>> 26 KSP preconditioned resid norm 9.445856732256e+14 true resid norm >>> 1.515432067895e+14 ||r(i)||/||b|| 6.225972628492e-03 >>> 27 KSP preconditioned resid norm 8.526344464253e+14 true resid norm >>> 1.348203382746e+14 ||r(i)||/||b|| 5.538933441126e-03 >>> 28 KSP preconditioned resid norm 7.744588594030e+14 true resid norm >>> 1.215509014949e+14 ||r(i)||/||b|| 4.993774394172e-03 >>> 29 KSP preconditioned resid norm 7.032530926217e+14 true resid norm >>> 1.102818922582e+14 ||r(i)||/||b|| 4.530800536453e-03 >>> 30 KSP preconditioned resid norm 6.388922318141e+14 true resid norm >>> 9.838037383987e+13 ||r(i)||/||b|| 4.041840790388e-03 >>> 31 KSP preconditioned resid norm 5.692116056101e+14 true resid norm >>> 9.317411265002e+13 ||r(i)||/||b|| 3.827947734067e-03 >>> 32 KSP preconditioned resid norm 5.208032879647e+14 true resid norm >>> 8.007380640801e+13 ||r(i)||/||b|| 3.289737214338e-03 >>> 33 KSP preconditioned resid norm 4.765676071483e+14 true resid norm >>> 7.802986158610e+13 ||r(i)||/||b|| 3.205764169389e-03 >>> 34 KSP preconditioned resid norm 4.409793613234e+14 true resid norm >>> 6.791885197478e+13 ||r(i)||/||b|| 2.790365350662e-03 >>> 35 KSP preconditioned resid norm 4.123618028942e+14 true resid norm >>> 6.437750648670e+13 ||r(i)||/||b|| 2.644873378149e-03 >>> 36 KSP preconditioned resid norm 3.827034276493e+14 true resid norm >>> 5.974308940262e+13 ||r(i)||/||b|| 2.454473857604e-03 >>> 37 KSP preconditioned resid norm 3.509107846575e+14 true resid norm >>> 5.419166518671e+13 ||r(i)||/||b|| 2.226400188387e-03 >>> 38 KSP preconditioned resid norm 3.170601404687e+14 true resid norm >>> 4.972345072887e+13 ||r(i)||/||b|| 2.042828905305e-03 >>> 39 KSP preconditioned resid norm 2.901553052614e+14 true resid norm >>> 4.518683026559e+13 ||r(i)||/||b|| 1.856447242750e-03 >>> 40 KSP preconditioned resid norm 2.667015211684e+14 true resid norm >>> 4.122270495295e+13 ||r(i)||/||b|| 1.693585863377e-03 >>> 41 KSP preconditioned resid norm 2.422095249996e+14 true resid norm >>> 3.839959453856e+13 ||r(i)||/||b|| 1.577601725654e-03 >>> 42 KSP preconditioned resid norm 2.195894402502e+14 true resid norm >>> 3.399029120498e+13 ||r(i)||/||b|| 1.396450736130e-03 >>> 43 KSP preconditioned resid norm 1.983452570770e+14 true resid norm >>> 3.143816040870e+13 ||r(i)||/||b|| 1.291599474113e-03 >>> 44 KSP preconditioned resid norm 1.779873290535e+14 true resid norm >>> 2.755766172822e+13 ||r(i)||/||b|| 1.132173795580e-03 >>> 45 KSP preconditioned resid norm 1.542400485092e+14 true resid norm >>> 2.400274515345e+13 ||r(i)||/||b|| 9.861242710910e-04 >>> 46 KSP preconditioned resid norm 1.223281336233e+14 true resid norm >>> 1.907355666953e+13 ||r(i)||/||b|| 7.836144177513e-04 >>> 0 KSP preconditioned resid norm 7.586062313136e+16 true resid norm >>> 4.608637352265e+16 ||r(i)||/||b|| 6.590757061047e-02 >>> 1 KSP preconditioned resid norm 2.763422737801e+16 true resid norm >>> 5.227955984017e+15 ||r(i)||/||b|| 7.476437216213e-03 >>> 2 KSP preconditioned resid norm 1.553456028419e+16 true resid norm >>> 2.906104653716e+15 ||r(i)||/||b|| 4.155985447023e-03 >>> 3 KSP preconditioned resid norm 1.040767126705e+16 true resid norm >>> 2.091654777018e+15 ||r(i)||/||b|| 2.991250436342e-03 >>> 4 KSP preconditioned resid norm 7.531366829970e+15 true resid norm >>> 1.318758653617e+15 ||r(i)||/||b|| 1.885940950392e-03 >>> 5 KSP preconditioned resid norm 5.773017012836e+15 true resid norm >>> 9.450385953724e+14 ||r(i)||/||b|| 1.351488372664e-03 >>> 6 KSP preconditioned resid norm 4.731507925306e+15 true resid norm >>> 7.935497854840e+14 ||r(i)||/||b|| 1.134846040642e-03 >>> 7 KSP preconditioned resid norm 3.788639277852e+15 true resid norm >>> 7.524409398539e+14 ||r(i)||/||b|| 1.076056773034e-03 >>> 8 KSP preconditioned resid norm 2.738698664678e+15 true resid norm >>> 4.915430614067e+14 ||r(i)||/||b|| 7.029498429037e-04 >>> 9 KSP preconditioned resid norm 1.837368414949e+15 true resid norm >>> 3.357340764188e+14 ||r(i)||/||b|| 4.801292802316e-04 >>> 10 KSP preconditioned resid norm 1.059331899595e+15 true resid norm >>> 1.989071439556e+14 ||r(i)||/||b|| 2.844547234496e-04 >>> 11 KSP preconditioned resid norm 4.945892712172e+14 true resid norm >>> 8.452757650210e+13 ||r(i)||/||b|| 1.208818744245e-04 >>> 12 KSP preconditioned resid norm 1.987888303244e+14 true resid norm >>> 3.594134462521e+13 ||r(i)||/||b|| 5.139928633259e-05 >>> 13 KSP preconditioned resid norm 1.163784016975e+14 true resid norm >>> 2.571324386412e+13 ||r(i)||/||b|| 3.677220197779e-05 >>> 14 KSP preconditioned resid norm 8.494315446492e+13 true resid norm >>> 1.429434206797e+13 ||r(i)||/||b|| 2.044216732982e-05 >>> 15 KSP preconditioned resid norm 8.642186180523e+13 true resid norm >>> 1.607306715480e+13 ||r(i)||/||b|| 2.298590076546e-05 >>> 16 KSP preconditioned resid norm 3.925619743667e+13 true resid norm >>> 1.371348923807e+13 ||r(i)||/||b|| 1.961149665703e-05 >>> 0 KSP preconditioned resid norm 3.953744769406e+16 true resid norm >>> 2.171194598493e+16 ||r(i)||/||b|| 1.950905346919e-01 >>> 1 KSP preconditioned resid norm 1.491502757726e+16 true resid norm >>> 3.091776185716e+15 ||r(i)||/||b|| 2.778084790914e-02 >>> 2 KSP preconditioned resid norm 1.008617531922e+16 true resid norm >>> 1.691017660922e+15 ||r(i)||/||b|| 1.519447127732e-02 >>> 3 KSP preconditioned resid norm 8.326895039450e+15 true resid norm >>> 1.670298498749e+15 ||r(i)||/||b|| 1.500830130298e-02 >>> 4 KSP preconditioned resid norm 7.300566603499e+15 true resid norm >>> 1.589496633697e+15 ||r(i)||/||b|| 1.428226416804e-02 >>> 5 KSP preconditioned resid norm 6.872352176548e+15 true resid norm >>> 1.096739362492e+15 ||r(i)||/||b|| 9.854642637499e-03 >>> 6 KSP preconditioned resid norm 6.961475959227e+15 true resid norm >>> 1.092150928708e+15 ||r(i)||/||b|| 9.813413721358e-03 >>> 7 KSP preconditioned resid norm 7.025042419735e+15 true resid norm >>> 1.296288429690e+15 ||r(i)||/||b|| 1.164767096595e-02 >>> 8 KSP preconditioned resid norm 6.408702665765e+15 true resid norm >>> 1.056644614043e+15 ||r(i)||/||b|| 9.494375256645e-03 >>> 9 KSP preconditioned resid norm 5.399106179995e+15 true resid norm >>> 1.078818901202e+15 ||r(i)||/||b|| 9.693620112044e-03 >>> 10 KSP preconditioned resid norm 4.030707639439e+15 true resid norm >>> 7.107969460630e+14 ||r(i)||/||b|| 6.386795378039e-03 >>> 11 KSP preconditioned resid norm 2.682586389004e+15 true resid norm >>> 4.368532286577e+14 ||r(i)||/||b|| 3.925301307393e-03 >>> 12 KSP preconditioned resid norm 1.621400972048e+15 true resid norm >>> 2.499302020485e+14 ||r(i)||/||b|| 2.245723012904e-03 >>> 13 KSP preconditioned resid norm 1.118646687359e+15 true resid norm >>> 1.825993809337e+14 ||r(i)||/||b|| 1.640728605602e-03 >>> 14 KSP preconditioned resid norm 8.665732278929e+14 true resid norm >>> 1.337846645968e+14 ||r(i)||/||b|| 1.202108819167e-03 >>> 15 KSP preconditioned resid norm 1.790412083411e+15 true resid norm >>> 2.690613625509e+14 ||r(i)||/||b|| 2.417624155910e-03 >>> 16 KSP preconditioned resid norm 4.902192144952e+14 true resid norm >>> 8.615757491331e+13 ||r(i)||/||b|| 7.741603340972e-04 >>> 17 KSP preconditioned resid norm 3.873085562421e+14 true resid norm >>> 6.966874756362e+13 ||r(i)||/||b|| 6.260016132563e-04 >>> 18 KSP preconditioned resid norm 3.069894649217e+14 true resid norm >>> 4.954148501038e+13 ||r(i)||/||b|| 4.451500941838e-04 >>> 19 KSP preconditioned resid norm 2.366855243734e+14 true resid norm >>> 4.444373941350e+13 ||r(i)||/||b|| 3.993448073197e-04 >>> 20 KSP preconditioned resid norm 1.853280795186e+14 true resid norm >>> 2.839442050460e+13 ||r(i)||/||b|| 2.551352459311e-04 >>> 21 KSP preconditioned resid norm 1.419103820891e+14 true resid norm >>> 2.714177608773e+13 ||r(i)||/||b|| 2.438797339086e-04 >>> 22 KSP preconditioned resid norm 1.116662401807e+14 true resid norm >>> 1.710034207306e+13 ||r(i)||/||b|| 1.536534256654e-04 >>> 23 KSP preconditioned resid norm 8.768706858701e+13 true resid norm >>> 1.892684059360e+13 ||r(i)||/||b|| 1.700652467538e-04 >>> 24 KSP preconditioned resid norm 7.191985915577e+13 true resid norm >>> 1.104970014738e+13 ||r(i)||/||b|| 9.928598345972e-05 >>> 25 KSP preconditioned resid norm 5.804472664630e+13 true resid norm >>> 1.353159760302e+13 ||r(i)||/||b|| 1.215868265996e-04 >>> 26 KSP preconditioned resid norm 4.574262022619e+13 true resid norm >>> 6.915191476157e+12 ||r(i)||/||b|| 6.213576634340e-05 >>> 27 KSP preconditioned resid norm 3.470073834736e+13 true resid norm >>> 8.786634213858e+12 ||r(i)||/||b|| 7.895142923224e-05 >>> 28 KSP preconditioned resid norm 2.674822501624e+13 true resid norm >>> 4.041336605553e+12 ||r(i)||/||b|| 3.631302877201e-05 >>> 29 KSP preconditioned resid norm 2.088626286399e+13 true resid norm >>> 5.261273540428e+12 ||r(i)||/||b|| 4.727465096287e-05 >>> 30 KSP preconditioned resid norm 1.567496199145e+13 true resid norm >>> 2.371012817670e+12 ||r(i)||/||b|| 2.130450023603e-05 >>> 31 KSP preconditioned resid norm 1.119841843422e+13 true resid norm >>> 3.387071792996e+12 ||r(i)||/||b|| 3.043419726606e-05 >>> 32 KSP preconditioned resid norm 7.994377199528e+12 true resid norm >>> 1.204577998308e+12 ||r(i)||/||b|| 1.082361599145e-05 >>> 33 KSP preconditioned resid norm 6.174157756589e+12 true resid norm >>> 1.362745552188e+12 ||r(i)||/||b|| 1.224481484110e-05 >>> 0 SNES Function norm 0.000000000000e+00 >>> 1 TS dt 3e-12 time 2e-12 >>> 0 KSP preconditioned resid norm 2.220368634013e+19 true resid norm >>> 1.204306153911e+19 ||r(i)||/||b|| 2.302397943981e+02 >>> 1 KSP preconditioned resid norm 5.390402695434e+18 true resid norm >>> 1.775960718595e+18 ||r(i)||/||b|| 3.395289722473e+01 >>> 2 KSP preconditioned resid norm 2.218493119258e+18 true resid norm >>> 5.320083037595e+17 ||r(i)||/||b|| 1.017095877804e+01 >>> 3 KSP preconditioned resid norm 1.401798813920e+18 true resid norm >>> 2.963744523802e+17 ||r(i)||/||b|| 5.666100165582e+00 >>> 4 KSP preconditioned resid norm 1.004143331118e+18 true resid norm >>> 1.656359344976e+17 ||r(i)||/||b|| 3.166635276239e+00 >>> 5 KSP preconditioned resid norm 8.673495646713e+17 true resid norm >>> 1.510463721187e+17 ||r(i)||/||b|| 2.887711363779e+00 >>> 6 KSP preconditioned resid norm 7.389956702949e+17 true resid norm >>> 1.526304376340e+17 ||r(i)||/||b|| 2.917995599842e+00 >>> 7 KSP preconditioned resid norm 5.566613599927e+17 true resid norm >>> 1.017416269196e+17 ||r(i)||/||b|| 1.945101018344e+00 >>> 8 KSP preconditioned resid norm 3.566897773081e+17 true resid norm >>> 6.307244640523e+16 ||r(i)||/||b|| 1.205821878877e+00 >>> 9 KSP preconditioned resid norm 1.893453815442e+17 true resid norm >>> 3.268203516935e+16 ||r(i)||/||b|| 6.248166243663e-01 >>> 10 KSP preconditioned resid norm 6.894933140555e+16 true resid norm >>> 1.152954614680e+16 ||r(i)||/||b|| 2.204223839364e-01 >>> 11 KSP preconditioned resid norm 5.103537427509e+16 true resid norm >>> 9.272910080166e+15 ||r(i)||/||b|| 1.772799136993e-01 >>> 12 KSP preconditioned resid norm 4.991235858989e+16 true resid norm >>> 8.396555957121e+15 ||r(i)||/||b|| 1.605257359967e-01 >>> 13 KSP preconditioned resid norm 4.918689912532e+16 true resid norm >>> 8.423517310572e+15 ||r(i)||/||b|| 1.610411843696e-01 >>> 14 KSP preconditioned resid norm 5.686949456502e+16 true resid norm >>> 9.761258523981e+15 ||r(i)||/||b|| 1.866161813032e-01 >>> 15 KSP preconditioned resid norm 1.616570395103e+17 true resid norm >>> 4.351134639515e+16 ||r(i)||/||b|| 8.318518854587e-01 >>> 16 KSP preconditioned resid norm 4.128330778786e+16 true resid norm >>> 7.104207079385e+15 ||r(i)||/||b|| 1.358185517866e-01 >>> 17 KSP preconditioned resid norm 2.161181135112e+16 true resid norm >>> 5.859480309696e+15 ||r(i)||/||b|| 1.120218091889e-01 >>> 18 KSP preconditioned resid norm 1.410375919962e+16 true resid norm >>> 2.425065017616e+15 ||r(i)||/||b|| 4.636250252850e-02 >>> 19 KSP preconditioned resid norm 1.082544604615e+16 true resid norm >>> 2.485306543509e+15 ||r(i)||/||b|| 4.751420274116e-02 >>> 20 KSP preconditioned resid norm 8.996941383859e+15 true resid norm >>> 1.577914541048e+15 ||r(i)||/||b|| 3.016664145813e-02 >>> 21 KSP preconditioned resid norm 7.748060569447e+15 true resid norm >>> 1.336297549587e+15 ||r(i)||/||b|| 2.554739690340e-02 >>> 22 KSP preconditioned resid norm 6.873596381580e+15 true resid norm >>> 1.173018262854e+15 ||r(i)||/||b|| 2.242581612555e-02 >>> 23 KSP preconditioned resid norm 6.175955090548e+15 true resid norm >>> 1.012331331012e+15 ||r(i)||/||b|| 1.935379610558e-02 >>> 24 KSP preconditioned resid norm 5.614288705672e+15 true resid norm >>> 9.369191861544e+14 ||r(i)||/||b|| 1.791206331440e-02 >>> 25 KSP preconditioned resid norm 5.220540757763e+15 true resid norm >>> 8.444260437985e+14 ||r(i)||/||b|| 1.614377524163e-02 >>> 26 KSP preconditioned resid norm 5.059721722058e+15 true resid norm >>> 8.452379209101e+14 ||r(i)||/||b|| 1.615929674492e-02 >>> 27 KSP preconditioned resid norm 4.893746226621e+15 true resid norm >>> 9.389168158486e+14 ||r(i)||/||b|| 1.795025408912e-02 >>> 28 KSP preconditioned resid norm 4.794914716345e+15 true resid norm >>> 7.493396637157e+14 ||r(i)||/||b|| 1.432590953288e-02 >>> 29 KSP preconditioned resid norm 4.860853428688e+15 true resid norm >>> 7.693465766642e+14 ||r(i)||/||b|| 1.470840259819e-02 >>> 30 KSP preconditioned resid norm 5.112203028306e+15 true resid norm >>> 8.019578406020e+14 ||r(i)||/||b|| 1.533186621496e-02 >>> 31 KSP preconditioned resid norm 5.649006086453e+15 true resid norm >>> 8.938106702489e+14 ||r(i)||/||b|| 1.708791276044e-02 >>> 32 KSP preconditioned resid norm 8.383819984522e+15 true resid norm >>> 1.347106201218e+15 ||r(i)||/||b|| 2.575403719341e-02 >>> 33 KSP preconditioned resid norm 1.617520068743e+16 true resid norm >>> 2.603058585140e+15 ||r(i)||/||b|| 4.976539159104e-02 >>> 34 KSP preconditioned resid norm 3.383001122808e+16 true resid norm >>> 5.541435636245e+15 ||r(i)||/||b|| 1.059414167582e-01 >>> 35 KSP preconditioned resid norm 6.256166371964e+15 true resid norm >>> 1.029049029941e+15 ||r(i)||/||b|| 1.967340582872e-02 >>> 36 KSP preconditioned resid norm 3.290396673886e+15 true resid norm >>> 5.409340387326e+14 ||r(i)||/||b|| 1.034160138236e-02 >>> 37 KSP preconditioned resid norm 2.075346751579e+15 true resid norm >>> 3.809761571158e+14 ||r(i)||/||b|| 7.283519377531e-03 >>> 38 KSP preconditioned resid norm 1.750045982315e+15 true resid norm >>> 2.837816436037e+14 ||r(i)||/||b|| 5.425350278670e-03 >>> 39 KSP preconditioned resid norm 1.596755323762e+15 true resid norm >>> 2.766061504317e+14 ||r(i)||/||b|| 5.288168876146e-03 >>> 40 KSP preconditioned resid norm 1.469812007135e+15 true resid norm >>> 2.359521384401e+14 ||r(i)||/||b|| 4.510943638858e-03 >>> 41 KSP preconditioned resid norm 1.314668041488e+15 true resid norm >>> 2.158813968118e+14 ||r(i)||/||b|| 4.127230293966e-03 >>> 42 KSP preconditioned resid norm 1.190446489712e+15 true resid norm >>> 1.941922420158e+14 ||r(i)||/||b|| 3.712576053042e-03 >>> 43 KSP preconditioned resid norm 1.100371263474e+15 true resid norm >>> 1.837040226442e+14 ||r(i)||/||b|| 3.512061801423e-03 >>> 44 KSP preconditioned resid norm 1.032027622009e+15 true resid norm >>> 1.624908335172e+14 ||r(i)||/||b|| 3.106507093655e-03 >>> 45 KSP preconditioned resid norm 8.066010465859e+14 true resid norm >>> 2.499636130742e+14 ||r(i)||/||b|| 4.778815643705e-03 >>> 46 KSP preconditioned resid norm 6.546087590187e+14 true resid norm >>> 1.014535455422e+14 ||r(i)||/||b|| 1.939593465560e-03 >>> 47 KSP preconditioned resid norm 6.561404017041e+14 true resid norm >>> 1.083911025925e+14 ||r(i)||/||b|| 2.072225994564e-03 >>> 48 KSP preconditioned resid norm 6.834623592895e+14 true resid norm >>> 1.073483632090e+14 ||r(i)||/||b|| 2.052290855937e-03 >>> 49 KSP preconditioned resid norm 8.808387787841e+14 true resid norm >>> 1.369903526339e+14 ||r(i)||/||b|| 2.618987748466e-03 >>> 50 KSP preconditioned resid norm 3.995486126369e+14 true resid norm >>> 1.237415046233e+14 ||r(i)||/||b|| 2.365695673849e-03 >>> 51 KSP preconditioned resid norm 3.217227010099e+14 true resid norm >>> 4.935649374236e+13 ||r(i)||/||b|| 9.435996764234e-04 >>> 0 KSP preconditioned resid norm 6.543260687832e+18 true resid norm >>> 3.546137973950e+18 ||r(i)||/||b|| 4.156987252753e+02 >>> 1 KSP preconditioned resid norm 1.591265056482e+18 true resid norm >>> 5.223494588784e+17 ||r(i)||/||b|| 6.123281321796e+01 >>> 2 KSP preconditioned resid norm 6.570987793466e+17 true resid norm >>> 1.552598280892e+17 ||r(i)||/||b|| 1.820045161730e+01 >>> 3 KSP preconditioned resid norm 4.166120135488e+17 true resid norm >>> 8.739095834512e+16 ||r(i)||/||b|| 1.024447166228e+01 >>> 4 KSP preconditioned resid norm 2.995238174583e+17 true resid norm >>> 4.935051588719e+16 ||r(i)||/||b|| 5.785151817748e+00 >>> 5 KSP preconditioned resid norm 2.594691834001e+17 true resid norm >>> 4.470128066210e+16 ||r(i)||/||b|| 5.240141676921e+00 >>> 6 KSP preconditioned resid norm 2.219773011977e+17 true resid norm >>> 4.652509796902e+16 ||r(i)||/||b|| 5.453940050023e+00 >>> 7 KSP preconditioned resid norm 1.675026903695e+17 true resid norm >>> 2.988993956885e+16 ||r(i)||/||b|| 3.503870934691e+00 >>> 8 KSP preconditioned resid norm 1.081971008730e+17 true resid norm >>> 1.908275119338e+16 ||r(i)||/||b|| 2.236990044975e+00 >>> 9 KSP preconditioned resid norm 5.710427760412e+16 true resid norm >>> 9.635681698385e+15 ||r(i)||/||b|| 1.129550127097e+00 >>> 10 KSP preconditioned resid norm 1.779335357179e+16 true resid norm >>> 2.952424081259e+15 ||r(i)||/||b|| 3.461001619416e-01 >>> 11 KSP preconditioned resid norm 1.125492898955e+16 true resid norm >>> 2.030196257790e+15 ||r(i)||/||b|| 2.379913028263e-01 >>> 12 KSP preconditioned resid norm 1.086499382749e+16 true resid norm >>> 1.877721745252e+15 ||r(i)||/||b|| 2.201173619461e-01 >>> 13 KSP preconditioned resid norm 1.075258231290e+16 true resid norm >>> 1.822442508498e+15 ||r(i)||/||b|| 2.136372113085e-01 >>> 14 KSP preconditioned resid norm 1.173739259962e+16 true resid norm >>> 2.130298543993e+15 ||r(i)||/||b|| 2.497258695795e-01 >>> 15 KSP preconditioned resid norm 4.943363278431e+16 true resid norm >>> 1.361259414632e+16 ||r(i)||/||b|| 1.595746718229e+00 >>> 16 KSP preconditioned resid norm 1.782643390767e+17 true resid norm >>> 3.375538781586e+16 ||r(i)||/||b|| 3.957001049961e+00 >>> 17 KSP preconditioned resid norm 3.253381322946e+16 true resid norm >>> 8.720742251900e+15 ||r(i)||/||b|| 1.022295653525e+00 >>> 18 KSP preconditioned resid norm 2.077141437527e+16 true resid norm >>> 3.948414579285e+15 ||r(i)||/||b|| 4.628559067708e-01 >>> 19 KSP preconditioned resid norm 1.484650052668e+16 true resid norm >>> 3.391990202104e+15 ||r(i)||/||b|| 3.976286352982e-01 >>> 20 KSP preconditioned resid norm 1.135224344133e+16 true resid norm >>> 2.492450462789e+15 ||r(i)||/||b|| 2.921794041305e-01 >>> 21 KSP preconditioned resid norm 8.560056764067e+15 true resid norm >>> 1.658566536392e+15 ||r(i)||/||b|| 1.944267256456e-01 >>> 22 KSP preconditioned resid norm 6.591497757947e+15 true resid norm >>> 1.287448297256e+15 ||r(i)||/||b|| 1.509221073626e-01 >>> 23 KSP preconditioned resid norm 4.783676751639e+15 true resid norm >>> 9.069671731055e+14 ||r(i)||/||b|| 1.063199177517e-01 >>> 24 KSP preconditioned resid norm 3.029245298062e+15 true resid norm >>> 6.184462851213e+14 ||r(i)||/||b|| 7.249783687627e-02 >>> 25 KSP preconditioned resid norm 1.567581368038e+15 true resid norm >>> 2.751547089998e+14 ||r(i)||/||b|| 3.225522036225e-02 >>> 26 KSP preconditioned resid norm 1.174427256669e+15 true resid norm >>> 2.270458347820e+14 ||r(i)||/||b|| 2.661562093503e-02 >>> 27 KSP preconditioned resid norm 1.068689317098e+15 true resid norm >>> 2.026010487900e+14 ||r(i)||/||b|| 2.375006227624e-02 >>> 28 KSP preconditioned resid norm 1.006302654001e+15 true resid norm >>> 1.659657859887e+14 ||r(i)||/||b|| 1.945546568735e-02 >>> 29 KSP preconditioned resid norm 9.398219810967e+14 true resid norm >>> 1.788496139471e+14 ||r(i)||/||b|| 2.096578223407e-02 >>> 30 KSP preconditioned resid norm 8.922669862958e+14 true resid norm >>> 1.448103190511e+14 ||r(i)||/||b|| 1.697549996037e-02 >>> 31 KSP preconditioned resid norm 8.611553511040e+14 true resid norm >>> 1.569494235094e+14 ||r(i)||/||b|| 1.839851572749e-02 >>> 32 KSP preconditioned resid norm 8.275858647114e+14 true resid norm >>> 1.327021986732e+14 ||r(i)||/||b|| 1.555611632568e-02 >>> 33 KSP preconditioned resid norm 7.965262562283e+14 true resid norm >>> 1.578752878362e+14 ||r(i)||/||b|| 1.850705087847e-02 >>> 34 KSP preconditioned resid norm 7.584298711071e+14 true resid norm >>> 1.400338111929e+14 ||r(i)||/||b|| 1.641557018817e-02 >>> 35 KSP preconditioned resid norm 7.453216416858e+14 true resid norm >>> 1.168697043482e+14 ||r(i)||/||b|| 1.370014011799e-02 >>> 36 KSP preconditioned resid norm 7.552391370873e+14 true resid norm >>> 1.200616262811e+14 ||r(i)||/||b|| 1.407431559803e-02 >>> 37 KSP preconditioned resid norm 7.900785942536e+14 true resid norm >>> 1.248526226554e+14 ||r(i)||/||b|| 1.463594379756e-02 >>> 38 KSP preconditioned resid norm 8.292853068973e+14 true resid norm >>> 1.319608976445e+14 ||r(i)||/||b|| 1.546921674790e-02 >>> 39 KSP preconditioned resid norm 8.694401463969e+14 true resid norm >>> 1.375708051207e+14 ||r(i)||/||b|| 1.612684242516e-02 >>> 40 KSP preconditioned resid norm 9.345453545993e+14 true resid norm >>> 1.486706547594e+14 ||r(i)||/||b|| 1.742803075440e-02 >>> 41 KSP preconditioned resid norm 1.186246552213e+15 true resid norm >>> 1.878863391090e+14 ||r(i)||/||b|| 2.202511922492e-02 >>> 42 KSP preconditioned resid norm 2.140906233315e+15 true resid norm >>> 3.526342198473e+14 ||r(i)||/||b|| 4.133781504155e-02 >>> 43 KSP preconditioned resid norm 1.599648547908e+16 true resid norm >>> 2.542842342413e+15 ||r(i)||/||b|| 2.980866306056e-01 >>> 44 KSP preconditioned resid norm 1.201331553169e+15 true resid norm >>> 1.958292468332e+14 ||r(i)||/||b|| 2.295623263342e-02 >>> 45 KSP preconditioned resid norm 6.227596295745e+14 true resid norm >>> 9.990152835185e+13 ||r(i)||/||b|| 1.171103276127e-02 >>> 46 KSP preconditioned resid norm 4.228157632423e+14 true resid norm >>> 6.914757907633e+13 ||r(i)||/||b|| 8.105877630555e-03 >>> 47 KSP preconditioned resid norm 2.986102023248e+14 true resid norm >>> 4.782856914894e+13 ||r(i)||/||b|| 5.606740452011e-03 >>> 48 KSP preconditioned resid norm 1.831310965020e+14 true resid norm >>> 3.038059690813e+13 ||r(i)||/||b|| 3.561388615047e-03 >>> 49 KSP preconditioned resid norm 1.519167951348e+14 true resid norm >>> 2.657532215634e+13 ||r(i)||/||b|| 3.115312383591e-03 >>> 50 KSP preconditioned resid norm 1.784468495322e+14 true resid norm >>> 2.872094801120e+13 ||r(i)||/||b|| 3.366835008863e-03 >>> 51 KSP preconditioned resid norm 1.082269784640e+14 true resid norm >>> 1.734196241756e+13 ||r(i)||/||b|| 2.032924754679e-03 >>> 52 KSP preconditioned resid norm 7.941561433572e+13 true resid norm >>> 2.262252591710e+13 ||r(i)||/||b|| 2.651942833395e-03 >>> 53 KSP preconditioned resid norm 1.279466501710e+14 true resid norm >>> 2.100188573520e+13 ||r(i)||/||b|| 2.461962053545e-03 >>> 54 KSP preconditioned resid norm 1.148448766150e+14 true resid norm >>> 1.892568589516e+13 ||r(i)||/||b|| 2.218577945746e-03 >>> 55 KSP preconditioned resid norm 4.111681106774e+13 true resid norm >>> 1.052257112253e+13 ||r(i)||/||b|| 1.233516415432e-03 >>> 0 KSP preconditioned resid norm 1.892564880723e+19 true resid norm >>> 1.028289393047e+19 ||r(i)||/||b|| 7.840877673877e+01 >>> 1 KSP preconditioned resid norm 4.581010831586e+18 true resid norm >>> 1.519789346059e+18 ||r(i)||/||b|| 1.158864657468e+01 >>> 2 KSP preconditioned resid norm 1.874273717290e+18 true resid norm >>> 4.647134076709e+17 ||r(i)||/||b|| 3.543517036739e+00 >>> 3 KSP preconditioned resid norm 1.173044525535e+18 true resid norm >>> 2.550993039028e+17 ||r(i)||/||b|| 1.945174627025e+00 >>> 4 KSP preconditioned resid norm 8.299292116891e+17 true resid norm >>> 1.381054316961e+17 ||r(i)||/||b|| 1.053076889978e+00 >>> 5 KSP preconditioned resid norm 7.051404070011e+17 true resid norm >>> 1.287518923939e+17 ||r(i)||/||b|| 9.817545968746e-01 >>> 6 KSP preconditioned resid norm 5.863125538322e+17 true resid norm >>> 1.166548434193e+17 ||r(i)||/||b|| 8.895125861467e-01 >>> 7 KSP preconditioned resid norm 4.330808264262e+17 true resid norm >>> 8.612437830885e+16 ||r(i)||/||b|| 6.567127110566e-01 >>> 8 KSP preconditioned resid norm 2.702964509584e+17 true resid norm >>> 4.804107972053e+16 ||r(i)||/||b|| 3.663212243137e-01 >>> 9 KSP preconditioned resid norm 1.547354820777e+17 true resid norm >>> 2.879372380790e+16 ||r(i)||/||b|| 2.195569337580e-01 >>> 10 KSP preconditioned resid norm 9.913511217220e+16 true resid norm >>> 1.649311378149e+16 ||r(i)||/||b|| 1.257627361485e-01 >>> 11 KSP preconditioned resid norm 9.246165206306e+16 true resid norm >>> 1.559859930326e+16 ||r(i)||/||b|| 1.189419144530e-01 >>> 12 KSP preconditioned resid norm 9.248856361042e+16 true resid norm >>> 1.507587985844e+16 ||r(i)||/||b|| 1.149560917339e-01 >>> 13 KSP preconditioned resid norm 7.088634726331e+16 true resid norm >>> 1.150049603396e+16 ||r(i)||/||b|| 8.769319532120e-02 >>> 14 KSP preconditioned resid norm 2.014973749241e+17 true resid norm >>> 3.263663179758e+16 ||r(i)||/||b|| 2.488597464319e-01 >>> 15 KSP preconditioned resid norm 1.687600147354e+16 true resid norm >>> 4.438944439784e+15 ||r(i)||/||b|| 3.384768975431e-02 >>> 16 KSP preconditioned resid norm 1.189486995696e+16 true resid norm >>> 1.895011071195e+15 ||r(i)||/||b|| 1.444977464550e-02 >>> 17 KSP preconditioned resid norm 1.038277428537e+16 true resid norm >>> 2.030560404119e+15 ||r(i)||/||b|| 1.548336085714e-02 >>> 18 KSP preconditioned resid norm 9.585935517274e+15 true resid norm >>> 1.570427582757e+15 ||r(i)||/||b|| 1.197477155297e-02 >>> 19 KSP preconditioned resid norm 8.986106113745e+15 true resid norm >>> 1.453955802798e+15 ||r(i)||/||b|| 1.108665485617e-02 >>> 20 KSP preconditioned resid norm 8.480772996688e+15 true resid norm >>> 1.519414795393e+15 ||r(i)||/||b|| 1.158579056355e-02 >>> 21 KSP preconditioned resid norm 8.009707085363e+15 true resid norm >>> 1.276353172356e+15 ||r(i)||/||b|| 9.732405255546e-03 >>> 22 KSP preconditioned resid norm 7.603487959173e+15 true resid norm >>> 1.348218761629e+15 ||r(i)||/||b|| 1.028039232831e-02 >>> 23 KSP preconditioned resid norm 7.264676993739e+15 true resid norm >>> 1.145141030012e+15 ||r(i)||/||b|| 8.731890843545e-03 >>> 24 KSP preconditioned resid norm 6.442617339194e+15 true resid norm >>> 1.846348609557e+15 ||r(i)||/||b|| 1.407871528071e-02 >>> 25 KSP preconditioned resid norm 5.874716908283e+15 true resid norm >>> 9.094105988409e+14 ||r(i)||/||b|| 6.934407093038e-03 >>> 26 KSP preconditioned resid norm 5.974445043424e+15 true resid norm >>> 9.497988242860e+14 ||r(i)||/||b|| 7.242374030480e-03 >>> 27 KSP preconditioned resid norm 6.328543343408e+15 true resid norm >>> 9.841709349231e+14 ||r(i)||/||b|| 7.504467091753e-03 >>> 28 KSP preconditioned resid norm 1.045930080864e+16 true resid norm >>> 1.637386301334e+15 ||r(i)||/||b|| 1.248534292045e-02 >>> 29 KSP preconditioned resid norm 1.763968686067e+16 true resid norm >>> 3.035143897342e+15 ||r(i)||/||b|| 2.314347710148e-02 >>> 30 KSP preconditioned resid norm 3.994992144337e+15 true resid norm >>> 6.366779249631e+14 ||r(i)||/||b|| 4.854775086713e-03 >>> 31 KSP preconditioned resid norm 2.497471912676e+15 true resid norm >>> 4.171284268487e+14 ||r(i)||/||b|| 3.180673642395e-03 >>> 32 KSP preconditioned resid norm 1.774822233295e+15 true resid norm >>> 3.030553568855e+14 ||r(i)||/||b|| 2.310847508319e-03 >>> 33 KSP preconditioned resid norm 1.554413264158e+15 true resid norm >>> 2.490797343960e+14 ||r(i)||/||b|| 1.899274408204e-03 >>> 34 KSP preconditioned resid norm 1.363683273075e+15 true resid norm >>> 2.337415683510e+14 ||r(i)||/||b|| 1.782318340667e-03 >>> 35 KSP preconditioned resid norm 1.216181955878e+15 true resid norm >>> 1.941708948493e+14 ||r(i)||/||b|| 1.480585372791e-03 >>> 36 KSP preconditioned resid norm 1.106171392348e+15 true resid norm >>> 1.781889599099e+14 ||r(i)||/||b|| 1.358720460346e-03 >>> 37 KSP preconditioned resid norm 9.985678909493e+14 true resid norm >>> 1.843602345870e+14 ||r(i)||/||b|| 1.405777456326e-03 >>> 38 KSP preconditioned resid norm 9.306373857300e+14 true resid norm >>> 1.483324768155e+14 ||r(i)||/||b|| 1.131059810243e-03 >>> 39 KSP preconditioned resid norm 8.621292024153e+14 true resid norm >>> 1.776747195898e+14 ||r(i)||/||b|| 1.354799292363e-03 >>> 0 KSP preconditioned resid norm 6.470804796325e+18 true resid norm >>> 3.511545401133e+18 ||r(i)||/||b|| 1.662319590124e+02 >>> 1 KSP preconditioned resid norm 1.571603694611e+18 true resid norm >>> 5.187449890798e+17 ||r(i)||/||b|| 2.455670820454e+01 >>> 2 KSP preconditioned resid norm 6.468415902839e+17 true resid norm >>> 1.566954416728e+17 ||r(i)||/||b|| 7.417756930949e+00 >>> 3 KSP preconditioned resid norm 4.084120869852e+17 true resid norm >>> 8.688409741290e+16 ||r(i)||/||b|| 4.112979349580e+00 >>> 4 KSP preconditioned resid norm 2.920961241699e+17 true resid norm >>> 4.832079671037e+16 ||r(i)||/||b|| 2.287443213923e+00 >>> 5 KSP preconditioned resid norm 2.515909976499e+17 true resid norm >>> 4.435653262803e+16 ||r(i)||/||b|| 2.099780145624e+00 >>> 6 KSP preconditioned resid norm 2.133092401708e+17 true resid norm >>> 4.355815215083e+16 ||r(i)||/||b|| 2.061985859746e+00 >>> 7 KSP preconditioned resid norm 1.601275153782e+17 true resid norm >>> 2.999237788966e+16 ||r(i)||/||b|| 1.419799877977e+00 >>> 8 KSP preconditioned resid norm 1.019875018318e+17 true resid norm >>> 1.811096161004e+16 ||r(i)||/||b|| 8.573491964719e-01 >>> 9 KSP preconditioned resid norm 5.552203589730e+16 true resid norm >>> 9.849208071572e+15 ||r(i)||/||b|| 4.662486072173e-01 >>> 10 KSP preconditioned resid norm 2.614195079413e+16 true resid norm >>> 4.400774101946e+15 ||r(i)||/||b|| 2.083268807806e-01 >>> 11 KSP preconditioned resid norm 2.230316715231e+16 true resid norm >>> 4.001821966241e+15 ||r(i)||/||b|| 1.894410093210e-01 >>> 12 KSP preconditioned resid norm 2.202763899809e+16 true resid norm >>> 3.668851069689e+15 ||r(i)||/||b|| 1.736786032846e-01 >>> 13 KSP preconditioned resid norm 2.171352861704e+16 true resid norm >>> 3.746836567903e+15 ||r(i)||/||b|| 1.773703346056e-01 >>> 14 KSP preconditioned resid norm 2.756407833586e+16 true resid norm >>> 4.737724354756e+15 ||r(i)||/||b|| 2.242776643292e-01 >>> 15 KSP preconditioned resid norm 1.909305469788e+16 true resid norm >>> 7.268311827741e+15 ||r(i)||/||b|| 3.440723601207e-01 >>> 16 KSP preconditioned resid norm 8.249997128004e+15 true resid norm >>> 1.447767843213e+15 ||r(i)||/||b|| 6.853543305884e-02 >>> 17 KSP preconditioned resid norm 5.434797146758e+15 true resid norm >>> 1.776552830818e+15 ||r(i)||/||b|| 8.409968364943e-02 >>> 18 KSP preconditioned resid norm 4.166874856572e+15 true resid norm >>> 7.323760066865e+14 ||r(i)||/||b|| 3.466972071212e-02 >>> 19 KSP preconditioned resid norm 3.463504540881e+15 true resid norm >>> 7.653814460137e+14 ||r(i)||/||b|| 3.623215497131e-02 >>> 20 KSP preconditioned resid norm 3.008102754263e+15 true resid norm >>> 5.633011056242e+14 ||r(i)||/||b|| 2.666593639130e-02 >>> 21 KSP preconditioned resid norm 2.629337883031e+15 true resid norm >>> 5.301916607139e+14 ||r(i)||/||b|| 2.509857864406e-02 >>> 22 KSP preconditioned resid norm 2.387457188115e+15 true resid norm >>> 3.879435274146e+14 ||r(i)||/||b|| 1.836473836491e-02 >>> 23 KSP preconditioned resid norm 2.223797660329e+15 true resid norm >>> 3.481194862204e+14 ||r(i)||/||b|| 1.647952042601e-02 >>> 24 KSP preconditioned resid norm 2.082583007435e+15 true resid norm >>> 3.393016280495e+14 ||r(i)||/||b|| 1.606209457197e-02 >>> 25 KSP preconditioned resid norm 1.971105202361e+15 true resid norm >>> 3.085385809365e+14 ||r(i)||/||b|| 1.460581222257e-02 >>> 26 KSP preconditioned resid norm 1.916167824316e+15 true resid norm >>> 3.157123008884e+14 ||r(i)||/||b|| 1.494540672721e-02 >>> 27 KSP preconditioned resid norm 1.825444848282e+15 true resid norm >>> 4.173235963436e+14 ||r(i)||/||b|| 1.975555233884e-02 >>> 28 KSP preconditioned resid norm 1.787001647353e+15 true resid norm >>> 2.787303202952e+14 ||r(i)||/||b|| 1.319472821393e-02 >>> 29 KSP preconditioned resid norm 1.896717764390e+15 true resid norm >>> 3.068021792990e+14 ||r(i)||/||b|| 1.452361324382e-02 >>> 30 KSP preconditioned resid norm 2.229137880818e+15 true resid norm >>> 3.494609542895e+14 ||r(i)||/||b|| 1.654302376702e-02 >>> 31 KSP preconditioned resid norm 3.112887805330e+15 true resid norm >>> 5.056849908847e+14 ||r(i)||/||b|| 2.393846499915e-02 >>> 32 KSP preconditioned resid norm 2.024054955579e+16 true resid norm >>> 3.265471021005e+15 ||r(i)||/||b|| 1.545831202253e-01 >>> 33 KSP preconditioned resid norm 2.565517746075e+15 true resid norm >>> 4.297071200158e+14 ||r(i)||/||b|| 2.034177212653e-02 >>> 34 KSP preconditioned resid norm 1.151189963639e+15 true resid norm >>> 1.893961732619e+14 ||r(i)||/||b|| 8.965766724994e-03 >>> 35 KSP preconditioned resid norm 7.095661584181e+14 true resid norm >>> 1.215463627012e+14 ||r(i)||/||b|| 5.753845579254e-03 >>> 36 KSP preconditioned resid norm 5.356379112526e+14 true resid norm >>> 8.697359204859e+13 ||r(i)||/||b|| 4.117215908392e-03 >>> 37 KSP preconditioned resid norm 4.260660574229e+14 true resid norm >>> 9.044225863386e+13 ||r(i)||/||b|| 4.281418040435e-03 >>> 38 KSP preconditioned resid norm 3.822193347318e+14 true resid norm >>> 6.067358042990e+13 ||r(i)||/||b|| 2.872207812522e-03 >>> 39 KSP preconditioned resid norm 3.556139762092e+14 true resid norm >>> 7.037316076443e+13 ||r(i)||/||b|| 3.331373238687e-03 >>> 40 KSP preconditioned resid norm 3.319733183305e+14 true resid norm >>> 5.228450610578e+13 ||r(i)||/||b|| 2.475080024071e-03 >>> 41 KSP preconditioned resid norm 3.093370284781e+14 true resid norm >>> 5.468964654818e+13 ||r(i)||/||b|| 2.588936221775e-03 >>> 42 KSP preconditioned resid norm 2.908716641400e+14 true resid norm >>> 4.719126609844e+13 ||r(i)||/||b|| 2.233972714489e-03 >>> 43 KSP preconditioned resid norm 2.664499581099e+14 true resid norm >>> 6.079882213095e+13 ||r(i)||/||b|| 2.878136590578e-03 >>> 44 KSP preconditioned resid norm 2.513459163383e+14 true resid norm >>> 3.896503433878e+13 ||r(i)||/||b|| 1.844553679708e-03 >>> 45 KSP preconditioned resid norm 2.506735910619e+14 true resid norm >>> 4.872960867423e+13 ||r(i)||/||b|| 2.306795836731e-03 >>> 46 KSP preconditioned resid norm 2.610309833791e+14 true resid norm >>> 4.046617714344e+13 ||r(i)||/||b|| 1.915615813519e-03 >>> 47 KSP preconditioned resid norm 3.265892246735e+14 true resid norm >>> 6.092635527969e+13 ||r(i)||/||b|| 2.884173842765e-03 >>> 48 KSP preconditioned resid norm 6.619675585826e+14 true resid norm >>> 1.029872445074e+14 ||r(i)||/||b|| 4.875281237212e-03 >>> 49 KSP preconditioned resid norm 6.614927581836e+14 true resid norm >>> 1.173864813817e+14 ||r(i)||/||b|| 5.556922247214e-03 >>> 50 KSP preconditioned resid norm 2.572515469657e+14 true resid norm >>> 4.031945474821e+13 ||r(i)||/||b|| 1.908670167541e-03 >>> 51 KSP preconditioned resid norm 1.432897312628e+14 true resid norm >>> 2.304000171834e+13 ||r(i)||/||b|| 1.090683497942e-03 >>> 52 KSP preconditioned resid norm 8.898633954299e+13 true resid norm >>> 1.420071536381e+13 ||r(i)||/||b|| 6.722432617682e-04 >>> 0 KSP preconditioned resid norm 6.694445614266e+18 true resid norm >>> 3.650633549648e+18 ||r(i)||/||b|| 5.229628447406e+00 >>> 1 KSP preconditioned resid norm 1.597624793291e+18 true resid norm >>> 5.366629107368e+17 ||r(i)||/||b|| 7.687837156178e-01 >>> 2 KSP preconditioned resid norm 6.300333922334e+17 true resid norm >>> 1.705382687722e+17 ||r(i)||/||b|| 2.443005493741e-01 >>> 3 KSP preconditioned resid norm 3.642099405484e+17 true resid norm >>> 9.482515314454e+16 ||r(i)||/||b|| 1.358395225569e-01 >>> 4 KSP preconditioned resid norm 2.247296120507e+17 true resid norm >>> 3.802694326323e+16 ||r(i)||/||b|| 5.447459504021e-02 >>> 5 KSP preconditioned resid norm 1.557575416788e+17 true resid norm >>> 3.828791753195e+16 ||r(i)||/||b|| 5.484844753491e-02 >>> 6 KSP preconditioned resid norm 1.004780833173e+17 true resid norm >>> 1.699813376802e+16 ||r(i)||/||b|| 2.435027309565e-02 >>> 7 KSP preconditioned resid norm 7.104370054381e+16 true resid norm >>> 1.472894912543e+16 ||r(i)||/||b|| 2.109960649273e-02 >>> 8 KSP preconditioned resid norm 5.767747590200e+16 true resid norm >>> 9.399551072817e+15 ||r(i)||/||b|| 1.346510380041e-02 >>> 9 KSP preconditioned resid norm 4.524165642090e+16 true resid norm >>> 1.246569325329e+16 ||r(i)||/||b|| 1.785743300921e-02 >>> 10 KSP preconditioned resid norm 3.195588487377e+16 true resid norm >>> 5.031148957130e+15 ||r(i)||/||b|| 7.207253029241e-03 >>> 11 KSP preconditioned resid norm 2.437445149875e+16 true resid norm >>> 4.973530315557e+15 ||r(i)||/||b|| 7.124712811776e-03 >>> 12 KSP preconditioned resid norm 1.257635618330e+16 true resid norm >>> 1.885290715052e+15 ||r(i)||/||b|| 2.700728468356e-03 >>> 13 KSP preconditioned resid norm 2.495249888250e+15 true resid norm >>> 4.460762730080e+14 ||r(i)||/||b|| 6.390159777228e-04 >>> 14 KSP preconditioned resid norm 7.692041830184e+14 true resid norm >>> 1.319158639688e+14 ||r(i)||/||b|| 1.889729400372e-04 >>> 15 KSP preconditioned resid norm 5.963386353631e+14 true resid norm >>> 1.441107752347e+14 ||r(i)||/||b|| 2.064424707371e-04 >>> 16 KSP preconditioned resid norm 3.269005832710e+14 true resid norm >>> 5.055770257989e+13 ||r(i)||/||b|| 7.242523689424e-05 >>> 17 KSP preconditioned resid norm 1.442136448583e+14 true resid norm >>> 7.615238793008e+13 ||r(i)||/||b|| 1.090902959284e-04 >>> 18 KSP preconditioned resid norm 5.418830608399e+13 true resid norm >>> 8.650648142713e+12 ||r(i)||/||b|| 1.239228068235e-05 >>> 19 KSP preconditioned resid norm 3.534476706711e+13 true resid norm >>> 1.235947718863e+13 ||r(i)||/||b|| 1.770527570673e-05 >>> 0 KSP preconditioned resid norm 4.827689913975e+18 true resid norm >>> 2.628955418091e+18 ||r(i)||/||b|| 2.387179080674e+01 >>> 1 KSP preconditioned resid norm 1.161014015330e+18 true resid norm >>> 3.885001078629e+17 ||r(i)||/||b|| 3.527710374804e+00 >>> 2 KSP preconditioned resid norm 4.696713579932e+17 true resid norm >>> 1.228966476828e+17 ||r(i)||/||b|| 1.115942493412e+00 >>> 3 KSP preconditioned resid norm 2.869928353163e+17 true resid norm >>> 6.759586363313e+16 ||r(i)||/||b|| 6.137929555390e-01 >>> 4 KSP preconditioned resid norm 1.966510821692e+17 true resid norm >>> 3.310379360335e+16 ||r(i)||/||b|| 3.005934716010e-01 >>> 5 KSP preconditioned resid norm 1.592497528073e+17 true resid norm >>> 3.239157026660e+16 ||r(i)||/||b|| 2.941262464874e-01 >>> 6 KSP preconditioned resid norm 1.230094082012e+17 true resid norm >>> 2.280559390645e+16 ||r(i)||/||b|| 2.070823883934e-01 >>> 7 KSP preconditioned resid norm 8.598887604154e+16 true resid norm >>> 1.938013722659e+16 ||r(i)||/||b|| 1.759781008439e-01 >>> 8 KSP preconditioned resid norm 5.549487092651e+16 true resid norm >>> 9.386421691835e+15 ||r(i)||/||b|| 8.523183524123e-02 >>> 9 KSP preconditioned resid norm 4.475548308156e+16 true resid norm >>> 7.579602548804e+15 ||r(i)||/||b|| 6.882531563606e-02 >>> 10 KSP preconditioned resid norm 4.379582896462e+16 true resid norm >>> 6.844431748768e+15 ||r(i)||/||b|| 6.214971991278e-02 >>> 11 KSP preconditioned resid norm 4.081432560058e+16 true resid norm >>> 7.719907534397e+15 ||r(i)||/||b|| 7.009933163578e-02 >>> 12 KSP preconditioned resid norm 4.084955665921e+16 true resid norm >>> 6.376903215279e+15 ||r(i)||/||b|| 5.790440511177e-02 >>> 13 KSP preconditioned resid norm 7.513638228384e+16 true resid norm >>> 1.130789675242e+16 ||r(i)||/||b|| 1.026794687656e-01 >>> 14 KSP preconditioned resid norm 2.195838265499e+15 true resid norm >>> 3.518796225338e+14 ||r(i)||/||b|| 3.195184171050e-03 >>> 15 KSP preconditioned resid norm 2.237408970122e+15 true resid norm >>> 3.676139977919e+14 ||r(i)||/||b|| 3.338057538948e-03 >>> 16 KSP preconditioned resid norm 1.675719433917e+15 true resid norm >>> 3.658174858611e+14 ||r(i)||/||b|| 3.321744612262e-03 >>> 17 KSP preconditioned resid norm 1.403876583239e+15 true resid norm >>> 2.164126267214e+14 ||r(i)||/||b|| 1.965098729891e-03 >>> 18 KSP preconditioned resid norm 1.133854982860e+15 true resid norm >>> 2.692061619571e+14 ||r(i)||/||b|| 2.444481613457e-03 >>> 19 KSP preconditioned resid norm 9.034784290513e+14 true resid norm >>> 1.408322491509e+14 ||r(i)||/||b|| 1.278803728446e-03 >>> 20 KSP preconditioned resid norm 7.964711449547e+14 true resid norm >>> 1.641000281547e+14 ||r(i)||/||b|| 1.490082911460e-03 >>> 21 KSP preconditioned resid norm 7.094597664828e+14 true resid norm >>> 1.088610845746e+14 ||r(i)||/||b|| 9.884949056480e-04 >>> 22 KSP preconditioned resid norm 6.287860765855e+14 true resid norm >>> 1.439087926971e+14 ||r(i)||/||b|| 1.306739768531e-03 >>> 23 KSP preconditioned resid norm 5.617871594119e+14 true resid norm >>> 8.640965180691e+13 ||r(i)||/||b|| 7.846284183530e-04 >>> 24 KSP preconditioned resid norm 4.992037484390e+14 true resid norm >>> 1.232499062518e+14 ||r(i)||/||b|| 1.119150198876e-03 >>> 25 KSP preconditioned resid norm 3.174300847299e+14 true resid norm >>> 1.199443314096e+14 ||r(i)||/||b|| 1.089134478340e-03 >>> 26 KSP preconditioned resid norm 2.311461716948e+14 true resid norm >>> 3.806900580053e+13 ||r(i)||/||b|| 3.456792520850e-04 >>> 27 KSP preconditioned resid norm 2.486080516907e+14 true resid norm >>> 3.795889860412e+13 ||r(i)||/||b|| 3.446794420688e-04 >>> 28 KSP preconditioned resid norm 3.709940932117e+14 true resid norm >>> 6.158317563060e+13 ||r(i)||/||b|| 5.591957458661e-04 >>> 29 KSP preconditioned resid norm 5.526902041486e+14 true resid norm >>> 8.543527855752e+13 ||r(i)||/||b|| 7.757807847199e-04 >>> 30 KSP preconditioned resid norm 1.980272137271e+15 true resid norm >>> 3.202004161054e+14 ||r(i)||/||b|| 2.907526425475e-03 >>> 31 KSP preconditioned resid norm 4.796148387368e+14 true resid norm >>> 7.312671161483e+13 ||r(i)||/||b|| 6.640148973394e-04 >>> 32 KSP preconditioned resid norm 1.482366487345e+14 true resid norm >>> 3.206675205939e+13 ||r(i)||/||b|| 2.911767889807e-04 >>> 33 KSP preconditioned resid norm 7.403721045502e+13 true resid norm >>> 1.122274917558e+13 ||r(i)||/||b|| 1.019063004083e-04 >>> 34 KSP preconditioned resid norm 3.421203410880e+13 true resid norm >>> 9.812523407473e+12 ||r(i)||/||b|| 8.910098073838e-05 >>> 35 KSP preconditioned resid norm 1.809542403951e+13 true resid norm >>> 2.774316021276e+12 ||r(i)||/||b|| 2.519171349804e-05 >>> 36 KSP preconditioned resid norm 1.070863017107e+13 true resid norm >>> 1.674563212179e+12 ||r(i)||/||b|| 1.520559170335e-05 >>> 37 KSP preconditioned resid norm 6.984609262190e+12 true resid norm >>> 1.277736482286e+12 ||r(i)||/||b|| 1.160227282721e-05 >>> 0 KSP preconditioned resid norm 9.569153877844e+15 true resid norm >>> 5.893805322157e+15 ||r(i)||/||b|| 1.119437717455e-01 >>> 1 KSP preconditioned resid norm 4.594033327917e+15 true resid norm >>> 8.149731925277e+14 ||r(i)||/||b|| 1.547916296116e-02 >>> 2 KSP preconditioned resid norm 3.788286152976e+15 true resid norm >>> 6.151807578016e+14 ||r(i)||/||b|| 1.168441279773e-02 >>> 3 KSP preconditioned resid norm 3.712195492568e+15 true resid norm >>> 6.206672982073e+14 ||r(i)||/||b|| 1.178862119847e-02 >>> 4 KSP preconditioned resid norm 3.886667577513e+15 true resid norm >>> 6.330116967950e+14 ||r(i)||/||b|| 1.202308407301e-02 >>> 5 KSP preconditioned resid norm 4.317240607103e+15 true resid norm >>> 6.816590875948e+14 ||r(i)||/||b|| 1.294706647725e-02 >>> 6 KSP preconditioned resid norm 5.249548366505e+15 true resid norm >>> 7.833242603614e+14 ||r(i)||/||b|| 1.487804014750e-02 >>> 7 KSP preconditioned resid norm 6.521511569741e+15 true resid norm >>> 1.182301409963e+15 ||r(i)||/||b|| 2.245599777001e-02 >>> 8 KSP preconditioned resid norm 7.063327681244e+15 true resid norm >>> 1.110968116855e+15 ||r(i)||/||b|| 2.110113152571e-02 >>> 9 KSP preconditioned resid norm 7.059112088082e+15 true resid norm >>> 1.341815038025e+15 ||r(i)||/||b|| 2.548571392011e-02 >>> 10 KSP preconditioned resid norm 6.417041101993e+15 true resid norm >>> 1.020408664201e+15 ||r(i)||/||b|| 1.938109393654e-02 >>> 11 KSP preconditioned resid norm 4.850416656929e+15 true resid norm >>> 8.501719778107e+14 ||r(i)||/||b|| 1.614770977770e-02 >>> 12 KSP preconditioned resid norm 2.420686956668e+15 true resid norm >>> 3.735196627993e+14 ||r(i)||/||b|| 7.094431795646e-03 >>> 13 KSP preconditioned resid norm 1.348084459782e+15 true resid norm >>> 2.753333760274e+14 ||r(i)||/||b|| 5.229534216894e-03 >>> 14 KSP preconditioned resid norm 1.225904445555e+15 true resid norm >>> 2.239730314894e+14 ||r(i)||/||b|| 4.254023426925e-03 >>> 15 KSP preconditioned resid norm 1.156415256406e+15 true resid norm >>> 2.064224387096e+14 ||r(i)||/||b|| 3.920676897010e-03 >>> 16 KSP preconditioned resid norm 1.062658328422e+15 true resid norm >>> 1.696405409630e+14 ||r(i)||/||b|| 3.222061292889e-03 >>> 17 KSP preconditioned resid norm 9.163863908341e+14 true resid norm >>> 1.701529770139e+14 ||r(i)||/||b|| 3.231794227925e-03 >>> 18 KSP preconditioned resid norm 8.234055963962e+14 true resid norm >>> 1.322040551676e+14 ||r(i)||/||b|| 2.511012795057e-03 >>> 19 KSP preconditioned resid norm 7.097124944641e+14 true resid norm >>> 1.155726966735e+14 ||r(i)||/||b|| 2.195125707289e-03 >>> 20 KSP preconditioned resid norm 6.309033027960e+14 true resid norm >>> 1.023450925395e+14 ||r(i)||/||b|| 1.943887701115e-03 >>> 21 KSP preconditioned resid norm 5.667708572251e+14 true resid norm >>> 8.941024380489e+13 ||r(i)||/||b|| 1.698210133710e-03 >>> 22 KSP preconditioned resid norm 5.205793964231e+14 true resid norm >>> 8.689007901528e+13 ||r(i)||/||b|| 1.650343477697e-03 >>> 23 KSP preconditioned resid norm 4.890134322830e+14 true resid norm >>> 7.566844928998e+13 ||r(i)||/||b|| 1.437205871699e-03 >>> 24 KSP preconditioned resid norm 4.617102016960e+14 true resid norm >>> 8.638231658778e+13 ||r(i)||/||b|| 1.640699310952e-03 >>> 25 KSP preconditioned resid norm 4.386224186730e+14 true resid norm >>> 6.736465167645e+13 ||r(i)||/||b|| 1.279488001179e-03 >>> 26 KSP preconditioned resid norm 4.119635069909e+14 true resid norm >>> 8.360600093542e+13 ||r(i)||/||b|| 1.587967463073e-03 >>> 27 KSP preconditioned resid norm 3.764202708342e+14 true resid norm >>> 5.766864517117e+13 ||r(i)||/||b|| 1.095327262956e-03 >>> 28 KSP preconditioned resid norm 3.345193309034e+14 true resid norm >>> 7.248849449141e+13 ||r(i)||/||b|| 1.376807518738e-03 >>> 0 KSP preconditioned resid norm 9.592548277384e+15 true resid norm >>> 5.933564471549e+15 ||r(i)||/||b|| 5.704186299543e-01 >>> 1 KSP preconditioned resid norm 4.627892277467e+15 true resid norm >>> 8.123960687021e+14 ||r(i)||/||b|| 7.809906755228e-02 >>> 2 KSP preconditioned resid norm 3.843955835291e+15 true resid norm >>> 6.217930929753e+14 ||r(i)||/||b|| 5.977559794129e-02 >>> 3 KSP preconditioned resid norm 3.802651796154e+15 true resid norm >>> 6.334637368548e+14 ||r(i)||/||b|| 6.089754626161e-02 >>> 4 KSP preconditioned resid norm 4.025480937197e+15 true resid norm >>> 6.491593110279e+14 ||r(i)||/||b|| 6.240642814182e-02 >>> 5 KSP preconditioned resid norm 4.519579789539e+15 true resid norm >>> 7.102235297384e+14 ||r(i)||/||b|| 6.827678956506e-02 >>> 6 KSP preconditioned resid norm 5.567308133052e+15 true resid norm >>> 8.274501588234e+14 ||r(i)||/||b|| 7.954628085944e-02 >>> 7 KSP preconditioned resid norm 7.053789984861e+15 true resid norm >>> 1.260366188593e+15 ||r(i)||/||b|| 1.211643284547e-01 >>> 8 KSP preconditioned resid norm 7.835759896085e+15 true resid norm >>> 1.231567591968e+15 ||r(i)||/||b|| 1.183957976482e-01 >>> 9 KSP preconditioned resid norm 8.126559485840e+15 true resid norm >>> 1.514279927927e+15 ||r(i)||/||b|| 1.455741293445e-01 >>> 10 KSP preconditioned resid norm 7.865908148994e+15 true resid norm >>> 1.247217430777e+15 ||r(i)||/||b|| 1.199002827947e-01 >>> 11 KSP preconditioned resid norm 6.705237228445e+15 true resid norm >>> 1.156150868849e+15 ||r(i)||/||b|| 1.111456693176e-01 >>> 12 KSP preconditioned resid norm 3.837356917105e+15 true resid norm >>> 5.892678306289e+14 ||r(i)||/||b|| 5.664880700887e-02 >>> 13 KSP preconditioned resid norm 1.588477046031e+15 true resid norm >>> 3.087098247775e+14 ||r(i)||/||b|| 2.967758017080e-02 >>> 14 KSP preconditioned resid norm 1.304101915863e+15 true resid norm >>> 2.634312731694e+14 ||r(i)||/||b|| 2.532476164182e-02 >>> 15 KSP preconditioned resid norm 1.188980488344e+15 true resid norm >>> 2.212236776838e+14 ||r(i)||/||b|| 2.126716710384e-02 >>> 16 KSP preconditioned resid norm 1.053945473260e+15 true resid norm >>> 1.687417173497e+14 ||r(i)||/||b|| 1.622185444993e-02 >>> 17 KSP preconditioned resid norm 8.630322699779e+14 true resid norm >>> 2.032342536888e+14 ||r(i)||/||b|| 1.953776774564e-02 >>> 18 KSP preconditioned resid norm 7.497367164260e+14 true resid norm >>> 1.258314731452e+14 ||r(i)||/||b|| 1.209671132095e-02 >>> 19 KSP preconditioned resid norm 6.482413928213e+14 true resid norm >>> 1.007668116028e+14 ||r(i)||/||b|| 9.687139474922e-03 >>> 20 KSP preconditioned resid norm 5.423124085977e+14 true resid norm >>> 1.262959277696e+14 ||r(i)||/||b|| 1.214136130695e-02 >>> 21 KSP preconditioned resid norm 4.512195524384e+14 true resid norm >>> 7.029203398299e+13 ||r(i)||/||b|| 6.757470305334e-03 >>> 22 KSP preconditioned resid norm 3.576286880164e+14 true resid norm >>> 9.097241124393e+13 ||r(i)||/||b|| 8.745562373884e-03 >>> 23 KSP preconditioned resid norm 2.908403868181e+14 true resid norm >>> 4.482717971847e+13 ||r(i)||/||b|| 4.309426241567e-03 >>> 24 KSP preconditioned resid norm 2.299942905294e+14 true resid norm >>> 6.485244464292e+13 ||r(i)||/||b|| 6.234539592478e-03 >>> 25 KSP preconditioned resid norm 1.957293334591e+14 true resid norm >>> 3.000864324033e+13 ||r(i)||/||b|| 2.884857701642e-03 >>> 26 KSP preconditioned resid norm 1.685971736537e+14 true resid norm >>> 4.568513443333e+13 ||r(i)||/||b|| 4.391905054321e-03 >>> 27 KSP preconditioned resid norm 1.478986486404e+14 true resid norm >>> 2.270701271232e+13 ||r(i)||/||b|| 2.182921099758e-03 >>> 28 KSP preconditioned resid norm 1.388836935911e+14 true resid norm >>> 2.435562734780e+13 ||r(i)||/||b|| 2.341409392284e-03 >>> 29 KSP preconditioned resid norm 1.300875297573e+14 true resid norm >>> 2.023811842122e+13 ||r(i)||/||b|| 1.945575857149e-03 >>> 30 KSP preconditioned resid norm 1.230463362777e+14 true resid norm >>> 2.013991197226e+13 ||r(i)||/||b|| 1.936134856156e-03 >>> 31 KSP preconditioned resid norm 1.148174878708e+14 true resid norm >>> 1.790715664590e+13 ||r(i)||/||b|| 1.721490650234e-03 >>> 32 KSP preconditioned resid norm 1.085020096185e+14 true resid norm >>> 1.697737976905e+13 ||r(i)||/||b|| 1.632107269502e-03 >>> 33 KSP preconditioned resid norm 1.008662380835e+14 true resid norm >>> 1.575408064147e+13 ||r(i)||/||b|| 1.514506354280e-03 >>> 34 KSP preconditioned resid norm 9.448474542578e+13 true resid norm >>> 1.450932025652e+13 ||r(i)||/||b|| 1.394842277686e-03 >>> 35 KSP preconditioned resid norm 8.955256800693e+13 true resid norm >>> 1.427784032077e+13 ||r(i)||/||b|| 1.372589133148e-03 >>> 36 KSP preconditioned resid norm 8.409700445745e+13 true resid norm >>> 1.287414873178e+13 ||r(i)||/||b|| 1.237646328210e-03 >>> 37 KSP preconditioned resid norm 7.999039854664e+13 true resid norm >>> 1.307106382082e+13 ||r(i)||/||b|| 1.256576607951e-03 >>> 38 KSP preconditioned resid norm 7.508174679474e+13 true resid norm >>> 1.150183263556e+13 ||r(i)||/||b|| 1.105719781996e-03 >>> 39 KSP preconditioned resid norm 7.047760131088e+13 true resid norm >>> 1.131577846513e+13 ||r(i)||/||b|| 1.087833608263e-03 >>> 40 KSP preconditioned resid norm 6.612188631103e+13 true resid norm >>> 1.012435424497e+13 ||r(i)||/||b|| 9.732969626060e-04 >>> 41 KSP preconditioned resid norm 6.096165352930e+13 true resid norm >>> 1.005634567468e+13 ||r(i)||/||b|| 9.667590113164e-04 >>> 42 KSP preconditioned resid norm 5.668115744306e+13 true resid norm >>> 8.662975702181e+12 ||r(i)||/||b|| 8.328084669947e-04 >>> 43 KSP preconditioned resid norm 5.243169528258e+13 true resid norm >>> 8.972837494107e+12 ||r(i)||/||b|| 8.625967906361e-04 >>> 0 KSP preconditioned resid norm 2.201027933913e+16 true resid norm >>> 1.374916449250e+16 ||r(i)||/||b|| 1.042532971848e-01 >>> 1 KSP preconditioned resid norm 1.033786235817e+16 true resid norm >>> 1.769244185634e+15 ||r(i)||/||b|| 1.341532716246e-02 >>> 2 KSP preconditioned resid norm 8.200152533928e+15 true resid norm >>> 1.329548327593e+15 ||r(i)||/||b|| 1.008132508660e-02 >>> 3 KSP preconditioned resid norm 7.725254307721e+15 true resid norm >>> 1.292264285665e+15 ||r(i)||/||b|| 9.798618140627e-03 >>> 4 KSP preconditioned resid norm 7.783851928031e+15 true resid norm >>> 1.241739377824e+15 ||r(i)||/||b|| 9.415512080968e-03 >>> 5 KSP preconditioned resid norm 8.247256237278e+15 true resid norm >>> 1.289875346607e+15 ||r(i)||/||b|| 9.780503965494e-03 >>> 6 KSP preconditioned resid norm 9.468674539247e+15 true resid norm >>> 1.409079612219e+15 ||r(i)||/||b|| 1.068437254131e-02 >>> 7 KSP preconditioned resid norm 1.106031993220e+16 true resid norm >>> 1.997396506013e+15 ||r(i)||/||b|| 1.514529640334e-02 >>> 8 KSP preconditioned resid norm 1.133839709036e+16 true resid norm >>> 1.772431554854e+15 ||r(i)||/||b|| 1.343949544924e-02 >>> 9 KSP preconditioned resid norm 1.055805824819e+16 true resid norm >>> 2.045498232098e+15 ||r(i)||/||b|| 1.551002864197e-02 >>> 10 KSP preconditioned resid norm 8.628048224174e+15 true resid norm >>> 1.368695682952e+15 ||r(i)||/||b|| 1.037816064155e-02 >>> 11 KSP preconditioned resid norm 5.442091241360e+15 true resid norm >>> 9.810024540837e+14 ||r(i)||/||b|| 7.438469475027e-03 >>> 12 KSP preconditioned resid norm 2.000681531075e+15 true resid norm >>> 3.042659606098e+14 ||r(i)||/||b|| 2.307102342980e-03 >>> 13 KSP preconditioned resid norm 8.105673271529e+14 true resid norm >>> 1.475101242619e+14 ||r(i)||/||b|| 1.118498278992e-03 >>> 0 KSP preconditioned resid norm 1.928660996668e+16 true resid norm >>> 1.203367101178e+16 ||r(i)||/||b|| 4.946564650623e-01 >>> 1 KSP preconditioned resid norm 9.233394049572e+15 true resid norm >>> 1.583660413052e+15 ||r(i)||/||b|| 6.509799553373e-02 >>> 2 KSP preconditioned resid norm 7.543213441610e+15 true resid norm >>> 1.216909563864e+15 ||r(i)||/||b|| 5.002232341004e-02 >>> 3 KSP preconditioned resid norm 7.341955893763e+15 true resid norm >>> 1.220114169499e+15 ||r(i)||/||b|| 5.015405203166e-02 >>> 4 KSP preconditioned resid norm 7.657405519141e+15 true resid norm >>> 1.212628294074e+15 ||r(i)||/||b|| 4.984633739730e-02 >>> 5 KSP preconditioned resid norm 8.417297612797e+15 true resid norm >>> 1.311888034770e+15 ||r(i)||/||b|| 5.392651146949e-02 >>> 6 KSP preconditioned resid norm 1.008979814671e+16 true resid norm >>> 1.496346817141e+15 ||r(i)||/||b|| 6.150888007071e-02 >>> 7 KSP preconditioned resid norm 1.235951387667e+16 true resid norm >>> 2.203847820374e+15 ||r(i)||/||b|| 9.059143891287e-02 >>> 8 KSP preconditioned resid norm 1.311708980126e+16 true resid norm >>> 2.061617788942e+15 ||r(i)||/||b|| 8.474492669688e-02 >>> 9 KSP preconditioned resid norm 1.289258338081e+16 true resid norm >>> 2.353735302483e+15 ||r(i)||/||b|| 9.675271854106e-02 >>> 10 KSP preconditioned resid norm 1.151089236912e+16 true resid norm >>> 1.820534806530e+15 ||r(i)||/||b|| 7.483496191971e-02 >>> 11 KSP preconditioned resid norm 8.373719062509e+15 true resid norm >>> 1.433022884373e+15 ||r(i)||/||b|| 5.890588446727e-02 >>> 12 KSP preconditioned resid norm 3.564533969408e+15 true resid norm >>> 5.439956922043e+14 ||r(i)||/||b|| 2.236150430334e-02 >>> 13 KSP preconditioned resid norm 1.257109874849e+15 true resid norm >>> 2.286041152149e+14 ||r(i)||/||b|| 9.397008063476e-03 >>> 14 KSP preconditioned resid norm 1.068785016666e+15 true resid norm >>> 1.971356784282e+14 ||r(i)||/||b|| 8.103465495567e-03 >>> 15 KSP preconditioned resid norm 9.963029863724e+14 true resid norm >>> 1.558476632901e+14 ||r(i)||/||b|| 6.406279026229e-03 >>> 16 KSP preconditioned resid norm 8.667425302785e+14 true resid norm >>> 2.560422780209e+14 ||r(i)||/||b|| 1.052488206037e-02 >>> 17 KSP preconditioned resid norm 7.240699142388e+14 true resid norm >>> 1.131314963782e+14 ||r(i)||/||b|| 4.650386904448e-03 >>> 18 KSP preconditioned resid norm 6.352047441583e+14 true resid norm >>> 1.637714957863e+14 ||r(i)||/||b|| 6.731996337968e-03 >>> 19 KSP preconditioned resid norm 5.532686184638e+14 true resid norm >>> 8.631039548890e+13 ||r(i)||/||b|| 3.547877874413e-03 >>> 20 KSP preconditioned resid norm 4.676701949378e+14 true resid norm >>> 1.246119427256e+14 ||r(i)||/||b|| 5.122302498784e-03 >>> 21 KSP preconditioned resid norm 4.001088470241e+14 true resid norm >>> 6.181513171397e+13 ||r(i)||/||b|| 2.540974779106e-03 >>> 22 KSP preconditioned resid norm 3.482451502111e+14 true resid norm >>> 6.477073783496e+13 ||r(i)||/||b|| 2.662468018741e-03 >>> 23 KSP preconditioned resid norm 3.041601141164e+14 true resid norm >>> 4.679581096972e+13 ||r(i)||/||b|| 1.923590100755e-03 >>> 24 KSP preconditioned resid norm 2.690997224286e+14 true resid norm >>> 4.294556821793e+13 ||r(i)||/||b|| 1.765321899192e-03 >>> 25 KSP preconditioned resid norm 2.482501027210e+14 true resid norm >>> 3.840071688811e+13 ||r(i)||/||b|| 1.578501095230e-03 >>> 26 KSP preconditioned resid norm 2.328482920528e+14 true resid norm >>> 3.868960801558e+13 ||r(i)||/||b|| 1.590376263145e-03 >>> 27 KSP preconditioned resid norm 2.171581409229e+14 true resid norm >>> 3.334869397249e+13 ||r(i)||/||b|| 1.370832479858e-03 >>> 28 KSP preconditioned resid norm 2.002313534549e+14 true resid norm >>> 3.534153915591e+13 ||r(i)||/||b|| 1.452750437635e-03 >>> 29 KSP preconditioned resid norm 1.813916789125e+14 true resid norm >>> 2.780025874135e+13 ||r(i)||/||b|| 1.142758323985e-03 >>> 30 KSP preconditioned resid norm 1.655865002619e+14 true resid norm >>> 2.941107681765e+13 ||r(i)||/||b|| 1.208972663292e-03 >>> 31 KSP preconditioned resid norm 1.477358437937e+14 true resid norm >>> 2.263549873151e+13 ||r(i)||/||b|| 9.304555340168e-04 >>> 32 KSP preconditioned resid norm 1.346388933582e+14 true resid norm >>> 2.267328899678e+13 ||r(i)||/||b|| 9.320089418685e-04 >>> 0 KSP preconditioned resid norm 6.130047183925e+16 true resid norm >>> 4.019546453762e+16 ||r(i)||/||b|| 5.748521837148e-02 >>> 1 KSP preconditioned resid norm 2.383257388870e+16 true resid norm >>> 4.333010621750e+15 ||r(i)||/||b|| 6.196820080636e-03 >>> 2 KSP preconditioned resid norm 1.380338515654e+16 true resid norm >>> 2.902899468751e+15 ||r(i)||/||b|| 4.151558186756e-03 >>> 3 KSP preconditioned resid norm 9.355621435823e+15 true resid norm >>> 1.914392486860e+15 ||r(i)||/||b|| 2.737852924995e-03 >>> 4 KSP preconditioned resid norm 6.905538449399e+15 true resid norm >>> 1.160515328667e+15 ||r(i)||/||b|| 1.659701607116e-03 >>> 5 KSP preconditioned resid norm 5.388961375964e+15 true resid norm >>> 9.343353155170e+14 ||r(i)||/||b|| 1.336232091419e-03 >>> 6 KSP preconditioned resid norm 4.457024362952e+15 true resid norm >>> 7.443375401358e+14 ||r(i)||/||b|| 1.064508310303e-03 >>> 7 KSP preconditioned resid norm 3.604925656449e+15 true resid norm >>> 7.177624609068e+14 ||r(i)||/||b|| 1.026502175773e-03 >>> 8 KSP preconditioned resid norm 2.616540301450e+15 true resid norm >>> 4.566819450076e+14 ||r(i)||/||b|| 6.531199884630e-04 >>> 9 KSP preconditioned resid norm 1.730734442406e+15 true resid norm >>> 3.416454932875e+14 ||r(i)||/||b|| 4.886015378397e-04 >>> 10 KSP preconditioned resid norm 9.605665765368e+14 true resid norm >>> 1.715807715453e+14 ||r(i)||/||b|| 2.453848520993e-04 >>> 11 KSP preconditioned resid norm 3.885455022454e+14 true resid norm >>> 7.526175493349e+13 ||r(i)||/||b|| 1.076349898463e-04 >>> 12 KSP preconditioned resid norm 1.085553513563e+14 true resid norm >>> 1.877760955419e+13 ||r(i)||/||b|| 2.685464636705e-05 >>> 13 KSP preconditioned resid norm 5.523836175310e+13 true resid norm >>> 1.483397872543e+13 ||r(i)||/||b|| 2.121469464673e-05 >>> 14 KSP preconditioned resid norm 4.329267915925e+13 true resid norm >>> 6.950523101530e+12 ||r(i)||/||b|| 9.940234374291e-06 >>> 0 KSP preconditioned resid norm 2.469234905564e+16 true resid norm >>> 1.581399294540e+16 ||r(i)||/||b|| 1.421323674244e-01 >>> 1 KSP preconditioned resid norm 1.094883401656e+16 true resid norm >>> 1.827560974935e+15 ||r(i)||/||b|| 1.642567875658e-02 >>> 2 KSP preconditioned resid norm 7.902898352613e+15 true resid norm >>> 1.331982197511e+15 ||r(i)||/||b|| 1.197153582609e-02 >>> 3 KSP preconditioned resid norm 6.757313480243e+15 true resid norm >>> 1.177585457449e+15 ||r(i)||/||b|| 1.058385503836e-02 >>> 4 KSP preconditioned resid norm 6.237102240639e+15 true resid norm >>> 9.859442147838e+14 ||r(i)||/||b|| 8.861429613598e-03 >>> 5 KSP preconditioned resid norm 6.056820105725e+15 true resid norm >>> 9.499166634556e+14 ||r(i)||/||b|| 8.537622642110e-03 >>> 6 KSP preconditioned resid norm 6.282037371920e+15 true resid norm >>> 9.440405134922e+14 ||r(i)||/||b|| 8.484809218675e-03 >>> 7 KSP preconditioned resid norm 6.512961686254e+15 true resid norm >>> 1.175538616906e+15 ||r(i)||/||b|| 1.056545852755e-02 >>> 8 KSP preconditioned resid norm 5.899315381141e+15 true resid norm >>> 9.360504236018e+14 ||r(i)||/||b|| 8.412996211297e-03 >>> 9 KSP preconditioned resid norm 4.860546577526e+15 true resid norm >>> 9.070630816401e+14 ||r(i)||/||b|| 8.152464949358e-03 >>> 10 KSP preconditioned resid norm 3.431569954082e+15 true resid norm >>> 5.519072654101e+14 ||r(i)||/||b|| 4.960409840975e-03 >>> 11 KSP preconditioned resid norm 1.788273694951e+15 true resid norm >>> 3.211455497861e+14 ||r(i)||/||b|| 2.886379008546e-03 >>> 12 KSP preconditioned resid norm 6.021242921522e+14 true resid norm >>> 9.258232372098e+13 ||r(i)||/||b|| 8.321076718289e-04 >>> 13 KSP preconditioned resid norm 3.678076314547e+14 true resid norm >>> 7.160952479278e+13 ||r(i)||/||b|| 6.436091962400e-04 >>> 14 KSP preconditioned resid norm 3.596149520932e+14 true resid norm >>> 5.543217762956e+13 ||r(i)||/||b|| 4.982110884443e-04 >>> 15 KSP preconditioned resid norm 1.443083471369e+14 true resid norm >>> 3.905973798110e+13 ||r(i)||/||b|| 3.510595362853e-04 >>> 16 KSP preconditioned resid norm 1.158730309561e+14 true resid norm >>> 1.821149714099e+13 ||r(i)||/||b|| 1.636805588525e-04 >>> 17 KSP preconditioned resid norm 9.042472136695e+13 true resid norm >>> 2.428568415416e+13 ||r(i)||/||b|| 2.182738916901e-04 >>> 18 KSP preconditioned resid norm 7.667332005406e+13 true resid norm >>> 1.178343148104e+13 ||r(i)||/||b|| 1.059066497984e-04 >>> 19 KSP preconditioned resid norm 6.435860750477e+13 true resid norm >>> 1.303667702134e+13 ||r(i)||/||b|| 1.171705194752e-04 >>> 20 KSP preconditioned resid norm 5.245023814970e+13 true resid norm >>> 8.751976411105e+12 ||r(i)||/||b|| 7.866066029293e-05 >>> 21 KSP preconditioned resid norm 4.161239918955e+13 true resid norm >>> 6.811593020631e+12 ||r(i)||/||b|| 6.122096078432e-05 >>> 22 KSP preconditioned resid norm 3.107777724805e+13 true resid norm >>> 6.092648679105e+12 ||r(i)||/||b|| 5.475926185348e-05 >>> 23 KSP preconditioned resid norm 2.312582950537e+13 true resid norm >>> 3.536350440396e+12 ||r(i)||/||b|| 3.178386773481e-05 >>> 24 KSP preconditioned resid norm 1.680324737468e+13 true resid norm >>> 4.383700522921e+12 ||r(i)||/||b|| 3.939964660118e-05 >>> 25 KSP preconditioned resid norm 1.300784122000e+13 true resid norm >>> 1.953230538144e+12 ||r(i)||/||b|| 1.755516658383e-05 >>> 26 KSP preconditioned resid norm 9.922666654442e+12 true resid norm >>> 3.054726768240e+12 ||r(i)||/||b|| 2.745514993611e-05 >>> 27 KSP preconditioned resid norm 7.226267347787e+12 true resid norm >>> 1.085334598547e+12 ||r(i)||/||b|| 9.754726492653e-06 >>> 0 SNES Function norm 0.000000000000e+00 >>> 2 TS dt 4e-12 time 5e-12 >>> 0 KSP preconditioned resid norm 2.220258026867e+19 true resid norm >>> 1.204245784292e+19 ||r(i)||/||b|| 2.302282866910e+02 >>> 1 KSP preconditioned resid norm 5.390122646802e+18 true resid norm >>> 1.775868988684e+18 ||r(i)||/||b|| 3.395114851018e+01 >>> 2 KSP preconditioned resid norm 2.218369826326e+18 true resid norm >>> 5.319781926844e+17 ||r(i)||/||b|| 1.017038460556e+01 >>> 3 KSP preconditioned resid norm 1.401713665358e+18 true resid norm >>> 2.963575488339e+17 ||r(i)||/||b|| 5.665777834223e+00 >>> 4 KSP preconditioned resid norm 1.004075064815e+18 true resid norm >>> 1.656255011022e+17 ||r(i)||/||b|| 3.166436274762e+00 >>> 5 KSP preconditioned resid norm 8.672862633339e+17 true resid norm >>> 1.510357773334e+17 ||r(i)||/||b|| 2.887509235913e+00 >>> 6 KSP preconditioned resid norm 7.389427240230e+17 true resid norm >>> 1.526197236053e+17 ||r(i)||/||b|| 2.917791196717e+00 >>> 7 KSP preconditioned resid norm 5.566238898025e+17 true resid norm >>> 1.017349876189e+17 ||r(i)||/||b|| 1.944974373301e+00 >>> 8 KSP preconditioned resid norm 3.566703993070e+17 true resid norm >>> 6.306898440820e+16 ||r(i)||/||b|| 1.205755869196e+00 >>> 9 KSP preconditioned resid norm 1.893399021800e+17 true resid norm >>> 3.268112455206e+16 ||r(i)||/||b|| 6.247993068277e-01 >>> 10 KSP preconditioned resid norm 6.895422631218e+16 true resid norm >>> 1.153042493210e+16 ||r(i)||/||b|| 2.204392169408e-01 >>> 11 KSP preconditioned resid norm 5.104209415340e+16 true resid norm >>> 9.274112790250e+15 ||r(i)||/||b|| 1.773029331827e-01 >>> 12 KSP preconditioned resid norm 4.991932910946e+16 true resid norm >>> 8.397724827599e+15 ||r(i)||/||b|| 1.605481060744e-01 >>> 13 KSP preconditioned resid norm 4.919289629176e+16 true resid norm >>> 8.425037065615e+15 ||r(i)||/||b|| 1.610702627509e-01 >>> 14 KSP preconditioned resid norm 5.687556145039e+16 true resid norm >>> 9.761935947260e+15 ||r(i)||/||b|| 1.866291596982e-01 >>> 15 KSP preconditioned resid norm 1.615122498031e+17 true resid norm >>> 4.345955527842e+16 ||r(i)||/||b|| 8.308618624718e-01 >>> 16 KSP preconditioned resid norm 4.126010760136e+16 true resid norm >>> 7.099883585094e+15 ||r(i)||/||b|| 1.357359149456e-01 >>> 17 KSP preconditioned resid norm 2.161057719247e+16 true resid norm >>> 5.853862804145e+15 ||r(i)||/||b|| 1.119144298865e-01 >>> 18 KSP preconditioned resid norm 1.411351808998e+16 true resid norm >>> 2.426558230375e+15 ||r(i)||/||b|| 4.639105664496e-02 >>> 19 KSP preconditioned resid norm 1.083945264815e+16 true resid norm >>> 2.485250997240e+15 ||r(i)||/||b|| 4.751314777723e-02 >>> 20 KSP preconditioned resid norm 9.014213050640e+15 true resid norm >>> 1.580604299892e+15 ||r(i)||/||b|| 3.021806882344e-02 >>> 21 KSP preconditioned resid norm 7.767372671831e+15 true resid norm >>> 1.339058517057e+15 ||r(i)||/||b|| 2.560018496078e-02 >>> 22 KSP preconditioned resid norm 6.894209717493e+15 true resid norm >>> 1.176401403286e+15 ||r(i)||/||b|| 2.249049845740e-02 >>> 23 KSP preconditioned resid norm 6.197326579110e+15 true resid norm >>> 1.015741229888e+15 ||r(i)||/||b|| 1.941898955585e-02 >>> 24 KSP preconditioned resid norm 5.635415722302e+15 true resid norm >>> 9.407688590092e+14 ||r(i)||/||b|| 1.798566417314e-02 >>> 25 KSP preconditioned resid norm 5.240066189102e+15 true resid norm >>> 8.479712706847e+14 ||r(i)||/||b|| 1.621155542826e-02 >>> 26 KSP preconditioned resid norm 5.077381341025e+15 true resid norm >>> 8.489262350542e+14 ||r(i)||/||b|| 1.622981248288e-02 >>> 27 KSP preconditioned resid norm 4.908170126730e+15 true resid norm >>> 9.447288544322e+14 ||r(i)||/||b|| 1.806137155559e-02 >>> 28 KSP preconditioned resid norm 4.807291179172e+15 true resid norm >>> 7.514342780137e+14 ||r(i)||/||b|| 1.436595657171e-02 >>> 29 KSP preconditioned resid norm 4.874566711284e+15 true resid norm >>> 7.721244730622e+14 ||r(i)||/||b|| 1.476151271311e-02 >>> 30 KSP preconditioned resid norm 5.131077173970e+15 true resid norm >>> 8.050523689285e+14 ||r(i)||/||b|| 1.539102980576e-02 >>> 31 KSP preconditioned resid norm 5.679198106771e+15 true resid norm >>> 8.990076632730e+14 ||r(i)||/||b|| 1.718727162987e-02 >>> 32 KSP preconditioned resid norm 8.479698883566e+15 true resid norm >>> 1.362468331537e+15 ||r(i)||/||b|| 2.604773491692e-02 >>> 33 KSP preconditioned resid norm 1.652298710742e+16 true resid norm >>> 2.658951324175e+15 ||r(i)||/||b|| 5.083395895960e-02 >>> 34 KSP preconditioned resid norm 3.352309602032e+16 true resid norm >>> 5.485872403557e+15 ||r(i)||/||b|| 1.048791717564e-01 >>> 35 KSP preconditioned resid norm 6.378357733800e+15 true resid norm >>> 1.047516366293e+15 ||r(i)||/||b|| 2.002646813784e-02 >>> 36 KSP preconditioned resid norm 3.388128874134e+15 true resid norm >>> 5.554504439336e+14 ||r(i)||/||b|| 1.061912823085e-02 >>> 37 KSP preconditioned resid norm 2.152326148598e+15 true resid norm >>> 3.939853481438e+14 ||r(i)||/||b|| 7.532230784419e-03 >>> 38 KSP preconditioned resid norm 1.818288964313e+15 true resid norm >>> 2.940442100180e+14 ||r(i)||/||b|| 5.621551311776e-03 >>> 39 KSP preconditioned resid norm 1.660791920226e+15 true resid norm >>> 2.869955671543e+14 ||r(i)||/||b|| 5.486795019399e-03 >>> 40 KSP preconditioned resid norm 1.530575944029e+15 true resid norm >>> 2.452126997689e+14 ||r(i)||/||b|| 4.687988156493e-03 >>> 41 KSP preconditioned resid norm 1.370742276038e+15 true resid norm >>> 2.241019638220e+14 ||r(i)||/||b|| 4.284392094026e-03 >>> 42 KSP preconditioned resid norm 1.242467362496e+15 true resid norm >>> 2.048557595739e+14 ||r(i)||/||b|| 3.916442238011e-03 >>> 43 KSP preconditioned resid norm 1.151462192104e+15 true resid norm >>> 1.847175725318e+14 ||r(i)||/||b|| 3.531439412157e-03 >>> 44 KSP preconditioned resid norm 1.085430472672e+15 true resid norm >>> 1.860586924987e+14 ||r(i)||/||b|| 3.557079008015e-03 >>> 45 KSP preconditioned resid norm 1.036632239565e+15 true resid norm >>> 1.638613644087e+14 ||r(i)||/||b|| 3.132709424833e-03 >>> 46 KSP preconditioned resid norm 1.028748911897e+15 true resid norm >>> 1.653234224458e+14 ||r(i)||/||b|| 3.160661120519e-03 >>> 47 KSP preconditioned resid norm 9.769896259678e+14 true resid norm >>> 1.886232302190e+14 ||r(i)||/||b|| 3.606107963166e-03 >>> 48 KSP preconditioned resid norm 1.056096824670e+15 true resid norm >>> 1.655410058571e+14 ||r(i)||/||b|| 3.164820890613e-03 >>> 49 KSP preconditioned resid norm 7.777695958066e+14 true resid norm >>> 2.880991932726e+14 ||r(i)||/||b|| 5.507894196468e-03 >>> 50 KSP preconditioned resid norm 1.009825961040e+15 true resid norm >>> 1.550789761056e+14 ||r(i)||/||b|| 2.964807303982e-03 >>> 51 KSP preconditioned resid norm 9.471596692833e+14 true resid norm >>> 2.053798093618e+14 ||r(i)||/||b|| 3.926461046995e-03 >>> 52 KSP preconditioned resid norm 8.991518749617e+14 true resid norm >>> 1.386233583178e+14 ||r(i)||/||b|| 2.650208013776e-03 >>> 53 KSP preconditioned resid norm 7.186213371144e+13 true resid norm >>> 1.103090149144e+13 ||r(i)||/||b|| 2.108893038414e-04 >>> 0 KSP preconditioned resid norm 6.542826642292e+18 true resid norm >>> 3.545903145424e+18 ||r(i)||/||b|| 4.156717341790e+02 >>> 1 KSP preconditioned resid norm 1.591157510957e+18 true resid norm >>> 5.223152980213e+17 ||r(i)||/||b|| 6.122888776501e+01 >>> 2 KSP preconditioned resid norm 6.570525108060e+17 true resid norm >>> 1.552502546069e+17 ||r(i)||/||b|| 1.819935286374e+01 >>> 3 KSP preconditioned resid norm 4.165816250413e+17 true resid norm >>> 8.738489300540e+16 ||r(i)||/||b|| 1.024377387845e+01 >>> 4 KSP preconditioned resid norm 2.995010683652e+17 true resid norm >>> 4.934680552583e+16 ||r(i)||/||b|| 5.784724339013e+00 >>> 5 KSP preconditioned resid norm 2.594493782706e+17 true resid norm >>> 4.469794627497e+16 ||r(i)||/||b|| 5.239757568206e+00 >>> 6 KSP preconditioned resid norm 2.219609972421e+17 true resid norm >>> 4.652118621437e+16 ||r(i)||/||b|| 5.453488535001e+00 >>> 7 KSP preconditioned resid norm 1.674912012123e+17 true resid norm >>> 2.988815541157e+16 ||r(i)||/||b|| 3.503666310619e+00 >>> 8 KSP preconditioned resid norm 1.081901407775e+17 true resid norm >>> 1.908145564246e+16 ||r(i)||/||b|| 2.236841062001e+00 >>> 9 KSP preconditioned resid norm 5.710123690945e+16 true resid norm >>> 9.635193037737e+15 ||r(i)||/||b|| 1.129494302267e+00 >>> 10 KSP preconditioned resid norm 1.779373863118e+16 true resid norm >>> 2.952490629534e+15 ||r(i)||/||b|| 3.461084101266e-01 >>> 11 KSP preconditioned resid norm 1.125622359514e+16 true resid norm >>> 2.030418337832e+15 ||r(i)||/||b|| 2.380176437376e-01 >>> 12 KSP preconditioned resid norm 1.086650353983e+16 true resid norm >>> 1.877894898988e+15 ||r(i)||/||b|| 2.201379443417e-01 >>> 13 KSP preconditioned resid norm 1.075411372217e+16 true resid norm >>> 1.822725565845e+15 ||r(i)||/||b|| 2.136706688859e-01 >>> 14 KSP preconditioned resid norm 1.173963298627e+16 true resid norm >>> 2.130483772307e+15 ||r(i)||/||b|| 2.497479056692e-01 >>> 15 KSP preconditioned resid norm 4.943816039086e+16 true resid norm >>> 1.362098103563e+16 ||r(i)||/||b|| 1.596731939959e+00 >>> 16 KSP preconditioned resid norm 1.779827523151e+17 true resid norm >>> 3.369509176778e+16 ||r(i)||/||b|| 3.949937901296e+00 >>> 17 KSP preconditioned resid norm 3.250308535899e+16 true resid norm >>> 8.719651405683e+15 ||r(i)||/||b|| 1.022169098418e+00 >>> 18 KSP preconditioned resid norm 2.074788387365e+16 true resid norm >>> 3.942900424212e+15 ||r(i)||/||b|| 4.622101027047e-01 >>> 19 KSP preconditioned resid norm 1.482962263858e+16 true resid norm >>> 3.390403569469e+15 ||r(i)||/||b|| 3.974431543926e-01 >>> 20 KSP preconditioned resid norm 1.133931148892e+16 true resid norm >>> 2.488262609273e+15 ||r(i)||/||b|| 2.916888565398e-01 >>> 21 KSP preconditioned resid norm 8.550489811072e+15 true resid norm >>> 1.657275963132e+15 ||r(i)||/||b|| 1.942756881268e-01 >>> 22 KSP preconditioned resid norm 6.584107021814e+15 true resid norm >>> 1.285719379545e+15 ||r(i)||/||b|| 1.507196283273e-01 >>> 23 KSP preconditioned resid norm 4.778317630297e+15 true resid norm >>> 9.061620975056e+14 ||r(i)||/||b|| 1.062256793459e-01 >>> 24 KSP preconditioned resid norm 3.026263700342e+15 true resid norm >>> 6.175894175717e+14 ||r(i)||/||b|| 7.239748342928e-02 >>> 25 KSP preconditioned resid norm 1.568620488361e+15 true resid norm >>> 2.753144451691e+14 ||r(i)||/||b|| 3.227398723952e-02 >>> 26 KSP preconditioned resid norm 1.178330886619e+15 true resid norm >>> 2.275547590347e+14 ||r(i)||/||b|| 2.667531442045e-02 >>> 27 KSP preconditioned resid norm 1.073508363176e+15 true resid norm >>> 2.033900922703e+14 ||r(i)||/||b|| 2.384258929293e-02 >>> 28 KSP preconditioned resid norm 1.011358091649e+15 true resid norm >>> 1.668017917552e+14 ||r(i)||/||b|| 1.955349235427e-02 >>> 29 KSP preconditioned resid norm 9.448460925412e+14 true resid norm >>> 1.798337339042e+14 ||r(i)||/||b|| 2.108117367287e-02 >>> 30 KSP preconditioned resid norm 8.970299972493e+14 true resid norm >>> 1.456018479256e+14 ||r(i)||/||b|| 1.706830958004e-02 >>> 31 KSP preconditioned resid norm 8.656072021210e+14 true resid norm >>> 1.578084000747e+14 ||r(i)||/||b|| 1.849923380218e-02 >>> 32 KSP preconditioned resid norm 8.314911059315e+14 true resid norm >>> 1.333543211106e+14 ||r(i)||/||b|| 1.563258206526e-02 >>> 33 KSP preconditioned resid norm 8.004072124456e+14 true resid norm >>> 1.582598207144e+14 ||r(i)||/||b|| 1.855215199888e-02 >>> 34 KSP preconditioned resid norm 7.705686923615e+14 true resid norm >>> 1.334906781329e+14 ||r(i)||/||b|| 1.564856664172e-02 >>> 35 KSP preconditioned resid norm 7.648238175354e+14 true resid norm >>> 1.202157111000e+14 ||r(i)||/||b|| 1.409239650919e-02 >>> 36 KSP preconditioned resid norm 7.709241111814e+14 true resid norm >>> 1.292601566237e+14 ||r(i)||/||b|| 1.515263989468e-02 >>> 37 KSP preconditioned resid norm 8.026855071425e+14 true resid norm >>> 1.264132558814e+14 ||r(i)||/||b|| 1.481890935550e-02 >>> 38 KSP preconditioned resid norm 8.397026301556e+14 true resid norm >>> 1.379015425365e+14 ||r(i)||/||b|| 1.616563424923e-02 >>> 39 KSP preconditioned resid norm 8.772641473437e+14 true resid norm >>> 1.384855714837e+14 ||r(i)||/||b|| 1.623409757587e-02 >>> 40 KSP preconditioned resid norm 9.403663540144e+14 true resid norm >>> 1.523096876425e+14 ||r(i)||/||b|| 1.785464221614e-02 >>> 41 KSP preconditioned resid norm 1.191025293024e+15 true resid norm >>> 1.883988208054e+14 ||r(i)||/||b|| 2.208522380610e-02 >>> 42 KSP preconditioned resid norm 2.105592809750e+15 true resid norm >>> 3.674437948704e+14 ||r(i)||/||b|| 4.307393438656e-02 >>> 43 KSP preconditioned resid norm 1.653069974154e+16 true resid norm >>> 2.622043044873e+15 ||r(i)||/||b|| 3.073713902652e-01 >>> 44 KSP preconditioned resid norm 1.224677806349e+15 true resid norm >>> 2.078786938958e+14 ||r(i)||/||b|| 2.436876971726e-02 >>> 45 KSP preconditioned resid norm 6.142260967855e+14 true resid norm >>> 9.787693095074e+13 ||r(i)||/||b|| 1.147371260744e-02 >>> 46 KSP preconditioned resid norm 4.190317366481e+14 true resid norm >>> 7.181248026289e+13 ||r(i)||/||b|| 8.418283574692e-03 >>> 47 KSP preconditioned resid norm 2.851377922408e+14 true resid norm >>> 4.551212829362e+13 ||r(i)||/||b|| 5.335200798815e-03 >>> 48 KSP preconditioned resid norm 1.874725496828e+14 true resid norm >>> 3.515118139860e+13 ||r(i)||/||b|| 4.120629337903e-03 >>> 49 KSP preconditioned resid norm 1.260137969667e+14 true resid norm >>> 2.018341951104e+13 ||r(i)||/||b|| 2.366019782758e-03 >>> 50 KSP preconditioned resid norm 8.902880553848e+13 true resid norm >>> 2.353973309483e+13 ||r(i)||/||b|| 2.759466707450e-03 >>> 51 KSP preconditioned resid norm 6.273228119323e+13 true resid norm >>> 1.015825242680e+13 ||r(i)||/||b|| 1.190810416783e-03 >>> 52 KSP preconditioned resid norm 4.428322011916e+13 true resid norm >>> 1.394037459429e+13 ||r(i)||/||b|| 1.634173141528e-03 >>> 0 KSP preconditioned resid norm 1.892236961136e+19 true resid norm >>> 1.028105836629e+19 ||r(i)||/||b|| 7.839481546009e+01 >>> 1 KSP preconditioned resid norm 4.580244542528e+18 true resid norm >>> 1.519483928522e+18 ||r(i)||/||b|| 1.158632291804e+01 >>> 2 KSP preconditioned resid norm 1.874052673028e+18 true resid norm >>> 4.645668462025e+17 ||r(i)||/||b|| 3.542401071890e+00 >>> 3 KSP preconditioned resid norm 1.172977406681e+18 true resid norm >>> 2.550460309107e+17 ||r(i)||/||b|| 1.944769284904e+00 >>> 4 KSP preconditioned resid norm 8.299170411979e+17 true resid norm >>> 1.381052822672e+17 ||r(i)||/||b|| 1.053076223446e+00 >>> 5 KSP preconditioned resid norm 7.051011041666e+17 true resid norm >>> 1.287511951247e+17 ||r(i)||/||b|| 9.817497209388e-01 >>> 6 KSP preconditioned resid norm 5.862228682149e+17 true resid norm >>> 1.166604371743e+17 ||r(i)||/||b|| 8.895556389170e-01 >>> 7 KSP preconditioned resid norm 4.329574557115e+17 true resid norm >>> 8.609505694316e+16 ||r(i)||/||b|| 6.564894255647e-01 >>> 8 KSP preconditioned resid norm 2.701974155126e+17 true resid norm >>> 4.802624326608e+16 ||r(i)||/||b|| 3.662082583277e-01 >>> 9 KSP preconditioned resid norm 1.546976756797e+17 true resid norm >>> 2.878555741954e+16 ||r(i)||/||b|| 2.194947622532e-01 >>> 10 KSP preconditioned resid norm 9.916116392638e+16 true resid norm >>> 1.649766880508e+16 ||r(i)||/||b|| 1.257975254509e-01 >>> 11 KSP preconditioned resid norm 9.248895040376e+16 true resid norm >>> 1.560136782751e+16 ||r(i)||/||b|| 1.189630783317e-01 >>> 12 KSP preconditioned resid norm 9.251831545179e+16 true resid norm >>> 1.508056895396e+16 ||r(i)||/||b|| 1.149918985048e-01 >>> 13 KSP preconditioned resid norm 7.096157383949e+16 true resid norm >>> 1.151221892162e+16 ||r(i)||/||b|| 8.778262370879e-02 >>> 14 KSP preconditioned resid norm 2.017629866031e+17 true resid norm >>> 3.268014415733e+16 ||r(i)||/||b|| 2.491916473135e-01 >>> 15 KSP preconditioned resid norm 1.669814416925e+16 true resid norm >>> 4.397468530146e+15 ||r(i)||/||b|| 3.353144410138e-02 >>> 16 KSP preconditioned resid norm 1.171914014315e+16 true resid norm >>> 1.865370183430e+15 ||r(i)||/||b|| 1.422376433288e-02 >>> 17 KSP preconditioned resid norm 1.021030261775e+16 true resid norm >>> 2.000923353135e+15 ||r(i)||/||b|| 1.525738026477e-02 >>> 18 KSP preconditioned resid norm 9.416598060128e+15 true resid norm >>> 1.541246281727e+15 ||r(i)||/||b|| 1.175226455582e-02 >>> 19 KSP preconditioned resid norm 8.821999311745e+15 true resid norm >>> 1.426601243635e+15 ||r(i)||/||b|| 1.087807667707e-02 >>> 20 KSP preconditioned resid norm 8.323362047556e+15 true resid norm >>> 1.490210405686e+15 ||r(i)||/||b|| 1.136310733664e-02 >>> 21 KSP preconditioned resid norm 7.861641988008e+15 true resid norm >>> 1.251748347072e+15 ||r(i)||/||b|| 9.544793655962e-03 >>> 22 KSP preconditioned resid norm 7.464952525350e+15 true resid norm >>> 1.325467789570e+15 ||r(i)||/||b|| 1.010691692037e-02 >>> 23 KSP preconditioned resid norm 7.139096535480e+15 true resid norm >>> 1.125905927178e+15 ||r(i)||/||b|| 8.585223839986e-03 >>> 24 KSP preconditioned resid norm 5.858113828393e+15 true resid norm >>> 2.042448137714e+15 ||r(i)||/||b|| 1.557401379686e-02 >>> 25 KSP preconditioned resid norm 5.027356938946e+15 true resid norm >>> 7.946522117539e+14 ||r(i)||/||b|| 6.059358022871e-03 >>> 26 KSP preconditioned resid norm 5.122614017475e+15 true resid norm >>> 7.976827488389e+14 ||r(i)||/||b|| 6.082466382639e-03 >>> 27 KSP preconditioned resid norm 5.425317067876e+15 true resid norm >>> 8.948524578548e+14 ||r(i)||/||b|| 6.823401910404e-03 >>> 28 KSP preconditioned resid norm 9.316529191423e+15 true resid norm >>> 1.453760069771e+15 ||r(i)||/||b|| 1.108516733711e-02 >>> 29 KSP preconditioned resid norm 9.523628080058e+15 true resid norm >>> 2.109739324167e+15 ||r(i)||/||b|| 1.608712051759e-02 >>> 30 KSP preconditioned resid norm 2.300173794637e+15 true resid norm >>> 3.594261469430e+14 ||r(i)||/||b|| 2.740685390281e-03 >>> 31 KSP preconditioned resid norm 1.385027169120e+15 true resid norm >>> 2.935564708865e+14 ||r(i)||/||b|| 2.238417927644e-03 >>> 32 KSP preconditioned resid norm 9.719758142764e+14 true resid norm >>> 1.523176597454e+14 ||r(i)||/||b|| 1.161448014555e-03 >>> 33 KSP preconditioned resid norm 8.580268959656e+14 true resid norm >>> 1.580637131624e+14 ||r(i)||/||b|| 1.205262647368e-03 >>> 0 KSP preconditioned resid norm 6.470797769938e+18 true resid norm >>> 3.511548722307e+18 ||r(i)||/||b|| 1.662246831388e+02 >>> 1 KSP preconditioned resid norm 1.571597715354e+18 true resid norm >>> 5.187477038218e+17 ||r(i)||/||b|| 2.455573865428e+01 >>> 2 KSP preconditioned resid norm 6.468337546861e+17 true resid norm >>> 1.566996108585e+17 ||r(i)||/||b|| 7.417622599812e+00 >>> 3 KSP preconditioned resid norm 4.084044703454e+17 true resid norm >>> 8.688412887362e+16 ||r(i)||/||b|| 4.112796926342e+00 >>> 4 KSP preconditioned resid norm 2.920894102259e+17 true resid norm >>> 4.831956260441e+16 ||r(i)||/||b|| 2.287282512214e+00 >>> 5 KSP preconditioned resid norm 2.515869150351e+17 true resid norm >>> 4.435569840236e+16 ||r(i)||/||b|| 2.099646764259e+00 >>> 6 KSP preconditioned resid norm 2.133085363553e+17 true resid norm >>> 4.355632900946e+16 ||r(i)||/||b|| 2.061807356479e+00 >>> 7 KSP preconditioned resid norm 1.601295664581e+17 true resid norm >>> 2.999352167191e+16 ||r(i)||/||b|| 1.419790534148e+00 >>> 8 KSP preconditioned resid norm 1.019894513782e+17 true resid norm >>> 1.811122934468e+16 ||r(i)||/||b|| 8.573235336161e-01 >>> 9 KSP preconditioned resid norm 5.552366492146e+16 true resid norm >>> 9.849650352808e+15 ||r(i)||/||b|| 4.662486949199e-01 >>> 10 KSP preconditioned resid norm 2.614329828139e+16 true resid norm >>> 4.400937150634e+15 ||r(i)||/||b|| 2.083252835795e-01 >>> 11 KSP preconditioned resid norm 2.230447954838e+16 true resid norm >>> 4.002038016135e+15 ||r(i)||/||b|| 1.894427654999e-01 >>> 12 KSP preconditioned resid norm 2.202888172936e+16 true resid norm >>> 3.669015023581e+15 ||r(i)||/||b|| 1.736785982356e-01 >>> 13 KSP preconditioned resid norm 2.171444770780e+16 true resid norm >>> 3.746639147837e+15 ||r(i)||/||b|| 1.773530582755e-01 >>> 14 KSP preconditioned resid norm 2.756620760322e+16 true resid norm >>> 4.738417417687e+15 ||r(i)||/||b|| 2.243004429444e-01 >>> 15 KSP preconditioned resid norm 1.911455835947e+16 true resid norm >>> 7.274462982878e+15 ||r(i)||/||b|| 3.443481494796e-01 >>> 16 KSP preconditioned resid norm 8.270124664006e+15 true resid norm >>> 1.450957167614e+15 ||r(i)||/||b|| 6.868334017480e-02 >>> 17 KSP preconditioned resid norm 5.454864800054e+15 true resid norm >>> 1.779587764442e+15 ||r(i)||/||b|| 8.423958647728e-02 >>> 18 KSP preconditioned resid norm 4.188178169772e+15 true resid norm >>> 7.356207570842e+14 ||r(i)||/||b|| 3.482176581514e-02 >>> 19 KSP preconditioned resid norm 3.485885419242e+15 true resid norm >>> 7.684000525717e+14 ||r(i)||/||b|| 3.637342533543e-02 >>> 20 KSP preconditioned resid norm 3.031526098420e+15 true resid norm >>> 5.668894180734e+14 ||r(i)||/||b|| 2.683460243492e-02 >>> 21 KSP preconditioned resid norm 2.653824581081e+15 true resid norm >>> 5.337896513987e+14 ||r(i)||/||b|| 2.526777290683e-02 >>> 22 KSP preconditioned resid norm 2.412624108418e+15 true resid norm >>> 3.917881686680e+14 ||r(i)||/||b|| 1.854590932504e-02 >>> 23 KSP preconditioned resid norm 2.249293018506e+15 true resid norm >>> 3.520070992955e+14 ||r(i)||/||b|| 1.666280982271e-02 >>> 24 KSP preconditioned resid norm 2.107969657718e+15 true resid norm >>> 3.433991961929e+14 ||r(i)||/||b|| 1.625534118740e-02 >>> 25 KSP preconditioned resid norm 1.995899579616e+15 true resid norm >>> 3.124097002184e+14 ||r(i)||/||b|| 1.478840464277e-02 >>> 26 KSP preconditioned resid norm 1.940326192847e+15 true resid norm >>> 3.197343858652e+14 ||r(i)||/||b|| 1.513513016106e-02 >>> 27 KSP preconditioned resid norm 1.843618399864e+15 true resid norm >>> 4.284525431489e+14 ||r(i)||/||b|| 2.028147517148e-02 >>> 28 KSP preconditioned resid norm 1.800569430365e+15 true resid norm >>> 2.810328443449e+14 ||r(i)||/||b|| 1.330313180793e-02 >>> 29 KSP preconditioned resid norm 1.911265522964e+15 true resid norm >>> 3.078500227429e+14 ||r(i)||/||b|| 1.457256513619e-02 >>> 30 KSP preconditioned resid norm 2.245578740852e+15 true resid norm >>> 3.522723048093e+14 ||r(i)||/||b|| 1.667536374293e-02 >>> 31 KSP preconditioned resid norm 3.129444954320e+15 true resid norm >>> 5.056459987900e+14 ||r(i)||/||b|| 2.393554883500e-02 >>> 32 KSP preconditioned resid norm 2.268235755551e+16 true resid norm >>> 3.663405821329e+15 ||r(i)||/||b|| 1.734130778226e-01 >>> 33 KSP preconditioned resid norm 2.666354866266e+15 true resid norm >>> 4.420681584183e+14 ||r(i)||/||b|| 2.092599174035e-02 >>> 34 KSP preconditioned resid norm 1.195729215971e+15 true resid norm >>> 1.967322086000e+14 ||r(i)||/||b|| 9.312628593187e-03 >>> 35 KSP preconditioned resid norm 7.390918117697e+14 true resid norm >>> 1.247185996576e+14 ||r(i)||/||b|| 5.903751122089e-03 >>> 36 KSP preconditioned resid norm 5.589675490221e+14 true resid norm >>> 9.067132579829e+13 ||r(i)||/||b|| 4.292069850790e-03 >>> 37 KSP preconditioned resid norm 4.470335591004e+14 true resid norm >>> 9.109636100840e+13 ||r(i)||/||b|| 4.312189561126e-03 >>> 38 KSP preconditioned resid norm 4.031608678043e+14 true resid norm >>> 6.399106058589e+13 ||r(i)||/||b|| 3.029117523568e-03 >>> 39 KSP preconditioned resid norm 3.769205171045e+14 true resid norm >>> 7.200688789282e+13 ||r(i)||/||b|| 3.408559319641e-03 >>> 40 KSP preconditioned resid norm 3.535103364248e+14 true resid norm >>> 5.569365215364e+13 ||r(i)||/||b|| 2.636346641945e-03 >>> 41 KSP preconditioned resid norm 3.299087996116e+14 true resid norm >>> 5.691184302265e+13 ||r(i)||/||b|| 2.694011623187e-03 >>> 42 KSP preconditioned resid norm 3.102396074559e+14 true resid norm >>> 5.072539063269e+13 ||r(i)||/||b|| 2.401166166782e-03 >>> 43 KSP preconditioned resid norm 2.923631153501e+14 true resid norm >>> 5.690604160219e+13 ||r(i)||/||b|| 2.693737003823e-03 >>> 44 KSP preconditioned resid norm 2.831787236990e+14 true resid norm >>> 4.398532156025e+13 ||r(i)||/||b|| 2.082114393761e-03 >>> 45 KSP preconditioned resid norm 2.880585984465e+14 true resid norm >>> 4.666639157225e+13 ||r(i)||/||b|| 2.209027060639e-03 >>> 46 KSP preconditioned resid norm 3.054400711070e+14 true resid norm >>> 4.771426500640e+13 ||r(i)||/||b|| 2.258629798159e-03 >>> 47 KSP preconditioned resid norm 3.761477931303e+14 true resid norm >>> 5.957625439087e+13 ||r(i)||/||b|| 2.820135726954e-03 >>> 48 KSP preconditioned resid norm 9.608784364697e+14 true resid norm >>> 1.580857575603e+14 ||r(i)||/||b|| 7.483238034628e-03 >>> 49 KSP preconditioned resid norm 1.902148260101e+15 true resid norm >>> 2.982038509627e+14 ||r(i)||/||b|| 1.411594841961e-02 >>> 50 KSP preconditioned resid norm 2.165055327372e+14 true resid norm >>> 4.058692191960e+13 ||r(i)||/||b|| 1.921245800409e-03 >>> 51 KSP preconditioned resid norm 2.195213865618e+14 true resid norm >>> 3.429224338721e+13 ||r(i)||/||b|| 1.623277289290e-03 >>> 52 KSP preconditioned resid norm 1.864119738188e+14 true resid norm >>> 5.012858400476e+13 ||r(i)||/||b|| 2.372915386152e-03 >>> 53 KSP preconditioned resid norm 2.971924491921e+13 true resid norm >>> 4.598158953060e+12 ||r(i)||/||b|| 2.176610878666e-04 >>> 0 KSP preconditioned resid norm 6.694681472101e+18 true resid norm >>> 3.650754008704e+18 ||r(i)||/||b|| 5.229798534711e+00 >>> 1 KSP preconditioned resid norm 1.597667311920e+18 true resid norm >>> 5.366749653247e+17 ||r(i)||/||b|| 7.688006205237e-01 >>> 2 KSP preconditioned resid norm 6.300446691769e+17 true resid norm >>> 1.705328302298e+17 ||r(i)||/||b|| 2.442926429799e-01 >>> 3 KSP preconditioned resid norm 3.642174434692e+17 true resid norm >>> 9.481577406412e+16 ||r(i)||/||b|| 1.358260225383e-01 >>> 4 KSP preconditioned resid norm 2.247332932327e+17 true resid norm >>> 3.802800659066e+16 ||r(i)||/||b|| 5.447609251997e-02 >>> 5 KSP preconditioned resid norm 1.557523729879e+17 true resid norm >>> 3.829141784861e+16 ||r(i)||/||b|| 5.485343588727e-02 >>> 6 KSP preconditioned resid norm 1.004692897232e+17 true resid norm >>> 1.699728149080e+16 ||r(i)||/||b|| 2.434904067015e-02 >>> 7 KSP preconditioned resid norm 7.103483293598e+16 true resid norm >>> 1.472858902835e+16 ||r(i)||/||b|| 2.109908066531e-02 >>> 8 KSP preconditioned resid norm 5.766848376574e+16 true resid norm >>> 9.397954931890e+15 ||r(i)||/||b|| 1.346281091931e-02 >>> 9 KSP preconditioned resid norm 4.523472389397e+16 true resid norm >>> 1.246375028587e+16 ||r(i)||/||b|| 1.785464121292e-02 >>> 10 KSP preconditioned resid norm 3.195342596604e+16 true resid norm >>> 5.030521805863e+15 ||r(i)||/||b|| 7.206351210301e-03 >>> 11 KSP preconditioned resid norm 2.437460193025e+16 true resid norm >>> 4.974645265358e+15 ||r(i)||/||b|| 7.126306636231e-03 >>> 12 KSP preconditioned resid norm 1.258045317564e+16 true resid norm >>> 1.885909944349e+15 ||r(i)||/||b|| 2.701614252848e-03 >>> 13 KSP preconditioned resid norm 2.495567071066e+15 true resid norm >>> 4.460991862355e+14 ||r(i)||/||b|| 6.390484992820e-04 >>> 14 KSP preconditioned resid norm 7.678105595258e+14 true resid norm >>> 1.316278119117e+14 ||r(i)||/||b|| 1.885602087190e-04 >>> 15 KSP preconditioned resid norm 5.950875767888e+14 true resid norm >>> 1.439411571316e+14 ||r(i)||/||b|| 2.061993908263e-04 >>> 16 KSP preconditioned resid norm 3.263039815738e+14 true resid norm >>> 5.046549667518e+13 ||r(i)||/||b|| 7.229311532250e-05 >>> 17 KSP preconditioned resid norm 1.436785883111e+14 true resid norm >>> 7.588987254543e+13 ||r(i)||/||b|| 1.087141843277e-04 >>> 18 KSP preconditioned resid norm 5.401544347363e+13 true resid norm >>> 8.625334293000e+12 ||r(i)||/||b|| 1.235601208391e-05 >>> 19 KSP preconditioned resid norm 3.528863285477e+13 true resid norm >>> 1.233589079533e+13 ||r(i)||/||b|| 1.767147922099e-05 >>> 0 KSP preconditioned resid norm 4.827871909881e+18 true resid norm >>> 2.629048338874e+18 ||r(i)||/||b|| 2.387243403092e+01 >>> 1 KSP preconditioned resid norm 1.161059375885e+18 true resid norm >>> 3.885115498370e+17 ||r(i)||/||b|| 3.527784638492e+00 >>> 2 KSP preconditioned resid norm 4.696929326234e+17 true resid norm >>> 1.228954029189e+17 ||r(i)||/||b|| 1.115921816843e+00 >>> 3 KSP preconditioned resid norm 2.870100194440e+17 true resid norm >>> 6.759377089469e+16 ||r(i)||/||b|| 6.137687971441e-01 >>> 4 KSP preconditioned resid norm 1.966638027618e+17 true resid norm >>> 3.310588396768e+16 ||r(i)||/||b|| 3.006099276942e-01 >>> 5 KSP preconditioned resid norm 1.592573327953e+17 true resid norm >>> 3.239413757211e+16 ||r(i)||/||b|| 2.941470876529e-01 >>> 6 KSP preconditioned resid norm 1.230114084345e+17 true resid norm >>> 2.280735920899e+16 ||r(i)||/||b|| 2.070966783247e-01 >>> 7 KSP preconditioned resid norm 8.598639005636e+16 true resid norm >>> 1.938027655308e+16 ||r(i)||/||b|| 1.759778877677e-01 >>> 8 KSP preconditioned resid norm 5.549090128976e+16 true resid norm >>> 9.386098364471e+15 ||r(i)||/||b|| 8.522818340779e-02 >>> 9 KSP preconditioned resid norm 4.474765035679e+16 true resid norm >>> 7.579175521646e+15 ||r(i)||/||b|| 6.882085999480e-02 >>> 10 KSP preconditioned resid norm 4.378403606928e+16 true resid norm >>> 6.842614495131e+15 ||r(i)||/||b|| 6.213269673237e-02 >>> 11 KSP preconditioned resid norm 4.079517449398e+16 true resid norm >>> 7.704845808673e+15 ||r(i)||/||b|| 6.996197847191e-02 >>> 12 KSP preconditioned resid norm 4.084405622719e+16 true resid norm >>> 6.377952548774e+15 ||r(i)||/||b|| 5.791344693880e-02 >>> 13 KSP preconditioned resid norm 7.523868588173e+16 true resid norm >>> 1.132249240798e+16 ||r(i)||/||b|| 1.028111385699e-01 >>> 14 KSP preconditioned resid norm 2.184581375390e+15 true resid norm >>> 3.502138477296e+14 ||r(i)||/||b|| 3.180031668881e-03 >>> 15 KSP preconditioned resid norm 2.222269400035e+15 true resid norm >>> 3.654767263686e+14 ||r(i)||/||b|| 3.318622526282e-03 >>> 16 KSP preconditioned resid norm 1.669508240008e+15 true resid norm >>> 3.629680148217e+14 ||r(i)||/||b|| 3.295842781223e-03 >>> 17 KSP preconditioned resid norm 1.401913603747e+15 true resid norm >>> 2.161155966391e+14 ||r(i)||/||b|| 1.962385113858e-03 >>> 18 KSP preconditioned resid norm 1.134372518022e+15 true resid norm >>> 2.686701598571e+14 ||r(i)||/||b|| 2.439594043376e-03 >>> 19 KSP preconditioned resid norm 9.057728167563e+14 true resid norm >>> 1.409896456102e+14 ||r(i)||/||b|| 1.280222186904e-03 >>> 20 KSP preconditioned resid norm 7.989861774441e+14 true resid norm >>> 1.659027150307e+14 ||r(i)||/||b|| 1.506439254676e-03 >>> 21 KSP preconditioned resid norm 7.119014585200e+14 true resid norm >>> 1.092105820565e+14 ||r(i)||/||b|| 9.916601292842e-04 >>> 22 KSP preconditioned resid norm 6.306860453827e+14 true resid norm >>> 1.460692513610e+14 ||r(i)||/||b|| 1.326346311515e-03 >>> 23 KSP preconditioned resid norm 5.637172379886e+14 true resid norm >>> 8.672554380666e+13 ||r(i)||/||b|| 7.874902080372e-04 >>> 24 KSP preconditioned resid norm 4.961067656606e+14 true resid norm >>> 1.292132364829e+14 ||r(i)||/||b|| 1.173289367962e-03 >>> 25 KSP preconditioned resid norm 4.386188245672e+14 true resid norm >>> 8.044020700160e+13 ||r(i)||/||b|| 7.304177358342e-04 >>> 26 KSP preconditioned resid norm 4.419078090293e+14 true resid norm >>> 6.703649620362e+13 ||r(i)||/||b|| 6.087085998464e-04 >>> 27 KSP preconditioned resid norm 4.436677280477e+14 true resid norm >>> 1.368871842229e+14 ||r(i)||/||b|| 1.242970784036e-03 >>> 28 KSP preconditioned resid norm 7.847692003240e+14 true resid norm >>> 1.196853324186e+14 ||r(i)||/||b|| 1.086773552386e-03 >>> 29 KSP preconditioned resid norm 1.771704167839e+15 true resid norm >>> 6.548256652554e+14 ||r(i)||/||b|| 5.945985193355e-03 >>> 30 KSP preconditioned resid norm 1.053899282998e+15 true resid norm >>> 1.880673778730e+14 ||r(i)||/||b|| 1.707700084953e-03 >>> 31 KSP preconditioned resid norm 2.462670432282e+14 true resid norm >>> 3.919805233311e+13 ||r(i)||/||b|| 3.559283808617e-04 >>> 32 KSP preconditioned resid norm 1.093017734594e+14 true resid norm >>> 1.903326631551e+13 ||r(i)||/||b|| 1.728269457018e-04 >>> 33 KSP preconditioned resid norm 6.432042300545e+13 true resid norm >>> 1.002471721316e+13 ||r(i)||/||b|| 9.102700654495e-05 >>> 34 KSP preconditioned resid norm 3.805689715853e+13 true resid norm >>> 7.047382673791e+12 ||r(i)||/||b|| 6.399204437707e-05 >>> 35 KSP preconditioned resid norm 2.527168724251e+13 true resid norm >>> 4.009865388649e+12 ||r(i)||/||b|| 3.641060742320e-05 >>> 36 KSP preconditioned resid norm 1.580947387777e+13 true resid norm >>> 2.531811988645e+12 ||r(i)||/||b|| 2.298950300149e-05 >>> 37 KSP preconditioned resid norm 1.075548264218e+13 true resid norm >>> 1.788042513750e+12 ||r(i)||/||b|| 1.623588517670e-05 >>> 38 KSP preconditioned resid norm 6.581274433602e+12 true resid norm >>> 1.035380330637e+12 ||r(i)||/||b|| 9.401519277742e-06 >>> 0 KSP preconditioned resid norm 9.555736051823e+15 true resid norm >>> 5.892741751019e+15 ||r(i)||/||b|| 1.119235791581e-01 >>> 1 KSP preconditioned resid norm 4.575036500991e+15 true resid norm >>> 8.101073993664e+14 ||r(i)||/||b|| 1.538674584270e-02 >>> 2 KSP preconditioned resid norm 3.765161451471e+15 true resid norm >>> 6.105904357791e+14 ||r(i)||/||b|| 1.159722754867e-02 >>> 3 KSP preconditioned resid norm 3.685525899039e+15 true resid norm >>> 6.160361102521e+14 ||r(i)||/||b|| 1.170065977151e-02 >>> 4 KSP preconditioned resid norm 3.856425370708e+15 true resid norm >>> 6.278594964367e+14 ||r(i)||/||b|| 1.192522683307e-02 >>> 5 KSP preconditioned resid norm 4.281981871434e+15 true resid norm >>> 6.756473177150e+14 ||r(i)||/||b|| 1.283288310305e-02 >>> 6 KSP preconditioned resid norm 5.205822014832e+15 true resid norm >>> 7.760311477213e+14 ||r(i)||/||b|| 1.473951977891e-02 >>> 7 KSP preconditioned resid norm 6.471690051175e+15 true resid norm >>> 1.171341036231e+15 ||r(i)||/||b|| 2.224782397211e-02 >>> 8 KSP preconditioned resid norm 7.016914241690e+15 true resid norm >>> 1.103022939349e+15 ||r(i)||/||b|| 2.095022664859e-02 >>> 9 KSP preconditioned resid norm 7.025590013206e+15 true resid norm >>> 1.330551872513e+15 ||r(i)||/||b|| 2.527178928238e-02 >>> 10 KSP preconditioned resid norm 6.405136813848e+15 true resid norm >>> 1.017867183616e+15 ||r(i)||/||b|| 1.933282385541e-02 >>> 11 KSP preconditioned resid norm 4.854020251396e+15 true resid norm >>> 8.473369029366e+14 ||r(i)||/||b|| 1.609386308386e-02 >>> 12 KSP preconditioned resid norm 2.353499572340e+15 true resid norm >>> 3.616493792413e+14 ||r(i)||/||b|| 6.868974517339e-03 >>> 13 KSP preconditioned resid norm 1.176204335431e+15 true resid norm >>> 2.390307465685e+14 ||r(i)||/||b|| 4.540021914275e-03 >>> 14 KSP preconditioned resid norm 1.050288991204e+15 true resid norm >>> 1.923001174300e+14 ||r(i)||/||b|| 3.652445385304e-03 >>> 15 KSP preconditioned resid norm 9.850224343838e+14 true resid norm >>> 1.772358652449e+14 ||r(i)||/||b|| 3.366323051568e-03 >>> 16 KSP preconditioned resid norm 8.835883676210e+14 true resid norm >>> 1.426145250921e+14 ||r(i)||/||b|| 2.708743868757e-03 >>> 17 KSP preconditioned resid norm 7.483591829964e+14 true resid norm >>> 1.680673516819e+14 ||r(i)||/||b|| 3.192181217957e-03 >>> 18 KSP preconditioned resid norm 6.652269160454e+14 true resid norm >>> 1.088301440117e+14 ||r(i)||/||b|| 2.067061438079e-03 >>> 19 KSP preconditioned resid norm 5.777832230782e+14 true resid norm >>> 9.220881620683e+13 ||r(i)||/||b|| 1.751364844390e-03 >>> 20 KSP preconditioned resid norm 5.145663259773e+14 true resid norm >>> 8.255284700142e+13 ||r(i)||/||b|| 1.567964539511e-03 >>> 21 KSP preconditioned resid norm 4.536494497263e+14 true resid norm >>> 7.162151119367e+13 ||r(i)||/||b|| 1.360340604800e-03 >>> 22 KSP preconditioned resid norm 4.039727797390e+14 true resid norm >>> 6.757078541538e+13 ||r(i)||/||b|| 1.283403289972e-03 >>> 23 KSP preconditioned resid norm 3.668438626905e+14 true resid norm >>> 5.700238490686e+13 ||r(i)||/||b|| 1.082672753854e-03 >>> 24 KSP preconditioned resid norm 3.368796010179e+14 true resid norm >>> 6.388426371627e+13 ||r(i)||/||b|| 1.213383472264e-03 >>> 0 KSP preconditioned resid norm 9.591788600089e+15 true resid norm >>> 5.932754803820e+15 ||r(i)||/||b|| 5.703607413375e-01 >>> 1 KSP preconditioned resid norm 4.627577110935e+15 true resid norm >>> 8.119149369849e+14 ||r(i)||/||b|| 7.805554429176e-02 >>> 2 KSP preconditioned resid norm 3.843546862524e+15 true resid norm >>> 6.215691855461e+14 ||r(i)||/||b|| 5.975616272433e-02 >>> 3 KSP preconditioned resid norm 3.802220665831e+15 true resid norm >>> 6.332285028379e+14 ||r(i)||/||b|| 6.087706137495e-02 >>> 4 KSP preconditioned resid norm 4.025305304206e+15 true resid norm >>> 6.486950645643e+14 ||r(i)||/||b|| 6.236397932519e-02 >>> 5 KSP preconditioned resid norm 4.519185013022e+15 true resid norm >>> 7.100155628952e+14 ||r(i)||/||b|| 6.825918417416e-02 >>> 6 KSP preconditioned resid norm 5.566187938033e+15 true resid norm >>> 8.272878114077e+14 ||r(i)||/||b|| 7.953345537054e-02 >>> 7 KSP preconditioned resid norm 7.046115999320e+15 true resid norm >>> 1.260178524183e+15 ||r(i)||/||b|| 1.211505246783e-01 >>> 8 KSP preconditioned resid norm 7.802424274220e+15 true resid norm >>> 1.227649475790e+15 ||r(i)||/||b|| 1.180232604022e-01 >>> 9 KSP preconditioned resid norm 8.076065684234e+15 true resid norm >>> 1.496448814554e+15 ||r(i)||/||b|| 1.438649806820e-01 >>> 10 KSP preconditioned resid norm 7.812303545308e+15 true resid norm >>> 1.236539634992e+15 ||r(i)||/||b|| 1.188779388714e-01 >>> 11 KSP preconditioned resid norm 6.692698687119e+15 true resid norm >>> 1.122795851888e+15 ||r(i)||/||b|| 1.079428858312e-01 >>> 12 KSP preconditioned resid norm 4.016544769049e+15 true resid norm >>> 6.096078457742e+14 ||r(i)||/||b|| 5.860622836074e-02 >>> 13 KSP preconditioned resid norm 1.711147574309e+15 true resid norm >>> 2.966258404639e+14 ||r(i)||/||b|| 2.851689305581e-02 >>> 14 KSP preconditioned resid norm 1.304839987953e+15 true resid norm >>> 2.700754536307e+14 ||r(i)||/||b|| 2.596440288593e-02 >>> 15 KSP preconditioned resid norm 1.171070442303e+15 true resid norm >>> 2.199042966295e+14 ||r(i)||/||b|| 2.114106882828e-02 >>> 16 KSP preconditioned resid norm 1.057836577493e+15 true resid norm >>> 1.660135728859e+14 ||r(i)||/||b|| 1.596014459292e-02 >>> 17 KSP preconditioned resid norm 8.905346433993e+14 true resid norm >>> 1.613310037432e+14 ||r(i)||/||b|| 1.550997368650e-02 >>> 18 KSP preconditioned resid norm 7.864866777768e+14 true resid norm >>> 1.268576094498e+14 ||r(i)||/||b|| 1.219578468395e-02 >>> 19 KSP preconditioned resid norm 6.743846682412e+14 true resid norm >>> 1.066594941052e+14 ||r(i)||/||b|| 1.025398657793e-02 >>> 20 KSP preconditioned resid norm 5.775523980161e+14 true resid norm >>> 1.005326356082e+14 ||r(i)||/||b|| 9.664965175573e-03 >>> 21 KSP preconditioned resid norm 4.897344092103e+14 true resid norm >>> 7.691922473584e+13 ||r(i)||/||b|| 7.394828792721e-03 >>> 22 KSP preconditioned resid norm 4.026909639422e+14 true resid norm >>> 7.643768412999e+13 ||r(i)||/||b|| 7.348534640001e-03 >>> 23 KSP preconditioned resid norm 3.340361168756e+14 true resid norm >>> 5.217079213762e+13 ||r(i)||/||b|| 5.015574157998e-03 >>> 24 KSP preconditioned resid norm 2.772587883161e+14 true resid norm >>> 6.149190087025e+13 ||r(i)||/||b|| 5.911683075799e-03 >>> 25 KSP preconditioned resid norm 2.446809593449e+14 true resid norm >>> 3.772191549633e+13 ||r(i)||/||b|| 3.626493997916e-03 >>> 26 KSP preconditioned resid norm 2.130893101456e+14 true resid norm >>> 5.402695231969e+13 ||r(i)||/||b|| 5.194020922190e-03 >>> 27 KSP preconditioned resid norm 1.886821885223e+14 true resid norm >>> 2.887742158299e+13 ||r(i)||/||b|| 2.776205679591e-03 >>> 28 KSP preconditioned resid norm 1.654479962436e+14 true resid norm >>> 4.199650238007e+13 ||r(i)||/||b|| 4.037442473714e-03 >>> 29 KSP preconditioned resid norm 1.453563006355e+14 true resid norm >>> 2.222396036725e+13 ||r(i)||/||b|| 2.136557961634e-03 >>> 30 KSP preconditioned resid norm 1.311692747504e+14 true resid norm >>> 2.933890714503e+13 ||r(i)||/||b|| 2.820571788757e-03 >>> 31 KSP preconditioned resid norm 1.173199137783e+14 true resid norm >>> 1.793346595553e+13 ||r(i)||/||b|| 1.724080174451e-03 >>> 32 KSP preconditioned resid norm 1.086053135569e+14 true resid norm >>> 2.009773461598e+13 ||r(i)||/||b|| 1.932147744820e-03 >>> 33 KSP preconditioned resid norm 9.929347161574e+13 true resid norm >>> 1.516318812509e+13 ||r(i)||/||b|| 1.457752343733e-03 >>> 34 KSP preconditioned resid norm 9.235712158960e+13 true resid norm >>> 1.538423395802e+13 ||r(i)||/||b|| 1.479003157109e-03 >>> 35 KSP preconditioned resid norm 8.718922541063e+13 true resid norm >>> 1.331044977413e+13 ||r(i)||/||b|| 1.279634546134e-03 >>> 36 KSP preconditioned resid norm 8.163106494702e+13 true resid norm >>> 1.369060193385e+13 ||r(i)||/||b|| 1.316181458118e-03 >>> 37 KSP preconditioned resid norm 7.751439049037e+13 true resid norm >>> 1.182966275510e+13 ||r(i)||/||b|| 1.137275252708e-03 >>> 38 KSP preconditioned resid norm 7.269591005999e+13 true resid norm >>> 1.225102361382e+13 ||r(i)||/||b|| 1.177783869649e-03 >>> 39 KSP preconditioned resid norm 6.809139852739e+13 true resid norm >>> 1.039056641301e+13 ||r(i)||/||b|| 9.989240004362e-04 >>> 40 KSP preconditioned resid norm 6.390023598760e+13 true resid norm >>> 1.042637257280e+13 ||r(i)||/||b|| 1.002366318301e-03 >>> 41 KSP preconditioned resid norm 5.919860950635e+13 true resid norm >>> 9.048648521202e+12 ||r(i)||/||b|| 8.699152500509e-04 >>> 42 KSP preconditioned resid norm 5.538975217584e+13 true resid norm >>> 8.775858022438e+12 ||r(i)||/||b|| 8.436898292725e-04 >>> 0 KSP preconditioned resid norm 2.200871087933e+16 true resid norm >>> 1.375121498750e+16 ||r(i)||/||b|| 1.042688751858e-01 >>> 1 KSP preconditioned resid norm 1.033852082172e+16 true resid norm >>> 1.769345381246e+15 ||r(i)||/||b|| 1.341609835098e-02 >>> 2 KSP preconditioned resid norm 8.201114088620e+15 true resid norm >>> 1.330146487456e+15 ||r(i)||/||b|| 1.008586355500e-02 >>> 3 KSP preconditioned resid norm 7.727276364323e+15 true resid norm >>> 1.292967289192e+15 ||r(i)||/||b|| 9.803951506731e-03 >>> 4 KSP preconditioned resid norm 7.788308934018e+15 true resid norm >>> 1.241309862269e+15 ||r(i)||/||b|| 9.412257986909e-03 >>> 5 KSP preconditioned resid norm 8.253685478294e+15 true resid norm >>> 1.290166199902e+15 ||r(i)||/||b|| 9.782712188615e-03 >>> 6 KSP preconditioned resid norm 9.476714658211e+15 true resid norm >>> 1.410015893632e+15 ||r(i)||/||b|| 1.069147499743e-02 >>> 7 KSP preconditioned resid norm 1.110541369388e+16 true resid norm >>> 1.990482855418e+15 ||r(i)||/||b|| 1.509287787295e-02 >>> 8 KSP preconditioned resid norm 1.149682905149e+16 true resid norm >>> 1.789313061240e+15 ||r(i)||/||b|| 1.356750370206e-02 >>> 9 KSP preconditioned resid norm 1.081618830942e+16 true resid norm >>> 1.972959789140e+15 ||r(i)||/||b|| 1.496000885648e-02 >>> 10 KSP preconditioned resid norm 8.800226542664e+15 true resid norm >>> 1.343393349540e+15 ||r(i)||/||b|| 1.018630816375e-02 >>> 11 KSP preconditioned resid norm 4.142701353678e+15 true resid norm >>> 6.194874024708e+14 ||r(i)||/||b|| 4.697276182950e-03 >>> 12 KSP preconditioned resid norm 3.300703256438e+15 true resid norm >>> 1.133601440349e+15 ||r(i)||/||b|| 8.595556625482e-03 >>> 13 KSP preconditioned resid norm 7.882205618989e+14 true resid norm >>> 1.186911081255e+14 ||r(i)||/||b|| 8.999778092374e-04 >>> 0 KSP preconditioned resid norm 1.928554431739e+16 true resid norm >>> 1.203381747345e+16 ||r(i)||/||b|| 4.946634802067e-01 >>> 1 KSP preconditioned resid norm 9.232243648078e+15 true resid norm >>> 1.583125971956e+15 ||r(i)||/||b|| 6.507615763833e-02 >>> 2 KSP preconditioned resid norm 7.541746545280e+15 true resid norm >>> 1.216565269124e+15 ||r(i)||/||b|| 5.000827137780e-02 >>> 3 KSP preconditioned resid norm 7.340199256082e+15 true resid norm >>> 1.219811880454e+15 ||r(i)||/||b|| 5.014172695523e-02 >>> 4 KSP preconditioned resid norm 7.655426776880e+15 true resid norm >>> 1.212245393966e+15 ||r(i)||/||b|| 4.983069809450e-02 >>> 5 KSP preconditioned resid norm 8.414958315915e+15 true resid norm >>> 1.311448634719e+15 ||r(i)||/||b|| 5.390855787813e-02 >>> 6 KSP preconditioned resid norm 1.008678778264e+16 true resid norm >>> 1.495852838926e+15 ||r(i)||/||b|| 6.148869823002e-02 >>> 7 KSP preconditioned resid norm 1.235685231672e+16 true resid norm >>> 2.203374571492e+15 ||r(i)||/||b|| 9.057216765485e-02 >>> 8 KSP preconditioned resid norm 1.311846317918e+16 true resid norm >>> 2.061677565336e+15 ||r(i)||/||b|| 8.474755428051e-02 >>> 9 KSP preconditioned resid norm 1.289737803434e+16 true resid norm >>> 2.355893652982e+15 ||r(i)||/||b|| 9.684163449812e-02 >>> 10 KSP preconditioned resid norm 1.151803850475e+16 true resid norm >>> 1.821777204240e+15 ||r(i)||/||b|| 7.488618254339e-02 >>> 11 KSP preconditioned resid norm 8.380110732685e+15 true resid norm >>> 1.434954901150e+15 ||r(i)||/||b|| 5.898542062058e-02 >>> 12 KSP preconditioned resid norm 3.553409074636e+15 true resid norm >>> 5.423523740165e+14 ||r(i)||/||b|| 2.229399884295e-02 >>> 13 KSP preconditioned resid norm 1.190458477552e+15 true resid norm >>> 2.162492452839e+14 ||r(i)||/||b|| 8.889166260017e-03 >>> 14 KSP preconditioned resid norm 9.902014249371e+14 true resid norm >>> 1.847474743657e+14 ||r(i)||/||b|| 7.594250854373e-03 >>> 15 KSP preconditioned resid norm 9.211725470007e+14 true resid norm >>> 1.442747306714e+14 ||r(i)||/||b|| 5.930573613672e-03 >>> 16 KSP preconditioned resid norm 8.405580663519e+14 true resid norm >>> 2.201247255509e+14 ||r(i)||/||b|| 9.048472196027e-03 >>> 17 KSP preconditioned resid norm 7.117085749555e+14 true resid norm >>> 1.108024781635e+14 ||r(i)||/||b|| 4.554659365975e-03 >>> 18 KSP preconditioned resid norm 6.347627750827e+14 true resid norm >>> 1.457120949776e+14 ||r(i)||/||b|| 5.989658075575e-03 >>> 19 KSP preconditioned resid norm 5.626851770343e+14 true resid norm >>> 8.750418282909e+13 ||r(i)||/||b|| 3.596956967843e-03 >>> 20 KSP preconditioned resid norm 4.865367652981e+14 true resid norm >>> 1.207991751614e+14 ||r(i)||/||b|| 4.965584738448e-03 >>> 21 KSP preconditioned resid norm 4.252070053539e+14 true resid norm >>> 6.577223220744e+13 ||r(i)||/||b|| 2.703640914986e-03 >>> 22 KSP preconditioned resid norm 3.618870286814e+14 true resid norm >>> 8.993615915309e+13 ||r(i)||/||b|| 3.696926065336e-03 >>> 23 KSP preconditioned resid norm 3.105522471772e+14 true resid norm >>> 4.771648977942e+13 ||r(i)||/||b|| 1.961439497451e-03 >>> 24 KSP preconditioned resid norm 2.735005931489e+14 true resid norm >>> 4.708110884633e+13 ||r(i)||/||b|| 1.935321455997e-03 >>> 25 KSP preconditioned resid norm 2.496548247596e+14 true resid norm >>> 3.887274137414e+13 ||r(i)||/||b|| 1.597907361960e-03 >>> 26 KSP preconditioned resid norm 2.329765393701e+14 true resid norm >>> 3.903456593754e+13 ||r(i)||/||b|| 1.604559341009e-03 >>> 27 KSP preconditioned resid norm 2.169265400605e+14 true resid norm >>> 3.334703074182e+13 ||r(i)||/||b|| 1.370766867430e-03 >>> 28 KSP preconditioned resid norm 1.998370208104e+14 true resid norm >>> 3.474062054769e+13 ||r(i)||/||b|| 1.428051929704e-03 >>> 29 KSP preconditioned resid norm 1.805920498739e+14 true resid norm >>> 2.773007978134e+13 ||r(i)||/||b|| 1.139875837515e-03 >>> 30 KSP preconditioned resid norm 1.643922523062e+14 true resid norm >>> 2.817409671160e+13 ||r(i)||/||b|| 1.158127648337e-03 >>> 31 KSP preconditioned resid norm 1.456868079593e+14 true resid norm >>> 2.236751337771e+13 ||r(i)||/||b|| 9.194415683468e-04 >>> 32 KSP preconditioned resid norm 1.315816507773e+14 true resid norm >>> 2.157979387266e+13 ||r(i)||/||b|| 8.870614800950e-04 >>> 0 KSP preconditioned resid norm 6.130010040591e+16 true resid norm >>> 4.019535303993e+16 ||r(i)||/||b|| 5.748506002867e-02 >>> 1 KSP preconditioned resid norm 2.383231267167e+16 true resid norm >>> 4.333028232365e+15 ||r(i)||/||b|| 6.196845386481e-03 >>> 2 KSP preconditioned resid norm 1.380314332389e+16 true resid norm >>> 2.902938687246e+15 ||r(i)||/||b|| 4.151614355275e-03 >>> 3 KSP preconditioned resid norm 9.355423030603e+15 true resid norm >>> 1.914374573200e+15 ||r(i)||/||b|| 2.737827359010e-03 >>> 4 KSP preconditioned resid norm 6.905389736952e+15 true resid norm >>> 1.160495154173e+15 ||r(i)||/||b|| 1.659672786910e-03 >>> 5 KSP preconditioned resid norm 5.388850338707e+15 true resid norm >>> 9.343171842551e+14 ||r(i)||/||b|| 1.336206187053e-03 >>> 6 KSP preconditioned resid norm 4.456941671648e+15 true resid norm >>> 7.443205431054e+14 ||r(i)||/||b|| 1.064484022780e-03 >>> 7 KSP preconditioned resid norm 3.604897929141e+15 true resid norm >>> 7.177355376747e+14 ||r(i)||/||b|| 1.026463691635e-03 >>> 8 KSP preconditioned resid norm 2.616552572944e+15 true resid norm >>> 4.566784838372e+14 ||r(i)||/||b|| 6.531150511630e-04 >>> 9 KSP preconditioned resid norm 1.730753896972e+15 true resid norm >>> 3.416577764278e+14 ||r(i)||/||b|| 4.886191139484e-04 >>> 10 KSP preconditioned resid norm 9.605912905151e+14 true resid norm >>> 1.715828170206e+14 ||r(i)||/||b|| 2.453877821777e-04 >>> 11 KSP preconditioned resid norm 3.885469679759e+14 true resid norm >>> 7.526615556743e+13 ||r(i)||/||b|| 1.076412854646e-04 >>> 12 KSP preconditioned resid norm 1.085402187481e+14 true resid norm >>> 1.877491024324e+13 ||r(i)||/||b|| 2.685078649001e-05 >>> 13 KSP preconditioned resid norm 5.521537327924e+13 true resid norm >>> 1.482832280216e+13 ||r(i)||/||b|| 2.120660628506e-05 >>> 14 KSP preconditioned resid norm 4.327780981098e+13 true resid norm >>> 6.949078996691e+12 ||r(i)||/||b|| 9.938169292156e-06 >>> 0 KSP preconditioned resid norm 2.469228993807e+16 true resid norm >>> 1.581397706747e+16 ||r(i)||/||b|| 1.421322431696e-01 >>> 1 KSP preconditioned resid norm 1.094866250702e+16 true resid norm >>> 1.827538407052e+15 ||r(i)||/||b|| 1.642547805430e-02 >>> 2 KSP preconditioned resid norm 7.902646639796e+15 true resid norm >>> 1.331972302022e+15 ||r(i)||/||b|| 1.197144844200e-02 >>> 3 KSP preconditioned resid norm 6.757113896005e+15 true resid norm >>> 1.177404713092e+15 ||r(i)||/||b|| 1.058223192536e-02 >>> 4 KSP preconditioned resid norm 6.236819639281e+15 true resid norm >>> 9.858933018833e+14 ||r(i)||/||b|| 8.860973171059e-03 >>> 5 KSP preconditioned resid norm 6.056207345085e+15 true resid norm >>> 9.498733496025e+14 ||r(i)||/||b|| 8.537234455953e-03 >>> 6 KSP preconditioned resid norm 6.281149615757e+15 true resid norm >>> 9.439016000829e+14 ||r(i)||/||b|| 8.483561799717e-03 >>> 7 KSP preconditioned resid norm 6.511636840454e+15 true resid norm >>> 1.175604965975e+15 ||r(i)||/||b|| 1.056605622877e-02 >>> 8 KSP preconditioned resid norm 5.897282381771e+15 true resid norm >>> 9.358325816295e+14 ||r(i)||/||b|| 8.411039391972e-03 >>> 9 KSP preconditioned resid norm 4.858643828149e+15 true resid norm >>> 9.074074922507e+14 ||r(i)||/||b|| 8.155561487934e-03 >>> 10 KSP preconditioned resid norm 3.431226959566e+15 true resid norm >>> 5.523076022464e+14 ||r(i)||/||b|| 4.964008616682e-03 >>> 11 KSP preconditioned resid norm 1.790601246330e+15 true resid norm >>> 3.233761759516e+14 ||r(i)||/||b|| 2.906427717678e-03 >>> 12 KSP preconditioned resid norm 6.061504971272e+14 true resid norm >>> 9.361691978333e+13 ||r(i)||/||b|| 8.414064817893e-04 >>> 13 KSP preconditioned resid norm 3.669164519745e+14 true resid norm >>> 7.433339410578e+13 ||r(i)||/||b|| 6.680907656304e-04 >>> 14 KSP preconditioned resid norm 1.028448839513e+15 true resid norm >>> 1.739193394480e+14 ||r(i)||/||b|| 1.563145421348e-03 >>> 15 KSP preconditioned resid norm 1.340577188948e+14 true resid norm >>> 2.477633902186e+13 ||r(i)||/||b|| 2.226838086134e-04 >>> 16 KSP preconditioned resid norm 1.124181660974e+14 true resid norm >>> 1.917872742131e+13 ||r(i)||/||b|| 1.723738144997e-04 >>> 17 KSP preconditioned resid norm 9.394070158671e+13 true resid norm >>> 1.682505899954e+13 ||r(i)||/||b|| 1.512196057236e-04 >>> 18 KSP preconditioned resid norm 9.034650415597e+13 true resid norm >>> 1.892962426081e+13 ||r(i)||/||b|| 1.701349348787e-04 >>> 19 KSP preconditioned resid norm 8.058877605551e+13 true resid norm >>> 1.275341142002e+13 ||r(i)||/||b|| 1.146246112195e-04 >>> 20 KSP preconditioned resid norm 6.303186171389e+13 true resid norm >>> 1.900109222650e+13 ||r(i)||/||b|| 1.707772718592e-04 >>> 21 KSP preconditioned resid norm 4.601610061053e+13 true resid norm >>> 6.959914424356e+12 ||r(i)||/||b|| 6.255404603042e-05 >>> 22 KSP preconditioned resid norm 2.822582332621e+13 true resid norm >>> 1.013252988889e+13 ||r(i)||/||b|| 9.106875493415e-05 >>> 23 KSP preconditioned resid norm 1.858033653578e+13 true resid norm >>> 2.785463510953e+12 ||r(i)||/||b|| 2.503507975190e-05 >>> 24 KSP preconditioned resid norm 9.505670647827e+12 true resid norm >>> 4.268589348463e+12 ||r(i)||/||b|| 3.836506001486e-05 >>> 25 KSP preconditioned resid norm 5.844140386668e+12 true resid norm >>> 8.921656809030e+11 ||r(i)||/||b|| 8.018571733391e-06 >>> 0 SNES Function norm 0.000000000000e+00 >>> 3 TS dt 5e-12 time 9e-12 >>> 0 KSP preconditioned resid norm 2.220122692212e+19 true resid norm >>> 1.204173138214e+19 ||r(i)||/||b|| 2.302144004449e+02 >>> 1 KSP preconditioned resid norm 5.389791318038e+18 true resid norm >>> 1.775764523088e+18 ||r(i)||/||b|| 3.394915166605e+01 >>> 2 KSP preconditioned resid norm 2.218229032455e+18 true resid norm >>> 5.319501602511e+17 ||r(i)||/||b|| 1.016984878025e+01 >>> 3 KSP preconditioned resid norm 1.401621992174e+18 true resid norm >>> 2.963396938661e+17 ||r(i)||/||b|| 5.665436537857e+00 >>> 4 KSP preconditioned resid norm 1.004007060031e+18 true resid norm >>> 1.656144361093e+17 ||r(i)||/||b|| 3.166224764861e+00 >>> 5 KSP preconditioned resid norm 8.672273682794e+17 true resid norm >>> 1.510259905938e+17 ||r(i)||/||b|| 2.887322160975e+00 >>> 6 KSP preconditioned resid norm 7.388939161376e+17 true resid norm >>> 1.526077665500e+17 ||r(i)||/||b|| 2.917562629878e+00 >>> 7 KSP preconditioned resid norm 5.565893055963e+17 true resid norm >>> 1.017298978392e+17 ||r(i)||/||b|| 1.944877085792e+00 >>> 8 KSP preconditioned resid norm 3.566491384538e+17 true resid norm >>> 6.306517676931e+16 ||r(i)||/||b|| 1.205683086440e+00 >>> 9 KSP preconditioned resid norm 1.893313432840e+17 true resid norm >>> 3.267997700517e+16 ||r(i)||/||b|| 6.247773741206e-01 >>> 10 KSP preconditioned resid norm 6.895898463878e+16 true resid norm >>> 1.153123828824e+16 ||r(i)||/||b|| 2.204547688925e-01 >>> 11 KSP preconditioned resid norm 5.104999855255e+16 true resid norm >>> 9.275473058095e+15 ||r(i)||/||b|| 1.773289405940e-01 >>> 12 KSP preconditioned resid norm 4.992747088606e+16 true resid norm >>> 8.399049844320e+15 ||r(i)||/||b|| 1.605734393881e-01 >>> 13 KSP preconditioned resid norm 4.920015136673e+16 true resid norm >>> 8.426577623649e+15 ||r(i)||/||b|| 1.610997167989e-01 >>> 14 KSP preconditioned resid norm 5.688370526613e+16 true resid norm >>> 9.763030126011e+15 ||r(i)||/||b|| 1.866500800972e-01 >>> 15 KSP preconditioned resid norm 1.613946025228e+17 true resid norm >>> 4.341380499600e+16 ||r(i)||/||b|| 8.299872145473e-01 >>> 16 KSP preconditioned resid norm 4.125620543313e+16 true resid norm >>> 7.099475531667e+15 ||r(i)||/||b|| 1.357281150965e-01 >>> 17 KSP preconditioned resid norm 2.163115342752e+16 true resid norm >>> 5.850497008723e+15 ||r(i)||/||b|| 1.118500835491e-01 >>> 18 KSP preconditioned resid norm 1.414953599369e+16 true resid norm >>> 2.432122296311e+15 ||r(i)||/||b|| 4.649743118207e-02 >>> 19 KSP preconditioned resid norm 1.088056550362e+16 true resid norm >>> 2.488538030816e+15 ||r(i)||/||b|| 4.757598991109e-02 >>> 20 KSP preconditioned resid norm 9.058853458567e+15 true resid norm >>> 1.587978910975e+15 ||r(i)||/||b|| 3.035905729067e-02 >>> 21 KSP preconditioned resid norm 7.813748136791e+15 true resid norm >>> 1.345999130102e+15 ||r(i)||/||b|| 2.573287618717e-02 >>> 22 KSP preconditioned resid norm 6.941551759120e+15 true resid norm >>> 1.184314230023e+15 ||r(i)||/||b|| 2.264177648138e-02 >>> 23 KSP preconditioned resid norm 6.244923673699e+15 true resid norm >>> 1.023439246806e+15 ||r(i)||/||b|| 1.956616080515e-02 >>> 24 KSP preconditioned resid norm 5.681880462174e+15 true resid norm >>> 9.491910634069e+14 ||r(i)||/||b|| 1.814668045943e-02 >>> 25 KSP preconditioned resid norm 5.283519017995e+15 true resid norm >>> 8.557005279630e+14 ||r(i)||/||b|| 1.635932390068e-02 >>> 26 KSP preconditioned resid norm 5.117675200632e+15 true resid norm >>> 8.569254569155e+14 ||r(i)||/||b|| 1.638274215138e-02 >>> 27 KSP preconditioned resid norm 4.944059018035e+15 true resid norm >>> 9.547778379458e+14 ||r(i)||/||b|| 1.825348868410e-02 >>> 28 KSP preconditioned resid norm 4.840428844487e+15 true resid norm >>> 7.569152804275e+14 ||r(i)||/||b|| 1.447074278120e-02 >>> 29 KSP preconditioned resid norm 4.909458338739e+15 true resid norm >>> 7.780937746322e+14 ||r(i)||/||b|| 1.487563425328e-02 >>> 30 KSP preconditioned resid norm 5.172548892535e+15 true resid norm >>> 8.119476371329e+14 ||r(i)||/||b|| 1.552285402683e-02 >>> 31 KSP preconditioned resid norm 5.734458283292e+15 true resid norm >>> 9.082595641858e+14 ||r(i)||/||b|| 1.736415008622e-02 >>> 32 KSP preconditioned resid norm 8.602942487649e+15 true resid norm >>> 1.383459272467e+15 ||r(i)||/||b|| 2.644904099284e-02 >>> 33 KSP preconditioned resid norm 1.681001901654e+16 true resid norm >>> 2.705594423024e+15 ||r(i)||/||b|| 5.172568446988e-02 >>> 34 KSP preconditioned resid norm 3.498037913905e+16 true resid norm >>> 5.724738344713e+15 ||r(i)||/||b|| 1.094458233545e-01 >>> 35 KSP preconditioned resid norm 6.660621055336e+15 true resid norm >>> 1.092449655644e+15 ||r(i)||/||b|| 2.088550512457e-02 >>> 36 KSP preconditioned resid norm 3.564205941873e+15 true resid norm >>> 5.832825274169e+14 ||r(i)||/||b|| 1.115122344769e-02 >>> 37 KSP preconditioned resid norm 2.275128811690e+15 true resid norm >>> 4.150874600928e+14 ||r(i)||/||b|| 7.935662050992e-03 >>> 38 KSP preconditioned resid norm 1.923021972098e+15 true resid norm >>> 3.105449109783e+14 ||r(i)||/||b|| 5.937012562674e-03 >>> 39 KSP preconditioned resid norm 1.756400018558e+15 true resid norm >>> 3.028037938881e+14 ||r(i)||/||b|| 5.789017513361e-03 >>> 40 KSP preconditioned resid norm 1.618977002834e+15 true resid norm >>> 2.591473194248e+14 ||r(i)||/||b|| 4.954390932253e-03 >>> 41 KSP preconditioned resid norm 1.449378745707e+15 true resid norm >>> 2.366294470672e+14 ||r(i)||/||b|| 4.523893164149e-03 >>> 42 KSP preconditioned resid norm 1.311727507569e+15 true resid norm >>> 2.162961405943e+14 ||r(i)||/||b|| 4.135160031831e-03 >>> 43 KSP preconditioned resid norm 1.220101213287e+15 true resid norm >>> 1.960093358179e+14 ||r(i)||/||b|| 3.747315921186e-03 >>> 44 KSP preconditioned resid norm 1.141924730669e+15 true resid norm >>> 1.884563341884e+14 ||r(i)||/||b|| 3.602917272310e-03 >>> 45 KSP preconditioned resid norm 1.062154013672e+15 true resid norm >>> 2.268601642641e+14 ||r(i)||/||b|| 4.337123545072e-03 >>> 46 KSP preconditioned resid norm 1.050847341809e+15 true resid norm >>> 1.644035317108e+14 ||r(i)||/||b|| 3.143074636258e-03 >>> 47 KSP preconditioned resid norm 8.268920862881e+14 true resid norm >>> 2.491484388741e+14 ||r(i)||/||b|| 4.763231852378e-03 >>> 48 KSP preconditioned resid norm 6.970056044801e+14 true resid norm >>> 1.086553600705e+14 ||r(i)||/||b|| 2.077278406232e-03 >>> 49 KSP preconditioned resid norm 8.789823309588e+14 true resid norm >>> 1.565701303869e+14 ||r(i)||/||b|| 2.993315292522e-03 >>> 50 KSP preconditioned resid norm 9.889574125697e+14 true resid norm >>> 1.557528361421e+14 ||r(i)||/||b|| 2.977690221792e-03 >>> 51 KSP preconditioned resid norm 3.685341349551e+14 true resid norm >>> 5.749337875960e+13 ||r(i)||/||b|| 1.099161183775e-03 >>> 52 KSP preconditioned resid norm 2.728364862859e+15 true resid norm >>> 4.745949443528e+14 ||r(i)||/||b|| 9.073329000715e-03 >>> 53 KSP preconditioned resid norm 8.484691724076e+15 true resid norm >>> 1.333040899485e+15 ||r(i)||/||b|| 2.548514011022e-02 >>> 54 KSP preconditioned resid norm 8.341541248747e+13 true resid norm >>> 1.303089350822e+13 ||r(i)||/||b|| 2.491252496055e-04 >>> 0 KSP preconditioned resid norm 6.542890621126e+18 true resid norm >>> 3.545938224251e+18 ||r(i)||/||b|| 4.156801777589e+02 >>> 1 KSP preconditioned resid norm 1.591173149842e+18 true resid norm >>> 5.223204285160e+17 ||r(i)||/||b|| 6.123012721646e+01 >>> 2 KSP preconditioned resid norm 6.570590492388e+17 true resid norm >>> 1.552517142001e+17 ||r(i)||/||b|| 1.819971360886e+01 >>> 3 KSP preconditioned resid norm 4.165858181866e+17 true resid norm >>> 8.738575659546e+16 ||r(i)||/||b|| 1.024398185698e+01 >>> 4 KSP preconditioned resid norm 2.995041215234e+17 true resid norm >>> 4.934730887237e+16 ||r(i)||/||b|| 5.784843622966e+00 >>> 5 KSP preconditioned resid norm 2.594520327186e+17 true resid norm >>> 4.469839192099e+16 ||r(i)||/||b|| 5.239864409419e+00 >>> 6 KSP preconditioned resid norm 2.219632225075e+17 true resid norm >>> 4.652171577038e+16 ||r(i)||/||b|| 5.453607439866e+00 >>> 7 KSP preconditioned resid norm 1.674928029034e+17 true resid norm >>> 2.988841295071e+16 ||r(i)||/||b|| 3.503733010156e+00 >>> 8 KSP preconditioned resid norm 1.081911414020e+17 true resid norm >>> 1.908165723430e+16 ||r(i)||/||b|| 2.236888002402e+00 >>> 9 KSP preconditioned resid norm 5.710171826154e+16 true resid norm >>> 9.635278906551e+15 ||r(i)||/||b|| 1.129516137997e+00 >>> 10 KSP preconditioned resid norm 1.779377501866e+16 true resid norm >>> 2.952494296810e+15 ||r(i)||/||b|| 3.461124465555e-01 >>> 11 KSP preconditioned resid norm 1.125612460139e+16 true resid norm >>> 2.030403352863e+15 ||r(i)||/||b|| 2.380183672880e-01 >>> 12 KSP preconditioned resid norm 1.086634567103e+16 true resid norm >>> 1.877885131752e+15 ||r(i)||/||b|| 2.201390932416e-01 >>> 13 KSP preconditioned resid norm 1.075395224737e+16 true resid norm >>> 1.822688133743e+15 ||r(i)||/||b|| 2.136685073225e-01 >>> 14 KSP preconditioned resid norm 1.173925328196e+16 true resid norm >>> 2.130469004546e+15 ||r(i)||/||b|| 2.497487769141e-01 >>> 15 KSP preconditioned resid norm 4.944016549258e+16 true resid norm >>> 1.361939461929e+16 ||r(i)||/||b|| 1.596562607209e+00 >>> 16 KSP preconditioned resid norm 1.780247590037e+17 true resid norm >>> 3.370503336823e+16 ||r(i)||/||b|| 3.951144485840e+00 >>> 17 KSP preconditioned resid norm 3.251104571145e+16 true resid norm >>> 8.719659549123e+15 ||r(i)||/||b|| 1.022180704274e+00 >>> 18 KSP preconditioned resid norm 2.075437272594e+16 true resid norm >>> 3.944434594682e+15 ||r(i)||/||b|| 4.623947654424e-01 >>> 19 KSP preconditioned resid norm 1.483444135710e+16 true resid norm >>> 3.390829411015e+15 ||r(i)||/||b|| 3.974972160206e-01 >>> 20 KSP preconditioned resid norm 1.134313319987e+16 true resid norm >>> 2.489467119154e+15 ||r(i)||/||b|| 2.918330972428e-01 >>> 21 KSP preconditioned resid norm 8.553477032620e+15 true resid norm >>> 1.657674168535e+15 ||r(i)||/||b|| 1.943243930000e-01 >>> 22 KSP preconditioned resid norm 6.586595023874e+15 true resid norm >>> 1.286251502369e+15 ||r(i)||/||b|| 1.507835780925e-01 >>> 23 KSP preconditioned resid norm 4.780400299659e+15 true resid norm >>> 9.064518348736e+14 ||r(i)||/||b|| 1.062607513219e-01 >>> 24 KSP preconditioned resid norm 3.028073960021e+15 true resid norm >>> 6.179142836679e+14 ||r(i)||/||b|| 7.243632094834e-02 >>> 25 KSP preconditioned resid norm 1.570529076731e+15 true resid norm >>> 2.755631911412e+14 ||r(i)||/||b|| 3.230348331903e-02 >>> 26 KSP preconditioned resid norm 1.180384548694e+15 true resid norm >>> 2.278461878994e+14 ||r(i)||/||b|| 2.670975575377e-02 >>> 27 KSP preconditioned resid norm 1.075603583388e+15 true resid norm >>> 2.036512758236e+14 ||r(i)||/||b|| 2.387345553745e-02 >>> 28 KSP preconditioned resid norm 1.013485206919e+15 true resid norm >>> 1.671016982388e+14 ||r(i)||/||b|| 1.958885328364e-02 >>> 29 KSP preconditioned resid norm 9.469966879519e+14 true resid norm >>> 1.801244790801e+14 ||r(i)||/||b|| 2.111547656715e-02 >>> 30 KSP preconditioned resid norm 8.991813841494e+14 true resid norm >>> 1.459100485336e+14 ||r(i)||/||b|| 1.710461690970e-02 >>> 31 KSP preconditioned resid norm 8.677534755734e+14 true resid norm >>> 1.581142867341e+14 ||r(i)||/||b|| 1.853528478482e-02 >>> 32 KSP preconditioned resid norm 8.336211910215e+14 true resid norm >>> 1.336646371267e+14 ||r(i)||/||b|| 1.566912241756e-02 >>> 33 KSP preconditioned resid norm 8.025169227655e+14 true resid norm >>> 1.586001394574e+14 ||r(i)||/||b|| 1.859223990744e-02 >>> 34 KSP preconditioned resid norm 7.724033526395e+14 true resid norm >>> 1.340849232530e+14 ||r(i)||/||b|| 1.571839135589e-02 >>> 35 KSP preconditioned resid norm 7.664262315912e+14 true resid norm >>> 1.204381005271e+14 ||r(i)||/||b|| 1.411861343033e-02 >>> 36 KSP preconditioned resid norm 7.727266103384e+14 true resid norm >>> 1.292430217432e+14 ||r(i)||/||b|| 1.515078911553e-02 >>> 37 KSP preconditioned resid norm 8.047214262262e+14 true resid norm >>> 1.267167533285e+14 ||r(i)||/||b|| 1.485464190786e-02 >>> 38 KSP preconditioned resid norm 8.419506152984e+14 true resid norm >>> 1.380565821207e+14 ||r(i)||/||b|| 1.618397754485e-02 >>> 39 KSP preconditioned resid norm 8.796981754885e+14 true resid norm >>> 1.388509364066e+14 ||r(i)||/||b|| 1.627709742171e-02 >>> 40 KSP preconditioned resid norm 9.432799067464e+14 true resid norm >>> 1.525727137977e+14 ||r(i)||/||b|| 1.788566206789e-02 >>> 41 KSP preconditioned resid norm 1.193594717717e+15 true resid norm >>> 1.887759239390e+14 ||r(i)||/||b|| 2.212966065874e-02 >>> 42 KSP preconditioned resid norm 2.117323854280e+15 true resid norm >>> 3.659661973675e+14 ||r(i)||/||b|| 4.290116870483e-02 >>> 43 KSP preconditioned resid norm 1.749515464589e+16 true resid norm >>> 2.774904343458e+15 ||r(i)||/||b|| 3.252940851772e-01 >>> 44 KSP preconditioned resid norm 1.219803820486e+15 true resid norm >>> 2.038867414461e+14 ||r(i)||/||b|| 2.390105849768e-02 >>> 45 KSP preconditioned resid norm 6.218668840796e+14 true resid norm >>> 9.918031040094e+13 ||r(i)||/||b|| 1.162662360435e-02 >>> 46 KSP preconditioned resid norm 4.213016096566e+14 true resid norm >>> 7.032265378826e+13 ||r(i)||/||b|| 8.243723206248e-03 >>> 47 KSP preconditioned resid norm 2.932264460599e+14 true resid norm >>> 4.681093096736e+13 ||r(i)||/||b|| 5.487511308712e-03 >>> 48 KSP preconditioned resid norm 1.890454719223e+14 true resid norm >>> 3.330632990756e+13 ||r(i)||/||b|| 3.904405621560e-03 >>> 49 KSP preconditioned resid norm 1.361587907879e+14 true resid norm >>> 2.180429132446e+13 ||r(i)||/||b|| 2.556054595557e-03 >>> 50 KSP preconditioned resid norm 1.041049830937e+14 true resid norm >>> 1.811134568183e+13 ||r(i)||/||b|| 2.123141159366e-03 >>> 51 KSP preconditioned resid norm 9.375298557199e+13 true resid norm >>> 1.575768352161e+13 ||r(i)||/||b|| 1.847228088333e-03 >>> 52 KSP preconditioned resid norm 1.220175150775e+14 true resid norm >>> 1.950548748269e+13 ||r(i)||/||b|| 2.286572408009e-03 >>> 53 KSP preconditioned resid norm 7.051895161295e+13 true resid norm >>> 1.295324819768e+13 ||r(i)||/||b|| 1.518472170932e-03 >>> 54 KSP preconditioned resid norm 5.292501274648e+13 true resid norm >>> 8.611444708372e+12 ||r(i)||/||b|| 1.009494988563e-03 >>> 0 KSP preconditioned resid norm 1.892610202067e+19 true resid norm >>> 1.028308040523e+19 ||r(i)||/||b|| 7.841022917690e+01 >>> 1 KSP preconditioned resid norm 4.581169591238e+18 true resid norm >>> 1.519782710159e+18 ||r(i)||/||b|| 1.158860048804e+01 >>> 2 KSP preconditioned resid norm 1.874457569158e+18 true resid norm >>> 4.646597685037e+17 ||r(i)||/||b|| 3.543109409036e+00 >>> 3 KSP preconditioned resid norm 1.173257076173e+18 true resid norm >>> 2.550995626245e+17 ||r(i)||/||b|| 1.945177357373e+00 >>> 4 KSP preconditioned resid norm 8.301427099842e+17 true resid norm >>> 1.381385637907e+17 ||r(i)||/||b|| 1.053329937932e+00 >>> 5 KSP preconditioned resid norm 7.053087339556e+17 true resid norm >>> 1.287851939525e+17 ||r(i)||/||b|| 9.820089092432e-01 >>> 6 KSP preconditioned resid norm 5.863912202870e+17 true resid norm >>> 1.166954217660e+17 ||r(i)||/||b|| 8.898223493329e-01 >>> 7 KSP preconditioned resid norm 4.330693111087e+17 true resid norm >>> 8.611440299479e+16 ||r(i)||/||b|| 6.566369033559e-01 >>> 8 KSP preconditioned resid norm 2.702458300388e+17 true resid norm >>> 4.803465439349e+16 ||r(i)||/||b|| 3.662723727716e-01 >>> 9 KSP preconditioned resid norm 1.546904751775e+17 true resid norm >>> 2.878502712633e+16 ||r(i)||/||b|| 2.194907055953e-01 >>> 10 KSP preconditioned resid norm 9.910545254967e+16 true resid norm >>> 1.648892832929e+16 ||r(i)||/||b|| 1.257308703453e-01 >>> 11 KSP preconditioned resid norm 9.242232753221e+16 true resid norm >>> 1.559261342084e+16 ||r(i)||/||b|| 1.188963174082e-01 >>> 12 KSP preconditioned resid norm 9.244969774564e+16 true resid norm >>> 1.506990436290e+16 ||r(i)||/||b|| 1.149105723385e-01 >>> 13 KSP preconditioned resid norm 7.091380549056e+16 true resid norm >>> 1.150476534699e+16 ||r(i)||/||b|| 8.772578370826e-02 >>> 14 KSP preconditioned resid norm 2.015813147523e+17 true resid norm >>> 3.265267485752e+16 ||r(i)||/||b|| 2.489821743992e-01 >>> 15 KSP preconditioned resid norm 1.668693825394e+16 true resid norm >>> 4.392881515438e+15 ||r(i)||/||b|| 3.349646533904e-02 >>> 16 KSP preconditioned resid norm 1.171106201192e+16 true resid norm >>> 1.864120995445e+15 ||r(i)||/||b|| 1.421423821522e-02 >>> 17 KSP preconditioned resid norm 1.020170882678e+16 true resid norm >>> 1.999245817269e+15 ||r(i)||/||b|| 1.524458786039e-02 >>> 18 KSP preconditioned resid norm 9.407404703690e+15 true resid norm >>> 1.539877936168e+15 ||r(i)||/||b|| 1.174182998880e-02 >>> 19 KSP preconditioned resid norm 8.812299187434e+15 true resid norm >>> 1.424937011002e+15 ||r(i)||/||b|| 1.086538597310e-02 >>> 20 KSP preconditioned resid norm 8.313368595823e+15 true resid norm >>> 1.488773269648e+15 ||r(i)||/||b|| 1.135214825376e-02 >>> 21 KSP preconditioned resid norm 7.851626197782e+15 true resid norm >>> 1.250085440746e+15 ||r(i)||/||b|| 9.532113144783e-03 >>> 22 KSP preconditioned resid norm 7.455044400782e+15 true resid norm >>> 1.324254107888e+15 ||r(i)||/||b|| 1.009766178966e-02 >>> 23 KSP preconditioned resid norm 7.129687083603e+15 true resid norm >>> 1.124533100660e+15 ||r(i)||/||b|| 8.574755293646e-03 >>> 24 KSP preconditioned resid norm 5.784281419699e+15 true resid norm >>> 2.057123935555e+15 ||r(i)||/||b|| 1.568591831199e-02 >>> 25 KSP preconditioned resid norm 4.924869719264e+15 true resid norm >>> 7.825081154476e+14 ||r(i)||/||b|| 5.966756871199e-03 >>> 26 KSP preconditioned resid norm 5.016674985414e+15 true resid norm >>> 7.817733101871e+14 ||r(i)||/||b|| 5.961153856674e-03 >>> 27 KSP preconditioned resid norm 5.308634596254e+15 true resid norm >>> 8.859615986698e+14 ||r(i)||/||b|| 6.755607197068e-03 >>> 28 KSP preconditioned resid norm 9.149615612203e+15 true resid norm >>> 1.429126026168e+15 ||r(i)||/||b|| 1.089732792301e-02 >>> 29 KSP preconditioned resid norm 8.897313413659e+15 true resid norm >>> 2.002400213933e+15 ||r(i)||/||b|| 1.526864066904e-02 >>> 30 KSP preconditioned resid norm 2.145204463192e+15 true resid norm >>> 3.349465461999e+14 ||r(i)||/||b|| 2.554024126484e-03 >>> 31 KSP preconditioned resid norm 1.279996253934e+15 true resid norm >>> 2.794251271078e+14 ||r(i)||/||b|| 2.130663905259e-03 >>> 32 KSP preconditioned resid norm 8.936440443654e+14 true resid norm >>> 1.396095375555e+14 ||r(i)||/||b|| 1.064546361948e-03 >>> 0 KSP preconditioned resid norm 6.470811776559e+18 true resid norm >>> 3.511555308471e+18 ||r(i)||/||b|| 1.662220839803e+02 >>> 1 KSP preconditioned resid norm 1.571601370240e+18 true resid norm >>> 5.187483735196e+17 ||r(i)||/||b|| 2.455534033589e+01 >>> 2 KSP preconditioned resid norm 6.468356854753e+17 true resid norm >>> 1.566993981909e+17 ||r(i)||/||b|| 7.417482635946e+00 >>> 3 KSP preconditioned resid norm 4.084058718904e+17 true resid norm >>> 8.688424998267e+16 ||r(i)||/||b|| 4.112730636007e+00 >>> 4 KSP preconditioned resid norm 2.920904676538e+17 true resid norm >>> 4.831974981420e+16 ||r(i)||/||b|| 2.287251319137e+00 >>> 5 KSP preconditioned resid norm 2.515876340477e+17 true resid norm >>> 4.435582123557e+16 ||r(i)||/||b|| 2.099615809737e+00 >>> 6 KSP preconditioned resid norm 2.133089016156e+17 true resid norm >>> 4.355659888860e+16 ||r(i)||/||b|| 2.061784025127e+00 >>> 7 KSP preconditioned resid norm 1.601295948028e+17 true resid norm >>> 2.999342671961e+16 ||r(i)||/||b|| 1.419761176199e+00 >>> 8 KSP preconditioned resid norm 1.019894482623e+17 true resid norm >>> 1.811123232414e+16 ||r(i)||/||b|| 8.573086612382e-01 >>> 9 KSP preconditioned resid norm 5.552358385594e+16 true resid norm >>> 9.849609129144e+15 ||r(i)||/||b|| 4.662385786400e-01 >>> 10 KSP preconditioned resid norm 2.614302078211e+16 true resid norm >>> 4.400897857400e+15 ||r(i)||/||b|| 2.083197754216e-01 >>> 11 KSP preconditioned resid norm 2.230412831734e+16 true resid norm >>> 4.001989581448e+15 ||r(i)||/||b|| 1.894371552944e-01 >>> 12 KSP preconditioned resid norm 2.202852939300e+16 true resid norm >>> 3.668964514289e+15 ||r(i)||/||b|| 1.736731658885e-01 >>> 13 KSP preconditioned resid norm 2.171413942501e+16 true resid norm >>> 3.746648405904e+15 ||r(i)||/||b|| 1.773503907139e-01 >>> 14 KSP preconditioned resid norm 2.756569557917e+16 true resid norm >>> 4.738274109991e+15 ||r(i)||/||b|| 2.242897314282e-01 >>> 15 KSP preconditioned resid norm 1.911125733606e+16 true resid norm >>> 7.273426207009e+15 ||r(i)||/||b|| 3.442930427122e-01 >>> 16 KSP preconditioned resid norm 8.267301781208e+15 true resid norm >>> 1.450502353903e+15 ||r(i)||/||b|| 6.866060844958e-02 >>> 17 KSP preconditioned resid norm 5.452175853074e+15 true resid norm >>> 1.779140334910e+15 ||r(i)||/||b|| 8.421693186737e-02 >>> 18 KSP preconditioned resid norm 4.185422899260e+15 true resid norm >>> 7.352001585133e+14 ||r(i)||/||b|| 3.480124667150e-02 >>> 19 KSP preconditioned resid norm 3.483100692842e+15 true resid norm >>> 7.680069742951e+14 ||r(i)||/||b|| 3.635418171280e-02 >>> 20 KSP preconditioned resid norm 3.028709630445e+15 true resid norm >>> 5.664408475792e+14 ||r(i)||/||b|| 2.681289909034e-02 >>> 21 KSP preconditioned resid norm 2.650986791756e+15 true resid norm >>> 5.333477625647e+14 ||r(i)||/||b|| 2.524641328184e-02 >>> 22 KSP preconditioned resid norm 2.409785359425e+15 true resid norm >>> 3.913611824173e+14 ||r(i)||/||b|| 1.852537283791e-02 >>> 23 KSP preconditioned resid norm 2.246473514462e+15 true resid norm >>> 3.515886291592e+14 ||r(i)||/||b|| 1.664270942896e-02 >>> 24 KSP preconditioned resid norm 2.105216345845e+15 true resid norm >>> 3.429537728013e+14 ||r(i)||/||b|| 1.623397207682e-02 >>> 25 KSP preconditioned resid norm 1.993254024352e+15 true resid norm >>> 3.120073169264e+14 ||r(i)||/||b|| 1.476909855627e-02 >>> 26 KSP preconditioned resid norm 1.937764630771e+15 true resid norm >>> 3.193060027359e+14 ||r(i)||/||b|| 1.511458728106e-02 >>> 27 KSP preconditioned resid norm 1.841409046328e+15 true resid norm >>> 4.276359096161e+14 ||r(i)||/||b|| 2.024246404712e-02 >>> 28 KSP preconditioned resid norm 1.798600034298e+15 true resid norm >>> 2.807262398909e+14 ||r(i)||/||b|| 1.328838549404e-02 >>> 29 KSP preconditioned resid norm 1.909156742948e+15 true resid norm >>> 3.075705616539e+14 ||r(i)||/||b|| 1.455908144342e-02 >>> 30 KSP preconditioned resid norm 2.243056909373e+15 true resid norm >>> 3.518787704137e+14 ||r(i)||/||b|| 1.665644348118e-02 >>> 31 KSP preconditioned resid norm 3.125791809917e+15 true resid norm >>> 5.051797337710e+14 ||r(i)||/||b|| 2.391305867501e-02 >>> 32 KSP preconditioned resid norm 2.268406906040e+16 true resid norm >>> 3.663605847107e+15 ||r(i)||/||b|| 1.734195093893e-01 >>> 33 KSP preconditioned resid norm 2.663631019961e+15 true resid norm >>> 4.418031676456e+14 ||r(i)||/||b|| 2.091308174984e-02 >>> 34 KSP preconditioned resid norm 1.194136462714e+15 true resid norm >>> 1.964734637439e+14 ||r(i)||/||b|| 9.300217630505e-03 >>> 35 KSP preconditioned resid norm 7.378290473645e+14 true resid norm >>> 1.245828993721e+14 ||r(i)||/||b|| 5.897224261846e-03 >>> 36 KSP preconditioned resid norm 5.578220476287e+14 true resid norm >>> 9.049566410806e+13 ||r(i)||/||b|| 4.283679611405e-03 >>> 37 KSP preconditioned resid norm 4.459209275003e+14 true resid norm >>> 9.101333159944e+13 ||r(i)||/||b|| 4.308183787380e-03 >>> 38 KSP preconditioned resid norm 4.020379023591e+14 true resid norm >>> 6.381912771158e+13 ||r(i)||/||b|| 3.020925907227e-03 >>> 39 KSP preconditioned resid norm 3.758207934261e+14 true resid norm >>> 7.185355109637e+13 ||r(i)||/||b|| 3.401241317717e-03 >>> 40 KSP preconditioned resid norm 3.524000257362e+14 true resid norm >>> 5.552480372659e+13 ||r(i)||/||b|| 2.628307908397e-03 >>> 41 KSP preconditioned resid norm 3.288683013967e+14 true resid norm >>> 5.672396056313e+13 ||r(i)||/||b|| 2.685070889720e-03 >>> 42 KSP preconditioned resid norm 3.091159619430e+14 true resid norm >>> 5.054650403677e+13 ||r(i)||/||b|| 2.392656387510e-03 >>> 43 KSP preconditioned resid norm 2.914764729338e+14 true resid norm >>> 5.652869757159e+13 ||r(i)||/||b|| 2.675827970692e-03 >>> 44 KSP preconditioned resid norm 2.823730992718e+14 true resid norm >>> 4.387571428023e+13 ||r(i)||/||b|| 2.076889589689e-03 >>> 45 KSP preconditioned resid norm 2.874969520743e+14 true resid norm >>> 4.617969867525e+13 ||r(i)||/||b|| 2.185950405754e-03 >>> 46 KSP preconditioned resid norm 3.038236713170e+14 true resid norm >>> 4.761130615995e+13 ||r(i)||/||b|| 2.253716611508e-03 >>> 47 KSP preconditioned resid norm 3.972150696557e+14 true resid norm >>> 6.226543710285e+13 ||r(i)||/||b|| 2.947380805938e-03 >>> 48 KSP preconditioned resid norm 7.035007479461e+14 true resid norm >>> 1.474495546291e+14 ||r(i)||/||b|| 6.979634406808e-03 >>> 49 KSP preconditioned resid norm 2.896572855920e+13 true resid norm >>> 4.634266191547e+12 ||r(i)||/||b|| 2.193664392014e-04 >>> 0 KSP preconditioned resid norm 6.694681597421e+18 true resid norm >>> 3.650754072308e+18 ||r(i)||/||b|| 5.229798618159e+00 >>> 1 KSP preconditioned resid norm 1.597667334547e+18 true resid norm >>> 5.366749713724e+17 ||r(i)||/||b|| 7.688006280602e-01 >>> 2 KSP preconditioned resid norm 6.300446759218e+17 true resid norm >>> 1.705328267289e+17 ||r(i)||/||b|| 2.442926376067e-01 >>> 3 KSP preconditioned resid norm 3.642174486076e+17 true resid norm >>> 9.481576843120e+16 ||r(i)||/||b|| 1.358260142699e-01 >>> 4 KSP preconditioned resid norm 2.247332959633e+17 true resid norm >>> 3.802800729913e+16 ||r(i)||/||b|| 5.447609345502e-02 >>> 5 KSP preconditioned resid norm 1.557523701694e+17 true resid norm >>> 3.829141999854e+16 ||r(i)||/||b|| 5.485343888670e-02 >>> 6 KSP preconditioned resid norm 1.004692846458e+17 true resid norm >>> 1.699728099753e+16 ||r(i)||/||b|| 2.434903992784e-02 >>> 7 KSP preconditioned resid norm 7.103482800944e+16 true resid norm >>> 1.472858884839e+16 ||r(i)||/||b|| 2.109908037658e-02 >>> 8 KSP preconditioned resid norm 5.766847884006e+16 true resid norm >>> 9.397954060610e+15 ||r(i)||/||b|| 1.346280965144e-02 >>> 9 KSP preconditioned resid norm 4.523472015836e+16 true resid norm >>> 1.246374917128e+16 ||r(i)||/||b|| 1.785463959006e-02 >>> 10 KSP preconditioned resid norm 3.195342481549e+16 true resid norm >>> 5.030521483345e+15 ||r(i)||/||b|| 7.206350737723e-03 >>> 11 KSP preconditioned resid norm 2.437460220100e+16 true resid norm >>> 4.974645944970e+15 ||r(i)||/||b|| 7.126307599347e-03 >>> 12 KSP preconditioned resid norm 1.258045555912e+16 true resid norm >>> 1.885910304740e+15 ||r(i)||/||b|| 2.701614765157e-03 >>> 13 KSP preconditioned resid norm 2.495567167662e+15 true resid norm >>> 4.460991814847e+14 ||r(i)||/||b|| 6.390484915397e-04 >>> 14 KSP preconditioned resid norm 7.678096336284e+14 true resid norm >>> 1.316276217008e+14 ||r(i)||/||b|| 1.885599359606e-04 >>> 15 KSP preconditioned resid norm 5.950867488217e+14 true resid norm >>> 1.439410415068e+14 ||r(i)||/||b|| 2.061992248885e-04 >>> 16 KSP preconditioned resid norm 3.263035982098e+14 true resid norm >>> 5.046543709851e+13 ||r(i)||/||b|| 7.229302987142e-05 >>> 17 KSP preconditioned resid norm 1.436782483869e+14 true resid norm >>> 7.588970331259e+13 ||r(i)||/||b|| 1.087139417380e-04 >>> 18 KSP preconditioned resid norm 5.401533723431e+13 true resid norm >>> 8.625318760812e+12 ||r(i)||/||b|| 1.235598981554e-05 >>> 19 KSP preconditioned resid norm 3.528859982338e+13 true resid norm >>> 1.233587619696e+13 ||r(i)||/||b|| 1.767145828255e-05 >>> 0 KSP preconditioned resid norm 4.827874033276e+18 true resid norm >>> 2.629049503582e+18 ||r(i)||/||b|| 2.387244449971e+01 >>> 1 KSP preconditioned resid norm 1.161059891938e+18 true resid norm >>> 3.885117259710e+17 ||r(i)||/||b|| 3.527786222014e+00 >>> 2 KSP preconditioned resid norm 4.696931551640e+17 true resid norm >>> 1.228954734511e+17 ||r(i)||/||b|| 1.115922452289e+00 >>> 3 KSP preconditioned resid norm 2.870101730651e+17 true resid norm >>> 6.759381584873e+16 ||r(i)||/||b|| 6.137692025860e-01 >>> 4 KSP preconditioned resid norm 1.966639305477e+17 true resid norm >>> 3.310590025777e+16 ||r(i)||/||b|| 3.006100742644e-01 >>> 5 KSP preconditioned resid norm 1.592574496560e+17 true resid norm >>> 3.239415772489e+16 ||r(i)||/||b|| 2.941472693263e-01 >>> 6 KSP preconditioned resid norm 1.230114823548e+17 true resid norm >>> 2.280737281236e+16 ||r(i)||/||b|| 2.070968009182e-01 >>> 7 KSP preconditioned resid norm 8.598640814452e+16 true resid norm >>> 1.938029193987e+16 ||r(i)||/||b|| 1.759780266946e-01 >>> 8 KSP preconditioned resid norm 5.549085767183e+16 true resid norm >>> 9.386094645880e+15 ||r(i)||/||b|| 8.522814925985e-02 >>> 9 KSP preconditioned resid norm 4.474754629313e+16 true resid norm >>> 7.579167805290e+15 ||r(i)||/||b|| 6.882078961970e-02 >>> 10 KSP preconditioned resid norm 4.378389532275e+16 true resid norm >>> 6.842593279419e+15 ||r(i)||/||b|| 6.213250380964e-02 >>> 11 KSP preconditioned resid norm 4.079498017991e+16 true resid norm >>> 7.704710537522e+15 ||r(i)||/||b|| 6.996074986140e-02 >>> 12 KSP preconditioned resid norm 4.084398127844e+16 true resid norm >>> 6.377957981626e+15 ||r(i)||/||b|| 5.791349601078e-02 >>> 13 KSP preconditioned resid norm 7.523960004025e+16 true resid norm >>> 1.132262464871e+16 ||r(i)||/||b|| 1.028123388886e-01 >>> 14 KSP preconditioned resid norm 2.184546709977e+15 true resid norm >>> 3.502101186681e+14 ||r(i)||/||b|| 3.179997793785e-03 >>> 15 KSP preconditioned resid norm 2.222188254453e+15 true resid norm >>> 3.654675413503e+14 ||r(i)||/||b|| 3.318539109075e-03 >>> 16 KSP preconditioned resid norm 1.669487552266e+15 true resid norm >>> 3.629534589137e+14 ||r(i)||/||b|| 3.295710595061e-03 >>> 17 KSP preconditioned resid norm 1.401917499993e+15 true resid norm >>> 2.161165604483e+14 ||r(i)||/||b|| 1.962393856693e-03 >>> 18 KSP preconditioned resid norm 1.134393254100e+15 true resid norm >>> 2.686682335159e+14 ||r(i)||/||b|| 2.439576540763e-03 >>> 19 KSP preconditioned resid norm 9.058009036885e+14 true resid norm >>> 1.409928994873e+14 ||r(i)||/||b|| 1.280251727203e-03 >>> 20 KSP preconditioned resid norm 7.990098022926e+14 true resid norm >>> 1.659150954933e+14 ||r(i)||/||b|| 1.506551665700e-03 >>> 21 KSP preconditioned resid norm 7.119179981764e+14 true resid norm >>> 1.092130722300e+14 ||r(i)||/||b|| 9.916827362515e-04 >>> 22 KSP preconditioned resid norm 6.306959704581e+14 true resid norm >>> 1.460783700778e+14 ||r(i)||/||b|| 1.326429105857e-03 >>> 23 KSP preconditioned resid norm 5.637228678118e+14 true resid norm >>> 8.672640313846e+13 ||r(i)||/||b|| 7.874980074592e-04 >>> 24 KSP preconditioned resid norm 4.960976390096e+14 true resid norm >>> 1.292271182132e+14 ||r(i)||/||b|| 1.173415412376e-03 >>> 25 KSP preconditioned resid norm 4.386404742743e+14 true resid norm >>> 8.037964656859e+13 ||r(i)||/||b|| 7.298678282782e-04 >>> 26 KSP preconditioned resid norm 4.419824802921e+14 true resid norm >>> 6.704823034514e+13 ||r(i)||/||b|| 6.088151461346e-04 >>> 27 KSP preconditioned resid norm 4.437076279615e+14 true resid norm >>> 1.370394110315e+14 ||r(i)||/||b|| 1.244353036968e-03 >>> 28 KSP preconditioned resid norm 7.848825686766e+14 true resid norm >>> 1.196984074237e+14 ||r(i)||/||b|| 1.086892271915e-03 >>> 29 KSP preconditioned resid norm 1.771839364121e+15 true resid norm >>> 6.554952272410e+14 ||r(i)||/||b|| 5.952064961431e-03 >>> 30 KSP preconditioned resid norm 1.052090569223e+15 true resid norm >>> 1.877636113350e+14 ||r(i)||/||b|| 1.704941799139e-03 >>> 31 KSP preconditioned resid norm 2.459236824172e+14 true resid norm >>> 3.913333475140e+13 ||r(i)||/||b|| 3.553407270074e-04 >>> 32 KSP preconditioned resid norm 1.091606883884e+14 true resid norm >>> 1.900051073024e+13 ||r(i)||/||b|| 1.725295158025e-04 >>> 33 KSP preconditioned resid norm 6.424114279947e+13 true resid norm >>> 1.001416869814e+13 ||r(i)||/||b|| 9.093122291205e-05 >>> 34 KSP preconditioned resid norm 3.801391191576e+13 true resid norm >>> 7.035993054761e+12 ||r(i)||/||b|| 6.388862342504e-05 >>> 35 KSP preconditioned resid norm 2.524482106623e+13 true resid norm >>> 4.006294246762e+12 ||r(i)||/||b|| 3.637818037470e-05 >>> 36 KSP preconditioned resid norm 1.579134058219e+13 true resid norm >>> 2.528643754315e+12 ||r(i)||/||b|| 2.296073451724e-05 >>> 37 KSP preconditioned resid norm 1.074218958170e+13 true resid norm >>> 1.786180378482e+12 ||r(i)||/||b|| 1.621897643756e-05 >>> 38 KSP preconditioned resid norm 6.572401461924e+12 true resid norm >>> 1.033900044668e+12 ||r(i)||/||b|| 9.388077858923e-06 >>> 0 KSP preconditioned resid norm 9.558506771869e+15 true resid norm >>> 5.892974320929e+15 ||r(i)||/||b|| 1.119279921775e-01 >>> 1 KSP preconditioned resid norm 4.577878026258e+15 true resid norm >>> 8.113551231445e+14 ||r(i)||/||b|| 1.541044384903e-02 >>> 2 KSP preconditioned resid norm 3.768474917795e+15 true resid norm >>> 6.114238708763e+14 ||r(i)||/||b|| 1.161305692331e-02 >>> 3 KSP preconditioned resid norm 3.689006206393e+15 true resid norm >>> 6.168086911615e+14 ||r(i)||/||b|| 1.171533331040e-02 >>> 4 KSP preconditioned resid norm 3.859762637542e+15 true resid norm >>> 6.287723791867e+14 ||r(i)||/||b|| 1.194256518123e-02 >>> 5 KSP preconditioned resid norm 4.285185348381e+15 true resid norm >>> 6.764146832146e+14 ||r(i)||/||b|| 1.284745753985e-02 >>> 6 KSP preconditioned resid norm 5.208557345650e+15 true resid norm >>> 7.765898979336e+14 ||r(i)||/||b|| 1.475013181583e-02 >>> 7 KSP preconditioned resid norm 6.471756089495e+15 true resid norm >>> 1.171552566673e+15 ||r(i)||/||b|| 2.225184081532e-02 >>> 8 KSP preconditioned resid norm 7.012039525053e+15 true resid norm >>> 1.102268075574e+15 ||r(i)||/||b|| 2.093588836833e-02 >>> 9 KSP preconditioned resid norm 7.014970322733e+15 true resid norm >>> 1.328280835862e+15 ||r(i)||/||b|| 2.522865346245e-02 >>> 10 KSP preconditioned resid norm 6.390016130531e+15 true resid norm >>> 1.015385597333e+15 ||r(i)||/||b|| 1.928568919633e-02 >>> 11 KSP preconditioned resid norm 4.845316666400e+15 true resid norm >>> 8.456481888760e+14 ||r(i)||/||b|| 1.606178793844e-02 >>> 12 KSP preconditioned resid norm 2.410560854094e+15 true resid norm >>> 3.709640804249e+14 ||r(i)||/||b|| 7.045892690295e-03 >>> 13 KSP preconditioned resid norm 1.334157374345e+15 true resid norm >>> 2.713196530598e+14 ||r(i)||/||b|| 5.153299904502e-03 >>> 14 KSP preconditioned resid norm 1.217530036393e+15 true resid norm >>> 2.232136761716e+14 ||r(i)||/||b|| 4.239600792372e-03 >>> 15 KSP preconditioned resid norm 1.145671346103e+15 true resid norm >>> 2.123551517116e+14 ||r(i)||/||b|| 4.033359805286e-03 >>> 16 KSP preconditioned resid norm 1.048828510236e+15 true resid norm >>> 1.668978069044e+14 ||r(i)||/||b|| 3.169967389692e-03 >>> 17 KSP preconditioned resid norm 9.144204694073e+14 true resid norm >>> 1.578839852075e+14 ||r(i)||/||b|| 2.998763697051e-03 >>> 18 KSP preconditioned resid norm 8.235850213860e+14 true resid norm >>> 1.321988157805e+14 ||r(i)||/||b|| 2.510913371198e-03 >>> 19 KSP preconditioned resid norm 6.978607225935e+14 true resid norm >>> 1.125789224246e+14 ||r(i)||/||b|| 2.138263644512e-03 >>> 20 KSP preconditioned resid norm 6.009340490988e+14 true resid norm >>> 1.003011158758e+14 ||r(i)||/||b|| 1.905065574997e-03 >>> 21 KSP preconditioned resid norm 5.123703002158e+14 true resid norm >>> 8.114653676231e+13 ||r(i)||/||b|| 1.541253777351e-03 >>> 22 KSP preconditioned resid norm 4.394549158605e+14 true resid norm >>> 8.003106905102e+13 ||r(i)||/||b|| 1.520067182184e-03 >>> 23 KSP preconditioned resid norm 3.852402878098e+14 true resid norm >>> 5.993269339949e+13 ||r(i)||/||b|| 1.138329419521e-03 >>> 24 KSP preconditioned resid norm 3.390788847804e+14 true resid norm >>> 7.376626581336e+13 ||r(i)||/||b|| 1.401076871080e-03 >>> 0 KSP preconditioned resid norm 9.592017367891e+15 true resid norm >>> 5.933298102242e+15 ||r(i)||/||b|| 5.704123038824e-01 >>> 1 KSP preconditioned resid norm 4.627662883687e+15 true resid norm >>> 8.121175703537e+14 ||r(i)||/||b|| 7.807493342594e-02 >>> 2 KSP preconditioned resid norm 3.843678575472e+15 true resid norm >>> 6.216594738645e+14 ||r(i)||/||b|| 5.976477274644e-02 >>> 3 KSP preconditioned resid norm 3.802359495548e+15 true resid norm >>> 6.332926539714e+14 ||r(i)||/||b|| 6.088315731972e-02 >>> 4 KSP preconditioned resid norm 4.025235313693e+15 true resid norm >>> 6.488898363947e+14 ||r(i)||/||b|| 6.238263107055e-02 >>> 5 KSP preconditioned resid norm 4.519136017232e+15 true resid norm >>> 7.100489487455e+14 ||r(i)||/||b|| 6.826231376612e-02 >>> 6 KSP preconditioned resid norm 5.566459161126e+15 true resid norm >>> 8.272746471009e+14 ||r(i)||/||b|| 7.953209652789e-02 >>> 7 KSP preconditioned resid norm 7.051037635374e+15 true resid norm >>> 1.259719337020e+15 ||r(i)||/||b|| 1.211062375246e-01 >>> 8 KSP preconditioned resid norm 7.825859719364e+15 true resid norm >>> 1.230378249193e+15 ||r(i)||/||b|| 1.182854593980e-01 >>> 9 KSP preconditioned resid norm 8.112136766042e+15 true resid norm >>> 1.509007778424e+15 ||r(i)||/||b|| 1.450721990763e-01 >>> 10 KSP preconditioned resid norm 7.849971536540e+15 true resid norm >>> 1.244897342905e+15 ||r(i)||/||b|| 1.196812884212e-01 >>> 11 KSP preconditioned resid norm 6.691547560102e+15 true resid norm >>> 1.152431148742e+15 ||r(i)||/||b|| 1.107918218994e-01 >>> 12 KSP preconditioned resid norm 3.838210718255e+15 true resid norm >>> 5.898239658450e+14 ||r(i)||/||b|| 5.670418735838e-02 >>> 13 KSP preconditioned resid norm 1.585805121666e+15 true resid norm >>> 3.093588848689e+14 ||r(i)||/||b|| 2.974098236828e-02 >>> 14 KSP preconditioned resid norm 1.284246821472e+15 true resid norm >>> 2.667115224243e+14 ||r(i)||/||b|| 2.564097258496e-02 >>> 15 KSP preconditioned resid norm 1.153005952878e+15 true resid norm >>> 2.281947914341e+14 ||r(i)||/||b|| 2.193807128394e-02 >>> 16 KSP preconditioned resid norm 1.029063568485e+15 true resid norm >>> 1.605944355705e+14 ||r(i)||/||b|| 1.543914369478e-02 >>> 17 KSP preconditioned resid norm 8.755986525312e+14 true resid norm >>> 1.474488326268e+14 ||r(i)||/||b|| 1.417535860732e-02 >>> 18 KSP preconditioned resid norm 7.769999155370e+14 true resid norm >>> 1.263847845681e+14 ||r(i)||/||b|| 1.215031419270e-02 >>> 19 KSP preconditioned resid norm 6.633003954457e+14 true resid norm >>> 1.037940616116e+14 ||r(i)||/||b|| 9.978499106727e-03 >>> 20 KSP preconditioned resid norm 5.634039280371e+14 true resid norm >>> 1.015458370156e+14 ||r(i)||/||b|| 9.762360468599e-03 >>> 21 KSP preconditioned resid norm 4.739327677772e+14 true resid norm >>> 7.392362794765e+13 ||r(i)||/||b|| 7.106831007366e-03 >>> 22 KSP preconditioned resid norm 3.847430187788e+14 true resid norm >>> 7.626771651358e+13 ||r(i)||/||b|| 7.332185765606e-03 >>> 23 KSP preconditioned resid norm 3.166693300565e+14 true resid norm >>> 4.910897850051e+13 ||r(i)||/||b|| 4.721213241789e-03 >>> 24 KSP preconditioned resid norm 2.603961739838e+14 true resid norm >>> 5.903512955155e+13 ||r(i)||/||b|| 5.675488350193e-03 >>> 25 KSP preconditioned resid norm 2.300385519514e+14 true resid norm >>> 3.524302891352e+13 ||r(i)||/||b|| 3.388175846206e-03 >>> 26 KSP preconditioned resid norm 1.995854205790e+14 true resid norm >>> 5.125398452570e+13 ||r(i)||/||b|| 4.927428707047e-03 >>> 27 KSP preconditioned resid norm 1.777736693158e+14 true resid norm >>> 2.713391594494e+13 ||r(i)||/||b|| 2.608586192838e-03 >>> 28 KSP preconditioned resid norm 1.569155636768e+14 true resid norm >>> 4.045925214031e+13 ||r(i)||/||b|| 3.889650381462e-03 >>> 29 KSP preconditioned resid norm 1.392286125087e+14 true resid norm >>> 2.127117420025e+13 ||r(i)||/||b|| 2.044956999086e-03 >>> 30 KSP preconditioned resid norm 1.268134787155e+14 true resid norm >>> 2.836210701297e+13 ||r(i)||/||b|| 2.726661382160e-03 >>> 31 KSP preconditioned resid norm 1.148332814766e+14 true resid norm >>> 1.759485016641e+13 ||r(i)||/||b|| 1.691524485529e-03 >>> 32 KSP preconditioned resid norm 1.072826684457e+14 true resid norm >>> 1.915166944365e+13 ||r(i)||/||b|| 1.841193161425e-03 >>> 33 KSP preconditioned resid norm 9.907876839152e+13 true resid norm >>> 1.514435361562e+13 ||r(i)||/||b|| 1.455939932199e-03 >>> 34 KSP preconditioned resid norm 9.246074413874e+13 true resid norm >>> 1.530899883812e+13 ||r(i)||/||b|| 1.471768508325e-03 >>> 35 KSP preconditioned resid norm 8.716833484735e+13 true resid norm >>> 1.331822154534e+13 ||r(i)||/||b|| 1.280380204127e-03 >>> 36 KSP preconditioned resid norm 8.124134503515e+13 true resid norm >>> 1.344545385639e+13 ||r(i)||/||b|| 1.292611997376e-03 >>> 37 KSP preconditioned resid norm 7.675303961825e+13 true resid norm >>> 1.172301521125e+13 ||r(i)||/||b|| 1.127021093474e-03 >>> 38 KSP preconditioned resid norm 7.135475879451e+13 true resid norm >>> 1.180708573448e+13 ||r(i)||/||b|| 1.135103421383e-03 >>> 39 KSP preconditioned resid norm 6.615399842578e+13 true resid norm >>> 1.009923425569e+13 ||r(i)||/||b|| 9.709148908363e-04 >>> 40 KSP preconditioned resid norm 6.137620595303e+13 true resid norm >>> 9.912104999293e+12 ||r(i)||/||b|| 9.529247564410e-04 >>> 41 KSP preconditioned resid norm 5.598390411354e+13 true resid norm >>> 8.555770285650e+12 ||r(i)||/||b|| 8.225301604653e-04 >>> 0 KSP preconditioned resid norm 2.200654561303e+16 true resid norm >>> 1.374535903551e+16 ||r(i)||/||b|| 1.042244800141e-01 >>> 1 KSP preconditioned resid norm 1.033788570768e+16 true resid norm >>> 1.766933748516e+15 ||r(i)||/||b|| 1.339781308605e-02 >>> 2 KSP preconditioned resid norm 8.199712159807e+15 true resid norm >>> 1.328504928257e+15 ||r(i)||/||b|| 1.007341714291e-02 >>> 3 KSP preconditioned resid norm 7.724963557325e+15 true resid norm >>> 1.290091137776e+15 ||r(i)||/||b|| 9.782143751798e-03 >>> 4 KSP preconditioned resid norm 7.785648142256e+15 true resid norm >>> 1.238722001104e+15 ||r(i)||/||b|| 9.392636170038e-03 >>> 5 KSP preconditioned resid norm 8.249045346662e+15 true resid norm >>> 1.288778250144e+15 ||r(i)||/||b|| 9.772188753144e-03 >>> 6 KSP preconditioned resid norm 9.469275305646e+15 true resid norm >>> 1.408770770519e+15 ||r(i)||/||b|| 1.068203461525e-02 >>> 7 KSP preconditioned resid norm 1.105611185924e+16 true resid norm >>> 1.994387714295e+15 ||r(i)||/||b|| 1.512248766524e-02 >>> 8 KSP preconditioned resid norm 1.130059557946e+16 true resid norm >>> 1.768360656538e+15 ||r(i)||/||b|| 1.340863264676e-02 >>> 9 KSP preconditioned resid norm 1.052660615325e+16 true resid norm >>> 1.953565171874e+15 ||r(i)||/||b|| 1.481294986084e-02 >>> 10 KSP preconditioned resid norm 8.559164814525e+15 true resid norm >>> 1.322544665666e+15 ||r(i)||/||b|| 1.002822332384e-02 >>> 11 KSP preconditioned resid norm 4.990037328467e+15 true resid norm >>> 7.740616289621e+14 ||r(i)||/||b|| 5.869338921523e-03 >>> 12 KSP preconditioned resid norm 1.334591152980e+17 true resid norm >>> 2.007497042766e+16 ||r(i)||/||b|| 1.522188942984e-01 >>> 13 KSP preconditioned resid norm 7.977709053830e+14 true resid norm >>> 2.666709181112e+14 ||r(i)||/||b|| 2.022037962283e-03 >>> 0 KSP preconditioned resid norm 1.928493598618e+16 true resid norm >>> 1.203377031534e+16 ||r(i)||/||b|| 4.946657154689e-01 >>> 1 KSP preconditioned resid norm 9.231841001478e+15 true resid norm >>> 1.582820058439e+15 ||r(i)||/||b|| 6.506413170177e-02 >>> 2 KSP preconditioned resid norm 7.541228065487e+15 true resid norm >>> 1.216411600127e+15 ||r(i)||/||b|| 5.000237653817e-02 >>> 3 KSP preconditioned resid norm 7.339567418801e+15 true resid norm >>> 1.219667619599e+15 ||r(i)||/||b|| 5.013621997708e-02 >>> 4 KSP preconditioned resid norm 7.654746286546e+15 true resid norm >>> 1.212045913869e+15 ||r(i)||/||b|| 4.982291862433e-02 >>> 5 KSP preconditioned resid norm 8.414088698130e+15 true resid norm >>> 1.311254037213e+15 ||r(i)||/||b|| 5.390101352132e-02 >>> 6 KSP preconditioned resid norm 1.008553581054e+16 true resid norm >>> 1.495637339777e+15 ||r(i)||/||b|| 6.148035863870e-02 >>> 7 KSP preconditioned resid norm 1.235521461592e+16 true resid norm >>> 2.202862093934e+15 ||r(i)||/||b|| 9.055186572627e-02 >>> 8 KSP preconditioned resid norm 1.311621757723e+16 true resid norm >>> 2.061404522413e+15 ||r(i)||/||b|| 8.473704551686e-02 >>> 9 KSP preconditioned resid norm 1.289521641859e+16 true resid norm >>> 2.354903652303e+15 ||r(i)||/||b|| 9.680175618294e-02 >>> 10 KSP preconditioned resid norm 1.151646040420e+16 true resid norm >>> 1.821568826477e+15 ||r(i)||/||b|| 7.487824872947e-02 >>> 11 KSP preconditioned resid norm 8.378553284195e+15 true resid norm >>> 1.434374156491e+15 ||r(i)||/||b|| 5.896204595714e-02 >>> 12 KSP preconditioned resid norm 3.545538676243e+15 true resid norm >>> 5.411687842502e+14 ||r(i)||/||b|| 2.224553376337e-02 >>> 13 KSP preconditioned resid norm 1.159722122499e+15 true resid norm >>> 2.107714204908e+14 ||r(i)||/||b|| 8.664067269472e-03 >>> 14 KSP preconditioned resid norm 9.539010029354e+14 true resid norm >>> 1.788314612223e+14 ||r(i)||/||b|| 7.351128565343e-03 >>> 15 KSP preconditioned resid norm 8.844744179759e+14 true resid norm >>> 1.382762666459e+14 ||r(i)||/||b|| 5.684048023218e-03 >>> 16 KSP preconditioned resid norm 7.998051624629e+14 true resid norm >>> 2.151800146054e+14 ||r(i)||/||b|| 8.845288973459e-03 >>> 17 KSP preconditioned resid norm 6.751752084065e+14 true resid norm >>> 1.049375541763e+14 ||r(i)||/||b|| 4.313611524562e-03 >>> 18 KSP preconditioned resid norm 6.016391198259e+14 true resid norm >>> 1.405509222696e+14 ||r(i)||/||b|| 5.777551066910e-03 >>> 19 KSP preconditioned resid norm 5.356608668579e+14 true resid norm >>> 8.319469793388e+13 ||r(i)||/||b|| 3.419839642795e-03 >>> 20 KSP preconditioned resid norm 4.666955314009e+14 true resid norm >>> 1.159814975417e+14 ||r(i)||/||b|| 4.767588956680e-03 >>> 21 KSP preconditioned resid norm 4.123539296814e+14 true resid norm >>> 6.367161120725e+13 ||r(i)||/||b|| 2.617314631039e-03 >>> 22 KSP preconditioned resid norm 3.572486233332e+14 true resid norm >>> 8.643344455842e+13 ||r(i)||/||b|| 3.552973056037e-03 >>> 23 KSP preconditioned resid norm 3.110551508962e+14 true resid norm >>> 4.773889823616e+13 ||r(i)||/||b|| 1.962377179627e-03 >>> 24 KSP preconditioned resid norm 2.787205893436e+14 true resid norm >>> 4.531540777818e+13 ||r(i)||/||b|| 1.862756062561e-03 >>> 25 KSP preconditioned resid norm 2.574052602586e+14 true resid norm >>> 4.020202768069e+13 ||r(i)||/||b|| 1.652563100745e-03 >>> 26 KSP preconditioned resid norm 2.417321052344e+14 true resid norm >>> 3.935243057179e+13 ||r(i)||/||b|| 1.617639169947e-03 >>> 27 KSP preconditioned resid norm 2.257728086711e+14 true resid norm >>> 3.470313603079e+13 ||r(i)||/||b|| 1.426523123165e-03 >>> 28 KSP preconditioned resid norm 2.082687407590e+14 true resid norm >>> 3.533906053355e+13 ||r(i)||/||b|| 1.452663729217e-03 >>> 29 KSP preconditioned resid norm 1.877437540892e+14 true resid norm >>> 2.879571658728e+13 ||r(i)||/||b|| 1.183690013589e-03 >>> 30 KSP preconditioned resid norm 1.698389677822e+14 true resid norm >>> 2.905672512335e+13 ||r(i)||/||b|| 1.194419150913e-03 >>> 31 KSP preconditioned resid norm 1.483411237433e+14 true resid norm >>> 2.274318996948e+13 ||r(i)||/||b|| 9.348920615478e-04 >>> 32 KSP preconditioned resid norm 1.319650244589e+14 true resid norm >>> 2.174098043497e+13 ||r(i)||/||b|| 8.936947739606e-04 >>> 0 KSP preconditioned resid norm 6.130010037542e+16 true resid norm >>> 4.019535297204e+16 ||r(i)||/||b|| 5.748505993040e-02 >>> 1 KSP preconditioned resid norm 2.383231272294e+16 true resid norm >>> 4.333028202793e+15 ||r(i)||/||b|| 6.196845344062e-03 >>> 2 KSP preconditioned resid norm 1.380314308424e+16 true resid norm >>> 2.902938638811e+15 ||r(i)||/||b|| 4.151614285921e-03 >>> 3 KSP preconditioned resid norm 9.355422663861e+15 true resid norm >>> 1.914374541722e+15 ||r(i)||/||b|| 2.737827313936e-03 >>> 4 KSP preconditioned resid norm 6.905389449720e+15 true resid norm >>> 1.160495113499e+15 ||r(i)||/||b|| 1.659672728707e-03 >>> 5 KSP preconditioned resid norm 5.388850196885e+15 true resid norm >>> 9.343171628690e+14 ||r(i)||/||b|| 1.336206156440e-03 >>> 6 KSP preconditioned resid norm 4.456941607393e+15 true resid norm >>> 7.443205302890e+14 ||r(i)||/||b|| 1.064484004428e-03 >>> 7 KSP preconditioned resid norm 3.604897945956e+15 true resid norm >>> 7.177354673261e+14 ||r(i)||/||b|| 1.026463591006e-03 >>> 8 KSP preconditioned resid norm 2.616552593649e+15 true resid norm >>> 4.566784798390e+14 ||r(i)||/||b|| 6.531150454316e-04 >>> 9 KSP preconditioned resid norm 1.730753896393e+15 true resid norm >>> 3.416577992882e+14 ||r(i)||/||b|| 4.886191466320e-04 >>> 10 KSP preconditioned resid norm 9.605913238033e+14 true resid norm >>> 1.715828091641e+14 ||r(i)||/||b|| 2.453877709368e-04 >>> 11 KSP preconditioned resid norm 3.885469473149e+14 true resid norm >>> 7.526615944264e+13 ||r(i)||/||b|| 1.076412910045e-04 >>> 12 KSP preconditioned resid norm 1.085402135701e+14 true resid norm >>> 1.877490961949e+13 ||r(i)||/||b|| 2.685078559740e-05 >>> 13 KSP preconditioned resid norm 5.521535097548e+13 true resid norm >>> 1.482831670997e+13 ||r(i)||/||b|| 2.120659757192e-05 >>> 14 KSP preconditioned resid norm 4.327778991064e+13 true resid norm >>> 6.949075957635e+12 ||r(i)||/||b|| 9.938164945670e-06 >>> 0 KSP preconditioned resid norm 2.469229041488e+16 true resid norm >>> 1.581397714283e+16 ||r(i)||/||b|| 1.421322439001e-01 >>> 1 KSP preconditioned resid norm 1.094866313372e+16 true resid norm >>> 1.827538507852e+15 ||r(i)||/||b|| 1.642547896642e-02 >>> 2 KSP preconditioned resid norm 7.902647446780e+15 true resid norm >>> 1.331972422469e+15 ||r(i)||/||b|| 1.197144952903e-02 >>> 3 KSP preconditioned resid norm 6.757114900348e+15 true resid norm >>> 1.177404752019e+15 ||r(i)||/||b|| 1.058223227919e-02 >>> 4 KSP preconditioned resid norm 6.236820681429e+15 true resid norm >>> 9.858934558838e+14 ||r(i)||/||b|| 8.860974558495e-03 >>> 5 KSP preconditioned resid norm 6.056208231119e+15 true resid norm >>> 9.498735329089e+14 ||r(i)||/||b|| 8.537236106662e-03 >>> 6 KSP preconditioned resid norm 6.281150442631e+15 true resid norm >>> 9.439017368810e+14 ||r(i)||/||b|| 8.483563032401e-03 >>> 7 KSP preconditioned resid norm 6.511637444968e+15 true resid norm >>> 1.175605283055e+15 ||r(i)||/||b|| 1.056605908257e-02 >>> 8 KSP preconditioned resid norm 5.897282453520e+15 true resid norm >>> 9.358326231502e+14 ||r(i)||/||b|| 8.411039768298e-03 >>> 9 KSP preconditioned resid norm 4.858643646570e+15 true resid norm >>> 9.074077133036e+14 ||r(i)||/||b|| 8.155563477757e-03 >>> 10 KSP preconditioned resid norm 3.431227043473e+15 true resid norm >>> 5.523077341885e+14 ||r(i)||/||b|| 4.964009804403e-03 >>> 11 KSP preconditioned resid norm 1.790602152143e+15 true resid norm >>> 3.233768236148e+14 ||r(i)||/||b|| 2.906433539808e-03 >>> 12 KSP preconditioned resid norm 6.061523931075e+14 true resid norm >>> 9.361733206500e+13 ||r(i)||/||b|| 8.414101875932e-04 >>> 13 KSP preconditioned resid norm 3.669173259935e+14 true resid norm >>> 7.433445474494e+13 ||r(i)||/||b|| 6.681002986515e-04 >>> 14 KSP preconditioned resid norm 1.034316883992e+15 true resid norm >>> 1.749783469552e+14 ||r(i)||/||b|| 1.572663528097e-03 >>> 15 KSP preconditioned resid norm 1.340579303848e+14 true resid norm >>> 2.477734260421e+13 ||r(i)||/||b|| 2.226928286548e-04 >>> 16 KSP preconditioned resid norm 1.124182191319e+14 true resid norm >>> 1.917839477947e+13 ||r(i)||/||b|| 1.723708248589e-04 >>> 17 KSP preconditioned resid norm 9.394060484065e+13 true resid norm >>> 1.682558945581e+13 ||r(i)||/||b|| 1.512243733944e-04 >>> 18 KSP preconditioned resid norm 9.034672558986e+13 true resid norm >>> 1.892870450718e+13 ||r(i)||/||b|| 1.701266684168e-04 >>> 19 KSP preconditioned resid norm 8.058964131802e+13 true resid norm >>> 1.275370577304e+13 ||r(i)||/||b|| 1.146272568369e-04 >>> 20 KSP preconditioned resid norm 6.303390378547e+13 true resid norm >>> 1.900069972515e+13 ||r(i)||/||b|| 1.707737442148e-04 >>> 21 KSP preconditioned resid norm 4.601826552473e+13 true resid norm >>> 6.960266416937e+12 ||r(i)||/||b|| 6.255720967895e-05 >>> 22 KSP preconditioned resid norm 2.822788042823e+13 true resid norm >>> 1.013300658091e+13 ||r(i)||/||b|| 9.107303936204e-05 >>> 23 KSP preconditioned resid norm 1.858182222289e+13 true resid norm >>> 2.785683196819e+12 ||r(i)||/||b|| 2.503705424529e-05 >>> 24 KSP preconditioned resid norm 9.506926663598e+12 true resid norm >>> 4.268961211449e+12 ||r(i)||/||b|| 3.836840224479e-05 >>> 25 KSP preconditioned resid norm 5.845171932356e+12 true resid norm >>> 8.923162702109e+11 ||r(i)||/||b|| 8.019925197071e-06 >>> 0 SNES Function norm 0.000000000000e+00 >>> 4 TS dt 6e-12 time 1.4e-11 >>> 0 KSP preconditioned resid norm 2.220105437401e+19 true resid norm >>> 1.204162959566e+19 ||r(i)||/||b|| 2.302124615573e+02 >>> 1 KSP preconditioned resid norm 5.389751984623e+18 true resid norm >>> 1.775745946542e+18 ||r(i)||/||b|| 3.394879756150e+01 >>> 2 KSP preconditioned resid norm 2.218217192238e+18 true resid norm >>> 5.319400119211e+17 ||r(i)||/||b|| 1.016965507636e+01 >>> 3 KSP preconditioned resid norm 1.401615472906e+18 true resid norm >>> 2.963370402601e+17 ||r(i)||/||b|| 5.665385980104e+00 >>> 4 KSP preconditioned resid norm 1.004001983833e+18 true resid norm >>> 1.656139793050e+17 ||r(i)||/||b|| 3.166216128906e+00 >>> 5 KSP preconditioned resid norm 8.672200498511e+17 true resid norm >>> 1.510248645324e+17 ||r(i)||/||b|| 2.887300721564e+00 >>> 6 KSP preconditioned resid norm 7.388847075384e+17 true resid norm >>> 1.526081348396e+17 ||r(i)||/||b|| 2.917569760470e+00 >>> 7 KSP preconditioned resid norm 5.565797182786e+17 true resid norm >>> 1.017271957075e+17 ||r(i)||/||b|| 1.944825486043e+00 >>> 8 KSP preconditioned resid norm 3.566434137336e+17 true resid norm >>> 6.306431985913e+16 ||r(i)||/||b|| 1.205666741023e+00 >>> 9 KSP preconditioned resid norm 1.893287605930e+17 true resid norm >>> 3.267942353405e+16 ||r(i)||/||b|| 6.247668120231e-01 >>> 10 KSP preconditioned resid norm 6.895870882324e+16 true resid norm >>> 1.153127225742e+16 ||r(i)||/||b|| 2.204554250883e-01 >>> 11 KSP preconditioned resid norm 5.104972946485e+16 true resid norm >>> 9.275497869859e+15 ||r(i)||/||b|| 1.773294203933e-01 >>> 12 KSP preconditioned resid norm 4.992720890366e+16 true resid norm >>> 8.399052910493e+15 ||r(i)||/||b|| 1.605735029394e-01 >>> 13 KSP preconditioned resid norm 4.919979261885e+16 true resid norm >>> 8.426637216788e+15 ||r(i)||/||b|| 1.611008610517e-01 >>> 14 KSP preconditioned resid norm 5.688292138515e+16 true resid norm >>> 9.762950550877e+15 ||r(i)||/||b|| 1.866485645090e-01 >>> 15 KSP preconditioned resid norm 1.614157649132e+17 true resid norm >>> 4.341833601566e+16 ||r(i)||/||b|| 8.300738642954e-01 >>> 16 KSP preconditioned resid norm 4.125579354715e+16 true resid norm >>> 7.099585962609e+15 ||r(i)||/||b|| 1.357302304896e-01 >>> 17 KSP preconditioned resid norm 2.163051916179e+16 true resid norm >>> 5.849969022355e+15 ||r(i)||/||b|| 1.118399929155e-01 >>> 18 KSP preconditioned resid norm 1.415003279507e+16 true resid norm >>> 2.432246275858e+15 ||r(i)||/||b|| 4.649980285729e-02 >>> 19 KSP preconditioned resid norm 1.088163683743e+16 true resid norm >>> 2.488576155885e+15 ||r(i)||/||b|| 4.757672024934e-02 >>> 20 KSP preconditioned resid norm 9.060374162401e+15 true resid norm >>> 1.588297599957e+15 ||r(i)||/||b|| 3.036515093466e-02 >>> 21 KSP preconditioned resid norm 7.815461281280e+15 true resid norm >>> 1.346385634634e+15 ||r(i)||/||b|| 2.574026619006e-02 >>> 22 KSP preconditioned resid norm 6.943207518072e+15 true resid norm >>> 1.184751633436e+15 ||r(i)||/||b|| 2.265013947659e-02 >>> 23 KSP preconditioned resid norm 6.246190157097e+15 true resid norm >>> 1.023840274541e+15 ||r(i)||/||b|| 1.957382827391e-02 >>> 24 KSP preconditioned resid norm 5.682318425500e+15 true resid norm >>> 9.496257176169e+14 ||r(i)||/||b|| 1.815499075718e-02 >>> 25 KSP preconditioned resid norm 5.282641218542e+15 true resid norm >>> 8.559439950771e+14 ||r(i)||/||b|| 1.636397901932e-02 >>> 26 KSP preconditioned resid norm 5.115699254805e+15 true resid norm >>> 8.571023693113e+14 ||r(i)||/||b|| 1.638612487439e-02 >>> 27 KSP preconditioned resid norm 4.941038732000e+15 true resid norm >>> 9.550184315260e+14 ||r(i)||/||b|| 1.825808892456e-02 >>> 28 KSP preconditioned resid norm 4.836641976772e+15 true resid norm >>> 7.564719174843e+14 ||r(i)||/||b|| 1.446226699132e-02 >>> 29 KSP preconditioned resid norm 4.905885514019e+15 true resid norm >>> 7.773698572205e+14 ||r(i)||/||b|| 1.486179482182e-02 >>> 30 KSP preconditioned resid norm 5.169672483375e+15 true resid norm >>> 8.117514654202e+14 ||r(i)||/||b|| 1.551910408326e-02 >>> 31 KSP preconditioned resid norm 5.732174203621e+15 true resid norm >>> 9.079138157636e+14 ||r(i)||/||b|| 1.735754058438e-02 >>> 32 KSP preconditioned resid norm 8.584579912384e+15 true resid norm >>> 1.381429969496e+15 ||r(i)||/||b|| 2.641024549212e-02 >>> 33 KSP preconditioned resid norm 1.658456221337e+16 true resid norm >>> 2.668310391151e+15 ||r(i)||/||b|| 5.101288811997e-02 >>> 34 KSP preconditioned resid norm 3.797117274875e+16 true resid norm >>> 6.214594485387e+15 ||r(i)||/||b|| 1.188109202908e-01 >>> 35 KSP preconditioned resid norm 6.896188371971e+15 true resid norm >>> 1.129650731508e+15 ||r(i)||/||b|| 2.159671774774e-02 >>> 36 KSP preconditioned resid norm 3.678389972715e+15 true resid norm >>> 6.014447539523e+14 ||r(i)||/||b|| 1.149845012239e-02 >>> 37 KSP preconditioned resid norm 2.344506104501e+15 true resid norm >>> 4.262436512363e+14 ||r(i)||/||b|| 8.148946900809e-03 >>> 38 KSP preconditioned resid norm 1.979277723710e+15 true resid norm >>> 3.194432632084e+14 ||r(i)||/||b|| 6.107131876701e-03 >>> 39 KSP preconditioned resid norm 1.806670892810e+15 true resid norm >>> 3.108273060962e+14 ||r(i)||/||b|| 5.942411588660e-03 >>> 40 KSP preconditioned resid norm 1.663957842585e+15 true resid norm >>> 2.662101226861e+14 ||r(i)||/||b|| 5.089418101442e-03 >>> 41 KSP preconditioned resid norm 1.486329834699e+15 true resid norm >>> 2.427881601889e+14 ||r(i)||/||b|| 4.641635880760e-03 >>> 42 KSP preconditioned resid norm 1.346929765546e+15 true resid norm >>> 2.207126725819e+14 ||r(i)||/||b|| 4.219595632660e-03 >>> 43 KSP preconditioned resid norm 1.242596801597e+15 true resid norm >>> 2.017748115787e+14 ||r(i)||/||b|| 3.857540682909e-03 >>> 44 KSP preconditioned resid norm 1.178815061343e+15 true resid norm >>> 1.881589524072e+14 ||r(i)||/||b|| 3.597232023587e-03 >>> 45 KSP preconditioned resid norm 9.347996687830e+14 true resid norm >>> 2.754924905110e+14 ||r(i)||/||b|| 5.266878861971e-03 >>> 46 KSP preconditioned resid norm 7.515414326072e+14 true resid norm >>> 1.170313953112e+14 ||r(i)||/||b|| 2.237411920043e-03 >>> 47 KSP preconditioned resid norm 7.344982573734e+14 true resid norm >>> 1.334969192887e+14 ||r(i)||/||b|| 2.552200610027e-03 >>> 48 KSP preconditioned resid norm 8.030126805909e+14 true resid norm >>> 1.311086080593e+14 ||r(i)||/||b|| 2.506540759528e-03 >>> 49 KSP preconditioned resid norm 9.282223496248e+14 true resid norm >>> 1.446635532628e+14 ||r(i)||/||b|| 2.765684862641e-03 >>> 50 KSP preconditioned resid norm 1.074003818336e+15 true resid norm >>> 3.137568207714e+14 ||r(i)||/||b|| 5.998418193015e-03 >>> 51 KSP preconditioned resid norm 1.638671425800e+15 true resid norm >>> 2.543484445494e+14 ||r(i)||/||b|| 4.862645960648e-03 >>> 52 KSP preconditioned resid norm 2.418883444028e+15 true resid norm >>> 6.581384678327e+14 ||r(i)||/||b|| 1.258232330779e-02 >>> 53 KSP preconditioned resid norm 8.572181890774e+14 true resid norm >>> 1.327918910006e+14 ||r(i)||/||b|| 2.538721844848e-03 >>> 54 KSP preconditioned resid norm 2.577275091696e+14 true resid norm >>> 8.010767963629e+13 ||r(i)||/||b|| 1.531502523990e-03 >>> 0 KSP preconditioned resid norm 6.542967041071e+18 true resid norm >>> 3.545980595607e+18 ||r(i)||/||b|| 4.156826068038e+02 >>> 1 KSP preconditioned resid norm 1.591191551470e+18 true resid norm >>> 5.223269779130e+17 ||r(i)||/||b|| 6.123052112913e+01 >>> 2 KSP preconditioned resid norm 6.570663371928e+17 true resid norm >>> 1.552539857829e+17 ||r(i)||/||b|| 1.819986877731e+01 >>> 3 KSP preconditioned resid norm 4.165904471738e+17 true resid norm >>> 8.738677344351e+16 ||r(i)||/||b|| 1.024403851227e+01 >>> 4 KSP preconditioned resid norm 2.995075360323e+17 true resid norm >>> 4.934782956985e+16 ||r(i)||/||b|| 5.784869342240e+00 >>> 5 KSP preconditioned resid norm 2.594552595600e+17 true resid norm >>> 4.469892332113e+16 ||r(i)||/||b|| 5.239894710780e+00 >>> 6 KSP preconditioned resid norm 2.219662091794e+17 true resid norm >>> 4.652217437246e+16 ||r(i)||/||b|| 5.453627902330e+00 >>> 7 KSP preconditioned resid norm 1.674952385698e+17 true resid norm >>> 2.988890119169e+16 ||r(i)||/||b|| 3.503768852333e+00 >>> 8 KSP preconditioned resid norm 1.081926357303e+17 true resid norm >>> 1.908191297028e+16 ||r(i)||/||b|| 2.236904323761e+00 >>> 9 KSP preconditioned resid norm 5.710239913499e+16 true resid norm >>> 9.635393149266e+15 ||r(i)||/||b|| 1.129522633832e+00 >>> 10 KSP preconditioned resid norm 1.779371916701e+16 true resid norm >>> 2.952475144259e+15 ||r(i)||/||b|| 3.461080881292e-01 >>> 11 KSP preconditioned resid norm 1.125590077314e+16 true resid norm >>> 2.030360591090e+15 ||r(i)||/||b|| 2.380119012218e-01 >>> 12 KSP preconditioned resid norm 1.086608349874e+16 true resid norm >>> 1.877856566597e+15 ||r(i)||/||b|| 2.201344005587e-01 >>> 13 KSP preconditioned resid norm 1.075368648574e+16 true resid norm >>> 1.822636849246e+15 ||r(i)||/||b|| 2.136611908395e-01 >>> 14 KSP preconditioned resid norm 1.173878698877e+16 true resid norm >>> 2.130421173676e+15 ||r(i)||/||b|| 2.497416449940e-01 >>> 15 KSP preconditioned resid norm 4.944017897832e+16 true resid norm >>> 1.361808358525e+16 ||r(i)||/||b|| 1.596399171332e+00 >>> 16 KSP preconditioned resid norm 1.780572750257e+17 true resid norm >>> 3.371295639234e+16 ||r(i)||/||b|| 3.952049149280e+00 >>> 17 KSP preconditioned resid norm 3.251503916379e+16 true resid norm >>> 8.719191629936e+15 ||r(i)||/||b|| 1.022119610706e+00 >>> 18 KSP preconditioned resid norm 2.075735168556e+16 true resid norm >>> 3.945307122742e+15 ||r(i)||/||b|| 4.624942255619e-01 >>> 19 KSP preconditioned resid norm 1.483613863843e+16 true resid norm >>> 3.390657011871e+15 ||r(i)||/||b|| 3.974745793076e-01 >>> 20 KSP preconditioned resid norm 1.134405248775e+16 true resid norm >>> 2.490065225313e+15 ||r(i)||/||b|| 2.919014292554e-01 >>> 21 KSP preconditioned resid norm 8.553736210329e+15 true resid norm >>> 1.657655420050e+15 ||r(i)||/||b|| 1.943210087055e-01 >>> 22 KSP preconditioned resid norm 6.586515952486e+15 true resid norm >>> 1.286392357530e+15 ||r(i)||/||b|| 1.507991694069e-01 >>> 23 KSP preconditioned resid norm 4.780127454522e+15 true resid norm >>> 9.064248964802e+14 ||r(i)||/||b|| 1.062569446397e-01 >>> 24 KSP preconditioned resid norm 3.027986246405e+15 true resid norm >>> 6.179735775190e+14 ||r(i)||/||b|| 7.244282948339e-02 >>> 25 KSP preconditioned resid norm 1.571830581724e+15 true resid norm >>> 2.758055375192e+14 ||r(i)||/||b|| 3.233169548331e-02 >>> 26 KSP preconditioned resid norm 1.182949949046e+15 true resid norm >>> 2.283703518838e+14 ||r(i)||/||b|| 2.677103854019e-02 >>> 27 KSP preconditioned resid norm 1.078564628165e+15 true resid norm >>> 2.041848112402e+14 ||r(i)||/||b|| 2.393585422076e-02 >>> 28 KSP preconditioned resid norm 1.016575362098e+15 true resid norm >>> 1.676746397025e+14 ||r(i)||/||b|| 1.965589755702e-02 >>> 29 KSP preconditioned resid norm 9.501065512919e+14 true resid norm >>> 1.807838083536e+14 ||r(i)||/||b|| 2.119263845308e-02 >>> 30 KSP preconditioned resid norm 9.022105299639e+14 true resid norm >>> 1.464542351381e+14 ||r(i)||/||b|| 1.716830552177e-02 >>> 31 KSP preconditioned resid norm 8.706508374697e+14 true resid norm >>> 1.587763993315e+14 ||r(i)||/||b|| 1.861278870358e-02 >>> 32 KSP preconditioned resid norm 8.363545887412e+14 true resid norm >>> 1.341498283380e+14 ||r(i)||/||b|| 1.572590397559e-02 >>> 33 KSP preconditioned resid norm 8.049250107078e+14 true resid norm >>> 1.594310708223e+14 ||r(i)||/||b|| 1.868953349802e-02 >>> 34 KSP preconditioned resid norm 7.730163706333e+14 true resid norm >>> 1.360746921537e+14 ||r(i)||/||b|| 1.595154886763e-02 >>> 35 KSP preconditioned resid norm 7.656443951451e+14 true resid norm >>> 1.202971650940e+14 ||r(i)||/||b|| 1.410200587090e-02 >>> 36 KSP preconditioned resid norm 7.731881795958e+14 true resid norm >>> 1.274146022068e+14 ||r(i)||/||b|| 1.493635753557e-02 >>> 37 KSP preconditioned resid norm 8.066446225342e+14 true resid norm >>> 1.271087970298e+14 ||r(i)||/||b|| 1.490050908979e-02 >>> 38 KSP preconditioned resid norm 8.449932808607e+14 true resid norm >>> 1.374808484319e+14 ||r(i)||/||b|| 1.611638753257e-02 >>> 39 KSP preconditioned resid norm 8.841110986186e+14 true resid norm >>> 1.396540752943e+14 ||r(i)||/||b|| 1.637114713515e-02 >>> 40 KSP preconditioned resid norm 9.488359303223e+14 true resid norm >>> 1.529468173159e+14 ||r(i)||/||b|| 1.792940768006e-02 >>> 41 KSP preconditioned resid norm 1.207535627829e+15 true resid norm >>> 1.911287075526e+14 ||r(i)||/||b|| 2.240533394033e-02 >>> 42 KSP preconditioned resid norm 2.145532208476e+15 true resid norm >>> 3.678432914805e+14 ||r(i)||/||b|| 4.312095178618e-02 >>> 43 KSP preconditioned resid norm 1.285566323841e+16 true resid norm >>> 2.041241798720e+15 ||r(i)||/||b|| 2.392874662258e-01 >>> 44 KSP preconditioned resid norm 1.228721458245e+15 true resid norm >>> 2.037912017178e+14 ||r(i)||/||b|| 2.388971278599e-02 >>> 45 KSP preconditioned resid norm 6.120797893644e+14 true resid norm >>> 9.792977962736e+13 ||r(i)||/||b|| 1.147995737192e-02 >>> 46 KSP preconditioned resid norm 4.263304414021e+14 true resid norm >>> 6.948352885735e+13 ||r(i)||/||b|| 8.145305262283e-03 >>> 47 KSP preconditioned resid norm 2.943667166575e+14 true resid norm >>> 4.740665276317e+13 ||r(i)||/||b|| 5.557312136691e-03 >>> 48 KSP preconditioned resid norm 1.946102625201e+14 true resid norm >>> 3.133411275125e+13 ||r(i)||/||b|| 3.673185828050e-03 >>> 49 KSP preconditioned resid norm 1.369383238938e+14 true resid norm >>> 2.605829767015e+13 ||r(i)||/||b|| 3.054720919178e-03 >>> 50 KSP preconditioned resid norm 9.297342410318e+13 true resid norm >>> 1.484605459745e+13 ||r(i)||/||b|| 1.740349815638e-03 >>> 51 KSP preconditioned resid norm 6.674823642018e+13 true resid norm >>> 2.111621172189e+13 ||r(i)||/||b|| 2.475377881439e-03 >>> 52 KSP preconditioned resid norm 5.794642992605e+13 true resid norm >>> 9.469052681739e+12 ||r(i)||/||b|| 1.110023136501e-03 >>> 53 KSP preconditioned resid norm 4.582540149696e+13 true resid norm >>> 7.443523694856e+12 ||r(i)||/||b|| 8.725776269382e-04 >>> 0 KSP preconditioned resid norm 1.892653228182e+19 true resid norm >>> 1.028330526045e+19 ||r(i)||/||b|| 7.841194129911e+01 >>> 1 KSP preconditioned resid norm 4.581281818866e+18 true resid norm >>> 1.519814562161e+18 ||r(i)||/||b|| 1.158884300480e+01 >>> 2 KSP preconditioned resid norm 1.874494262231e+18 true resid norm >>> 4.646649902787e+17 ||r(i)||/||b|| 3.543149115843e+00 >>> 3 KSP preconditioned resid norm 1.173263764111e+18 true resid norm >>> 2.551017670341e+17 ||r(i)||/||b|| 1.945194105918e+00 >>> 4 KSP preconditioned resid norm 8.301325588811e+17 true resid norm >>> 1.381381450838e+17 ||r(i)||/||b|| 1.053326712486e+00 >>> 5 KSP preconditioned resid norm 7.052947600755e+17 true resid norm >>> 1.287824897888e+17 ||r(i)||/||b|| 9.819882590194e-01 >>> 6 KSP preconditioned resid norm 5.863848966069e+17 true resid norm >>> 1.166961566059e+17 ||r(i)||/||b|| 8.898279249584e-01 >>> 7 KSP preconditioned resid norm 4.330736686179e+17 true resid norm >>> 8.611330077675e+16 ||r(i)||/||b|| 6.566284783504e-01 >>> 8 KSP preconditioned resid norm 2.702589924096e+17 true resid norm >>> 4.803726753137e+16 ||r(i)||/||b|| 3.662922870069e-01 >>> 9 KSP preconditioned resid norm 1.546977321412e+17 true resid norm >>> 2.878601764016e+16 ||r(i)||/||b|| 2.194982516096e-01 >>> 10 KSP preconditioned resid norm 9.910027101694e+16 true resid norm >>> 1.648820746608e+16 ||r(i)||/||b|| 1.257253697341e-01 >>> 11 KSP preconditioned resid norm 9.241668505040e+16 true resid norm >>> 1.559203946128e+16 ||r(i)||/||b|| 1.188919371745e-01 >>> 12 KSP preconditioned resid norm 9.244329678182e+16 true resid norm >>> 1.506886748242e+16 ||r(i)||/||b|| 1.149026623784e-01 >>> 13 KSP preconditioned resid norm 7.090985777634e+16 true resid norm >>> 1.150411231671e+16 ||r(i)||/||b|| 8.772080151562e-02 >>> 14 KSP preconditioned resid norm 2.014608975967e+17 true resid norm >>> 3.263298275594e+16 ||r(i)||/||b|| 2.488320110574e-01 >>> 15 KSP preconditioned resid norm 1.667109940317e+16 true resid norm >>> 4.390284971711e+15 ||r(i)||/||b|| 3.347666521311e-02 >>> 16 KSP preconditioned resid norm 1.169352181037e+16 true resid norm >>> 1.861078956987e+15 ||r(i)||/||b|| 1.419104171589e-02 >>> 17 KSP preconditioned resid norm 1.018452985290e+16 true resid norm >>> 1.996695440591e+15 ||r(i)||/||b|| 1.522514033324e-02 >>> 18 KSP preconditioned resid norm 9.390559252345e+15 true resid norm >>> 1.536845912378e+15 ||r(i)||/||b|| 1.171870992984e-02 >>> 19 KSP preconditioned resid norm 8.796086996183e+15 true resid norm >>> 1.422370571244e+15 ||r(i)||/||b|| 1.084581609835e-02 >>> 20 KSP preconditioned resid norm 8.297954532675e+15 true resid norm >>> 1.485654737712e+15 ||r(i)||/||b|| 1.132836856767e-02 >>> 21 KSP preconditioned resid norm 7.837153561450e+15 true resid norm >>> 1.247795221846e+15 ||r(i)||/||b|| 9.514649542211e-03 >>> 22 KSP preconditioned resid norm 7.441474574169e+15 true resid norm >>> 1.321880544976e+15 ||r(i)||/||b|| 1.007956265733e-02 >>> 23 KSP preconditioned resid norm 7.117292465449e+15 true resid norm >>> 1.122736430687e+15 ||r(i)||/||b|| 8.561055114844e-03 >>> 24 KSP preconditioned resid norm 5.694316647476e+15 true resid norm >>> 2.071287152890e+15 ||r(i)||/||b|| 1.579391475139e-02 >>> 25 KSP preconditioned resid norm 4.803477075252e+15 true resid norm >>> 7.688470974380e+14 ||r(i)||/||b|| 5.862589113657e-03 >>> 26 KSP preconditioned resid norm 4.890644639484e+15 true resid norm >>> 7.629702578172e+14 ||r(i)||/||b|| 5.817777217899e-03 >>> 27 KSP preconditioned resid norm 5.168669214975e+15 true resid norm >>> 8.754612707062e+14 ||r(i)||/||b|| 6.675540211016e-03 >>> 28 KSP preconditioned resid norm 8.939764002038e+15 true resid norm >>> 1.397979783339e+15 ||r(i)||/||b|| 1.065983221661e-02 >>> 29 KSP preconditioned resid norm 8.205557149295e+15 true resid norm >>> 1.863983796228e+15 ||r(i)||/||b|| 1.421319160626e-02 >>> 30 KSP preconditioned resid norm 1.967760781914e+15 true resid norm >>> 3.069470515034e+14 ||r(i)||/||b|| 2.340523165932e-03 >>> 31 KSP preconditioned resid norm 1.158003829729e+15 true resid norm >>> 2.612136672864e+14 ||r(i)||/||b|| 1.991798378735e-03 >>> 32 KSP preconditioned resid norm 8.020817038350e+14 true resid norm >>> 1.249397423467e+14 ||r(i)||/||b|| 9.526866600465e-04 >>> 0 KSP preconditioned resid norm 6.470845423033e+18 true resid norm >>> 3.511573094792e+18 ||r(i)||/||b|| 1.662215647083e+02 >>> 1 KSP preconditioned resid norm 1.571609661901e+18 true resid norm >>> 5.187508694693e+17 ||r(i)||/||b|| 2.455525739870e+01 >>> 2 KSP preconditioned resid norm 6.468393005498e+17 true resid norm >>> 1.566999779328e+17 ||r(i)||/||b|| 7.417449336415e+00 >>> 3 KSP preconditioned resid norm 4.084082755984e+17 true resid norm >>> 8.688467105425e+16 ||r(i)||/||b|| 4.112716888400e+00 >>> 4 KSP preconditioned resid norm 2.920922708944e+17 true resid norm >>> 4.832004038605e+16 ||r(i)||/||b|| 2.287246343141e+00 >>> 5 KSP preconditioned resid norm 2.515891595070e+17 true resid norm >>> 4.435607154638e+16 ||r(i)||/||b|| 2.099610464520e+00 >>> 6 KSP preconditioned resid norm 2.133101049372e+17 true resid norm >>> 4.355693207474e+16 ||r(i)||/||b|| 2.061782912647e+00 >>> 7 KSP preconditioned resid norm 1.601303798951e+17 true resid norm >>> 2.999351550015e+16 ||r(i)||/||b|| 1.419753752223e+00 >>> 8 KSP preconditioned resid norm 1.019899000725e+17 true resid norm >>> 1.811131044506e+16 ||r(i)||/||b|| 8.573053386130e-01 >>> 9 KSP preconditioned resid norm 5.552370825926e+16 true resid norm >>> 9.849613114257e+15 ||r(i)||/||b|| 4.662349492457e-01 >>> 10 KSP preconditioned resid norm 2.614277580792e+16 true resid norm >>> 4.400858558792e+15 ||r(i)||/||b|| 2.083162092759e-01 >>> 11 KSP preconditioned resid norm 2.230379890809e+16 true resid norm >>> 4.001941486401e+15 ||r(i)||/||b|| 1.894333273960e-01 >>> 12 KSP preconditioned resid norm 2.202819377737e+16 true resid norm >>> 3.668912744052e+15 ||r(i)||/||b|| 1.736692931151e-01 >>> 13 KSP preconditioned resid norm 2.171381974288e+16 true resid norm >>> 3.746629440349e+15 ||r(i)||/||b|| 1.773480406489e-01 >>> 14 KSP preconditioned resid norm 2.756519085776e+16 true resid norm >>> 4.738155898553e+15 ||r(i)||/||b|| 2.242822991374e-01 >>> 15 KSP preconditioned resid norm 1.910971370209e+16 true resid norm >>> 7.272906924885e+15 ||r(i)||/||b|| 3.442656428895e-01 >>> 16 KSP preconditioned resid norm 8.265967788035e+15 true resid norm >>> 1.450280377529e+15 ||r(i)||/||b|| 6.864953885656e-02 >>> 17 KSP preconditioned resid norm 5.450920356042e+15 true resid norm >>> 1.778917951827e+15 ||r(i)||/||b|| 8.420571563182e-02 >>> 18 KSP preconditioned resid norm 4.184159413509e+15 true resid norm >>> 7.350039137637e+14 ||r(i)||/||b|| 3.479167236863e-02 >>> 19 KSP preconditioned resid norm 3.481860538622e+15 true resid norm >>> 7.678205813365e+14 ||r(i)||/||b|| 3.634506103098e-02 >>> 20 KSP preconditioned resid norm 3.027493421078e+15 true resid norm >>> 5.662352682787e+14 ||r(i)||/||b|| 2.680294834981e-02 >>> 21 KSP preconditioned resid norm 2.649808113090e+15 true resid norm >>> 5.331458151920e+14 ||r(i)||/||b|| 2.523664728788e-02 >>> 22 KSP preconditioned resid norm 2.408644968014e+15 true resid norm >>> 3.911896234018e+14 ||r(i)||/||b|| 1.851710032633e-02 >>> 23 KSP preconditioned resid norm 2.245371706567e+15 true resid norm >>> 3.514281238053e+14 ||r(i)||/||b|| 1.663497556353e-02 >>> 24 KSP preconditioned resid norm 2.104176299070e+15 true resid norm >>> 3.427795705053e+14 ||r(i)||/||b|| 1.622559320890e-02 >>> 25 KSP preconditioned resid norm 1.992290690493e+15 true resid norm >>> 3.118626867030e+14 ||r(i)||/||b|| 1.476213148881e-02 >>> 26 KSP preconditioned resid norm 1.936852873636e+15 true resid norm >>> 3.191431362940e+14 ||r(i)||/||b|| 1.510675416649e-02 >>> 27 KSP preconditioned resid norm 1.840320660654e+15 true resid norm >>> 4.277077423449e+14 ||r(i)||/||b|| 2.024569850927e-02 >>> 28 KSP preconditioned resid norm 1.797325493063e+15 true resid norm >>> 2.805438081480e+14 ||r(i)||/||b|| 1.327964120375e-02 >>> 29 KSP preconditioned resid norm 1.907733126356e+15 true resid norm >>> 3.072709450643e+14 ||r(i)||/||b|| 1.454477976088e-02 >>> 30 KSP preconditioned resid norm 2.241086833192e+15 true resid norm >>> 3.515931723170e+14 ||r(i)||/||b|| 1.664278819369e-02 >>> 31 KSP preconditioned resid norm 3.121906541214e+15 true resid norm >>> 5.044161632328e+14 ||r(i)||/||b|| 2.387671896708e-02 >>> 32 KSP preconditioned resid norm 2.288516501159e+16 true resid norm >>> 3.696560518057e+15 ||r(i)||/||b|| 1.749780103571e-01 >>> 33 KSP preconditioned resid norm 2.665344016377e+15 true resid norm >>> 4.419048360378e+14 ||r(i)||/||b|| 2.091772300206e-02 >>> 34 KSP preconditioned resid norm 1.193724857597e+15 true resid norm >>> 1.964465911853e+14 ||r(i)||/||b|| 9.298869448811e-03 >>> 35 KSP preconditioned resid norm 7.370892713374e+14 true resid norm >>> 1.243994812053e+14 ||r(i)||/||b|| 5.888493805102e-03 >>> 36 KSP preconditioned resid norm 5.570336976172e+14 true resid norm >>> 9.038862530299e+13 ||r(i)||/||b|| 4.278577812313e-03 >>> 37 KSP preconditioned resid norm 4.452876529911e+14 true resid norm >>> 9.072905275923e+13 ||r(i)||/||b|| 4.294692067355e-03 >>> 38 KSP preconditioned resid norm 4.015373409323e+14 true resid norm >>> 6.374984875718e+13 ||r(i)||/||b|| 3.017621824831e-03 >>> 39 KSP preconditioned resid norm 3.753907061691e+14 true resid norm >>> 7.170117945509e+13 ||r(i)||/||b|| 3.394000898950e-03 >>> 40 KSP preconditioned resid norm 3.520150126535e+14 true resid norm >>> 5.546554172401e+13 ||r(i)||/||b|| 2.625481197140e-03 >>> 41 KSP preconditioned resid norm 3.283953946659e+14 true resid norm >>> 5.672854000420e+13 ||r(i)||/||b|| 2.685265671132e-03 >>> 42 KSP preconditioned resid norm 3.086422594501e+14 true resid norm >>> 5.033158713486e+13 ||r(i)||/||b|| 2.382463625836e-03 >>> 43 KSP preconditioned resid norm 2.903330837015e+14 true resid norm >>> 5.711996083788e+13 ||r(i)||/||b|| 2.703793715879e-03 >>> 44 KSP preconditioned resid norm 2.805776139313e+14 true resid norm >>> 4.359341431583e+13 ||r(i)||/||b|| 2.063509812540e-03 >>> 45 KSP preconditioned resid norm 2.850165946773e+14 true resid norm >>> 4.589953632859e+13 ||r(i)||/||b|| 2.172670920403e-03 >>> 46 KSP preconditioned resid norm 3.036441891069e+14 true resid norm >>> 4.745822530443e+13 ||r(i)||/||b|| 2.246452018920e-03 >>> 47 KSP preconditioned resid norm 3.630944657131e+14 true resid norm >>> 5.763700439451e+13 ||r(i)||/||b|| 2.728268157016e-03 >>> 48 KSP preconditioned resid norm 1.221109219167e+15 true resid norm >>> 1.980487895228e+14 ||r(i)||/||b|| 9.374710078478e-03 >>> 49 KSP preconditioned resid norm 3.188684128549e+15 true resid norm >>> 5.020159499531e+14 ||r(i)||/||b|| 2.376310401554e-02 >>> 50 KSP preconditioned resid norm 3.895126724245e+13 true resid norm >>> 6.299322456844e+12 ||r(i)||/||b|| 2.981806749036e-04 >>> 0 KSP preconditioned resid norm 6.694681597053e+18 true resid norm >>> 3.650754072115e+18 ||r(i)||/||b|| 5.229798617888e+00 >>> 1 KSP preconditioned resid norm 1.597667334483e+18 true resid norm >>> 5.366749713509e+17 ||r(i)||/||b|| 7.688006280303e-01 >>> 2 KSP preconditioned resid norm 6.300446759111e+17 true resid norm >>> 1.705328267334e+17 ||r(i)||/||b|| 2.442926376135e-01 >>> 3 KSP preconditioned resid norm 3.642174486049e+17 true resid norm >>> 9.481576844129e+16 ||r(i)||/||b|| 1.358260142845e-01 >>> 4 KSP preconditioned resid norm 2.247332959635e+17 true resid norm >>> 3.802800729856e+16 ||r(i)||/||b|| 5.447609345427e-02 >>> 5 KSP preconditioned resid norm 1.557523701769e+17 true resid norm >>> 3.829141999541e+16 ||r(i)||/||b|| 5.485343888228e-02 >>> 6 KSP preconditioned resid norm 1.004692846569e+17 true resid norm >>> 1.699728099873e+16 ||r(i)||/||b|| 2.434903992959e-02 >>> 7 KSP preconditioned resid norm 7.103482802189e+16 true resid norm >>> 1.472858884963e+16 ||r(i)||/||b|| 2.109908037838e-02 >>> 8 KSP preconditioned resid norm 5.766847885215e+16 true resid norm >>> 9.397954062732e+15 ||r(i)||/||b|| 1.346280965449e-02 >>> 9 KSP preconditioned resid norm 4.523472016725e+16 true resid norm >>> 1.246374917365e+16 ||r(i)||/||b|| 1.785463959348e-02 >>> 10 KSP preconditioned resid norm 3.195342481960e+16 true resid norm >>> 5.030521484214e+15 ||r(i)||/||b|| 7.206350738975e-03 >>> 11 KSP preconditioned resid norm 2.437460220226e+16 true resid norm >>> 4.974645944208e+15 ||r(i)||/||b|| 7.126307598263e-03 >>> 12 KSP preconditioned resid norm 1.258045555577e+16 true resid norm >>> 1.885910304237e+15 ||r(i)||/||b|| 2.701614764439e-03 >>> 13 KSP preconditioned resid norm 2.495567167302e+15 true resid norm >>> 4.460991814541e+14 ||r(i)||/||b|| 6.390484914966e-04 >>> 14 KSP preconditioned resid norm 7.678096351052e+14 true resid norm >>> 1.316276220022e+14 ||r(i)||/||b|| 1.885599363926e-04 >>> 15 KSP preconditioned resid norm 5.950867501843e+14 true resid norm >>> 1.439410416931e+14 ||r(i)||/||b|| 2.061992251557e-04 >>> 16 KSP preconditioned resid norm 3.263035989760e+14 true resid norm >>> 5.046543721443e+13 ||r(i)||/||b|| 7.229303003757e-05 >>> 17 KSP preconditioned resid norm 1.436782488882e+14 true resid norm >>> 7.588970356348e+13 ||r(i)||/||b|| 1.087139420975e-04 >>> 18 KSP preconditioned resid norm 5.401533739797e+13 true resid norm >>> 8.625318785849e+12 ||r(i)||/||b|| 1.235598985142e-05 >>> 19 KSP preconditioned resid norm 3.528859989845e+13 true resid norm >>> 1.233587622450e+13 ||r(i)||/||b|| 1.767145832202e-05 >>> 0 KSP preconditioned resid norm 4.827874033841e+18 true resid norm >>> 2.629049503905e+18 ||r(i)||/||b|| 2.387244450275e+01 >>> 1 KSP preconditioned resid norm 1.161059892070e+18 true resid norm >>> 3.885117260269e+17 ||r(i)||/||b|| 3.527786222537e+00 >>> 2 KSP preconditioned resid norm 4.696931552127e+17 true resid norm >>> 1.228954734891e+17 ||r(i)||/||b|| 1.115922452639e+00 >>> 3 KSP preconditioned resid norm 2.870101730937e+17 true resid norm >>> 6.759381587642e+16 ||r(i)||/||b|| 6.137692028400e-01 >>> 4 KSP preconditioned resid norm 1.966639305793e+17 true resid norm >>> 3.310590026030e+16 ||r(i)||/||b|| 3.006100742887e-01 >>> 5 KSP preconditioned resid norm 1.592574497021e+17 true resid norm >>> 3.239415772705e+16 ||r(i)||/||b|| 2.941472693472e-01 >>> 6 KSP preconditioned resid norm 1.230114824013e+17 true resid norm >>> 2.280737281616e+16 ||r(i)||/||b|| 2.070968009535e-01 >>> 7 KSP preconditioned resid norm 8.598640818077e+16 true resid norm >>> 1.938029194893e+16 ||r(i)||/||b|| 1.759780267776e-01 >>> 8 KSP preconditioned resid norm 5.549085768793e+16 true resid norm >>> 9.386094648379e+15 ||r(i)||/||b|| 8.522814928290e-02 >>> 9 KSP preconditioned resid norm 4.474754629124e+16 true resid norm >>> 7.579167807628e+15 ||r(i)||/||b|| 6.882078964122e-02 >>> 10 KSP preconditioned resid norm 4.378389530908e+16 true resid norm >>> 6.842593278076e+15 ||r(i)||/||b|| 6.213250379770e-02 >>> 11 KSP preconditioned resid norm 4.079498014536e+16 true resid norm >>> 7.704710526729e+15 ||r(i)||/||b|| 6.996074976369e-02 >>> 12 KSP preconditioned resid norm 4.084398125729e+16 true resid norm >>> 6.377957983596e+15 ||r(i)||/||b|| 5.791349602893e-02 >>> 13 KSP preconditioned resid norm 7.523960017073e+16 true resid norm >>> 1.132262466824e+16 ||r(i)||/||b|| 1.028123390664e-01 >>> 14 KSP preconditioned resid norm 2.184546755774e+15 true resid norm >>> 3.502101254985e+14 ||r(i)||/||b|| 3.179997855820e-03 >>> 15 KSP preconditioned resid norm 2.222188316359e+15 true resid norm >>> 3.654675514798e+14 ||r(i)||/||b|| 3.318539201068e-03 >>> 16 KSP preconditioned resid norm 1.669487598659e+15 true resid norm >>> 3.629534674524e+14 ||r(i)||/||b|| 3.295710672608e-03 >>> 17 KSP preconditioned resid norm 1.401917540926e+15 true resid norm >>> 2.161165666331e+14 ||r(i)||/||b|| 1.962393912861e-03 >>> 18 KSP preconditioned resid norm 1.134393288674e+15 true resid norm >>> 2.686682412462e+14 ||r(i)||/||b|| 2.439576610967e-03 >>> 19 KSP preconditioned resid norm 9.058009317761e+14 true resid norm >>> 1.409929037370e+14 ||r(i)||/||b|| 1.280251765797e-03 >>> 20 KSP preconditioned resid norm 7.990098269893e+14 true resid norm >>> 1.659151006791e+14 ||r(i)||/||b|| 1.506551712794e-03 >>> 21 KSP preconditioned resid norm 7.119180202197e+14 true resid norm >>> 1.092130755589e+14 ||r(i)||/||b|| 9.916827664832e-04 >>> 22 KSP preconditioned resid norm 6.306959900843e+14 true resid norm >>> 1.460783745609e+14 ||r(i)||/||b|| 1.326429146571e-03 >>> 23 KSP preconditioned resid norm 5.637228851054e+14 true resid norm >>> 8.672640572399e+13 ||r(i)||/||b|| 7.874980309399e-04 >>> 24 KSP preconditioned resid norm 4.960976586203e+14 true resid norm >>> 1.292271181476e+14 ||r(i)||/||b|| 1.173415411786e-03 >>> 25 KSP preconditioned resid norm 4.386404759546e+14 true resid norm >>> 8.037967670881e+13 ||r(i)||/||b|| 7.298681019622e-04 >>> 26 KSP preconditioned resid norm 4.419824575339e+14 true resid norm >>> 6.704822670651e+13 ||r(i)||/||b|| 6.088151130974e-04 >>> 27 KSP preconditioned resid norm 4.437076214979e+14 true resid norm >>> 1.370393448160e+14 ||r(i)||/||b|| 1.244352435720e-03 >>> 28 KSP preconditioned resid norm 7.848825938920e+14 true resid norm >>> 1.196984132854e+14 ||r(i)||/||b|| 1.086892325146e-03 >>> 29 KSP preconditioned resid norm 1.771840029177e+15 true resid norm >>> 6.554951997284e+14 ||r(i)||/||b|| 5.952064711635e-03 >>> 30 KSP preconditioned resid norm 1.052090974187e+15 true resid norm >>> 1.877636782214e+14 ||r(i)||/||b|| 1.704942406491e-03 >>> 31 KSP preconditioned resid norm 2.459238084389e+14 true resid norm >>> 3.913335964196e+13 ||r(i)||/||b|| 3.553409530216e-04 >>> 32 KSP preconditioned resid norm 1.091607449298e+14 true resid norm >>> 1.900052367477e+13 ||r(i)||/||b|| 1.725296333429e-04 >>> 33 KSP preconditioned resid norm 6.424117612547e+13 true resid norm >>> 1.001417309383e+13 ||r(i)||/||b|| 9.093126282646e-05 >>> 34 KSP preconditioned resid norm 3.801393062657e+13 true resid norm >>> 7.035998067262e+12 ||r(i)||/||b|| 6.388866894011e-05 >>> 35 KSP preconditioned resid norm 2.524483314829e+13 true resid norm >>> 4.006295820236e+12 ||r(i)||/||b|| 3.637819466241e-05 >>> 36 KSP preconditioned resid norm 1.579134906712e+13 true resid norm >>> 2.528645237998e+12 ||r(i)||/||b|| 2.296074798956e-05 >>> 37 KSP preconditioned resid norm 1.074219589287e+13 true resid norm >>> 1.786181249150e+12 ||r(i)||/||b|| 1.621898434352e-05 >>> 38 KSP preconditioned resid norm 6.572405677048e+12 true resid norm >>> 1.033900747383e+12 ||r(i)||/||b|| 9.388084239790e-06 >>> 0 KSP preconditioned resid norm 9.565416792628e+15 true resid norm >>> 5.898678788482e+15 ||r(i)||/||b|| 1.120363504643e-01 >>> 1 KSP preconditioned resid norm 4.584205668003e+15 true resid norm >>> 8.149165580746e+14 ||r(i)||/||b|| 1.547808931009e-02 >>> 2 KSP preconditioned resid norm 3.775899089492e+15 true resid norm >>> 6.134705686008e+14 ||r(i)||/||b|| 1.165193191355e-02 >>> 3 KSP preconditioned resid norm 3.695846031998e+15 true resid norm >>> 6.195110682482e+14 ||r(i)||/||b|| 1.176666193357e-02 >>> 4 KSP preconditioned resid norm 3.863984235758e+15 true resid norm >>> 6.333337603322e+14 ||r(i)||/||b|| 1.202920275504e-02 >>> 5 KSP preconditioned resid norm 4.289553507880e+15 true resid norm >>> 6.781260017663e+14 ||r(i)||/||b|| 1.287996263523e-02 >>> 6 KSP preconditioned resid norm 5.214022047398e+15 true resid norm >>> 7.772905386519e+14 ||r(i)||/||b|| 1.476344081849e-02 >>> 7 KSP preconditioned resid norm 6.520856882448e+15 true resid norm >>> 1.156631144536e+15 ||r(i)||/||b|| 2.196843342618e-02 >>> 8 KSP preconditioned resid norm 7.245755589579e+15 true resid norm >>> 1.125518734778e+15 ||r(i)||/||b|| 2.137750095326e-02 >>> 9 KSP preconditioned resid norm 7.402293382659e+15 true resid norm >>> 1.342363548018e+15 ||r(i)||/||b|| 2.549613537355e-02 >>> 10 KSP preconditioned resid norm 6.814780966449e+15 true resid norm >>> 1.040341685629e+15 ||r(i)||/||b|| 1.975969363197e-02 >>> 11 KSP preconditioned resid norm 4.531806636058e+15 true resid norm >>> 6.813426758243e+14 ||r(i)||/||b|| 1.294105842210e-02 >>> 12 KSP preconditioned resid norm 4.019547715129e+15 true resid norm >>> 1.067806433320e+15 ||r(i)||/||b|| 2.028134436224e-02 >>> 13 KSP preconditioned resid norm 1.578855518519e+15 true resid norm >>> 2.462140978868e+14 ||r(i)||/||b|| 4.676458907031e-03 >>> 14 KSP preconditioned resid norm 1.216192317102e+15 true resid norm >>> 3.547355821423e+14 ||r(i)||/||b|| 6.737657944804e-03 >>> 15 KSP preconditioned resid norm 1.068138625918e+15 true resid norm >>> 1.711364338225e+14 ||r(i)||/||b|| 3.250473905172e-03 >>> 16 KSP preconditioned resid norm 1.009888628733e+15 true resid norm >>> 1.725105447584e+14 ||r(i)||/||b|| 3.276573033454e-03 >>> 17 KSP preconditioned resid norm 9.012147556868e+14 true resid norm >>> 1.461953215911e+14 ||r(i)||/||b|| 2.776755757239e-03 >>> 18 KSP preconditioned resid norm 8.184497465452e+14 true resid norm >>> 1.394097869305e+14 ||r(i)||/||b|| 2.647874940607e-03 >>> 19 KSP preconditioned resid norm 6.946418330368e+14 true resid norm >>> 1.094539907094e+14 ||r(i)||/||b|| 2.078910566681e-03 >>> 20 KSP preconditioned resid norm 5.822189800664e+14 true resid norm >>> 1.086460879389e+14 ||r(i)||/||b|| 2.063565693502e-03 >>> 21 KSP preconditioned resid norm 4.765744627590e+14 true resid norm >>> 7.425312666441e+13 ||r(i)||/||b|| 1.410324179423e-03 >>> 22 KSP preconditioned resid norm 3.785099184439e+14 true resid norm >>> 8.229511757789e+13 ||r(i)||/||b|| 1.563069454208e-03 >>> 23 KSP preconditioned resid norm 3.069654165781e+14 true resid norm >>> 4.744586000483e+13 ||r(i)||/||b|| 9.011612922478e-04 >>> 0 KSP preconditioned resid norm 9.591954950201e+15 true resid norm >>> 5.933293318674e+15 ||r(i)||/||b|| 5.704140552403e-01 >>> 1 KSP preconditioned resid norm 4.627616930759e+15 true resid norm >>> 8.120897483902e+14 ||r(i)||/||b|| 7.807256134471e-02 >>> 2 KSP preconditioned resid norm 3.843607704549e+15 true resid norm >>> 6.216441009903e+14 ||r(i)||/||b|| 5.976352651335e-02 >>> 3 KSP preconditioned resid norm 3.802257051246e+15 true resid norm >>> 6.332758127660e+14 ||r(i)||/||b|| 6.088177425993e-02 >>> 4 KSP preconditioned resid norm 4.025094330467e+15 true resid norm >>> 6.488587775849e+14 ||r(i)||/||b|| 6.237988697367e-02 >>> 5 KSP preconditioned resid norm 4.518903996358e+15 true resid norm >>> 7.100173385586e+14 ||r(i)||/||b|| 6.825953945399e-02 >>> 6 KSP preconditioned resid norm 5.566041493723e+15 true resid norm >>> 8.272295550982e+14 ||r(i)||/||b|| 7.952806979104e-02 >>> 7 KSP preconditioned resid norm 7.050235529471e+15 true resid norm >>> 1.259531120704e+15 ||r(i)||/||b|| 1.210886122891e-01 >>> 8 KSP preconditioned resid norm 7.824426472314e+15 true resid norm >>> 1.230246429464e+15 ||r(i)||/||b|| 1.182732450740e-01 >>> 9 KSP preconditioned resid norm 8.110342869221e+15 true resid norm >>> 1.508507345577e+15 ||r(i)||/||b|| 1.450246509206e-01 >>> 10 KSP preconditioned resid norm 7.848071091100e+15 true resid norm >>> 1.244716713436e+15 ||r(i)||/||b|| 1.196643870448e-01 >>> 11 KSP preconditioned resid norm 6.689698623315e+15 true resid norm >>> 1.152050117302e+15 ||r(i)||/||b|| 1.107556198481e-01 >>> 12 KSP preconditioned resid norm 3.836259156428e+15 true resid norm >>> 5.895445738700e+14 ||r(i)||/||b|| 5.667754703239e-02 >>> 13 KSP preconditioned resid norm 1.583619332521e+15 true resid norm >>> 3.090404276347e+14 ||r(i)||/||b|| 2.971048186772e-02 >>> 14 KSP preconditioned resid norm 1.282447478166e+15 true resid norm >>> 2.666239295029e+14 ||r(i)||/||b|| 2.563265098882e-02 >>> 15 KSP preconditioned resid norm 1.151484666969e+15 true resid norm >>> 2.280596910606e+14 ||r(i)||/||b|| 2.192516806902e-02 >>> 16 KSP preconditioned resid norm 1.027737067851e+15 true resid norm >>> 1.605668418840e+14 ||r(i)||/||b|| 1.543655074795e-02 >>> 17 KSP preconditioned resid norm 8.747586168174e+14 true resid norm >>> 1.470663481317e+14 ||r(i)||/||b|| 1.413864232250e-02 >>> 18 KSP preconditioned resid norm 7.765674155007e+14 true resid norm >>> 1.265290838281e+14 ||r(i)||/||b|| 1.216423391459e-02 >>> 19 KSP preconditioned resid norm 6.633620818064e+14 true resid norm >>> 1.039325479930e+14 ||r(i)||/||b|| 9.991851571798e-03 >>> 20 KSP preconditioned resid norm 5.640296981466e+14 true resid norm >>> 1.018948133357e+14 ||r(i)||/||b|| 9.795948145668e-03 >>> 21 KSP preconditioned resid norm 4.751732697839e+14 true resid norm >>> 7.431313741469e+13 ||r(i)||/||b|| 7.144305159654e-03 >>> 22 KSP preconditioned resid norm 3.865078030089e+14 true resid norm >>> 7.686807394274e+13 ||r(i)||/||b|| 7.389931260973e-03 >>> 23 KSP preconditioned resid norm 3.183297004119e+14 true resid norm >>> 4.953673129366e+13 ||r(i)||/||b|| 4.762354777175e-03 >>> 24 KSP preconditioned resid norm 2.612111917568e+14 true resid norm >>> 5.971714747402e+13 ||r(i)||/||b|| 5.741078087414e-03 >>> 25 KSP preconditioned resid norm 2.296313138344e+14 true resid norm >>> 3.527848689641e+13 ||r(i)||/||b|| 3.391597834880e-03 >>> 26 KSP preconditioned resid norm 1.981337219705e+14 true resid norm >>> 5.164126794406e+13 ||r(i)||/||b|| 4.964680403211e-03 >>> 27 KSP preconditioned resid norm 1.751049134665e+14 true resid norm >>> 2.675806111216e+13 ||r(i)||/||b|| 2.572462430151e-03 >>> 28 KSP preconditioned resid norm 1.533798667157e+14 true resid norm >>> 3.995390488198e+13 ||r(i)||/||b|| 3.841082461690e-03 >>> 29 KSP preconditioned resid norm 1.350471089065e+14 true resid norm >>> 2.063214947275e+13 ||r(i)||/||b|| 1.983530463939e-03 >>> 30 KSP preconditioned resid norm 1.221880384902e+14 true resid norm >>> 2.772903253208e+13 ||r(i)||/||b|| 2.665809533591e-03 >>> 31 KSP preconditioned resid norm 1.099123503540e+14 true resid norm >>> 1.682484274214e+13 ||r(i)||/||b|| 1.617504185596e-03 >>> 32 KSP preconditioned resid norm 1.022845787224e+14 true resid norm >>> 1.853354426028e+13 ||r(i)||/||b|| 1.781775073585e-03 >>> 33 KSP preconditioned resid norm 9.416346598487e+13 true resid norm >>> 1.438140346990e+13 ||r(i)||/||b|| 1.382597190584e-03 >>> 34 KSP preconditioned resid norm 8.782252733286e+13 true resid norm >>> 1.465353724876e+13 ||r(i)||/||b|| 1.408759546637e-03 >>> 35 KSP preconditioned resid norm 8.286435025538e+13 true resid norm >>> 1.264742128603e+13 ||r(i)||/||b|| 1.215895873779e-03 >>> 36 KSP preconditioned resid norm 7.744963280138e+13 true resid norm >>> 1.290903292754e+13 ||r(i)||/||b|| 1.241046654183e-03 >>> 37 KSP preconditioned resid norm 7.340074137876e+13 true resid norm >>> 1.120214736846e+13 ||r(i)||/||b|| 1.076950348592e-03 >>> 38 KSP preconditioned resid norm 6.860608702874e+13 true resid norm >>> 1.142914172047e+13 ||r(i)||/||b|| 1.098773097257e-03 >>> 39 KSP preconditioned resid norm 6.401516623396e+13 true resid norm >>> 9.766911914727e+12 ||r(i)||/||b|| 9.389698997216e-04 >>> 40 KSP preconditioned resid norm 5.980495528832e+13 true resid norm >>> 9.707430083091e+12 ||r(i)||/||b|| 9.332514443927e-04 >>> 41 KSP preconditioned resid norm 5.502664117143e+13 true resid norm >>> 8.404607061129e+12 ||r(i)||/||b|| 8.080008418514e-04 >>> 0 KSP preconditioned resid norm 2.200733964807e+16 true resid norm >>> 1.374508262416e+16 ||r(i)||/||b|| 1.042223674371e-01 >>> 1 KSP preconditioned resid norm 1.033612993855e+16 true resid norm >>> 1.767505005693e+15 ||r(i)||/||b|| 1.340214251070e-02 >>> 2 KSP preconditioned resid norm 8.198008013445e+15 true resid norm >>> 1.328272543525e+15 ||r(i)||/||b|| 1.007165346861e-02 >>> 3 KSP preconditioned resid norm 7.723889011566e+15 true resid norm >>> 1.289831076731e+15 ||r(i)||/||b|| 9.780170267923e-03 >>> 4 KSP preconditioned resid norm 7.785469249486e+15 true resid norm >>> 1.238697760343e+15 ||r(i)||/||b|| 9.392450860579e-03 >>> 5 KSP preconditioned resid norm 8.250177532689e+15 true resid norm >>> 1.288999707195e+15 ||r(i)||/||b|| 9.773866391570e-03 >>> 6 KSP preconditioned resid norm 9.471871112821e+15 true resid norm >>> 1.409248720218e+15 ||r(i)||/||b|| 1.068565696875e-02 >>> 7 KSP preconditioned resid norm 1.106232045691e+16 true resid norm >>> 1.993194297637e+15 ||r(i)||/||b|| 1.511343613874e-02 >>> 8 KSP preconditioned resid norm 1.131471113452e+16 true resid norm >>> 1.768387012694e+15 ||r(i)||/||b|| 1.340883034665e-02 >>> 9 KSP preconditioned resid norm 1.055003108779e+16 true resid norm >>> 1.935924228946e+15 ||r(i)||/||b|| 1.467918468275e-02 >>> 10 KSP preconditioned resid norm 8.569038225337e+15 true resid norm >>> 1.316932619063e+15 ||r(i)||/||b|| 9.985668261662e-03 >>> 11 KSP preconditioned resid norm 4.806783660704e+15 true resid norm >>> 7.346602562794e+14 ||r(i)||/||b|| 5.570576275535e-03 >>> 12 KSP preconditioned resid norm 6.311127886872e+15 true resid norm >>> 1.031465284608e+15 ||r(i)||/||b|| 7.821106415332e-03 >>> 13 KSP preconditioned resid norm 9.706706695861e+14 true resid norm >>> 2.189061895566e+14 ||r(i)||/||b|| 1.659860616781e-03 >>> 14 KSP preconditioned resid norm 6.844878378158e+14 true resid norm >>> 1.339168617782e+14 ||r(i)||/||b|| 1.015427317239e-03 >>> 0 KSP preconditioned resid norm 1.928571496492e+16 true resid norm >>> 1.203387628158e+16 ||r(i)||/||b|| 4.946666172277e-01 >>> 1 KSP preconditioned resid norm 9.232456268915e+15 true resid norm >>> 1.583223656595e+15 ||r(i)||/||b|| 6.508026775390e-02 >>> 2 KSP preconditioned resid norm 7.541947576336e+15 true resid norm >>> 1.216650042172e+15 ||r(i)||/||b|| 5.001182882627e-02 >>> 3 KSP preconditioned resid norm 7.340367763150e+15 true resid norm >>> 1.219887358936e+15 ||r(i)||/||b|| 5.014490253381e-02 >>> 4 KSP preconditioned resid norm 7.655525114233e+15 true resid norm >>> 1.212298323935e+15 ||r(i)||/||b|| 4.983294633744e-02 >>> 5 KSP preconditioned resid norm 8.414914513850e+15 true resid norm >>> 1.311466433986e+15 ||r(i)||/||b|| 5.390936796484e-02 >>> 6 KSP preconditioned resid norm 1.008643582787e+16 true resid norm >>> 1.495807999597e+15 ||r(i)||/||b|| 6.148694451139e-02 >>> 7 KSP preconditioned resid norm 1.235608743666e+16 true resid norm >>> 2.203234082057e+15 ||r(i)||/||b|| 9.056652443733e-02 >>> 8 KSP preconditioned resid norm 1.311751365507e+16 true resid norm >>> 2.061480991281e+15 ||r(i)||/||b|| 8.473959716509e-02 >>> 9 KSP preconditioned resid norm 1.289621791383e+16 true resid norm >>> 2.355792563000e+15 ||r(i)||/||b|| 9.683761996231e-02 >>> 10 KSP preconditioned resid norm 1.151686987354e+16 true resid norm >>> 1.821530468335e+15 ||r(i)||/||b|| 7.487614911976e-02 >>> 11 KSP preconditioned resid norm 8.379707704664e+15 true resid norm >>> 1.435040037498e+15 ||r(i)||/||b|| 5.898900606299e-02 >>> 12 KSP preconditioned resid norm 3.557464484347e+15 true resid norm >>> 5.430083136124e+14 ||r(i)||/||b|| 2.232099444401e-02 >>> 13 KSP preconditioned resid norm 1.204545626126e+15 true resid norm >>> 2.194281091480e+14 ||r(i)||/||b|| 9.019850124522e-03 >>> 14 KSP preconditioned resid norm 1.005673666245e+15 true resid norm >>> 1.883222474780e+14 ||r(i)||/||b|| 7.741207149622e-03 >>> 15 KSP preconditioned resid norm 9.379382960505e+14 true resid norm >>> 1.480571711312e+14 ||r(i)||/||b|| 6.086063898783e-03 >>> 16 KSP preconditioned resid norm 8.825746357103e+14 true resid norm >>> 2.025244331207e+14 ||r(i)||/||b|| 8.325004669617e-03 >>> 17 KSP preconditioned resid norm 7.524485997458e+14 true resid norm >>> 1.181787653170e+14 ||r(i)||/||b|| 4.857876938371e-03 >>> 18 KSP preconditioned resid norm 6.768878630597e+14 true resid norm >>> 1.383948785570e+14 ||r(i)||/||b|| 5.688884015051e-03 >>> 19 KSP preconditioned resid norm 6.018014871695e+14 true resid norm >>> 9.381957375927e+13 ||r(i)||/||b|| 3.856563761775e-03 >>> 20 KSP preconditioned resid norm 5.246326908653e+14 true resid norm >>> 1.170575496120e+14 ||r(i)||/||b|| 4.811788050053e-03 >>> 21 KSP preconditioned resid norm 4.606732721936e+14 true resid norm >>> 7.159556750064e+13 ||r(i)||/||b|| 2.943019884477e-03 >>> 22 KSP preconditioned resid norm 3.919505243136e+14 true resid norm >>> 9.177422576855e+13 ||r(i)||/||b|| 3.772487330545e-03 >>> 23 KSP preconditioned resid norm 3.382362447667e+14 true resid norm >>> 5.210255290014e+13 ||r(i)||/||b|| 2.141736626583e-03 >>> 24 KSP preconditioned resid norm 2.828412329842e+14 true resid norm >>> 7.512251290427e+13 ||r(i)||/||b|| 3.087999117364e-03 >>> 25 KSP preconditioned resid norm 2.453088437412e+14 true resid norm >>> 3.770320821115e+13 ||r(i)||/||b|| 1.549834652446e-03 >>> 26 KSP preconditioned resid norm 2.155681340535e+14 true resid norm >>> 5.419527743350e+13 ||r(i)||/||b|| 2.227760526238e-03 >>> 27 KSP preconditioned resid norm 1.897133167426e+14 true resid norm >>> 2.950727994670e+13 ||r(i)||/||b|| 1.212931395777e-03 >>> 28 KSP preconditioned resid norm 1.747505144272e+14 true resid norm >>> 2.784171913979e+13 ||r(i)||/||b|| 1.144466562762e-03 >>> 29 KSP preconditioned resid norm 1.591031918648e+14 true resid norm >>> 2.517709500033e+13 ||r(i)||/||b|| 1.034934058155e-03 >>> 30 KSP preconditioned resid norm 1.457115463937e+14 true resid norm >>> 2.260424017892e+13 ||r(i)||/||b|| 9.291738391412e-04 >>> 31 KSP preconditioned resid norm 1.296458013341e+14 true resid norm >>> 2.025545529149e+13 ||r(i)||/||b|| 8.326242779131e-04 >>> 0 KSP preconditioned resid norm 6.130010037612e+16 true resid norm >>> 4.019535297216e+16 ||r(i)||/||b|| 5.748505993058e-02 >>> 1 KSP preconditioned resid norm 2.383231272358e+16 true resid norm >>> 4.333028202730e+15 ||r(i)||/||b|| 6.196845343971e-03 >>> 2 KSP preconditioned resid norm 1.380314308485e+16 true resid norm >>> 2.902938638700e+15 ||r(i)||/||b|| 4.151614285761e-03 >>> 3 KSP preconditioned resid norm 9.355422664357e+15 true resid norm >>> 1.914374541766e+15 ||r(i)||/||b|| 2.737827313999e-03 >>> 4 KSP preconditioned resid norm 6.905389450100e+15 true resid norm >>> 1.160495113546e+15 ||r(i)||/||b|| 1.659672728773e-03 >>> 5 KSP preconditioned resid norm 5.388850197186e+15 true resid norm >>> 9.343171629150e+14 ||r(i)||/||b|| 1.336206156506e-03 >>> 6 KSP preconditioned resid norm 4.456941607624e+15 true resid norm >>> 7.443205303349e+14 ||r(i)||/||b|| 1.064484004494e-03 >>> 7 KSP preconditioned resid norm 3.604897946036e+15 true resid norm >>> 7.177354673946e+14 ||r(i)||/||b|| 1.026463591104e-03 >>> 8 KSP preconditioned resid norm 2.616552593607e+15 true resid norm >>> 4.566784798486e+14 ||r(i)||/||b|| 6.531150454453e-04 >>> 9 KSP preconditioned resid norm 1.730753896325e+15 true resid norm >>> 3.416577992536e+14 ||r(i)||/||b|| 4.886191465826e-04 >>> 10 KSP preconditioned resid norm 9.605913237294e+14 true resid norm >>> 1.715828091570e+14 ||r(i)||/||b|| 2.453877709266e-04 >>> 11 KSP preconditioned resid norm 3.885469473062e+14 true resid norm >>> 7.526615943221e+13 ||r(i)||/||b|| 1.076412909896e-04 >>> 12 KSP preconditioned resid norm 1.085402136107e+14 true resid norm >>> 1.877490962768e+13 ||r(i)||/||b|| 2.685078560912e-05 >>> 13 KSP preconditioned resid norm 5.521535103061e+13 true resid norm >>> 1.482831672696e+13 ||r(i)||/||b|| 2.120659759623e-05 >>> 14 KSP preconditioned resid norm 4.327778995114e+13 true resid norm >>> 6.949075963503e+12 ||r(i)||/||b|| 9.938164954062e-06 >>> 0 KSP preconditioned resid norm 2.469229041475e+16 true resid norm >>> 1.581397714280e+16 ||r(i)||/||b|| 1.421322438999e-01 >>> 1 KSP preconditioned resid norm 1.094866313351e+16 true resid norm >>> 1.827538507817e+15 ||r(i)||/||b|| 1.642547896610e-02 >>> 2 KSP preconditioned resid norm 7.902647446500e+15 true resid norm >>> 1.331972422409e+15 ||r(i)||/||b|| 1.197144952849e-02 >>> 3 KSP preconditioned resid norm 6.757114899989e+15 true resid norm >>> 1.177404752006e+15 ||r(i)||/||b|| 1.058223227906e-02 >>> 4 KSP preconditioned resid norm 6.236820681061e+15 true resid norm >>> 9.858934558297e+14 ||r(i)||/||b|| 8.860974558006e-03 >>> 5 KSP preconditioned resid norm 6.056208230839e+15 true resid norm >>> 9.498735328449e+14 ||r(i)||/||b|| 8.537236106085e-03 >>> 6 KSP preconditioned resid norm 6.281150442401e+15 true resid norm >>> 9.439017368432e+14 ||r(i)||/||b|| 8.483563032058e-03 >>> 7 KSP preconditioned resid norm 6.511637444792e+15 true resid norm >>> 1.175605282966e+15 ||r(i)||/||b|| 1.056605908176e-02 >>> 8 KSP preconditioned resid norm 5.897282453446e+15 true resid norm >>> 9.358326231383e+14 ||r(i)||/||b|| 8.411039768188e-03 >>> 9 KSP preconditioned resid norm 4.858643646559e+15 true resid norm >>> 9.074077132105e+14 ||r(i)||/||b|| 8.155563476918e-03 >>> 10 KSP preconditioned resid norm 3.431227043388e+15 true resid norm >>> 5.523077341519e+14 ||r(i)||/||b|| 4.964009804073e-03 >>> 11 KSP preconditioned resid norm 1.790602151882e+15 true resid norm >>> 3.233768234561e+14 ||r(i)||/||b|| 2.906433538380e-03 >>> 12 KSP preconditioned resid norm 6.061523925825e+14 true resid norm >>> 9.361733195934e+13 ||r(i)||/||b|| 8.414101866433e-04 >>> 13 KSP preconditioned resid norm 3.669173256365e+14 true resid norm >>> 7.433445448820e+13 ||r(i)||/||b|| 6.681002963438e-04 >>> 14 KSP preconditioned resid norm 1.034315631221e+15 true resid norm >>> 1.749781208578e+14 ||r(i)||/||b|| 1.572661495987e-03 >>> 15 KSP preconditioned resid norm 1.340579302998e+14 true resid norm >>> 2.477734238014e+13 ||r(i)||/||b|| 2.226928266408e-04 >>> 16 KSP preconditioned resid norm 1.124182190968e+14 true resid norm >>> 1.917839484668e+13 ||r(i)||/||b|| 1.723708254629e-04 >>> 17 KSP preconditioned resid norm 9.394060483803e+13 true resid norm >>> 1.682558933028e+13 ||r(i)||/||b|| 1.512243722661e-04 >>> 18 KSP preconditioned resid norm 9.034672550576e+13 true resid norm >>> 1.892870471110e+13 ||r(i)||/||b|| 1.701266702495e-04 >>> 19 KSP preconditioned resid norm 8.058964108831e+13 true resid norm >>> 1.275370569496e+13 ||r(i)||/||b|| 1.146272561351e-04 >>> 20 KSP preconditioned resid norm 6.303390314758e+13 true resid norm >>> 1.900069985546e+13 ||r(i)||/||b|| 1.707737453860e-04 >>> 21 KSP preconditioned resid norm 4.601826493253e+13 true resid norm >>> 6.960266318594e+12 ||r(i)||/||b|| 6.255720879505e-05 >>> 22 KSP preconditioned resid norm 2.822787946131e+13 true resid norm >>> 1.013300639920e+13 ||r(i)||/||b|| 9.107303772884e-05 >>> 23 KSP preconditioned resid norm 1.858182164082e+13 true resid norm >>> 2.785683110677e+12 ||r(i)||/||b|| 2.503705347106e-05 >>> 24 KSP preconditioned resid norm 9.506925688468e+12 true resid norm >>> 4.268960894748e+12 ||r(i)||/||b|| 3.836839939834e-05 >>> 25 KSP preconditioned resid norm 5.845171242291e+12 true resid norm >>> 8.923161711832e+11 ||r(i)||/||b|| 8.019924307031e-06 >>> 0 SNES Function norm 0.000000000000e+00 >>> 5 TS dt 7e-12 time 2e-11 >>> TS Object: 1 MPI processes >>> type: theta >>> maximum steps=10 >>> maximum time=2e-11 >>> total number of nonlinear solver iterations=0 >>> total number of nonlinear solve failures=0 >>> total number of linear solver iterations=0 >>> total number of rejected steps=5 >>> Theta=1 >>> Extrapolation=no >>> SNES Object: 1 MPI processes >>> type: newtonls >>> maximum iterations=50, maximum function evaluations=10000 >>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>> total number of linear solver iterations=0 >>> total number of function evaluations=1 >>> SNESLineSearch Object: 1 MPI processes >>> type: bt >>> interpolation: cubic >>> alpha=1.000000e-04 >>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>> lambda=1.000000e-08 >>> maximum iterations=40 >>> KSP Object: 1 MPI processes >>> type: gmres >>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>> Orthogonalization with no iterative refinement >>> GMRES: happy breakdown tolerance 1e-30 >>> maximum iterations=10000, initial guess is zero >>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>> left preconditioning >>> using DEFAULT norm type for convergence test >>> PC Object: 1 MPI processes >>> type: ilu >>> PC has not been set up so information may be incomplete >>> ILU: out-of-place factorization >>> 0 levels of fill >>> tolerance for zero pivot 2.22045e-14 >>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>> matrix ordering: natural >>> linear system matrix = precond matrix: >>> Mat Object: 1 MPI processes >>> type: seqaij >>> rows=20904, cols=20904, bs=4 >>> total: nonzeros=410816, allocated nonzeros=410816 >>> total number of mallocs used during MatSetValues calls =0 >>> using I-node routines: found 5226 nodes, limit used is 5 >>> >>> On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley >>> wrote: >>> >>>> On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone >>>> wrote: >>>> >>>>> Hi Jed and Matt, >>>>> >>>>> Thank you very much for your replies. >>>>> >>>>> This time I copied the whole output from my screen, from compile to >>>>> execution. This time I put the TSView() routine just after the TSSolve() >>>>> routine to make the tsview output. And I intentionally to do only 5 time >>>>> steps to make the test. Thank you in advance! >>>>> >>>> >>>> You have to make the command line arguments work. Quit calling TSView() >>>> in the code. We need many more options >>>> for debugging. First track down why you are not processing options. Do >>>> the examples take options for you? >>>> >>>> Once that works, run with -ts_view -ts_monitor -snes_monitor >>>> -ksp_monitor_true_residual >>>> >>>> Matt >>>> >>>> >>>>> host206-47:streamer_Implicit_SG GPL$ make pStreamer >>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx -o >>>>> pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>>>> -Wno-unknown-pragmas -g -O0 -fPIC >>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >>>>> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first >>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >>>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >>>>> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >>>>> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >>>>> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >>>>> -lpthread -lssl -lcrypto >>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >>>>> -lmpichf90 -lgfortran >>>>> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >>>>> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >>>>> -lclang_rt.osx -lmpichcxx -lc++ >>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>> -lclang_rt.osx >>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >>>>> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>> -lclang_rt.osx -ldl >>>>> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >>>>> >>>>> time=0.000000 step=0 >>>>> dt = 0.001000 >>>>> >>>>> time=0.002000 step=1 >>>>> dt = 0.002000 >>>>> >>>>> time=0.005000 step=2 >>>>> dt = 0.003000 >>>>> >>>>> time=0.009000 step=3 >>>>> dt = 0.004000 >>>>> >>>>> time=0.014000 step=4 >>>>> dt = 0.005000 >>>>> >>>>> time=0.020000 step=5 >>>>> dt = 0.006000 >>>>> >>>>> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >>>>> TS Object: 1 MPI processes >>>>> type: theta >>>>> maximum steps=10 >>>>> maximum time=2e-11 >>>>> total number of nonlinear solver iterations=0 >>>>> total number of nonlinear solve failures=0 >>>>> total number of linear solver iterations=0 >>>>> total number of rejected steps=5 >>>>> Theta=1 >>>>> Extrapolation=no >>>>> SNES Object: 1 MPI processes >>>>> type: newtonls >>>>> maximum iterations=50, maximum function evaluations=10000 >>>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>> total number of linear solver iterations=0 >>>>> total number of function evaluations=1 >>>>> SNESLineSearch Object: 1 MPI processes >>>>> type: bt >>>>> interpolation: cubic >>>>> alpha=1.000000e-04 >>>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>>> lambda=1.000000e-08 >>>>> maximum iterations=40 >>>>> KSP Object: 1 MPI processes >>>>> type: gmres >>>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>>> Orthogonalization with no iterative refinement >>>>> GMRES: happy breakdown tolerance 1e-30 >>>>> maximum iterations=10000, initial guess is zero >>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>>> left preconditioning >>>>> using DEFAULT norm type for convergence test >>>>> PC Object: 1 MPI processes >>>>> type: ilu >>>>> PC has not been set up so information may be incomplete >>>>> ILU: out-of-place factorization >>>>> 0 levels of fill >>>>> tolerance for zero pivot 2.22045e-14 >>>>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>>>> matrix ordering: natural >>>>> linear system matrix = precond matrix: >>>>> Mat Object: 1 MPI processes >>>>> type: seqaij >>>>> rows=20904, cols=20904, bs=4 >>>>> total: nonzeros=410816, allocated nonzeros=410816 >>>>> total number of mallocs used during MatSetValues calls =0 >>>>> using I-node routines: found 5226 nodes, limit used is 5 >>>>> >>>>> >>>>> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >>>>> >>>>>> Sharp Stone writes: >>>>>> >>>>>> > Hi Matt, >>>>>> > >>>>>> > Thank you very much for your reply. >>>>>> > >>>>>> > The ts_view output is attached below. I found my code results say >>>>>> "SNES has >>>>>> > not been set up", and PC "has not been set up". Does this cause the >>>>>> > problem? If so I do not see example ex17 explicitly set up the snes >>>>>> object? >>>>>> >>>>>> Those objects are set up on the first time step (inside TSSolve). >>>>>> >>>>>> > PS: I don't know why the -ts_view option does not work in my >>>>>> command line, >>>>>> >>>>>> Are you sure it doesn't output after the TSSolve? The example does. >>>>>> >>>>>> > so I use TSView() routine just before the TSSolve() routine. >>>>>> >>>>>> You should call it *after* the solve so we can see what happened >>>>>> during >>>>>> the solve. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Feng >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Mon Oct 27 15:16:30 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 27 Oct 2014 14:16:30 -0600 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: <87lho11d5t.fsf@jedbrown.org> Sharp Stone writes: > Hi Matt, > > That is why we asked this. We do not think the calculation of DAE > F-functions are wrong. But even if the F-functions are not accurate, we > expect the TS solver would still call TSSetIJacobian(), but actually it > does not call TSSetIJacobian() routine at all... The solver never calls TSSetIJacobian. It does call your function (FormIJacobian) if it has to solve something. But as Matt points out, your FormIFunction produces a residual of 0, so SNES does not need to solve anything. It looks like your FormIFunction contains an implicit solve. Note that this will typically make the corresponding Jacobian dense. It is worth considering whether you can reformulate to retain sparsity (perhaps adding some algebraic variables). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Mon Oct 27 15:28:12 2014 From: jed at jedbrown.org (Jed Brown) Date: Mon, 27 Oct 2014 14:28:12 -0600 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <7911368.Xkc5LuFApY@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> <7911368.Xkc5LuFApY@besikovitch-ii> Message-ID: <87ioj51cmb.fsf@jedbrown.org> Filippo Leonardi writes: >> -ksp_chebyshev_estimate_eigenvalues_random > > Can try this but the problem, to me, seems deeper (because inherited from each > solve afterwards). Are you changing the operator between solves? Or just solving different right-hand sides? If the latter, the spectral estimates are reused. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From alpkalpalp at gmail.com Mon Oct 27 16:00:35 2014 From: alpkalpalp at gmail.com (Alp Kalpalp) Date: Mon, 27 Oct 2014 23:00:35 +0200 Subject: [petsc-users] FETI-DP implementation and call sequence In-Reply-To: References: <871trs7pnj.fsf@jedbrown.org> <1F4A79CC-DDB5-40C1-A94C-870CCF03330F@gmail.com> <87egvpy3gt.fsf@jedbrown.org> <87lhpxwhda.fsf@jedbrown.org> Message-ID: Hi, Are there any some simpler examples for BDDC/FETIDP algo? Since it uses MATIS etc. it is somehow hard to track the ex59. A simple case for example unstructured membrane solution etc. can be useful I thnk. On Mon, Sep 22, 2014 at 6:39 PM, Stefano Zampini wrote: > Sorry for late reply. I just pushed a fix for the crash. It is in master. > > Stefano > On Fri, 5 Sep 2014, Jed Brown wrote: > > > Satish Balay writes: > > > Perhaps the following is the fix [with proper comments, more error > > > checks?]. But someone more familiar with this code should check this.. > > > > > > Satish > > > > > > -------------- > > > $ git diff |cat > > > diff --git a/src/ksp/pc/impls/is/pcis.c b/src/ksp/pc/impls/is/pcis.c > > > index dab5836..0fa0217 100644 > > > --- a/src/ksp/pc/impls/is/pcis.c > > > +++ b/src/ksp/pc/impls/is/pcis.c > > > @@ -140,6 +140,8 @@ PetscErrorCode PCISSetUp(PC pc) > > > ierr = > PetscObjectTypeCompare((PetscObject)pc->pmat,MATIS,&flg);CHKERRQ(ierr); > > > if (!flg) > SETERRQ(PetscObjectComm((PetscObject)pc),PETSC_ERR_ARG_WRONG,"Preconditioner > type of Neumann Neumman requires matrix of type MATIS"); > > > matis = (Mat_IS*)pc->pmat->data; > > > + PetscObjectReference((PetscObject)pc->pmat); > > > + pcis->pmat = pc->pmat; > > > > Uh, PCISSetUp can be called more than once? > > I have no idea.. > > > > > And simply destroying the pcis->pmat reference is not enough because > > that extra reference could significantly increase the peak memory usage. > > Curently the object (pc->pmat) is destroyed at the end anyway [perhaps > duing PCDestroy()]. This fix changes the order a bit so that its > destoryed only after its last use. > > > The right solution is to not hold that reference and not hold the info. > > > > > pcis->pure_neumann = matis->pure_neumann; > > > > > > @@ -378,8 +380,9 @@ PetscErrorCode PCISDestroy(PC pc) > > > ierr = VecScatterDestroy(&pcis->global_to_B);CHKERRQ(ierr); > > > ierr = PetscFree(pcis->work_N);CHKERRQ(ierr); > > > if (pcis->ISLocalToGlobalMappingGetInfoWasCalled) { > > > - ierr = > ISLocalToGlobalMappingRestoreInfo((ISLocalToGlobalMapping)0,&(pcis->n_neigh),&(pcis->neigh),&(pcis->n_shared),&(pcis->shared));CHKERRQ(ierr); > > > + ierr = > ISLocalToGlobalMappingRestoreInfo(((Mat_IS*)pcis->pmat->data)->mapping,&(pcis->n_neigh),&(pcis->neigh),&(pcis->n_shared),&(pcis->shared));CHKERRQ(ierr); > > > } > > > > Why not restore the info at the place it is gotten, like we do with > > every other accessor? > > Looks like this info is stashed in 'pcis->n_neigh, pcis->neigh' etc - > and reused later multple times. [perhaps preventing multiple > mallocs/frees] > > $ git grep -l 'pcis->n_neigh' > src/ksp/pc/impls/bddc/bddcfetidp.c > src/ksp/pc/impls/is/nn/nn.c > src/ksp/pc/impls/is/pcis.c > > Or perhaps this info should be stashed in the IS so multiple > ISLocalToGlobalMappingGetInfo() calls are cheap [but then the malloc'd > memory will live until IS is destroyed anyway] > > I guess there are 2 issues you are touching on. A fix for this crash - > and code cleanup. My patch gets the examples working. > > But I'll defer both isses to Stefano [asuming he is aquainted with the > above sources]. > > Satish > > > > > > + ierr = MatDestroy(&pcis->pmat);CHKERRQ(ierr); > > > ierr = > PetscObjectComposeFunction((PetscObject)pc,"PCISSetUseStiffnessScaling_C",NULL);CHKERRQ(ierr); > > > ierr = > PetscObjectComposeFunction((PetscObject)pc,"PCISSetSubdomainScalingFactor_C",NULL);CHKERRQ(ierr); > > > ierr = > PetscObjectComposeFunction((PetscObject)pc,"PCISSetSubdomainDiagonalScaling_C",NULL);CHKERRQ(ierr); > > > diff --git a/src/ksp/pc/impls/is/pcis.h b/src/ksp/pc/impls/is/pcis.h > > > index 4a42cf9..736ea8c 100644 > > > --- a/src/ksp/pc/impls/is/pcis.h > > > +++ b/src/ksp/pc/impls/is/pcis.h > > > @@ -73,6 +73,7 @@ typedef struct { > > > /* We need: > */ > > > /* proc[k].loc_to_glob(proc[k].shared[i][m]) == > proc[l].loc_to_glob(proc[l].shared[j][m]) */ > > > /* for all 0 <= m < proc[k].n_shared[i], or equiv'ly, for all 0 <= > m < proc[l].n_shared[j] */ > > > + Mat pmat; > > > } PC_IS; > > > > > > PETSC_EXTERN PetscErrorCode PCISSetUp(PC pc); > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Mon Oct 27 17:00:14 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Mon, 27 Oct 2014 17:00:14 -0500 Subject: [petsc-users] MatZeroRows and MATELEMENTAL In-Reply-To: References: Message-ID: > On Oct 26, 2014, at 4:32 AM, Francesco Ballarin wrote: > > Dear PETSc users, > I have a code for PETSc 3.3 in which the PLAPACK interface, now discontinued, was used. I am trying to update it to PETSc 3.5.2, using the Elemental interface. > > I see that MatZeroRows is not implemented for MATELEMENTAL. What function should I use instead? Does it also support the version with optional arguments x and b? Unfortunately due to Elemental?s complicated parallel storage of matrix entries we don?t have any routines to do this. (We don?t even have routines to generically set values INSERT_VALUES) on other processes. cc:ing Elemental?s author to see what he suggests. Barry > > Thanks, > best regards, > Francesco From jychang48 at gmail.com Mon Oct 27 18:56:09 2014 From: jychang48 at gmail.com (Justin Chang) Date: Mon, 27 Oct 2014 18:56:09 -0500 Subject: [petsc-users] Partitioner type for DMPlex? Message-ID: Hi all, I just noticed that in petsc-dev the DMPlexDistribute() function no longer takes as input the partitioner type. I am assuming that it only uses Chaco and that METIS/ParMETIS are no longer options. Is there any particular reason for this? Thanks, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From baagaard at usgs.gov Mon Oct 27 18:59:55 2014 From: baagaard at usgs.gov (Aagaard, Brad) Date: Mon, 27 Oct 2014 16:59:55 -0700 Subject: [petsc-users] Partitioner type for DMPlex? In-Reply-To: References: Message-ID: Matt made it a PETSc parameter (petscpartitioner_type). Use -help and I think you should get "chaco", "parmetis", and "shell" as partitioner types. Regards, Brad On Mon, Oct 27, 2014 at 4:56 PM, Justin Chang wrote: > Hi all, > > I just noticed that in petsc-dev the DMPlexDistribute() function no longer > takes as input the partitioner type. I am assuming that it only uses Chaco > and that METIS/ParMETIS are no longer options. Is there any particular > reason for this? > > Thanks, > Justin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack.poulson at gmail.com Tue Oct 28 00:20:29 2014 From: jack.poulson at gmail.com (Jack Poulson) Date: Mon, 27 Oct 2014 22:20:29 -0700 Subject: [petsc-users] Fwd: [dev@libelemental.org] Re: [elemental] Fwd: MatZeroRows and MATELEMENTAL In-Reply-To: <544F1B42.3060308@stanford.edu> References: <544F1B42.3060308@stanford.edu> Message-ID: ---------- Forwarded message ---------- From: Jack Poulson Date: Mon, Oct 27, 2014 at 9:27 PM Subject: [dev at libelemental.org] Re: [elemental] Fwd: [petsc-users] MatZeroRows and MATELEMENTAL To: "dev at libele >> Elemental" , Matthew Knepley < knepley at gmail.com> Hi Matt, At first glance, I would have suggested EntrywiseFill [1,2,3], which takes an arbitrary function, say fill : (i,j) |-> T and sets each entry (i,j) of a matrix of type T to fill(i,j), but the documentation for MatZeroRows [4] states that only a subset of rows are to be modified. I would instead implement this as follows (in C++), for the slightly more general case where the off-diagonal value can be arbitrary: template void ModifyRows ( AbstractDistMatrix& A, const std::vector& rowInd, T diagVal, T offDiagVal ) { DEBUG_ONLY(CallStackEntry cse("ModifyRows")) const Int numRows = rowInd.size(); const Int localWidth = A.LocalWidth(); for( Int s=0; s rowInd(rows,rows+numRows); Jack P.S. The release candidate for 0.85 is coming in a few hours. [1] https://github.com/elemental/Elemental/blob/396a8cebc46ea13c4fbdc51a1f21fce2c1a75edf/include/El/blas_like/level1.hpp#L176 [2] https://github.com/elemental/Elemental/blob/396a8cebc46ea13c4fbdc51a1f21fce2c1a75edf/include/El/blas_like/level1.h#L314 [3] https://github.com/elemental/Elemental/blob/396a8cebc46ea13c4fbdc51a1f21fce2c1a75edf/include/El/blas_like/level1.py#L485 [4] http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatZeroRows.html On 10/27/2014 02:36 PM, Matthew Knepley wrote: > I do not know the answer to this one. Is there something in that does > the equivalent of > MatZeroRows(), or do you do it by hand? > > Thanks, > > Matt > > ---------- Forwarded message ---------- > From: *Francesco Ballarin* > > Date: Sun, Oct 26, 2014 at 4:32 AM > Subject: [petsc-users] MatZeroRows and MATELEMENTAL > To: petsc-users at mcs.anl.gov > > > Dear PETSc users, > I have a code for PETSc 3.3 in which the PLAPACK interface, now > discontinued, was used. I am trying to update it to PETSc 3.5.2, using > the Elemental interface. > > I see that MatZeroRows is not implemented for MATELEMENTAL. What > function should I use instead? Does it also support the version with > optional arguments x and b? > > Thanks, > best regards, > Francesco > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener > > -- > You received this message because you are subscribed to the Google > Groups "elemental-dev" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to elemental-dev+unsubscribe at googlegroups.com > . > For more options, visit https://groups.google.com/d/optout. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From stali at geology.wisc.edu Tue Oct 28 05:39:42 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 05:39:42 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> Message-ID: <544F726E.6060503@geology.wisc.edu> Mark When I replace "-fieldsplit_0_ksp_type preonly" with "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to kill it). With MUMPS , i.e., with '-fieldsplit_0_pc_type lu -fieldsplit_0_pc_factor_mat_solver_package mumps -fieldsplit_0_ksp_type preonly' it works fine but takes more time, and will be an issue for larger problems. The output for this run is attached. I will work on passing rigid body modes (as Matt mentioned) but short of that what is the best set of options for solving the following problem (i.e., linear elasticity with constraints): |K cG'| | u | = |F| |G 0 | |l/c| |d| where c is a scaling factor (so that cG' terms are more or less of the same order as K)? The constraints are used to impose slip between surfaces and so on. Tabrez On 10/27/2014 01:17 PM, Mark Adams wrote: > The null space for GAMG is not critical but useful for elasticity. If > you in fact have an indefinite operator (eg, not "pinned) the you > need to use an iterative coarse grid solver. You are > using '-fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_type preonly'. > And you have a hard elasticity problem. You are going to want to > start with a stronger solver. Use cg instead of preonly. As Matt said > start with MUMPS, then go to CG/GAMG, then you can see how far you can > cut the _0_ solver down. > Mark > > On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley > wrote: > > On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali > > wrote: > > Matt > > On 10/23/2014 09:54 AM, Matthew Knepley wrote: >> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >> > wrote: >> >> Matt >> >> Sorry about that (I always forget it). The output for the >> smallest problem is now attached (see log.txt). I am also >> attaching some results that compare results obtained >> using FS/LSC and the direct solver (MUMPS), again for the >> smallest problem. The difference, as you can see is >> insignificant O(1E-6). >> >> >> 1) How do you use MUMPS if you have a saddle point > I simply used -pc_type lu -pc_factor_mat_solver_package mumps. > >> >> 2) You can see from the output that something is seriously >> wrong with the preconditioner. It looks like it has a null space. >> Did you add the elastic null modes to GAMG? Without this, >> it is not going to work. We have helper functions for this: >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >> >> you could just copy that code. And then use >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >> >> I don't see it in the output, so I think this is your problem. >> >> In order to test, I would first use MUMPS as the A00 solver >> and get the Schur stuff worked out. Then I would >> replace MUMPS with GAMG and tune it until I get back my >> original convergence. > I will try this with MatNullSpaceCreateRigidBody. Btw does it > matter if some nodes are pinned? > > > No these are null modes of the operator, not of the particular > problem. > > Matt > > Tabrez > >> >> Thanks, >> >> Matt >> >> Also, I did pass 'upper' and 'full' to >> '-pc_fieldsplit_schur_factorization_type' but the >> iteration count doesn't improve (in fact, it increases >> slightly). The attached log is with 'upper'. >> >> Regards, >> >> Tabrez >> >> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>> > >>> wrote: >>> >>> Hello >>> >>> I am using the following options (below) for solving >>> linear elasticity/poroelasticity problems involving >>> slip between two surfaces involving non-trivial >>> geometries, i.e., elements with high aspect ratios, >>> large contrasts in material properties etc. The >>> constraints are imposed using Lagrange Multipliers. >>> >>> A picture (shows displacement magnitude) is >>> attached. The boundary nodes, i.e., the base and the >>> four side are pinned. >>> >>> The following options appear to work well for the >>> saddle point problem: >>> >>> -pc_type fieldsplit -pc_fieldsplit_type schur >>> -pc_fieldsplit_detect_saddle_point >>> -fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_type >>> preonly -fieldsplit_1_pc_type lsc >>> -fieldsplit_1_ksp_type preonly >>> -pc_fieldsplit_schur_fact_type lower -ksp_monitor >>> >>> However, the number of iterations keep on increasing >>> with the problems size (see attached plot), e.g., >>> >>> 120K Tets *507* Iterations (KSP Residual norm >>> 8.827362494659e-05)in 17 secs on 3 cores >>> 1 Million Tets *1374* Iterations (KSP Residual norm >>> 7.164704416296e-05)in 117 secs on 20 cores >>> 8 Million Tets *2495* Iterations (KSP Residual norm >>> 9.101247550026e-05) in 225 secs on 160 cores >>> >>> So what other options should I try to improve solver >>> performance? Any tips/insights would be appreciated >>> as preconditioning is black magic to me. >>> >>> >>> For reports, always run with >>> >>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >>> >>> so that we can see exactly what you used. >>> >>> I believe the default is a diagonal factorization. Since >>> your outer iterates are increasing, I would strengthen this >>> to either upper or full >>> >>> -pc_fieldsplit_schur_factorization_type >>> >>> Thanks, >>> >>> Matt >>> >>> Thanks in advance. >>> >>> Tabrez >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they >>> begin their experiments is infinitely more interesting >>> than any results to which their experiments lead. >>> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin >> their experiments is infinitely more interesting than any >> results to which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to > which their experiments lead. > -- Norbert Wiener > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- salloc: Granted job allocation 78969 Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 9.127510702346e+05 true resid norm 1.362125072634e+14 ||r(i)||/||b|| 7.756982470220e+11 2 KSP preconditioned resid norm 8.626930780275e+05 true resid norm 7.140706098955e+14 ||r(i)||/||b|| 4.066464464051e+12 3 KSP preconditioned resid norm 6.917392651646e+05 true resid norm 2.788588220631e+15 ||r(i)||/||b|| 1.588035517346e+13 4 KSP preconditioned resid norm 5.948180320596e+05 true resid norm 3.828406259238e+15 ||r(i)||/||b|| 2.180187476056e+13 5 KSP preconditioned resid norm 5.365265868551e+05 true resid norm 4.509015181942e+15 ||r(i)||/||b|| 2.567778277265e+13 6 KSP preconditioned resid norm 5.020807291688e+05 true resid norm 4.938429989057e+15 ||r(i)||/||b|| 2.812319927527e+13 7 KSP preconditioned resid norm 4.691107247107e+05 true resid norm 5.504613096315e+15 ||r(i)||/||b|| 3.134747913486e+13 8 KSP preconditioned resid norm 4.500805942145e+05 true resid norm 5.950219821547e+15 ||r(i)||/||b|| 3.388510480939e+13 9 KSP preconditioned resid norm 4.250032501744e+05 true resid norm 6.670720166034e+15 ||r(i)||/||b|| 3.798818510228e+13 10 KSP preconditioned resid norm 4.051645463810e+05 true resid norm 7.414211051141e+15 ||r(i)||/||b|| 4.222219112596e+13 11 KSP preconditioned resid norm 3.880013874032e+05 true resid norm 8.151575238507e+15 ||r(i)||/||b|| 4.642130704452e+13 12 KSP preconditioned resid norm 3.741408602796e+05 true resid norm 8.948407390773e+15 ||r(i)||/||b|| 5.095907906048e+13 13 KSP preconditioned resid norm 3.532148969165e+05 true resid norm 1.028219119134e+16 ||r(i)||/||b|| 5.855466464062e+13 14 KSP preconditioned resid norm 3.306246816868e+05 true resid norm 1.207912805635e+16 ||r(i)||/||b|| 6.878779817734e+13 15 KSP preconditioned resid norm 3.077008490960e+05 true resid norm 1.412194821609e+16 ||r(i)||/||b|| 8.042117934570e+13 16 KSP preconditioned resid norm 2.709436838755e+05 true resid norm 1.756809508461e+16 ||r(i)||/||b|| 1.000461766282e+14 17 KSP preconditioned resid norm 2.491233804162e+05 true resid norm 1.976045004357e+16 ||r(i)||/||b|| 1.125311233683e+14 18 KSP preconditioned resid norm 2.169453185969e+05 true resid norm 2.315544602061e+16 ||r(i)||/||b|| 1.318648283338e+14 19 KSP preconditioned resid norm 1.853345270881e+05 true resid norm 2.647069144539e+16 ||r(i)||/||b|| 1.507443726291e+14 20 KSP preconditioned resid norm 1.684382143134e+05 true resid norm 2.847996525788e+16 ||r(i)||/||b|| 1.621867152263e+14 21 KSP preconditioned resid norm 1.487793180708e+05 true resid norm 3.106944054176e+16 ||r(i)||/||b|| 1.769331689754e+14 22 KSP preconditioned resid norm 1.324108267213e+05 true resid norm 3.364687707291e+16 ||r(i)||/||b|| 1.916110648544e+14 23 KSP preconditioned resid norm 1.154783368235e+05 true resid norm 3.664744963424e+16 ||r(i)||/||b|| 2.086986210755e+14 24 KSP preconditioned resid norm 9.690290840840e+04 true resid norm 4.059423491278e+16 ||r(i)||/||b|| 2.311746365563e+14 25 KSP preconditioned resid norm 8.541667474325e+04 true resid norm 4.382080823729e+16 ||r(i)||/||b|| 2.495492140602e+14 26 KSP preconditioned resid norm 6.738137388808e+04 true resid norm 4.935828200762e+16 ||r(i)||/||b|| 2.810838270181e+14 27 KSP preconditioned resid norm 5.559761934660e+04 true resid norm 5.372591255719e+16 ||r(i)||/||b|| 3.059564575057e+14 28 KSP preconditioned resid norm 4.606357787962e+04 true resid norm 5.783519870706e+16 ||r(i)||/||b|| 3.293578772945e+14 29 KSP preconditioned resid norm 3.846753020730e+04 true resid norm 6.227329505424e+16 ||r(i)||/||b|| 3.546317939545e+14 30 KSP preconditioned resid norm 3.331905232453e+04 true resid norm 6.606924388231e+16 ||r(i)||/||b|| 3.762488312653e+14 31 KSP preconditioned resid norm 3.290201988582e+04 true resid norm 6.618297473845e+16 ||r(i)||/||b|| 3.768965017878e+14 32 KSP preconditioned resid norm 3.162870028104e+04 true resid norm 6.654393352106e+16 ||r(i)||/||b|| 3.789520773039e+14 33 KSP preconditioned resid norm 2.991966449152e+04 true resid norm 6.709255863611e+16 ||r(i)||/||b|| 3.820763685204e+14 34 KSP preconditioned resid norm 2.922952746973e+04 true resid norm 6.743695964917e+16 ||r(i)||/||b|| 3.840376514265e+14 35 KSP preconditioned resid norm 2.798614888145e+04 true resid norm 6.838110987651e+16 ||r(i)||/||b|| 3.894143652907e+14 36 KSP preconditioned resid norm 2.701665220461e+04 true resid norm 6.929642686679e+16 ||r(i)||/||b|| 3.946268806396e+14 37 KSP preconditioned resid norm 2.601811428056e+04 true resid norm 7.044504608577e+16 ||r(i)||/||b|| 4.011679974032e+14 38 KSP preconditioned resid norm 2.531056406767e+04 true resid norm 7.153872989114e+16 ||r(i)||/||b|| 4.073962698846e+14 39 KSP preconditioned resid norm 2.332617529703e+04 true resid norm 7.497512777257e+16 ||r(i)||/||b|| 4.269657489747e+14 40 KSP preconditioned resid norm 2.254584604090e+04 true resid norm 7.640212636531e+16 ||r(i)||/||b|| 4.350921709100e+14 41 KSP preconditioned resid norm 2.082879229685e+04 true resid norm 7.986664049248e+16 ||r(i)||/||b|| 4.548217654180e+14 42 KSP preconditioned resid norm 1.914379086156e+04 true resid norm 8.367676210994e+16 ||r(i)||/||b|| 4.765195134368e+14 43 KSP preconditioned resid norm 1.815822267369e+04 true resid norm 8.615765209492e+16 ||r(i)||/||b|| 4.906475994038e+14 44 KSP preconditioned resid norm 1.677428596939e+04 true resid norm 8.993611308328e+16 ||r(i)||/||b|| 5.121650475736e+14 45 KSP preconditioned resid norm 1.534247135375e+04 true resid norm 9.414497408723e+16 ||r(i)||/||b|| 5.361335227769e+14 46 KSP preconditioned resid norm 1.412608783430e+04 true resid norm 9.852630241654e+16 ||r(i)||/||b|| 5.610841589039e+14 47 KSP preconditioned resid norm 1.177668978657e+04 true resid norm 1.077880272442e+17 ||r(i)||/||b|| 6.138275072022e+14 48 KSP preconditioned resid norm 1.065678918714e+04 true resid norm 1.124882415134e+17 ||r(i)||/||b|| 6.405941238848e+14 49 KSP preconditioned resid norm 9.962410636884e+03 true resid norm 1.158017305255e+17 ||r(i)||/||b|| 6.594636658224e+14 50 KSP preconditioned resid norm 9.371192462863e+03 true resid norm 1.195573007708e+17 ||r(i)||/||b|| 6.808507565847e+14 51 KSP preconditioned resid norm 8.608163058470e+03 true resid norm 1.259685427089e+17 ||r(i)||/||b|| 7.173612741030e+14 52 KSP preconditioned resid norm 7.720924959940e+03 true resid norm 1.350861356613e+17 ||r(i)||/||b|| 7.692838252129e+14 53 KSP preconditioned resid norm 6.628413884374e+03 true resid norm 1.478135305653e+17 ||r(i)||/||b|| 8.417633508790e+14 54 KSP preconditioned resid norm 5.583298606838e+03 true resid norm 1.607393601806e+17 ||r(i)||/||b|| 9.153729156340e+14 55 KSP preconditioned resid norm 4.854896331327e+03 true resid norm 1.721936891600e+17 ||r(i)||/||b|| 9.806026297673e+14 56 KSP preconditioned resid norm 4.262399411003e+03 true resid norm 1.837125351729e+17 ||r(i)||/||b|| 1.046199753258e+15 57 KSP preconditioned resid norm 3.584024669247e+03 true resid norm 2.026508285264e+17 ||r(i)||/||b|| 1.154048887313e+15 58 KSP preconditioned resid norm 3.302611896991e+03 true resid norm 2.155979885039e+17 ||r(i)||/||b|| 1.227779923473e+15 59 KSP preconditioned resid norm 3.106452236021e+03 true resid norm 2.294151324082e+17 ||r(i)||/||b|| 1.306465313830e+15 60 KSP preconditioned resid norm 2.894632302133e+03 true resid norm 2.500169543959e+17 ||r(i)||/||b|| 1.423787852871e+15 61 KSP preconditioned resid norm 2.892427608148e+03 true resid norm 2.503572781245e+17 ||r(i)||/||b|| 1.425725916599e+15 62 KSP preconditioned resid norm 2.881855812307e+03 true resid norm 2.520690203632e+17 ||r(i)||/||b|| 1.435473886742e+15 63 KSP preconditioned resid norm 2.872986380486e+03 true resid norm 2.536302493244e+17 ||r(i)||/||b|| 1.444364719110e+15 64 KSP preconditioned resid norm 2.865635949494e+03 true resid norm 2.551732099856e+17 ||r(i)||/||b|| 1.453151517797e+15 65 KSP preconditioned resid norm 2.860836435783e+03 true resid norm 2.565769302199e+17 ||r(i)||/||b|| 1.461145374947e+15 66 KSP preconditioned resid norm 2.860026342234e+03 true resid norm 2.574294644148e+17 ||r(i)||/||b|| 1.466000357017e+15 67 KSP preconditioned resid norm 2.860026336219e+03 true resid norm 2.574330960365e+17 ||r(i)||/||b|| 1.466021038250e+15 68 KSP preconditioned resid norm 2.859446352247e+03 true resid norm 2.564102967021e+17 ||r(i)||/||b|| 1.460196436188e+15 69 KSP preconditioned resid norm 2.855291192171e+03 true resid norm 2.530743143761e+17 ||r(i)||/||b|| 1.441198800109e+15 70 KSP preconditioned resid norm 2.843683581105e+03 true resid norm 2.467887076760e+17 ||r(i)||/||b|| 1.405403745773e+15 71 KSP preconditioned resid norm 2.825924372436e+03 true resid norm 2.382078643091e+17 ||r(i)||/||b|| 1.356537857527e+15 72 KSP preconditioned resid norm 2.803340022921e+03 true resid norm 2.288941643098e+17 ||r(i)||/||b|| 1.303498522829e+15 73 KSP preconditioned resid norm 2.768188650590e+03 true resid norm 2.170599057122e+17 ||r(i)||/||b|| 1.236105198726e+15 74 KSP preconditioned resid norm 2.728040723580e+03 true resid norm 2.050447447412e+17 ||r(i)||/||b|| 1.167681678080e+15 75 KSP preconditioned resid norm 2.663723517309e+03 true resid norm 1.886813389061e+17 ||r(i)||/||b|| 1.074495923874e+15 76 KSP preconditioned resid norm 2.580490699705e+03 true resid norm 1.697855415273e+17 ||r(i)||/||b|| 9.668887944168e+14 77 KSP preconditioned resid norm 2.507785655255e+03 true resid norm 1.549544648278e+17 ||r(i)||/||b|| 8.824292948570e+14 78 KSP preconditioned resid norm 2.381407885460e+03 true resid norm 1.327524813313e+17 ||r(i)||/||b|| 7.559942117309e+14 79 KSP preconditioned resid norm 2.275211715962e+03 true resid norm 1.165811444580e+17 ||r(i)||/||b|| 6.639022451665e+14 80 KSP preconditioned resid norm 2.161471490944e+03 true resid norm 1.015723094108e+17 ||r(i)||/||b|| 5.784304535530e+14 81 KSP preconditioned resid norm 1.982790705425e+03 true resid norm 8.141089236173e+16 ||r(i)||/||b|| 4.636159172328e+14 82 KSP preconditioned resid norm 1.740676429181e+03 true resid norm 5.840122410223e+16 ||r(i)||/||b|| 3.325812590208e+14 83 KSP preconditioned resid norm 1.587150102469e+03 true resid norm 4.629492748351e+16 ||r(i)||/||b|| 2.636387422598e+14 84 KSP preconditioned resid norm 1.322911469852e+03 true resid norm 2.928137569559e+16 ||r(i)||/||b|| 1.667505595029e+14 85 KSP preconditioned resid norm 1.181708294070e+03 true resid norm 2.197349343473e+16 ||r(i)||/||b|| 1.251338858722e+14 86 KSP preconditioned resid norm 9.902319656225e+02 true resid norm 1.423216418217e+16 ||r(i)||/||b|| 8.104883339450e+13 87 KSP preconditioned resid norm 8.359003992039e+02 true resid norm 9.303072101775e+15 ||r(i)||/||b|| 5.297881131659e+13 88 KSP preconditioned resid norm 6.981353830695e+02 true resid norm 5.935654404148e+15 ||r(i)||/||b|| 3.380215817717e+13 89 KSP preconditioned resid norm 5.800715479524e+02 true resid norm 3.708718763345e+15 ||r(i)||/||b|| 2.112028257333e+13 90 KSP preconditioned resid norm 4.883815028666e+02 true resid norm 2.388206870627e+15 ||r(i)||/||b|| 1.360027739222e+13 91 KSP preconditioned resid norm 4.789803991475e+02 true resid norm 2.397467761477e+15 ||r(i)||/||b|| 1.365301599121e+13 92 KSP preconditioned resid norm 4.571508470005e+02 true resid norm 2.421108357131e+15 ||r(i)||/||b|| 1.378764363279e+13 93 KSP preconditioned resid norm 4.504410932746e+02 true resid norm 2.430412703020e+15 ||r(i)||/||b|| 1.384062969803e+13 94 KSP preconditioned resid norm 4.405021383131e+02 true resid norm 2.456970033422e+15 ||r(i)||/||b|| 1.399186745917e+13 95 KSP preconditioned resid norm 4.293421518077e+02 true resid norm 2.491683359417e+15 ||r(i)||/||b|| 1.418955170025e+13 96 KSP preconditioned resid norm 4.173639525791e+02 true resid norm 2.540094021569e+15 ||r(i)||/||b|| 1.446523905468e+13 97 KSP preconditioned resid norm 4.071406525465e+02 true resid norm 2.585692506997e+15 ||r(i)||/||b|| 1.472491172295e+13 98 KSP preconditioned resid norm 3.923770601661e+02 true resid norm 2.664481005387e+15 ||r(i)||/||b|| 1.517359372224e+13 99 KSP preconditioned resid norm 3.696685994238e+02 true resid norm 2.790356355448e+15 ||r(i)||/||b|| 1.589042428609e+13 100 KSP preconditioned resid norm 3.522302545921e+02 true resid norm 2.891433818038e+15 ||r(i)||/||b|| 1.646603670319e+13 101 KSP preconditioned resid norm 3.289451418467e+02 true resid norm 3.043740223578e+15 ||r(i)||/||b|| 1.733338592215e+13 102 KSP preconditioned resid norm 3.041026598143e+02 true resid norm 3.223224072832e+15 ||r(i)||/||b|| 1.835550430197e+13 103 KSP preconditioned resid norm 2.721797727812e+02 true resid norm 3.434699332293e+15 ||r(i)||/||b|| 1.955980625153e+13 104 KSP preconditioned resid norm 2.495065013936e+02 true resid norm 3.595571585792e+15 ||r(i)||/||b|| 2.047593596341e+13 105 KSP preconditioned resid norm 2.226326085921e+02 true resid norm 3.801867914908e+15 ||r(i)||/||b|| 2.165074512064e+13 106 KSP preconditioned resid norm 2.073865356308e+02 true resid norm 3.926005989758e+15 ||r(i)||/||b|| 2.235768231007e+13 107 KSP preconditioned resid norm 1.889509629828e+02 true resid norm 4.115952048686e+15 ||r(i)||/||b|| 2.343938051752e+13 108 KSP preconditioned resid norm 1.749949837765e+02 true resid norm 4.278800226377e+15 ||r(i)||/||b|| 2.436676265374e+13 109 KSP preconditioned resid norm 1.618193234597e+02 true resid norm 4.465831577902e+15 ||r(i)||/||b|| 2.543186228688e+13 110 KSP preconditioned resid norm 1.465181121950e+02 true resid norm 4.714656055446e+15 ||r(i)||/||b|| 2.684885926407e+13 111 KSP preconditioned resid norm 1.318107867392e+02 true resid norm 5.037611920944e+15 ||r(i)||/||b|| 2.868801709007e+13 112 KSP preconditioned resid norm 1.242084520074e+02 true resid norm 5.234477391902e+15 ||r(i)||/||b|| 2.980911972440e+13 113 KSP preconditioned resid norm 1.155780632405e+02 true resid norm 5.582223270103e+15 ||r(i)||/||b|| 3.178945085220e+13 114 KSP preconditioned resid norm 1.021440284034e+02 true resid norm 6.169174994158e+15 ||r(i)||/||b|| 3.513200310810e+13 115 KSP preconditioned resid norm 9.495516699860e+01 true resid norm 6.557142438207e+15 ||r(i)||/||b|| 3.734138660964e+13 116 KSP preconditioned resid norm 8.974660320529e+01 true resid norm 6.966234434027e+15 ||r(i)||/||b|| 3.967106947360e+13 117 KSP preconditioned resid norm 8.668620937542e+01 true resid norm 7.370496578775e+15 ||r(i)||/||b|| 4.197324746972e+13 118 KSP preconditioned resid norm 8.544499762555e+01 true resid norm 7.721074005539e+15 ||r(i)||/||b|| 4.396970360176e+13 119 KSP preconditioned resid norm 8.540357047452e+01 true resid norm 7.783986871592e+15 ||r(i)||/||b|| 4.432797760238e+13 120 KSP preconditioned resid norm 8.509182838139e+01 true resid norm 7.567150037172e+15 ||r(i)||/||b|| 4.309314274229e+13 121 KSP preconditioned resid norm 8.501310700513e+01 true resid norm 7.594940200282e+15 ||r(i)||/||b|| 4.325140119625e+13 122 KSP preconditioned resid norm 8.493474829240e+01 true resid norm 7.626607956863e+15 ||r(i)||/||b|| 4.343174163458e+13 123 KSP preconditioned resid norm 8.488333741555e+01 true resid norm 7.656182294989e+15 ||r(i)||/||b|| 4.360016054635e+13 124 KSP preconditioned resid norm 8.487226574331e+01 true resid norm 7.669732385858e+15 ||r(i)||/||b|| 4.367732513237e+13 125 KSP preconditioned resid norm 8.487223372854e+01 true resid norm 7.668341900577e+15 ||r(i)||/||b|| 4.366940664517e+13 126 KSP preconditioned resid norm 8.485866512717e+01 true resid norm 7.639207309985e+15 ||r(i)||/||b|| 4.350349199236e+13 127 KSP preconditioned resid norm 8.480919141670e+01 true resid norm 7.584336229378e+15 ||r(i)||/||b|| 4.319101407169e+13 128 KSP preconditioned resid norm 8.461824281885e+01 true resid norm 7.465571488217e+15 ||r(i)||/||b|| 4.251467675600e+13 129 KSP preconditioned resid norm 8.425430504400e+01 true resid norm 7.263958891880e+15 ||r(i)||/||b|| 4.136654035724e+13 130 KSP preconditioned resid norm 8.380956322835e+01 true resid norm 7.064381851222e+15 ||r(i)||/||b|| 4.022999597012e+13 131 KSP preconditioned resid norm 8.314252321333e+01 true resid norm 6.808447921810e+15 ||r(i)||/||b|| 3.877251233381e+13 132 KSP preconditioned resid norm 8.226839402109e+01 true resid norm 6.516419146749e+15 ||r(i)||/||b|| 3.710947702636e+13 133 KSP preconditioned resid norm 8.078979668705e+01 true resid norm 6.095156115716e+15 ||r(i)||/||b|| 3.471048297455e+13 134 KSP preconditioned resid norm 7.933264464976e+01 true resid norm 5.736990471656e+15 ||r(i)||/||b|| 3.267081372667e+13 135 KSP preconditioned resid norm 7.629871013849e+01 true resid norm 5.098475273910e+15 ||r(i)||/||b|| 2.903461959488e+13 136 KSP preconditioned resid norm 7.384550751976e+01 true resid norm 4.620263860149e+15 ||r(i)||/||b|| 2.631131787455e+13 137 KSP preconditioned resid norm 7.019186335982e+01 true resid norm 3.995768284315e+15 ||r(i)||/||b|| 2.275496219783e+13 138 KSP preconditioned resid norm 6.549021374298e+01 true resid norm 3.271622487708e+15 ||r(i)||/||b|| 1.863112191104e+13 139 KSP preconditioned resid norm 6.216358544968e+01 true resid norm 2.825089859247e+15 ||r(i)||/||b|| 1.608822343502e+13 140 KSP preconditioned resid norm 5.819590402733e+01 true resid norm 2.365235725586e+15 ||r(i)||/||b|| 1.346946211469e+13 141 KSP preconditioned resid norm 5.322146614464e+01 true resid norm 1.862541582703e+15 ||r(i)||/||b|| 1.060673700040e+13 142 KSP preconditioned resid norm 4.838864093542e+01 true resid norm 1.460907819000e+15 ||r(i)||/||b|| 8.319527017206e+12 143 KSP preconditioned resid norm 4.288902076387e+01 true resid norm 1.075226017243e+15 ||r(i)||/||b|| 6.123159711867e+12 144 KSP preconditioned resid norm 3.774714312484e+01 true resid norm 7.813012695376e+14 ||r(i)||/||b|| 4.449327285375e+12 145 KSP preconditioned resid norm 3.210844216241e+01 true resid norm 5.182913258583e+14 ||r(i)||/||b|| 2.951547409208e+12 146 KSP preconditioned resid norm 2.633903472489e+01 true resid norm 3.154832514028e+14 ||r(i)||/||b|| 1.796603043249e+12 147 KSP preconditioned resid norm 2.195326834454e+01 true resid norm 1.974701889296e+14 ||r(i)||/||b|| 1.124546361191e+12 148 KSP preconditioned resid norm 1.932058797915e+01 true resid norm 1.401755026989e+14 ||r(i)||/||b|| 7.982665755408e+11 149 KSP preconditioned resid norm 1.720408988995e+01 true resid norm 1.028932403836e+14 ||r(i)||/||b|| 5.859528452969e+11 150 KSP preconditioned resid norm 1.559612800537e+01 true resid norm 8.129959387556e+13 ||r(i)||/||b|| 4.629820984862e+11 151 KSP preconditioned resid norm 1.528043313778e+01 true resid norm 8.168488419825e+13 ||r(i)||/||b|| 4.651762364102e+11 152 KSP preconditioned resid norm 1.473378998995e+01 true resid norm 8.241472575976e+13 ||r(i)||/||b|| 4.693325127407e+11 153 KSP preconditioned resid norm 1.446554996245e+01 true resid norm 8.283307200666e+13 ||r(i)||/||b|| 4.717148963917e+11 154 KSP preconditioned resid norm 1.433069476310e+01 true resid norm 8.320633972939e+13 ||r(i)||/||b|| 4.738405684318e+11 155 KSP preconditioned resid norm 1.398451011406e+01 true resid norm 8.447592357675e+13 ||r(i)||/||b|| 4.810705503521e+11 156 KSP preconditioned resid norm 1.338238273977e+01 true resid norm 8.694936201334e+13 ||r(i)||/||b|| 4.951562015007e+11 157 KSP preconditioned resid norm 1.285596644400e+01 true resid norm 8.942599864914e+13 ||r(i)||/||b|| 5.092600656428e+11 158 KSP preconditioned resid norm 1.259640614474e+01 true resid norm 9.096283789856e+13 ||r(i)||/||b|| 5.180120043281e+11 159 KSP preconditioned resid norm 1.150402605336e+01 true resid norm 9.770715990572e+13 ||r(i)||/||b|| 5.564193346343e+11 160 KSP preconditioned resid norm 1.091253930358e+01 true resid norm 1.013341154326e+14 ||r(i)||/||b|| 5.770739947734e+11 161 KSP preconditioned resid norm 1.005190049488e+01 true resid norm 1.071270024087e+14 ||r(i)||/||b|| 6.100631259688e+11 162 KSP preconditioned resid norm 9.249803666300e+00 true resid norm 1.126347826847e+14 ||r(i)||/||b|| 6.414286414481e+11 163 KSP preconditioned resid norm 8.498257679037e+00 true resid norm 1.184209546140e+14 ||r(i)||/||b|| 6.743795320290e+11 164 KSP preconditioned resid norm 7.683747445800e+00 true resid norm 1.250523348873e+14 ||r(i)||/||b|| 7.121436856789e+11 165 KSP preconditioned resid norm 7.203608831235e+00 true resid norm 1.292064073553e+14 ||r(i)||/||b|| 7.358001530340e+11 166 KSP preconditioned resid norm 6.554687342538e+00 true resid norm 1.355010457000e+14 ||r(i)||/||b|| 7.716466404650e+11 167 KSP preconditioned resid norm 6.040626373592e+00 true resid norm 1.410685680534e+14 ||r(i)||/||b|| 8.033523730485e+11 168 KSP preconditioned resid norm 5.602778891604e+00 true resid norm 1.472337491989e+14 ||r(i)||/||b|| 8.384616321263e+11 169 KSP preconditioned resid norm 5.137313672639e+00 true resid norm 1.546288196744e+14 ||r(i)||/||b|| 8.805748221682e+11 170 KSP preconditioned resid norm 4.772073867658e+00 true resid norm 1.617510890095e+14 ||r(i)||/||b|| 9.211344737672e+11 171 KSP preconditioned resid norm 4.429551648207e+00 true resid norm 1.699873935479e+14 ||r(i)||/||b|| 9.680382942806e+11 172 KSP preconditioned resid norm 4.058041912274e+00 true resid norm 1.807915228232e+14 ||r(i)||/||b|| 1.029565273761e+12 173 KSP preconditioned resid norm 3.659353082764e+00 true resid norm 1.979585093233e+14 ||r(i)||/||b|| 1.127327231179e+12 174 KSP preconditioned resid norm 3.345837180280e+00 true resid norm 2.142616541693e+14 ||r(i)||/||b|| 1.220169813201e+12 175 KSP preconditioned resid norm 3.144285418302e+00 true resid norm 2.281976140099e+14 ||r(i)||/||b|| 1.299531832417e+12 176 KSP preconditioned resid norm 3.022396101784e+00 true resid norm 2.394374033937e+14 ||r(i)||/||b|| 1.363539793925e+12 177 KSP preconditioned resid norm 2.937050870466e+00 true resid norm 2.511983521409e+14 ||r(i)||/||b|| 1.430515635644e+12 178 KSP preconditioned resid norm 2.894843034244e+00 true resid norm 2.630993320056e+14 ||r(i)||/||b|| 1.498288921698e+12 179 KSP preconditioned resid norm 2.891858751192e+00 true resid norm 2.664327984692e+14 ||r(i)||/||b|| 1.517272230531e+12 180 KSP preconditioned resid norm 2.881378316904e+00 true resid norm 2.599860822183e+14 ||r(i)||/||b|| 1.480559695131e+12 181 KSP preconditioned resid norm 2.879541972597e+00 true resid norm 2.605905686926e+14 ||r(i)||/||b|| 1.484002103672e+12 182 KSP preconditioned resid norm 2.877331259337e+00 true resid norm 2.614134463899e+14 ||r(i)||/||b|| 1.488688198951e+12 183 KSP preconditioned resid norm 2.874456654944e+00 true resid norm 2.627919787038e+14 ||r(i)||/||b|| 1.496538616809e+12 184 KSP preconditioned resid norm 2.874349383523e+00 true resid norm 2.629889220630e+14 ||r(i)||/||b|| 1.497660162998e+12 185 KSP preconditioned resid norm 2.874315970654e+00 true resid norm 2.632240084014e+14 ||r(i)||/||b|| 1.498998924499e+12 186 KSP preconditioned resid norm 2.873426488098e+00 true resid norm 2.616629867267e+14 ||r(i)||/||b|| 1.490109272579e+12 187 KSP preconditioned resid norm 2.871828875691e+00 true resid norm 2.600477959076e+14 ||r(i)||/||b|| 1.480911140101e+12 188 KSP preconditioned resid norm 2.866311172083e+00 true resid norm 2.569199084288e+14 ||r(i)||/||b|| 1.463098555318e+12 189 KSP preconditioned resid norm 2.846490699873e+00 true resid norm 2.470737050314e+14 ||r(i)||/||b|| 1.407026738796e+12 190 KSP preconditioned resid norm 2.819730703425e+00 true resid norm 2.362845722118e+14 ||r(i)||/||b|| 1.345585160610e+12 191 KSP preconditioned resid norm 2.792935326546e+00 true resid norm 2.273870208563e+14 ||r(i)||/||b|| 1.294915694729e+12 192 KSP preconditioned resid norm 2.756924664914e+00 true resid norm 2.167287516776e+14 ||r(i)||/||b|| 1.234219354251e+12 193 KSP preconditioned resid norm 2.712358962874e+00 true resid norm 2.044300382742e+14 ||r(i)||/||b|| 1.164181069080e+12 194 KSP preconditioned resid norm 2.634447232139e+00 true resid norm 1.861632836558e+14 ||r(i)||/||b|| 1.060156190448e+12 195 KSP preconditioned resid norm 2.554392023213e+00 true resid norm 1.695435830513e+14 ||r(i)||/||b|| 9.655108977062e+11 196 KSP preconditioned resid norm 2.423823542815e+00 true resid norm 1.453834575243e+14 ||r(i)||/||b|| 8.279246554760e+11 197 KSP preconditioned resid norm 2.301144929663e+00 true resid norm 1.254689122515e+14 ||r(i)||/||b|| 7.145159959579e+11 198 KSP preconditioned resid norm 2.127715354714e+00 true resid norm 1.001847989583e+14 ||r(i)||/||b|| 5.705289073048e+11 199 KSP preconditioned resid norm 2.009569485205e+00 true resid norm 8.505075510895e+13 ||r(i)||/||b|| 4.843440809612e+11 200 KSP preconditioned resid norm 1.877928091553e+00 true resid norm 7.067964496444e+13 ||r(i)||/||b|| 4.025039829348e+11 201 KSP preconditioned resid norm 1.742305371081e+00 true resid norm 5.778287055757e+13 ||r(i)||/||b|| 3.290598807695e+11 202 KSP preconditioned resid norm 1.552693582614e+00 true resid norm 4.295277252766e+13 ||r(i)||/||b|| 2.446059545033e+11 203 KSP preconditioned resid norm 1.405130331702e+00 true resid norm 3.341307717615e+13 ||r(i)||/||b|| 1.902796293370e+11 204 KSP preconditioned resid norm 1.207664877323e+00 true resid norm 2.297639302916e+13 ||r(i)||/||b|| 1.308451635880e+11 205 KSP preconditioned resid norm 1.053317227920e+00 true resid norm 1.622811796763e+13 ||r(i)||/||b|| 9.241532156523e+10 206 KSP preconditioned resid norm 8.888835507681e-01 true resid norm 1.056119524799e+13 ||r(i)||/||b|| 6.014352723484e+10 207 KSP preconditioned resid norm 7.509558493683e-01 true resid norm 6.806255926965e+12 ||r(i)||/||b|| 3.876002943783e+10 208 KSP preconditioned resid norm 6.625677908839e-01 true resid norm 4.847382413253e+12 ||r(i)||/||b|| 2.760470470847e+10 209 KSP preconditioned resid norm 5.988641055583e-01 true resid norm 3.692435549416e+12 ||r(i)||/||b|| 2.102755349320e+10 210 KSP preconditioned resid norm 5.549019306325e-01 true resid norm 3.047376495457e+12 ||r(i)||/||b|| 1.735409363673e+10 211 KSP preconditioned resid norm 5.455063254750e-01 true resid norm 3.068911085737e+12 ||r(i)||/||b|| 1.747672807218e+10 212 KSP preconditioned resid norm 5.256033146031e-01 true resid norm 3.117144722512e+12 ||r(i)||/||b|| 1.775140730866e+10 213 KSP preconditioned resid norm 5.188506724092e-01 true resid norm 3.135841625267e+12 ||r(i)||/||b|| 1.785788177993e+10 214 KSP preconditioned resid norm 5.064655860471e-01 true resid norm 3.177028328480e+12 ||r(i)||/||b|| 1.809243038435e+10 215 KSP preconditioned resid norm 4.942961912759e-01 true resid norm 3.227522585036e+12 ||r(i)||/||b|| 1.837998331971e+10 216 KSP preconditioned resid norm 4.748831446073e-01 true resid norm 3.319195929690e+12 ||r(i)||/||b|| 1.890204149318e+10 217 KSP preconditioned resid norm 4.515763724804e-01 true resid norm 3.439076398391e+12 ||r(i)||/||b|| 1.958473261525e+10 218 KSP preconditioned resid norm 4.373634838140e-01 true resid norm 3.528928306121e+12 ||r(i)||/||b|| 2.009641813311e+10 219 KSP preconditioned resid norm 4.009407832729e-01 true resid norm 3.761064602302e+12 ||r(i)||/||b|| 2.141837983572e+10 220 KSP preconditioned resid norm 3.740641614089e-01 true resid norm 3.936779171425e+12 ||r(i)||/||b|| 2.241903305019e+10 221 KSP preconditioned resid norm 3.517620162262e-01 true resid norm 4.097027645608e+12 ||r(i)||/||b|| 2.333161048533e+10 222 KSP preconditioned resid norm 3.229170697014e-01 true resid norm 4.313508830841e+12 ||r(i)||/||b|| 2.456442000680e+10 223 KSP preconditioned resid norm 2.994650896035e-01 true resid norm 4.502568103319e+12 ||r(i)||/||b|| 2.564106817363e+10 224 KSP preconditioned resid norm 2.773846058866e-01 true resid norm 4.710002710255e+12 ||r(i)||/||b|| 2.682235955578e+10 225 KSP preconditioned resid norm 2.501291804794e-01 true resid norm 4.969587839164e+12 ||r(i)||/||b|| 2.830063591596e+10 226 KSP preconditioned resid norm 2.304709168381e-01 true resid norm 5.185431689243e+12 ||r(i)||/||b|| 2.952981596337e+10 227 KSP preconditioned resid norm 2.100777905891e-01 true resid norm 5.450168294124e+12 ||r(i)||/||b|| 3.103742876968e+10 228 KSP preconditioned resid norm 1.902333022433e-01 true resid norm 5.764157521664e+12 ||r(i)||/||b|| 3.282552369782e+10 229 KSP preconditioned resid norm 1.696089510455e-01 true resid norm 6.179725445319e+12 ||r(i)||/||b|| 3.519208545029e+10 230 KSP preconditioned resid norm 1.581595533833e-01 true resid norm 6.486809236950e+12 ||r(i)||/||b|| 3.694085554227e+10 231 KSP preconditioned resid norm 1.508618057318e-01 true resid norm 6.725919801784e+12 ||r(i)||/||b|| 3.830253406734e+10 232 KSP preconditioned resid norm 1.423058782095e-01 true resid norm 7.059550776628e+12 ||r(i)||/||b|| 4.020248413462e+10 233 KSP preconditioned resid norm 1.315340285601e-01 true resid norm 7.576206972091e+12 ||r(i)||/||b|| 4.314471986014e+10 234 KSP preconditioned resid norm 1.237625992155e-01 true resid norm 8.062848589615e+12 ||r(i)||/||b|| 4.591602961154e+10 235 KSP preconditioned resid norm 1.167453140487e-01 true resid norm 8.626376541470e+12 ||r(i)||/||b|| 4.912518898453e+10 236 KSP preconditioned resid norm 1.118147547475e-01 true resid norm 9.193248341542e+12 ||r(i)||/||b|| 5.235339078799e+10 237 KSP preconditioned resid norm 1.093201267993e-01 true resid norm 9.616798592267e+12 ||r(i)||/||b|| 5.476541001892e+10 238 KSP preconditioned resid norm 1.083570175037e-01 true resid norm 9.931570457757e+12 ||r(i)||/||b|| 5.655796188643e+10 239 KSP preconditioned resid norm 1.082842938953e-01 true resid norm 1.003792921081e+13 ||r(i)||/||b|| 5.716365001267e+10 240 KSP preconditioned resid norm 1.078314038130e-01 true resid norm 9.849622726373e+12 ||r(i)||/||b|| 5.609128879701e+10 241 KSP preconditioned resid norm 1.077292227053e-01 true resid norm 9.870891496109e+12 ||r(i)||/||b|| 5.621240944688e+10 242 KSP preconditioned resid norm 1.075373948198e-01 true resid norm 9.914536589062e+12 ||r(i)||/||b|| 5.646095800365e+10 243 KSP preconditioned resid norm 1.074752343474e-01 true resid norm 9.944186669264e+12 ||r(i)||/||b|| 5.662980824875e+10 244 KSP preconditioned resid norm 1.074752341803e-01 true resid norm 9.944105914292e+12 ||r(i)||/||b|| 5.662934836815e+10 245 KSP preconditioned resid norm 1.074690883936e-01 true resid norm 9.924451054116e+12 ||r(i)||/||b|| 5.651741855428e+10 246 KSP preconditioned resid norm 1.073632398521e-01 true resid norm 9.811970192974e+12 ||r(i)||/||b|| 5.587686645988e+10 247 KSP preconditioned resid norm 1.072870146724e-01 true resid norm 9.751045941996e+12 ||r(i)||/||b|| 5.552991715520e+10 248 KSP preconditioned resid norm 1.067010059812e-01 true resid norm 9.474108561586e+12 ||r(i)||/||b|| 5.395282379692e+10 249 KSP preconditioned resid norm 1.055793587548e-01 true resid norm 9.010840121971e+12 ||r(i)||/||b|| 5.131461880584e+10 250 KSP preconditioned resid norm 1.047262495938e-01 true resid norm 8.722549728520e+12 ||r(i)||/||b|| 4.967287270391e+10 251 KSP preconditioned resid norm 1.029466337065e-01 true resid norm 8.186021332697e+12 ||r(i)||/||b|| 4.661746946320e+10 252 KSP preconditioned resid norm 1.004304639605e-01 true resid norm 7.495626733836e+12 ||r(i)||/||b|| 4.268583432301e+10 253 KSP preconditioned resid norm 9.795540774722e-02 true resid norm 6.920483082409e+12 ||r(i)||/||b|| 3.941052626826e+10 254 KSP preconditioned resid norm 9.555358753906e-02 true resid norm 6.409473015860e+12 ||r(i)||/||b|| 3.650044392123e+10 255 KSP preconditioned resid norm 9.212232118013e-02 true resid norm 5.754525445666e+12 ||r(i)||/||b|| 3.277067128656e+10 256 KSP preconditioned resid norm 8.758697690555e-02 true resid norm 4.960269002563e+12 ||r(i)||/||b|| 2.824756732952e+10 257 KSP preconditioned resid norm 8.129606182327e-02 true resid norm 4.021896818151e+12 ||r(i)||/||b|| 2.290375806320e+10 258 KSP preconditioned resid norm 7.531177419725e-02 true resid norm 3.231819088719e+12 ||r(i)||/||b|| 1.840445089939e+10 259 KSP preconditioned resid norm 7.147261886049e-02 true resid norm 2.780265759853e+12 ||r(i)||/||b|| 1.583296071339e+10 260 KSP preconditioned resid norm 6.629473978281e-02 true resid norm 2.268089888953e+12 ||r(i)||/||b|| 1.291623938430e+10 261 KSP preconditioned resid norm 6.208831287626e-02 true resid norm 1.905334327725e+12 ||r(i)||/||b|| 1.085043163584e+10 262 KSP preconditioned resid norm 5.627448281017e-02 true resid norm 1.483140584032e+12 ||r(i)||/||b|| 8.446137393950e+09 263 KSP preconditioned resid norm 4.989994432902e-02 true resid norm 1.096525121051e+12 ||r(i)||/||b|| 6.244453107160e+09 264 KSP preconditioned resid norm 4.497048672753e-02 true resid norm 8.467526728715e+11 ||r(i)||/||b|| 4.822057659783e+09 265 KSP preconditioned resid norm 3.747768169066e-02 true resid norm 5.348775283731e+11 ||r(i)||/||b|| 3.046001938193e+09 266 KSP preconditioned resid norm 3.272507533606e-02 true resid norm 3.761068823582e+11 ||r(i)||/||b|| 2.141840387491e+09 267 KSP preconditioned resid norm 2.833978687737e-02 true resid norm 2.584061281222e+11 ||r(i)||/||b|| 1.471562227517e+09 268 KSP preconditioned resid norm 2.427354160616e-02 true resid norm 1.695159727985e+11 ||r(i)||/||b|| 9.653536637995e+08 269 KSP preconditioned resid norm 2.223267559447e-02 true resid norm 1.346280124342e+11 ||r(i)||/||b|| 7.666749209995e+08 270 KSP preconditioned resid norm 2.051292146863e-02 true resid norm 1.110744657811e+11 ||r(i)||/||b|| 6.325430030351e+08 271 KSP preconditioned resid norm 2.015852923458e-02 true resid norm 1.116963414691e+11 ||r(i)||/||b|| 6.360844390658e+08 272 KSP preconditioned resid norm 1.928845567194e-02 true resid norm 1.133138718159e+11 ||r(i)||/||b|| 6.452958946045e+08 273 KSP preconditioned resid norm 1.901639059774e-02 true resid norm 1.139195524924e+11 ||r(i)||/||b|| 6.487451038471e+08 274 KSP preconditioned resid norm 1.867369139213e-02 true resid norm 1.148722990709e+11 ||r(i)||/||b|| 6.541707719128e+08 275 KSP preconditioned resid norm 1.820272269340e-02 true resid norm 1.164715379777e+11 ||r(i)||/||b|| 6.632780619961e+08 276 KSP preconditioned resid norm 1.749869482046e-02 true resid norm 1.194547750039e+11 ||r(i)||/||b|| 6.802668964145e+08 277 KSP preconditioned resid norm 1.687368081121e-02 true resid norm 1.224936003716e+11 ||r(i)||/||b|| 6.975722933867e+08 278 KSP preconditioned resid norm 1.639575587312e-02 true resid norm 1.252604862821e+11 ||r(i)||/||b|| 7.133290589993e+08 279 KSP preconditioned resid norm 1.496512503896e-02 true resid norm 1.337998974184e+11 ||r(i)||/||b|| 7.619590004206e+08 280 KSP preconditioned resid norm 1.418029241433e-02 true resid norm 1.386300487682e+11 ||r(i)||/||b|| 7.894655782685e+08 281 KSP preconditioned resid norm 1.331740374144e-02 true resid norm 1.444340453171e+11 ||r(i)||/||b|| 8.225179758723e+08 282 KSP preconditioned resid norm 1.222653019600e-02 true resid norm 1.521351041691e+11 ||r(i)||/||b|| 8.663736978743e+08 283 KSP preconditioned resid norm 1.131424510923e-02 true resid norm 1.595359004069e+11 ||r(i)||/||b|| 9.085194947878e+08 284 KSP preconditioned resid norm 1.037042507094e-02 true resid norm 1.680264407990e+11 ||r(i)||/||b|| 9.568711287950e+08 285 KSP preconditioned resid norm 9.688182631346e-03 true resid norm 1.743226122924e+11 ||r(i)||/||b|| 9.927263471485e+08 286 KSP preconditioned resid norm 9.052103564731e-03 true resid norm 1.811486798072e+11 ||r(i)||/||b|| 1.031599198927e+09 287 KSP preconditioned resid norm 8.110652749532e-03 true resid norm 1.926557561601e+11 ||r(i)||/||b|| 1.097129296968e+09 288 KSP preconditioned resid norm 7.508113362095e-03 true resid norm 2.010760992496e+11 ||r(i)||/||b|| 1.145081173818e+09 289 KSP preconditioned resid norm 6.917230248586e-03 true resid norm 2.109956414742e+11 ||r(i)||/||b|| 1.201570637741e+09 290 KSP preconditioned resid norm 6.270792144381e-03 true resid norm 2.235532529095e+11 ||r(i)||/||b|| 1.273083286417e+09 291 KSP preconditioned resid norm 5.723434422299e-03 true resid norm 2.366863243489e+11 ||r(i)||/||b|| 1.347873044702e+09 292 KSP preconditioned resid norm 5.384057623534e-03 true resid norm 2.476827107725e+11 ||r(i)||/||b|| 1.410494883501e+09 293 KSP preconditioned resid norm 5.001836333933e-03 true resid norm 2.646740880851e+11 ||r(i)||/||b|| 1.507256787828e+09 294 KSP preconditioned resid norm 4.705081336940e-03 true resid norm 2.809588439377e+11 ||r(i)||/||b|| 1.599994648849e+09 295 KSP preconditioned resid norm 4.424589025495e-03 true resid norm 3.023646578354e+11 ||r(i)||/||b|| 1.721895733046e+09 296 KSP preconditioned resid norm 4.204809966953e-03 true resid norm 3.239120797248e+11 ||r(i)||/||b|| 1.844603241506e+09 297 KSP preconditioned resid norm 4.020983239105e-03 true resid norm 3.491084935202e+11 ||r(i)||/||b|| 1.988090902111e+09 298 KSP preconditioned resid norm 3.962918432092e-03 true resid norm 3.653308451948e+11 ||r(i)||/||b|| 2.080473385992e+09 299 KSP preconditioned resid norm 3.957383148812e-03 true resid norm 3.700697096946e+11 ||r(i)||/||b|| 2.107460106662e+09 300 KSP preconditioned resid norm 3.939097482021e-03 true resid norm 3.622457576700e+11 ||r(i)||/||b|| 2.062904536897e+09 301 KSP preconditioned resid norm 3.936139794992e-03 true resid norm 3.628684964620e+11 ||r(i)||/||b|| 2.066450888102e+09 302 KSP preconditioned resid norm 3.931455705928e-03 true resid norm 3.639581395965e+11 ||r(i)||/||b|| 2.072656149912e+09 303 KSP preconditioned resid norm 3.926424749526e-03 true resid norm 3.655814698003e+11 ||r(i)||/||b|| 2.081900634275e+09 304 KSP preconditioned resid norm 3.926009575006e-03 true resid norm 3.661253634305e+11 ||r(i)||/||b|| 2.084997980796e+09 305 KSP preconditioned resid norm 3.925339982792e-03 true resid norm 3.649034775025e+11 ||r(i)||/||b|| 2.078039627327e+09 306 KSP preconditioned resid norm 3.923356505149e-03 true resid norm 3.626519814581e+11 ||r(i)||/||b|| 2.065217885991e+09 307 KSP preconditioned resid norm 3.917014224631e-03 true resid norm 3.580396053341e+11 ||r(i)||/||b|| 2.038951486923e+09 308 KSP preconditioned resid norm 3.903303707944e-03 true resid norm 3.514819854106e+11 ||r(i)||/||b|| 2.001607381146e+09 309 KSP preconditioned resid norm 3.868422381327e-03 true resid norm 3.361181915994e+11 ||r(i)||/||b|| 1.914114182714e+09 310 KSP preconditioned resid norm 3.830516524577e-03 true resid norm 3.221653520608e+11 ||r(i)||/||b|| 1.834656037581e+09 311 KSP preconditioned resid norm 3.796415788584e-03 true resid norm 3.115912412675e+11 ||r(i)||/||b|| 1.774438959348e+09 312 KSP preconditioned resid norm 3.690310261976e-03 true resid norm 2.821305118145e+11 ||r(i)||/||b|| 1.606667022308e+09 313 KSP preconditioned resid norm 3.593395172499e-03 true resid norm 2.581263239915e+11 ||r(i)||/||b|| 1.469968808688e+09 314 KSP preconditioned resid norm 3.480205774823e-03 true resid norm 2.333504532643e+11 ||r(i)||/||b|| 1.328876042116e+09 315 KSP preconditioned resid norm 3.353602368302e-03 true resid norm 2.081533340394e+11 ||r(i)||/||b|| 1.185384364256e+09 316 KSP preconditioned resid norm 3.194409146950e-03 true resid norm 1.800154987321e+11 ||r(i)||/||b|| 1.025145998768e+09 317 KSP preconditioned resid norm 3.071672481669e-03 true resid norm 1.608433892393e+11 ||r(i)||/||b|| 9.159653366981e+08 318 KSP preconditioned resid norm 2.839844293931e-03 true resid norm 1.286439364064e+11 ||r(i)||/||b|| 7.325970130446e+08 319 KSP preconditioned resid norm 2.631447463216e-03 true resid norm 1.035755023455e+11 ||r(i)||/||b|| 5.898381669786e+08 320 KSP preconditioned resid norm 2.421709878855e-03 true resid norm 8.242231041182e+10 ||r(i)||/||b|| 4.693757055534e+08 321 KSP preconditioned resid norm 2.249677749119e-03 true resid norm 6.796005580822e+10 ||r(i)||/||b|| 3.870165612326e+08 322 KSP preconditioned resid norm 2.051679744073e-03 true resid norm 5.362063892849e+10 ||r(i)||/||b|| 3.053569489078e+08 323 KSP preconditioned resid norm 1.852618445705e-03 true resid norm 4.140290200052e+10 ||r(i)||/||b|| 2.357798057510e+08 324 KSP preconditioned resid norm 1.586867536205e-03 true resid norm 2.821223751515e+10 ||r(i)||/||b|| 1.606620685922e+08 325 KSP preconditioned resid norm 1.397039923183e-03 true resid norm 2.041649071932e+10 ||r(i)||/||b|| 1.162671209825e+08 326 KSP preconditioned resid norm 1.221057962383e-03 true resid norm 1.446679933564e+10 ||r(i)||/||b|| 8.238502550265e+07 327 KSP preconditioned resid norm 1.033496887800e-03 true resid norm 9.360606892139e+09 ||r(i)||/||b|| 5.330645843890e+07 328 KSP preconditioned resid norm 9.116911789257e-04 true resid norm 6.661951473115e+09 ||r(i)||/||b|| 3.793824945494e+07 329 KSP preconditioned resid norm 8.282610842511e-04 true resid norm 5.144839330536e+09 ||r(i)||/||b|| 2.929865201137e+07 330 KSP preconditioned resid norm 7.530731044695e-04 true resid norm 4.075530635865e+09 ||r(i)||/||b|| 2.320919006220e+07 331 KSP preconditioned resid norm 7.379285385788e-04 true resid norm 4.106395550093e+09 ||r(i)||/||b|| 2.338495850184e+07 332 KSP preconditioned resid norm 7.208624023923e-04 true resid norm 4.143156222668e+09 ||r(i)||/||b|| 2.359430190098e+07 333 KSP preconditioned resid norm 7.065775764655e-04 true resid norm 4.177514984231e+09 ||r(i)||/||b|| 2.378996698086e+07 334 KSP preconditioned resid norm 6.922452814868e-04 true resid norm 4.218601053057e+09 ||r(i)||/||b|| 2.402394249607e+07 335 KSP preconditioned resid norm 6.744135487420e-04 true resid norm 4.282753533716e+09 ||r(i)||/||b|| 2.438927581083e+07 336 KSP preconditioned resid norm 6.469130429753e-04 true resid norm 4.403672957096e+09 ||r(i)||/||b|| 2.507788353585e+07 337 KSP preconditioned resid norm 6.112369725507e-04 true resid norm 4.573122433971e+09 ||r(i)||/||b|| 2.604285852098e+07 338 KSP preconditioned resid norm 5.788565956748e-04 true resid norm 4.751840775440e+09 ||r(i)||/||b|| 2.706061751370e+07 339 KSP preconditioned resid norm 5.410631845567e-04 true resid norm 4.973946362559e+09 ||r(i)||/||b|| 2.832545668334e+07 340 KSP preconditioned resid norm 5.009844336682e-04 true resid norm 5.235146248376e+09 ||r(i)||/||b|| 2.981292870497e+07 341 KSP preconditioned resid norm 4.659768311587e-04 true resid norm 5.473524476167e+09 ||r(i)||/||b|| 3.117043674253e+07 342 KSP preconditioned resid norm 4.188668678783e-04 true resid norm 5.805084466010e+09 ||r(i)||/||b|| 3.305859303647e+07 343 KSP preconditioned resid norm 3.879486344930e-04 true resid norm 6.063617165803e+09 ||r(i)||/||b|| 3.453087605994e+07 344 KSP preconditioned resid norm 3.714346767924e-04 true resid norm 6.230541109727e+09 ||r(i)||/||b|| 3.548146872789e+07 345 KSP preconditioned resid norm 3.513107945823e-04 true resid norm 6.444239345039e+09 ||r(i)||/||b|| 3.669843000299e+07 346 KSP preconditioned resid norm 3.188508242839e-04 true resid norm 6.808853174711e+09 ||r(i)||/||b|| 3.877482015393e+07 347 KSP preconditioned resid norm 2.976989911735e-04 true resid norm 7.088594142323e+09 ||r(i)||/||b|| 4.036787928305e+07 348 KSP preconditioned resid norm 2.760902621649e-04 true resid norm 7.404251197607e+09 ||r(i)||/||b|| 4.216547209859e+07 349 KSP preconditioned resid norm 2.491238069081e-04 true resid norm 7.856460563260e+09 ||r(i)||/||b|| 4.474069826006e+07 350 KSP preconditioned resid norm 2.261931392463e-04 true resid norm 8.300304774785e+09 ||r(i)||/||b|| 4.726828683286e+07 351 KSP preconditioned resid norm 2.108262997601e-04 true resid norm 8.660994767295e+09 ||r(i)||/||b|| 4.932233165246e+07 352 KSP preconditioned resid norm 1.952804314878e-04 true resid norm 9.171687969602e+09 ||r(i)||/||b|| 5.223060953203e+07 353 KSP preconditioned resid norm 1.857561963852e-04 true resid norm 9.612624044306e+09 ||r(i)||/||b|| 5.474163694844e+07 354 KSP preconditioned resid norm 1.755682138867e-04 true resid norm 1.022824097483e+10 ||r(i)||/||b|| 5.824743082478e+07 355 KSP preconditioned resid norm 1.629922761938e-04 true resid norm 1.128526888154e+10 ||r(i)||/||b|| 6.426695656995e+07 356 KSP preconditioned resid norm 1.523745456214e-04 true resid norm 1.237228264869e+10 ||r(i)||/||b|| 7.045724474987e+07 357 KSP preconditioned resid norm 1.479156994814e-04 true resid norm 1.303748936001e+10 ||r(i)||/||b|| 7.424544078444e+07 358 KSP preconditioned resid norm 1.460139994385e-04 true resid norm 1.352484512512e+10 ||r(i)||/||b|| 7.702081743870e+07 359 KSP preconditioned resid norm 1.458891367415e-04 true resid norm 1.366183537709e+10 ||r(i)||/||b|| 7.780094475922e+07 360 KSP preconditioned resid norm 1.455077198104e-04 true resid norm 1.345578836961e+10 ||r(i)||/||b|| 7.662755543012e+07 361 KSP preconditioned resid norm 1.453704662869e-04 true resid norm 1.350023349252e+10 ||r(i)||/||b|| 7.688065996966e+07 362 KSP preconditioned resid norm 1.452981914522e-04 true resid norm 1.353022367766e+10 ||r(i)||/||b|| 7.705144703254e+07 363 KSP preconditioned resid norm 1.452129596669e-04 true resid norm 1.358122759269e+10 ||r(i)||/||b|| 7.734190235326e+07 364 KSP preconditioned resid norm 1.452121241088e-04 true resid norm 1.358710317940e+10 ||r(i)||/||b|| 7.737536244000e+07 365 KSP preconditioned resid norm 1.452046967752e-04 true resid norm 1.356520699403e+10 ||r(i)||/||b|| 7.725066880542e+07 366 KSP preconditioned resid norm 1.450384393428e-04 true resid norm 1.342016311361e+10 ||r(i)||/||b|| 7.642467796180e+07 367 KSP preconditioned resid norm 1.448709937803e-04 true resid norm 1.330896122194e+10 ||r(i)||/||b|| 7.579140929825e+07 368 KSP preconditioned resid norm 1.440619441880e-04 true resid norm 1.296138556368e+10 ||r(i)||/||b|| 7.381204753305e+07 369 KSP preconditioned resid norm 1.424189519513e-04 true resid norm 1.234231668356e+10 ||r(i)||/||b|| 7.028659561425e+07 370 KSP preconditioned resid norm 1.405178593540e-04 true resid norm 1.171445058083e+10 ||r(i)||/||b|| 6.671104557822e+07 371 KSP preconditioned resid norm 1.384125397526e-04 true resid norm 1.109105276720e+10 ||r(i)||/||b|| 6.316094139954e+07 372 KSP preconditioned resid norm 1.349489907590e-04 true resid norm 1.017955796073e+10 ||r(i)||/||b|| 5.797019248993e+07 373 KSP preconditioned resid norm 1.299224645965e-04 true resid norm 9.012691544632e+09 ||r(i)||/||b|| 5.132516222319e+07 374 KSP preconditioned resid norm 1.244108688000e-04 true resid norm 7.863710058584e+09 ||r(i)||/||b|| 4.478198243380e+07 375 KSP preconditioned resid norm 1.196624061573e-04 true resid norm 6.954908378743e+09 ||r(i)||/||b|| 3.960657024804e+07 376 KSP preconditioned resid norm 1.150069930686e-04 true resid norm 6.146973261882e+09 ||r(i)||/||b|| 3.500556945562e+07 377 KSP preconditioned resid norm 1.078887656664e-04 true resid norm 5.102072199579e+09 ||r(i)||/||b|| 2.905510324203e+07 378 KSP preconditioned resid norm 1.020396071938e-04 true resid norm 4.354821250020e+09 ||r(i)||/||b|| 2.479968453413e+07 379 KSP preconditioned resid norm 9.618818827534e-05 true resid norm 3.697423694068e+09 ||r(i)||/||b|| 2.105595980580e+07 380 KSP preconditioned resid norm 9.035897971170e-05 true resid norm 3.119110900940e+09 ||r(i)||/||b|| 1.776260423316e+07 KSP Object: 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-10, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, blocksize = 1, factorization LOWER Preconditioner for the Schur complement formed from S itself Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_1_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_) 4 MPI processes type: lsc KSP Object: (fieldsplit_1_lsc_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 package used to perform factorization: petsc total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=9549, allocated nonzeros=9549 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines linear system matrix = precond matrix: Mat Object: (fieldsplit_1_) 4 MPI processes type: schurcomplement rows=3183, cols=3183 Schur complement A11 - A10 inv(A00) A01 A11 Mat Object: (fieldsplit_1_) 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=3183, allocated nonzeros=3183 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines A10 Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=71946 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines KSP of A00 KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 A01 Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=3183 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 3757 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=75129, cols=75129 total: nonzeros=3.0016e+06, allocated nonzeros=1.26217e+07 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Recovering stress ... Cleaning up ... Finished 0.00user 0.02system 0:26.40elapsed 0%CPU (0avgtext+0avgdata 17056maxresident)k 1168inputs+0outputs (6major+1797minor)pagefaults 0swaps salloc: Relinquishing job allocation 78969 salloc: Job allocation 78969 has been revoked. From mfadams at lbl.gov Tue Oct 28 07:05:14 2014 From: mfadams at lbl.gov (Mark Adams) Date: Tue, 28 Oct 2014 08:05:14 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <7911368.Xkc5LuFApY@besikovitch-ii> References: <9621705.dtimTkejkQ@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> <7911368.Xkc5LuFApY@besikovitch-ii> Message-ID: On Mon, Oct 27, 2014 at 3:02 PM, Filippo Leonardi < filippo.leonardi at sam.math.ethz.ch> wrote: > On Monday 27 October 2014 14:31:45 Mark Adams wrote: > > > > I'm not sure what you are asking. starting i=0 or i=1, and with and > > without mg. So four possibilities? The solutions will be different for > > i=0 or 1 in this code: > > > > Yes, 4 cases: > i = 0, MG: wrong solution *(for *any* solution, even afterwards, i.e. i >= > 0)* > (and error, if debug) > OK, so the first time there is no RHS and so not eigen estimate and the next solves do not check to see if a valid eigen estimate has been created. Jed: could we just check for zero iterations in the eigen estimator and then call the random thing. Or something along those lines. > i = 1, MG: ok > i = 0, CG: (or anything): ok > i = 1, CG (or anything): ok > > > -ksp_chebyshev_estimate_eigenvalues_random > > Can try this but the problem, to me, seems deeper (because inherited from > each > solve afterwards). -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 28 07:08:11 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 07:08:11 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544F726E.6060503@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> Message-ID: On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali wrote: > Mark > > When I replace "-fieldsplit_0_ksp_type preonly" with > "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to kill it). > That means something in the setup is wrong. It should be about the same or faster. Run with -fieldsplit_0_ksp_monitor so we can see what is happening. Matt > With MUMPS , i.e., with '-fieldsplit_0_pc_type lu > -fieldsplit_0_pc_factor_mat_solver_package mumps -fieldsplit_0_ksp_type > preonly' it works fine but takes more time, and will be an issue for larger > problems. The output for this run is attached. > > I will work on passing rigid body modes (as Matt mentioned) but short of > that what is the best set of options for solving the following problem > (i.e., linear elasticity with constraints): > > |K cG'| | u | = |F| > |G 0 | |l/c| |d| > > where c is a scaling factor (so that cG' terms are more or less of the > same order as K)? The constraints are used to impose slip between surfaces > and so on. > > Tabrez > > On 10/27/2014 01:17 PM, Mark Adams wrote: > > The null space for GAMG is not critical but useful for elasticity. If you > in fact have an indefinite operator (eg, not "pinned) the you need to use > an iterative coarse grid solver. You are using '-fieldsplit_0_pc_type gamg > -fieldsplit_0_ksp_type preonly'. And you have a hard elasticity problem. > You are going to want to start with a stronger solver. Use cg instead of > preonly. As Matt said start with MUMPS, then go to CG/GAMG, then you can > see how far you can cut the _0_ solver down. > Mark > > On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley > wrote: > >> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >> wrote: >> >>> Matt >>> >>> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>> >>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>> wrote: >>> >>>> Matt >>>> >>>> Sorry about that (I always forget it). The output for the smallest >>>> problem is now attached (see log.txt). I am also attaching some results >>>> that compare results obtained using FS/LSC and the direct solver (MUMPS), >>>> again for the smallest problem. The difference, as you can see is >>>> insignificant O(1E-6). >>>> >>> >>> 1) How do you use MUMPS if you have a saddle point >>> >>> I simply used -pc_type lu -pc_factor_mat_solver_package mumps. >>> >>> >>> 2) You can see from the output that something is seriously wrong with >>> the preconditioner. It looks like it has a null space. >>> Did you add the elastic null modes to GAMG? Without this, it is not >>> going to work. We have helper functions for this: >>> >>> >>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>> >>> you could just copy that code. And then use >>> >>> >>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>> >>> I don't see it in the output, so I think this is your problem. >>> >>> In order to test, I would first use MUMPS as the A00 solver and get >>> the Schur stuff worked out. Then I would >>> replace MUMPS with GAMG and tune it until I get back my original >>> convergence. >>> >>> I will try this with MatNullSpaceCreateRigidBody. Btw does it matter if >>> some nodes are pinned? >>> >> >> No these are null modes of the operator, not of the particular problem. >> >> Matt >> >> >>> Tabrez >>> >>> >>> Thanks, >>> >>> Matt >>> >>> >>>> Also, I did pass 'upper' and 'full' to >>>> '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't >>>> improve (in fact, it increases slightly). The attached log is with 'upper'. >>>> >>>> Regards, >>>> >>>> Tabrez >>>> >>>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>> >>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>>> wrote: >>>> >>>>> Hello >>>>> >>>>> I am using the following options (below) for solving linear >>>>> elasticity/poroelasticity problems involving slip between two surfaces >>>>> involving non-trivial geometries, i.e., elements with high aspect ratios, >>>>> large contrasts in material properties etc. The constraints are imposed >>>>> using Lagrange Multipliers. >>>>> >>>>> A picture (shows displacement magnitude) is attached. The boundary >>>>> nodes, i.e., the base and the four side are pinned. >>>>> >>>>> The following options appear to work well for the saddle point problem: >>>>> >>>>> -pc_type fieldsplit -pc_fieldsplit_type schur >>>>> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >>>>> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >>>>> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >>>>> -ksp_monitor >>>>> >>>>> However, the number of iterations keep on increasing with the problems >>>>> size (see attached plot), e.g., >>>>> >>>>> 120K Tets *507* Iterations (KSP Residual norm >>>>> 8.827362494659e-05) in 17 secs on 3 cores >>>>> 1 Million Tets *1374* Iterations (KSP Residual norm >>>>> 7.164704416296e-05) in 117 secs on 20 cores >>>>> 8 Million Tets *2495* Iterations (KSP Residual norm >>>>> 9.101247550026e-05) in 225 secs on 160 cores >>>>> >>>>> So what other options should I try to improve solver performance? Any >>>>> tips/insights would be appreciated as preconditioning is black magic to me. >>>>> >>>> >>>> For reports, always run with >>>> >>>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >>>> >>>> so that we can see exactly what you used. >>>> >>>> I believe the default is a diagonal factorization. Since your outer >>>> iterates are increasing, I would strengthen this >>>> to either upper or full >>>> >>>> -pc_fieldsplit_schur_factorization_type >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Thanks in advance. >>>>> >>>>> Tabrez >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Tue Oct 28 08:00:38 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 08:00:38 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> Message-ID: <544F9376.8030008@geology.wisc.edu> Matt With MUMPS it is indeed the same but not faster. E.g., see 1.txt and 2.txt With GAMG, cg takes forever (see 4.txt). For reference GAMG with preonly is attached again (3.txt) All logs include -fieldsplit_0_ksp_monitor. Also, the true residual shows small oscillations. Is that normal? Tabrez On 10/28/2014 07:08 AM, Matthew Knepley wrote: > On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali > wrote: > > Mark > > When I replace "-fieldsplit_0_ksp_type preonly" with > "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to kill > it). > > > That means something in the setup is wrong. It should be about the > same or faster. Run with -fieldsplit_0_ksp_monitor > so we can see what is happening. > > Matt > > With MUMPS , i.e., with '-fieldsplit_0_pc_type lu > -fieldsplit_0_pc_factor_mat_solver_package mumps > -fieldsplit_0_ksp_type preonly' it works fine but takes more time, > and will be an issue for larger problems. The output for this run > is attached. > > I will work on passing rigid body modes (as Matt mentioned) but > short of that what is the best set of options for solving the > following problem (i.e., linear elasticity with constraints): > > |K cG'| | u | = |F| > |G 0 | |l/c| |d| > > where c is a scaling factor (so that cG' terms are more or less of > the same order as K)? The constraints are used to impose slip > between surfaces and so on. > > Tabrez > > On 10/27/2014 01:17 PM, Mark Adams wrote: >> The null space for GAMG is not critical but useful for >> elasticity. If you in fact have an indefinite operator (eg, not >> "pinned) the you need to use an iterative coarse grid solver. >> You are using '-fieldsplit_0_pc_type gamg -fieldsplit_0_ksp_type >> preonly'. And you have a hard elasticity problem. You are going >> to want to start with a stronger solver. Use cg instead of >> preonly. As Matt said start with MUMPS, then go to CG/GAMG, then >> you can see how far you can cut the _0_ solver down. >> Mark >> >> On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley >> > wrote: >> >> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >> > wrote: >> >> Matt >> >> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>> > >>> wrote: >>> >>> Matt >>> >>> Sorry about that (I always forget it). The output >>> for the smallest problem is now attached (see >>> log.txt). I am also attaching some results that >>> compare results obtained using FS/LSC and the direct >>> solver (MUMPS), again for the smallest problem. The >>> difference, as you can see is insignificant O(1E-6). >>> >>> >>> 1) How do you use MUMPS if you have a saddle point >> I simply used -pc_type lu -pc_factor_mat_solver_package >> mumps. >> >>> >>> 2) You can see from the output that something is >>> seriously wrong with the preconditioner. It looks like >>> it has a null space. >>> Did you add the elastic null modes to GAMG? Without >>> this, it is not going to work. We have helper functions >>> for this: >>> >>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>> >>> you could just copy that code. And then use >>> >>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>> >>> I don't see it in the output, so I think this is your >>> problem. >>> >>> In order to test, I would first use MUMPS as the A00 >>> solver and get the Schur stuff worked out. Then I would >>> replace MUMPS with GAMG and tune it until I get back my >>> original convergence. >> I will try this with MatNullSpaceCreateRigidBody. Btw >> does it matter if some nodes are pinned? >> >> >> No these are null modes of the operator, not of the >> particular problem. >> >> Matt >> >> Tabrez >> >>> >>> Thanks, >>> >>> Matt >>> >>> Also, I did pass 'upper' and 'full' to >>> '-pc_fieldsplit_schur_factorization_type' but the >>> iteration count doesn't improve (in fact, it >>> increases slightly). The attached log is with 'upper'. >>> >>> Regards, >>> >>> Tabrez >>> >>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>>> >>> > wrote: >>>> >>>> Hello >>>> >>>> I am using the following options (below) for >>>> solving linear elasticity/poroelasticity >>>> problems involving slip between two surfaces >>>> involving non-trivial geometries, i.e., >>>> elements with high aspect ratios, large >>>> contrasts in material properties etc. The >>>> constraints are imposed using Lagrange >>>> Multipliers. >>>> >>>> A picture (shows displacement magnitude) is >>>> attached. The boundary nodes, i.e., the base >>>> and the four side are pinned. >>>> >>>> The following options appear to work well for >>>> the saddle point problem: >>>> >>>> -pc_type fieldsplit -pc_fieldsplit_type schur >>>> -pc_fieldsplit_detect_saddle_point >>>> -fieldsplit_0_pc_type gamg >>>> -fieldsplit_0_ksp_type preonly >>>> -fieldsplit_1_pc_type lsc >>>> -fieldsplit_1_ksp_type preonly >>>> -pc_fieldsplit_schur_fact_type lower -ksp_monitor >>>> >>>> However, the number of iterations keep on >>>> increasing with the problems size (see attached >>>> plot), e.g., >>>> >>>> 120K Tets *507* Iterations (KSP Residual norm >>>> 8.827362494659e-05)in 17 secs on 3 cores >>>> 1 Million Tets *1374* Iterations (KSP Residual >>>> norm 7.164704416296e-05)in 117 secs on 20 cores >>>> 8 Million Tets *2495* Iterations (KSP Residual >>>> norm 9.101247550026e-05) in 225 secs on 160 cores >>>> >>>> So what other options should I try to improve >>>> solver performance? Any tips/insights would be >>>> appreciated as preconditioning is black magic >>>> to me. >>>> >>>> >>>> For reports, always run with >>>> >>>> -ksp_view -ksp_monitor_true_residual >>>> -ksp_converged_reason >>>> >>>> so that we can see exactly what you used. >>>> >>>> I believe the default is a diagonal factorization. >>>> Since your outer iterates are increasing, I would >>>> strengthen this >>>> to either upper or full >>>> >>>> -pc_fieldsplit_schur_factorization_type >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> Thanks in advance. >>>> >>>> Tabrez >>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before >>>> they begin their experiments is infinitely more >>>> interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they >>> begin their experiments is infinitely more interesting >>> than any results to which their experiments lead. >>> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin >> their experiments is infinitely more interesting than any >> results to which their experiments lead. >> -- Norbert Wiener >> >> > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- salloc: Granted job allocation 79010 Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 9.127510702346e+05 true resid norm 1.362125072634e+14 ||r(i)||/||b|| 7.756982470220e+11 2 KSP preconditioned resid norm 8.626930780275e+05 true resid norm 7.140706098955e+14 ||r(i)||/||b|| 4.066464464051e+12 3 KSP preconditioned resid norm 6.917392651646e+05 true resid norm 2.788588220631e+15 ||r(i)||/||b|| 1.588035517346e+13 4 KSP preconditioned resid norm 5.948180320596e+05 true resid norm 3.828406259238e+15 ||r(i)||/||b|| 2.180187476056e+13 5 KSP preconditioned resid norm 5.365265868551e+05 true resid norm 4.509015181942e+15 ||r(i)||/||b|| 2.567778277265e+13 6 KSP preconditioned resid norm 5.020807291688e+05 true resid norm 4.938429989057e+15 ||r(i)||/||b|| 2.812319927527e+13 7 KSP preconditioned resid norm 4.691107247107e+05 true resid norm 5.504613096315e+15 ||r(i)||/||b|| 3.134747913486e+13 8 KSP preconditioned resid norm 4.500805942145e+05 true resid norm 5.950219821547e+15 ||r(i)||/||b|| 3.388510480939e+13 9 KSP preconditioned resid norm 4.250032501744e+05 true resid norm 6.670720166034e+15 ||r(i)||/||b|| 3.798818510228e+13 10 KSP preconditioned resid norm 4.051645463810e+05 true resid norm 7.414211051141e+15 ||r(i)||/||b|| 4.222219112596e+13 11 KSP preconditioned resid norm 3.880013874032e+05 true resid norm 8.151575238507e+15 ||r(i)||/||b|| 4.642130704452e+13 12 KSP preconditioned resid norm 3.741408602796e+05 true resid norm 8.948407390773e+15 ||r(i)||/||b|| 5.095907906048e+13 13 KSP preconditioned resid norm 3.532148969165e+05 true resid norm 1.028219119134e+16 ||r(i)||/||b|| 5.855466464062e+13 14 KSP preconditioned resid norm 3.306246816868e+05 true resid norm 1.207912805635e+16 ||r(i)||/||b|| 6.878779817734e+13 15 KSP preconditioned resid norm 3.077008490960e+05 true resid norm 1.412194821609e+16 ||r(i)||/||b|| 8.042117934569e+13 16 KSP preconditioned resid norm 2.709436838755e+05 true resid norm 1.756809508461e+16 ||r(i)||/||b|| 1.000461766282e+14 17 KSP preconditioned resid norm 2.491233804162e+05 true resid norm 1.976045004356e+16 ||r(i)||/||b|| 1.125311233683e+14 18 KSP preconditioned resid norm 2.169453185969e+05 true resid norm 2.315544602061e+16 ||r(i)||/||b|| 1.318648283338e+14 19 KSP preconditioned resid norm 1.853345270881e+05 true resid norm 2.647069144540e+16 ||r(i)||/||b|| 1.507443726291e+14 20 KSP preconditioned resid norm 1.684382143134e+05 true resid norm 2.847996525788e+16 ||r(i)||/||b|| 1.621867152263e+14 21 KSP preconditioned resid norm 1.487793180709e+05 true resid norm 3.106944054176e+16 ||r(i)||/||b|| 1.769331689754e+14 22 KSP preconditioned resid norm 1.324108267214e+05 true resid norm 3.364687707291e+16 ||r(i)||/||b|| 1.916110648544e+14 23 KSP preconditioned resid norm 1.154783368236e+05 true resid norm 3.664744963424e+16 ||r(i)||/||b|| 2.086986210755e+14 24 KSP preconditioned resid norm 9.690290840858e+04 true resid norm 4.059423491277e+16 ||r(i)||/||b|| 2.311746365563e+14 25 KSP preconditioned resid norm 8.541667474357e+04 true resid norm 4.382080823727e+16 ||r(i)||/||b|| 2.495492140601e+14 26 KSP preconditioned resid norm 6.738137388843e+04 true resid norm 4.935828200765e+16 ||r(i)||/||b|| 2.810838270183e+14 27 KSP preconditioned resid norm 5.559761934719e+04 true resid norm 5.372591255721e+16 ||r(i)||/||b|| 3.059564575058e+14 28 KSP preconditioned resid norm 4.606357788000e+04 true resid norm 5.783519870721e+16 ||r(i)||/||b|| 3.293578772954e+14 29 KSP preconditioned resid norm 3.846753020759e+04 true resid norm 6.227329505443e+16 ||r(i)||/||b|| 3.546317939556e+14 30 KSP preconditioned resid norm 3.331905232454e+04 true resid norm 6.606924388252e+16 ||r(i)||/||b|| 3.762488312665e+14 31 KSP preconditioned resid norm 3.290201988589e+04 true resid norm 6.618297473864e+16 ||r(i)||/||b|| 3.768965017889e+14 32 KSP preconditioned resid norm 3.162870028106e+04 true resid norm 6.654393352127e+16 ||r(i)||/||b|| 3.789520773051e+14 33 KSP preconditioned resid norm 2.991966449137e+04 true resid norm 6.709255863637e+16 ||r(i)||/||b|| 3.820763685218e+14 34 KSP preconditioned resid norm 2.922952746962e+04 true resid norm 6.743695964938e+16 ||r(i)||/||b|| 3.840376514277e+14 35 KSP preconditioned resid norm 2.798614888132e+04 true resid norm 6.838110987678e+16 ||r(i)||/||b|| 3.894143652923e+14 36 KSP preconditioned resid norm 2.701665220432e+04 true resid norm 6.929642686721e+16 ||r(i)||/||b|| 3.946268806420e+14 37 KSP preconditioned resid norm 2.601811428041e+04 true resid norm 7.044504608605e+16 ||r(i)||/||b|| 4.011679974047e+14 38 KSP preconditioned resid norm 2.531056406740e+04 true resid norm 7.153872989162e+16 ||r(i)||/||b|| 4.073962698874e+14 39 KSP preconditioned resid norm 2.332617529628e+04 true resid norm 7.497512777369e+16 ||r(i)||/||b|| 4.269657489810e+14 40 KSP preconditioned resid norm 2.254584604071e+04 true resid norm 7.640212636547e+16 ||r(i)||/||b|| 4.350921709109e+14 41 KSP preconditioned resid norm 2.082879229582e+04 true resid norm 7.986664049456e+16 ||r(i)||/||b|| 4.548217654299e+14 42 KSP preconditioned resid norm 1.914379086213e+04 true resid norm 8.367676210833e+16 ||r(i)||/||b|| 4.765195134276e+14 43 KSP preconditioned resid norm 1.815822267408e+04 true resid norm 8.615765209333e+16 ||r(i)||/||b|| 4.906475993948e+14 44 KSP preconditioned resid norm 1.677428596734e+04 true resid norm 8.993611308867e+16 ||r(i)||/||b|| 5.121650476043e+14 45 KSP preconditioned resid norm 1.534247135245e+04 true resid norm 9.414497409112e+16 ||r(i)||/||b|| 5.361335227990e+14 46 KSP preconditioned resid norm 1.412608783355e+04 true resid norm 9.852630241878e+16 ||r(i)||/||b|| 5.610841589166e+14 47 KSP preconditioned resid norm 1.177668977489e+04 true resid norm 1.077880272896e+17 ||r(i)||/||b|| 6.138275074610e+14 48 KSP preconditioned resid norm 1.065678918035e+04 true resid norm 1.124882415442e+17 ||r(i)||/||b|| 6.405941240601e+14 49 KSP preconditioned resid norm 9.962410637891e+03 true resid norm 1.158017305203e+17 ||r(i)||/||b|| 6.594636657927e+14 50 KSP preconditioned resid norm 9.371192463165e+03 true resid norm 1.195573007678e+17 ||r(i)||/||b|| 6.808507565679e+14 51 KSP preconditioned resid norm 8.608163058609e+03 true resid norm 1.259685427075e+17 ||r(i)||/||b|| 7.173612740950e+14 52 KSP preconditioned resid norm 7.720924960100e+03 true resid norm 1.350861356595e+17 ||r(i)||/||b|| 7.692838252031e+14 53 KSP preconditioned resid norm 6.628413884735e+03 true resid norm 1.478135305618e+17 ||r(i)||/||b|| 8.417633508586e+14 54 KSP preconditioned resid norm 5.583298607076e+03 true resid norm 1.607393601792e+17 ||r(i)||/||b|| 9.153729156262e+14 55 KSP preconditioned resid norm 4.854896331522e+03 true resid norm 1.721936891597e+17 ||r(i)||/||b|| 9.806026297653e+14 56 KSP preconditioned resid norm 4.262399411032e+03 true resid norm 1.837125351753e+17 ||r(i)||/||b|| 1.046199753271e+15 57 KSP preconditioned resid norm 3.584024669343e+03 true resid norm 2.026508285280e+17 ||r(i)||/||b|| 1.154048887322e+15 58 KSP preconditioned resid norm 3.302611897192e+03 true resid norm 2.155979885016e+17 ||r(i)||/||b|| 1.227779923460e+15 59 KSP preconditioned resid norm 3.106452236261e+03 true resid norm 2.294151324033e+17 ||r(i)||/||b|| 1.306465313802e+15 60 KSP preconditioned resid norm 2.894632302136e+03 true resid norm 2.500169544012e+17 ||r(i)||/||b|| 1.423787852901e+15 61 KSP preconditioned resid norm 2.892427608155e+03 true resid norm 2.503572781296e+17 ||r(i)||/||b|| 1.425725916628e+15 62 KSP preconditioned resid norm 2.881855812247e+03 true resid norm 2.520690203791e+17 ||r(i)||/||b|| 1.435473886833e+15 63 KSP preconditioned resid norm 2.872986380440e+03 true resid norm 2.536302493377e+17 ||r(i)||/||b|| 1.444364719185e+15 64 KSP preconditioned resid norm 2.865635949494e+03 true resid norm 2.551732099897e+17 ||r(i)||/||b|| 1.453151517821e+15 65 KSP preconditioned resid norm 2.860836435780e+03 true resid norm 2.565769302251e+17 ||r(i)||/||b|| 1.461145374977e+15 66 KSP preconditioned resid norm 2.860026342231e+03 true resid norm 2.574294644220e+17 ||r(i)||/||b|| 1.466000357059e+15 67 KSP preconditioned resid norm 2.860026336216e+03 true resid norm 2.574330960398e+17 ||r(i)||/||b|| 1.466021038269e+15 68 KSP preconditioned resid norm 2.859446352241e+03 true resid norm 2.564102967024e+17 ||r(i)||/||b|| 1.460196436189e+15 69 KSP preconditioned resid norm 2.855291192203e+03 true resid norm 2.530743143999e+17 ||r(i)||/||b|| 1.441198800245e+15 70 KSP preconditioned resid norm 2.843683581066e+03 true resid norm 2.467887076608e+17 ||r(i)||/||b|| 1.405403745686e+15 71 KSP preconditioned resid norm 2.825924372442e+03 true resid norm 2.382078643151e+17 ||r(i)||/||b|| 1.356537857561e+15 72 KSP preconditioned resid norm 2.803340022966e+03 true resid norm 2.288941643308e+17 ||r(i)||/||b|| 1.303498522949e+15 73 KSP preconditioned resid norm 2.768188650268e+03 true resid norm 2.170599056138e+17 ||r(i)||/||b|| 1.236105198165e+15 74 KSP preconditioned resid norm 2.728040723494e+03 true resid norm 2.050447447185e+17 ||r(i)||/||b|| 1.167681677951e+15 75 KSP preconditioned resid norm 2.663723516353e+03 true resid norm 1.886813386912e+17 ||r(i)||/||b|| 1.074495922651e+15 76 KSP preconditioned resid norm 2.580490699661e+03 true resid norm 1.697855415282e+17 ||r(i)||/||b|| 9.668887944221e+14 77 KSP preconditioned resid norm 2.507785655285e+03 true resid norm 1.549544648438e+17 ||r(i)||/||b|| 8.824292949480e+14 78 KSP preconditioned resid norm 2.381407885412e+03 true resid norm 1.327524813262e+17 ||r(i)||/||b|| 7.559942117023e+14 79 KSP preconditioned resid norm 2.275211715418e+03 true resid norm 1.165811443906e+17 ||r(i)||/||b|| 6.639022447827e+14 80 KSP preconditioned resid norm 2.161471491933e+03 true resid norm 1.015723095329e+17 ||r(i)||/||b|| 5.784304542479e+14 81 KSP preconditioned resid norm 1.982790704653e+03 true resid norm 8.141089228178e+16 ||r(i)||/||b|| 4.636159167775e+14 82 KSP preconditioned resid norm 1.740676429272e+03 true resid norm 5.840122411450e+16 ||r(i)||/||b|| 3.325812590907e+14 83 KSP preconditioned resid norm 1.587150103736e+03 true resid norm 4.629492757460e+16 ||r(i)||/||b|| 2.636387427785e+14 84 KSP preconditioned resid norm 1.322911470027e+03 true resid norm 2.928137570587e+16 ||r(i)||/||b|| 1.667505595614e+14 85 KSP preconditioned resid norm 1.181708293679e+03 true resid norm 2.197349341693e+16 ||r(i)||/||b|| 1.251338857709e+14 86 KSP preconditioned resid norm 9.902319658992e+02 true resid norm 1.423216419310e+16 ||r(i)||/||b|| 8.104883345673e+13 87 KSP preconditioned resid norm 8.359003989923e+02 true resid norm 9.303072097139e+15 ||r(i)||/||b|| 5.297881129018e+13 88 KSP preconditioned resid norm 6.981353832080e+02 true resid norm 5.935654408160e+15 ||r(i)||/||b|| 3.380215820001e+13 89 KSP preconditioned resid norm 5.800715473848e+02 true resid norm 3.708718754357e+15 ||r(i)||/||b|| 2.112028252214e+13 90 KSP preconditioned resid norm 4.883814959546e+02 true resid norm 2.388206782439e+15 ||r(i)||/||b|| 1.360027689001e+13 91 KSP preconditioned resid norm 4.789803948167e+02 true resid norm 2.397467672188e+15 ||r(i)||/||b|| 1.365301548272e+13 92 KSP preconditioned resid norm 4.571508437042e+02 true resid norm 2.421108269234e+15 ||r(i)||/||b|| 1.378764313224e+13 93 KSP preconditioned resid norm 4.504410856546e+02 true resid norm 2.430412620122e+15 ||r(i)||/||b|| 1.384062922594e+13 94 KSP preconditioned resid norm 4.405021402898e+02 true resid norm 2.456969922842e+15 ||r(i)||/||b|| 1.399186682945e+13 95 KSP preconditioned resid norm 4.293421489705e+02 true resid norm 2.491683261285e+15 ||r(i)||/||b|| 1.418955114141e+13 96 KSP preconditioned resid norm 4.173639470445e+02 true resid norm 2.540093934790e+15 ||r(i)||/||b|| 1.446523856049e+13 97 KSP preconditioned resid norm 4.071406469759e+02 true resid norm 2.585692419976e+15 ||r(i)||/||b|| 1.472491122739e+13 98 KSP preconditioned resid norm 3.923770532776e+02 true resid norm 2.664480924262e+15 ||r(i)||/||b|| 1.517359326025e+13 99 KSP preconditioned resid norm 3.696685915067e+02 true resid norm 2.790356275911e+15 ||r(i)||/||b|| 1.589042383315e+13 100 KSP preconditioned resid norm 3.522302502819e+02 true resid norm 2.891433713993e+15 ||r(i)||/||b|| 1.646603611067e+13 101 KSP preconditioned resid norm 3.289451342229e+02 true resid norm 3.043740140594e+15 ||r(i)||/||b|| 1.733338544958e+13 102 KSP preconditioned resid norm 3.041026415553e+02 true resid norm 3.223224053568e+15 ||r(i)||/||b|| 1.835550419226e+13 103 KSP preconditioned resid norm 2.721797724150e+02 true resid norm 3.434699181372e+15 ||r(i)||/||b|| 1.955980539207e+13 104 KSP preconditioned resid norm 2.495064983369e+02 true resid norm 3.595571452999e+15 ||r(i)||/||b|| 2.047593520719e+13 105 KSP preconditioned resid norm 2.226326062999e+02 true resid norm 3.801867767252e+15 ||r(i)||/||b|| 2.165074427978e+13 106 KSP preconditioned resid norm 2.073865351924e+02 true resid norm 3.926005822571e+15 ||r(i)||/||b|| 2.235768135798e+13 107 KSP preconditioned resid norm 1.889509603881e+02 true resid norm 4.115951898164e+15 ||r(i)||/||b|| 2.343937966033e+13 108 KSP preconditioned resid norm 1.749949839579e+02 true resid norm 4.278800041159e+15 ||r(i)||/||b|| 2.436676159897e+13 109 KSP preconditioned resid norm 1.618193207581e+02 true resid norm 4.465831426392e+15 ||r(i)||/||b|| 2.543186142407e+13 110 KSP preconditioned resid norm 1.465181110970e+02 true resid norm 4.714655861628e+15 ||r(i)||/||b|| 2.684885816033e+13 111 KSP preconditioned resid norm 1.318107852393e+02 true resid norm 5.037611722496e+15 ||r(i)||/||b|| 2.868801595995e+13 112 KSP preconditioned resid norm 1.242084503143e+02 true resid norm 5.234477191653e+15 ||r(i)||/||b|| 2.980911858403e+13 113 KSP preconditioned resid norm 1.155780621405e+02 true resid norm 5.582223035717e+15 ||r(i)||/||b|| 3.178944951743e+13 114 KSP preconditioned resid norm 1.021440269876e+02 true resid norm 6.169174742477e+15 ||r(i)||/||b|| 3.513200167484e+13 115 KSP preconditioned resid norm 9.495516434823e+01 true resid norm 6.557142244926e+15 ||r(i)||/||b|| 3.734138550895e+13 116 KSP preconditioned resid norm 8.974660093572e+01 true resid norm 6.966234194713e+15 ||r(i)||/||b|| 3.967106811076e+13 117 KSP preconditioned resid norm 8.668620724647e+01 true resid norm 7.370496290054e+15 ||r(i)||/||b|| 4.197324582552e+13 118 KSP preconditioned resid norm 8.544499517816e+01 true resid norm 7.721073757102e+15 ||r(i)||/||b|| 4.396970218697e+13 119 KSP preconditioned resid norm 8.540356800271e+01 true resid norm 7.783986645065e+15 ||r(i)||/||b|| 4.432797631237e+13 120 KSP preconditioned resid norm 8.509182595205e+01 true resid norm 7.567149827193e+15 ||r(i)||/||b|| 4.309314154651e+13 121 KSP preconditioned resid norm 8.501310458315e+01 true resid norm 7.594939989774e+15 ||r(i)||/||b|| 4.325139999746e+13 122 KSP preconditioned resid norm 8.493474592259e+01 true resid norm 7.626607729139e+15 ||r(i)||/||b|| 4.343174033775e+13 123 KSP preconditioned resid norm 8.488333505978e+01 true resid norm 7.656182047437e+15 ||r(i)||/||b|| 4.360015913659e+13 124 KSP preconditioned resid norm 8.487226334182e+01 true resid norm 7.669732176286e+15 ||r(i)||/||b|| 4.367732393890e+13 125 KSP preconditioned resid norm 8.487223132681e+01 true resid norm 7.668341686089e+15 ||r(i)||/||b|| 4.366940542371e+13 126 KSP preconditioned resid norm 8.485866275463e+01 true resid norm 7.639207136251e+15 ||r(i)||/||b|| 4.350349100299e+13 127 KSP preconditioned resid norm 8.480918903522e+01 true resid norm 7.584336028944e+15 ||r(i)||/||b|| 4.319101293027e+13 128 KSP preconditioned resid norm 8.461824044155e+01 true resid norm 7.465571292783e+15 ||r(i)||/||b|| 4.251467564305e+13 129 KSP preconditioned resid norm 8.425430254488e+01 true resid norm 7.263958636771e+15 ||r(i)||/||b|| 4.136653890445e+13 130 KSP preconditioned resid norm 8.380956078491e+01 true resid norm 7.064381616384e+15 ||r(i)||/||b|| 4.022999463278e+13 131 KSP preconditioned resid norm 8.314252078293e+01 true resid norm 6.808447695490e+15 ||r(i)||/||b|| 3.877251104497e+13 132 KSP preconditioned resid norm 8.226839163275e+01 true resid norm 6.516418948240e+15 ||r(i)||/||b|| 3.710947589590e+13 133 KSP preconditioned resid norm 8.078979388845e+01 true resid norm 6.095155817867e+15 ||r(i)||/||b|| 3.471048127837e+13 134 KSP preconditioned resid norm 7.933264183989e+01 true resid norm 5.736990178593e+15 ||r(i)||/||b|| 3.267081205775e+13 135 KSP preconditioned resid norm 7.629870771673e+01 true resid norm 5.098475075145e+15 ||r(i)||/||b|| 2.903461846295e+13 136 KSP preconditioned resid norm 7.384550440347e+01 true resid norm 4.620263540645e+15 ||r(i)||/||b|| 2.631131605505e+13 137 KSP preconditioned resid norm 7.019186089415e+01 true resid norm 3.995768085741e+15 ||r(i)||/||b|| 2.275496106699e+13 138 KSP preconditioned resid norm 6.549021211843e+01 true resid norm 3.271622423957e+15 ||r(i)||/||b|| 1.863112154799e+13 139 KSP preconditioned resid norm 6.216358320616e+01 true resid norm 2.825089716686e+15 ||r(i)||/||b|| 1.608822262317e+13 140 KSP preconditioned resid norm 5.819590212463e+01 true resid norm 2.365235629981e+15 ||r(i)||/||b|| 1.346946157024e+13 141 KSP preconditioned resid norm 5.322146455266e+01 true resid norm 1.862541523452e+15 ||r(i)||/||b|| 1.060673666298e+13 142 KSP preconditioned resid norm 4.838864075967e+01 true resid norm 1.460907869921e+15 ||r(i)||/||b|| 8.319527307186e+12 143 KSP preconditioned resid norm 4.288901894843e+01 true resid norm 1.075225948117e+15 ||r(i)||/||b|| 6.123159318213e+12 144 KSP preconditioned resid norm 3.774714223463e+01 true resid norm 7.813012561145e+14 ||r(i)||/||b|| 4.449327208934e+12 145 KSP preconditioned resid norm 3.210844047850e+01 true resid norm 5.182912787837e+14 ||r(i)||/||b|| 2.951547141129e+12 146 KSP preconditioned resid norm 2.633903242416e+01 true resid norm 3.154831961047e+14 ||r(i)||/||b|| 1.796602728339e+12 147 KSP preconditioned resid norm 2.195326889592e+01 true resid norm 1.974702121211e+14 ||r(i)||/||b|| 1.124546493262e+12 148 KSP preconditioned resid norm 1.932058732465e+01 true resid norm 1.401754968147e+14 ||r(i)||/||b|| 7.982665420318e+11 149 KSP preconditioned resid norm 1.720408947068e+01 true resid norm 1.028932390640e+14 ||r(i)||/||b|| 5.859528377818e+11 150 KSP preconditioned resid norm 1.559613054012e+01 true resid norm 8.129962713198e+13 ||r(i)||/||b|| 4.629822878737e+11 151 KSP preconditioned resid norm 1.528043547944e+01 true resid norm 8.168491802735e+13 ||r(i)||/||b|| 4.651764290589e+11 152 KSP preconditioned resid norm 1.473379333578e+01 true resid norm 8.241475904895e+13 ||r(i)||/||b|| 4.693327023149e+11 153 KSP preconditioned resid norm 1.446555352901e+01 true resid norm 8.283310540263e+13 ||r(i)||/||b|| 4.717150865739e+11 154 KSP preconditioned resid norm 1.433069800411e+01 true resid norm 8.320637495219e+13 ||r(i)||/||b|| 4.738407690174e+11 155 KSP preconditioned resid norm 1.398451209809e+01 true resid norm 8.447596474853e+13 ||r(i)||/||b|| 4.810707848158e+11 156 KSP preconditioned resid norm 1.338238396578e+01 true resid norm 8.694940793013e+13 ||r(i)||/||b|| 4.951564629860e+11 157 KSP preconditioned resid norm 1.285596711425e+01 true resid norm 8.942604823864e+13 ||r(i)||/||b|| 5.092603480434e+11 158 KSP preconditioned resid norm 1.259640711755e+01 true resid norm 9.096288627685e+13 ||r(i)||/||b|| 5.180122798311e+11 159 KSP preconditioned resid norm 1.150402739583e+01 true resid norm 9.770720834982e+13 ||r(i)||/||b|| 5.564196105120e+11 160 KSP preconditioned resid norm 1.091254034929e+01 true resid norm 1.013341675049e+14 ||r(i)||/||b|| 5.770742913128e+11 161 KSP preconditioned resid norm 1.005189915540e+01 true resid norm 1.071270712788e+14 ||r(i)||/||b|| 6.100635181677e+11 162 KSP preconditioned resid norm 9.249803512226e+00 true resid norm 1.126348443433e+14 ||r(i)||/||b|| 6.414289925795e+11 163 KSP preconditioned resid norm 8.498257694031e+00 true resid norm 1.184210190463e+14 ||r(i)||/||b|| 6.743798989555e+11 164 KSP preconditioned resid norm 7.683744736255e+00 true resid norm 1.250524243614e+14 ||r(i)||/||b|| 7.121441952129e+11 165 KSP preconditioned resid norm 7.203608048105e+00 true resid norm 1.292064833777e+14 ||r(i)||/||b|| 7.358005859637e+11 166 KSP preconditioned resid norm 6.554687010295e+00 true resid norm 1.355011219866e+14 ||r(i)||/||b|| 7.716470748995e+11 167 KSP preconditioned resid norm 6.040625671715e+00 true resid norm 1.410686541010e+14 ||r(i)||/||b|| 8.033528630691e+11 168 KSP preconditioned resid norm 5.602777727500e+00 true resid norm 1.472338492425e+14 ||r(i)||/||b|| 8.384622018512e+11 169 KSP preconditioned resid norm 5.137313560334e+00 true resid norm 1.546289114528e+14 ||r(i)||/||b|| 8.805753448241e+11 170 KSP preconditioned resid norm 4.772073752601e+00 true resid norm 1.617511854452e+14 ||r(i)||/||b|| 9.211350229458e+11 171 KSP preconditioned resid norm 4.429552108805e+00 true resid norm 1.699874815352e+14 ||r(i)||/||b|| 9.680387953478e+11 172 KSP preconditioned resid norm 4.058042066487e+00 true resid norm 1.807916301249e+14 ||r(i)||/||b|| 1.029565884819e+12 173 KSP preconditioned resid norm 3.659352883143e+00 true resid norm 1.979586476733e+14 ||r(i)||/||b|| 1.127328019050e+12 174 KSP preconditioned resid norm 3.345836787538e+00 true resid norm 2.142618164480e+14 ||r(i)||/||b|| 1.220170737340e+12 175 KSP preconditioned resid norm 3.144285119620e+00 true resid norm 2.281978037714e+14 ||r(i)||/||b|| 1.299532913064e+12 176 KSP preconditioned resid norm 3.022396672033e+00 true resid norm 2.394375521335e+14 ||r(i)||/||b|| 1.363540640964e+12 177 KSP preconditioned resid norm 2.937051784737e+00 true resid norm 2.511984768128e+14 ||r(i)||/||b|| 1.430516345621e+12 178 KSP preconditioned resid norm 2.894844006212e+00 true resid norm 2.630994673035e+14 ||r(i)||/||b|| 1.498289692188e+12 179 KSP preconditioned resid norm 2.891859803054e+00 true resid norm 2.664328909271e+14 ||r(i)||/||b|| 1.517272757057e+12 180 KSP preconditioned resid norm 2.881379220023e+00 true resid norm 2.599861276412e+14 ||r(i)||/||b|| 1.480559953803e+12 181 KSP preconditioned resid norm 2.879542844584e+00 true resid norm 2.605906188316e+14 ||r(i)||/||b|| 1.484002389202e+12 182 KSP preconditioned resid norm 2.877332104952e+00 true resid norm 2.614134991883e+14 ||r(i)||/||b|| 1.488688499626e+12 183 KSP preconditioned resid norm 2.874457453304e+00 true resid norm 2.627920384570e+14 ||r(i)||/||b|| 1.496538957089e+12 184 KSP preconditioned resid norm 2.874350178426e+00 true resid norm 2.629889841321e+14 ||r(i)||/||b|| 1.497660516467e+12 185 KSP preconditioned resid norm 2.874316759497e+00 true resid norm 2.632240907437e+14 ||r(i)||/||b|| 1.498999393418e+12 186 KSP preconditioned resid norm 2.873427310817e+00 true resid norm 2.616631062224e+14 ||r(i)||/||b|| 1.490109953079e+12 187 KSP preconditioned resid norm 2.871829724110e+00 true resid norm 2.600479248375e+14 ||r(i)||/||b|| 1.480911874327e+12 188 KSP preconditioned resid norm 2.866312069964e+00 true resid norm 2.569200516311e+14 ||r(i)||/||b|| 1.463099370821e+12 189 KSP preconditioned resid norm 2.846491732666e+00 true resid norm 2.470738838019e+14 ||r(i)||/||b|| 1.407027756852e+12 190 KSP preconditioned resid norm 2.819731886872e+00 true resid norm 2.362847839597e+14 ||r(i)||/||b|| 1.345586366464e+12 191 KSP preconditioned resid norm 2.792936766447e+00 true resid norm 2.273872960182e+14 ||r(i)||/||b|| 1.294917261711e+12 192 KSP preconditioned resid norm 2.756926455345e+00 true resid norm 2.167291058783e+14 ||r(i)||/||b|| 1.234221371341e+12 193 KSP preconditioned resid norm 2.712360422860e+00 true resid norm 2.044302781008e+14 ||r(i)||/||b|| 1.164182434836e+12 194 KSP preconditioned resid norm 2.634448843059e+00 true resid norm 1.861635296282e+14 ||r(i)||/||b|| 1.060157591203e+12 195 KSP preconditioned resid norm 2.554394059240e+00 true resid norm 1.695438918584e+14 ||r(i)||/||b|| 9.655126562901e+11 196 KSP preconditioned resid norm 2.423825539214e+00 true resid norm 1.453837282179e+14 ||r(i)||/||b|| 8.279261970125e+11 197 KSP preconditioned resid norm 2.301147456622e+00 true resid norm 1.254692423355e+14 ||r(i)||/||b|| 7.145178757091e+11 198 KSP preconditioned resid norm 2.127717296192e+00 true resid norm 1.001850107010e+14 ||r(i)||/||b|| 5.705301131295e+11 199 KSP preconditioned resid norm 2.009570889375e+00 true resid norm 8.505087899865e+13 ||r(i)||/||b|| 4.843447864840e+11 200 KSP preconditioned resid norm 1.877929467118e+00 true resid norm 7.067975283976e+13 ||r(i)||/||b|| 4.025045972594e+11 201 KSP preconditioned resid norm 1.742306645778e+00 true resid norm 5.778295583468e+13 ||r(i)||/||b|| 3.290603664026e+11 202 KSP preconditioned resid norm 1.552694773463e+00 true resid norm 4.295283799072e+13 ||r(i)||/||b|| 2.446063273000e+11 203 KSP preconditioned resid norm 1.405131270730e+00 true resid norm 3.341312001580e+13 ||r(i)||/||b|| 1.902798732988e+11 204 KSP preconditioned resid norm 1.207664193636e+00 true resid norm 2.297635104050e+13 ||r(i)||/||b|| 1.308449244724e+11 205 KSP preconditioned resid norm 1.053318237111e+00 true resid norm 1.622815103212e+13 ||r(i)||/||b|| 9.241550985974e+10 206 KSP preconditioned resid norm 8.888845765981e-01 true resid norm 1.056122171702e+13 ||r(i)||/||b|| 6.014367796979e+10 207 KSP preconditioned resid norm 7.509568465149e-01 true resid norm 6.806276967032e+12 ||r(i)||/||b|| 3.876014925607e+10 208 KSP preconditioned resid norm 6.625686251463e-01 true resid norm 4.847396589685e+12 ||r(i)||/||b|| 2.760478543991e+10 209 KSP preconditioned resid norm 5.988642618667e-01 true resid norm 3.692436407290e+12 ||r(i)||/||b|| 2.102755837859e+10 210 KSP preconditioned resid norm 5.549023555358e-01 true resid norm 3.047380948800e+12 ||r(i)||/||b|| 1.735411899747e+10 211 KSP preconditioned resid norm 5.455066689212e-01 true resid norm 3.068915702741e+12 ||r(i)||/||b|| 1.747675436493e+10 212 KSP preconditioned resid norm 5.256037924156e-01 true resid norm 3.117149021671e+12 ||r(i)||/||b|| 1.775143179136e+10 213 KSP preconditioned resid norm 5.188512405250e-01 true resid norm 3.135845749577e+12 ||r(i)||/||b|| 1.785790526691e+10 214 KSP preconditioned resid norm 5.064661544592e-01 true resid norm 3.177032622850e+12 ||r(i)||/||b|| 1.809245483978e+10 215 KSP preconditioned resid norm 4.942966107592e-01 true resid norm 3.227527663801e+12 ||r(i)||/||b|| 1.838001224208e+10 216 KSP preconditioned resid norm 4.748832777350e-01 true resid norm 3.319202455933e+12 ||r(i)||/||b|| 1.890207865861e+10 217 KSP preconditioned resid norm 4.515766007515e-01 true resid norm 3.439082377839e+12 ||r(i)||/||b|| 1.958476666681e+10 218 KSP preconditioned resid norm 4.373636013530e-01 true resid norm 3.528935103695e+12 ||r(i)||/||b|| 2.009645684370e+10 219 KSP preconditioned resid norm 4.009409058224e-01 true resid norm 3.761071581232e+12 ||r(i)||/||b|| 2.141841957908e+10 220 KSP preconditioned resid norm 3.740642218035e-01 true resid norm 3.936786773390e+12 ||r(i)||/||b|| 2.241907634160e+10 221 KSP preconditioned resid norm 3.517621550751e-01 true resid norm 4.097034865102e+12 ||r(i)||/||b|| 2.333165159865e+10 222 KSP preconditioned resid norm 3.229172571887e-01 true resid norm 4.313515792451e+12 ||r(i)||/||b|| 2.456445965153e+10 223 KSP preconditioned resid norm 2.994652885566e-01 true resid norm 4.502575429318e+12 ||r(i)||/||b|| 2.564110989347e+10 224 KSP preconditioned resid norm 2.773847957709e-01 true resid norm 4.710010351261e+12 ||r(i)||/||b|| 2.682240306952e+10 225 KSP preconditioned resid norm 2.501295155918e-01 true resid norm 4.969594294575e+12 ||r(i)||/||b|| 2.830067267801e+10 226 KSP preconditioned resid norm 2.304712314567e-01 true resid norm 5.185438809988e+12 ||r(i)||/||b|| 2.952985651435e+10 227 KSP preconditioned resid norm 2.100779846797e-01 true resid norm 5.450176316298e+12 ||r(i)||/||b|| 3.103747445408e+10 228 KSP preconditioned resid norm 1.902335220167e-01 true resid norm 5.764165785042e+12 ||r(i)||/||b|| 3.282557075581e+10 229 KSP preconditioned resid norm 1.696094706888e-01 true resid norm 6.179726816553e+12 ||r(i)||/||b|| 3.519209325915e+10 230 KSP preconditioned resid norm 1.581599504430e-01 true resid norm 6.486811193342e+12 ||r(i)||/||b|| 3.694086668347e+10 231 KSP preconditioned resid norm 1.508622285730e-01 true resid norm 6.725920163898e+12 ||r(i)||/||b|| 3.830253612950e+10 232 KSP preconditioned resid norm 1.423061981474e-01 true resid norm 7.059553066573e+12 ||r(i)||/||b|| 4.020249717532e+10 233 KSP preconditioned resid norm 1.315343744406e-01 true resid norm 7.576207381506e+12 ||r(i)||/||b|| 4.314472219166e+10 234 KSP preconditioned resid norm 1.237631119715e-01 true resid norm 8.062833402738e+12 ||r(i)||/||b|| 4.591594312584e+10 235 KSP preconditioned resid norm 1.167455701157e-01 true resid norm 8.626377159952e+12 ||r(i)||/||b|| 4.912519250663e+10 236 KSP preconditioned resid norm 1.118150542335e-01 true resid norm 9.193240713930e+12 ||r(i)||/||b|| 5.235334735053e+10 237 KSP preconditioned resid norm 1.093203112720e-01 true resid norm 9.616801357038e+12 ||r(i)||/||b|| 5.476542576364e+10 238 KSP preconditioned resid norm 1.083571577732e-01 true resid norm 9.931584526963e+12 ||r(i)||/||b|| 5.655804200725e+10 239 KSP preconditioned resid norm 1.082844361418e-01 true resid norm 1.003794171129e+13 ||r(i)||/||b|| 5.716372119998e+10 240 KSP preconditioned resid norm 1.078315530435e-01 true resid norm 9.849635902358e+12 ||r(i)||/||b|| 5.609136383115e+10 241 KSP preconditioned resid norm 1.077293716089e-01 true resid norm 9.870905184873e+12 ||r(i)||/||b|| 5.621248740118e+10 242 KSP preconditioned resid norm 1.075375538902e-01 true resid norm 9.914548970249e+12 ||r(i)||/||b|| 5.646102851161e+10 243 KSP preconditioned resid norm 1.074753898786e-01 true resid norm 9.944199906958e+12 ||r(i)||/||b|| 5.662988363431e+10 244 KSP preconditioned resid norm 1.074753897188e-01 true resid norm 9.944120931810e+12 ||r(i)||/||b|| 5.662943388939e+10 245 KSP preconditioned resid norm 1.074692455282e-01 true resid norm 9.924468762730e+12 ||r(i)||/||b|| 5.651751940068e+10 246 KSP preconditioned resid norm 1.073634021511e-01 true resid norm 9.811991743004e+12 ||r(i)||/||b|| 5.587698918224e+10 247 KSP preconditioned resid norm 1.072871763676e-01 true resid norm 9.751064980653e+12 ||r(i)||/||b|| 5.553002557588e+10 248 KSP preconditioned resid norm 1.067011749299e-01 true resid norm 9.474130637214e+12 ||r(i)||/||b|| 5.395294951244e+10 249 KSP preconditioned resid norm 1.055795514542e-01 true resid norm 9.010869435560e+12 ||r(i)||/||b|| 5.131478573984e+10 250 KSP preconditioned resid norm 1.047264906315e-01 true resid norm 8.722590914578e+12 ||r(i)||/||b|| 4.967310724884e+10 251 KSP preconditioned resid norm 1.029468452861e-01 true resid norm 8.186051408096e+12 ||r(i)||/||b|| 4.661764073554e+10 252 KSP preconditioned resid norm 1.004306739339e-01 true resid norm 7.495653525966e+12 ||r(i)||/||b|| 4.268598689791e+10 253 KSP preconditioned resid norm 9.795465882615e-02 true resid norm 6.920294332192e+12 ||r(i)||/||b|| 3.940945138009e+10 254 KSP preconditioned resid norm 9.555297895830e-02 true resid norm 6.409331737743e+12 ||r(i)||/||b|| 3.649963937552e+10 255 KSP preconditioned resid norm 9.212236801156e-02 true resid norm 5.754520651725e+12 ||r(i)||/||b|| 3.277064398619e+10 256 KSP preconditioned resid norm 8.758716129780e-02 true resid norm 4.960284730854e+12 ||r(i)||/||b|| 2.824765689845e+10 257 KSP preconditioned resid norm 8.129614388311e-02 true resid norm 4.021897523341e+12 ||r(i)||/||b|| 2.290376207910e+10 258 KSP preconditioned resid norm 7.531167948772e-02 true resid norm 3.231798425642e+12 ||r(i)||/||b|| 1.840433322801e+10 259 KSP preconditioned resid norm 7.147274373805e-02 true resid norm 2.780272287343e+12 ||r(i)||/||b|| 1.583299788591e+10 260 KSP preconditioned resid norm 6.629497651684e-02 true resid norm 2.268106077708e+12 ||r(i)||/||b|| 1.291633157545e+10 261 KSP preconditioned resid norm 6.208848335548e-02 true resid norm 1.905343450029e+12 ||r(i)||/||b|| 1.085048358521e+10 262 KSP preconditioned resid norm 5.627463420064e-02 true resid norm 1.483147396155e+12 ||r(i)||/||b|| 8.446176187388e+09 263 KSP preconditioned resid norm 4.990060996976e-02 true resid norm 1.096560027659e+12 ||r(i)||/||b|| 6.244651892097e+09 264 KSP preconditioned resid norm 4.497149523606e-02 true resid norm 8.467973504088e+11 ||r(i)||/||b|| 4.822312087867e+09 265 KSP preconditioned resid norm 3.747771448386e-02 true resid norm 5.348773301719e+11 ||r(i)||/||b|| 3.046000809484e+09 266 KSP preconditioned resid norm 3.272493579045e-02 true resid norm 3.761018286666e+11 ||r(i)||/||b|| 2.141811607904e+09 267 KSP preconditioned resid norm 2.833957133035e-02 true resid norm 2.584003424653e+11 ||r(i)||/||b|| 1.471529279559e+09 268 KSP preconditioned resid norm 2.427356932428e-02 true resid norm 1.695160658437e+11 ||r(i)||/||b|| 9.653541936699e+08 269 KSP preconditioned resid norm 2.223266993764e-02 true resid norm 1.346276265138e+11 ||r(i)||/||b|| 7.666727232729e+08 270 KSP preconditioned resid norm 2.051295333624e-02 true resid norm 1.110746629442e+11 ||r(i)||/||b|| 6.325441258323e+08 271 KSP preconditioned resid norm 2.015855323507e-02 true resid norm 1.116965466597e+11 ||r(i)||/||b|| 6.360856075776e+08 272 KSP preconditioned resid norm 1.928852034379e-02 true resid norm 1.133139863240e+11 ||r(i)||/||b|| 6.452965467012e+08 273 KSP preconditioned resid norm 1.901631826643e-02 true resid norm 1.139199965486e+11 ||r(i)||/||b|| 6.487476326432e+08 274 KSP preconditioned resid norm 1.867363596983e-02 true resid norm 1.148727414952e+11 ||r(i)||/||b|| 6.541732914152e+08 275 KSP preconditioned resid norm 1.820280164400e-02 true resid norm 1.164715762055e+11 ||r(i)||/||b|| 6.632782796940e+08 276 KSP preconditioned resid norm 1.749874598398e-02 true resid norm 1.194548458241e+11 ||r(i)||/||b|| 6.802672997192e+08 277 KSP preconditioned resid norm 1.687372343702e-02 true resid norm 1.224937051098e+11 ||r(i)||/||b|| 6.975728898463e+08 278 KSP preconditioned resid norm 1.639581534582e-02 true resid norm 1.252604851014e+11 ||r(i)||/||b|| 7.133290522755e+08 279 KSP preconditioned resid norm 1.496517609431e-02 true resid norm 1.337999448784e+11 ||r(i)||/||b|| 7.619592706937e+08 280 KSP preconditioned resid norm 1.418035417318e-02 true resid norm 1.386300317614e+11 ||r(i)||/||b|| 7.894654814188e+08 281 KSP preconditioned resid norm 1.331745056385e-02 true resid norm 1.444341207454e+11 ||r(i)||/||b|| 8.225184054188e+08 282 KSP preconditioned resid norm 1.222655360483e-02 true resid norm 1.521352895752e+11 ||r(i)||/||b|| 8.663747537182e+08 283 KSP preconditioned resid norm 1.131430015656e-02 true resid norm 1.595358392621e+11 ||r(i)||/||b|| 9.085191465823e+08 284 KSP preconditioned resid norm 1.037045062652e-02 true resid norm 1.680266580128e+11 ||r(i)||/||b|| 9.568723657767e+08 285 KSP preconditioned resid norm 9.688191710241e-03 true resid norm 1.743229906865e+11 ||r(i)||/||b|| 9.927285020137e+08 286 KSP preconditioned resid norm 9.052121728457e-03 true resid norm 1.811489298466e+11 ||r(i)||/||b|| 1.031600622843e+09 287 KSP preconditioned resid norm 8.110693591993e-03 true resid norm 1.926557318145e+11 ||r(i)||/||b|| 1.097129158325e+09 288 KSP preconditioned resid norm 7.508148910847e-03 true resid norm 2.010760971618e+11 ||r(i)||/||b|| 1.145081161929e+09 289 KSP preconditioned resid norm 6.917244373311e-03 true resid norm 2.109959323352e+11 ||r(i)||/||b|| 1.201572294126e+09 290 KSP preconditioned resid norm 6.270817241633e-03 true resid norm 2.235532624417e+11 ||r(i)||/||b|| 1.273083340701e+09 291 KSP preconditioned resid norm 5.723459551474e-03 true resid norm 2.366863981842e+11 ||r(i)||/||b|| 1.347873465176e+09 292 KSP preconditioned resid norm 5.384076682559e-03 true resid norm 2.476826665094e+11 ||r(i)||/||b|| 1.410494631433e+09 293 KSP preconditioned resid norm 5.001871189708e-03 true resid norm 2.646732099310e+11 ||r(i)||/||b|| 1.507251786946e+09 294 KSP preconditioned resid norm 4.705091159541e-03 true resid norm 2.809592007224e+11 ||r(i)||/||b|| 1.599996680654e+09 295 KSP preconditioned resid norm 4.424608317205e-03 true resid norm 3.023640626614e+11 ||r(i)||/||b|| 1.721892343670e+09 296 KSP preconditioned resid norm 4.204818967276e-03 true resid norm 3.239124009068e+11 ||r(i)||/||b|| 1.844605070563e+09 297 KSP preconditioned resid norm 4.020995274001e-03 true resid norm 3.491082273440e+11 ||r(i)||/||b|| 1.988089386300e+09 298 KSP preconditioned resid norm 3.962926420244e-03 true resid norm 3.653307295648e+11 ||r(i)||/||b|| 2.080472727506e+09 299 KSP preconditioned resid norm 3.957390293488e-03 true resid norm 3.700702150773e+11 ||r(i)||/||b|| 2.107462984698e+09 300 KSP preconditioned resid norm 3.939107198479e-03 true resid norm 3.622470106797e+11 ||r(i)||/||b|| 2.062911672493e+09 301 KSP preconditioned resid norm 3.936151280373e-03 true resid norm 3.628693131542e+11 ||r(i)||/||b|| 2.066455538973e+09 302 KSP preconditioned resid norm 3.931466595581e-03 true resid norm 3.639592089146e+11 ||r(i)||/||b|| 2.072662239428e+09 303 KSP preconditioned resid norm 3.926435182664e-03 true resid norm 3.655828398113e+11 ||r(i)||/||b|| 2.081908436166e+09 304 KSP preconditioned resid norm 3.926021426467e-03 true resid norm 3.661260862992e+11 ||r(i)||/||b|| 2.085002097364e+09 305 KSP preconditioned resid norm 3.925349557399e-03 true resid norm 3.649007689721e+11 ||r(i)||/||b|| 2.078024202882e+09 306 KSP preconditioned resid norm 3.923358477472e-03 true resid norm 3.626439026392e+11 ||r(i)||/||b|| 2.065171879014e+09 307 KSP preconditioned resid norm 3.917022083572e-03 true resid norm 3.580385367023e+11 ||r(i)||/||b|| 2.038945401315e+09 308 KSP preconditioned resid norm 3.903315312325e-03 true resid norm 3.514826655816e+11 ||r(i)||/||b|| 2.001611254560e+09 309 KSP preconditioned resid norm 3.868431844102e-03 true resid norm 3.361181687167e+11 ||r(i)||/||b|| 1.914114052402e+09 310 KSP preconditioned resid norm 3.830529010142e-03 true resid norm 3.221666140909e+11 ||r(i)||/||b|| 1.834663224546e+09 311 KSP preconditioned resid norm 3.796432004576e-03 true resid norm 3.115935002888e+11 ||r(i)||/||b|| 1.774451823943e+09 312 KSP preconditioned resid norm 3.690322145629e-03 true resid norm 2.821316991888e+11 ||r(i)||/||b|| 1.606673784125e+09 313 KSP preconditioned resid norm 3.593417830510e-03 true resid norm 2.581298804645e+11 ||r(i)||/||b|| 1.469989061967e+09 314 KSP preconditioned resid norm 3.480230295411e-03 true resid norm 2.333539117608e+11 ||r(i)||/||b|| 1.328895737442e+09 315 KSP preconditioned resid norm 3.353622106193e-03 true resid norm 2.081556998192e+11 ||r(i)||/||b|| 1.185397836817e+09 316 KSP preconditioned resid norm 3.194404159008e-03 true resid norm 1.800136692851e+11 ||r(i)||/||b|| 1.025135580497e+09 317 KSP preconditioned resid norm 3.071654543561e-03 true resid norm 1.608397945341e+11 ||r(i)||/||b|| 9.159448656958e+08 318 KSP preconditioned resid norm 2.839856706992e-03 true resid norm 1.286448140701e+11 ||r(i)||/||b|| 7.326020111329e+08 319 KSP preconditioned resid norm 2.631456530872e-03 true resid norm 1.035759604459e+11 ||r(i)||/||b|| 5.898407757529e+08 320 KSP preconditioned resid norm 2.421742413444e-03 true resid norm 8.242488267052e+10 ||r(i)||/||b|| 4.693903539628e+08 321 KSP preconditioned resid norm 2.249880781719e-03 true resid norm 6.797555823014e+10 ||r(i)||/||b|| 3.871048438856e+08 322 KSP preconditioned resid norm 2.051711845259e-03 true resid norm 5.362248662101e+10 ||r(i)||/||b|| 3.053674710829e+08 323 KSP preconditioned resid norm 1.852622242819e-03 true resid norm 4.140287521870e+10 ||r(i)||/||b|| 2.357796532349e+08 324 KSP preconditioned resid norm 1.586793121641e-03 true resid norm 2.820883331841e+10 ||r(i)||/||b|| 1.606426824911e+08 325 KSP preconditioned resid norm 1.397002344352e-03 true resid norm 2.041497331206e+10 ||r(i)||/||b|| 1.162584797044e+08 326 KSP preconditioned resid norm 1.221037138397e-03 true resid norm 1.446611906140e+10 ||r(i)||/||b|| 8.238115150056e+07 327 KSP preconditioned resid norm 1.033473918443e-03 true resid norm 9.359998416615e+09 ||r(i)||/||b|| 5.330299331365e+07 328 KSP preconditioned resid norm 9.116864046740e-04 true resid norm 6.661822582821e+09 ||r(i)||/||b|| 3.793751545497e+07 329 KSP preconditioned resid norm 8.282517125284e-04 true resid norm 5.144655662558e+09 ||r(i)||/||b|| 2.929760606536e+07 330 KSP preconditioned resid norm 7.530584930796e-04 true resid norm 4.075336468774e+09 ||r(i)||/||b|| 2.320808432621e+07 331 KSP preconditioned resid norm 7.379013029880e-04 true resid norm 4.106224221999e+09 ||r(i)||/||b|| 2.338398282857e+07 332 KSP preconditioned resid norm 7.208253294439e-04 true resid norm 4.143001104326e+09 ||r(i)||/||b|| 2.359341853844e+07 333 KSP preconditioned resid norm 7.065478031531e-04 true resid norm 4.177342705301e+09 ||r(i)||/||b|| 2.378898589280e+07 334 KSP preconditioned resid norm 6.922256661224e-04 true resid norm 4.218395844337e+09 ||r(i)||/||b|| 2.402277388059e+07 335 KSP preconditioned resid norm 6.743886958324e-04 true resid norm 4.282558054647e+09 ||r(i)||/||b|| 2.438816260342e+07 336 KSP preconditioned resid norm 6.468482934965e-04 true resid norm 4.403654061697e+09 ||r(i)||/||b|| 2.507777593099e+07 337 KSP preconditioned resid norm 6.112298327485e-04 true resid norm 4.572827652221e+09 ||r(i)||/||b|| 2.604117980812e+07 338 KSP preconditioned resid norm 5.788752885710e-04 true resid norm 4.751394641190e+09 ||r(i)||/||b|| 2.705807688389e+07 339 KSP preconditioned resid norm 5.410834722063e-04 true resid norm 4.973481545718e+09 ||r(i)||/||b|| 2.832280966057e+07 340 KSP preconditioned resid norm 5.010118139868e-04 true resid norm 5.234602366049e+09 ||r(i)||/||b|| 2.980983142282e+07 341 KSP preconditioned resid norm 4.660413373564e-04 true resid norm 5.472737511902e+09 ||r(i)||/||b|| 3.116595516582e+07 342 KSP preconditioned resid norm 4.188717184922e-04 true resid norm 5.804580276171e+09 ||r(i)||/||b|| 3.305572179372e+07 343 KSP preconditioned resid norm 3.873339125732e-04 true resid norm 6.068604268666e+09 ||r(i)||/||b|| 3.455927643981e+07 344 KSP preconditioned resid norm 3.623884936388e-04 true resid norm 6.323395887446e+09 ||r(i)||/||b|| 3.601025488528e+07 345 KSP preconditioned resid norm 3.275102770812e-04 true resid norm 6.705624685656e+09 ||r(i)||/||b|| 3.818695814616e+07 346 KSP preconditioned resid norm 3.059476371251e-04 true resid norm 6.975976972838e+09 ||r(i)||/||b|| 3.972655094464e+07 347 KSP preconditioned resid norm 2.842430817112e-04 true resid norm 7.278309985054e+09 ||r(i)||/||b|| 4.144826646332e+07 348 KSP preconditioned resid norm 2.701042874873e-04 true resid norm 7.506173036350e+09 ||r(i)||/||b|| 4.274589303963e+07 349 KSP preconditioned resid norm 2.489935977407e-04 true resid norm 7.858370531617e+09 ||r(i)||/||b|| 4.475157508141e+07 350 KSP preconditioned resid norm 2.261790623467e-04 true resid norm 8.300057569323e+09 ||r(i)||/||b|| 4.726687905580e+07 351 KSP preconditioned resid norm 2.108197466213e-04 true resid norm 8.660561620706e+09 ||r(i)||/||b|| 4.931986498433e+07 352 KSP preconditioned resid norm 1.952751185189e-04 true resid norm 9.171201474632e+09 ||r(i)||/||b|| 5.222783905740e+07 353 KSP preconditioned resid norm 1.857517778822e-04 true resid norm 9.612041070020e+09 ||r(i)||/||b|| 5.473831704676e+07 354 KSP preconditioned resid norm 1.755607178806e-04 true resid norm 1.022779610633e+10 ||r(i)||/||b|| 5.824489740313e+07 355 KSP preconditioned resid norm 1.629796228709e-04 true resid norm 1.128529235952e+10 ||r(i)||/||b|| 6.426709027154e+07 356 KSP preconditioned resid norm 1.523670690967e-04 true resid norm 1.237182813014e+10 ||r(i)||/||b|| 7.045465637343e+07 357 KSP preconditioned resid norm 1.479074333582e-04 true resid norm 1.303729898576e+10 ||r(i)||/||b|| 7.424435664783e+07 358 KSP preconditioned resid norm 1.460074843662e-04 true resid norm 1.352434774547e+10 ||r(i)||/||b|| 7.701798497833e+07 359 KSP preconditioned resid norm 1.458829647533e-04 true resid norm 1.366121581535e+10 ||r(i)||/||b|| 7.779741650057e+07 360 KSP preconditioned resid norm 1.455003008377e-04 true resid norm 1.345493890124e+10 ||r(i)||/||b|| 7.662271790721e+07 361 KSP preconditioned resid norm 1.453612812516e-04 true resid norm 1.349985427846e+10 ||r(i)||/||b|| 7.687850043463e+07 362 KSP preconditioned resid norm 1.452896929232e-04 true resid norm 1.352954023042e+10 ||r(i)||/||b|| 7.704755496096e+07 363 KSP preconditioned resid norm 1.452045529904e-04 true resid norm 1.358035370495e+10 ||r(i)||/||b|| 7.733692576772e+07 364 KSP preconditioned resid norm 1.452037661685e-04 true resid norm 1.358635691883e+10 ||r(i)||/||b|| 7.737111266129e+07 365 KSP preconditioned resid norm 1.451947097272e-04 true resid norm 1.356254561296e+10 ||r(i)||/||b|| 7.723551286509e+07 366 KSP preconditioned resid norm 1.450283003291e-04 true resid norm 1.341859300371e+10 ||r(i)||/||b|| 7.641573655459e+07 367 KSP preconditioned resid norm 1.448630740135e-04 true resid norm 1.330853779826e+10 ||r(i)||/||b|| 7.578899799982e+07 368 KSP preconditioned resid norm 1.440567965769e-04 true resid norm 1.296196745350e+10 ||r(i)||/||b|| 7.381536125899e+07 369 KSP preconditioned resid norm 1.424149319794e-04 true resid norm 1.234303836654e+10 ||r(i)||/||b|| 7.029070542941e+07 370 KSP preconditioned resid norm 1.405170637750e-04 true resid norm 1.171613546588e+10 ||r(i)||/||b|| 6.672064060300e+07 371 KSP preconditioned resid norm 1.384124482954e-04 true resid norm 1.109270199634e+10 ||r(i)||/||b|| 6.317033337224e+07 372 KSP preconditioned resid norm 1.349411971564e-04 true resid norm 1.017892860063e+10 ||r(i)||/||b|| 5.796660843193e+07 373 KSP preconditioned resid norm 1.298701178596e-04 true resid norm 9.002393037879e+09 ||r(i)||/||b|| 5.126651464526e+07 374 KSP preconditioned resid norm 1.243592082079e-04 true resid norm 7.854364119834e+09 ||r(i)||/||b|| 4.472875950699e+07 375 KSP preconditioned resid norm 1.196434867064e-04 true resid norm 6.952300103787e+09 ||r(i)||/||b|| 3.959171673458e+07 376 KSP preconditioned resid norm 1.150026535868e-04 true resid norm 6.147028883649e+09 ||r(i)||/||b|| 3.500588620853e+07 377 KSP preconditioned resid norm 1.078820003518e-04 true resid norm 5.101678614659e+09 ||r(i)||/||b|| 2.905286186832e+07 378 KSP preconditioned resid norm 1.020452351121e-04 true resid norm 4.355966470087e+09 ||r(i)||/||b|| 2.480620629352e+07 379 KSP preconditioned resid norm 9.619060714772e-05 true resid norm 3.698072276069e+09 ||r(i)||/||b|| 2.105965332802e+07 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 KSP Object: 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-10, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, blocksize = 1, factorization LOWER Preconditioner for the Schur complement formed from S itself Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_1_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_) 4 MPI processes type: lsc KSP Object: (fieldsplit_1_lsc_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 package used to perform factorization: petsc total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=9549, allocated nonzeros=9549 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines linear system matrix = precond matrix: Mat Object: (fieldsplit_1_) 4 MPI processes type: schurcomplement rows=3183, cols=3183 Schur complement A11 - A10 inv(A00) A01 A11 Mat Object: (fieldsplit_1_) 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=3183, allocated nonzeros=3183 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines A10 Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=71946 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines KSP of A00 KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 A01 Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=3183 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 3757 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=75129, cols=75129 total: nonzeros=3.0016e+06, allocated nonzeros=1.26217e+07 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Recovering stress ... Cleaning up ... Finished 0.00user 0.02system 0:26.34elapsed 0%CPU (0avgtext+0avgdata 17072maxresident)k 904inputs+0outputs (3major+1798minor)pagefaults 0swaps salloc: Relinquishing job allocation 79010 -------------- next part -------------- salloc: Granted job allocation 79009 Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 0.000000000000e+00 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.432612442706e-02 1 KSP Residual norm 3.137946166373e-15 1 KSP preconditioned resid norm 9.127510702346e+05 true resid norm 1.362125072634e+14 ||r(i)||/||b|| 7.756982470220e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.663988751024e-01 1 KSP Residual norm 2.456064991848e-14 2 KSP preconditioned resid norm 8.626930780275e+05 true resid norm 7.140706098955e+14 ||r(i)||/||b|| 4.066464464051e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.451385401868e-01 1 KSP Residual norm 2.569197214229e-14 3 KSP preconditioned resid norm 6.917392651646e+05 true resid norm 2.788588220631e+15 ||r(i)||/||b|| 1.588035517345e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.081370774413e-02 1 KSP Residual norm 1.573306002072e-14 4 KSP preconditioned resid norm 5.948180320596e+05 true resid norm 3.828406259238e+15 ||r(i)||/||b|| 2.180187476056e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.380360375235e-01 1 KSP Residual norm 2.643171503486e-14 5 KSP preconditioned resid norm 5.365265868551e+05 true resid norm 4.509015181942e+15 ||r(i)||/||b|| 2.567778277265e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.277633702520e-01 1 KSP Residual norm 2.000627015696e-14 6 KSP preconditioned resid norm 5.020807291688e+05 true resid norm 4.938429989056e+15 ||r(i)||/||b|| 2.812319927527e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.250134450112e-01 1 KSP Residual norm 3.007637391635e-14 7 KSP preconditioned resid norm 4.691107247107e+05 true resid norm 5.504613096315e+15 ||r(i)||/||b|| 3.134747913486e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.969378496446e-02 1 KSP Residual norm 2.097049353738e-14 8 KSP preconditioned resid norm 4.500805942145e+05 true resid norm 5.950219821547e+15 ||r(i)||/||b|| 3.388510480939e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.908705156881e-02 1 KSP Residual norm 1.517652960611e-14 9 KSP preconditioned resid norm 4.250032501743e+05 true resid norm 6.670720166035e+15 ||r(i)||/||b|| 3.798818510228e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.051056574003e-02 1 KSP Residual norm 1.611299744434e-14 10 KSP preconditioned resid norm 4.051645463809e+05 true resid norm 7.414211051143e+15 ||r(i)||/||b|| 4.222219112597e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.908456136614e-02 1 KSP Residual norm 6.643673163588e-15 11 KSP preconditioned resid norm 3.880013874030e+05 true resid norm 8.151575238509e+15 ||r(i)||/||b|| 4.642130704453e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.302530902167e-02 1 KSP Residual norm 1.118497427955e-14 12 KSP preconditioned resid norm 3.741408602795e+05 true resid norm 8.948407390774e+15 ||r(i)||/||b|| 5.095907906048e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.220040527806e-02 1 KSP Residual norm 2.195172558135e-14 13 KSP preconditioned resid norm 3.532148969164e+05 true resid norm 1.028219119134e+16 ||r(i)||/||b|| 5.855466464062e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.664221635474e-02 1 KSP Residual norm 1.095313151883e-14 14 KSP preconditioned resid norm 3.306246816868e+05 true resid norm 1.207912805635e+16 ||r(i)||/||b|| 6.878779817733e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.908823169273e-02 1 KSP Residual norm 4.567677409800e-15 15 KSP preconditioned resid norm 3.077008490958e+05 true resid norm 1.412194821609e+16 ||r(i)||/||b|| 8.042117934570e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.464400632944e-02 1 KSP Residual norm 1.248385796725e-14 16 KSP preconditioned resid norm 2.709436838753e+05 true resid norm 1.756809508461e+16 ||r(i)||/||b|| 1.000461766282e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.730924456396e-02 1 KSP Residual norm 8.833952331029e-15 17 KSP preconditioned resid norm 2.491233804160e+05 true resid norm 1.976045004357e+16 ||r(i)||/||b|| 1.125311233683e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.360294528433e-02 1 KSP Residual norm 8.866990379709e-15 18 KSP preconditioned resid norm 2.169453185967e+05 true resid norm 2.315544602061e+16 ||r(i)||/||b|| 1.318648283338e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.672923469416e-02 1 KSP Residual norm 6.628932206411e-15 19 KSP preconditioned resid norm 1.853345270878e+05 true resid norm 2.647069144539e+16 ||r(i)||/||b|| 1.507443726291e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.439173838058e-02 1 KSP Residual norm 8.710300289138e-15 20 KSP preconditioned resid norm 1.684382143130e+05 true resid norm 2.847996525787e+16 ||r(i)||/||b|| 1.621867152262e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.126040666118e-02 1 KSP Residual norm 2.157718718490e-15 21 KSP preconditioned resid norm 1.487793180702e+05 true resid norm 3.106944054176e+16 ||r(i)||/||b|| 1.769331689754e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.982119365260e-02 1 KSP Residual norm 6.500808717418e-15 22 KSP preconditioned resid norm 1.324108267210e+05 true resid norm 3.364687707286e+16 ||r(i)||/||b|| 1.916110648541e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.423850086514e-02 1 KSP Residual norm 8.994789776599e-15 23 KSP preconditioned resid norm 1.154783368229e+05 true resid norm 3.664744963422e+16 ||r(i)||/||b|| 2.086986210754e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.393990608456e-02 1 KSP Residual norm 2.406129385825e-15 24 KSP preconditioned resid norm 9.690290840659e+04 true resid norm 4.059423491285e+16 ||r(i)||/||b|| 2.311746365567e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.622348172052e-02 1 KSP Residual norm 3.428873840323e-15 25 KSP preconditioned resid norm 8.541667474243e+04 true resid norm 4.382080823706e+16 ||r(i)||/||b|| 2.495492140589e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.075620151853e-02 1 KSP Residual norm 5.698030139326e-15 26 KSP preconditioned resid norm 6.738137389098e+04 true resid norm 4.935828200691e+16 ||r(i)||/||b|| 2.810838270141e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.010803475004e-02 1 KSP Residual norm 2.156132132236e-15 27 KSP preconditioned resid norm 5.559761934926e+04 true resid norm 5.372591255703e+16 ||r(i)||/||b|| 3.059564575047e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.826237158796e-02 1 KSP Residual norm 1.017098814718e-15 28 KSP preconditioned resid norm 4.606357788249e+04 true resid norm 5.783519870719e+16 ||r(i)||/||b|| 3.293578772953e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.651629400227e-02 1 KSP Residual norm 2.722822698494e-15 29 KSP preconditioned resid norm 3.846753020935e+04 true resid norm 6.227329505519e+16 ||r(i)||/||b|| 3.546317939599e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.027548486581e-02 1 KSP Residual norm 3.494353474393e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.263440621161e+02 1 KSP Residual norm 2.976783624382e-11 30 KSP preconditioned resid norm 3.331905232453e+04 true resid norm 6.606924388293e+16 ||r(i)||/||b|| 3.762488312689e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.258485129873e-02 1 KSP Residual norm 1.640294240474e-15 31 KSP preconditioned resid norm 3.290201988669e+04 true resid norm 6.618297473877e+16 ||r(i)||/||b|| 3.768965017896e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.873187189707e-01 1 KSP Residual norm 3.007064345094e-14 32 KSP preconditioned resid norm 3.162870028135e+04 true resid norm 6.654393352155e+16 ||r(i)||/||b|| 3.789520773067e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.581656246293e-01 1 KSP Residual norm 2.053212713329e-14 33 KSP preconditioned resid norm 2.991966449596e+04 true resid norm 6.709255863545e+16 ||r(i)||/||b|| 3.820763685166e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.232182407060e-02 1 KSP Residual norm 9.699695737156e-15 34 KSP preconditioned resid norm 2.922952747341e+04 true resid norm 6.743695964733e+16 ||r(i)||/||b|| 3.840376514161e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.224840558679e-01 1 KSP Residual norm 2.098402045938e-14 35 KSP preconditioned resid norm 2.798614888072e+04 true resid norm 6.838110987781e+16 ||r(i)||/||b|| 3.894143652981e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.112708896553e-01 1 KSP Residual norm 2.324449816880e-14 36 KSP preconditioned resid norm 2.701665220391e+04 true resid norm 6.929642686803e+16 ||r(i)||/||b|| 3.946268806467e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.287500625843e-01 1 KSP Residual norm 2.070816800358e-14 37 KSP preconditioned resid norm 2.601811427966e+04 true resid norm 7.044504608715e+16 ||r(i)||/||b|| 4.011679974110e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.310462940439e-02 1 KSP Residual norm 1.033991542240e-14 38 KSP preconditioned resid norm 2.531056406749e+04 true resid norm 7.153872989067e+16 ||r(i)||/||b|| 4.073962698820e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.049341398055e-02 1 KSP Residual norm 4.340873415037e-15 39 KSP preconditioned resid norm 2.332617529409e+04 true resid norm 7.497512777546e+16 ||r(i)||/||b|| 4.269657489911e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.919505340201e-02 1 KSP Residual norm 4.836385484814e-15 40 KSP preconditioned resid norm 2.254584603894e+04 true resid norm 7.640212636613e+16 ||r(i)||/||b|| 4.350921709147e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.385283041415e-02 1 KSP Residual norm 1.246243453282e-14 41 KSP preconditioned resid norm 2.082879229166e+04 true resid norm 7.986664049992e+16 ||r(i)||/||b|| 4.548217654604e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.962557904950e-02 1 KSP Residual norm 1.180258599058e-14 42 KSP preconditioned resid norm 1.914379086052e+04 true resid norm 8.367676211170e+16 ||r(i)||/||b|| 4.765195134468e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.016394666901e-02 1 KSP Residual norm 9.028629004183e-15 43 KSP preconditioned resid norm 1.815822266994e+04 true resid norm 8.615765210331e+16 ||r(i)||/||b|| 4.906475994516e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.203801613990e-02 1 KSP Residual norm 6.332115504097e-15 44 KSP preconditioned resid norm 1.677428595429e+04 true resid norm 8.993611312333e+16 ||r(i)||/||b|| 5.121650478016e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.522972725508e-02 1 KSP Residual norm 6.698886160069e-15 45 KSP preconditioned resid norm 1.534247135186e+04 true resid norm 9.414497409761e+16 ||r(i)||/||b|| 5.361335228360e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.458547496485e-02 1 KSP Residual norm 4.892131084318e-15 46 KSP preconditioned resid norm 1.412608783199e+04 true resid norm 9.852630241846e+16 ||r(i)||/||b|| 5.610841589148e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.620925013348e-02 1 KSP Residual norm 6.700211558332e-15 47 KSP preconditioned resid norm 1.177668960206e+04 true resid norm 1.077880279666e+17 ||r(i)||/||b|| 6.138275113166e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.177239640098e-02 1 KSP Residual norm 9.278074637413e-15 48 KSP preconditioned resid norm 1.065678908965e+04 true resid norm 1.124882419582e+17 ||r(i)||/||b|| 6.405941264176e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.503343368650e-02 1 KSP Residual norm 1.422030224668e-14 49 KSP preconditioned resid norm 9.962410632557e+03 true resid norm 1.158017305506e+17 ||r(i)||/||b|| 6.594636659650e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.674003101964e-02 1 KSP Residual norm 8.935395883314e-15 50 KSP preconditioned resid norm 9.371192454299e+03 true resid norm 1.195573008344e+17 ||r(i)||/||b|| 6.808507569471e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.760463738960e-02 1 KSP Residual norm 8.072963700989e-15 51 KSP preconditioned resid norm 8.608163058302e+03 true resid norm 1.259685427142e+17 ||r(i)||/||b|| 7.173612741331e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.252122536889e-02 1 KSP Residual norm 4.505661163138e-15 52 KSP preconditioned resid norm 7.720924960576e+03 true resid norm 1.350861356563e+17 ||r(i)||/||b|| 7.692838251844e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.552699268498e-02 1 KSP Residual norm 3.880587833025e-15 53 KSP preconditioned resid norm 6.628413883383e+03 true resid norm 1.478135305786e+17 ||r(i)||/||b|| 8.417633509545e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.354164979975e-01 1 KSP Residual norm 1.073467575675e-14 54 KSP preconditioned resid norm 5.583298607651e+03 true resid norm 1.607393601737e+17 ||r(i)||/||b|| 9.153729155947e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.768247575683e-01 1 KSP Residual norm 7.925034358398e-15 55 KSP preconditioned resid norm 4.854896331648e+03 true resid norm 1.721936891633e+17 ||r(i)||/||b|| 9.806026297857e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.050327743689e-01 1 KSP Residual norm 1.458510169247e-14 56 KSP preconditioned resid norm 4.262399410809e+03 true resid norm 1.837125351848e+17 ||r(i)||/||b|| 1.046199753325e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.402303362087e-01 1 KSP Residual norm 1.021937235406e-14 57 KSP preconditioned resid norm 3.584024669708e+03 true resid norm 2.026508285120e+17 ||r(i)||/||b|| 1.154048887232e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.547784188892e-01 1 KSP Residual norm 2.095367572283e-14 58 KSP preconditioned resid norm 3.302611897269e+03 true resid norm 2.155979884936e+17 ||r(i)||/||b|| 1.227779923415e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.240632014175e-01 1 KSP Residual norm 1.749305570860e-14 59 KSP preconditioned resid norm 3.106452236462e+03 true resid norm 2.294151323914e+17 ||r(i)||/||b|| 1.306465313734e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.529538123727e-01 1 KSP Residual norm 1.780729312291e-14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.362276533049e+03 1 KSP Residual norm 9.408788105556e-11 60 KSP preconditioned resid norm 2.894632302157e+03 true resid norm 2.500169544004e+17 ||r(i)||/||b|| 1.423787852896e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.165306005194e-01 1 KSP Residual norm 4.440352683747e-14 61 KSP preconditioned resid norm 2.892427608173e+03 true resid norm 2.503572781299e+17 ||r(i)||/||b|| 1.425725916630e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.944839539559e-01 1 KSP Residual norm 2.299886915476e-14 62 KSP preconditioned resid norm 2.881855812195e+03 true resid norm 2.520690203889e+17 ||r(i)||/||b|| 1.435473886889e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.198280520481e-01 1 KSP Residual norm 4.423326697447e-14 63 KSP preconditioned resid norm 2.872986380420e+03 true resid norm 2.536302493423e+17 ||r(i)||/||b|| 1.444364719211e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.089918279476e-01 1 KSP Residual norm 1.800237076950e-14 64 KSP preconditioned resid norm 2.865635949476e+03 true resid norm 2.551732099922e+17 ||r(i)||/||b|| 1.453151517835e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.719087534002e-01 1 KSP Residual norm 1.837140972890e-14 65 KSP preconditioned resid norm 2.860836435778e+03 true resid norm 2.565769302236e+17 ||r(i)||/||b|| 1.461145374968e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.481116351416e-01 1 KSP Residual norm 2.007161710282e-14 66 KSP preconditioned resid norm 2.860026342229e+03 true resid norm 2.574294644210e+17 ||r(i)||/||b|| 1.466000357053e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.525429042937e-01 1 KSP Residual norm 2.156117826644e-14 67 KSP preconditioned resid norm 2.860026336213e+03 true resid norm 2.574330960350e+17 ||r(i)||/||b|| 1.466021038242e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.882198589149e-01 1 KSP Residual norm 7.689815713863e-15 68 KSP preconditioned resid norm 2.859446352239e+03 true resid norm 2.564102966971e+17 ||r(i)||/||b|| 1.460196436160e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.626191755428e-01 1 KSP Residual norm 1.723112875186e-14 69 KSP preconditioned resid norm 2.855291192202e+03 true resid norm 2.530743143937e+17 ||r(i)||/||b|| 1.441198800209e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.657349055826e-01 1 KSP Residual norm 1.239710116466e-14 70 KSP preconditioned resid norm 2.843683581024e+03 true resid norm 2.467887076324e+17 ||r(i)||/||b|| 1.405403745525e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.701637412937e-01 1 KSP Residual norm 1.019116797445e-14 71 KSP preconditioned resid norm 2.825924372468e+03 true resid norm 2.382078643186e+17 ||r(i)||/||b|| 1.356537857581e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.796494252151e-01 1 KSP Residual norm 1.629015568887e-14 72 KSP preconditioned resid norm 2.803340023027e+03 true resid norm 2.288941643475e+17 ||r(i)||/||b|| 1.303498523044e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.509030408957e-01 1 KSP Residual norm 7.503202555850e-15 73 KSP preconditioned resid norm 2.768188650420e+03 true resid norm 2.170599056557e+17 ||r(i)||/||b|| 1.236105198404e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.403279864061e-01 1 KSP Residual norm 7.083414306415e-15 74 KSP preconditioned resid norm 2.728040723726e+03 true resid norm 2.050447447767e+17 ||r(i)||/||b|| 1.167681678282e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.235758010958e-01 1 KSP Residual norm 6.082600419538e-15 75 KSP preconditioned resid norm 2.663723516783e+03 true resid norm 1.886813387898e+17 ||r(i)||/||b|| 1.074495923212e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.194778522578e-01 1 KSP Residual norm 1.644080208873e-14 76 KSP preconditioned resid norm 2.580490699769e+03 true resid norm 1.697855415479e+17 ||r(i)||/||b|| 9.668887945342e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.162122535052e-01 1 KSP Residual norm 4.736563840197e-15 77 KSP preconditioned resid norm 2.507785655569e+03 true resid norm 1.549544648976e+17 ||r(i)||/||b|| 8.824292952547e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.292215567615e-02 1 KSP Residual norm 5.820466774404e-15 78 KSP preconditioned resid norm 2.381407885104e+03 true resid norm 1.327524812787e+17 ||r(i)||/||b|| 7.559942114318e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.878020432997e-02 1 KSP Residual norm 9.375574109779e-15 79 KSP preconditioned resid norm 2.275211715198e+03 true resid norm 1.165811443595e+17 ||r(i)||/||b|| 6.639022446051e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.317922828486e-02 1 KSP Residual norm 6.709439480018e-15 80 KSP preconditioned resid norm 2.161471491914e+03 true resid norm 1.015723095291e+17 ||r(i)||/||b|| 5.784304542263e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.004691390359e-02 1 KSP Residual norm 5.059825151652e-15 81 KSP preconditioned resid norm 1.982790704180e+03 true resid norm 8.141089223231e+16 ||r(i)||/||b|| 4.636159164957e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.658868321392e-02 1 KSP Residual norm 5.284454678312e-15 82 KSP preconditioned resid norm 1.740676428371e+03 true resid norm 5.840122403494e+16 ||r(i)||/||b|| 3.325812586376e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.546598555059e-02 1 KSP Residual norm 5.389178370759e-15 83 KSP preconditioned resid norm 1.587150103801e+03 true resid norm 4.629492757889e+16 ||r(i)||/||b|| 2.636387428029e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.579942189825e-02 1 KSP Residual norm 2.925284718872e-15 84 KSP preconditioned resid norm 1.322911467320e+03 true resid norm 2.928137554973e+16 ||r(i)||/||b|| 1.667505586723e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.844872261644e-02 1 KSP Residual norm 2.358375413277e-15 85 KSP preconditioned resid norm 1.181708293490e+03 true resid norm 2.197349340824e+16 ||r(i)||/||b|| 1.251338857214e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.306304458857e-02 1 KSP Residual norm 5.376181300056e-15 86 KSP preconditioned resid norm 9.902319641871e+02 true resid norm 1.423216413086e+16 ||r(i)||/||b|| 8.104883310228e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.279208851959e-02 1 KSP Residual norm 3.873435620041e-15 87 KSP preconditioned resid norm 8.359003981497e+02 true resid norm 9.303072073000e+15 ||r(i)||/||b|| 5.297881115271e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.411795608184e-02 1 KSP Residual norm 2.149885048897e-15 88 KSP preconditioned resid norm 6.981353816351e+02 true resid norm 5.935654373576e+15 ||r(i)||/||b|| 3.380215800307e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.806354454189e-02 1 KSP Residual norm 4.211558319536e-15 89 KSP preconditioned resid norm 5.800715464910e+02 true resid norm 3.708718738882e+15 ||r(i)||/||b|| 2.112028243402e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.378367843596e-02 1 KSP Residual norm 4.841752762035e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.262347348058e+01 1 KSP Residual norm 1.008816625874e-12 90 KSP preconditioned resid norm 4.883814974933e+02 true resid norm 2.388206801554e+15 ||r(i)||/||b|| 1.360027699886e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.704733431426e-02 1 KSP Residual norm 2.819647156081e-15 91 KSP preconditioned resid norm 4.789803955726e+02 true resid norm 2.397467691582e+15 ||r(i)||/||b|| 1.365301559317e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.389489186629e-02 1 KSP Residual norm 1.387938523458e-14 92 KSP preconditioned resid norm 4.571508435040e+02 true resid norm 2.421108288771e+15 ||r(i)||/||b|| 1.378764324350e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.462628653018e-02 1 KSP Residual norm 2.052394328917e-14 93 KSP preconditioned resid norm 4.504410874594e+02 true resid norm 2.430412637471e+15 ||r(i)||/||b|| 1.384062932474e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.668189141494e-01 1 KSP Residual norm 1.877558049570e-14 94 KSP preconditioned resid norm 4.405021386174e+02 true resid norm 2.456969950409e+15 ||r(i)||/||b|| 1.399186698643e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.776534479465e-01 1 KSP Residual norm 2.310772953932e-14 95 KSP preconditioned resid norm 4.293421493884e+02 true resid norm 2.491683283369e+15 ||r(i)||/||b|| 1.418955126717e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.606412366904e-01 1 KSP Residual norm 2.122396211016e-14 96 KSP preconditioned resid norm 4.173639478383e+02 true resid norm 2.540093955020e+15 ||r(i)||/||b|| 1.446523867570e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.346533923868e-02 1 KSP Residual norm 1.503078855333e-14 97 KSP preconditioned resid norm 4.071406475666e+02 true resid norm 2.585692441776e+15 ||r(i)||/||b|| 1.472491135153e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.920241840132e-02 1 KSP Residual norm 4.032077331726e-14 98 KSP preconditioned resid norm 3.923770546792e+02 true resid norm 2.664480942542e+15 ||r(i)||/||b|| 1.517359336435e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.231708914902e-02 1 KSP Residual norm 3.770470752765e-14 99 KSP preconditioned resid norm 3.696685934377e+02 true resid norm 2.790356292077e+15 ||r(i)||/||b|| 1.589042392521e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.209831489793e-02 1 KSP Residual norm 7.913084380445e-15 100 KSP preconditioned resid norm 3.522302511012e+02 true resid norm 2.891433737297e+15 ||r(i)||/||b|| 1.646603624339e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.717762359557e-02 1 KSP Residual norm 6.387846873809e-15 101 KSP preconditioned resid norm 3.289451358269e+02 true resid norm 3.043740158723e+15 ||r(i)||/||b|| 1.733338555282e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.905335730113e-02 1 KSP Residual norm 1.253434192456e-14 102 KSP preconditioned resid norm 3.041026456067e+02 true resid norm 3.223224057165e+15 ||r(i)||/||b|| 1.835550421275e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.361347595838e-02 1 KSP Residual norm 1.589751445909e-14 103 KSP preconditioned resid norm 2.721797720469e+02 true resid norm 3.434699217170e+15 ||r(i)||/||b|| 1.955980559593e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.666368265704e-02 1 KSP Residual norm 2.047784953441e-14 104 KSP preconditioned resid norm 2.495064987590e+02 true resid norm 3.595571483255e+15 ||r(i)||/||b|| 2.047593537949e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.972669728289e-02 1 KSP Residual norm 1.587845242165e-14 105 KSP preconditioned resid norm 2.226326066385e+02 true resid norm 3.801867800407e+15 ||r(i)||/||b|| 2.165074446859e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.235885856035e-02 1 KSP Residual norm 8.108441479733e-15 106 KSP preconditioned resid norm 2.073865351310e+02 true resid norm 3.926005860070e+15 ||r(i)||/||b|| 2.235768157153e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.288096354148e-02 1 KSP Residual norm 8.226581209870e-15 107 KSP preconditioned resid norm 1.889509610510e+02 true resid norm 4.115951929688e+15 ||r(i)||/||b|| 2.343937983985e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.077517535712e-01 1 KSP Residual norm 1.263977595841e-14 108 KSP preconditioned resid norm 1.749949838100e+02 true resid norm 4.278800082488e+15 ||r(i)||/||b|| 2.436676183433e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.124921729581e-01 1 KSP Residual norm 8.947834815232e-15 109 KSP preconditioned resid norm 1.618193213731e+02 true resid norm 4.465831458563e+15 ||r(i)||/||b|| 2.543186160728e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.266675799102e-01 1 KSP Residual norm 5.644406386791e-15 110 KSP preconditioned resid norm 1.465181112551e+02 true resid norm 4.714655904952e+15 ||r(i)||/||b|| 2.684885840704e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.616539885274e-01 1 KSP Residual norm 1.087196543493e-14 111 KSP preconditioned resid norm 1.318107859469e+02 true resid norm 5.037611757045e+15 ||r(i)||/||b|| 2.868801615670e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.019418195399e-01 1 KSP Residual norm 8.813045811269e-15 112 KSP preconditioned resid norm 1.242084506988e+02 true resid norm 5.234477234429e+15 ||r(i)||/||b|| 2.980911882762e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.743305360210e-01 1 KSP Residual norm 1.078322696740e-14 113 KSP preconditioned resid norm 1.155780623602e+02 true resid norm 5.582223086850e+15 ||r(i)||/||b|| 3.178944980861e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.373146309370e-01 1 KSP Residual norm 1.121934786619e-14 114 KSP preconditioned resid norm 1.021440272152e+02 true resid norm 6.169174800806e+15 ||r(i)||/||b|| 3.513200200701e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.577417995151e-01 1 KSP Residual norm 1.608185800429e-14 115 KSP preconditioned resid norm 9.495516498203e+01 true resid norm 6.557142282859e+15 ||r(i)||/||b|| 3.734138572497e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.326441461139e-01 1 KSP Residual norm 1.475599163182e-14 116 KSP preconditioned resid norm 8.974660138822e+01 true resid norm 6.966234249946e+15 ||r(i)||/||b|| 3.967106842530e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.608712631594e-01 1 KSP Residual norm 3.057682945540e-14 117 KSP preconditioned resid norm 8.668620768379e+01 true resid norm 7.370496356350e+15 ||r(i)||/||b|| 4.197324620306e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.693428002447e-01 1 KSP Residual norm 1.397475726963e-14 118 KSP preconditioned resid norm 8.544499569894e+01 true resid norm 7.721073813452e+15 ||r(i)||/||b|| 4.396970250787e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.692697763536e-01 1 KSP Residual norm 2.527602062420e-14 119 KSP preconditioned resid norm 8.540356853154e+01 true resid norm 7.783986693879e+15 ||r(i)||/||b|| 4.432797659035e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.544114362896e-01 1 KSP Residual norm 1.003176996544e-14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.149544086266e+01 1 KSP Residual norm 3.620617423011e-12 120 KSP preconditioned resid norm 8.509182647946e+01 true resid norm 7.567149874280e+15 ||r(i)||/||b|| 4.309314181466e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.405465613062e-01 1 KSP Residual norm 4.069686873734e-14 121 KSP preconditioned resid norm 8.501310511443e+01 true resid norm 7.594940035779e+15 ||r(i)||/||b|| 4.325140025944e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.631956253110e-01 1 KSP Residual norm 2.619300070627e-14 122 KSP preconditioned resid norm 8.493474643777e+01 true resid norm 7.626607781411e+15 ||r(i)||/||b|| 4.343174063543e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.862156946512e-01 1 KSP Residual norm 3.067356159902e-14 123 KSP preconditioned resid norm 8.488333556760e+01 true resid norm 7.656182108667e+15 ||r(i)||/||b|| 4.360015948528e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.292116881356e-01 1 KSP Residual norm 3.302487897840e-14 124 KSP preconditioned resid norm 8.487226386783e+01 true resid norm 7.669732222341e+15 ||r(i)||/||b|| 4.367732420118e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.672163974636e-01 1 KSP Residual norm 1.388193683457e-14 125 KSP preconditioned resid norm 8.487223185288e+01 true resid norm 7.668341733505e+15 ||r(i)||/||b|| 4.366940569373e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.953837842424e-01 1 KSP Residual norm 3.224928125577e-14 126 KSP preconditioned resid norm 8.485866327100e+01 true resid norm 7.639207170331e+15 ||r(i)||/||b|| 4.350349119707e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.920848608123e-01 1 KSP Residual norm 3.937923070844e-14 127 KSP preconditioned resid norm 8.480918955298e+01 true resid norm 7.584336070638e+15 ||r(i)||/||b|| 4.319101316770e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.760053086227e-01 1 KSP Residual norm 4.078484056150e-14 128 KSP preconditioned resid norm 8.461824097068e+01 true resid norm 7.465571340945e+15 ||r(i)||/||b|| 4.251467591732e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.572514546188e-01 1 KSP Residual norm 1.500036257427e-14 129 KSP preconditioned resid norm 8.425430309267e+01 true resid norm 7.263958693975e+15 ||r(i)||/||b|| 4.136653923021e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.803203692911e-01 1 KSP Residual norm 1.100923154553e-14 130 KSP preconditioned resid norm 8.380956131992e+01 true resid norm 7.064381669302e+15 ||r(i)||/||b|| 4.022999493413e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.475756125120e-01 1 KSP Residual norm 6.340526362351e-15 131 KSP preconditioned resid norm 8.314252130468e+01 true resid norm 6.808447742483e+15 ||r(i)||/||b|| 3.877251131258e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.425784809582e-01 1 KSP Residual norm 2.287330326722e-14 132 KSP preconditioned resid norm 8.226839212542e+01 true resid norm 6.516418982759e+15 ||r(i)||/||b|| 3.710947609247e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.327916764308e-01 1 KSP Residual norm 1.160678799057e-14 133 KSP preconditioned resid norm 8.078979442602e+01 true resid norm 6.095155862228e+15 ||r(i)||/||b|| 3.471048153099e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.235477772951e-01 1 KSP Residual norm 1.387437826422e-14 134 KSP preconditioned resid norm 7.933264236809e+01 true resid norm 5.736990224682e+15 ||r(i)||/||b|| 3.267081232022e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.442608308610e-02 1 KSP Residual norm 8.580779610678e-15 135 KSP preconditioned resid norm 7.629870801296e+01 true resid norm 5.098475070340e+15 ||r(i)||/||b|| 2.903461843560e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.044574609148e-01 1 KSP Residual norm 5.344151663174e-15 136 KSP preconditioned resid norm 7.384550506962e+01 true resid norm 4.620263607105e+15 ||r(i)||/||b|| 2.631131643353e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.312986062637e-02 1 KSP Residual norm 1.831837803733e-14 137 KSP preconditioned resid norm 7.019186147043e+01 true resid norm 3.995768136048e+15 ||r(i)||/||b|| 2.275496135348e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.996095146259e-02 1 KSP Residual norm 3.547339901511e-15 138 KSP preconditioned resid norm 6.549021238982e+01 true resid norm 3.271622426563e+15 ||r(i)||/||b|| 1.863112156283e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.727364488516e-02 1 KSP Residual norm 7.265180868689e-15 139 KSP preconditioned resid norm 6.216358370903e+01 true resid norm 2.825089749412e+15 ||r(i)||/||b|| 1.608822280954e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.718899482930e-02 1 KSP Residual norm 3.920662921030e-15 140 KSP preconditioned resid norm 5.819590254270e+01 true resid norm 2.365235650999e+15 ||r(i)||/||b|| 1.346946168993e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.707477036849e-02 1 KSP Residual norm 5.128671758811e-15 141 KSP preconditioned resid norm 5.322146484911e+01 true resid norm 1.862541531651e+15 ||r(i)||/||b|| 1.060673670967e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.739912345712e-02 1 KSP Residual norm 5.218644156412e-15 142 KSP preconditioned resid norm 4.838864057319e+01 true resid norm 1.460907841679e+15 ||r(i)||/||b|| 8.319527146358e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.324734710895e-02 1 KSP Residual norm 1.713271438493e-15 143 KSP preconditioned resid norm 4.288901931377e+01 true resid norm 1.075225961416e+15 ||r(i)||/||b|| 6.123159393948e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.909274848556e-02 1 KSP Residual norm 3.791735116392e-15 144 KSP preconditioned resid norm 3.774714230712e+01 true resid norm 7.813012526156e+14 ||r(i)||/||b|| 4.449327189008e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.455399761613e-02 1 KSP Residual norm 8.432667272675e-15 145 KSP preconditioned resid norm 3.210844092000e+01 true resid norm 5.182912921441e+14 ||r(i)||/||b|| 2.951547217213e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.425351119629e-02 1 KSP Residual norm 7.840730412096e-15 146 KSP preconditioned resid norm 2.633903313561e+01 true resid norm 3.154832144343e+14 ||r(i)||/||b|| 1.796602832722e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.267810003654e-02 1 KSP Residual norm 1.853325783419e-15 147 KSP preconditioned resid norm 2.195326852966e+01 true resid norm 1.974702011318e+14 ||r(i)||/||b|| 1.124546430680e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.059533160127e-02 1 KSP Residual norm 3.880033145575e-15 148 KSP preconditioned resid norm 1.932058747257e+01 true resid norm 1.401754982427e+14 ||r(i)||/||b|| 7.982665501638e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.922159692313e-02 1 KSP Residual norm 4.254698830779e-15 149 KSP preconditioned resid norm 1.720408953179e+01 true resid norm 1.028932388368e+14 ||r(i)||/||b|| 5.859528364877e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.720490765467e-02 1 KSP Residual norm 2.995400749171e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.698566365971e-01 1 KSP Residual norm 4.242734101262e-14 150 KSP preconditioned resid norm 1.559612972721e+01 true resid norm 8.129961677145e+13 ||r(i)||/||b|| 4.629822288729e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.983152808428e-02 1 KSP Residual norm 2.548021685714e-15 151 KSP preconditioned resid norm 1.528043473124e+01 true resid norm 8.168490748905e+13 ||r(i)||/||b|| 4.651763690458e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.775434224923e-02 1 KSP Residual norm 2.821242593794e-14 152 KSP preconditioned resid norm 1.473379227056e+01 true resid norm 8.241474869184e+13 ||r(i)||/||b|| 4.693326433336e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.611027613575e-01 1 KSP Residual norm 3.837537072715e-14 153 KSP preconditioned resid norm 1.446555241196e+01 true resid norm 8.283309499757e+13 ||r(i)||/||b|| 4.717150273196e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.003053001324e-01 1 KSP Residual norm 4.720744628996e-14 154 KSP preconditioned resid norm 1.433069699697e+01 true resid norm 8.320636394822e+13 ||r(i)||/||b|| 4.738407063524e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.383592836455e-01 1 KSP Residual norm 2.115917312023e-14 155 KSP preconditioned resid norm 1.398451149982e+01 true resid norm 8.447595182963e+13 ||r(i)||/||b|| 4.810707112457e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.419635827286e-02 1 KSP Residual norm 2.426183427348e-14 156 KSP preconditioned resid norm 1.338238360243e+01 true resid norm 8.694939355383e+13 ||r(i)||/||b|| 4.951563811164e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.008329724485e-02 1 KSP Residual norm 2.030808580056e-14 157 KSP preconditioned resid norm 1.285596687709e+01 true resid norm 8.942603293712e+13 ||r(i)||/||b|| 5.092602609048e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.375596191492e-02 1 KSP Residual norm 2.845017599242e-14 158 KSP preconditioned resid norm 1.259640678944e+01 true resid norm 9.096287132985e+13 ||r(i)||/||b|| 5.180121947114e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.009801642531e-02 1 KSP Residual norm 1.975915069711e-14 159 KSP preconditioned resid norm 1.150402693981e+01 true resid norm 9.770719346254e+13 ||r(i)||/||b|| 5.564195257325e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.933590874665e-02 1 KSP Residual norm 9.647642941187e-15 160 KSP preconditioned resid norm 1.091253999431e+01 true resid norm 1.013341514372e+14 ||r(i)||/||b|| 5.770741998110e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.164615120399e-02 1 KSP Residual norm 9.800066509091e-15 161 KSP preconditioned resid norm 1.005189955046e+01 true resid norm 1.071270499652e+14 ||r(i)||/||b|| 6.100633967920e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.939988206716e-02 1 KSP Residual norm 6.919428925773e-15 162 KSP preconditioned resid norm 9.249803514171e+00 true resid norm 1.126348254529e+14 ||r(i)||/||b|| 6.414288850030e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.189802241163e-02 1 KSP Residual norm 6.350798872997e-15 163 KSP preconditioned resid norm 8.498257632424e+00 true resid norm 1.184209993877e+14 ||r(i)||/||b|| 6.743797870045e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.196680027343e-02 1 KSP Residual norm 7.075248718013e-15 164 KSP preconditioned resid norm 7.683745538480e+00 true resid norm 1.250523967921e+14 ||r(i)||/||b|| 7.121440382118e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.816725942938e-02 1 KSP Residual norm 6.556663955684e-15 165 KSP preconditioned resid norm 7.203608281457e+00 true resid norm 1.292064598020e+14 ||r(i)||/||b|| 7.358004517055e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.595327197871e-02 1 KSP Residual norm 1.268691476289e-14 166 KSP preconditioned resid norm 6.554687098085e+00 true resid norm 1.355010983788e+14 ||r(i)||/||b|| 7.716469404586e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.049820618324e-01 1 KSP Residual norm 1.049392572381e-14 167 KSP preconditioned resid norm 6.040625888585e+00 true resid norm 1.410686273098e+14 ||r(i)||/||b|| 8.033527104998e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.082599133933e-01 1 KSP Residual norm 1.076305957967e-14 168 KSP preconditioned resid norm 5.602778075991e+00 true resid norm 1.472338183013e+14 ||r(i)||/||b|| 8.384620256483e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.214867871266e-01 1 KSP Residual norm 7.544134595691e-15 169 KSP preconditioned resid norm 5.137313587881e+00 true resid norm 1.546288829821e+14 ||r(i)||/||b|| 8.805751826902e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.451422200433e-01 1 KSP Residual norm 8.798919031561e-15 170 KSP preconditioned resid norm 4.772073779271e+00 true resid norm 1.617511555644e+14 ||r(i)||/||b|| 9.211348527816e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.538997464631e-01 1 KSP Residual norm 1.154848627985e-14 171 KSP preconditioned resid norm 4.429551959412e+00 true resid norm 1.699874542588e+14 ||r(i)||/||b|| 9.680386400149e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.772777372295e-01 1 KSP Residual norm 1.088781590122e-14 172 KSP preconditioned resid norm 4.058042007762e+00 true resid norm 1.807915970742e+14 ||r(i)||/||b|| 1.029565696604e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.218964362063e-01 1 KSP Residual norm 1.367276410642e-14 173 KSP preconditioned resid norm 3.659352909972e+00 true resid norm 1.979586060760e+14 ||r(i)||/||b|| 1.127327782163e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.042932733202e-01 1 KSP Residual norm 1.514786491688e-14 174 KSP preconditioned resid norm 3.345836830163e+00 true resid norm 2.142617703762e+14 ||r(i)||/||b|| 1.220170474971e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.423019487483e-01 1 KSP Residual norm 1.585379300763e-14 175 KSP preconditioned resid norm 3.144285094141e+00 true resid norm 2.281977537693e+14 ||r(i)||/||b|| 1.299532628314e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.444560138891e-01 1 KSP Residual norm 1.826154859425e-14 176 KSP preconditioned resid norm 3.022396458018e+00 true resid norm 2.394375101324e+14 ||r(i)||/||b|| 1.363540401777e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.264841471998e-01 1 KSP Residual norm 2.281531782132e-14 177 KSP preconditioned resid norm 2.937051498184e+00 true resid norm 2.511984383257e+14 ||r(i)||/||b|| 1.430516126446e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.438912207642e-01 1 KSP Residual norm 1.255498343538e-14 178 KSP preconditioned resid norm 2.894843703130e+00 true resid norm 2.630994251974e+14 ||r(i)||/||b|| 1.498289452403e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.690538188432e-01 1 KSP Residual norm 2.250517560947e-14 179 KSP preconditioned resid norm 2.891859475040e+00 true resid norm 2.664328621553e+14 ||r(i)||/||b|| 1.517272593208e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.805084972468e-01 1 KSP Residual norm 1.040787396795e-14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.456335165427e+00 1 KSP Residual norm 1.187478904410e-13 180 KSP preconditioned resid norm 2.881378939656e+00 true resid norm 2.599861140094e+14 ||r(i)||/||b|| 1.480559876173e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.527765332166e-01 1 KSP Residual norm 4.441865472488e-14 181 KSP preconditioned resid norm 2.879542572807e+00 true resid norm 2.605906041815e+14 ||r(i)||/||b|| 1.484002305773e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.392188934233e-01 1 KSP Residual norm 2.345546419849e-14 182 KSP preconditioned resid norm 2.877331841910e+00 true resid norm 2.614134834872e+14 ||r(i)||/||b|| 1.488688410212e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.683164945024e-01 1 KSP Residual norm 3.403904075903e-14 183 KSP preconditioned resid norm 2.874457206174e+00 true resid norm 2.627920202652e+14 ||r(i)||/||b|| 1.496538853491e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.072401669340e-01 1 KSP Residual norm 3.345257461923e-14 184 KSP preconditioned resid norm 2.874349932663e+00 true resid norm 2.629889650379e+14 ||r(i)||/||b|| 1.497660407730e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.170144502745e-01 1 KSP Residual norm 5.819885505904e-14 185 KSP preconditioned resid norm 2.874316515576e+00 true resid norm 2.632240654583e+14 ||r(i)||/||b|| 1.498999249424e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.800384503143e-01 1 KSP Residual norm 1.591860780959e-14 186 KSP preconditioned resid norm 2.873427056139e+00 true resid norm 2.616630691857e+14 ||r(i)||/||b|| 1.490109742164e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.018927052803e-01 1 KSP Residual norm 3.129174808875e-14 187 KSP preconditioned resid norm 2.871829462138e+00 true resid norm 2.600478851962e+14 ||r(i)||/||b|| 1.480911648579e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.637917563527e-01 1 KSP Residual norm 3.193446569020e-14 188 KSP preconditioned resid norm 2.866311791868e+00 true resid norm 2.569200071429e+14 ||r(i)||/||b|| 1.463099117472e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.379315055220e-01 1 KSP Residual norm 1.105628872594e-14 189 KSP preconditioned resid norm 2.846491410206e+00 true resid norm 2.470738274589e+14 ||r(i)||/||b|| 1.407027435992e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.842917846689e-01 1 KSP Residual norm 1.179166615236e-14 190 KSP preconditioned resid norm 2.819731512218e+00 true resid norm 2.362847153980e+14 ||r(i)||/||b|| 1.345585976022e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.576426740575e-01 1 KSP Residual norm 1.707484813968e-14 191 KSP preconditioned resid norm 2.792936291745e+00 true resid norm 2.273872017288e+14 ||r(i)||/||b|| 1.294916724756e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.220909275811e-01 1 KSP Residual norm 1.197393115191e-14 192 KSP preconditioned resid norm 2.756925837105e+00 true resid norm 2.167289779578e+14 ||r(i)||/||b|| 1.234220642864e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.143566345141e-01 1 KSP Residual norm 1.148499584951e-14 193 KSP preconditioned resid norm 2.712359961087e+00 true resid norm 2.044302013060e+14 ||r(i)||/||b|| 1.164181997508e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.107756532337e-01 1 KSP Residual norm 1.632152981890e-14 194 KSP preconditioned resid norm 2.634448327016e+00 true resid norm 1.861634496794e+14 ||r(i)||/||b|| 1.060157135913e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.043405317003e-01 1 KSP Residual norm 7.856186790365e-15 195 KSP preconditioned resid norm 2.554393404317e+00 true resid norm 1.695437914179e+14 ||r(i)||/||b|| 9.655120843051e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.897340056754e-02 1 KSP Residual norm 1.233077367022e-14 196 KSP preconditioned resid norm 2.423824906921e+00 true resid norm 1.453836419572e+14 ||r(i)||/||b|| 8.279257057783e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.502812970292e-02 1 KSP Residual norm 7.704791381187e-15 197 KSP preconditioned resid norm 2.301146621552e+00 true resid norm 1.254691319924e+14 ||r(i)||/||b|| 7.145172473306e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.907420812051e-02 1 KSP Residual norm 7.888966824591e-15 198 KSP preconditioned resid norm 2.127716661822e+00 true resid norm 1.001849406975e+14 ||r(i)||/||b|| 5.705297144763e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.716253367866e-02 1 KSP Residual norm 8.361962613217e-15 199 KSP preconditioned resid norm 2.009570453213e+00 true resid norm 8.505084049811e+13 ||r(i)||/||b|| 4.843445672325e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.819264528613e-02 1 KSP Residual norm 6.951257832233e-15 200 KSP preconditioned resid norm 1.877929062377e+00 true resid norm 7.067972158531e+13 ||r(i)||/||b|| 4.025044192727e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.672402963659e-02 1 KSP Residual norm 5.297852273456e-15 201 KSP preconditioned resid norm 1.742306242225e+00 true resid norm 5.778292864225e+13 ||r(i)||/||b|| 3.290602115481e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.871933633585e-02 1 KSP Residual norm 6.192002480595e-15 202 KSP preconditioned resid norm 1.552694419760e+00 true resid norm 4.295281880799e+13 ||r(i)||/||b|| 2.446062180589e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.918330495290e-02 1 KSP Residual norm 4.483591375824e-15 203 KSP preconditioned resid norm 1.405131030219e+00 true resid norm 3.341310972617e+13 ||r(i)||/||b|| 1.902798147018e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.301758775286e-02 1 KSP Residual norm 4.985298204897e-15 204 KSP preconditioned resid norm 1.207664654845e+00 true resid norm 2.297637594674e+13 ||r(i)||/||b|| 1.308450663076e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.487114864986e-02 1 KSP Residual norm 6.686322509946e-15 205 KSP preconditioned resid norm 1.053317934875e+00 true resid norm 1.622814116375e+13 ||r(i)||/||b|| 9.241545366171e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.464047516038e-02 1 KSP Residual norm 6.373082739763e-15 206 KSP preconditioned resid norm 8.888842477271e-01 true resid norm 1.056121318385e+13 ||r(i)||/||b|| 6.014362937534e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.308297823471e-02 1 KSP Residual norm 3.228552462555e-15 207 KSP preconditioned resid norm 7.509565241464e-01 true resid norm 6.806270118230e+12 ||r(i)||/||b|| 3.876011025375e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.177134799295e-02 1 KSP Residual norm 6.514276661277e-15 208 KSP preconditioned resid norm 6.625683546609e-01 true resid norm 4.847391962688e+12 ||r(i)||/||b|| 2.760475909025e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.577304365446e-02 1 KSP Residual norm 3.006506436827e-15 209 KSP preconditioned resid norm 5.988642171909e-01 true resid norm 3.692436202320e+12 ||r(i)||/||b|| 2.102755721133e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.086253992372e-02 1 KSP Residual norm 4.986918085249e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.885326783279e-02 1 KSP Residual norm 1.404475689675e-15 210 KSP preconditioned resid norm 5.549022199891e-01 true resid norm 3.047379519414e+12 ||r(i)||/||b|| 1.735411085745e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.211250667338e-02 1 KSP Residual norm 1.949072374552e-15 211 KSP preconditioned resid norm 5.455065633447e-01 true resid norm 3.068914211850e+12 ||r(i)||/||b|| 1.747674587466e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.362579277780e-02 1 KSP Residual norm 1.739560075111e-14 212 KSP preconditioned resid norm 5.256035528838e-01 true resid norm 3.117147854387e+12 ||r(i)||/||b|| 1.775142514396e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.498397648598e-02 1 KSP Residual norm 3.157070253707e-14 213 KSP preconditioned resid norm 5.188510171595e-01 true resid norm 3.135844570587e+12 ||r(i)||/||b|| 1.785789855284e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.747223733427e-01 1 KSP Residual norm 2.229812801717e-14 214 KSP preconditioned resid norm 5.064659649982e-01 true resid norm 3.177031272671e+12 ||r(i)||/||b|| 1.809244715083e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.236869632461e-01 1 KSP Residual norm 2.922225997772e-14 215 KSP preconditioned resid norm 4.942964291547e-01 true resid norm 3.227526236207e+12 ||r(i)||/||b|| 1.838000411226e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.670277153783e-01 1 KSP Residual norm 2.489463845279e-14 216 KSP preconditioned resid norm 4.748832016015e-01 true resid norm 3.319200539705e+12 ||r(i)||/||b|| 1.890206774614e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.233207557671e-01 1 KSP Residual norm 2.664414905660e-14 217 KSP preconditioned resid norm 4.515765272474e-01 true resid norm 3.439080465531e+12 ||r(i)||/||b|| 1.958475577666e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.069169676118e-01 1 KSP Residual norm 1.742013956853e-14 218 KSP preconditioned resid norm 4.373635821286e-01 true resid norm 3.528932804201e+12 ||r(i)||/||b|| 2.009644374862e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.848611903705e-02 1 KSP Residual norm 2.037010741050e-14 219 KSP preconditioned resid norm 4.009408906413e-01 true resid norm 3.761069194307e+12 ||r(i)||/||b|| 2.141840598610e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.406641258785e-02 1 KSP Residual norm 1.727865745898e-14 220 KSP preconditioned resid norm 3.740642106294e-01 true resid norm 3.936784283271e+12 ||r(i)||/||b|| 2.241906216095e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.464274294020e-02 1 KSP Residual norm 1.362084954420e-14 221 KSP preconditioned resid norm 3.517621171323e-01 true resid norm 4.097032495403e+12 ||r(i)||/||b|| 2.333163810377e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.084819938922e-02 1 KSP Residual norm 9.867274238529e-15 222 KSP preconditioned resid norm 3.229172006626e-01 true resid norm 4.313513535838e+12 ||r(i)||/||b|| 2.456444680066e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.419457027247e-02 1 KSP Residual norm 1.371011247946e-14 223 KSP preconditioned resid norm 2.994652166016e-01 true resid norm 4.502573148283e+12 ||r(i)||/||b|| 2.564109690352e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.126563841148e-02 1 KSP Residual norm 6.406045790162e-15 224 KSP preconditioned resid norm 2.773847405024e-01 true resid norm 4.710007856899e+12 ||r(i)||/||b|| 2.682238886471e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.654362398008e-02 1 KSP Residual norm 9.059827849603e-15 225 KSP preconditioned resid norm 2.501293925886e-01 true resid norm 4.969592381216e+12 ||r(i)||/||b|| 2.830066178188e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.053285546577e-01 1 KSP Residual norm 8.400900669162e-15 226 KSP preconditioned resid norm 2.304711303177e-01 true resid norm 5.185436529100e+12 ||r(i)||/||b|| 2.952984352523e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.066853240120e-01 1 KSP Residual norm 9.778263593415e-15 227 KSP preconditioned resid norm 2.100779081353e-01 true resid norm 5.450173944695e+12 ||r(i)||/||b|| 3.103746094835e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.277024603368e-01 1 KSP Residual norm 9.776514604684e-15 228 KSP preconditioned resid norm 1.902334377364e-01 true resid norm 5.764163371932e+12 ||r(i)||/||b|| 3.282555701372e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.543910821598e-01 1 KSP Residual norm 1.181391499448e-14 229 KSP preconditioned resid norm 1.696092543322e-01 true resid norm 6.179727454390e+12 ||r(i)||/||b|| 3.519209689148e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.979926773607e-01 1 KSP Residual norm 1.071619560571e-14 230 KSP preconditioned resid norm 1.581594612138e-01 true resid norm 6.486820673125e+12 ||r(i)||/||b|| 3.694092066861e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.854916535800e-01 1 KSP Residual norm 1.132398649899e-14 231 KSP preconditioned resid norm 1.508614670436e-01 true resid norm 6.725942677263e+12 ||r(i)||/||b|| 3.830266433782e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.739948371107e-01 1 KSP Residual norm 1.000596545317e-14 232 KSP preconditioned resid norm 1.423059331638e-01 true resid norm 7.059559742076e+12 ||r(i)||/||b|| 4.020253519074e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.165302109324e-01 1 KSP Residual norm 1.436273080547e-14 233 KSP preconditioned resid norm 1.315342545891e-01 true resid norm 7.576207806606e+12 ||r(i)||/||b|| 4.314472461251e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.911479359997e-01 1 KSP Residual norm 1.198219600577e-14 234 KSP preconditioned resid norm 1.237629426248e-01 true resid norm 8.062838626274e+12 ||r(i)||/||b|| 4.591597287265e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.061700420075e-01 1 KSP Residual norm 1.407708677210e-14 235 KSP preconditioned resid norm 1.167454862829e-01 true resid norm 8.626377131458e+12 ||r(i)||/||b|| 4.912519234437e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.548961874561e-01 1 KSP Residual norm 1.604305454787e-14 236 KSP preconditioned resid norm 1.118149553209e-01 true resid norm 9.193243533084e+12 ||r(i)||/||b|| 5.235336340495e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.770259787025e-01 1 KSP Residual norm 1.893803176438e-14 237 KSP preconditioned resid norm 1.093202512722e-01 true resid norm 9.616800691160e+12 ||r(i)||/||b|| 5.476542197162e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.417889391749e-01 1 KSP Residual norm 1.690901852777e-14 238 KSP preconditioned resid norm 1.083571127873e-01 true resid norm 9.931580028790e+12 ||r(i)||/||b|| 5.655801639121e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.187112380794e-01 1 KSP Residual norm 1.620010267590e-14 239 KSP preconditioned resid norm 1.082843905303e-01 true resid norm 1.003793770978e+13 ||r(i)||/||b|| 5.716369841234e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.871816129329e-01 1 KSP Residual norm 9.856603275885e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.305816419653e-02 1 KSP Residual norm 5.125943974901e-15 240 KSP preconditioned resid norm 1.078315051119e-01 true resid norm 9.849631662842e+12 ||r(i)||/||b|| 5.609133968810e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.631975659100e-01 1 KSP Residual norm 4.341548554567e-14 241 KSP preconditioned resid norm 1.077293243842e-01 true resid norm 9.870900635153e+12 ||r(i)||/||b|| 5.621246149159e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.124036568843e-01 1 KSP Residual norm 2.822614733058e-14 242 KSP preconditioned resid norm 1.075375019689e-01 true resid norm 9.914545172593e+12 ||r(i)||/||b|| 5.646100688485e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.583603326980e-01 1 KSP Residual norm 3.059478151273e-14 243 KSP preconditioned resid norm 1.074753395368e-01 true resid norm 9.944195858321e+12 ||r(i)||/||b|| 5.662986057827e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.464411713550e-01 1 KSP Residual norm 2.581305253579e-14 244 KSP preconditioned resid norm 1.074753393734e-01 true resid norm 9.944116001255e+12 ||r(i)||/||b|| 5.662940581103e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.844774702344e-01 1 KSP Residual norm 2.734713125893e-14 245 KSP preconditioned resid norm 1.074691947874e-01 true resid norm 9.924463134528e+12 ||r(i)||/||b|| 5.651748734939e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.072421499599e-01 1 KSP Residual norm 3.536389172473e-14 246 KSP preconditioned resid norm 1.073633495242e-01 true resid norm 9.811984249327e+12 ||r(i)||/||b|| 5.587694650751e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.090367772690e-01 1 KSP Residual norm 2.022736372740e-14 247 KSP preconditioned resid norm 1.072871234598e-01 true resid norm 9.751058413422e+12 ||r(i)||/||b|| 5.552998817703e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.467686068074e-01 1 KSP Residual norm 2.528166708051e-14 248 KSP preconditioned resid norm 1.067011226183e-01 true resid norm 9.474124334909e+12 ||r(i)||/||b|| 5.395291362229e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.509317584687e-01 1 KSP Residual norm 8.128514111878e-15 249 KSP preconditioned resid norm 1.055794830704e-01 true resid norm 9.010857808915e+12 ||r(i)||/||b|| 5.131471952883e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.686711500127e-01 1 KSP Residual norm 1.486145068985e-14 250 KSP preconditioned resid norm 1.047264121276e-01 true resid norm 8.722577180229e+12 ||r(i)||/||b|| 4.967302903495e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.142587525903e-01 1 KSP Residual norm 1.667766960153e-14 251 KSP preconditioned resid norm 1.029467816886e-01 true resid norm 8.186043104130e+12 ||r(i)||/||b|| 4.661759344640e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.321819326806e-01 1 KSP Residual norm 1.013710018293e-14 252 KSP preconditioned resid norm 1.004305894140e-01 true resid norm 7.495640695546e+12 ||r(i)||/||b|| 4.268591383168e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.307320262440e-01 1 KSP Residual norm 1.163000419497e-14 253 KSP preconditioned resid norm 9.795525188712e-02 true resid norm 6.920433832017e+12 ||r(i)||/||b|| 3.941024579885e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.033010276042e-01 1 KSP Residual norm 2.057300677142e-14 254 KSP preconditioned resid norm 9.555346666676e-02 true resid norm 6.409436126785e+12 ||r(i)||/||b|| 3.650023384661e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.001483252949e-01 1 KSP Residual norm 3.022216578126e-14 255 KSP preconditioned resid norm 9.212243463287e-02 true resid norm 5.754536094258e+12 ||r(i)||/||b|| 3.277073192779e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.638821676481e-02 1 KSP Residual norm 7.181126476117e-15 256 KSP preconditioned resid norm 8.758708194374e-02 true resid norm 4.960276502644e+12 ||r(i)||/||b|| 2.824761004072e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.754618472946e-02 1 KSP Residual norm 8.117223947162e-15 257 KSP preconditioned resid norm 8.129609425535e-02 true resid norm 4.021894178468e+12 ||r(i)||/||b|| 2.290374303083e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.467147798403e-02 1 KSP Residual norm 6.784150236776e-15 258 KSP preconditioned resid norm 7.531175302380e-02 true resid norm 3.231810388767e+12 ||r(i)||/||b|| 1.840440135521e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.848478043310e-02 1 KSP Residual norm 4.316675777061e-15 259 KSP preconditioned resid norm 7.147270516207e-02 true resid norm 2.780270355409e+12 ||r(i)||/||b|| 1.583298688400e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.058373929961e-02 1 KSP Residual norm 5.289429363899e-15 260 KSP preconditioned resid norm 6.629489690362e-02 true resid norm 2.268100553137e+12 ||r(i)||/||b|| 1.291630011432e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.613908443002e-02 1 KSP Residual norm 8.820012013287e-15 261 KSP preconditioned resid norm 6.208842131630e-02 true resid norm 1.905339947374e+12 ||r(i)||/||b|| 1.085046363842e+10 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.231798995468e-02 1 KSP Residual norm 7.630919261092e-15 262 KSP preconditioned resid norm 5.627458278077e-02 true resid norm 1.483145028925e+12 ||r(i)||/||b|| 8.446162706573e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.916578646329e-02 1 KSP Residual norm 2.949348030580e-15 263 KSP preconditioned resid norm 4.990039343986e-02 true resid norm 1.096548664620e+12 ||r(i)||/||b|| 6.244587182255e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.483292043233e-02 1 KSP Residual norm 4.418728740841e-15 264 KSP preconditioned resid norm 4.497116392959e-02 true resid norm 8.467826607120e+11 ||r(i)||/||b|| 4.822228433493e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.343913990765e-02 1 KSP Residual norm 4.058113449723e-15 265 KSP preconditioned resid norm 3.747770397703e-02 true resid norm 5.348773999832e+11 ||r(i)||/||b|| 3.046001207043e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.638852525403e-02 1 KSP Residual norm 6.127995084333e-15 266 KSP preconditioned resid norm 3.272498533581e-02 true resid norm 3.761035937528e+11 ||r(i)||/||b|| 2.141821659655e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.134376559679e-02 1 KSP Residual norm 2.972817521485e-15 267 KSP preconditioned resid norm 2.833964662941e-02 true resid norm 2.584023473356e+11 ||r(i)||/||b|| 1.471540696824e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.350744666503e-02 1 KSP Residual norm 7.530068076823e-15 268 KSP preconditioned resid norm 2.427356052923e-02 true resid norm 1.695160385358e+11 ||r(i)||/||b|| 9.653540381580e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.167316747264e-02 1 KSP Residual norm 5.083477541133e-15 269 KSP preconditioned resid norm 2.223267214607e-02 true resid norm 1.346277567493e+11 ||r(i)||/||b|| 7.666734649336e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.410458980223e-02 1 KSP Residual norm 5.490623589024e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.051400199236e-03 1 KSP Residual norm 4.641030410088e-17 270 KSP preconditioned resid norm 2.051294383185e-02 true resid norm 1.110745980536e+11 ||r(i)||/||b|| 6.325437562959e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.136682259001e-02 1 KSP Residual norm 2.027427674949e-15 271 KSP preconditioned resid norm 2.015854957434e-02 true resid norm 1.116964726126e+11 ||r(i)||/||b|| 6.360851858972e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.642231017164e-02 1 KSP Residual norm 1.134158249079e-14 272 KSP preconditioned resid norm 1.928855547464e-02 true resid norm 1.133138446679e+11 ||r(i)||/||b|| 6.452957400029e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.765186556955e-02 1 KSP Residual norm 2.234729757801e-14 273 KSP preconditioned resid norm 1.901642751801e-02 true resid norm 1.139196332011e+11 ||r(i)||/||b|| 6.487455634645e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.089884619353e-01 1 KSP Residual norm 3.975660349867e-14 274 KSP preconditioned resid norm 1.867376689073e-02 true resid norm 1.148722535933e+11 ||r(i)||/||b|| 6.541705129284e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.440256051332e-01 1 KSP Residual norm 3.175387513871e-14 275 KSP preconditioned resid norm 1.820278690528e-02 true resid norm 1.164715122429e+11 ||r(i)||/||b|| 6.632779154420e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.908548901254e-02 1 KSP Residual norm 1.514068651114e-14 276 KSP preconditioned resid norm 1.749874765680e-02 true resid norm 1.194547409423e+11 ||r(i)||/||b|| 6.802667024419e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.148081477012e-01 1 KSP Residual norm 3.134244668133e-14 277 KSP preconditioned resid norm 1.687371324494e-02 true resid norm 1.224936496306e+11 ||r(i)||/||b|| 6.975725739051e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.748609335503e-02 1 KSP Residual norm 2.910045348598e-14 278 KSP preconditioned resid norm 1.639579962064e-02 true resid norm 1.252604619142e+11 ||r(i)||/||b|| 7.133289202302e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.037786907532e-02 1 KSP Residual norm 3.140553015773e-14 279 KSP preconditioned resid norm 1.496516824281e-02 true resid norm 1.337998766917e+11 ||r(i)||/||b|| 7.619588823868e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.843201195034e-02 1 KSP Residual norm 1.476997112843e-14 280 KSP preconditioned resid norm 1.418033786605e-02 true resid norm 1.386300123404e+11 ||r(i)||/||b|| 7.894653708208e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.555563116090e-02 1 KSP Residual norm 1.378906607762e-14 281 KSP preconditioned resid norm 1.331743441195e-02 true resid norm 1.444341012579e+11 ||r(i)||/||b|| 8.225182944417e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.574073569061e-02 1 KSP Residual norm 6.115370955524e-15 282 KSP preconditioned resid norm 1.222654523959e-02 true resid norm 1.521352340423e+11 ||r(i)||/||b|| 8.663744374713e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.383344046193e-02 1 KSP Residual norm 7.856819645912e-15 283 KSP preconditioned resid norm 1.131428865743e-02 true resid norm 1.595357992507e+11 ||r(i)||/||b|| 9.085189187270e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.311734144150e-02 1 KSP Residual norm 1.115920122563e-14 284 KSP preconditioned resid norm 1.037046304893e-02 true resid norm 1.680264075889e+11 ||r(i)||/||b|| 9.568709396714e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.592477540254e-02 1 KSP Residual norm 1.156068322093e-14 285 KSP preconditioned resid norm 9.688198762175e-03 true resid norm 1.743227657802e+11 ||r(i)||/||b|| 9.927272212254e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.426922320142e-02 1 KSP Residual norm 3.949284064332e-15 286 KSP preconditioned resid norm 9.052118320697e-03 true resid norm 1.811488172856e+11 ||r(i)||/||b|| 1.031599981835e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.186368600513e-02 1 KSP Residual norm 6.582672988054e-15 287 KSP preconditioned resid norm 8.110683448667e-03 true resid norm 1.926556955885e+11 ||r(i)||/||b|| 1.097128952026e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.262092759332e-01 1 KSP Residual norm 7.546825301272e-15 288 KSP preconditioned resid norm 7.508139123502e-03 true resid norm 2.010760697421e+11 ||r(i)||/||b|| 1.145081005780e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.232480811525e-01 1 KSP Residual norm 7.782003264457e-15 289 KSP preconditioned resid norm 6.917241898136e-03 true resid norm 2.109957958410e+11 ||r(i)||/||b|| 1.201571516824e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.458453127507e-01 1 KSP Residual norm 7.940187999184e-15 290 KSP preconditioned resid norm 6.270807515459e-03 true resid norm 2.235532907593e+11 ||r(i)||/||b|| 1.273083501963e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.819572252522e-01 1 KSP Residual norm 1.436046038556e-14 291 KSP preconditioned resid norm 5.723445930406e-03 true resid norm 2.366865076416e+11 ||r(i)||/||b|| 1.347874088510e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.049783709059e-01 1 KSP Residual norm 1.076610048324e-14 292 KSP preconditioned resid norm 5.384061758061e-03 true resid norm 2.476830100424e+11 ||r(i)||/||b|| 1.410496587773e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.979746680572e-01 1 KSP Residual norm 1.027368530784e-14 293 KSP preconditioned resid norm 5.001851591856e-03 true resid norm 2.646739127192e+11 ||r(i)||/||b|| 1.507255789160e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.598181997627e-01 1 KSP Residual norm 1.148559409323e-14 294 KSP preconditioned resid norm 4.705087086658e-03 true resid norm 2.809591417686e+11 ||r(i)||/||b|| 1.599996344926e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.769227109196e-01 1 KSP Residual norm 1.366070705781e-14 295 KSP preconditioned resid norm 4.424601809451e-03 true resid norm 3.023642732917e+11 ||r(i)||/||b|| 1.721893543160e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.397692724522e-01 1 KSP Residual norm 1.290250727597e-14 296 KSP preconditioned resid norm 4.204816135054e-03 true resid norm 3.239122788695e+11 ||r(i)||/||b|| 1.844604375589e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.716721838185e-01 1 KSP Residual norm 2.596154084612e-14 297 KSP preconditioned resid norm 4.020991299471e-03 true resid norm 3.491083171568e+11 ||r(i)||/||b|| 1.988089897763e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.261036489188e-01 1 KSP Residual norm 1.822196692684e-14 298 KSP preconditioned resid norm 3.962923728563e-03 true resid norm 3.653307902232e+11 ||r(i)||/||b|| 2.080473072941e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.697803614075e-01 1 KSP Residual norm 2.384951372987e-14 299 KSP preconditioned resid norm 3.957387938402e-03 true resid norm 3.700700457571e+11 ||r(i)||/||b|| 2.107462020459e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.629908750446e-01 1 KSP Residual norm 1.183986125549e-14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.422441367284e-03 1 KSP Residual norm 1.636073791245e-16 300 KSP preconditioned resid norm 3.939103706412e-03 true resid norm 3.622465539444e+11 ||r(i)||/||b|| 2.062909071493e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.689675100410e-01 1 KSP Residual norm 4.110406281660e-14 301 KSP preconditioned resid norm 3.936147418317e-03 true resid norm 3.628689205460e+11 ||r(i)||/||b|| 2.066453303161e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.941237217684e-01 1 KSP Residual norm 3.654342959634e-14 302 KSP preconditioned resid norm 3.931462764084e-03 true resid norm 3.639587685780e+11 ||r(i)||/||b|| 2.072659731814e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.714532365736e-01 1 KSP Residual norm 2.465173478608e-14 303 KSP preconditioned resid norm 3.926430936071e-03 true resid norm 3.655824828049e+11 ||r(i)||/||b|| 2.081906403098e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.465762168832e-01 1 KSP Residual norm 1.776189431029e-14 304 KSP preconditioned resid norm 3.926017212455e-03 true resid norm 3.661256993569e+11 ||r(i)||/||b|| 2.084999893818e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.372160993447e-01 1 KSP Residual norm 2.732388886982e-14 305 KSP preconditioned resid norm 3.925345705573e-03 true resid norm 3.649008716223e+11 ||r(i)||/||b|| 2.078024787451e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.757222143455e-01 1 KSP Residual norm 2.703342797030e-14 306 KSP preconditioned resid norm 3.923355352898e-03 true resid norm 3.626445385881e+11 ||r(i)||/||b|| 2.065175500595e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.082277875927e-01 1 KSP Residual norm 4.030968503734e-14 307 KSP preconditioned resid norm 3.917018032874e-03 true resid norm 3.580382977176e+11 ||r(i)||/||b|| 2.038944040353e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.876152800596e-01 1 KSP Residual norm 3.861888673792e-14 308 KSP preconditioned resid norm 3.903310183725e-03 true resid norm 3.514819371400e+11 ||r(i)||/||b|| 2.001607106256e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.350617692262e-01 1 KSP Residual norm 1.714074604894e-14 309 KSP preconditioned resid norm 3.868426817964e-03 true resid norm 3.361175125711e+11 ||r(i)||/||b|| 1.914110315807e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.737802075693e-01 1 KSP Residual norm 1.065291850979e-14 310 KSP preconditioned resid norm 3.830522327434e-03 true resid norm 3.221653934227e+11 ||r(i)||/||b|| 1.834656273127e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.406895635003e-01 1 KSP Residual norm 7.661562948385e-15 311 KSP preconditioned resid norm 3.796427162378e-03 true resid norm 3.115929492422e+11 ||r(i)||/||b|| 1.774448685863e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.156924079072e-01 1 KSP Residual norm 2.624744488131e-14 312 KSP preconditioned resid norm 3.690308765222e-03 true resid norm 2.821289622466e+11 ||r(i)||/||b|| 1.606658197882e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.372604233523e-01 1 KSP Residual norm 5.427199790975e-15 313 KSP preconditioned resid norm 3.593413434155e-03 true resid norm 2.581294623390e+11 ||r(i)||/||b|| 1.469986680841e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.172594571495e-01 1 KSP Residual norm 1.734038988634e-14 314 KSP preconditioned resid norm 3.480212917648e-03 true resid norm 2.333509621663e+11 ||r(i)||/||b|| 1.328878940193e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.030133709721e-01 1 KSP Residual norm 5.340025165053e-15 315 KSP preconditioned resid norm 3.353610727554e-03 true resid norm 2.081539379027e+11 ||r(i)||/||b|| 1.185387803116e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.546183183988e-02 1 KSP Residual norm 9.474779979399e-15 316 KSP preconditioned resid norm 3.194400816588e-03 true resid norm 1.800135422579e+11 ||r(i)||/||b|| 1.025134857107e+09 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.082679756430e-02 1 KSP Residual norm 4.137707563338e-15 317 KSP preconditioned resid norm 3.071660466318e-03 true resid norm 1.608410018372e+11 ||r(i)||/||b|| 9.159517410037e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.986802333543e-02 1 KSP Residual norm 5.891987058382e-15 318 KSP preconditioned resid norm 2.839836959694e-03 true resid norm 1.286425107952e+11 ||r(i)||/||b|| 7.325888945235e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.065370739430e-02 1 KSP Residual norm 8.586015622114e-15 319 KSP preconditioned resid norm 2.631447805934e-03 true resid norm 1.035751986247e+11 ||r(i)||/||b|| 5.898364373600e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.709365439290e-02 1 KSP Residual norm 4.626622373775e-15 320 KSP preconditioned resid norm 2.421698672480e-03 true resid norm 8.242093003462e+10 ||r(i)||/||b|| 4.693678446293e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.533068594810e-02 1 KSP Residual norm 6.352988984759e-15 321 KSP preconditioned resid norm 2.249646162938e-03 true resid norm 6.795739661676e+10 ||r(i)||/||b|| 3.870014177616e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.580381531151e-02 1 KSP Residual norm 5.767752580444e-15 322 KSP preconditioned resid norm 2.051650980233e-03 true resid norm 5.361859971127e+10 ||r(i)||/||b|| 3.053453360445e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.957321613322e-02 1 KSP Residual norm 4.444123762553e-15 323 KSP preconditioned resid norm 1.852638069944e-03 true resid norm 4.140385628580e+10 ||r(i)||/||b|| 2.357852401816e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.032160513475e-02 1 KSP Residual norm 1.437828688493e-14 324 KSP preconditioned resid norm 1.586960792034e-03 true resid norm 2.821637588504e+10 ||r(i)||/||b|| 1.606856356371e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.112779650514e-02 1 KSP Residual norm 5.727563482493e-15 325 KSP preconditioned resid norm 1.397072877149e-03 true resid norm 2.041761460730e+10 ||r(i)||/||b|| 1.162735212606e+08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.763371056357e-02 1 KSP Residual norm 5.639127741743e-15 326 KSP preconditioned resid norm 1.221051611758e-03 true resid norm 1.446657118383e+10 ||r(i)||/||b|| 8.238372623165e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.087280082280e-02 1 KSP Residual norm 1.532244282116e-15 327 KSP preconditioned resid norm 1.033482057986e-03 true resid norm 9.360214184176e+09 ||r(i)||/||b|| 5.330422205925e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.412350779009e-02 1 KSP Residual norm 6.040755977793e-15 328 KSP preconditioned resid norm 9.116882133916e-04 true resid norm 6.661870351799e+09 ||r(i)||/||b|| 3.793778748809e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.694453546376e-02 1 KSP Residual norm 6.632693106872e-15 329 KSP preconditioned resid norm 8.282548180128e-04 true resid norm 5.144717363016e+09 ||r(i)||/||b|| 2.929795743498e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.284103666232e-02 1 KSP Residual norm 9.000495611341e-15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.857159406248e-05 1 KSP Residual norm 1.807976737916e-18 330 KSP preconditioned resid norm 7.530630893338e-04 true resid norm 4.075399789633e+09 ||r(i)||/||b|| 2.320844492363e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.117414233737e-02 1 KSP Residual norm 2.753811269095e-15 331 KSP preconditioned resid norm 7.378843605824e-04 true resid norm 4.106334585676e+09 ||r(i)||/||b|| 2.338461132380e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.058784968427e-01 1 KSP Residual norm 2.806454406138e-14 332 KSP preconditioned resid norm 7.208109673083e-04 true resid norm 4.143108086041e+09 ||r(i)||/||b|| 2.359402777419e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.042506304248e-01 1 KSP Residual norm 3.102912293216e-14 333 KSP preconditioned resid norm 7.065440015463e-04 true resid norm 4.177434934991e+09 ||r(i)||/||b|| 2.378951111923e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.422215659160e-01 1 KSP Residual norm 2.501684006696e-14 334 KSP preconditioned resid norm 6.922130798635e-04 true resid norm 4.218518046304e+09 ||r(i)||/||b|| 2.402346979210e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.637430765000e-02 1 KSP Residual norm 2.696302183947e-14 335 KSP preconditioned resid norm 6.743986454519e-04 true resid norm 4.282620435472e+09 ||r(i)||/||b|| 2.438851784758e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.564667165661e-02 1 KSP Residual norm 2.321690857301e-14 336 KSP preconditioned resid norm 6.468451089725e-04 true resid norm 4.403779004208e+09 ||r(i)||/||b|| 2.507848744926e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.229076988553e-01 1 KSP Residual norm 3.248769277105e-14 337 KSP preconditioned resid norm 6.112383860139e-04 true resid norm 4.572889268801e+09 ||r(i)||/||b|| 2.604153070008e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.020334312596e-02 1 KSP Residual norm 2.060848308948e-14 338 KSP preconditioned resid norm 5.788902799897e-04 true resid norm 4.751423567671e+09 ||r(i)||/||b|| 2.705824161341e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.292620453901e-02 1 KSP Residual norm 1.569420098068e-14 339 KSP preconditioned resid norm 5.410935892311e-04 true resid norm 4.973526607240e+09 ||r(i)||/||b|| 2.832306627535e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.732626069050e-02 1 KSP Residual norm 1.904390383312e-14 340 KSP preconditioned resid norm 5.009948786042e-04 true resid norm 5.234837867811e+09 ||r(i)||/||b|| 2.981117255006e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.824656168048e-02 1 KSP Residual norm 1.465230609570e-14 341 KSP preconditioned resid norm 4.659625875994e-04 true resid norm 5.473363530127e+09 ||r(i)||/||b|| 3.116952019262e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.680257126080e-02 1 KSP Residual norm 1.499795900289e-14 342 KSP preconditioned resid norm 4.188737491331e-04 true resid norm 5.804840588010e+09 ||r(i)||/||b|| 3.305720420853e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.663881661976e-02 1 KSP Residual norm 1.348626999566e-14 343 KSP preconditioned resid norm 3.888174107377e-04 true resid norm 6.055882640284e+09 ||r(i)||/||b|| 3.448682975313e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.550588930007e-02 1 KSP Residual norm 5.810506118595e-15 344 KSP preconditioned resid norm 3.757521942558e-04 true resid norm 6.184885124251e+09 ||r(i)||/||b|| 3.522146861034e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.699014290239e-02 1 KSP Residual norm 1.512483488204e-14 345 KSP preconditioned resid norm 3.517425591232e-04 true resid norm 6.438959263708e+09 ||r(i)||/||b|| 3.666836117954e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.581820536421e-02 1 KSP Residual norm 1.297802479441e-14 346 KSP preconditioned resid norm 3.188163998220e-04 true resid norm 6.808945943561e+09 ||r(i)||/||b|| 3.877534845075e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.112282608412e-01 1 KSP Residual norm 8.934160118744e-15 347 KSP preconditioned resid norm 2.976271547968e-04 true resid norm 7.089267610639e+09 ||r(i)||/||b|| 4.037171452698e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.097268867979e-01 1 KSP Residual norm 1.346905677201e-14 348 KSP preconditioned resid norm 2.760598176598e-04 true resid norm 7.404379197336e+09 ||r(i)||/||b|| 4.216620102700e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.275714537379e-01 1 KSP Residual norm 1.393995566652e-14 349 KSP preconditioned resid norm 2.491152197103e-04 true resid norm 7.856285580077e+09 ||r(i)||/||b|| 4.473970177192e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.636809622968e-01 1 KSP Residual norm 6.528601051804e-15 350 KSP preconditioned resid norm 2.261875421951e-04 true resid norm 8.300044966044e+09 ||r(i)||/||b|| 4.726680728308e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.844767798499e-01 1 KSP Residual norm 9.965383200474e-15 351 KSP preconditioned resid norm 2.108222499389e-04 true resid norm 8.660693905734e+09 ||r(i)||/||b|| 4.932061831651e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.804402847968e-01 1 KSP Residual norm 1.044990721389e-14 352 KSP preconditioned resid norm 1.952757682812e-04 true resid norm 9.171398383610e+09 ||r(i)||/||b|| 5.222896040780e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.243993926974e-01 1 KSP Residual norm 1.060036179430e-14 353 KSP preconditioned resid norm 1.857516307760e-04 true resid norm 9.612319190889e+09 ||r(i)||/||b|| 5.473990087981e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.169232562665e-01 1 KSP Residual norm 7.847038296565e-15 354 KSP preconditioned resid norm 1.755630407243e-04 true resid norm 1.022795215943e+10 ||r(i)||/||b|| 5.824578608891e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.706403088480e-01 1 KSP Residual norm 1.385640888601e-14 355 KSP preconditioned resid norm 1.629837424048e-04 true resid norm 1.128528581762e+10 ||r(i)||/||b|| 6.426705301698e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.864462189446e-01 1 KSP Residual norm 1.750159592573e-14 356 KSP preconditioned resid norm 1.523697398411e-04 true resid norm 1.237194867639e+10 ||r(i)||/||b|| 7.045534285603e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.396910599064e-01 1 KSP Residual norm 1.558489229106e-14 357 KSP preconditioned resid norm 1.479102506497e-04 true resid norm 1.303733768943e+10 ||r(i)||/||b|| 7.424457705610e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.704655085365e-01 1 KSP Residual norm 2.552079543082e-14 358 KSP preconditioned resid norm 1.460096349773e-04 true resid norm 1.352450072155e+10 ||r(i)||/||b|| 7.701885614123e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.330524337806e-01 1 KSP Residual norm 1.249387256428e-14 359 KSP preconditioned resid norm 1.458849827096e-04 true resid norm 1.366141869934e+10 ||r(i)||/||b|| 7.779857187726e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.506916160917e-01 1 KSP Residual norm 1.241916700050e-14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.271223582046e-04 1 KSP Residual norm 6.097927485411e-18 360 KSP preconditioned resid norm 1.455026626572e-04 true resid norm 1.345524299294e+10 ||r(i)||/||b|| 7.662444963809e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.737717097093e-01 1 KSP Residual norm 3.950702964754e-14 361 KSP preconditioned resid norm 1.453653589202e-04 true resid norm 1.349948577101e+10 ||r(i)||/||b|| 7.687640187127e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.329927151475e-01 1 KSP Residual norm 1.442345915700e-14 362 KSP preconditioned resid norm 1.452932335715e-04 true resid norm 1.352941874316e+10 ||r(i)||/||b|| 7.704686311958e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.848191576197e-01 1 KSP Residual norm 2.318196826569e-14 363 KSP preconditioned resid norm 1.452077085053e-04 true resid norm 1.358040481119e+10 ||r(i)||/||b|| 7.733721680577e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.838397768184e-01 1 KSP Residual norm 3.018720109462e-14 364 KSP preconditioned resid norm 1.452068424347e-04 true resid norm 1.358658974421e+10 ||r(i)||/||b|| 7.737243854715e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.893211055741e-01 1 KSP Residual norm 2.205052232884e-14 365 KSP preconditioned resid norm 1.451988195240e-04 true resid norm 1.356415073143e+10 ||r(i)||/||b|| 7.724465363790e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.985197960881e-01 1 KSP Residual norm 3.139653106468e-14 366 KSP preconditioned resid norm 1.450322012808e-04 true resid norm 1.341939854662e+10 ||r(i)||/||b|| 7.642032393234e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.767284477804e-01 1 KSP Residual norm 2.500766293079e-14 367 KSP preconditioned resid norm 1.448660392702e-04 true resid norm 1.330873409029e+10 ||r(i)||/||b|| 7.579011583687e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.340344154347e-01 1 KSP Residual norm 3.224068360290e-14 368 KSP preconditioned resid norm 1.440561644899e-04 true resid norm 1.296085311722e+10 ||r(i)||/||b|| 7.380901537549e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.431596358775e-01 1 KSP Residual norm 1.982158693092e-14 369 KSP preconditioned resid norm 1.424259451750e-04 true resid norm 1.234613349126e+10 ||r(i)||/||b|| 7.030833143798e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.579827213754e-01 1 KSP Residual norm 2.479550477092e-14 370 KSP preconditioned resid norm 1.405134706181e-04 true resid norm 1.171425599420e+10 ||r(i)||/||b|| 6.670993745307e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.368445142149e-01 1 KSP Residual norm 9.282781706174e-15 371 KSP preconditioned resid norm 1.384045644474e-04 true resid norm 1.108995050694e+10 ||r(i)||/||b|| 6.315466428613e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.249655070777e-01 1 KSP Residual norm 1.216181515764e-14 372 KSP preconditioned resid norm 1.349464774298e-04 true resid norm 1.017989050712e+10 ||r(i)||/||b|| 5.797208626355e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.225625228406e-01 1 KSP Residual norm 1.648618859724e-14 373 KSP preconditioned resid norm 1.299650235031e-04 true resid norm 9.022746031412e+09 ||r(i)||/||b|| 5.138242016467e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.228291793551e-01 1 KSP Residual norm 1.862062891308e-14 374 KSP preconditioned resid norm 1.244499402556e-04 true resid norm 7.872007787314e+09 ||r(i)||/||b|| 4.482923605066e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.134427886212e-01 1 KSP Residual norm 1.107258931901e-14 375 KSP preconditioned resid norm 1.196895128318e-04 true resid norm 6.960466866168e+09 ||r(i)||/||b|| 3.963822452883e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.950649015460e-02 1 KSP Residual norm 9.566681063980e-15 376 KSP preconditioned resid norm 1.150083249018e-04 true resid norm 6.147642417833e+09 ||r(i)||/||b|| 3.500938014165e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.775509097828e-02 1 KSP Residual norm 3.520658949477e-15 377 KSP preconditioned resid norm 1.078871827785e-04 true resid norm 5.102194105304e+09 ||r(i)||/||b|| 2.905579746650e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.068583860496e-02 1 KSP Residual norm 5.813080004255e-15 378 KSP preconditioned resid norm 1.020385331017e-04 true resid norm 4.355003319652e+09 ||r(i)||/||b|| 2.480072137793e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.338182300117e-02 1 KSP Residual norm 7.875764136671e-15 379 KSP preconditioned resid norm 9.619002775224e-05 true resid norm 3.697867140899e+09 ||r(i)||/||b|| 2.105848513139e+07 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.881890909876e-02 1 KSP Residual norm 4.836863157033e-15 380 KSP preconditioned resid norm 9.035467035326e-05 true resid norm 3.118903114672e+09 ||r(i)||/||b|| 1.776142093915e+07 KSP Object: 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-10, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, blocksize = 1, factorization LOWER Preconditioner for the Schur complement formed from S itself Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_0_) 4 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_1_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_) 4 MPI processes type: lsc KSP Object: (fieldsplit_1_lsc_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 package used to perform factorization: petsc total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=9549, allocated nonzeros=9549 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines linear system matrix = precond matrix: Mat Object: (fieldsplit_1_) 4 MPI processes type: schurcomplement rows=3183, cols=3183 Schur complement A11 - A10 inv(A00) A01 A11 Mat Object: (fieldsplit_1_) 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=3183, allocated nonzeros=3183 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines A10 Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=71946 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines KSP of A00 KSP Object: (fieldsplit_0_) 4 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 matrix ordering: natural factor fill ratio given 0, needed 0 Factored matrix follows: Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=71946 package used to perform factorization: mumps total: nonzeros=6.47715e+07, allocated nonzeros=6.47715e+07 total number of mallocs used during MatSetValues calls =0 MUMPS run parameters: SYM (matrix type): 0 PAR (host participation): 1 ICNTL(1) (output for error): 6 ICNTL(2) (output of diagnostic msg): 0 ICNTL(3) (output for global info): 0 ICNTL(4) (level of printing): 0 ICNTL(5) (input mat struct): 0 ICNTL(6) (matrix prescaling): 7 ICNTL(7) (sequentia matrix ordering):7 ICNTL(8) (scalling strategy): 77 ICNTL(10) (max num of refinements): 0 ICNTL(11) (error analysis): 0 ICNTL(12) (efficiency control): 1 ICNTL(13) (efficiency control): 0 ICNTL(14) (percentage of estimated workspace increase): 20 ICNTL(18) (input mat struct): 3 ICNTL(19) (Shur complement info): 0 ICNTL(20) (rhs sparse pattern): 0 ICNTL(21) (somumpstion struct): 1 ICNTL(22) (in-core/out-of-core facility): 0 ICNTL(23) (max size of memory can be allocated locally):0 ICNTL(24) (detection of null pivot rows): 0 ICNTL(25) (computation of a null space basis): 0 ICNTL(26) (Schur options for rhs or solution): 0 ICNTL(27) (experimental parameter): -8 ICNTL(28) (use parallel or sequential ordering): 1 ICNTL(29) (parallel ordering): 0 ICNTL(30) (user-specified set of entries in inv(A)): 0 ICNTL(31) (factors is discarded in the solve phase): 0 ICNTL(33) (compute determinant): 0 CNTL(1) (relative pivoting threshold): 0.01 CNTL(2) (stopping criterion of refinement): 1.49012e-08 CNTL(3) (absomumpste pivoting threshold): 0 CNTL(4) (vamumpse of static pivoting): -1 CNTL(5) (fixation for null pivots): 0 RINFO(1) (local estimated flops for the elimination after analysis): [0] 1.6957e+10 [1] 1.67964e+10 [2] 1.70959e+10 [3] 1.7073e+10 RINFO(2) (local estimated flops for the assembly after factorization): [0] 3.99956e+07 [1] 3.71513e+07 [2] 4.43658e+07 [3] 2.95262e+07 RINFO(3) (local estimated flops for the elimination after factorization): [0] 1.732e+10 [1] 1.7234e+10 [2] 1.70959e+10 [3] 1.62723e+10 INFO(15) (estimated size of (in MB) MUMPS internal data for running numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(16) (size of (in MB) MUMPS internal data used during numerical factorization): [0] 243 [1] 237 [2] 256 [3] 212 INFO(23) (num of pivots eliminated on this processor after factorization): [0] 23748 [1] 15630 [2] 20355 [3] 12213 RINFOG(1) (global estimated flops for the elimination after analysis): 6.79222e+10 RINFOG(2) (global estimated flops for the assembly after factorization): 1.51039e+08 RINFOG(3) (global estimated flops for the elimination after factorization): 6.79222e+10 (RINFOG(12) RINFOG(13))*2^INFOG(34) (determinant): (0,0)*(2^0) INFOG(3) (estimated real workspace for factors on all processors after analysis): 64771542 INFOG(4) (estimated integer workspace for factors on all processors after analysis): 1209207 INFOG(5) (estimated maximum front size in the complete tree): 2562 INFOG(6) (number of nodes in the complete tree): 3832 INFOG(7) (ordering option effectively use after analysis): 5 INFOG(8) (structural symmetry in percent of the permuted matrix after analysis): 100 INFOG(9) (total real/complex workspace to store the matrix factors after factorization): 64771542 INFOG(10) (total integer space store the matrix factors after factorization): 1208659 INFOG(11) (order of largest frontal matrix after factorization): 2562 INFOG(12) (number of off-diagonal pivots): 0 INFOG(13) (number of delayed pivots after factorization): 0 INFOG(14) (number of memory compress after factorization): 0 INFOG(15) (number of steps of iterative refinement after solution): 0 INFOG(16) (estimated size (in MB) of all MUMPS internal data for factorization after analysis: value on the most memory consuming processor): 256 INFOG(17) (estimated size of all MUMPS internal data for factorization after analysis: sum over all processors): 948 INFOG(18) (size of all MUMPS internal data allocated during factorization: value on the most memory consuming processor): 256 INFOG(19) (size of all MUMPS internal data allocated during factorization: sum over all processors): 948 INFOG(20) (estimated number of entries in the factors): 64771542 INFOG(21) (size in MB of memory effectively used during factorization - value on the most memory consuming processor): 212 INFOG(22) (size in MB of memory effectively used during factorization - sum over all processors): 789 INFOG(23) (after analysis: value of ICNTL(6) effectively used): 0 INFOG(24) (after analysis: value of ICNTL(12) effectively used): 1 INFOG(25) (after factorization: number of pivots modified by static pivoting): 0 INFOG(28) (after factorization: number of null pivots encountered): 0 INFOG(29) (after factorization: effective number of entries in the factors (sum over all processors)): 58207698 INFOG(30, 31) (after solution: size in Mbytes of memory used during solution phase): 229, 844 INFOG(32) (after analysis: type of analysis done): 1 INFOG(33) (value used for ICNTL(8)): 7 INFOG(34) (exponent of the determinant if determinant is requested): 0 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 A01 Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=3183 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 3757 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=75129, cols=75129 total: nonzeros=3.0016e+06, allocated nonzeros=1.26217e+07 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Recovering stress ... Cleaning up ... Finished 0.01user 0.03system 0:40.70elapsed 0%CPU (0avgtext+0avgdata 17056maxresident)k 904inputs+0outputs (3major+1802minor)pagefaults 0swaps salloc: Relinquishing job allocation 79009 -------------- next part -------------- salloc: Granted job allocation 79008 Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 1 KSP preconditioned resid norm 7.598278213552e+05 true resid norm 6.423512943905e+15 ||r(i)||/||b|| 3.658039801496e+13 2 KSP preconditioned resid norm 4.421363398396e+05 true resid norm 1.938122873265e+16 ||r(i)||/||b|| 1.103715470414e+14 3 KSP preconditioned resid norm 3.187714266989e+05 true resid norm 2.595569488051e+16 ||r(i)||/||b|| 1.478115881100e+14 4 KSP preconditioned resid norm 2.009250685731e+05 true resid norm 4.269068986797e+16 ||r(i)||/||b|| 2.431134552917e+14 5 KSP preconditioned resid norm 1.299557266250e+05 true resid norm 5.664566692593e+16 ||r(i)||/||b|| 3.225837730956e+14 6 KSP preconditioned resid norm 1.070108696541e+05 true resid norm 6.757083396954e+16 ||r(i)||/||b|| 3.848000342482e+14 7 KSP preconditioned resid norm 9.205344823934e+04 true resid norm 8.073944995171e+16 ||r(i)||/||b|| 4.597922103582e+14 8 KSP preconditioned resid norm 8.698599347251e+04 true resid norm 9.596166057921e+16 ||r(i)||/||b|| 5.464791258021e+14 9 KSP preconditioned resid norm 8.674528784539e+04 true resid norm 1.014395579435e+17 ||r(i)||/||b|| 5.776744651155e+14 10 KSP preconditioned resid norm 7.633154167998e+04 true resid norm 5.663942256556e+16 ||r(i)||/||b|| 3.225482129294e+14 11 KSP preconditioned resid norm 5.726963014918e+04 true resid norm 3.255239259030e+16 ||r(i)||/||b|| 1.853782327040e+14 12 KSP preconditioned resid norm 3.933249871854e+04 true resid norm 5.893837891722e+16 ||r(i)||/||b|| 3.356402295716e+14 13 KSP preconditioned resid norm 2.986641999324e+04 true resid norm 7.405028344525e+16 ||r(i)||/||b|| 4.216989776782e+14 14 KSP preconditioned resid norm 2.633745666937e+04 true resid norm 8.472086806682e+16 ||r(i)||/||b|| 4.824654517117e+14 15 KSP preconditioned resid norm 2.608339084208e+04 true resid norm 8.969329343812e+16 ||r(i)||/||b|| 5.107822467070e+14 16 KSP preconditioned resid norm 2.488827926174e+04 true resid norm 7.386717952238e+16 ||r(i)||/||b|| 4.206562438291e+14 17 KSP preconditioned resid norm 2.127496521837e+04 true resid norm 3.884168517181e+16 ||r(i)||/||b|| 2.211942772692e+14 18 KSP preconditioned resid norm 1.563809118446e+04 true resid norm 2.054810743673e+16 ||r(i)||/||b|| 1.170166472853e+14 19 KSP preconditioned resid norm 1.136562830178e+04 true resid norm 4.048500849886e+16 ||r(i)||/||b|| 2.305526177747e+14 20 KSP preconditioned resid norm 9.662175529016e+03 true resid norm 5.237903358199e+16 ||r(i)||/||b|| 2.982862979806e+14 21 KSP preconditioned resid norm 9.530658333081e+03 true resid norm 5.684529709698e+16 ||r(i)||/||b|| 3.237206200478e+14 22 KSP preconditioned resid norm 9.299377613844e+03 true resid norm 4.717621951212e+16 ||r(i)||/||b|| 2.686574934409e+14 23 KSP preconditioned resid norm 7.559054781177e+03 true resid norm 1.967879836939e+16 ||r(i)||/||b|| 1.120661362551e+14 24 KSP preconditioned resid norm 5.773171779442e+03 true resid norm 1.503038207188e+16 ||r(i)||/||b|| 8.559449685985e+13 25 KSP preconditioned resid norm 4.675136883937e+03 true resid norm 3.083933334216e+16 ||r(i)||/||b|| 1.756227625014e+14 26 KSP preconditioned resid norm 4.619378203964e+03 true resid norm 3.484189753965e+16 ||r(i)||/||b|| 1.984164258291e+14 27 KSP preconditioned resid norm 4.393843972597e+03 true resid norm 2.542715244756e+16 ||r(i)||/||b|| 1.448016630528e+14 28 KSP preconditioned resid norm 3.659887718417e+03 true resid norm 6.968079702291e+15 ||r(i)||/||b|| 3.968157784311e+13 29 KSP preconditioned resid norm 2.468848759677e+03 true resid norm 1.079956395636e+16 ||r(i)||/||b|| 6.150098106151e+13 30 KSP preconditioned resid norm 1.864796910441e+03 true resid norm 1.493669666429e+16 ||r(i)||/||b|| 8.506098046037e+13 31 KSP preconditioned resid norm 1.851274436483e+03 true resid norm 1.517346418548e+16 ||r(i)||/||b|| 8.640931590201e+13 32 KSP preconditioned resid norm 1.844524113469e+03 true resid norm 1.551800988833e+16 ||r(i)||/||b|| 8.837142278256e+13 33 KSP preconditioned resid norm 1.843313509513e+03 true resid norm 1.522935935101e+16 ||r(i)||/||b|| 8.672762574585e+13 34 KSP preconditioned resid norm 1.818448641203e+03 true resid norm 1.340161908290e+16 ||r(i)||/||b|| 7.631907406096e+13 35 KSP preconditioned resid norm 1.761501351737e+03 true resid norm 9.701610689032e+15 ||r(i)||/||b|| 5.524839499665e+13 36 KSP preconditioned resid norm 1.587903182351e+03 true resid norm 3.006466867338e+15 ||r(i)||/||b|| 1.712112290991e+13 37 KSP preconditioned resid norm 1.424674192381e+03 true resid norm 3.330040454395e+15 ||r(i)||/||b|| 1.896379851515e+13 38 KSP preconditioned resid norm 1.223589331013e+03 true resid norm 7.090044535635e+15 ||r(i)||/||b|| 4.037613893242e+13 39 KSP preconditioned resid norm 1.217042925271e+03 true resid norm 7.803451047400e+15 ||r(i)||/||b|| 4.443882151354e+13 40 KSP preconditioned resid norm 1.210937465891e+03 true resid norm 6.896760418758e+15 ||r(i)||/||b|| 3.927543126871e+13 41 KSP preconditioned resid norm 1.139515965864e+03 true resid norm 4.397710228518e+15 ||r(i)||/||b|| 2.504392719661e+13 42 KSP preconditioned resid norm 9.844313363552e+02 true resid norm 2.063347155848e+15 ||r(i)||/||b|| 1.175027759158e+13 43 KSP preconditioned resid norm 8.134473256619e+02 true resid norm 3.162170274309e+15 ||r(i)||/||b|| 1.800781725443e+13 44 KSP preconditioned resid norm 7.167053701822e+02 true resid norm 6.294910273859e+15 ||r(i)||/||b|| 3.584803600415e+13 45 KSP preconditioned resid norm 6.962717083756e+02 true resid norm 6.711029585626e+15 ||r(i)||/||b|| 3.821773778245e+13 46 KSP preconditioned resid norm 6.927406658772e+02 true resid norm 6.102891293465e+15 ||r(i)||/||b|| 3.475453299566e+13 47 KSP preconditioned resid norm 6.104613558502e+02 true resid norm 1.348418985388e+15 ||r(i)||/||b|| 7.678929521455e+12 48 KSP preconditioned resid norm 4.947332360362e+02 true resid norm 3.525127520461e+15 ||r(i)||/||b|| 2.007477355118e+13 49 KSP preconditioned resid norm 4.128569311177e+02 true resid norm 7.381606168160e+15 ||r(i)||/||b|| 4.203651397281e+13 50 KSP preconditioned resid norm 4.128555858135e+02 true resid norm 7.386790205685e+15 ||r(i)||/||b|| 4.206603584933e+13 51 KSP preconditioned resid norm 3.995142906102e+02 true resid norm 5.174163353743e+15 ||r(i)||/||b|| 2.946564543844e+13 52 KSP preconditioned resid norm 3.398724348924e+02 true resid norm 1.795681743007e+15 ||r(i)||/||b|| 1.022598591161e+13 53 KSP preconditioned resid norm 2.582635865429e+02 true resid norm 1.065592994159e+15 ||r(i)||/||b|| 6.068301907178e+12 54 KSP preconditioned resid norm 1.942336466476e+02 true resid norm 2.315650760727e+15 ||r(i)||/||b|| 1.318708738205e+13 55 KSP preconditioned resid norm 1.780428610647e+02 true resid norm 2.677037580411e+15 ||r(i)||/||b|| 1.524510046879e+13 56 KSP preconditioned resid norm 1.700699391492e+02 true resid norm 3.421212998216e+15 ||r(i)||/||b|| 1.948300474547e+13 57 KSP preconditioned resid norm 1.696493267328e+02 true resid norm 3.269908568092e+15 ||r(i)||/||b|| 1.862136154124e+13 58 KSP preconditioned resid norm 1.506929748059e+02 true resid norm 1.808390502696e+15 ||r(i)||/||b|| 1.029835931410e+13 59 KSP preconditioned resid norm 1.240787458056e+02 true resid norm 6.663534486569e+14 ||r(i)||/||b|| 3.794726434487e+12 60 KSP preconditioned resid norm 1.157829983215e+02 true resid norm 1.130780847653e+15 ||r(i)||/||b|| 6.439531427128e+12 61 KSP preconditioned resid norm 1.157796703883e+02 true resid norm 1.131509662988e+15 ||r(i)||/||b|| 6.443681859341e+12 62 KSP preconditioned resid norm 1.157785293613e+02 true resid norm 1.132964508976e+15 ||r(i)||/||b|| 6.451966865653e+12 63 KSP preconditioned resid norm 1.157199903582e+02 true resid norm 1.115006939543e+15 ||r(i)||/||b|| 6.349702724058e+12 64 KSP preconditioned resid norm 1.146344365975e+02 true resid norm 9.897225252928e+14 ||r(i)||/||b|| 5.636237400897e+12 65 KSP preconditioned resid norm 1.099927272891e+02 true resid norm 6.314461681629e+14 ||r(i)||/||b|| 3.595937668085e+12 66 KSP preconditioned resid norm 1.059910545350e+02 true resid norm 4.913476628990e+14 ||r(i)||/||b|| 2.798109574858e+12 67 KSP preconditioned resid norm 1.029682260529e+02 true resid norm 5.270094367253e+14 ||r(i)||/||b|| 3.001195003638e+12 68 KSP preconditioned resid norm 1.023121023363e+02 true resid norm 5.711176277756e+14 ||r(i)||/||b|| 3.252380795342e+12 69 KSP preconditioned resid norm 1.022818674567e+02 true resid norm 5.528815353501e+14 ||r(i)||/||b|| 3.148530530699e+12 70 KSP preconditioned resid norm 1.016711964340e+02 true resid norm 4.892143108744e+14 ||r(i)||/||b|| 2.785960635976e+12 71 KSP preconditioned resid norm 9.783670449849e+01 true resid norm 8.075062190937e+14 ||r(i)||/||b|| 4.598558320339e+12 72 KSP preconditioned resid norm 9.297125493721e+01 true resid norm 1.418932304648e+15 ||r(i)||/||b|| 8.080486318556e+12 73 KSP preconditioned resid norm 9.024342817965e+01 true resid norm 1.822117383425e+15 ||r(i)||/||b|| 1.037653067686e+13 74 KSP preconditioned resid norm 9.022915567731e+01 true resid norm 1.793734928753e+15 ||r(i)||/||b|| 1.021489926153e+13 75 KSP preconditioned resid norm 8.654020807346e+01 true resid norm 1.378342523434e+15 ||r(i)||/||b|| 7.849337044771e+12 76 KSP preconditioned resid norm 7.824470332642e+01 true resid norm 6.887264923035e+14 ||r(i)||/||b|| 3.922135665005e+12 77 KSP preconditioned resid norm 6.933902305081e+01 true resid norm 4.001788567778e+14 ||r(i)||/||b|| 2.278924629862e+12 78 KSP preconditioned resid norm 6.111029596781e+01 true resid norm 8.990216751018e+14 ||r(i)||/||b|| 5.119717355050e+12 79 KSP preconditioned resid norm 6.052457451972e+01 true resid norm 1.038799252649e+15 ||r(i)||/||b|| 5.915717840284e+12 80 KSP preconditioned resid norm 5.982916701975e+01 true resid norm 8.122329384891e+14 ||r(i)||/||b|| 4.625475877492e+12 81 KSP preconditioned resid norm 5.472847286515e+01 true resid norm 3.261724168907e+14 ||r(i)||/||b|| 1.857475330953e+12 82 KSP preconditioned resid norm 4.705564259934e+01 true resid norm 3.873293003694e+14 ||r(i)||/||b|| 2.205749423112e+12 83 KSP preconditioned resid norm 3.875293844888e+01 true resid norm 9.915723130017e+14 ||r(i)||/||b|| 5.646771507580e+12 84 KSP preconditioned resid norm 3.752007823180e+01 true resid norm 1.183842304947e+15 ||r(i)||/||b|| 6.741703967917e+12 85 KSP preconditioned resid norm 3.751814586336e+01 true resid norm 1.191376622304e+15 ||r(i)||/||b|| 6.784610136259e+12 86 KSP preconditioned resid norm 3.545906287093e+01 true resid norm 8.010012535302e+14 ||r(i)||/||b|| 4.561514068780e+12 87 KSP preconditioned resid norm 3.016927381502e+01 true resid norm 2.096547884375e+14 ||r(i)||/||b|| 1.193934794521e+12 88 KSP preconditioned resid norm 2.595797020041e+01 true resid norm 5.781551449568e+14 ||r(i)||/||b|| 3.292457803325e+12 89 KSP preconditioned resid norm 2.431814395444e+01 true resid norm 8.373221520170e+14 ||r(i)||/||b|| 4.768353057743e+12 90 KSP preconditioned resid norm 2.419007128340e+01 true resid norm 9.458145602756e+14 ||r(i)||/||b|| 5.386191849438e+12 91 KSP preconditioned resid norm 2.418985479856e+01 true resid norm 9.456196991434e+14 ||r(i)||/||b|| 5.385082161043e+12 92 KSP preconditioned resid norm 2.417422094832e+01 true resid norm 9.338606389867e+14 ||r(i)||/||b|| 5.318117074404e+12 93 KSP preconditioned resid norm 2.399682638989e+01 true resid norm 8.239131879372e+14 ||r(i)||/||b|| 4.691992155650e+12 94 KSP preconditioned resid norm 2.347990061529e+01 true resid norm 5.758811247477e+14 ||r(i)||/||b|| 3.279507792159e+12 95 KSP preconditioned resid norm 2.317576427891e+01 true resid norm 4.648997976941e+14 ||r(i)||/||b|| 2.647495192309e+12 96 KSP preconditioned resid norm 2.257226431826e+01 true resid norm 2.698576382857e+14 ||r(i)||/||b|| 1.536775889155e+12 97 KSP preconditioned resid norm 2.214410432961e+01 true resid norm 2.012456758637e+14 ||r(i)||/||b|| 1.146046872820e+12 98 KSP preconditioned resid norm 2.213931822377e+01 true resid norm 2.023089301094e+14 ||r(i)||/||b|| 1.152101856104e+12 99 KSP preconditioned resid norm 2.208588382827e+01 true resid norm 2.030851676254e+14 ||r(i)||/||b|| 1.156522346502e+12 100 KSP preconditioned resid norm 2.202243648243e+01 true resid norm 2.215590489413e+14 ||r(i)||/||b|| 1.261726763045e+12 101 KSP preconditioned resid norm 2.168083689592e+01 true resid norm 2.512890878838e+14 ||r(i)||/||b|| 1.431032354396e+12 102 KSP preconditioned resid norm 2.168020292686e+01 true resid norm 2.479141826676e+14 ||r(i)||/||b|| 1.411813061596e+12 103 KSP preconditioned resid norm 2.157247007086e+01 true resid norm 2.214827950420e+14 ||r(i)||/||b|| 1.261292514992e+12 104 KSP preconditioned resid norm 2.084234283050e+01 true resid norm 1.903742949023e+14 ||r(i)||/||b|| 1.084136910777e+12 105 KSP preconditioned resid norm 1.953449641542e+01 true resid norm 2.170389187419e+14 ||r(i)||/||b|| 1.235985682857e+12 106 KSP preconditioned resid norm 1.807383475067e+01 true resid norm 4.030037324419e+14 ||r(i)||/||b|| 2.295011633506e+12 107 KSP preconditioned resid norm 1.608363181466e+01 true resid norm 6.615648433348e+14 ||r(i)||/||b|| 3.767456451513e+12 108 KSP preconditioned resid norm 1.540664146839e+01 true resid norm 8.161690838364e+14 ||r(i)||/||b|| 4.647891301062e+12 109 KSP preconditioned resid norm 1.539806765978e+01 true resid norm 8.031109823862e+14 ||r(i)||/||b|| 4.573528479264e+12 110 KSP preconditioned resid norm 1.500376511467e+01 true resid norm 6.721986290227e+14 ||r(i)||/||b|| 3.828013364259e+12 111 KSP preconditioned resid norm 1.422725209437e+01 true resid norm 3.479679053556e+14 ||r(i)||/||b|| 1.981595520317e+12 112 KSP preconditioned resid norm 1.379945740031e+01 true resid norm 2.034814218620e+14 ||r(i)||/||b|| 1.158778921342e+12 113 KSP preconditioned resid norm 1.363797458150e+01 true resid norm 1.739177470708e+14 ||r(i)||/||b|| 9.904207347713e+11 114 KSP preconditioned resid norm 1.358289054680e+01 true resid norm 1.810806633853e+14 ||r(i)||/||b|| 1.031211861374e+12 115 KSP preconditioned resid norm 1.281735475027e+01 true resid norm 3.060385425933e+14 ||r(i)||/||b|| 1.742817644137e+12 116 KSP preconditioned resid norm 1.199583599528e+01 true resid norm 5.270983461652e+14 ||r(i)||/||b|| 3.001701322023e+12 117 KSP preconditioned resid norm 1.174988815172e+01 true resid norm 5.706455305712e+14 ||r(i)||/||b|| 3.249692312609e+12 118 KSP preconditioned resid norm 1.163364583819e+01 true resid norm 5.557577297715e+14 ||r(i)||/||b|| 3.164909782617e+12 119 KSP preconditioned resid norm 1.150734102719e+01 true resid norm 4.444630681836e+14 ||r(i)||/||b|| 2.531112816163e+12 120 KSP preconditioned resid norm 1.089200826845e+01 true resid norm 1.939041645161e+14 ||r(i)||/||b|| 1.104238689437e+12 121 KSP preconditioned resid norm 1.075988028467e+01 true resid norm 1.998888904135e+14 ||r(i)||/||b|| 1.138320298246e+12 122 KSP preconditioned resid norm 1.064140507137e+01 true resid norm 2.105261696882e+14 ||r(i)||/||b|| 1.198897106149e+12 123 KSP preconditioned resid norm 1.052013508415e+01 true resid norm 2.341474414652e+14 ||r(i)||/||b|| 1.333414702793e+12 124 KSP preconditioned resid norm 1.051711998999e+01 true resid norm 2.302186000312e+14 ||r(i)||/||b|| 1.311040873293e+12 125 KSP preconditioned resid norm 1.043636031947e+01 true resid norm 2.126432353421e+14 ||r(i)||/||b|| 1.210953297974e+12 126 KSP preconditioned resid norm 1.034172555340e+01 true resid norm 1.912541379018e+14 ||r(i)||/||b|| 1.089147410078e+12 127 KSP preconditioned resid norm 1.018172019219e+01 true resid norm 1.659024395687e+14 ||r(i)||/||b|| 9.447754404906e+11 128 KSP preconditioned resid norm 9.985693057631e+00 true resid norm 1.356785077811e+14 ||r(i)||/||b|| 7.726572453501e+11 129 KSP preconditioned resid norm 9.979396625910e+00 true resid norm 1.307609335369e+14 ||r(i)||/||b|| 7.446528146452e+11 130 KSP preconditioned resid norm 9.931647767858e+00 true resid norm 1.472008427511e+14 ||r(i)||/||b|| 8.382742376323e+11 131 KSP preconditioned resid norm 9.849459584026e+00 true resid norm 1.801125916915e+14 ||r(i)||/||b|| 1.025698920375e+12 132 KSP preconditioned resid norm 9.688338620908e+00 true resid norm 2.431561636866e+14 ||r(i)||/||b|| 1.384717260652e+12 133 KSP preconditioned resid norm 9.653638081152e+00 true resid norm 2.681392129647e+14 ||r(i)||/||b|| 1.526989860427e+12 134 KSP preconditioned resid norm 9.625227378606e+00 true resid norm 2.556456989644e+14 ||r(i)||/||b|| 1.455842231595e+12 135 KSP preconditioned resid norm 9.510469551367e+00 true resid norm 2.356857223521e+14 ||r(i)||/||b|| 1.342174851265e+12 136 KSP preconditioned resid norm 9.311262628453e+00 true resid norm 1.813247990861e+14 ||r(i)||/||b|| 1.032602156869e+12 137 KSP preconditioned resid norm 8.496681822499e+00 true resid norm 1.110559696168e+14 ||r(i)||/||b|| 6.324376717218e+11 138 KSP preconditioned resid norm 7.616859591822e+00 true resid norm 2.432558891525e+14 ||r(i)||/||b|| 1.385285173765e+12 139 KSP preconditioned resid norm 7.369595364139e+00 true resid norm 3.141468295058e+14 ||r(i)||/||b|| 1.788992434329e+12 140 KSP preconditioned resid norm 7.357820656078e+00 true resid norm 3.377604377039e+14 ||r(i)||/||b|| 1.923466388690e+12 141 KSP preconditioned resid norm 7.266040336282e+00 true resid norm 2.854752590002e+14 ||r(i)||/||b|| 1.625714572204e+12 142 KSP preconditioned resid norm 6.816659148098e+00 true resid norm 1.249887885912e+14 ||r(i)||/||b|| 7.117818044425e+11 143 KSP preconditioned resid norm 6.378163918028e+00 true resid norm 8.055787212918e+13 ||r(i)||/||b|| 4.587581672922e+11 144 KSP preconditioned resid norm 6.261029449882e+00 true resid norm 9.977655883777e+13 ||r(i)||/||b|| 5.682040756704e+11 145 KSP preconditioned resid norm 6.214633120628e+00 true resid norm 1.318013178069e+14 ||r(i)||/||b|| 7.505775587870e+11 146 KSP preconditioned resid norm 6.207644628966e+00 true resid norm 1.193014418655e+14 ||r(i)||/||b|| 6.793937001933e+11 147 KSP preconditioned resid norm 5.891477122112e+00 true resid norm 7.112628078462e+13 ||r(i)||/||b|| 4.050474690635e+11 148 KSP preconditioned resid norm 5.541483303411e+00 true resid norm 1.650592277942e+14 ||r(i)||/||b|| 9.399735474158e+11 149 KSP preconditioned resid norm 5.492410219709e+00 true resid norm 2.258077005605e+14 ||r(i)||/||b|| 1.285921836459e+12 150 KSP preconditioned resid norm 5.458541392235e+00 true resid norm 1.900989296854e+14 ||r(i)||/||b|| 1.082568770521e+12 151 KSP preconditioned resid norm 5.458387299042e+00 true resid norm 1.902472823779e+14 ||r(i)||/||b|| 1.083413604273e+12 152 KSP preconditioned resid norm 5.457860124922e+00 true resid norm 1.913469274966e+14 ||r(i)||/||b|| 1.089675825034e+12 153 KSP preconditioned resid norm 5.457399009783e+00 true resid norm 1.891753206004e+14 ||r(i)||/||b|| 1.077309033640e+12 154 KSP preconditioned resid norm 5.437863987680e+00 true resid norm 1.653562940607e+14 ||r(i)||/||b|| 9.416652700543e+11 155 KSP preconditioned resid norm 5.416787761257e+00 true resid norm 1.461507401107e+14 ||r(i)||/||b|| 8.322941496525e+11 156 KSP preconditioned resid norm 5.398307109159e+00 true resid norm 1.306020506252e+14 ||r(i)||/||b|| 7.437480137675e+11 157 KSP preconditioned resid norm 5.382855191804e+00 true resid norm 1.164603795666e+14 ||r(i)||/||b|| 6.632145174648e+11 158 KSP preconditioned resid norm 5.380423128513e+00 true resid norm 1.099542663154e+14 ||r(i)||/||b|| 6.261637300937e+11 159 KSP preconditioned resid norm 5.374859084371e+00 true resid norm 1.230562839833e+14 ||r(i)||/||b|| 7.007766444409e+11 160 KSP preconditioned resid norm 5.343330818539e+00 true resid norm 1.617549407255e+14 ||r(i)||/||b|| 9.211564083851e+11 161 KSP preconditioned resid norm 5.313386300458e+00 true resid norm 1.949614968387e+14 ||r(i)||/||b|| 1.110259948759e+12 162 KSP preconditioned resid norm 5.305156807574e+00 true resid norm 2.167041713453e+14 ||r(i)||/||b|| 1.234079375030e+12 163 KSP preconditioned resid norm 5.302288854295e+00 true resid norm 2.071969491865e+14 ||r(i)||/||b|| 1.179937977072e+12 164 KSP preconditioned resid norm 5.269823647432e+00 true resid norm 2.047500468533e+14 ||r(i)||/||b|| 1.166003442802e+12 165 KSP preconditioned resid norm 5.167981346807e+00 true resid norm 1.347204187772e+14 ||r(i)||/||b|| 7.672011534261e+11 166 KSP preconditioned resid norm 4.943638894138e+00 true resid norm 1.104946705459e+14 ||r(i)||/||b|| 6.292412053026e+11 167 KSP preconditioned resid norm 4.585756505323e+00 true resid norm 6.368799627739e+13 ||r(i)||/||b|| 3.626881852574e+11 168 KSP preconditioned resid norm 4.220001997990e+00 true resid norm 1.634659962016e+14 ||r(i)||/||b|| 9.309004675769e+11 169 KSP preconditioned resid norm 4.206275934587e+00 true resid norm 1.707558711430e+14 ||r(i)||/||b|| 9.724145937511e+11 170 KSP preconditioned resid norm 4.187313597963e+00 true resid norm 1.607630116424e+14 ||r(i)||/||b|| 9.155076051558e+11 171 KSP preconditioned resid norm 4.079389132676e+00 true resid norm 8.609086263452e+13 ||r(i)||/||b|| 4.902672491086e+11 172 KSP preconditioned resid norm 4.000951705220e+00 true resid norm 4.832575423627e+13 ||r(i)||/||b|| 2.752038237913e+11 173 KSP preconditioned resid norm 3.968309939241e+00 true resid norm 4.867187563114e+13 ||r(i)||/||b|| 2.771749038680e+11 174 KSP preconditioned resid norm 3.965355241668e+00 true resid norm 4.914226983036e+13 ||r(i)||/||b|| 2.798536883869e+11 175 KSP preconditioned resid norm 3.925713966351e+00 true resid norm 5.051835159459e+13 ||r(i)||/||b|| 2.876901509389e+11 176 KSP preconditioned resid norm 3.871442062684e+00 true resid norm 1.134027479202e+14 ||r(i)||/||b|| 6.458020231509e+11 177 KSP preconditioned resid norm 3.670840390486e+00 true resid norm 1.451295211343e+14 ||r(i)||/||b|| 8.264785473579e+11 178 KSP preconditioned resid norm 3.627801487329e+00 true resid norm 1.548327670525e+14 ||r(i)||/||b|| 8.817362546007e+11 179 KSP preconditioned resid norm 3.625353909567e+00 true resid norm 1.448111660342e+14 ||r(i)||/||b|| 8.246655898108e+11 180 KSP preconditioned resid norm 3.489187683674e+00 true resid norm 6.526040773648e+13 ||r(i)||/||b|| 3.716426993245e+11 181 KSP preconditioned resid norm 3.477988311428e+00 true resid norm 6.605679306842e+13 ||r(i)||/||b|| 3.761779268037e+11 182 KSP preconditioned resid norm 3.450252485717e+00 true resid norm 6.902743913726e+13 ||r(i)||/||b|| 3.930950586766e+11 183 KSP preconditioned resid norm 3.432891398819e+00 true resid norm 7.443017997285e+13 ||r(i)||/||b|| 4.238623992056e+11 184 KSP preconditioned resid norm 3.429401963752e+00 true resid norm 7.193653821770e+13 ||r(i)||/||b|| 4.096616949015e+11 185 KSP preconditioned resid norm 3.402753697392e+00 true resid norm 6.619060775227e+13 ||r(i)||/||b|| 3.769399700093e+11 186 KSP preconditioned resid norm 3.365344754595e+00 true resid norm 5.787263460112e+13 ||r(i)||/||b|| 3.295710659215e+11 187 KSP preconditioned resid norm 3.333860758000e+00 true resid norm 5.286042303103e+13 ||r(i)||/||b|| 3.010276978657e+11 188 KSP preconditioned resid norm 3.280951184012e+00 true resid norm 4.487501136122e+13 ||r(i)||/||b|| 2.555526533307e+11 189 KSP preconditioned resid norm 3.274901345142e+00 true resid norm 4.216755107484e+13 ||r(i)||/||b|| 2.401343026945e+11 190 KSP preconditioned resid norm 3.269766489834e+00 true resid norm 4.566718330079e+13 ||r(i)||/||b|| 2.600638865295e+11 191 KSP preconditioned resid norm 3.224282149858e+00 true resid norm 6.014715696957e+13 ||r(i)||/||b|| 3.425239367662e+11 192 KSP preconditioned resid norm 3.190061156246e+00 true resid norm 7.489485410637e+13 ||r(i)||/||b|| 4.265086092934e+11 193 KSP preconditioned resid norm 3.174670622367e+00 true resid norm 8.375034387468e+13 ||r(i)||/||b|| 4.769385443105e+11 194 KSP preconditioned resid norm 3.174669599011e+00 true resid norm 8.368810612367e+13 ||r(i)||/||b|| 4.765841149315e+11 195 KSP preconditioned resid norm 3.161385505086e+00 true resid norm 7.567270930158e+13 ||r(i)||/||b|| 4.309383119946e+11 196 KSP preconditioned resid norm 3.096104483416e+00 true resid norm 5.310942114269e+13 ||r(i)||/||b|| 3.024456836484e+11 197 KSP preconditioned resid norm 2.890744457445e+00 true resid norm 3.994989614098e+13 ||r(i)||/||b|| 2.275052785377e+11 198 KSP preconditioned resid norm 2.467527000711e+00 true resid norm 1.522868258783e+14 ||r(i)||/||b|| 8.672377173844e+11 199 KSP preconditioned resid norm 2.383286040847e+00 true resid norm 1.874146376472e+14 ||r(i)||/||b|| 1.067282357618e+12 200 KSP preconditioned resid norm 2.377677655552e+00 true resid norm 2.016380246410e+14 ||r(i)||/||b|| 1.148281206986e+12 201 KSP preconditioned resid norm 2.338075769967e+00 true resid norm 1.712742596292e+14 ||r(i)||/||b|| 9.753666944655e+11 202 KSP preconditioned resid norm 2.263046833056e+00 true resid norm 1.100049031547e+14 ||r(i)||/||b|| 6.264520950039e+11 203 KSP preconditioned resid norm 2.087852652579e+00 true resid norm 2.994461865954e+13 ||r(i)||/||b|| 1.705275724573e+11 204 KSP preconditioned resid norm 2.007978318730e+00 true resid norm 3.902414615090e+13 ||r(i)||/||b|| 2.222333497044e+11 205 KSP preconditioned resid norm 2.007794121017e+00 true resid norm 3.997900843259e+13 ||r(i)||/||b|| 2.276710662030e+11 206 KSP preconditioned resid norm 1.990340363260e+00 true resid norm 2.955605641295e+13 ||r(i)||/||b|| 1.683148016949e+11 207 KSP preconditioned resid norm 1.933402028333e+00 true resid norm 3.966903044733e+13 ||r(i)||/||b|| 2.259058143578e+11 208 KSP preconditioned resid norm 1.832793003811e+00 true resid norm 9.682342679101e+13 ||r(i)||/||b|| 5.513866820410e+11 209 KSP preconditioned resid norm 1.813616245556e+00 true resid norm 1.223607688510e+14 ||r(i)||/||b|| 6.968158490648e+11 210 KSP preconditioned resid norm 1.773130779026e+00 true resid norm 1.066236841099e+14 ||r(i)||/||b|| 6.071968464333e+11 211 KSP preconditioned resid norm 1.772584830402e+00 true resid norm 1.069079970412e+14 ||r(i)||/||b|| 6.088159418220e+11 212 KSP preconditioned resid norm 1.770976105151e+00 true resid norm 1.087835209619e+14 ||r(i)||/||b|| 6.194966101890e+11 213 KSP preconditioned resid norm 1.770973048299e+00 true resid norm 1.089542042522e+14 ||r(i)||/||b|| 6.204686114521e+11 214 KSP preconditioned resid norm 1.758886456895e+00 true resid norm 8.479357845122e+13 ||r(i)||/||b|| 4.828795202789e+11 215 KSP preconditioned resid norm 1.742030866566e+00 true resid norm 5.582748508382e+13 ||r(i)||/||b|| 3.179244196087e+11 216 KSP preconditioned resid norm 1.718022070763e+00 true resid norm 2.704018886646e+13 ||r(i)||/||b|| 1.539875267276e+11 217 KSP preconditioned resid norm 1.696292837687e+00 true resid norm 1.749811486465e+13 ||r(i)||/||b|| 9.964765570648e+10 218 KSP preconditioned resid norm 1.690048480269e+00 true resid norm 2.513788931053e+13 ||r(i)||/||b|| 1.431543774046e+11 219 KSP preconditioned resid norm 1.686282962967e+00 true resid norm 1.945553472563e+13 ||r(i)||/||b|| 1.107947022248e+11 220 KSP preconditioned resid norm 1.655955991493e+00 true resid norm 3.243058909820e+13 ||r(i)||/||b|| 1.846845904152e+11 221 KSP preconditioned resid norm 1.612547703436e+00 true resid norm 6.996479117954e+13 ||r(i)||/||b|| 3.984330584730e+11 222 KSP preconditioned resid norm 1.592432434959e+00 true resid norm 8.751037594164e+13 ||r(i)||/||b|| 4.983510440998e+11 223 KSP preconditioned resid norm 1.592307656748e+00 true resid norm 8.560617673493e+13 ||r(i)||/||b|| 4.875070767116e+11 224 KSP preconditioned resid norm 1.566116384754e+00 true resid norm 8.332212832354e+13 ||r(i)||/||b|| 4.744999572891e+11 225 KSP preconditioned resid norm 1.511745201901e+00 true resid norm 5.089325612967e+13 ||r(i)||/||b|| 2.898251442409e+11 226 KSP preconditioned resid norm 1.445254159360e+00 true resid norm 3.385040442580e+13 ||r(i)||/||b|| 1.927701053421e+11 227 KSP preconditioned resid norm 1.360552734183e+00 true resid norm 1.476185891824e+13 ||r(i)||/||b|| 8.406532054739e+10 228 KSP preconditioned resid norm 1.246018368109e+00 true resid norm 2.657896590600e+13 ||r(i)||/||b|| 1.513609702600e+11 229 KSP preconditioned resid norm 1.234565282845e+00 true resid norm 3.292979573825e+13 ||r(i)||/||b|| 1.875274550197e+11 230 KSP preconditioned resid norm 1.234556093198e+00 true resid norm 3.303205109670e+13 ||r(i)||/||b|| 1.881097752771e+11 231 KSP preconditioned resid norm 1.223622511917e+00 true resid norm 2.208248107041e+13 ||r(i)||/||b|| 1.257545448678e+11 232 KSP preconditioned resid norm 1.210980434409e+00 true resid norm 1.542295263348e+13 ||r(i)||/||b|| 8.783009403508e+10 233 KSP preconditioned resid norm 1.182304375476e+00 true resid norm 1.477289800812e+13 ||r(i)||/||b|| 8.412818557237e+10 234 KSP preconditioned resid norm 1.181925536452e+00 true resid norm 1.421319533202e+13 ||r(i)||/||b|| 8.094081024669e+10 235 KSP preconditioned resid norm 1.181097014157e+00 true resid norm 1.340812930501e+13 ||r(i)||/||b|| 7.635614824732e+10 236 KSP preconditioned resid norm 1.139325286816e+00 true resid norm 1.259155720183e+13 ||r(i)||/||b|| 7.170596184572e+10 237 KSP preconditioned resid norm 1.054521342867e+00 true resid norm 4.023699395379e+13 ||r(i)||/||b|| 2.291402331728e+11 238 KSP preconditioned resid norm 9.221108351721e-01 true resid norm 2.190315343620e+13 ||r(i)||/||b|| 1.247333160960e+11 239 KSP preconditioned resid norm 8.694114847957e-01 true resid norm 2.406683225069e+13 ||r(i)||/||b|| 1.370549589263e+11 240 KSP preconditioned resid norm 8.572287007726e-01 true resid norm 3.648678138668e+13 ||r(i)||/||b|| 2.077836531295e+11 241 KSP preconditioned resid norm 8.569741281629e-01 true resid norm 3.634105281283e+13 ||r(i)||/||b|| 2.069537631177e+11 242 KSP preconditioned resid norm 8.546636466972e-01 true resid norm 3.506620503540e+13 ||r(i)||/||b|| 1.996938043514e+11 243 KSP preconditioned resid norm 8.436981690934e-01 true resid norm 2.951646328876e+13 ||r(i)||/||b|| 1.680893281489e+11 244 KSP preconditioned resid norm 8.173174100365e-01 true resid norm 1.824446706210e+13 ||r(i)||/||b|| 1.038979562321e+11 245 KSP preconditioned resid norm 7.819864387751e-01 true resid norm 7.776718832744e+12 ||r(i)||/||b|| 4.428658782763e+10 246 KSP preconditioned resid norm 7.494727650930e-01 true resid norm 9.390814897668e+12 ||r(i)||/||b|| 5.347848593774e+10 247 KSP preconditioned resid norm 7.397424145173e-01 true resid norm 1.239122155493e+13 ||r(i)||/||b|| 7.056509737419e+10 248 KSP preconditioned resid norm 7.246093261829e-01 true resid norm 2.036047657616e+13 ||r(i)||/||b|| 1.159481335889e+11 249 KSP preconditioned resid norm 7.245038778291e-01 true resid norm 1.963876697176e+13 ||r(i)||/||b|| 1.118381668448e+11 250 KSP preconditioned resid norm 7.171029204552e-01 true resid norm 9.527294292595e+12 ||r(i)||/||b|| 5.425570404734e+10 251 KSP preconditioned resid norm 6.952339007733e-01 true resid norm 8.813498953048e+12 ||r(i)||/||b|| 5.019080718329e+10 252 KSP preconditioned resid norm 6.616924086753e-01 true resid norm 2.108382041551e+13 ||r(i)||/||b|| 1.200674069174e+11 253 KSP preconditioned resid norm 6.448191214386e-01 true resid norm 3.210805413021e+13 ||r(i)||/||b|| 1.828478295017e+11 254 KSP preconditioned resid norm 6.422075541827e-01 true resid norm 3.557756209357e+13 ||r(i)||/||b|| 2.026058627343e+11 255 KSP preconditioned resid norm 6.416647613441e-01 true resid norm 3.380009792729e+13 ||r(i)||/||b|| 1.924836216447e+11 256 KSP preconditioned resid norm 6.402280834143e-01 true resid norm 2.953288783796e+13 ||r(i)||/||b|| 1.681828621002e+11 257 KSP preconditioned resid norm 6.350183274762e-01 true resid norm 2.019635706586e+13 ||r(i)||/||b|| 1.150135115120e+11 258 KSP preconditioned resid norm 6.268519717549e-01 true resid norm 8.111866952331e+12 ||r(i)||/||b|| 4.619517767800e+10 259 KSP preconditioned resid norm 6.259448365356e-01 true resid norm 6.274138275600e+12 ||r(i)||/||b|| 3.572974435119e+10 260 KSP preconditioned resid norm 6.185273691470e-01 true resid norm 1.328310119998e+13 ||r(i)||/||b|| 7.564414254497e+10 261 KSP preconditioned resid norm 5.799540271653e-01 true resid norm 3.941950358614e+13 ||r(i)||/||b|| 2.244848174707e+11 262 KSP preconditioned resid norm 5.592595784538e-01 true resid norm 6.141084463330e+13 ||r(i)||/||b|| 3.497203413052e+11 263 KSP preconditioned resid norm 5.534201157748e-01 true resid norm 7.435564413187e+13 ||r(i)||/||b|| 4.234379350917e+11 264 KSP preconditioned resid norm 5.528810608860e-01 true resid norm 7.763287354101e+13 ||r(i)||/||b|| 4.421009871039e+11 265 KSP preconditioned resid norm 5.469595863469e-01 true resid norm 7.004373972883e+13 ||r(i)||/||b|| 3.988826519246e+11 266 KSP preconditioned resid norm 4.945292416960e-01 true resid norm 4.228774811832e+13 ||r(i)||/||b|| 2.408187966356e+11 267 KSP preconditioned resid norm 4.281890068775e-01 true resid norm 1.125884213676e+13 ||r(i)||/||b|| 6.411646246328e+10 268 KSP preconditioned resid norm 4.084231601522e-01 true resid norm 2.127490364486e+13 ||r(i)||/||b|| 1.211555810435e+11 269 KSP preconditioned resid norm 4.001208915779e-01 true resid norm 3.267071702508e+13 ||r(i)||/||b|| 1.860520625782e+11 270 KSP preconditioned resid norm 3.984419205712e-01 true resid norm 2.966020373146e+13 ||r(i)||/||b|| 1.689078962207e+11 271 KSP preconditioned resid norm 3.983169885607e-01 true resid norm 2.985688873765e+13 ||r(i)||/||b|| 1.700279711506e+11 272 KSP preconditioned resid norm 3.983124521919e-01 true resid norm 2.995123552363e+13 ||r(i)||/||b|| 1.705652539447e+11 273 KSP preconditioned resid norm 3.970518487040e-01 true resid norm 2.679057592981e+13 ||r(i)||/||b|| 1.525660396609e+11 274 KSP preconditioned resid norm 3.939367745543e-01 true resid norm 2.092683959757e+13 ||r(i)||/||b|| 1.191734380174e+11 275 KSP preconditioned resid norm 3.908410919474e-01 true resid norm 1.686677707108e+13 ||r(i)||/||b|| 9.605233520626e+10 276 KSP preconditioned resid norm 3.864892438875e-01 true resid norm 1.308673657633e+13 ||r(i)||/||b|| 7.452589211849e+10 277 KSP preconditioned resid norm 3.810077620712e-01 true resid norm 8.711820645292e+12 ||r(i)||/||b|| 4.961177309405e+10 278 KSP preconditioned resid norm 3.799753478575e-01 true resid norm 7.283942147984e+12 ||r(i)||/||b|| 4.148034030880e+10 279 KSP preconditioned resid norm 3.797915436438e-01 true resid norm 8.282722196918e+12 ||r(i)||/||b|| 4.716815818018e+10 280 KSP preconditioned resid norm 3.782267745986e-01 true resid norm 1.343417477314e+13 ||r(i)||/||b|| 7.650447107297e+10 281 KSP preconditioned resid norm 3.767711703315e-01 true resid norm 1.850915006024e+13 ||r(i)||/||b|| 1.054052637606e+11 282 KSP preconditioned resid norm 3.760725811208e-01 true resid norm 2.146321377790e+13 ||r(i)||/||b|| 1.222279630371e+11 283 KSP preconditioned resid norm 3.760725798486e-01 true resid norm 2.145927689153e+13 ||r(i)||/||b|| 1.222055433935e+11 284 KSP preconditioned resid norm 3.726641668459e-01 true resid norm 1.486048146819e+13 ||r(i)||/||b|| 8.462695281341e+10 285 KSP preconditioned resid norm 3.629918107955e-01 true resid norm 5.461478467526e+12 ||r(i)||/||b|| 3.110183755165e+10 286 KSP preconditioned resid norm 3.465171750557e-01 true resid norm 1.207028659269e+13 ||r(i)||/||b|| 6.873744811775e+10 287 KSP preconditioned resid norm 3.331409716393e-01 true resid norm 2.006105462605e+13 ||r(i)||/||b|| 1.142429958854e+11 288 KSP preconditioned resid norm 3.239240346132e-01 true resid norm 3.105697669784e+13 ||r(i)||/||b|| 1.768621903107e+11 289 KSP preconditioned resid norm 3.238957055659e-01 true resid norm 3.157956806648e+13 ||r(i)||/||b|| 1.798382254539e+11 290 KSP preconditioned resid norm 3.157544915738e-01 true resid norm 2.092815634087e+13 ||r(i)||/||b|| 1.191809365613e+11 291 KSP preconditioned resid norm 2.852382588503e-01 true resid norm 2.979472210968e+12 ||r(i)||/||b|| 1.696739468006e+10 292 KSP preconditioned resid norm 2.694150546753e-01 true resid norm 1.436731569255e+13 ||r(i)||/||b|| 8.181848951342e+10 293 KSP preconditioned resid norm 2.641931974470e-01 true resid norm 1.861155923250e+13 ||r(i)||/||b|| 1.059884599516e+11 294 KSP preconditioned resid norm 2.641198814870e-01 true resid norm 1.793981239430e+13 ||r(i)||/||b|| 1.021630194300e+11 295 KSP preconditioned resid norm 2.637318754754e-01 true resid norm 1.659027423690e+13 ||r(i)||/||b|| 9.447771648676e+10 296 KSP preconditioned resid norm 2.595722151521e-01 true resid norm 1.208243067646e+13 ||r(i)||/||b|| 6.880660582345e+10 297 KSP preconditioned resid norm 2.491740116565e-01 true resid norm 2.771289873763e+12 ||r(i)||/||b|| 1.578184514959e+10 298 KSP preconditioned resid norm 2.415305724877e-01 true resid norm 7.852236246693e+12 ||r(i)||/||b|| 4.471664176906e+10 299 KSP preconditioned resid norm 2.408045892302e-01 true resid norm 9.473449724245e+12 ||r(i)||/||b|| 5.394907187295e+10 300 KSP preconditioned resid norm 2.340555162432e-01 true resid norm 3.319865257484e+12 ||r(i)||/||b|| 1.890585315782e+10 301 KSP preconditioned resid norm 2.337976114563e-01 true resid norm 3.340283381850e+12 ||r(i)||/||b|| 1.902212958203e+10 302 KSP preconditioned resid norm 2.333327903686e-01 true resid norm 3.415389327731e+12 ||r(i)||/||b|| 1.944984030942e+10 303 KSP preconditioned resid norm 2.332588418708e-01 true resid norm 3.475181431476e+12 ||r(i)||/||b|| 1.979034230143e+10 304 KSP preconditioned resid norm 2.326010662742e-01 true resid norm 3.181161063381e+12 ||r(i)||/||b|| 1.811596533927e+10 305 KSP preconditioned resid norm 2.315413201851e-01 true resid norm 2.933253862391e+12 ||r(i)||/||b|| 1.670419203669e+10 306 KSP preconditioned resid norm 2.304630572502e-01 true resid norm 2.715749947702e+12 ||r(i)||/||b|| 1.546555831110e+10 307 KSP preconditioned resid norm 2.292865323373e-01 true resid norm 2.503342572809e+12 ||r(i)||/||b|| 1.425594818300e+10 308 KSP preconditioned resid norm 2.291299261731e-01 true resid norm 2.431749785546e+12 ||r(i)||/||b|| 1.384824406907e+10 309 KSP preconditioned resid norm 2.286328421233e-01 true resid norm 2.568787793883e+12 ||r(i)||/||b|| 1.462864335089e+10 310 KSP preconditioned resid norm 2.219499874016e-01 true resid norm 3.330410350857e+12 ||r(i)||/||b|| 1.896590498865e+10 311 KSP preconditioned resid norm 2.147861496159e-01 true resid norm 4.183733540267e+12 ||r(i)||/||b|| 2.382538019741e+10 312 KSP preconditioned resid norm 2.085908097212e-01 true resid norm 5.269197084207e+12 ||r(i)||/||b|| 3.000684022011e+10 313 KSP preconditioned resid norm 2.077913842950e-01 true resid norm 5.622549320884e+12 ||r(i)||/||b|| 3.201909824309e+10 314 KSP preconditioned resid norm 2.074683531125e-01 true resid norm 5.380881874823e+12 ||r(i)||/||b|| 3.064285887978e+10 315 KSP preconditioned resid norm 2.049851408085e-01 true resid norm 4.681933765100e+12 ||r(i)||/||b|| 2.666251350354e+10 316 KSP preconditioned resid norm 1.972228619533e-01 true resid norm 3.213437404698e+12 ||r(i)||/||b|| 1.829977152479e+10 317 KSP preconditioned resid norm 1.890934720736e-01 true resid norm 1.861733116751e+12 ||r(i)||/||b|| 1.060213297663e+10 318 KSP preconditioned resid norm 1.708640605511e-01 true resid norm 5.900505719991e+12 ||r(i)||/||b|| 3.360199467359e+10 319 KSP preconditioned resid norm 1.663553073051e-01 true resid norm 7.175237786099e+12 ||r(i)||/||b|| 4.086129449097e+10 320 KSP preconditioned resid norm 1.658645928614e-01 true resid norm 7.758811670795e+12 ||r(i)||/||b|| 4.418461074482e+10 321 KSP preconditioned resid norm 1.622193976534e-01 true resid norm 5.257250867666e+12 ||r(i)||/||b|| 2.993880932180e+10 322 KSP preconditioned resid norm 1.538134435071e-01 true resid norm 1.761918308713e+12 ||r(i)||/||b|| 1.003371108074e+10 323 KSP preconditioned resid norm 1.475401894061e-01 true resid norm 2.439997291723e+12 ||r(i)||/||b|| 1.389521168029e+10 324 KSP preconditioned resid norm 1.465929441051e-01 true resid norm 3.261115459698e+12 ||r(i)||/||b|| 1.857128685351e+10 325 KSP preconditioned resid norm 1.465393365067e-01 true resid norm 3.532319290848e+12 ||r(i)||/||b|| 2.011572899495e+10 326 KSP preconditioned resid norm 1.454914804114e-01 true resid norm 2.395332308598e+12 ||r(i)||/||b|| 1.364085508845e+10 327 KSP preconditioned resid norm 1.390786123266e-01 true resid norm 2.528026009523e+12 ||r(i)||/||b|| 1.439651455957e+10 328 KSP preconditioned resid norm 1.374343106603e-01 true resid norm 3.812158646060e+12 ||r(i)||/||b|| 2.170934841835e+10 329 KSP preconditioned resid norm 1.374341181714e-01 true resid norm 3.791013558642e+12 ||r(i)||/||b|| 2.158893211024e+10 330 KSP preconditioned resid norm 1.352522554178e-01 true resid norm 1.974296331181e+12 ||r(i)||/||b|| 1.124315405367e+10 331 KSP preconditioned resid norm 1.350655877373e-01 true resid norm 1.991730189414e+12 ||r(i)||/||b|| 1.134243578295e+10 332 KSP preconditioned resid norm 1.348111499178e-01 true resid norm 2.041450090606e+12 ||r(i)||/||b|| 1.162557894631e+10 333 KSP preconditioned resid norm 1.347849508207e-01 true resid norm 2.070098875209e+12 ||r(i)||/||b|| 1.178872704807e+10 334 KSP preconditioned resid norm 1.347064718924e-01 true resid norm 1.986607521799e+12 ||r(i)||/||b|| 1.131326339365e+10 335 KSP preconditioned resid norm 1.343983613005e-01 true resid norm 1.851271628553e+12 ||r(i)||/||b|| 1.054255725763e+10 336 KSP preconditioned resid norm 1.336229788762e-01 true resid norm 1.667659941047e+12 ||r(i)||/||b|| 9.496931808159e+09 337 KSP preconditioned resid norm 1.325162653711e-01 true resid norm 1.468835090963e+12 ||r(i)||/||b|| 8.364670969763e+09 338 KSP preconditioned resid norm 1.322680103842e-01 true resid norm 1.408300477178e+12 ||r(i)||/||b|| 8.019940557400e+09 339 KSP preconditioned resid norm 1.319145624423e-01 true resid norm 1.509021321087e+12 ||r(i)||/||b|| 8.593522114840e+09 340 KSP preconditioned resid norm 1.301020989018e-01 true resid norm 1.939032411134e+12 ||r(i)||/||b|| 1.104233430875e+10 341 KSP preconditioned resid norm 1.265364340713e-01 true resid norm 3.097593434152e+12 ||r(i)||/||b|| 1.764006731197e+10 342 KSP preconditioned resid norm 1.233025029546e-01 true resid norm 4.405615884105e+12 ||r(i)||/||b|| 2.508894804898e+10 343 KSP preconditioned resid norm 1.217458096040e-01 true resid norm 5.448234354847e+12 ||r(i)||/||b|| 3.102641543957e+10 344 KSP preconditioned resid norm 1.216044613872e-01 true resid norm 5.205265234000e+12 ||r(i)||/||b|| 2.964276334398e+10 345 KSP preconditioned resid norm 1.202483376431e-01 true resid norm 4.373819937611e+12 ||r(i)||/||b|| 2.490787759919e+10 346 KSP preconditioned resid norm 1.142262444908e-01 true resid norm 2.410382502863e+12 ||r(i)||/||b|| 1.372656241110e+10 347 KSP preconditioned resid norm 1.034106328028e-01 true resid norm 1.402453552596e+12 ||r(i)||/||b|| 7.986643694730e+09 348 KSP preconditioned resid norm 9.616103472769e-02 true resid norm 3.393076323304e+12 ||r(i)||/||b|| 1.932277298816e+10 349 KSP preconditioned resid norm 9.259589822073e-02 true resid norm 4.915943835511e+12 ||r(i)||/||b|| 2.799514591043e+10 350 KSP preconditioned resid norm 9.235272729717e-02 true resid norm 5.600460755060e+12 ||r(i)||/||b|| 3.189330904698e+10 351 KSP preconditioned resid norm 9.219524424992e-02 true resid norm 5.004185758598e+12 ||r(i)||/||b|| 2.849766294376e+10 352 KSP preconditioned resid norm 9.013478975958e-02 true resid norm 1.354932500749e+12 ||r(i)||/||b|| 7.716022462104e+09 353 KSP preconditioned resid norm 8.753727009852e-02 true resid norm 2.615102379560e+12 ||r(i)||/||b|| 1.489239404195e+10 354 KSP preconditioned resid norm 8.737394501163e-02 true resid norm 3.256844376132e+12 ||r(i)||/||b|| 1.854696403542e+10 355 KSP preconditioned resid norm 8.731164789228e-02 true resid norm 3.679087592003e+12 ||r(i)||/||b|| 2.095154000975e+10 356 KSP preconditioned resid norm 8.686035696790e-02 true resid norm 2.606571260249e+12 ||r(i)||/||b|| 1.484381132052e+10 357 KSP preconditioned resid norm 8.510853060356e-02 true resid norm 9.410247297091e+11 ||r(i)||/||b|| 5.358914888984e+09 358 KSP preconditioned resid norm 8.456101573028e-02 true resid norm 2.187983263684e+12 ||r(i)||/||b|| 1.246005096192e+10 359 KSP preconditioned resid norm 8.455208355692e-02 true resid norm 2.388461620241e+12 ||r(i)||/||b|| 1.360172813145e+10 360 KSP preconditioned resid norm 8.282898790796e-02 true resid norm 8.621883722570e+11 ||r(i)||/||b|| 4.909960343577e+09 361 KSP preconditioned resid norm 8.272531817972e-02 true resid norm 8.633101635311e+11 ||r(i)||/||b|| 4.916348681494e+09 362 KSP preconditioned resid norm 8.254591608780e-02 true resid norm 8.673672359825e+11 ||r(i)||/||b|| 4.939452756530e+09 363 KSP preconditioned resid norm 8.246541279676e-02 true resid norm 8.718796820962e+11 ||r(i)||/||b|| 4.965150077653e+09 364 KSP preconditioned resid norm 8.241801614853e-02 true resid norm 8.671459105212e+11 ||r(i)||/||b|| 4.938192359993e+09 365 KSP preconditioned resid norm 8.212818717225e-02 true resid norm 8.587064176355e+11 ||r(i)||/||b|| 4.890131429549e+09 366 KSP preconditioned resid norm 8.194338791810e-02 true resid norm 8.545028564125e+11 ||r(i)||/||b|| 4.866193135354e+09 367 KSP preconditioned resid norm 8.171085097872e-02 true resid norm 8.509831240026e+11 ||r(i)||/||b|| 4.846149085691e+09 368 KSP preconditioned resid norm 8.161620694246e-02 true resid norm 8.497340417759e+11 ||r(i)||/||b|| 4.839035855686e+09 369 KSP preconditioned resid norm 8.156115260727e-02 true resid norm 8.491439509018e+11 ||r(i)||/||b|| 4.835675426708e+09 370 KSP preconditioned resid norm 8.063518312109e-02 true resid norm 8.485155476082e+11 ||r(i)||/||b|| 4.832096817496e+09 371 KSP preconditioned resid norm 7.923271766690e-02 true resid norm 8.588597971885e+11 ||r(i)||/||b|| 4.891004890091e+09 372 KSP preconditioned resid norm 7.803394074240e-02 true resid norm 9.006066172395e+11 ||r(i)||/||b|| 5.128743228390e+09 373 KSP preconditioned resid norm 7.794468296789e-02 true resid norm 9.117020640296e+11 ||r(i)||/||b|| 5.191929192718e+09 374 KSP preconditioned resid norm 7.789560350182e-02 true resid norm 9.012718632073e+11 ||r(i)||/||b|| 5.132531647980e+09 375 KSP preconditioned resid norm 7.726439270554e-02 true resid norm 8.586478043639e+11 ||r(i)||/||b|| 4.889797640729e+09 376 KSP preconditioned resid norm 7.571815304778e-02 true resid norm 7.859006686039e+11 ||r(i)||/||b|| 4.475519783148e+09 377 KSP preconditioned resid norm 7.229778169081e-02 true resid norm 7.199635191238e+11 ||r(i)||/||b|| 4.100023198488e+09 378 KSP preconditioned resid norm 6.397340351388e-02 true resid norm 1.019822843376e+12 ||r(i)||/||b|| 5.807651645013e+09 379 KSP preconditioned resid norm 6.206292899595e-02 true resid norm 1.188216539089e+12 ||r(i)||/||b|| 6.766614204318e+09 380 KSP preconditioned resid norm 6.196521462769e-02 true resid norm 1.124045327968e+12 ||r(i)||/||b|| 6.401174223978e+09 381 KSP preconditioned resid norm 5.991532104364e-02 true resid norm 7.442069871361e+11 ||r(i)||/||b|| 4.238084056604e+09 382 KSP preconditioned resid norm 5.520548141478e-02 true resid norm 7.033003686501e+11 ||r(i)||/||b|| 4.005130468944e+09 383 KSP preconditioned resid norm 5.363722028702e-02 true resid norm 9.204398617490e+11 ||r(i)||/||b|| 5.241688899150e+09 384 KSP preconditioned resid norm 5.357854244551e-02 true resid norm 9.679742638260e+11 ||r(i)||/||b|| 5.512386158198e+09 385 KSP preconditioned resid norm 5.353846275978e-02 true resid norm 9.122890587478e+11 ||r(i)||/||b|| 5.195271989815e+09 386 KSP preconditioned resid norm 5.127828730979e-02 true resid norm 5.397803438917e+11 ||r(i)||/||b|| 3.073922321421e+09 387 KSP preconditioned resid norm 4.779149659066e-02 true resid norm 7.151556879314e+11 ||r(i)||/||b|| 4.072643728696e+09 388 KSP preconditioned resid norm 4.453398637579e-02 true resid norm 1.396877441705e+12 ||r(i)||/||b|| 7.954889052439e+09 389 KSP preconditioned resid norm 4.229884991114e-02 true resid norm 2.320750680264e+12 ||r(i)||/||b|| 1.321613022639e+10 390 KSP preconditioned resid norm 4.227141787457e-02 true resid norm 2.410982268811e+12 ||r(i)||/||b|| 1.372997793735e+10 391 KSP preconditioned resid norm 4.226967821924e-02 true resid norm 2.405619775441e+12 ||r(i)||/||b|| 1.369943979669e+10 392 KSP preconditioned resid norm 4.221372313301e-02 true resid norm 2.329085067743e+12 ||r(i)||/||b|| 1.326359260622e+10 393 KSP preconditioned resid norm 4.155609966799e-02 true resid norm 1.810164248293e+12 ||r(i)||/||b|| 1.030846037881e+10 394 KSP preconditioned resid norm 3.984904493432e-02 true resid norm 7.675743853622e+11 ||r(i)||/||b|| 4.371155902983e+09 395 KSP preconditioned resid norm 3.825027307918e-02 true resid norm 4.185151745600e+11 ||r(i)||/||b|| 2.383345654379e+09 396 KSP preconditioned resid norm 3.763881169814e-02 true resid norm 8.271169350618e+11 ||r(i)||/||b|| 4.710236743304e+09 397 KSP preconditioned resid norm 3.693767673821e-02 true resid norm 1.445874605711e+12 ||r(i)||/||b|| 8.233916397230e+09 398 KSP preconditioned resid norm 3.655336108976e-02 true resid norm 1.951696829437e+12 ||r(i)||/||b|| 1.111445519746e+10 399 KSP preconditioned resid norm 3.631459017681e-02 true resid norm 1.521939263936e+12 ||r(i)||/||b|| 8.667086766314e+09 400 KSP preconditioned resid norm 3.535829901366e-02 true resid norm 4.129825555092e+11 ||r(i)||/||b|| 2.351838687909e+09 401 KSP preconditioned resid norm 3.306227938654e-02 true resid norm 1.743545317737e+12 ||r(i)||/||b|| 9.929081210999e+09 402 KSP preconditioned resid norm 3.237217236740e-02 true resid norm 2.637614832312e+12 ||r(i)||/||b|| 1.502059717459e+10 403 KSP preconditioned resid norm 3.229588503023e-02 true resid norm 2.838708688793e+12 ||r(i)||/||b|| 1.616577947167e+10 404 KSP preconditioned resid norm 3.221312023156e-02 true resid norm 2.631900255728e+12 ||r(i)||/||b|| 1.498805400269e+10 405 KSP preconditioned resid norm 3.128674650292e-02 true resid norm 1.788603572226e+12 ||r(i)||/||b|| 1.018567739092e+10 406 KSP preconditioned resid norm 2.906677117599e-02 true resid norm 5.532805319168e+11 ||r(i)||/||b|| 3.150802722465e+09 407 KSP preconditioned resid norm 2.646322183153e-02 true resid norm 1.125449605044e+12 ||r(i)||/||b|| 6.409171252212e+09 408 KSP preconditioned resid norm 2.555613075126e-02 true resid norm 1.788020242410e+12 ||r(i)||/||b|| 1.018235546458e+10 409 KSP preconditioned resid norm 2.524431021639e-02 true resid norm 2.496704664068e+12 ||r(i)||/||b|| 1.421814685126e+10 410 KSP preconditioned resid norm 2.518030459049e-02 true resid norm 2.153170063032e+12 ||r(i)||/||b|| 1.226179795813e+10 411 KSP preconditioned resid norm 2.483081199074e-02 true resid norm 1.288184341021e+12 ||r(i)||/||b|| 7.335907364507e+09 412 KSP preconditioned resid norm 2.421322742969e-02 true resid norm 4.654342665955e+11 ||r(i)||/||b|| 2.650538867212e+09 413 KSP preconditioned resid norm 2.406880750280e-02 true resid norm 2.461138537084e+11 ||r(i)||/||b|| 1.401560610879e+09 414 KSP preconditioned resid norm 2.406137336500e-02 true resid norm 2.570585037746e+11 ||r(i)||/||b|| 1.463887823270e+09 415 KSP preconditioned resid norm 2.405151767205e-02 true resid norm 2.456408679897e+11 ||r(i)||/||b|| 1.398867068265e+09 416 KSP preconditioned resid norm 2.366488268271e-02 true resid norm 5.476635939531e+11 ||r(i)||/||b|| 3.118815579583e+09 417 KSP preconditioned resid norm 2.318091177565e-02 true resid norm 1.255400569078e+12 ||r(i)||/||b|| 7.149211480715e+09 418 KSP preconditioned resid norm 2.317653711428e-02 true resid norm 1.290066344908e+12 ||r(i)||/||b|| 7.346624934756e+09 419 KSP preconditioned resid norm 2.297691251598e-02 true resid norm 1.023329583721e+12 ||r(i)||/||b|| 5.827621707918e+09 420 KSP preconditioned resid norm 2.223563066277e-02 true resid norm 6.947559472807e+11 ||r(i)||/||b|| 3.956471995421e+09 421 KSP preconditioned resid norm 2.207591596678e-02 true resid norm 7.277570512278e+11 ||r(i)||/||b|| 4.144405533948e+09 422 KSP preconditioned resid norm 2.192159198929e-02 true resid norm 7.832565151219e+11 ||r(i)||/||b|| 4.460461949898e+09 423 KSP preconditioned resid norm 2.185247147135e-02 true resid norm 8.595230163286e+11 ||r(i)||/||b|| 4.894781767374e+09 424 KSP preconditioned resid norm 2.185084575688e-02 true resid norm 8.423052356273e+11 ||r(i)||/||b|| 4.796730548907e+09 425 KSP preconditioned resid norm 2.184027206302e-02 true resid norm 7.874820929027e+11 ||r(i)||/||b|| 4.484525623221e+09 426 KSP preconditioned resid norm 2.180574611184e-02 true resid norm 6.882090989991e+11 ||r(i)||/||b|| 3.919189231616e+09 427 KSP preconditioned resid norm 2.166857906443e-02 true resid norm 5.212834704447e+11 ||r(i)||/||b|| 2.968586970091e+09 428 KSP preconditioned resid norm 2.159097506805e-02 true resid norm 4.034244503683e+11 ||r(i)||/||b|| 2.297407523315e+09 429 KSP preconditioned resid norm 2.158895718874e-02 true resid norm 4.137932807816e+11 ||r(i)||/||b|| 2.356455577982e+09 430 KSP preconditioned resid norm 2.144608377764e-02 true resid norm 5.707555847347e+11 ||r(i)||/||b|| 3.250319045230e+09 431 KSP preconditioned resid norm 2.102844428927e-02 true resid norm 8.954067244875e+11 ||r(i)||/||b|| 5.099131060069e+09 432 KSP preconditioned resid norm 2.043855604252e-02 true resid norm 1.579549885579e+12 ||r(i)||/||b|| 8.995165729961e+09 433 KSP preconditioned resid norm 2.009798102446e-02 true resid norm 2.188070885454e+12 ||r(i)||/||b|| 1.246054994733e+10 434 KSP preconditioned resid norm 1.996946332503e-02 true resid norm 2.549838325591e+12 ||r(i)||/||b|| 1.452073057818e+10 435 KSP preconditioned resid norm 1.995647707137e-02 true resid norm 2.680808177026e+12 ||r(i)||/||b|| 1.526657313121e+10 436 KSP preconditioned resid norm 1.988125326167e-02 true resid norm 2.348426065382e+12 ||r(i)||/||b|| 1.337373504663e+10 437 KSP preconditioned resid norm 1.898974930191e-02 true resid norm 1.218972146414e+12 ||r(i)||/||b|| 6.941760166809e+09 438 KSP preconditioned resid norm 1.696428871787e-02 true resid norm 3.026205745736e+11 ||r(i)||/||b|| 1.723353118783e+09 439 KSP preconditioned resid norm 1.589383908182e-02 true resid norm 1.469775790684e+12 ||r(i)||/||b|| 8.370028033799e+09 440 KSP preconditioned resid norm 1.578856928747e-02 true resid norm 1.654607025409e+12 ||r(i)||/||b|| 9.422598518346e+09 441 KSP preconditioned resid norm 1.551247109488e-02 true resid norm 1.442783527791e+12 ||r(i)||/||b|| 8.216313434240e+09 442 KSP preconditioned resid norm 1.452922185421e-02 true resid norm 4.930974509265e+11 ||r(i)||/||b|| 2.808074206835e+09 443 KSP preconditioned resid norm 1.322597574636e-02 true resid norm 6.258068580126e+11 ||r(i)||/||b|| 3.563823120854e+09 444 KSP preconditioned resid norm 1.248114500958e-02 true resid norm 1.255462163215e+12 ||r(i)||/||b|| 7.149562244861e+09 445 KSP preconditioned resid norm 1.232420552748e-02 true resid norm 1.428758773085e+12 ||r(i)||/||b|| 8.136445749113e+09 446 KSP preconditioned resid norm 1.210183666940e-02 true resid norm 1.113947512959e+12 ||r(i)||/||b|| 6.343669538405e+09 447 KSP preconditioned resid norm 1.098375596479e-02 true resid norm 6.623811396906e+11 ||r(i)||/||b|| 3.772105067598e+09 448 KSP preconditioned resid norm 1.016183389490e-02 true resid norm 1.004416279978e+11 ||r(i)||/||b|| 5.719914883828e+08 449 KSP preconditioned resid norm 8.946579568395e-03 true resid norm 3.185680683629e+11 ||r(i)||/||b|| 1.814170351541e+09 450 KSP preconditioned resid norm 8.941903973955e-03 true resid norm 3.415328417489e+11 ||r(i)||/||b|| 1.944949343989e+09 451 KSP preconditioned resid norm 8.941859110708e-03 true resid norm 3.414222834181e+11 ||r(i)||/||b|| 1.944319740254e+09 452 KSP preconditioned resid norm 8.937586314421e-03 true resid norm 3.373377321037e+11 ||r(i)||/||b|| 1.921059179545e+09 453 KSP preconditioned resid norm 8.857435678093e-03 true resid norm 2.969067995457e+11 ||r(i)||/||b|| 1.690814511557e+09 454 KSP preconditioned resid norm 8.484561477096e-03 true resid norm 1.648904346090e+11 ||r(i)||/||b|| 9.390123098578e+08 455 KSP preconditioned resid norm 8.064680909874e-03 true resid norm 6.485855267074e+10 ||r(i)||/||b|| 3.693542290782e+08 456 KSP preconditioned resid norm 7.880264947424e-03 true resid norm 7.274673519887e+10 ||r(i)||/||b|| 4.142755764801e+08 457 KSP preconditioned resid norm 7.593688239054e-03 true resid norm 1.445361395141e+11 ||r(i)||/||b|| 8.230993783529e+08 458 KSP preconditioned resid norm 7.462467793953e-03 true resid norm 2.021789922620e+11 ||r(i)||/||b|| 1.151361890572e+09 459 KSP preconditioned resid norm 7.446133179161e-03 true resid norm 1.788849926823e+11 ||r(i)||/||b|| 1.018708032250e+09 460 KSP preconditioned resid norm 7.348667644300e-03 true resid norm 1.054645854854e+11 ||r(i)||/||b|| 6.005960519154e+08 461 KSP preconditioned resid norm 7.111670463818e-03 true resid norm 8.554318742785e+10 ||r(i)||/||b|| 4.871483673973e+08 462 KSP preconditioned resid norm 6.928819885161e-03 true resid norm 1.869275057925e+11 ||r(i)||/||b|| 1.064508256081e+09 463 KSP preconditioned resid norm 6.911497814174e-03 true resid norm 2.062115664765e+11 ||r(i)||/||b|| 1.174326453900e+09 464 KSP preconditioned resid norm 6.905631098974e-03 true resid norm 1.906811421053e+11 ||r(i)||/||b|| 1.085884333553e+09 465 KSP preconditioned resid norm 6.716726820012e-03 true resid norm 9.813136700632e+10 ||r(i)||/||b|| 5.588350944710e+08 466 KSP preconditioned resid norm 6.199015524483e-03 true resid norm 1.333586080679e+11 ||r(i)||/||b|| 7.594459611812e+08 467 KSP preconditioned resid norm 5.532796739445e-03 true resid norm 3.569112996540e+11 ||r(i)||/||b|| 2.032526050993e+09 468 KSP preconditioned resid norm 5.360292734614e-03 true resid norm 4.755992375611e+11 ||r(i)||/||b|| 2.708425990190e+09 469 KSP preconditioned resid norm 5.342219637791e-03 true resid norm 5.151772931536e+11 ||r(i)||/||b|| 2.933813724110e+09 470 KSP preconditioned resid norm 5.332377875131e-03 true resid norm 4.601284207857e+11 ||r(i)||/||b|| 2.620323321105e+09 471 KSP preconditioned resid norm 5.189844708887e-03 true resid norm 1.790760088328e+11 ||r(i)||/||b|| 1.019795824378e+09 472 KSP preconditioned resid norm 5.049866465566e-03 true resid norm 5.152313973554e+10 ||r(i)||/||b|| 2.934121834836e+08 473 KSP preconditioned resid norm 4.924601896546e-03 true resid norm 1.677928034924e+11 ||r(i)||/||b|| 9.555406191908e+08 474 KSP preconditioned resid norm 4.924351655351e-03 true resid norm 1.738933523340e+11 ||r(i)||/||b|| 9.902818124731e+08 475 KSP preconditioned resid norm 4.924289830625e-03 true resid norm 1.772937150451e+11 ||r(i)||/||b|| 1.009646079729e+09 476 KSP preconditioned resid norm 4.876495146769e-03 true resid norm 7.947838694965e+10 ||r(i)||/||b|| 4.526107526511e+08 477 KSP preconditioned resid norm 4.778812651601e-03 true resid norm 1.558755330076e+11 ||r(i)||/||b|| 8.876745618800e+08 478 KSP preconditioned resid norm 4.755402622058e-03 true resid norm 2.205314865771e+11 ||r(i)||/||b|| 1.255875036646e+09 479 KSP preconditioned resid norm 4.755321537318e-03 true resid norm 2.242726552323e+11 ||r(i)||/||b|| 1.277180113734e+09 480 KSP preconditioned resid norm 4.644535254442e-03 true resid norm 7.822908083997e+10 ||r(i)||/||b|| 4.454962476857e+08 481 KSP preconditioned resid norm 4.632047119055e-03 true resid norm 7.991822757006e+10 ||r(i)||/||b|| 4.551155417125e+08 482 KSP preconditioned resid norm 4.612904266289e-03 true resid norm 8.633846252556e+10 ||r(i)||/||b|| 4.916772723532e+08 483 KSP preconditioned resid norm 4.610144234326e-03 true resid norm 9.030386603281e+10 ||r(i)||/||b|| 5.142593142751e+08 484 KSP preconditioned resid norm 4.604192845375e-03 true resid norm 8.230498241659e+10 ||r(i)||/||b|| 4.687075501684e+08 485 KSP preconditioned resid norm 4.564896521330e-03 true resid norm 6.060884575469e+10 ||r(i)||/||b|| 3.451531459958e+08 486 KSP preconditioned resid norm 4.546464306921e-03 true resid norm 4.872040768046e+10 ||r(i)||/||b|| 2.774512824938e+08 487 KSP preconditioned resid norm 4.526348213322e-03 true resid norm 4.510654738498e+10 ||r(i)||/||b|| 2.568711966229e+08 488 KSP preconditioned resid norm 4.523746573565e-03 true resid norm 4.683371377026e+10 ||r(i)||/||b|| 2.667070036592e+08 489 KSP preconditioned resid norm 4.522949666646e-03 true resid norm 4.552503866918e+10 ||r(i)||/||b|| 2.592544062272e+08 490 KSP preconditioned resid norm 4.504493562917e-03 true resid norm 5.209398625762e+10 ||r(i)||/||b|| 2.966630203957e+08 491 KSP preconditioned resid norm 4.424828850570e-03 true resid norm 1.544598998033e+11 ||r(i)||/||b|| 8.796128631636e+08 492 KSP preconditioned resid norm 4.338171328969e-03 true resid norm 2.844045719702e+11 ||r(i)||/||b|| 1.619617260959e+09 493 KSP preconditioned resid norm 4.255536926378e-03 true resid norm 4.382320759656e+11 ||r(i)||/||b|| 2.495628778479e+09 494 KSP preconditioned resid norm 4.229260600674e-03 true resid norm 5.282692430595e+11 ||r(i)||/||b|| 3.008369304916e+09 495 KSP preconditioned resid norm 4.223256517614e-03 true resid norm 4.857613635841e+11 ||r(i)||/||b|| 2.766296911887e+09 496 KSP preconditioned resid norm 4.136338595951e-03 true resid norm 3.512867560712e+11 ||r(i)||/||b|| 2.000495595897e+09 497 KSP preconditioned resid norm 3.946602463360e-03 true resid norm 1.832798358993e+11 ||r(i)||/||b|| 1.043735632490e+09 498 KSP preconditioned resid norm 3.453206785283e-03 true resid norm 1.333858086954e+11 ||r(i)||/||b|| 7.596008623685e+08 499 KSP preconditioned resid norm 3.126641781407e-03 true resid norm 3.661688364605e+11 ||r(i)||/||b|| 2.085245549495e+09 500 KSP preconditioned resid norm 3.072454365635e-03 true resid norm 4.367597724160e+11 ||r(i)||/||b|| 2.487244355452e+09 501 KSP preconditioned resid norm 3.069829058366e-03 true resid norm 4.195014946846e+11 ||r(i)||/||b|| 2.388962515907e+09 502 KSP preconditioned resid norm 2.883500007164e-03 true resid norm 1.789975834398e+11 ||r(i)||/||b|| 1.019349210179e+09 503 KSP preconditioned resid norm 2.412663722535e-03 true resid norm 1.641891705673e+11 ||r(i)||/||b|| 9.350187757928e+08 504 KSP preconditioned resid norm 2.352077771113e-03 true resid norm 2.578756112588e+11 ||r(i)||/||b|| 1.468541058542e+09 505 KSP preconditioned resid norm 2.349699627521e-03 true resid norm 2.420151436910e+11 ||r(i)||/||b|| 1.378219419681e+09 506 KSP preconditioned resid norm 2.253261082551e-03 true resid norm 1.477198125861e+11 ||r(i)||/||b|| 8.412296489915e+08 507 KSP preconditioned resid norm 2.117496752659e-03 true resid norm 2.502313428203e+10 ||r(i)||/||b|| 1.425008744611e+08 508 KSP preconditioned resid norm 2.020618048397e-03 true resid norm 6.554994550831e+10 ||r(i)||/||b|| 3.732915489534e+08 509 KSP preconditioned resid norm 1.992845917187e-03 true resid norm 9.910197077001e+10 ||r(i)||/||b|| 5.643624550136e+08 510 KSP preconditioned resid norm 1.962007082941e-03 true resid norm 8.209270536124e+10 ||r(i)||/||b|| 4.674986821795e+08 511 KSP preconditioned resid norm 1.961075306017e-03 true resid norm 8.239042222491e+10 ||r(i)||/||b|| 4.691941098161e+08 512 KSP preconditioned resid norm 1.956798179749e-03 true resid norm 8.497102060464e+10 ||r(i)||/||b|| 4.838900116803e+08 513 KSP preconditioned resid norm 1.956585953911e-03 true resid norm 8.663197416314e+10 ||r(i)||/||b|| 4.933487522145e+08 514 KSP preconditioned resid norm 1.911127535137e-03 true resid norm 5.254354959573e+10 ||r(i)||/||b|| 2.992231780516e+08 515 KSP preconditioned resid norm 1.825050318119e-03 true resid norm 1.205423016569e+10 ||r(i)||/||b|| 6.864601053593e+07 516 KSP preconditioned resid norm 1.702727067567e-03 true resid norm 8.691898767780e+10 ||r(i)||/||b|| 4.949832267915e+08 517 KSP preconditioned resid norm 1.575077603748e-03 true resid norm 1.678911784643e+11 ||r(i)||/||b|| 9.561008415579e+08 518 KSP preconditioned resid norm 1.553332075472e-03 true resid norm 1.915078454545e+11 ||r(i)||/||b|| 1.090592215022e+09 519 KSP preconditioned resid norm 1.552121970944e-03 true resid norm 1.846348983938e+11 ||r(i)||/||b|| 1.051452395236e+09 520 KSP preconditioned resid norm 1.518013706982e-03 true resid norm 1.426334653778e+11 ||r(i)||/||b|| 8.122640958829e+08 521 KSP preconditioned resid norm 1.410696251789e-03 true resid norm 2.885210286931e+10 ||r(i)||/||b|| 1.643059515478e+08 522 KSP preconditioned resid norm 1.267570105467e-03 true resid norm 1.187734543987e+11 ||r(i)||/||b|| 6.763869355376e+08 523 KSP preconditioned resid norm 1.227286861495e-03 true resid norm 1.525664114203e+11 ||r(i)||/||b|| 8.688298914012e+08 524 KSP preconditioned resid norm 1.217056207751e-03 true resid norm 1.718144113257e+11 ||r(i)||/||b|| 9.784427315529e+08 525 KSP preconditioned resid norm 1.186114694759e-03 true resid norm 1.373831599556e+11 ||r(i)||/||b|| 7.823648392420e+08 526 KSP preconditioned resid norm 1.115137377131e-03 true resid norm 7.634117128305e+10 ||r(i)||/||b|| 4.347450460284e+08 527 KSP preconditioned resid norm 1.036168964439e-03 true resid norm 2.813784237214e+10 ||r(i)||/||b|| 1.602384057203e+08 528 KSP preconditioned resid norm 1.008529357209e-03 true resid norm 9.431002050913e+09 ||r(i)||/||b|| 5.370734233977e+07 529 KSP preconditioned resid norm 1.002545899539e-03 true resid norm 1.580928344462e+10 ||r(i)||/||b|| 9.003015729646e+07 530 KSP preconditioned resid norm 1.001044030393e-03 true resid norm 1.102846358442e+10 ||r(i)||/||b|| 6.280451069913e+07 531 KSP preconditioned resid norm 9.852021598370e-04 true resid norm 4.195448622877e+10 ||r(i)||/||b|| 2.389209484224e+08 532 KSP preconditioned resid norm 9.691131299573e-04 true resid norm 7.663773271522e+10 ||r(i)||/||b|| 4.364338937538e+08 533 KSP preconditioned resid norm 9.562269910907e-04 true resid norm 1.076463013683e+11 ||r(i)||/||b|| 6.130204116154e+08 534 KSP preconditioned resid norm 9.545583556159e-04 true resid norm 1.135933278122e+11 ||r(i)||/||b|| 6.468873308892e+08 535 KSP preconditioned resid norm 9.470351105911e-04 true resid norm 9.687360218422e+10 ||r(i)||/||b|| 5.516724191245e+08 536 KSP preconditioned resid norm 9.174176771380e-04 true resid norm 6.066146213665e+10 ||r(i)||/||b|| 3.454527839371e+08 537 KSP preconditioned resid norm 8.636721736575e-04 true resid norm 1.993342436665e+10 ||r(i)||/||b|| 1.135161715249e+08 538 KSP preconditioned resid norm 8.192013866524e-04 true resid norm 1.450978609755e+10 ||r(i)||/||b|| 8.262982501873e+07 539 KSP preconditioned resid norm 8.061723688304e-04 true resid norm 3.106291289771e+10 ||r(i)||/||b|| 1.768959955752e+08 540 KSP preconditioned resid norm 8.061240703341e-04 true resid norm 3.052293614382e+10 ||r(i)||/||b|| 1.738209547449e+08 541 KSP preconditioned resid norm 8.061159019417e-04 true resid norm 3.058586668252e+10 ||r(i)||/||b|| 1.741793293872e+08 542 KSP preconditioned resid norm 8.055832977323e-04 true resid norm 2.940404373948e+10 ||r(i)||/||b|| 1.674491252111e+08 543 KSP preconditioned resid norm 8.011598611197e-04 true resid norm 2.471293892149e+10 ||r(i)||/||b|| 1.407343847147e+08 544 KSP preconditioned resid norm 7.921014666310e-04 true resid norm 1.837280168337e+10 ||r(i)||/||b|| 1.046287917681e+08 545 KSP preconditioned resid norm 7.833595745957e-04 true resid norm 1.321607741808e+10 ||r(i)||/||b|| 7.526245784384e+07 546 KSP preconditioned resid norm 7.773918326701e-04 true resid norm 1.030955860607e+10 ||r(i)||/||b|| 5.871051564180e+07 547 KSP preconditioned resid norm 7.745620001638e-04 true resid norm 9.112905031628e+09 ||r(i)||/||b|| 5.189585450213e+07 548 KSP preconditioned resid norm 7.740136037520e-04 true resid norm 8.419501078298e+09 ||r(i)||/||b|| 4.794708179482e+07 549 KSP preconditioned resid norm 7.722928268624e-04 true resid norm 1.049955839169e+10 ||r(i)||/||b|| 5.979251981015e+07 550 KSP preconditioned resid norm 7.664750762967e-04 true resid norm 1.858263223266e+10 ||r(i)||/||b|| 1.058237274794e+08 551 KSP preconditioned resid norm 7.563299862673e-04 true resid norm 3.297697455153e+10 ||r(i)||/||b|| 1.877961272840e+08 552 KSP preconditioned resid norm 7.543705761967e-04 true resid norm 3.822881948391e+10 ||r(i)||/||b|| 2.177041510736e+08 553 KSP preconditioned resid norm 7.542158399799e-04 true resid norm 4.001213009543e+10 ||r(i)||/||b|| 2.278596862961e+08 554 KSP preconditioned resid norm 7.521293265612e-04 true resid norm 3.403626346312e+10 ||r(i)||/||b|| 1.938285289211e+08 555 KSP preconditioned resid norm 7.416950965955e-04 true resid norm 2.488167973246e+10 ||r(i)||/||b|| 1.416953240139e+08 556 KSP preconditioned resid norm 7.054460671726e-04 true resid norm 1.069038545685e+10 ||r(i)||/||b|| 6.087923514128e+07 557 KSP preconditioned resid norm 6.723596233551e-04 true resid norm 4.826313315080e+10 ||r(i)||/||b|| 2.748472114126e+08 558 KSP preconditioned resid norm 6.659385082178e-04 true resid norm 5.999069197269e+10 ||r(i)||/||b|| 3.416329053460e+08 559 KSP preconditioned resid norm 6.658202350976e-04 true resid norm 6.156816562653e+10 ||r(i)||/||b|| 3.506162474237e+08 560 KSP preconditioned resid norm 6.547040506518e-04 true resid norm 4.666674715963e+10 ||r(i)||/||b|| 2.657561680144e+08 561 KSP preconditioned resid norm 5.844352666647e-04 true resid norm 5.248501827612e+09 ||r(i)||/||b|| 2.988898559292e+07 562 KSP preconditioned resid norm 5.077133854698e-04 true resid norm 4.911855677739e+10 ||r(i)||/||b|| 2.797186481179e+08 563 KSP preconditioned resid norm 4.920929576139e-04 true resid norm 6.479587544472e+10 ||r(i)||/||b|| 3.689972969922e+08 564 KSP preconditioned resid norm 4.918835649437e-04 true resid norm 6.729439671602e+10 ||r(i)||/||b|| 3.832257889951e+08 565 KSP preconditioned resid norm 4.703399310010e-04 true resid norm 3.485904866337e+10 ||r(i)||/||b|| 1.985140974517e+08 566 KSP preconditioned resid norm 4.474811061246e-04 true resid norm 3.541098025014e+09 ||r(i)||/||b|| 2.016572182483e+07 567 KSP preconditioned resid norm 4.230407891772e-04 true resid norm 3.192300153359e+10 ||r(i)||/||b|| 1.817939984131e+08 568 KSP preconditioned resid norm 4.218651971064e-04 true resid norm 3.754767103675e+10 ||r(i)||/||b|| 2.138251705964e+08 569 KSP preconditioned resid norm 4.165861285478e-04 true resid norm 2.702192475393e+10 ||r(i)||/||b|| 1.538835168950e+08 570 KSP preconditioned resid norm 3.988746281883e-04 true resid norm 1.564881336083e+10 ||r(i)||/||b|| 8.911631784665e+07 571 KSP preconditioned resid norm 3.979325947012e-04 true resid norm 1.580464073361e+10 ||r(i)||/||b|| 9.000371814734e+07 572 KSP preconditioned resid norm 3.918646655240e-04 true resid norm 1.724849279013e+10 ||r(i)||/||b|| 9.822611660180e+07 573 KSP preconditioned resid norm 3.813318868871e-04 true resid norm 2.183096822238e+10 ||r(i)||/||b|| 1.243222382520e+08 574 KSP preconditioned resid norm 3.812199206269e-04 true resid norm 2.117350124532e+10 ||r(i)||/||b|| 1.205781181867e+08 575 KSP preconditioned resid norm 3.775534780325e-04 true resid norm 1.545759995873e+10 ||r(i)||/||b|| 8.802740241740e+07 576 KSP preconditioned resid norm 3.663265117637e-04 true resid norm 2.709812573731e+09 ||r(i)||/||b|| 1.543174636039e+07 577 KSP preconditioned resid norm 3.469600788306e-04 true resid norm 2.104442127151e+10 ||r(i)||/||b|| 1.198430380430e+08 578 KSP preconditioned resid norm 3.218221400959e-04 true resid norm 4.827651986055e+10 ||r(i)||/||b|| 2.749234455815e+08 579 KSP preconditioned resid norm 3.159142633650e-04 true resid norm 5.947894664883e+10 ||r(i)||/||b|| 3.387186358812e+08 580 KSP preconditioned resid norm 3.158757784362e-04 true resid norm 5.853711398982e+10 ||r(i)||/||b|| 3.333551200245e+08 581 KSP preconditioned resid norm 3.039420517331e-04 true resid norm 4.030471897495e+10 ||r(i)||/||b|| 2.295259112669e+08 582 KSP preconditioned resid norm 2.757600327684e-04 true resid norm 4.482197601150e+09 ||r(i)||/||b|| 2.552506294665e+07 583 KSP preconditioned resid norm 2.596438684754e-04 true resid norm 1.172878967238e+10 ||r(i)||/||b|| 6.679270333789e+07 584 KSP preconditioned resid norm 2.528824194106e-04 true resid norm 2.182545628648e+10 ||r(i)||/||b|| 1.242908490712e+08 585 KSP preconditioned resid norm 2.509394596484e-04 true resid norm 2.387508027483e+10 ||r(i)||/||b|| 1.359629764460e+08 586 KSP preconditioned resid norm 2.507880787449e-04 true resid norm 2.241355174256e+10 ||r(i)||/||b|| 1.276399146124e+08 587 KSP preconditioned resid norm 2.479827266195e-04 true resid norm 1.577647541424e+10 ||r(i)||/||b|| 8.984332326657e+07 588 KSP preconditioned resid norm 2.416639962173e-04 true resid norm 6.931502976215e+09 ||r(i)||/||b|| 3.947328197609e+07 589 KSP preconditioned resid norm 2.397800547684e-04 true resid norm 3.531160199136e+09 ||r(i)||/||b|| 2.010912823980e+07 590 KSP preconditioned resid norm 2.397581300989e-04 true resid norm 3.272823788840e+09 ||r(i)||/||b|| 1.863796303893e+07 591 KSP preconditioned resid norm 2.381927700788e-04 true resid norm 4.322328355329e+09 ||r(i)||/||b|| 2.461464512799e+07 592 KSP preconditioned resid norm 2.295688690620e-04 true resid norm 1.084571125107e+10 ||r(i)||/||b|| 6.176377906981e+07 593 KSP preconditioned resid norm 2.191296967620e-04 true resid norm 1.440682509655e+10 ||r(i)||/||b|| 8.204348629258e+07 594 KSP preconditioned resid norm 2.131890464017e-04 true resid norm 1.802610318890e+10 ||r(i)||/||b|| 1.026544252447e+08 595 KSP preconditioned resid norm 2.128615750391e-04 true resid norm 1.926485527393e+10 ||r(i)||/||b|| 1.097088275178e+08 596 KSP preconditioned resid norm 2.103093424487e-04 true resid norm 1.601158355520e+10 ||r(i)||/||b|| 9.118220892741e+07 597 KSP preconditioned resid norm 2.002703326724e-04 true resid norm 9.011830263785e+09 ||r(i)||/||b|| 5.132025743099e+07 598 KSP preconditioned resid norm 1.812528643223e-04 true resid norm 1.855743689471e+09 ||r(i)||/||b|| 1.056802459455e+07 599 KSP preconditioned resid norm 1.648481554293e-04 true resid norm 1.185373601681e+10 ||r(i)||/||b|| 6.750424343282e+07 600 KSP preconditioned resid norm 1.648277142351e-04 true resid norm 1.174086964094e+10 ||r(i)||/||b|| 6.686149592254e+07 601 KSP preconditioned resid norm 1.648266487826e-04 true resid norm 1.175065536477e+10 ||r(i)||/||b|| 6.691722332212e+07 602 KSP preconditioned resid norm 1.645364378275e-04 true resid norm 1.128883836019e+10 ||r(i)||/||b|| 6.428728391279e+07 603 KSP preconditioned resid norm 1.617283351697e-04 true resid norm 8.666047026016e+09 ||r(i)||/||b|| 4.935110307963e+07 604 KSP preconditioned resid norm 1.563495331577e-04 true resid norm 4.638424448370e+09 ||r(i)||/||b|| 2.641473816047e+07 605 KSP preconditioned resid norm 1.532577905154e-04 true resid norm 2.037953363833e+09 ||r(i)||/||b|| 1.160566590835e+07 606 KSP preconditioned resid norm 1.529790769593e-04 true resid norm 1.685952897620e+09 ||r(i)||/||b|| 9.601105900775e+06 607 KSP preconditioned resid norm 1.528899627964e-04 true resid norm 1.550807594371e+09 ||r(i)||/||b|| 8.831485130041e+06 608 KSP preconditioned resid norm 1.528768842246e-04 true resid norm 1.498982994341e+09 ||r(i)||/||b|| 8.536356200960e+06 609 KSP preconditioned resid norm 1.523478881665e-04 true resid norm 2.726800476436e+09 ||r(i)||/||b|| 1.552848847764e+07 610 KSP preconditioned resid norm 1.505914549230e-04 true resid norm 6.968816215625e+09 ||r(i)||/||b|| 3.968577211361e+07 611 KSP preconditioned resid norm 1.474807681861e-04 true resid norm 1.307410306868e+10 ||r(i)||/||b|| 7.445394725865e+07 612 KSP preconditioned resid norm 1.468510511764e-04 true resid norm 1.567946182273e+10 ||r(i)||/||b|| 8.929085364105e+07 613 KSP preconditioned resid norm 1.468151717479e-04 true resid norm 1.504109300596e+10 ||r(i)||/||b|| 8.565549311459e+07 614 KSP preconditioned resid norm 1.461845224480e-04 true resid norm 1.281876147133e+10 ||r(i)||/||b|| 7.299983681440e+07 615 KSP preconditioned resid norm 1.436626916314e-04 true resid norm 8.812476417186e+09 ||r(i)||/||b|| 5.018498408164e+07 616 KSP preconditioned resid norm 1.384034870969e-04 true resid norm 1.718785095522e+09 ||r(i)||/||b|| 9.788077559032e+06 617 KSP preconditioned resid norm 1.324630273582e-04 true resid norm 9.086766624892e+09 ||r(i)||/||b|| 5.174700241291e+07 618 KSP preconditioned resid norm 1.296278937868e-04 true resid norm 1.379006862770e+10 ||r(i)||/||b|| 7.853120301305e+07 619 KSP preconditioned resid norm 1.295296207841e-04 true resid norm 1.451894315250e+10 ||r(i)||/||b|| 8.268197229661e+07 620 KSP preconditioned resid norm 1.261318135745e-04 true resid norm 1.061166004551e+10 ||r(i)||/||b|| 6.043091240788e+07 621 KSP preconditioned resid norm 1.159951195720e-04 true resid norm 4.640385578344e+09 ||r(i)||/||b|| 2.642590633522e+07 622 KSP preconditioned resid norm 1.019086335187e-04 true resid norm 5.487423615005e+09 ||r(i)||/||b|| 3.124958907478e+07 623 KSP preconditioned resid norm 9.684434695139e-05 true resid norm 9.319082628344e+09 ||r(i)||/||b|| 5.306998750623e+07 624 KSP preconditioned resid norm 9.654335078091e-05 true resid norm 1.026751692044e+10 ||r(i)||/||b|| 5.847109811327e+07 625 KSP preconditioned resid norm 9.586472328416e-05 true resid norm 8.659828448981e+09 ||r(i)||/||b|| 4.931568974350e+07 626 KSP preconditioned resid norm 9.282628606304e-05 true resid norm 4.798713692328e+09 ||r(i)||/||b|| 2.732754776991e+07 KSP Object: 4 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-10, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 4 MPI processes type: fieldsplit FieldSplit with Schur preconditioner, blocksize = 1, factorization LOWER Preconditioner for the Schur complement formed from S itself Split info: Split number 0 Defined by IS Split number 1 Defined by IS KSP solver for A00 block KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: gamg MG: type is MULTIPLICATIVE, levels=3 cycles=v Cycles per PCApply=1 Using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 1.01207 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 package used to perform factorization: petsc total: nonzeros=503, allocated nonzeros=503 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 18 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 18 nodes, limit used is 5 Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.0821344, max = 1.72482 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=1019, cols=1019 total: nonzeros=34719, allocated nonzeros=34719 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.156902, max = 3.29495 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 KSP solver for S = A11 - A10 inv(A00) A01 KSP Object: (fieldsplit_1_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_) 4 MPI processes type: lsc KSP Object: (fieldsplit_1_lsc_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_1_lsc_sub_) 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 package used to perform factorization: petsc total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=0, cols=0 total: nonzeros=0, allocated nonzeros=0 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=9549, allocated nonzeros=9549 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines linear system matrix = precond matrix: Mat Object: (fieldsplit_1_) 4 MPI processes type: schurcomplement rows=3183, cols=3183 Schur complement A11 - A10 inv(A00) A01 A11 Mat Object: (fieldsplit_1_) 4 MPI processes type: mpiaij rows=3183, cols=3183 total: nonzeros=3183, allocated nonzeros=3183 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines A10 Mat Object: 4 MPI processes type: mpiaij rows=3183, cols=71946 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines KSP of A00 KSP Object: (fieldsplit_0_) 4 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_) 4 MPI processes type: gamg MG: type is MULTIPLICATIVE, levels=3 cycles=v Cycles per PCApply=1 Using Galerkin computed coarse grid matrices Coarse grid solver -- level ------------------------------- KSP Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_) 4 MPI processes type: bjacobi block Jacobi: number of blocks = 4 Local solve is same for all blocks, in the following KSP and PC objects: KSP Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_coarse_sub_) 1 MPI processes type: lu LU: out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd factor fill ratio given 5, needed 1.01207 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 package used to perform factorization: petsc total: nonzeros=503, allocated nonzeros=503 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 15 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node routines: found 18 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=23, cols=23 total: nonzeros=497, allocated nonzeros=497 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 18 nodes, limit used is 5 Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.0821344, max = 1.72482 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_1_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=1019, cols=1019 total: nonzeros=34719, allocated nonzeros=34719 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: chebyshev Chebyshev: eigenvalue estimates: min = 0.156902, max = 3.29495 maximum iterations=2 tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using nonzero initial guess using NONE norm type for convergence test PC Object: (fieldsplit_0_mg_levels_2_) 4 MPI processes type: sor SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1 linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: (fieldsplit_0_) 4 MPI processes type: mpiaij rows=71946, cols=71946 total: nonzeros=2.96023e+06, allocated nonzeros=2.96023e+06 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 A01 Mat Object: 4 MPI processes type: mpiaij rows=71946, cols=3183 total: nonzeros=19098, allocated nonzeros=19098 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 3757 nodes, limit used is 5 linear system matrix = precond matrix: Mat Object: 4 MPI processes type: mpiaij rows=75129, cols=75129 total: nonzeros=3.0016e+06, allocated nonzeros=1.26217e+07 total number of mallocs used during MatSetValues calls =0 using I-node (on process 0) routines: found 6259 nodes, limit used is 5 Recovering stress ... Cleaning up ... Finished 0.01user 0.02system 0:16.80elapsed 0%CPU (0avgtext+0avgdata 17056maxresident)k 904inputs+0outputs (3major+1806minor)pagefaults 0swaps salloc: Relinquishing job allocation 79008 -------------- next part -------------- salloc: Granted job allocation 79007 Reading input ... Partitioning mesh ... Reading mesh data ... Forming [K] ... Applying constraints ... Forming RHS ... Setting up solver ... Solving ... Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 0.000000000000e+00 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.458554290527e-03 1 KSP Residual norm 2.443523132385e-04 2 KSP Residual norm 1.526975625938e-04 3 KSP Residual norm 2.014215872825e-04 4 KSP Residual norm 1.476966271575e-04 5 KSP Residual norm 9.122757243249e-05 6 KSP Residual norm 1.047856181139e-04 7 KSP Residual norm 1.147488524595e-04 8 KSP Residual norm 1.045417032000e-04 9 KSP Residual norm 1.079705732917e-04 10 KSP Residual norm 8.497852252670e-05 11 KSP Residual norm 6.831445728983e-05 12 KSP Residual norm 6.954068462504e-05 13 KSP Residual norm 6.093026980090e-05 14 KSP Residual norm 5.221858684731e-05 15 KSP Residual norm 4.677505495486e-05 16 KSP Residual norm 4.144856671247e-05 17 KSP Residual norm 3.542015548564e-05 18 KSP Residual norm 3.077721287010e-05 19 KSP Residual norm 2.900753681509e-05 20 KSP Residual norm 2.876772232961e-05 21 KSP Residual norm 2.782865244825e-05 22 KSP Residual norm 2.536365456623e-05 23 KSP Residual norm 2.243212162085e-05 24 KSP Residual norm 1.967158760290e-05 25 KSP Residual norm 1.680895651382e-05 26 KSP Residual norm 1.356166055062e-05 27 KSP Residual norm 1.054810017706e-05 28 KSP Residual norm 8.465627851240e-06 29 KSP Residual norm 6.516458160634e-06 30 KSP Residual norm 5.852661265719e-06 31 KSP Residual norm 5.778044905151e-06 32 KSP Residual norm 5.526592479324e-06 33 KSP Residual norm 5.377995774695e-06 34 KSP Residual norm 4.766413673254e-06 35 KSP Residual norm 3.725169370300e-06 36 KSP Residual norm 2.858872754067e-06 37 KSP Residual norm 2.429697062121e-06 38 KSP Residual norm 2.147367585432e-06 39 KSP Residual norm 2.001025086617e-06 40 KSP Residual norm 1.863348682851e-06 41 KSP Residual norm 1.740846296551e-06 42 KSP Residual norm 1.607907436767e-06 43 KSP Residual norm 1.360087402410e-06 44 KSP Residual norm 1.229680860159e-06 45 KSP Residual norm 1.107487969698e-06 46 KSP Residual norm 9.858758145950e-07 47 KSP Residual norm 8.476936047597e-07 48 KSP Residual norm 7.205399850111e-07 49 KSP Residual norm 6.287921992929e-07 50 KSP Residual norm 4.887400516822e-07 51 KSP Residual norm 3.786489336871e-07 52 KSP Residual norm 3.250515666351e-07 53 KSP Residual norm 2.935685470083e-07 54 KSP Residual norm 2.683763781902e-07 55 KSP Residual norm 2.520124366175e-07 56 KSP Residual norm 2.081511612269e-07 57 KSP Residual norm 1.926821572998e-07 58 KSP Residual norm 1.739477707764e-07 59 KSP Residual norm 1.527781085294e-07 60 KSP Residual norm 1.386486593697e-07 61 KSP Residual norm 1.380585309557e-07 62 KSP Residual norm 1.241062083013e-07 63 KSP Residual norm 1.075235601659e-07 64 KSP Residual norm 1.022859259930e-07 65 KSP Residual norm 9.416544583143e-08 66 KSP Residual norm 7.443131271013e-08 67 KSP Residual norm 5.726970118462e-08 68 KSP Residual norm 4.763134320298e-08 69 KSP Residual norm 3.795994540583e-08 70 KSP Residual norm 3.095116957380e-08 71 KSP Residual norm 2.504320617006e-08 72 KSP Residual norm 2.197758476947e-08 73 KSP Residual norm 1.949130087394e-08 74 KSP Residual norm 1.447911930994e-08 1 KSP preconditioned resid norm 9.127511072040e+05 true resid norm 1.362119811364e+14 ||r(i)||/||b|| 7.756952508524e+11 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.118159765853e-03 1 KSP Residual norm 6.335215968361e-04 2 KSP Residual norm 1.003527880099e-03 3 KSP Residual norm 2.425186737584e-03 4 KSP Residual norm 1.545924874557e-03 5 KSP Residual norm 1.135719649755e-03 6 KSP Residual norm 1.122356230013e-03 7 KSP Residual norm 7.845893770358e-04 8 KSP Residual norm 5.530803114162e-04 9 KSP Residual norm 5.458296360672e-04 10 KSP Residual norm 3.887287897385e-04 11 KSP Residual norm 3.684623158531e-04 12 KSP Residual norm 3.395071253152e-04 13 KSP Residual norm 2.772093983820e-04 14 KSP Residual norm 2.501947745913e-04 15 KSP Residual norm 2.218579795335e-04 16 KSP Residual norm 2.181304610977e-04 17 KSP Residual norm 1.935956621137e-04 18 KSP Residual norm 1.715388915405e-04 19 KSP Residual norm 1.522412948980e-04 20 KSP Residual norm 1.277575863633e-04 21 KSP Residual norm 1.056621401075e-04 22 KSP Residual norm 7.498521323183e-05 23 KSP Residual norm 6.062531233758e-05 24 KSP Residual norm 5.479547449311e-05 25 KSP Residual norm 4.885757554560e-05 26 KSP Residual norm 4.029801003083e-05 27 KSP Residual norm 3.591733350388e-05 28 KSP Residual norm 3.337761887714e-05 29 KSP Residual norm 3.198393580179e-05 30 KSP Residual norm 3.069961534982e-05 31 KSP Residual norm 2.955390544445e-05 32 KSP Residual norm 2.598688701551e-05 33 KSP Residual norm 2.203998557497e-05 34 KSP Residual norm 1.870530768035e-05 35 KSP Residual norm 1.516729851782e-05 36 KSP Residual norm 1.328472104150e-05 37 KSP Residual norm 1.112484772139e-05 38 KSP Residual norm 9.348277197652e-06 39 KSP Residual norm 8.081145286199e-06 40 KSP Residual norm 7.348530292858e-06 41 KSP Residual norm 7.157909756123e-06 42 KSP Residual norm 6.806355468615e-06 43 KSP Residual norm 6.072841029014e-06 44 KSP Residual norm 5.173333653906e-06 45 KSP Residual norm 3.812675096393e-06 46 KSP Residual norm 2.975956719089e-06 47 KSP Residual norm 2.377042720804e-06 48 KSP Residual norm 2.153333750758e-06 49 KSP Residual norm 1.993041019384e-06 50 KSP Residual norm 2.082172224275e-06 51 KSP Residual norm 2.145205085477e-06 52 KSP Residual norm 1.831483680779e-06 53 KSP Residual norm 1.460931622910e-06 54 KSP Residual norm 1.261318452725e-06 55 KSP Residual norm 1.159293828002e-06 56 KSP Residual norm 9.154894198031e-07 57 KSP Residual norm 7.611453343045e-07 58 KSP Residual norm 6.786785453445e-07 59 KSP Residual norm 6.625510979426e-07 60 KSP Residual norm 5.976351205378e-07 61 KSP Residual norm 4.781690371194e-07 62 KSP Residual norm 3.938124531241e-07 63 KSP Residual norm 2.955258894181e-07 64 KSP Residual norm 2.188814810276e-07 65 KSP Residual norm 1.629996919271e-07 66 KSP Residual norm 1.298294484605e-07 67 KSP Residual norm 1.003724704125e-07 68 KSP Residual norm 7.619728392688e-08 69 KSP Residual norm 5.935877330937e-08 70 KSP Residual norm 5.251359439852e-08 71 KSP Residual norm 4.697585582364e-08 72 KSP Residual norm 4.087074088091e-08 73 KSP Residual norm 3.256494389770e-08 74 KSP Residual norm 2.438036988038e-08 75 KSP Residual norm 2.016970749018e-08 2 KSP preconditioned resid norm 8.626934097311e+05 true resid norm 7.140669210036e+14 ||r(i)||/||b|| 4.066443456678e+12 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.121634416290e-03 1 KSP Residual norm 5.832590584079e-04 2 KSP Residual norm 8.933603398054e-04 3 KSP Residual norm 2.014663205673e-03 4 KSP Residual norm 1.165196521648e-03 5 KSP Residual norm 9.626894660113e-04 6 KSP Residual norm 9.220499099680e-04 7 KSP Residual norm 7.532447592303e-04 8 KSP Residual norm 5.781332011526e-04 9 KSP Residual norm 5.373510027896e-04 10 KSP Residual norm 4.083768312978e-04 11 KSP Residual norm 3.895941176461e-04 12 KSP Residual norm 3.680749227535e-04 13 KSP Residual norm 3.243100026463e-04 14 KSP Residual norm 3.191216677156e-04 15 KSP Residual norm 2.816828178821e-04 16 KSP Residual norm 2.684434998550e-04 17 KSP Residual norm 2.492988047891e-04 18 KSP Residual norm 1.946006691240e-04 19 KSP Residual norm 1.577535528055e-04 20 KSP Residual norm 1.197098848127e-04 21 KSP Residual norm 9.833084120150e-05 22 KSP Residual norm 7.714351102884e-05 23 KSP Residual norm 6.471381928356e-05 24 KSP Residual norm 5.764240452275e-05 25 KSP Residual norm 4.766450480650e-05 26 KSP Residual norm 3.967647015185e-05 27 KSP Residual norm 3.601749096709e-05 28 KSP Residual norm 3.135071159310e-05 29 KSP Residual norm 2.743607943832e-05 30 KSP Residual norm 2.333891385128e-05 31 KSP Residual norm 2.065291384346e-05 32 KSP Residual norm 1.632157339076e-05 33 KSP Residual norm 1.385904616531e-05 34 KSP Residual norm 1.216123385251e-05 35 KSP Residual norm 1.119791939626e-05 36 KSP Residual norm 1.018093123240e-05 37 KSP Residual norm 8.436740028435e-06 38 KSP Residual norm 7.318549276463e-06 39 KSP Residual norm 6.247323885315e-06 40 KSP Residual norm 5.755644460560e-06 41 KSP Residual norm 5.524921970483e-06 42 KSP Residual norm 5.476592921821e-06 43 KSP Residual norm 5.345262846172e-06 44 KSP Residual norm 4.383201474491e-06 45 KSP Residual norm 3.058930013757e-06 46 KSP Residual norm 2.326928382096e-06 47 KSP Residual norm 1.833177617474e-06 48 KSP Residual norm 1.486141936302e-06 49 KSP Residual norm 1.268775280871e-06 50 KSP Residual norm 1.014764686856e-06 51 KSP Residual norm 8.712527430261e-07 52 KSP Residual norm 7.448330261622e-07 53 KSP Residual norm 6.331415929439e-07 54 KSP Residual norm 5.667390248714e-07 55 KSP Residual norm 5.222098846876e-07 56 KSP Residual norm 4.269429224062e-07 57 KSP Residual norm 3.988402738064e-07 58 KSP Residual norm 3.986042871115e-07 59 KSP Residual norm 3.766809542061e-07 60 KSP Residual norm 3.631236153372e-07 61 KSP Residual norm 3.423133948992e-07 62 KSP Residual norm 3.054262356047e-07 63 KSP Residual norm 2.504194459945e-07 64 KSP Residual norm 1.904813285389e-07 65 KSP Residual norm 1.588771497818e-07 66 KSP Residual norm 1.453420252582e-07 67 KSP Residual norm 1.217809505240e-07 68 KSP Residual norm 1.009800586139e-07 69 KSP Residual norm 9.254027255920e-08 70 KSP Residual norm 8.562839467242e-08 71 KSP Residual norm 7.073129996800e-08 72 KSP Residual norm 5.324506678657e-08 73 KSP Residual norm 4.342118023774e-08 74 KSP Residual norm 3.584371992560e-08 75 KSP Residual norm 3.149002455231e-08 76 KSP Residual norm 2.765584156115e-08 77 KSP Residual norm 2.203726144736e-08 78 KSP Residual norm 1.724483256904e-08 3 KSP preconditioned resid norm 6.917411796180e+05 true resid norm 2.788569211433e+15 ||r(i)||/||b|| 1.588024692053e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.112183529852e-03 1 KSP Residual norm 5.455688303432e-04 2 KSP Residual norm 8.453301842306e-04 3 KSP Residual norm 1.072687137328e-03 4 KSP Residual norm 7.111872271491e-04 5 KSP Residual norm 7.872689524016e-04 6 KSP Residual norm 6.182681208124e-04 7 KSP Residual norm 6.176186968745e-04 8 KSP Residual norm 5.146246114208e-04 9 KSP Residual norm 4.461815584742e-04 10 KSP Residual norm 3.755214099804e-04 11 KSP Residual norm 3.030289038255e-04 12 KSP Residual norm 2.521097757210e-04 13 KSP Residual norm 2.043666764161e-04 14 KSP Residual norm 1.789058586696e-04 15 KSP Residual norm 1.658756267215e-04 16 KSP Residual norm 1.448707635576e-04 17 KSP Residual norm 1.263635532211e-04 18 KSP Residual norm 1.175427664014e-04 19 KSP Residual norm 1.068846873611e-04 20 KSP Residual norm 1.016024893865e-04 21 KSP Residual norm 8.769970709747e-05 22 KSP Residual norm 7.471728877888e-05 23 KSP Residual norm 6.986780473420e-05 24 KSP Residual norm 6.238626893677e-05 25 KSP Residual norm 5.363693929622e-05 26 KSP Residual norm 4.611227663271e-05 27 KSP Residual norm 4.196926220422e-05 28 KSP Residual norm 4.116254475716e-05 29 KSP Residual norm 4.214702881118e-05 30 KSP Residual norm 3.736499055738e-05 31 KSP Residual norm 3.086684423347e-05 32 KSP Residual norm 2.277655266281e-05 33 KSP Residual norm 1.897354363037e-05 34 KSP Residual norm 1.700980333378e-05 35 KSP Residual norm 1.541184065153e-05 36 KSP Residual norm 1.392044716605e-05 37 KSP Residual norm 1.195512207648e-05 38 KSP Residual norm 9.752529615505e-06 39 KSP Residual norm 9.242458286882e-06 40 KSP Residual norm 8.955493048486e-06 41 KSP Residual norm 8.225498630135e-06 42 KSP Residual norm 7.928867871685e-06 43 KSP Residual norm 7.270708896890e-06 44 KSP Residual norm 5.822540627390e-06 45 KSP Residual norm 5.078410819163e-06 46 KSP Residual norm 4.625181589827e-06 47 KSP Residual norm 3.837005260832e-06 48 KSP Residual norm 3.300458858425e-06 49 KSP Residual norm 3.151686106849e-06 50 KSP Residual norm 3.235837806350e-06 51 KSP Residual norm 2.919310420550e-06 52 KSP Residual norm 2.410548522036e-06 53 KSP Residual norm 1.911164510170e-06 54 KSP Residual norm 1.539027190792e-06 55 KSP Residual norm 1.462542934336e-06 56 KSP Residual norm 1.437185862915e-06 57 KSP Residual norm 1.444828984891e-06 58 KSP Residual norm 1.156933380849e-06 59 KSP Residual norm 1.037462808644e-06 60 KSP Residual norm 8.517004248871e-07 61 KSP Residual norm 6.178096950995e-07 62 KSP Residual norm 5.428127074882e-07 63 KSP Residual norm 5.092075166993e-07 64 KSP Residual norm 4.303559611141e-07 65 KSP Residual norm 3.182951762614e-07 66 KSP Residual norm 2.410774738927e-07 67 KSP Residual norm 1.827720465310e-07 68 KSP Residual norm 1.470587842470e-07 69 KSP Residual norm 1.214345122157e-07 70 KSP Residual norm 9.937590548964e-08 71 KSP Residual norm 7.966377026472e-08 72 KSP Residual norm 6.203672035646e-08 73 KSP Residual norm 5.685137232389e-08 74 KSP Residual norm 4.932632522158e-08 75 KSP Residual norm 3.973551506940e-08 76 KSP Residual norm 3.343764268923e-08 77 KSP Residual norm 2.808344449328e-08 78 KSP Residual norm 2.370724255012e-08 79 KSP Residual norm 1.979003937285e-08 80 KSP Residual norm 1.568358754511e-08 81 KSP Residual norm 1.319722769947e-08 82 KSP Residual norm 1.135448200817e-08 83 KSP Residual norm 9.108819584852e-09 4 KSP preconditioned resid norm 5.948195614265e+05 true resid norm 3.828393420795e+15 ||r(i)||/||b|| 2.180180164865e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.366134666029e-03 1 KSP Residual norm 6.190808191358e-04 2 KSP Residual norm 9.027995442355e-04 3 KSP Residual norm 1.887849221230e-03 4 KSP Residual norm 1.339477414022e-03 5 KSP Residual norm 1.150633711807e-03 6 KSP Residual norm 9.400210086583e-04 7 KSP Residual norm 8.457671922067e-04 8 KSP Residual norm 6.363782000781e-04 9 KSP Residual norm 5.502091992996e-04 10 KSP Residual norm 4.164886285662e-04 11 KSP Residual norm 3.417283869063e-04 12 KSP Residual norm 2.638604858957e-04 13 KSP Residual norm 2.045911104075e-04 14 KSP Residual norm 2.027281229106e-04 15 KSP Residual norm 1.859072831032e-04 16 KSP Residual norm 1.837118459745e-04 17 KSP Residual norm 1.708411329875e-04 18 KSP Residual norm 1.447258634246e-04 19 KSP Residual norm 1.401489027174e-04 20 KSP Residual norm 1.154278749177e-04 21 KSP Residual norm 9.522454071119e-05 22 KSP Residual norm 8.088954011184e-05 23 KSP Residual norm 6.707919256941e-05 24 KSP Residual norm 5.760475943156e-05 25 KSP Residual norm 5.255878572114e-05 26 KSP Residual norm 4.645626237734e-05 27 KSP Residual norm 3.979799793971e-05 28 KSP Residual norm 3.541020304317e-05 29 KSP Residual norm 3.284484054434e-05 30 KSP Residual norm 3.408695693072e-05 31 KSP Residual norm 3.476264544016e-05 32 KSP Residual norm 3.191018564080e-05 33 KSP Residual norm 2.789884934232e-05 34 KSP Residual norm 2.360312270493e-05 35 KSP Residual norm 2.224559533273e-05 36 KSP Residual norm 1.965779470703e-05 37 KSP Residual norm 1.520750257749e-05 38 KSP Residual norm 1.190569367477e-05 39 KSP Residual norm 9.575387075546e-06 40 KSP Residual norm 8.584494703270e-06 41 KSP Residual norm 7.975779046071e-06 42 KSP Residual norm 7.583538058107e-06 43 KSP Residual norm 7.079686038979e-06 44 KSP Residual norm 6.461134253262e-06 45 KSP Residual norm 5.492538180627e-06 46 KSP Residual norm 4.332693373689e-06 47 KSP Residual norm 3.288246918674e-06 48 KSP Residual norm 2.891220636972e-06 49 KSP Residual norm 2.805230665973e-06 50 KSP Residual norm 2.529889749263e-06 51 KSP Residual norm 2.289109317321e-06 52 KSP Residual norm 2.083997675248e-06 53 KSP Residual norm 1.678328539832e-06 54 KSP Residual norm 1.463627970874e-06 55 KSP Residual norm 1.326615803618e-06 56 KSP Residual norm 1.183287739943e-06 57 KSP Residual norm 1.099902486703e-06 58 KSP Residual norm 9.711017638628e-07 59 KSP Residual norm 8.717737263683e-07 60 KSP Residual norm 7.158813669130e-07 61 KSP Residual norm 5.555211434149e-07 62 KSP Residual norm 4.364635863369e-07 63 KSP Residual norm 3.273158469623e-07 64 KSP Residual norm 2.408346771072e-07 65 KSP Residual norm 1.791539103301e-07 66 KSP Residual norm 1.471306759533e-07 67 KSP Residual norm 1.117364600410e-07 68 KSP Residual norm 9.605693098442e-08 69 KSP Residual norm 8.362357228390e-08 70 KSP Residual norm 6.995263596925e-08 71 KSP Residual norm 6.244814108441e-08 72 KSP Residual norm 5.145266972274e-08 73 KSP Residual norm 4.096018528648e-08 74 KSP Residual norm 3.211624329837e-08 75 KSP Residual norm 2.803590960630e-08 76 KSP Residual norm 2.578079591351e-08 77 KSP Residual norm 2.191141346209e-08 5 KSP preconditioned resid norm 5.365287940041e+05 true resid norm 4.508998341363e+15 ||r(i)||/||b|| 2.567768686951e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.488312829038e-03 1 KSP Residual norm 6.460112936415e-04 2 KSP Residual norm 1.056712758220e-03 3 KSP Residual norm 1.745811175619e-03 4 KSP Residual norm 1.093364613342e-03 5 KSP Residual norm 9.414266168559e-04 6 KSP Residual norm 7.787487001053e-04 7 KSP Residual norm 5.599829545994e-04 8 KSP Residual norm 4.408760623419e-04 9 KSP Residual norm 4.446319636040e-04 10 KSP Residual norm 3.740975490781e-04 11 KSP Residual norm 3.571189071501e-04 12 KSP Residual norm 3.260620628953e-04 13 KSP Residual norm 2.681837780733e-04 14 KSP Residual norm 2.448807205660e-04 15 KSP Residual norm 1.960603432773e-04 16 KSP Residual norm 1.757459789551e-04 17 KSP Residual norm 1.383133100717e-04 18 KSP Residual norm 1.084657557844e-04 19 KSP Residual norm 8.876117375686e-05 20 KSP Residual norm 7.411090297872e-05 21 KSP Residual norm 6.177947581896e-05 22 KSP Residual norm 5.998575104883e-05 23 KSP Residual norm 6.202820594681e-05 24 KSP Residual norm 5.799270894202e-05 25 KSP Residual norm 6.024437873714e-05 26 KSP Residual norm 5.453680776174e-05 27 KSP Residual norm 4.400235720632e-05 28 KSP Residual norm 4.043795686355e-05 29 KSP Residual norm 3.540341520985e-05 30 KSP Residual norm 3.006268590824e-05 31 KSP Residual norm 2.568008761880e-05 32 KSP Residual norm 2.052479757148e-05 33 KSP Residual norm 1.968115039631e-05 34 KSP Residual norm 1.676990886905e-05 35 KSP Residual norm 1.427190005100e-05 36 KSP Residual norm 1.223456101213e-05 37 KSP Residual norm 1.076187951184e-05 38 KSP Residual norm 9.830057519170e-06 39 KSP Residual norm 8.863874585165e-06 40 KSP Residual norm 8.568008042638e-06 41 KSP Residual norm 7.819005069427e-06 42 KSP Residual norm 7.463291453451e-06 43 KSP Residual norm 7.206096105249e-06 44 KSP Residual norm 6.477872180838e-06 45 KSP Residual norm 5.514540533508e-06 46 KSP Residual norm 4.548975927847e-06 47 KSP Residual norm 4.103764888127e-06 48 KSP Residual norm 3.871548983945e-06 49 KSP Residual norm 3.690238177172e-06 50 KSP Residual norm 3.285270031352e-06 51 KSP Residual norm 2.885431612205e-06 52 KSP Residual norm 2.407232211460e-06 53 KSP Residual norm 2.005443492844e-06 54 KSP Residual norm 1.613518039061e-06 55 KSP Residual norm 1.342306029637e-06 56 KSP Residual norm 1.136987079652e-06 57 KSP Residual norm 1.036955019472e-06 58 KSP Residual norm 1.002879761731e-06 59 KSP Residual norm 9.393157341316e-07 60 KSP Residual norm 7.523704309410e-07 61 KSP Residual norm 5.652190214322e-07 62 KSP Residual norm 4.381366902976e-07 63 KSP Residual norm 3.515412993628e-07 64 KSP Residual norm 2.880607395009e-07 65 KSP Residual norm 2.501032500012e-07 66 KSP Residual norm 2.188061022144e-07 67 KSP Residual norm 1.904015122725e-07 68 KSP Residual norm 1.660059731128e-07 69 KSP Residual norm 1.439741622467e-07 70 KSP Residual norm 1.173835837313e-07 71 KSP Residual norm 1.029374441887e-07 72 KSP Residual norm 8.879549261250e-08 73 KSP Residual norm 7.932687645151e-08 74 KSP Residual norm 7.075714692934e-08 75 KSP Residual norm 5.873245093168e-08 76 KSP Residual norm 4.975099776932e-08 77 KSP Residual norm 3.789187803489e-08 78 KSP Residual norm 2.841682897460e-08 79 KSP Residual norm 2.356667833821e-08 80 KSP Residual norm 2.008965117265e-08 81 KSP Residual norm 1.667308433084e-08 82 KSP Residual norm 1.483050619945e-08 6 KSP preconditioned resid norm 5.020843588217e+05 true resid norm 4.938401705202e+15 ||r(i)||/||b|| 2.812303820536e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.769514873888e-03 1 KSP Residual norm 7.554255342708e-04 2 KSP Residual norm 1.202681910651e-03 3 KSP Residual norm 2.038479102336e-03 4 KSP Residual norm 1.320280561979e-03 5 KSP Residual norm 1.002093237465e-03 6 KSP Residual norm 8.517597405316e-04 7 KSP Residual norm 6.589603937718e-04 8 KSP Residual norm 5.229862226745e-04 9 KSP Residual norm 4.574646622057e-04 10 KSP Residual norm 3.920956735403e-04 11 KSP Residual norm 3.337687236588e-04 12 KSP Residual norm 2.528757959888e-04 13 KSP Residual norm 1.781583443076e-04 14 KSP Residual norm 1.617656168152e-04 15 KSP Residual norm 1.374378055356e-04 16 KSP Residual norm 1.172504417565e-04 17 KSP Residual norm 1.018598102463e-04 18 KSP Residual norm 8.915959904021e-05 19 KSP Residual norm 9.036094657626e-05 20 KSP Residual norm 8.386103130714e-05 21 KSP Residual norm 7.043820218181e-05 22 KSP Residual norm 6.110769472221e-05 23 KSP Residual norm 5.197530241963e-05 24 KSP Residual norm 4.239057154037e-05 25 KSP Residual norm 3.853336656959e-05 26 KSP Residual norm 3.783247867377e-05 27 KSP Residual norm 3.374702046416e-05 28 KSP Residual norm 2.981912122697e-05 29 KSP Residual norm 2.852801491964e-05 30 KSP Residual norm 2.528568531242e-05 31 KSP Residual norm 1.932794755519e-05 32 KSP Residual norm 1.572979284664e-05 33 KSP Residual norm 1.282607648586e-05 34 KSP Residual norm 1.050688214084e-05 35 KSP Residual norm 9.721266551595e-06 36 KSP Residual norm 8.806047021634e-06 37 KSP Residual norm 7.510563478671e-06 38 KSP Residual norm 6.315274300959e-06 39 KSP Residual norm 5.475208261207e-06 40 KSP Residual norm 4.880013023308e-06 41 KSP Residual norm 4.027825691436e-06 42 KSP Residual norm 3.616763004376e-06 43 KSP Residual norm 3.154009626456e-06 44 KSP Residual norm 2.735595632597e-06 45 KSP Residual norm 2.192881534181e-06 46 KSP Residual norm 1.780721015588e-06 47 KSP Residual norm 1.473095387198e-06 48 KSP Residual norm 1.259999717299e-06 49 KSP Residual norm 1.088330476906e-06 50 KSP Residual norm 8.847386412245e-07 51 KSP Residual norm 7.456259219502e-07 52 KSP Residual norm 6.188172638722e-07 53 KSP Residual norm 5.080014079749e-07 54 KSP Residual norm 4.211944223204e-07 55 KSP Residual norm 3.499397153718e-07 56 KSP Residual norm 2.978554863615e-07 57 KSP Residual norm 2.605161821846e-07 58 KSP Residual norm 2.312101778210e-07 59 KSP Residual norm 2.183027965109e-07 60 KSP Residual norm 1.986979980414e-07 61 KSP Residual norm 1.971501196954e-07 62 KSP Residual norm 1.997808288405e-07 63 KSP Residual norm 1.801005728651e-07 64 KSP Residual norm 1.724328152593e-07 65 KSP Residual norm 1.662819679382e-07 66 KSP Residual norm 1.600228513100e-07 67 KSP Residual norm 1.439887883000e-07 68 KSP Residual norm 1.204805806011e-07 69 KSP Residual norm 1.011600152477e-07 70 KSP Residual norm 8.182701259323e-08 71 KSP Residual norm 6.860467259226e-08 72 KSP Residual norm 5.608141910120e-08 73 KSP Residual norm 4.770127115125e-08 74 KSP Residual norm 4.543428546534e-08 75 KSP Residual norm 4.007728483313e-08 76 KSP Residual norm 3.463284158971e-08 77 KSP Residual norm 3.293514608251e-08 78 KSP Residual norm 2.987452930718e-08 79 KSP Residual norm 2.571116300800e-08 80 KSP Residual norm 2.394291171936e-08 81 KSP Residual norm 2.137522468758e-08 82 KSP Residual norm 1.805059519812e-08 83 KSP Residual norm 1.586340314220e-08 7 KSP preconditioned resid norm 4.691102991691e+05 true resid norm 5.504651150469e+15 ||r(i)||/||b|| 3.134769584433e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.527422325655e-03 1 KSP Residual norm 6.018453284980e-04 2 KSP Residual norm 7.874985643985e-04 3 KSP Residual norm 1.166196559295e-03 4 KSP Residual norm 5.330064308678e-04 5 KSP Residual norm 6.341587189666e-04 6 KSP Residual norm 6.625149610418e-04 7 KSP Residual norm 4.785656112730e-04 8 KSP Residual norm 4.703646902479e-04 9 KSP Residual norm 4.430265312971e-04 10 KSP Residual norm 4.007430377692e-04 11 KSP Residual norm 3.658240807850e-04 12 KSP Residual norm 2.858403156771e-04 13 KSP Residual norm 2.065252933060e-04 14 KSP Residual norm 2.001905572058e-04 15 KSP Residual norm 1.898029574879e-04 16 KSP Residual norm 1.674888020043e-04 17 KSP Residual norm 1.478289449473e-04 18 KSP Residual norm 1.422716276178e-04 19 KSP Residual norm 1.326107688631e-04 20 KSP Residual norm 1.057705886133e-04 21 KSP Residual norm 9.083121059249e-05 22 KSP Residual norm 8.735642387423e-05 23 KSP Residual norm 7.143879099212e-05 24 KSP Residual norm 6.184941755080e-05 25 KSP Residual norm 5.503446822490e-05 26 KSP Residual norm 4.509501821798e-05 27 KSP Residual norm 4.068271819980e-05 28 KSP Residual norm 3.740655856657e-05 29 KSP Residual norm 3.540939518352e-05 30 KSP Residual norm 3.441643749841e-05 31 KSP Residual norm 3.048728643928e-05 32 KSP Residual norm 2.560618875028e-05 33 KSP Residual norm 2.442189390263e-05 34 KSP Residual norm 2.288011439488e-05 35 KSP Residual norm 1.919028530538e-05 36 KSP Residual norm 1.506850956179e-05 37 KSP Residual norm 1.136038711293e-05 38 KSP Residual norm 9.282310045553e-06 39 KSP Residual norm 8.245337078777e-06 40 KSP Residual norm 7.967169020193e-06 41 KSP Residual norm 7.454169708382e-06 42 KSP Residual norm 6.536449227047e-06 43 KSP Residual norm 5.474238251264e-06 44 KSP Residual norm 4.421274577576e-06 45 KSP Residual norm 3.630078833617e-06 46 KSP Residual norm 3.129858660419e-06 47 KSP Residual norm 2.813374761293e-06 48 KSP Residual norm 2.587719864297e-06 49 KSP Residual norm 2.224641633428e-06 50 KSP Residual norm 1.872844390087e-06 51 KSP Residual norm 1.732747111978e-06 52 KSP Residual norm 1.489933469199e-06 53 KSP Residual norm 1.194830480675e-06 54 KSP Residual norm 1.085916253766e-06 55 KSP Residual norm 9.824508220387e-07 56 KSP Residual norm 1.010269768433e-06 57 KSP Residual norm 1.016322527198e-06 58 KSP Residual norm 9.104609424094e-07 59 KSP Residual norm 8.195207568388e-07 60 KSP Residual norm 6.884598594270e-07 61 KSP Residual norm 5.899005691761e-07 62 KSP Residual norm 4.812008714856e-07 63 KSP Residual norm 3.840555394231e-07 64 KSP Residual norm 3.319490925880e-07 65 KSP Residual norm 2.774601986995e-07 66 KSP Residual norm 2.346167139757e-07 67 KSP Residual norm 2.348872575486e-07 68 KSP Residual norm 2.119788835727e-07 69 KSP Residual norm 1.769423196698e-07 70 KSP Residual norm 1.460596452810e-07 71 KSP Residual norm 1.193532978581e-07 72 KSP Residual norm 1.034328374361e-07 73 KSP Residual norm 9.641065250990e-08 74 KSP Residual norm 8.330389561692e-08 75 KSP Residual norm 6.478808679032e-08 76 KSP Residual norm 4.839997978880e-08 77 KSP Residual norm 3.603684065269e-08 78 KSP Residual norm 2.864204842742e-08 79 KSP Residual norm 2.222153649771e-08 80 KSP Residual norm 1.812607266569e-08 81 KSP Residual norm 1.527061479378e-08 8 KSP preconditioned resid norm 4.500808358610e+05 true resid norm 5.950241483066e+15 ||r(i)||/||b|| 3.388522816666e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.929657397867e-04 1 KSP Residual norm 4.902876453791e-04 2 KSP Residual norm 7.559847546165e-04 3 KSP Residual norm 1.054393061562e-03 4 KSP Residual norm 6.055772034347e-04 5 KSP Residual norm 5.635524283518e-04 6 KSP Residual norm 4.568396742433e-04 7 KSP Residual norm 3.584099288095e-04 8 KSP Residual norm 3.178576484198e-04 9 KSP Residual norm 2.547543900920e-04 10 KSP Residual norm 1.784251514264e-04 11 KSP Residual norm 1.647555699220e-04 12 KSP Residual norm 1.414872559198e-04 13 KSP Residual norm 1.117786292845e-04 14 KSP Residual norm 1.090192322288e-04 15 KSP Residual norm 8.900454547982e-05 16 KSP Residual norm 8.346961909198e-05 17 KSP Residual norm 7.446711921831e-05 18 KSP Residual norm 5.799234857060e-05 19 KSP Residual norm 5.033824206817e-05 20 KSP Residual norm 4.665813278179e-05 21 KSP Residual norm 4.256100960811e-05 22 KSP Residual norm 3.908892921780e-05 23 KSP Residual norm 3.276750994252e-05 24 KSP Residual norm 2.954099933430e-05 25 KSP Residual norm 2.536999847468e-05 26 KSP Residual norm 2.186412716443e-05 27 KSP Residual norm 1.994227883577e-05 28 KSP Residual norm 1.815190381738e-05 29 KSP Residual norm 1.757818035909e-05 30 KSP Residual norm 1.659161868569e-05 31 KSP Residual norm 1.368718205339e-05 32 KSP Residual norm 1.173228497672e-05 33 KSP Residual norm 1.084139960618e-05 34 KSP Residual norm 1.046181691502e-05 35 KSP Residual norm 9.856219072178e-06 36 KSP Residual norm 8.949732576945e-06 37 KSP Residual norm 8.031018084880e-06 38 KSP Residual norm 6.827642170103e-06 39 KSP Residual norm 5.969773433539e-06 40 KSP Residual norm 5.381266124258e-06 41 KSP Residual norm 4.568703418801e-06 42 KSP Residual norm 4.255442440527e-06 43 KSP Residual norm 3.467073470314e-06 44 KSP Residual norm 2.779168301414e-06 45 KSP Residual norm 2.427111019689e-06 46 KSP Residual norm 2.187905281589e-06 47 KSP Residual norm 2.072694161060e-06 48 KSP Residual norm 1.934433997310e-06 49 KSP Residual norm 1.871616071656e-06 50 KSP Residual norm 1.770825491245e-06 51 KSP Residual norm 1.591519924697e-06 52 KSP Residual norm 1.316226263058e-06 53 KSP Residual norm 1.008636192991e-06 54 KSP Residual norm 8.324978134901e-07 55 KSP Residual norm 7.440883250546e-07 56 KSP Residual norm 6.241613621927e-07 57 KSP Residual norm 5.889489323059e-07 58 KSP Residual norm 5.485295952931e-07 59 KSP Residual norm 5.172101894151e-07 60 KSP Residual norm 4.755149723296e-07 61 KSP Residual norm 4.239045087594e-07 62 KSP Residual norm 3.583844835911e-07 63 KSP Residual norm 2.973104934794e-07 64 KSP Residual norm 2.737087525212e-07 65 KSP Residual norm 2.504228001347e-07 66 KSP Residual norm 2.175835643014e-07 67 KSP Residual norm 1.782239566180e-07 68 KSP Residual norm 1.398876654733e-07 69 KSP Residual norm 1.243251281054e-07 70 KSP Residual norm 1.096852436426e-07 71 KSP Residual norm 9.509159609015e-08 72 KSP Residual norm 8.493077703498e-08 73 KSP Residual norm 8.014336951229e-08 74 KSP Residual norm 7.381630016116e-08 75 KSP Residual norm 6.609518299482e-08 76 KSP Residual norm 5.653639079658e-08 77 KSP Residual norm 4.894221048879e-08 78 KSP Residual norm 4.305927517180e-08 79 KSP Residual norm 3.915247682279e-08 80 KSP Residual norm 3.543175807015e-08 81 KSP Residual norm 2.860488312897e-08 82 KSP Residual norm 2.407236546821e-08 83 KSP Residual norm 2.044171572991e-08 84 KSP Residual norm 1.635455342896e-08 85 KSP Residual norm 1.411433457232e-08 86 KSP Residual norm 1.215986886164e-08 87 KSP Residual norm 1.077340974712e-08 88 KSP Residual norm 9.395234868584e-09 9 KSP preconditioned resid norm 4.250017833169e+05 true resid norm 6.670775950304e+15 ||r(i)||/||b|| 3.798850278059e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.200254261475e-04 1 KSP Residual norm 5.026832893650e-04 2 KSP Residual norm 7.666367615071e-04 3 KSP Residual norm 8.705943922492e-04 4 KSP Residual norm 5.646459712524e-04 5 KSP Residual norm 6.588493249227e-04 6 KSP Residual norm 5.231050716195e-04 7 KSP Residual norm 3.685995698304e-04 8 KSP Residual norm 3.034729621818e-04 9 KSP Residual norm 2.521184397522e-04 10 KSP Residual norm 2.104732600449e-04 11 KSP Residual norm 1.703912268183e-04 12 KSP Residual norm 1.354329037777e-04 13 KSP Residual norm 1.161211341991e-04 14 KSP Residual norm 1.012831477647e-04 15 KSP Residual norm 8.660495544495e-05 16 KSP Residual norm 6.316880992678e-05 17 KSP Residual norm 4.690754740217e-05 18 KSP Residual norm 3.748021802120e-05 19 KSP Residual norm 3.012542988496e-05 20 KSP Residual norm 2.449929942689e-05 21 KSP Residual norm 2.164049659353e-05 22 KSP Residual norm 1.989638436244e-05 23 KSP Residual norm 1.774693897896e-05 24 KSP Residual norm 1.592703834611e-05 25 KSP Residual norm 1.401764611911e-05 26 KSP Residual norm 1.175314669513e-05 27 KSP Residual norm 9.736243974266e-06 28 KSP Residual norm 8.675432629691e-06 29 KSP Residual norm 8.355692597077e-06 30 KSP Residual norm 7.838784741412e-06 31 KSP Residual norm 7.469193190213e-06 32 KSP Residual norm 6.401811844866e-06 33 KSP Residual norm 5.699841128770e-06 34 KSP Residual norm 5.098776305195e-06 35 KSP Residual norm 5.060202209001e-06 36 KSP Residual norm 4.828789978454e-06 37 KSP Residual norm 4.327525907492e-06 38 KSP Residual norm 3.917921800783e-06 39 KSP Residual norm 3.620194481807e-06 40 KSP Residual norm 3.120745519737e-06 41 KSP Residual norm 2.682541335973e-06 42 KSP Residual norm 2.384578475669e-06 43 KSP Residual norm 2.383680000394e-06 44 KSP Residual norm 2.152159496154e-06 45 KSP Residual norm 1.728707913893e-06 46 KSP Residual norm 1.496837742317e-06 47 KSP Residual norm 1.201019823118e-06 48 KSP Residual norm 9.486848633177e-07 49 KSP Residual norm 7.494833422803e-07 50 KSP Residual norm 6.817552715651e-07 51 KSP Residual norm 6.344596909119e-07 52 KSP Residual norm 5.797101100600e-07 53 KSP Residual norm 4.655820368251e-07 54 KSP Residual norm 4.165240240650e-07 55 KSP Residual norm 3.460916052170e-07 56 KSP Residual norm 2.600177993180e-07 57 KSP Residual norm 2.246119587309e-07 58 KSP Residual norm 2.011821254588e-07 59 KSP Residual norm 1.819845981449e-07 60 KSP Residual norm 1.666569264012e-07 61 KSP Residual norm 1.436604560548e-07 62 KSP Residual norm 1.276587088656e-07 63 KSP Residual norm 1.144636358343e-07 64 KSP Residual norm 1.032454211393e-07 65 KSP Residual norm 1.047993675460e-07 66 KSP Residual norm 1.000193922051e-07 67 KSP Residual norm 9.180687880429e-08 68 KSP Residual norm 8.214049905738e-08 69 KSP Residual norm 6.745832360433e-08 70 KSP Residual norm 5.581906787117e-08 71 KSP Residual norm 4.630074362701e-08 72 KSP Residual norm 3.511733315896e-08 73 KSP Residual norm 2.847611309449e-08 74 KSP Residual norm 2.379609914911e-08 75 KSP Residual norm 2.204080814832e-08 76 KSP Residual norm 1.868848687109e-08 77 KSP Residual norm 1.558881848560e-08 78 KSP Residual norm 1.419681957837e-08 79 KSP Residual norm 1.253008934831e-08 80 KSP Residual norm 1.181813806603e-08 81 KSP Residual norm 1.138361116878e-08 82 KSP Residual norm 1.068134133328e-08 83 KSP Residual norm 9.791384955514e-09 84 KSP Residual norm 8.445017061187e-09 10 KSP preconditioned resid norm 4.051635153763e+05 true resid norm 7.414259071687e+15 ||r(i)||/||b|| 4.222246459170e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.411279505320e-03 1 KSP Residual norm 7.209624613977e-04 2 KSP Residual norm 1.013550000279e-03 3 KSP Residual norm 1.329722578055e-03 4 KSP Residual norm 9.299374430975e-04 5 KSP Residual norm 9.942662240769e-04 6 KSP Residual norm 6.792656465301e-04 7 KSP Residual norm 4.683452755646e-04 8 KSP Residual norm 3.992715074611e-04 9 KSP Residual norm 3.511431604995e-04 10 KSP Residual norm 3.111979031550e-04 11 KSP Residual norm 2.766221017318e-04 12 KSP Residual norm 2.109137983476e-04 13 KSP Residual norm 1.662041206546e-04 14 KSP Residual norm 1.398102151659e-04 15 KSP Residual norm 1.400556053078e-04 16 KSP Residual norm 1.379179889368e-04 17 KSP Residual norm 1.268873434229e-04 18 KSP Residual norm 1.047101319592e-04 19 KSP Residual norm 9.331480445280e-05 20 KSP Residual norm 8.923232325618e-05 21 KSP Residual norm 7.570662411081e-05 22 KSP Residual norm 6.676541969887e-05 23 KSP Residual norm 6.221192581325e-05 24 KSP Residual norm 5.958345257685e-05 25 KSP Residual norm 5.665881061978e-05 26 KSP Residual norm 5.120506512044e-05 27 KSP Residual norm 4.246868334557e-05 28 KSP Residual norm 3.831321898572e-05 29 KSP Residual norm 3.587959246487e-05 30 KSP Residual norm 3.392221891532e-05 31 KSP Residual norm 2.901134960129e-05 32 KSP Residual norm 2.336738717744e-05 33 KSP Residual norm 2.041689746657e-05 34 KSP Residual norm 1.815689228967e-05 35 KSP Residual norm 1.564169691250e-05 36 KSP Residual norm 1.367788319918e-05 37 KSP Residual norm 1.204491060549e-05 38 KSP Residual norm 1.005190704326e-05 39 KSP Residual norm 8.444382862709e-06 40 KSP Residual norm 7.461525396347e-06 41 KSP Residual norm 6.648844443497e-06 42 KSP Residual norm 5.512834496401e-06 43 KSP Residual norm 4.646669659544e-06 44 KSP Residual norm 4.019218409906e-06 45 KSP Residual norm 3.313439333579e-06 46 KSP Residual norm 2.946430101063e-06 47 KSP Residual norm 3.013172151866e-06 48 KSP Residual norm 3.121787615489e-06 49 KSP Residual norm 2.899554392415e-06 50 KSP Residual norm 2.631906303388e-06 51 KSP Residual norm 2.222379968828e-06 52 KSP Residual norm 1.891132078561e-06 53 KSP Residual norm 1.487269144552e-06 54 KSP Residual norm 1.330743450427e-06 55 KSP Residual norm 1.153298630154e-06 56 KSP Residual norm 9.837001884336e-07 57 KSP Residual norm 9.462488905838e-07 58 KSP Residual norm 8.159219117588e-07 59 KSP Residual norm 7.053692968344e-07 60 KSP Residual norm 5.852092375603e-07 61 KSP Residual norm 5.152501907863e-07 62 KSP Residual norm 4.916268459686e-07 63 KSP Residual norm 4.395264829083e-07 64 KSP Residual norm 3.748901354410e-07 65 KSP Residual norm 3.235816789575e-07 66 KSP Residual norm 2.721611241940e-07 67 KSP Residual norm 2.362337360615e-07 68 KSP Residual norm 1.990056944395e-07 69 KSP Residual norm 1.783713283092e-07 70 KSP Residual norm 1.488298351018e-07 71 KSP Residual norm 1.199228386691e-07 72 KSP Residual norm 1.101628376416e-07 73 KSP Residual norm 1.064611915990e-07 74 KSP Residual norm 9.245248268909e-08 75 KSP Residual norm 8.045119268170e-08 76 KSP Residual norm 7.164372389583e-08 77 KSP Residual norm 5.897550067188e-08 78 KSP Residual norm 5.081315281605e-08 79 KSP Residual norm 4.511493511029e-08 80 KSP Residual norm 4.134344906919e-08 81 KSP Residual norm 3.639898286485e-08 82 KSP Residual norm 3.159308391246e-08 83 KSP Residual norm 2.524885503680e-08 84 KSP Residual norm 1.945248424195e-08 85 KSP Residual norm 1.761737475196e-08 86 KSP Residual norm 1.541422422114e-08 87 KSP Residual norm 1.212892098515e-08 11 KSP preconditioned resid norm 3.880015722902e+05 true resid norm 8.151564986314e+15 ||r(i)||/||b|| 4.642124866068e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.122716868788e-03 1 KSP Residual norm 6.008808533038e-04 2 KSP Residual norm 8.746901486366e-04 3 KSP Residual norm 9.675719644774e-04 4 KSP Residual norm 6.304857576912e-04 5 KSP Residual norm 6.570214073237e-04 6 KSP Residual norm 5.052071154901e-04 7 KSP Residual norm 4.304615634874e-04 8 KSP Residual norm 3.834416080452e-04 9 KSP Residual norm 3.159183513395e-04 10 KSP Residual norm 2.937905902811e-04 11 KSP Residual norm 2.405098072791e-04 12 KSP Residual norm 1.793551829062e-04 13 KSP Residual norm 1.538545824359e-04 14 KSP Residual norm 1.343317570233e-04 15 KSP Residual norm 1.174204668875e-04 16 KSP Residual norm 1.147415887521e-04 17 KSP Residual norm 1.090709538620e-04 18 KSP Residual norm 9.906670909342e-05 19 KSP Residual norm 8.811596697872e-05 20 KSP Residual norm 7.939148563411e-05 21 KSP Residual norm 7.155753313988e-05 22 KSP Residual norm 7.256620200825e-05 23 KSP Residual norm 6.755550431329e-05 24 KSP Residual norm 6.686160083169e-05 25 KSP Residual norm 5.955937553796e-05 26 KSP Residual norm 4.746574986832e-05 27 KSP Residual norm 4.126635464198e-05 28 KSP Residual norm 3.404096975607e-05 29 KSP Residual norm 3.076848529963e-05 30 KSP Residual norm 2.784101022334e-05 31 KSP Residual norm 2.391938044908e-05 32 KSP Residual norm 1.950884595540e-05 33 KSP Residual norm 1.705158155397e-05 34 KSP Residual norm 1.520903833371e-05 35 KSP Residual norm 1.249213038474e-05 36 KSP Residual norm 1.058160083137e-05 37 KSP Residual norm 9.284507036373e-06 38 KSP Residual norm 8.757954083754e-06 39 KSP Residual norm 8.195893501203e-06 40 KSP Residual norm 7.019599668936e-06 41 KSP Residual norm 5.620825670329e-06 42 KSP Residual norm 4.932913838607e-06 43 KSP Residual norm 4.363079222424e-06 44 KSP Residual norm 3.202394102474e-06 45 KSP Residual norm 2.540107419165e-06 46 KSP Residual norm 2.079814007785e-06 47 KSP Residual norm 1.900196960299e-06 48 KSP Residual norm 1.845644821932e-06 49 KSP Residual norm 1.700279138062e-06 50 KSP Residual norm 1.518564339413e-06 51 KSP Residual norm 1.292591923086e-06 52 KSP Residual norm 1.054511094545e-06 53 KSP Residual norm 8.437084330216e-07 54 KSP Residual norm 7.666930913055e-07 55 KSP Residual norm 7.396953754101e-07 56 KSP Residual norm 6.403041855562e-07 57 KSP Residual norm 6.338848815188e-07 58 KSP Residual norm 6.445911176637e-07 59 KSP Residual norm 5.587648495519e-07 60 KSP Residual norm 4.493675868183e-07 61 KSP Residual norm 3.751369595937e-07 62 KSP Residual norm 3.452352201801e-07 63 KSP Residual norm 3.306976430352e-07 64 KSP Residual norm 3.161092208678e-07 65 KSP Residual norm 2.735594513429e-07 66 KSP Residual norm 2.150194814698e-07 67 KSP Residual norm 1.794154549059e-07 68 KSP Residual norm 1.485349674374e-07 69 KSP Residual norm 1.171367396097e-07 70 KSP Residual norm 8.618131070106e-08 71 KSP Residual norm 6.828400533145e-08 72 KSP Residual norm 5.751941286285e-08 73 KSP Residual norm 4.660222107664e-08 74 KSP Residual norm 4.004667546977e-08 75 KSP Residual norm 3.579380735855e-08 76 KSP Residual norm 3.077152509197e-08 77 KSP Residual norm 2.696045956599e-08 78 KSP Residual norm 2.495900645150e-08 79 KSP Residual norm 2.403357077445e-08 80 KSP Residual norm 2.394915077750e-08 81 KSP Residual norm 2.217405978792e-08 82 KSP Residual norm 2.053050255384e-08 83 KSP Residual norm 1.710274427764e-08 84 KSP Residual norm 1.311093930139e-08 85 KSP Residual norm 1.030213845590e-08 12 KSP preconditioned resid norm 3.741403203347e+05 true resid norm 8.948453074716e+15 ||r(i)||/||b|| 5.095933921981e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.100806057211e-03 1 KSP Residual norm 6.577291415384e-04 2 KSP Residual norm 1.048309555084e-03 3 KSP Residual norm 1.117110319452e-03 4 KSP Residual norm 6.625286495026e-04 5 KSP Residual norm 6.160736713854e-04 6 KSP Residual norm 4.468025399915e-04 7 KSP Residual norm 3.425480962131e-04 8 KSP Residual norm 2.813853772798e-04 9 KSP Residual norm 2.223382544106e-04 10 KSP Residual norm 1.797286574771e-04 11 KSP Residual norm 1.546165261470e-04 12 KSP Residual norm 1.424932511566e-04 13 KSP Residual norm 1.289292806564e-04 14 KSP Residual norm 1.082005147602e-04 15 KSP Residual norm 9.287904294910e-05 16 KSP Residual norm 8.201311042269e-05 17 KSP Residual norm 6.809773868750e-05 18 KSP Residual norm 6.025972381858e-05 19 KSP Residual norm 5.674859783110e-05 20 KSP Residual norm 5.663754637967e-05 21 KSP Residual norm 5.402993686216e-05 22 KSP Residual norm 4.870753390569e-05 23 KSP Residual norm 4.663393563839e-05 24 KSP Residual norm 4.161348668758e-05 25 KSP Residual norm 3.317451239796e-05 26 KSP Residual norm 2.731027132468e-05 27 KSP Residual norm 2.462064152491e-05 28 KSP Residual norm 2.430835583592e-05 29 KSP Residual norm 2.417411815501e-05 30 KSP Residual norm 2.061414764703e-05 31 KSP Residual norm 1.780807411335e-05 32 KSP Residual norm 1.505486222387e-05 33 KSP Residual norm 1.330518334944e-05 34 KSP Residual norm 1.193912739898e-05 35 KSP Residual norm 1.120691851760e-05 36 KSP Residual norm 1.022333587184e-05 37 KSP Residual norm 9.384895313995e-06 38 KSP Residual norm 7.740221628598e-06 39 KSP Residual norm 6.287892496063e-06 40 KSP Residual norm 6.011999913701e-06 41 KSP Residual norm 5.437488480571e-06 42 KSP Residual norm 5.083442598145e-06 43 KSP Residual norm 4.677988691041e-06 44 KSP Residual norm 3.865991896605e-06 45 KSP Residual norm 3.331218522568e-06 46 KSP Residual norm 2.839017185796e-06 47 KSP Residual norm 2.571079213875e-06 48 KSP Residual norm 2.364850078864e-06 49 KSP Residual norm 1.993638556637e-06 50 KSP Residual norm 1.571108954534e-06 51 KSP Residual norm 1.312909890124e-06 52 KSP Residual norm 1.186715359094e-06 53 KSP Residual norm 9.981062704104e-07 54 KSP Residual norm 8.560047123095e-07 55 KSP Residual norm 7.384475525843e-07 56 KSP Residual norm 5.877495150335e-07 57 KSP Residual norm 4.817942529451e-07 58 KSP Residual norm 3.872015792209e-07 59 KSP Residual norm 3.069987194596e-07 60 KSP Residual norm 2.422136466907e-07 61 KSP Residual norm 1.995139647837e-07 62 KSP Residual norm 1.609971638595e-07 63 KSP Residual norm 1.304093109524e-07 64 KSP Residual norm 1.079522417317e-07 65 KSP Residual norm 9.449448515878e-08 66 KSP Residual norm 8.395371583770e-08 67 KSP Residual norm 7.513531914746e-08 68 KSP Residual norm 6.347499399425e-08 69 KSP Residual norm 5.142261546478e-08 70 KSP Residual norm 4.099511898433e-08 71 KSP Residual norm 3.570300378045e-08 72 KSP Residual norm 3.299806467809e-08 73 KSP Residual norm 3.061456314443e-08 74 KSP Residual norm 2.623432508532e-08 75 KSP Residual norm 2.422341421294e-08 76 KSP Residual norm 2.344073736418e-08 77 KSP Residual norm 2.119450666598e-08 78 KSP Residual norm 1.846946786987e-08 79 KSP Residual norm 1.612504341588e-08 80 KSP Residual norm 1.525327989441e-08 81 KSP Residual norm 1.324228014875e-08 82 KSP Residual norm 1.142821576457e-08 83 KSP Residual norm 9.525344010032e-09 13 KSP preconditioned resid norm 3.532140696819e+05 true resid norm 1.028226620724e+16 ||r(i)||/||b|| 5.855509183857e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.227266045511e-03 1 KSP Residual norm 5.447297984266e-04 2 KSP Residual norm 8.246023615042e-04 3 KSP Residual norm 1.016008283971e-03 4 KSP Residual norm 4.657979761309e-04 5 KSP Residual norm 5.061163020278e-04 6 KSP Residual norm 4.426729389847e-04 7 KSP Residual norm 3.238243221960e-04 8 KSP Residual norm 2.646832838726e-04 9 KSP Residual norm 2.059711839729e-04 10 KSP Residual norm 1.480021196030e-04 11 KSP Residual norm 1.167905945715e-04 12 KSP Residual norm 9.427696513830e-05 13 KSP Residual norm 8.203541359160e-05 14 KSP Residual norm 7.466639075924e-05 15 KSP Residual norm 6.970570901991e-05 16 KSP Residual norm 7.211965322671e-05 17 KSP Residual norm 6.298075380451e-05 18 KSP Residual norm 5.074960900055e-05 19 KSP Residual norm 4.514720236487e-05 20 KSP Residual norm 3.853683174505e-05 21 KSP Residual norm 3.738423001375e-05 22 KSP Residual norm 3.709620904948e-05 23 KSP Residual norm 3.142770872224e-05 24 KSP Residual norm 2.848631585846e-05 25 KSP Residual norm 2.562061058508e-05 26 KSP Residual norm 2.169952343122e-05 27 KSP Residual norm 1.843706154138e-05 28 KSP Residual norm 1.619100785386e-05 29 KSP Residual norm 1.420378837809e-05 30 KSP Residual norm 1.232002380120e-05 31 KSP Residual norm 1.047042057508e-05 32 KSP Residual norm 8.127398704757e-06 33 KSP Residual norm 6.752017594785e-06 34 KSP Residual norm 5.981342215874e-06 35 KSP Residual norm 5.549067177236e-06 36 KSP Residual norm 5.328108423986e-06 37 KSP Residual norm 4.738309219215e-06 38 KSP Residual norm 4.212749550733e-06 39 KSP Residual norm 3.393898149761e-06 40 KSP Residual norm 2.877927596096e-06 41 KSP Residual norm 2.723313626964e-06 42 KSP Residual norm 2.555495230140e-06 43 KSP Residual norm 2.372522752692e-06 44 KSP Residual norm 2.212579417735e-06 45 KSP Residual norm 1.756195477009e-06 46 KSP Residual norm 1.512099855840e-06 47 KSP Residual norm 1.328880831649e-06 48 KSP Residual norm 1.306042041683e-06 49 KSP Residual norm 1.263516418337e-06 50 KSP Residual norm 1.200549322100e-06 51 KSP Residual norm 1.132632757850e-06 52 KSP Residual norm 1.039805417880e-06 53 KSP Residual norm 8.439350435988e-07 54 KSP Residual norm 6.860715496219e-07 55 KSP Residual norm 5.508211742582e-07 56 KSP Residual norm 4.283152270733e-07 57 KSP Residual norm 3.833253253564e-07 58 KSP Residual norm 3.660761718311e-07 59 KSP Residual norm 3.788499665563e-07 60 KSP Residual norm 3.923257840629e-07 61 KSP Residual norm 3.332719937087e-07 62 KSP Residual norm 2.762242120717e-07 63 KSP Residual norm 2.491688031842e-07 64 KSP Residual norm 2.075512941491e-07 65 KSP Residual norm 1.830265280253e-07 66 KSP Residual norm 1.696931286980e-07 67 KSP Residual norm 1.496252483305e-07 68 KSP Residual norm 1.359020030144e-07 69 KSP Residual norm 1.177107263386e-07 70 KSP Residual norm 9.572491139085e-08 71 KSP Residual norm 8.271075371502e-08 72 KSP Residual norm 7.843118965192e-08 73 KSP Residual norm 7.624679608790e-08 74 KSP Residual norm 6.323188798955e-08 75 KSP Residual norm 5.450929130695e-08 76 KSP Residual norm 4.728026130130e-08 77 KSP Residual norm 3.690713027026e-08 78 KSP Residual norm 2.859737703704e-08 79 KSP Residual norm 2.188241157878e-08 80 KSP Residual norm 1.806077048037e-08 81 KSP Residual norm 1.518605739665e-08 82 KSP Residual norm 1.229808906365e-08 83 KSP Residual norm 1.061568380973e-08 14 KSP preconditioned resid norm 3.306243387330e+05 true resid norm 1.207917655474e+16 ||r(i)||/||b|| 6.878807436431e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.459432290838e-03 1 KSP Residual norm 5.177694546374e-04 2 KSP Residual norm 7.633997715428e-04 3 KSP Residual norm 9.877976742848e-04 4 KSP Residual norm 4.200357547174e-04 5 KSP Residual norm 4.211594841571e-04 6 KSP Residual norm 3.232605114025e-04 7 KSP Residual norm 2.605123987139e-04 8 KSP Residual norm 2.332504809664e-04 9 KSP Residual norm 2.124526459244e-04 10 KSP Residual norm 1.839763132207e-04 11 KSP Residual norm 1.600669335903e-04 12 KSP Residual norm 1.351695692210e-04 13 KSP Residual norm 1.119178924099e-04 14 KSP Residual norm 1.069523586410e-04 15 KSP Residual norm 9.804367761906e-05 16 KSP Residual norm 1.054276789421e-04 17 KSP Residual norm 9.539107180572e-05 18 KSP Residual norm 7.764874131566e-05 19 KSP Residual norm 7.302935596796e-05 20 KSP Residual norm 6.474901449940e-05 21 KSP Residual norm 5.984922018873e-05 22 KSP Residual norm 5.405159983081e-05 23 KSP Residual norm 4.695603825105e-05 24 KSP Residual norm 4.086670619422e-05 25 KSP Residual norm 3.469454967883e-05 26 KSP Residual norm 2.756686133114e-05 27 KSP Residual norm 2.326495609470e-05 28 KSP Residual norm 2.126439346970e-05 29 KSP Residual norm 1.948357948039e-05 30 KSP Residual norm 1.755298230261e-05 31 KSP Residual norm 1.607621313879e-05 32 KSP Residual norm 1.531089867882e-05 33 KSP Residual norm 1.454208559897e-05 34 KSP Residual norm 1.389506093092e-05 35 KSP Residual norm 1.258612479997e-05 36 KSP Residual norm 1.138599379362e-05 37 KSP Residual norm 9.413442310771e-06 38 KSP Residual norm 7.507957864729e-06 39 KSP Residual norm 6.045511467564e-06 40 KSP Residual norm 5.489633705423e-06 41 KSP Residual norm 4.922716726317e-06 42 KSP Residual norm 4.149569625497e-06 43 KSP Residual norm 3.548235691340e-06 44 KSP Residual norm 3.034898161568e-06 45 KSP Residual norm 2.411746177837e-06 46 KSP Residual norm 2.030258462211e-06 47 KSP Residual norm 1.710648711823e-06 48 KSP Residual norm 1.627083264795e-06 49 KSP Residual norm 1.681648902998e-06 50 KSP Residual norm 1.513821678675e-06 51 KSP Residual norm 1.331216028835e-06 52 KSP Residual norm 1.144376302051e-06 53 KSP Residual norm 9.106034615301e-07 54 KSP Residual norm 8.177818316452e-07 55 KSP Residual norm 7.307874915659e-07 56 KSP Residual norm 6.516247159255e-07 57 KSP Residual norm 6.250263205941e-07 58 KSP Residual norm 5.706483785491e-07 59 KSP Residual norm 4.958566052577e-07 60 KSP Residual norm 3.936161272306e-07 61 KSP Residual norm 3.164833774631e-07 62 KSP Residual norm 2.878817619036e-07 63 KSP Residual norm 2.667989753790e-07 64 KSP Residual norm 2.370790828550e-07 65 KSP Residual norm 2.222940416236e-07 66 KSP Residual norm 2.058942142729e-07 67 KSP Residual norm 1.771742345211e-07 68 KSP Residual norm 1.539046780393e-07 69 KSP Residual norm 1.316500082810e-07 70 KSP Residual norm 1.131952124929e-07 71 KSP Residual norm 1.023916402035e-07 72 KSP Residual norm 8.685913831782e-08 73 KSP Residual norm 7.412771219194e-08 74 KSP Residual norm 6.097126422574e-08 75 KSP Residual norm 4.863425079193e-08 76 KSP Residual norm 3.950374358809e-08 77 KSP Residual norm 3.265334139673e-08 78 KSP Residual norm 2.800050071506e-08 79 KSP Residual norm 2.687040208216e-08 80 KSP Residual norm 2.504901206204e-08 81 KSP Residual norm 2.217777011225e-08 82 KSP Residual norm 1.899252358757e-08 83 KSP Residual norm 1.532282428560e-08 84 KSP Residual norm 1.151789037598e-08 15 KSP preconditioned resid norm 3.076994286453e+05 true resid norm 1.412211206135e+16 ||r(i)||/||b|| 8.042211240599e+13 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.764234654438e-04 1 KSP Residual norm 5.489953732263e-04 2 KSP Residual norm 8.749783110445e-04 3 KSP Residual norm 8.888547231329e-04 4 KSP Residual norm 4.529115004058e-04 5 KSP Residual norm 4.860423654622e-04 6 KSP Residual norm 3.287254848119e-04 7 KSP Residual norm 2.232670268195e-04 8 KSP Residual norm 1.876484330318e-04 9 KSP Residual norm 1.525064956599e-04 10 KSP Residual norm 1.208670435123e-04 11 KSP Residual norm 1.028323886916e-04 12 KSP Residual norm 7.772774008753e-05 13 KSP Residual norm 6.752649158855e-05 14 KSP Residual norm 6.504161985039e-05 15 KSP Residual norm 5.904951311588e-05 16 KSP Residual norm 5.543844939544e-05 17 KSP Residual norm 4.615173256967e-05 18 KSP Residual norm 3.799649666412e-05 19 KSP Residual norm 3.730733034732e-05 20 KSP Residual norm 3.606579011292e-05 21 KSP Residual norm 3.299019027953e-05 22 KSP Residual norm 3.133517799975e-05 23 KSP Residual norm 2.938428020786e-05 24 KSP Residual norm 2.721543127547e-05 25 KSP Residual norm 2.551548018198e-05 26 KSP Residual norm 2.213065569460e-05 27 KSP Residual norm 1.939231079258e-05 28 KSP Residual norm 1.775454795321e-05 29 KSP Residual norm 1.568041345539e-05 30 KSP Residual norm 1.450072435892e-05 31 KSP Residual norm 1.228486363809e-05 32 KSP Residual norm 9.164130602858e-06 33 KSP Residual norm 7.457318710014e-06 34 KSP Residual norm 6.244901632835e-06 35 KSP Residual norm 5.747392763597e-06 36 KSP Residual norm 5.657878370579e-06 37 KSP Residual norm 4.983694063528e-06 38 KSP Residual norm 4.473040049383e-06 39 KSP Residual norm 3.932591714728e-06 40 KSP Residual norm 3.444085299692e-06 41 KSP Residual norm 2.836538436918e-06 42 KSP Residual norm 2.263428801519e-06 43 KSP Residual norm 1.893766046004e-06 44 KSP Residual norm 1.616942774810e-06 45 KSP Residual norm 1.431508738610e-06 46 KSP Residual norm 1.264420486557e-06 47 KSP Residual norm 1.124190695961e-06 48 KSP Residual norm 1.061696091503e-06 49 KSP Residual norm 1.017025023220e-06 50 KSP Residual norm 9.320447033483e-07 51 KSP Residual norm 9.209793239494e-07 52 KSP Residual norm 8.175080114526e-07 53 KSP Residual norm 6.490646229583e-07 54 KSP Residual norm 5.019016355281e-07 55 KSP Residual norm 4.198722502674e-07 56 KSP Residual norm 3.756672730086e-07 57 KSP Residual norm 3.605745424031e-07 58 KSP Residual norm 3.048796236760e-07 59 KSP Residual norm 2.579874371926e-07 60 KSP Residual norm 2.133016847868e-07 61 KSP Residual norm 1.871571346571e-07 62 KSP Residual norm 1.684993102107e-07 63 KSP Residual norm 1.399548580012e-07 64 KSP Residual norm 1.163755908148e-07 65 KSP Residual norm 1.010738340893e-07 66 KSP Residual norm 8.957923623371e-08 67 KSP Residual norm 8.812367491528e-08 68 KSP Residual norm 8.495145999334e-08 69 KSP Residual norm 7.402801205774e-08 70 KSP Residual norm 5.738516273128e-08 71 KSP Residual norm 5.165840544128e-08 72 KSP Residual norm 4.781686500394e-08 73 KSP Residual norm 4.216651531229e-08 74 KSP Residual norm 4.143466233621e-08 75 KSP Residual norm 3.678394992614e-08 76 KSP Residual norm 3.155260951661e-08 77 KSP Residual norm 2.665212150931e-08 78 KSP Residual norm 2.277533135085e-08 79 KSP Residual norm 1.942564024377e-08 80 KSP Residual norm 1.642312107108e-08 81 KSP Residual norm 1.359787956392e-08 82 KSP Residual norm 1.084745939350e-08 83 KSP Residual norm 8.897116416397e-09 16 KSP preconditioned resid norm 2.709447111564e+05 true resid norm 1.756802993155e+16 ||r(i)||/||b|| 1.000458055968e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.006665543077e-03 1 KSP Residual norm 5.895749726952e-04 2 KSP Residual norm 8.025235967352e-04 3 KSP Residual norm 7.152189058079e-04 4 KSP Residual norm 4.846790001718e-04 5 KSP Residual norm 4.677406752836e-04 6 KSP Residual norm 3.724837178428e-04 7 KSP Residual norm 2.742675706877e-04 8 KSP Residual norm 2.388662852402e-04 9 KSP Residual norm 1.945942299314e-04 10 KSP Residual norm 1.594878261667e-04 11 KSP Residual norm 1.327032521003e-04 12 KSP Residual norm 1.061558146642e-04 13 KSP Residual norm 9.159945027617e-05 14 KSP Residual norm 8.979385708770e-05 15 KSP Residual norm 8.846500098469e-05 16 KSP Residual norm 7.478542512343e-05 17 KSP Residual norm 5.796343876666e-05 18 KSP Residual norm 5.057527325271e-05 19 KSP Residual norm 4.559837389235e-05 20 KSP Residual norm 3.995001619217e-05 21 KSP Residual norm 3.709269854324e-05 22 KSP Residual norm 3.524419479059e-05 23 KSP Residual norm 3.197962503758e-05 24 KSP Residual norm 2.850779201751e-05 25 KSP Residual norm 2.619239942769e-05 26 KSP Residual norm 2.158099863421e-05 27 KSP Residual norm 1.838456422539e-05 28 KSP Residual norm 1.679939483916e-05 29 KSP Residual norm 1.559798356750e-05 30 KSP Residual norm 1.389078783891e-05 31 KSP Residual norm 1.230756735543e-05 32 KSP Residual norm 1.036043808588e-05 33 KSP Residual norm 8.406954609656e-06 34 KSP Residual norm 7.702409645961e-06 35 KSP Residual norm 7.763526406491e-06 36 KSP Residual norm 6.921207319542e-06 37 KSP Residual norm 5.613224065244e-06 38 KSP Residual norm 4.543750542402e-06 39 KSP Residual norm 4.244335245093e-06 40 KSP Residual norm 3.863807735471e-06 41 KSP Residual norm 3.265515940597e-06 42 KSP Residual norm 3.042299669021e-06 43 KSP Residual norm 2.938283558895e-06 44 KSP Residual norm 2.634864096186e-06 45 KSP Residual norm 2.122139872450e-06 46 KSP Residual norm 1.834700632190e-06 47 KSP Residual norm 1.678691839842e-06 48 KSP Residual norm 1.631914047786e-06 49 KSP Residual norm 1.527765669795e-06 50 KSP Residual norm 1.398370393210e-06 51 KSP Residual norm 1.317678290275e-06 52 KSP Residual norm 1.146021421948e-06 53 KSP Residual norm 9.024594194983e-07 54 KSP Residual norm 7.664948872376e-07 55 KSP Residual norm 6.402294319727e-07 56 KSP Residual norm 5.892367718234e-07 57 KSP Residual norm 5.376061205510e-07 58 KSP Residual norm 4.538359670196e-07 59 KSP Residual norm 3.878851824806e-07 60 KSP Residual norm 3.363043882561e-07 61 KSP Residual norm 2.836355279035e-07 62 KSP Residual norm 2.652346207214e-07 63 KSP Residual norm 2.518746871928e-07 64 KSP Residual norm 2.185671835739e-07 65 KSP Residual norm 2.325660767046e-07 66 KSP Residual norm 2.107215034343e-07 67 KSP Residual norm 1.962801545641e-07 68 KSP Residual norm 1.843457663841e-07 69 KSP Residual norm 1.520648017635e-07 70 KSP Residual norm 1.187434097302e-07 71 KSP Residual norm 9.419538658356e-08 72 KSP Residual norm 7.587845565008e-08 73 KSP Residual norm 6.895613663244e-08 74 KSP Residual norm 5.785708345932e-08 75 KSP Residual norm 4.601479926402e-08 76 KSP Residual norm 4.006359993181e-08 77 KSP Residual norm 3.432622256133e-08 78 KSP Residual norm 3.009006020118e-08 79 KSP Residual norm 2.711199022491e-08 80 KSP Residual norm 2.329136306763e-08 81 KSP Residual norm 1.978765156074e-08 82 KSP Residual norm 1.687387852349e-08 83 KSP Residual norm 1.252086360136e-08 84 KSP Residual norm 1.009300072156e-08 85 KSP Residual norm 8.301525016312e-09 17 KSP preconditioned resid norm 2.491269146610e+05 true resid norm 1.976009758694e+16 ||r(i)||/||b|| 1.125291162105e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.047460439316e-03 1 KSP Residual norm 4.919547070189e-04 2 KSP Residual norm 7.436613474197e-04 3 KSP Residual norm 1.016601540646e-03 4 KSP Residual norm 4.834002402018e-04 5 KSP Residual norm 3.841456559995e-04 6 KSP Residual norm 3.091528583130e-04 7 KSP Residual norm 2.244716515111e-04 8 KSP Residual norm 2.036058849090e-04 9 KSP Residual norm 1.659843045113e-04 10 KSP Residual norm 1.430423906551e-04 11 KSP Residual norm 1.143870290674e-04 12 KSP Residual norm 8.247368346501e-05 13 KSP Residual norm 6.837045207442e-05 14 KSP Residual norm 6.741780718848e-05 15 KSP Residual norm 6.420177699846e-05 16 KSP Residual norm 5.832569400485e-05 17 KSP Residual norm 5.257713558213e-05 18 KSP Residual norm 4.463237663341e-05 19 KSP Residual norm 4.289717643227e-05 20 KSP Residual norm 3.925425113427e-05 21 KSP Residual norm 3.642518503266e-05 22 KSP Residual norm 3.382442626620e-05 23 KSP Residual norm 3.011084282428e-05 24 KSP Residual norm 2.619779074135e-05 25 KSP Residual norm 2.126870383081e-05 26 KSP Residual norm 1.916858295220e-05 27 KSP Residual norm 1.787278551239e-05 28 KSP Residual norm 1.630367244253e-05 29 KSP Residual norm 1.448418257010e-05 30 KSP Residual norm 1.297702817759e-05 31 KSP Residual norm 1.163154365095e-05 32 KSP Residual norm 1.112403023070e-05 33 KSP Residual norm 9.562485089284e-06 34 KSP Residual norm 8.090875215548e-06 35 KSP Residual norm 7.365316880616e-06 36 KSP Residual norm 6.490558913849e-06 37 KSP Residual norm 5.490720696473e-06 38 KSP Residual norm 4.889993822583e-06 39 KSP Residual norm 4.240229108611e-06 40 KSP Residual norm 4.055809490180e-06 41 KSP Residual norm 3.711705751009e-06 42 KSP Residual norm 3.072702194758e-06 43 KSP Residual norm 2.847854845271e-06 44 KSP Residual norm 2.591918260342e-06 45 KSP Residual norm 2.106166307304e-06 46 KSP Residual norm 1.825735977728e-06 47 KSP Residual norm 1.652433417938e-06 48 KSP Residual norm 1.663190707162e-06 49 KSP Residual norm 1.516622999196e-06 50 KSP Residual norm 1.181290840831e-06 51 KSP Residual norm 1.029068488627e-06 52 KSP Residual norm 8.423105866540e-07 53 KSP Residual norm 6.587332873009e-07 54 KSP Residual norm 5.961621074795e-07 55 KSP Residual norm 4.998087026841e-07 56 KSP Residual norm 4.511507612384e-07 57 KSP Residual norm 4.460497909512e-07 58 KSP Residual norm 4.105831049055e-07 59 KSP Residual norm 3.676751228872e-07 60 KSP Residual norm 2.958046501445e-07 61 KSP Residual norm 2.417258417823e-07 62 KSP Residual norm 2.030642525828e-07 63 KSP Residual norm 1.560066698513e-07 64 KSP Residual norm 1.372480903474e-07 65 KSP Residual norm 1.295951349641e-07 66 KSP Residual norm 1.252167878690e-07 67 KSP Residual norm 1.229654012693e-07 68 KSP Residual norm 1.189378346191e-07 69 KSP Residual norm 9.854574354458e-08 70 KSP Residual norm 8.419440019896e-08 71 KSP Residual norm 7.514176463563e-08 72 KSP Residual norm 6.898096360473e-08 73 KSP Residual norm 6.590957946669e-08 74 KSP Residual norm 5.820989756204e-08 75 KSP Residual norm 4.650085553019e-08 76 KSP Residual norm 3.617544162585e-08 77 KSP Residual norm 2.682816549403e-08 78 KSP Residual norm 2.069272280887e-08 79 KSP Residual norm 1.720897137942e-08 80 KSP Residual norm 1.491930893360e-08 81 KSP Residual norm 1.302497792373e-08 82 KSP Residual norm 1.151142679282e-08 83 KSP Residual norm 1.013626025008e-08 18 KSP preconditioned resid norm 2.169456154145e+05 true resid norm 2.315539961842e+16 ||r(i)||/||b|| 1.318645640842e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.104926636821e-03 1 KSP Residual norm 4.901360946873e-04 2 KSP Residual norm 7.483102349257e-04 3 KSP Residual norm 9.196903999372e-04 4 KSP Residual norm 3.695002350688e-04 5 KSP Residual norm 3.467871154561e-04 6 KSP Residual norm 2.910736640461e-04 7 KSP Residual norm 2.168040412436e-04 8 KSP Residual norm 1.748089027602e-04 9 KSP Residual norm 1.355236100671e-04 10 KSP Residual norm 1.039409243028e-04 11 KSP Residual norm 8.905983498733e-05 12 KSP Residual norm 6.833132900943e-05 13 KSP Residual norm 5.462032713138e-05 14 KSP Residual norm 4.617069050036e-05 15 KSP Residual norm 4.124988498795e-05 16 KSP Residual norm 4.100007304891e-05 17 KSP Residual norm 3.861618642049e-05 18 KSP Residual norm 3.311774686138e-05 19 KSP Residual norm 2.861282278629e-05 20 KSP Residual norm 2.650770404091e-05 21 KSP Residual norm 2.623837784316e-05 22 KSP Residual norm 2.477307973651e-05 23 KSP Residual norm 2.193020294856e-05 24 KSP Residual norm 1.978158125720e-05 25 KSP Residual norm 1.628800002949e-05 26 KSP Residual norm 1.337066349183e-05 27 KSP Residual norm 1.056084362902e-05 28 KSP Residual norm 8.327146674636e-06 29 KSP Residual norm 7.476520001124e-06 30 KSP Residual norm 6.758688660092e-06 31 KSP Residual norm 6.344916596862e-06 32 KSP Residual norm 5.639172949263e-06 33 KSP Residual norm 5.019162446101e-06 34 KSP Residual norm 4.329666750214e-06 35 KSP Residual norm 3.932025165108e-06 36 KSP Residual norm 3.824661744409e-06 37 KSP Residual norm 3.612800792369e-06 38 KSP Residual norm 3.115762198290e-06 39 KSP Residual norm 2.644784127796e-06 40 KSP Residual norm 2.410462138367e-06 41 KSP Residual norm 2.231325971284e-06 42 KSP Residual norm 2.192679923140e-06 43 KSP Residual norm 2.095458366674e-06 44 KSP Residual norm 1.728215328947e-06 45 KSP Residual norm 1.424776653802e-06 46 KSP Residual norm 1.267997887003e-06 47 KSP Residual norm 1.142675191748e-06 48 KSP Residual norm 1.023886780243e-06 49 KSP Residual norm 9.001970836361e-07 50 KSP Residual norm 7.768838339449e-07 51 KSP Residual norm 6.724756885041e-07 52 KSP Residual norm 5.987063594814e-07 53 KSP Residual norm 5.447521724472e-07 54 KSP Residual norm 4.292668513514e-07 55 KSP Residual norm 3.205033569052e-07 56 KSP Residual norm 2.709805126595e-07 57 KSP Residual norm 2.789370480631e-07 58 KSP Residual norm 2.966717039899e-07 59 KSP Residual norm 2.983681045338e-07 60 KSP Residual norm 2.743015317437e-07 61 KSP Residual norm 2.253539038506e-07 62 KSP Residual norm 1.971698697952e-07 63 KSP Residual norm 1.747084246538e-07 64 KSP Residual norm 1.522096036417e-07 65 KSP Residual norm 1.304048372684e-07 66 KSP Residual norm 1.071001473850e-07 67 KSP Residual norm 9.071875432813e-08 68 KSP Residual norm 7.484051013814e-08 69 KSP Residual norm 6.824342689597e-08 70 KSP Residual norm 6.458097324937e-08 71 KSP Residual norm 5.498262048912e-08 72 KSP Residual norm 4.680346480838e-08 73 KSP Residual norm 3.941139784530e-08 74 KSP Residual norm 3.222185743391e-08 75 KSP Residual norm 2.724758428997e-08 76 KSP Residual norm 2.435119329492e-08 77 KSP Residual norm 2.133393055823e-08 78 KSP Residual norm 1.939351540643e-08 79 KSP Residual norm 1.750266617565e-08 80 KSP Residual norm 1.511157745552e-08 81 KSP Residual norm 1.298075717751e-08 82 KSP Residual norm 1.060778061111e-08 19 KSP preconditioned resid norm 1.853355800655e+05 true resid norm 2.647055850521e+16 ||r(i)||/||b|| 1.507436155659e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.003394072838e-03 1 KSP Residual norm 5.696683252227e-04 2 KSP Residual norm 7.732470294927e-04 3 KSP Residual norm 7.060559354310e-04 4 KSP Residual norm 4.241476918480e-04 5 KSP Residual norm 3.959355987071e-04 6 KSP Residual norm 3.448304989909e-04 7 KSP Residual norm 2.622563588950e-04 8 KSP Residual norm 2.134878116972e-04 9 KSP Residual norm 1.971741545468e-04 10 KSP Residual norm 1.905991690342e-04 11 KSP Residual norm 1.875717643270e-04 12 KSP Residual norm 1.654126756784e-04 13 KSP Residual norm 1.546681671198e-04 14 KSP Residual norm 1.407597959742e-04 15 KSP Residual norm 1.301279903165e-04 16 KSP Residual norm 1.335640781539e-04 17 KSP Residual norm 1.276671033626e-04 18 KSP Residual norm 1.126807978273e-04 19 KSP Residual norm 9.249888114060e-05 20 KSP Residual norm 7.957117048478e-05 21 KSP Residual norm 6.549436075359e-05 22 KSP Residual norm 6.123550879741e-05 23 KSP Residual norm 5.500682467595e-05 24 KSP Residual norm 4.608792819387e-05 25 KSP Residual norm 3.704104678580e-05 26 KSP Residual norm 3.044396889081e-05 27 KSP Residual norm 2.621815700709e-05 28 KSP Residual norm 2.312945424914e-05 29 KSP Residual norm 2.136627206694e-05 30 KSP Residual norm 1.748794168727e-05 31 KSP Residual norm 1.519585022652e-05 32 KSP Residual norm 1.346370869511e-05 33 KSP Residual norm 1.244136226406e-05 34 KSP Residual norm 1.036703813047e-05 35 KSP Residual norm 9.127586098080e-06 36 KSP Residual norm 8.589276252940e-06 37 KSP Residual norm 7.439087298358e-06 38 KSP Residual norm 6.032096842967e-06 39 KSP Residual norm 5.371001354826e-06 40 KSP Residual norm 5.052628561343e-06 41 KSP Residual norm 4.235581891213e-06 42 KSP Residual norm 4.006341188218e-06 43 KSP Residual norm 3.674875844409e-06 44 KSP Residual norm 3.118907549004e-06 45 KSP Residual norm 2.838510594690e-06 46 KSP Residual norm 2.552724420933e-06 47 KSP Residual norm 2.485876074870e-06 48 KSP Residual norm 2.329343858497e-06 49 KSP Residual norm 2.129840644699e-06 50 KSP Residual norm 1.911315218659e-06 51 KSP Residual norm 1.708689794884e-06 52 KSP Residual norm 1.337006783095e-06 53 KSP Residual norm 1.010201566588e-06 54 KSP Residual norm 7.812537281814e-07 55 KSP Residual norm 6.065635032670e-07 56 KSP Residual norm 4.745408866103e-07 57 KSP Residual norm 4.497898757200e-07 58 KSP Residual norm 4.410483435916e-07 59 KSP Residual norm 4.054157667251e-07 60 KSP Residual norm 3.677507331387e-07 61 KSP Residual norm 3.232768365893e-07 62 KSP Residual norm 2.752489805134e-07 63 KSP Residual norm 2.168346903127e-07 64 KSP Residual norm 1.935391974554e-07 65 KSP Residual norm 1.716880482916e-07 66 KSP Residual norm 1.538439103468e-07 67 KSP Residual norm 1.484143529597e-07 68 KSP Residual norm 1.304001182758e-07 69 KSP Residual norm 1.096602465484e-07 70 KSP Residual norm 9.276994685310e-08 71 KSP Residual norm 7.584105921700e-08 72 KSP Residual norm 6.162511740874e-08 73 KSP Residual norm 5.973182970634e-08 74 KSP Residual norm 5.537962543187e-08 75 KSP Residual norm 5.097661590262e-08 76 KSP Residual norm 4.342942447113e-08 77 KSP Residual norm 3.218873643226e-08 78 KSP Residual norm 2.245432765023e-08 79 KSP Residual norm 1.719369765218e-08 80 KSP Residual norm 1.277469549039e-08 81 KSP Residual norm 9.288274152713e-09 20 KSP preconditioned resid norm 1.684388285095e+05 true resid norm 2.847986111485e+16 ||r(i)||/||b|| 1.621861221562e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.537382633878e-04 1 KSP Residual norm 5.805828678495e-04 2 KSP Residual norm 7.965481811167e-04 3 KSP Residual norm 7.111994300477e-04 4 KSP Residual norm 4.204604609856e-04 5 KSP Residual norm 3.377502975996e-04 6 KSP Residual norm 2.471202748238e-04 7 KSP Residual norm 1.839752037246e-04 8 KSP Residual norm 1.492243582648e-04 9 KSP Residual norm 1.339640368056e-04 10 KSP Residual norm 1.102253289921e-04 11 KSP Residual norm 9.211440024755e-05 12 KSP Residual norm 8.135009310268e-05 13 KSP Residual norm 7.426343872420e-05 14 KSP Residual norm 7.141467046487e-05 15 KSP Residual norm 6.751872578598e-05 16 KSP Residual norm 6.099760203763e-05 17 KSP Residual norm 5.130445074494e-05 18 KSP Residual norm 4.237438001810e-05 19 KSP Residual norm 3.896618823252e-05 20 KSP Residual norm 3.726344868786e-05 21 KSP Residual norm 3.454550895663e-05 22 KSP Residual norm 3.202807156908e-05 23 KSP Residual norm 2.613934877233e-05 24 KSP Residual norm 1.993928582774e-05 25 KSP Residual norm 1.665949348566e-05 26 KSP Residual norm 1.469515167173e-05 27 KSP Residual norm 1.309464698227e-05 28 KSP Residual norm 1.120570391983e-05 29 KSP Residual norm 1.000025419139e-05 30 KSP Residual norm 8.992698775246e-06 31 KSP Residual norm 8.546494427485e-06 32 KSP Residual norm 7.619433610613e-06 33 KSP Residual norm 6.684353305851e-06 34 KSP Residual norm 5.805665463552e-06 35 KSP Residual norm 5.538825845230e-06 36 KSP Residual norm 5.576972541327e-06 37 KSP Residual norm 5.153810552137e-06 38 KSP Residual norm 4.479657577698e-06 39 KSP Residual norm 4.113747150514e-06 40 KSP Residual norm 3.768004090406e-06 41 KSP Residual norm 3.265930621579e-06 42 KSP Residual norm 2.674433544064e-06 43 KSP Residual norm 2.249961923301e-06 44 KSP Residual norm 1.896980704269e-06 45 KSP Residual norm 1.590077828881e-06 46 KSP Residual norm 1.433599415304e-06 47 KSP Residual norm 1.264942493409e-06 48 KSP Residual norm 1.147269407683e-06 49 KSP Residual norm 1.071149972893e-06 50 KSP Residual norm 9.960572128471e-07 51 KSP Residual norm 9.054498144213e-07 52 KSP Residual norm 7.913472707524e-07 53 KSP Residual norm 6.676027659634e-07 54 KSP Residual norm 5.767292660234e-07 55 KSP Residual norm 4.747825693063e-07 56 KSP Residual norm 3.873369549872e-07 57 KSP Residual norm 3.615075410476e-07 58 KSP Residual norm 3.565791759558e-07 59 KSP Residual norm 3.395980578492e-07 60 KSP Residual norm 3.075627537411e-07 61 KSP Residual norm 2.547754484475e-07 62 KSP Residual norm 2.307419116310e-07 63 KSP Residual norm 2.078221774477e-07 64 KSP Residual norm 1.949611630826e-07 65 KSP Residual norm 1.896807262403e-07 66 KSP Residual norm 1.593277329169e-07 67 KSP Residual norm 1.420542254058e-07 68 KSP Residual norm 1.204522352312e-07 69 KSP Residual norm 9.055826906855e-08 70 KSP Residual norm 7.923260846165e-08 71 KSP Residual norm 6.764523107534e-08 72 KSP Residual norm 5.145174923238e-08 73 KSP Residual norm 4.778528619223e-08 74 KSP Residual norm 4.740345233515e-08 75 KSP Residual norm 3.974844378837e-08 76 KSP Residual norm 3.524092573411e-08 77 KSP Residual norm 2.934993321551e-08 78 KSP Residual norm 2.313698716287e-08 79 KSP Residual norm 1.964932311386e-08 80 KSP Residual norm 1.693938270624e-08 81 KSP Residual norm 1.386799725078e-08 82 KSP Residual norm 1.113391565463e-08 83 KSP Residual norm 8.631102817980e-09 21 KSP preconditioned resid norm 1.487806487874e+05 true resid norm 3.106924793675e+16 ||r(i)||/||b|| 1.769320721351e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.810993976025e-04 1 KSP Residual norm 4.913918413805e-04 2 KSP Residual norm 5.907714832462e-04 3 KSP Residual norm 4.976750762555e-04 4 KSP Residual norm 3.115264025829e-04 5 KSP Residual norm 3.179209881946e-04 6 KSP Residual norm 2.532800518764e-04 7 KSP Residual norm 1.636904534251e-04 8 KSP Residual norm 1.357757591304e-04 9 KSP Residual norm 1.298703024028e-04 10 KSP Residual norm 1.238484927286e-04 11 KSP Residual norm 1.308135661460e-04 12 KSP Residual norm 1.257468923606e-04 13 KSP Residual norm 1.084311062611e-04 14 KSP Residual norm 1.016543604295e-04 15 KSP Residual norm 9.676356124751e-05 16 KSP Residual norm 8.575190221132e-05 17 KSP Residual norm 7.943696412936e-05 18 KSP Residual norm 6.756756438112e-05 19 KSP Residual norm 5.663282149118e-05 20 KSP Residual norm 4.994174797292e-05 21 KSP Residual norm 4.259147023293e-05 22 KSP Residual norm 3.717689923584e-05 23 KSP Residual norm 3.117252650288e-05 24 KSP Residual norm 2.817084233291e-05 25 KSP Residual norm 2.677111718607e-05 26 KSP Residual norm 2.223989377061e-05 27 KSP Residual norm 1.840409375961e-05 28 KSP Residual norm 1.629031423294e-05 29 KSP Residual norm 1.480731244178e-05 30 KSP Residual norm 1.450633839369e-05 31 KSP Residual norm 1.383421165283e-05 32 KSP Residual norm 1.126659173232e-05 33 KSP Residual norm 9.046222000982e-06 34 KSP Residual norm 7.423634843579e-06 35 KSP Residual norm 6.448874764476e-06 36 KSP Residual norm 5.769499084658e-06 37 KSP Residual norm 4.878605782202e-06 38 KSP Residual norm 4.623523488208e-06 39 KSP Residual norm 4.629385226328e-06 40 KSP Residual norm 4.513863767124e-06 41 KSP Residual norm 4.176769507365e-06 42 KSP Residual norm 3.736387002214e-06 43 KSP Residual norm 3.481315629465e-06 44 KSP Residual norm 3.132910696786e-06 45 KSP Residual norm 2.509756400312e-06 46 KSP Residual norm 2.113275513921e-06 47 KSP Residual norm 1.881671265557e-06 48 KSP Residual norm 1.699549513344e-06 49 KSP Residual norm 1.532260506137e-06 50 KSP Residual norm 1.412358396491e-06 51 KSP Residual norm 1.123774096400e-06 52 KSP Residual norm 9.429909862981e-07 53 KSP Residual norm 7.748560647461e-07 54 KSP Residual norm 5.999205728586e-07 55 KSP Residual norm 4.561585762233e-07 56 KSP Residual norm 3.706628650552e-07 57 KSP Residual norm 3.265410172797e-07 58 KSP Residual norm 3.293590423008e-07 59 KSP Residual norm 3.391473249581e-07 60 KSP Residual norm 3.141948913859e-07 61 KSP Residual norm 2.734434019163e-07 62 KSP Residual norm 2.122393814236e-07 63 KSP Residual norm 1.799312618961e-07 64 KSP Residual norm 1.524589865814e-07 65 KSP Residual norm 1.420346798007e-07 66 KSP Residual norm 1.329670334278e-07 67 KSP Residual norm 1.236468730866e-07 68 KSP Residual norm 1.081920532807e-07 69 KSP Residual norm 8.640326460684e-08 70 KSP Residual norm 7.233329781509e-08 71 KSP Residual norm 5.873858313210e-08 72 KSP Residual norm 4.862676487180e-08 73 KSP Residual norm 3.863378706232e-08 74 KSP Residual norm 2.830135007971e-08 75 KSP Residual norm 2.354496966257e-08 76 KSP Residual norm 1.945874592626e-08 77 KSP Residual norm 1.466758952108e-08 78 KSP Residual norm 1.243539154133e-08 79 KSP Residual norm 9.933994823062e-09 80 KSP Residual norm 8.686746036269e-09 22 KSP preconditioned resid norm 1.324121731957e+05 true resid norm 3.364664152788e+16 ||r(i)||/||b|| 1.916097234807e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.236570261854e-04 1 KSP Residual norm 4.706242811827e-04 2 KSP Residual norm 5.562488873553e-04 3 KSP Residual norm 5.609783800068e-04 4 KSP Residual norm 3.454545651402e-04 5 KSP Residual norm 2.877563228972e-04 6 KSP Residual norm 2.462870604430e-04 7 KSP Residual norm 1.716609037117e-04 8 KSP Residual norm 1.458496918538e-04 9 KSP Residual norm 1.246082437876e-04 10 KSP Residual norm 1.053101658509e-04 11 KSP Residual norm 1.029900301184e-04 12 KSP Residual norm 1.036363762928e-04 13 KSP Residual norm 9.824668678896e-05 14 KSP Residual norm 9.680288224021e-05 15 KSP Residual norm 7.812856544877e-05 16 KSP Residual norm 6.294674948349e-05 17 KSP Residual norm 5.560159461833e-05 18 KSP Residual norm 4.530925425629e-05 19 KSP Residual norm 3.766396771754e-05 20 KSP Residual norm 3.269389848494e-05 21 KSP Residual norm 3.117697933641e-05 22 KSP Residual norm 2.917855994036e-05 23 KSP Residual norm 2.776502616972e-05 24 KSP Residual norm 2.774930874360e-05 25 KSP Residual norm 2.630073186780e-05 26 KSP Residual norm 2.264785016088e-05 27 KSP Residual norm 1.950080597855e-05 28 KSP Residual norm 1.638213857905e-05 29 KSP Residual norm 1.440300714192e-05 30 KSP Residual norm 1.299794654707e-05 31 KSP Residual norm 1.128794452692e-05 32 KSP Residual norm 9.351390145761e-06 33 KSP Residual norm 8.372411327498e-06 34 KSP Residual norm 7.863111974186e-06 35 KSP Residual norm 7.081928409805e-06 36 KSP Residual norm 6.328178952492e-06 37 KSP Residual norm 5.770678494521e-06 38 KSP Residual norm 4.779970847015e-06 39 KSP Residual norm 4.425298365097e-06 40 KSP Residual norm 4.079181634024e-06 41 KSP Residual norm 3.595296107917e-06 42 KSP Residual norm 3.173499176922e-06 43 KSP Residual norm 2.805457693519e-06 44 KSP Residual norm 2.578107803412e-06 45 KSP Residual norm 2.075042306164e-06 46 KSP Residual norm 1.755417060717e-06 47 KSP Residual norm 1.494531720767e-06 48 KSP Residual norm 1.131456404472e-06 49 KSP Residual norm 9.072178494548e-07 50 KSP Residual norm 7.929093739793e-07 51 KSP Residual norm 6.369327555831e-07 52 KSP Residual norm 4.950467387802e-07 53 KSP Residual norm 4.162092175812e-07 54 KSP Residual norm 3.860153378126e-07 55 KSP Residual norm 3.438318181045e-07 56 KSP Residual norm 2.702820435485e-07 57 KSP Residual norm 2.073784010026e-07 58 KSP Residual norm 1.651892741294e-07 59 KSP Residual norm 1.503215788499e-07 60 KSP Residual norm 1.369953027125e-07 61 KSP Residual norm 1.111914925219e-07 62 KSP Residual norm 9.430048851337e-08 63 KSP Residual norm 7.984752571888e-08 64 KSP Residual norm 6.530383059670e-08 65 KSP Residual norm 5.609768876872e-08 66 KSP Residual norm 4.851481014568e-08 67 KSP Residual norm 4.362544737446e-08 68 KSP Residual norm 3.837273733659e-08 69 KSP Residual norm 3.267537354674e-08 70 KSP Residual norm 2.757925624430e-08 71 KSP Residual norm 2.346703494081e-08 72 KSP Residual norm 2.055401461567e-08 73 KSP Residual norm 1.802215498353e-08 74 KSP Residual norm 1.651584385615e-08 75 KSP Residual norm 1.528391030961e-08 76 KSP Residual norm 1.335118078025e-08 77 KSP Residual norm 1.123265559933e-08 78 KSP Residual norm 9.495952915143e-09 79 KSP Residual norm 7.787131255500e-09 23 KSP preconditioned resid norm 1.154787064400e+05 true resid norm 3.664739272329e+16 ||r(i)||/||b|| 2.086982969810e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.561371442395e-03 1 KSP Residual norm 5.546154721467e-04 2 KSP Residual norm 5.137400913198e-04 3 KSP Residual norm 8.288166432585e-04 4 KSP Residual norm 3.970276322695e-04 5 KSP Residual norm 2.404563958565e-04 6 KSP Residual norm 2.405035565019e-04 7 KSP Residual norm 1.695008772452e-04 8 KSP Residual norm 1.248158759227e-04 9 KSP Residual norm 1.235448001048e-04 10 KSP Residual norm 1.116421920902e-04 11 KSP Residual norm 1.034494992550e-04 12 KSP Residual norm 9.646733806247e-05 13 KSP Residual norm 8.533215401184e-05 14 KSP Residual norm 8.558747902630e-05 15 KSP Residual norm 8.287881075581e-05 16 KSP Residual norm 6.964571124122e-05 17 KSP Residual norm 5.336826311381e-05 18 KSP Residual norm 4.271570823568e-05 19 KSP Residual norm 3.445993009786e-05 20 KSP Residual norm 2.489905393520e-05 21 KSP Residual norm 1.902151191460e-05 22 KSP Residual norm 1.699915744743e-05 23 KSP Residual norm 1.563527709466e-05 24 KSP Residual norm 1.461637347331e-05 25 KSP Residual norm 1.426211606485e-05 26 KSP Residual norm 1.297804137498e-05 27 KSP Residual norm 1.184760374627e-05 28 KSP Residual norm 1.120599992241e-05 29 KSP Residual norm 1.094260507369e-05 30 KSP Residual norm 9.730912037432e-06 31 KSP Residual norm 8.125057600001e-06 32 KSP Residual norm 6.194194833738e-06 33 KSP Residual norm 5.505758411786e-06 34 KSP Residual norm 5.018672482728e-06 35 KSP Residual norm 4.617820329755e-06 36 KSP Residual norm 4.290480808024e-06 37 KSP Residual norm 4.077490465777e-06 38 KSP Residual norm 3.786772056769e-06 39 KSP Residual norm 3.383737529423e-06 40 KSP Residual norm 2.969295091019e-06 41 KSP Residual norm 2.611618958681e-06 42 KSP Residual norm 2.325420372727e-06 43 KSP Residual norm 1.968790355182e-06 44 KSP Residual norm 1.752245246567e-06 45 KSP Residual norm 1.439594540088e-06 46 KSP Residual norm 1.219326411444e-06 47 KSP Residual norm 1.029236420209e-06 48 KSP Residual norm 8.760970900512e-07 49 KSP Residual norm 7.429762117272e-07 50 KSP Residual norm 6.763390133587e-07 51 KSP Residual norm 6.660291833363e-07 52 KSP Residual norm 6.411817883412e-07 53 KSP Residual norm 5.410486176647e-07 54 KSP Residual norm 4.959970164233e-07 55 KSP Residual norm 4.670367172974e-07 56 KSP Residual norm 4.068916423305e-07 57 KSP Residual norm 3.724788298233e-07 58 KSP Residual norm 3.515328700066e-07 59 KSP Residual norm 3.148652656014e-07 60 KSP Residual norm 2.862742320464e-07 61 KSP Residual norm 2.407543434259e-07 62 KSP Residual norm 2.100653986253e-07 63 KSP Residual norm 1.803017590707e-07 64 KSP Residual norm 1.555781249095e-07 65 KSP Residual norm 1.433530752521e-07 66 KSP Residual norm 1.321537121012e-07 67 KSP Residual norm 1.132092024354e-07 68 KSP Residual norm 8.942805346621e-08 69 KSP Residual norm 7.110908541938e-08 70 KSP Residual norm 5.547505324779e-08 71 KSP Residual norm 4.431699413762e-08 72 KSP Residual norm 3.826647519651e-08 73 KSP Residual norm 3.644532359166e-08 74 KSP Residual norm 2.916786706694e-08 75 KSP Residual norm 2.500274485671e-08 76 KSP Residual norm 2.012316307652e-08 77 KSP Residual norm 1.517678510171e-08 24 KSP preconditioned resid norm 9.690339957219e+04 true resid norm 4.059415187015e+16 ||r(i)||/||b|| 2.311741636480e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.556379584761e-03 1 KSP Residual norm 5.880421344453e-04 2 KSP Residual norm 4.821259040552e-04 3 KSP Residual norm 6.527388586404e-04 4 KSP Residual norm 3.151411320522e-04 5 KSP Residual norm 2.112469967696e-04 6 KSP Residual norm 1.961263520180e-04 7 KSP Residual norm 1.442249596814e-04 8 KSP Residual norm 1.230318626522e-04 9 KSP Residual norm 1.142902071994e-04 10 KSP Residual norm 9.188953457618e-05 11 KSP Residual norm 8.151038952014e-05 12 KSP Residual norm 7.979013714226e-05 13 KSP Residual norm 6.750838636857e-05 14 KSP Residual norm 5.644018931222e-05 15 KSP Residual norm 4.650616005477e-05 16 KSP Residual norm 3.700155124834e-05 17 KSP Residual norm 2.760156237493e-05 18 KSP Residual norm 2.145589868147e-05 19 KSP Residual norm 1.851807700041e-05 20 KSP Residual norm 1.594718920076e-05 21 KSP Residual norm 1.387886974918e-05 22 KSP Residual norm 1.217932491402e-05 23 KSP Residual norm 1.079699866349e-05 24 KSP Residual norm 1.004503911227e-05 25 KSP Residual norm 1.065822418695e-05 26 KSP Residual norm 9.608929912134e-06 27 KSP Residual norm 8.097127754963e-06 28 KSP Residual norm 7.457033495404e-06 29 KSP Residual norm 6.722010154956e-06 30 KSP Residual norm 5.796871056490e-06 31 KSP Residual norm 5.302460347968e-06 32 KSP Residual norm 4.568449546353e-06 33 KSP Residual norm 4.061053565689e-06 34 KSP Residual norm 3.469318836675e-06 35 KSP Residual norm 3.060564125877e-06 36 KSP Residual norm 2.863167602510e-06 37 KSP Residual norm 2.747194414911e-06 38 KSP Residual norm 2.416301399707e-06 39 KSP Residual norm 2.104212895593e-06 40 KSP Residual norm 1.971671732801e-06 41 KSP Residual norm 1.805369446860e-06 42 KSP Residual norm 1.639537698386e-06 43 KSP Residual norm 1.431106743958e-06 44 KSP Residual norm 1.288705217354e-06 45 KSP Residual norm 1.212898252127e-06 46 KSP Residual norm 1.119049981475e-06 47 KSP Residual norm 1.094697639307e-06 48 KSP Residual norm 9.996240591188e-07 49 KSP Residual norm 8.490281680693e-07 50 KSP Residual norm 7.068362841619e-07 51 KSP Residual norm 6.077406959056e-07 52 KSP Residual norm 5.126824929047e-07 53 KSP Residual norm 4.130004917292e-07 54 KSP Residual norm 3.527425680000e-07 55 KSP Residual norm 2.984846749382e-07 56 KSP Residual norm 2.445708088157e-07 57 KSP Residual norm 2.155906130165e-07 58 KSP Residual norm 1.809125784884e-07 59 KSP Residual norm 1.442086482951e-07 60 KSP Residual norm 1.136136701697e-07 61 KSP Residual norm 9.220393875071e-08 62 KSP Residual norm 7.748596207000e-08 63 KSP Residual norm 6.053968956046e-08 64 KSP Residual norm 4.764097896279e-08 65 KSP Residual norm 4.150184523880e-08 66 KSP Residual norm 3.661923466782e-08 67 KSP Residual norm 2.864045491650e-08 68 KSP Residual norm 2.296038006028e-08 69 KSP Residual norm 2.190301906963e-08 70 KSP Residual norm 2.132992436822e-08 71 KSP Residual norm 2.019488349175e-08 72 KSP Residual norm 1.876187802996e-08 73 KSP Residual norm 1.613489776597e-08 74 KSP Residual norm 1.354484515904e-08 25 KSP preconditioned resid norm 8.541750248371e+04 true resid norm 4.382054411186e+16 ||r(i)||/||b|| 2.495477099279e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.263160981846e-03 1 KSP Residual norm 5.453863094632e-04 2 KSP Residual norm 5.465013343199e-04 3 KSP Residual norm 6.990937414369e-04 4 KSP Residual norm 3.588790812459e-04 5 KSP Residual norm 2.422243339963e-04 6 KSP Residual norm 2.132798833348e-04 7 KSP Residual norm 1.385817991349e-04 8 KSP Residual norm 1.198158582849e-04 9 KSP Residual norm 1.167098732958e-04 10 KSP Residual norm 1.041730138467e-04 11 KSP Residual norm 9.718092852428e-05 12 KSP Residual norm 1.017281492243e-04 13 KSP Residual norm 9.524346237582e-05 14 KSP Residual norm 8.107307810974e-05 15 KSP Residual norm 6.479791655019e-05 16 KSP Residual norm 5.143584203258e-05 17 KSP Residual norm 3.927492572006e-05 18 KSP Residual norm 3.502501673341e-05 19 KSP Residual norm 3.428225753375e-05 20 KSP Residual norm 3.022513398514e-05 21 KSP Residual norm 2.585093494495e-05 22 KSP Residual norm 2.255662419475e-05 23 KSP Residual norm 2.031245914470e-05 24 KSP Residual norm 1.895129486522e-05 25 KSP Residual norm 1.873167003650e-05 26 KSP Residual norm 1.767149809833e-05 27 KSP Residual norm 1.533669672743e-05 28 KSP Residual norm 1.337823819625e-05 29 KSP Residual norm 1.164935426879e-05 30 KSP Residual norm 1.030469085932e-05 31 KSP Residual norm 9.251274258045e-06 32 KSP Residual norm 7.889319407405e-06 33 KSP Residual norm 7.398648887843e-06 34 KSP Residual norm 6.719989233090e-06 35 KSP Residual norm 6.224124778495e-06 36 KSP Residual norm 5.906840948528e-06 37 KSP Residual norm 5.882272084145e-06 38 KSP Residual norm 5.130111966192e-06 39 KSP Residual norm 4.143513923016e-06 40 KSP Residual norm 3.593655808976e-06 41 KSP Residual norm 3.259585052371e-06 42 KSP Residual norm 2.965155208210e-06 43 KSP Residual norm 2.496942520079e-06 44 KSP Residual norm 2.139425725541e-06 45 KSP Residual norm 1.883717681849e-06 46 KSP Residual norm 1.634940574794e-06 47 KSP Residual norm 1.462141403815e-06 48 KSP Residual norm 1.198755279973e-06 49 KSP Residual norm 9.194929820665e-07 50 KSP Residual norm 7.373590476703e-07 51 KSP Residual norm 6.349903500526e-07 52 KSP Residual norm 4.988886313989e-07 53 KSP Residual norm 4.062403989953e-07 54 KSP Residual norm 3.539366182036e-07 55 KSP Residual norm 3.151049803850e-07 56 KSP Residual norm 2.604651077725e-07 57 KSP Residual norm 2.313045737876e-07 58 KSP Residual norm 1.868022189852e-07 59 KSP Residual norm 1.587127262635e-07 60 KSP Residual norm 1.336758372606e-07 61 KSP Residual norm 1.026283395787e-07 62 KSP Residual norm 8.337435004763e-08 63 KSP Residual norm 7.379664552643e-08 64 KSP Residual norm 6.493089452340e-08 65 KSP Residual norm 5.710022886919e-08 66 KSP Residual norm 5.549855150226e-08 67 KSP Residual norm 5.249395746603e-08 68 KSP Residual norm 4.913946936156e-08 69 KSP Residual norm 4.021655118277e-08 70 KSP Residual norm 3.217659488217e-08 71 KSP Residual norm 2.787984477306e-08 72 KSP Residual norm 2.565797697943e-08 73 KSP Residual norm 2.488147056304e-08 74 KSP Residual norm 2.328565929184e-08 75 KSP Residual norm 2.076902358323e-08 76 KSP Residual norm 1.950959738939e-08 77 KSP Residual norm 1.723016112698e-08 78 KSP Residual norm 1.481096851263e-08 79 KSP Residual norm 1.222480916986e-08 26 KSP preconditioned resid norm 6.738179237233e+04 true resid norm 4.935818323004e+16 ||r(i)||/||b|| 2.810832645030e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.532727541976e-03 1 KSP Residual norm 8.981129308270e-04 2 KSP Residual norm 4.096783864848e-04 3 KSP Residual norm 4.447317134801e-04 4 KSP Residual norm 3.283447255887e-04 5 KSP Residual norm 1.992638886033e-04 6 KSP Residual norm 2.084015185779e-04 7 KSP Residual norm 1.628211883442e-04 8 KSP Residual norm 1.009145418314e-04 9 KSP Residual norm 9.211134174709e-05 10 KSP Residual norm 8.821799290148e-05 11 KSP Residual norm 8.097534529147e-05 12 KSP Residual norm 8.643643984467e-05 13 KSP Residual norm 7.714889764732e-05 14 KSP Residual norm 7.165927997121e-05 15 KSP Residual norm 6.503150014903e-05 16 KSP Residual norm 5.702863105054e-05 17 KSP Residual norm 4.638791482164e-05 18 KSP Residual norm 3.648950094184e-05 19 KSP Residual norm 3.139922867124e-05 20 KSP Residual norm 2.949160237399e-05 21 KSP Residual norm 2.903589835737e-05 22 KSP Residual norm 2.867593287288e-05 23 KSP Residual norm 2.656223924611e-05 24 KSP Residual norm 2.440204151754e-05 25 KSP Residual norm 2.118657498380e-05 26 KSP Residual norm 1.826875212691e-05 27 KSP Residual norm 1.615747344423e-05 28 KSP Residual norm 1.324436188513e-05 29 KSP Residual norm 1.095085310540e-05 30 KSP Residual norm 9.865457945830e-06 31 KSP Residual norm 8.844360249318e-06 32 KSP Residual norm 7.013022656729e-06 33 KSP Residual norm 5.925900762305e-06 34 KSP Residual norm 5.476203232789e-06 35 KSP Residual norm 5.047108232130e-06 36 KSP Residual norm 4.542341419913e-06 37 KSP Residual norm 4.024443913505e-06 38 KSP Residual norm 3.777402182145e-06 39 KSP Residual norm 3.516395778645e-06 40 KSP Residual norm 3.067438356702e-06 41 KSP Residual norm 2.581799296313e-06 42 KSP Residual norm 2.163930954787e-06 43 KSP Residual norm 1.789094058217e-06 44 KSP Residual norm 1.632487994790e-06 45 KSP Residual norm 1.495577674001e-06 46 KSP Residual norm 1.342875805099e-06 47 KSP Residual norm 1.280885115626e-06 48 KSP Residual norm 1.262726664063e-06 49 KSP Residual norm 1.077173225008e-06 50 KSP Residual norm 8.570281319406e-07 51 KSP Residual norm 7.276861091825e-07 52 KSP Residual norm 6.280839861458e-07 53 KSP Residual norm 5.562558050428e-07 54 KSP Residual norm 4.688731967329e-07 55 KSP Residual norm 3.903576210168e-07 56 KSP Residual norm 3.249526367587e-07 57 KSP Residual norm 3.069134430532e-07 58 KSP Residual norm 2.722985340041e-07 59 KSP Residual norm 2.315377531067e-07 60 KSP Residual norm 2.201907468262e-07 61 KSP Residual norm 1.941356329188e-07 62 KSP Residual norm 1.721379303241e-07 63 KSP Residual norm 1.522248727037e-07 64 KSP Residual norm 1.511021089591e-07 65 KSP Residual norm 1.416684233369e-07 66 KSP Residual norm 1.227387882380e-07 67 KSP Residual norm 1.055908672073e-07 68 KSP Residual norm 9.629719013476e-08 69 KSP Residual norm 8.119707222122e-08 70 KSP Residual norm 6.199254637314e-08 71 KSP Residual norm 5.113871963314e-08 72 KSP Residual norm 4.125768123565e-08 73 KSP Residual norm 3.233628356351e-08 74 KSP Residual norm 2.630359020570e-08 75 KSP Residual norm 2.359214974889e-08 27 KSP preconditioned resid norm 5.559954270401e+04 true resid norm 5.372513401960e+16 ||r(i)||/||b|| 3.059520239169e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.458298763858e-03 1 KSP Residual norm 8.860325879629e-04 2 KSP Residual norm 4.588496391882e-04 3 KSP Residual norm 5.823852512060e-04 4 KSP Residual norm 4.684206540287e-04 5 KSP Residual norm 2.258498957237e-04 6 KSP Residual norm 1.601370153283e-04 7 KSP Residual norm 1.285999085506e-04 8 KSP Residual norm 1.079431492907e-04 9 KSP Residual norm 1.068766482225e-04 10 KSP Residual norm 8.595115134436e-05 11 KSP Residual norm 6.772446087951e-05 12 KSP Residual norm 7.997266800771e-05 13 KSP Residual norm 8.215819893902e-05 14 KSP Residual norm 7.809073521416e-05 15 KSP Residual norm 7.049933338204e-05 16 KSP Residual norm 6.153117053635e-05 17 KSP Residual norm 5.152907441677e-05 18 KSP Residual norm 4.131243606234e-05 19 KSP Residual norm 3.653936713893e-05 20 KSP Residual norm 3.670612866217e-05 21 KSP Residual norm 3.469717857852e-05 22 KSP Residual norm 2.954091202217e-05 23 KSP Residual norm 2.497407543625e-05 24 KSP Residual norm 2.264578796404e-05 25 KSP Residual norm 2.203727754619e-05 26 KSP Residual norm 2.059452450624e-05 27 KSP Residual norm 1.823633324603e-05 28 KSP Residual norm 1.534306868733e-05 29 KSP Residual norm 1.205296972946e-05 30 KSP Residual norm 1.010201361142e-05 31 KSP Residual norm 8.966613986159e-06 32 KSP Residual norm 7.161708702442e-06 33 KSP Residual norm 6.294437066998e-06 34 KSP Residual norm 5.743344950855e-06 35 KSP Residual norm 5.265851938943e-06 36 KSP Residual norm 4.569796627003e-06 37 KSP Residual norm 3.865843219107e-06 38 KSP Residual norm 3.495988681295e-06 39 KSP Residual norm 3.321436596148e-06 40 KSP Residual norm 2.898443695496e-06 41 KSP Residual norm 2.518640123053e-06 42 KSP Residual norm 2.412762687014e-06 43 KSP Residual norm 2.094921708489e-06 44 KSP Residual norm 1.885836807155e-06 45 KSP Residual norm 1.635967124173e-06 46 KSP Residual norm 1.275692301594e-06 47 KSP Residual norm 1.066201426259e-06 48 KSP Residual norm 1.015176008824e-06 49 KSP Residual norm 9.594540463366e-07 50 KSP Residual norm 8.891235336799e-07 51 KSP Residual norm 8.386074587068e-07 52 KSP Residual norm 7.394466893496e-07 53 KSP Residual norm 6.247670879141e-07 54 KSP Residual norm 5.512270753400e-07 55 KSP Residual norm 4.583853117419e-07 56 KSP Residual norm 3.513592819269e-07 57 KSP Residual norm 3.007522853264e-07 58 KSP Residual norm 2.799635694475e-07 59 KSP Residual norm 2.373251646745e-07 60 KSP Residual norm 2.053307975849e-07 61 KSP Residual norm 1.781955416543e-07 62 KSP Residual norm 1.520456708599e-07 63 KSP Residual norm 1.305278155902e-07 64 KSP Residual norm 1.247296876179e-07 65 KSP Residual norm 1.251804981556e-07 66 KSP Residual norm 1.172659686614e-07 67 KSP Residual norm 1.022193992228e-07 68 KSP Residual norm 8.689306315290e-08 69 KSP Residual norm 6.975176862999e-08 70 KSP Residual norm 5.369088629595e-08 71 KSP Residual norm 4.201788088418e-08 72 KSP Residual norm 3.585898255279e-08 73 KSP Residual norm 3.060707840494e-08 74 KSP Residual norm 2.526507406218e-08 75 KSP Residual norm 2.302835156508e-08 28 KSP preconditioned resid norm 4.606515567878e+04 true resid norm 5.783437414963e+16 ||r(i)||/||b|| 3.293531816336e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.330167966816e-03 1 KSP Residual norm 1.166614978432e-03 2 KSP Residual norm 4.977076305009e-04 3 KSP Residual norm 5.095461459787e-04 4 KSP Residual norm 4.747858390030e-04 5 KSP Residual norm 2.440217064195e-04 6 KSP Residual norm 1.751104956341e-04 7 KSP Residual norm 1.412153092133e-04 8 KSP Residual norm 1.038130149122e-04 9 KSP Residual norm 9.692495691618e-05 10 KSP Residual norm 8.503472373424e-05 11 KSP Residual norm 7.023573382913e-05 12 KSP Residual norm 7.927166546752e-05 13 KSP Residual norm 7.838446576939e-05 14 KSP Residual norm 7.190482942204e-05 15 KSP Residual norm 6.706937890436e-05 16 KSP Residual norm 5.785140194637e-05 17 KSP Residual norm 4.539131844559e-05 18 KSP Residual norm 3.428795842485e-05 19 KSP Residual norm 2.953860301033e-05 20 KSP Residual norm 2.657054843252e-05 21 KSP Residual norm 2.269546682447e-05 22 KSP Residual norm 1.842762634814e-05 23 KSP Residual norm 1.586278074104e-05 24 KSP Residual norm 1.418321548295e-05 25 KSP Residual norm 1.428621000145e-05 26 KSP Residual norm 1.381084130888e-05 27 KSP Residual norm 1.315707270932e-05 28 KSP Residual norm 1.241605930814e-05 29 KSP Residual norm 1.084224179468e-05 30 KSP Residual norm 8.810422424630e-06 31 KSP Residual norm 7.745459019640e-06 32 KSP Residual norm 6.472052902758e-06 33 KSP Residual norm 5.578443757592e-06 34 KSP Residual norm 4.766765111330e-06 35 KSP Residual norm 4.074383764034e-06 36 KSP Residual norm 3.487468681544e-06 37 KSP Residual norm 3.160861250108e-06 38 KSP Residual norm 3.016671170262e-06 39 KSP Residual norm 2.839181283894e-06 40 KSP Residual norm 2.599154368057e-06 41 KSP Residual norm 2.362026226551e-06 42 KSP Residual norm 2.348722714966e-06 43 KSP Residual norm 2.177595113405e-06 44 KSP Residual norm 1.973295820098e-06 45 KSP Residual norm 1.723782728176e-06 46 KSP Residual norm 1.311679210459e-06 47 KSP Residual norm 1.007973849961e-06 48 KSP Residual norm 8.436412131832e-07 49 KSP Residual norm 7.477532633586e-07 50 KSP Residual norm 6.696960026556e-07 51 KSP Residual norm 6.386178770376e-07 52 KSP Residual norm 6.042337366551e-07 53 KSP Residual norm 5.341764196295e-07 54 KSP Residual norm 4.945411067002e-07 55 KSP Residual norm 4.571638274256e-07 56 KSP Residual norm 3.801193049780e-07 57 KSP Residual norm 3.246928135180e-07 58 KSP Residual norm 3.094877886287e-07 59 KSP Residual norm 2.771049744814e-07 60 KSP Residual norm 2.500935768214e-07 61 KSP Residual norm 2.310724465447e-07 62 KSP Residual norm 2.085745148626e-07 63 KSP Residual norm 1.744735959340e-07 64 KSP Residual norm 1.509478279392e-07 65 KSP Residual norm 1.358954071493e-07 66 KSP Residual norm 1.167340661337e-07 67 KSP Residual norm 1.021900109069e-07 68 KSP Residual norm 8.899532715016e-08 69 KSP Residual norm 8.006765682760e-08 70 KSP Residual norm 7.018250953598e-08 71 KSP Residual norm 5.664446960496e-08 72 KSP Residual norm 4.364335238503e-08 73 KSP Residual norm 3.225710840994e-08 29 KSP preconditioned resid norm 3.847038156005e+04 true resid norm 6.227152566671e+16 ||r(i)||/||b|| 3.546217177080e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.202966787944e-03 1 KSP Residual norm 1.476228608294e-03 2 KSP Residual norm 6.078893181096e-04 3 KSP Residual norm 5.162213519161e-04 4 KSP Residual norm 4.907152857467e-04 5 KSP Residual norm 2.637215509866e-04 6 KSP Residual norm 2.010791109289e-04 7 KSP Residual norm 1.921418660075e-04 8 KSP Residual norm 1.360429566542e-04 9 KSP Residual norm 1.232082276212e-04 10 KSP Residual norm 1.118122749298e-04 11 KSP Residual norm 8.855241262449e-05 12 KSP Residual norm 9.578167955567e-05 13 KSP Residual norm 9.728393408392e-05 14 KSP Residual norm 9.148679109032e-05 15 KSP Residual norm 8.400718910102e-05 16 KSP Residual norm 7.067989753275e-05 17 KSP Residual norm 5.597927515879e-05 18 KSP Residual norm 4.443794083219e-05 19 KSP Residual norm 3.905097938906e-05 20 KSP Residual norm 3.773247510791e-05 21 KSP Residual norm 3.640986535985e-05 22 KSP Residual norm 3.319083246437e-05 23 KSP Residual norm 2.998367684259e-05 24 KSP Residual norm 2.626685507765e-05 25 KSP Residual norm 2.594007510751e-05 26 KSP Residual norm 2.526568911596e-05 27 KSP Residual norm 2.269335562029e-05 28 KSP Residual norm 1.865868014539e-05 29 KSP Residual norm 1.587284482795e-05 30 KSP Residual norm 1.348975295845e-05 31 KSP Residual norm 1.221250531481e-05 32 KSP Residual norm 1.012410350609e-05 33 KSP Residual norm 8.650012518139e-06 34 KSP Residual norm 7.468922340080e-06 35 KSP Residual norm 6.551584695498e-06 36 KSP Residual norm 5.790829479362e-06 37 KSP Residual norm 5.322203833066e-06 38 KSP Residual norm 4.893153971012e-06 39 KSP Residual norm 4.345808327390e-06 40 KSP Residual norm 3.689817967514e-06 41 KSP Residual norm 3.212922552460e-06 42 KSP Residual norm 2.992187105683e-06 43 KSP Residual norm 2.655583659965e-06 44 KSP Residual norm 2.269001782011e-06 45 KSP Residual norm 2.062820568696e-06 46 KSP Residual norm 1.793993524875e-06 47 KSP Residual norm 1.620561353911e-06 48 KSP Residual norm 1.558144431224e-06 49 KSP Residual norm 1.365019964765e-06 50 KSP Residual norm 1.134587733675e-06 51 KSP Residual norm 1.026575836641e-06 52 KSP Residual norm 9.187620210473e-07 53 KSP Residual norm 7.544642568055e-07 54 KSP Residual norm 6.394745655338e-07 55 KSP Residual norm 5.742379060414e-07 56 KSP Residual norm 4.971178596498e-07 57 KSP Residual norm 4.608218030992e-07 58 KSP Residual norm 4.359319043829e-07 59 KSP Residual norm 3.863190705487e-07 60 KSP Residual norm 3.459223040039e-07 61 KSP Residual norm 2.859792196232e-07 62 KSP Residual norm 2.319741419507e-07 63 KSP Residual norm 1.823871484709e-07 64 KSP Residual norm 1.585690065684e-07 65 KSP Residual norm 1.405853273542e-07 66 KSP Residual norm 1.301176318742e-07 67 KSP Residual norm 1.303446840750e-07 68 KSP Residual norm 1.191442369748e-07 69 KSP Residual norm 1.016544010795e-07 70 KSP Residual norm 8.381356465844e-08 71 KSP Residual norm 6.958741218310e-08 72 KSP Residual norm 5.667207918522e-08 73 KSP Residual norm 4.463762060517e-08 74 KSP Residual norm 3.570573769498e-08 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.561030501163e+02 1 KSP Residual norm 8.836165701345e+01 2 KSP Residual norm 3.062123346477e+01 3 KSP Residual norm 1.751946280215e+01 4 KSP Residual norm 1.593042090879e+01 5 KSP Residual norm 9.001478876471e+00 6 KSP Residual norm 5.144242671251e+00 7 KSP Residual norm 4.274948494095e+00 8 KSP Residual norm 3.198965978367e+00 9 KSP Residual norm 2.861732283102e+00 10 KSP Residual norm 2.753783810750e+00 11 KSP Residual norm 1.953048037505e+00 12 KSP Residual norm 1.666516164050e+00 13 KSP Residual norm 1.550244629808e+00 14 KSP Residual norm 1.435050153277e+00 15 KSP Residual norm 1.415962906052e+00 16 KSP Residual norm 1.362117136956e+00 17 KSP Residual norm 1.389161559294e+00 18 KSP Residual norm 1.202872558983e+00 19 KSP Residual norm 8.623780487822e-01 20 KSP Residual norm 6.992608823968e-01 21 KSP Residual norm 6.210826811507e-01 22 KSP Residual norm 5.560173122598e-01 23 KSP Residual norm 5.046947259723e-01 24 KSP Residual norm 4.696872623868e-01 25 KSP Residual norm 4.869911928727e-01 26 KSP Residual norm 4.944861821388e-01 27 KSP Residual norm 4.766650251589e-01 28 KSP Residual norm 4.204824246722e-01 29 KSP Residual norm 3.514495840277e-01 30 KSP Residual norm 2.571778777698e-01 31 KSP Residual norm 2.152328565921e-01 32 KSP Residual norm 1.871195935424e-01 33 KSP Residual norm 1.737397580891e-01 34 KSP Residual norm 1.637274475870e-01 35 KSP Residual norm 1.633080010112e-01 36 KSP Residual norm 1.574801409758e-01 37 KSP Residual norm 1.287109783196e-01 38 KSP Residual norm 9.939546805435e-02 39 KSP Residual norm 8.347268333523e-02 40 KSP Residual norm 7.315246118620e-02 41 KSP Residual norm 6.168543819919e-02 42 KSP Residual norm 5.571146020382e-02 43 KSP Residual norm 4.921127306288e-02 44 KSP Residual norm 4.196320755575e-02 45 KSP Residual norm 4.045096390520e-02 46 KSP Residual norm 3.583856895768e-02 47 KSP Residual norm 2.966981574858e-02 48 KSP Residual norm 2.674067714817e-02 49 KSP Residual norm 2.470758707879e-02 50 KSP Residual norm 2.160278212171e-02 51 KSP Residual norm 1.799655748966e-02 52 KSP Residual norm 1.585798913990e-02 53 KSP Residual norm 1.313791274135e-02 54 KSP Residual norm 1.033589043269e-02 55 KSP Residual norm 9.693683874679e-03 56 KSP Residual norm 8.950684424769e-03 57 KSP Residual norm 6.944499386194e-03 58 KSP Residual norm 5.111071810144e-03 59 KSP Residual norm 3.768053417896e-03 60 KSP Residual norm 2.986414434813e-03 61 KSP Residual norm 2.477901432098e-03 30 KSP preconditioned resid norm 3.341483541882e+04 true resid norm 6.606668648019e+16 ||r(i)||/||b|| 3.762342674607e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.797259945442e-03 1 KSP Residual norm 2.679501064942e-03 2 KSP Residual norm 9.584759442441e-04 3 KSP Residual norm 5.797289656897e-04 4 KSP Residual norm 5.520259770641e-04 5 KSP Residual norm 3.116626077413e-04 6 KSP Residual norm 1.868471808470e-04 7 KSP Residual norm 1.673264745192e-04 8 KSP Residual norm 1.379472773140e-04 9 KSP Residual norm 1.342369259729e-04 10 KSP Residual norm 1.300813713935e-04 11 KSP Residual norm 9.811989055758e-05 12 KSP Residual norm 9.454850504853e-05 13 KSP Residual norm 9.484914974655e-05 14 KSP Residual norm 9.202930116087e-05 15 KSP Residual norm 8.890497265314e-05 16 KSP Residual norm 8.143285754864e-05 17 KSP Residual norm 7.295428613524e-05 18 KSP Residual norm 5.614277397680e-05 19 KSP Residual norm 4.108344334938e-05 20 KSP Residual norm 3.778468854239e-05 21 KSP Residual norm 3.714372508864e-05 22 KSP Residual norm 3.506071987150e-05 23 KSP Residual norm 3.240235906008e-05 24 KSP Residual norm 3.006912713671e-05 25 KSP Residual norm 3.004714161980e-05 26 KSP Residual norm 2.968594648008e-05 27 KSP Residual norm 2.807930547027e-05 28 KSP Residual norm 2.434595876551e-05 29 KSP Residual norm 1.979734001230e-05 30 KSP Residual norm 1.440035923661e-05 31 KSP Residual norm 1.175020281064e-05 32 KSP Residual norm 9.350918145626e-06 33 KSP Residual norm 7.646393875997e-06 34 KSP Residual norm 6.306665340007e-06 35 KSP Residual norm 5.483551514071e-06 36 KSP Residual norm 4.675905845790e-06 37 KSP Residual norm 3.954450758555e-06 38 KSP Residual norm 3.383919840203e-06 39 KSP Residual norm 3.151287951345e-06 40 KSP Residual norm 2.835856823407e-06 41 KSP Residual norm 2.195852572361e-06 42 KSP Residual norm 1.830496232383e-06 43 KSP Residual norm 1.505310977498e-06 44 KSP Residual norm 1.270094286825e-06 45 KSP Residual norm 1.233744824204e-06 46 KSP Residual norm 1.111131007245e-06 47 KSP Residual norm 9.290548502840e-07 48 KSP Residual norm 7.996753766714e-07 49 KSP Residual norm 6.861434803219e-07 50 KSP Residual norm 5.550478072304e-07 51 KSP Residual norm 4.534223790442e-07 52 KSP Residual norm 3.921218050596e-07 53 KSP Residual norm 3.242424310659e-07 54 KSP Residual norm 2.652693775270e-07 55 KSP Residual norm 2.500495028937e-07 56 KSP Residual norm 2.201771254649e-07 57 KSP Residual norm 1.679933859797e-07 58 KSP Residual norm 1.245564232331e-07 59 KSP Residual norm 1.007838988898e-07 60 KSP Residual norm 9.771698237786e-08 61 KSP Residual norm 9.800243337093e-08 62 KSP Residual norm 9.455738173231e-08 63 KSP Residual norm 7.946004819149e-08 64 KSP Residual norm 6.440610372555e-08 31 KSP preconditioned resid norm 3.296191763413e+04 true resid norm 6.615646640361e+16 ||r(i)||/||b|| 3.767455430449e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.904144722677e-03 1 KSP Residual norm 9.373684549838e-04 2 KSP Residual norm 6.896595968003e-04 3 KSP Residual norm 1.659195972770e-03 4 KSP Residual norm 1.734578486636e-03 5 KSP Residual norm 1.171151355373e-03 6 KSP Residual norm 1.048638276144e-03 7 KSP Residual norm 8.371975388609e-04 8 KSP Residual norm 5.775672536419e-04 9 KSP Residual norm 5.761628584397e-04 10 KSP Residual norm 5.025132838387e-04 11 KSP Residual norm 4.181865883556e-04 12 KSP Residual norm 3.728074434143e-04 13 KSP Residual norm 2.938452282494e-04 14 KSP Residual norm 2.787110154328e-04 15 KSP Residual norm 2.809873705928e-04 16 KSP Residual norm 3.076063930203e-04 17 KSP Residual norm 2.830617191700e-04 18 KSP Residual norm 2.328607785399e-04 19 KSP Residual norm 1.913047163477e-04 20 KSP Residual norm 1.497771428830e-04 21 KSP Residual norm 1.163652522617e-04 22 KSP Residual norm 9.533307337996e-05 23 KSP Residual norm 8.395041556320e-05 24 KSP Residual norm 7.130152392621e-05 25 KSP Residual norm 5.669716728571e-05 26 KSP Residual norm 4.601740554886e-05 27 KSP Residual norm 4.179109301190e-05 28 KSP Residual norm 3.839908391757e-05 29 KSP Residual norm 3.234709171163e-05 30 KSP Residual norm 2.875606376627e-05 31 KSP Residual norm 2.695776138209e-05 32 KSP Residual norm 2.172186613244e-05 33 KSP Residual norm 1.813899202853e-05 34 KSP Residual norm 1.549698141987e-05 35 KSP Residual norm 1.271397247367e-05 36 KSP Residual norm 1.021722108874e-05 37 KSP Residual norm 8.699684443694e-06 38 KSP Residual norm 7.808715453059e-06 39 KSP Residual norm 7.268042292887e-06 40 KSP Residual norm 6.512367842549e-06 41 KSP Residual norm 5.607165686161e-06 42 KSP Residual norm 5.113585837530e-06 43 KSP Residual norm 4.184641994601e-06 44 KSP Residual norm 3.605545114354e-06 45 KSP Residual norm 3.117061030589e-06 46 KSP Residual norm 2.513810214960e-06 47 KSP Residual norm 1.960496319790e-06 48 KSP Residual norm 1.717924307862e-06 49 KSP Residual norm 1.588866094637e-06 50 KSP Residual norm 1.442720205623e-06 51 KSP Residual norm 1.290576562383e-06 52 KSP Residual norm 1.118197050529e-06 53 KSP Residual norm 9.118619804648e-07 54 KSP Residual norm 7.707014685986e-07 55 KSP Residual norm 7.630101183250e-07 56 KSP Residual norm 6.521313997880e-07 57 KSP Residual norm 5.898581200397e-07 58 KSP Residual norm 5.456276588104e-07 59 KSP Residual norm 4.252023915298e-07 60 KSP Residual norm 3.394011554638e-07 61 KSP Residual norm 2.667137005476e-07 62 KSP Residual norm 2.227852737240e-07 63 KSP Residual norm 1.931766295071e-07 64 KSP Residual norm 1.827060691266e-07 65 KSP Residual norm 1.638484435979e-07 66 KSP Residual norm 1.332291816230e-07 67 KSP Residual norm 1.002097721413e-07 68 KSP Residual norm 8.261219925635e-08 69 KSP Residual norm 7.574036729502e-08 70 KSP Residual norm 7.096904751142e-08 71 KSP Residual norm 6.521151897598e-08 72 KSP Residual norm 6.278867079833e-08 73 KSP Residual norm 6.107563816735e-08 74 KSP Residual norm 5.478639174238e-08 75 KSP Residual norm 5.068904104777e-08 76 KSP Residual norm 4.423794391261e-08 77 KSP Residual norm 3.640955493348e-08 78 KSP Residual norm 3.041644716433e-08 79 KSP Residual norm 2.862829135848e-08 32 KSP preconditioned resid norm 3.204272803125e+04 true resid norm 6.639310280626e+16 ||r(i)||/||b|| 3.780931317972e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.028523756621e-03 1 KSP Residual norm 8.876569712406e-04 2 KSP Residual norm 9.188193328623e-04 3 KSP Residual norm 1.266560799388e-03 4 KSP Residual norm 1.234268416084e-03 5 KSP Residual norm 1.175764976746e-03 6 KSP Residual norm 1.267098102358e-03 7 KSP Residual norm 1.082536923131e-03 8 KSP Residual norm 9.335329621098e-04 9 KSP Residual norm 8.197719400363e-04 10 KSP Residual norm 5.855156348359e-04 11 KSP Residual norm 5.009262627853e-04 12 KSP Residual norm 4.308927868747e-04 13 KSP Residual norm 3.218764927142e-04 14 KSP Residual norm 2.643860477153e-04 15 KSP Residual norm 2.325623755896e-04 16 KSP Residual norm 2.150121250574e-04 17 KSP Residual norm 2.012349369169e-04 18 KSP Residual norm 1.884310415131e-04 19 KSP Residual norm 1.887882895914e-04 20 KSP Residual norm 1.620922731117e-04 21 KSP Residual norm 1.260687411049e-04 22 KSP Residual norm 9.266081005252e-05 23 KSP Residual norm 7.164301468863e-05 24 KSP Residual norm 6.060010692426e-05 25 KSP Residual norm 5.264197558682e-05 26 KSP Residual norm 4.125033470114e-05 27 KSP Residual norm 3.327424455051e-05 28 KSP Residual norm 2.745008555280e-05 29 KSP Residual norm 2.382719610643e-05 30 KSP Residual norm 2.238919683223e-05 31 KSP Residual norm 2.047008359348e-05 32 KSP Residual norm 1.710298147383e-05 33 KSP Residual norm 1.499276775809e-05 34 KSP Residual norm 1.365783854728e-05 35 KSP Residual norm 1.157089747420e-05 36 KSP Residual norm 9.491111576942e-06 37 KSP Residual norm 8.021066958218e-06 38 KSP Residual norm 6.886928664489e-06 39 KSP Residual norm 5.306066294273e-06 40 KSP Residual norm 4.291953383754e-06 41 KSP Residual norm 3.861521821874e-06 42 KSP Residual norm 3.489108780704e-06 43 KSP Residual norm 3.506837509708e-06 44 KSP Residual norm 3.497377191985e-06 45 KSP Residual norm 2.979103396428e-06 46 KSP Residual norm 2.218547467196e-06 47 KSP Residual norm 1.926877053237e-06 48 KSP Residual norm 1.780413795970e-06 49 KSP Residual norm 1.510574363467e-06 50 KSP Residual norm 1.344482159893e-06 51 KSP Residual norm 1.162842933482e-06 52 KSP Residual norm 9.686110235355e-07 53 KSP Residual norm 8.217617451062e-07 54 KSP Residual norm 6.807628842970e-07 55 KSP Residual norm 6.289036905607e-07 56 KSP Residual norm 5.983342704629e-07 57 KSP Residual norm 5.918636606763e-07 58 KSP Residual norm 5.703757954302e-07 59 KSP Residual norm 4.584962449423e-07 60 KSP Residual norm 3.962550272201e-07 61 KSP Residual norm 3.154476086813e-07 62 KSP Residual norm 2.667411808795e-07 63 KSP Residual norm 2.408476589671e-07 64 KSP Residual norm 2.214192017955e-07 65 KSP Residual norm 1.959114317829e-07 66 KSP Residual norm 1.732010527668e-07 67 KSP Residual norm 1.428592882355e-07 68 KSP Residual norm 1.141232657625e-07 69 KSP Residual norm 9.634172841510e-08 70 KSP Residual norm 8.430685594210e-08 71 KSP Residual norm 8.209006692464e-08 72 KSP Residual norm 7.783587416129e-08 73 KSP Residual norm 6.985928952313e-08 74 KSP Residual norm 5.847962598254e-08 75 KSP Residual norm 5.182937693660e-08 76 KSP Residual norm 4.771355973899e-08 77 KSP Residual norm 3.958143414171e-08 78 KSP Residual norm 3.147673914128e-08 79 KSP Residual norm 2.747339961043e-08 80 KSP Residual norm 2.396477243292e-08 81 KSP Residual norm 2.056277979285e-08 82 KSP Residual norm 1.789212741682e-08 33 KSP preconditioned resid norm 2.994817324340e+04 true resid norm 6.707071577786e+16 ||r(i)||/||b|| 3.819519785712e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.497608192516e-03 1 KSP Residual norm 1.252043818295e-03 2 KSP Residual norm 5.696723552997e-04 3 KSP Residual norm 6.797518492903e-04 4 KSP Residual norm 7.142091157792e-04 5 KSP Residual norm 4.926896702847e-04 6 KSP Residual norm 5.269037257366e-04 7 KSP Residual norm 4.761410133646e-04 8 KSP Residual norm 3.783683691436e-04 9 KSP Residual norm 4.086614529668e-04 10 KSP Residual norm 3.709875689061e-04 11 KSP Residual norm 3.045979386093e-04 12 KSP Residual norm 2.939427016228e-04 13 KSP Residual norm 2.085106407861e-04 14 KSP Residual norm 1.498141300579e-04 15 KSP Residual norm 1.238623890144e-04 16 KSP Residual norm 1.149556534544e-04 17 KSP Residual norm 1.013265998930e-04 18 KSP Residual norm 8.878643389115e-05 19 KSP Residual norm 8.859973774871e-05 20 KSP Residual norm 9.207371837586e-05 21 KSP Residual norm 8.560349806843e-05 22 KSP Residual norm 7.462046857923e-05 23 KSP Residual norm 6.455341087625e-05 24 KSP Residual norm 6.006355831581e-05 25 KSP Residual norm 5.627721416734e-05 26 KSP Residual norm 4.808613710491e-05 27 KSP Residual norm 4.021137771455e-05 28 KSP Residual norm 3.335847835798e-05 29 KSP Residual norm 2.730499637432e-05 30 KSP Residual norm 2.534401897779e-05 31 KSP Residual norm 2.477626864724e-05 32 KSP Residual norm 2.211349910047e-05 33 KSP Residual norm 1.991776226779e-05 34 KSP Residual norm 1.745023494380e-05 35 KSP Residual norm 1.475407128769e-05 36 KSP Residual norm 1.200720574114e-05 37 KSP Residual norm 1.035195074627e-05 38 KSP Residual norm 9.965243200742e-06 39 KSP Residual norm 9.709371137245e-06 40 KSP Residual norm 8.540665491641e-06 41 KSP Residual norm 7.301730369111e-06 42 KSP Residual norm 6.943881464745e-06 43 KSP Residual norm 6.049212969577e-06 44 KSP Residual norm 5.561340028762e-06 45 KSP Residual norm 5.057262335930e-06 46 KSP Residual norm 4.109471328295e-06 47 KSP Residual norm 3.517028197167e-06 48 KSP Residual norm 3.441769119869e-06 49 KSP Residual norm 3.004705874164e-06 50 KSP Residual norm 2.445452493814e-06 51 KSP Residual norm 2.202899953692e-06 52 KSP Residual norm 1.915823231868e-06 53 KSP Residual norm 1.567616029765e-06 54 KSP Residual norm 1.430359292474e-06 55 KSP Residual norm 1.389146056863e-06 56 KSP Residual norm 1.222334222011e-06 57 KSP Residual norm 1.096157570403e-06 58 KSP Residual norm 9.717051709099e-07 59 KSP Residual norm 8.175803053283e-07 60 KSP Residual norm 7.175769704252e-07 61 KSP Residual norm 6.226268138772e-07 62 KSP Residual norm 5.739329779006e-07 63 KSP Residual norm 5.420049760341e-07 64 KSP Residual norm 5.166886898884e-07 65 KSP Residual norm 4.617547348970e-07 66 KSP Residual norm 3.659692698579e-07 67 KSP Residual norm 2.835236244727e-07 68 KSP Residual norm 2.399596906063e-07 69 KSP Residual norm 2.041867739512e-07 70 KSP Residual norm 1.631059225505e-07 71 KSP Residual norm 1.306121888779e-07 72 KSP Residual norm 1.121152431501e-07 73 KSP Residual norm 1.005781454747e-07 74 KSP Residual norm 9.569674217905e-08 75 KSP Residual norm 9.285958701444e-08 76 KSP Residual norm 8.342188701241e-08 77 KSP Residual norm 6.515847445233e-08 78 KSP Residual norm 5.257140633432e-08 79 KSP Residual norm 4.674491135263e-08 80 KSP Residual norm 3.842818899627e-08 81 KSP Residual norm 3.467828976262e-08 34 KSP preconditioned resid norm 2.929463537058e+04 true resid norm 6.741363970494e+16 ||r(i)||/||b|| 3.839048498195e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.589139700298e-03 1 KSP Residual norm 9.983027718487e-04 2 KSP Residual norm 7.007204205488e-04 3 KSP Residual norm 1.225224334150e-03 4 KSP Residual norm 1.359268313499e-03 5 KSP Residual norm 1.151010144169e-03 6 KSP Residual norm 1.314584298993e-03 7 KSP Residual norm 9.906027188392e-04 8 KSP Residual norm 5.557936682819e-04 9 KSP Residual norm 4.491607146994e-04 10 KSP Residual norm 3.567735763910e-04 11 KSP Residual norm 2.794890581112e-04 12 KSP Residual norm 2.755508236802e-04 13 KSP Residual norm 2.190681644481e-04 14 KSP Residual norm 1.829754524297e-04 15 KSP Residual norm 1.673165003873e-04 16 KSP Residual norm 1.605229614111e-04 17 KSP Residual norm 1.496662818939e-04 18 KSP Residual norm 1.359863660043e-04 19 KSP Residual norm 1.274587727249e-04 20 KSP Residual norm 1.094541880668e-04 21 KSP Residual norm 8.730828177944e-05 22 KSP Residual norm 7.070154511965e-05 23 KSP Residual norm 5.962533554374e-05 24 KSP Residual norm 4.979028126356e-05 25 KSP Residual norm 4.414515735896e-05 26 KSP Residual norm 4.030419076830e-05 27 KSP Residual norm 3.799196089126e-05 28 KSP Residual norm 3.559352262423e-05 29 KSP Residual norm 3.157738649482e-05 30 KSP Residual norm 2.796533992761e-05 31 KSP Residual norm 2.662187181728e-05 32 KSP Residual norm 2.285914119979e-05 33 KSP Residual norm 2.050372960136e-05 34 KSP Residual norm 1.877713021908e-05 35 KSP Residual norm 1.546143623532e-05 36 KSP Residual norm 1.230947537342e-05 37 KSP Residual norm 1.047004633844e-05 38 KSP Residual norm 1.000630157756e-05 39 KSP Residual norm 1.031209247156e-05 40 KSP Residual norm 9.324091175017e-06 41 KSP Residual norm 7.856465251838e-06 42 KSP Residual norm 7.267846769612e-06 43 KSP Residual norm 6.629384892986e-06 44 KSP Residual norm 5.940062001487e-06 45 KSP Residual norm 4.866405176769e-06 46 KSP Residual norm 4.109903372555e-06 47 KSP Residual norm 3.709492951897e-06 48 KSP Residual norm 3.684341808762e-06 49 KSP Residual norm 3.634997061392e-06 50 KSP Residual norm 3.324597818681e-06 51 KSP Residual norm 2.871224282297e-06 52 KSP Residual norm 2.461711329504e-06 53 KSP Residual norm 2.101038752901e-06 54 KSP Residual norm 2.043265823365e-06 55 KSP Residual norm 1.933269066075e-06 56 KSP Residual norm 1.718206916819e-06 57 KSP Residual norm 1.546637466467e-06 58 KSP Residual norm 1.283708430958e-06 59 KSP Residual norm 9.179091529614e-07 60 KSP Residual norm 7.173003333839e-07 61 KSP Residual norm 5.753656753846e-07 62 KSP Residual norm 4.602861709338e-07 63 KSP Residual norm 3.506801284338e-07 64 KSP Residual norm 2.928185807349e-07 65 KSP Residual norm 2.453094164898e-07 66 KSP Residual norm 1.946637405251e-07 67 KSP Residual norm 1.526312961529e-07 68 KSP Residual norm 1.293714316702e-07 69 KSP Residual norm 1.095362693575e-07 70 KSP Residual norm 8.860329645016e-08 71 KSP Residual norm 7.376945042652e-08 72 KSP Residual norm 6.788874756726e-08 73 KSP Residual norm 6.169937240609e-08 74 KSP Residual norm 5.225979194791e-08 75 KSP Residual norm 4.488850373250e-08 76 KSP Residual norm 3.770217755811e-08 77 KSP Residual norm 2.894005449234e-08 78 KSP Residual norm 2.220190849960e-08 35 KSP preconditioned resid norm 2.807666085858e+04 true resid norm 6.829390007285e+16 ||r(i)||/||b|| 3.889177259351e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.098662399116e-03 1 KSP Residual norm 1.180547810119e-03 2 KSP Residual norm 7.436704036245e-04 3 KSP Residual norm 9.930700090533e-04 4 KSP Residual norm 1.150959993373e-03 5 KSP Residual norm 9.673225003264e-04 6 KSP Residual norm 1.065935339232e-03 7 KSP Residual norm 8.308041540495e-04 8 KSP Residual norm 6.158700416638e-04 9 KSP Residual norm 5.407803669533e-04 10 KSP Residual norm 3.993953740973e-04 11 KSP Residual norm 3.111767806300e-04 12 KSP Residual norm 3.158121400737e-04 13 KSP Residual norm 2.917974185044e-04 14 KSP Residual norm 2.618667748065e-04 15 KSP Residual norm 2.299670955265e-04 16 KSP Residual norm 1.996762500852e-04 17 KSP Residual norm 1.597174224603e-04 18 KSP Residual norm 1.279315695407e-04 19 KSP Residual norm 1.188389407269e-04 20 KSP Residual norm 1.166072840864e-04 21 KSP Residual norm 1.002838515736e-04 22 KSP Residual norm 7.894440720864e-05 23 KSP Residual norm 6.525893622434e-05 24 KSP Residual norm 5.504880153731e-05 25 KSP Residual norm 5.322585078355e-05 26 KSP Residual norm 4.878772351575e-05 27 KSP Residual norm 4.321419437546e-05 28 KSP Residual norm 3.715287862839e-05 29 KSP Residual norm 3.121464511085e-05 30 KSP Residual norm 2.739541677709e-05 31 KSP Residual norm 2.476287838252e-05 32 KSP Residual norm 2.011894148168e-05 33 KSP Residual norm 1.774369907698e-05 34 KSP Residual norm 1.609978187105e-05 35 KSP Residual norm 1.382882493105e-05 36 KSP Residual norm 1.175604056930e-05 37 KSP Residual norm 1.099471731396e-05 38 KSP Residual norm 1.137546827360e-05 39 KSP Residual norm 1.078631905743e-05 40 KSP Residual norm 9.016333412395e-06 41 KSP Residual norm 7.240946919130e-06 42 KSP Residual norm 6.666750884017e-06 43 KSP Residual norm 6.123543105095e-06 44 KSP Residual norm 6.004987237460e-06 45 KSP Residual norm 5.911099981011e-06 46 KSP Residual norm 5.226322369404e-06 47 KSP Residual norm 4.569614713640e-06 48 KSP Residual norm 4.070573611138e-06 49 KSP Residual norm 3.542094674392e-06 50 KSP Residual norm 3.022366098988e-06 51 KSP Residual norm 2.678088384944e-06 52 KSP Residual norm 2.392334411702e-06 53 KSP Residual norm 2.094546520202e-06 54 KSP Residual norm 1.882135762528e-06 55 KSP Residual norm 1.674145655792e-06 56 KSP Residual norm 1.422304233988e-06 57 KSP Residual norm 1.299727387385e-06 58 KSP Residual norm 1.222209535345e-06 59 KSP Residual norm 1.140815252444e-06 60 KSP Residual norm 1.025476070598e-06 61 KSP Residual norm 8.887532497863e-07 62 KSP Residual norm 7.340876430948e-07 63 KSP Residual norm 6.261403311382e-07 64 KSP Residual norm 5.531483214585e-07 65 KSP Residual norm 4.730880210327e-07 66 KSP Residual norm 3.793560254514e-07 67 KSP Residual norm 2.881108070948e-07 68 KSP Residual norm 2.303591779009e-07 69 KSP Residual norm 2.060969773595e-07 70 KSP Residual norm 1.760921569719e-07 71 KSP Residual norm 1.724202858344e-07 72 KSP Residual norm 1.648948025189e-07 73 KSP Residual norm 1.410951065221e-07 74 KSP Residual norm 1.126471720254e-07 75 KSP Residual norm 1.020990935966e-07 76 KSP Residual norm 9.646852948285e-08 77 KSP Residual norm 8.294022486813e-08 78 KSP Residual norm 6.396943520277e-08 79 KSP Residual norm 4.993732239955e-08 80 KSP Residual norm 4.342332192073e-08 81 KSP Residual norm 4.044956810083e-08 82 KSP Residual norm 3.478198886369e-08 83 KSP Residual norm 2.494949872366e-08 36 KSP preconditioned resid norm 2.704691824793e+04 true resid norm 6.925862867453e+16 ||r(i)||/||b|| 3.944116288095e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.108401652889e-03 1 KSP Residual norm 1.558686954017e-03 2 KSP Residual norm 8.971126816303e-04 3 KSP Residual norm 1.057367938200e-03 4 KSP Residual norm 1.399800497171e-03 5 KSP Residual norm 1.170317714523e-03 6 KSP Residual norm 1.262867794196e-03 7 KSP Residual norm 1.094444816447e-03 8 KSP Residual norm 6.607770391877e-04 9 KSP Residual norm 5.809202363281e-04 10 KSP Residual norm 5.098379368017e-04 11 KSP Residual norm 3.853970598283e-04 12 KSP Residual norm 3.550528841499e-04 13 KSP Residual norm 2.678250361276e-04 14 KSP Residual norm 1.992474091741e-04 15 KSP Residual norm 1.585196677539e-04 16 KSP Residual norm 1.413610604272e-04 17 KSP Residual norm 1.396173588980e-04 18 KSP Residual norm 1.320710568247e-04 19 KSP Residual norm 1.314864457738e-04 20 KSP Residual norm 1.316300114140e-04 21 KSP Residual norm 1.149778459873e-04 22 KSP Residual norm 9.272519564161e-05 23 KSP Residual norm 7.555776782060e-05 24 KSP Residual norm 6.327403158519e-05 25 KSP Residual norm 5.637755173175e-05 26 KSP Residual norm 5.240568823100e-05 27 KSP Residual norm 4.828961568493e-05 28 KSP Residual norm 4.031619101225e-05 29 KSP Residual norm 3.248286666178e-05 30 KSP Residual norm 2.990088184326e-05 31 KSP Residual norm 2.974469609612e-05 32 KSP Residual norm 2.713156428506e-05 33 KSP Residual norm 2.463204444812e-05 34 KSP Residual norm 2.185469452404e-05 35 KSP Residual norm 1.901166412802e-05 36 KSP Residual norm 1.667186961890e-05 37 KSP Residual norm 1.367461489296e-05 38 KSP Residual norm 1.266748964257e-05 39 KSP Residual norm 1.197367175093e-05 40 KSP Residual norm 1.039944114118e-05 41 KSP Residual norm 8.404657078122e-06 42 KSP Residual norm 7.150701407321e-06 43 KSP Residual norm 6.032530367428e-06 44 KSP Residual norm 5.439436120565e-06 45 KSP Residual norm 5.003886250907e-06 46 KSP Residual norm 4.172386795958e-06 47 KSP Residual norm 3.720793813426e-06 48 KSP Residual norm 3.834769373591e-06 49 KSP Residual norm 3.848491703341e-06 50 KSP Residual norm 3.710364079224e-06 51 KSP Residual norm 3.645318788266e-06 52 KSP Residual norm 3.386484957173e-06 53 KSP Residual norm 2.866833072668e-06 54 KSP Residual norm 2.421079647777e-06 55 KSP Residual norm 2.150517237402e-06 56 KSP Residual norm 1.793458801437e-06 57 KSP Residual norm 1.489835872192e-06 58 KSP Residual norm 1.205834005242e-06 59 KSP Residual norm 8.941326022839e-07 60 KSP Residual norm 6.944210427043e-07 61 KSP Residual norm 5.434662369430e-07 62 KSP Residual norm 4.354458673228e-07 63 KSP Residual norm 3.531039219787e-07 64 KSP Residual norm 3.092506626382e-07 65 KSP Residual norm 2.574472375606e-07 66 KSP Residual norm 2.017550955167e-07 67 KSP Residual norm 1.652123030157e-07 68 KSP Residual norm 1.381341422540e-07 69 KSP Residual norm 1.078009310583e-07 70 KSP Residual norm 8.110227070400e-08 71 KSP Residual norm 6.524416703523e-08 72 KSP Residual norm 5.483520579305e-08 73 KSP Residual norm 4.696525450788e-08 74 KSP Residual norm 3.838140100999e-08 37 KSP preconditioned resid norm 2.615281748771e+04 true resid norm 7.028336818809e+16 ||r(i)||/||b|| 4.002472797368e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.857560560081e-03 1 KSP Residual norm 1.104770959879e-03 2 KSP Residual norm 7.662633853876e-04 3 KSP Residual norm 1.115532985323e-03 4 KSP Residual norm 1.095273469154e-03 5 KSP Residual norm 8.888003970555e-04 6 KSP Residual norm 1.015926068748e-03 7 KSP Residual norm 7.278166395353e-04 8 KSP Residual norm 5.113633663251e-04 9 KSP Residual norm 4.457152106190e-04 10 KSP Residual norm 3.412574268163e-04 11 KSP Residual norm 2.675765106861e-04 12 KSP Residual norm 2.528170499816e-04 13 KSP Residual norm 2.001673807049e-04 14 KSP Residual norm 1.675234696539e-04 15 KSP Residual norm 1.477392742362e-04 16 KSP Residual norm 1.412322202718e-04 17 KSP Residual norm 1.284334569411e-04 18 KSP Residual norm 1.069470659362e-04 19 KSP Residual norm 9.576615902980e-05 20 KSP Residual norm 9.220340363418e-05 21 KSP Residual norm 7.977547307586e-05 22 KSP Residual norm 6.066604820007e-05 23 KSP Residual norm 4.573346593964e-05 24 KSP Residual norm 3.882953490518e-05 25 KSP Residual norm 3.454137606841e-05 26 KSP Residual norm 3.108810158657e-05 27 KSP Residual norm 2.970589301021e-05 28 KSP Residual norm 2.822496124694e-05 29 KSP Residual norm 2.430865198627e-05 30 KSP Residual norm 2.119627239434e-05 31 KSP Residual norm 1.925528920007e-05 32 KSP Residual norm 1.647278029984e-05 33 KSP Residual norm 1.536893588300e-05 34 KSP Residual norm 1.344143411297e-05 35 KSP Residual norm 1.082326808862e-05 36 KSP Residual norm 8.741951217121e-06 37 KSP Residual norm 7.367445273265e-06 38 KSP Residual norm 7.293542037939e-06 39 KSP Residual norm 7.637502867865e-06 40 KSP Residual norm 7.233085243491e-06 41 KSP Residual norm 6.062750695567e-06 42 KSP Residual norm 5.359944185252e-06 43 KSP Residual norm 4.836168198110e-06 44 KSP Residual norm 4.573736256242e-06 45 KSP Residual norm 4.141315180545e-06 46 KSP Residual norm 3.270700347947e-06 47 KSP Residual norm 2.687983511988e-06 48 KSP Residual norm 2.489653819132e-06 49 KSP Residual norm 2.148799445605e-06 50 KSP Residual norm 1.727205813320e-06 51 KSP Residual norm 1.516309971034e-06 52 KSP Residual norm 1.381133033019e-06 53 KSP Residual norm 1.266130156460e-06 54 KSP Residual norm 1.236586086348e-06 55 KSP Residual norm 1.195719052432e-06 56 KSP Residual norm 9.712831224611e-07 57 KSP Residual norm 8.400776886181e-07 58 KSP Residual norm 7.915763772898e-07 59 KSP Residual norm 7.124033682623e-07 60 KSP Residual norm 6.650462593872e-07 61 KSP Residual norm 6.314529869288e-07 62 KSP Residual norm 5.672033354059e-07 63 KSP Residual norm 5.015958118280e-07 64 KSP Residual norm 4.405544707050e-07 65 KSP Residual norm 3.965605576749e-07 66 KSP Residual norm 3.363776376485e-07 67 KSP Residual norm 2.679789904113e-07 68 KSP Residual norm 2.430799808722e-07 69 KSP Residual norm 2.190582094554e-07 70 KSP Residual norm 1.894083000238e-07 71 KSP Residual norm 1.670692618900e-07 72 KSP Residual norm 1.741406381315e-07 73 KSP Residual norm 1.687684374006e-07 74 KSP Residual norm 1.471750477195e-07 75 KSP Residual norm 1.252336783031e-07 76 KSP Residual norm 1.083570099164e-07 77 KSP Residual norm 8.503665420745e-08 78 KSP Residual norm 6.677870581341e-08 79 KSP Residual norm 5.609795421669e-08 80 KSP Residual norm 4.514482703256e-08 81 KSP Residual norm 3.999462867830e-08 82 KSP Residual norm 3.616652411385e-08 83 KSP Residual norm 2.973110998452e-08 84 KSP Residual norm 2.539814341891e-08 38 KSP preconditioned resid norm 2.541030501437e+04 true resid norm 7.136706613174e+16 ||r(i)||/||b|| 4.064186850804e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.686148294626e-03 1 KSP Residual norm 1.345532220632e-03 2 KSP Residual norm 6.833658915971e-04 3 KSP Residual norm 8.850434281419e-04 4 KSP Residual norm 1.108232958902e-03 5 KSP Residual norm 8.277792446713e-04 6 KSP Residual norm 8.042714435278e-04 7 KSP Residual norm 6.131289088356e-04 8 KSP Residual norm 3.666900651662e-04 9 KSP Residual norm 3.204191828695e-04 10 KSP Residual norm 2.663663109431e-04 11 KSP Residual norm 2.080551324219e-04 12 KSP Residual norm 2.114945974589e-04 13 KSP Residual norm 1.707454155199e-04 14 KSP Residual norm 1.344706069423e-04 15 KSP Residual norm 1.139425554226e-04 16 KSP Residual norm 9.817678376105e-05 17 KSP Residual norm 8.727400604560e-05 18 KSP Residual norm 7.635506906036e-05 19 KSP Residual norm 6.290025387422e-05 20 KSP Residual norm 5.488244385311e-05 21 KSP Residual norm 4.513638929966e-05 22 KSP Residual norm 3.773083539899e-05 23 KSP Residual norm 3.293679796727e-05 24 KSP Residual norm 2.798587934912e-05 25 KSP Residual norm 2.521377074225e-05 26 KSP Residual norm 2.348846990094e-05 27 KSP Residual norm 2.212262645163e-05 28 KSP Residual norm 1.890876233764e-05 29 KSP Residual norm 1.496684565614e-05 30 KSP Residual norm 1.238950229664e-05 31 KSP Residual norm 1.137429905982e-05 32 KSP Residual norm 9.799017814421e-06 33 KSP Residual norm 8.307878475766e-06 34 KSP Residual norm 7.340899704929e-06 35 KSP Residual norm 6.677689108296e-06 36 KSP Residual norm 5.679332575601e-06 37 KSP Residual norm 4.737834499624e-06 38 KSP Residual norm 4.379505883757e-06 39 KSP Residual norm 4.284330021289e-06 40 KSP Residual norm 3.890747717197e-06 41 KSP Residual norm 3.306036088807e-06 42 KSP Residual norm 2.996607191238e-06 43 KSP Residual norm 2.593561320509e-06 44 KSP Residual norm 2.335203258803e-06 45 KSP Residual norm 2.106120581207e-06 46 KSP Residual norm 1.759927495153e-06 47 KSP Residual norm 1.572705467268e-06 48 KSP Residual norm 1.687547116646e-06 49 KSP Residual norm 1.719386795049e-06 50 KSP Residual norm 1.544998385804e-06 51 KSP Residual norm 1.333446714173e-06 52 KSP Residual norm 1.094811287372e-06 53 KSP Residual norm 9.063892297805e-07 54 KSP Residual norm 8.096564319340e-07 55 KSP Residual norm 7.123907398728e-07 56 KSP Residual norm 5.950651432794e-07 57 KSP Residual norm 5.554692830084e-07 58 KSP Residual norm 5.730539381964e-07 59 KSP Residual norm 5.250470947769e-07 60 KSP Residual norm 4.622863074135e-07 61 KSP Residual norm 3.983957170675e-07 62 KSP Residual norm 3.471127522064e-07 63 KSP Residual norm 2.963892116118e-07 64 KSP Residual norm 2.787883709356e-07 65 KSP Residual norm 2.493539929251e-07 66 KSP Residual norm 2.087433845302e-07 67 KSP Residual norm 1.639409560074e-07 68 KSP Residual norm 1.252802573172e-07 69 KSP Residual norm 1.040374486829e-07 70 KSP Residual norm 8.889574737155e-08 71 KSP Residual norm 7.352962046160e-08 72 KSP Residual norm 6.415826766357e-08 73 KSP Residual norm 5.526206322940e-08 74 KSP Residual norm 4.356753510787e-08 75 KSP Residual norm 3.933407322765e-08 76 KSP Residual norm 3.906055827691e-08 77 KSP Residual norm 3.753675323316e-08 78 KSP Residual norm 3.516027149416e-08 39 KSP preconditioned resid norm 2.376113533031e+04 true resid norm 7.421817678319e+16 ||r(i)||/||b|| 4.226550908175e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.448485343978e-03 1 KSP Residual norm 1.877712348206e-03 2 KSP Residual norm 7.362219243307e-04 3 KSP Residual norm 6.265769132019e-04 4 KSP Residual norm 7.052791781458e-04 5 KSP Residual norm 4.973103509996e-04 6 KSP Residual norm 5.063675386823e-04 7 KSP Residual norm 5.049992167159e-04 8 KSP Residual norm 3.649104901125e-04 9 KSP Residual norm 3.291398447245e-04 10 KSP Residual norm 2.886394241620e-04 11 KSP Residual norm 2.160852349208e-04 12 KSP Residual norm 2.082404816652e-04 13 KSP Residual norm 1.738379583037e-04 14 KSP Residual norm 1.390340353887e-04 15 KSP Residual norm 1.152863288511e-04 16 KSP Residual norm 9.628953279095e-05 17 KSP Residual norm 8.104871654253e-05 18 KSP Residual norm 6.458748504441e-05 19 KSP Residual norm 5.436750017737e-05 20 KSP Residual norm 4.960330291745e-05 21 KSP Residual norm 4.423910886965e-05 22 KSP Residual norm 3.751608231988e-05 23 KSP Residual norm 3.097507312067e-05 24 KSP Residual norm 2.654398710180e-05 25 KSP Residual norm 2.410632090359e-05 26 KSP Residual norm 2.262168986884e-05 27 KSP Residual norm 2.127761613142e-05 28 KSP Residual norm 1.878551205942e-05 29 KSP Residual norm 1.588904694442e-05 30 KSP Residual norm 1.352796741991e-05 31 KSP Residual norm 1.284549676333e-05 32 KSP Residual norm 1.189560781697e-05 33 KSP Residual norm 1.096187226724e-05 34 KSP Residual norm 1.050800727191e-05 35 KSP Residual norm 9.429216902562e-06 36 KSP Residual norm 8.010134659411e-06 37 KSP Residual norm 6.562086453106e-06 38 KSP Residual norm 6.135151491360e-06 39 KSP Residual norm 6.304572091082e-06 40 KSP Residual norm 5.864883908410e-06 41 KSP Residual norm 4.871422156216e-06 42 KSP Residual norm 4.521521599310e-06 43 KSP Residual norm 3.901020257269e-06 44 KSP Residual norm 3.342878888644e-06 45 KSP Residual norm 2.901185387312e-06 46 KSP Residual norm 2.370929031289e-06 47 KSP Residual norm 2.057534349427e-06 48 KSP Residual norm 2.065635150279e-06 49 KSP Residual norm 2.002078270314e-06 50 KSP Residual norm 1.737075879530e-06 51 KSP Residual norm 1.621980201008e-06 52 KSP Residual norm 1.499306959800e-06 53 KSP Residual norm 1.251524020016e-06 54 KSP Residual norm 1.087278539075e-06 55 KSP Residual norm 1.004359438887e-06 56 KSP Residual norm 8.040096448021e-07 57 KSP Residual norm 6.496437761905e-07 58 KSP Residual norm 5.864154948851e-07 59 KSP Residual norm 4.941356700083e-07 60 KSP Residual norm 4.177694065962e-07 61 KSP Residual norm 3.355073217862e-07 62 KSP Residual norm 2.726079569897e-07 63 KSP Residual norm 2.204324775599e-07 64 KSP Residual norm 2.062625086352e-07 65 KSP Residual norm 1.965139460809e-07 66 KSP Residual norm 1.762017948813e-07 67 KSP Residual norm 1.485340874688e-07 68 KSP Residual norm 1.183767999852e-07 69 KSP Residual norm 1.015263370843e-07 70 KSP Residual norm 8.579107873501e-08 71 KSP Residual norm 7.300894666406e-08 72 KSP Residual norm 6.601708822975e-08 73 KSP Residual norm 6.035259302725e-08 74 KSP Residual norm 5.102759488665e-08 40 KSP preconditioned resid norm 2.266470152499e+04 true resid norm 7.617450569713e+16 ||r(i)||/||b|| 4.337959246487e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 4.894568495092e-03 1 KSP Residual norm 1.730141626821e-03 2 KSP Residual norm 7.321237484290e-04 3 KSP Residual norm 7.477820481054e-04 4 KSP Residual norm 8.675933051419e-04 5 KSP Residual norm 6.353133919162e-04 6 KSP Residual norm 7.216165540451e-04 7 KSP Residual norm 6.852258520951e-04 8 KSP Residual norm 4.634745820735e-04 9 KSP Residual norm 3.801324619125e-04 10 KSP Residual norm 2.893552023669e-04 11 KSP Residual norm 1.935192439807e-04 12 KSP Residual norm 1.776839577824e-04 13 KSP Residual norm 1.497245878292e-04 14 KSP Residual norm 1.278722324474e-04 15 KSP Residual norm 1.088897777225e-04 16 KSP Residual norm 9.195038416523e-05 17 KSP Residual norm 7.839274698735e-05 18 KSP Residual norm 6.361079279202e-05 19 KSP Residual norm 5.512071401534e-05 20 KSP Residual norm 5.358343883296e-05 21 KSP Residual norm 5.010757822471e-05 22 KSP Residual norm 4.467826573978e-05 23 KSP Residual norm 3.689462107862e-05 24 KSP Residual norm 3.042819913212e-05 25 KSP Residual norm 2.770721844164e-05 26 KSP Residual norm 2.560987357069e-05 27 KSP Residual norm 2.383241083705e-05 28 KSP Residual norm 2.130615728769e-05 29 KSP Residual norm 1.864709534240e-05 30 KSP Residual norm 1.691734844476e-05 31 KSP Residual norm 1.663343086351e-05 32 KSP Residual norm 1.384476692638e-05 33 KSP Residual norm 1.096664875448e-05 34 KSP Residual norm 9.041538100897e-06 35 KSP Residual norm 8.035116378152e-06 36 KSP Residual norm 7.487358771470e-06 37 KSP Residual norm 6.922286567347e-06 38 KSP Residual norm 6.683978271630e-06 39 KSP Residual norm 6.571513786902e-06 40 KSP Residual norm 6.392196001859e-06 41 KSP Residual norm 5.885985199009e-06 42 KSP Residual norm 5.342015656913e-06 43 KSP Residual norm 4.498991783463e-06 44 KSP Residual norm 3.932284705947e-06 45 KSP Residual norm 3.573012327624e-06 46 KSP Residual norm 2.812525717644e-06 47 KSP Residual norm 2.357823053723e-06 48 KSP Residual norm 2.321096234255e-06 49 KSP Residual norm 2.254448087145e-06 50 KSP Residual norm 1.900644375418e-06 51 KSP Residual norm 1.687852279266e-06 52 KSP Residual norm 1.548995817152e-06 53 KSP Residual norm 1.295446188831e-06 54 KSP Residual norm 1.042834938211e-06 55 KSP Residual norm 8.481533694890e-07 56 KSP Residual norm 6.756185007410e-07 57 KSP Residual norm 5.746887988703e-07 58 KSP Residual norm 5.526723766353e-07 59 KSP Residual norm 5.008099478074e-07 60 KSP Residual norm 4.465353560522e-07 61 KSP Residual norm 3.927688485383e-07 62 KSP Residual norm 3.420693756600e-07 63 KSP Residual norm 2.673713349059e-07 64 KSP Residual norm 2.400243253714e-07 65 KSP Residual norm 2.165205347772e-07 66 KSP Residual norm 1.886361334908e-07 67 KSP Residual norm 1.600299088365e-07 68 KSP Residual norm 1.230937818800e-07 69 KSP Residual norm 9.728878010846e-08 70 KSP Residual norm 7.739080437147e-08 71 KSP Residual norm 6.400565474022e-08 72 KSP Residual norm 5.774380293199e-08 73 KSP Residual norm 5.262337181248e-08 74 KSP Residual norm 4.380556816512e-08 41 KSP preconditioned resid norm 2.105290133097e+04 true resid norm 7.939938805261e+16 ||r(i)||/||b|| 4.521608724809e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 6.841749150660e-03 1 KSP Residual norm 2.363150428293e-03 2 KSP Residual norm 8.844816246439e-04 3 KSP Residual norm 7.399280175224e-04 4 KSP Residual norm 8.152875281328e-04 5 KSP Residual norm 5.705714452426e-04 6 KSP Residual norm 5.545991807728e-04 7 KSP Residual norm 5.568312728534e-04 8 KSP Residual norm 3.984359252430e-04 9 KSP Residual norm 3.518179049850e-04 10 KSP Residual norm 2.865683517061e-04 11 KSP Residual norm 1.946304172583e-04 12 KSP Residual norm 1.695859517312e-04 13 KSP Residual norm 1.509675120733e-04 14 KSP Residual norm 1.380468936414e-04 15 KSP Residual norm 1.259986345381e-04 16 KSP Residual norm 1.083130710512e-04 17 KSP Residual norm 8.715524198467e-05 18 KSP Residual norm 6.396011558164e-05 19 KSP Residual norm 4.942927362900e-05 20 KSP Residual norm 4.505356838945e-05 21 KSP Residual norm 4.308999656278e-05 22 KSP Residual norm 3.715629535318e-05 23 KSP Residual norm 3.138894546386e-05 24 KSP Residual norm 2.535823915175e-05 25 KSP Residual norm 2.287812607685e-05 26 KSP Residual norm 2.075648186285e-05 27 KSP Residual norm 1.871789703307e-05 28 KSP Residual norm 1.570953441385e-05 29 KSP Residual norm 1.365960268182e-05 30 KSP Residual norm 1.228732972138e-05 31 KSP Residual norm 1.234470804237e-05 32 KSP Residual norm 1.087278841960e-05 33 KSP Residual norm 8.844936092204e-06 34 KSP Residual norm 7.120485131028e-06 35 KSP Residual norm 5.976178118320e-06 36 KSP Residual norm 5.047847474608e-06 37 KSP Residual norm 4.379867890795e-06 38 KSP Residual norm 4.058224319163e-06 39 KSP Residual norm 4.109843619958e-06 40 KSP Residual norm 4.033356075586e-06 41 KSP Residual norm 3.594404957626e-06 42 KSP Residual norm 3.275464848295e-06 43 KSP Residual norm 2.817569724154e-06 44 KSP Residual norm 2.458220178731e-06 45 KSP Residual norm 2.361313029935e-06 46 KSP Residual norm 2.068544301005e-06 47 KSP Residual norm 1.847217535112e-06 48 KSP Residual norm 1.759979870877e-06 49 KSP Residual norm 1.673099545886e-06 50 KSP Residual norm 1.431318427249e-06 51 KSP Residual norm 1.316592746098e-06 52 KSP Residual norm 1.275241058966e-06 53 KSP Residual norm 1.116010261181e-06 54 KSP Residual norm 9.526046844932e-07 55 KSP Residual norm 8.963645445155e-07 56 KSP Residual norm 7.578147693963e-07 57 KSP Residual norm 6.304157989137e-07 58 KSP Residual norm 5.459278326444e-07 59 KSP Residual norm 4.811425356981e-07 60 KSP Residual norm 4.310702242542e-07 61 KSP Residual norm 3.630664598016e-07 62 KSP Residual norm 2.928688468898e-07 63 KSP Residual norm 2.366372451740e-07 64 KSP Residual norm 2.104069142208e-07 65 KSP Residual norm 1.944937696921e-07 66 KSP Residual norm 1.728402991375e-07 67 KSP Residual norm 1.568978663183e-07 68 KSP Residual norm 1.361617646301e-07 69 KSP Residual norm 1.183901202293e-07 70 KSP Residual norm 9.653275466982e-08 71 KSP Residual norm 7.510865937253e-08 72 KSP Residual norm 6.081414656161e-08 42 KSP preconditioned resid norm 1.957509013646e+04 true resid norm 8.268430184905e+16 ||r(i)||/||b|| 4.708676852745e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.649614691567e-03 1 KSP Residual norm 2.655921634485e-03 2 KSP Residual norm 1.025468401791e-03 3 KSP Residual norm 8.333481282778e-04 4 KSP Residual norm 8.830350770686e-04 5 KSP Residual norm 6.147874518018e-04 6 KSP Residual norm 5.650671931018e-04 7 KSP Residual norm 5.486362169857e-04 8 KSP Residual norm 4.135381544613e-04 9 KSP Residual norm 3.714173034486e-04 10 KSP Residual norm 2.941708580862e-04 11 KSP Residual norm 2.027756832023e-04 12 KSP Residual norm 1.889580376599e-04 13 KSP Residual norm 1.601802062105e-04 14 KSP Residual norm 1.301604007311e-04 15 KSP Residual norm 1.126774027860e-04 16 KSP Residual norm 9.488363031767e-05 17 KSP Residual norm 7.834050311818e-05 18 KSP Residual norm 6.275094909709e-05 19 KSP Residual norm 5.555941215709e-05 20 KSP Residual norm 5.712672361791e-05 21 KSP Residual norm 5.686853276368e-05 22 KSP Residual norm 5.142459357070e-05 23 KSP Residual norm 4.366557473552e-05 24 KSP Residual norm 3.906910599964e-05 25 KSP Residual norm 3.748264814786e-05 26 KSP Residual norm 3.517382806903e-05 27 KSP Residual norm 3.181529121120e-05 28 KSP Residual norm 2.700856766821e-05 29 KSP Residual norm 2.308954582075e-05 30 KSP Residual norm 2.153599585848e-05 31 KSP Residual norm 2.160546126970e-05 32 KSP Residual norm 1.943580581710e-05 33 KSP Residual norm 1.703535852862e-05 34 KSP Residual norm 1.526636492585e-05 35 KSP Residual norm 1.354904880899e-05 36 KSP Residual norm 1.152431331920e-05 37 KSP Residual norm 9.735247027132e-06 38 KSP Residual norm 8.772392325721e-06 39 KSP Residual norm 8.276278668864e-06 40 KSP Residual norm 7.596661624973e-06 41 KSP Residual norm 6.660893665535e-06 42 KSP Residual norm 6.236759123373e-06 43 KSP Residual norm 5.287754883860e-06 44 KSP Residual norm 4.360171131362e-06 45 KSP Residual norm 3.634321358329e-06 46 KSP Residual norm 2.688742933622e-06 47 KSP Residual norm 2.072035396840e-06 48 KSP Residual norm 1.865985960310e-06 49 KSP Residual norm 1.698149417692e-06 50 KSP Residual norm 1.463358701737e-06 51 KSP Residual norm 1.328707597483e-06 52 KSP Residual norm 1.267224015116e-06 53 KSP Residual norm 1.089330038047e-06 54 KSP Residual norm 9.620801983557e-07 55 KSP Residual norm 8.756610241621e-07 56 KSP Residual norm 7.263046369919e-07 57 KSP Residual norm 5.956130503293e-07 58 KSP Residual norm 5.333969013251e-07 59 KSP Residual norm 4.828929062570e-07 60 KSP Residual norm 4.315887747104e-07 61 KSP Residual norm 3.586215658232e-07 62 KSP Residual norm 2.766908910091e-07 63 KSP Residual norm 2.196647300234e-07 64 KSP Residual norm 2.035240864530e-07 65 KSP Residual norm 1.958604191608e-07 66 KSP Residual norm 1.734196309683e-07 67 KSP Residual norm 1.506753401343e-07 68 KSP Residual norm 1.327210061708e-07 69 KSP Residual norm 1.190025995448e-07 70 KSP Residual norm 1.030255361311e-07 71 KSP Residual norm 8.636906819522e-08 72 KSP Residual norm 7.635632800147e-08 43 KSP preconditioned resid norm 1.834656977901e+04 true resid norm 8.563281376977e+16 ||r(i)||/||b|| 4.876587683708e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.945468330826e-03 1 KSP Residual norm 2.743230503785e-03 2 KSP Residual norm 1.025370266502e-03 3 KSP Residual norm 8.421479310026e-04 4 KSP Residual norm 9.064995869749e-04 5 KSP Residual norm 6.059268893194e-04 6 KSP Residual norm 5.842139628834e-04 7 KSP Residual norm 5.970750243976e-04 8 KSP Residual norm 4.435449958101e-04 9 KSP Residual norm 3.848288393754e-04 10 KSP Residual norm 3.022705887708e-04 11 KSP Residual norm 1.938018447489e-04 12 KSP Residual norm 1.628274811088e-04 13 KSP Residual norm 1.340146358043e-04 14 KSP Residual norm 1.138902061175e-04 15 KSP Residual norm 1.028285126872e-04 16 KSP Residual norm 8.995223394877e-05 17 KSP Residual norm 7.744319971273e-05 18 KSP Residual norm 6.248590678175e-05 19 KSP Residual norm 5.304165616064e-05 20 KSP Residual norm 5.345453716135e-05 21 KSP Residual norm 5.465024666066e-05 22 KSP Residual norm 5.092642329793e-05 23 KSP Residual norm 4.479307896396e-05 24 KSP Residual norm 3.862618956547e-05 25 KSP Residual norm 3.597311506456e-05 26 KSP Residual norm 3.297347388597e-05 27 KSP Residual norm 2.997406922532e-05 28 KSP Residual norm 2.584914519934e-05 29 KSP Residual norm 2.232555319789e-05 30 KSP Residual norm 1.898275402831e-05 31 KSP Residual norm 1.872239444191e-05 32 KSP Residual norm 1.716940343790e-05 33 KSP Residual norm 1.529771178742e-05 34 KSP Residual norm 1.312981904051e-05 35 KSP Residual norm 1.156591472825e-05 36 KSP Residual norm 1.016524168932e-05 37 KSP Residual norm 9.444977752200e-06 38 KSP Residual norm 9.046253105976e-06 39 KSP Residual norm 9.017750862843e-06 40 KSP Residual norm 8.504566358579e-06 41 KSP Residual norm 7.028099112108e-06 42 KSP Residual norm 6.129136112890e-06 43 KSP Residual norm 5.015873754027e-06 44 KSP Residual norm 4.118497518847e-06 45 KSP Residual norm 3.543665125623e-06 46 KSP Residual norm 2.750817019829e-06 47 KSP Residual norm 2.160128157912e-06 48 KSP Residual norm 1.861468667569e-06 49 KSP Residual norm 1.571563674113e-06 50 KSP Residual norm 1.292490587135e-06 51 KSP Residual norm 1.228608305736e-06 52 KSP Residual norm 1.218676155406e-06 53 KSP Residual norm 1.061788333870e-06 54 KSP Residual norm 8.963965528049e-07 55 KSP Residual norm 8.563929561929e-07 56 KSP Residual norm 7.194148267358e-07 57 KSP Residual norm 5.726184076211e-07 58 KSP Residual norm 4.938754885524e-07 59 KSP Residual norm 4.551459893873e-07 60 KSP Residual norm 4.497025569600e-07 61 KSP Residual norm 4.008373575411e-07 62 KSP Residual norm 3.247223498568e-07 63 KSP Residual norm 2.512433938424e-07 64 KSP Residual norm 2.039831245713e-07 65 KSP Residual norm 1.751591580972e-07 66 KSP Residual norm 1.539373591788e-07 67 KSP Residual norm 1.328495597157e-07 68 KSP Residual norm 1.105405388401e-07 69 KSP Residual norm 9.843392816202e-08 70 KSP Residual norm 8.652307832273e-08 71 KSP Residual norm 6.939616605567e-08 44 KSP preconditioned resid norm 1.673348104017e+04 true resid norm 8.997778410888e+16 ||r(i)||/||b|| 5.124023542803e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.068940190048e-02 1 KSP Residual norm 3.696226261142e-03 2 KSP Residual norm 1.365499441580e-03 3 KSP Residual norm 1.036803391587e-03 4 KSP Residual norm 1.063043512076e-03 5 KSP Residual norm 6.832245879458e-04 6 KSP Residual norm 5.688343876472e-04 7 KSP Residual norm 5.592497261418e-04 8 KSP Residual norm 4.118306049072e-04 9 KSP Residual norm 3.609255859944e-04 10 KSP Residual norm 2.875765835321e-04 11 KSP Residual norm 1.873537601536e-04 12 KSP Residual norm 1.680585023560e-04 13 KSP Residual norm 1.544529441896e-04 14 KSP Residual norm 1.368378046171e-04 15 KSP Residual norm 1.257726559882e-04 16 KSP Residual norm 1.117857134263e-04 17 KSP Residual norm 9.568960682874e-05 18 KSP Residual norm 7.517130278619e-05 19 KSP Residual norm 6.293296057179e-05 20 KSP Residual norm 6.317854600704e-05 21 KSP Residual norm 6.387343805195e-05 22 KSP Residual norm 6.094869667116e-05 23 KSP Residual norm 5.381497186561e-05 24 KSP Residual norm 4.668954303764e-05 25 KSP Residual norm 4.364914362555e-05 26 KSP Residual norm 4.026829236648e-05 27 KSP Residual norm 3.642498195651e-05 28 KSP Residual norm 3.218609574910e-05 29 KSP Residual norm 2.946702341324e-05 30 KSP Residual norm 2.738101831327e-05 31 KSP Residual norm 2.670531030981e-05 32 KSP Residual norm 2.284648832803e-05 33 KSP Residual norm 1.864392868907e-05 34 KSP Residual norm 1.559774662327e-05 35 KSP Residual norm 1.345859036667e-05 36 KSP Residual norm 1.132447688899e-05 37 KSP Residual norm 9.485766681601e-06 38 KSP Residual norm 9.022173096874e-06 39 KSP Residual norm 8.682472839293e-06 40 KSP Residual norm 7.847656785606e-06 41 KSP Residual norm 6.427838531119e-06 42 KSP Residual norm 5.410236913360e-06 43 KSP Residual norm 4.364958921999e-06 44 KSP Residual norm 3.794094458332e-06 45 KSP Residual norm 3.583990867507e-06 46 KSP Residual norm 3.037667233348e-06 47 KSP Residual norm 2.491681748670e-06 48 KSP Residual norm 2.142623860458e-06 49 KSP Residual norm 1.680106972358e-06 50 KSP Residual norm 1.294449947079e-06 51 KSP Residual norm 1.141842217116e-06 52 KSP Residual norm 1.057488908235e-06 53 KSP Residual norm 9.243193715263e-07 54 KSP Residual norm 8.768571106897e-07 55 KSP Residual norm 9.093688602513e-07 56 KSP Residual norm 8.082819565107e-07 57 KSP Residual norm 6.406098166720e-07 58 KSP Residual norm 5.443475292529e-07 59 KSP Residual norm 4.941574695216e-07 60 KSP Residual norm 4.717477106585e-07 61 KSP Residual norm 4.354051278597e-07 62 KSP Residual norm 3.757179860061e-07 63 KSP Residual norm 3.085232616450e-07 64 KSP Residual norm 2.636112561660e-07 65 KSP Residual norm 2.326530352069e-07 66 KSP Residual norm 1.972264881966e-07 67 KSP Residual norm 1.743024090012e-07 68 KSP Residual norm 1.628420778541e-07 69 KSP Residual norm 1.552845988308e-07 70 KSP Residual norm 1.438983865413e-07 71 KSP Residual norm 1.198978899178e-07 72 KSP Residual norm 1.059726521179e-07 45 KSP preconditioned resid norm 1.566228445630e+04 true resid norm 9.320890532567e+16 ||r(i)||/||b|| 5.308028309629e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.023035159946e-02 1 KSP Residual norm 3.536441251213e-03 2 KSP Residual norm 1.299726708917e-03 3 KSP Residual norm 9.428501552426e-04 4 KSP Residual norm 9.630344648589e-04 5 KSP Residual norm 6.368977362211e-04 6 KSP Residual norm 5.513053110152e-04 7 KSP Residual norm 5.617556710062e-04 8 KSP Residual norm 4.347941713256e-04 9 KSP Residual norm 3.906150283847e-04 10 KSP Residual norm 3.120084082186e-04 11 KSP Residual norm 1.984641078560e-04 12 KSP Residual norm 1.681185546211e-04 13 KSP Residual norm 1.371417088701e-04 14 KSP Residual norm 1.121381601931e-04 15 KSP Residual norm 1.004023393828e-04 16 KSP Residual norm 8.905858437048e-05 17 KSP Residual norm 7.916812527319e-05 18 KSP Residual norm 6.416002845146e-05 19 KSP Residual norm 5.123151421311e-05 20 KSP Residual norm 4.983864355666e-05 21 KSP Residual norm 5.093238155446e-05 22 KSP Residual norm 4.875813942182e-05 23 KSP Residual norm 4.380301171124e-05 24 KSP Residual norm 3.902866695035e-05 25 KSP Residual norm 3.602001554898e-05 26 KSP Residual norm 3.237738235880e-05 27 KSP Residual norm 2.883863304348e-05 28 KSP Residual norm 2.566135573405e-05 29 KSP Residual norm 2.339711398578e-05 30 KSP Residual norm 2.172070412002e-05 31 KSP Residual norm 2.215652395288e-05 32 KSP Residual norm 2.041810770762e-05 33 KSP Residual norm 1.840317638983e-05 34 KSP Residual norm 1.632981195663e-05 35 KSP Residual norm 1.411967595011e-05 36 KSP Residual norm 1.177600970864e-05 37 KSP Residual norm 9.485871252377e-06 38 KSP Residual norm 7.853507742643e-06 39 KSP Residual norm 7.386855929668e-06 40 KSP Residual norm 6.644163066161e-06 41 KSP Residual norm 5.256380649064e-06 42 KSP Residual norm 4.384025386803e-06 43 KSP Residual norm 3.552475448562e-06 44 KSP Residual norm 2.764945944119e-06 45 KSP Residual norm 2.471277442834e-06 46 KSP Residual norm 2.059282947255e-06 47 KSP Residual norm 1.680435974468e-06 48 KSP Residual norm 1.438952643236e-06 49 KSP Residual norm 1.302221630097e-06 50 KSP Residual norm 1.150403423476e-06 51 KSP Residual norm 1.059786968767e-06 52 KSP Residual norm 1.018262542155e-06 53 KSP Residual norm 8.896479556252e-07 54 KSP Residual norm 7.386690416481e-07 55 KSP Residual norm 6.614686577572e-07 56 KSP Residual norm 5.505979420336e-07 57 KSP Residual norm 4.300348202747e-07 58 KSP Residual norm 3.612676247250e-07 59 KSP Residual norm 3.102019883359e-07 60 KSP Residual norm 2.949675441403e-07 61 KSP Residual norm 2.705807583346e-07 62 KSP Residual norm 2.296994756951e-07 63 KSP Residual norm 1.715661988897e-07 64 KSP Residual norm 1.350853774156e-07 65 KSP Residual norm 1.150777370489e-07 66 KSP Residual norm 9.713973641981e-08 46 KSP preconditioned resid norm 1.458094822686e+04 true resid norm 9.677010485817e+16 ||r(i)||/||b|| 5.510830261531e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.165338459409e-02 1 KSP Residual norm 4.021154451202e-03 2 KSP Residual norm 1.428210877005e-03 3 KSP Residual norm 9.534950746545e-04 4 KSP Residual norm 9.336632914576e-04 5 KSP Residual norm 5.542260730215e-04 6 KSP Residual norm 4.285308139791e-04 7 KSP Residual norm 4.169386015716e-04 8 KSP Residual norm 3.236925930430e-04 9 KSP Residual norm 2.956336777945e-04 10 KSP Residual norm 2.548255791119e-04 11 KSP Residual norm 1.709957602369e-04 12 KSP Residual norm 1.444251024216e-04 13 KSP Residual norm 1.271527953630e-04 14 KSP Residual norm 1.109463616545e-04 15 KSP Residual norm 1.026935813105e-04 16 KSP Residual norm 9.036300059143e-05 17 KSP Residual norm 7.974732054415e-05 18 KSP Residual norm 6.246581512295e-05 19 KSP Residual norm 4.815085281768e-05 20 KSP Residual norm 4.508466254521e-05 21 KSP Residual norm 4.605830327172e-05 22 KSP Residual norm 4.480079189875e-05 23 KSP Residual norm 4.226825723677e-05 24 KSP Residual norm 3.886440867565e-05 25 KSP Residual norm 3.714483739506e-05 26 KSP Residual norm 3.466813048214e-05 27 KSP Residual norm 3.157816438163e-05 28 KSP Residual norm 2.734108066183e-05 29 KSP Residual norm 2.289084423499e-05 30 KSP Residual norm 1.868739824367e-05 31 KSP Residual norm 1.703050999555e-05 32 KSP Residual norm 1.488926580684e-05 33 KSP Residual norm 1.316289040688e-05 34 KSP Residual norm 1.176666274689e-05 35 KSP Residual norm 1.067015554096e-05 36 KSP Residual norm 9.683841885193e-06 37 KSP Residual norm 8.289029385390e-06 38 KSP Residual norm 7.639491477433e-06 39 KSP Residual norm 7.531435789549e-06 40 KSP Residual norm 7.097915007145e-06 41 KSP Residual norm 5.932511395006e-06 42 KSP Residual norm 5.082346006282e-06 43 KSP Residual norm 4.127554478438e-06 44 KSP Residual norm 3.308466758627e-06 45 KSP Residual norm 3.080558700489e-06 46 KSP Residual norm 2.614399158331e-06 47 KSP Residual norm 2.134658529762e-06 48 KSP Residual norm 1.867579067836e-06 49 KSP Residual norm 1.676055453007e-06 50 KSP Residual norm 1.467406593581e-06 51 KSP Residual norm 1.384527346995e-06 52 KSP Residual norm 1.332916986381e-06 53 KSP Residual norm 1.153092883405e-06 54 KSP Residual norm 9.480760410664e-07 55 KSP Residual norm 8.687028794959e-07 56 KSP Residual norm 6.933949456396e-07 57 KSP Residual norm 5.186478760568e-07 58 KSP Residual norm 4.365943019232e-07 59 KSP Residual norm 3.798127739025e-07 60 KSP Residual norm 3.436195382948e-07 61 KSP Residual norm 3.027833517334e-07 62 KSP Residual norm 2.441374170129e-07 63 KSP Residual norm 1.686263620605e-07 64 KSP Residual norm 1.223747277648e-07 65 KSP Residual norm 9.678364258946e-08 47 KSP preconditioned resid norm 1.324524139961e+04 true resid norm 1.020728786373e+17 ||r(i)||/||b|| 5.812810777672e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.537737235992e-02 1 KSP Residual norm 5.303141895115e-03 2 KSP Residual norm 1.863822726293e-03 3 KSP Residual norm 1.178842406727e-03 4 KSP Residual norm 1.110742074982e-03 5 KSP Residual norm 6.563003990071e-04 6 KSP Residual norm 4.483556622294e-04 7 KSP Residual norm 4.278233237450e-04 8 KSP Residual norm 3.326725098230e-04 9 KSP Residual norm 3.120271251636e-04 10 KSP Residual norm 2.770975930024e-04 11 KSP Residual norm 1.858335130086e-04 12 KSP Residual norm 1.563279523640e-04 13 KSP Residual norm 1.348893147587e-04 14 KSP Residual norm 1.168594300982e-04 15 KSP Residual norm 1.063801307429e-04 16 KSP Residual norm 9.499221736723e-05 17 KSP Residual norm 8.796818715069e-05 18 KSP Residual norm 7.185287198326e-05 19 KSP Residual norm 5.558252909753e-05 20 KSP Residual norm 5.079665310198e-05 21 KSP Residual norm 5.038704247280e-05 22 KSP Residual norm 4.872418680346e-05 23 KSP Residual norm 4.526028706568e-05 24 KSP Residual norm 4.087094903599e-05 25 KSP Residual norm 3.832350172879e-05 26 KSP Residual norm 3.500901798843e-05 27 KSP Residual norm 3.018377313734e-05 28 KSP Residual norm 2.546885541859e-05 29 KSP Residual norm 2.248558918461e-05 30 KSP Residual norm 2.020541661036e-05 31 KSP Residual norm 2.083469298231e-05 32 KSP Residual norm 1.925807109951e-05 33 KSP Residual norm 1.745158197653e-05 34 KSP Residual norm 1.578755104637e-05 35 KSP Residual norm 1.409002102669e-05 36 KSP Residual norm 1.186049749594e-05 37 KSP Residual norm 9.321894263541e-06 38 KSP Residual norm 7.937422102674e-06 39 KSP Residual norm 7.516031927365e-06 40 KSP Residual norm 7.247292645578e-06 41 KSP Residual norm 6.267071304596e-06 42 KSP Residual norm 5.438391466313e-06 43 KSP Residual norm 4.408785153766e-06 44 KSP Residual norm 3.561595662088e-06 45 KSP Residual norm 3.332953606826e-06 46 KSP Residual norm 2.851848227886e-06 47 KSP Residual norm 2.391843396566e-06 48 KSP Residual norm 2.174700887059e-06 49 KSP Residual norm 2.018935879442e-06 50 KSP Residual norm 1.802401396384e-06 51 KSP Residual norm 1.659281636257e-06 52 KSP Residual norm 1.604130360363e-06 53 KSP Residual norm 1.423234953733e-06 54 KSP Residual norm 1.174340220542e-06 55 KSP Residual norm 1.051131959718e-06 56 KSP Residual norm 8.249639230385e-07 57 KSP Residual norm 5.804529359420e-07 58 KSP Residual norm 4.390716428111e-07 59 KSP Residual norm 3.478998227942e-07 60 KSP Residual norm 3.059031637501e-07 61 KSP Residual norm 2.870663424107e-07 62 KSP Residual norm 2.694912628313e-07 63 KSP Residual norm 2.033324690014e-07 64 KSP Residual norm 1.570231981101e-07 65 KSP Residual norm 1.361584972948e-07 48 KSP preconditioned resid norm 1.101180815503e+04 true resid norm 1.109424710055e+17 ||r(i)||/||b|| 6.317913237796e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.529136856971e-02 1 KSP Residual norm 8.720225705961e-03 2 KSP Residual norm 3.028019961647e-03 3 KSP Residual norm 1.805563664619e-03 4 KSP Residual norm 1.671546733155e-03 5 KSP Residual norm 9.600033404930e-04 6 KSP Residual norm 5.897667072834e-04 7 KSP Residual norm 5.263190838057e-04 8 KSP Residual norm 4.138638419305e-04 9 KSP Residual norm 3.765030082530e-04 10 KSP Residual norm 3.586739527397e-04 11 KSP Residual norm 2.573711674052e-04 12 KSP Residual norm 2.161858824945e-04 13 KSP Residual norm 1.882692548809e-04 14 KSP Residual norm 1.682163508274e-04 15 KSP Residual norm 1.582097462802e-04 16 KSP Residual norm 1.467037090156e-04 17 KSP Residual norm 1.437081284073e-04 18 KSP Residual norm 1.225327698188e-04 19 KSP Residual norm 9.266339101892e-05 20 KSP Residual norm 8.298615331556e-05 21 KSP Residual norm 8.096719774835e-05 22 KSP Residual norm 7.677743588621e-05 23 KSP Residual norm 7.125044954353e-05 24 KSP Residual norm 6.440721127719e-05 25 KSP Residual norm 5.967430005539e-05 26 KSP Residual norm 5.339200488487e-05 27 KSP Residual norm 4.622250973352e-05 28 KSP Residual norm 3.864097511643e-05 29 KSP Residual norm 3.276978268988e-05 30 KSP Residual norm 2.736378460015e-05 31 KSP Residual norm 2.683611762354e-05 32 KSP Residual norm 2.602328357325e-05 33 KSP Residual norm 2.551761795412e-05 34 KSP Residual norm 2.432374830800e-05 35 KSP Residual norm 2.279692777872e-05 36 KSP Residual norm 1.935194015836e-05 37 KSP Residual norm 1.462745596173e-05 38 KSP Residual norm 1.145233584938e-05 39 KSP Residual norm 9.969109032963e-06 40 KSP Residual norm 9.283300264275e-06 41 KSP Residual norm 8.432012598578e-06 42 KSP Residual norm 7.724490979256e-06 43 KSP Residual norm 6.641157282154e-06 44 KSP Residual norm 5.498796121628e-06 45 KSP Residual norm 5.151701677424e-06 46 KSP Residual norm 4.436927309378e-06 47 KSP Residual norm 3.735597158709e-06 48 KSP Residual norm 3.473169996960e-06 49 KSP Residual norm 3.199012134481e-06 50 KSP Residual norm 2.675036782820e-06 51 KSP Residual norm 2.187995874784e-06 52 KSP Residual norm 1.929616177918e-06 53 KSP Residual norm 1.623796547579e-06 54 KSP Residual norm 1.292046651603e-06 55 KSP Residual norm 1.189419090537e-06 56 KSP Residual norm 1.042996713745e-06 57 KSP Residual norm 7.995361730562e-07 58 KSP Residual norm 6.102546806294e-07 59 KSP Residual norm 4.839025498701e-07 60 KSP Residual norm 4.298002047011e-07 61 KSP Residual norm 4.093053943545e-07 62 KSP Residual norm 3.952050688298e-07 63 KSP Residual norm 3.240418403540e-07 64 KSP Residual norm 2.577727357944e-07 65 KSP Residual norm 2.166886066207e-07 49 KSP preconditioned resid norm 1.016098975847e+04 true resid norm 1.147543884061e+17 ||r(i)||/||b|| 6.534992983613e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.014751730764e-02 1 KSP Residual norm 6.965128967426e-03 2 KSP Residual norm 2.458950705186e-03 3 KSP Residual norm 1.436187370269e-03 4 KSP Residual norm 1.355915832117e-03 5 KSP Residual norm 8.244585228380e-04 6 KSP Residual norm 5.857960158146e-04 7 KSP Residual norm 5.529150413455e-04 8 KSP Residual norm 4.163269483849e-04 9 KSP Residual norm 3.820156321200e-04 10 KSP Residual norm 3.374410716302e-04 11 KSP Residual norm 2.222488901810e-04 12 KSP Residual norm 1.757560361732e-04 13 KSP Residual norm 1.439389488094e-04 14 KSP Residual norm 1.139941586897e-04 15 KSP Residual norm 9.756740483185e-05 16 KSP Residual norm 8.321865352518e-05 17 KSP Residual norm 7.746631438270e-05 18 KSP Residual norm 6.158238212134e-05 19 KSP Residual norm 4.509068143151e-05 20 KSP Residual norm 3.655819397263e-05 21 KSP Residual norm 3.355416133263e-05 22 KSP Residual norm 3.213755535085e-05 23 KSP Residual norm 3.083530032586e-05 24 KSP Residual norm 2.801595650808e-05 25 KSP Residual norm 2.515459514272e-05 26 KSP Residual norm 2.236450847735e-05 27 KSP Residual norm 1.922790666296e-05 28 KSP Residual norm 1.525739031535e-05 29 KSP Residual norm 1.264085919461e-05 30 KSP Residual norm 1.036479736113e-05 31 KSP Residual norm 9.535822218185e-06 32 KSP Residual norm 8.530241395454e-06 33 KSP Residual norm 7.184685480567e-06 34 KSP Residual norm 6.349632644878e-06 35 KSP Residual norm 5.961643760947e-06 36 KSP Residual norm 5.479859104209e-06 37 KSP Residual norm 4.340697217846e-06 38 KSP Residual norm 3.448868263597e-06 39 KSP Residual norm 3.210147815538e-06 40 KSP Residual norm 3.160156370872e-06 41 KSP Residual norm 2.909990886827e-06 42 KSP Residual norm 2.696138777541e-06 43 KSP Residual norm 2.348395859547e-06 44 KSP Residual norm 1.966644147929e-06 45 KSP Residual norm 1.869077764919e-06 46 KSP Residual norm 1.669314521407e-06 47 KSP Residual norm 1.427225330931e-06 48 KSP Residual norm 1.269773342409e-06 49 KSP Residual norm 1.151370695893e-06 50 KSP Residual norm 1.038246251457e-06 51 KSP Residual norm 9.715853220357e-07 52 KSP Residual norm 9.694384981127e-07 53 KSP Residual norm 9.028560327428e-07 54 KSP Residual norm 8.108626112536e-07 55 KSP Residual norm 7.974114946454e-07 56 KSP Residual norm 7.140184536740e-07 57 KSP Residual norm 5.594907194899e-07 58 KSP Residual norm 4.357519314403e-07 59 KSP Residual norm 3.539158008487e-07 60 KSP Residual norm 3.233131067159e-07 61 KSP Residual norm 3.114888799006e-07 62 KSP Residual norm 3.043194243786e-07 63 KSP Residual norm 2.407968266722e-07 64 KSP Residual norm 1.903583628524e-07 50 KSP preconditioned resid norm 9.426536560751e+03 true resid norm 1.191223652352e+17 ||r(i)||/||b|| 6.783739008296e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.249841924119e-02 1 KSP Residual norm 7.781664635687e-03 2 KSP Residual norm 2.726547275219e-03 3 KSP Residual norm 1.542887226069e-03 4 KSP Residual norm 1.408720582229e-03 5 KSP Residual norm 8.123388922393e-04 6 KSP Residual norm 5.507207900451e-04 7 KSP Residual norm 4.998200915810e-04 8 KSP Residual norm 3.708927774712e-04 9 KSP Residual norm 3.229767342502e-04 10 KSP Residual norm 3.007584984555e-04 11 KSP Residual norm 2.055581010868e-04 12 KSP Residual norm 1.596103201953e-04 13 KSP Residual norm 1.305743529813e-04 14 KSP Residual norm 1.076132614924e-04 15 KSP Residual norm 9.787201491072e-05 16 KSP Residual norm 8.740312357648e-05 17 KSP Residual norm 8.481258240273e-05 18 KSP Residual norm 7.137120181873e-05 19 KSP Residual norm 5.312239061753e-05 20 KSP Residual norm 4.449250413237e-05 21 KSP Residual norm 4.035542983360e-05 22 KSP Residual norm 3.684508535649e-05 23 KSP Residual norm 3.282088297511e-05 24 KSP Residual norm 2.786661940131e-05 25 KSP Residual norm 2.414164978471e-05 26 KSP Residual norm 2.084937802557e-05 27 KSP Residual norm 1.849171584439e-05 28 KSP Residual norm 1.490210491450e-05 29 KSP Residual norm 1.222357533759e-05 30 KSP Residual norm 1.014570804217e-05 31 KSP Residual norm 9.676284617596e-06 32 KSP Residual norm 8.939032502326e-06 33 KSP Residual norm 7.755974776723e-06 34 KSP Residual norm 6.817504502032e-06 35 KSP Residual norm 6.252582488409e-06 36 KSP Residual norm 5.672989803458e-06 37 KSP Residual norm 4.550319728838e-06 38 KSP Residual norm 3.552445446494e-06 39 KSP Residual norm 3.102724226900e-06 40 KSP Residual norm 2.985390695811e-06 41 KSP Residual norm 2.751827341512e-06 42 KSP Residual norm 2.671934578975e-06 43 KSP Residual norm 2.382593075155e-06 44 KSP Residual norm 1.997153271735e-06 45 KSP Residual norm 1.885446072722e-06 46 KSP Residual norm 1.660379467724e-06 47 KSP Residual norm 1.397007873130e-06 48 KSP Residual norm 1.229646316961e-06 49 KSP Residual norm 1.119330806309e-06 50 KSP Residual norm 9.971479274286e-07 51 KSP Residual norm 9.275441622520e-07 52 KSP Residual norm 9.799288840118e-07 53 KSP Residual norm 9.451732953376e-07 54 KSP Residual norm 8.435198407802e-07 55 KSP Residual norm 8.572036080193e-07 56 KSP Residual norm 8.066579589692e-07 57 KSP Residual norm 6.384472013951e-07 58 KSP Residual norm 4.991355060566e-07 59 KSP Residual norm 3.987833326552e-07 60 KSP Residual norm 3.426524136347e-07 61 KSP Residual norm 3.045644311966e-07 62 KSP Residual norm 2.782838213557e-07 63 KSP Residual norm 2.300840683363e-07 64 KSP Residual norm 1.897317716222e-07 51 KSP preconditioned resid norm 8.625804677687e+03 true resid norm 1.258184275912e+17 ||r(i)||/||b|| 7.165064037536e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 2.896225439208e-02 1 KSP Residual norm 1.000114132932e-02 2 KSP Residual norm 3.495186055002e-03 3 KSP Residual norm 1.996484783996e-03 4 KSP Residual norm 1.828327706984e-03 5 KSP Residual norm 1.054374961256e-03 6 KSP Residual norm 6.355563734858e-04 7 KSP Residual norm 5.490374043826e-04 8 KSP Residual norm 4.044195347461e-04 9 KSP Residual norm 3.586864348861e-04 10 KSP Residual norm 3.338818106887e-04 11 KSP Residual norm 2.290409216084e-04 12 KSP Residual norm 1.878410883568e-04 13 KSP Residual norm 1.680414430761e-04 14 KSP Residual norm 1.464149517926e-04 15 KSP Residual norm 1.383934457305e-04 16 KSP Residual norm 1.286312297590e-04 17 KSP Residual norm 1.296675718557e-04 18 KSP Residual norm 1.124701015547e-04 19 KSP Residual norm 8.623514687789e-05 20 KSP Residual norm 7.330970765560e-05 21 KSP Residual norm 6.694552841670e-05 22 KSP Residual norm 6.044529459289e-05 23 KSP Residual norm 5.566730717127e-05 24 KSP Residual norm 4.970944537541e-05 25 KSP Residual norm 4.793832887048e-05 26 KSP Residual norm 4.723039776080e-05 27 KSP Residual norm 4.529586894585e-05 28 KSP Residual norm 3.999420887683e-05 29 KSP Residual norm 3.507093574033e-05 30 KSP Residual norm 2.869940217284e-05 31 KSP Residual norm 2.716055635395e-05 32 KSP Residual norm 2.540820341108e-05 33 KSP Residual norm 2.331954568011e-05 34 KSP Residual norm 2.105868070728e-05 35 KSP Residual norm 1.933715467956e-05 36 KSP Residual norm 1.709696453273e-05 37 KSP Residual norm 1.333517916126e-05 38 KSP Residual norm 1.064257887776e-05 39 KSP Residual norm 9.351185163763e-06 40 KSP Residual norm 8.634942057860e-06 41 KSP Residual norm 7.509476540536e-06 42 KSP Residual norm 6.742035044856e-06 43 KSP Residual norm 5.819701721514e-06 44 KSP Residual norm 4.958336558898e-06 45 KSP Residual norm 4.667553348271e-06 46 KSP Residual norm 3.895071317867e-06 47 KSP Residual norm 3.083836980055e-06 48 KSP Residual norm 2.627159089299e-06 49 KSP Residual norm 2.374917833700e-06 50 KSP Residual norm 2.154811548644e-06 51 KSP Residual norm 1.932602948388e-06 52 KSP Residual norm 1.849071939491e-06 53 KSP Residual norm 1.661532462426e-06 54 KSP Residual norm 1.405369306770e-06 55 KSP Residual norm 1.321654099292e-06 56 KSP Residual norm 1.151407842631e-06 57 KSP Residual norm 8.828233702174e-07 58 KSP Residual norm 6.981318578454e-07 59 KSP Residual norm 5.691179635192e-07 60 KSP Residual norm 4.910801158627e-07 61 KSP Residual norm 4.395115698343e-07 62 KSP Residual norm 3.847207943129e-07 63 KSP Residual norm 2.839765832047e-07 52 KSP preconditioned resid norm 7.732139191549e+03 true resid norm 1.349578176487e+17 ||r(i)||/||b|| 7.685530842599e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 3.787459797456e-02 1 KSP Residual norm 1.306543726641e-02 2 KSP Residual norm 4.552019316152e-03 3 KSP Residual norm 2.673940737973e-03 4 KSP Residual norm 2.455975780412e-03 5 KSP Residual norm 1.396437062816e-03 6 KSP Residual norm 8.271037518237e-04 7 KSP Residual norm 6.872546481628e-04 8 KSP Residual norm 5.251319056442e-04 9 KSP Residual norm 4.828630648123e-04 10 KSP Residual norm 4.575698097434e-04 11 KSP Residual norm 3.205492493711e-04 12 KSP Residual norm 2.764488252880e-04 13 KSP Residual norm 2.608338461779e-04 14 KSP Residual norm 2.455740551764e-04 15 KSP Residual norm 2.460713809335e-04 16 KSP Residual norm 2.412622649133e-04 17 KSP Residual norm 2.465729413177e-04 18 KSP Residual norm 2.148404270203e-04 19 KSP Residual norm 1.574627594820e-04 20 KSP Residual norm 1.320872666101e-04 21 KSP Residual norm 1.195631830924e-04 22 KSP Residual norm 1.085991333173e-04 23 KSP Residual norm 9.816982180059e-05 24 KSP Residual norm 8.941460303292e-05 25 KSP Residual norm 8.881125578828e-05 26 KSP Residual norm 8.742332111047e-05 27 KSP Residual norm 8.299357351975e-05 28 KSP Residual norm 7.334607914548e-05 29 KSP Residual norm 6.136062744331e-05 30 KSP Residual norm 4.821351366598e-05 31 KSP Residual norm 4.279817364688e-05 32 KSP Residual norm 3.838508511205e-05 33 KSP Residual norm 3.502632131077e-05 34 KSP Residual norm 3.095180135225e-05 35 KSP Residual norm 2.758721576792e-05 36 KSP Residual norm 2.359640371988e-05 37 KSP Residual norm 1.838106081348e-05 38 KSP Residual norm 1.463793449263e-05 39 KSP Residual norm 1.319359280582e-05 40 KSP Residual norm 1.258373653515e-05 41 KSP Residual norm 1.136721262300e-05 42 KSP Residual norm 1.057606660182e-05 43 KSP Residual norm 9.178960772006e-06 44 KSP Residual norm 7.577487185675e-06 45 KSP Residual norm 6.793643591561e-06 46 KSP Residual norm 5.517000969765e-06 47 KSP Residual norm 4.310791162269e-06 48 KSP Residual norm 3.801213837702e-06 49 KSP Residual norm 3.520991737270e-06 50 KSP Residual norm 3.168836007215e-06 51 KSP Residual norm 2.917871088374e-06 52 KSP Residual norm 2.788057311734e-06 53 KSP Residual norm 2.428598525515e-06 54 KSP Residual norm 1.976041504167e-06 55 KSP Residual norm 1.863103404246e-06 56 KSP Residual norm 1.603125510216e-06 57 KSP Residual norm 1.152030559125e-06 58 KSP Residual norm 7.945228000465e-07 59 KSP Residual norm 5.793664702319e-07 60 KSP Residual norm 4.825770010647e-07 61 KSP Residual norm 4.213693415355e-07 62 KSP Residual norm 3.790411982450e-07 63 KSP Residual norm 3.186239009831e-07 53 KSP preconditioned resid norm 6.615977705299e+03 true resid norm 1.479645949961e+17 ||r(i)||/||b|| 8.426236273432e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 5.534299543885e-02 1 KSP Residual norm 1.908845669056e-02 2 KSP Residual norm 6.675424341165e-03 3 KSP Residual norm 3.865402939741e-03 4 KSP Residual norm 3.523146775089e-03 5 KSP Residual norm 1.985996689575e-03 6 KSP Residual norm 1.142253430123e-03 7 KSP Residual norm 9.762937638121e-04 8 KSP Residual norm 7.422361091932e-04 9 KSP Residual norm 6.707384236066e-04 10 KSP Residual norm 6.441654898371e-04 11 KSP Residual norm 4.605729290877e-04 12 KSP Residual norm 3.975308948994e-04 13 KSP Residual norm 3.683314363720e-04 14 KSP Residual norm 3.402185283451e-04 15 KSP Residual norm 3.344095429710e-04 16 KSP Residual norm 3.216162395950e-04 17 KSP Residual norm 3.219828398117e-04 18 KSP Residual norm 2.712905047165e-04 19 KSP Residual norm 1.963628429018e-04 20 KSP Residual norm 1.615188451505e-04 21 KSP Residual norm 1.446186832760e-04 22 KSP Residual norm 1.291746550311e-04 23 KSP Residual norm 1.175761622082e-04 24 KSP Residual norm 1.090909997489e-04 25 KSP Residual norm 1.121856284236e-04 26 KSP Residual norm 1.126690856171e-04 27 KSP Residual norm 1.076815099431e-04 28 KSP Residual norm 9.539951594810e-05 29 KSP Residual norm 8.161318611896e-05 30 KSP Residual norm 6.202852712847e-05 31 KSP Residual norm 5.468156361474e-05 32 KSP Residual norm 4.875029542327e-05 33 KSP Residual norm 4.468761814285e-05 34 KSP Residual norm 4.095854702023e-05 35 KSP Residual norm 3.877659589576e-05 36 KSP Residual norm 3.496736295495e-05 37 KSP Residual norm 2.778422143147e-05 38 KSP Residual norm 2.228734290922e-05 39 KSP Residual norm 1.987670987355e-05 40 KSP Residual norm 1.865655677751e-05 41 KSP Residual norm 1.658341661346e-05 42 KSP Residual norm 1.501570195641e-05 43 KSP Residual norm 1.277429215131e-05 44 KSP Residual norm 1.066377559656e-05 45 KSP Residual norm 1.001338999738e-05 46 KSP Residual norm 8.603321916745e-06 47 KSP Residual norm 7.168230995531e-06 48 KSP Residual norm 6.577881405729e-06 49 KSP Residual norm 6.182423765918e-06 50 KSP Residual norm 5.388222545195e-06 51 KSP Residual norm 4.376214086067e-06 52 KSP Residual norm 3.821061360066e-06 53 KSP Residual norm 3.242234396330e-06 54 KSP Residual norm 2.657702673126e-06 55 KSP Residual norm 2.490268249837e-06 56 KSP Residual norm 2.215518093240e-06 57 KSP Residual norm 1.673059323179e-06 58 KSP Residual norm 1.218050121133e-06 59 KSP Residual norm 9.118615192059e-07 60 KSP Residual norm 7.554474239578e-07 61 KSP Residual norm 6.760870134778e-07 62 KSP Residual norm 6.128925428863e-07 63 KSP Residual norm 4.872665529947e-07 54 KSP preconditioned resid norm 5.588411829570e+03 true resid norm 1.606666312672e+17 ||r(i)||/||b|| 9.149587415484e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 7.117524502481e-02 1 KSP Residual norm 2.452091515298e-02 2 KSP Residual norm 8.565659841127e-03 3 KSP Residual norm 4.915899435939e-03 4 KSP Residual norm 4.488899314170e-03 5 KSP Residual norm 2.534088372379e-03 6 KSP Residual norm 1.459261970655e-03 7 KSP Residual norm 1.208628459326e-03 8 KSP Residual norm 9.141846939026e-04 9 KSP Residual norm 8.241150986565e-04 10 KSP Residual norm 7.876633703423e-04 11 KSP Residual norm 5.593527590639e-04 12 KSP Residual norm 4.815295234214e-04 13 KSP Residual norm 4.471015821100e-04 14 KSP Residual norm 4.174339016644e-04 15 KSP Residual norm 4.142527530944e-04 16 KSP Residual norm 3.966636464920e-04 17 KSP Residual norm 3.987710186403e-04 18 KSP Residual norm 3.409946844508e-04 19 KSP Residual norm 2.435692452505e-04 20 KSP Residual norm 1.995643042580e-04 21 KSP Residual norm 1.788208294162e-04 22 KSP Residual norm 1.606543376267e-04 23 KSP Residual norm 1.434276753393e-04 24 KSP Residual norm 1.318874022217e-04 25 KSP Residual norm 1.357640431484e-04 26 KSP Residual norm 1.371426380330e-04 27 KSP Residual norm 1.320730748297e-04 28 KSP Residual norm 1.174040308615e-04 29 KSP Residual norm 9.969159465049e-05 30 KSP Residual norm 7.526918193918e-05 31 KSP Residual norm 6.409408645616e-05 32 KSP Residual norm 5.545743202136e-05 33 KSP Residual norm 5.093333858239e-05 34 KSP Residual norm 4.731292167660e-05 35 KSP Residual norm 4.630335662995e-05 36 KSP Residual norm 4.394877655871e-05 37 KSP Residual norm 3.627167435510e-05 38 KSP Residual norm 2.892500785312e-05 39 KSP Residual norm 2.558467050303e-05 40 KSP Residual norm 2.322979730074e-05 41 KSP Residual norm 1.985998939806e-05 42 KSP Residual norm 1.806209559269e-05 43 KSP Residual norm 1.571481200276e-05 44 KSP Residual norm 1.306026693550e-05 45 KSP Residual norm 1.236227601277e-05 46 KSP Residual norm 1.091785477771e-05 47 KSP Residual norm 9.014979945176e-06 48 KSP Residual norm 8.081703382362e-06 49 KSP Residual norm 7.331529431001e-06 50 KSP Residual norm 6.428883296168e-06 51 KSP Residual norm 5.600272525901e-06 52 KSP Residual norm 5.041587219015e-06 53 KSP Residual norm 4.189151729950e-06 54 KSP Residual norm 3.413833986449e-06 55 KSP Residual norm 3.211983588033e-06 56 KSP Residual norm 2.816520152414e-06 57 KSP Residual norm 2.074732309036e-06 58 KSP Residual norm 1.501228226841e-06 59 KSP Residual norm 1.123841774593e-06 60 KSP Residual norm 9.168761012380e-07 61 KSP Residual norm 7.736451894018e-07 62 KSP Residual norm 6.715326781752e-07 55 KSP preconditioned resid norm 4.862818612284e+03 true resid norm 1.720548582098e+17 ||r(i)||/||b|| 9.798120201027e+14 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 8.097048986072e-02 1 KSP Residual norm 2.785049534840e-02 2 KSP Residual norm 9.684456395891e-03 3 KSP Residual norm 5.650376247637e-03 4 KSP Residual norm 5.199262421606e-03 5 KSP Residual norm 2.966573500979e-03 6 KSP Residual norm 1.692045148297e-03 7 KSP Residual norm 1.387209750694e-03 8 KSP Residual norm 1.050455468777e-03 9 KSP Residual norm 9.497006304171e-04 10 KSP Residual norm 8.961278488919e-04 11 KSP Residual norm 6.291680836294e-04 12 KSP Residual norm 5.504589413296e-04 13 KSP Residual norm 5.281798966696e-04 14 KSP Residual norm 5.053366004482e-04 15 KSP Residual norm 5.080776569570e-04 16 KSP Residual norm 5.051760021010e-04 17 KSP Residual norm 5.265107423845e-04 18 KSP Residual norm 4.652135565841e-04 19 KSP Residual norm 3.399176827505e-04 20 KSP Residual norm 2.834763365470e-04 21 KSP Residual norm 2.549028565209e-04 22 KSP Residual norm 2.277981488194e-04 23 KSP Residual norm 2.051433406325e-04 24 KSP Residual norm 1.894263801135e-04 25 KSP Residual norm 1.940292985477e-04 26 KSP Residual norm 1.916170631285e-04 27 KSP Residual norm 1.807390612590e-04 28 KSP Residual norm 1.561156995800e-04 29 KSP Residual norm 1.257244859992e-04 30 KSP Residual norm 8.808334349971e-05 31 KSP Residual norm 6.983541834365e-05 32 KSP Residual norm 5.802738647411e-05 33 KSP Residual norm 5.228163110035e-05 34 KSP Residual norm 4.906259915446e-05 35 KSP Residual norm 4.913092591751e-05 36 KSP Residual norm 4.775218606894e-05 37 KSP Residual norm 3.885868942540e-05 38 KSP Residual norm 2.948912342050e-05 39 KSP Residual norm 2.356093479027e-05 40 KSP Residual norm 1.970979939765e-05 41 KSP Residual norm 1.608494721158e-05 42 KSP Residual norm 1.403225031077e-05 43 KSP Residual norm 1.234528305432e-05 44 KSP Residual norm 1.088762436300e-05 45 KSP Residual norm 1.071930332901e-05 46 KSP Residual norm 9.560015373745e-06 47 KSP Residual norm 7.729596621484e-06 48 KSP Residual norm 6.694020308090e-06 49 KSP Residual norm 6.080095146805e-06 50 KSP Residual norm 5.340801611436e-06 51 KSP Residual norm 4.570337941684e-06 52 KSP Residual norm 4.216431725451e-06 53 KSP Residual norm 3.671477390153e-06 54 KSP Residual norm 2.960721494223e-06 55 KSP Residual norm 2.801199472463e-06 56 KSP Residual norm 2.593094018395e-06 57 KSP Residual norm 2.041275135301e-06 58 KSP Residual norm 1.489128390812e-06 59 KSP Residual norm 1.086192246907e-06 60 KSP Residual norm 8.784624368373e-07 61 KSP Residual norm 7.598945731528e-07 56 KSP preconditioned resid norm 4.266164340989e+03 true resid norm 1.836361597449e+17 ||r(i)||/||b|| 1.045764813128e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 9.734111293474e-02 1 KSP Residual norm 3.351236092044e-02 2 KSP Residual norm 1.172250434111e-02 3 KSP Residual norm 6.495880831233e-03 4 KSP Residual norm 5.883235272948e-03 5 KSP Residual norm 3.318964425320e-03 6 KSP Residual norm 1.912855153202e-03 7 KSP Residual norm 1.593440648588e-03 8 KSP Residual norm 1.157658519626e-03 9 KSP Residual norm 1.000884603435e-03 10 KSP Residual norm 9.600579737453e-04 11 KSP Residual norm 6.730787319291e-04 12 KSP Residual norm 5.519402301365e-04 13 KSP Residual norm 4.991750748235e-04 14 KSP Residual norm 4.497907231049e-04 15 KSP Residual norm 4.367545309703e-04 16 KSP Residual norm 4.133743983384e-04 17 KSP Residual norm 4.278837545250e-04 18 KSP Residual norm 3.784761981932e-04 19 KSP Residual norm 2.721189776360e-04 20 KSP Residual norm 2.137712676530e-04 21 KSP Residual norm 1.834616735391e-04 22 KSP Residual norm 1.590450704979e-04 23 KSP Residual norm 1.418151037482e-04 24 KSP Residual norm 1.282077249439e-04 25 KSP Residual norm 1.303003751437e-04 26 KSP Residual norm 1.327957401822e-04 27 KSP Residual norm 1.299175303303e-04 28 KSP Residual norm 1.162352848207e-04 29 KSP Residual norm 1.010247527231e-04 30 KSP Residual norm 7.812742759124e-05 31 KSP Residual norm 6.774559877373e-05 32 KSP Residual norm 5.931013488744e-05 33 KSP Residual norm 5.402446545501e-05 34 KSP Residual norm 4.992005024758e-05 35 KSP Residual norm 4.958704503613e-05 36 KSP Residual norm 4.854802962807e-05 37 KSP Residual norm 4.153167020452e-05 38 KSP Residual norm 3.406039604803e-05 39 KSP Residual norm 3.051357349430e-05 40 KSP Residual norm 2.786768746962e-05 41 KSP Residual norm 2.368381621025e-05 42 KSP Residual norm 2.100606643935e-05 43 KSP Residual norm 1.782726753663e-05 44 KSP Residual norm 1.448786755593e-05 45 KSP Residual norm 1.367238826875e-05 46 KSP Residual norm 1.230515391224e-05 47 KSP Residual norm 1.064571473618e-05 48 KSP Residual norm 9.979454169078e-06 49 KSP Residual norm 9.490434858101e-06 50 KSP Residual norm 8.375194042996e-06 51 KSP Residual norm 6.924545850403e-06 52 KSP Residual norm 6.081694373014e-06 53 KSP Residual norm 5.057411735333e-06 54 KSP Residual norm 3.984780896761e-06 55 KSP Residual norm 3.715181539819e-06 56 KSP Residual norm 3.439471845442e-06 57 KSP Residual norm 2.694381357429e-06 58 KSP Residual norm 2.022736769713e-06 59 KSP Residual norm 1.514617114149e-06 60 KSP Residual norm 1.213176250354e-06 61 KSP Residual norm 1.033439723532e-06 62 KSP Residual norm 9.339196905519e-07 57 KSP preconditioned resid norm 3.588300009114e+03 true resid norm 2.025168832433e+17 ||r(i)||/||b|| 1.153286100375e+15 Residual norms for fieldsplit_0_ solve. 0 KSP Residual norm 1.438264043302e-01 1 KSP Residual norm 4.949969620921e-02 2 KSP Residual norm 1.722525991671e-02 3 KSP Residual norm 1.003128270574e-02 4 KSP Residual norm 9.136248811885e-03 5 KSP Residual norm 5.154880279509e-03 6 KSP Residual norm 2.959435490126e-03 7 KSP Residual norm 2.469024976228e-03 8 KSP Residual norm 1.872358716273e-03 9 KSP Residual norm 1.703625757661e-03 10 KSP Residual norm 1.646023298693e-03 11 KSP Residual norm 1.180248093195e-03 12 KSP Residual norm 1.027162664013e-03 13 KSP Residual norm 9.645198407274e-04 14 KSP Residual norm 8.991999559088e-04 15 KSP Residual norm 8.854476916254e-04 16 KSP Residual norm 8.520628731759e-04 17 KSP Residual norm 8.594115514554e-04 18 KSP Residual norm 7.296494206375e-04 19 KSP Residual norm 5.194112854287e-04 20 KSP Residual norm 4.247355731663e-04 21 KSP Residual norm 3.806791627345e-04 22 KSP Residual norm 3.431911915923e-04 23 KSP Residual norm 3.141771958831e-04 24 KSP Residual norm 2.949977399556e-04 25 KSP Residual norm 3.063472948783e-04 26 KSP Residual norm 3.093611083713e-04 27 KSP Residual norm 2.945437149512e-04 28 KSP Residual norm 2.570348340777e-04 29 KSP Residual norm 2.119598600564e-04 30 KSP Residual norm 1.538243415304e-04 31 KSP Residual norm 1.290977588615e-04 32 KSP Residual norm 1.132740495052e-04 33 KSP Residual norm 1.065776006388e-04 34 KSP Residual norm 1.012380201501e-04 35 KSP Residual norm 1.005203450168e-04 36 KSP Residual norm 9.550144740830e-05 37 KSP Residual norm 7.663249899939e-05 38 KSP Residual norm 5.843432016332e-05 39 KSP Residual norm 4.856472448830e-05 40 KSP Residual norm 4.246126578709e-05 41 KSP Residual norm 3.612834309081e-05 42 KSP Residual norm 3.312665767316e-05 43 KSP Residual norm 2.975232974953e-05 44 KSP Residual norm 2.578464738439e-05 45 KSP Residual norm 2.497910656697e-05 46 KSP Residual norm 2.185533006985e-05 47 KSP Residual norm 1.773658919152e-05 48 KSP Residual norm 1.568563766309e-05 49 KSP Residual norm 1.430107230766e-05 50 KSP Residual norm 1.241428021738e-05 51 KSP Residual norm 1.038130970723e-05 52 KSP Residual norm 9.204344254369e-06 53 KSP Residual norm 7.670690020288e-06 54 KSP Residual norm 6.070490800828e-06 55 KSP Residual norm 5.693507146990e-06 56 KSP Residual norm 5.220329757465e-06 57 KSP Residual norm 4.048429857886e-06 58 KSP Residual norm 2.991885369648e-06 59 KSP Residual norm 2.202617597307e-06 60 KSP Residual norm 1.727572505277e-06 [mpiexec at aci-service-1.chtc.wisc.edu] Sending Ctrl-C to processes as requested [mpiexec at aci-service-1.chtc.wisc.edu] Press Ctrl-C again to force abort 0.03user 0.12system 1:26.53elapsed 0%CPU (0avgtext+0avgdata 17072maxresident)k 904inputs+0outputs (3major+1805minor)pagefaults 0swaps salloc: Relinquishing job allocation 79007 From knepley at gmail.com Tue Oct 28 08:44:55 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 08:44:55 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544F9376.8030008@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> Message-ID: On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali wrote: > Matt > > With MUMPS it is indeed the same but not faster. E.g., see 1.txt and 2.txt > There is a big problem here. This system is never actually being solved. Look at 1.txt: 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 The preconditioned residual did drop 10 order of magnitude, but the true residual (b - Ax) is still very very large. This looks like the system is singular and the preconditioner is masking this. This needs to be figured out first. I would go back to a small serial problem, and make sure it is actually being solved. Matt > With GAMG, cg takes forever (see 4.txt). For reference GAMG with preonly > is attached again (3.txt) > > All logs include -fieldsplit_0_ksp_monitor. > > Also, the true residual shows small oscillations. Is that normal? > > Tabrez > > > On 10/28/2014 07:08 AM, Matthew Knepley wrote: > > On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali > wrote: > >> Mark >> >> When I replace "-fieldsplit_0_ksp_type preonly" with >> "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to kill it). >> > > That means something in the setup is wrong. It should be about the same > or faster. Run with -fieldsplit_0_ksp_monitor > so we can see what is happening. > > Matt > > >> With MUMPS , i.e., with '-fieldsplit_0_pc_type lu >> -fieldsplit_0_pc_factor_mat_solver_package mumps -fieldsplit_0_ksp_type >> preonly' it works fine but takes more time, and will be an issue for larger >> problems. The output for this run is attached. >> >> I will work on passing rigid body modes (as Matt mentioned) but short of >> that what is the best set of options for solving the following problem >> (i.e., linear elasticity with constraints): >> >> |K cG'| | u | = |F| >> |G 0 | |l/c| |d| >> >> where c is a scaling factor (so that cG' terms are more or less of the >> same order as K)? The constraints are used to impose slip between surfaces >> and so on. >> >> Tabrez >> >> On 10/27/2014 01:17 PM, Mark Adams wrote: >> >> The null space for GAMG is not critical but useful for elasticity. If >> you in fact have an indefinite operator (eg, not "pinned) the you need to >> use an iterative coarse grid solver. You are using '-fieldsplit_0_pc_type >> gamg -fieldsplit_0_ksp_type preonly'. And you have a hard elasticity >> problem. You are going to want to start with a stronger solver. Use cg >> instead of preonly. As Matt said start with MUMPS, then go to CG/GAMG, >> then you can see how far you can cut the _0_ solver down. >> Mark >> >> On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley >> wrote: >> >>> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >>> wrote: >>> >>>> Matt >>>> >>>> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>>> >>>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>>> wrote: >>>> >>>>> Matt >>>>> >>>>> Sorry about that (I always forget it). The output for the smallest >>>>> problem is now attached (see log.txt). I am also attaching some results >>>>> that compare results obtained using FS/LSC and the direct solver (MUMPS), >>>>> again for the smallest problem. The difference, as you can see is >>>>> insignificant O(1E-6). >>>>> >>>> >>>> 1) How do you use MUMPS if you have a saddle point >>>> >>>> I simply used -pc_type lu -pc_factor_mat_solver_package mumps. >>>> >>>> >>>> 2) You can see from the output that something is seriously wrong with >>>> the preconditioner. It looks like it has a null space. >>>> Did you add the elastic null modes to GAMG? Without this, it is not >>>> going to work. We have helper functions for this: >>>> >>>> >>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>>> >>>> you could just copy that code. And then use >>>> >>>> >>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>>> >>>> I don't see it in the output, so I think this is your problem. >>>> >>>> In order to test, I would first use MUMPS as the A00 solver and get >>>> the Schur stuff worked out. Then I would >>>> replace MUMPS with GAMG and tune it until I get back my original >>>> convergence. >>>> >>>> I will try this with MatNullSpaceCreateRigidBody. Btw does it matter if >>>> some nodes are pinned? >>>> >>> >>> No these are null modes of the operator, not of the particular problem. >>> >>> Matt >>> >>> >>>> Tabrez >>>> >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> >>>>> Also, I did pass 'upper' and 'full' to >>>>> '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't >>>>> improve (in fact, it increases slightly). The attached log is with 'upper'. >>>>> >>>>> Regards, >>>>> >>>>> Tabrez >>>>> >>>>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>>> >>>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>>>> wrote: >>>>> >>>>>> Hello >>>>>> >>>>>> I am using the following options (below) for solving linear >>>>>> elasticity/poroelasticity problems involving slip between two surfaces >>>>>> involving non-trivial geometries, i.e., elements with high aspect ratios, >>>>>> large contrasts in material properties etc. The constraints are imposed >>>>>> using Lagrange Multipliers. >>>>>> >>>>>> A picture (shows displacement magnitude) is attached. The boundary >>>>>> nodes, i.e., the base and the four side are pinned. >>>>>> >>>>>> The following options appear to work well for the saddle point >>>>>> problem: >>>>>> >>>>>> -pc_type fieldsplit -pc_fieldsplit_type schur >>>>>> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >>>>>> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >>>>>> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >>>>>> -ksp_monitor >>>>>> >>>>>> However, the number of iterations keep on increasing with the >>>>>> problems size (see attached plot), e.g., >>>>>> >>>>>> 120K Tets *507* Iterations (KSP Residual norm >>>>>> 8.827362494659e-05) in 17 secs on 3 cores >>>>>> 1 Million Tets *1374* Iterations (KSP Residual norm >>>>>> 7.164704416296e-05) in 117 secs on 20 cores >>>>>> 8 Million Tets *2495* Iterations (KSP Residual norm >>>>>> 9.101247550026e-05) in 225 secs on 160 cores >>>>>> >>>>>> So what other options should I try to improve solver performance? Any >>>>>> tips/insights would be appreciated as preconditioning is black magic to me. >>>>>> >>>>> >>>>> For reports, always run with >>>>> >>>>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >>>>> >>>>> so that we can see exactly what you used. >>>>> >>>>> I believe the default is a diagonal factorization. Since your outer >>>>> iterates are increasing, I would strengthen this >>>>> to either upper or full >>>>> >>>>> -pc_fieldsplit_schur_factorization_type >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> Tabrez >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 09:03:51 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 08:03:51 -0600 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> Message-ID: <871tpsz3y0.fsf@jedbrown.org> Matthew Knepley writes: > On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali wrote: > >> Matt >> >> With MUMPS it is indeed the same but not faster. E.g., see 1.txt and 2.txt >> > > There is a big problem here. This system is never actually being solved. > Look at 1.txt: > > 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm > 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 > 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm > 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 > > The preconditioned residual did drop 10 order of magnitude, but the true > residual (b - Ax) is still very very large. > This looks like the system is singular and the preconditioner is masking > this. This needs to be figured out first. It could also be very poor scaling between the fields. It's important to nondimensionalize so that you don't ask to solve for velocity 1e15 more accurately than pressure, or vice-versa. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From knepley at gmail.com Tue Oct 28 09:06:01 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 09:06:01 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <871tpsz3y0.fsf@jedbrown.org> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <871tpsz3y0.fsf@jedbrown.org> Message-ID: On Tue, Oct 28, 2014 at 9:03 AM, Jed Brown wrote: > Matthew Knepley writes: > > > On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali > wrote: > > > >> Matt > >> > >> With MUMPS it is indeed the same but not faster. E.g., see 1.txt and > 2.txt > >> > > > > There is a big problem here. This system is never actually being solved. > > Look at 1.txt: > > > > 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm > > 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 > > 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm > > 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 > > > > The preconditioned residual did drop 10 order of magnitude, but the true > > residual (b - Ax) is still very very large. > > This looks like the system is singular and the preconditioner is masking > > this. This needs to be figured out first. > > It could also be very poor scaling between the fields. It's important > to nondimensionalize so that you don't ask to solve for velocity 1e15 > more accurately than pressure, or vice-versa. > Yes, this was a big deal in PyLith. There is a large gap between displacements and the Lagrange multipliers. Without non-dimensionalization, we could not solve the system. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 09:10:05 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 08:10:05 -0600 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: References: <9621705.dtimTkejkQ@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> <7911368.Xkc5LuFApY@besikovitch-ii> Message-ID: <87y4s0xp36.fsf@jedbrown.org> Mark Adams writes: > OK, so the first time there is no RHS and so not eigen estimate and the > next solves do not check to see if a valid eigen estimate has been created. I think so. I'm hoping Filippo will confirm. > Jed: could we just check for zero iterations in the eigen estimator and > then call the random thing. Or something along those lines. Yeah, that should work; an alternative is to defer the estimate until the next solve. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From thronesf at gmail.com Tue Oct 28 09:22:11 2014 From: thronesf at gmail.com (Sharp Stone) Date: Tue, 28 Oct 2014 10:22:11 -0400 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: Hi Matt and Jed, Thank you for your replies. Now I see Petsc will calculate functions' residue in SNES before form the Jacobian. In order to get a initial non-zero residues of the functions, we cannot initialize some of the discretized spatial nodes of the functions to zero, am I right? In my problem, initially only a small region will be initialized with some values, while other spatial nodes remain zero. I don't know how the SNES calculate the initial residue? If it is a sum residue on all discretized nodes, then I should not get a zero residue, or at least, I should get a small residue which is not zero. But I still get the exact zero residue. So I wonder how the SNES calculate the residue? Thanks in advance! On Mon, Oct 27, 2014 at 4:09 PM, Matthew Knepley wrote: > On Mon, Oct 27, 2014 at 2:58 PM, Sharp Stone wrote: > >> Hi Matt, >> >> That is why we asked this. We do not think the calculation of DAE >> F-functions are wrong. But even if the F-functions are not accurate, we >> expect the TS solver would still call TSSetIJacobian(), but actually it >> does not call TSSetIJacobian() routine at all... >> > > SNES is not solving anything. It gets an initial residual of 0.0 so it > never iterates and never forms a Jacobian. Something > is wrong with your residual calculation. Also, you seem to be solving a > linear equations somewhere. Give that solve a > prefix so that it does not interfere with the other output. > > Matt > > >> Thank you in advance! >> >> >> On Mon, Oct 27, 2014 at 3:39 PM, Matthew Knepley >> wrote: >> >>> On Mon, Oct 27, 2014 at 2:31 PM, Sharp Stone wrote: >>> >>>> Hi Matt, >>>> >>>> Thank you for your reply and suggestion. I use the command line options >>>> to make the output on the screen. It was very long and attached below. >>>> Thanks a million in advance! >>>> >>> >>> Notice that you never actually solve anything. There is something wrong >>> with your formulation. Try something where you >>> know the exact solution first. Also run with >>> >>> -snes_converged_reason -ksp_converged_reason >>> >>> Matt >>> >>> >>>> >>>> >>>> 0 KSP preconditioned resid norm 5.607581107219e-11 true resid norm >>>> 9.787719465322e-12 ||r(i)||/||b|| 6.796750851600e-16 >>>> 0 TS dt 2e-12 time 0 >>>> 0 KSP preconditioned resid norm 3.622586114352e+17 true resid norm >>>> 5.225681998280e+16 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 3.704153274287e+17 true resid norm >>>> 1.035782762954e+17 ||r(i)||/||b|| 1.982100639294e+00 >>>> 2 KSP preconditioned resid norm 2.842418683024e+17 true resid norm >>>> 7.903974218109e+16 ||r(i)||/||b|| 1.512524914587e+00 >>>> 3 KSP preconditioned resid norm 2.393970839759e+17 true resid norm >>>> 4.943510895366e+16 ||r(i)||/||b|| 9.460030091754e-01 >>>> 4 KSP preconditioned resid norm 2.271787411557e+17 true resid norm >>>> 4.669216610635e+16 ||r(i)||/||b|| 8.935133466162e-01 >>>> 5 KSP preconditioned resid norm 2.360179849286e+17 true resid norm >>>> 4.108243474851e+16 ||r(i)||/||b|| 7.861640789094e-01 >>>> 6 KSP preconditioned resid norm 2.619090922116e+17 true resid norm >>>> 4.955060344596e+16 ||r(i)||/||b|| 9.482131416009e-01 >>>> 7 KSP preconditioned resid norm 2.333689240438e+17 true resid norm >>>> 4.336784470641e+16 ||r(i)||/||b|| 8.298982739609e-01 >>>> 8 KSP preconditioned resid norm 3.244607818183e+17 true resid norm >>>> 5.407474341938e+16 ||r(i)||/||b|| 1.034788252274e+00 >>>> 9 KSP preconditioned resid norm 3.024748487424e+17 true resid norm >>>> 6.388129145873e+16 ||r(i)||/||b|| 1.222448887624e+00 >>>> 10 KSP preconditioned resid norm 2.127860348138e+17 true resid norm >>>> 3.704786520226e+16 ||r(i)||/||b|| 7.089575143389e-01 >>>> 11 KSP preconditioned resid norm 4.652609350461e+16 true resid norm >>>> 8.497318285406e+15 ||r(i)||/||b|| 1.626068767331e-01 >>>> 12 KSP preconditioned resid norm 2.920260091494e+16 true resid norm >>>> 4.651858168992e+15 ||r(i)||/||b|| 8.901915904036e-02 >>>> 13 KSP preconditioned resid norm 2.527529714994e+16 true resid norm >>>> 3.813962069025e+15 ||r(i)||/||b|| 7.298496292505e-02 >>>> 14 KSP preconditioned resid norm 2.224122069397e+16 true resid norm >>>> 4.355442499437e+15 ||r(i)||/||b|| 8.334687225268e-02 >>>> 15 KSP preconditioned resid norm 2.040909225547e+16 true resid norm >>>> 5.536568280983e+15 ||r(i)||/||b|| 1.059492001772e-01 >>>> 16 KSP preconditioned resid norm 1.834652616545e+16 true resid norm >>>> 3.131287565910e+15 ||r(i)||/||b|| 5.992112736559e-02 >>>> 17 KSP preconditioned resid norm 1.760978175265e+16 true resid norm >>>> 3.245011634605e+15 ||r(i)||/||b|| 6.209738050790e-02 >>>> 18 KSP preconditioned resid norm 1.385338263500e+16 true resid norm >>>> 3.596318213647e+15 ||r(i)||/||b|| 6.882007391247e-02 >>>> 19 KSP preconditioned resid norm 1.488783377120e+16 true resid norm >>>> 2.508274919240e+15 ||r(i)||/||b|| 4.799899649588e-02 >>>> 20 KSP preconditioned resid norm 1.320707251811e+16 true resid norm >>>> 2.042806338080e+15 ||r(i)||/||b|| 3.909166954193e-02 >>>> 21 KSP preconditioned resid norm 7.616453043037e+15 true resid norm >>>> 1.348500002677e+15 ||r(i)||/||b|| 2.580524423647e-02 >>>> 22 KSP preconditioned resid norm 8.879333117289e+15 true resid norm >>>> 1.392661876801e+15 ||r(i)||/||b|| 2.665033726238e-02 >>>> 23 KSP preconditioned resid norm 8.820610261864e+15 true resid norm >>>> 1.365093600480e+15 ||r(i)||/||b|| 2.612278360851e-02 >>>> 24 KSP preconditioned resid norm 7.993346710634e+15 true resid norm >>>> 1.790642691323e+15 ||r(i)||/||b|| 3.426620088847e-02 >>>> 25 KSP preconditioned resid norm 7.525623400757e+15 true resid norm >>>> 1.148549131012e+15 ||r(i)||/||b|| 2.197893272859e-02 >>>> 26 KSP preconditioned resid norm 7.109494224141e+15 true resid norm >>>> 1.253098862904e+15 ||r(i)||/||b|| 2.397962339302e-02 >>>> 27 KSP preconditioned resid norm 9.199926655972e+15 true resid norm >>>> 1.867276980934e+15 ||r(i)||/||b|| 3.573269444158e-02 >>>> 28 KSP preconditioned resid norm 1.121259577312e+16 true resid norm >>>> 3.202544820818e+15 ||r(i)||/||b|| 6.128472459426e-02 >>>> 29 KSP preconditioned resid norm 4.340241766754e+15 true resid norm >>>> 6.879443404198e+14 ||r(i)||/||b|| 1.316468052680e-02 >>>> 30 KSP preconditioned resid norm 4.050053526588e+15 true resid norm >>>> 6.673524276162e+14 ||r(i)||/||b|| 1.277062836652e-02 >>>> 31 KSP preconditioned resid norm 3.675002682668e+15 true resid norm >>>> 5.703629309888e+14 ||r(i)||/||b|| 1.091461231618e-02 >>>> 32 KSP preconditioned resid norm 3.175854297440e+15 true resid norm >>>> 4.918363244864e+14 ||r(i)||/||b|| 9.411906898435e-03 >>>> 33 KSP preconditioned resid norm 2.534108954658e+15 true resid norm >>>> 3.888660709280e+14 ||r(i)||/||b|| 7.441441539229e-03 >>>> 34 KSP preconditioned resid norm 1.517593193539e+15 true resid norm >>>> 2.347716358056e+14 ||r(i)||/||b|| 4.492650641255e-03 >>>> 35 KSP preconditioned resid norm 1.238737221149e+15 true resid norm >>>> 4.944967490035e+14 ||r(i)||/||b|| 9.462817468922e-03 >>>> 36 KSP preconditioned resid norm 8.673662184926e+14 true resid norm >>>> 1.322920275286e+14 ||r(i)||/||b|| 2.531574396837e-03 >>>> 37 KSP preconditioned resid norm 7.206551748402e+14 true resid norm >>>> 1.263301396525e+14 ||r(i)||/||b|| 2.417486171069e-03 >>>> 38 KSP preconditioned resid norm 5.143409845488e+14 true resid norm >>>> 7.953540049469e+13 ||r(i)||/||b|| 1.522009960056e-03 >>>> 39 KSP preconditioned resid norm 4.261910201431e+14 true resid norm >>>> 7.194412501613e+13 ||r(i)||/||b|| 1.376741352417e-03 >>>> 40 KSP preconditioned resid norm 4.660682952219e+14 true resid norm >>>> 7.611539723637e+13 ||r(i)||/||b|| 1.456563894654e-03 >>>> 41 KSP preconditioned resid norm 3.209169783039e+14 true resid norm >>>> 4.903825175831e+13 ||r(i)||/||b|| 9.384086474158e-04 >>>> 0 KSP preconditioned resid norm 5.647841057638e+16 true resid norm >>>> 8.145493103696e+15 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 5.871590984109e+16 true resid norm >>>> 1.629681152051e+16 ||r(i)||/||b|| 2.000715157823e+00 >>>> 2 KSP preconditioned resid norm 4.554798588549e+16 true resid norm >>>> 1.256325647853e+16 ||r(i)||/||b|| 1.542356775532e+00 >>>> 3 KSP preconditioned resid norm 3.879559549735e+16 true resid norm >>>> 7.930640596410e+15 ||r(i)||/||b|| 9.736231429392e-01 >>>> 4 KSP preconditioned resid norm 3.752982759961e+16 true resid norm >>>> 7.738765648717e+15 ||r(i)||/||b|| 9.500671782787e-01 >>>> 5 KSP preconditioned resid norm 3.847367502586e+16 true resid norm >>>> 6.717373650268e+15 ||r(i)||/||b|| 8.246736649031e-01 >>>> 6 KSP preconditioned resid norm 4.072409022879e+16 true resid norm >>>> 7.643006165053e+15 ||r(i)||/||b|| 9.383110473184e-01 >>>> 7 KSP preconditioned resid norm 3.699615223300e+16 true resid norm >>>> 6.864744274739e+15 ||r(i)||/||b|| 8.427659550315e-01 >>>> 8 KSP preconditioned resid norm 4.452919179646e+16 true resid norm >>>> 7.438964756180e+15 ||r(i)||/||b|| 9.132614393603e-01 >>>> 9 KSP preconditioned resid norm 4.218921641123e+16 true resid norm >>>> 8.817640722426e+15 ||r(i)||/||b|| 1.082517732220e+00 >>>> 10 KSP preconditioned resid norm 2.723066166891e+18 true resid norm >>>> 4.777495288747e+17 ||r(i)||/||b|| 5.865200826920e+01 >>>> 11 KSP preconditioned resid norm 1.543090703715e+16 true resid norm >>>> 2.978277930467e+15 ||r(i)||/||b|| 3.656350686879e-01 >>>> 12 KSP preconditioned resid norm 5.543831189057e+15 true resid norm >>>> 9.036431670746e+14 ||r(i)||/||b|| 1.109378100958e-01 >>>> 13 KSP preconditioned resid norm 4.835578439031e+15 true resid norm >>>> 7.302084941558e+14 ||r(i)||/||b|| 8.964570773800e-02 >>>> 14 KSP preconditioned resid norm 4.632268883924e+15 true resid norm >>>> 7.764942849443e+14 ||r(i)||/||b|| 9.532808819050e-02 >>>> 15 KSP preconditioned resid norm 4.279662320804e+15 true resid norm >>>> 7.279053691611e+14 ||r(i)||/||b|| 8.936295935612e-02 >>>> 16 KSP preconditioned resid norm 4.003909782590e+15 true resid norm >>>> 6.398763789786e+14 ||r(i)||/||b|| 7.855588002257e-02 >>>> 17 KSP preconditioned resid norm 3.778065018292e+15 true resid norm >>>> 9.619695351776e+14 ||r(i)||/||b|| 1.180983794267e-01 >>>> 18 KSP preconditioned resid norm 4.432967840881e+15 true resid norm >>>> 1.765362859847e+15 ||r(i)||/||b|| 2.167287894512e-01 >>>> 19 KSP preconditioned resid norm 3.349713073090e+15 true resid norm >>>> 5.343310168427e+14 ||r(i)||/||b|| 6.559836341894e-02 >>>> 20 KSP preconditioned resid norm 3.170973083244e+15 true resid norm >>>> 4.905381423867e+14 ||r(i)||/||b|| 6.022203151385e-02 >>>> 21 KSP preconditioned resid norm 7.415620721710e+15 true resid norm >>>> 1.207250421775e+15 ||r(i)||/||b|| 1.482108457286e-01 >>>> 22 KSP preconditioned resid norm 1.972990201442e+15 true resid norm >>>> 3.077655556372e+14 ||r(i)||/||b|| 3.778353891154e-02 >>>> 23 KSP preconditioned resid norm 2.209676808077e+15 true resid norm >>>> 3.428403970507e+14 ||r(i)||/||b|| 4.208958164793e-02 >>>> 24 KSP preconditioned resid norm 2.003292171306e+15 true resid norm >>>> 4.370066270665e+14 ||r(i)||/||b|| 5.365011319796e-02 >>>> 25 KSP preconditioned resid norm 1.896876965875e+15 true resid norm >>>> 2.911542588080e+14 ||r(i)||/||b|| 3.574421524903e-02 >>>> 26 KSP preconditioned resid norm 1.776666236555e+15 true resid norm >>>> 2.752957912495e+14 ||r(i)||/||b|| 3.379731438537e-02 >>>> 27 KSP preconditioned resid norm 2.106240629900e+15 true resid norm >>>> 3.329282684307e+14 ||r(i)||/||b|| 4.087269661791e-02 >>>> 28 KSP preconditioned resid norm 1.993718847257e+15 true resid norm >>>> 3.265440954005e+14 ||r(i)||/||b|| 4.008892908550e-02 >>>> 29 KSP preconditioned resid norm 1.305019272390e+15 true resid norm >>>> 2.016059917794e+14 ||r(i)||/||b|| 2.475061843560e-02 >>>> 30 KSP preconditioned resid norm 1.188165021558e+15 true resid norm >>>> 2.349019593212e+14 ||r(i)||/||b|| 2.883827367242e-02 >>>> 31 KSP preconditioned resid norm 1.072198412006e+15 true resid norm >>>> 1.660526922017e+14 ||r(i)||/||b|| 2.038583669371e-02 >>>> 32 KSP preconditioned resid norm 9.531727749289e+14 true resid norm >>>> 1.481127635008e+14 ||r(i)||/||b|| 1.818340051551e-02 >>>> 33 KSP preconditioned resid norm 8.113851256730e+14 true resid norm >>>> 1.246949236714e+14 ||r(i)||/||b|| 1.530845610990e-02 >>>> 34 KSP preconditioned resid norm 1.978771889204e+15 true resid norm >>>> 3.123099443158e+14 ||r(i)||/||b|| 3.834144113069e-02 >>>> 35 KSP preconditioned resid norm 5.701229171299e+14 true resid norm >>>> 9.665965877556e+13 ||r(i)||/||b|| 1.186664300676e-02 >>>> 36 KSP preconditioned resid norm 4.923209980376e+14 true resid norm >>>> 8.358449899120e+13 ||r(i)||/||b|| 1.026144125679e-02 >>>> 37 KSP preconditioned resid norm 4.337271696165e+14 true resid norm >>>> 7.056948273999e+13 ||r(i)||/||b|| 8.663623164566e-03 >>>> 38 KSP preconditioned resid norm 3.355561922811e+14 true resid norm >>>> 5.206223795577e+13 ||r(i)||/||b|| 6.391539136181e-03 >>>> 39 KSP preconditioned resid norm 2.948796430248e+14 true resid norm >>>> 4.783720648479e+13 ||r(i)||/||b|| 5.872843531485e-03 >>>> 40 KSP preconditioned resid norm 2.903844355011e+14 true resid norm >>>> 4.553096332971e+13 ||r(i)||/||b|| 5.589712341546e-03 >>>> 41 KSP preconditioned resid norm 2.470740134237e+14 true resid norm >>>> 3.845088511805e+13 ||r(i)||/||b|| 4.720510425650e-03 >>>> 42 KSP preconditioned resid norm 2.144333147632e+14 true resid norm >>>> 4.199300915233e+13 ||r(i)||/||b|| 5.155367344584e-03 >>>> 43 KSP preconditioned resid norm 1.666875797521e+14 true resid norm >>>> 2.786670988964e+13 ||r(i)||/||b|| 3.421120064173e-03 >>>> 44 KSP preconditioned resid norm 1.097596670806e+14 true resid norm >>>> 1.683182082477e+13 ||r(i)||/||b|| 2.066396792741e-03 >>>> 45 KSP preconditioned resid norm 1.086486049833e+14 true resid norm >>>> 1.868163126673e+13 ||r(i)||/||b|| 2.293492981813e-03 >>>> 46 KSP preconditioned resid norm 9.986155841385e+13 true resid norm >>>> 1.544797785039e+13 ||r(i)||/||b|| 1.896506160367e-03 >>>> 47 KSP preconditioned resid norm 9.193832565781e+13 true resid norm >>>> 1.424417809082e+13 ||r(i)||/||b|| 1.748718943038e-03 >>>> 48 KSP preconditioned resid norm 7.868838175665e+13 true resid norm >>>> 1.234477959732e+13 ||r(i)||/||b|| 1.515534963957e-03 >>>> 49 KSP preconditioned resid norm 6.573261298706e+13 true resid norm >>>> 1.037919549254e+13 ||r(i)||/||b|| 1.274225557668e-03 >>>> 50 KSP preconditioned resid norm 6.394833480259e+13 true resid norm >>>> 1.032066032237e+13 ||r(i)||/||b|| 1.267039354277e-03 >>>> 51 KSP preconditioned resid norm 6.650072664422e+13 true resid norm >>>> 2.391163155719e+13 ||r(i)||/||b|| 2.935565870940e-03 >>>> 52 KSP preconditioned resid norm 6.239223696051e+13 true resid norm >>>> 9.858797480463e+12 ||r(i)||/||b|| 1.210337711291e-03 >>>> 53 KSP preconditioned resid norm 6.247432830217e+13 true resid norm >>>> 9.562224327829e+12 ||r(i)||/||b|| 1.173928233208e-03 >>>> 54 KSP preconditioned resid norm 6.218810094746e+13 true resid norm >>>> 9.709963371483e+12 ||r(i)||/||b|| 1.192065753156e-03 >>>> 55 KSP preconditioned resid norm 6.285033387840e+13 true resid norm >>>> 9.623710764220e+12 ||r(i)||/||b|| 1.181476755515e-03 >>>> 56 KSP preconditioned resid norm 5.407058856328e+13 true resid norm >>>> 9.490503427247e+12 ||r(i)||/||b|| 1.165123253611e-03 >>>> 0 KSP preconditioned resid norm 9.075431914728e+17 true resid norm >>>> 1.310345788054e+17 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 8.641655528143e+17 true resid norm >>>> 2.471138002109e+17 ||r(i)||/||b|| 1.885867092974e+00 >>>> 2 KSP preconditioned resid norm 6.384433252903e+17 true resid norm >>>> 1.816534419632e+17 ||r(i)||/||b|| 1.386301567259e+00 >>>> 3 KSP preconditioned resid norm 5.179227150533e+17 true resid norm >>>> 1.107056703740e+17 ||r(i)||/||b|| 8.448584441089e-01 >>>> 4 KSP preconditioned resid norm 4.625114083211e+17 true resid norm >>>> 9.438923102711e+16 ||r(i)||/||b|| 7.203383403647e-01 >>>> 5 KSP preconditioned resid norm 4.977065165555e+17 true resid norm >>>> 8.614996614554e+16 ||r(i)||/||b|| 6.574597860420e-01 >>>> 6 KSP preconditioned resid norm 6.630931668382e+17 true resid norm >>>> 1.310543864565e+17 ||r(i)||/||b|| 1.000151163542e+00 >>>> 7 KSP preconditioned resid norm 5.452278853431e+17 true resid norm >>>> 1.019521297074e+17 ||r(i)||/||b|| 7.780551564087e-01 >>>> 8 KSP preconditioned resid norm 1.958476842674e+18 true resid norm >>>> 3.244825770314e+17 ||r(i)||/||b|| 2.476312588551e+00 >>>> 9 KSP preconditioned resid norm 1.577830260132e+18 true resid norm >>>> 3.469004540765e+17 ||r(i)||/||b|| 2.647396261651e+00 >>>> 10 KSP preconditioned resid norm 1.081897998863e+17 true resid norm >>>> 1.828781621979e+16 ||r(i)||/||b|| 1.395648109568e-01 >>>> 11 KSP preconditioned resid norm 5.614465792481e+16 true resid norm >>>> 9.930812946447e+15 ||r(i)||/||b|| 7.578772746079e-02 >>>> 12 KSP preconditioned resid norm 4.960081715579e+16 true resid norm >>>> 8.035214375965e+15 ||r(i)||/||b|| 6.132132792135e-02 >>>> 13 KSP preconditioned resid norm 3.329600867897e+16 true resid norm >>>> 6.602562228940e+15 ||r(i)||/||b|| 5.038793797127e-02 >>>> 14 KSP preconditioned resid norm 3.130960732609e+16 true resid norm >>>> 4.730473520325e+15 ||r(i)||/||b|| 3.610095566719e-02 >>>> 15 KSP preconditioned resid norm 2.560518921527e+16 true resid norm >>>> 3.945110867578e+15 ||r(i)||/||b|| 3.010740297365e-02 >>>> 16 KSP preconditioned resid norm 1.584886017619e+16 true resid norm >>>> 3.416780637867e+15 ||r(i)||/||b|| 2.607541207073e-02 >>>> 17 KSP preconditioned resid norm 1.443824757417e+16 true resid norm >>>> 2.286337927483e+15 ||r(i)||/||b|| 1.744835560449e-02 >>>> 18 KSP preconditioned resid norm 1.039145582094e+16 true resid norm >>>> 1.711526931765e+15 ||r(i)||/||b|| 1.306164332627e-02 >>>> 19 KSP preconditioned resid norm 1.018242603191e+16 true resid norm >>>> 1.657426513724e+15 ||r(i)||/||b|| 1.264877201754e-02 >>>> 20 KSP preconditioned resid norm 8.198983478967e+15 true resid norm >>>> 1.311419876212e+15 ||r(i)||/||b|| 1.000819698256e-02 >>>> 21 KSP preconditioned resid norm 6.481494503642e+15 true resid norm >>>> 1.039872641976e+15 ||r(i)||/||b|| 7.935864345546e-03 >>>> 22 KSP preconditioned resid norm 6.325821399407e+15 true resid norm >>>> 1.165377948147e+15 ||r(i)||/||b|| 8.893667295849e-03 >>>> 23 KSP preconditioned resid norm 5.405294478066e+15 true resid norm >>>> 1.922243319095e+15 ||r(i)||/||b|| 1.466974089297e-02 >>>> 24 KSP preconditioned resid norm 4.960676496180e+15 true resid norm >>>> 8.018184223519e+14 ||r(i)||/||b|| 6.119136106374e-03 >>>> 25 KSP preconditioned resid norm 4.419138742847e+15 true resid norm >>>> 7.148418830282e+14 ||r(i)||/||b|| 5.455368266491e-03 >>>> 26 KSP preconditioned resid norm 2.795870478230e+15 true resid norm >>>> 4.843283048525e+14 ||r(i)||/||b|| 3.696186985663e-03 >>>> 27 KSP preconditioned resid norm 2.911201681995e+15 true resid norm >>>> 4.441742607418e+14 ||r(i)||/||b|| 3.389748452592e-03 >>>> 28 KSP preconditioned resid norm 2.776697567476e+15 true resid norm >>>> 5.074732182061e+14 ||r(i)||/||b|| 3.872819089681e-03 >>>> 29 KSP preconditioned resid norm 1.941501506934e+15 true resid norm >>>> 2.950418833153e+14 ||r(i)||/||b|| 2.251633774878e-03 >>>> 30 KSP preconditioned resid norm 8.413150178496e+14 true resid norm >>>> 1.695476752862e+14 ||r(i)||/||b|| 1.293915520864e-03 >>>> 0 KSP preconditioned resid norm 1.415973479575e+17 true resid norm >>>> 2.042710742436e+16 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 1.440919303250e+17 true resid norm >>>> 4.045863378014e+16 ||r(i)||/||b|| 1.980634503928e+00 >>>> 2 KSP preconditioned resid norm 1.100352174605e+17 true resid norm >>>> 3.072773288453e+16 ||r(i)||/||b|| 1.504262558872e+00 >>>> 3 KSP preconditioned resid norm 9.218323125716e+16 true resid norm >>>> 1.914173874994e+16 ||r(i)||/||b|| 9.370753456316e-01 >>>> 4 KSP preconditioned resid norm 8.672799672395e+16 true resid norm >>>> 1.779286318286e+16 ||r(i)||/||b|| 8.710417394506e-01 >>>> 5 KSP preconditioned resid norm 9.059066321416e+16 true resid norm >>>> 1.574500663517e+16 ||r(i)||/||b|| 7.707898288327e-01 >>>> 6 KSP preconditioned resid norm 1.027848583976e+17 true resid norm >>>> 1.954996111922e+16 ||r(i)||/||b|| 9.570596909823e-01 >>>> 7 KSP preconditioned resid norm 9.069106612200e+16 true resid norm >>>> 1.686428629204e+16 ||r(i)||/||b|| 8.255836688812e-01 >>>> 8 KSP preconditioned resid norm 1.359707392060e+17 true resid norm >>>> 2.263432282464e+16 ||r(i)||/||b|| 1.108053252691e+00 >>>> 9 KSP preconditioned resid norm 1.256447896822e+17 true resid norm >>>> 2.668588264948e+16 ||r(i)||/||b|| 1.306395570116e+00 >>>> 10 KSP preconditioned resid norm 5.661761773340e+16 true resid norm >>>> 9.807578010522e+15 ||r(i)||/||b|| 4.801256392682e-01 >>>> 11 KSP preconditioned resid norm 1.597811133832e+16 true resid norm >>>> 2.854210532053e+15 ||r(i)||/||b|| 1.397266129148e-01 >>>> 12 KSP preconditioned resid norm 1.141170314391e+16 true resid norm >>>> 1.817769027418e+15 ||r(i)||/||b|| 8.898807793264e-02 >>>> 13 KSP preconditioned resid norm 9.557798166843e+15 true resid norm >>>> 1.455009507586e+15 ||r(i)||/||b|| 7.122934624852e-02 >>>> 14 KSP preconditioned resid norm 7.997842098400e+15 true resid norm >>>> 2.021581420109e+15 ||r(i)||/||b|| 9.896562338034e-02 >>>> 15 KSP preconditioned resid norm 7.518666379332e+15 true resid norm >>>> 2.232979183117e+15 ||r(i)||/||b|| 1.093145072735e-01 >>>> 16 KSP preconditioned resid norm 6.614897634500e+15 true resid norm >>>> 1.222952133628e+15 ||r(i)||/||b|| 5.986908024822e-02 >>>> 17 KSP preconditioned resid norm 6.199437182741e+15 true resid norm >>>> 1.163257046730e+15 ||r(i)||/||b|| 5.694673369870e-02 >>>> 18 KSP preconditioned resid norm 4.584922270319e+15 true resid norm >>>> 9.962391827866e+14 ||r(i)||/||b|| 4.877044811535e-02 >>>> 19 KSP preconditioned resid norm 5.012002077991e+15 true resid norm >>>> 8.755757394511e+14 ||r(i)||/||b|| 4.286342266977e-02 >>>> 20 KSP preconditioned resid norm 4.312819580189e+15 true resid norm >>>> 6.706322150678e+14 ||r(i)||/||b|| 3.283050317090e-02 >>>> 21 KSP preconditioned resid norm 2.607093537931e+15 true resid norm >>>> 5.192589254045e+14 ||r(i)||/||b|| 2.542009079490e-02 >>>> 22 KSP preconditioned resid norm 2.989363098445e+15 true resid norm >>>> 4.636989016003e+14 ||r(i)||/||b|| 2.270017442838e-02 >>>> 23 KSP preconditioned resid norm 3.016276187245e+15 true resid norm >>>> 4.674022912685e+14 ||r(i)||/||b|| 2.288147222994e-02 >>>> 24 KSP preconditioned resid norm 2.901859180715e+15 true resid norm >>>> 4.492447164798e+14 ||r(i)||/||b|| 2.199257619530e-02 >>>> 25 KSP preconditioned resid norm 2.843000955863e+15 true resid norm >>>> 5.109023428494e+14 ||r(i)||/||b|| 2.501099799574e-02 >>>> 26 KSP preconditioned resid norm 2.741940909323e+15 true resid norm >>>> 4.263494245202e+14 ||r(i)||/||b|| 2.087174731415e-02 >>>> 27 KSP preconditioned resid norm 1.296293793548e+15 true resid norm >>>> 2.124064840417e+14 ||r(i)||/||b|| 1.039826538477e-02 >>>> 28 KSP preconditioned resid norm 9.720512789714e+14 true resid norm >>>> 2.091841490290e+14 ||r(i)||/||b|| 1.024051740089e-02 >>>> 29 KSP preconditioned resid norm 1.459179957917e+15 true resid norm >>>> 2.244723575034e+14 ||r(i)||/||b|| 1.098894487800e-02 >>>> 30 KSP preconditioned resid norm 1.339547884067e+15 true resid norm >>>> 2.746667338979e+14 ||r(i)||/||b|| 1.344618835119e-02 >>>> 31 KSP preconditioned resid norm 1.186062213325e+15 true resid norm >>>> 1.831856176134e+14 ||r(i)||/||b|| 8.967770806105e-03 >>>> 32 KSP preconditioned resid norm 9.909356538944e+14 true resid norm >>>> 1.788908155328e+14 ||r(i)||/||b|| 8.757520671747e-03 >>>> 33 KSP preconditioned resid norm 7.609550150745e+14 true resid norm >>>> 1.203572113192e+14 ||r(i)||/||b|| 5.892033992812e-03 >>>> 34 KSP preconditioned resid norm 5.051711143811e+14 true resid norm >>>> 7.920683224412e+13 ||r(i)||/||b|| 3.877535404237e-03 >>>> 35 KSP preconditioned resid norm 5.443461822227e+14 true resid norm >>>> 8.557198028123e+13 ||r(i)||/||b|| 4.189138408269e-03 >>>> 36 KSP preconditioned resid norm 4.648220720251e+14 true resid norm >>>> 7.162139608842e+13 ||r(i)||/||b|| 3.506193735634e-03 >>>> 37 KSP preconditioned resid norm 3.842876907480e+14 true resid norm >>>> 6.998069618798e+13 ||r(i)||/||b|| 3.425873998417e-03 >>>> 38 KSP preconditioned resid norm 2.822934553902e+14 true resid norm >>>> 5.594039593962e+13 ||r(i)||/||b|| 2.738537316003e-03 >>>> 39 KSP preconditioned resid norm 2.288268021627e+14 true resid norm >>>> 3.624972614121e+13 ||r(i)||/||b|| 1.774589293929e-03 >>>> 40 KSP preconditioned resid norm 3.336880274482e+14 true resid norm >>>> 5.633095049288e+13 ||r(i)||/||b|| 2.757656741242e-03 >>>> 41 KSP preconditioned resid norm 1.655073694079e+14 true resid norm >>>> 2.539321805860e+13 ||r(i)||/||b|| 1.243113747388e-03 >>>> 42 KSP preconditioned resid norm 1.317678812236e+14 true resid norm >>>> 2.429513710603e+13 ||r(i)||/||b|| 1.189357680523e-03 >>>> 0 KSP preconditioned resid norm 4.735355646927e+18 true resid norm >>>> 6.977368406398e+17 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 1.655056377336e+18 true resid norm >>>> 5.433322725028e+17 ||r(i)||/||b|| 7.787065851426e-01 >>>> 2 KSP preconditioned resid norm 8.847192945348e+17 true resid norm >>>> 2.725627800798e+17 ||r(i)||/||b|| 3.906383670810e-01 >>>> 3 KSP preconditioned resid norm 5.344946494044e+17 true resid norm >>>> 1.588736353868e+17 ||r(i)||/||b|| 2.276985048419e-01 >>>> 4 KSP preconditioned resid norm 3.198139750974e+17 true resid norm >>>> 6.624530455539e+16 ||r(i)||/||b|| 9.494310848578e-02 >>>> 5 KSP preconditioned resid norm 3.286368493485e+17 true resid norm >>>> 6.812427209313e+16 ||r(i)||/||b|| 9.763605434775e-02 >>>> 6 KSP preconditioned resid norm 4.284030910438e+17 true resid norm >>>> 1.696553569581e+17 ||r(i)||/||b|| 2.431509232084e-01 >>>> 7 KSP preconditioned resid norm 1.587968810131e+18 true resid norm >>>> 3.732963000458e+17 ||r(i)||/||b|| 5.350101618592e-01 >>>> 8 KSP preconditioned resid norm 6.621891209840e+16 true resid norm >>>> 1.229792367414e+16 ||r(i)||/||b|| 1.762544695628e-02 >>>> 9 KSP preconditioned resid norm 5.638286198157e+16 true resid norm >>>> 2.091986582514e+16 ||r(i)||/||b|| 2.998245843800e-02 >>>> 10 KSP preconditioned resid norm 1.309067125882e+16 true resid norm >>>> 2.099826104432e+15 ||r(i)||/||b|| 3.009481486611e-03 >>>> 11 KSP preconditioned resid norm 1.075925809079e+16 true resid norm >>>> 2.578010577039e+15 ||r(i)||/||b|| 3.694817912546e-03 >>>> 12 KSP preconditioned resid norm 9.319606428298e+15 true resid norm >>>> 2.026298982436e+15 ||r(i)||/||b|| 2.904102040216e-03 >>>> 13 KSP preconditioned resid norm 4.070270220193e+15 true resid norm >>>> 2.216532493559e+15 ||r(i)||/||b|| 3.176745678968e-03 >>>> 14 KSP preconditioned resid norm 3.400982828163e+15 true resid norm >>>> 9.161069365806e+14 ||r(i)||/||b|| 1.312969135671e-03 >>>> 15 KSP preconditioned resid norm 4.173720697476e+14 true resid norm >>>> 7.019272258636e+13 ||r(i)||/||b|| 1.006005681483e-04 >>>> 16 KSP preconditioned resid norm 2.871344308441e+14 true resid norm >>>> 8.503915375563e+13 ||r(i)||/||b|| 1.218785490496e-04 >>>> 17 KSP preconditioned resid norm 8.731930916187e+13 true resid norm >>>> 2.939915802842e+13 ||r(i)||/||b|| 4.213502328681e-05 >>>> 18 KSP preconditioned resid norm 7.273170647297e+13 true resid norm >>>> 2.554194915512e+13 ||r(i)||/||b|| 3.660685184933e-05 >>>> 19 KSP preconditioned resid norm 5.198384943683e+13 true resid norm >>>> 8.213409786802e+12 ||r(i)||/||b|| 1.177150081293e-05 >>>> 20 KSP preconditioned resid norm 2.826075121950e+13 true resid norm >>>> 1.225537290577e+13 ||r(i)||/||b|| 1.756446297795e-05 >>>> 0 KSP preconditioned resid norm 7.535755573391e+17 true resid norm >>>> 1.091530949677e+17 ||r(i)||/||b|| 1.000000000000e+00 >>>> 1 KSP preconditioned resid norm 5.749658535033e+17 true resid norm >>>> 1.739986551510e+17 ||r(i)||/||b|| 1.594078987889e+00 >>>> 2 KSP preconditioned resid norm 3.832130567981e+17 true resid norm >>>> 1.138657236506e+17 ||r(i)||/||b|| 1.043174485197e+00 >>>> 3 KSP preconditioned resid norm 2.820261830557e+17 true resid norm >>>> 6.703201541200e+16 ||r(i)||/||b|| 6.141100756863e-01 >>>> 4 KSP preconditioned resid norm 2.196333397214e+17 true resid norm >>>> 4.464718002673e+16 ||r(i)||/||b|| 4.090326530819e-01 >>>> 5 KSP preconditioned resid norm 2.438749906836e+17 true resid norm >>>> 4.259961207836e+16 ||r(i)||/||b|| 3.902739733672e-01 >>>> 6 KSP preconditioned resid norm 7.293863369143e+17 true resid norm >>>> 1.861518427200e+17 ||r(i)||/||b|| 1.705419738901e+00 >>>> 7 KSP preconditioned resid norm 3.735172838302e+17 true resid norm >>>> 7.178953918634e+16 ||r(i)||/||b|| 6.576958647631e-01 >>>> 8 KSP preconditioned resid norm 2.306462817079e+17 true resid norm >>>> 3.812340554474e+16 ||r(i)||/||b|| 3.492654565226e-01 >>>> 9 KSP preconditioned resid norm 2.324630150277e+17 true resid norm >>>> 5.879635462988e+16 ||r(i)||/||b|| 5.386595281360e-01 >>>> 10 KSP preconditioned resid norm 2.486084300490e+16 true resid norm >>>> 3.943582532938e+15 ||r(i)||/||b|| 3.612891172811e-02 >>>> 11 KSP preconditioned resid norm 2.044336976942e+16 true resid norm >>>> 4.427406712026e+15 ||r(i)||/||b|| 4.056143999707e-02 >>>> 12 KSP preconditioned resid norm 1.903964203761e+16 true resid norm >>>> 3.227987898262e+15 ||r(i)||/||b|| 2.957303133931e-02 >>>> 13 KSP preconditioned resid norm 6.823460749064e+15 true resid norm >>>> 1.876444880951e+15 ||r(i)||/||b|| 1.719094526368e-02 >>>> 14 KSP preconditioned resid norm 7.090194359812e+15 true resid norm >>>> 1.450790953432e+15 ||r(i)||/||b|| 1.329134051455e-02 >>>> 15 KSP preconditioned resid norm 4.884314111756e+15 true resid norm >>>> 8.008003608330e+14 ||r(i)||/||b|| 7.336487903253e-03 >>>> 16 KSP preconditioned resid norm 1.603299777176e+15 true resid norm >>>> 2.941609026573e+14 ||r(i)||/||b|| 2.694938725689e-03 >>>> 17 KSP preconditioned resid norm 1.548805654760e+15 true resid norm >>>> 2.838534773138e+14 ||r(i)||/||b|| 2.600507822503e-03 >>>> 18 KSP preconditioned resid norm 1.022454633110e+15 true resid norm >>>> 2.215058790848e+14 ||r(i)||/||b|| 2.029313773928e-03 >>>> 19 KSP preconditioned resid norm 9.587995256078e+14 true resid norm >>>> 2.226912518453e+14 ||r(i)||/||b|| 2.040173500451e-03 >>>> 20 KSP preconditioned resid norm 7.709554287637e+14 true resid norm >>>> 1.253194818485e+14 ||r(i)||/||b|| 1.148107452983e-03 >>>> 21 KSP preconditioned resid norm 3.692994290799e+14 true resid norm >>>> 6.513442758709e+13 ||r(i)||/||b|| 5.967254305192e-04 >>>> 22 KSP preconditioned resid norm 3.577186864519e+14 true resid norm >>>> 5.884014914924e+13 ||r(i)||/||b|| 5.390607491859e-04 >>>> 23 KSP preconditioned resid norm 1.723423887037e+14 true resid norm >>>> 4.313917181929e+13 ||r(i)||/||b|| 3.952171198816e-04 >>>> 24 KSP preconditioned resid norm 1.656260736170e+14 true resid norm >>>> 3.420915571267e+13 ||r(i)||/||b|| 3.134052746997e-04 >>>> 25 KSP preconditioned resid norm 1.188400863617e+14 true resid norm >>>> 1.877160245085e+13 ||r(i)||/||b|| 1.719749903236e-04 >>>> 26 KSP preconditioned resid norm 8.416275062676e+13 true resid norm >>>> 1.813244070153e+13 ||r(i)||/||b|| 1.661193455568e-04 >>>> 27 KSP preconditioned resid norm 5.224466611453e+13 true resid norm >>>> 8.293327735079e+12 ||r(i)||/||b|| 7.597886012791e-05 >>>> 28 KSP preconditioned resid norm 4.471999764526e+13 true resid norm >>>> 9.742703301619e+12 ||r(i)||/||b|| 8.925723365429e-05 >>>> 29 KSP preconditioned resid norm 9.410327220658e+14 true resid norm >>>> 1.612776349391e+14 ||r(i)||/||b|| 1.477536069746e-03 >>>> 30 KSP preconditioned resid norm 5.783713583723e+14 true resid norm >>>> 1.102180319983e+14 ||r(i)||/||b|| 1.009756361291e-03 >>>> 31 KSP preconditioned resid norm 1.833775181615e+13 true resid norm >>>> 4.198845362007e+12 ||r(i)||/||b|| 3.846748791915e-05 >>>> 32 KSP preconditioned resid norm 1.462922686654e+13 true resid norm >>>> 3.704838629537e+12 ||r(i)||/||b|| 3.394167275452e-05 >>>> 33 KSP preconditioned resid norm 1.053043620710e+13 true resid norm >>>> 1.904310347574e+12 ||r(i)||/||b|| 1.744623318411e-05 >>>> 34 KSP preconditioned resid norm 8.014167149326e+13 true resid norm >>>> 1.458222541169e+13 ||r(i)||/||b|| 1.335942459168e-04 >>>> 35 KSP preconditioned resid norm 5.245679087504e+12 true resid norm >>>> 1.495201096266e+12 ||r(i)||/||b|| 1.369820156459e-05 >>>> 0 KSP preconditioned resid norm 1.124208361458e+16 true resid norm >>>> 6.487794900041e+15 ||r(i)||/||b|| 1.232063762752e-01 >>>> 1 KSP preconditioned resid norm 5.136632948471e+15 true resid norm >>>> 1.030125368299e+15 ||r(i)||/||b|| 1.956258107611e-02 >>>> 2 KSP preconditioned resid norm 4.225773164973e+15 true resid norm >>>> 7.107211019673e+14 ||r(i)||/||b|| 1.349693892375e-02 >>>> 3 KSP preconditioned resid norm 4.144329493920e+15 true resid norm >>>> 7.618007419548e+14 ||r(i)||/||b|| 1.446696609651e-02 >>>> 4 KSP preconditioned resid norm 4.333716562490e+15 true resid norm >>>> 8.456727780045e+14 ||r(i)||/||b|| 1.605973679776e-02 >>>> 5 KSP preconditioned resid norm 4.885745055980e+15 true resid norm >>>> 7.972847407816e+14 ||r(i)||/||b|| 1.514082446881e-02 >>>> 6 KSP preconditioned resid norm 6.011621624545e+15 true resid norm >>>> 9.523510473075e+14 ||r(i)||/||b|| 1.808560894548e-02 >>>> 7 KSP preconditioned resid norm 7.637951309150e+15 true resid norm >>>> 1.487731882292e+15 ||r(i)||/||b|| 2.825275103643e-02 >>>> 8 KSP preconditioned resid norm 8.460249904060e+15 true resid norm >>>> 1.336944180115e+15 ||r(i)||/||b|| 2.538921933447e-02 >>>> 9 KSP preconditioned resid norm 8.381050292195e+15 true resid norm >>>> 1.752911449852e+15 ||r(i)||/||b|| 3.328863982216e-02 >>>> 10 KSP preconditioned resid norm 7.434169027728e+15 true resid norm >>>> 1.180498665014e+15 ||r(i)||/||b|| 2.241824301708e-02 >>>> 11 KSP preconditioned resid norm 5.682629267055e+15 true resid norm >>>> 9.983854763086e+14 ||r(i)||/||b|| 1.895982511114e-02 >>>> 12 KSP preconditioned resid norm 3.541898147790e+15 true resid norm >>>> 5.480576756264e+14 ||r(i)||/||b|| 1.040788145188e-02 >>>> 13 KSP preconditioned resid norm 2.510644084392e+15 true resid norm >>>> 4.650369826671e+14 ||r(i)||/||b|| 8.831278169411e-03 >>>> 14 KSP preconditioned resid norm 2.282038802861e+15 true resid norm >>>> 3.635360161467e+14 ||r(i)||/||b|| 6.903725516148e-03 >>>> 15 KSP preconditioned resid norm 2.130130680848e+15 true resid norm >>>> 3.489368266499e+14 ||r(i)||/||b|| 6.626479816775e-03 >>>> 16 KSP preconditioned resid norm 1.929984942209e+15 true resid norm >>>> 3.145683546766e+14 ||r(i)||/||b|| 5.973805841227e-03 >>>> 17 KSP preconditioned resid norm 1.692158321133e+15 true resid norm >>>> 2.756020615583e+14 ||r(i)||/||b|| 5.233817008974e-03 >>>> 18 KSP preconditioned resid norm 1.473781775606e+15 true resid norm >>>> 2.951884895649e+14 ||r(i)||/||b|| 5.605772789952e-03 >>>> 19 KSP preconditioned resid norm 1.275239549902e+15 true resid norm >>>> 1.993708462138e+14 ||r(i)||/||b|| 3.786149204063e-03 >>>> 20 KSP preconditioned resid norm 1.159341081209e+15 true resid norm >>>> 2.011812256624e+14 ||r(i)||/||b|| 3.820529189093e-03 >>>> 21 KSP preconditioned resid norm 1.026465449352e+15 true resid norm >>>> 1.599378003014e+14 ||r(i)||/||b|| 3.037296509548e-03 >>>> 22 KSP preconditioned resid norm 9.313333291328e+14 true resid norm >>>> 1.626450968383e+14 ||r(i)||/||b|| 3.088709385718e-03 >>>> 23 KSP preconditioned resid norm 8.475207473455e+14 true resid norm >>>> 1.311587742521e+14 ||r(i)||/||b|| 2.490768826892e-03 >>>> 24 KSP preconditioned resid norm 7.894756750239e+14 true resid norm >>>> 1.443612018407e+14 ||r(i)||/||b|| 2.741489339222e-03 >>>> 25 KSP preconditioned resid norm 7.370222420808e+14 true resid norm >>>> 1.135397001576e+14 ||r(i)||/||b|| 2.156174052249e-03 >>>> 26 KSP preconditioned resid norm 6.874978587995e+14 true resid norm >>>> 1.225152011526e+14 ||r(i)||/||b|| 2.326623175549e-03 >>>> 27 KSP preconditioned resid norm 6.203646083349e+14 true resid norm >>>> 9.547951255722e+13 ||r(i)||/||b|| 1.813202317882e-03 >>>> 28 KSP preconditioned resid norm 5.655995299941e+14 true resid norm >>>> 9.635247011837e+13 ||r(i)||/||b|| 1.829780205963e-03 >>>> 29 KSP preconditioned resid norm 5.170668664072e+14 true resid norm >>>> 7.945307015395e+13 ||r(i)||/||b|| 1.508852392597e-03 >>>> 30 KSP preconditioned resid norm 4.771681972381e+14 true resid norm >>>> 7.711681382438e+13 ||r(i)||/||b|| 1.464485750178e-03 >>>> 31 KSP preconditioned resid norm 4.347723667483e+14 true resid norm >>>> 6.696963877348e+13 ||r(i)||/||b|| 1.271785967477e-03 >>>> 32 KSP preconditioned resid norm 4.053435837983e+14 true resid norm >>>> 6.425971653178e+13 ||r(i)||/||b|| 1.220323227897e-03 >>>> 33 KSP preconditioned resid norm 3.774514402447e+14 true resid norm >>>> 5.808746381526e+13 ||r(i)||/||b|| 1.103109150945e-03 >>>> 34 KSP preconditioned resid norm 3.533225997981e+14 true resid norm >>>> 5.617172311710e+13 ||r(i)||/||b|| 1.066728304612e-03 >>>> 0 KSP preconditioned resid norm 1.117659190837e+16 true resid norm >>>> 6.478857863821e+15 ||r(i)||/||b|| 6.224768870229e-01 >>>> 1 KSP preconditioned resid norm 5.124485271748e+15 true resid norm >>>> 1.017923791746e+15 ||r(i)||/||b|| 9.780026764454e-02 >>>> 2 KSP preconditioned resid norm 4.246328499709e+15 true resid norm >>>> 7.102918780385e+14 ||r(i)||/||b|| 6.824355255393e-02 >>>> 3 KSP preconditioned resid norm 4.210000472667e+15 true resid norm >>>> 7.697163273299e+14 ||r(i)||/||b|| 7.395294562682e-02 >>>> 4 KSP preconditioned resid norm 4.471596919519e+15 true resid norm >>>> 8.475637573508e+14 ||r(i)||/||b|| 8.143238520100e-02 >>>> 5 KSP preconditioned resid norm 5.135403291038e+15 true resid norm >>>> 8.255731593574e+14 ||r(i)||/||b|| 7.931956851781e-02 >>>> 6 KSP preconditioned resid norm 6.474268664840e+15 true resid norm >>>> 1.007942498042e+15 ||r(i)||/||b|| 9.684128308834e-02 >>>> 7 KSP preconditioned resid norm 8.529281532903e+15 true resid norm >>>> 1.642696055823e+15 ||r(i)||/||b|| 1.578272511369e-01 >>>> 8 KSP preconditioned resid norm 9.784577605090e+15 true resid norm >>>> 1.538078623395e+15 ||r(i)||/||b|| 1.477757983909e-01 >>>> 9 KSP preconditioned resid norm 1.010006302597e+16 true resid norm >>>> 2.046930405602e+15 ||r(i)||/||b|| 1.966653526923e-01 >>>> 10 KSP preconditioned resid norm 9.531884877717e+15 true resid norm >>>> 1.482620346274e+15 ||r(i)||/||b|| 1.424474679309e-01 >>>> 11 KSP preconditioned resid norm 7.931210548314e+15 true resid norm >>>> 1.282982557872e+15 ||r(i)||/||b|| 1.232666321002e-01 >>>> 12 KSP preconditioned resid norm 5.236526900965e+15 true resid norm >>>> 7.915538690853e+14 ||r(i)||/||b|| 7.605105694487e-02 >>>> 13 KSP preconditioned resid norm 3.885416517845e+15 true resid norm >>>> 6.196979959619e+14 ||r(i)||/||b|| 5.953945703529e-02 >>>> 14 KSP preconditioned resid norm 3.337626627490e+15 true resid norm >>>> 7.828597862542e+14 ||r(i)||/||b|| 7.521574526958e-02 >>>> 15 KSP preconditioned resid norm 3.054301414200e+15 true resid norm >>>> 4.972852337202e+14 ||r(i)||/||b|| 4.777826134714e-02 >>>> 16 KSP preconditioned resid norm 2.784835626497e+15 true resid norm >>>> 4.900149140724e+14 ||r(i)||/||b|| 4.707974225054e-02 >>>> 17 KSP preconditioned resid norm 2.551999131175e+15 true resid norm >>>> 4.223064940199e+14 ||r(i)||/||b|| 4.057444032459e-02 >>>> 18 KSP preconditioned resid norm 2.358735848485e+15 true resid norm >>>> 3.867160573888e+14 ||r(i)||/||b|| 3.715497586533e-02 >>>> 19 KSP preconditioned resid norm 2.111367549832e+15 true resid norm >>>> 3.463614183500e+14 ||r(i)||/||b|| 3.327777549857e-02 >>>> 20 KSP preconditioned resid norm 1.933287196882e+15 true resid norm >>>> 3.127201079650e+14 ||r(i)||/||b|| 3.004557954614e-02 >>>> 21 KSP preconditioned resid norm 1.712007284199e+15 true resid norm >>>> 2.787684847304e+14 ||r(i)||/||b|| 2.678356929917e-02 >>>> 22 KSP preconditioned resid norm 1.556420935486e+15 true resid norm >>>> 2.516757515483e+14 ||r(i)||/||b|| 2.418054874112e-02 >>>> 23 KSP preconditioned resid norm 1.424455851932e+15 true resid norm >>>> 2.267048417314e+14 ||r(i)||/||b|| 2.178138911521e-02 >>>> 24 KSP preconditioned resid norm 1.341093049266e+15 true resid norm >>>> 2.200423231461e+14 ||r(i)||/||b|| 2.114126643990e-02 >>>> 25 KSP preconditioned resid norm 1.265242012282e+15 true resid norm >>>> 1.979405460182e+14 ||r(i)||/||b|| 1.901776786755e-02 >>>> 26 KSP preconditioned resid norm 1.190343462827e+15 true resid norm >>>> 1.929502180424e+14 ||r(i)||/||b|| 1.853830622649e-02 >>>> 27 KSP preconditioned resid norm 1.080874612432e+15 true resid norm >>>> 1.688981747210e+14 ||r(i)||/||b|| 1.622742962325e-02 >>>> 28 KSP preconditioned resid norm 9.890419339482e+14 true resid norm >>>> 1.571112460256e+14 ||r(i)||/||b|| 1.509496293914e-02 >>>> 29 KSP preconditioned resid norm 9.054720544549e+14 true resid norm >>>> 1.411421952126e+14 ||r(i)||/||b|| 1.356068556375e-02 >>>> 30 KSP preconditioned resid norm 8.347895469778e+14 true resid norm >>>> 1.302020126144e+14 ||r(i)||/||b|| 1.250957270554e-02 >>>> 31 KSP preconditioned resid norm 7.581633555103e+14 true resid norm >>>> 1.198729260222e+14 ||r(i)||/||b|| 1.151717284080e-02 >>>> 32 KSP preconditioned resid norm 7.073149016328e+14 true resid norm >>>> 1.097657041094e+14 ||r(i)||/||b|| 1.054608933119e-02 >>>> 33 KSP preconditioned resid norm 6.626892210136e+14 true resid norm >>>> 1.045304064903e+14 ||r(i)||/||b|| 1.004309145207e-02 >>>> 34 KSP preconditioned resid norm 6.277118615935e+14 true resid norm >>>> 9.735143814138e+13 ||r(i)||/||b|| 9.353349222222e-03 >>>> 35 KSP preconditioned resid norm 6.007031548284e+14 true resid norm >>>> 9.325545854852e+13 ||r(i)||/||b|| 8.959814948147e-03 >>>> 36 KSP preconditioned resid norm 5.720600717487e+14 true resid norm >>>> 8.974565981239e+13 ||r(i)||/||b|| 8.622599865294e-03 >>>> 37 KSP preconditioned resid norm 5.429834406497e+14 true resid norm >>>> 8.373407335142e+13 ||r(i)||/||b|| 8.045017565303e-03 >>>> 38 KSP preconditioned resid norm 5.122271874419e+14 true resid norm >>>> 8.063599233502e+13 ||r(i)||/||b|| 7.747359572588e-03 >>>> 39 KSP preconditioned resid norm 4.877390474820e+14 true resid norm >>>> 7.540978455988e+13 ||r(i)||/||b|| 7.245235029160e-03 >>>> 40 KSP preconditioned resid norm 4.664511103016e+14 true resid norm >>>> 7.249582547116e+13 ||r(i)||/||b|| 6.965267136580e-03 >>>> 41 KSP preconditioned resid norm 4.430412921029e+14 true resid norm >>>> 6.907345898754e+13 ||r(i)||/||b|| 6.636452385624e-03 >>>> 42 KSP preconditioned resid norm 4.184290025581e+14 true resid norm >>>> 6.506826627971e+13 ||r(i)||/||b|| 6.251640750440e-03 >>>> 43 KSP preconditioned resid norm 3.961557540211e+14 true resid norm >>>> 6.169515842630e+13 ||r(i)||/||b|| 5.927558679138e-03 >>>> 44 KSP preconditioned resid norm 3.628856632080e+14 true resid norm >>>> 5.666822624927e+13 ||r(i)||/||b|| 5.444580172956e-03 >>>> 45 KSP preconditioned resid norm 3.439726178254e+14 true resid norm >>>> 5.290740915351e+13 ||r(i)||/||b|| 5.083247702381e-03 >>>> 46 KSP preconditioned resid norm 3.808130490503e+14 true resid norm >>>> 7.046303156753e+13 ||r(i)||/||b|| 6.769959993302e-03 >>>> 47 KSP preconditioned resid norm 2.548925306353e+14 true resid norm >>>> 3.911087819219e+13 ||r(i)||/||b|| 3.757702085387e-03 >>>> 48 KSP preconditioned resid norm 1.722230117103e+14 true resid norm >>>> 2.660198748286e+13 ||r(i)||/||b|| 2.555870603278e-03 >>>> 49 KSP preconditioned resid norm 8.498499404656e+13 true resid norm >>>> 1.314954336818e+13 ||r(i)||/||b|| 1.263384225066e-03 >>>> 50 KSP preconditioned resid norm 1.168755498979e+14 true resid norm >>>> 2.563042385986e+13 ||r(i)||/||b|| 2.462524536378e-03 >>>> 51 KSP preconditioned resid norm 5.069372366763e+13 true resid norm >>>> 7.752336990767e+12 ||r(i)||/||b|| 7.448304467540e-04 >>>> 0 KSP preconditioned resid norm 2.551555721110e+16 true resid norm >>>> 1.503348678660e+16 ||r(i)||/||b|| 1.139781099215e-01 >>>> 1 KSP preconditioned resid norm 1.135435821514e+16 true resid norm >>>> 2.150528625386e+15 ||r(i)||/||b|| 1.630448022690e-02 >>>> 2 KSP preconditioned resid norm 8.921910642422e+15 true resid norm >>>> 1.462019059252e+15 ||r(i)||/||b|| 1.108446572695e-02 >>>> 3 KSP preconditioned resid norm 8.401630743802e+15 true resid norm >>>> 1.508644008325e+15 ||r(i)||/||b|| 1.143795814332e-02 >>>> 4 KSP preconditioned resid norm 8.507938797573e+15 true resid norm >>>> 1.437822857483e+15 ||r(i)||/||b|| 1.090101943907e-02 >>>> 5 KSP preconditioned resid norm 9.195685552195e+15 true resid norm >>>> 1.421948919345e+15 ||r(i)||/||b|| 1.078066935052e-02 >>>> 6 KSP preconditioned resid norm 1.067998630800e+16 true resid norm >>>> 1.627316531548e+15 ||r(i)||/||b|| 1.233768753335e-02 >>>> 7 KSP preconditioned resid norm 1.295276293823e+16 true resid norm >>>> 2.221334900964e+15 ||r(i)||/||b|| 1.684130615262e-02 >>>> 8 KSP preconditioned resid norm 1.417997817727e+16 true resid norm >>>> 2.186215590047e+15 ||r(i)||/||b|| 1.657504505585e-02 >>>> 9 KSP preconditioned resid norm 1.359372910192e+16 true resid norm >>>> 2.107067442870e+15 ||r(i)||/||b|| 1.597497427073e-02 >>>> 10 KSP preconditioned resid norm 1.179811021994e+16 true resid norm >>>> 3.232410469947e+15 ||r(i)||/||b|| 2.450689191967e-02 >>>> 11 KSP preconditioned resid norm 7.523875759744e+15 true resid norm >>>> 1.204652282825e+15 ||r(i)||/||b|| 9.133209897209e-03 >>>> 12 KSP preconditioned resid norm 3.961884677960e+15 true resid norm >>>> 7.586571607759e+14 ||r(i)||/||b|| 5.751846560354e-03 >>>> 13 KSP preconditioned resid norm 2.175818220311e+15 true resid norm >>>> 3.487257134817e+14 ||r(i)||/||b|| 2.643904123366e-03 >>>> 14 KSP preconditioned resid norm 1.788448374434e+15 true resid norm >>>> 3.516091940967e+14 ||r(i)||/||b|| 2.665765563440e-03 >>>> 15 KSP preconditioned resid norm 1.604495215225e+15 true resid norm >>>> 2.568394922627e+14 ||r(i)||/||b|| 1.947258164179e-03 >>>> 16 KSP preconditioned resid norm 1.391089499920e+15 true resid norm >>>> 2.519771358465e+14 ||r(i)||/||b|| 1.910393649516e-03 >>>> 17 KSP preconditioned resid norm 1.231055803595e+15 true resid norm >>>> 2.082047844556e+14 ||r(i)||/||b|| 1.578528530720e-03 >>>> 18 KSP preconditioned resid norm 1.101068654831e+15 true resid norm >>>> 1.838589386072e+14 ||r(i)||/||b|| 1.393947699032e-03 >>>> 19 KSP preconditioned resid norm 9.356667448723e+14 true resid norm >>>> 1.565965285410e+14 ||r(i)||/||b|| 1.187254600129e-03 >>>> 20 KSP preconditioned resid norm 8.181513038456e+14 true resid norm >>>> 1.328185221533e+14 ||r(i)||/||b|| 1.006978908651e-03 >>>> 0 KSP preconditioned resid norm 2.272810589988e+16 true resid norm >>>> 1.323636987884e+16 ||r(i)||/||b|| 5.438005326146e-01 >>>> 1 KSP preconditioned resid norm 1.026969580669e+16 true resid norm >>>> 1.999385093393e+15 ||r(i)||/||b|| 8.214236143606e-02 >>>> 2 KSP preconditioned resid norm 8.322628960001e+15 true resid norm >>>> 1.379596337850e+15 ||r(i)||/||b|| 5.667907667911e-02 >>>> 3 KSP preconditioned resid norm 8.076035080201e+15 true resid norm >>>> 1.472961452514e+15 ||r(i)||/||b|| 6.051487150403e-02 >>>> 4 KSP preconditioned resid norm 8.405382352119e+15 true resid norm >>>> 1.516410203314e+15 ||r(i)||/||b|| 6.229991181666e-02 >>>> 5 KSP preconditioned resid norm 9.416373856303e+15 true resid norm >>>> 1.470293081560e+15 ||r(i)||/||b|| 6.040524465323e-02 >>>> 6 KSP preconditioned resid norm 1.145894310132e+16 true resid norm >>>> 1.749278072065e+15 ||r(i)||/||b|| 7.186701157399e-02 >>>> 7 KSP preconditioned resid norm 1.439168534642e+16 true resid norm >>>> 2.685145033101e+15 ||r(i)||/||b|| 1.103159939254e-01 >>>> 8 KSP preconditioned resid norm 1.556059742979e+16 true resid norm >>>> 2.450465189043e+15 ||r(i)||/||b|| 1.006744513151e-01 >>>> 9 KSP preconditioned resid norm 1.521075410830e+16 true resid norm >>>> 2.889962061237e+15 ||r(i)||/||b|| 1.187306582184e-01 >>>> 10 KSP preconditioned resid norm 1.335253374250e+16 true resid norm >>>> 2.051263914753e+15 ||r(i)||/||b|| 8.427374118333e-02 >>>> 11 KSP preconditioned resid norm 9.966633192923e+15 true resid norm >>>> 1.537352185287e+15 ||r(i)||/||b|| 6.316028826844e-02 >>>> 12 KSP preconditioned resid norm 5.467457389741e+15 true resid norm >>>> 8.244964108686e+14 ||r(i)||/||b|| 3.387345559797e-02 >>>> 13 KSP preconditioned resid norm 7.465607339270e+15 true resid norm >>>> 1.255988023601e+15 ||r(i)||/||b|| 5.160077592599e-02 >>>> 14 KSP preconditioned resid norm 3.395725692367e+15 true resid norm >>>> 8.163584192182e+14 ||r(i)||/||b|| 3.353911587836e-02 >>>> 15 KSP preconditioned resid norm 2.979451058224e+15 true resid norm >>>> 4.736799198293e+14 ||r(i)||/||b|| 1.946057680844e-02 >>>> 16 KSP preconditioned resid norm 2.589762095203e+15 true resid norm >>>> 5.000061827618e+14 ||r(i)||/||b|| 2.054216004731e-02 >>>> 17 KSP preconditioned resid norm 2.297387399767e+15 true resid norm >>>> 3.733072000155e+14 ||r(i)||/||b|| 1.533688285047e-02 >>>> 18 KSP preconditioned resid norm 2.076822069911e+15 true resid norm >>>> 3.563941616114e+14 ||r(i)||/||b|| 1.464203075912e-02 >>>> 19 KSP preconditioned resid norm 1.810897092306e+15 true resid norm >>>> 2.945299484347e+14 ||r(i)||/||b|| 1.210041305100e-02 >>>> 20 KSP preconditioned resid norm 1.630399807803e+15 true resid norm >>>> 2.670493324597e+14 ||r(i)||/||b|| 1.097140458866e-02 >>>> 21 KSP preconditioned resid norm 1.416015877821e+15 true resid norm >>>> 2.289562555093e+14 ||r(i)||/||b|| 9.406395773998e-03 >>>> 22 KSP preconditioned resid norm 1.270953330458e+15 true resid norm >>>> 2.078621859272e+14 ||r(i)||/||b|| 8.539770983461e-03 >>>> 23 KSP preconditioned resid norm 1.149572826465e+15 true resid norm >>>> 1.837793084226e+14 ||r(i)||/||b|| 7.550354569913e-03 >>>> 24 KSP preconditioned resid norm 1.075403245776e+15 true resid norm >>>> 1.776412893147e+14 ||r(i)||/||b|| 7.298181346390e-03 >>>> 25 KSP preconditioned resid norm 1.008390645992e+15 true resid norm >>>> 1.577927704554e+14 ||r(i)||/||b|| 6.482728527673e-03 >>>> 26 KSP preconditioned resid norm 9.445856732256e+14 true resid norm >>>> 1.515432067895e+14 ||r(i)||/||b|| 6.225972628492e-03 >>>> 27 KSP preconditioned resid norm 8.526344464253e+14 true resid norm >>>> 1.348203382746e+14 ||r(i)||/||b|| 5.538933441126e-03 >>>> 28 KSP preconditioned resid norm 7.744588594030e+14 true resid norm >>>> 1.215509014949e+14 ||r(i)||/||b|| 4.993774394172e-03 >>>> 29 KSP preconditioned resid norm 7.032530926217e+14 true resid norm >>>> 1.102818922582e+14 ||r(i)||/||b|| 4.530800536453e-03 >>>> 30 KSP preconditioned resid norm 6.388922318141e+14 true resid norm >>>> 9.838037383987e+13 ||r(i)||/||b|| 4.041840790388e-03 >>>> 31 KSP preconditioned resid norm 5.692116056101e+14 true resid norm >>>> 9.317411265002e+13 ||r(i)||/||b|| 3.827947734067e-03 >>>> 32 KSP preconditioned resid norm 5.208032879647e+14 true resid norm >>>> 8.007380640801e+13 ||r(i)||/||b|| 3.289737214338e-03 >>>> 33 KSP preconditioned resid norm 4.765676071483e+14 true resid norm >>>> 7.802986158610e+13 ||r(i)||/||b|| 3.205764169389e-03 >>>> 34 KSP preconditioned resid norm 4.409793613234e+14 true resid norm >>>> 6.791885197478e+13 ||r(i)||/||b|| 2.790365350662e-03 >>>> 35 KSP preconditioned resid norm 4.123618028942e+14 true resid norm >>>> 6.437750648670e+13 ||r(i)||/||b|| 2.644873378149e-03 >>>> 36 KSP preconditioned resid norm 3.827034276493e+14 true resid norm >>>> 5.974308940262e+13 ||r(i)||/||b|| 2.454473857604e-03 >>>> 37 KSP preconditioned resid norm 3.509107846575e+14 true resid norm >>>> 5.419166518671e+13 ||r(i)||/||b|| 2.226400188387e-03 >>>> 38 KSP preconditioned resid norm 3.170601404687e+14 true resid norm >>>> 4.972345072887e+13 ||r(i)||/||b|| 2.042828905305e-03 >>>> 39 KSP preconditioned resid norm 2.901553052614e+14 true resid norm >>>> 4.518683026559e+13 ||r(i)||/||b|| 1.856447242750e-03 >>>> 40 KSP preconditioned resid norm 2.667015211684e+14 true resid norm >>>> 4.122270495295e+13 ||r(i)||/||b|| 1.693585863377e-03 >>>> 41 KSP preconditioned resid norm 2.422095249996e+14 true resid norm >>>> 3.839959453856e+13 ||r(i)||/||b|| 1.577601725654e-03 >>>> 42 KSP preconditioned resid norm 2.195894402502e+14 true resid norm >>>> 3.399029120498e+13 ||r(i)||/||b|| 1.396450736130e-03 >>>> 43 KSP preconditioned resid norm 1.983452570770e+14 true resid norm >>>> 3.143816040870e+13 ||r(i)||/||b|| 1.291599474113e-03 >>>> 44 KSP preconditioned resid norm 1.779873290535e+14 true resid norm >>>> 2.755766172822e+13 ||r(i)||/||b|| 1.132173795580e-03 >>>> 45 KSP preconditioned resid norm 1.542400485092e+14 true resid norm >>>> 2.400274515345e+13 ||r(i)||/||b|| 9.861242710910e-04 >>>> 46 KSP preconditioned resid norm 1.223281336233e+14 true resid norm >>>> 1.907355666953e+13 ||r(i)||/||b|| 7.836144177513e-04 >>>> 0 KSP preconditioned resid norm 7.586062313136e+16 true resid norm >>>> 4.608637352265e+16 ||r(i)||/||b|| 6.590757061047e-02 >>>> 1 KSP preconditioned resid norm 2.763422737801e+16 true resid norm >>>> 5.227955984017e+15 ||r(i)||/||b|| 7.476437216213e-03 >>>> 2 KSP preconditioned resid norm 1.553456028419e+16 true resid norm >>>> 2.906104653716e+15 ||r(i)||/||b|| 4.155985447023e-03 >>>> 3 KSP preconditioned resid norm 1.040767126705e+16 true resid norm >>>> 2.091654777018e+15 ||r(i)||/||b|| 2.991250436342e-03 >>>> 4 KSP preconditioned resid norm 7.531366829970e+15 true resid norm >>>> 1.318758653617e+15 ||r(i)||/||b|| 1.885940950392e-03 >>>> 5 KSP preconditioned resid norm 5.773017012836e+15 true resid norm >>>> 9.450385953724e+14 ||r(i)||/||b|| 1.351488372664e-03 >>>> 6 KSP preconditioned resid norm 4.731507925306e+15 true resid norm >>>> 7.935497854840e+14 ||r(i)||/||b|| 1.134846040642e-03 >>>> 7 KSP preconditioned resid norm 3.788639277852e+15 true resid norm >>>> 7.524409398539e+14 ||r(i)||/||b|| 1.076056773034e-03 >>>> 8 KSP preconditioned resid norm 2.738698664678e+15 true resid norm >>>> 4.915430614067e+14 ||r(i)||/||b|| 7.029498429037e-04 >>>> 9 KSP preconditioned resid norm 1.837368414949e+15 true resid norm >>>> 3.357340764188e+14 ||r(i)||/||b|| 4.801292802316e-04 >>>> 10 KSP preconditioned resid norm 1.059331899595e+15 true resid norm >>>> 1.989071439556e+14 ||r(i)||/||b|| 2.844547234496e-04 >>>> 11 KSP preconditioned resid norm 4.945892712172e+14 true resid norm >>>> 8.452757650210e+13 ||r(i)||/||b|| 1.208818744245e-04 >>>> 12 KSP preconditioned resid norm 1.987888303244e+14 true resid norm >>>> 3.594134462521e+13 ||r(i)||/||b|| 5.139928633259e-05 >>>> 13 KSP preconditioned resid norm 1.163784016975e+14 true resid norm >>>> 2.571324386412e+13 ||r(i)||/||b|| 3.677220197779e-05 >>>> 14 KSP preconditioned resid norm 8.494315446492e+13 true resid norm >>>> 1.429434206797e+13 ||r(i)||/||b|| 2.044216732982e-05 >>>> 15 KSP preconditioned resid norm 8.642186180523e+13 true resid norm >>>> 1.607306715480e+13 ||r(i)||/||b|| 2.298590076546e-05 >>>> 16 KSP preconditioned resid norm 3.925619743667e+13 true resid norm >>>> 1.371348923807e+13 ||r(i)||/||b|| 1.961149665703e-05 >>>> 0 KSP preconditioned resid norm 3.953744769406e+16 true resid norm >>>> 2.171194598493e+16 ||r(i)||/||b|| 1.950905346919e-01 >>>> 1 KSP preconditioned resid norm 1.491502757726e+16 true resid norm >>>> 3.091776185716e+15 ||r(i)||/||b|| 2.778084790914e-02 >>>> 2 KSP preconditioned resid norm 1.008617531922e+16 true resid norm >>>> 1.691017660922e+15 ||r(i)||/||b|| 1.519447127732e-02 >>>> 3 KSP preconditioned resid norm 8.326895039450e+15 true resid norm >>>> 1.670298498749e+15 ||r(i)||/||b|| 1.500830130298e-02 >>>> 4 KSP preconditioned resid norm 7.300566603499e+15 true resid norm >>>> 1.589496633697e+15 ||r(i)||/||b|| 1.428226416804e-02 >>>> 5 KSP preconditioned resid norm 6.872352176548e+15 true resid norm >>>> 1.096739362492e+15 ||r(i)||/||b|| 9.854642637499e-03 >>>> 6 KSP preconditioned resid norm 6.961475959227e+15 true resid norm >>>> 1.092150928708e+15 ||r(i)||/||b|| 9.813413721358e-03 >>>> 7 KSP preconditioned resid norm 7.025042419735e+15 true resid norm >>>> 1.296288429690e+15 ||r(i)||/||b|| 1.164767096595e-02 >>>> 8 KSP preconditioned resid norm 6.408702665765e+15 true resid norm >>>> 1.056644614043e+15 ||r(i)||/||b|| 9.494375256645e-03 >>>> 9 KSP preconditioned resid norm 5.399106179995e+15 true resid norm >>>> 1.078818901202e+15 ||r(i)||/||b|| 9.693620112044e-03 >>>> 10 KSP preconditioned resid norm 4.030707639439e+15 true resid norm >>>> 7.107969460630e+14 ||r(i)||/||b|| 6.386795378039e-03 >>>> 11 KSP preconditioned resid norm 2.682586389004e+15 true resid norm >>>> 4.368532286577e+14 ||r(i)||/||b|| 3.925301307393e-03 >>>> 12 KSP preconditioned resid norm 1.621400972048e+15 true resid norm >>>> 2.499302020485e+14 ||r(i)||/||b|| 2.245723012904e-03 >>>> 13 KSP preconditioned resid norm 1.118646687359e+15 true resid norm >>>> 1.825993809337e+14 ||r(i)||/||b|| 1.640728605602e-03 >>>> 14 KSP preconditioned resid norm 8.665732278929e+14 true resid norm >>>> 1.337846645968e+14 ||r(i)||/||b|| 1.202108819167e-03 >>>> 15 KSP preconditioned resid norm 1.790412083411e+15 true resid norm >>>> 2.690613625509e+14 ||r(i)||/||b|| 2.417624155910e-03 >>>> 16 KSP preconditioned resid norm 4.902192144952e+14 true resid norm >>>> 8.615757491331e+13 ||r(i)||/||b|| 7.741603340972e-04 >>>> 17 KSP preconditioned resid norm 3.873085562421e+14 true resid norm >>>> 6.966874756362e+13 ||r(i)||/||b|| 6.260016132563e-04 >>>> 18 KSP preconditioned resid norm 3.069894649217e+14 true resid norm >>>> 4.954148501038e+13 ||r(i)||/||b|| 4.451500941838e-04 >>>> 19 KSP preconditioned resid norm 2.366855243734e+14 true resid norm >>>> 4.444373941350e+13 ||r(i)||/||b|| 3.993448073197e-04 >>>> 20 KSP preconditioned resid norm 1.853280795186e+14 true resid norm >>>> 2.839442050460e+13 ||r(i)||/||b|| 2.551352459311e-04 >>>> 21 KSP preconditioned resid norm 1.419103820891e+14 true resid norm >>>> 2.714177608773e+13 ||r(i)||/||b|| 2.438797339086e-04 >>>> 22 KSP preconditioned resid norm 1.116662401807e+14 true resid norm >>>> 1.710034207306e+13 ||r(i)||/||b|| 1.536534256654e-04 >>>> 23 KSP preconditioned resid norm 8.768706858701e+13 true resid norm >>>> 1.892684059360e+13 ||r(i)||/||b|| 1.700652467538e-04 >>>> 24 KSP preconditioned resid norm 7.191985915577e+13 true resid norm >>>> 1.104970014738e+13 ||r(i)||/||b|| 9.928598345972e-05 >>>> 25 KSP preconditioned resid norm 5.804472664630e+13 true resid norm >>>> 1.353159760302e+13 ||r(i)||/||b|| 1.215868265996e-04 >>>> 26 KSP preconditioned resid norm 4.574262022619e+13 true resid norm >>>> 6.915191476157e+12 ||r(i)||/||b|| 6.213576634340e-05 >>>> 27 KSP preconditioned resid norm 3.470073834736e+13 true resid norm >>>> 8.786634213858e+12 ||r(i)||/||b|| 7.895142923224e-05 >>>> 28 KSP preconditioned resid norm 2.674822501624e+13 true resid norm >>>> 4.041336605553e+12 ||r(i)||/||b|| 3.631302877201e-05 >>>> 29 KSP preconditioned resid norm 2.088626286399e+13 true resid norm >>>> 5.261273540428e+12 ||r(i)||/||b|| 4.727465096287e-05 >>>> 30 KSP preconditioned resid norm 1.567496199145e+13 true resid norm >>>> 2.371012817670e+12 ||r(i)||/||b|| 2.130450023603e-05 >>>> 31 KSP preconditioned resid norm 1.119841843422e+13 true resid norm >>>> 3.387071792996e+12 ||r(i)||/||b|| 3.043419726606e-05 >>>> 32 KSP preconditioned resid norm 7.994377199528e+12 true resid norm >>>> 1.204577998308e+12 ||r(i)||/||b|| 1.082361599145e-05 >>>> 33 KSP preconditioned resid norm 6.174157756589e+12 true resid norm >>>> 1.362745552188e+12 ||r(i)||/||b|| 1.224481484110e-05 >>>> 0 SNES Function norm 0.000000000000e+00 >>>> 1 TS dt 3e-12 time 2e-12 >>>> 0 KSP preconditioned resid norm 2.220368634013e+19 true resid norm >>>> 1.204306153911e+19 ||r(i)||/||b|| 2.302397943981e+02 >>>> 1 KSP preconditioned resid norm 5.390402695434e+18 true resid norm >>>> 1.775960718595e+18 ||r(i)||/||b|| 3.395289722473e+01 >>>> 2 KSP preconditioned resid norm 2.218493119258e+18 true resid norm >>>> 5.320083037595e+17 ||r(i)||/||b|| 1.017095877804e+01 >>>> 3 KSP preconditioned resid norm 1.401798813920e+18 true resid norm >>>> 2.963744523802e+17 ||r(i)||/||b|| 5.666100165582e+00 >>>> 4 KSP preconditioned resid norm 1.004143331118e+18 true resid norm >>>> 1.656359344976e+17 ||r(i)||/||b|| 3.166635276239e+00 >>>> 5 KSP preconditioned resid norm 8.673495646713e+17 true resid norm >>>> 1.510463721187e+17 ||r(i)||/||b|| 2.887711363779e+00 >>>> 6 KSP preconditioned resid norm 7.389956702949e+17 true resid norm >>>> 1.526304376340e+17 ||r(i)||/||b|| 2.917995599842e+00 >>>> 7 KSP preconditioned resid norm 5.566613599927e+17 true resid norm >>>> 1.017416269196e+17 ||r(i)||/||b|| 1.945101018344e+00 >>>> 8 KSP preconditioned resid norm 3.566897773081e+17 true resid norm >>>> 6.307244640523e+16 ||r(i)||/||b|| 1.205821878877e+00 >>>> 9 KSP preconditioned resid norm 1.893453815442e+17 true resid norm >>>> 3.268203516935e+16 ||r(i)||/||b|| 6.248166243663e-01 >>>> 10 KSP preconditioned resid norm 6.894933140555e+16 true resid norm >>>> 1.152954614680e+16 ||r(i)||/||b|| 2.204223839364e-01 >>>> 11 KSP preconditioned resid norm 5.103537427509e+16 true resid norm >>>> 9.272910080166e+15 ||r(i)||/||b|| 1.772799136993e-01 >>>> 12 KSP preconditioned resid norm 4.991235858989e+16 true resid norm >>>> 8.396555957121e+15 ||r(i)||/||b|| 1.605257359967e-01 >>>> 13 KSP preconditioned resid norm 4.918689912532e+16 true resid norm >>>> 8.423517310572e+15 ||r(i)||/||b|| 1.610411843696e-01 >>>> 14 KSP preconditioned resid norm 5.686949456502e+16 true resid norm >>>> 9.761258523981e+15 ||r(i)||/||b|| 1.866161813032e-01 >>>> 15 KSP preconditioned resid norm 1.616570395103e+17 true resid norm >>>> 4.351134639515e+16 ||r(i)||/||b|| 8.318518854587e-01 >>>> 16 KSP preconditioned resid norm 4.128330778786e+16 true resid norm >>>> 7.104207079385e+15 ||r(i)||/||b|| 1.358185517866e-01 >>>> 17 KSP preconditioned resid norm 2.161181135112e+16 true resid norm >>>> 5.859480309696e+15 ||r(i)||/||b|| 1.120218091889e-01 >>>> 18 KSP preconditioned resid norm 1.410375919962e+16 true resid norm >>>> 2.425065017616e+15 ||r(i)||/||b|| 4.636250252850e-02 >>>> 19 KSP preconditioned resid norm 1.082544604615e+16 true resid norm >>>> 2.485306543509e+15 ||r(i)||/||b|| 4.751420274116e-02 >>>> 20 KSP preconditioned resid norm 8.996941383859e+15 true resid norm >>>> 1.577914541048e+15 ||r(i)||/||b|| 3.016664145813e-02 >>>> 21 KSP preconditioned resid norm 7.748060569447e+15 true resid norm >>>> 1.336297549587e+15 ||r(i)||/||b|| 2.554739690340e-02 >>>> 22 KSP preconditioned resid norm 6.873596381580e+15 true resid norm >>>> 1.173018262854e+15 ||r(i)||/||b|| 2.242581612555e-02 >>>> 23 KSP preconditioned resid norm 6.175955090548e+15 true resid norm >>>> 1.012331331012e+15 ||r(i)||/||b|| 1.935379610558e-02 >>>> 24 KSP preconditioned resid norm 5.614288705672e+15 true resid norm >>>> 9.369191861544e+14 ||r(i)||/||b|| 1.791206331440e-02 >>>> 25 KSP preconditioned resid norm 5.220540757763e+15 true resid norm >>>> 8.444260437985e+14 ||r(i)||/||b|| 1.614377524163e-02 >>>> 26 KSP preconditioned resid norm 5.059721722058e+15 true resid norm >>>> 8.452379209101e+14 ||r(i)||/||b|| 1.615929674492e-02 >>>> 27 KSP preconditioned resid norm 4.893746226621e+15 true resid norm >>>> 9.389168158486e+14 ||r(i)||/||b|| 1.795025408912e-02 >>>> 28 KSP preconditioned resid norm 4.794914716345e+15 true resid norm >>>> 7.493396637157e+14 ||r(i)||/||b|| 1.432590953288e-02 >>>> 29 KSP preconditioned resid norm 4.860853428688e+15 true resid norm >>>> 7.693465766642e+14 ||r(i)||/||b|| 1.470840259819e-02 >>>> 30 KSP preconditioned resid norm 5.112203028306e+15 true resid norm >>>> 8.019578406020e+14 ||r(i)||/||b|| 1.533186621496e-02 >>>> 31 KSP preconditioned resid norm 5.649006086453e+15 true resid norm >>>> 8.938106702489e+14 ||r(i)||/||b|| 1.708791276044e-02 >>>> 32 KSP preconditioned resid norm 8.383819984522e+15 true resid norm >>>> 1.347106201218e+15 ||r(i)||/||b|| 2.575403719341e-02 >>>> 33 KSP preconditioned resid norm 1.617520068743e+16 true resid norm >>>> 2.603058585140e+15 ||r(i)||/||b|| 4.976539159104e-02 >>>> 34 KSP preconditioned resid norm 3.383001122808e+16 true resid norm >>>> 5.541435636245e+15 ||r(i)||/||b|| 1.059414167582e-01 >>>> 35 KSP preconditioned resid norm 6.256166371964e+15 true resid norm >>>> 1.029049029941e+15 ||r(i)||/||b|| 1.967340582872e-02 >>>> 36 KSP preconditioned resid norm 3.290396673886e+15 true resid norm >>>> 5.409340387326e+14 ||r(i)||/||b|| 1.034160138236e-02 >>>> 37 KSP preconditioned resid norm 2.075346751579e+15 true resid norm >>>> 3.809761571158e+14 ||r(i)||/||b|| 7.283519377531e-03 >>>> 38 KSP preconditioned resid norm 1.750045982315e+15 true resid norm >>>> 2.837816436037e+14 ||r(i)||/||b|| 5.425350278670e-03 >>>> 39 KSP preconditioned resid norm 1.596755323762e+15 true resid norm >>>> 2.766061504317e+14 ||r(i)||/||b|| 5.288168876146e-03 >>>> 40 KSP preconditioned resid norm 1.469812007135e+15 true resid norm >>>> 2.359521384401e+14 ||r(i)||/||b|| 4.510943638858e-03 >>>> 41 KSP preconditioned resid norm 1.314668041488e+15 true resid norm >>>> 2.158813968118e+14 ||r(i)||/||b|| 4.127230293966e-03 >>>> 42 KSP preconditioned resid norm 1.190446489712e+15 true resid norm >>>> 1.941922420158e+14 ||r(i)||/||b|| 3.712576053042e-03 >>>> 43 KSP preconditioned resid norm 1.100371263474e+15 true resid norm >>>> 1.837040226442e+14 ||r(i)||/||b|| 3.512061801423e-03 >>>> 44 KSP preconditioned resid norm 1.032027622009e+15 true resid norm >>>> 1.624908335172e+14 ||r(i)||/||b|| 3.106507093655e-03 >>>> 45 KSP preconditioned resid norm 8.066010465859e+14 true resid norm >>>> 2.499636130742e+14 ||r(i)||/||b|| 4.778815643705e-03 >>>> 46 KSP preconditioned resid norm 6.546087590187e+14 true resid norm >>>> 1.014535455422e+14 ||r(i)||/||b|| 1.939593465560e-03 >>>> 47 KSP preconditioned resid norm 6.561404017041e+14 true resid norm >>>> 1.083911025925e+14 ||r(i)||/||b|| 2.072225994564e-03 >>>> 48 KSP preconditioned resid norm 6.834623592895e+14 true resid norm >>>> 1.073483632090e+14 ||r(i)||/||b|| 2.052290855937e-03 >>>> 49 KSP preconditioned resid norm 8.808387787841e+14 true resid norm >>>> 1.369903526339e+14 ||r(i)||/||b|| 2.618987748466e-03 >>>> 50 KSP preconditioned resid norm 3.995486126369e+14 true resid norm >>>> 1.237415046233e+14 ||r(i)||/||b|| 2.365695673849e-03 >>>> 51 KSP preconditioned resid norm 3.217227010099e+14 true resid norm >>>> 4.935649374236e+13 ||r(i)||/||b|| 9.435996764234e-04 >>>> 0 KSP preconditioned resid norm 6.543260687832e+18 true resid norm >>>> 3.546137973950e+18 ||r(i)||/||b|| 4.156987252753e+02 >>>> 1 KSP preconditioned resid norm 1.591265056482e+18 true resid norm >>>> 5.223494588784e+17 ||r(i)||/||b|| 6.123281321796e+01 >>>> 2 KSP preconditioned resid norm 6.570987793466e+17 true resid norm >>>> 1.552598280892e+17 ||r(i)||/||b|| 1.820045161730e+01 >>>> 3 KSP preconditioned resid norm 4.166120135488e+17 true resid norm >>>> 8.739095834512e+16 ||r(i)||/||b|| 1.024447166228e+01 >>>> 4 KSP preconditioned resid norm 2.995238174583e+17 true resid norm >>>> 4.935051588719e+16 ||r(i)||/||b|| 5.785151817748e+00 >>>> 5 KSP preconditioned resid norm 2.594691834001e+17 true resid norm >>>> 4.470128066210e+16 ||r(i)||/||b|| 5.240141676921e+00 >>>> 6 KSP preconditioned resid norm 2.219773011977e+17 true resid norm >>>> 4.652509796902e+16 ||r(i)||/||b|| 5.453940050023e+00 >>>> 7 KSP preconditioned resid norm 1.675026903695e+17 true resid norm >>>> 2.988993956885e+16 ||r(i)||/||b|| 3.503870934691e+00 >>>> 8 KSP preconditioned resid norm 1.081971008730e+17 true resid norm >>>> 1.908275119338e+16 ||r(i)||/||b|| 2.236990044975e+00 >>>> 9 KSP preconditioned resid norm 5.710427760412e+16 true resid norm >>>> 9.635681698385e+15 ||r(i)||/||b|| 1.129550127097e+00 >>>> 10 KSP preconditioned resid norm 1.779335357179e+16 true resid norm >>>> 2.952424081259e+15 ||r(i)||/||b|| 3.461001619416e-01 >>>> 11 KSP preconditioned resid norm 1.125492898955e+16 true resid norm >>>> 2.030196257790e+15 ||r(i)||/||b|| 2.379913028263e-01 >>>> 12 KSP preconditioned resid norm 1.086499382749e+16 true resid norm >>>> 1.877721745252e+15 ||r(i)||/||b|| 2.201173619461e-01 >>>> 13 KSP preconditioned resid norm 1.075258231290e+16 true resid norm >>>> 1.822442508498e+15 ||r(i)||/||b|| 2.136372113085e-01 >>>> 14 KSP preconditioned resid norm 1.173739259962e+16 true resid norm >>>> 2.130298543993e+15 ||r(i)||/||b|| 2.497258695795e-01 >>>> 15 KSP preconditioned resid norm 4.943363278431e+16 true resid norm >>>> 1.361259414632e+16 ||r(i)||/||b|| 1.595746718229e+00 >>>> 16 KSP preconditioned resid norm 1.782643390767e+17 true resid norm >>>> 3.375538781586e+16 ||r(i)||/||b|| 3.957001049961e+00 >>>> 17 KSP preconditioned resid norm 3.253381322946e+16 true resid norm >>>> 8.720742251900e+15 ||r(i)||/||b|| 1.022295653525e+00 >>>> 18 KSP preconditioned resid norm 2.077141437527e+16 true resid norm >>>> 3.948414579285e+15 ||r(i)||/||b|| 4.628559067708e-01 >>>> 19 KSP preconditioned resid norm 1.484650052668e+16 true resid norm >>>> 3.391990202104e+15 ||r(i)||/||b|| 3.976286352982e-01 >>>> 20 KSP preconditioned resid norm 1.135224344133e+16 true resid norm >>>> 2.492450462789e+15 ||r(i)||/||b|| 2.921794041305e-01 >>>> 21 KSP preconditioned resid norm 8.560056764067e+15 true resid norm >>>> 1.658566536392e+15 ||r(i)||/||b|| 1.944267256456e-01 >>>> 22 KSP preconditioned resid norm 6.591497757947e+15 true resid norm >>>> 1.287448297256e+15 ||r(i)||/||b|| 1.509221073626e-01 >>>> 23 KSP preconditioned resid norm 4.783676751639e+15 true resid norm >>>> 9.069671731055e+14 ||r(i)||/||b|| 1.063199177517e-01 >>>> 24 KSP preconditioned resid norm 3.029245298062e+15 true resid norm >>>> 6.184462851213e+14 ||r(i)||/||b|| 7.249783687627e-02 >>>> 25 KSP preconditioned resid norm 1.567581368038e+15 true resid norm >>>> 2.751547089998e+14 ||r(i)||/||b|| 3.225522036225e-02 >>>> 26 KSP preconditioned resid norm 1.174427256669e+15 true resid norm >>>> 2.270458347820e+14 ||r(i)||/||b|| 2.661562093503e-02 >>>> 27 KSP preconditioned resid norm 1.068689317098e+15 true resid norm >>>> 2.026010487900e+14 ||r(i)||/||b|| 2.375006227624e-02 >>>> 28 KSP preconditioned resid norm 1.006302654001e+15 true resid norm >>>> 1.659657859887e+14 ||r(i)||/||b|| 1.945546568735e-02 >>>> 29 KSP preconditioned resid norm 9.398219810967e+14 true resid norm >>>> 1.788496139471e+14 ||r(i)||/||b|| 2.096578223407e-02 >>>> 30 KSP preconditioned resid norm 8.922669862958e+14 true resid norm >>>> 1.448103190511e+14 ||r(i)||/||b|| 1.697549996037e-02 >>>> 31 KSP preconditioned resid norm 8.611553511040e+14 true resid norm >>>> 1.569494235094e+14 ||r(i)||/||b|| 1.839851572749e-02 >>>> 32 KSP preconditioned resid norm 8.275858647114e+14 true resid norm >>>> 1.327021986732e+14 ||r(i)||/||b|| 1.555611632568e-02 >>>> 33 KSP preconditioned resid norm 7.965262562283e+14 true resid norm >>>> 1.578752878362e+14 ||r(i)||/||b|| 1.850705087847e-02 >>>> 34 KSP preconditioned resid norm 7.584298711071e+14 true resid norm >>>> 1.400338111929e+14 ||r(i)||/||b|| 1.641557018817e-02 >>>> 35 KSP preconditioned resid norm 7.453216416858e+14 true resid norm >>>> 1.168697043482e+14 ||r(i)||/||b|| 1.370014011799e-02 >>>> 36 KSP preconditioned resid norm 7.552391370873e+14 true resid norm >>>> 1.200616262811e+14 ||r(i)||/||b|| 1.407431559803e-02 >>>> 37 KSP preconditioned resid norm 7.900785942536e+14 true resid norm >>>> 1.248526226554e+14 ||r(i)||/||b|| 1.463594379756e-02 >>>> 38 KSP preconditioned resid norm 8.292853068973e+14 true resid norm >>>> 1.319608976445e+14 ||r(i)||/||b|| 1.546921674790e-02 >>>> 39 KSP preconditioned resid norm 8.694401463969e+14 true resid norm >>>> 1.375708051207e+14 ||r(i)||/||b|| 1.612684242516e-02 >>>> 40 KSP preconditioned resid norm 9.345453545993e+14 true resid norm >>>> 1.486706547594e+14 ||r(i)||/||b|| 1.742803075440e-02 >>>> 41 KSP preconditioned resid norm 1.186246552213e+15 true resid norm >>>> 1.878863391090e+14 ||r(i)||/||b|| 2.202511922492e-02 >>>> 42 KSP preconditioned resid norm 2.140906233315e+15 true resid norm >>>> 3.526342198473e+14 ||r(i)||/||b|| 4.133781504155e-02 >>>> 43 KSP preconditioned resid norm 1.599648547908e+16 true resid norm >>>> 2.542842342413e+15 ||r(i)||/||b|| 2.980866306056e-01 >>>> 44 KSP preconditioned resid norm 1.201331553169e+15 true resid norm >>>> 1.958292468332e+14 ||r(i)||/||b|| 2.295623263342e-02 >>>> 45 KSP preconditioned resid norm 6.227596295745e+14 true resid norm >>>> 9.990152835185e+13 ||r(i)||/||b|| 1.171103276127e-02 >>>> 46 KSP preconditioned resid norm 4.228157632423e+14 true resid norm >>>> 6.914757907633e+13 ||r(i)||/||b|| 8.105877630555e-03 >>>> 47 KSP preconditioned resid norm 2.986102023248e+14 true resid norm >>>> 4.782856914894e+13 ||r(i)||/||b|| 5.606740452011e-03 >>>> 48 KSP preconditioned resid norm 1.831310965020e+14 true resid norm >>>> 3.038059690813e+13 ||r(i)||/||b|| 3.561388615047e-03 >>>> 49 KSP preconditioned resid norm 1.519167951348e+14 true resid norm >>>> 2.657532215634e+13 ||r(i)||/||b|| 3.115312383591e-03 >>>> 50 KSP preconditioned resid norm 1.784468495322e+14 true resid norm >>>> 2.872094801120e+13 ||r(i)||/||b|| 3.366835008863e-03 >>>> 51 KSP preconditioned resid norm 1.082269784640e+14 true resid norm >>>> 1.734196241756e+13 ||r(i)||/||b|| 2.032924754679e-03 >>>> 52 KSP preconditioned resid norm 7.941561433572e+13 true resid norm >>>> 2.262252591710e+13 ||r(i)||/||b|| 2.651942833395e-03 >>>> 53 KSP preconditioned resid norm 1.279466501710e+14 true resid norm >>>> 2.100188573520e+13 ||r(i)||/||b|| 2.461962053545e-03 >>>> 54 KSP preconditioned resid norm 1.148448766150e+14 true resid norm >>>> 1.892568589516e+13 ||r(i)||/||b|| 2.218577945746e-03 >>>> 55 KSP preconditioned resid norm 4.111681106774e+13 true resid norm >>>> 1.052257112253e+13 ||r(i)||/||b|| 1.233516415432e-03 >>>> 0 KSP preconditioned resid norm 1.892564880723e+19 true resid norm >>>> 1.028289393047e+19 ||r(i)||/||b|| 7.840877673877e+01 >>>> 1 KSP preconditioned resid norm 4.581010831586e+18 true resid norm >>>> 1.519789346059e+18 ||r(i)||/||b|| 1.158864657468e+01 >>>> 2 KSP preconditioned resid norm 1.874273717290e+18 true resid norm >>>> 4.647134076709e+17 ||r(i)||/||b|| 3.543517036739e+00 >>>> 3 KSP preconditioned resid norm 1.173044525535e+18 true resid norm >>>> 2.550993039028e+17 ||r(i)||/||b|| 1.945174627025e+00 >>>> 4 KSP preconditioned resid norm 8.299292116891e+17 true resid norm >>>> 1.381054316961e+17 ||r(i)||/||b|| 1.053076889978e+00 >>>> 5 KSP preconditioned resid norm 7.051404070011e+17 true resid norm >>>> 1.287518923939e+17 ||r(i)||/||b|| 9.817545968746e-01 >>>> 6 KSP preconditioned resid norm 5.863125538322e+17 true resid norm >>>> 1.166548434193e+17 ||r(i)||/||b|| 8.895125861467e-01 >>>> 7 KSP preconditioned resid norm 4.330808264262e+17 true resid norm >>>> 8.612437830885e+16 ||r(i)||/||b|| 6.567127110566e-01 >>>> 8 KSP preconditioned resid norm 2.702964509584e+17 true resid norm >>>> 4.804107972053e+16 ||r(i)||/||b|| 3.663212243137e-01 >>>> 9 KSP preconditioned resid norm 1.547354820777e+17 true resid norm >>>> 2.879372380790e+16 ||r(i)||/||b|| 2.195569337580e-01 >>>> 10 KSP preconditioned resid norm 9.913511217220e+16 true resid norm >>>> 1.649311378149e+16 ||r(i)||/||b|| 1.257627361485e-01 >>>> 11 KSP preconditioned resid norm 9.246165206306e+16 true resid norm >>>> 1.559859930326e+16 ||r(i)||/||b|| 1.189419144530e-01 >>>> 12 KSP preconditioned resid norm 9.248856361042e+16 true resid norm >>>> 1.507587985844e+16 ||r(i)||/||b|| 1.149560917339e-01 >>>> 13 KSP preconditioned resid norm 7.088634726331e+16 true resid norm >>>> 1.150049603396e+16 ||r(i)||/||b|| 8.769319532120e-02 >>>> 14 KSP preconditioned resid norm 2.014973749241e+17 true resid norm >>>> 3.263663179758e+16 ||r(i)||/||b|| 2.488597464319e-01 >>>> 15 KSP preconditioned resid norm 1.687600147354e+16 true resid norm >>>> 4.438944439784e+15 ||r(i)||/||b|| 3.384768975431e-02 >>>> 16 KSP preconditioned resid norm 1.189486995696e+16 true resid norm >>>> 1.895011071195e+15 ||r(i)||/||b|| 1.444977464550e-02 >>>> 17 KSP preconditioned resid norm 1.038277428537e+16 true resid norm >>>> 2.030560404119e+15 ||r(i)||/||b|| 1.548336085714e-02 >>>> 18 KSP preconditioned resid norm 9.585935517274e+15 true resid norm >>>> 1.570427582757e+15 ||r(i)||/||b|| 1.197477155297e-02 >>>> 19 KSP preconditioned resid norm 8.986106113745e+15 true resid norm >>>> 1.453955802798e+15 ||r(i)||/||b|| 1.108665485617e-02 >>>> 20 KSP preconditioned resid norm 8.480772996688e+15 true resid norm >>>> 1.519414795393e+15 ||r(i)||/||b|| 1.158579056355e-02 >>>> 21 KSP preconditioned resid norm 8.009707085363e+15 true resid norm >>>> 1.276353172356e+15 ||r(i)||/||b|| 9.732405255546e-03 >>>> 22 KSP preconditioned resid norm 7.603487959173e+15 true resid norm >>>> 1.348218761629e+15 ||r(i)||/||b|| 1.028039232831e-02 >>>> 23 KSP preconditioned resid norm 7.264676993739e+15 true resid norm >>>> 1.145141030012e+15 ||r(i)||/||b|| 8.731890843545e-03 >>>> 24 KSP preconditioned resid norm 6.442617339194e+15 true resid norm >>>> 1.846348609557e+15 ||r(i)||/||b|| 1.407871528071e-02 >>>> 25 KSP preconditioned resid norm 5.874716908283e+15 true resid norm >>>> 9.094105988409e+14 ||r(i)||/||b|| 6.934407093038e-03 >>>> 26 KSP preconditioned resid norm 5.974445043424e+15 true resid norm >>>> 9.497988242860e+14 ||r(i)||/||b|| 7.242374030480e-03 >>>> 27 KSP preconditioned resid norm 6.328543343408e+15 true resid norm >>>> 9.841709349231e+14 ||r(i)||/||b|| 7.504467091753e-03 >>>> 28 KSP preconditioned resid norm 1.045930080864e+16 true resid norm >>>> 1.637386301334e+15 ||r(i)||/||b|| 1.248534292045e-02 >>>> 29 KSP preconditioned resid norm 1.763968686067e+16 true resid norm >>>> 3.035143897342e+15 ||r(i)||/||b|| 2.314347710148e-02 >>>> 30 KSP preconditioned resid norm 3.994992144337e+15 true resid norm >>>> 6.366779249631e+14 ||r(i)||/||b|| 4.854775086713e-03 >>>> 31 KSP preconditioned resid norm 2.497471912676e+15 true resid norm >>>> 4.171284268487e+14 ||r(i)||/||b|| 3.180673642395e-03 >>>> 32 KSP preconditioned resid norm 1.774822233295e+15 true resid norm >>>> 3.030553568855e+14 ||r(i)||/||b|| 2.310847508319e-03 >>>> 33 KSP preconditioned resid norm 1.554413264158e+15 true resid norm >>>> 2.490797343960e+14 ||r(i)||/||b|| 1.899274408204e-03 >>>> 34 KSP preconditioned resid norm 1.363683273075e+15 true resid norm >>>> 2.337415683510e+14 ||r(i)||/||b|| 1.782318340667e-03 >>>> 35 KSP preconditioned resid norm 1.216181955878e+15 true resid norm >>>> 1.941708948493e+14 ||r(i)||/||b|| 1.480585372791e-03 >>>> 36 KSP preconditioned resid norm 1.106171392348e+15 true resid norm >>>> 1.781889599099e+14 ||r(i)||/||b|| 1.358720460346e-03 >>>> 37 KSP preconditioned resid norm 9.985678909493e+14 true resid norm >>>> 1.843602345870e+14 ||r(i)||/||b|| 1.405777456326e-03 >>>> 38 KSP preconditioned resid norm 9.306373857300e+14 true resid norm >>>> 1.483324768155e+14 ||r(i)||/||b|| 1.131059810243e-03 >>>> 39 KSP preconditioned resid norm 8.621292024153e+14 true resid norm >>>> 1.776747195898e+14 ||r(i)||/||b|| 1.354799292363e-03 >>>> 0 KSP preconditioned resid norm 6.470804796325e+18 true resid norm >>>> 3.511545401133e+18 ||r(i)||/||b|| 1.662319590124e+02 >>>> 1 KSP preconditioned resid norm 1.571603694611e+18 true resid norm >>>> 5.187449890798e+17 ||r(i)||/||b|| 2.455670820454e+01 >>>> 2 KSP preconditioned resid norm 6.468415902839e+17 true resid norm >>>> 1.566954416728e+17 ||r(i)||/||b|| 7.417756930949e+00 >>>> 3 KSP preconditioned resid norm 4.084120869852e+17 true resid norm >>>> 8.688409741290e+16 ||r(i)||/||b|| 4.112979349580e+00 >>>> 4 KSP preconditioned resid norm 2.920961241699e+17 true resid norm >>>> 4.832079671037e+16 ||r(i)||/||b|| 2.287443213923e+00 >>>> 5 KSP preconditioned resid norm 2.515909976499e+17 true resid norm >>>> 4.435653262803e+16 ||r(i)||/||b|| 2.099780145624e+00 >>>> 6 KSP preconditioned resid norm 2.133092401708e+17 true resid norm >>>> 4.355815215083e+16 ||r(i)||/||b|| 2.061985859746e+00 >>>> 7 KSP preconditioned resid norm 1.601275153782e+17 true resid norm >>>> 2.999237788966e+16 ||r(i)||/||b|| 1.419799877977e+00 >>>> 8 KSP preconditioned resid norm 1.019875018318e+17 true resid norm >>>> 1.811096161004e+16 ||r(i)||/||b|| 8.573491964719e-01 >>>> 9 KSP preconditioned resid norm 5.552203589730e+16 true resid norm >>>> 9.849208071572e+15 ||r(i)||/||b|| 4.662486072173e-01 >>>> 10 KSP preconditioned resid norm 2.614195079413e+16 true resid norm >>>> 4.400774101946e+15 ||r(i)||/||b|| 2.083268807806e-01 >>>> 11 KSP preconditioned resid norm 2.230316715231e+16 true resid norm >>>> 4.001821966241e+15 ||r(i)||/||b|| 1.894410093210e-01 >>>> 12 KSP preconditioned resid norm 2.202763899809e+16 true resid norm >>>> 3.668851069689e+15 ||r(i)||/||b|| 1.736786032846e-01 >>>> 13 KSP preconditioned resid norm 2.171352861704e+16 true resid norm >>>> 3.746836567903e+15 ||r(i)||/||b|| 1.773703346056e-01 >>>> 14 KSP preconditioned resid norm 2.756407833586e+16 true resid norm >>>> 4.737724354756e+15 ||r(i)||/||b|| 2.242776643292e-01 >>>> 15 KSP preconditioned resid norm 1.909305469788e+16 true resid norm >>>> 7.268311827741e+15 ||r(i)||/||b|| 3.440723601207e-01 >>>> 16 KSP preconditioned resid norm 8.249997128004e+15 true resid norm >>>> 1.447767843213e+15 ||r(i)||/||b|| 6.853543305884e-02 >>>> 17 KSP preconditioned resid norm 5.434797146758e+15 true resid norm >>>> 1.776552830818e+15 ||r(i)||/||b|| 8.409968364943e-02 >>>> 18 KSP preconditioned resid norm 4.166874856572e+15 true resid norm >>>> 7.323760066865e+14 ||r(i)||/||b|| 3.466972071212e-02 >>>> 19 KSP preconditioned resid norm 3.463504540881e+15 true resid norm >>>> 7.653814460137e+14 ||r(i)||/||b|| 3.623215497131e-02 >>>> 20 KSP preconditioned resid norm 3.008102754263e+15 true resid norm >>>> 5.633011056242e+14 ||r(i)||/||b|| 2.666593639130e-02 >>>> 21 KSP preconditioned resid norm 2.629337883031e+15 true resid norm >>>> 5.301916607139e+14 ||r(i)||/||b|| 2.509857864406e-02 >>>> 22 KSP preconditioned resid norm 2.387457188115e+15 true resid norm >>>> 3.879435274146e+14 ||r(i)||/||b|| 1.836473836491e-02 >>>> 23 KSP preconditioned resid norm 2.223797660329e+15 true resid norm >>>> 3.481194862204e+14 ||r(i)||/||b|| 1.647952042601e-02 >>>> 24 KSP preconditioned resid norm 2.082583007435e+15 true resid norm >>>> 3.393016280495e+14 ||r(i)||/||b|| 1.606209457197e-02 >>>> 25 KSP preconditioned resid norm 1.971105202361e+15 true resid norm >>>> 3.085385809365e+14 ||r(i)||/||b|| 1.460581222257e-02 >>>> 26 KSP preconditioned resid norm 1.916167824316e+15 true resid norm >>>> 3.157123008884e+14 ||r(i)||/||b|| 1.494540672721e-02 >>>> 27 KSP preconditioned resid norm 1.825444848282e+15 true resid norm >>>> 4.173235963436e+14 ||r(i)||/||b|| 1.975555233884e-02 >>>> 28 KSP preconditioned resid norm 1.787001647353e+15 true resid norm >>>> 2.787303202952e+14 ||r(i)||/||b|| 1.319472821393e-02 >>>> 29 KSP preconditioned resid norm 1.896717764390e+15 true resid norm >>>> 3.068021792990e+14 ||r(i)||/||b|| 1.452361324382e-02 >>>> 30 KSP preconditioned resid norm 2.229137880818e+15 true resid norm >>>> 3.494609542895e+14 ||r(i)||/||b|| 1.654302376702e-02 >>>> 31 KSP preconditioned resid norm 3.112887805330e+15 true resid norm >>>> 5.056849908847e+14 ||r(i)||/||b|| 2.393846499915e-02 >>>> 32 KSP preconditioned resid norm 2.024054955579e+16 true resid norm >>>> 3.265471021005e+15 ||r(i)||/||b|| 1.545831202253e-01 >>>> 33 KSP preconditioned resid norm 2.565517746075e+15 true resid norm >>>> 4.297071200158e+14 ||r(i)||/||b|| 2.034177212653e-02 >>>> 34 KSP preconditioned resid norm 1.151189963639e+15 true resid norm >>>> 1.893961732619e+14 ||r(i)||/||b|| 8.965766724994e-03 >>>> 35 KSP preconditioned resid norm 7.095661584181e+14 true resid norm >>>> 1.215463627012e+14 ||r(i)||/||b|| 5.753845579254e-03 >>>> 36 KSP preconditioned resid norm 5.356379112526e+14 true resid norm >>>> 8.697359204859e+13 ||r(i)||/||b|| 4.117215908392e-03 >>>> 37 KSP preconditioned resid norm 4.260660574229e+14 true resid norm >>>> 9.044225863386e+13 ||r(i)||/||b|| 4.281418040435e-03 >>>> 38 KSP preconditioned resid norm 3.822193347318e+14 true resid norm >>>> 6.067358042990e+13 ||r(i)||/||b|| 2.872207812522e-03 >>>> 39 KSP preconditioned resid norm 3.556139762092e+14 true resid norm >>>> 7.037316076443e+13 ||r(i)||/||b|| 3.331373238687e-03 >>>> 40 KSP preconditioned resid norm 3.319733183305e+14 true resid norm >>>> 5.228450610578e+13 ||r(i)||/||b|| 2.475080024071e-03 >>>> 41 KSP preconditioned resid norm 3.093370284781e+14 true resid norm >>>> 5.468964654818e+13 ||r(i)||/||b|| 2.588936221775e-03 >>>> 42 KSP preconditioned resid norm 2.908716641400e+14 true resid norm >>>> 4.719126609844e+13 ||r(i)||/||b|| 2.233972714489e-03 >>>> 43 KSP preconditioned resid norm 2.664499581099e+14 true resid norm >>>> 6.079882213095e+13 ||r(i)||/||b|| 2.878136590578e-03 >>>> 44 KSP preconditioned resid norm 2.513459163383e+14 true resid norm >>>> 3.896503433878e+13 ||r(i)||/||b|| 1.844553679708e-03 >>>> 45 KSP preconditioned resid norm 2.506735910619e+14 true resid norm >>>> 4.872960867423e+13 ||r(i)||/||b|| 2.306795836731e-03 >>>> 46 KSP preconditioned resid norm 2.610309833791e+14 true resid norm >>>> 4.046617714344e+13 ||r(i)||/||b|| 1.915615813519e-03 >>>> 47 KSP preconditioned resid norm 3.265892246735e+14 true resid norm >>>> 6.092635527969e+13 ||r(i)||/||b|| 2.884173842765e-03 >>>> 48 KSP preconditioned resid norm 6.619675585826e+14 true resid norm >>>> 1.029872445074e+14 ||r(i)||/||b|| 4.875281237212e-03 >>>> 49 KSP preconditioned resid norm 6.614927581836e+14 true resid norm >>>> 1.173864813817e+14 ||r(i)||/||b|| 5.556922247214e-03 >>>> 50 KSP preconditioned resid norm 2.572515469657e+14 true resid norm >>>> 4.031945474821e+13 ||r(i)||/||b|| 1.908670167541e-03 >>>> 51 KSP preconditioned resid norm 1.432897312628e+14 true resid norm >>>> 2.304000171834e+13 ||r(i)||/||b|| 1.090683497942e-03 >>>> 52 KSP preconditioned resid norm 8.898633954299e+13 true resid norm >>>> 1.420071536381e+13 ||r(i)||/||b|| 6.722432617682e-04 >>>> 0 KSP preconditioned resid norm 6.694445614266e+18 true resid norm >>>> 3.650633549648e+18 ||r(i)||/||b|| 5.229628447406e+00 >>>> 1 KSP preconditioned resid norm 1.597624793291e+18 true resid norm >>>> 5.366629107368e+17 ||r(i)||/||b|| 7.687837156178e-01 >>>> 2 KSP preconditioned resid norm 6.300333922334e+17 true resid norm >>>> 1.705382687722e+17 ||r(i)||/||b|| 2.443005493741e-01 >>>> 3 KSP preconditioned resid norm 3.642099405484e+17 true resid norm >>>> 9.482515314454e+16 ||r(i)||/||b|| 1.358395225569e-01 >>>> 4 KSP preconditioned resid norm 2.247296120507e+17 true resid norm >>>> 3.802694326323e+16 ||r(i)||/||b|| 5.447459504021e-02 >>>> 5 KSP preconditioned resid norm 1.557575416788e+17 true resid norm >>>> 3.828791753195e+16 ||r(i)||/||b|| 5.484844753491e-02 >>>> 6 KSP preconditioned resid norm 1.004780833173e+17 true resid norm >>>> 1.699813376802e+16 ||r(i)||/||b|| 2.435027309565e-02 >>>> 7 KSP preconditioned resid norm 7.104370054381e+16 true resid norm >>>> 1.472894912543e+16 ||r(i)||/||b|| 2.109960649273e-02 >>>> 8 KSP preconditioned resid norm 5.767747590200e+16 true resid norm >>>> 9.399551072817e+15 ||r(i)||/||b|| 1.346510380041e-02 >>>> 9 KSP preconditioned resid norm 4.524165642090e+16 true resid norm >>>> 1.246569325329e+16 ||r(i)||/||b|| 1.785743300921e-02 >>>> 10 KSP preconditioned resid norm 3.195588487377e+16 true resid norm >>>> 5.031148957130e+15 ||r(i)||/||b|| 7.207253029241e-03 >>>> 11 KSP preconditioned resid norm 2.437445149875e+16 true resid norm >>>> 4.973530315557e+15 ||r(i)||/||b|| 7.124712811776e-03 >>>> 12 KSP preconditioned resid norm 1.257635618330e+16 true resid norm >>>> 1.885290715052e+15 ||r(i)||/||b|| 2.700728468356e-03 >>>> 13 KSP preconditioned resid norm 2.495249888250e+15 true resid norm >>>> 4.460762730080e+14 ||r(i)||/||b|| 6.390159777228e-04 >>>> 14 KSP preconditioned resid norm 7.692041830184e+14 true resid norm >>>> 1.319158639688e+14 ||r(i)||/||b|| 1.889729400372e-04 >>>> 15 KSP preconditioned resid norm 5.963386353631e+14 true resid norm >>>> 1.441107752347e+14 ||r(i)||/||b|| 2.064424707371e-04 >>>> 16 KSP preconditioned resid norm 3.269005832710e+14 true resid norm >>>> 5.055770257989e+13 ||r(i)||/||b|| 7.242523689424e-05 >>>> 17 KSP preconditioned resid norm 1.442136448583e+14 true resid norm >>>> 7.615238793008e+13 ||r(i)||/||b|| 1.090902959284e-04 >>>> 18 KSP preconditioned resid norm 5.418830608399e+13 true resid norm >>>> 8.650648142713e+12 ||r(i)||/||b|| 1.239228068235e-05 >>>> 19 KSP preconditioned resid norm 3.534476706711e+13 true resid norm >>>> 1.235947718863e+13 ||r(i)||/||b|| 1.770527570673e-05 >>>> 0 KSP preconditioned resid norm 4.827689913975e+18 true resid norm >>>> 2.628955418091e+18 ||r(i)||/||b|| 2.387179080674e+01 >>>> 1 KSP preconditioned resid norm 1.161014015330e+18 true resid norm >>>> 3.885001078629e+17 ||r(i)||/||b|| 3.527710374804e+00 >>>> 2 KSP preconditioned resid norm 4.696713579932e+17 true resid norm >>>> 1.228966476828e+17 ||r(i)||/||b|| 1.115942493412e+00 >>>> 3 KSP preconditioned resid norm 2.869928353163e+17 true resid norm >>>> 6.759586363313e+16 ||r(i)||/||b|| 6.137929555390e-01 >>>> 4 KSP preconditioned resid norm 1.966510821692e+17 true resid norm >>>> 3.310379360335e+16 ||r(i)||/||b|| 3.005934716010e-01 >>>> 5 KSP preconditioned resid norm 1.592497528073e+17 true resid norm >>>> 3.239157026660e+16 ||r(i)||/||b|| 2.941262464874e-01 >>>> 6 KSP preconditioned resid norm 1.230094082012e+17 true resid norm >>>> 2.280559390645e+16 ||r(i)||/||b|| 2.070823883934e-01 >>>> 7 KSP preconditioned resid norm 8.598887604154e+16 true resid norm >>>> 1.938013722659e+16 ||r(i)||/||b|| 1.759781008439e-01 >>>> 8 KSP preconditioned resid norm 5.549487092651e+16 true resid norm >>>> 9.386421691835e+15 ||r(i)||/||b|| 8.523183524123e-02 >>>> 9 KSP preconditioned resid norm 4.475548308156e+16 true resid norm >>>> 7.579602548804e+15 ||r(i)||/||b|| 6.882531563606e-02 >>>> 10 KSP preconditioned resid norm 4.379582896462e+16 true resid norm >>>> 6.844431748768e+15 ||r(i)||/||b|| 6.214971991278e-02 >>>> 11 KSP preconditioned resid norm 4.081432560058e+16 true resid norm >>>> 7.719907534397e+15 ||r(i)||/||b|| 7.009933163578e-02 >>>> 12 KSP preconditioned resid norm 4.084955665921e+16 true resid norm >>>> 6.376903215279e+15 ||r(i)||/||b|| 5.790440511177e-02 >>>> 13 KSP preconditioned resid norm 7.513638228384e+16 true resid norm >>>> 1.130789675242e+16 ||r(i)||/||b|| 1.026794687656e-01 >>>> 14 KSP preconditioned resid norm 2.195838265499e+15 true resid norm >>>> 3.518796225338e+14 ||r(i)||/||b|| 3.195184171050e-03 >>>> 15 KSP preconditioned resid norm 2.237408970122e+15 true resid norm >>>> 3.676139977919e+14 ||r(i)||/||b|| 3.338057538948e-03 >>>> 16 KSP preconditioned resid norm 1.675719433917e+15 true resid norm >>>> 3.658174858611e+14 ||r(i)||/||b|| 3.321744612262e-03 >>>> 17 KSP preconditioned resid norm 1.403876583239e+15 true resid norm >>>> 2.164126267214e+14 ||r(i)||/||b|| 1.965098729891e-03 >>>> 18 KSP preconditioned resid norm 1.133854982860e+15 true resid norm >>>> 2.692061619571e+14 ||r(i)||/||b|| 2.444481613457e-03 >>>> 19 KSP preconditioned resid norm 9.034784290513e+14 true resid norm >>>> 1.408322491509e+14 ||r(i)||/||b|| 1.278803728446e-03 >>>> 20 KSP preconditioned resid norm 7.964711449547e+14 true resid norm >>>> 1.641000281547e+14 ||r(i)||/||b|| 1.490082911460e-03 >>>> 21 KSP preconditioned resid norm 7.094597664828e+14 true resid norm >>>> 1.088610845746e+14 ||r(i)||/||b|| 9.884949056480e-04 >>>> 22 KSP preconditioned resid norm 6.287860765855e+14 true resid norm >>>> 1.439087926971e+14 ||r(i)||/||b|| 1.306739768531e-03 >>>> 23 KSP preconditioned resid norm 5.617871594119e+14 true resid norm >>>> 8.640965180691e+13 ||r(i)||/||b|| 7.846284183530e-04 >>>> 24 KSP preconditioned resid norm 4.992037484390e+14 true resid norm >>>> 1.232499062518e+14 ||r(i)||/||b|| 1.119150198876e-03 >>>> 25 KSP preconditioned resid norm 3.174300847299e+14 true resid norm >>>> 1.199443314096e+14 ||r(i)||/||b|| 1.089134478340e-03 >>>> 26 KSP preconditioned resid norm 2.311461716948e+14 true resid norm >>>> 3.806900580053e+13 ||r(i)||/||b|| 3.456792520850e-04 >>>> 27 KSP preconditioned resid norm 2.486080516907e+14 true resid norm >>>> 3.795889860412e+13 ||r(i)||/||b|| 3.446794420688e-04 >>>> 28 KSP preconditioned resid norm 3.709940932117e+14 true resid norm >>>> 6.158317563060e+13 ||r(i)||/||b|| 5.591957458661e-04 >>>> 29 KSP preconditioned resid norm 5.526902041486e+14 true resid norm >>>> 8.543527855752e+13 ||r(i)||/||b|| 7.757807847199e-04 >>>> 30 KSP preconditioned resid norm 1.980272137271e+15 true resid norm >>>> 3.202004161054e+14 ||r(i)||/||b|| 2.907526425475e-03 >>>> 31 KSP preconditioned resid norm 4.796148387368e+14 true resid norm >>>> 7.312671161483e+13 ||r(i)||/||b|| 6.640148973394e-04 >>>> 32 KSP preconditioned resid norm 1.482366487345e+14 true resid norm >>>> 3.206675205939e+13 ||r(i)||/||b|| 2.911767889807e-04 >>>> 33 KSP preconditioned resid norm 7.403721045502e+13 true resid norm >>>> 1.122274917558e+13 ||r(i)||/||b|| 1.019063004083e-04 >>>> 34 KSP preconditioned resid norm 3.421203410880e+13 true resid norm >>>> 9.812523407473e+12 ||r(i)||/||b|| 8.910098073838e-05 >>>> 35 KSP preconditioned resid norm 1.809542403951e+13 true resid norm >>>> 2.774316021276e+12 ||r(i)||/||b|| 2.519171349804e-05 >>>> 36 KSP preconditioned resid norm 1.070863017107e+13 true resid norm >>>> 1.674563212179e+12 ||r(i)||/||b|| 1.520559170335e-05 >>>> 37 KSP preconditioned resid norm 6.984609262190e+12 true resid norm >>>> 1.277736482286e+12 ||r(i)||/||b|| 1.160227282721e-05 >>>> 0 KSP preconditioned resid norm 9.569153877844e+15 true resid norm >>>> 5.893805322157e+15 ||r(i)||/||b|| 1.119437717455e-01 >>>> 1 KSP preconditioned resid norm 4.594033327917e+15 true resid norm >>>> 8.149731925277e+14 ||r(i)||/||b|| 1.547916296116e-02 >>>> 2 KSP preconditioned resid norm 3.788286152976e+15 true resid norm >>>> 6.151807578016e+14 ||r(i)||/||b|| 1.168441279773e-02 >>>> 3 KSP preconditioned resid norm 3.712195492568e+15 true resid norm >>>> 6.206672982073e+14 ||r(i)||/||b|| 1.178862119847e-02 >>>> 4 KSP preconditioned resid norm 3.886667577513e+15 true resid norm >>>> 6.330116967950e+14 ||r(i)||/||b|| 1.202308407301e-02 >>>> 5 KSP preconditioned resid norm 4.317240607103e+15 true resid norm >>>> 6.816590875948e+14 ||r(i)||/||b|| 1.294706647725e-02 >>>> 6 KSP preconditioned resid norm 5.249548366505e+15 true resid norm >>>> 7.833242603614e+14 ||r(i)||/||b|| 1.487804014750e-02 >>>> 7 KSP preconditioned resid norm 6.521511569741e+15 true resid norm >>>> 1.182301409963e+15 ||r(i)||/||b|| 2.245599777001e-02 >>>> 8 KSP preconditioned resid norm 7.063327681244e+15 true resid norm >>>> 1.110968116855e+15 ||r(i)||/||b|| 2.110113152571e-02 >>>> 9 KSP preconditioned resid norm 7.059112088082e+15 true resid norm >>>> 1.341815038025e+15 ||r(i)||/||b|| 2.548571392011e-02 >>>> 10 KSP preconditioned resid norm 6.417041101993e+15 true resid norm >>>> 1.020408664201e+15 ||r(i)||/||b|| 1.938109393654e-02 >>>> 11 KSP preconditioned resid norm 4.850416656929e+15 true resid norm >>>> 8.501719778107e+14 ||r(i)||/||b|| 1.614770977770e-02 >>>> 12 KSP preconditioned resid norm 2.420686956668e+15 true resid norm >>>> 3.735196627993e+14 ||r(i)||/||b|| 7.094431795646e-03 >>>> 13 KSP preconditioned resid norm 1.348084459782e+15 true resid norm >>>> 2.753333760274e+14 ||r(i)||/||b|| 5.229534216894e-03 >>>> 14 KSP preconditioned resid norm 1.225904445555e+15 true resid norm >>>> 2.239730314894e+14 ||r(i)||/||b|| 4.254023426925e-03 >>>> 15 KSP preconditioned resid norm 1.156415256406e+15 true resid norm >>>> 2.064224387096e+14 ||r(i)||/||b|| 3.920676897010e-03 >>>> 16 KSP preconditioned resid norm 1.062658328422e+15 true resid norm >>>> 1.696405409630e+14 ||r(i)||/||b|| 3.222061292889e-03 >>>> 17 KSP preconditioned resid norm 9.163863908341e+14 true resid norm >>>> 1.701529770139e+14 ||r(i)||/||b|| 3.231794227925e-03 >>>> 18 KSP preconditioned resid norm 8.234055963962e+14 true resid norm >>>> 1.322040551676e+14 ||r(i)||/||b|| 2.511012795057e-03 >>>> 19 KSP preconditioned resid norm 7.097124944641e+14 true resid norm >>>> 1.155726966735e+14 ||r(i)||/||b|| 2.195125707289e-03 >>>> 20 KSP preconditioned resid norm 6.309033027960e+14 true resid norm >>>> 1.023450925395e+14 ||r(i)||/||b|| 1.943887701115e-03 >>>> 21 KSP preconditioned resid norm 5.667708572251e+14 true resid norm >>>> 8.941024380489e+13 ||r(i)||/||b|| 1.698210133710e-03 >>>> 22 KSP preconditioned resid norm 5.205793964231e+14 true resid norm >>>> 8.689007901528e+13 ||r(i)||/||b|| 1.650343477697e-03 >>>> 23 KSP preconditioned resid norm 4.890134322830e+14 true resid norm >>>> 7.566844928998e+13 ||r(i)||/||b|| 1.437205871699e-03 >>>> 24 KSP preconditioned resid norm 4.617102016960e+14 true resid norm >>>> 8.638231658778e+13 ||r(i)||/||b|| 1.640699310952e-03 >>>> 25 KSP preconditioned resid norm 4.386224186730e+14 true resid norm >>>> 6.736465167645e+13 ||r(i)||/||b|| 1.279488001179e-03 >>>> 26 KSP preconditioned resid norm 4.119635069909e+14 true resid norm >>>> 8.360600093542e+13 ||r(i)||/||b|| 1.587967463073e-03 >>>> 27 KSP preconditioned resid norm 3.764202708342e+14 true resid norm >>>> 5.766864517117e+13 ||r(i)||/||b|| 1.095327262956e-03 >>>> 28 KSP preconditioned resid norm 3.345193309034e+14 true resid norm >>>> 7.248849449141e+13 ||r(i)||/||b|| 1.376807518738e-03 >>>> 0 KSP preconditioned resid norm 9.592548277384e+15 true resid norm >>>> 5.933564471549e+15 ||r(i)||/||b|| 5.704186299543e-01 >>>> 1 KSP preconditioned resid norm 4.627892277467e+15 true resid norm >>>> 8.123960687021e+14 ||r(i)||/||b|| 7.809906755228e-02 >>>> 2 KSP preconditioned resid norm 3.843955835291e+15 true resid norm >>>> 6.217930929753e+14 ||r(i)||/||b|| 5.977559794129e-02 >>>> 3 KSP preconditioned resid norm 3.802651796154e+15 true resid norm >>>> 6.334637368548e+14 ||r(i)||/||b|| 6.089754626161e-02 >>>> 4 KSP preconditioned resid norm 4.025480937197e+15 true resid norm >>>> 6.491593110279e+14 ||r(i)||/||b|| 6.240642814182e-02 >>>> 5 KSP preconditioned resid norm 4.519579789539e+15 true resid norm >>>> 7.102235297384e+14 ||r(i)||/||b|| 6.827678956506e-02 >>>> 6 KSP preconditioned resid norm 5.567308133052e+15 true resid norm >>>> 8.274501588234e+14 ||r(i)||/||b|| 7.954628085944e-02 >>>> 7 KSP preconditioned resid norm 7.053789984861e+15 true resid norm >>>> 1.260366188593e+15 ||r(i)||/||b|| 1.211643284547e-01 >>>> 8 KSP preconditioned resid norm 7.835759896085e+15 true resid norm >>>> 1.231567591968e+15 ||r(i)||/||b|| 1.183957976482e-01 >>>> 9 KSP preconditioned resid norm 8.126559485840e+15 true resid norm >>>> 1.514279927927e+15 ||r(i)||/||b|| 1.455741293445e-01 >>>> 10 KSP preconditioned resid norm 7.865908148994e+15 true resid norm >>>> 1.247217430777e+15 ||r(i)||/||b|| 1.199002827947e-01 >>>> 11 KSP preconditioned resid norm 6.705237228445e+15 true resid norm >>>> 1.156150868849e+15 ||r(i)||/||b|| 1.111456693176e-01 >>>> 12 KSP preconditioned resid norm 3.837356917105e+15 true resid norm >>>> 5.892678306289e+14 ||r(i)||/||b|| 5.664880700887e-02 >>>> 13 KSP preconditioned resid norm 1.588477046031e+15 true resid norm >>>> 3.087098247775e+14 ||r(i)||/||b|| 2.967758017080e-02 >>>> 14 KSP preconditioned resid norm 1.304101915863e+15 true resid norm >>>> 2.634312731694e+14 ||r(i)||/||b|| 2.532476164182e-02 >>>> 15 KSP preconditioned resid norm 1.188980488344e+15 true resid norm >>>> 2.212236776838e+14 ||r(i)||/||b|| 2.126716710384e-02 >>>> 16 KSP preconditioned resid norm 1.053945473260e+15 true resid norm >>>> 1.687417173497e+14 ||r(i)||/||b|| 1.622185444993e-02 >>>> 17 KSP preconditioned resid norm 8.630322699779e+14 true resid norm >>>> 2.032342536888e+14 ||r(i)||/||b|| 1.953776774564e-02 >>>> 18 KSP preconditioned resid norm 7.497367164260e+14 true resid norm >>>> 1.258314731452e+14 ||r(i)||/||b|| 1.209671132095e-02 >>>> 19 KSP preconditioned resid norm 6.482413928213e+14 true resid norm >>>> 1.007668116028e+14 ||r(i)||/||b|| 9.687139474922e-03 >>>> 20 KSP preconditioned resid norm 5.423124085977e+14 true resid norm >>>> 1.262959277696e+14 ||r(i)||/||b|| 1.214136130695e-02 >>>> 21 KSP preconditioned resid norm 4.512195524384e+14 true resid norm >>>> 7.029203398299e+13 ||r(i)||/||b|| 6.757470305334e-03 >>>> 22 KSP preconditioned resid norm 3.576286880164e+14 true resid norm >>>> 9.097241124393e+13 ||r(i)||/||b|| 8.745562373884e-03 >>>> 23 KSP preconditioned resid norm 2.908403868181e+14 true resid norm >>>> 4.482717971847e+13 ||r(i)||/||b|| 4.309426241567e-03 >>>> 24 KSP preconditioned resid norm 2.299942905294e+14 true resid norm >>>> 6.485244464292e+13 ||r(i)||/||b|| 6.234539592478e-03 >>>> 25 KSP preconditioned resid norm 1.957293334591e+14 true resid norm >>>> 3.000864324033e+13 ||r(i)||/||b|| 2.884857701642e-03 >>>> 26 KSP preconditioned resid norm 1.685971736537e+14 true resid norm >>>> 4.568513443333e+13 ||r(i)||/||b|| 4.391905054321e-03 >>>> 27 KSP preconditioned resid norm 1.478986486404e+14 true resid norm >>>> 2.270701271232e+13 ||r(i)||/||b|| 2.182921099758e-03 >>>> 28 KSP preconditioned resid norm 1.388836935911e+14 true resid norm >>>> 2.435562734780e+13 ||r(i)||/||b|| 2.341409392284e-03 >>>> 29 KSP preconditioned resid norm 1.300875297573e+14 true resid norm >>>> 2.023811842122e+13 ||r(i)||/||b|| 1.945575857149e-03 >>>> 30 KSP preconditioned resid norm 1.230463362777e+14 true resid norm >>>> 2.013991197226e+13 ||r(i)||/||b|| 1.936134856156e-03 >>>> 31 KSP preconditioned resid norm 1.148174878708e+14 true resid norm >>>> 1.790715664590e+13 ||r(i)||/||b|| 1.721490650234e-03 >>>> 32 KSP preconditioned resid norm 1.085020096185e+14 true resid norm >>>> 1.697737976905e+13 ||r(i)||/||b|| 1.632107269502e-03 >>>> 33 KSP preconditioned resid norm 1.008662380835e+14 true resid norm >>>> 1.575408064147e+13 ||r(i)||/||b|| 1.514506354280e-03 >>>> 34 KSP preconditioned resid norm 9.448474542578e+13 true resid norm >>>> 1.450932025652e+13 ||r(i)||/||b|| 1.394842277686e-03 >>>> 35 KSP preconditioned resid norm 8.955256800693e+13 true resid norm >>>> 1.427784032077e+13 ||r(i)||/||b|| 1.372589133148e-03 >>>> 36 KSP preconditioned resid norm 8.409700445745e+13 true resid norm >>>> 1.287414873178e+13 ||r(i)||/||b|| 1.237646328210e-03 >>>> 37 KSP preconditioned resid norm 7.999039854664e+13 true resid norm >>>> 1.307106382082e+13 ||r(i)||/||b|| 1.256576607951e-03 >>>> 38 KSP preconditioned resid norm 7.508174679474e+13 true resid norm >>>> 1.150183263556e+13 ||r(i)||/||b|| 1.105719781996e-03 >>>> 39 KSP preconditioned resid norm 7.047760131088e+13 true resid norm >>>> 1.131577846513e+13 ||r(i)||/||b|| 1.087833608263e-03 >>>> 40 KSP preconditioned resid norm 6.612188631103e+13 true resid norm >>>> 1.012435424497e+13 ||r(i)||/||b|| 9.732969626060e-04 >>>> 41 KSP preconditioned resid norm 6.096165352930e+13 true resid norm >>>> 1.005634567468e+13 ||r(i)||/||b|| 9.667590113164e-04 >>>> 42 KSP preconditioned resid norm 5.668115744306e+13 true resid norm >>>> 8.662975702181e+12 ||r(i)||/||b|| 8.328084669947e-04 >>>> 43 KSP preconditioned resid norm 5.243169528258e+13 true resid norm >>>> 8.972837494107e+12 ||r(i)||/||b|| 8.625967906361e-04 >>>> 0 KSP preconditioned resid norm 2.201027933913e+16 true resid norm >>>> 1.374916449250e+16 ||r(i)||/||b|| 1.042532971848e-01 >>>> 1 KSP preconditioned resid norm 1.033786235817e+16 true resid norm >>>> 1.769244185634e+15 ||r(i)||/||b|| 1.341532716246e-02 >>>> 2 KSP preconditioned resid norm 8.200152533928e+15 true resid norm >>>> 1.329548327593e+15 ||r(i)||/||b|| 1.008132508660e-02 >>>> 3 KSP preconditioned resid norm 7.725254307721e+15 true resid norm >>>> 1.292264285665e+15 ||r(i)||/||b|| 9.798618140627e-03 >>>> 4 KSP preconditioned resid norm 7.783851928031e+15 true resid norm >>>> 1.241739377824e+15 ||r(i)||/||b|| 9.415512080968e-03 >>>> 5 KSP preconditioned resid norm 8.247256237278e+15 true resid norm >>>> 1.289875346607e+15 ||r(i)||/||b|| 9.780503965494e-03 >>>> 6 KSP preconditioned resid norm 9.468674539247e+15 true resid norm >>>> 1.409079612219e+15 ||r(i)||/||b|| 1.068437254131e-02 >>>> 7 KSP preconditioned resid norm 1.106031993220e+16 true resid norm >>>> 1.997396506013e+15 ||r(i)||/||b|| 1.514529640334e-02 >>>> 8 KSP preconditioned resid norm 1.133839709036e+16 true resid norm >>>> 1.772431554854e+15 ||r(i)||/||b|| 1.343949544924e-02 >>>> 9 KSP preconditioned resid norm 1.055805824819e+16 true resid norm >>>> 2.045498232098e+15 ||r(i)||/||b|| 1.551002864197e-02 >>>> 10 KSP preconditioned resid norm 8.628048224174e+15 true resid norm >>>> 1.368695682952e+15 ||r(i)||/||b|| 1.037816064155e-02 >>>> 11 KSP preconditioned resid norm 5.442091241360e+15 true resid norm >>>> 9.810024540837e+14 ||r(i)||/||b|| 7.438469475027e-03 >>>> 12 KSP preconditioned resid norm 2.000681531075e+15 true resid norm >>>> 3.042659606098e+14 ||r(i)||/||b|| 2.307102342980e-03 >>>> 13 KSP preconditioned resid norm 8.105673271529e+14 true resid norm >>>> 1.475101242619e+14 ||r(i)||/||b|| 1.118498278992e-03 >>>> 0 KSP preconditioned resid norm 1.928660996668e+16 true resid norm >>>> 1.203367101178e+16 ||r(i)||/||b|| 4.946564650623e-01 >>>> 1 KSP preconditioned resid norm 9.233394049572e+15 true resid norm >>>> 1.583660413052e+15 ||r(i)||/||b|| 6.509799553373e-02 >>>> 2 KSP preconditioned resid norm 7.543213441610e+15 true resid norm >>>> 1.216909563864e+15 ||r(i)||/||b|| 5.002232341004e-02 >>>> 3 KSP preconditioned resid norm 7.341955893763e+15 true resid norm >>>> 1.220114169499e+15 ||r(i)||/||b|| 5.015405203166e-02 >>>> 4 KSP preconditioned resid norm 7.657405519141e+15 true resid norm >>>> 1.212628294074e+15 ||r(i)||/||b|| 4.984633739730e-02 >>>> 5 KSP preconditioned resid norm 8.417297612797e+15 true resid norm >>>> 1.311888034770e+15 ||r(i)||/||b|| 5.392651146949e-02 >>>> 6 KSP preconditioned resid norm 1.008979814671e+16 true resid norm >>>> 1.496346817141e+15 ||r(i)||/||b|| 6.150888007071e-02 >>>> 7 KSP preconditioned resid norm 1.235951387667e+16 true resid norm >>>> 2.203847820374e+15 ||r(i)||/||b|| 9.059143891287e-02 >>>> 8 KSP preconditioned resid norm 1.311708980126e+16 true resid norm >>>> 2.061617788942e+15 ||r(i)||/||b|| 8.474492669688e-02 >>>> 9 KSP preconditioned resid norm 1.289258338081e+16 true resid norm >>>> 2.353735302483e+15 ||r(i)||/||b|| 9.675271854106e-02 >>>> 10 KSP preconditioned resid norm 1.151089236912e+16 true resid norm >>>> 1.820534806530e+15 ||r(i)||/||b|| 7.483496191971e-02 >>>> 11 KSP preconditioned resid norm 8.373719062509e+15 true resid norm >>>> 1.433022884373e+15 ||r(i)||/||b|| 5.890588446727e-02 >>>> 12 KSP preconditioned resid norm 3.564533969408e+15 true resid norm >>>> 5.439956922043e+14 ||r(i)||/||b|| 2.236150430334e-02 >>>> 13 KSP preconditioned resid norm 1.257109874849e+15 true resid norm >>>> 2.286041152149e+14 ||r(i)||/||b|| 9.397008063476e-03 >>>> 14 KSP preconditioned resid norm 1.068785016666e+15 true resid norm >>>> 1.971356784282e+14 ||r(i)||/||b|| 8.103465495567e-03 >>>> 15 KSP preconditioned resid norm 9.963029863724e+14 true resid norm >>>> 1.558476632901e+14 ||r(i)||/||b|| 6.406279026229e-03 >>>> 16 KSP preconditioned resid norm 8.667425302785e+14 true resid norm >>>> 2.560422780209e+14 ||r(i)||/||b|| 1.052488206037e-02 >>>> 17 KSP preconditioned resid norm 7.240699142388e+14 true resid norm >>>> 1.131314963782e+14 ||r(i)||/||b|| 4.650386904448e-03 >>>> 18 KSP preconditioned resid norm 6.352047441583e+14 true resid norm >>>> 1.637714957863e+14 ||r(i)||/||b|| 6.731996337968e-03 >>>> 19 KSP preconditioned resid norm 5.532686184638e+14 true resid norm >>>> 8.631039548890e+13 ||r(i)||/||b|| 3.547877874413e-03 >>>> 20 KSP preconditioned resid norm 4.676701949378e+14 true resid norm >>>> 1.246119427256e+14 ||r(i)||/||b|| 5.122302498784e-03 >>>> 21 KSP preconditioned resid norm 4.001088470241e+14 true resid norm >>>> 6.181513171397e+13 ||r(i)||/||b|| 2.540974779106e-03 >>>> 22 KSP preconditioned resid norm 3.482451502111e+14 true resid norm >>>> 6.477073783496e+13 ||r(i)||/||b|| 2.662468018741e-03 >>>> 23 KSP preconditioned resid norm 3.041601141164e+14 true resid norm >>>> 4.679581096972e+13 ||r(i)||/||b|| 1.923590100755e-03 >>>> 24 KSP preconditioned resid norm 2.690997224286e+14 true resid norm >>>> 4.294556821793e+13 ||r(i)||/||b|| 1.765321899192e-03 >>>> 25 KSP preconditioned resid norm 2.482501027210e+14 true resid norm >>>> 3.840071688811e+13 ||r(i)||/||b|| 1.578501095230e-03 >>>> 26 KSP preconditioned resid norm 2.328482920528e+14 true resid norm >>>> 3.868960801558e+13 ||r(i)||/||b|| 1.590376263145e-03 >>>> 27 KSP preconditioned resid norm 2.171581409229e+14 true resid norm >>>> 3.334869397249e+13 ||r(i)||/||b|| 1.370832479858e-03 >>>> 28 KSP preconditioned resid norm 2.002313534549e+14 true resid norm >>>> 3.534153915591e+13 ||r(i)||/||b|| 1.452750437635e-03 >>>> 29 KSP preconditioned resid norm 1.813916789125e+14 true resid norm >>>> 2.780025874135e+13 ||r(i)||/||b|| 1.142758323985e-03 >>>> 30 KSP preconditioned resid norm 1.655865002619e+14 true resid norm >>>> 2.941107681765e+13 ||r(i)||/||b|| 1.208972663292e-03 >>>> 31 KSP preconditioned resid norm 1.477358437937e+14 true resid norm >>>> 2.263549873151e+13 ||r(i)||/||b|| 9.304555340168e-04 >>>> 32 KSP preconditioned resid norm 1.346388933582e+14 true resid norm >>>> 2.267328899678e+13 ||r(i)||/||b|| 9.320089418685e-04 >>>> 0 KSP preconditioned resid norm 6.130047183925e+16 true resid norm >>>> 4.019546453762e+16 ||r(i)||/||b|| 5.748521837148e-02 >>>> 1 KSP preconditioned resid norm 2.383257388870e+16 true resid norm >>>> 4.333010621750e+15 ||r(i)||/||b|| 6.196820080636e-03 >>>> 2 KSP preconditioned resid norm 1.380338515654e+16 true resid norm >>>> 2.902899468751e+15 ||r(i)||/||b|| 4.151558186756e-03 >>>> 3 KSP preconditioned resid norm 9.355621435823e+15 true resid norm >>>> 1.914392486860e+15 ||r(i)||/||b|| 2.737852924995e-03 >>>> 4 KSP preconditioned resid norm 6.905538449399e+15 true resid norm >>>> 1.160515328667e+15 ||r(i)||/||b|| 1.659701607116e-03 >>>> 5 KSP preconditioned resid norm 5.388961375964e+15 true resid norm >>>> 9.343353155170e+14 ||r(i)||/||b|| 1.336232091419e-03 >>>> 6 KSP preconditioned resid norm 4.457024362952e+15 true resid norm >>>> 7.443375401358e+14 ||r(i)||/||b|| 1.064508310303e-03 >>>> 7 KSP preconditioned resid norm 3.604925656449e+15 true resid norm >>>> 7.177624609068e+14 ||r(i)||/||b|| 1.026502175773e-03 >>>> 8 KSP preconditioned resid norm 2.616540301450e+15 true resid norm >>>> 4.566819450076e+14 ||r(i)||/||b|| 6.531199884630e-04 >>>> 9 KSP preconditioned resid norm 1.730734442406e+15 true resid norm >>>> 3.416454932875e+14 ||r(i)||/||b|| 4.886015378397e-04 >>>> 10 KSP preconditioned resid norm 9.605665765368e+14 true resid norm >>>> 1.715807715453e+14 ||r(i)||/||b|| 2.453848520993e-04 >>>> 11 KSP preconditioned resid norm 3.885455022454e+14 true resid norm >>>> 7.526175493349e+13 ||r(i)||/||b|| 1.076349898463e-04 >>>> 12 KSP preconditioned resid norm 1.085553513563e+14 true resid norm >>>> 1.877760955419e+13 ||r(i)||/||b|| 2.685464636705e-05 >>>> 13 KSP preconditioned resid norm 5.523836175310e+13 true resid norm >>>> 1.483397872543e+13 ||r(i)||/||b|| 2.121469464673e-05 >>>> 14 KSP preconditioned resid norm 4.329267915925e+13 true resid norm >>>> 6.950523101530e+12 ||r(i)||/||b|| 9.940234374291e-06 >>>> 0 KSP preconditioned resid norm 2.469234905564e+16 true resid norm >>>> 1.581399294540e+16 ||r(i)||/||b|| 1.421323674244e-01 >>>> 1 KSP preconditioned resid norm 1.094883401656e+16 true resid norm >>>> 1.827560974935e+15 ||r(i)||/||b|| 1.642567875658e-02 >>>> 2 KSP preconditioned resid norm 7.902898352613e+15 true resid norm >>>> 1.331982197511e+15 ||r(i)||/||b|| 1.197153582609e-02 >>>> 3 KSP preconditioned resid norm 6.757313480243e+15 true resid norm >>>> 1.177585457449e+15 ||r(i)||/||b|| 1.058385503836e-02 >>>> 4 KSP preconditioned resid norm 6.237102240639e+15 true resid norm >>>> 9.859442147838e+14 ||r(i)||/||b|| 8.861429613598e-03 >>>> 5 KSP preconditioned resid norm 6.056820105725e+15 true resid norm >>>> 9.499166634556e+14 ||r(i)||/||b|| 8.537622642110e-03 >>>> 6 KSP preconditioned resid norm 6.282037371920e+15 true resid norm >>>> 9.440405134922e+14 ||r(i)||/||b|| 8.484809218675e-03 >>>> 7 KSP preconditioned resid norm 6.512961686254e+15 true resid norm >>>> 1.175538616906e+15 ||r(i)||/||b|| 1.056545852755e-02 >>>> 8 KSP preconditioned resid norm 5.899315381141e+15 true resid norm >>>> 9.360504236018e+14 ||r(i)||/||b|| 8.412996211297e-03 >>>> 9 KSP preconditioned resid norm 4.860546577526e+15 true resid norm >>>> 9.070630816401e+14 ||r(i)||/||b|| 8.152464949358e-03 >>>> 10 KSP preconditioned resid norm 3.431569954082e+15 true resid norm >>>> 5.519072654101e+14 ||r(i)||/||b|| 4.960409840975e-03 >>>> 11 KSP preconditioned resid norm 1.788273694951e+15 true resid norm >>>> 3.211455497861e+14 ||r(i)||/||b|| 2.886379008546e-03 >>>> 12 KSP preconditioned resid norm 6.021242921522e+14 true resid norm >>>> 9.258232372098e+13 ||r(i)||/||b|| 8.321076718289e-04 >>>> 13 KSP preconditioned resid norm 3.678076314547e+14 true resid norm >>>> 7.160952479278e+13 ||r(i)||/||b|| 6.436091962400e-04 >>>> 14 KSP preconditioned resid norm 3.596149520932e+14 true resid norm >>>> 5.543217762956e+13 ||r(i)||/||b|| 4.982110884443e-04 >>>> 15 KSP preconditioned resid norm 1.443083471369e+14 true resid norm >>>> 3.905973798110e+13 ||r(i)||/||b|| 3.510595362853e-04 >>>> 16 KSP preconditioned resid norm 1.158730309561e+14 true resid norm >>>> 1.821149714099e+13 ||r(i)||/||b|| 1.636805588525e-04 >>>> 17 KSP preconditioned resid norm 9.042472136695e+13 true resid norm >>>> 2.428568415416e+13 ||r(i)||/||b|| 2.182738916901e-04 >>>> 18 KSP preconditioned resid norm 7.667332005406e+13 true resid norm >>>> 1.178343148104e+13 ||r(i)||/||b|| 1.059066497984e-04 >>>> 19 KSP preconditioned resid norm 6.435860750477e+13 true resid norm >>>> 1.303667702134e+13 ||r(i)||/||b|| 1.171705194752e-04 >>>> 20 KSP preconditioned resid norm 5.245023814970e+13 true resid norm >>>> 8.751976411105e+12 ||r(i)||/||b|| 7.866066029293e-05 >>>> 21 KSP preconditioned resid norm 4.161239918955e+13 true resid norm >>>> 6.811593020631e+12 ||r(i)||/||b|| 6.122096078432e-05 >>>> 22 KSP preconditioned resid norm 3.107777724805e+13 true resid norm >>>> 6.092648679105e+12 ||r(i)||/||b|| 5.475926185348e-05 >>>> 23 KSP preconditioned resid norm 2.312582950537e+13 true resid norm >>>> 3.536350440396e+12 ||r(i)||/||b|| 3.178386773481e-05 >>>> 24 KSP preconditioned resid norm 1.680324737468e+13 true resid norm >>>> 4.383700522921e+12 ||r(i)||/||b|| 3.939964660118e-05 >>>> 25 KSP preconditioned resid norm 1.300784122000e+13 true resid norm >>>> 1.953230538144e+12 ||r(i)||/||b|| 1.755516658383e-05 >>>> 26 KSP preconditioned resid norm 9.922666654442e+12 true resid norm >>>> 3.054726768240e+12 ||r(i)||/||b|| 2.745514993611e-05 >>>> 27 KSP preconditioned resid norm 7.226267347787e+12 true resid norm >>>> 1.085334598547e+12 ||r(i)||/||b|| 9.754726492653e-06 >>>> 0 SNES Function norm 0.000000000000e+00 >>>> 2 TS dt 4e-12 time 5e-12 >>>> 0 KSP preconditioned resid norm 2.220258026867e+19 true resid norm >>>> 1.204245784292e+19 ||r(i)||/||b|| 2.302282866910e+02 >>>> 1 KSP preconditioned resid norm 5.390122646802e+18 true resid norm >>>> 1.775868988684e+18 ||r(i)||/||b|| 3.395114851018e+01 >>>> 2 KSP preconditioned resid norm 2.218369826326e+18 true resid norm >>>> 5.319781926844e+17 ||r(i)||/||b|| 1.017038460556e+01 >>>> 3 KSP preconditioned resid norm 1.401713665358e+18 true resid norm >>>> 2.963575488339e+17 ||r(i)||/||b|| 5.665777834223e+00 >>>> 4 KSP preconditioned resid norm 1.004075064815e+18 true resid norm >>>> 1.656255011022e+17 ||r(i)||/||b|| 3.166436274762e+00 >>>> 5 KSP preconditioned resid norm 8.672862633339e+17 true resid norm >>>> 1.510357773334e+17 ||r(i)||/||b|| 2.887509235913e+00 >>>> 6 KSP preconditioned resid norm 7.389427240230e+17 true resid norm >>>> 1.526197236053e+17 ||r(i)||/||b|| 2.917791196717e+00 >>>> 7 KSP preconditioned resid norm 5.566238898025e+17 true resid norm >>>> 1.017349876189e+17 ||r(i)||/||b|| 1.944974373301e+00 >>>> 8 KSP preconditioned resid norm 3.566703993070e+17 true resid norm >>>> 6.306898440820e+16 ||r(i)||/||b|| 1.205755869196e+00 >>>> 9 KSP preconditioned resid norm 1.893399021800e+17 true resid norm >>>> 3.268112455206e+16 ||r(i)||/||b|| 6.247993068277e-01 >>>> 10 KSP preconditioned resid norm 6.895422631218e+16 true resid norm >>>> 1.153042493210e+16 ||r(i)||/||b|| 2.204392169408e-01 >>>> 11 KSP preconditioned resid norm 5.104209415340e+16 true resid norm >>>> 9.274112790250e+15 ||r(i)||/||b|| 1.773029331827e-01 >>>> 12 KSP preconditioned resid norm 4.991932910946e+16 true resid norm >>>> 8.397724827599e+15 ||r(i)||/||b|| 1.605481060744e-01 >>>> 13 KSP preconditioned resid norm 4.919289629176e+16 true resid norm >>>> 8.425037065615e+15 ||r(i)||/||b|| 1.610702627509e-01 >>>> 14 KSP preconditioned resid norm 5.687556145039e+16 true resid norm >>>> 9.761935947260e+15 ||r(i)||/||b|| 1.866291596982e-01 >>>> 15 KSP preconditioned resid norm 1.615122498031e+17 true resid norm >>>> 4.345955527842e+16 ||r(i)||/||b|| 8.308618624718e-01 >>>> 16 KSP preconditioned resid norm 4.126010760136e+16 true resid norm >>>> 7.099883585094e+15 ||r(i)||/||b|| 1.357359149456e-01 >>>> 17 KSP preconditioned resid norm 2.161057719247e+16 true resid norm >>>> 5.853862804145e+15 ||r(i)||/||b|| 1.119144298865e-01 >>>> 18 KSP preconditioned resid norm 1.411351808998e+16 true resid norm >>>> 2.426558230375e+15 ||r(i)||/||b|| 4.639105664496e-02 >>>> 19 KSP preconditioned resid norm 1.083945264815e+16 true resid norm >>>> 2.485250997240e+15 ||r(i)||/||b|| 4.751314777723e-02 >>>> 20 KSP preconditioned resid norm 9.014213050640e+15 true resid norm >>>> 1.580604299892e+15 ||r(i)||/||b|| 3.021806882344e-02 >>>> 21 KSP preconditioned resid norm 7.767372671831e+15 true resid norm >>>> 1.339058517057e+15 ||r(i)||/||b|| 2.560018496078e-02 >>>> 22 KSP preconditioned resid norm 6.894209717493e+15 true resid norm >>>> 1.176401403286e+15 ||r(i)||/||b|| 2.249049845740e-02 >>>> 23 KSP preconditioned resid norm 6.197326579110e+15 true resid norm >>>> 1.015741229888e+15 ||r(i)||/||b|| 1.941898955585e-02 >>>> 24 KSP preconditioned resid norm 5.635415722302e+15 true resid norm >>>> 9.407688590092e+14 ||r(i)||/||b|| 1.798566417314e-02 >>>> 25 KSP preconditioned resid norm 5.240066189102e+15 true resid norm >>>> 8.479712706847e+14 ||r(i)||/||b|| 1.621155542826e-02 >>>> 26 KSP preconditioned resid norm 5.077381341025e+15 true resid norm >>>> 8.489262350542e+14 ||r(i)||/||b|| 1.622981248288e-02 >>>> 27 KSP preconditioned resid norm 4.908170126730e+15 true resid norm >>>> 9.447288544322e+14 ||r(i)||/||b|| 1.806137155559e-02 >>>> 28 KSP preconditioned resid norm 4.807291179172e+15 true resid norm >>>> 7.514342780137e+14 ||r(i)||/||b|| 1.436595657171e-02 >>>> 29 KSP preconditioned resid norm 4.874566711284e+15 true resid norm >>>> 7.721244730622e+14 ||r(i)||/||b|| 1.476151271311e-02 >>>> 30 KSP preconditioned resid norm 5.131077173970e+15 true resid norm >>>> 8.050523689285e+14 ||r(i)||/||b|| 1.539102980576e-02 >>>> 31 KSP preconditioned resid norm 5.679198106771e+15 true resid norm >>>> 8.990076632730e+14 ||r(i)||/||b|| 1.718727162987e-02 >>>> 32 KSP preconditioned resid norm 8.479698883566e+15 true resid norm >>>> 1.362468331537e+15 ||r(i)||/||b|| 2.604773491692e-02 >>>> 33 KSP preconditioned resid norm 1.652298710742e+16 true resid norm >>>> 2.658951324175e+15 ||r(i)||/||b|| 5.083395895960e-02 >>>> 34 KSP preconditioned resid norm 3.352309602032e+16 true resid norm >>>> 5.485872403557e+15 ||r(i)||/||b|| 1.048791717564e-01 >>>> 35 KSP preconditioned resid norm 6.378357733800e+15 true resid norm >>>> 1.047516366293e+15 ||r(i)||/||b|| 2.002646813784e-02 >>>> 36 KSP preconditioned resid norm 3.388128874134e+15 true resid norm >>>> 5.554504439336e+14 ||r(i)||/||b|| 1.061912823085e-02 >>>> 37 KSP preconditioned resid norm 2.152326148598e+15 true resid norm >>>> 3.939853481438e+14 ||r(i)||/||b|| 7.532230784419e-03 >>>> 38 KSP preconditioned resid norm 1.818288964313e+15 true resid norm >>>> 2.940442100180e+14 ||r(i)||/||b|| 5.621551311776e-03 >>>> 39 KSP preconditioned resid norm 1.660791920226e+15 true resid norm >>>> 2.869955671543e+14 ||r(i)||/||b|| 5.486795019399e-03 >>>> 40 KSP preconditioned resid norm 1.530575944029e+15 true resid norm >>>> 2.452126997689e+14 ||r(i)||/||b|| 4.687988156493e-03 >>>> 41 KSP preconditioned resid norm 1.370742276038e+15 true resid norm >>>> 2.241019638220e+14 ||r(i)||/||b|| 4.284392094026e-03 >>>> 42 KSP preconditioned resid norm 1.242467362496e+15 true resid norm >>>> 2.048557595739e+14 ||r(i)||/||b|| 3.916442238011e-03 >>>> 43 KSP preconditioned resid norm 1.151462192104e+15 true resid norm >>>> 1.847175725318e+14 ||r(i)||/||b|| 3.531439412157e-03 >>>> 44 KSP preconditioned resid norm 1.085430472672e+15 true resid norm >>>> 1.860586924987e+14 ||r(i)||/||b|| 3.557079008015e-03 >>>> 45 KSP preconditioned resid norm 1.036632239565e+15 true resid norm >>>> 1.638613644087e+14 ||r(i)||/||b|| 3.132709424833e-03 >>>> 46 KSP preconditioned resid norm 1.028748911897e+15 true resid norm >>>> 1.653234224458e+14 ||r(i)||/||b|| 3.160661120519e-03 >>>> 47 KSP preconditioned resid norm 9.769896259678e+14 true resid norm >>>> 1.886232302190e+14 ||r(i)||/||b|| 3.606107963166e-03 >>>> 48 KSP preconditioned resid norm 1.056096824670e+15 true resid norm >>>> 1.655410058571e+14 ||r(i)||/||b|| 3.164820890613e-03 >>>> 49 KSP preconditioned resid norm 7.777695958066e+14 true resid norm >>>> 2.880991932726e+14 ||r(i)||/||b|| 5.507894196468e-03 >>>> 50 KSP preconditioned resid norm 1.009825961040e+15 true resid norm >>>> 1.550789761056e+14 ||r(i)||/||b|| 2.964807303982e-03 >>>> 51 KSP preconditioned resid norm 9.471596692833e+14 true resid norm >>>> 2.053798093618e+14 ||r(i)||/||b|| 3.926461046995e-03 >>>> 52 KSP preconditioned resid norm 8.991518749617e+14 true resid norm >>>> 1.386233583178e+14 ||r(i)||/||b|| 2.650208013776e-03 >>>> 53 KSP preconditioned resid norm 7.186213371144e+13 true resid norm >>>> 1.103090149144e+13 ||r(i)||/||b|| 2.108893038414e-04 >>>> 0 KSP preconditioned resid norm 6.542826642292e+18 true resid norm >>>> 3.545903145424e+18 ||r(i)||/||b|| 4.156717341790e+02 >>>> 1 KSP preconditioned resid norm 1.591157510957e+18 true resid norm >>>> 5.223152980213e+17 ||r(i)||/||b|| 6.122888776501e+01 >>>> 2 KSP preconditioned resid norm 6.570525108060e+17 true resid norm >>>> 1.552502546069e+17 ||r(i)||/||b|| 1.819935286374e+01 >>>> 3 KSP preconditioned resid norm 4.165816250413e+17 true resid norm >>>> 8.738489300540e+16 ||r(i)||/||b|| 1.024377387845e+01 >>>> 4 KSP preconditioned resid norm 2.995010683652e+17 true resid norm >>>> 4.934680552583e+16 ||r(i)||/||b|| 5.784724339013e+00 >>>> 5 KSP preconditioned resid norm 2.594493782706e+17 true resid norm >>>> 4.469794627497e+16 ||r(i)||/||b|| 5.239757568206e+00 >>>> 6 KSP preconditioned resid norm 2.219609972421e+17 true resid norm >>>> 4.652118621437e+16 ||r(i)||/||b|| 5.453488535001e+00 >>>> 7 KSP preconditioned resid norm 1.674912012123e+17 true resid norm >>>> 2.988815541157e+16 ||r(i)||/||b|| 3.503666310619e+00 >>>> 8 KSP preconditioned resid norm 1.081901407775e+17 true resid norm >>>> 1.908145564246e+16 ||r(i)||/||b|| 2.236841062001e+00 >>>> 9 KSP preconditioned resid norm 5.710123690945e+16 true resid norm >>>> 9.635193037737e+15 ||r(i)||/||b|| 1.129494302267e+00 >>>> 10 KSP preconditioned resid norm 1.779373863118e+16 true resid norm >>>> 2.952490629534e+15 ||r(i)||/||b|| 3.461084101266e-01 >>>> 11 KSP preconditioned resid norm 1.125622359514e+16 true resid norm >>>> 2.030418337832e+15 ||r(i)||/||b|| 2.380176437376e-01 >>>> 12 KSP preconditioned resid norm 1.086650353983e+16 true resid norm >>>> 1.877894898988e+15 ||r(i)||/||b|| 2.201379443417e-01 >>>> 13 KSP preconditioned resid norm 1.075411372217e+16 true resid norm >>>> 1.822725565845e+15 ||r(i)||/||b|| 2.136706688859e-01 >>>> 14 KSP preconditioned resid norm 1.173963298627e+16 true resid norm >>>> 2.130483772307e+15 ||r(i)||/||b|| 2.497479056692e-01 >>>> 15 KSP preconditioned resid norm 4.943816039086e+16 true resid norm >>>> 1.362098103563e+16 ||r(i)||/||b|| 1.596731939959e+00 >>>> 16 KSP preconditioned resid norm 1.779827523151e+17 true resid norm >>>> 3.369509176778e+16 ||r(i)||/||b|| 3.949937901296e+00 >>>> 17 KSP preconditioned resid norm 3.250308535899e+16 true resid norm >>>> 8.719651405683e+15 ||r(i)||/||b|| 1.022169098418e+00 >>>> 18 KSP preconditioned resid norm 2.074788387365e+16 true resid norm >>>> 3.942900424212e+15 ||r(i)||/||b|| 4.622101027047e-01 >>>> 19 KSP preconditioned resid norm 1.482962263858e+16 true resid norm >>>> 3.390403569469e+15 ||r(i)||/||b|| 3.974431543926e-01 >>>> 20 KSP preconditioned resid norm 1.133931148892e+16 true resid norm >>>> 2.488262609273e+15 ||r(i)||/||b|| 2.916888565398e-01 >>>> 21 KSP preconditioned resid norm 8.550489811072e+15 true resid norm >>>> 1.657275963132e+15 ||r(i)||/||b|| 1.942756881268e-01 >>>> 22 KSP preconditioned resid norm 6.584107021814e+15 true resid norm >>>> 1.285719379545e+15 ||r(i)||/||b|| 1.507196283273e-01 >>>> 23 KSP preconditioned resid norm 4.778317630297e+15 true resid norm >>>> 9.061620975056e+14 ||r(i)||/||b|| 1.062256793459e-01 >>>> 24 KSP preconditioned resid norm 3.026263700342e+15 true resid norm >>>> 6.175894175717e+14 ||r(i)||/||b|| 7.239748342928e-02 >>>> 25 KSP preconditioned resid norm 1.568620488361e+15 true resid norm >>>> 2.753144451691e+14 ||r(i)||/||b|| 3.227398723952e-02 >>>> 26 KSP preconditioned resid norm 1.178330886619e+15 true resid norm >>>> 2.275547590347e+14 ||r(i)||/||b|| 2.667531442045e-02 >>>> 27 KSP preconditioned resid norm 1.073508363176e+15 true resid norm >>>> 2.033900922703e+14 ||r(i)||/||b|| 2.384258929293e-02 >>>> 28 KSP preconditioned resid norm 1.011358091649e+15 true resid norm >>>> 1.668017917552e+14 ||r(i)||/||b|| 1.955349235427e-02 >>>> 29 KSP preconditioned resid norm 9.448460925412e+14 true resid norm >>>> 1.798337339042e+14 ||r(i)||/||b|| 2.108117367287e-02 >>>> 30 KSP preconditioned resid norm 8.970299972493e+14 true resid norm >>>> 1.456018479256e+14 ||r(i)||/||b|| 1.706830958004e-02 >>>> 31 KSP preconditioned resid norm 8.656072021210e+14 true resid norm >>>> 1.578084000747e+14 ||r(i)||/||b|| 1.849923380218e-02 >>>> 32 KSP preconditioned resid norm 8.314911059315e+14 true resid norm >>>> 1.333543211106e+14 ||r(i)||/||b|| 1.563258206526e-02 >>>> 33 KSP preconditioned resid norm 8.004072124456e+14 true resid norm >>>> 1.582598207144e+14 ||r(i)||/||b|| 1.855215199888e-02 >>>> 34 KSP preconditioned resid norm 7.705686923615e+14 true resid norm >>>> 1.334906781329e+14 ||r(i)||/||b|| 1.564856664172e-02 >>>> 35 KSP preconditioned resid norm 7.648238175354e+14 true resid norm >>>> 1.202157111000e+14 ||r(i)||/||b|| 1.409239650919e-02 >>>> 36 KSP preconditioned resid norm 7.709241111814e+14 true resid norm >>>> 1.292601566237e+14 ||r(i)||/||b|| 1.515263989468e-02 >>>> 37 KSP preconditioned resid norm 8.026855071425e+14 true resid norm >>>> 1.264132558814e+14 ||r(i)||/||b|| 1.481890935550e-02 >>>> 38 KSP preconditioned resid norm 8.397026301556e+14 true resid norm >>>> 1.379015425365e+14 ||r(i)||/||b|| 1.616563424923e-02 >>>> 39 KSP preconditioned resid norm 8.772641473437e+14 true resid norm >>>> 1.384855714837e+14 ||r(i)||/||b|| 1.623409757587e-02 >>>> 40 KSP preconditioned resid norm 9.403663540144e+14 true resid norm >>>> 1.523096876425e+14 ||r(i)||/||b|| 1.785464221614e-02 >>>> 41 KSP preconditioned resid norm 1.191025293024e+15 true resid norm >>>> 1.883988208054e+14 ||r(i)||/||b|| 2.208522380610e-02 >>>> 42 KSP preconditioned resid norm 2.105592809750e+15 true resid norm >>>> 3.674437948704e+14 ||r(i)||/||b|| 4.307393438656e-02 >>>> 43 KSP preconditioned resid norm 1.653069974154e+16 true resid norm >>>> 2.622043044873e+15 ||r(i)||/||b|| 3.073713902652e-01 >>>> 44 KSP preconditioned resid norm 1.224677806349e+15 true resid norm >>>> 2.078786938958e+14 ||r(i)||/||b|| 2.436876971726e-02 >>>> 45 KSP preconditioned resid norm 6.142260967855e+14 true resid norm >>>> 9.787693095074e+13 ||r(i)||/||b|| 1.147371260744e-02 >>>> 46 KSP preconditioned resid norm 4.190317366481e+14 true resid norm >>>> 7.181248026289e+13 ||r(i)||/||b|| 8.418283574692e-03 >>>> 47 KSP preconditioned resid norm 2.851377922408e+14 true resid norm >>>> 4.551212829362e+13 ||r(i)||/||b|| 5.335200798815e-03 >>>> 48 KSP preconditioned resid norm 1.874725496828e+14 true resid norm >>>> 3.515118139860e+13 ||r(i)||/||b|| 4.120629337903e-03 >>>> 49 KSP preconditioned resid norm 1.260137969667e+14 true resid norm >>>> 2.018341951104e+13 ||r(i)||/||b|| 2.366019782758e-03 >>>> 50 KSP preconditioned resid norm 8.902880553848e+13 true resid norm >>>> 2.353973309483e+13 ||r(i)||/||b|| 2.759466707450e-03 >>>> 51 KSP preconditioned resid norm 6.273228119323e+13 true resid norm >>>> 1.015825242680e+13 ||r(i)||/||b|| 1.190810416783e-03 >>>> 52 KSP preconditioned resid norm 4.428322011916e+13 true resid norm >>>> 1.394037459429e+13 ||r(i)||/||b|| 1.634173141528e-03 >>>> 0 KSP preconditioned resid norm 1.892236961136e+19 true resid norm >>>> 1.028105836629e+19 ||r(i)||/||b|| 7.839481546009e+01 >>>> 1 KSP preconditioned resid norm 4.580244542528e+18 true resid norm >>>> 1.519483928522e+18 ||r(i)||/||b|| 1.158632291804e+01 >>>> 2 KSP preconditioned resid norm 1.874052673028e+18 true resid norm >>>> 4.645668462025e+17 ||r(i)||/||b|| 3.542401071890e+00 >>>> 3 KSP preconditioned resid norm 1.172977406681e+18 true resid norm >>>> 2.550460309107e+17 ||r(i)||/||b|| 1.944769284904e+00 >>>> 4 KSP preconditioned resid norm 8.299170411979e+17 true resid norm >>>> 1.381052822672e+17 ||r(i)||/||b|| 1.053076223446e+00 >>>> 5 KSP preconditioned resid norm 7.051011041666e+17 true resid norm >>>> 1.287511951247e+17 ||r(i)||/||b|| 9.817497209388e-01 >>>> 6 KSP preconditioned resid norm 5.862228682149e+17 true resid norm >>>> 1.166604371743e+17 ||r(i)||/||b|| 8.895556389170e-01 >>>> 7 KSP preconditioned resid norm 4.329574557115e+17 true resid norm >>>> 8.609505694316e+16 ||r(i)||/||b|| 6.564894255647e-01 >>>> 8 KSP preconditioned resid norm 2.701974155126e+17 true resid norm >>>> 4.802624326608e+16 ||r(i)||/||b|| 3.662082583277e-01 >>>> 9 KSP preconditioned resid norm 1.546976756797e+17 true resid norm >>>> 2.878555741954e+16 ||r(i)||/||b|| 2.194947622532e-01 >>>> 10 KSP preconditioned resid norm 9.916116392638e+16 true resid norm >>>> 1.649766880508e+16 ||r(i)||/||b|| 1.257975254509e-01 >>>> 11 KSP preconditioned resid norm 9.248895040376e+16 true resid norm >>>> 1.560136782751e+16 ||r(i)||/||b|| 1.189630783317e-01 >>>> 12 KSP preconditioned resid norm 9.251831545179e+16 true resid norm >>>> 1.508056895396e+16 ||r(i)||/||b|| 1.149918985048e-01 >>>> 13 KSP preconditioned resid norm 7.096157383949e+16 true resid norm >>>> 1.151221892162e+16 ||r(i)||/||b|| 8.778262370879e-02 >>>> 14 KSP preconditioned resid norm 2.017629866031e+17 true resid norm >>>> 3.268014415733e+16 ||r(i)||/||b|| 2.491916473135e-01 >>>> 15 KSP preconditioned resid norm 1.669814416925e+16 true resid norm >>>> 4.397468530146e+15 ||r(i)||/||b|| 3.353144410138e-02 >>>> 16 KSP preconditioned resid norm 1.171914014315e+16 true resid norm >>>> 1.865370183430e+15 ||r(i)||/||b|| 1.422376433288e-02 >>>> 17 KSP preconditioned resid norm 1.021030261775e+16 true resid norm >>>> 2.000923353135e+15 ||r(i)||/||b|| 1.525738026477e-02 >>>> 18 KSP preconditioned resid norm 9.416598060128e+15 true resid norm >>>> 1.541246281727e+15 ||r(i)||/||b|| 1.175226455582e-02 >>>> 19 KSP preconditioned resid norm 8.821999311745e+15 true resid norm >>>> 1.426601243635e+15 ||r(i)||/||b|| 1.087807667707e-02 >>>> 20 KSP preconditioned resid norm 8.323362047556e+15 true resid norm >>>> 1.490210405686e+15 ||r(i)||/||b|| 1.136310733664e-02 >>>> 21 KSP preconditioned resid norm 7.861641988008e+15 true resid norm >>>> 1.251748347072e+15 ||r(i)||/||b|| 9.544793655962e-03 >>>> 22 KSP preconditioned resid norm 7.464952525350e+15 true resid norm >>>> 1.325467789570e+15 ||r(i)||/||b|| 1.010691692037e-02 >>>> 23 KSP preconditioned resid norm 7.139096535480e+15 true resid norm >>>> 1.125905927178e+15 ||r(i)||/||b|| 8.585223839986e-03 >>>> 24 KSP preconditioned resid norm 5.858113828393e+15 true resid norm >>>> 2.042448137714e+15 ||r(i)||/||b|| 1.557401379686e-02 >>>> 25 KSP preconditioned resid norm 5.027356938946e+15 true resid norm >>>> 7.946522117539e+14 ||r(i)||/||b|| 6.059358022871e-03 >>>> 26 KSP preconditioned resid norm 5.122614017475e+15 true resid norm >>>> 7.976827488389e+14 ||r(i)||/||b|| 6.082466382639e-03 >>>> 27 KSP preconditioned resid norm 5.425317067876e+15 true resid norm >>>> 8.948524578548e+14 ||r(i)||/||b|| 6.823401910404e-03 >>>> 28 KSP preconditioned resid norm 9.316529191423e+15 true resid norm >>>> 1.453760069771e+15 ||r(i)||/||b|| 1.108516733711e-02 >>>> 29 KSP preconditioned resid norm 9.523628080058e+15 true resid norm >>>> 2.109739324167e+15 ||r(i)||/||b|| 1.608712051759e-02 >>>> 30 KSP preconditioned resid norm 2.300173794637e+15 true resid norm >>>> 3.594261469430e+14 ||r(i)||/||b|| 2.740685390281e-03 >>>> 31 KSP preconditioned resid norm 1.385027169120e+15 true resid norm >>>> 2.935564708865e+14 ||r(i)||/||b|| 2.238417927644e-03 >>>> 32 KSP preconditioned resid norm 9.719758142764e+14 true resid norm >>>> 1.523176597454e+14 ||r(i)||/||b|| 1.161448014555e-03 >>>> 33 KSP preconditioned resid norm 8.580268959656e+14 true resid norm >>>> 1.580637131624e+14 ||r(i)||/||b|| 1.205262647368e-03 >>>> 0 KSP preconditioned resid norm 6.470797769938e+18 true resid norm >>>> 3.511548722307e+18 ||r(i)||/||b|| 1.662246831388e+02 >>>> 1 KSP preconditioned resid norm 1.571597715354e+18 true resid norm >>>> 5.187477038218e+17 ||r(i)||/||b|| 2.455573865428e+01 >>>> 2 KSP preconditioned resid norm 6.468337546861e+17 true resid norm >>>> 1.566996108585e+17 ||r(i)||/||b|| 7.417622599812e+00 >>>> 3 KSP preconditioned resid norm 4.084044703454e+17 true resid norm >>>> 8.688412887362e+16 ||r(i)||/||b|| 4.112796926342e+00 >>>> 4 KSP preconditioned resid norm 2.920894102259e+17 true resid norm >>>> 4.831956260441e+16 ||r(i)||/||b|| 2.287282512214e+00 >>>> 5 KSP preconditioned resid norm 2.515869150351e+17 true resid norm >>>> 4.435569840236e+16 ||r(i)||/||b|| 2.099646764259e+00 >>>> 6 KSP preconditioned resid norm 2.133085363553e+17 true resid norm >>>> 4.355632900946e+16 ||r(i)||/||b|| 2.061807356479e+00 >>>> 7 KSP preconditioned resid norm 1.601295664581e+17 true resid norm >>>> 2.999352167191e+16 ||r(i)||/||b|| 1.419790534148e+00 >>>> 8 KSP preconditioned resid norm 1.019894513782e+17 true resid norm >>>> 1.811122934468e+16 ||r(i)||/||b|| 8.573235336161e-01 >>>> 9 KSP preconditioned resid norm 5.552366492146e+16 true resid norm >>>> 9.849650352808e+15 ||r(i)||/||b|| 4.662486949199e-01 >>>> 10 KSP preconditioned resid norm 2.614329828139e+16 true resid norm >>>> 4.400937150634e+15 ||r(i)||/||b|| 2.083252835795e-01 >>>> 11 KSP preconditioned resid norm 2.230447954838e+16 true resid norm >>>> 4.002038016135e+15 ||r(i)||/||b|| 1.894427654999e-01 >>>> 12 KSP preconditioned resid norm 2.202888172936e+16 true resid norm >>>> 3.669015023581e+15 ||r(i)||/||b|| 1.736785982356e-01 >>>> 13 KSP preconditioned resid norm 2.171444770780e+16 true resid norm >>>> 3.746639147837e+15 ||r(i)||/||b|| 1.773530582755e-01 >>>> 14 KSP preconditioned resid norm 2.756620760322e+16 true resid norm >>>> 4.738417417687e+15 ||r(i)||/||b|| 2.243004429444e-01 >>>> 15 KSP preconditioned resid norm 1.911455835947e+16 true resid norm >>>> 7.274462982878e+15 ||r(i)||/||b|| 3.443481494796e-01 >>>> 16 KSP preconditioned resid norm 8.270124664006e+15 true resid norm >>>> 1.450957167614e+15 ||r(i)||/||b|| 6.868334017480e-02 >>>> 17 KSP preconditioned resid norm 5.454864800054e+15 true resid norm >>>> 1.779587764442e+15 ||r(i)||/||b|| 8.423958647728e-02 >>>> 18 KSP preconditioned resid norm 4.188178169772e+15 true resid norm >>>> 7.356207570842e+14 ||r(i)||/||b|| 3.482176581514e-02 >>>> 19 KSP preconditioned resid norm 3.485885419242e+15 true resid norm >>>> 7.684000525717e+14 ||r(i)||/||b|| 3.637342533543e-02 >>>> 20 KSP preconditioned resid norm 3.031526098420e+15 true resid norm >>>> 5.668894180734e+14 ||r(i)||/||b|| 2.683460243492e-02 >>>> 21 KSP preconditioned resid norm 2.653824581081e+15 true resid norm >>>> 5.337896513987e+14 ||r(i)||/||b|| 2.526777290683e-02 >>>> 22 KSP preconditioned resid norm 2.412624108418e+15 true resid norm >>>> 3.917881686680e+14 ||r(i)||/||b|| 1.854590932504e-02 >>>> 23 KSP preconditioned resid norm 2.249293018506e+15 true resid norm >>>> 3.520070992955e+14 ||r(i)||/||b|| 1.666280982271e-02 >>>> 24 KSP preconditioned resid norm 2.107969657718e+15 true resid norm >>>> 3.433991961929e+14 ||r(i)||/||b|| 1.625534118740e-02 >>>> 25 KSP preconditioned resid norm 1.995899579616e+15 true resid norm >>>> 3.124097002184e+14 ||r(i)||/||b|| 1.478840464277e-02 >>>> 26 KSP preconditioned resid norm 1.940326192847e+15 true resid norm >>>> 3.197343858652e+14 ||r(i)||/||b|| 1.513513016106e-02 >>>> 27 KSP preconditioned resid norm 1.843618399864e+15 true resid norm >>>> 4.284525431489e+14 ||r(i)||/||b|| 2.028147517148e-02 >>>> 28 KSP preconditioned resid norm 1.800569430365e+15 true resid norm >>>> 2.810328443449e+14 ||r(i)||/||b|| 1.330313180793e-02 >>>> 29 KSP preconditioned resid norm 1.911265522964e+15 true resid norm >>>> 3.078500227429e+14 ||r(i)||/||b|| 1.457256513619e-02 >>>> 30 KSP preconditioned resid norm 2.245578740852e+15 true resid norm >>>> 3.522723048093e+14 ||r(i)||/||b|| 1.667536374293e-02 >>>> 31 KSP preconditioned resid norm 3.129444954320e+15 true resid norm >>>> 5.056459987900e+14 ||r(i)||/||b|| 2.393554883500e-02 >>>> 32 KSP preconditioned resid norm 2.268235755551e+16 true resid norm >>>> 3.663405821329e+15 ||r(i)||/||b|| 1.734130778226e-01 >>>> 33 KSP preconditioned resid norm 2.666354866266e+15 true resid norm >>>> 4.420681584183e+14 ||r(i)||/||b|| 2.092599174035e-02 >>>> 34 KSP preconditioned resid norm 1.195729215971e+15 true resid norm >>>> 1.967322086000e+14 ||r(i)||/||b|| 9.312628593187e-03 >>>> 35 KSP preconditioned resid norm 7.390918117697e+14 true resid norm >>>> 1.247185996576e+14 ||r(i)||/||b|| 5.903751122089e-03 >>>> 36 KSP preconditioned resid norm 5.589675490221e+14 true resid norm >>>> 9.067132579829e+13 ||r(i)||/||b|| 4.292069850790e-03 >>>> 37 KSP preconditioned resid norm 4.470335591004e+14 true resid norm >>>> 9.109636100840e+13 ||r(i)||/||b|| 4.312189561126e-03 >>>> 38 KSP preconditioned resid norm 4.031608678043e+14 true resid norm >>>> 6.399106058589e+13 ||r(i)||/||b|| 3.029117523568e-03 >>>> 39 KSP preconditioned resid norm 3.769205171045e+14 true resid norm >>>> 7.200688789282e+13 ||r(i)||/||b|| 3.408559319641e-03 >>>> 40 KSP preconditioned resid norm 3.535103364248e+14 true resid norm >>>> 5.569365215364e+13 ||r(i)||/||b|| 2.636346641945e-03 >>>> 41 KSP preconditioned resid norm 3.299087996116e+14 true resid norm >>>> 5.691184302265e+13 ||r(i)||/||b|| 2.694011623187e-03 >>>> 42 KSP preconditioned resid norm 3.102396074559e+14 true resid norm >>>> 5.072539063269e+13 ||r(i)||/||b|| 2.401166166782e-03 >>>> 43 KSP preconditioned resid norm 2.923631153501e+14 true resid norm >>>> 5.690604160219e+13 ||r(i)||/||b|| 2.693737003823e-03 >>>> 44 KSP preconditioned resid norm 2.831787236990e+14 true resid norm >>>> 4.398532156025e+13 ||r(i)||/||b|| 2.082114393761e-03 >>>> 45 KSP preconditioned resid norm 2.880585984465e+14 true resid norm >>>> 4.666639157225e+13 ||r(i)||/||b|| 2.209027060639e-03 >>>> 46 KSP preconditioned resid norm 3.054400711070e+14 true resid norm >>>> 4.771426500640e+13 ||r(i)||/||b|| 2.258629798159e-03 >>>> 47 KSP preconditioned resid norm 3.761477931303e+14 true resid norm >>>> 5.957625439087e+13 ||r(i)||/||b|| 2.820135726954e-03 >>>> 48 KSP preconditioned resid norm 9.608784364697e+14 true resid norm >>>> 1.580857575603e+14 ||r(i)||/||b|| 7.483238034628e-03 >>>> 49 KSP preconditioned resid norm 1.902148260101e+15 true resid norm >>>> 2.982038509627e+14 ||r(i)||/||b|| 1.411594841961e-02 >>>> 50 KSP preconditioned resid norm 2.165055327372e+14 true resid norm >>>> 4.058692191960e+13 ||r(i)||/||b|| 1.921245800409e-03 >>>> 51 KSP preconditioned resid norm 2.195213865618e+14 true resid norm >>>> 3.429224338721e+13 ||r(i)||/||b|| 1.623277289290e-03 >>>> 52 KSP preconditioned resid norm 1.864119738188e+14 true resid norm >>>> 5.012858400476e+13 ||r(i)||/||b|| 2.372915386152e-03 >>>> 53 KSP preconditioned resid norm 2.971924491921e+13 true resid norm >>>> 4.598158953060e+12 ||r(i)||/||b|| 2.176610878666e-04 >>>> 0 KSP preconditioned resid norm 6.694681472101e+18 true resid norm >>>> 3.650754008704e+18 ||r(i)||/||b|| 5.229798534711e+00 >>>> 1 KSP preconditioned resid norm 1.597667311920e+18 true resid norm >>>> 5.366749653247e+17 ||r(i)||/||b|| 7.688006205237e-01 >>>> 2 KSP preconditioned resid norm 6.300446691769e+17 true resid norm >>>> 1.705328302298e+17 ||r(i)||/||b|| 2.442926429799e-01 >>>> 3 KSP preconditioned resid norm 3.642174434692e+17 true resid norm >>>> 9.481577406412e+16 ||r(i)||/||b|| 1.358260225383e-01 >>>> 4 KSP preconditioned resid norm 2.247332932327e+17 true resid norm >>>> 3.802800659066e+16 ||r(i)||/||b|| 5.447609251997e-02 >>>> 5 KSP preconditioned resid norm 1.557523729879e+17 true resid norm >>>> 3.829141784861e+16 ||r(i)||/||b|| 5.485343588727e-02 >>>> 6 KSP preconditioned resid norm 1.004692897232e+17 true resid norm >>>> 1.699728149080e+16 ||r(i)||/||b|| 2.434904067015e-02 >>>> 7 KSP preconditioned resid norm 7.103483293598e+16 true resid norm >>>> 1.472858902835e+16 ||r(i)||/||b|| 2.109908066531e-02 >>>> 8 KSP preconditioned resid norm 5.766848376574e+16 true resid norm >>>> 9.397954931890e+15 ||r(i)||/||b|| 1.346281091931e-02 >>>> 9 KSP preconditioned resid norm 4.523472389397e+16 true resid norm >>>> 1.246375028587e+16 ||r(i)||/||b|| 1.785464121292e-02 >>>> 10 KSP preconditioned resid norm 3.195342596604e+16 true resid norm >>>> 5.030521805863e+15 ||r(i)||/||b|| 7.206351210301e-03 >>>> 11 KSP preconditioned resid norm 2.437460193025e+16 true resid norm >>>> 4.974645265358e+15 ||r(i)||/||b|| 7.126306636231e-03 >>>> 12 KSP preconditioned resid norm 1.258045317564e+16 true resid norm >>>> 1.885909944349e+15 ||r(i)||/||b|| 2.701614252848e-03 >>>> 13 KSP preconditioned resid norm 2.495567071066e+15 true resid norm >>>> 4.460991862355e+14 ||r(i)||/||b|| 6.390484992820e-04 >>>> 14 KSP preconditioned resid norm 7.678105595258e+14 true resid norm >>>> 1.316278119117e+14 ||r(i)||/||b|| 1.885602087190e-04 >>>> 15 KSP preconditioned resid norm 5.950875767888e+14 true resid norm >>>> 1.439411571316e+14 ||r(i)||/||b|| 2.061993908263e-04 >>>> 16 KSP preconditioned resid norm 3.263039815738e+14 true resid norm >>>> 5.046549667518e+13 ||r(i)||/||b|| 7.229311532250e-05 >>>> 17 KSP preconditioned resid norm 1.436785883111e+14 true resid norm >>>> 7.588987254543e+13 ||r(i)||/||b|| 1.087141843277e-04 >>>> 18 KSP preconditioned resid norm 5.401544347363e+13 true resid norm >>>> 8.625334293000e+12 ||r(i)||/||b|| 1.235601208391e-05 >>>> 19 KSP preconditioned resid norm 3.528863285477e+13 true resid norm >>>> 1.233589079533e+13 ||r(i)||/||b|| 1.767147922099e-05 >>>> 0 KSP preconditioned resid norm 4.827871909881e+18 true resid norm >>>> 2.629048338874e+18 ||r(i)||/||b|| 2.387243403092e+01 >>>> 1 KSP preconditioned resid norm 1.161059375885e+18 true resid norm >>>> 3.885115498370e+17 ||r(i)||/||b|| 3.527784638492e+00 >>>> 2 KSP preconditioned resid norm 4.696929326234e+17 true resid norm >>>> 1.228954029189e+17 ||r(i)||/||b|| 1.115921816843e+00 >>>> 3 KSP preconditioned resid norm 2.870100194440e+17 true resid norm >>>> 6.759377089469e+16 ||r(i)||/||b|| 6.137687971441e-01 >>>> 4 KSP preconditioned resid norm 1.966638027618e+17 true resid norm >>>> 3.310588396768e+16 ||r(i)||/||b|| 3.006099276942e-01 >>>> 5 KSP preconditioned resid norm 1.592573327953e+17 true resid norm >>>> 3.239413757211e+16 ||r(i)||/||b|| 2.941470876529e-01 >>>> 6 KSP preconditioned resid norm 1.230114084345e+17 true resid norm >>>> 2.280735920899e+16 ||r(i)||/||b|| 2.070966783247e-01 >>>> 7 KSP preconditioned resid norm 8.598639005636e+16 true resid norm >>>> 1.938027655308e+16 ||r(i)||/||b|| 1.759778877677e-01 >>>> 8 KSP preconditioned resid norm 5.549090128976e+16 true resid norm >>>> 9.386098364471e+15 ||r(i)||/||b|| 8.522818340779e-02 >>>> 9 KSP preconditioned resid norm 4.474765035679e+16 true resid norm >>>> 7.579175521646e+15 ||r(i)||/||b|| 6.882085999480e-02 >>>> 10 KSP preconditioned resid norm 4.378403606928e+16 true resid norm >>>> 6.842614495131e+15 ||r(i)||/||b|| 6.213269673237e-02 >>>> 11 KSP preconditioned resid norm 4.079517449398e+16 true resid norm >>>> 7.704845808673e+15 ||r(i)||/||b|| 6.996197847191e-02 >>>> 12 KSP preconditioned resid norm 4.084405622719e+16 true resid norm >>>> 6.377952548774e+15 ||r(i)||/||b|| 5.791344693880e-02 >>>> 13 KSP preconditioned resid norm 7.523868588173e+16 true resid norm >>>> 1.132249240798e+16 ||r(i)||/||b|| 1.028111385699e-01 >>>> 14 KSP preconditioned resid norm 2.184581375390e+15 true resid norm >>>> 3.502138477296e+14 ||r(i)||/||b|| 3.180031668881e-03 >>>> 15 KSP preconditioned resid norm 2.222269400035e+15 true resid norm >>>> 3.654767263686e+14 ||r(i)||/||b|| 3.318622526282e-03 >>>> 16 KSP preconditioned resid norm 1.669508240008e+15 true resid norm >>>> 3.629680148217e+14 ||r(i)||/||b|| 3.295842781223e-03 >>>> 17 KSP preconditioned resid norm 1.401913603747e+15 true resid norm >>>> 2.161155966391e+14 ||r(i)||/||b|| 1.962385113858e-03 >>>> 18 KSP preconditioned resid norm 1.134372518022e+15 true resid norm >>>> 2.686701598571e+14 ||r(i)||/||b|| 2.439594043376e-03 >>>> 19 KSP preconditioned resid norm 9.057728167563e+14 true resid norm >>>> 1.409896456102e+14 ||r(i)||/||b|| 1.280222186904e-03 >>>> 20 KSP preconditioned resid norm 7.989861774441e+14 true resid norm >>>> 1.659027150307e+14 ||r(i)||/||b|| 1.506439254676e-03 >>>> 21 KSP preconditioned resid norm 7.119014585200e+14 true resid norm >>>> 1.092105820565e+14 ||r(i)||/||b|| 9.916601292842e-04 >>>> 22 KSP preconditioned resid norm 6.306860453827e+14 true resid norm >>>> 1.460692513610e+14 ||r(i)||/||b|| 1.326346311515e-03 >>>> 23 KSP preconditioned resid norm 5.637172379886e+14 true resid norm >>>> 8.672554380666e+13 ||r(i)||/||b|| 7.874902080372e-04 >>>> 24 KSP preconditioned resid norm 4.961067656606e+14 true resid norm >>>> 1.292132364829e+14 ||r(i)||/||b|| 1.173289367962e-03 >>>> 25 KSP preconditioned resid norm 4.386188245672e+14 true resid norm >>>> 8.044020700160e+13 ||r(i)||/||b|| 7.304177358342e-04 >>>> 26 KSP preconditioned resid norm 4.419078090293e+14 true resid norm >>>> 6.703649620362e+13 ||r(i)||/||b|| 6.087085998464e-04 >>>> 27 KSP preconditioned resid norm 4.436677280477e+14 true resid norm >>>> 1.368871842229e+14 ||r(i)||/||b|| 1.242970784036e-03 >>>> 28 KSP preconditioned resid norm 7.847692003240e+14 true resid norm >>>> 1.196853324186e+14 ||r(i)||/||b|| 1.086773552386e-03 >>>> 29 KSP preconditioned resid norm 1.771704167839e+15 true resid norm >>>> 6.548256652554e+14 ||r(i)||/||b|| 5.945985193355e-03 >>>> 30 KSP preconditioned resid norm 1.053899282998e+15 true resid norm >>>> 1.880673778730e+14 ||r(i)||/||b|| 1.707700084953e-03 >>>> 31 KSP preconditioned resid norm 2.462670432282e+14 true resid norm >>>> 3.919805233311e+13 ||r(i)||/||b|| 3.559283808617e-04 >>>> 32 KSP preconditioned resid norm 1.093017734594e+14 true resid norm >>>> 1.903326631551e+13 ||r(i)||/||b|| 1.728269457018e-04 >>>> 33 KSP preconditioned resid norm 6.432042300545e+13 true resid norm >>>> 1.002471721316e+13 ||r(i)||/||b|| 9.102700654495e-05 >>>> 34 KSP preconditioned resid norm 3.805689715853e+13 true resid norm >>>> 7.047382673791e+12 ||r(i)||/||b|| 6.399204437707e-05 >>>> 35 KSP preconditioned resid norm 2.527168724251e+13 true resid norm >>>> 4.009865388649e+12 ||r(i)||/||b|| 3.641060742320e-05 >>>> 36 KSP preconditioned resid norm 1.580947387777e+13 true resid norm >>>> 2.531811988645e+12 ||r(i)||/||b|| 2.298950300149e-05 >>>> 37 KSP preconditioned resid norm 1.075548264218e+13 true resid norm >>>> 1.788042513750e+12 ||r(i)||/||b|| 1.623588517670e-05 >>>> 38 KSP preconditioned resid norm 6.581274433602e+12 true resid norm >>>> 1.035380330637e+12 ||r(i)||/||b|| 9.401519277742e-06 >>>> 0 KSP preconditioned resid norm 9.555736051823e+15 true resid norm >>>> 5.892741751019e+15 ||r(i)||/||b|| 1.119235791581e-01 >>>> 1 KSP preconditioned resid norm 4.575036500991e+15 true resid norm >>>> 8.101073993664e+14 ||r(i)||/||b|| 1.538674584270e-02 >>>> 2 KSP preconditioned resid norm 3.765161451471e+15 true resid norm >>>> 6.105904357791e+14 ||r(i)||/||b|| 1.159722754867e-02 >>>> 3 KSP preconditioned resid norm 3.685525899039e+15 true resid norm >>>> 6.160361102521e+14 ||r(i)||/||b|| 1.170065977151e-02 >>>> 4 KSP preconditioned resid norm 3.856425370708e+15 true resid norm >>>> 6.278594964367e+14 ||r(i)||/||b|| 1.192522683307e-02 >>>> 5 KSP preconditioned resid norm 4.281981871434e+15 true resid norm >>>> 6.756473177150e+14 ||r(i)||/||b|| 1.283288310305e-02 >>>> 6 KSP preconditioned resid norm 5.205822014832e+15 true resid norm >>>> 7.760311477213e+14 ||r(i)||/||b|| 1.473951977891e-02 >>>> 7 KSP preconditioned resid norm 6.471690051175e+15 true resid norm >>>> 1.171341036231e+15 ||r(i)||/||b|| 2.224782397211e-02 >>>> 8 KSP preconditioned resid norm 7.016914241690e+15 true resid norm >>>> 1.103022939349e+15 ||r(i)||/||b|| 2.095022664859e-02 >>>> 9 KSP preconditioned resid norm 7.025590013206e+15 true resid norm >>>> 1.330551872513e+15 ||r(i)||/||b|| 2.527178928238e-02 >>>> 10 KSP preconditioned resid norm 6.405136813848e+15 true resid norm >>>> 1.017867183616e+15 ||r(i)||/||b|| 1.933282385541e-02 >>>> 11 KSP preconditioned resid norm 4.854020251396e+15 true resid norm >>>> 8.473369029366e+14 ||r(i)||/||b|| 1.609386308386e-02 >>>> 12 KSP preconditioned resid norm 2.353499572340e+15 true resid norm >>>> 3.616493792413e+14 ||r(i)||/||b|| 6.868974517339e-03 >>>> 13 KSP preconditioned resid norm 1.176204335431e+15 true resid norm >>>> 2.390307465685e+14 ||r(i)||/||b|| 4.540021914275e-03 >>>> 14 KSP preconditioned resid norm 1.050288991204e+15 true resid norm >>>> 1.923001174300e+14 ||r(i)||/||b|| 3.652445385304e-03 >>>> 15 KSP preconditioned resid norm 9.850224343838e+14 true resid norm >>>> 1.772358652449e+14 ||r(i)||/||b|| 3.366323051568e-03 >>>> 16 KSP preconditioned resid norm 8.835883676210e+14 true resid norm >>>> 1.426145250921e+14 ||r(i)||/||b|| 2.708743868757e-03 >>>> 17 KSP preconditioned resid norm 7.483591829964e+14 true resid norm >>>> 1.680673516819e+14 ||r(i)||/||b|| 3.192181217957e-03 >>>> 18 KSP preconditioned resid norm 6.652269160454e+14 true resid norm >>>> 1.088301440117e+14 ||r(i)||/||b|| 2.067061438079e-03 >>>> 19 KSP preconditioned resid norm 5.777832230782e+14 true resid norm >>>> 9.220881620683e+13 ||r(i)||/||b|| 1.751364844390e-03 >>>> 20 KSP preconditioned resid norm 5.145663259773e+14 true resid norm >>>> 8.255284700142e+13 ||r(i)||/||b|| 1.567964539511e-03 >>>> 21 KSP preconditioned resid norm 4.536494497263e+14 true resid norm >>>> 7.162151119367e+13 ||r(i)||/||b|| 1.360340604800e-03 >>>> 22 KSP preconditioned resid norm 4.039727797390e+14 true resid norm >>>> 6.757078541538e+13 ||r(i)||/||b|| 1.283403289972e-03 >>>> 23 KSP preconditioned resid norm 3.668438626905e+14 true resid norm >>>> 5.700238490686e+13 ||r(i)||/||b|| 1.082672753854e-03 >>>> 24 KSP preconditioned resid norm 3.368796010179e+14 true resid norm >>>> 6.388426371627e+13 ||r(i)||/||b|| 1.213383472264e-03 >>>> 0 KSP preconditioned resid norm 9.591788600089e+15 true resid norm >>>> 5.932754803820e+15 ||r(i)||/||b|| 5.703607413375e-01 >>>> 1 KSP preconditioned resid norm 4.627577110935e+15 true resid norm >>>> 8.119149369849e+14 ||r(i)||/||b|| 7.805554429176e-02 >>>> 2 KSP preconditioned resid norm 3.843546862524e+15 true resid norm >>>> 6.215691855461e+14 ||r(i)||/||b|| 5.975616272433e-02 >>>> 3 KSP preconditioned resid norm 3.802220665831e+15 true resid norm >>>> 6.332285028379e+14 ||r(i)||/||b|| 6.087706137495e-02 >>>> 4 KSP preconditioned resid norm 4.025305304206e+15 true resid norm >>>> 6.486950645643e+14 ||r(i)||/||b|| 6.236397932519e-02 >>>> 5 KSP preconditioned resid norm 4.519185013022e+15 true resid norm >>>> 7.100155628952e+14 ||r(i)||/||b|| 6.825918417416e-02 >>>> 6 KSP preconditioned resid norm 5.566187938033e+15 true resid norm >>>> 8.272878114077e+14 ||r(i)||/||b|| 7.953345537054e-02 >>>> 7 KSP preconditioned resid norm 7.046115999320e+15 true resid norm >>>> 1.260178524183e+15 ||r(i)||/||b|| 1.211505246783e-01 >>>> 8 KSP preconditioned resid norm 7.802424274220e+15 true resid norm >>>> 1.227649475790e+15 ||r(i)||/||b|| 1.180232604022e-01 >>>> 9 KSP preconditioned resid norm 8.076065684234e+15 true resid norm >>>> 1.496448814554e+15 ||r(i)||/||b|| 1.438649806820e-01 >>>> 10 KSP preconditioned resid norm 7.812303545308e+15 true resid norm >>>> 1.236539634992e+15 ||r(i)||/||b|| 1.188779388714e-01 >>>> 11 KSP preconditioned resid norm 6.692698687119e+15 true resid norm >>>> 1.122795851888e+15 ||r(i)||/||b|| 1.079428858312e-01 >>>> 12 KSP preconditioned resid norm 4.016544769049e+15 true resid norm >>>> 6.096078457742e+14 ||r(i)||/||b|| 5.860622836074e-02 >>>> 13 KSP preconditioned resid norm 1.711147574309e+15 true resid norm >>>> 2.966258404639e+14 ||r(i)||/||b|| 2.851689305581e-02 >>>> 14 KSP preconditioned resid norm 1.304839987953e+15 true resid norm >>>> 2.700754536307e+14 ||r(i)||/||b|| 2.596440288593e-02 >>>> 15 KSP preconditioned resid norm 1.171070442303e+15 true resid norm >>>> 2.199042966295e+14 ||r(i)||/||b|| 2.114106882828e-02 >>>> 16 KSP preconditioned resid norm 1.057836577493e+15 true resid norm >>>> 1.660135728859e+14 ||r(i)||/||b|| 1.596014459292e-02 >>>> 17 KSP preconditioned resid norm 8.905346433993e+14 true resid norm >>>> 1.613310037432e+14 ||r(i)||/||b|| 1.550997368650e-02 >>>> 18 KSP preconditioned resid norm 7.864866777768e+14 true resid norm >>>> 1.268576094498e+14 ||r(i)||/||b|| 1.219578468395e-02 >>>> 19 KSP preconditioned resid norm 6.743846682412e+14 true resid norm >>>> 1.066594941052e+14 ||r(i)||/||b|| 1.025398657793e-02 >>>> 20 KSP preconditioned resid norm 5.775523980161e+14 true resid norm >>>> 1.005326356082e+14 ||r(i)||/||b|| 9.664965175573e-03 >>>> 21 KSP preconditioned resid norm 4.897344092103e+14 true resid norm >>>> 7.691922473584e+13 ||r(i)||/||b|| 7.394828792721e-03 >>>> 22 KSP preconditioned resid norm 4.026909639422e+14 true resid norm >>>> 7.643768412999e+13 ||r(i)||/||b|| 7.348534640001e-03 >>>> 23 KSP preconditioned resid norm 3.340361168756e+14 true resid norm >>>> 5.217079213762e+13 ||r(i)||/||b|| 5.015574157998e-03 >>>> 24 KSP preconditioned resid norm 2.772587883161e+14 true resid norm >>>> 6.149190087025e+13 ||r(i)||/||b|| 5.911683075799e-03 >>>> 25 KSP preconditioned resid norm 2.446809593449e+14 true resid norm >>>> 3.772191549633e+13 ||r(i)||/||b|| 3.626493997916e-03 >>>> 26 KSP preconditioned resid norm 2.130893101456e+14 true resid norm >>>> 5.402695231969e+13 ||r(i)||/||b|| 5.194020922190e-03 >>>> 27 KSP preconditioned resid norm 1.886821885223e+14 true resid norm >>>> 2.887742158299e+13 ||r(i)||/||b|| 2.776205679591e-03 >>>> 28 KSP preconditioned resid norm 1.654479962436e+14 true resid norm >>>> 4.199650238007e+13 ||r(i)||/||b|| 4.037442473714e-03 >>>> 29 KSP preconditioned resid norm 1.453563006355e+14 true resid norm >>>> 2.222396036725e+13 ||r(i)||/||b|| 2.136557961634e-03 >>>> 30 KSP preconditioned resid norm 1.311692747504e+14 true resid norm >>>> 2.933890714503e+13 ||r(i)||/||b|| 2.820571788757e-03 >>>> 31 KSP preconditioned resid norm 1.173199137783e+14 true resid norm >>>> 1.793346595553e+13 ||r(i)||/||b|| 1.724080174451e-03 >>>> 32 KSP preconditioned resid norm 1.086053135569e+14 true resid norm >>>> 2.009773461598e+13 ||r(i)||/||b|| 1.932147744820e-03 >>>> 33 KSP preconditioned resid norm 9.929347161574e+13 true resid norm >>>> 1.516318812509e+13 ||r(i)||/||b|| 1.457752343733e-03 >>>> 34 KSP preconditioned resid norm 9.235712158960e+13 true resid norm >>>> 1.538423395802e+13 ||r(i)||/||b|| 1.479003157109e-03 >>>> 35 KSP preconditioned resid norm 8.718922541063e+13 true resid norm >>>> 1.331044977413e+13 ||r(i)||/||b|| 1.279634546134e-03 >>>> 36 KSP preconditioned resid norm 8.163106494702e+13 true resid norm >>>> 1.369060193385e+13 ||r(i)||/||b|| 1.316181458118e-03 >>>> 37 KSP preconditioned resid norm 7.751439049037e+13 true resid norm >>>> 1.182966275510e+13 ||r(i)||/||b|| 1.137275252708e-03 >>>> 38 KSP preconditioned resid norm 7.269591005999e+13 true resid norm >>>> 1.225102361382e+13 ||r(i)||/||b|| 1.177783869649e-03 >>>> 39 KSP preconditioned resid norm 6.809139852739e+13 true resid norm >>>> 1.039056641301e+13 ||r(i)||/||b|| 9.989240004362e-04 >>>> 40 KSP preconditioned resid norm 6.390023598760e+13 true resid norm >>>> 1.042637257280e+13 ||r(i)||/||b|| 1.002366318301e-03 >>>> 41 KSP preconditioned resid norm 5.919860950635e+13 true resid norm >>>> 9.048648521202e+12 ||r(i)||/||b|| 8.699152500509e-04 >>>> 42 KSP preconditioned resid norm 5.538975217584e+13 true resid norm >>>> 8.775858022438e+12 ||r(i)||/||b|| 8.436898292725e-04 >>>> 0 KSP preconditioned resid norm 2.200871087933e+16 true resid norm >>>> 1.375121498750e+16 ||r(i)||/||b|| 1.042688751858e-01 >>>> 1 KSP preconditioned resid norm 1.033852082172e+16 true resid norm >>>> 1.769345381246e+15 ||r(i)||/||b|| 1.341609835098e-02 >>>> 2 KSP preconditioned resid norm 8.201114088620e+15 true resid norm >>>> 1.330146487456e+15 ||r(i)||/||b|| 1.008586355500e-02 >>>> 3 KSP preconditioned resid norm 7.727276364323e+15 true resid norm >>>> 1.292967289192e+15 ||r(i)||/||b|| 9.803951506731e-03 >>>> 4 KSP preconditioned resid norm 7.788308934018e+15 true resid norm >>>> 1.241309862269e+15 ||r(i)||/||b|| 9.412257986909e-03 >>>> 5 KSP preconditioned resid norm 8.253685478294e+15 true resid norm >>>> 1.290166199902e+15 ||r(i)||/||b|| 9.782712188615e-03 >>>> 6 KSP preconditioned resid norm 9.476714658211e+15 true resid norm >>>> 1.410015893632e+15 ||r(i)||/||b|| 1.069147499743e-02 >>>> 7 KSP preconditioned resid norm 1.110541369388e+16 true resid norm >>>> 1.990482855418e+15 ||r(i)||/||b|| 1.509287787295e-02 >>>> 8 KSP preconditioned resid norm 1.149682905149e+16 true resid norm >>>> 1.789313061240e+15 ||r(i)||/||b|| 1.356750370206e-02 >>>> 9 KSP preconditioned resid norm 1.081618830942e+16 true resid norm >>>> 1.972959789140e+15 ||r(i)||/||b|| 1.496000885648e-02 >>>> 10 KSP preconditioned resid norm 8.800226542664e+15 true resid norm >>>> 1.343393349540e+15 ||r(i)||/||b|| 1.018630816375e-02 >>>> 11 KSP preconditioned resid norm 4.142701353678e+15 true resid norm >>>> 6.194874024708e+14 ||r(i)||/||b|| 4.697276182950e-03 >>>> 12 KSP preconditioned resid norm 3.300703256438e+15 true resid norm >>>> 1.133601440349e+15 ||r(i)||/||b|| 8.595556625482e-03 >>>> 13 KSP preconditioned resid norm 7.882205618989e+14 true resid norm >>>> 1.186911081255e+14 ||r(i)||/||b|| 8.999778092374e-04 >>>> 0 KSP preconditioned resid norm 1.928554431739e+16 true resid norm >>>> 1.203381747345e+16 ||r(i)||/||b|| 4.946634802067e-01 >>>> 1 KSP preconditioned resid norm 9.232243648078e+15 true resid norm >>>> 1.583125971956e+15 ||r(i)||/||b|| 6.507615763833e-02 >>>> 2 KSP preconditioned resid norm 7.541746545280e+15 true resid norm >>>> 1.216565269124e+15 ||r(i)||/||b|| 5.000827137780e-02 >>>> 3 KSP preconditioned resid norm 7.340199256082e+15 true resid norm >>>> 1.219811880454e+15 ||r(i)||/||b|| 5.014172695523e-02 >>>> 4 KSP preconditioned resid norm 7.655426776880e+15 true resid norm >>>> 1.212245393966e+15 ||r(i)||/||b|| 4.983069809450e-02 >>>> 5 KSP preconditioned resid norm 8.414958315915e+15 true resid norm >>>> 1.311448634719e+15 ||r(i)||/||b|| 5.390855787813e-02 >>>> 6 KSP preconditioned resid norm 1.008678778264e+16 true resid norm >>>> 1.495852838926e+15 ||r(i)||/||b|| 6.148869823002e-02 >>>> 7 KSP preconditioned resid norm 1.235685231672e+16 true resid norm >>>> 2.203374571492e+15 ||r(i)||/||b|| 9.057216765485e-02 >>>> 8 KSP preconditioned resid norm 1.311846317918e+16 true resid norm >>>> 2.061677565336e+15 ||r(i)||/||b|| 8.474755428051e-02 >>>> 9 KSP preconditioned resid norm 1.289737803434e+16 true resid norm >>>> 2.355893652982e+15 ||r(i)||/||b|| 9.684163449812e-02 >>>> 10 KSP preconditioned resid norm 1.151803850475e+16 true resid norm >>>> 1.821777204240e+15 ||r(i)||/||b|| 7.488618254339e-02 >>>> 11 KSP preconditioned resid norm 8.380110732685e+15 true resid norm >>>> 1.434954901150e+15 ||r(i)||/||b|| 5.898542062058e-02 >>>> 12 KSP preconditioned resid norm 3.553409074636e+15 true resid norm >>>> 5.423523740165e+14 ||r(i)||/||b|| 2.229399884295e-02 >>>> 13 KSP preconditioned resid norm 1.190458477552e+15 true resid norm >>>> 2.162492452839e+14 ||r(i)||/||b|| 8.889166260017e-03 >>>> 14 KSP preconditioned resid norm 9.902014249371e+14 true resid norm >>>> 1.847474743657e+14 ||r(i)||/||b|| 7.594250854373e-03 >>>> 15 KSP preconditioned resid norm 9.211725470007e+14 true resid norm >>>> 1.442747306714e+14 ||r(i)||/||b|| 5.930573613672e-03 >>>> 16 KSP preconditioned resid norm 8.405580663519e+14 true resid norm >>>> 2.201247255509e+14 ||r(i)||/||b|| 9.048472196027e-03 >>>> 17 KSP preconditioned resid norm 7.117085749555e+14 true resid norm >>>> 1.108024781635e+14 ||r(i)||/||b|| 4.554659365975e-03 >>>> 18 KSP preconditioned resid norm 6.347627750827e+14 true resid norm >>>> 1.457120949776e+14 ||r(i)||/||b|| 5.989658075575e-03 >>>> 19 KSP preconditioned resid norm 5.626851770343e+14 true resid norm >>>> 8.750418282909e+13 ||r(i)||/||b|| 3.596956967843e-03 >>>> 20 KSP preconditioned resid norm 4.865367652981e+14 true resid norm >>>> 1.207991751614e+14 ||r(i)||/||b|| 4.965584738448e-03 >>>> 21 KSP preconditioned resid norm 4.252070053539e+14 true resid norm >>>> 6.577223220744e+13 ||r(i)||/||b|| 2.703640914986e-03 >>>> 22 KSP preconditioned resid norm 3.618870286814e+14 true resid norm >>>> 8.993615915309e+13 ||r(i)||/||b|| 3.696926065336e-03 >>>> 23 KSP preconditioned resid norm 3.105522471772e+14 true resid norm >>>> 4.771648977942e+13 ||r(i)||/||b|| 1.961439497451e-03 >>>> 24 KSP preconditioned resid norm 2.735005931489e+14 true resid norm >>>> 4.708110884633e+13 ||r(i)||/||b|| 1.935321455997e-03 >>>> 25 KSP preconditioned resid norm 2.496548247596e+14 true resid norm >>>> 3.887274137414e+13 ||r(i)||/||b|| 1.597907361960e-03 >>>> 26 KSP preconditioned resid norm 2.329765393701e+14 true resid norm >>>> 3.903456593754e+13 ||r(i)||/||b|| 1.604559341009e-03 >>>> 27 KSP preconditioned resid norm 2.169265400605e+14 true resid norm >>>> 3.334703074182e+13 ||r(i)||/||b|| 1.370766867430e-03 >>>> 28 KSP preconditioned resid norm 1.998370208104e+14 true resid norm >>>> 3.474062054769e+13 ||r(i)||/||b|| 1.428051929704e-03 >>>> 29 KSP preconditioned resid norm 1.805920498739e+14 true resid norm >>>> 2.773007978134e+13 ||r(i)||/||b|| 1.139875837515e-03 >>>> 30 KSP preconditioned resid norm 1.643922523062e+14 true resid norm >>>> 2.817409671160e+13 ||r(i)||/||b|| 1.158127648337e-03 >>>> 31 KSP preconditioned resid norm 1.456868079593e+14 true resid norm >>>> 2.236751337771e+13 ||r(i)||/||b|| 9.194415683468e-04 >>>> 32 KSP preconditioned resid norm 1.315816507773e+14 true resid norm >>>> 2.157979387266e+13 ||r(i)||/||b|| 8.870614800950e-04 >>>> 0 KSP preconditioned resid norm 6.130010040591e+16 true resid norm >>>> 4.019535303993e+16 ||r(i)||/||b|| 5.748506002867e-02 >>>> 1 KSP preconditioned resid norm 2.383231267167e+16 true resid norm >>>> 4.333028232365e+15 ||r(i)||/||b|| 6.196845386481e-03 >>>> 2 KSP preconditioned resid norm 1.380314332389e+16 true resid norm >>>> 2.902938687246e+15 ||r(i)||/||b|| 4.151614355275e-03 >>>> 3 KSP preconditioned resid norm 9.355423030603e+15 true resid norm >>>> 1.914374573200e+15 ||r(i)||/||b|| 2.737827359010e-03 >>>> 4 KSP preconditioned resid norm 6.905389736952e+15 true resid norm >>>> 1.160495154173e+15 ||r(i)||/||b|| 1.659672786910e-03 >>>> 5 KSP preconditioned resid norm 5.388850338707e+15 true resid norm >>>> 9.343171842551e+14 ||r(i)||/||b|| 1.336206187053e-03 >>>> 6 KSP preconditioned resid norm 4.456941671648e+15 true resid norm >>>> 7.443205431054e+14 ||r(i)||/||b|| 1.064484022780e-03 >>>> 7 KSP preconditioned resid norm 3.604897929141e+15 true resid norm >>>> 7.177355376747e+14 ||r(i)||/||b|| 1.026463691635e-03 >>>> 8 KSP preconditioned resid norm 2.616552572944e+15 true resid norm >>>> 4.566784838372e+14 ||r(i)||/||b|| 6.531150511630e-04 >>>> 9 KSP preconditioned resid norm 1.730753896972e+15 true resid norm >>>> 3.416577764278e+14 ||r(i)||/||b|| 4.886191139484e-04 >>>> 10 KSP preconditioned resid norm 9.605912905151e+14 true resid norm >>>> 1.715828170206e+14 ||r(i)||/||b|| 2.453877821777e-04 >>>> 11 KSP preconditioned resid norm 3.885469679759e+14 true resid norm >>>> 7.526615556743e+13 ||r(i)||/||b|| 1.076412854646e-04 >>>> 12 KSP preconditioned resid norm 1.085402187481e+14 true resid norm >>>> 1.877491024324e+13 ||r(i)||/||b|| 2.685078649001e-05 >>>> 13 KSP preconditioned resid norm 5.521537327924e+13 true resid norm >>>> 1.482832280216e+13 ||r(i)||/||b|| 2.120660628506e-05 >>>> 14 KSP preconditioned resid norm 4.327780981098e+13 true resid norm >>>> 6.949078996691e+12 ||r(i)||/||b|| 9.938169292156e-06 >>>> 0 KSP preconditioned resid norm 2.469228993807e+16 true resid norm >>>> 1.581397706747e+16 ||r(i)||/||b|| 1.421322431696e-01 >>>> 1 KSP preconditioned resid norm 1.094866250702e+16 true resid norm >>>> 1.827538407052e+15 ||r(i)||/||b|| 1.642547805430e-02 >>>> 2 KSP preconditioned resid norm 7.902646639796e+15 true resid norm >>>> 1.331972302022e+15 ||r(i)||/||b|| 1.197144844200e-02 >>>> 3 KSP preconditioned resid norm 6.757113896005e+15 true resid norm >>>> 1.177404713092e+15 ||r(i)||/||b|| 1.058223192536e-02 >>>> 4 KSP preconditioned resid norm 6.236819639281e+15 true resid norm >>>> 9.858933018833e+14 ||r(i)||/||b|| 8.860973171059e-03 >>>> 5 KSP preconditioned resid norm 6.056207345085e+15 true resid norm >>>> 9.498733496025e+14 ||r(i)||/||b|| 8.537234455953e-03 >>>> 6 KSP preconditioned resid norm 6.281149615757e+15 true resid norm >>>> 9.439016000829e+14 ||r(i)||/||b|| 8.483561799717e-03 >>>> 7 KSP preconditioned resid norm 6.511636840454e+15 true resid norm >>>> 1.175604965975e+15 ||r(i)||/||b|| 1.056605622877e-02 >>>> 8 KSP preconditioned resid norm 5.897282381771e+15 true resid norm >>>> 9.358325816295e+14 ||r(i)||/||b|| 8.411039391972e-03 >>>> 9 KSP preconditioned resid norm 4.858643828149e+15 true resid norm >>>> 9.074074922507e+14 ||r(i)||/||b|| 8.155561487934e-03 >>>> 10 KSP preconditioned resid norm 3.431226959566e+15 true resid norm >>>> 5.523076022464e+14 ||r(i)||/||b|| 4.964008616682e-03 >>>> 11 KSP preconditioned resid norm 1.790601246330e+15 true resid norm >>>> 3.233761759516e+14 ||r(i)||/||b|| 2.906427717678e-03 >>>> 12 KSP preconditioned resid norm 6.061504971272e+14 true resid norm >>>> 9.361691978333e+13 ||r(i)||/||b|| 8.414064817893e-04 >>>> 13 KSP preconditioned resid norm 3.669164519745e+14 true resid norm >>>> 7.433339410578e+13 ||r(i)||/||b|| 6.680907656304e-04 >>>> 14 KSP preconditioned resid norm 1.028448839513e+15 true resid norm >>>> 1.739193394480e+14 ||r(i)||/||b|| 1.563145421348e-03 >>>> 15 KSP preconditioned resid norm 1.340577188948e+14 true resid norm >>>> 2.477633902186e+13 ||r(i)||/||b|| 2.226838086134e-04 >>>> 16 KSP preconditioned resid norm 1.124181660974e+14 true resid norm >>>> 1.917872742131e+13 ||r(i)||/||b|| 1.723738144997e-04 >>>> 17 KSP preconditioned resid norm 9.394070158671e+13 true resid norm >>>> 1.682505899954e+13 ||r(i)||/||b|| 1.512196057236e-04 >>>> 18 KSP preconditioned resid norm 9.034650415597e+13 true resid norm >>>> 1.892962426081e+13 ||r(i)||/||b|| 1.701349348787e-04 >>>> 19 KSP preconditioned resid norm 8.058877605551e+13 true resid norm >>>> 1.275341142002e+13 ||r(i)||/||b|| 1.146246112195e-04 >>>> 20 KSP preconditioned resid norm 6.303186171389e+13 true resid norm >>>> 1.900109222650e+13 ||r(i)||/||b|| 1.707772718592e-04 >>>> 21 KSP preconditioned resid norm 4.601610061053e+13 true resid norm >>>> 6.959914424356e+12 ||r(i)||/||b|| 6.255404603042e-05 >>>> 22 KSP preconditioned resid norm 2.822582332621e+13 true resid norm >>>> 1.013252988889e+13 ||r(i)||/||b|| 9.106875493415e-05 >>>> 23 KSP preconditioned resid norm 1.858033653578e+13 true resid norm >>>> 2.785463510953e+12 ||r(i)||/||b|| 2.503507975190e-05 >>>> 24 KSP preconditioned resid norm 9.505670647827e+12 true resid norm >>>> 4.268589348463e+12 ||r(i)||/||b|| 3.836506001486e-05 >>>> 25 KSP preconditioned resid norm 5.844140386668e+12 true resid norm >>>> 8.921656809030e+11 ||r(i)||/||b|| 8.018571733391e-06 >>>> 0 SNES Function norm 0.000000000000e+00 >>>> 3 TS dt 5e-12 time 9e-12 >>>> 0 KSP preconditioned resid norm 2.220122692212e+19 true resid norm >>>> 1.204173138214e+19 ||r(i)||/||b|| 2.302144004449e+02 >>>> 1 KSP preconditioned resid norm 5.389791318038e+18 true resid norm >>>> 1.775764523088e+18 ||r(i)||/||b|| 3.394915166605e+01 >>>> 2 KSP preconditioned resid norm 2.218229032455e+18 true resid norm >>>> 5.319501602511e+17 ||r(i)||/||b|| 1.016984878025e+01 >>>> 3 KSP preconditioned resid norm 1.401621992174e+18 true resid norm >>>> 2.963396938661e+17 ||r(i)||/||b|| 5.665436537857e+00 >>>> 4 KSP preconditioned resid norm 1.004007060031e+18 true resid norm >>>> 1.656144361093e+17 ||r(i)||/||b|| 3.166224764861e+00 >>>> 5 KSP preconditioned resid norm 8.672273682794e+17 true resid norm >>>> 1.510259905938e+17 ||r(i)||/||b|| 2.887322160975e+00 >>>> 6 KSP preconditioned resid norm 7.388939161376e+17 true resid norm >>>> 1.526077665500e+17 ||r(i)||/||b|| 2.917562629878e+00 >>>> 7 KSP preconditioned resid norm 5.565893055963e+17 true resid norm >>>> 1.017298978392e+17 ||r(i)||/||b|| 1.944877085792e+00 >>>> 8 KSP preconditioned resid norm 3.566491384538e+17 true resid norm >>>> 6.306517676931e+16 ||r(i)||/||b|| 1.205683086440e+00 >>>> 9 KSP preconditioned resid norm 1.893313432840e+17 true resid norm >>>> 3.267997700517e+16 ||r(i)||/||b|| 6.247773741206e-01 >>>> 10 KSP preconditioned resid norm 6.895898463878e+16 true resid norm >>>> 1.153123828824e+16 ||r(i)||/||b|| 2.204547688925e-01 >>>> 11 KSP preconditioned resid norm 5.104999855255e+16 true resid norm >>>> 9.275473058095e+15 ||r(i)||/||b|| 1.773289405940e-01 >>>> 12 KSP preconditioned resid norm 4.992747088606e+16 true resid norm >>>> 8.399049844320e+15 ||r(i)||/||b|| 1.605734393881e-01 >>>> 13 KSP preconditioned resid norm 4.920015136673e+16 true resid norm >>>> 8.426577623649e+15 ||r(i)||/||b|| 1.610997167989e-01 >>>> 14 KSP preconditioned resid norm 5.688370526613e+16 true resid norm >>>> 9.763030126011e+15 ||r(i)||/||b|| 1.866500800972e-01 >>>> 15 KSP preconditioned resid norm 1.613946025228e+17 true resid norm >>>> 4.341380499600e+16 ||r(i)||/||b|| 8.299872145473e-01 >>>> 16 KSP preconditioned resid norm 4.125620543313e+16 true resid norm >>>> 7.099475531667e+15 ||r(i)||/||b|| 1.357281150965e-01 >>>> 17 KSP preconditioned resid norm 2.163115342752e+16 true resid norm >>>> 5.850497008723e+15 ||r(i)||/||b|| 1.118500835491e-01 >>>> 18 KSP preconditioned resid norm 1.414953599369e+16 true resid norm >>>> 2.432122296311e+15 ||r(i)||/||b|| 4.649743118207e-02 >>>> 19 KSP preconditioned resid norm 1.088056550362e+16 true resid norm >>>> 2.488538030816e+15 ||r(i)||/||b|| 4.757598991109e-02 >>>> 20 KSP preconditioned resid norm 9.058853458567e+15 true resid norm >>>> 1.587978910975e+15 ||r(i)||/||b|| 3.035905729067e-02 >>>> 21 KSP preconditioned resid norm 7.813748136791e+15 true resid norm >>>> 1.345999130102e+15 ||r(i)||/||b|| 2.573287618717e-02 >>>> 22 KSP preconditioned resid norm 6.941551759120e+15 true resid norm >>>> 1.184314230023e+15 ||r(i)||/||b|| 2.264177648138e-02 >>>> 23 KSP preconditioned resid norm 6.244923673699e+15 true resid norm >>>> 1.023439246806e+15 ||r(i)||/||b|| 1.956616080515e-02 >>>> 24 KSP preconditioned resid norm 5.681880462174e+15 true resid norm >>>> 9.491910634069e+14 ||r(i)||/||b|| 1.814668045943e-02 >>>> 25 KSP preconditioned resid norm 5.283519017995e+15 true resid norm >>>> 8.557005279630e+14 ||r(i)||/||b|| 1.635932390068e-02 >>>> 26 KSP preconditioned resid norm 5.117675200632e+15 true resid norm >>>> 8.569254569155e+14 ||r(i)||/||b|| 1.638274215138e-02 >>>> 27 KSP preconditioned resid norm 4.944059018035e+15 true resid norm >>>> 9.547778379458e+14 ||r(i)||/||b|| 1.825348868410e-02 >>>> 28 KSP preconditioned resid norm 4.840428844487e+15 true resid norm >>>> 7.569152804275e+14 ||r(i)||/||b|| 1.447074278120e-02 >>>> 29 KSP preconditioned resid norm 4.909458338739e+15 true resid norm >>>> 7.780937746322e+14 ||r(i)||/||b|| 1.487563425328e-02 >>>> 30 KSP preconditioned resid norm 5.172548892535e+15 true resid norm >>>> 8.119476371329e+14 ||r(i)||/||b|| 1.552285402683e-02 >>>> 31 KSP preconditioned resid norm 5.734458283292e+15 true resid norm >>>> 9.082595641858e+14 ||r(i)||/||b|| 1.736415008622e-02 >>>> 32 KSP preconditioned resid norm 8.602942487649e+15 true resid norm >>>> 1.383459272467e+15 ||r(i)||/||b|| 2.644904099284e-02 >>>> 33 KSP preconditioned resid norm 1.681001901654e+16 true resid norm >>>> 2.705594423024e+15 ||r(i)||/||b|| 5.172568446988e-02 >>>> 34 KSP preconditioned resid norm 3.498037913905e+16 true resid norm >>>> 5.724738344713e+15 ||r(i)||/||b|| 1.094458233545e-01 >>>> 35 KSP preconditioned resid norm 6.660621055336e+15 true resid norm >>>> 1.092449655644e+15 ||r(i)||/||b|| 2.088550512457e-02 >>>> 36 KSP preconditioned resid norm 3.564205941873e+15 true resid norm >>>> 5.832825274169e+14 ||r(i)||/||b|| 1.115122344769e-02 >>>> 37 KSP preconditioned resid norm 2.275128811690e+15 true resid norm >>>> 4.150874600928e+14 ||r(i)||/||b|| 7.935662050992e-03 >>>> 38 KSP preconditioned resid norm 1.923021972098e+15 true resid norm >>>> 3.105449109783e+14 ||r(i)||/||b|| 5.937012562674e-03 >>>> 39 KSP preconditioned resid norm 1.756400018558e+15 true resid norm >>>> 3.028037938881e+14 ||r(i)||/||b|| 5.789017513361e-03 >>>> 40 KSP preconditioned resid norm 1.618977002834e+15 true resid norm >>>> 2.591473194248e+14 ||r(i)||/||b|| 4.954390932253e-03 >>>> 41 KSP preconditioned resid norm 1.449378745707e+15 true resid norm >>>> 2.366294470672e+14 ||r(i)||/||b|| 4.523893164149e-03 >>>> 42 KSP preconditioned resid norm 1.311727507569e+15 true resid norm >>>> 2.162961405943e+14 ||r(i)||/||b|| 4.135160031831e-03 >>>> 43 KSP preconditioned resid norm 1.220101213287e+15 true resid norm >>>> 1.960093358179e+14 ||r(i)||/||b|| 3.747315921186e-03 >>>> 44 KSP preconditioned resid norm 1.141924730669e+15 true resid norm >>>> 1.884563341884e+14 ||r(i)||/||b|| 3.602917272310e-03 >>>> 45 KSP preconditioned resid norm 1.062154013672e+15 true resid norm >>>> 2.268601642641e+14 ||r(i)||/||b|| 4.337123545072e-03 >>>> 46 KSP preconditioned resid norm 1.050847341809e+15 true resid norm >>>> 1.644035317108e+14 ||r(i)||/||b|| 3.143074636258e-03 >>>> 47 KSP preconditioned resid norm 8.268920862881e+14 true resid norm >>>> 2.491484388741e+14 ||r(i)||/||b|| 4.763231852378e-03 >>>> 48 KSP preconditioned resid norm 6.970056044801e+14 true resid norm >>>> 1.086553600705e+14 ||r(i)||/||b|| 2.077278406232e-03 >>>> 49 KSP preconditioned resid norm 8.789823309588e+14 true resid norm >>>> 1.565701303869e+14 ||r(i)||/||b|| 2.993315292522e-03 >>>> 50 KSP preconditioned resid norm 9.889574125697e+14 true resid norm >>>> 1.557528361421e+14 ||r(i)||/||b|| 2.977690221792e-03 >>>> 51 KSP preconditioned resid norm 3.685341349551e+14 true resid norm >>>> 5.749337875960e+13 ||r(i)||/||b|| 1.099161183775e-03 >>>> 52 KSP preconditioned resid norm 2.728364862859e+15 true resid norm >>>> 4.745949443528e+14 ||r(i)||/||b|| 9.073329000715e-03 >>>> 53 KSP preconditioned resid norm 8.484691724076e+15 true resid norm >>>> 1.333040899485e+15 ||r(i)||/||b|| 2.548514011022e-02 >>>> 54 KSP preconditioned resid norm 8.341541248747e+13 true resid norm >>>> 1.303089350822e+13 ||r(i)||/||b|| 2.491252496055e-04 >>>> 0 KSP preconditioned resid norm 6.542890621126e+18 true resid norm >>>> 3.545938224251e+18 ||r(i)||/||b|| 4.156801777589e+02 >>>> 1 KSP preconditioned resid norm 1.591173149842e+18 true resid norm >>>> 5.223204285160e+17 ||r(i)||/||b|| 6.123012721646e+01 >>>> 2 KSP preconditioned resid norm 6.570590492388e+17 true resid norm >>>> 1.552517142001e+17 ||r(i)||/||b|| 1.819971360886e+01 >>>> 3 KSP preconditioned resid norm 4.165858181866e+17 true resid norm >>>> 8.738575659546e+16 ||r(i)||/||b|| 1.024398185698e+01 >>>> 4 KSP preconditioned resid norm 2.995041215234e+17 true resid norm >>>> 4.934730887237e+16 ||r(i)||/||b|| 5.784843622966e+00 >>>> 5 KSP preconditioned resid norm 2.594520327186e+17 true resid norm >>>> 4.469839192099e+16 ||r(i)||/||b|| 5.239864409419e+00 >>>> 6 KSP preconditioned resid norm 2.219632225075e+17 true resid norm >>>> 4.652171577038e+16 ||r(i)||/||b|| 5.453607439866e+00 >>>> 7 KSP preconditioned resid norm 1.674928029034e+17 true resid norm >>>> 2.988841295071e+16 ||r(i)||/||b|| 3.503733010156e+00 >>>> 8 KSP preconditioned resid norm 1.081911414020e+17 true resid norm >>>> 1.908165723430e+16 ||r(i)||/||b|| 2.236888002402e+00 >>>> 9 KSP preconditioned resid norm 5.710171826154e+16 true resid norm >>>> 9.635278906551e+15 ||r(i)||/||b|| 1.129516137997e+00 >>>> 10 KSP preconditioned resid norm 1.779377501866e+16 true resid norm >>>> 2.952494296810e+15 ||r(i)||/||b|| 3.461124465555e-01 >>>> 11 KSP preconditioned resid norm 1.125612460139e+16 true resid norm >>>> 2.030403352863e+15 ||r(i)||/||b|| 2.380183672880e-01 >>>> 12 KSP preconditioned resid norm 1.086634567103e+16 true resid norm >>>> 1.877885131752e+15 ||r(i)||/||b|| 2.201390932416e-01 >>>> 13 KSP preconditioned resid norm 1.075395224737e+16 true resid norm >>>> 1.822688133743e+15 ||r(i)||/||b|| 2.136685073225e-01 >>>> 14 KSP preconditioned resid norm 1.173925328196e+16 true resid norm >>>> 2.130469004546e+15 ||r(i)||/||b|| 2.497487769141e-01 >>>> 15 KSP preconditioned resid norm 4.944016549258e+16 true resid norm >>>> 1.361939461929e+16 ||r(i)||/||b|| 1.596562607209e+00 >>>> 16 KSP preconditioned resid norm 1.780247590037e+17 true resid norm >>>> 3.370503336823e+16 ||r(i)||/||b|| 3.951144485840e+00 >>>> 17 KSP preconditioned resid norm 3.251104571145e+16 true resid norm >>>> 8.719659549123e+15 ||r(i)||/||b|| 1.022180704274e+00 >>>> 18 KSP preconditioned resid norm 2.075437272594e+16 true resid norm >>>> 3.944434594682e+15 ||r(i)||/||b|| 4.623947654424e-01 >>>> 19 KSP preconditioned resid norm 1.483444135710e+16 true resid norm >>>> 3.390829411015e+15 ||r(i)||/||b|| 3.974972160206e-01 >>>> 20 KSP preconditioned resid norm 1.134313319987e+16 true resid norm >>>> 2.489467119154e+15 ||r(i)||/||b|| 2.918330972428e-01 >>>> 21 KSP preconditioned resid norm 8.553477032620e+15 true resid norm >>>> 1.657674168535e+15 ||r(i)||/||b|| 1.943243930000e-01 >>>> 22 KSP preconditioned resid norm 6.586595023874e+15 true resid norm >>>> 1.286251502369e+15 ||r(i)||/||b|| 1.507835780925e-01 >>>> 23 KSP preconditioned resid norm 4.780400299659e+15 true resid norm >>>> 9.064518348736e+14 ||r(i)||/||b|| 1.062607513219e-01 >>>> 24 KSP preconditioned resid norm 3.028073960021e+15 true resid norm >>>> 6.179142836679e+14 ||r(i)||/||b|| 7.243632094834e-02 >>>> 25 KSP preconditioned resid norm 1.570529076731e+15 true resid norm >>>> 2.755631911412e+14 ||r(i)||/||b|| 3.230348331903e-02 >>>> 26 KSP preconditioned resid norm 1.180384548694e+15 true resid norm >>>> 2.278461878994e+14 ||r(i)||/||b|| 2.670975575377e-02 >>>> 27 KSP preconditioned resid norm 1.075603583388e+15 true resid norm >>>> 2.036512758236e+14 ||r(i)||/||b|| 2.387345553745e-02 >>>> 28 KSP preconditioned resid norm 1.013485206919e+15 true resid norm >>>> 1.671016982388e+14 ||r(i)||/||b|| 1.958885328364e-02 >>>> 29 KSP preconditioned resid norm 9.469966879519e+14 true resid norm >>>> 1.801244790801e+14 ||r(i)||/||b|| 2.111547656715e-02 >>>> 30 KSP preconditioned resid norm 8.991813841494e+14 true resid norm >>>> 1.459100485336e+14 ||r(i)||/||b|| 1.710461690970e-02 >>>> 31 KSP preconditioned resid norm 8.677534755734e+14 true resid norm >>>> 1.581142867341e+14 ||r(i)||/||b|| 1.853528478482e-02 >>>> 32 KSP preconditioned resid norm 8.336211910215e+14 true resid norm >>>> 1.336646371267e+14 ||r(i)||/||b|| 1.566912241756e-02 >>>> 33 KSP preconditioned resid norm 8.025169227655e+14 true resid norm >>>> 1.586001394574e+14 ||r(i)||/||b|| 1.859223990744e-02 >>>> 34 KSP preconditioned resid norm 7.724033526395e+14 true resid norm >>>> 1.340849232530e+14 ||r(i)||/||b|| 1.571839135589e-02 >>>> 35 KSP preconditioned resid norm 7.664262315912e+14 true resid norm >>>> 1.204381005271e+14 ||r(i)||/||b|| 1.411861343033e-02 >>>> 36 KSP preconditioned resid norm 7.727266103384e+14 true resid norm >>>> 1.292430217432e+14 ||r(i)||/||b|| 1.515078911553e-02 >>>> 37 KSP preconditioned resid norm 8.047214262262e+14 true resid norm >>>> 1.267167533285e+14 ||r(i)||/||b|| 1.485464190786e-02 >>>> 38 KSP preconditioned resid norm 8.419506152984e+14 true resid norm >>>> 1.380565821207e+14 ||r(i)||/||b|| 1.618397754485e-02 >>>> 39 KSP preconditioned resid norm 8.796981754885e+14 true resid norm >>>> 1.388509364066e+14 ||r(i)||/||b|| 1.627709742171e-02 >>>> 40 KSP preconditioned resid norm 9.432799067464e+14 true resid norm >>>> 1.525727137977e+14 ||r(i)||/||b|| 1.788566206789e-02 >>>> 41 KSP preconditioned resid norm 1.193594717717e+15 true resid norm >>>> 1.887759239390e+14 ||r(i)||/||b|| 2.212966065874e-02 >>>> 42 KSP preconditioned resid norm 2.117323854280e+15 true resid norm >>>> 3.659661973675e+14 ||r(i)||/||b|| 4.290116870483e-02 >>>> 43 KSP preconditioned resid norm 1.749515464589e+16 true resid norm >>>> 2.774904343458e+15 ||r(i)||/||b|| 3.252940851772e-01 >>>> 44 KSP preconditioned resid norm 1.219803820486e+15 true resid norm >>>> 2.038867414461e+14 ||r(i)||/||b|| 2.390105849768e-02 >>>> 45 KSP preconditioned resid norm 6.218668840796e+14 true resid norm >>>> 9.918031040094e+13 ||r(i)||/||b|| 1.162662360435e-02 >>>> 46 KSP preconditioned resid norm 4.213016096566e+14 true resid norm >>>> 7.032265378826e+13 ||r(i)||/||b|| 8.243723206248e-03 >>>> 47 KSP preconditioned resid norm 2.932264460599e+14 true resid norm >>>> 4.681093096736e+13 ||r(i)||/||b|| 5.487511308712e-03 >>>> 48 KSP preconditioned resid norm 1.890454719223e+14 true resid norm >>>> 3.330632990756e+13 ||r(i)||/||b|| 3.904405621560e-03 >>>> 49 KSP preconditioned resid norm 1.361587907879e+14 true resid norm >>>> 2.180429132446e+13 ||r(i)||/||b|| 2.556054595557e-03 >>>> 50 KSP preconditioned resid norm 1.041049830937e+14 true resid norm >>>> 1.811134568183e+13 ||r(i)||/||b|| 2.123141159366e-03 >>>> 51 KSP preconditioned resid norm 9.375298557199e+13 true resid norm >>>> 1.575768352161e+13 ||r(i)||/||b|| 1.847228088333e-03 >>>> 52 KSP preconditioned resid norm 1.220175150775e+14 true resid norm >>>> 1.950548748269e+13 ||r(i)||/||b|| 2.286572408009e-03 >>>> 53 KSP preconditioned resid norm 7.051895161295e+13 true resid norm >>>> 1.295324819768e+13 ||r(i)||/||b|| 1.518472170932e-03 >>>> 54 KSP preconditioned resid norm 5.292501274648e+13 true resid norm >>>> 8.611444708372e+12 ||r(i)||/||b|| 1.009494988563e-03 >>>> 0 KSP preconditioned resid norm 1.892610202067e+19 true resid norm >>>> 1.028308040523e+19 ||r(i)||/||b|| 7.841022917690e+01 >>>> 1 KSP preconditioned resid norm 4.581169591238e+18 true resid norm >>>> 1.519782710159e+18 ||r(i)||/||b|| 1.158860048804e+01 >>>> 2 KSP preconditioned resid norm 1.874457569158e+18 true resid norm >>>> 4.646597685037e+17 ||r(i)||/||b|| 3.543109409036e+00 >>>> 3 KSP preconditioned resid norm 1.173257076173e+18 true resid norm >>>> 2.550995626245e+17 ||r(i)||/||b|| 1.945177357373e+00 >>>> 4 KSP preconditioned resid norm 8.301427099842e+17 true resid norm >>>> 1.381385637907e+17 ||r(i)||/||b|| 1.053329937932e+00 >>>> 5 KSP preconditioned resid norm 7.053087339556e+17 true resid norm >>>> 1.287851939525e+17 ||r(i)||/||b|| 9.820089092432e-01 >>>> 6 KSP preconditioned resid norm 5.863912202870e+17 true resid norm >>>> 1.166954217660e+17 ||r(i)||/||b|| 8.898223493329e-01 >>>> 7 KSP preconditioned resid norm 4.330693111087e+17 true resid norm >>>> 8.611440299479e+16 ||r(i)||/||b|| 6.566369033559e-01 >>>> 8 KSP preconditioned resid norm 2.702458300388e+17 true resid norm >>>> 4.803465439349e+16 ||r(i)||/||b|| 3.662723727716e-01 >>>> 9 KSP preconditioned resid norm 1.546904751775e+17 true resid norm >>>> 2.878502712633e+16 ||r(i)||/||b|| 2.194907055953e-01 >>>> 10 KSP preconditioned resid norm 9.910545254967e+16 true resid norm >>>> 1.648892832929e+16 ||r(i)||/||b|| 1.257308703453e-01 >>>> 11 KSP preconditioned resid norm 9.242232753221e+16 true resid norm >>>> 1.559261342084e+16 ||r(i)||/||b|| 1.188963174082e-01 >>>> 12 KSP preconditioned resid norm 9.244969774564e+16 true resid norm >>>> 1.506990436290e+16 ||r(i)||/||b|| 1.149105723385e-01 >>>> 13 KSP preconditioned resid norm 7.091380549056e+16 true resid norm >>>> 1.150476534699e+16 ||r(i)||/||b|| 8.772578370826e-02 >>>> 14 KSP preconditioned resid norm 2.015813147523e+17 true resid norm >>>> 3.265267485752e+16 ||r(i)||/||b|| 2.489821743992e-01 >>>> 15 KSP preconditioned resid norm 1.668693825394e+16 true resid norm >>>> 4.392881515438e+15 ||r(i)||/||b|| 3.349646533904e-02 >>>> 16 KSP preconditioned resid norm 1.171106201192e+16 true resid norm >>>> 1.864120995445e+15 ||r(i)||/||b|| 1.421423821522e-02 >>>> 17 KSP preconditioned resid norm 1.020170882678e+16 true resid norm >>>> 1.999245817269e+15 ||r(i)||/||b|| 1.524458786039e-02 >>>> 18 KSP preconditioned resid norm 9.407404703690e+15 true resid norm >>>> 1.539877936168e+15 ||r(i)||/||b|| 1.174182998880e-02 >>>> 19 KSP preconditioned resid norm 8.812299187434e+15 true resid norm >>>> 1.424937011002e+15 ||r(i)||/||b|| 1.086538597310e-02 >>>> 20 KSP preconditioned resid norm 8.313368595823e+15 true resid norm >>>> 1.488773269648e+15 ||r(i)||/||b|| 1.135214825376e-02 >>>> 21 KSP preconditioned resid norm 7.851626197782e+15 true resid norm >>>> 1.250085440746e+15 ||r(i)||/||b|| 9.532113144783e-03 >>>> 22 KSP preconditioned resid norm 7.455044400782e+15 true resid norm >>>> 1.324254107888e+15 ||r(i)||/||b|| 1.009766178966e-02 >>>> 23 KSP preconditioned resid norm 7.129687083603e+15 true resid norm >>>> 1.124533100660e+15 ||r(i)||/||b|| 8.574755293646e-03 >>>> 24 KSP preconditioned resid norm 5.784281419699e+15 true resid norm >>>> 2.057123935555e+15 ||r(i)||/||b|| 1.568591831199e-02 >>>> 25 KSP preconditioned resid norm 4.924869719264e+15 true resid norm >>>> 7.825081154476e+14 ||r(i)||/||b|| 5.966756871199e-03 >>>> 26 KSP preconditioned resid norm 5.016674985414e+15 true resid norm >>>> 7.817733101871e+14 ||r(i)||/||b|| 5.961153856674e-03 >>>> 27 KSP preconditioned resid norm 5.308634596254e+15 true resid norm >>>> 8.859615986698e+14 ||r(i)||/||b|| 6.755607197068e-03 >>>> 28 KSP preconditioned resid norm 9.149615612203e+15 true resid norm >>>> 1.429126026168e+15 ||r(i)||/||b|| 1.089732792301e-02 >>>> 29 KSP preconditioned resid norm 8.897313413659e+15 true resid norm >>>> 2.002400213933e+15 ||r(i)||/||b|| 1.526864066904e-02 >>>> 30 KSP preconditioned resid norm 2.145204463192e+15 true resid norm >>>> 3.349465461999e+14 ||r(i)||/||b|| 2.554024126484e-03 >>>> 31 KSP preconditioned resid norm 1.279996253934e+15 true resid norm >>>> 2.794251271078e+14 ||r(i)||/||b|| 2.130663905259e-03 >>>> 32 KSP preconditioned resid norm 8.936440443654e+14 true resid norm >>>> 1.396095375555e+14 ||r(i)||/||b|| 1.064546361948e-03 >>>> 0 KSP preconditioned resid norm 6.470811776559e+18 true resid norm >>>> 3.511555308471e+18 ||r(i)||/||b|| 1.662220839803e+02 >>>> 1 KSP preconditioned resid norm 1.571601370240e+18 true resid norm >>>> 5.187483735196e+17 ||r(i)||/||b|| 2.455534033589e+01 >>>> 2 KSP preconditioned resid norm 6.468356854753e+17 true resid norm >>>> 1.566993981909e+17 ||r(i)||/||b|| 7.417482635946e+00 >>>> 3 KSP preconditioned resid norm 4.084058718904e+17 true resid norm >>>> 8.688424998267e+16 ||r(i)||/||b|| 4.112730636007e+00 >>>> 4 KSP preconditioned resid norm 2.920904676538e+17 true resid norm >>>> 4.831974981420e+16 ||r(i)||/||b|| 2.287251319137e+00 >>>> 5 KSP preconditioned resid norm 2.515876340477e+17 true resid norm >>>> 4.435582123557e+16 ||r(i)||/||b|| 2.099615809737e+00 >>>> 6 KSP preconditioned resid norm 2.133089016156e+17 true resid norm >>>> 4.355659888860e+16 ||r(i)||/||b|| 2.061784025127e+00 >>>> 7 KSP preconditioned resid norm 1.601295948028e+17 true resid norm >>>> 2.999342671961e+16 ||r(i)||/||b|| 1.419761176199e+00 >>>> 8 KSP preconditioned resid norm 1.019894482623e+17 true resid norm >>>> 1.811123232414e+16 ||r(i)||/||b|| 8.573086612382e-01 >>>> 9 KSP preconditioned resid norm 5.552358385594e+16 true resid norm >>>> 9.849609129144e+15 ||r(i)||/||b|| 4.662385786400e-01 >>>> 10 KSP preconditioned resid norm 2.614302078211e+16 true resid norm >>>> 4.400897857400e+15 ||r(i)||/||b|| 2.083197754216e-01 >>>> 11 KSP preconditioned resid norm 2.230412831734e+16 true resid norm >>>> 4.001989581448e+15 ||r(i)||/||b|| 1.894371552944e-01 >>>> 12 KSP preconditioned resid norm 2.202852939300e+16 true resid norm >>>> 3.668964514289e+15 ||r(i)||/||b|| 1.736731658885e-01 >>>> 13 KSP preconditioned resid norm 2.171413942501e+16 true resid norm >>>> 3.746648405904e+15 ||r(i)||/||b|| 1.773503907139e-01 >>>> 14 KSP preconditioned resid norm 2.756569557917e+16 true resid norm >>>> 4.738274109991e+15 ||r(i)||/||b|| 2.242897314282e-01 >>>> 15 KSP preconditioned resid norm 1.911125733606e+16 true resid norm >>>> 7.273426207009e+15 ||r(i)||/||b|| 3.442930427122e-01 >>>> 16 KSP preconditioned resid norm 8.267301781208e+15 true resid norm >>>> 1.450502353903e+15 ||r(i)||/||b|| 6.866060844958e-02 >>>> 17 KSP preconditioned resid norm 5.452175853074e+15 true resid norm >>>> 1.779140334910e+15 ||r(i)||/||b|| 8.421693186737e-02 >>>> 18 KSP preconditioned resid norm 4.185422899260e+15 true resid norm >>>> 7.352001585133e+14 ||r(i)||/||b|| 3.480124667150e-02 >>>> 19 KSP preconditioned resid norm 3.483100692842e+15 true resid norm >>>> 7.680069742951e+14 ||r(i)||/||b|| 3.635418171280e-02 >>>> 20 KSP preconditioned resid norm 3.028709630445e+15 true resid norm >>>> 5.664408475792e+14 ||r(i)||/||b|| 2.681289909034e-02 >>>> 21 KSP preconditioned resid norm 2.650986791756e+15 true resid norm >>>> 5.333477625647e+14 ||r(i)||/||b|| 2.524641328184e-02 >>>> 22 KSP preconditioned resid norm 2.409785359425e+15 true resid norm >>>> 3.913611824173e+14 ||r(i)||/||b|| 1.852537283791e-02 >>>> 23 KSP preconditioned resid norm 2.246473514462e+15 true resid norm >>>> 3.515886291592e+14 ||r(i)||/||b|| 1.664270942896e-02 >>>> 24 KSP preconditioned resid norm 2.105216345845e+15 true resid norm >>>> 3.429537728013e+14 ||r(i)||/||b|| 1.623397207682e-02 >>>> 25 KSP preconditioned resid norm 1.993254024352e+15 true resid norm >>>> 3.120073169264e+14 ||r(i)||/||b|| 1.476909855627e-02 >>>> 26 KSP preconditioned resid norm 1.937764630771e+15 true resid norm >>>> 3.193060027359e+14 ||r(i)||/||b|| 1.511458728106e-02 >>>> 27 KSP preconditioned resid norm 1.841409046328e+15 true resid norm >>>> 4.276359096161e+14 ||r(i)||/||b|| 2.024246404712e-02 >>>> 28 KSP preconditioned resid norm 1.798600034298e+15 true resid norm >>>> 2.807262398909e+14 ||r(i)||/||b|| 1.328838549404e-02 >>>> 29 KSP preconditioned resid norm 1.909156742948e+15 true resid norm >>>> 3.075705616539e+14 ||r(i)||/||b|| 1.455908144342e-02 >>>> 30 KSP preconditioned resid norm 2.243056909373e+15 true resid norm >>>> 3.518787704137e+14 ||r(i)||/||b|| 1.665644348118e-02 >>>> 31 KSP preconditioned resid norm 3.125791809917e+15 true resid norm >>>> 5.051797337710e+14 ||r(i)||/||b|| 2.391305867501e-02 >>>> 32 KSP preconditioned resid norm 2.268406906040e+16 true resid norm >>>> 3.663605847107e+15 ||r(i)||/||b|| 1.734195093893e-01 >>>> 33 KSP preconditioned resid norm 2.663631019961e+15 true resid norm >>>> 4.418031676456e+14 ||r(i)||/||b|| 2.091308174984e-02 >>>> 34 KSP preconditioned resid norm 1.194136462714e+15 true resid norm >>>> 1.964734637439e+14 ||r(i)||/||b|| 9.300217630505e-03 >>>> 35 KSP preconditioned resid norm 7.378290473645e+14 true resid norm >>>> 1.245828993721e+14 ||r(i)||/||b|| 5.897224261846e-03 >>>> 36 KSP preconditioned resid norm 5.578220476287e+14 true resid norm >>>> 9.049566410806e+13 ||r(i)||/||b|| 4.283679611405e-03 >>>> 37 KSP preconditioned resid norm 4.459209275003e+14 true resid norm >>>> 9.101333159944e+13 ||r(i)||/||b|| 4.308183787380e-03 >>>> 38 KSP preconditioned resid norm 4.020379023591e+14 true resid norm >>>> 6.381912771158e+13 ||r(i)||/||b|| 3.020925907227e-03 >>>> 39 KSP preconditioned resid norm 3.758207934261e+14 true resid norm >>>> 7.185355109637e+13 ||r(i)||/||b|| 3.401241317717e-03 >>>> 40 KSP preconditioned resid norm 3.524000257362e+14 true resid norm >>>> 5.552480372659e+13 ||r(i)||/||b|| 2.628307908397e-03 >>>> 41 KSP preconditioned resid norm 3.288683013967e+14 true resid norm >>>> 5.672396056313e+13 ||r(i)||/||b|| 2.685070889720e-03 >>>> 42 KSP preconditioned resid norm 3.091159619430e+14 true resid norm >>>> 5.054650403677e+13 ||r(i)||/||b|| 2.392656387510e-03 >>>> 43 KSP preconditioned resid norm 2.914764729338e+14 true resid norm >>>> 5.652869757159e+13 ||r(i)||/||b|| 2.675827970692e-03 >>>> 44 KSP preconditioned resid norm 2.823730992718e+14 true resid norm >>>> 4.387571428023e+13 ||r(i)||/||b|| 2.076889589689e-03 >>>> 45 KSP preconditioned resid norm 2.874969520743e+14 true resid norm >>>> 4.617969867525e+13 ||r(i)||/||b|| 2.185950405754e-03 >>>> 46 KSP preconditioned resid norm 3.038236713170e+14 true resid norm >>>> 4.761130615995e+13 ||r(i)||/||b|| 2.253716611508e-03 >>>> 47 KSP preconditioned resid norm 3.972150696557e+14 true resid norm >>>> 6.226543710285e+13 ||r(i)||/||b|| 2.947380805938e-03 >>>> 48 KSP preconditioned resid norm 7.035007479461e+14 true resid norm >>>> 1.474495546291e+14 ||r(i)||/||b|| 6.979634406808e-03 >>>> 49 KSP preconditioned resid norm 2.896572855920e+13 true resid norm >>>> 4.634266191547e+12 ||r(i)||/||b|| 2.193664392014e-04 >>>> 0 KSP preconditioned resid norm 6.694681597421e+18 true resid norm >>>> 3.650754072308e+18 ||r(i)||/||b|| 5.229798618159e+00 >>>> 1 KSP preconditioned resid norm 1.597667334547e+18 true resid norm >>>> 5.366749713724e+17 ||r(i)||/||b|| 7.688006280602e-01 >>>> 2 KSP preconditioned resid norm 6.300446759218e+17 true resid norm >>>> 1.705328267289e+17 ||r(i)||/||b|| 2.442926376067e-01 >>>> 3 KSP preconditioned resid norm 3.642174486076e+17 true resid norm >>>> 9.481576843120e+16 ||r(i)||/||b|| 1.358260142699e-01 >>>> 4 KSP preconditioned resid norm 2.247332959633e+17 true resid norm >>>> 3.802800729913e+16 ||r(i)||/||b|| 5.447609345502e-02 >>>> 5 KSP preconditioned resid norm 1.557523701694e+17 true resid norm >>>> 3.829141999854e+16 ||r(i)||/||b|| 5.485343888670e-02 >>>> 6 KSP preconditioned resid norm 1.004692846458e+17 true resid norm >>>> 1.699728099753e+16 ||r(i)||/||b|| 2.434903992784e-02 >>>> 7 KSP preconditioned resid norm 7.103482800944e+16 true resid norm >>>> 1.472858884839e+16 ||r(i)||/||b|| 2.109908037658e-02 >>>> 8 KSP preconditioned resid norm 5.766847884006e+16 true resid norm >>>> 9.397954060610e+15 ||r(i)||/||b|| 1.346280965144e-02 >>>> 9 KSP preconditioned resid norm 4.523472015836e+16 true resid norm >>>> 1.246374917128e+16 ||r(i)||/||b|| 1.785463959006e-02 >>>> 10 KSP preconditioned resid norm 3.195342481549e+16 true resid norm >>>> 5.030521483345e+15 ||r(i)||/||b|| 7.206350737723e-03 >>>> 11 KSP preconditioned resid norm 2.437460220100e+16 true resid norm >>>> 4.974645944970e+15 ||r(i)||/||b|| 7.126307599347e-03 >>>> 12 KSP preconditioned resid norm 1.258045555912e+16 true resid norm >>>> 1.885910304740e+15 ||r(i)||/||b|| 2.701614765157e-03 >>>> 13 KSP preconditioned resid norm 2.495567167662e+15 true resid norm >>>> 4.460991814847e+14 ||r(i)||/||b|| 6.390484915397e-04 >>>> 14 KSP preconditioned resid norm 7.678096336284e+14 true resid norm >>>> 1.316276217008e+14 ||r(i)||/||b|| 1.885599359606e-04 >>>> 15 KSP preconditioned resid norm 5.950867488217e+14 true resid norm >>>> 1.439410415068e+14 ||r(i)||/||b|| 2.061992248885e-04 >>>> 16 KSP preconditioned resid norm 3.263035982098e+14 true resid norm >>>> 5.046543709851e+13 ||r(i)||/||b|| 7.229302987142e-05 >>>> 17 KSP preconditioned resid norm 1.436782483869e+14 true resid norm >>>> 7.588970331259e+13 ||r(i)||/||b|| 1.087139417380e-04 >>>> 18 KSP preconditioned resid norm 5.401533723431e+13 true resid norm >>>> 8.625318760812e+12 ||r(i)||/||b|| 1.235598981554e-05 >>>> 19 KSP preconditioned resid norm 3.528859982338e+13 true resid norm >>>> 1.233587619696e+13 ||r(i)||/||b|| 1.767145828255e-05 >>>> 0 KSP preconditioned resid norm 4.827874033276e+18 true resid norm >>>> 2.629049503582e+18 ||r(i)||/||b|| 2.387244449971e+01 >>>> 1 KSP preconditioned resid norm 1.161059891938e+18 true resid norm >>>> 3.885117259710e+17 ||r(i)||/||b|| 3.527786222014e+00 >>>> 2 KSP preconditioned resid norm 4.696931551640e+17 true resid norm >>>> 1.228954734511e+17 ||r(i)||/||b|| 1.115922452289e+00 >>>> 3 KSP preconditioned resid norm 2.870101730651e+17 true resid norm >>>> 6.759381584873e+16 ||r(i)||/||b|| 6.137692025860e-01 >>>> 4 KSP preconditioned resid norm 1.966639305477e+17 true resid norm >>>> 3.310590025777e+16 ||r(i)||/||b|| 3.006100742644e-01 >>>> 5 KSP preconditioned resid norm 1.592574496560e+17 true resid norm >>>> 3.239415772489e+16 ||r(i)||/||b|| 2.941472693263e-01 >>>> 6 KSP preconditioned resid norm 1.230114823548e+17 true resid norm >>>> 2.280737281236e+16 ||r(i)||/||b|| 2.070968009182e-01 >>>> 7 KSP preconditioned resid norm 8.598640814452e+16 true resid norm >>>> 1.938029193987e+16 ||r(i)||/||b|| 1.759780266946e-01 >>>> 8 KSP preconditioned resid norm 5.549085767183e+16 true resid norm >>>> 9.386094645880e+15 ||r(i)||/||b|| 8.522814925985e-02 >>>> 9 KSP preconditioned resid norm 4.474754629313e+16 true resid norm >>>> 7.579167805290e+15 ||r(i)||/||b|| 6.882078961970e-02 >>>> 10 KSP preconditioned resid norm 4.378389532275e+16 true resid norm >>>> 6.842593279419e+15 ||r(i)||/||b|| 6.213250380964e-02 >>>> 11 KSP preconditioned resid norm 4.079498017991e+16 true resid norm >>>> 7.704710537522e+15 ||r(i)||/||b|| 6.996074986140e-02 >>>> 12 KSP preconditioned resid norm 4.084398127844e+16 true resid norm >>>> 6.377957981626e+15 ||r(i)||/||b|| 5.791349601078e-02 >>>> 13 KSP preconditioned resid norm 7.523960004025e+16 true resid norm >>>> 1.132262464871e+16 ||r(i)||/||b|| 1.028123388886e-01 >>>> 14 KSP preconditioned resid norm 2.184546709977e+15 true resid norm >>>> 3.502101186681e+14 ||r(i)||/||b|| 3.179997793785e-03 >>>> 15 KSP preconditioned resid norm 2.222188254453e+15 true resid norm >>>> 3.654675413503e+14 ||r(i)||/||b|| 3.318539109075e-03 >>>> 16 KSP preconditioned resid norm 1.669487552266e+15 true resid norm >>>> 3.629534589137e+14 ||r(i)||/||b|| 3.295710595061e-03 >>>> 17 KSP preconditioned resid norm 1.401917499993e+15 true resid norm >>>> 2.161165604483e+14 ||r(i)||/||b|| 1.962393856693e-03 >>>> 18 KSP preconditioned resid norm 1.134393254100e+15 true resid norm >>>> 2.686682335159e+14 ||r(i)||/||b|| 2.439576540763e-03 >>>> 19 KSP preconditioned resid norm 9.058009036885e+14 true resid norm >>>> 1.409928994873e+14 ||r(i)||/||b|| 1.280251727203e-03 >>>> 20 KSP preconditioned resid norm 7.990098022926e+14 true resid norm >>>> 1.659150954933e+14 ||r(i)||/||b|| 1.506551665700e-03 >>>> 21 KSP preconditioned resid norm 7.119179981764e+14 true resid norm >>>> 1.092130722300e+14 ||r(i)||/||b|| 9.916827362515e-04 >>>> 22 KSP preconditioned resid norm 6.306959704581e+14 true resid norm >>>> 1.460783700778e+14 ||r(i)||/||b|| 1.326429105857e-03 >>>> 23 KSP preconditioned resid norm 5.637228678118e+14 true resid norm >>>> 8.672640313846e+13 ||r(i)||/||b|| 7.874980074592e-04 >>>> 24 KSP preconditioned resid norm 4.960976390096e+14 true resid norm >>>> 1.292271182132e+14 ||r(i)||/||b|| 1.173415412376e-03 >>>> 25 KSP preconditioned resid norm 4.386404742743e+14 true resid norm >>>> 8.037964656859e+13 ||r(i)||/||b|| 7.298678282782e-04 >>>> 26 KSP preconditioned resid norm 4.419824802921e+14 true resid norm >>>> 6.704823034514e+13 ||r(i)||/||b|| 6.088151461346e-04 >>>> 27 KSP preconditioned resid norm 4.437076279615e+14 true resid norm >>>> 1.370394110315e+14 ||r(i)||/||b|| 1.244353036968e-03 >>>> 28 KSP preconditioned resid norm 7.848825686766e+14 true resid norm >>>> 1.196984074237e+14 ||r(i)||/||b|| 1.086892271915e-03 >>>> 29 KSP preconditioned resid norm 1.771839364121e+15 true resid norm >>>> 6.554952272410e+14 ||r(i)||/||b|| 5.952064961431e-03 >>>> 30 KSP preconditioned resid norm 1.052090569223e+15 true resid norm >>>> 1.877636113350e+14 ||r(i)||/||b|| 1.704941799139e-03 >>>> 31 KSP preconditioned resid norm 2.459236824172e+14 true resid norm >>>> 3.913333475140e+13 ||r(i)||/||b|| 3.553407270074e-04 >>>> 32 KSP preconditioned resid norm 1.091606883884e+14 true resid norm >>>> 1.900051073024e+13 ||r(i)||/||b|| 1.725295158025e-04 >>>> 33 KSP preconditioned resid norm 6.424114279947e+13 true resid norm >>>> 1.001416869814e+13 ||r(i)||/||b|| 9.093122291205e-05 >>>> 34 KSP preconditioned resid norm 3.801391191576e+13 true resid norm >>>> 7.035993054761e+12 ||r(i)||/||b|| 6.388862342504e-05 >>>> 35 KSP preconditioned resid norm 2.524482106623e+13 true resid norm >>>> 4.006294246762e+12 ||r(i)||/||b|| 3.637818037470e-05 >>>> 36 KSP preconditioned resid norm 1.579134058219e+13 true resid norm >>>> 2.528643754315e+12 ||r(i)||/||b|| 2.296073451724e-05 >>>> 37 KSP preconditioned resid norm 1.074218958170e+13 true resid norm >>>> 1.786180378482e+12 ||r(i)||/||b|| 1.621897643756e-05 >>>> 38 KSP preconditioned resid norm 6.572401461924e+12 true resid norm >>>> 1.033900044668e+12 ||r(i)||/||b|| 9.388077858923e-06 >>>> 0 KSP preconditioned resid norm 9.558506771869e+15 true resid norm >>>> 5.892974320929e+15 ||r(i)||/||b|| 1.119279921775e-01 >>>> 1 KSP preconditioned resid norm 4.577878026258e+15 true resid norm >>>> 8.113551231445e+14 ||r(i)||/||b|| 1.541044384903e-02 >>>> 2 KSP preconditioned resid norm 3.768474917795e+15 true resid norm >>>> 6.114238708763e+14 ||r(i)||/||b|| 1.161305692331e-02 >>>> 3 KSP preconditioned resid norm 3.689006206393e+15 true resid norm >>>> 6.168086911615e+14 ||r(i)||/||b|| 1.171533331040e-02 >>>> 4 KSP preconditioned resid norm 3.859762637542e+15 true resid norm >>>> 6.287723791867e+14 ||r(i)||/||b|| 1.194256518123e-02 >>>> 5 KSP preconditioned resid norm 4.285185348381e+15 true resid norm >>>> 6.764146832146e+14 ||r(i)||/||b|| 1.284745753985e-02 >>>> 6 KSP preconditioned resid norm 5.208557345650e+15 true resid norm >>>> 7.765898979336e+14 ||r(i)||/||b|| 1.475013181583e-02 >>>> 7 KSP preconditioned resid norm 6.471756089495e+15 true resid norm >>>> 1.171552566673e+15 ||r(i)||/||b|| 2.225184081532e-02 >>>> 8 KSP preconditioned resid norm 7.012039525053e+15 true resid norm >>>> 1.102268075574e+15 ||r(i)||/||b|| 2.093588836833e-02 >>>> 9 KSP preconditioned resid norm 7.014970322733e+15 true resid norm >>>> 1.328280835862e+15 ||r(i)||/||b|| 2.522865346245e-02 >>>> 10 KSP preconditioned resid norm 6.390016130531e+15 true resid norm >>>> 1.015385597333e+15 ||r(i)||/||b|| 1.928568919633e-02 >>>> 11 KSP preconditioned resid norm 4.845316666400e+15 true resid norm >>>> 8.456481888760e+14 ||r(i)||/||b|| 1.606178793844e-02 >>>> 12 KSP preconditioned resid norm 2.410560854094e+15 true resid norm >>>> 3.709640804249e+14 ||r(i)||/||b|| 7.045892690295e-03 >>>> 13 KSP preconditioned resid norm 1.334157374345e+15 true resid norm >>>> 2.713196530598e+14 ||r(i)||/||b|| 5.153299904502e-03 >>>> 14 KSP preconditioned resid norm 1.217530036393e+15 true resid norm >>>> 2.232136761716e+14 ||r(i)||/||b|| 4.239600792372e-03 >>>> 15 KSP preconditioned resid norm 1.145671346103e+15 true resid norm >>>> 2.123551517116e+14 ||r(i)||/||b|| 4.033359805286e-03 >>>> 16 KSP preconditioned resid norm 1.048828510236e+15 true resid norm >>>> 1.668978069044e+14 ||r(i)||/||b|| 3.169967389692e-03 >>>> 17 KSP preconditioned resid norm 9.144204694073e+14 true resid norm >>>> 1.578839852075e+14 ||r(i)||/||b|| 2.998763697051e-03 >>>> 18 KSP preconditioned resid norm 8.235850213860e+14 true resid norm >>>> 1.321988157805e+14 ||r(i)||/||b|| 2.510913371198e-03 >>>> 19 KSP preconditioned resid norm 6.978607225935e+14 true resid norm >>>> 1.125789224246e+14 ||r(i)||/||b|| 2.138263644512e-03 >>>> 20 KSP preconditioned resid norm 6.009340490988e+14 true resid norm >>>> 1.003011158758e+14 ||r(i)||/||b|| 1.905065574997e-03 >>>> 21 KSP preconditioned resid norm 5.123703002158e+14 true resid norm >>>> 8.114653676231e+13 ||r(i)||/||b|| 1.541253777351e-03 >>>> 22 KSP preconditioned resid norm 4.394549158605e+14 true resid norm >>>> 8.003106905102e+13 ||r(i)||/||b|| 1.520067182184e-03 >>>> 23 KSP preconditioned resid norm 3.852402878098e+14 true resid norm >>>> 5.993269339949e+13 ||r(i)||/||b|| 1.138329419521e-03 >>>> 24 KSP preconditioned resid norm 3.390788847804e+14 true resid norm >>>> 7.376626581336e+13 ||r(i)||/||b|| 1.401076871080e-03 >>>> 0 KSP preconditioned resid norm 9.592017367891e+15 true resid norm >>>> 5.933298102242e+15 ||r(i)||/||b|| 5.704123038824e-01 >>>> 1 KSP preconditioned resid norm 4.627662883687e+15 true resid norm >>>> 8.121175703537e+14 ||r(i)||/||b|| 7.807493342594e-02 >>>> 2 KSP preconditioned resid norm 3.843678575472e+15 true resid norm >>>> 6.216594738645e+14 ||r(i)||/||b|| 5.976477274644e-02 >>>> 3 KSP preconditioned resid norm 3.802359495548e+15 true resid norm >>>> 6.332926539714e+14 ||r(i)||/||b|| 6.088315731972e-02 >>>> 4 KSP preconditioned resid norm 4.025235313693e+15 true resid norm >>>> 6.488898363947e+14 ||r(i)||/||b|| 6.238263107055e-02 >>>> 5 KSP preconditioned resid norm 4.519136017232e+15 true resid norm >>>> 7.100489487455e+14 ||r(i)||/||b|| 6.826231376612e-02 >>>> 6 KSP preconditioned resid norm 5.566459161126e+15 true resid norm >>>> 8.272746471009e+14 ||r(i)||/||b|| 7.953209652789e-02 >>>> 7 KSP preconditioned resid norm 7.051037635374e+15 true resid norm >>>> 1.259719337020e+15 ||r(i)||/||b|| 1.211062375246e-01 >>>> 8 KSP preconditioned resid norm 7.825859719364e+15 true resid norm >>>> 1.230378249193e+15 ||r(i)||/||b|| 1.182854593980e-01 >>>> 9 KSP preconditioned resid norm 8.112136766042e+15 true resid norm >>>> 1.509007778424e+15 ||r(i)||/||b|| 1.450721990763e-01 >>>> 10 KSP preconditioned resid norm 7.849971536540e+15 true resid norm >>>> 1.244897342905e+15 ||r(i)||/||b|| 1.196812884212e-01 >>>> 11 KSP preconditioned resid norm 6.691547560102e+15 true resid norm >>>> 1.152431148742e+15 ||r(i)||/||b|| 1.107918218994e-01 >>>> 12 KSP preconditioned resid norm 3.838210718255e+15 true resid norm >>>> 5.898239658450e+14 ||r(i)||/||b|| 5.670418735838e-02 >>>> 13 KSP preconditioned resid norm 1.585805121666e+15 true resid norm >>>> 3.093588848689e+14 ||r(i)||/||b|| 2.974098236828e-02 >>>> 14 KSP preconditioned resid norm 1.284246821472e+15 true resid norm >>>> 2.667115224243e+14 ||r(i)||/||b|| 2.564097258496e-02 >>>> 15 KSP preconditioned resid norm 1.153005952878e+15 true resid norm >>>> 2.281947914341e+14 ||r(i)||/||b|| 2.193807128394e-02 >>>> 16 KSP preconditioned resid norm 1.029063568485e+15 true resid norm >>>> 1.605944355705e+14 ||r(i)||/||b|| 1.543914369478e-02 >>>> 17 KSP preconditioned resid norm 8.755986525312e+14 true resid norm >>>> 1.474488326268e+14 ||r(i)||/||b|| 1.417535860732e-02 >>>> 18 KSP preconditioned resid norm 7.769999155370e+14 true resid norm >>>> 1.263847845681e+14 ||r(i)||/||b|| 1.215031419270e-02 >>>> 19 KSP preconditioned resid norm 6.633003954457e+14 true resid norm >>>> 1.037940616116e+14 ||r(i)||/||b|| 9.978499106727e-03 >>>> 20 KSP preconditioned resid norm 5.634039280371e+14 true resid norm >>>> 1.015458370156e+14 ||r(i)||/||b|| 9.762360468599e-03 >>>> 21 KSP preconditioned resid norm 4.739327677772e+14 true resid norm >>>> 7.392362794765e+13 ||r(i)||/||b|| 7.106831007366e-03 >>>> 22 KSP preconditioned resid norm 3.847430187788e+14 true resid norm >>>> 7.626771651358e+13 ||r(i)||/||b|| 7.332185765606e-03 >>>> 23 KSP preconditioned resid norm 3.166693300565e+14 true resid norm >>>> 4.910897850051e+13 ||r(i)||/||b|| 4.721213241789e-03 >>>> 24 KSP preconditioned resid norm 2.603961739838e+14 true resid norm >>>> 5.903512955155e+13 ||r(i)||/||b|| 5.675488350193e-03 >>>> 25 KSP preconditioned resid norm 2.300385519514e+14 true resid norm >>>> 3.524302891352e+13 ||r(i)||/||b|| 3.388175846206e-03 >>>> 26 KSP preconditioned resid norm 1.995854205790e+14 true resid norm >>>> 5.125398452570e+13 ||r(i)||/||b|| 4.927428707047e-03 >>>> 27 KSP preconditioned resid norm 1.777736693158e+14 true resid norm >>>> 2.713391594494e+13 ||r(i)||/||b|| 2.608586192838e-03 >>>> 28 KSP preconditioned resid norm 1.569155636768e+14 true resid norm >>>> 4.045925214031e+13 ||r(i)||/||b|| 3.889650381462e-03 >>>> 29 KSP preconditioned resid norm 1.392286125087e+14 true resid norm >>>> 2.127117420025e+13 ||r(i)||/||b|| 2.044956999086e-03 >>>> 30 KSP preconditioned resid norm 1.268134787155e+14 true resid norm >>>> 2.836210701297e+13 ||r(i)||/||b|| 2.726661382160e-03 >>>> 31 KSP preconditioned resid norm 1.148332814766e+14 true resid norm >>>> 1.759485016641e+13 ||r(i)||/||b|| 1.691524485529e-03 >>>> 32 KSP preconditioned resid norm 1.072826684457e+14 true resid norm >>>> 1.915166944365e+13 ||r(i)||/||b|| 1.841193161425e-03 >>>> 33 KSP preconditioned resid norm 9.907876839152e+13 true resid norm >>>> 1.514435361562e+13 ||r(i)||/||b|| 1.455939932199e-03 >>>> 34 KSP preconditioned resid norm 9.246074413874e+13 true resid norm >>>> 1.530899883812e+13 ||r(i)||/||b|| 1.471768508325e-03 >>>> 35 KSP preconditioned resid norm 8.716833484735e+13 true resid norm >>>> 1.331822154534e+13 ||r(i)||/||b|| 1.280380204127e-03 >>>> 36 KSP preconditioned resid norm 8.124134503515e+13 true resid norm >>>> 1.344545385639e+13 ||r(i)||/||b|| 1.292611997376e-03 >>>> 37 KSP preconditioned resid norm 7.675303961825e+13 true resid norm >>>> 1.172301521125e+13 ||r(i)||/||b|| 1.127021093474e-03 >>>> 38 KSP preconditioned resid norm 7.135475879451e+13 true resid norm >>>> 1.180708573448e+13 ||r(i)||/||b|| 1.135103421383e-03 >>>> 39 KSP preconditioned resid norm 6.615399842578e+13 true resid norm >>>> 1.009923425569e+13 ||r(i)||/||b|| 9.709148908363e-04 >>>> 40 KSP preconditioned resid norm 6.137620595303e+13 true resid norm >>>> 9.912104999293e+12 ||r(i)||/||b|| 9.529247564410e-04 >>>> 41 KSP preconditioned resid norm 5.598390411354e+13 true resid norm >>>> 8.555770285650e+12 ||r(i)||/||b|| 8.225301604653e-04 >>>> 0 KSP preconditioned resid norm 2.200654561303e+16 true resid norm >>>> 1.374535903551e+16 ||r(i)||/||b|| 1.042244800141e-01 >>>> 1 KSP preconditioned resid norm 1.033788570768e+16 true resid norm >>>> 1.766933748516e+15 ||r(i)||/||b|| 1.339781308605e-02 >>>> 2 KSP preconditioned resid norm 8.199712159807e+15 true resid norm >>>> 1.328504928257e+15 ||r(i)||/||b|| 1.007341714291e-02 >>>> 3 KSP preconditioned resid norm 7.724963557325e+15 true resid norm >>>> 1.290091137776e+15 ||r(i)||/||b|| 9.782143751798e-03 >>>> 4 KSP preconditioned resid norm 7.785648142256e+15 true resid norm >>>> 1.238722001104e+15 ||r(i)||/||b|| 9.392636170038e-03 >>>> 5 KSP preconditioned resid norm 8.249045346662e+15 true resid norm >>>> 1.288778250144e+15 ||r(i)||/||b|| 9.772188753144e-03 >>>> 6 KSP preconditioned resid norm 9.469275305646e+15 true resid norm >>>> 1.408770770519e+15 ||r(i)||/||b|| 1.068203461525e-02 >>>> 7 KSP preconditioned resid norm 1.105611185924e+16 true resid norm >>>> 1.994387714295e+15 ||r(i)||/||b|| 1.512248766524e-02 >>>> 8 KSP preconditioned resid norm 1.130059557946e+16 true resid norm >>>> 1.768360656538e+15 ||r(i)||/||b|| 1.340863264676e-02 >>>> 9 KSP preconditioned resid norm 1.052660615325e+16 true resid norm >>>> 1.953565171874e+15 ||r(i)||/||b|| 1.481294986084e-02 >>>> 10 KSP preconditioned resid norm 8.559164814525e+15 true resid norm >>>> 1.322544665666e+15 ||r(i)||/||b|| 1.002822332384e-02 >>>> 11 KSP preconditioned resid norm 4.990037328467e+15 true resid norm >>>> 7.740616289621e+14 ||r(i)||/||b|| 5.869338921523e-03 >>>> 12 KSP preconditioned resid norm 1.334591152980e+17 true resid norm >>>> 2.007497042766e+16 ||r(i)||/||b|| 1.522188942984e-01 >>>> 13 KSP preconditioned resid norm 7.977709053830e+14 true resid norm >>>> 2.666709181112e+14 ||r(i)||/||b|| 2.022037962283e-03 >>>> 0 KSP preconditioned resid norm 1.928493598618e+16 true resid norm >>>> 1.203377031534e+16 ||r(i)||/||b|| 4.946657154689e-01 >>>> 1 KSP preconditioned resid norm 9.231841001478e+15 true resid norm >>>> 1.582820058439e+15 ||r(i)||/||b|| 6.506413170177e-02 >>>> 2 KSP preconditioned resid norm 7.541228065487e+15 true resid norm >>>> 1.216411600127e+15 ||r(i)||/||b|| 5.000237653817e-02 >>>> 3 KSP preconditioned resid norm 7.339567418801e+15 true resid norm >>>> 1.219667619599e+15 ||r(i)||/||b|| 5.013621997708e-02 >>>> 4 KSP preconditioned resid norm 7.654746286546e+15 true resid norm >>>> 1.212045913869e+15 ||r(i)||/||b|| 4.982291862433e-02 >>>> 5 KSP preconditioned resid norm 8.414088698130e+15 true resid norm >>>> 1.311254037213e+15 ||r(i)||/||b|| 5.390101352132e-02 >>>> 6 KSP preconditioned resid norm 1.008553581054e+16 true resid norm >>>> 1.495637339777e+15 ||r(i)||/||b|| 6.148035863870e-02 >>>> 7 KSP preconditioned resid norm 1.235521461592e+16 true resid norm >>>> 2.202862093934e+15 ||r(i)||/||b|| 9.055186572627e-02 >>>> 8 KSP preconditioned resid norm 1.311621757723e+16 true resid norm >>>> 2.061404522413e+15 ||r(i)||/||b|| 8.473704551686e-02 >>>> 9 KSP preconditioned resid norm 1.289521641859e+16 true resid norm >>>> 2.354903652303e+15 ||r(i)||/||b|| 9.680175618294e-02 >>>> 10 KSP preconditioned resid norm 1.151646040420e+16 true resid norm >>>> 1.821568826477e+15 ||r(i)||/||b|| 7.487824872947e-02 >>>> 11 KSP preconditioned resid norm 8.378553284195e+15 true resid norm >>>> 1.434374156491e+15 ||r(i)||/||b|| 5.896204595714e-02 >>>> 12 KSP preconditioned resid norm 3.545538676243e+15 true resid norm >>>> 5.411687842502e+14 ||r(i)||/||b|| 2.224553376337e-02 >>>> 13 KSP preconditioned resid norm 1.159722122499e+15 true resid norm >>>> 2.107714204908e+14 ||r(i)||/||b|| 8.664067269472e-03 >>>> 14 KSP preconditioned resid norm 9.539010029354e+14 true resid norm >>>> 1.788314612223e+14 ||r(i)||/||b|| 7.351128565343e-03 >>>> 15 KSP preconditioned resid norm 8.844744179759e+14 true resid norm >>>> 1.382762666459e+14 ||r(i)||/||b|| 5.684048023218e-03 >>>> 16 KSP preconditioned resid norm 7.998051624629e+14 true resid norm >>>> 2.151800146054e+14 ||r(i)||/||b|| 8.845288973459e-03 >>>> 17 KSP preconditioned resid norm 6.751752084065e+14 true resid norm >>>> 1.049375541763e+14 ||r(i)||/||b|| 4.313611524562e-03 >>>> 18 KSP preconditioned resid norm 6.016391198259e+14 true resid norm >>>> 1.405509222696e+14 ||r(i)||/||b|| 5.777551066910e-03 >>>> 19 KSP preconditioned resid norm 5.356608668579e+14 true resid norm >>>> 8.319469793388e+13 ||r(i)||/||b|| 3.419839642795e-03 >>>> 20 KSP preconditioned resid norm 4.666955314009e+14 true resid norm >>>> 1.159814975417e+14 ||r(i)||/||b|| 4.767588956680e-03 >>>> 21 KSP preconditioned resid norm 4.123539296814e+14 true resid norm >>>> 6.367161120725e+13 ||r(i)||/||b|| 2.617314631039e-03 >>>> 22 KSP preconditioned resid norm 3.572486233332e+14 true resid norm >>>> 8.643344455842e+13 ||r(i)||/||b|| 3.552973056037e-03 >>>> 23 KSP preconditioned resid norm 3.110551508962e+14 true resid norm >>>> 4.773889823616e+13 ||r(i)||/||b|| 1.962377179627e-03 >>>> 24 KSP preconditioned resid norm 2.787205893436e+14 true resid norm >>>> 4.531540777818e+13 ||r(i)||/||b|| 1.862756062561e-03 >>>> 25 KSP preconditioned resid norm 2.574052602586e+14 true resid norm >>>> 4.020202768069e+13 ||r(i)||/||b|| 1.652563100745e-03 >>>> 26 KSP preconditioned resid norm 2.417321052344e+14 true resid norm >>>> 3.935243057179e+13 ||r(i)||/||b|| 1.617639169947e-03 >>>> 27 KSP preconditioned resid norm 2.257728086711e+14 true resid norm >>>> 3.470313603079e+13 ||r(i)||/||b|| 1.426523123165e-03 >>>> 28 KSP preconditioned resid norm 2.082687407590e+14 true resid norm >>>> 3.533906053355e+13 ||r(i)||/||b|| 1.452663729217e-03 >>>> 29 KSP preconditioned resid norm 1.877437540892e+14 true resid norm >>>> 2.879571658728e+13 ||r(i)||/||b|| 1.183690013589e-03 >>>> 30 KSP preconditioned resid norm 1.698389677822e+14 true resid norm >>>> 2.905672512335e+13 ||r(i)||/||b|| 1.194419150913e-03 >>>> 31 KSP preconditioned resid norm 1.483411237433e+14 true resid norm >>>> 2.274318996948e+13 ||r(i)||/||b|| 9.348920615478e-04 >>>> 32 KSP preconditioned resid norm 1.319650244589e+14 true resid norm >>>> 2.174098043497e+13 ||r(i)||/||b|| 8.936947739606e-04 >>>> 0 KSP preconditioned resid norm 6.130010037542e+16 true resid norm >>>> 4.019535297204e+16 ||r(i)||/||b|| 5.748505993040e-02 >>>> 1 KSP preconditioned resid norm 2.383231272294e+16 true resid norm >>>> 4.333028202793e+15 ||r(i)||/||b|| 6.196845344062e-03 >>>> 2 KSP preconditioned resid norm 1.380314308424e+16 true resid norm >>>> 2.902938638811e+15 ||r(i)||/||b|| 4.151614285921e-03 >>>> 3 KSP preconditioned resid norm 9.355422663861e+15 true resid norm >>>> 1.914374541722e+15 ||r(i)||/||b|| 2.737827313936e-03 >>>> 4 KSP preconditioned resid norm 6.905389449720e+15 true resid norm >>>> 1.160495113499e+15 ||r(i)||/||b|| 1.659672728707e-03 >>>> 5 KSP preconditioned resid norm 5.388850196885e+15 true resid norm >>>> 9.343171628690e+14 ||r(i)||/||b|| 1.336206156440e-03 >>>> 6 KSP preconditioned resid norm 4.456941607393e+15 true resid norm >>>> 7.443205302890e+14 ||r(i)||/||b|| 1.064484004428e-03 >>>> 7 KSP preconditioned resid norm 3.604897945956e+15 true resid norm >>>> 7.177354673261e+14 ||r(i)||/||b|| 1.026463591006e-03 >>>> 8 KSP preconditioned resid norm 2.616552593649e+15 true resid norm >>>> 4.566784798390e+14 ||r(i)||/||b|| 6.531150454316e-04 >>>> 9 KSP preconditioned resid norm 1.730753896393e+15 true resid norm >>>> 3.416577992882e+14 ||r(i)||/||b|| 4.886191466320e-04 >>>> 10 KSP preconditioned resid norm 9.605913238033e+14 true resid norm >>>> 1.715828091641e+14 ||r(i)||/||b|| 2.453877709368e-04 >>>> 11 KSP preconditioned resid norm 3.885469473149e+14 true resid norm >>>> 7.526615944264e+13 ||r(i)||/||b|| 1.076412910045e-04 >>>> 12 KSP preconditioned resid norm 1.085402135701e+14 true resid norm >>>> 1.877490961949e+13 ||r(i)||/||b|| 2.685078559740e-05 >>>> 13 KSP preconditioned resid norm 5.521535097548e+13 true resid norm >>>> 1.482831670997e+13 ||r(i)||/||b|| 2.120659757192e-05 >>>> 14 KSP preconditioned resid norm 4.327778991064e+13 true resid norm >>>> 6.949075957635e+12 ||r(i)||/||b|| 9.938164945670e-06 >>>> 0 KSP preconditioned resid norm 2.469229041488e+16 true resid norm >>>> 1.581397714283e+16 ||r(i)||/||b|| 1.421322439001e-01 >>>> 1 KSP preconditioned resid norm 1.094866313372e+16 true resid norm >>>> 1.827538507852e+15 ||r(i)||/||b|| 1.642547896642e-02 >>>> 2 KSP preconditioned resid norm 7.902647446780e+15 true resid norm >>>> 1.331972422469e+15 ||r(i)||/||b|| 1.197144952903e-02 >>>> 3 KSP preconditioned resid norm 6.757114900348e+15 true resid norm >>>> 1.177404752019e+15 ||r(i)||/||b|| 1.058223227919e-02 >>>> 4 KSP preconditioned resid norm 6.236820681429e+15 true resid norm >>>> 9.858934558838e+14 ||r(i)||/||b|| 8.860974558495e-03 >>>> 5 KSP preconditioned resid norm 6.056208231119e+15 true resid norm >>>> 9.498735329089e+14 ||r(i)||/||b|| 8.537236106662e-03 >>>> 6 KSP preconditioned resid norm 6.281150442631e+15 true resid norm >>>> 9.439017368810e+14 ||r(i)||/||b|| 8.483563032401e-03 >>>> 7 KSP preconditioned resid norm 6.511637444968e+15 true resid norm >>>> 1.175605283055e+15 ||r(i)||/||b|| 1.056605908257e-02 >>>> 8 KSP preconditioned resid norm 5.897282453520e+15 true resid norm >>>> 9.358326231502e+14 ||r(i)||/||b|| 8.411039768298e-03 >>>> 9 KSP preconditioned resid norm 4.858643646570e+15 true resid norm >>>> 9.074077133036e+14 ||r(i)||/||b|| 8.155563477757e-03 >>>> 10 KSP preconditioned resid norm 3.431227043473e+15 true resid norm >>>> 5.523077341885e+14 ||r(i)||/||b|| 4.964009804403e-03 >>>> 11 KSP preconditioned resid norm 1.790602152143e+15 true resid norm >>>> 3.233768236148e+14 ||r(i)||/||b|| 2.906433539808e-03 >>>> 12 KSP preconditioned resid norm 6.061523931075e+14 true resid norm >>>> 9.361733206500e+13 ||r(i)||/||b|| 8.414101875932e-04 >>>> 13 KSP preconditioned resid norm 3.669173259935e+14 true resid norm >>>> 7.433445474494e+13 ||r(i)||/||b|| 6.681002986515e-04 >>>> 14 KSP preconditioned resid norm 1.034316883992e+15 true resid norm >>>> 1.749783469552e+14 ||r(i)||/||b|| 1.572663528097e-03 >>>> 15 KSP preconditioned resid norm 1.340579303848e+14 true resid norm >>>> 2.477734260421e+13 ||r(i)||/||b|| 2.226928286548e-04 >>>> 16 KSP preconditioned resid norm 1.124182191319e+14 true resid norm >>>> 1.917839477947e+13 ||r(i)||/||b|| 1.723708248589e-04 >>>> 17 KSP preconditioned resid norm 9.394060484065e+13 true resid norm >>>> 1.682558945581e+13 ||r(i)||/||b|| 1.512243733944e-04 >>>> 18 KSP preconditioned resid norm 9.034672558986e+13 true resid norm >>>> 1.892870450718e+13 ||r(i)||/||b|| 1.701266684168e-04 >>>> 19 KSP preconditioned resid norm 8.058964131802e+13 true resid norm >>>> 1.275370577304e+13 ||r(i)||/||b|| 1.146272568369e-04 >>>> 20 KSP preconditioned resid norm 6.303390378547e+13 true resid norm >>>> 1.900069972515e+13 ||r(i)||/||b|| 1.707737442148e-04 >>>> 21 KSP preconditioned resid norm 4.601826552473e+13 true resid norm >>>> 6.960266416937e+12 ||r(i)||/||b|| 6.255720967895e-05 >>>> 22 KSP preconditioned resid norm 2.822788042823e+13 true resid norm >>>> 1.013300658091e+13 ||r(i)||/||b|| 9.107303936204e-05 >>>> 23 KSP preconditioned resid norm 1.858182222289e+13 true resid norm >>>> 2.785683196819e+12 ||r(i)||/||b|| 2.503705424529e-05 >>>> 24 KSP preconditioned resid norm 9.506926663598e+12 true resid norm >>>> 4.268961211449e+12 ||r(i)||/||b|| 3.836840224479e-05 >>>> 25 KSP preconditioned resid norm 5.845171932356e+12 true resid norm >>>> 8.923162702109e+11 ||r(i)||/||b|| 8.019925197071e-06 >>>> 0 SNES Function norm 0.000000000000e+00 >>>> 4 TS dt 6e-12 time 1.4e-11 >>>> 0 KSP preconditioned resid norm 2.220105437401e+19 true resid norm >>>> 1.204162959566e+19 ||r(i)||/||b|| 2.302124615573e+02 >>>> 1 KSP preconditioned resid norm 5.389751984623e+18 true resid norm >>>> 1.775745946542e+18 ||r(i)||/||b|| 3.394879756150e+01 >>>> 2 KSP preconditioned resid norm 2.218217192238e+18 true resid norm >>>> 5.319400119211e+17 ||r(i)||/||b|| 1.016965507636e+01 >>>> 3 KSP preconditioned resid norm 1.401615472906e+18 true resid norm >>>> 2.963370402601e+17 ||r(i)||/||b|| 5.665385980104e+00 >>>> 4 KSP preconditioned resid norm 1.004001983833e+18 true resid norm >>>> 1.656139793050e+17 ||r(i)||/||b|| 3.166216128906e+00 >>>> 5 KSP preconditioned resid norm 8.672200498511e+17 true resid norm >>>> 1.510248645324e+17 ||r(i)||/||b|| 2.887300721564e+00 >>>> 6 KSP preconditioned resid norm 7.388847075384e+17 true resid norm >>>> 1.526081348396e+17 ||r(i)||/||b|| 2.917569760470e+00 >>>> 7 KSP preconditioned resid norm 5.565797182786e+17 true resid norm >>>> 1.017271957075e+17 ||r(i)||/||b|| 1.944825486043e+00 >>>> 8 KSP preconditioned resid norm 3.566434137336e+17 true resid norm >>>> 6.306431985913e+16 ||r(i)||/||b|| 1.205666741023e+00 >>>> 9 KSP preconditioned resid norm 1.893287605930e+17 true resid norm >>>> 3.267942353405e+16 ||r(i)||/||b|| 6.247668120231e-01 >>>> 10 KSP preconditioned resid norm 6.895870882324e+16 true resid norm >>>> 1.153127225742e+16 ||r(i)||/||b|| 2.204554250883e-01 >>>> 11 KSP preconditioned resid norm 5.104972946485e+16 true resid norm >>>> 9.275497869859e+15 ||r(i)||/||b|| 1.773294203933e-01 >>>> 12 KSP preconditioned resid norm 4.992720890366e+16 true resid norm >>>> 8.399052910493e+15 ||r(i)||/||b|| 1.605735029394e-01 >>>> 13 KSP preconditioned resid norm 4.919979261885e+16 true resid norm >>>> 8.426637216788e+15 ||r(i)||/||b|| 1.611008610517e-01 >>>> 14 KSP preconditioned resid norm 5.688292138515e+16 true resid norm >>>> 9.762950550877e+15 ||r(i)||/||b|| 1.866485645090e-01 >>>> 15 KSP preconditioned resid norm 1.614157649132e+17 true resid norm >>>> 4.341833601566e+16 ||r(i)||/||b|| 8.300738642954e-01 >>>> 16 KSP preconditioned resid norm 4.125579354715e+16 true resid norm >>>> 7.099585962609e+15 ||r(i)||/||b|| 1.357302304896e-01 >>>> 17 KSP preconditioned resid norm 2.163051916179e+16 true resid norm >>>> 5.849969022355e+15 ||r(i)||/||b|| 1.118399929155e-01 >>>> 18 KSP preconditioned resid norm 1.415003279507e+16 true resid norm >>>> 2.432246275858e+15 ||r(i)||/||b|| 4.649980285729e-02 >>>> 19 KSP preconditioned resid norm 1.088163683743e+16 true resid norm >>>> 2.488576155885e+15 ||r(i)||/||b|| 4.757672024934e-02 >>>> 20 KSP preconditioned resid norm 9.060374162401e+15 true resid norm >>>> 1.588297599957e+15 ||r(i)||/||b|| 3.036515093466e-02 >>>> 21 KSP preconditioned resid norm 7.815461281280e+15 true resid norm >>>> 1.346385634634e+15 ||r(i)||/||b|| 2.574026619006e-02 >>>> 22 KSP preconditioned resid norm 6.943207518072e+15 true resid norm >>>> 1.184751633436e+15 ||r(i)||/||b|| 2.265013947659e-02 >>>> 23 KSP preconditioned resid norm 6.246190157097e+15 true resid norm >>>> 1.023840274541e+15 ||r(i)||/||b|| 1.957382827391e-02 >>>> 24 KSP preconditioned resid norm 5.682318425500e+15 true resid norm >>>> 9.496257176169e+14 ||r(i)||/||b|| 1.815499075718e-02 >>>> 25 KSP preconditioned resid norm 5.282641218542e+15 true resid norm >>>> 8.559439950771e+14 ||r(i)||/||b|| 1.636397901932e-02 >>>> 26 KSP preconditioned resid norm 5.115699254805e+15 true resid norm >>>> 8.571023693113e+14 ||r(i)||/||b|| 1.638612487439e-02 >>>> 27 KSP preconditioned resid norm 4.941038732000e+15 true resid norm >>>> 9.550184315260e+14 ||r(i)||/||b|| 1.825808892456e-02 >>>> 28 KSP preconditioned resid norm 4.836641976772e+15 true resid norm >>>> 7.564719174843e+14 ||r(i)||/||b|| 1.446226699132e-02 >>>> 29 KSP preconditioned resid norm 4.905885514019e+15 true resid norm >>>> 7.773698572205e+14 ||r(i)||/||b|| 1.486179482182e-02 >>>> 30 KSP preconditioned resid norm 5.169672483375e+15 true resid norm >>>> 8.117514654202e+14 ||r(i)||/||b|| 1.551910408326e-02 >>>> 31 KSP preconditioned resid norm 5.732174203621e+15 true resid norm >>>> 9.079138157636e+14 ||r(i)||/||b|| 1.735754058438e-02 >>>> 32 KSP preconditioned resid norm 8.584579912384e+15 true resid norm >>>> 1.381429969496e+15 ||r(i)||/||b|| 2.641024549212e-02 >>>> 33 KSP preconditioned resid norm 1.658456221337e+16 true resid norm >>>> 2.668310391151e+15 ||r(i)||/||b|| 5.101288811997e-02 >>>> 34 KSP preconditioned resid norm 3.797117274875e+16 true resid norm >>>> 6.214594485387e+15 ||r(i)||/||b|| 1.188109202908e-01 >>>> 35 KSP preconditioned resid norm 6.896188371971e+15 true resid norm >>>> 1.129650731508e+15 ||r(i)||/||b|| 2.159671774774e-02 >>>> 36 KSP preconditioned resid norm 3.678389972715e+15 true resid norm >>>> 6.014447539523e+14 ||r(i)||/||b|| 1.149845012239e-02 >>>> 37 KSP preconditioned resid norm 2.344506104501e+15 true resid norm >>>> 4.262436512363e+14 ||r(i)||/||b|| 8.148946900809e-03 >>>> 38 KSP preconditioned resid norm 1.979277723710e+15 true resid norm >>>> 3.194432632084e+14 ||r(i)||/||b|| 6.107131876701e-03 >>>> 39 KSP preconditioned resid norm 1.806670892810e+15 true resid norm >>>> 3.108273060962e+14 ||r(i)||/||b|| 5.942411588660e-03 >>>> 40 KSP preconditioned resid norm 1.663957842585e+15 true resid norm >>>> 2.662101226861e+14 ||r(i)||/||b|| 5.089418101442e-03 >>>> 41 KSP preconditioned resid norm 1.486329834699e+15 true resid norm >>>> 2.427881601889e+14 ||r(i)||/||b|| 4.641635880760e-03 >>>> 42 KSP preconditioned resid norm 1.346929765546e+15 true resid norm >>>> 2.207126725819e+14 ||r(i)||/||b|| 4.219595632660e-03 >>>> 43 KSP preconditioned resid norm 1.242596801597e+15 true resid norm >>>> 2.017748115787e+14 ||r(i)||/||b|| 3.857540682909e-03 >>>> 44 KSP preconditioned resid norm 1.178815061343e+15 true resid norm >>>> 1.881589524072e+14 ||r(i)||/||b|| 3.597232023587e-03 >>>> 45 KSP preconditioned resid norm 9.347996687830e+14 true resid norm >>>> 2.754924905110e+14 ||r(i)||/||b|| 5.266878861971e-03 >>>> 46 KSP preconditioned resid norm 7.515414326072e+14 true resid norm >>>> 1.170313953112e+14 ||r(i)||/||b|| 2.237411920043e-03 >>>> 47 KSP preconditioned resid norm 7.344982573734e+14 true resid norm >>>> 1.334969192887e+14 ||r(i)||/||b|| 2.552200610027e-03 >>>> 48 KSP preconditioned resid norm 8.030126805909e+14 true resid norm >>>> 1.311086080593e+14 ||r(i)||/||b|| 2.506540759528e-03 >>>> 49 KSP preconditioned resid norm 9.282223496248e+14 true resid norm >>>> 1.446635532628e+14 ||r(i)||/||b|| 2.765684862641e-03 >>>> 50 KSP preconditioned resid norm 1.074003818336e+15 true resid norm >>>> 3.137568207714e+14 ||r(i)||/||b|| 5.998418193015e-03 >>>> 51 KSP preconditioned resid norm 1.638671425800e+15 true resid norm >>>> 2.543484445494e+14 ||r(i)||/||b|| 4.862645960648e-03 >>>> 52 KSP preconditioned resid norm 2.418883444028e+15 true resid norm >>>> 6.581384678327e+14 ||r(i)||/||b|| 1.258232330779e-02 >>>> 53 KSP preconditioned resid norm 8.572181890774e+14 true resid norm >>>> 1.327918910006e+14 ||r(i)||/||b|| 2.538721844848e-03 >>>> 54 KSP preconditioned resid norm 2.577275091696e+14 true resid norm >>>> 8.010767963629e+13 ||r(i)||/||b|| 1.531502523990e-03 >>>> 0 KSP preconditioned resid norm 6.542967041071e+18 true resid norm >>>> 3.545980595607e+18 ||r(i)||/||b|| 4.156826068038e+02 >>>> 1 KSP preconditioned resid norm 1.591191551470e+18 true resid norm >>>> 5.223269779130e+17 ||r(i)||/||b|| 6.123052112913e+01 >>>> 2 KSP preconditioned resid norm 6.570663371928e+17 true resid norm >>>> 1.552539857829e+17 ||r(i)||/||b|| 1.819986877731e+01 >>>> 3 KSP preconditioned resid norm 4.165904471738e+17 true resid norm >>>> 8.738677344351e+16 ||r(i)||/||b|| 1.024403851227e+01 >>>> 4 KSP preconditioned resid norm 2.995075360323e+17 true resid norm >>>> 4.934782956985e+16 ||r(i)||/||b|| 5.784869342240e+00 >>>> 5 KSP preconditioned resid norm 2.594552595600e+17 true resid norm >>>> 4.469892332113e+16 ||r(i)||/||b|| 5.239894710780e+00 >>>> 6 KSP preconditioned resid norm 2.219662091794e+17 true resid norm >>>> 4.652217437246e+16 ||r(i)||/||b|| 5.453627902330e+00 >>>> 7 KSP preconditioned resid norm 1.674952385698e+17 true resid norm >>>> 2.988890119169e+16 ||r(i)||/||b|| 3.503768852333e+00 >>>> 8 KSP preconditioned resid norm 1.081926357303e+17 true resid norm >>>> 1.908191297028e+16 ||r(i)||/||b|| 2.236904323761e+00 >>>> 9 KSP preconditioned resid norm 5.710239913499e+16 true resid norm >>>> 9.635393149266e+15 ||r(i)||/||b|| 1.129522633832e+00 >>>> 10 KSP preconditioned resid norm 1.779371916701e+16 true resid norm >>>> 2.952475144259e+15 ||r(i)||/||b|| 3.461080881292e-01 >>>> 11 KSP preconditioned resid norm 1.125590077314e+16 true resid norm >>>> 2.030360591090e+15 ||r(i)||/||b|| 2.380119012218e-01 >>>> 12 KSP preconditioned resid norm 1.086608349874e+16 true resid norm >>>> 1.877856566597e+15 ||r(i)||/||b|| 2.201344005587e-01 >>>> 13 KSP preconditioned resid norm 1.075368648574e+16 true resid norm >>>> 1.822636849246e+15 ||r(i)||/||b|| 2.136611908395e-01 >>>> 14 KSP preconditioned resid norm 1.173878698877e+16 true resid norm >>>> 2.130421173676e+15 ||r(i)||/||b|| 2.497416449940e-01 >>>> 15 KSP preconditioned resid norm 4.944017897832e+16 true resid norm >>>> 1.361808358525e+16 ||r(i)||/||b|| 1.596399171332e+00 >>>> 16 KSP preconditioned resid norm 1.780572750257e+17 true resid norm >>>> 3.371295639234e+16 ||r(i)||/||b|| 3.952049149280e+00 >>>> 17 KSP preconditioned resid norm 3.251503916379e+16 true resid norm >>>> 8.719191629936e+15 ||r(i)||/||b|| 1.022119610706e+00 >>>> 18 KSP preconditioned resid norm 2.075735168556e+16 true resid norm >>>> 3.945307122742e+15 ||r(i)||/||b|| 4.624942255619e-01 >>>> 19 KSP preconditioned resid norm 1.483613863843e+16 true resid norm >>>> 3.390657011871e+15 ||r(i)||/||b|| 3.974745793076e-01 >>>> 20 KSP preconditioned resid norm 1.134405248775e+16 true resid norm >>>> 2.490065225313e+15 ||r(i)||/||b|| 2.919014292554e-01 >>>> 21 KSP preconditioned resid norm 8.553736210329e+15 true resid norm >>>> 1.657655420050e+15 ||r(i)||/||b|| 1.943210087055e-01 >>>> 22 KSP preconditioned resid norm 6.586515952486e+15 true resid norm >>>> 1.286392357530e+15 ||r(i)||/||b|| 1.507991694069e-01 >>>> 23 KSP preconditioned resid norm 4.780127454522e+15 true resid norm >>>> 9.064248964802e+14 ||r(i)||/||b|| 1.062569446397e-01 >>>> 24 KSP preconditioned resid norm 3.027986246405e+15 true resid norm >>>> 6.179735775190e+14 ||r(i)||/||b|| 7.244282948339e-02 >>>> 25 KSP preconditioned resid norm 1.571830581724e+15 true resid norm >>>> 2.758055375192e+14 ||r(i)||/||b|| 3.233169548331e-02 >>>> 26 KSP preconditioned resid norm 1.182949949046e+15 true resid norm >>>> 2.283703518838e+14 ||r(i)||/||b|| 2.677103854019e-02 >>>> 27 KSP preconditioned resid norm 1.078564628165e+15 true resid norm >>>> 2.041848112402e+14 ||r(i)||/||b|| 2.393585422076e-02 >>>> 28 KSP preconditioned resid norm 1.016575362098e+15 true resid norm >>>> 1.676746397025e+14 ||r(i)||/||b|| 1.965589755702e-02 >>>> 29 KSP preconditioned resid norm 9.501065512919e+14 true resid norm >>>> 1.807838083536e+14 ||r(i)||/||b|| 2.119263845308e-02 >>>> 30 KSP preconditioned resid norm 9.022105299639e+14 true resid norm >>>> 1.464542351381e+14 ||r(i)||/||b|| 1.716830552177e-02 >>>> 31 KSP preconditioned resid norm 8.706508374697e+14 true resid norm >>>> 1.587763993315e+14 ||r(i)||/||b|| 1.861278870358e-02 >>>> 32 KSP preconditioned resid norm 8.363545887412e+14 true resid norm >>>> 1.341498283380e+14 ||r(i)||/||b|| 1.572590397559e-02 >>>> 33 KSP preconditioned resid norm 8.049250107078e+14 true resid norm >>>> 1.594310708223e+14 ||r(i)||/||b|| 1.868953349802e-02 >>>> 34 KSP preconditioned resid norm 7.730163706333e+14 true resid norm >>>> 1.360746921537e+14 ||r(i)||/||b|| 1.595154886763e-02 >>>> 35 KSP preconditioned resid norm 7.656443951451e+14 true resid norm >>>> 1.202971650940e+14 ||r(i)||/||b|| 1.410200587090e-02 >>>> 36 KSP preconditioned resid norm 7.731881795958e+14 true resid norm >>>> 1.274146022068e+14 ||r(i)||/||b|| 1.493635753557e-02 >>>> 37 KSP preconditioned resid norm 8.066446225342e+14 true resid norm >>>> 1.271087970298e+14 ||r(i)||/||b|| 1.490050908979e-02 >>>> 38 KSP preconditioned resid norm 8.449932808607e+14 true resid norm >>>> 1.374808484319e+14 ||r(i)||/||b|| 1.611638753257e-02 >>>> 39 KSP preconditioned resid norm 8.841110986186e+14 true resid norm >>>> 1.396540752943e+14 ||r(i)||/||b|| 1.637114713515e-02 >>>> 40 KSP preconditioned resid norm 9.488359303223e+14 true resid norm >>>> 1.529468173159e+14 ||r(i)||/||b|| 1.792940768006e-02 >>>> 41 KSP preconditioned resid norm 1.207535627829e+15 true resid norm >>>> 1.911287075526e+14 ||r(i)||/||b|| 2.240533394033e-02 >>>> 42 KSP preconditioned resid norm 2.145532208476e+15 true resid norm >>>> 3.678432914805e+14 ||r(i)||/||b|| 4.312095178618e-02 >>>> 43 KSP preconditioned resid norm 1.285566323841e+16 true resid norm >>>> 2.041241798720e+15 ||r(i)||/||b|| 2.392874662258e-01 >>>> 44 KSP preconditioned resid norm 1.228721458245e+15 true resid norm >>>> 2.037912017178e+14 ||r(i)||/||b|| 2.388971278599e-02 >>>> 45 KSP preconditioned resid norm 6.120797893644e+14 true resid norm >>>> 9.792977962736e+13 ||r(i)||/||b|| 1.147995737192e-02 >>>> 46 KSP preconditioned resid norm 4.263304414021e+14 true resid norm >>>> 6.948352885735e+13 ||r(i)||/||b|| 8.145305262283e-03 >>>> 47 KSP preconditioned resid norm 2.943667166575e+14 true resid norm >>>> 4.740665276317e+13 ||r(i)||/||b|| 5.557312136691e-03 >>>> 48 KSP preconditioned resid norm 1.946102625201e+14 true resid norm >>>> 3.133411275125e+13 ||r(i)||/||b|| 3.673185828050e-03 >>>> 49 KSP preconditioned resid norm 1.369383238938e+14 true resid norm >>>> 2.605829767015e+13 ||r(i)||/||b|| 3.054720919178e-03 >>>> 50 KSP preconditioned resid norm 9.297342410318e+13 true resid norm >>>> 1.484605459745e+13 ||r(i)||/||b|| 1.740349815638e-03 >>>> 51 KSP preconditioned resid norm 6.674823642018e+13 true resid norm >>>> 2.111621172189e+13 ||r(i)||/||b|| 2.475377881439e-03 >>>> 52 KSP preconditioned resid norm 5.794642992605e+13 true resid norm >>>> 9.469052681739e+12 ||r(i)||/||b|| 1.110023136501e-03 >>>> 53 KSP preconditioned resid norm 4.582540149696e+13 true resid norm >>>> 7.443523694856e+12 ||r(i)||/||b|| 8.725776269382e-04 >>>> 0 KSP preconditioned resid norm 1.892653228182e+19 true resid norm >>>> 1.028330526045e+19 ||r(i)||/||b|| 7.841194129911e+01 >>>> 1 KSP preconditioned resid norm 4.581281818866e+18 true resid norm >>>> 1.519814562161e+18 ||r(i)||/||b|| 1.158884300480e+01 >>>> 2 KSP preconditioned resid norm 1.874494262231e+18 true resid norm >>>> 4.646649902787e+17 ||r(i)||/||b|| 3.543149115843e+00 >>>> 3 KSP preconditioned resid norm 1.173263764111e+18 true resid norm >>>> 2.551017670341e+17 ||r(i)||/||b|| 1.945194105918e+00 >>>> 4 KSP preconditioned resid norm 8.301325588811e+17 true resid norm >>>> 1.381381450838e+17 ||r(i)||/||b|| 1.053326712486e+00 >>>> 5 KSP preconditioned resid norm 7.052947600755e+17 true resid norm >>>> 1.287824897888e+17 ||r(i)||/||b|| 9.819882590194e-01 >>>> 6 KSP preconditioned resid norm 5.863848966069e+17 true resid norm >>>> 1.166961566059e+17 ||r(i)||/||b|| 8.898279249584e-01 >>>> 7 KSP preconditioned resid norm 4.330736686179e+17 true resid norm >>>> 8.611330077675e+16 ||r(i)||/||b|| 6.566284783504e-01 >>>> 8 KSP preconditioned resid norm 2.702589924096e+17 true resid norm >>>> 4.803726753137e+16 ||r(i)||/||b|| 3.662922870069e-01 >>>> 9 KSP preconditioned resid norm 1.546977321412e+17 true resid norm >>>> 2.878601764016e+16 ||r(i)||/||b|| 2.194982516096e-01 >>>> 10 KSP preconditioned resid norm 9.910027101694e+16 true resid norm >>>> 1.648820746608e+16 ||r(i)||/||b|| 1.257253697341e-01 >>>> 11 KSP preconditioned resid norm 9.241668505040e+16 true resid norm >>>> 1.559203946128e+16 ||r(i)||/||b|| 1.188919371745e-01 >>>> 12 KSP preconditioned resid norm 9.244329678182e+16 true resid norm >>>> 1.506886748242e+16 ||r(i)||/||b|| 1.149026623784e-01 >>>> 13 KSP preconditioned resid norm 7.090985777634e+16 true resid norm >>>> 1.150411231671e+16 ||r(i)||/||b|| 8.772080151562e-02 >>>> 14 KSP preconditioned resid norm 2.014608975967e+17 true resid norm >>>> 3.263298275594e+16 ||r(i)||/||b|| 2.488320110574e-01 >>>> 15 KSP preconditioned resid norm 1.667109940317e+16 true resid norm >>>> 4.390284971711e+15 ||r(i)||/||b|| 3.347666521311e-02 >>>> 16 KSP preconditioned resid norm 1.169352181037e+16 true resid norm >>>> 1.861078956987e+15 ||r(i)||/||b|| 1.419104171589e-02 >>>> 17 KSP preconditioned resid norm 1.018452985290e+16 true resid norm >>>> 1.996695440591e+15 ||r(i)||/||b|| 1.522514033324e-02 >>>> 18 KSP preconditioned resid norm 9.390559252345e+15 true resid norm >>>> 1.536845912378e+15 ||r(i)||/||b|| 1.171870992984e-02 >>>> 19 KSP preconditioned resid norm 8.796086996183e+15 true resid norm >>>> 1.422370571244e+15 ||r(i)||/||b|| 1.084581609835e-02 >>>> 20 KSP preconditioned resid norm 8.297954532675e+15 true resid norm >>>> 1.485654737712e+15 ||r(i)||/||b|| 1.132836856767e-02 >>>> 21 KSP preconditioned resid norm 7.837153561450e+15 true resid norm >>>> 1.247795221846e+15 ||r(i)||/||b|| 9.514649542211e-03 >>>> 22 KSP preconditioned resid norm 7.441474574169e+15 true resid norm >>>> 1.321880544976e+15 ||r(i)||/||b|| 1.007956265733e-02 >>>> 23 KSP preconditioned resid norm 7.117292465449e+15 true resid norm >>>> 1.122736430687e+15 ||r(i)||/||b|| 8.561055114844e-03 >>>> 24 KSP preconditioned resid norm 5.694316647476e+15 true resid norm >>>> 2.071287152890e+15 ||r(i)||/||b|| 1.579391475139e-02 >>>> 25 KSP preconditioned resid norm 4.803477075252e+15 true resid norm >>>> 7.688470974380e+14 ||r(i)||/||b|| 5.862589113657e-03 >>>> 26 KSP preconditioned resid norm 4.890644639484e+15 true resid norm >>>> 7.629702578172e+14 ||r(i)||/||b|| 5.817777217899e-03 >>>> 27 KSP preconditioned resid norm 5.168669214975e+15 true resid norm >>>> 8.754612707062e+14 ||r(i)||/||b|| 6.675540211016e-03 >>>> 28 KSP preconditioned resid norm 8.939764002038e+15 true resid norm >>>> 1.397979783339e+15 ||r(i)||/||b|| 1.065983221661e-02 >>>> 29 KSP preconditioned resid norm 8.205557149295e+15 true resid norm >>>> 1.863983796228e+15 ||r(i)||/||b|| 1.421319160626e-02 >>>> 30 KSP preconditioned resid norm 1.967760781914e+15 true resid norm >>>> 3.069470515034e+14 ||r(i)||/||b|| 2.340523165932e-03 >>>> 31 KSP preconditioned resid norm 1.158003829729e+15 true resid norm >>>> 2.612136672864e+14 ||r(i)||/||b|| 1.991798378735e-03 >>>> 32 KSP preconditioned resid norm 8.020817038350e+14 true resid norm >>>> 1.249397423467e+14 ||r(i)||/||b|| 9.526866600465e-04 >>>> 0 KSP preconditioned resid norm 6.470845423033e+18 true resid norm >>>> 3.511573094792e+18 ||r(i)||/||b|| 1.662215647083e+02 >>>> 1 KSP preconditioned resid norm 1.571609661901e+18 true resid norm >>>> 5.187508694693e+17 ||r(i)||/||b|| 2.455525739870e+01 >>>> 2 KSP preconditioned resid norm 6.468393005498e+17 true resid norm >>>> 1.566999779328e+17 ||r(i)||/||b|| 7.417449336415e+00 >>>> 3 KSP preconditioned resid norm 4.084082755984e+17 true resid norm >>>> 8.688467105425e+16 ||r(i)||/||b|| 4.112716888400e+00 >>>> 4 KSP preconditioned resid norm 2.920922708944e+17 true resid norm >>>> 4.832004038605e+16 ||r(i)||/||b|| 2.287246343141e+00 >>>> 5 KSP preconditioned resid norm 2.515891595070e+17 true resid norm >>>> 4.435607154638e+16 ||r(i)||/||b|| 2.099610464520e+00 >>>> 6 KSP preconditioned resid norm 2.133101049372e+17 true resid norm >>>> 4.355693207474e+16 ||r(i)||/||b|| 2.061782912647e+00 >>>> 7 KSP preconditioned resid norm 1.601303798951e+17 true resid norm >>>> 2.999351550015e+16 ||r(i)||/||b|| 1.419753752223e+00 >>>> 8 KSP preconditioned resid norm 1.019899000725e+17 true resid norm >>>> 1.811131044506e+16 ||r(i)||/||b|| 8.573053386130e-01 >>>> 9 KSP preconditioned resid norm 5.552370825926e+16 true resid norm >>>> 9.849613114257e+15 ||r(i)||/||b|| 4.662349492457e-01 >>>> 10 KSP preconditioned resid norm 2.614277580792e+16 true resid norm >>>> 4.400858558792e+15 ||r(i)||/||b|| 2.083162092759e-01 >>>> 11 KSP preconditioned resid norm 2.230379890809e+16 true resid norm >>>> 4.001941486401e+15 ||r(i)||/||b|| 1.894333273960e-01 >>>> 12 KSP preconditioned resid norm 2.202819377737e+16 true resid norm >>>> 3.668912744052e+15 ||r(i)||/||b|| 1.736692931151e-01 >>>> 13 KSP preconditioned resid norm 2.171381974288e+16 true resid norm >>>> 3.746629440349e+15 ||r(i)||/||b|| 1.773480406489e-01 >>>> 14 KSP preconditioned resid norm 2.756519085776e+16 true resid norm >>>> 4.738155898553e+15 ||r(i)||/||b|| 2.242822991374e-01 >>>> 15 KSP preconditioned resid norm 1.910971370209e+16 true resid norm >>>> 7.272906924885e+15 ||r(i)||/||b|| 3.442656428895e-01 >>>> 16 KSP preconditioned resid norm 8.265967788035e+15 true resid norm >>>> 1.450280377529e+15 ||r(i)||/||b|| 6.864953885656e-02 >>>> 17 KSP preconditioned resid norm 5.450920356042e+15 true resid norm >>>> 1.778917951827e+15 ||r(i)||/||b|| 8.420571563182e-02 >>>> 18 KSP preconditioned resid norm 4.184159413509e+15 true resid norm >>>> 7.350039137637e+14 ||r(i)||/||b|| 3.479167236863e-02 >>>> 19 KSP preconditioned resid norm 3.481860538622e+15 true resid norm >>>> 7.678205813365e+14 ||r(i)||/||b|| 3.634506103098e-02 >>>> 20 KSP preconditioned resid norm 3.027493421078e+15 true resid norm >>>> 5.662352682787e+14 ||r(i)||/||b|| 2.680294834981e-02 >>>> 21 KSP preconditioned resid norm 2.649808113090e+15 true resid norm >>>> 5.331458151920e+14 ||r(i)||/||b|| 2.523664728788e-02 >>>> 22 KSP preconditioned resid norm 2.408644968014e+15 true resid norm >>>> 3.911896234018e+14 ||r(i)||/||b|| 1.851710032633e-02 >>>> 23 KSP preconditioned resid norm 2.245371706567e+15 true resid norm >>>> 3.514281238053e+14 ||r(i)||/||b|| 1.663497556353e-02 >>>> 24 KSP preconditioned resid norm 2.104176299070e+15 true resid norm >>>> 3.427795705053e+14 ||r(i)||/||b|| 1.622559320890e-02 >>>> 25 KSP preconditioned resid norm 1.992290690493e+15 true resid norm >>>> 3.118626867030e+14 ||r(i)||/||b|| 1.476213148881e-02 >>>> 26 KSP preconditioned resid norm 1.936852873636e+15 true resid norm >>>> 3.191431362940e+14 ||r(i)||/||b|| 1.510675416649e-02 >>>> 27 KSP preconditioned resid norm 1.840320660654e+15 true resid norm >>>> 4.277077423449e+14 ||r(i)||/||b|| 2.024569850927e-02 >>>> 28 KSP preconditioned resid norm 1.797325493063e+15 true resid norm >>>> 2.805438081480e+14 ||r(i)||/||b|| 1.327964120375e-02 >>>> 29 KSP preconditioned resid norm 1.907733126356e+15 true resid norm >>>> 3.072709450643e+14 ||r(i)||/||b|| 1.454477976088e-02 >>>> 30 KSP preconditioned resid norm 2.241086833192e+15 true resid norm >>>> 3.515931723170e+14 ||r(i)||/||b|| 1.664278819369e-02 >>>> 31 KSP preconditioned resid norm 3.121906541214e+15 true resid norm >>>> 5.044161632328e+14 ||r(i)||/||b|| 2.387671896708e-02 >>>> 32 KSP preconditioned resid norm 2.288516501159e+16 true resid norm >>>> 3.696560518057e+15 ||r(i)||/||b|| 1.749780103571e-01 >>>> 33 KSP preconditioned resid norm 2.665344016377e+15 true resid norm >>>> 4.419048360378e+14 ||r(i)||/||b|| 2.091772300206e-02 >>>> 34 KSP preconditioned resid norm 1.193724857597e+15 true resid norm >>>> 1.964465911853e+14 ||r(i)||/||b|| 9.298869448811e-03 >>>> 35 KSP preconditioned resid norm 7.370892713374e+14 true resid norm >>>> 1.243994812053e+14 ||r(i)||/||b|| 5.888493805102e-03 >>>> 36 KSP preconditioned resid norm 5.570336976172e+14 true resid norm >>>> 9.038862530299e+13 ||r(i)||/||b|| 4.278577812313e-03 >>>> 37 KSP preconditioned resid norm 4.452876529911e+14 true resid norm >>>> 9.072905275923e+13 ||r(i)||/||b|| 4.294692067355e-03 >>>> 38 KSP preconditioned resid norm 4.015373409323e+14 true resid norm >>>> 6.374984875718e+13 ||r(i)||/||b|| 3.017621824831e-03 >>>> 39 KSP preconditioned resid norm 3.753907061691e+14 true resid norm >>>> 7.170117945509e+13 ||r(i)||/||b|| 3.394000898950e-03 >>>> 40 KSP preconditioned resid norm 3.520150126535e+14 true resid norm >>>> 5.546554172401e+13 ||r(i)||/||b|| 2.625481197140e-03 >>>> 41 KSP preconditioned resid norm 3.283953946659e+14 true resid norm >>>> 5.672854000420e+13 ||r(i)||/||b|| 2.685265671132e-03 >>>> 42 KSP preconditioned resid norm 3.086422594501e+14 true resid norm >>>> 5.033158713486e+13 ||r(i)||/||b|| 2.382463625836e-03 >>>> 43 KSP preconditioned resid norm 2.903330837015e+14 true resid norm >>>> 5.711996083788e+13 ||r(i)||/||b|| 2.703793715879e-03 >>>> 44 KSP preconditioned resid norm 2.805776139313e+14 true resid norm >>>> 4.359341431583e+13 ||r(i)||/||b|| 2.063509812540e-03 >>>> 45 KSP preconditioned resid norm 2.850165946773e+14 true resid norm >>>> 4.589953632859e+13 ||r(i)||/||b|| 2.172670920403e-03 >>>> 46 KSP preconditioned resid norm 3.036441891069e+14 true resid norm >>>> 4.745822530443e+13 ||r(i)||/||b|| 2.246452018920e-03 >>>> 47 KSP preconditioned resid norm 3.630944657131e+14 true resid norm >>>> 5.763700439451e+13 ||r(i)||/||b|| 2.728268157016e-03 >>>> 48 KSP preconditioned resid norm 1.221109219167e+15 true resid norm >>>> 1.980487895228e+14 ||r(i)||/||b|| 9.374710078478e-03 >>>> 49 KSP preconditioned resid norm 3.188684128549e+15 true resid norm >>>> 5.020159499531e+14 ||r(i)||/||b|| 2.376310401554e-02 >>>> 50 KSP preconditioned resid norm 3.895126724245e+13 true resid norm >>>> 6.299322456844e+12 ||r(i)||/||b|| 2.981806749036e-04 >>>> 0 KSP preconditioned resid norm 6.694681597053e+18 true resid norm >>>> 3.650754072115e+18 ||r(i)||/||b|| 5.229798617888e+00 >>>> 1 KSP preconditioned resid norm 1.597667334483e+18 true resid norm >>>> 5.366749713509e+17 ||r(i)||/||b|| 7.688006280303e-01 >>>> 2 KSP preconditioned resid norm 6.300446759111e+17 true resid norm >>>> 1.705328267334e+17 ||r(i)||/||b|| 2.442926376135e-01 >>>> 3 KSP preconditioned resid norm 3.642174486049e+17 true resid norm >>>> 9.481576844129e+16 ||r(i)||/||b|| 1.358260142845e-01 >>>> 4 KSP preconditioned resid norm 2.247332959635e+17 true resid norm >>>> 3.802800729856e+16 ||r(i)||/||b|| 5.447609345427e-02 >>>> 5 KSP preconditioned resid norm 1.557523701769e+17 true resid norm >>>> 3.829141999541e+16 ||r(i)||/||b|| 5.485343888228e-02 >>>> 6 KSP preconditioned resid norm 1.004692846569e+17 true resid norm >>>> 1.699728099873e+16 ||r(i)||/||b|| 2.434903992959e-02 >>>> 7 KSP preconditioned resid norm 7.103482802189e+16 true resid norm >>>> 1.472858884963e+16 ||r(i)||/||b|| 2.109908037838e-02 >>>> 8 KSP preconditioned resid norm 5.766847885215e+16 true resid norm >>>> 9.397954062732e+15 ||r(i)||/||b|| 1.346280965449e-02 >>>> 9 KSP preconditioned resid norm 4.523472016725e+16 true resid norm >>>> 1.246374917365e+16 ||r(i)||/||b|| 1.785463959348e-02 >>>> 10 KSP preconditioned resid norm 3.195342481960e+16 true resid norm >>>> 5.030521484214e+15 ||r(i)||/||b|| 7.206350738975e-03 >>>> 11 KSP preconditioned resid norm 2.437460220226e+16 true resid norm >>>> 4.974645944208e+15 ||r(i)||/||b|| 7.126307598263e-03 >>>> 12 KSP preconditioned resid norm 1.258045555577e+16 true resid norm >>>> 1.885910304237e+15 ||r(i)||/||b|| 2.701614764439e-03 >>>> 13 KSP preconditioned resid norm 2.495567167302e+15 true resid norm >>>> 4.460991814541e+14 ||r(i)||/||b|| 6.390484914966e-04 >>>> 14 KSP preconditioned resid norm 7.678096351052e+14 true resid norm >>>> 1.316276220022e+14 ||r(i)||/||b|| 1.885599363926e-04 >>>> 15 KSP preconditioned resid norm 5.950867501843e+14 true resid norm >>>> 1.439410416931e+14 ||r(i)||/||b|| 2.061992251557e-04 >>>> 16 KSP preconditioned resid norm 3.263035989760e+14 true resid norm >>>> 5.046543721443e+13 ||r(i)||/||b|| 7.229303003757e-05 >>>> 17 KSP preconditioned resid norm 1.436782488882e+14 true resid norm >>>> 7.588970356348e+13 ||r(i)||/||b|| 1.087139420975e-04 >>>> 18 KSP preconditioned resid norm 5.401533739797e+13 true resid norm >>>> 8.625318785849e+12 ||r(i)||/||b|| 1.235598985142e-05 >>>> 19 KSP preconditioned resid norm 3.528859989845e+13 true resid norm >>>> 1.233587622450e+13 ||r(i)||/||b|| 1.767145832202e-05 >>>> 0 KSP preconditioned resid norm 4.827874033841e+18 true resid norm >>>> 2.629049503905e+18 ||r(i)||/||b|| 2.387244450275e+01 >>>> 1 KSP preconditioned resid norm 1.161059892070e+18 true resid norm >>>> 3.885117260269e+17 ||r(i)||/||b|| 3.527786222537e+00 >>>> 2 KSP preconditioned resid norm 4.696931552127e+17 true resid norm >>>> 1.228954734891e+17 ||r(i)||/||b|| 1.115922452639e+00 >>>> 3 KSP preconditioned resid norm 2.870101730937e+17 true resid norm >>>> 6.759381587642e+16 ||r(i)||/||b|| 6.137692028400e-01 >>>> 4 KSP preconditioned resid norm 1.966639305793e+17 true resid norm >>>> 3.310590026030e+16 ||r(i)||/||b|| 3.006100742887e-01 >>>> 5 KSP preconditioned resid norm 1.592574497021e+17 true resid norm >>>> 3.239415772705e+16 ||r(i)||/||b|| 2.941472693472e-01 >>>> 6 KSP preconditioned resid norm 1.230114824013e+17 true resid norm >>>> 2.280737281616e+16 ||r(i)||/||b|| 2.070968009535e-01 >>>> 7 KSP preconditioned resid norm 8.598640818077e+16 true resid norm >>>> 1.938029194893e+16 ||r(i)||/||b|| 1.759780267776e-01 >>>> 8 KSP preconditioned resid norm 5.549085768793e+16 true resid norm >>>> 9.386094648379e+15 ||r(i)||/||b|| 8.522814928290e-02 >>>> 9 KSP preconditioned resid norm 4.474754629124e+16 true resid norm >>>> 7.579167807628e+15 ||r(i)||/||b|| 6.882078964122e-02 >>>> 10 KSP preconditioned resid norm 4.378389530908e+16 true resid norm >>>> 6.842593278076e+15 ||r(i)||/||b|| 6.213250379770e-02 >>>> 11 KSP preconditioned resid norm 4.079498014536e+16 true resid norm >>>> 7.704710526729e+15 ||r(i)||/||b|| 6.996074976369e-02 >>>> 12 KSP preconditioned resid norm 4.084398125729e+16 true resid norm >>>> 6.377957983596e+15 ||r(i)||/||b|| 5.791349602893e-02 >>>> 13 KSP preconditioned resid norm 7.523960017073e+16 true resid norm >>>> 1.132262466824e+16 ||r(i)||/||b|| 1.028123390664e-01 >>>> 14 KSP preconditioned resid norm 2.184546755774e+15 true resid norm >>>> 3.502101254985e+14 ||r(i)||/||b|| 3.179997855820e-03 >>>> 15 KSP preconditioned resid norm 2.222188316359e+15 true resid norm >>>> 3.654675514798e+14 ||r(i)||/||b|| 3.318539201068e-03 >>>> 16 KSP preconditioned resid norm 1.669487598659e+15 true resid norm >>>> 3.629534674524e+14 ||r(i)||/||b|| 3.295710672608e-03 >>>> 17 KSP preconditioned resid norm 1.401917540926e+15 true resid norm >>>> 2.161165666331e+14 ||r(i)||/||b|| 1.962393912861e-03 >>>> 18 KSP preconditioned resid norm 1.134393288674e+15 true resid norm >>>> 2.686682412462e+14 ||r(i)||/||b|| 2.439576610967e-03 >>>> 19 KSP preconditioned resid norm 9.058009317761e+14 true resid norm >>>> 1.409929037370e+14 ||r(i)||/||b|| 1.280251765797e-03 >>>> 20 KSP preconditioned resid norm 7.990098269893e+14 true resid norm >>>> 1.659151006791e+14 ||r(i)||/||b|| 1.506551712794e-03 >>>> 21 KSP preconditioned resid norm 7.119180202197e+14 true resid norm >>>> 1.092130755589e+14 ||r(i)||/||b|| 9.916827664832e-04 >>>> 22 KSP preconditioned resid norm 6.306959900843e+14 true resid norm >>>> 1.460783745609e+14 ||r(i)||/||b|| 1.326429146571e-03 >>>> 23 KSP preconditioned resid norm 5.637228851054e+14 true resid norm >>>> 8.672640572399e+13 ||r(i)||/||b|| 7.874980309399e-04 >>>> 24 KSP preconditioned resid norm 4.960976586203e+14 true resid norm >>>> 1.292271181476e+14 ||r(i)||/||b|| 1.173415411786e-03 >>>> 25 KSP preconditioned resid norm 4.386404759546e+14 true resid norm >>>> 8.037967670881e+13 ||r(i)||/||b|| 7.298681019622e-04 >>>> 26 KSP preconditioned resid norm 4.419824575339e+14 true resid norm >>>> 6.704822670651e+13 ||r(i)||/||b|| 6.088151130974e-04 >>>> 27 KSP preconditioned resid norm 4.437076214979e+14 true resid norm >>>> 1.370393448160e+14 ||r(i)||/||b|| 1.244352435720e-03 >>>> 28 KSP preconditioned resid norm 7.848825938920e+14 true resid norm >>>> 1.196984132854e+14 ||r(i)||/||b|| 1.086892325146e-03 >>>> 29 KSP preconditioned resid norm 1.771840029177e+15 true resid norm >>>> 6.554951997284e+14 ||r(i)||/||b|| 5.952064711635e-03 >>>> 30 KSP preconditioned resid norm 1.052090974187e+15 true resid norm >>>> 1.877636782214e+14 ||r(i)||/||b|| 1.704942406491e-03 >>>> 31 KSP preconditioned resid norm 2.459238084389e+14 true resid norm >>>> 3.913335964196e+13 ||r(i)||/||b|| 3.553409530216e-04 >>>> 32 KSP preconditioned resid norm 1.091607449298e+14 true resid norm >>>> 1.900052367477e+13 ||r(i)||/||b|| 1.725296333429e-04 >>>> 33 KSP preconditioned resid norm 6.424117612547e+13 true resid norm >>>> 1.001417309383e+13 ||r(i)||/||b|| 9.093126282646e-05 >>>> 34 KSP preconditioned resid norm 3.801393062657e+13 true resid norm >>>> 7.035998067262e+12 ||r(i)||/||b|| 6.388866894011e-05 >>>> 35 KSP preconditioned resid norm 2.524483314829e+13 true resid norm >>>> 4.006295820236e+12 ||r(i)||/||b|| 3.637819466241e-05 >>>> 36 KSP preconditioned resid norm 1.579134906712e+13 true resid norm >>>> 2.528645237998e+12 ||r(i)||/||b|| 2.296074798956e-05 >>>> 37 KSP preconditioned resid norm 1.074219589287e+13 true resid norm >>>> 1.786181249150e+12 ||r(i)||/||b|| 1.621898434352e-05 >>>> 38 KSP preconditioned resid norm 6.572405677048e+12 true resid norm >>>> 1.033900747383e+12 ||r(i)||/||b|| 9.388084239790e-06 >>>> 0 KSP preconditioned resid norm 9.565416792628e+15 true resid norm >>>> 5.898678788482e+15 ||r(i)||/||b|| 1.120363504643e-01 >>>> 1 KSP preconditioned resid norm 4.584205668003e+15 true resid norm >>>> 8.149165580746e+14 ||r(i)||/||b|| 1.547808931009e-02 >>>> 2 KSP preconditioned resid norm 3.775899089492e+15 true resid norm >>>> 6.134705686008e+14 ||r(i)||/||b|| 1.165193191355e-02 >>>> 3 KSP preconditioned resid norm 3.695846031998e+15 true resid norm >>>> 6.195110682482e+14 ||r(i)||/||b|| 1.176666193357e-02 >>>> 4 KSP preconditioned resid norm 3.863984235758e+15 true resid norm >>>> 6.333337603322e+14 ||r(i)||/||b|| 1.202920275504e-02 >>>> 5 KSP preconditioned resid norm 4.289553507880e+15 true resid norm >>>> 6.781260017663e+14 ||r(i)||/||b|| 1.287996263523e-02 >>>> 6 KSP preconditioned resid norm 5.214022047398e+15 true resid norm >>>> 7.772905386519e+14 ||r(i)||/||b|| 1.476344081849e-02 >>>> 7 KSP preconditioned resid norm 6.520856882448e+15 true resid norm >>>> 1.156631144536e+15 ||r(i)||/||b|| 2.196843342618e-02 >>>> 8 KSP preconditioned resid norm 7.245755589579e+15 true resid norm >>>> 1.125518734778e+15 ||r(i)||/||b|| 2.137750095326e-02 >>>> 9 KSP preconditioned resid norm 7.402293382659e+15 true resid norm >>>> 1.342363548018e+15 ||r(i)||/||b|| 2.549613537355e-02 >>>> 10 KSP preconditioned resid norm 6.814780966449e+15 true resid norm >>>> 1.040341685629e+15 ||r(i)||/||b|| 1.975969363197e-02 >>>> 11 KSP preconditioned resid norm 4.531806636058e+15 true resid norm >>>> 6.813426758243e+14 ||r(i)||/||b|| 1.294105842210e-02 >>>> 12 KSP preconditioned resid norm 4.019547715129e+15 true resid norm >>>> 1.067806433320e+15 ||r(i)||/||b|| 2.028134436224e-02 >>>> 13 KSP preconditioned resid norm 1.578855518519e+15 true resid norm >>>> 2.462140978868e+14 ||r(i)||/||b|| 4.676458907031e-03 >>>> 14 KSP preconditioned resid norm 1.216192317102e+15 true resid norm >>>> 3.547355821423e+14 ||r(i)||/||b|| 6.737657944804e-03 >>>> 15 KSP preconditioned resid norm 1.068138625918e+15 true resid norm >>>> 1.711364338225e+14 ||r(i)||/||b|| 3.250473905172e-03 >>>> 16 KSP preconditioned resid norm 1.009888628733e+15 true resid norm >>>> 1.725105447584e+14 ||r(i)||/||b|| 3.276573033454e-03 >>>> 17 KSP preconditioned resid norm 9.012147556868e+14 true resid norm >>>> 1.461953215911e+14 ||r(i)||/||b|| 2.776755757239e-03 >>>> 18 KSP preconditioned resid norm 8.184497465452e+14 true resid norm >>>> 1.394097869305e+14 ||r(i)||/||b|| 2.647874940607e-03 >>>> 19 KSP preconditioned resid norm 6.946418330368e+14 true resid norm >>>> 1.094539907094e+14 ||r(i)||/||b|| 2.078910566681e-03 >>>> 20 KSP preconditioned resid norm 5.822189800664e+14 true resid norm >>>> 1.086460879389e+14 ||r(i)||/||b|| 2.063565693502e-03 >>>> 21 KSP preconditioned resid norm 4.765744627590e+14 true resid norm >>>> 7.425312666441e+13 ||r(i)||/||b|| 1.410324179423e-03 >>>> 22 KSP preconditioned resid norm 3.785099184439e+14 true resid norm >>>> 8.229511757789e+13 ||r(i)||/||b|| 1.563069454208e-03 >>>> 23 KSP preconditioned resid norm 3.069654165781e+14 true resid norm >>>> 4.744586000483e+13 ||r(i)||/||b|| 9.011612922478e-04 >>>> 0 KSP preconditioned resid norm 9.591954950201e+15 true resid norm >>>> 5.933293318674e+15 ||r(i)||/||b|| 5.704140552403e-01 >>>> 1 KSP preconditioned resid norm 4.627616930759e+15 true resid norm >>>> 8.120897483902e+14 ||r(i)||/||b|| 7.807256134471e-02 >>>> 2 KSP preconditioned resid norm 3.843607704549e+15 true resid norm >>>> 6.216441009903e+14 ||r(i)||/||b|| 5.976352651335e-02 >>>> 3 KSP preconditioned resid norm 3.802257051246e+15 true resid norm >>>> 6.332758127660e+14 ||r(i)||/||b|| 6.088177425993e-02 >>>> 4 KSP preconditioned resid norm 4.025094330467e+15 true resid norm >>>> 6.488587775849e+14 ||r(i)||/||b|| 6.237988697367e-02 >>>> 5 KSP preconditioned resid norm 4.518903996358e+15 true resid norm >>>> 7.100173385586e+14 ||r(i)||/||b|| 6.825953945399e-02 >>>> 6 KSP preconditioned resid norm 5.566041493723e+15 true resid norm >>>> 8.272295550982e+14 ||r(i)||/||b|| 7.952806979104e-02 >>>> 7 KSP preconditioned resid norm 7.050235529471e+15 true resid norm >>>> 1.259531120704e+15 ||r(i)||/||b|| 1.210886122891e-01 >>>> 8 KSP preconditioned resid norm 7.824426472314e+15 true resid norm >>>> 1.230246429464e+15 ||r(i)||/||b|| 1.182732450740e-01 >>>> 9 KSP preconditioned resid norm 8.110342869221e+15 true resid norm >>>> 1.508507345577e+15 ||r(i)||/||b|| 1.450246509206e-01 >>>> 10 KSP preconditioned resid norm 7.848071091100e+15 true resid norm >>>> 1.244716713436e+15 ||r(i)||/||b|| 1.196643870448e-01 >>>> 11 KSP preconditioned resid norm 6.689698623315e+15 true resid norm >>>> 1.152050117302e+15 ||r(i)||/||b|| 1.107556198481e-01 >>>> 12 KSP preconditioned resid norm 3.836259156428e+15 true resid norm >>>> 5.895445738700e+14 ||r(i)||/||b|| 5.667754703239e-02 >>>> 13 KSP preconditioned resid norm 1.583619332521e+15 true resid norm >>>> 3.090404276347e+14 ||r(i)||/||b|| 2.971048186772e-02 >>>> 14 KSP preconditioned resid norm 1.282447478166e+15 true resid norm >>>> 2.666239295029e+14 ||r(i)||/||b|| 2.563265098882e-02 >>>> 15 KSP preconditioned resid norm 1.151484666969e+15 true resid norm >>>> 2.280596910606e+14 ||r(i)||/||b|| 2.192516806902e-02 >>>> 16 KSP preconditioned resid norm 1.027737067851e+15 true resid norm >>>> 1.605668418840e+14 ||r(i)||/||b|| 1.543655074795e-02 >>>> 17 KSP preconditioned resid norm 8.747586168174e+14 true resid norm >>>> 1.470663481317e+14 ||r(i)||/||b|| 1.413864232250e-02 >>>> 18 KSP preconditioned resid norm 7.765674155007e+14 true resid norm >>>> 1.265290838281e+14 ||r(i)||/||b|| 1.216423391459e-02 >>>> 19 KSP preconditioned resid norm 6.633620818064e+14 true resid norm >>>> 1.039325479930e+14 ||r(i)||/||b|| 9.991851571798e-03 >>>> 20 KSP preconditioned resid norm 5.640296981466e+14 true resid norm >>>> 1.018948133357e+14 ||r(i)||/||b|| 9.795948145668e-03 >>>> 21 KSP preconditioned resid norm 4.751732697839e+14 true resid norm >>>> 7.431313741469e+13 ||r(i)||/||b|| 7.144305159654e-03 >>>> 22 KSP preconditioned resid norm 3.865078030089e+14 true resid norm >>>> 7.686807394274e+13 ||r(i)||/||b|| 7.389931260973e-03 >>>> 23 KSP preconditioned resid norm 3.183297004119e+14 true resid norm >>>> 4.953673129366e+13 ||r(i)||/||b|| 4.762354777175e-03 >>>> 24 KSP preconditioned resid norm 2.612111917568e+14 true resid norm >>>> 5.971714747402e+13 ||r(i)||/||b|| 5.741078087414e-03 >>>> 25 KSP preconditioned resid norm 2.296313138344e+14 true resid norm >>>> 3.527848689641e+13 ||r(i)||/||b|| 3.391597834880e-03 >>>> 26 KSP preconditioned resid norm 1.981337219705e+14 true resid norm >>>> 5.164126794406e+13 ||r(i)||/||b|| 4.964680403211e-03 >>>> 27 KSP preconditioned resid norm 1.751049134665e+14 true resid norm >>>> 2.675806111216e+13 ||r(i)||/||b|| 2.572462430151e-03 >>>> 28 KSP preconditioned resid norm 1.533798667157e+14 true resid norm >>>> 3.995390488198e+13 ||r(i)||/||b|| 3.841082461690e-03 >>>> 29 KSP preconditioned resid norm 1.350471089065e+14 true resid norm >>>> 2.063214947275e+13 ||r(i)||/||b|| 1.983530463939e-03 >>>> 30 KSP preconditioned resid norm 1.221880384902e+14 true resid norm >>>> 2.772903253208e+13 ||r(i)||/||b|| 2.665809533591e-03 >>>> 31 KSP preconditioned resid norm 1.099123503540e+14 true resid norm >>>> 1.682484274214e+13 ||r(i)||/||b|| 1.617504185596e-03 >>>> 32 KSP preconditioned resid norm 1.022845787224e+14 true resid norm >>>> 1.853354426028e+13 ||r(i)||/||b|| 1.781775073585e-03 >>>> 33 KSP preconditioned resid norm 9.416346598487e+13 true resid norm >>>> 1.438140346990e+13 ||r(i)||/||b|| 1.382597190584e-03 >>>> 34 KSP preconditioned resid norm 8.782252733286e+13 true resid norm >>>> 1.465353724876e+13 ||r(i)||/||b|| 1.408759546637e-03 >>>> 35 KSP preconditioned resid norm 8.286435025538e+13 true resid norm >>>> 1.264742128603e+13 ||r(i)||/||b|| 1.215895873779e-03 >>>> 36 KSP preconditioned resid norm 7.744963280138e+13 true resid norm >>>> 1.290903292754e+13 ||r(i)||/||b|| 1.241046654183e-03 >>>> 37 KSP preconditioned resid norm 7.340074137876e+13 true resid norm >>>> 1.120214736846e+13 ||r(i)||/||b|| 1.076950348592e-03 >>>> 38 KSP preconditioned resid norm 6.860608702874e+13 true resid norm >>>> 1.142914172047e+13 ||r(i)||/||b|| 1.098773097257e-03 >>>> 39 KSP preconditioned resid norm 6.401516623396e+13 true resid norm >>>> 9.766911914727e+12 ||r(i)||/||b|| 9.389698997216e-04 >>>> 40 KSP preconditioned resid norm 5.980495528832e+13 true resid norm >>>> 9.707430083091e+12 ||r(i)||/||b|| 9.332514443927e-04 >>>> 41 KSP preconditioned resid norm 5.502664117143e+13 true resid norm >>>> 8.404607061129e+12 ||r(i)||/||b|| 8.080008418514e-04 >>>> 0 KSP preconditioned resid norm 2.200733964807e+16 true resid norm >>>> 1.374508262416e+16 ||r(i)||/||b|| 1.042223674371e-01 >>>> 1 KSP preconditioned resid norm 1.033612993855e+16 true resid norm >>>> 1.767505005693e+15 ||r(i)||/||b|| 1.340214251070e-02 >>>> 2 KSP preconditioned resid norm 8.198008013445e+15 true resid norm >>>> 1.328272543525e+15 ||r(i)||/||b|| 1.007165346861e-02 >>>> 3 KSP preconditioned resid norm 7.723889011566e+15 true resid norm >>>> 1.289831076731e+15 ||r(i)||/||b|| 9.780170267923e-03 >>>> 4 KSP preconditioned resid norm 7.785469249486e+15 true resid norm >>>> 1.238697760343e+15 ||r(i)||/||b|| 9.392450860579e-03 >>>> 5 KSP preconditioned resid norm 8.250177532689e+15 true resid norm >>>> 1.288999707195e+15 ||r(i)||/||b|| 9.773866391570e-03 >>>> 6 KSP preconditioned resid norm 9.471871112821e+15 true resid norm >>>> 1.409248720218e+15 ||r(i)||/||b|| 1.068565696875e-02 >>>> 7 KSP preconditioned resid norm 1.106232045691e+16 true resid norm >>>> 1.993194297637e+15 ||r(i)||/||b|| 1.511343613874e-02 >>>> 8 KSP preconditioned resid norm 1.131471113452e+16 true resid norm >>>> 1.768387012694e+15 ||r(i)||/||b|| 1.340883034665e-02 >>>> 9 KSP preconditioned resid norm 1.055003108779e+16 true resid norm >>>> 1.935924228946e+15 ||r(i)||/||b|| 1.467918468275e-02 >>>> 10 KSP preconditioned resid norm 8.569038225337e+15 true resid norm >>>> 1.316932619063e+15 ||r(i)||/||b|| 9.985668261662e-03 >>>> 11 KSP preconditioned resid norm 4.806783660704e+15 true resid norm >>>> 7.346602562794e+14 ||r(i)||/||b|| 5.570576275535e-03 >>>> 12 KSP preconditioned resid norm 6.311127886872e+15 true resid norm >>>> 1.031465284608e+15 ||r(i)||/||b|| 7.821106415332e-03 >>>> 13 KSP preconditioned resid norm 9.706706695861e+14 true resid norm >>>> 2.189061895566e+14 ||r(i)||/||b|| 1.659860616781e-03 >>>> 14 KSP preconditioned resid norm 6.844878378158e+14 true resid norm >>>> 1.339168617782e+14 ||r(i)||/||b|| 1.015427317239e-03 >>>> 0 KSP preconditioned resid norm 1.928571496492e+16 true resid norm >>>> 1.203387628158e+16 ||r(i)||/||b|| 4.946666172277e-01 >>>> 1 KSP preconditioned resid norm 9.232456268915e+15 true resid norm >>>> 1.583223656595e+15 ||r(i)||/||b|| 6.508026775390e-02 >>>> 2 KSP preconditioned resid norm 7.541947576336e+15 true resid norm >>>> 1.216650042172e+15 ||r(i)||/||b|| 5.001182882627e-02 >>>> 3 KSP preconditioned resid norm 7.340367763150e+15 true resid norm >>>> 1.219887358936e+15 ||r(i)||/||b|| 5.014490253381e-02 >>>> 4 KSP preconditioned resid norm 7.655525114233e+15 true resid norm >>>> 1.212298323935e+15 ||r(i)||/||b|| 4.983294633744e-02 >>>> 5 KSP preconditioned resid norm 8.414914513850e+15 true resid norm >>>> 1.311466433986e+15 ||r(i)||/||b|| 5.390936796484e-02 >>>> 6 KSP preconditioned resid norm 1.008643582787e+16 true resid norm >>>> 1.495807999597e+15 ||r(i)||/||b|| 6.148694451139e-02 >>>> 7 KSP preconditioned resid norm 1.235608743666e+16 true resid norm >>>> 2.203234082057e+15 ||r(i)||/||b|| 9.056652443733e-02 >>>> 8 KSP preconditioned resid norm 1.311751365507e+16 true resid norm >>>> 2.061480991281e+15 ||r(i)||/||b|| 8.473959716509e-02 >>>> 9 KSP preconditioned resid norm 1.289621791383e+16 true resid norm >>>> 2.355792563000e+15 ||r(i)||/||b|| 9.683761996231e-02 >>>> 10 KSP preconditioned resid norm 1.151686987354e+16 true resid norm >>>> 1.821530468335e+15 ||r(i)||/||b|| 7.487614911976e-02 >>>> 11 KSP preconditioned resid norm 8.379707704664e+15 true resid norm >>>> 1.435040037498e+15 ||r(i)||/||b|| 5.898900606299e-02 >>>> 12 KSP preconditioned resid norm 3.557464484347e+15 true resid norm >>>> 5.430083136124e+14 ||r(i)||/||b|| 2.232099444401e-02 >>>> 13 KSP preconditioned resid norm 1.204545626126e+15 true resid norm >>>> 2.194281091480e+14 ||r(i)||/||b|| 9.019850124522e-03 >>>> 14 KSP preconditioned resid norm 1.005673666245e+15 true resid norm >>>> 1.883222474780e+14 ||r(i)||/||b|| 7.741207149622e-03 >>>> 15 KSP preconditioned resid norm 9.379382960505e+14 true resid norm >>>> 1.480571711312e+14 ||r(i)||/||b|| 6.086063898783e-03 >>>> 16 KSP preconditioned resid norm 8.825746357103e+14 true resid norm >>>> 2.025244331207e+14 ||r(i)||/||b|| 8.325004669617e-03 >>>> 17 KSP preconditioned resid norm 7.524485997458e+14 true resid norm >>>> 1.181787653170e+14 ||r(i)||/||b|| 4.857876938371e-03 >>>> 18 KSP preconditioned resid norm 6.768878630597e+14 true resid norm >>>> 1.383948785570e+14 ||r(i)||/||b|| 5.688884015051e-03 >>>> 19 KSP preconditioned resid norm 6.018014871695e+14 true resid norm >>>> 9.381957375927e+13 ||r(i)||/||b|| 3.856563761775e-03 >>>> 20 KSP preconditioned resid norm 5.246326908653e+14 true resid norm >>>> 1.170575496120e+14 ||r(i)||/||b|| 4.811788050053e-03 >>>> 21 KSP preconditioned resid norm 4.606732721936e+14 true resid norm >>>> 7.159556750064e+13 ||r(i)||/||b|| 2.943019884477e-03 >>>> 22 KSP preconditioned resid norm 3.919505243136e+14 true resid norm >>>> 9.177422576855e+13 ||r(i)||/||b|| 3.772487330545e-03 >>>> 23 KSP preconditioned resid norm 3.382362447667e+14 true resid norm >>>> 5.210255290014e+13 ||r(i)||/||b|| 2.141736626583e-03 >>>> 24 KSP preconditioned resid norm 2.828412329842e+14 true resid norm >>>> 7.512251290427e+13 ||r(i)||/||b|| 3.087999117364e-03 >>>> 25 KSP preconditioned resid norm 2.453088437412e+14 true resid norm >>>> 3.770320821115e+13 ||r(i)||/||b|| 1.549834652446e-03 >>>> 26 KSP preconditioned resid norm 2.155681340535e+14 true resid norm >>>> 5.419527743350e+13 ||r(i)||/||b|| 2.227760526238e-03 >>>> 27 KSP preconditioned resid norm 1.897133167426e+14 true resid norm >>>> 2.950727994670e+13 ||r(i)||/||b|| 1.212931395777e-03 >>>> 28 KSP preconditioned resid norm 1.747505144272e+14 true resid norm >>>> 2.784171913979e+13 ||r(i)||/||b|| 1.144466562762e-03 >>>> 29 KSP preconditioned resid norm 1.591031918648e+14 true resid norm >>>> 2.517709500033e+13 ||r(i)||/||b|| 1.034934058155e-03 >>>> 30 KSP preconditioned resid norm 1.457115463937e+14 true resid norm >>>> 2.260424017892e+13 ||r(i)||/||b|| 9.291738391412e-04 >>>> 31 KSP preconditioned resid norm 1.296458013341e+14 true resid norm >>>> 2.025545529149e+13 ||r(i)||/||b|| 8.326242779131e-04 >>>> 0 KSP preconditioned resid norm 6.130010037612e+16 true resid norm >>>> 4.019535297216e+16 ||r(i)||/||b|| 5.748505993058e-02 >>>> 1 KSP preconditioned resid norm 2.383231272358e+16 true resid norm >>>> 4.333028202730e+15 ||r(i)||/||b|| 6.196845343971e-03 >>>> 2 KSP preconditioned resid norm 1.380314308485e+16 true resid norm >>>> 2.902938638700e+15 ||r(i)||/||b|| 4.151614285761e-03 >>>> 3 KSP preconditioned resid norm 9.355422664357e+15 true resid norm >>>> 1.914374541766e+15 ||r(i)||/||b|| 2.737827313999e-03 >>>> 4 KSP preconditioned resid norm 6.905389450100e+15 true resid norm >>>> 1.160495113546e+15 ||r(i)||/||b|| 1.659672728773e-03 >>>> 5 KSP preconditioned resid norm 5.388850197186e+15 true resid norm >>>> 9.343171629150e+14 ||r(i)||/||b|| 1.336206156506e-03 >>>> 6 KSP preconditioned resid norm 4.456941607624e+15 true resid norm >>>> 7.443205303349e+14 ||r(i)||/||b|| 1.064484004494e-03 >>>> 7 KSP preconditioned resid norm 3.604897946036e+15 true resid norm >>>> 7.177354673946e+14 ||r(i)||/||b|| 1.026463591104e-03 >>>> 8 KSP preconditioned resid norm 2.616552593607e+15 true resid norm >>>> 4.566784798486e+14 ||r(i)||/||b|| 6.531150454453e-04 >>>> 9 KSP preconditioned resid norm 1.730753896325e+15 true resid norm >>>> 3.416577992536e+14 ||r(i)||/||b|| 4.886191465826e-04 >>>> 10 KSP preconditioned resid norm 9.605913237294e+14 true resid norm >>>> 1.715828091570e+14 ||r(i)||/||b|| 2.453877709266e-04 >>>> 11 KSP preconditioned resid norm 3.885469473062e+14 true resid norm >>>> 7.526615943221e+13 ||r(i)||/||b|| 1.076412909896e-04 >>>> 12 KSP preconditioned resid norm 1.085402136107e+14 true resid norm >>>> 1.877490962768e+13 ||r(i)||/||b|| 2.685078560912e-05 >>>> 13 KSP preconditioned resid norm 5.521535103061e+13 true resid norm >>>> 1.482831672696e+13 ||r(i)||/||b|| 2.120659759623e-05 >>>> 14 KSP preconditioned resid norm 4.327778995114e+13 true resid norm >>>> 6.949075963503e+12 ||r(i)||/||b|| 9.938164954062e-06 >>>> 0 KSP preconditioned resid norm 2.469229041475e+16 true resid norm >>>> 1.581397714280e+16 ||r(i)||/||b|| 1.421322438999e-01 >>>> 1 KSP preconditioned resid norm 1.094866313351e+16 true resid norm >>>> 1.827538507817e+15 ||r(i)||/||b|| 1.642547896610e-02 >>>> 2 KSP preconditioned resid norm 7.902647446500e+15 true resid norm >>>> 1.331972422409e+15 ||r(i)||/||b|| 1.197144952849e-02 >>>> 3 KSP preconditioned resid norm 6.757114899989e+15 true resid norm >>>> 1.177404752006e+15 ||r(i)||/||b|| 1.058223227906e-02 >>>> 4 KSP preconditioned resid norm 6.236820681061e+15 true resid norm >>>> 9.858934558297e+14 ||r(i)||/||b|| 8.860974558006e-03 >>>> 5 KSP preconditioned resid norm 6.056208230839e+15 true resid norm >>>> 9.498735328449e+14 ||r(i)||/||b|| 8.537236106085e-03 >>>> 6 KSP preconditioned resid norm 6.281150442401e+15 true resid norm >>>> 9.439017368432e+14 ||r(i)||/||b|| 8.483563032058e-03 >>>> 7 KSP preconditioned resid norm 6.511637444792e+15 true resid norm >>>> 1.175605282966e+15 ||r(i)||/||b|| 1.056605908176e-02 >>>> 8 KSP preconditioned resid norm 5.897282453446e+15 true resid norm >>>> 9.358326231383e+14 ||r(i)||/||b|| 8.411039768188e-03 >>>> 9 KSP preconditioned resid norm 4.858643646559e+15 true resid norm >>>> 9.074077132105e+14 ||r(i)||/||b|| 8.155563476918e-03 >>>> 10 KSP preconditioned resid norm 3.431227043388e+15 true resid norm >>>> 5.523077341519e+14 ||r(i)||/||b|| 4.964009804073e-03 >>>> 11 KSP preconditioned resid norm 1.790602151882e+15 true resid norm >>>> 3.233768234561e+14 ||r(i)||/||b|| 2.906433538380e-03 >>>> 12 KSP preconditioned resid norm 6.061523925825e+14 true resid norm >>>> 9.361733195934e+13 ||r(i)||/||b|| 8.414101866433e-04 >>>> 13 KSP preconditioned resid norm 3.669173256365e+14 true resid norm >>>> 7.433445448820e+13 ||r(i)||/||b|| 6.681002963438e-04 >>>> 14 KSP preconditioned resid norm 1.034315631221e+15 true resid norm >>>> 1.749781208578e+14 ||r(i)||/||b|| 1.572661495987e-03 >>>> 15 KSP preconditioned resid norm 1.340579302998e+14 true resid norm >>>> 2.477734238014e+13 ||r(i)||/||b|| 2.226928266408e-04 >>>> 16 KSP preconditioned resid norm 1.124182190968e+14 true resid norm >>>> 1.917839484668e+13 ||r(i)||/||b|| 1.723708254629e-04 >>>> 17 KSP preconditioned resid norm 9.394060483803e+13 true resid norm >>>> 1.682558933028e+13 ||r(i)||/||b|| 1.512243722661e-04 >>>> 18 KSP preconditioned resid norm 9.034672550576e+13 true resid norm >>>> 1.892870471110e+13 ||r(i)||/||b|| 1.701266702495e-04 >>>> 19 KSP preconditioned resid norm 8.058964108831e+13 true resid norm >>>> 1.275370569496e+13 ||r(i)||/||b|| 1.146272561351e-04 >>>> 20 KSP preconditioned resid norm 6.303390314758e+13 true resid norm >>>> 1.900069985546e+13 ||r(i)||/||b|| 1.707737453860e-04 >>>> 21 KSP preconditioned resid norm 4.601826493253e+13 true resid norm >>>> 6.960266318594e+12 ||r(i)||/||b|| 6.255720879505e-05 >>>> 22 KSP preconditioned resid norm 2.822787946131e+13 true resid norm >>>> 1.013300639920e+13 ||r(i)||/||b|| 9.107303772884e-05 >>>> 23 KSP preconditioned resid norm 1.858182164082e+13 true resid norm >>>> 2.785683110677e+12 ||r(i)||/||b|| 2.503705347106e-05 >>>> 24 KSP preconditioned resid norm 9.506925688468e+12 true resid norm >>>> 4.268960894748e+12 ||r(i)||/||b|| 3.836839939834e-05 >>>> 25 KSP preconditioned resid norm 5.845171242291e+12 true resid norm >>>> 8.923161711832e+11 ||r(i)||/||b|| 8.019924307031e-06 >>>> 0 SNES Function norm 0.000000000000e+00 >>>> 5 TS dt 7e-12 time 2e-11 >>>> TS Object: 1 MPI processes >>>> type: theta >>>> maximum steps=10 >>>> maximum time=2e-11 >>>> total number of nonlinear solver iterations=0 >>>> total number of nonlinear solve failures=0 >>>> total number of linear solver iterations=0 >>>> total number of rejected steps=5 >>>> Theta=1 >>>> Extrapolation=no >>>> SNES Object: 1 MPI processes >>>> type: newtonls >>>> maximum iterations=50, maximum function evaluations=10000 >>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>> total number of linear solver iterations=0 >>>> total number of function evaluations=1 >>>> SNESLineSearch Object: 1 MPI processes >>>> type: bt >>>> interpolation: cubic >>>> alpha=1.000000e-04 >>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>> lambda=1.000000e-08 >>>> maximum iterations=40 >>>> KSP Object: 1 MPI processes >>>> type: gmres >>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>> Orthogonalization with no iterative refinement >>>> GMRES: happy breakdown tolerance 1e-30 >>>> maximum iterations=10000, initial guess is zero >>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>> left preconditioning >>>> using DEFAULT norm type for convergence test >>>> PC Object: 1 MPI processes >>>> type: ilu >>>> PC has not been set up so information may be incomplete >>>> ILU: out-of-place factorization >>>> 0 levels of fill >>>> tolerance for zero pivot 2.22045e-14 >>>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>>> matrix ordering: natural >>>> linear system matrix = precond matrix: >>>> Mat Object: 1 MPI processes >>>> type: seqaij >>>> rows=20904, cols=20904, bs=4 >>>> total: nonzeros=410816, allocated nonzeros=410816 >>>> total number of mallocs used during MatSetValues calls =0 >>>> using I-node routines: found 5226 nodes, limit used is 5 >>>> >>>> On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley >>>> wrote: >>>> >>>>> On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone >>>>> wrote: >>>>> >>>>>> Hi Jed and Matt, >>>>>> >>>>>> Thank you very much for your replies. >>>>>> >>>>>> This time I copied the whole output from my screen, from compile to >>>>>> execution. This time I put the TSView() routine just after the TSSolve() >>>>>> routine to make the tsview output. And I intentionally to do only 5 time >>>>>> steps to make the test. Thank you in advance! >>>>>> >>>>> >>>>> You have to make the command line arguments work. Quit calling >>>>> TSView() in the code. We need many more options >>>>> for debugging. First track down why you are not processing options. Do >>>>> the examples take options for you? >>>>> >>>>> Once that works, run with -ts_view -ts_monitor -snes_monitor >>>>> -ksp_monitor_true_residual >>>>> >>>>> Matt >>>>> >>>>> >>>>>> host206-47:streamer_Implicit_SG GPL$ make pStreamer >>>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx >>>>>> -o pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>>>>> -Wno-unknown-pragmas -g -O0 -fPIC >>>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >>>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >>>>>> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >>>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >>>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first >>>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >>>>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >>>>>> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >>>>>> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >>>>>> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >>>>>> -lpthread -lssl -lcrypto >>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >>>>>> -lmpichf90 -lgfortran >>>>>> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >>>>>> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >>>>>> -lclang_rt.osx -lmpichcxx -lc++ >>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>> -lclang_rt.osx >>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >>>>>> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>> -lclang_rt.osx -ldl >>>>>> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >>>>>> >>>>>> time=0.000000 step=0 >>>>>> dt = 0.001000 >>>>>> >>>>>> time=0.002000 step=1 >>>>>> dt = 0.002000 >>>>>> >>>>>> time=0.005000 step=2 >>>>>> dt = 0.003000 >>>>>> >>>>>> time=0.009000 step=3 >>>>>> dt = 0.004000 >>>>>> >>>>>> time=0.014000 step=4 >>>>>> dt = 0.005000 >>>>>> >>>>>> time=0.020000 step=5 >>>>>> dt = 0.006000 >>>>>> >>>>>> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >>>>>> TS Object: 1 MPI processes >>>>>> type: theta >>>>>> maximum steps=10 >>>>>> maximum time=2e-11 >>>>>> total number of nonlinear solver iterations=0 >>>>>> total number of nonlinear solve failures=0 >>>>>> total number of linear solver iterations=0 >>>>>> total number of rejected steps=5 >>>>>> Theta=1 >>>>>> Extrapolation=no >>>>>> SNES Object: 1 MPI processes >>>>>> type: newtonls >>>>>> maximum iterations=50, maximum function evaluations=10000 >>>>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>>> total number of linear solver iterations=0 >>>>>> total number of function evaluations=1 >>>>>> SNESLineSearch Object: 1 MPI processes >>>>>> type: bt >>>>>> interpolation: cubic >>>>>> alpha=1.000000e-04 >>>>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>>>> lambda=1.000000e-08 >>>>>> maximum iterations=40 >>>>>> KSP Object: 1 MPI processes >>>>>> type: gmres >>>>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>>>> Orthogonalization with no iterative refinement >>>>>> GMRES: happy breakdown tolerance 1e-30 >>>>>> maximum iterations=10000, initial guess is zero >>>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>>>> left preconditioning >>>>>> using DEFAULT norm type for convergence test >>>>>> PC Object: 1 MPI processes >>>>>> type: ilu >>>>>> PC has not been set up so information may be incomplete >>>>>> ILU: out-of-place factorization >>>>>> 0 levels of fill >>>>>> tolerance for zero pivot 2.22045e-14 >>>>>> using diagonal shift on blocks to prevent zero pivot >>>>>> [INBLOCKS] >>>>>> matrix ordering: natural >>>>>> linear system matrix = precond matrix: >>>>>> Mat Object: 1 MPI processes >>>>>> type: seqaij >>>>>> rows=20904, cols=20904, bs=4 >>>>>> total: nonzeros=410816, allocated nonzeros=410816 >>>>>> total number of mallocs used during MatSetValues calls =0 >>>>>> using I-node routines: found 5226 nodes, limit used is 5 >>>>>> >>>>>> >>>>>> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >>>>>> >>>>>>> Sharp Stone writes: >>>>>>> >>>>>>> > Hi Matt, >>>>>>> > >>>>>>> > Thank you very much for your reply. >>>>>>> > >>>>>>> > The ts_view output is attached below. I found my code results say >>>>>>> "SNES has >>>>>>> > not been set up", and PC "has not been set up". Does this cause the >>>>>>> > problem? If so I do not see example ex17 explicitly set up the >>>>>>> snes object? >>>>>>> >>>>>>> Those objects are set up on the first time step (inside TSSolve). >>>>>>> >>>>>>> > PS: I don't know why the -ts_view option does not work in my >>>>>>> command line, >>>>>>> >>>>>>> Are you sure it doesn't output after the TSSolve? The example does. >>>>>>> >>>>>>> > so I use TSView() routine just before the TSSolve() routine. >>>>>>> >>>>>>> You should call it *after* the solve so we can see what happened >>>>>>> during >>>>>>> the solve. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> >>>>>> Feng >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> >>>> Feng >>>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their >>> experiments is infinitely more interesting than any results to which their >>> experiments lead. >>> -- Norbert Wiener >>> >> >> >> >> -- >> Best regards, >> >> Feng >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -- Best regards, Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 28 09:25:18 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 09:25:18 -0500 Subject: [petsc-users] TSSetIJacobian is never called In-Reply-To: References: <8738a92vvz.fsf@jedbrown.org> Message-ID: On Tue, Oct 28, 2014 at 9:22 AM, Sharp Stone wrote: > Hi Matt and Jed, > > Thank you for your replies. Now I see Petsc will calculate functions' > residue in SNES before form the Jacobian. In order to get a initial > non-zero residues of the functions, we cannot initialize some of the > discretized spatial nodes of the functions to zero, am I right? In my > problem, initially only a small region will be initialized with some > values, while other spatial nodes remain zero. I don't know how the SNES > calculate the initial residue? If it is a sum residue on all discretized > nodes, then I should not get a zero residue, or at least, I should get a > small residue which is not zero. But I still get the exact zero residue. So > I wonder how the SNES calculate the residue? > You are calculating the residual in your callback function. Print out the vector F which you pass back. Matt > Thanks in advance! > > TS Object: 1 MPI processes >>>>> type: theta >>>>> maximum steps=10 >>>>> maximum time=2e-11 >>>>> total number of nonlinear solver iterations=0 >>>>> total number of nonlinear solve failures=0 >>>>> total number of linear solver iterations=0 >>>>> total number of rejected steps=5 >>>>> Theta=1 >>>>> Extrapolation=no >>>>> SNES Object: 1 MPI processes >>>>> type: newtonls >>>>> maximum iterations=50, maximum function evaluations=10000 >>>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>> total number of linear solver iterations=0 >>>>> total number of function evaluations=1 >>>>> SNESLineSearch Object: 1 MPI processes >>>>> type: bt >>>>> interpolation: cubic >>>>> alpha=1.000000e-04 >>>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>>> lambda=1.000000e-08 >>>>> maximum iterations=40 >>>>> KSP Object: 1 MPI processes >>>>> type: gmres >>>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>>> Orthogonalization with no iterative refinement >>>>> GMRES: happy breakdown tolerance 1e-30 >>>>> maximum iterations=10000, initial guess is zero >>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>>> left preconditioning >>>>> using DEFAULT norm type for convergence test >>>>> PC Object: 1 MPI processes >>>>> type: ilu >>>>> PC has not been set up so information may be incomplete >>>>> ILU: out-of-place factorization >>>>> 0 levels of fill >>>>> tolerance for zero pivot 2.22045e-14 >>>>> using diagonal shift on blocks to prevent zero pivot [INBLOCKS] >>>>> matrix ordering: natural >>>>> linear system matrix = precond matrix: >>>>> Mat Object: 1 MPI processes >>>>> type: seqaij >>>>> rows=20904, cols=20904, bs=4 >>>>> total: nonzeros=410816, allocated nonzeros=410816 >>>>> total number of mallocs used during MatSetValues calls =0 >>>>> using I-node routines: found 5226 nodes, limit used is 5 >>>>> >>>>> On Mon, Oct 27, 2014 at 3:09 PM, Matthew Knepley >>>>> wrote: >>>>> >>>>>> On Mon, Oct 27, 2014 at 2:05 PM, Sharp Stone >>>>>> wrote: >>>>>> >>>>>>> Hi Jed and Matt, >>>>>>> >>>>>>> Thank you very much for your replies. >>>>>>> >>>>>>> This time I copied the whole output from my screen, from compile to >>>>>>> execution. This time I put the TSView() routine just after the TSSolve() >>>>>>> routine to make the tsview output. And I intentionally to do only 5 time >>>>>>> steps to make the test. Thank you in advance! >>>>>>> >>>>>> >>>>>> You have to make the command line arguments work. Quit calling >>>>>> TSView() in the code. We need many more options >>>>>> for debugging. First track down why you are not processing options. >>>>>> Do the examples take options for you? >>>>>> >>>>>> Once that works, run with -ts_view -ts_monitor -snes_monitor >>>>>> -ksp_monitor_true_residual >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> host206-47:streamer_Implicit_SG GPL$ make pStreamer >>>>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicxx >>>>>>> -o pStreamer.o -c -Wall -Wwrite-strings -Wno-strict-aliasing >>>>>>> -Wno-unknown-pragmas -g -O0 -fPIC >>>>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/include >>>>>>> -I/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/include >>>>>>> -I/opt/X11/include -Wno-unused `pwd`/pStreamer.cc >>>>>>> /Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/bin/mpicc >>>>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first >>>>>>> -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >>>>>>> -Wl,-commons,use_dylibs -Wl,-search_paths_first -fPIC -Wall >>>>>>> -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -O0 >>>>>>> -Wno-unused -o pStreamer flux.o field.o glob_streamer.o input.o >>>>>>> lookuptable.o nrutil.o pStreamer.o someutil.o photoionization.o heating.o >>>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib >>>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -lpetsc >>>>>>> -lsuperlu_4.3 -llapack -lblas -lparmetis -lmetis -L/opt/X11/lib -lX11 >>>>>>> -lpthread -lssl -lcrypto >>>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/5.1/lib/darwin >>>>>>> -lmpichf90 -lgfortran >>>>>>> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin13/4.8.2 >>>>>>> -L/usr/local/gfortran/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm >>>>>>> -lclang_rt.osx -lmpichcxx -lc++ >>>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>>> -lclang_rt.osx >>>>>>> -L/Users/GPL/local/PETSc/petsc-3.5.2/arch-darwin-cxx-debug/lib -ldl >>>>>>> -lpmpich -lmpich -lopa -lmpl -lpthread -lSystem >>>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>>> -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1/lib/darwin >>>>>>> -lclang_rt.osx -ldl >>>>>>> host206-47:streamer_Implicit_SG GPL$ mpiexec -n 1 ./pStreamer >>>>>>> >>>>>>> time=0.000000 step=0 >>>>>>> dt = 0.001000 >>>>>>> >>>>>>> time=0.002000 step=1 >>>>>>> dt = 0.002000 >>>>>>> >>>>>>> time=0.005000 step=2 >>>>>>> dt = 0.003000 >>>>>>> >>>>>>> time=0.009000 step=3 >>>>>>> dt = 0.004000 >>>>>>> >>>>>>> time=0.014000 step=4 >>>>>>> dt = 0.005000 >>>>>>> >>>>>>> time=0.020000 step=5 >>>>>>> dt = 0.006000 >>>>>>> >>>>>>> Step time elapse = 0.854144 s Total time elapse = 5.276643 s >>>>>>> TS Object: 1 MPI processes >>>>>>> type: theta >>>>>>> maximum steps=10 >>>>>>> maximum time=2e-11 >>>>>>> total number of nonlinear solver iterations=0 >>>>>>> total number of nonlinear solve failures=0 >>>>>>> total number of linear solver iterations=0 >>>>>>> total number of rejected steps=5 >>>>>>> Theta=1 >>>>>>> Extrapolation=no >>>>>>> SNES Object: 1 MPI processes >>>>>>> type: newtonls >>>>>>> maximum iterations=50, maximum function evaluations=10000 >>>>>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 >>>>>>> total number of linear solver iterations=0 >>>>>>> total number of function evaluations=1 >>>>>>> SNESLineSearch Object: 1 MPI processes >>>>>>> type: bt >>>>>>> interpolation: cubic >>>>>>> alpha=1.000000e-04 >>>>>>> maxstep=1.000000e+08, minlambda=1.000000e-12 >>>>>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, >>>>>>> lambda=1.000000e-08 >>>>>>> maximum iterations=40 >>>>>>> KSP Object: 1 MPI processes >>>>>>> type: gmres >>>>>>> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt >>>>>>> Orthogonalization with no iterative refinement >>>>>>> GMRES: happy breakdown tolerance 1e-30 >>>>>>> maximum iterations=10000, initial guess is zero >>>>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000 >>>>>>> left preconditioning >>>>>>> using DEFAULT norm type for convergence test >>>>>>> PC Object: 1 MPI processes >>>>>>> type: ilu >>>>>>> PC has not been set up so information may be incomplete >>>>>>> ILU: out-of-place factorization >>>>>>> 0 levels of fill >>>>>>> tolerance for zero pivot 2.22045e-14 >>>>>>> using diagonal shift on blocks to prevent zero pivot >>>>>>> [INBLOCKS] >>>>>>> matrix ordering: natural >>>>>>> linear system matrix = precond matrix: >>>>>>> Mat Object: 1 MPI processes >>>>>>> type: seqaij >>>>>>> rows=20904, cols=20904, bs=4 >>>>>>> total: nonzeros=410816, allocated nonzeros=410816 >>>>>>> total number of mallocs used during MatSetValues calls =0 >>>>>>> using I-node routines: found 5226 nodes, limit used is 5 >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 27, 2014 at 2:46 PM, Jed Brown wrote: >>>>>>> >>>>>>>> Sharp Stone writes: >>>>>>>> >>>>>>>> > Hi Matt, >>>>>>>> > >>>>>>>> > Thank you very much for your reply. >>>>>>>> > >>>>>>>> > The ts_view output is attached below. I found my code results say >>>>>>>> "SNES has >>>>>>>> > not been set up", and PC "has not been set up". Does this cause >>>>>>>> the >>>>>>>> > problem? If so I do not see example ex17 explicitly set up the >>>>>>>> snes object? >>>>>>>> >>>>>>>> Those objects are set up on the first time step (inside TSSolve). >>>>>>>> >>>>>>>> > PS: I don't know why the -ts_view option does not work in my >>>>>>>> command line, >>>>>>>> >>>>>>>> Are you sure it doesn't output after the TSSolve? The example does. >>>>>>>> >>>>>>>> > so I use TSView() routine just before the TSSolve() routine. >>>>>>>> >>>>>>>> You should call it *after* the solve so we can see what happened >>>>>>>> during >>>>>>>> the solve. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> >>>>>>> Feng >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> >>>>> Feng >>>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Feng >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > > > -- > Best regards, > > Feng > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From salazardetroya at gmail.com Tue Oct 28 09:48:36 2014 From: salazardetroya at gmail.com (Miguel Angel Salazar de Troya) Date: Tue, 28 Oct 2014 09:48:36 -0500 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: References: <87wq7vrjg5.fsf@jedbrown.org> <87lhoam3ta.fsf@jedbrown.org> Message-ID: Sorry to be pushy, but could anyone help me on this? Thanks Miguel On Thu, Oct 23, 2014 at 10:46 AM, Miguel Angel Salazar de Troya < salazardetroya at gmail.com> wrote: > I decided to implement the continuous adjoint because it is clearer to me > how to do it and the gradient will converge anyways. I was trying to > interpolate the solution, but in the adjoint analysis I need to do it once > the simulation has finished. I have saved all the solutions for each time > step. My idea was to reset the TS at the time step immediately previous to > the time step in which I want to interpolate to: > > interpolate_timestep : where I want to interpolate to > previous_timestep : last time step in our time step history which is > smaller that the interpolate_timestep > Current_Sol : Solution at the time step previous_timestep > > TSSetTime(ts,previous_timestep); > TSSetSolution(ts,Current_Sol); > TSSetRetainStages(ts,PETSC_TRUE); > > TSInterpolate(ts,interpolate_timestep,X); > > Vector X should have a close value to Current_Sol, but it's nowhere close. > I also compare it to the solution of the next time step after > previous_timestep (therefore interpolate_timestep is between these guys) > and it's not close either. I've read that TSInterpolate() has to be > extended to support continuous adjoints. > > Thanks > Miguel > > On Tue, Oct 21, 2014 at 6:04 PM, Miguel Angel Salazar de Troya < > salazardetroya at gmail.com> wrote: > >> That might be a reasonable argument, but I'm not sure. This is one of the >> papers that explains it. I'm re-reading to see if I skipped any details: >> >> http://www.sciencedirect.com/science/article/pii/S0377042709006062 >> >> Miguel >> >> On Mon, Oct 20, 2014 at 11:42 PM, Jed Brown wrote: >> >>> Miguel Angel Salazar de Troya writes: >>> >>> > Thanks for your response >>> > >>> > I'm struggling with this problem because the literature is not clear >>> for me >>> > on how to calculate the discrete adjoint with adaptive time stepping >>> > algorithms. They cover the details when automatic differentiation >>> tools are >>> > used. They mention that because the time step depend on the solution, >>> it >>> > also depends on the parameter. Hence, there are terms that represent >>> the >>> > derivative of the time step w.r.t. the parameters. What it is >>> confusing is >>> > that they mention these terms must be removed. I don't understand >>> this. I'm >>> > planning to hard-code the discrete adjoint problem (and use the TS if >>> > possible). >>> >>> Are they suggesting that the time step sizes for a given run should be >>> frozen (at least locally) so that you have consistent gradients for a >>> while? >>> >> >> >> >> -- >> *Miguel Angel Salazar de Troya* >> Graduate Research Assistant >> Department of Mechanical Science and Engineering >> University of Illinois at Urbana-Champaign >> (217) 550-2360 >> salaza11 at illinois.edu >> >> > > > -- > *Miguel Angel Salazar de Troya* > Graduate Research Assistant > Department of Mechanical Science and Engineering > University of Illinois at Urbana-Champaign > (217) 550-2360 > salaza11 at illinois.edu > > -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaza11 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Tue Oct 28 09:49:21 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 09:49:21 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> Message-ID: <544FACF1.9050109@geology.wisc.edu> Matt The system is certainly not singular (checked using -pc_type svd -pc_svd_monitor). It also gives the correct solution (compared to the analytical solution for a trivial problem). Tabrez On 10/28/2014 08:44 AM, Matthew Knepley wrote: > On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali > wrote: > > Matt > > With MUMPS it is indeed the same but not faster. E.g., see 1.txt > and 2.txt > > > There is a big problem here. This system is never actually being > solved. Look at 1.txt: > > 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm > 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 > 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm > 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 > > The preconditioned residual did drop 10 order of magnitude, but the > true residual (b - Ax) is still very very large. > This looks like the system is singular and the preconditioner is > masking this. This needs to be figured out first. > I would go back to a small serial problem, and make sure it is > actually being solved. > > Matt > > With GAMG, cg takes forever (see 4.txt). For reference GAMG with > preonly is attached again (3.txt) > > All logs include -fieldsplit_0_ksp_monitor. > > Also, the true residual shows small oscillations. Is that normal? > > Tabrez > > > On 10/28/2014 07:08 AM, Matthew Knepley wrote: >> On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali >> > wrote: >> >> Mark >> >> When I replace "-fieldsplit_0_ksp_type preonly" with >> "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to >> kill it). >> >> >> That means something in the setup is wrong. It should be about >> the same or faster. Run with -fieldsplit_0_ksp_monitor >> so we can see what is happening. >> >> Matt >> >> With MUMPS , i.e., with '-fieldsplit_0_pc_type lu >> -fieldsplit_0_pc_factor_mat_solver_package mumps >> -fieldsplit_0_ksp_type preonly' it works fine but takes more >> time, and will be an issue for larger problems. The output >> for this run is attached. >> >> I will work on passing rigid body modes (as Matt mentioned) >> but short of that what is the best set of options for solving >> the following problem (i.e., linear elasticity with constraints): >> >> |K cG'| | u | = |F| >> |G 0 | |l/c| |d| >> >> where c is a scaling factor (so that cG' terms are more or >> less of the same order as K)? The constraints are used to >> impose slip between surfaces and so on. >> >> Tabrez >> >> On 10/27/2014 01:17 PM, Mark Adams wrote: >>> The null space for GAMG is not critical but useful for >>> elasticity. If you in fact have an indefinite operator (eg, >>> not "pinned) the you need to use an iterative coarse grid >>> solver. You are using '-fieldsplit_0_pc_type gamg >>> -fieldsplit_0_ksp_type preonly'. And you have a hard >>> elasticity problem. You are going to want to start with a >>> stronger solver. Use cg instead of preonly. As Matt said >>> start with MUMPS, then go to CG/GAMG, then you can see how >>> far you can cut the _0_ solver down. >>> Mark >>> >>> On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley >>> > wrote: >>> >>> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >>> > >>> wrote: >>> >>> Matt >>> >>> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>>> >>> > wrote: >>>> >>>> Matt >>>> >>>> Sorry about that (I always forget it). The >>>> output for the smallest problem is now attached >>>> (see log.txt). I am also attaching some results >>>> that compare results obtained using FS/LSC and >>>> the direct solver (MUMPS), again for the >>>> smallest problem. The difference, as you can >>>> see is insignificant O(1E-6). >>>> >>>> >>>> 1) How do you use MUMPS if you have a saddle point >>> I simply used -pc_type lu >>> -pc_factor_mat_solver_package mumps. >>> >>>> >>>> 2) You can see from the output that something is >>>> seriously wrong with the preconditioner. It looks >>>> like it has a null space. >>>> Did you add the elastic null modes to GAMG? >>>> Without this, it is not going to work. We have >>>> helper functions for this: >>>> >>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>>> >>>> you could just copy that code. And then use >>>> >>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>>> >>>> I don't see it in the output, so I think this is >>>> your problem. >>>> >>>> In order to test, I would first use MUMPS as the >>>> A00 solver and get the Schur stuff worked out. Then >>>> I would >>>> replace MUMPS with GAMG and tune it until I get >>>> back my original convergence. >>> I will try this with MatNullSpaceCreateRigidBody. >>> Btw does it matter if some nodes are pinned? >>> >>> >>> No these are null modes of the operator, not of the >>> particular problem. >>> >>> Matt >>> >>> Tabrez >>> >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>> Also, I did pass 'upper' and 'full' to >>>> '-pc_fieldsplit_schur_factorization_type' but >>>> the iteration count doesn't improve (in fact, >>>> it increases slightly). The attached log is >>>> with 'upper'. >>>> >>>> Regards, >>>> >>>> Tabrez >>>> >>>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>>>> >>>> > wrote: >>>>> >>>>> Hello >>>>> >>>>> I am using the following options (below) >>>>> for solving linear >>>>> elasticity/poroelasticity problems >>>>> involving slip between two surfaces >>>>> involving non-trivial geometries, i.e., >>>>> elements with high aspect ratios, large >>>>> contrasts in material properties etc. The >>>>> constraints are imposed using Lagrange >>>>> Multipliers. >>>>> >>>>> A picture (shows displacement magnitude) >>>>> is attached. The boundary nodes, i.e., the >>>>> base and the four side are pinned. >>>>> >>>>> The following options appear to work well >>>>> for the saddle point problem: >>>>> >>>>> -pc_type fieldsplit -pc_fieldsplit_type >>>>> schur -pc_fieldsplit_detect_saddle_point >>>>> -fieldsplit_0_pc_type gamg >>>>> -fieldsplit_0_ksp_type preonly >>>>> -fieldsplit_1_pc_type lsc >>>>> -fieldsplit_1_ksp_type preonly >>>>> -pc_fieldsplit_schur_fact_type lower >>>>> -ksp_monitor >>>>> >>>>> However, the number of iterations keep on >>>>> increasing with the problems size (see >>>>> attached plot), e.g., >>>>> >>>>> 120K Tets *507* Iterations (KSP Residual >>>>> norm 8.827362494659e-05)in 17 secs on 3 >>>>> cores >>>>> 1 Million Tets *1374* Iterations (KSP >>>>> Residual norm 7.164704416296e-05)in 117 >>>>> secs on 20 cores >>>>> 8 Million Tets *2495* Iterations (KSP >>>>> Residual norm 9.101247550026e-05) in 225 >>>>> secs on 160 cores >>>>> >>>>> So what other options should I try to >>>>> improve solver performance? Any >>>>> tips/insights would be appreciated as >>>>> preconditioning is black magic to me. >>>>> >>>>> >>>>> For reports, always run with >>>>> >>>>> -ksp_view -ksp_monitor_true_residual >>>>> -ksp_converged_reason >>>>> >>>>> so that we can see exactly what you used. >>>>> >>>>> I believe the default is a diagonal >>>>> factorization. Since your outer iterates are >>>>> increasing, I would strengthen this >>>>> to either upper or full >>>>> >>>>> -pc_fieldsplit_schur_factorization_type >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> Thanks in advance. >>>>> >>>>> Tabrez >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted >>>>> before they begin their experiments is >>>>> infinitely more interesting than any results >>>>> to which their experiments lead. >>>>> -- Norbert Wiener >>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before >>>> they begin their experiments is infinitely more >>>> interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they >>> begin their experiments is infinitely more interesting >>> than any results to which their experiments lead. >>> -- Norbert Wiener >>> >>> >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 28 09:55:56 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 09:55:56 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FACF1.9050109@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> Message-ID: On Tue, Oct 28, 2014 at 9:49 AM, Tabrez Ali wrote: > Matt > > The system is certainly not singular (checked using -pc_type svd > -pc_svd_monitor). It also gives the correct solution (compared to the > analytical solution for a trivial problem). > Something is very wrong with the preconditioner then. Move back to full Schur complement with a tight tolerance on the solver for S. This should converge in 1 iterate with a good true residual. Perhaps there is something wrong with LSC for your system. It is not guaranteed to work for all saddle points. Matt > > Tabrez > > > On 10/28/2014 08:44 AM, Matthew Knepley wrote: > > On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali > wrote: > >> Matt >> >> With MUMPS it is indeed the same but not faster. E.g., see 1.txt and 2.txt >> > > There is a big problem here. This system is never actually being solved. > Look at 1.txt: > > 0 KSP preconditioned resid norm 9.320055451716e+05 true resid norm > 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 > 380 KSP preconditioned resid norm 9.034871458425e-05 true resid norm > 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 > > The preconditioned residual did drop 10 order of magnitude, but the true > residual (b - Ax) is still very very large. > This looks like the system is singular and the preconditioner is masking > this. This needs to be figured out first. > I would go back to a small serial problem, and make sure it is actually > being solved. > > Matt > > >> With GAMG, cg takes forever (see 4.txt). For reference GAMG with >> preonly is attached again (3.txt) >> >> All logs include -fieldsplit_0_ksp_monitor. >> >> Also, the true residual shows small oscillations. Is that normal? >> >> Tabrez >> >> >> On 10/28/2014 07:08 AM, Matthew Knepley wrote: >> >> On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali >> wrote: >> >>> Mark >>> >>> When I replace "-fieldsplit_0_ksp_type preonly" with >>> "-fieldsplit_0_ksp_type cg" then it becomes very slow (had to kill it). >>> >> >> That means something in the setup is wrong. It should be about the same >> or faster. Run with -fieldsplit_0_ksp_monitor >> so we can see what is happening. >> >> Matt >> >> >>> With MUMPS , i.e., with '-fieldsplit_0_pc_type lu >>> -fieldsplit_0_pc_factor_mat_solver_package mumps -fieldsplit_0_ksp_type >>> preonly' it works fine but takes more time, and will be an issue for larger >>> problems. The output for this run is attached. >>> >>> I will work on passing rigid body modes (as Matt mentioned) but short of >>> that what is the best set of options for solving the following problem >>> (i.e., linear elasticity with constraints): >>> >>> |K cG'| | u | = |F| >>> |G 0 | |l/c| |d| >>> >>> where c is a scaling factor (so that cG' terms are more or less of the >>> same order as K)? The constraints are used to impose slip between surfaces >>> and so on. >>> >>> Tabrez >>> >>> On 10/27/2014 01:17 PM, Mark Adams wrote: >>> >>> The null space for GAMG is not critical but useful for elasticity. If >>> you in fact have an indefinite operator (eg, not "pinned) the you need to >>> use an iterative coarse grid solver. You are using '-fieldsplit_0_pc_type >>> gamg -fieldsplit_0_ksp_type preonly'. And you have a hard elasticity >>> problem. You are going to want to start with a stronger solver. Use cg >>> instead of preonly. As Matt said start with MUMPS, then go to CG/GAMG, >>> then you can see how far you can cut the _0_ solver down. >>> Mark >>> >>> On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley >>> wrote: >>> >>>> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >>>> wrote: >>>> >>>>> Matt >>>>> >>>>> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>>>> >>>>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>>>> wrote: >>>>> >>>>>> Matt >>>>>> >>>>>> Sorry about that (I always forget it). The output for the smallest >>>>>> problem is now attached (see log.txt). I am also attaching some results >>>>>> that compare results obtained using FS/LSC and the direct solver (MUMPS), >>>>>> again for the smallest problem. The difference, as you can see is >>>>>> insignificant O(1E-6). >>>>>> >>>>> >>>>> 1) How do you use MUMPS if you have a saddle point >>>>> >>>>> I simply used -pc_type lu -pc_factor_mat_solver_package mumps. >>>>> >>>>> >>>>> 2) You can see from the output that something is seriously wrong >>>>> with the preconditioner. It looks like it has a null space. >>>>> Did you add the elastic null modes to GAMG? Without this, it is >>>>> not going to work. We have helper functions for this: >>>>> >>>>> >>>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>>>> >>>>> you could just copy that code. And then use >>>>> >>>>> >>>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>>>> >>>>> I don't see it in the output, so I think this is your problem. >>>>> >>>>> In order to test, I would first use MUMPS as the A00 solver and get >>>>> the Schur stuff worked out. Then I would >>>>> replace MUMPS with GAMG and tune it until I get back my original >>>>> convergence. >>>>> >>>>> I will try this with MatNullSpaceCreateRigidBody. Btw does it matter >>>>> if some nodes are pinned? >>>>> >>>> >>>> No these are null modes of the operator, not of the particular >>>> problem. >>>> >>>> Matt >>>> >>>> >>>>> Tabrez >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> >>>>>> Also, I did pass 'upper' and 'full' to >>>>>> '-pc_fieldsplit_schur_factorization_type' but the iteration count doesn't >>>>>> improve (in fact, it increases slightly). The attached log is with 'upper'. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Tabrez >>>>>> >>>>>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>>>> >>>>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez Ali >>>>>> wrote: >>>>>> >>>>>>> Hello >>>>>>> >>>>>>> I am using the following options (below) for solving linear >>>>>>> elasticity/poroelasticity problems involving slip between two surfaces >>>>>>> involving non-trivial geometries, i.e., elements with high aspect ratios, >>>>>>> large contrasts in material properties etc. The constraints are imposed >>>>>>> using Lagrange Multipliers. >>>>>>> >>>>>>> A picture (shows displacement magnitude) is attached. The boundary >>>>>>> nodes, i.e., the base and the four side are pinned. >>>>>>> >>>>>>> The following options appear to work well for the saddle point >>>>>>> problem: >>>>>>> >>>>>>> -pc_type fieldsplit -pc_fieldsplit_type schur >>>>>>> -pc_fieldsplit_detect_saddle_point -fieldsplit_0_pc_type gamg >>>>>>> -fieldsplit_0_ksp_type preonly -fieldsplit_1_pc_type lsc >>>>>>> -fieldsplit_1_ksp_type preonly -pc_fieldsplit_schur_fact_type lower >>>>>>> -ksp_monitor >>>>>>> >>>>>>> However, the number of iterations keep on increasing with the >>>>>>> problems size (see attached plot), e.g., >>>>>>> >>>>>>> 120K Tets *507* Iterations (KSP Residual norm >>>>>>> 8.827362494659e-05) in 17 secs on 3 cores >>>>>>> 1 Million Tets *1374* Iterations (KSP Residual norm >>>>>>> 7.164704416296e-05) in 117 secs on 20 cores >>>>>>> 8 Million Tets *2495* Iterations (KSP Residual norm >>>>>>> 9.101247550026e-05) in 225 secs on 160 cores >>>>>>> >>>>>>> So what other options should I try to improve solver performance? >>>>>>> Any tips/insights would be appreciated as preconditioning is black magic to >>>>>>> me. >>>>>>> >>>>>> >>>>>> For reports, always run with >>>>>> >>>>>> -ksp_view -ksp_monitor_true_residual -ksp_converged_reason >>>>>> >>>>>> so that we can see exactly what you used. >>>>>> >>>>>> I believe the default is a diagonal factorization. Since your outer >>>>>> iterates are increasing, I would strengthen this >>>>>> to either upper or full >>>>>> >>>>>> -pc_fieldsplit_schur_factorization_type >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Thanks in advance. >>>>>>> >>>>>>> Tabrez >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted before they begin their >>>>> experiments is infinitely more interesting than any results to which their >>>>> experiments lead. >>>>> -- Norbert Wiener >>>>> >>>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>> >>> >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 10:03:12 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 09:03:12 -0600 Subject: [petsc-users] Adaptive controllers in TS In-Reply-To: References: <87wq7vrjg5.fsf@jedbrown.org> <87lhoam3ta.fsf@jedbrown.org> Message-ID: <87vbn4xmmn.fsf@jedbrown.org> Miguel Angel Salazar de Troya writes: > Sorry to be pushy, but could anyone help me on this? Your last email didn't have a specific question, so I'm not sure how to answer. What you want to do is possible, but not trivial and might require library modifications to avoid doing something perverse. Hong might be able to comment on his status or even somewhere that you could help. > Thanks > Miguel > > On Thu, Oct 23, 2014 at 10:46 AM, Miguel Angel Salazar de Troya < > salazardetroya at gmail.com> wrote: > >> I decided to implement the continuous adjoint because it is clearer to me >> how to do it and the gradient will converge anyways. I was trying to >> interpolate the solution, but in the adjoint analysis I need to do it once >> the simulation has finished. I have saved all the solutions for each time >> step. My idea was to reset the TS at the time step immediately previous to >> the time step in which I want to interpolate to: >> >> interpolate_timestep : where I want to interpolate to >> previous_timestep : last time step in our time step history which is >> smaller that the interpolate_timestep >> Current_Sol : Solution at the time step previous_timestep >> >> TSSetTime(ts,previous_timestep); >> TSSetSolution(ts,Current_Sol); >> TSSetRetainStages(ts,PETSC_TRUE); Presumably this is a fragment and you did TSStep in here, otherwise how do you expect valid stage values? >> TSInterpolate(ts,interpolate_timestep,X); >> >> Vector X should have a close value to Current_Sol, but it's nowhere close. >> I also compare it to the solution of the next time step after >> previous_timestep (therefore interpolate_timestep is between these guys) >> and it's not close either. I've read that TSInterpolate() has to be >> extended to support continuous adjoints. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From lb2653 at columbia.edu Tue Oct 28 10:04:30 2014 From: lb2653 at columbia.edu (Luc Berger-Vergiat) Date: Tue, 28 Oct 2014 11:04:30 -0400 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <544E943F.5070007@purdue.edu> References: <544E943F.5070007@purdue.edu> Message-ID: <544FB07E.1020502@columbi.edu> Hi, I am running a code on CETUS and I use PETSc for as a linear solver. Here is my submission command: qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU Here is "4nodes_32cores_LU": #!/bin/sh LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" echo "Cobalt location args: $LOCARGS" >&2 ################################ # 32 cores on 4 nodes jobs # ################################ runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile I get the following error message: [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [7]PETSC ERROR: Argument out of range [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib -lblas" --with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node Thanks for the help Best, Luc On 10/27/2014 02:51 PM, Michael Povolotskyi wrote: > Hello, > will the eigenvectors that correspond to degenerate eigenvalues of a > Hermitian matrix be orthogonal? > I'm using Krylov Schur method. > Thank you, > Michael. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 10:10:38 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 09:10:38 -0600 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FAE89.8000405@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <871tpsz3y0.fsf@jedbrown.org> <544FAE89.8000405@geology.wisc.edu> Message-ID: <87sii8xma9.fsf@jedbrown.org> [re-adding list to Cc] Tabrez Ali writes: > Jed > > The reason I did not nondimenionalize was because it didnt make gmres > with asm go any faster for my test cases. The true residual spikes up by 15 orders of magnitude when you start solving (with MUMPS on fieldsplit_0_). That is a major problem and will hopefully be resolved if you scale the fields reasonably. The scales should be balanced so that residuals (respectively errors) are balanced when your sense of quality of solution is balanced. Note that triangular fieldsplit is a nonnormal preconditioner, so this is more important there. If don't nondimensionalize, you can use Schur Complement Reduction, but with a scale imbalance of 1e15, you can't combine with an outer iteration (usually not desirable for SCR, but you may have to solve the Schur complement problem very accurately to keep errors small). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From rongliang.chan at gmail.com Tue Oct 28 10:34:19 2014 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 28 Oct 2014 09:34:19 -0600 Subject: [petsc-users] Book for the PETSc finite volume method Message-ID: <544FB77B.3060705@gmail.com> Hello, I have a very elementary question. I am trying to learn the PETSc finite volume method codes. Can you tell me which book did you refer to for the PETSc FVM? Thanks. Best regards, Rongliang From knepley at gmail.com Tue Oct 28 10:35:45 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 10:35:45 -0500 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <544FB77B.3060705@gmail.com> References: <544FB77B.3060705@gmail.com> Message-ID: On Tue, Oct 28, 2014 at 10:34 AM, Rongliang Chen wrote: > Hello, > > I have a very elementary question. I am trying to learn the PETSc finite > volume method codes. Can you tell me which book did you refer to for the > PETSc FVM? PETSc has some support for operations done in finite volume codes, but it is very new and primitive. There are no full applications that use this. Right now it is an experiment. Thanks, Matt > Thanks. > > Best regards, > Rongliang > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 10:44:07 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 09:44:07 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: References: <544FB77B.3060705@gmail.com> Message-ID: <87ppdcxkqg.fsf@jedbrown.org> Matthew Knepley writes: > On Tue, Oct 28, 2014 at 10:34 AM, Rongliang Chen > wrote: > >> Hello, >> >> I have a very elementary question. I am trying to learn the PETSc finite >> volume method codes. Can you tell me which book did you refer to for the >> PETSc FVM? > > > PETSc has some support for operations done in finite volume codes, but it > is very new and primitive. There are no full applications > that use this. Right now it is an experiment. There are, however, many full applications based on finite volume methods that use PETSc for algebraic solvers and even (structured) grid management. Matt's comment refers to an experimental discretization interface called PetscFV that attempts to hide the boilerplate for a certain class of finite volume methods. If you are looking for a book on finite volume methods for hyperbolic problems, I recommend LeVeque's and the PyClaw package (which can use PETSc for parallelism). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From rongliang.chan at gmail.com Tue Oct 28 11:00:28 2014 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 28 Oct 2014 10:00:28 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <87ppdcxkqg.fsf@jedbrown.org> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> Message-ID: <544FBD9C.7030706@gmail.com> Thanks Matt and Jed, I am trying to write a parallel FVM code on unstructured grid for compressible NS equations. I found that the finite volume code on the structured mesh is easy to write but for the unstructured mesh, it is really untrivial. I want to follow the PETSc's DMPlexFV but it is not easy to read, so I want to find a good book for it. Best regards, Rongliang On 10/28/2014 09:44 AM, Jed Brown wrote: > Matthew Knepley writes: > >> On Tue, Oct 28, 2014 at 10:34 AM, Rongliang Chen >> wrote: >> >>> Hello, >>> >>> I have a very elementary question. I am trying to learn the PETSc finite >>> volume method codes. Can you tell me which book did you refer to for the >>> PETSc FVM? >> >> PETSc has some support for operations done in finite volume codes, but it >> is very new and primitive. There are no full applications >> that use this. Right now it is an experiment. > There are, however, many full applications based on finite volume > methods that use PETSc for algebraic solvers and even (structured) grid > management. Matt's comment refers to an experimental discretization > interface called PetscFV that attempts to hide the boilerplate for a > certain class of finite volume methods. > > If you are looking for a book on finite volume methods for hyperbolic > problems, I recommend LeVeque's and the PyClaw package (which can use > PETSc for parallelism). From stali at geology.wisc.edu Tue Oct 28 11:11:35 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 11:11:35 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> Message-ID: <544FC037.2060908@geology.wisc.edu> Matt and Jed But it does appear to work well in practice (as in I get the correct solution). Based on what I have tried, LSC (with GAMG on 00) is the only solver I am able to use on relatively large problems with non trivial geometries (e.g., see my original email). So why is that? I will experiment with your other suggestions. Thanks, Tabrez On 10/28/2014 09:55 AM, Matthew Knepley wrote: > On Tue, Oct 28, 2014 at 9:49 AM, Tabrez Ali > wrote: > > Matt > > The system is certainly not singular (checked using -pc_type svd > -pc_svd_monitor). It also gives the correct solution (compared to > the analytical solution for a trivial problem). > > > Something is very wrong with the preconditioner then. Move back to > full Schur complement with a tight tolerance > on the solver for S. This should converge in 1 iterate with a good > true residual. Perhaps there is something wrong > with LSC for your system. It is not guaranteed to work for all saddle > points. > > Matt > > > Tabrez > > > On 10/28/2014 08:44 AM, Matthew Knepley wrote: >> On Tue, Oct 28, 2014 at 8:00 AM, Tabrez Ali >> > wrote: >> >> Matt >> >> With MUMPS it is indeed the same but not faster. E.g., see >> 1.txt and 2.txt >> >> >> There is a big problem here. This system is never actually being >> solved. Look at 1.txt: >> >> 0 KSP preconditioned resid norm 9.320055451716e+05 true resid >> norm 1.755998647494e+02 ||r(i)||/||b|| 1.000000000000e+00 >> 380 KSP preconditioned resid norm 9.034871458425e-05 true resid >> norm 3.118451181896e+09 ||r(i)||/||b|| 1.775884728811e+07 >> >> The preconditioned residual did drop 10 order of magnitude, but >> the true residual (b - Ax) is still very very large. >> This looks like the system is singular and the preconditioner is >> masking this. This needs to be figured out first. >> I would go back to a small serial problem, and make sure it is >> actually being solved. >> >> Matt >> >> With GAMG, cg takes forever (see 4.txt). For reference GAMG >> with preonly is attached again (3.txt) >> >> All logs include -fieldsplit_0_ksp_monitor. >> >> Also, the true residual shows small oscillations. Is that normal? >> >> Tabrez >> >> >> On 10/28/2014 07:08 AM, Matthew Knepley wrote: >>> On Tue, Oct 28, 2014 at 5:39 AM, Tabrez Ali >>> > wrote: >>> >>> Mark >>> >>> When I replace "-fieldsplit_0_ksp_type preonly" with >>> "-fieldsplit_0_ksp_type cg" then it becomes very slow >>> (had to kill it). >>> >>> >>> That means something in the setup is wrong. It should be >>> about the same or faster. Run with -fieldsplit_0_ksp_monitor >>> so we can see what is happening. >>> >>> Matt >>> >>> With MUMPS , i.e., with '-fieldsplit_0_pc_type lu >>> -fieldsplit_0_pc_factor_mat_solver_package mumps >>> -fieldsplit_0_ksp_type preonly' it works fine but takes >>> more time, and will be an issue for larger problems. The >>> output for this run is attached. >>> >>> I will work on passing rigid body modes (as Matt >>> mentioned) but short of that what is the best set of >>> options for solving the following problem (i.e., linear >>> elasticity with constraints): >>> >>> |K cG'| | u | = |F| >>> |G 0 | |l/c| |d| >>> >>> where c is a scaling factor (so that cG' terms are more >>> or less of the same order as K)? The constraints are >>> used to impose slip between surfaces and so on. >>> >>> Tabrez >>> >>> On 10/27/2014 01:17 PM, Mark Adams wrote: >>>> The null space for GAMG is not critical but useful for >>>> elasticity. If you in fact have an indefinite operator >>>> (eg, not "pinned) the you need to use an iterative >>>> coarse grid solver. You are >>>> using '-fieldsplit_0_pc_type gamg >>>> -fieldsplit_0_ksp_type preonly'. And you have a hard >>>> elasticity problem. You are going to want to start >>>> with a stronger solver. Use cg instead of preonly. As >>>> Matt said start with MUMPS, then go to CG/GAMG, then >>>> you can see how far you can cut the _0_ solver down. >>>> Mark >>>> >>>> On Thu, Oct 23, 2014 at 11:51 AM, Matthew Knepley >>>> > wrote: >>>> >>>> On Thu, Oct 23, 2014 at 10:48 AM, Tabrez Ali >>>> >>> > wrote: >>>> >>>> Matt >>>> >>>> On 10/23/2014 09:54 AM, Matthew Knepley wrote: >>>>> On Thu, Oct 23, 2014 at 9:27 AM, Tabrez Ali >>>>> >>>> > wrote: >>>>> >>>>> Matt >>>>> >>>>> Sorry about that (I always forget it). The >>>>> output for the smallest problem is now >>>>> attached (see log.txt). I am also >>>>> attaching some results that compare >>>>> results obtained using FS/LSC and the >>>>> direct solver (MUMPS), again for the >>>>> smallest problem. The difference, as you >>>>> can see is insignificant O(1E-6). >>>>> >>>>> >>>>> 1) How do you use MUMPS if you have a saddle point >>>> I simply used -pc_type lu >>>> -pc_factor_mat_solver_package mumps. >>>> >>>>> >>>>> 2) You can see from the output that something >>>>> is seriously wrong with the preconditioner. It >>>>> looks like it has a null space. >>>>> Did you add the elastic null modes to >>>>> GAMG? Without this, it is not going to work. >>>>> We have helper functions for this: >>>>> >>>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DM/DMPlexCreateRigidBody.html >>>>> >>>>> you could just copy that code. And then use >>>>> >>>>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSetNearNullSpace.html >>>>> >>>>> I don't see it in the output, so I think this >>>>> is your problem. >>>>> >>>>> In order to test, I would first use MUMPS as >>>>> the A00 solver and get the Schur stuff worked >>>>> out. Then I would >>>>> replace MUMPS with GAMG and tune it until I >>>>> get back my original convergence. >>>> I will try this with >>>> MatNullSpaceCreateRigidBody. Btw does it matter >>>> if some nodes are pinned? >>>> >>>> >>>> No these are null modes of the operator, not of the >>>> particular problem. >>>> >>>> Matt >>>> >>>> Tabrez >>>> >>>>> >>>>> Thanks, >>>>> >>>>> Matt >>>>> >>>>> Also, I did pass 'upper' and 'full' to >>>>> '-pc_fieldsplit_schur_factorization_type' >>>>> but the iteration count doesn't improve >>>>> (in fact, it increases slightly). The >>>>> attached log is with 'upper'. >>>>> >>>>> Regards, >>>>> >>>>> Tabrez >>>>> >>>>> On 10/23/2014 07:46 AM, Matthew Knepley wrote: >>>>>> On Thu, Oct 23, 2014 at 7:20 AM, Tabrez >>>>>> Ali >>>>> > wrote: >>>>>> >>>>>> Hello >>>>>> >>>>>> I am using the following options >>>>>> (below) for solving linear >>>>>> elasticity/poroelasticity problems >>>>>> involving slip between two surfaces >>>>>> involving non-trivial geometries, >>>>>> i.e., elements with high aspect >>>>>> ratios, large contrasts in material >>>>>> properties etc. The constraints are >>>>>> imposed using Lagrange Multipliers. >>>>>> >>>>>> A picture (shows displacement >>>>>> magnitude) is attached. The boundary >>>>>> nodes, i.e., the base and the four >>>>>> side are pinned. >>>>>> >>>>>> The following options appear to work >>>>>> well for the saddle point problem: >>>>>> >>>>>> -pc_type fieldsplit >>>>>> -pc_fieldsplit_type schur >>>>>> -pc_fieldsplit_detect_saddle_point >>>>>> -fieldsplit_0_pc_type gamg >>>>>> -fieldsplit_0_ksp_type preonly >>>>>> -fieldsplit_1_pc_type lsc >>>>>> -fieldsplit_1_ksp_type preonly >>>>>> -pc_fieldsplit_schur_fact_type lower >>>>>> -ksp_monitor >>>>>> >>>>>> However, the number of iterations >>>>>> keep on increasing with the problems >>>>>> size (see attached plot), e.g., >>>>>> >>>>>> 120K Tets *507* Iterations (KSP >>>>>> Residual norm 8.827362494659e-05)in >>>>>> 17 secs on 3 cores >>>>>> 1 Million Tets *1374* Iterations (KSP >>>>>> Residual norm 7.164704416296e-05)in >>>>>> 117 secs on 20 cores >>>>>> 8 Million Tets *2495* Iterations (KSP >>>>>> Residual norm 9.101247550026e-05) in >>>>>> 225 secs on 160 cores >>>>>> >>>>>> So what other options should I try to >>>>>> improve solver performance? Any >>>>>> tips/insights would be appreciated as >>>>>> preconditioning is black magic to me. >>>>>> >>>>>> >>>>>> For reports, always run with >>>>>> >>>>>> -ksp_view -ksp_monitor_true_residual >>>>>> -ksp_converged_reason >>>>>> >>>>>> so that we can see exactly what you used. >>>>>> >>>>>> I believe the default is a diagonal >>>>>> factorization. Since your outer iterates >>>>>> are increasing, I would strengthen this >>>>>> to either upper or full >>>>>> >>>>>> -pc_fieldsplit_schur_factorization_type >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Matt >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> Tabrez >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted >>>>>> before they begin their experiments is >>>>>> infinitely more interesting than any >>>>>> results to which their experiments lead. >>>>>> -- Norbert Wiener >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> What most experimenters take for granted >>>>> before they begin their experiments is >>>>> infinitely more interesting than any results >>>>> to which their experiments lead. >>>>> -- Norbert Wiener >>>> >>>> >>>> >>>> >>>> -- >>>> What most experimenters take for granted before >>>> they begin their experiments is infinitely more >>>> interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> >>> >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin >>> their experiments is infinitely more interesting than any >>> results to which their experiments lead. >>> -- Norbert Wiener >> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to >> which their experiments lead. >> -- Norbert Wiener > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 11:15:10 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 10:15:10 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <544FBD9C.7030706@gmail.com> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> Message-ID: <87k33kxjap.fsf@jedbrown.org> Rongliang Chen writes: > Thanks Matt and Jed, > > I am trying to write a parallel FVM code on unstructured grid for > compressible NS equations. I found that the finite volume code on the > structured mesh is easy to write but for the unstructured mesh, it is > really untrivial. I want to follow the PETSc's DMPlexFV but it is not > easy to read, so I want to find a good book for it. There is no book specific to PETSc's interfaces. What sort of equations are you solving? For hyperbolic systems, you can look at src/ts/examples/tutorials/ex11.c. Make sure to set up global/tags so you can jump to the library implementations, or look at an older version that doesn't hide the actual discretization in overly-specialized library functions. (Matt and I wrote this example together, but disagree about which parts belong in the library.) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Tue Oct 28 11:18:08 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 10:18:08 -0600 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FC037.2060908@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> Message-ID: <87h9yoxj5r.fsf@jedbrown.org> Tabrez Ali writes: > Matt and Jed > > But it does appear to work well in practice (as in I get the correct > solution). Your "correct" solution has a residual seven orders of magnitude bigger than the initial guess. Does that not give you pause? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From rongliang.chan at gmail.com Tue Oct 28 11:33:37 2014 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 28 Oct 2014 10:33:37 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <87k33kxjap.fsf@jedbrown.org> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> <87k33kxjap.fsf@jedbrown.org> Message-ID: <544FC561.6040204@gmail.com> Hi Jed, I am solving the 3D compressible Navier-Stokes equations. I have read the ex11.c for several days and still do not fully understand this. I will look at the older version as you suggested. Maybe I need to go to your office to ask you some questions. :-) Best regards, Rongliang On 10/28/2014 10:15 AM, Jed Brown wrote: > Rongliang Chen writes: > >> Thanks Matt and Jed, >> >> I am trying to write a parallel FVM code on unstructured grid for >> compressible NS equations. I found that the finite volume code on the >> structured mesh is easy to write but for the unstructured mesh, it is >> really untrivial. I want to follow the PETSc's DMPlexFV but it is not >> easy to read, so I want to find a good book for it. > There is no book specific to PETSc's interfaces. What sort of equations > are you solving? For hyperbolic systems, you can look at > src/ts/examples/tutorials/ex11.c. Make sure to set up global/tags so > you can jump to the library implementations, or look at an older version > that doesn't hide the actual discretization in overly-specialized > library functions. (Matt and I wrote this example together, but > disagree about which parts belong in the library.) From bsmith at mcs.anl.gov Tue Oct 28 11:33:47 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Oct 2014 11:33:47 -0500 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <544FB07E.1020502@columbi.edu> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> Message-ID: <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> Hmm, this should never happen. In the code ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); for (i=0; iB->rmap->n; i++) { for (j=0; jilen[i]; j++) { PetscInt data,gid1 = aj[B->i[i] + j] + 1; ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind Barry > On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat wrote: > > Hi, > I am running a code on CETUS and I use PETSc for as a linear solver. > Here is my submission command: > qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU > > Here is "4nodes_32cores_LU": > #!/bin/sh > > LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" > echo "Cobalt location args: $LOCARGS" >&2 > > ################################ > # 32 cores on 4 nodes jobs # > ################################ > runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile > > I get the following error message: > > [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [7]PETSC ERROR: Argument out of range > [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown > [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 > [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib -lblas" --with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" > > [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h > [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c > [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c > [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c > > Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... > > I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node > > Thanks for the help > Best, > Luc > > On 10/27/2014 02:51 PM, Michael Povolotskyi wrote: >> Hello, >> will the eigenvectors that correspond to degenerate eigenvalues of a Hermitian matrix be orthogonal? >> I'm using Krylov Schur method. >> Thank you, >> Michael. >> > From stali at geology.wisc.edu Tue Oct 28 11:56:14 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 11:56:14 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <87h9yoxj5r.fsf@jedbrown.org> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> <87h9yoxj5r.fsf@jedbrown.org> Message-ID: <544FCAAE.3040609@geology.wisc.edu> Jed You're right and it is precisely what I am trying to understand as well. I am attaching the comparison to the analytical solution. As you can see the agreement is almost perfect, except at the discontinuity (y=0 in the analytical solution). So why am I getting the correct solution when according to true residuals I should not? Tabrez On 10/28/2014 11:18 AM, Jed Brown wrote: > Tabrez Ali writes: > >> Matt and Jed >> >> But it does appear to work well in practice (as in I get the correct >> solution). > Your "correct" solution has a residual seven orders of magnitude bigger > than the initial guess. Does that not give you pause? -------------- next part -------------- A non-text attachment was scrubbed... Name: petsc_vs_analytical.pdf Type: application/pdf Size: 19410 bytes Desc: not available URL: From knepley at gmail.com Tue Oct 28 11:57:56 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 11:57:56 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FCAAE.3040609@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> <87h9yoxj5r.fsf@jedbrown.org> <544FCAAE.3040609@geology.wisc.edu> Message-ID: On Tue, Oct 28, 2014 at 11:56 AM, Tabrez Ali wrote: > Jed > > You're right and it is precisely what I am trying to understand as well. I > am attaching the comparison to the analytical solution. As you can see the > agreement is almost perfect, except at the discontinuity (y=0 in the > analytical solution). > > So why am I getting the correct solution when according to true residuals > I should not? The discontinuty is generating the large residuals. Your saddle is not actually enforcing the discontinuty, so you get the bulk mostly right by solving the (1,1) block, but not the slip. At least it looks that way to me. Matt > > Tabrez > > > On 10/28/2014 11:18 AM, Jed Brown wrote: > >> Tabrez Ali writes: >> >> Matt and Jed >>> >>> But it does appear to work well in practice (as in I get the correct >>> solution). >>> >> Your "correct" solution has a residual seven orders of magnitude bigger >> than the initial guess. Does that not give you pause? >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From stali at geology.wisc.edu Tue Oct 28 12:05:42 2014 From: stali at geology.wisc.edu (Tabrez Ali) Date: Tue, 28 Oct 2014 12:05:42 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> <87h9yoxj5r.fsf@jedbrown.org> <544FCAAE.3040609@geology.wisc.edu> Message-ID: <544FCCE6.3050700@geology.wisc.edu> Matt The analytical (Okada) solution is wrong at the fault. PETSc is right. It is a strike slip fault and constraints are being enforced correctly. Tabrez On 10/28/2014 11:57 AM, Matthew Knepley wrote: > On Tue, Oct 28, 2014 at 11:56 AM, Tabrez Ali > wrote: > > Jed > > You're right and it is precisely what I am trying to understand as > well. I am attaching the comparison to the analytical solution. As > you can see the agreement is almost perfect, except at the > discontinuity (y=0 in the analytical solution). > > So why am I getting the correct solution when according to true > residuals I should not? > > > The discontinuty is generating the large residuals. Your saddle is not > actually enforcing > the discontinuty, so you get the bulk mostly right by solving the > (1,1) block, but not > the slip. At least it looks that way to me. > > Matt > > > Tabrez > > > On 10/28/2014 11:18 AM, Jed Brown wrote: > > Tabrez Ali > writes: > > Matt and Jed > > But it does appear to work well in practice (as in I get > the correct > solution). > > Your "correct" solution has a residual seven orders of > magnitude bigger > than the initial guess. Does that not give you pause? > > > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which > their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Tue Oct 28 12:08:02 2014 From: knepley at gmail.com (Matthew Knepley) Date: Tue, 28 Oct 2014 12:08:02 -0500 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FCCE6.3050700@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> <87h9yoxj5r.fsf@jedbrown.org> <544FCAAE.3040609@geology.wisc.edu> <544FCCE6.3050700@geology.wisc.edu> Message-ID: On Tue, Oct 28, 2014 at 12:05 PM, Tabrez Ali wrote: > Matt > > The analytical (Okada) solution is wrong at the fault. PETSc is right. It > is a strike slip fault and constraints are being enforced correctly. > If the solution is right and the residual is large, then the function calculating the residual is wrong, since the residual is supposed to define the solution. Matt > > Tabrez > > > > On 10/28/2014 11:57 AM, Matthew Knepley wrote: > > On Tue, Oct 28, 2014 at 11:56 AM, Tabrez Ali > wrote: > >> Jed >> >> You're right and it is precisely what I am trying to understand as well. >> I am attaching the comparison to the analytical solution. As you can see >> the agreement is almost perfect, except at the discontinuity (y=0 in the >> analytical solution). >> >> So why am I getting the correct solution when according to true residuals >> I should not? > > > The discontinuty is generating the large residuals. Your saddle is not > actually enforcing > the discontinuty, so you get the bulk mostly right by solving the (1,1) > block, but not > the slip. At least it looks that way to me. > > Matt > > >> >> Tabrez >> >> >> On 10/28/2014 11:18 AM, Jed Brown wrote: >> >>> Tabrez Ali writes: >>> >>> Matt and Jed >>>> >>>> But it does appear to work well in practice (as in I get the correct >>>> solution). >>>> >>> Your "correct" solution has a residual seven orders of magnitude bigger >>> than the initial guess. Does that not give you pause? >>> >> >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Tue Oct 28 12:08:16 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 11:08:16 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <544FC561.6040204@gmail.com> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> <87k33kxjap.fsf@jedbrown.org> <544FC561.6040204@gmail.com> Message-ID: <87egtsxgu7.fsf@jedbrown.org> Rongliang Chen writes: > Hi Jed, > > I am solving the 3D compressible Navier-Stokes equations. I have read > the ex11.c for several days and still do not fully understand this. I > will look at the older version as you suggested. Maybe I need to go to > your office to ask you some questions. :-) Are you on campus? ex11.c has not implemented diffusion, but it has an Euler example and I can draw pictures to help explain the reconstruction. Did you read the cited Berger 2005 paper on slope reconstruction and limiters for irregular grids? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jed at jedbrown.org Tue Oct 28 12:20:56 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 11:20:56 -0600 Subject: [petsc-users] Fieldsplit with LSC for constrained elasticity/poroelasticity? In-Reply-To: <544FCAAE.3040609@geology.wisc.edu> References: <5448F270.8010902@geology.wisc.edu> <54491042.8080307@geology.wisc.edu> <54492357.4070304@geology.wisc.edu> <544F726E.6060503@geology.wisc.edu> <544F9376.8030008@geology.wisc.edu> <544FACF1.9050109@geology.wisc.edu> <544FC037.2060908@geology.wisc.edu> <87h9yoxj5r.fsf@jedbrown.org> <544FCAAE.3040609@geology.wisc.edu> Message-ID: <87bnowxg93.fsf@jedbrown.org> Tabrez Ali writes: > Jed > > You're right and it is precisely what I am trying to understand as well. > I am attaching the comparison to the analytical solution. As you can see > the agreement is almost perfect, except at the discontinuity (y=0 in the > analytical solution). > > So why am I getting the correct solution when according to true > residuals I should not? Assuming (I'm not convinced) that the solution is "correct", your system is sufficiently ill-conditioned that the residual is huge. Take any system A and add a single row and column that are all zero except an entry on the diagonal. Augment the right-hand side with a corresponding 0. The solution is obviously the solution of A x = b with an extra 0 regardless of what the diagonal entry is. If the diagonal entry is 1e20, then [x; 1e-10] has a huge residual even though it is pretty accurate. If the diagonal entry is 1e-20, then [x; 1e10] is a nonsensical solution with a tiny residual. This is ill-conditioning. You should nondimensionalize your problem to avoid this spurious ill-conditioning, otherwise you should assume your results are nonsense any any case where you don't already know the answer. Sorry. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From lb2653 at columbia.edu Tue Oct 28 12:30:35 2014 From: lb2653 at columbia.edu (Luc Berger-Vergiat) Date: Tue, 28 Oct 2014 13:30:35 -0400 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> Message-ID: <544FD2BB.6070500@columbi.edu> Ok, I'm recompiling PETSc in debug mode then. Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? Would this work for example: runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile Best, Luc On 10/28/2014 12:33 PM, Barry Smith wrote: > Hmm, this should never happen. In the code > > ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); > for (i=0; iB->rmap->n; i++) { > for (j=0; jilen[i]; j++) { > PetscInt data,gid1 = aj[B->i[i] + j] + 1; > ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); > > Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind > > Barry > >> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat wrote: >> >> Hi, >> I am running a code on CETUS and I use PETSc for as a linear solver. >> Here is my submission command: >> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >> >> Here is "4nodes_32cores_LU": >> #!/bin/sh >> >> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >> echo "Cobalt location args: $LOCARGS" >&2 >> >> ################################ >> # 32 cores on 4 nodes jobs # >> ################################ >> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >> >> I get the following error message: >> >> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >> [7]PETSC ERROR: Argument out of range >> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib -lblas" --with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >> >> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >> >> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >> >> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >> >> Thanks for the help >> Best, >> Luc >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rongliang.chan at gmail.com Tue Oct 28 12:34:40 2014 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 28 Oct 2014 11:34:40 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <87egtsxgu7.fsf@jedbrown.org> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> <87k33kxjap.fsf@jedbrown.org> <544FC561.6040204@gmail.com> <87egtsxgu7.fsf@jedbrown.org> Message-ID: <544FD3B0.9040504@gmail.com> Yes. I am at CU Boulder. I will read the Berger paper first, and then if possible, I will go to your office to talk with you when you are free. Thanks. Best, Rongliang On 10/28/2014 11:08 AM, Jed Brown wrote: > Rongliang Chen writes: > >> Hi Jed, >> >> I am solving the 3D compressible Navier-Stokes equations. I have read >> the ex11.c for several days and still do not fully understand this. I >> will look at the older version as you suggested. Maybe I need to go to >> your office to ask you some questions. :-) > Are you on campus? > > ex11.c has not implemented diffusion, but it has an Euler example and I > can draw pictures to help explain the reconstruction. Did you read the > cited Berger 2005 paper on slope reconstruction and limiters for > irregular grids? From jed at jedbrown.org Tue Oct 28 12:41:20 2014 From: jed at jedbrown.org (Jed Brown) Date: Tue, 28 Oct 2014 11:41:20 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <544FD3B0.9040504@gmail.com> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> <87k33kxjap.fsf@jedbrown.org> <544FC561.6040204@gmail.com> <87egtsxgu7.fsf@jedbrown.org> <544FD3B0.9040504@gmail.com> Message-ID: <8761f4xfb3.fsf@jedbrown.org> Rongliang Chen writes: > Yes. I am at CU Boulder. I will read the Berger paper first, and then if > possible, I will go to your office to talk with you when you are free. I can meet before 5pm this afternoon or tomorrow any time other than 12-1pm. Note that the example does not currently define a diffusion/viscosity, and there is more than one way to do that. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From rongliang.chan at gmail.com Tue Oct 28 12:49:04 2014 From: rongliang.chan at gmail.com (Rongliang Chen) Date: Tue, 28 Oct 2014 11:49:04 -0600 Subject: [petsc-users] Book for the PETSc finite volume method In-Reply-To: <8761f4xfb3.fsf@jedbrown.org> References: <544FB77B.3060705@gmail.com> <87ppdcxkqg.fsf@jedbrown.org> <544FBD9C.7030706@gmail.com> <87k33kxjap.fsf@jedbrown.org> <544FC561.6040204@gmail.com> <87egtsxgu7.fsf@jedbrown.org> <544FD3B0.9040504@gmail.com> <8761f4xfb3.fsf@jedbrown.org> Message-ID: <544FD710.9090303@gmail.com> Great. I will go to your office about 3PM this afternoon. Thank you very much. Best regards, Rongliang On 10/28/2014 11:41 AM, Jed Brown wrote: > Rongliang Chen writes: > >> Yes. I am at CU Boulder. I will read the Berger paper first, and then if >> possible, I will go to your office to talk with you when you are free. > I can meet before 5pm this afternoon or tomorrow any time other than 12-1pm. > > Note that the example does not currently define a diffusion/viscosity, > and there is more than one way to do that. From bsmith at mcs.anl.gov Tue Oct 28 13:25:20 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Oct 2014 13:25:20 -0500 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <544FD2BB.6070500@columbi.edu> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> Message-ID: <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> You need to pass some options to valgrind telling it to check for memory corruption issues > On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat wrote: > > Ok, I'm recompiling PETSc in debug mode then. > Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? > Would this work for example: > runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile > > > Best, > Luc > > On 10/28/2014 12:33 PM, Barry Smith wrote: >> Hmm, this should never happen. In the code >> >> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >> for (i=0; iB->rmap->n; i++) { >> for (j=0; jilen[i]; j++) { >> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >> >> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >> >> >> Barry >> >> >>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>> wrote: >>> >>> Hi, >>> I am running a code on CETUS and I use PETSc for as a linear solver. >>> Here is my submission command: >>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>> >>> Here is "4nodes_32cores_LU": >>> #!/bin/sh >>> >>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>> echo "Cobalt location args: $LOCARGS" >&2 >>> >>> ################################ >>> # 32 cores on 4 nodes jobs # >>> ################################ >>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>> >>> I get the following error message: >>> >>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [7]PETSC ERROR: Argument out of range >>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>> >>> -lblas" - >>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>> >>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>> >>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>> >>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>> >>> Thanks for the help >>> Best, >>> Luc >>> >>> >>> > From lb2653 at columbia.edu Tue Oct 28 13:50:36 2014 From: lb2653 at columbia.edu (Luc Berger-Vergiat) Date: Tue, 28 Oct 2014 14:50:36 -0400 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> Message-ID: <544FE57C.8020303@columbi.edu> Yes, I am running with --leak-check=full Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... Hopefully the queue will go up fast and I can give you some details about the issue. Best, Luc On 10/28/2014 02:25 PM, Barry Smith wrote: > You need to pass some options to valgrind telling it to check for memory corruption issues > > >> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat wrote: >> >> Ok, I'm recompiling PETSc in debug mode then. >> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >> Would this work for example: >> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >> >> >> Best, >> Luc >> >> On 10/28/2014 12:33 PM, Barry Smith wrote: >>> Hmm, this should never happen. In the code >>> >>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>> for (i=0; iB->rmap->n; i++) { >>> for (j=0; jilen[i]; j++) { >>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>> >>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>> >>> >>> Barry >>> >>> >>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>> wrote: >>>> >>>> Hi, >>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>> Here is my submission command: >>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>> >>>> Here is "4nodes_32cores_LU": >>>> #!/bin/sh >>>> >>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>> echo "Cobalt location args: $LOCARGS" >&2 >>>> >>>> ################################ >>>> # 32 cores on 4 nodes jobs # >>>> ################################ >>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>> >>>> I get the following error message: >>>> >>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>> [7]PETSC ERROR: Argument out of range >>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>> >>>> -lblas" - >>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>> >>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>> >>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>> >>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>> >>>> Thanks for the help >>>> Best, >>>> Luc >>>> >>>> >>>> From bsmith at mcs.anl.gov Tue Oct 28 13:53:54 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Tue, 28 Oct 2014 13:53:54 -0500 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <544FE57C.8020303@columbi.edu> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> <544FE57C.8020303@columbi.edu> Message-ID: <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> You don?t care about checking for leaks. I use -q --tool=memcheck --num-callers=20 --track-origins=yes > On Oct 28, 2014, at 1:50 PM, Luc Berger-Vergiat wrote: > > Yes, I am running with --leak-check=full > Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... > Hopefully the queue will go up fast and I can give you some details about the issue. > > Best, > Luc > > On 10/28/2014 02:25 PM, Barry Smith wrote: >> You need to pass some options to valgrind telling it to check for memory corruption issues >> >> >>> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat wrote: >>> >>> Ok, I'm recompiling PETSc in debug mode then. >>> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >>> Would this work for example: >>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >>> >>> >>> Best, >>> Luc >>> >>> On 10/28/2014 12:33 PM, Barry Smith wrote: >>>> Hmm, this should never happen. In the code >>>> >>>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>>> for (i=0; iB->rmap->n; i++) { >>>> for (j=0; jilen[i]; j++) { >>>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>>> >>>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>>> >>>> >>>> Barry >>>> >>>> >>>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>>> wrote: >>>>> >>>>> Hi, >>>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>>> Here is my submission command: >>>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>>> >>>>> Here is "4nodes_32cores_LU": >>>>> #!/bin/sh >>>>> >>>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>>> echo "Cobalt location args: $LOCARGS" >&2 >>>>> >>>>> ################################ >>>>> # 32 cores on 4 nodes jobs # >>>>> ################################ >>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>>> >>>>> I get the following error message: >>>>> >>>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>> [7]PETSC ERROR: Argument out of range >>>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>>> -lblas" - >>>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>>> >>>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>>> >>>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>>> >>>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>>> >>>>> Thanks for the help >>>>> Best, >>>>> Luc >>>>> >>>>> >>>>> > > > From francesco.ballarin at polimi.it Tue Oct 28 15:30:35 2014 From: francesco.ballarin at polimi.it (Francesco Ballarin) Date: Tue, 28 Oct 2014 21:30:35 +0100 Subject: [petsc-users] MatZeroRows and MATELEMENTAL In-Reply-To: References: Message-ID: Thanks Jack, Barry and Matt. I patched my PETSc 3.5.2 as follows, but I am not really sure that this is the correct way (for example, I see in MatSetValues_Elemental two calls to functions P2RO and RO2E for the computation of the local row index but I do not understand what is their purpose, nor if I should apply them also to rowInd[s]). Thanks ====== diff --git a/petsc/src/mat/impls/elemental/matelem.cxx b/petsc/src/mat/impls/elemental/matelem.cxx --- a/petsc/src/mat/impls/elemental/matelem.cxx +++ b/petsc/src/mat/impls/elemental/matelem.cxx @@ -772,6 +772,36 @@ static PetscErrorCode MatZeroEntries_Elemental(Mat A) } #undef __FUNCT__ +#define __FUNCT__ "MatZeroRows_Elemental" +PetscErrorCode MatZeroRows_Elemental(Mat AA,PetscInt N,const PetscInt rows[],PetscScalar diag,Vec /*x*/,Vec /*b*/) +{ + // TODO: consider also the optional arguments x and b. + Mat_Elemental * a( (Mat_Elemental*)AA->data ); + elem::DistMatrix * A( a->emat ); + + PetscFunctionBegin; + std::vector rowInd(rows,rows+N); + const elem::Int localWidth( A->LocalWidth() ); + for( elem::Int s(0); s < N; ++s ) + { + const elem::Int row = rowInd[s]; + if( A->IsLocalRow(row) ) + { + const elem::Int iLoc(A->LocalRow(row)); + for( elem::Int jLoc(0); jLoc < localWidth; ++jLoc ) + { + const elem::Int j(A->GlobalCol(jLoc)); + if( row == j ) + A->SetLocal( iLoc, jLoc, diag ); + else + A->SetLocal( iLoc, jLoc, 0. ); + } + } + } + PetscFunctionReturn(0); +} + +#undef __FUNCT__ #define __FUNCT__ "MatGetOwnershipIS_Elemental" static PetscErrorCode MatGetOwnershipIS_Elemental(Mat A,IS *rows,IS *cols) { @@ -957,7 +987,7 @@ static struct _MatOps MatOps_Values = { MatAssemblyEnd_Elemental, 0, //MatSetOption_Elemental, MatZeroEntries_Elemental, -/*24*/ 0, +/*24*/ MatZeroRows_Elemental, MatLUFactorSymbolic_Elemental, MatLUFactorNumeric_Elemental, MatCholeskyFactorSymbolic_Elemental, ====== On Mon, Oct 27, 2014 at 11:00 PM, Barry Smith wrote: > > > On Oct 26, 2014, at 4:32 AM, Francesco Ballarin < > francesco.ballarin at polimi.it> wrote: > > > > Dear PETSc users, > > I have a code for PETSc 3.3 in which the PLAPACK interface, now > discontinued, was used. I am trying to update it to PETSc 3.5.2, using the > Elemental interface. > > > > I see that MatZeroRows is not implemented for MATELEMENTAL. What > function should I use instead? Does it also support the version with > optional arguments x and b? > > Unfortunately due to Elemental?s complicated parallel storage of matrix > entries we don?t have any routines to do this. (We don?t even have routines > to generically set values INSERT_VALUES) on other processes. cc:ing > Elemental?s author to see what he suggests. > > Barry > > > > > Thanks, > > best regards, > > Francesco > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Wed Oct 29 08:00:17 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Wed, 29 Oct 2014 14:00:17 +0100 Subject: [petsc-users] Problem with Preallocating Message-ID: <2312349.jPYSpgGm9R@asaru> Hello, I try to preallocate a sparse matrix like it was recommended in another posting, but get an error which kind of surprises me. Somehow I think it might be related to the order of assembly calls... My code creates the matrix: MatSetType(_matrixC.matrix, MATSBAIJ); MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); Than I enter a loop to get the number of elements per row. The entire code is a bit too long, but right before the SetPreallocaiton I do: for (int r = 0; r < n; r++) { cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; } which results in the output: Matrix Size = 9 Preallocated row 0 with 9 elements. Preallocated row 1 with 8 elements. Preallocated row 2 with 7 elements. Preallocated row 3 with 6 elements. Preallocated row 4 with 5 elements. Preallocated row 5 with 4 elements. Preallocated row 6 with 1 elements. Preallocated row 7 with 1 elements. Preallocated row 8 with 1 elements. at least in rows 0 to 5 it's a dense upper triangular matrix (incl. main diagonal). ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, nnz); Now more or less the same loops as above with MatSetValues call for each row. Since the KSPSolver requires that the diagonal is set, even if it's merely with zeros I do: _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm sure you get the point ;-) petsc::Vector zeros(_matrixC); MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); [ some more code ] ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); But when I run the code I get an error message: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Argument out of range [0]PETSC ERROR: New nonzero at (0,0) caused a malloc Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn off this check [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by lindnefn Wed Oct 29 13:48:49 2014 [0]PETSC ERROR: Configure options --with-debugging=1 [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c [0]PETSC ERROR: #2 MatSetValues() line 1135 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c I'm surprised that I get this error message since I do have preallocated 9 elements and I do not understand why inserting an element at (0, 0) could be a problem / needing a malloc. Has the preallocation got lost somewhere? Thanks once again.... Florian From jollage at gmail.com Wed Oct 29 08:56:39 2014 From: jollage at gmail.com (Mengqi Zhang) Date: Wed, 29 Oct 2014 11:56:39 -0200 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> References: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> Message-ID: Hi, Jose Thanks for your reply. I will specify the problem more exactly. I want to solve a big generalized eigenvalue problem. The matrices are sparse. My question lies on how to allocate the matrices among the processors. Do I have to do it by myself? Or there is a routine to do so? I notice that one should do something like from petsc4py import PETSc A = PETSc.Mat().create() A.setType('aij') A.setSizes([M,N]) A.setPreallocationNNZ([diag_nz, offdiag_nz]) # optional A.setUp() I have several question regarding these lines. (1) I presume M and N are the dimension of the matrix. Then how do the processors divide the matrix? I guess setPreallocationNNZ does the allocation of the matrix among the processors. What does nz mean here? Why here appears diag and offdiag? (2) I actually saw somewhere else that people use A.setPreallocationNNZ(5), with a single parameter 5. What does 5 mean here? (3) I want to make sure that the matrix so generated is sparse (since it uses aij). It is sparse right? I feel it tricky since if the matrix is stored as sparse, will the allocation/parallelization destroy the efficiency of sparse matrix? After the matrix is set up, I would like to use SLEPc4py to solve the generalized eigenvalue problem. The example code I got online is like E = SLEPc.EPS(); E.create()E.setOperators(A)E.setProblemType(SLEPc.EPS.ProblemType.GNHEP)E.setFromOptions()E.solve() I'm afraid this script is not designed for the parallel computation since in the options there is no indication of parallelization. Do you know how to set it up? Thank you very much for your time. I appreciate it very much. Best, Mengqi 2014-10-26 15:21 GMT-02:00 Jose E. Roman : > > El 26/10/2014, a las 17:10, Mengqi Zhang escribi?: > > > Dear all PETSc users, > > > > I'm new to PETSc/SLEPc. And I'm currently trying to use > Python/PETSc4py/SLEPc4py to solve a big generalized eigenvalue problem, A x > = B lambda x. > > > > After several days' digging and trying, I can write a script for solving > a smaller eigenvalue problem locally, but when the problem size increases, > I realize I should go parallel. My question is how to do it in parallel? > How could I preallocate the matrices among the processors? I presume there > should be a way to do that, but I couldn't find a clear guide on the > Internet. > > > > Currently, the matrices are implemented locally, and then I use > > > > E = SLEPc.EPS(); E.create() > > > > E.setOperators(A,B) > > > > E.setProblemType(SLEPc.EPS.ProblemType.GNHEP) > > > > E.setFromOptions() > > > > to locally solve the eigenvalue problem. > > > > > > > > Could you help me and guide me to set the code up? Any suggestion is > welcomed and appreciated. Thank you very much. > > > > > > > > Best regards, > > > > Mengqi > > > > You may want to do A.setPreallocationNNZ(5) or something like that. > Also, take into account that in generalized eigenproblems you will > probably want to use a parallel linear solver, see discussion in section > 3.4.1 of SLEPc's users guide. > > Jose > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jroman at dsic.upv.es Wed Oct 29 09:23:22 2014 From: jroman at dsic.upv.es (Jose E. Roman) Date: Wed, 29 Oct 2014 15:23:22 +0100 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: References: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> Message-ID: El 29/10/2014, a las 14:56, Mengqi Zhang escribi?: > Hi, Jose > > Thanks for your reply. > I will specify the problem more exactly. I want to solve a big generalized eigenvalue problem. The matrices are sparse. My question lies on how to allocate the matrices among the processors. Do I have to do it by myself? Or there is a routine to do so? I notice that one should do something like > from petsc4py import PETSc > > > A = PETSc.Mat().create() > > > A.setType('aij') > > > A.setSizes([M,N]) > > > A.setPreallocationNNZ([diag_nz, offdiag_nz]) # optional > A.setUp() > I have several question regarding these lines. > (1) I presume M and N are the dimension of the matrix. Then how do the processors divide the matrix? I guess setPreallocationNNZ does the allocation of the matrix among the processors. What does nz mean here? Why here appears diag and offdiag? > (2) I actually saw somewhere else that people use A.setPreallocationNNZ(5), with a single parameter 5. What does 5 mean here? > (3) I want to make sure that the matrix so generated is sparse (since it uses aij). It is sparse right? I feel it tricky since if the matrix is stored as sparse, will the allocation/parallelization destroy the efficiency of sparse matrix? > > > After the matrix is set up, I would like to use SLEPc4py to solve the generalized eigenvalue problem. The example code I got online is like > E = SLEPc.EPS(); E.create() > E.setOperators(A) > E.setProblemType(SLEPc.EPS.ProblemType.GNHEP) > E.setFromOptions() > E.solve() > I'm afraid this script is not designed for the parallel computation since in the options there is no indication of parallelization. Do you know how to set it up? > > Thank you very much for your time. I appreciate it very much. > Best, > Mengqi > Almost all examples under slepc4py/demo are parallel. You can take for instance ex1.py as a basis for your own scripts. In these examples, every process fills its part of the matrix (locally owned rows), as obtained from getOwnershipRange(). See PETSc's documentation here: http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html The meaning of the preallocation arguments is also explained in PETSc's manpages: http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html petsc4py/slepc4py are just python wrappers to PETSc/SLEPc, so you must understand how PETSc and SLEPc work, and dig into their manuals and manpages. Jose From mfadams at lbl.gov Wed Oct 29 09:30:14 2014 From: mfadams at lbl.gov (Mark Adams) Date: Wed, 29 Oct 2014 10:30:14 -0400 Subject: [petsc-users] Multiple solves with PCMG fail In-Reply-To: <87y4s0xp36.fsf@jedbrown.org> References: <9621705.dtimTkejkQ@besikovitch-ii> <5202897.4tZeA0US6V@besikovitch-ii> <7911368.Xkc5LuFApY@besikovitch-ii> <87y4s0xp36.fsf@jedbrown.org> Message-ID: > > Yeah, that should work; an alternative is to defer the estimate until > the next solve. > Sounds better. As I said one of my users has a zero RHS on the first solve. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Oct 29 09:39:43 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 29 Oct 2014 09:39:43 -0500 Subject: [petsc-users] Problem with Preallocating In-Reply-To: <2312349.jPYSpgGm9R@asaru> References: <2312349.jPYSpgGm9R@asaru> Message-ID: Can you send a small example? This looks like it should work. Matt On Wed, Oct 29, 2014 at 8:00 AM, Florian Lindner wrote: > Hello, > > I try to preallocate a sparse matrix like it was recommended in another > posting, but get an error which kind of surprises me. Somehow I think it > might be related to the order of assembly calls... > > My code creates the matrix: > > MatSetType(_matrixC.matrix, MATSBAIJ); > MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); > MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > > Than I enter a loop to get the number of elements per row. The entire > code is a bit too long, but right before the SetPreallocaiton I do: > > for (int r = 0; r < n; r++) { > cout << "Preallocated row " << r << " with " << nnz[r] << " > elements." << endl; > } > > which results in the output: > > Matrix Size = 9 > Preallocated row 0 with 9 elements. > Preallocated row 1 with 8 elements. > Preallocated row 2 with 7 elements. > Preallocated row 3 with 6 elements. > Preallocated row 4 with 5 elements. > Preallocated row 5 with 4 elements. > Preallocated row 6 with 1 elements. > Preallocated row 7 with 1 elements. > Preallocated row 8 with 1 elements. > > at least in rows 0 to 5 it's a dense upper triangular matrix (incl. main > diagonal). > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, > nnz); > > Now more or less the same loops as above with MatSetValues call for each > row. > > Since the KSPSolver requires that the diagonal is set, even if it's merely > with zeros I do: > > _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm > sure you get the point ;-) > petsc::Vector zeros(_matrixC); > MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); > ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); > CHKERRV(ierr); > [ some more code ] > ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); > CHKERRV(ierr); > > But when I run the code I get an error message: > > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: Argument out of range > [0]PETSC ERROR: New nonzero at (0,0) caused a malloc > Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn > off this check > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by > lindnefn Wed Oct 29 13:48:49 2014 > [0]PETSC ERROR: Configure options --with-debugging=1 > [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in > /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c > [0]PETSC ERROR: #2 MatSetValues() line 1135 in > /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in > /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in > /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > > I'm surprised that I get this error message since I do have preallocated 9 > elements and I do not understand why inserting an element at (0, 0) could > be a problem / needing a malloc. Has the preallocation got lost somewhere? > > Thanks once again.... > > Florian > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bisheshkh at gmail.com Wed Oct 29 11:18:50 2014 From: bisheshkh at gmail.com (Bishesh Khanal) Date: Wed, 29 Oct 2014 17:18:50 +0100 Subject: [petsc-users] analysing petsc log information and standard execution time for Stokes like equations Message-ID: Dear all, The computer cluster I'm using to execute my Petsc based code (Stokes-like equation solver) is providing me results with a big variation on the execution times for almost the same problems. When I look at the -log_summary, and the time taken and Mflop/s for Petsc routines it looks like the issue is with cluster. I'd like to confirm this before looking into cluster related issues. I've provided below (after the questions) the relevant -log_summary outputs for two problems (also a file attached if the email does not show up the outputs nicely). The tow problems P1 and P2 solves Stokes-like equation of the same size and using same combination of KSP and PC using 64 processors (8 nodes with 8 proc/node). P1 solves the equation 6 times while P2 only once. The operator is slightly different due to slightly different boundary but since the no. of iterations from ksp_monitor were almost the same, I guess this is not an issue. Now the problem is that P2 case execution was much slower than P1. In other experiments too, the execution is quite fast sometimes but slow most of the other times. I can see in the log_summary that different Petsc routines are running much slower for P2 and has smaller Mflop/s rate. My two questions: 1. Do these outputs confirm that the issue is with the cluster and not with my code ? If yes, what kinds of things I should focus/learn while submitting jobs to the cluster ? Any pointer would be helpful. 2. If I set the equations for the constant viscosity and as: div(grad(u)) - grad(p) = f div(u) + kp = g with k=1 in some regions and 0 in most of the other regions; with f and g are functions spatially varying. and solve the system with 64 to 128 processors using ksp and pc as: -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type hypre What order of execution time for solving this system should I target to be reasonable with around, say 128 processors ? -log_summary output for P1 ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-opt named nef017 with 64 processors, by bkhanal Tue Oct 28 05:52:24 2014 Using Petsc Release Version 3.4.3, Oct, 15, 2013 Max Max/Min Avg Total Time (sec): 4.221e+04 1.00780 4.201e+04 Objects: 9.980e+02 1.00000 9.980e+02 Flops: 2.159e+11 1.08499 2.106e+11 1.348e+13 Flops/sec: 5.154e+06 1.08499 5.013e+06 3.208e+08 MPI Messages: 1.316e+05 3.69736 7.413e+04 4.744e+06 MPI Message Lengths: 1.986e+09 2.61387 1.581e+04 7.502e+10 MPI Reductions: 8.128e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 4.2010e+04 100.0% 1.3477e+13 100.0% 4.744e+06 100.0% 1.581e+04 100.0% 8.127e+03 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecView 6 1.0 3.8704e+01424.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0 VecMDot 3228 1.0 1.0614e+01 1.7 5.93e+09 1.1 0.0e+00 0.0e+00 3.2e+03 0 3 0 0 40 0 3 0 0 40 35184 VecNorm 4383 1.0 4.0579e+01 9.8 2.73e+09 1.1 0.0e+00 0.0e+00 4.4e+03 0 1 0 0 54 0 1 0 0 54 4239 VecScale 4680 1.0 1.2393e+00 1.3 1.39e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 70518 VecCopy 1494 1.0 1.1592e+00 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 5477 1.0 9.7614e+02288.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 1113 1.0 7.9877e-01 2.3 6.17e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 48645 VecAYPX 375 1.0 7.9671e-02 1.4 4.99e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 39491 VecMAXPY 4317 1.0 7.3185e+00 2.1 8.89e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 4 0 0 0 76569 VecScatterBegin 5133 1.0 6.7937e+00 2.4 0.00e+00 0.0 4.7e+06 1.5e+04 1.2e+01 0 0100 96 0 0 0100 96 0 0 VecScatterEnd 5121 1.0 2.5840e+02113.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 3984 1.0 7.2473e+00 1.6 3.92e+09 1.1 0.0e+00 0.0e+00 4.0e+03 0 2 0 0 49 0 2 0 0 49 34118 MatMult 910 1.0 7.5977e+03 1.0 2.11e+11 1.1 4.7e+06 1.5e+04 6.2e+03 18 98 99 92 76 18 98 99 92 76 1736 MatMultAdd 702 1.0 5.5198e+01 6.3 4.98e+09 1.1 6.6e+05 6.2e+03 0.0e+00 0 2 14 5 0 0 2 14 5 0 5624 MatConvert 6 1.0 3.3739e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 MatAssemblyBegin 55 1.0 5.7320e+0319.7 0.00e+00 0.0 0.0e+00 0.0e+00 6.2e+01 11 0 0 0 1 11 0 0 0 1 0 MatAssemblyEnd 55 1.0 1.6179e+00 1.3 0.00e+00 0.0 9.4e+03 3.6e+03 4.0e+01 0 0 0 0 0 0 0 0 0 0 0 MatGetRowIJ 12 1.0 1.3590e-05 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatZeroEntries 20 1.0 6.9117e-01 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 18 1.0 4.7298e-02 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 3.6e+01 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 3228 1.0 1.4727e+01 1.7 1.19e+10 1.1 0.0e+00 0.0e+00 3.2e+03 0 6 0 0 40 0 6 0 0 40 50718 KSPSetUp 18 1.0 5.4130e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 6 1.0 3.2363e+04 1.0 2.15e+11 1.1 4.7e+06 1.5e+04 7.7e+03 77100 99 92 95 77100 99 92 95 415 PCSetUp 18 1.0 2.5724e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 2.0e+02 61 0 0 0 2 61 0 0 0 2 0 PCApply 18 1.0 3.2248e+04 1.0 2.12e+11 1.1 4.7e+06 1.5e+04 7.6e+03 77 98 99 91 93 77 98 99 91 93 411 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 859 849 1446200816 0 Vector Scatter 26 16 16352 0 Matrix 20 20 1832984916 0 Distributed Mesh 3 3 5457592 0 Bipartite Graph 6 6 4848 0 Index Set 61 61 1882648 0 IS L to G Mapping 5 5 3961468 0 Krylov Solver 5 5 57888 0 DMKSP interface 1 1 656 0 Preconditioner 5 5 4440 0 Viewer 7 6 4272 0 ======================================================================================================================== Average time to get PetscTime(): 0 Average time for MPI_Barrier(): 1.27792e-05 Average time for zero size MPI_Send(): 5.20423e-06 #PETSc Option Table entries: -log_summary output for P2: ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-opt named nef001 with 64 processors, by bkhanal Wed Oct 29 14:24:36 2014 Using Petsc Release Version 3.4.3, Oct, 15, 2013 Max Max/Min Avg Total Time (sec): 1.958e+04 1.00194 1.955e+04 Objects: 3.190e+02 1.00000 3.190e+02 Flops: 3.638e+10 1.08499 3.548e+10 2.271e+12 Flops/sec: 1.861e+06 1.08676 1.815e+06 1.161e+08 MPI Messages: 2.253e+04 3.68455 1.270e+04 8.131e+05 MPI Message Lengths: 3.403e+08 2.51345 1.616e+04 1.314e+10 MPI Reductions: 1.544e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.9554e+04 100.0% 2.2709e+12 100.0% 8.131e+05 100.0% 1.616e+04 100.0% 1.543e+03 99.9% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecView 1 1.0 4.4869e+02189.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 0 1 0 0 0 0 0 VecMDot 544 1.0 1.8271e+01 2.2 1.00e+09 1.1 0.0e+00 0.0e+00 5.4e+02 0 3 0 0 35 0 3 0 0 35 3456 VecNorm 738 1.0 2.0433e+0218.1 4.60e+08 1.1 0.0e+00 0.0e+00 7.4e+02 1 1 0 0 48 1 1 0 0 48 142 VecScale 788 1.0 4.1195e+00 9.0 2.34e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 3573 VecCopy 251 1.0 7.6140e+0046.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 926 1.0 3.9087e+0141.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 187 1.0 6.0848e+0032.3 1.04e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1073 VecAYPX 63 1.0 4.6702e-0116.2 8.38e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1131 VecMAXPY 727 1.0 1.0997e+01 4.9 1.50e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 4 0 0 0 8610 VecScatterBegin 864 1.0 2.0978e+0234.1 0.00e+00 0.0 8.0e+05 1.5e+04 2.0e+00 0 0 98 92 0 0 0 98 92 0 0 VecScatterEnd 862 1.0 5.4781e+02114.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 VecNormalize 671 1.0 1.6922e+01 2.2 6.61e+08 1.1 0.0e+00 0.0e+00 6.7e+02 0 2 0 0 43 0 2 0 0 43 2461 MatMult 152 1.0 6.3271e+03 1.0 3.56e+10 1.1 7.9e+05 1.5e+04 1.0e+03 32 98 98 89 68 32 98 98 89 68 351 MatMultAdd 118 1.0 4.5234e+02183.7 8.36e+08 1.1 1.1e+05 6.2e+03 0.0e+00 1 2 14 5 0 1 2 14 5 0 115 MatConvert 1 1.0 3.6065e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 MatAssemblyBegin 10 1.0 1.0849e+03 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 5 0 0 0 1 5 0 0 0 1 0 MatAssemblyEnd 10 1.0 1.3957e+01 1.1 0.00e+00 0.0 9.4e+03 3.6e+03 4.0e+01 0 0 1 0 3 0 0 1 0 3 0 MatGetRowIJ 2 1.0 2.2221e-03582.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 1.0 3.7378e-01 9.8 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 544 1.0 2.0370e+01 1.7 2.00e+09 1.1 0.0e+00 0.0e+00 5.4e+02 0 6 0 0 35 0 6 0 0 35 6200 KSPSetUp 3 1.0 4.2598e+01 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 1 0 0 0 0 1 0 KSPSolve 1 1.0 1.5113e+04 1.0 3.63e+10 1.1 7.9e+05 1.5e+04 1.3e+03 77100 98 89 84 77100 98 89 84 150 PCSetUp 3 1.0 1.1794e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 1.3e+02 60 0 1 2 8 60 0 1 2 8 0 PCApply 3 1.0 1.4940e+04 1.0 3.58e+10 1.1 7.9e+05 1.5e+04 1.3e+03 76 98 97 88 83 76 98 97 88 83 149 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 215 215 743212688 0 Vector Scatter 16 16 16352 0 Matrix 20 20 1832984916 0 Distributed Mesh 3 3 5457592 0 Bipartite Graph 6 6 4848 0 Index Set 41 41 1867368 0 IS L to G Mapping 5 5 3961468 0 Krylov Solver 5 5 57888 0 DMKSP interface 1 1 656 0 Preconditioner 5 5 4440 0 Viewer 2 1 712 0 ======================================================================================================================== Average time to get PetscTime(): 9.53674e-08 Average time for MPI_Barrier(): 0.000274992 Average time for zero size MPI_Send(): 1.67042e-05 #PETSc Option Table entries: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- P1: ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-opt named nef017 with 64 processors, by bkhanal Tue Oct 28 05:52:24 2014 Using Petsc Release Version 3.4.3, Oct, 15, 2013 Max Max/Min Avg Total Time (sec): 4.221e+04 1.00780 4.201e+04 Objects: 9.980e+02 1.00000 9.980e+02 Flops: 2.159e+11 1.08499 2.106e+11 1.348e+13 Flops/sec: 5.154e+06 1.08499 5.013e+06 3.208e+08 MPI Messages: 1.316e+05 3.69736 7.413e+04 4.744e+06 MPI Message Lengths: 1.986e+09 2.61387 1.581e+04 7.502e+10 MPI Reductions: 8.128e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 4.2010e+04 100.0% 1.3477e+13 100.0% 4.744e+06 100.0% 1.581e+04 100.0% 8.127e+03 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecView 6 1.0 3.8704e+01424.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0 VecMDot 3228 1.0 1.0614e+01 1.7 5.93e+09 1.1 0.0e+00 0.0e+00 3.2e+03 0 3 0 0 40 0 3 0 0 40 35184 VecNorm 4383 1.0 4.0579e+01 9.8 2.73e+09 1.1 0.0e+00 0.0e+00 4.4e+03 0 1 0 0 54 0 1 0 0 54 4239 VecScale 4680 1.0 1.2393e+00 1.3 1.39e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 70518 VecCopy 1494 1.0 1.1592e+00 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 5477 1.0 9.7614e+02288.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 1113 1.0 7.9877e-01 2.3 6.17e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 48645 VecAYPX 375 1.0 7.9671e-02 1.4 4.99e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 39491 VecMAXPY 4317 1.0 7.3185e+00 2.1 8.89e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 4 0 0 0 76569 VecScatterBegin 5133 1.0 6.7937e+00 2.4 0.00e+00 0.0 4.7e+06 1.5e+04 1.2e+01 0 0100 96 0 0 0100 96 0 0 VecScatterEnd 5121 1.0 2.5840e+02113.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 3984 1.0 7.2473e+00 1.6 3.92e+09 1.1 0.0e+00 0.0e+00 4.0e+03 0 2 0 0 49 0 2 0 0 49 34118 MatMult 910 1.0 7.5977e+03 1.0 2.11e+11 1.1 4.7e+06 1.5e+04 6.2e+03 18 98 99 92 76 18 98 99 92 76 1736 MatMultAdd 702 1.0 5.5198e+01 6.3 4.98e+09 1.1 6.6e+05 6.2e+03 0.0e+00 0 2 14 5 0 0 2 14 5 0 5624 MatConvert 6 1.0 3.3739e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 MatAssemblyBegin 55 1.0 5.7320e+0319.7 0.00e+00 0.0 0.0e+00 0.0e+00 6.2e+01 11 0 0 0 1 11 0 0 0 1 0 MatAssemblyEnd 55 1.0 1.6179e+00 1.3 0.00e+00 0.0 9.4e+03 3.6e+03 4.0e+01 0 0 0 0 0 0 0 0 0 0 0 MatGetRowIJ 12 1.0 1.3590e-05 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatZeroEntries 20 1.0 6.9117e-01 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 18 1.0 4.7298e-02 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 3.6e+01 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 3228 1.0 1.4727e+01 1.7 1.19e+10 1.1 0.0e+00 0.0e+00 3.2e+03 0 6 0 0 40 0 6 0 0 40 50718 KSPSetUp 18 1.0 5.4130e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 6 1.0 3.2363e+04 1.0 2.15e+11 1.1 4.7e+06 1.5e+04 7.7e+03 77100 99 92 95 77100 99 92 95 415 PCSetUp 18 1.0 2.5724e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 2.0e+02 61 0 0 0 2 61 0 0 0 2 0 PCApply 18 1.0 3.2248e+04 1.0 2.12e+11 1.1 4.7e+06 1.5e+04 7.6e+03 77 98 99 91 93 77 98 99 91 93 411 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 859 849 1446200816 0 Vector Scatter 26 16 16352 0 Matrix 20 20 1832984916 0 Distributed Mesh 3 3 5457592 0 Bipartite Graph 6 6 4848 0 Index Set 61 61 1882648 0 IS L to G Mapping 5 5 3961468 0 Krylov Solver 5 5 57888 0 DMKSP interface 1 1 656 0 Preconditioner 5 5 4440 0 Viewer 7 6 4272 0 ======================================================================================================================== Average time to get PetscTime(): 0 Average time for MPI_Barrier(): 1.27792e-05 Average time for zero size MPI_Send(): 5.20423e-06 #PETSc Option Table entries: ... P2: ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a arch-linux2-cxx-opt named nef001 with 64 processors, by bkhanal Wed Oct 29 14:24:36 2014 Using Petsc Release Version 3.4.3, Oct, 15, 2013 Max Max/Min Avg Total Time (sec): 1.958e+04 1.00194 1.955e+04 Objects: 3.190e+02 1.00000 3.190e+02 Flops: 3.638e+10 1.08499 3.548e+10 2.271e+12 Flops/sec: 1.861e+06 1.08676 1.815e+06 1.161e+08 MPI Messages: 2.253e+04 3.68455 1.270e+04 8.131e+05 MPI Message Lengths: 3.403e+08 2.51345 1.616e+04 1.314e+10 MPI Reductions: 1.544e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 1.9554e+04 100.0% 2.2709e+12 100.0% 8.131e+05 100.0% 1.616e+04 100.0% 1.543e+03 99.9% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %f - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecView 1 1.0 4.4869e+02189.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 0 1 0 0 0 0 0 VecMDot 544 1.0 1.8271e+01 2.2 1.00e+09 1.1 0.0e+00 0.0e+00 5.4e+02 0 3 0 0 35 0 3 0 0 35 3456 VecNorm 738 1.0 2.0433e+0218.1 4.60e+08 1.1 0.0e+00 0.0e+00 7.4e+02 1 1 0 0 48 1 1 0 0 48 142 VecScale 788 1.0 4.1195e+00 9.0 2.34e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 3573 VecCopy 251 1.0 7.6140e+0046.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 926 1.0 3.9087e+0141.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 187 1.0 6.0848e+0032.3 1.04e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1073 VecAYPX 63 1.0 4.6702e-0116.2 8.38e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1131 VecMAXPY 727 1.0 1.0997e+01 4.9 1.50e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 0 4 0 0 0 8610 VecScatterBegin 864 1.0 2.0978e+0234.1 0.00e+00 0.0 8.0e+05 1.5e+04 2.0e+00 0 0 98 92 0 0 0 98 92 0 0 VecScatterEnd 862 1.0 5.4781e+02114.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 VecNormalize 671 1.0 1.6922e+01 2.2 6.61e+08 1.1 0.0e+00 0.0e+00 6.7e+02 0 2 0 0 43 0 2 0 0 43 2461 MatMult 152 1.0 6.3271e+03 1.0 3.56e+10 1.1 7.9e+05 1.5e+04 1.0e+03 32 98 98 89 68 32 98 98 89 68 351 MatMultAdd 118 1.0 4.5234e+02183.7 8.36e+08 1.1 1.1e+05 6.2e+03 0.0e+00 1 2 14 5 0 1 2 14 5 0 115 MatConvert 1 1.0 3.6065e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 MatAssemblyBegin 10 1.0 1.0849e+03 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 5 0 0 0 1 5 0 0 0 1 0 MatAssemblyEnd 10 1.0 1.3957e+01 1.1 0.00e+00 0.0 9.4e+03 3.6e+03 4.0e+01 0 0 1 0 3 0 0 1 0 3 0 MatGetRowIJ 2 1.0 2.2221e-03582.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 1.0 3.7378e-01 9.8 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPGMRESOrthog 544 1.0 2.0370e+01 1.7 2.00e+09 1.1 0.0e+00 0.0e+00 5.4e+02 0 6 0 0 35 0 6 0 0 35 6200 KSPSetUp 3 1.0 4.2598e+01 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 1 0 0 0 0 1 0 KSPSolve 1 1.0 1.5113e+04 1.0 3.63e+10 1.1 7.9e+05 1.5e+04 1.3e+03 77100 98 89 84 77100 98 89 84 150 PCSetUp 3 1.0 1.1794e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 1.3e+02 60 0 1 2 8 60 0 1 2 8 0 PCApply 3 1.0 1.4940e+04 1.0 3.58e+10 1.1 7.9e+05 1.5e+04 1.3e+03 76 98 97 88 83 76 98 97 88 83 149 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 215 215 743212688 0 Vector Scatter 16 16 16352 0 Matrix 20 20 1832984916 0 Distributed Mesh 3 3 5457592 0 Bipartite Graph 6 6 4848 0 Index Set 41 41 1867368 0 IS L to G Mapping 5 5 3961468 0 Krylov Solver 5 5 57888 0 DMKSP interface 1 1 656 0 Preconditioner 5 5 4440 0 Viewer 2 1 712 0 ======================================================================================================================== Average time to get PetscTime(): 9.53674e-08 Average time for MPI_Barrier(): 0.000274992 Average time for zero size MPI_Send(): 1.67042e-05 #PETSc Option Table entries: ... From knepley at gmail.com Wed Oct 29 11:35:14 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 29 Oct 2014 11:35:14 -0500 Subject: [petsc-users] analysing petsc log information and standard execution time for Stokes like equations In-Reply-To: References: Message-ID: On Wed, Oct 29, 2014 at 11:18 AM, Bishesh Khanal wrote: > Dear all, > The computer cluster I'm using to execute my Petsc based code (Stokes-like > equation solver) is providing me results with a big variation on the > execution times for almost the same problems. > When I look at the -log_summary, and the time taken and Mflop/s for Petsc > routines it looks like the issue is with cluster. > I'd like to confirm this before looking into cluster related issues. > I've provided below (after the questions) the relevant -log_summary > outputs for two problems (also a file attached if the email does not show > up the outputs nicely). > > The tow problems P1 and P2 solves Stokes-like equation of the same size > and using same combination of KSP and PC using 64 processors (8 nodes with > 8 proc/node). > P1 solves the equation 6 times while P2 only once. > The operator is slightly different due to slightly different boundary but > since the no. of iterations from ksp_monitor were almost the same, I guess > this is not an issue. > > Now the problem is that P2 case execution was much slower than P1. In > other experiments too, the execution is quite fast sometimes but slow most > of the other times. > > I can see in the log_summary that different Petsc routines are running > much slower for P2 and has smaller Mflop/s rate. > > My two questions: > 1. Do these outputs confirm that the issue is with the cluster and not > with my code ? If yes, what kinds of things I should focus/learn while > submitting jobs to the cluster ? Any pointer would be helpful. > There are a lot of variables controlling performance on a cluster. You will have to narrow this down little by little. However, there are enormous differences below in the simplest operations, e.g. VecScale, but roughly the same number of flops were computed. Thus, something is going very wrong. My first two guesses, and things you should eliminate 1) Someone else was running at the same time 2) The mapping from processes to nodes was very different You can usually control 2 with your submission system or mpiexec call. Thanks, Matt > 2. If I set the equations for the constant viscosity and as: > div(grad(u)) - grad(p) = f > div(u) + kp = g > with k=1 in some regions and 0 in most of the other regions; with f and g > are functions spatially varying. > and solve the system with 64 to 128 processors using ksp and pc as: > -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition self > -pc_fieldsplit_dm_splits 0 -pc_fieldsplit_0_fields 0,1,2 > -pc_fieldsplit_1_fields 3 -fieldsplit_0_pc_type hypre > > What order of execution time for solving this system should I target to be > reasonable with around, say 128 processors ? > > -log_summary output for P1 > ---------------------------------------------- PETSc Performance Summary: > ---------------------------------------------- > > /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a > arch-linux2-cxx-opt named nef017 with 64 processors, by bkhanal Tue Oct 28 > 05:52:24 2014 > Using Petsc Release Version 3.4.3, Oct, 15, 2013 > > Max Max/Min Avg Total > Time (sec): 4.221e+04 1.00780 4.201e+04 > Objects: 9.980e+02 1.00000 9.980e+02 > Flops: 2.159e+11 1.08499 2.106e+11 1.348e+13 > Flops/sec: 5.154e+06 1.08499 5.013e+06 3.208e+08 > MPI Messages: 1.316e+05 3.69736 7.413e+04 4.744e+06 > MPI Message Lengths: 1.986e+09 2.61387 1.581e+04 7.502e+10 > MPI Reductions: 8.128e+03 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N > --> 2N flops > and VecAXPY() for complex vectors of length N > --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts > %Total Avg %Total counts %Total > 0: Main Stage: 4.2010e+04 100.0% 1.3477e+13 100.0% 4.744e+06 > 100.0% 1.581e+04 100.0% 8.127e+03 100.0% > > > ------------------------------------------------------------------------------------------------------------------------ > See the 'Profiling' chapter of the users' manual for details on > interpreting output. > Phase summary info: > Count: number of times phase was executed > Time and Flops: Max - maximum over all processors > Ratio - ratio of maximum to minimum over all processors > Mess: number of messages sent > Avg. len: average message length (bytes) > Reduct: number of global reductions > Global: entire computation > Stage: stages of a computation. Set stages with PetscLogStagePush() and > PetscLogStagePop(). > %T - percent time in this phase %f - percent flops in this > phase > %M - percent messages in this phase %L - percent message lengths > in this phase > %R - percent reductions in this phase > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > over all processors) > > ------------------------------------------------------------------------------------------------------------------------ > Event Count Time (sec) > Flops --- Global --- --- Stage --- Total > Max Ratio Max Ratio Max Ratio Mess Avg len > Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s > > ------------------------------------------------------------------------------------------------------------------------ > > --- Event Stage 0: Main Stage > > VecView 6 1.0 3.8704e+01424.2 0.00e+00 0.0 0.0e+00 0.0e+00 > 1.2e+01 0 0 0 0 0 0 0 0 0 0 0 > VecMDot 3228 1.0 1.0614e+01 1.7 5.93e+09 1.1 0.0e+00 0.0e+00 > 3.2e+03 0 3 0 0 40 0 3 0 0 40 35184 > VecNorm 4383 1.0 4.0579e+01 9.8 2.73e+09 1.1 0.0e+00 0.0e+00 > 4.4e+03 0 1 0 0 54 0 1 0 0 54 4239 > VecScale 4680 1.0 1.2393e+00 1.3 1.39e+09 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 70518 > VecCopy 1494 1.0 1.1592e+00 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecSet 5477 1.0 9.7614e+02288.4 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecAXPY 1113 1.0 7.9877e-01 2.3 6.17e+08 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 48645 > VecAYPX 375 1.0 7.9671e-02 1.4 4.99e+07 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 39491 > VecMAXPY 4317 1.0 7.3185e+00 2.1 8.89e+09 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 4 0 0 0 0 4 0 0 0 76569 > VecScatterBegin 5133 1.0 6.7937e+00 2.4 0.00e+00 0.0 4.7e+06 1.5e+04 > 1.2e+01 0 0100 96 0 0 0100 96 0 0 > VecScatterEnd 5121 1.0 2.5840e+02113.4 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecNormalize 3984 1.0 7.2473e+00 1.6 3.92e+09 1.1 0.0e+00 0.0e+00 > 4.0e+03 0 2 0 0 49 0 2 0 0 49 34118 > MatMult 910 1.0 7.5977e+03 1.0 2.11e+11 1.1 4.7e+06 1.5e+04 > 6.2e+03 18 98 99 92 76 18 98 99 92 76 1736 > MatMultAdd 702 1.0 5.5198e+01 6.3 4.98e+09 1.1 6.6e+05 6.2e+03 > 0.0e+00 0 2 14 5 0 0 2 14 5 0 5624 > MatConvert 6 1.0 3.3739e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > MatAssemblyBegin 55 1.0 5.7320e+0319.7 0.00e+00 0.0 0.0e+00 0.0e+00 > 6.2e+01 11 0 0 0 1 11 0 0 0 1 0 > MatAssemblyEnd 55 1.0 1.6179e+00 1.3 0.00e+00 0.0 9.4e+03 3.6e+03 > 4.0e+01 0 0 0 0 0 0 0 0 0 0 0 > MatGetRowIJ 12 1.0 1.3590e-05 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatZeroEntries 20 1.0 6.9117e-01 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatView 18 1.0 4.7298e-02 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 > 3.6e+01 0 0 0 0 0 0 0 0 0 0 0 > KSPGMRESOrthog 3228 1.0 1.4727e+01 1.7 1.19e+10 1.1 0.0e+00 0.0e+00 > 3.2e+03 0 6 0 0 40 0 6 0 0 40 50718 > KSPSetUp 18 1.0 5.4130e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 > 1.0e+01 0 0 0 0 0 0 0 0 0 0 0 > KSPSolve 6 1.0 3.2363e+04 1.0 2.15e+11 1.1 4.7e+06 1.5e+04 > 7.7e+03 77100 99 92 95 77100 99 92 95 415 > PCSetUp 18 1.0 2.5724e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 > 2.0e+02 61 0 0 0 2 61 0 0 0 2 0 > PCApply 18 1.0 3.2248e+04 1.0 2.12e+11 1.1 4.7e+06 1.5e+04 > 7.6e+03 77 98 99 91 93 77 98 99 91 93 411 > > ------------------------------------------------------------------------------------------------------------------------ > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Vector 859 849 1446200816 0 > Vector Scatter 26 16 16352 0 > Matrix 20 20 1832984916 0 > Distributed Mesh 3 3 5457592 0 > Bipartite Graph 6 6 4848 0 > Index Set 61 61 1882648 0 > IS L to G Mapping 5 5 3961468 0 > Krylov Solver 5 5 57888 0 > DMKSP interface 1 1 656 0 > Preconditioner 5 5 4440 0 > Viewer 7 6 4272 0 > > ======================================================================================================================== > Average time to get PetscTime(): 0 > Average time for MPI_Barrier(): 1.27792e-05 > Average time for zero size MPI_Send(): 5.20423e-06 > #PETSc Option Table entries: > > > > > -log_summary output for P2: > ---------------------------------------------- PETSc Performance Summary: > ---------------------------------------------- > > /epi/asclepios2/bkhanal/works/AdLemModel/build/src/AdLemMain on a > arch-linux2-cxx-opt named nef001 with 64 processors, by bkhanal Wed Oct 29 > 14:24:36 2014 > Using Petsc Release Version 3.4.3, Oct, 15, 2013 > > Max Max/Min Avg Total > Time (sec): 1.958e+04 1.00194 1.955e+04 > Objects: 3.190e+02 1.00000 3.190e+02 > Flops: 3.638e+10 1.08499 3.548e+10 2.271e+12 > Flops/sec: 1.861e+06 1.08676 1.815e+06 1.161e+08 > MPI Messages: 2.253e+04 3.68455 1.270e+04 8.131e+05 > MPI Message Lengths: 3.403e+08 2.51345 1.616e+04 1.314e+10 > MPI Reductions: 1.544e+03 1.00000 > > Flop counting convention: 1 flop = 1 real number operation of type > (multiply/divide/add/subtract) > e.g., VecAXPY() for real vectors of length N > --> 2N flops > and VecAXPY() for complex vectors of length N > --> 8N flops > > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages > --- -- Message Lengths -- -- Reductions -- > Avg %Total Avg %Total counts > %Total Avg %Total counts %Total > 0: Main Stage: 1.9554e+04 100.0% 2.2709e+12 100.0% 8.131e+05 > 100.0% 1.616e+04 100.0% 1.543e+03 99.9% > > > ------------------------------------------------------------------------------------------------------------------------ > See the 'Profiling' chapter of the users' manual for details on > interpreting output. > Phase summary info: > Count: number of times phase was executed > Time and Flops: Max - maximum over all processors > Ratio - ratio of maximum to minimum over all processors > Mess: number of messages sent > Avg. len: average message length (bytes) > Reduct: number of global reductions > Global: entire computation > Stage: stages of a computation. Set stages with PetscLogStagePush() and > PetscLogStagePop(). > %T - percent time in this phase %f - percent flops in this > phase > %M - percent messages in this phase %L - percent message lengths > in this phase > %R - percent reductions in this phase > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time > over all processors) > > ------------------------------------------------------------------------------------------------------------------------ > Event Count Time (sec) > Flops --- Global --- --- Stage --- Total > Max Ratio Max Ratio Max Ratio Mess Avg len > Reduct %T %f %M %L %R %T %f %M %L %R Mflop/s > > ------------------------------------------------------------------------------------------------------------------------ > > --- Event Stage 0: Main Stage > > VecView 1 1.0 4.4869e+02189.5 0.00e+00 0.0 0.0e+00 0.0e+00 > 2.0e+00 1 0 0 0 0 1 0 0 0 0 0 > VecMDot 544 1.0 1.8271e+01 2.2 1.00e+09 1.1 0.0e+00 0.0e+00 > 5.4e+02 0 3 0 0 35 0 3 0 0 35 3456 > VecNorm 738 1.0 2.0433e+0218.1 4.60e+08 1.1 0.0e+00 0.0e+00 > 7.4e+02 1 1 0 0 48 1 1 0 0 48 142 > VecScale 788 1.0 4.1195e+00 9.0 2.34e+08 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 1 0 0 0 0 1 0 0 0 3573 > VecCopy 251 1.0 7.6140e+0046.9 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecSet 926 1.0 3.9087e+0141.5 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > VecAXPY 187 1.0 6.0848e+0032.3 1.04e+08 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 1073 > VecAYPX 63 1.0 4.6702e-0116.2 8.38e+06 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 1131 > VecMAXPY 727 1.0 1.0997e+01 4.9 1.50e+09 1.1 0.0e+00 0.0e+00 > 0.0e+00 0 4 0 0 0 0 4 0 0 0 8610 > VecScatterBegin 864 1.0 2.0978e+0234.1 0.00e+00 0.0 8.0e+05 1.5e+04 > 2.0e+00 0 0 98 92 0 0 0 98 92 0 0 > VecScatterEnd 862 1.0 5.4781e+02114.5 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 > VecNormalize 671 1.0 1.6922e+01 2.2 6.61e+08 1.1 0.0e+00 0.0e+00 > 6.7e+02 0 2 0 0 43 0 2 0 0 43 2461 > MatMult 152 1.0 6.3271e+03 1.0 3.56e+10 1.1 7.9e+05 1.5e+04 > 1.0e+03 32 98 98 89 68 32 98 98 89 68 351 > MatMultAdd 118 1.0 4.5234e+02183.7 8.36e+08 1.1 1.1e+05 6.2e+03 > 0.0e+00 1 2 14 5 0 1 2 14 5 0 115 > MatConvert 1 1.0 3.6065e+02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 > MatAssemblyBegin 10 1.0 1.0849e+03 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 > 1.2e+01 5 0 0 0 1 5 0 0 0 1 0 > MatAssemblyEnd 10 1.0 1.3957e+01 1.1 0.00e+00 0.0 9.4e+03 3.6e+03 > 4.0e+01 0 0 1 0 3 0 0 1 0 3 0 > MatGetRowIJ 2 1.0 2.2221e-03582.5 0.00e+00 0.0 0.0e+00 0.0e+00 > 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 > MatView 3 1.0 3.7378e-01 9.8 0.00e+00 0.0 0.0e+00 0.0e+00 > 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 > KSPGMRESOrthog 544 1.0 2.0370e+01 1.7 2.00e+09 1.1 0.0e+00 0.0e+00 > 5.4e+02 0 6 0 0 35 0 6 0 0 35 6200 > KSPSetUp 3 1.0 4.2598e+01 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 > 1.0e+01 0 0 0 0 1 0 0 0 0 1 0 > KSPSolve 1 1.0 1.5113e+04 1.0 3.63e+10 1.1 7.9e+05 1.5e+04 > 1.3e+03 77100 98 89 84 77100 98 89 84 150 > PCSetUp 3 1.0 1.1794e+04 1.0 0.00e+00 0.0 7.7e+03 3.3e+04 > 1.3e+02 60 0 1 2 8 60 0 1 2 8 0 > PCApply 3 1.0 1.4940e+04 1.0 3.58e+10 1.1 7.9e+05 1.5e+04 > 1.3e+03 76 98 97 88 83 76 98 97 88 83 149 > > ------------------------------------------------------------------------------------------------------------------------ > > Memory usage is given in bytes: > > Object Type Creations Destructions Memory Descendants' Mem. > Reports information only for process 0. > > --- Event Stage 0: Main Stage > > Vector 215 215 743212688 0 > Vector Scatter 16 16 16352 0 > Matrix 20 20 1832984916 0 > Distributed Mesh 3 3 5457592 0 > Bipartite Graph 6 6 4848 0 > Index Set 41 41 1867368 0 > IS L to G Mapping 5 5 3961468 0 > Krylov Solver 5 5 57888 0 > DMKSP interface 1 1 656 0 > Preconditioner 5 5 4440 0 > Viewer 2 1 712 0 > > ======================================================================================================================== > Average time to get PetscTime(): 9.53674e-08 > Average time for MPI_Barrier(): 0.000274992 > Average time for zero size MPI_Send(): 1.67042e-05 > #PETSc Option Table entries: > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 29 13:00:17 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 29 Oct 2014 13:00:17 -0500 Subject: [petsc-users] Problem with Preallocating In-Reply-To: <2312349.jPYSpgGm9R@asaru> References: <2312349.jPYSpgGm9R@asaru> Message-ID: <5E018295-7CE9-4AAF-AA56-F25143BFE462@mcs.anl.gov> MatAssembly flushes out any preallocated space that was not used. Hence when you try to put a diagonal entry in later it is as if you did not preallocate. You should set zeros in the diagonal as part of your initial setting values in the matrix before calling MatAssembly. Barry > On Oct 29, 2014, at 8:00 AM, Florian Lindner wrote: > > Hello, > > I try to preallocate a sparse matrix like it was recommended in another posting, but get an error which kind of surprises me. Somehow I think it might be related to the order of assembly calls... > > My code creates the matrix: > > MatSetType(_matrixC.matrix, MATSBAIJ); > MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); > MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > > Than I enter a loop to get the number of elements per row. The entire code is a bit too long, but right before the SetPreallocaiton I do: > > for (int r = 0; r < n; r++) { > cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; > } > > which results in the output: > > Matrix Size = 9 > Preallocated row 0 with 9 elements. > Preallocated row 1 with 8 elements. > Preallocated row 2 with 7 elements. > Preallocated row 3 with 6 elements. > Preallocated row 4 with 5 elements. > Preallocated row 5 with 4 elements. > Preallocated row 6 with 1 elements. > Preallocated row 7 with 1 elements. > Preallocated row 8 with 1 elements. > > at least in rows 0 to 5 it's a dense upper triangular matrix (incl. main diagonal). > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, nnz); > > Now more or less the same loops as above with MatSetValues call for each row. > > Since the KSPSolver requires that the diagonal is set, even if it's merely with zeros I do: > > _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm sure you get the point ;-) > petsc::Vector zeros(_matrixC); > MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); > ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); > [ some more code ] > ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); > > But when I run the code I get an error message: > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > [0]PETSC ERROR: Argument out of range > [0]PETSC ERROR: New nonzero at (0,0) caused a malloc > Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn off this check > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by lindnefn Wed Oct 29 13:48:49 2014 > [0]PETSC ERROR: Configure options --with-debugging=1 > [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c > [0]PETSC ERROR: #2 MatSetValues() line 1135 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > > I'm surprised that I get this error message since I do have preallocated 9 elements and I do not understand why inserting an element at (0, 0) could be a problem / needing a malloc. Has the preallocation got lost somewhere? > > Thanks once again.... > > Florian From jychang48 at gmail.com Wed Oct 29 17:28:29 2014 From: jychang48 at gmail.com (Justin Chang) Date: Wed, 29 Oct 2014 17:28:29 -0500 Subject: [petsc-users] Using DMPlexProjectFunction on certain mesh points Message-ID: Hi all, Is there a way to have DMPlexProjectFunction() work only on select DMPlex mesh points? Kind of like the way DMPlexAddBoundary() works where it projects the boundary values on certain mesh points with certain label values and ID's. Also, I don't see anywhere in the online documentation regarding DMPlexAddBoundary. I think I saw an old post in the archives where it talks about what this function does, but can someone perhaps explain what each argument corresponds with? Thanks, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zonexo at gmail.com Wed Oct 29 20:52:38 2014 From: zonexo at gmail.com (TAY wee-beng) Date: Thu, 30 Oct 2014 09:52:38 +0800 Subject: [petsc-users] Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type Message-ID: <545199E6.7040706@gmail.com> Hi, When compiling using gnu gfortran, I got the following error msg: Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type My code is : /module PETSc_solvers// // //use set_matrix// // //use ibm// // //implicit none// // //contains// // //subroutine semi_momentum_simple_xyz(du,dv,dw)// // //!discretise momentum eqn using semi-implicit mtd// // //!simple except east,west// // //#include "finclude/petsc.h90"// // //...// // //call KSPSetTolerances(ksp_semi_xyz,tol,PETSC_DEFAULT_REAL,PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,ierr) -> error/ There's no problem when using ifort though. -- Thank you Yours sincerely, TAY wee-beng -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Wed Oct 29 20:53:47 2014 From: knepley at gmail.com (Matthew Knepley) Date: Wed, 29 Oct 2014 20:53:47 -0500 Subject: [petsc-users] Using DMPlexProjectFunction on certain mesh points In-Reply-To: References: Message-ID: On Wed, Oct 29, 2014 at 5:28 PM, Justin Chang wrote: > Hi all, > > Is there a way to have DMPlexProjectFunction() work only on select DMPlex > mesh points? Kind of like the way DMPlexAddBoundary() works where it > projects the boundary values on certain mesh points with certain label > values and ID's. > I think what you want is DMPlexProjectFunctionLabelLocal() > Also, I don't see anywhere in the online documentation regarding > DMPlexAddBoundary. I think I saw an old post in the archives where it talks > about what this function does, but can someone perhaps explain what each > argument corresponds with? Thanks, > > Justin > I have not documented it yet since I am not sure its exactly what we want, but I will add it now: + dm - The DM . isEssential - Flag for essential boundary conditions, e.g. Dirichlet, rather than natural, e.g. Neumann . name - The boundary condition name . lablename - Name of the label marking points with constraints . field - DM field constrained by the condition . bcFunc - Function giving boundary values . numids - The number of label ids for constrained points . ids - An array of label ids for constrained points - ctx - A user context which is passed to bcFunc PetscErrorCode DMPlexAddBoundary(DM dm, PetscBool isEssential, const char name[], const char labelname[], PetscInt field, void (*bcFunc)(), PetscInt numids, const PetscInt *ids, void *ctx) Thanks, Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Wed Oct 29 21:02:40 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Wed, 29 Oct 2014 21:02:40 -0500 Subject: [petsc-users] Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type In-Reply-To: <545199E6.7040706@gmail.com> References: <545199E6.7040706@gmail.com> Message-ID: <1F860363-1461-4A41-A1CA-4B94C88070DB@mcs.anl.gov> Hmm, could you send us a (simple) code that reproduces the problem; I don?t think we have a petsc_default_real anywhere. Barry > On Oct 29, 2014, at 8:52 PM, TAY wee-beng wrote: > > Hi, > > When compiling using gnu gfortran, I got the following error msg: > > Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type > > My code is : > > module PETSc_solvers > > use set_matrix > > use ibm > > implicit none > > contains > > subroutine semi_momentum_simple_xyz(du,dv,dw) > > !discretise momentum eqn using semi-implicit mtd > > !simple except east,west > > #include "finclude/petsc.h90" > > ... > > call KSPSetTolerances(ksp_semi_xyz,tol,PETSC_DEFAULT_REAL,PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,ierr) -> error > > There's no problem when using ifort though. > -- > Thank you > > Yours sincerely, > > TAY wee-beng > From jollage at gmail.com Wed Oct 29 21:30:30 2014 From: jollage at gmail.com (Mengqi Zhang) Date: Thu, 30 Oct 2014 00:30:30 -0200 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: References: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> Message-ID: Hi again, I read the demo's and also consulted the src folder. Now, the PETSc part assembling the matrices is clear to me. I put together a very simple example by modifying the ex1.py. My code is attached. The problem now for me is that it seems I can't switch the SLEPc.EPS to a parallel one. I tried to run the attached code on my laptop which has 2 physical cores. The results of parallel running are (I use mpirun -np 2 python matricesEigenvalue.py to run parallelly) from 0 to 8192 on rank 0 # The matrix is of 128*128, the first half is allocated on rank 0 from 8192 to 16384 on rank 1 # so it's from 0 to 8192, the rank 1 for the rest ****************************** Using SLEPc4py for solving the EVP ****************************** Elapsed time of EPS is 4.9312 on rank 0 ****************************** *** SLEPc Solution Results *** ****************************** Number of iterations of the method: 90 Solution method: krylovschur Number of requested eigenvalues: 3 Stopping condition: tol=1e-08, maxit=1820 Number of converged eigenpairs 3 k ||Ax-kx||/||kx|| ----------------- ------------------ 129012.869061 7.58376e-09 128984.178324 7.86259e-09 128955.487588 8.10532e-09 while, the results of serial running are from 0 to 16384 on rank 0 ****************************** Using SLEPc4py for solving the EVP ****************************** Elapsed time of EPS is 4.5439 on rank 0 ****************************** *** SLEPc Solution Results *** ****************************** Number of iterations of the method: 90 Solution method: krylovschur Number of requested eigenvalues: 3 Stopping condition: tol=1e-08, maxit=1820 Number of converged eigenpairs 3 k ||Ax-kx||/||kx|| ----------------- ------------------ 129012.869061 7.58376e-09 128984.178324 7.86259e-09 128955.487588 8.10532e-09 It seems that the parallel running doesn't work, the elapsed time is roughly the same. You see that in the process of assembling the matrix in the parallel case, the two cores are working according to from 0 to 8192 on rank 0 from 8192 to 16384 on rank 1 but in the SLEPc phase, only rank 0 is working (the print command only works for rank 0, but not for rank 1). I also tried to run the ex1.py in parallel and in serial, the computation times for a relative large matrix are also roughly the same for the two runnings, or in some cases, parallel running is even longer. Could you help me on this? Thanks a lot. 2014-10-29 15:37 GMT-02:00 Mengqi Zhang : > Thanks a lot, Jose. I'll try to read and I'll come back to you if I have > any problem. > > Mengqi > > 2014-10-29 12:23 GMT-02:00 Jose E. Roman : > > >> El 29/10/2014, a las 14:56, Mengqi Zhang escribi?: >> >> > Hi, Jose >> > >> > Thanks for your reply. >> > I will specify the problem more exactly. I want to solve a big >> generalized eigenvalue problem. The matrices are sparse. My question lies >> on how to allocate the matrices among the processors. Do I have to do it by >> myself? Or there is a routine to do so? I notice that one should do >> something like >> > from petsc4py import PETSc >> > >> > >> > A = PETSc.Mat().create() >> > >> > >> > A.setType('aij') >> > >> > >> > A.setSizes([M,N]) >> > >> > >> > A.setPreallocationNNZ([diag_nz, offdiag_nz]) # optional >> > A.setUp() >> > I have several question regarding these lines. >> > (1) I presume M and N are the dimension of the matrix. Then how do the >> processors divide the matrix? I guess setPreallocationNNZ does the >> allocation of the matrix among the processors. What does nz mean here? Why >> here appears diag and offdiag? >> > (2) I actually saw somewhere else that people use >> A.setPreallocationNNZ(5), with a single parameter 5. What does 5 mean here? >> > (3) I want to make sure that the matrix so generated is sparse (since >> it uses aij). It is sparse right? I feel it tricky since if the matrix is >> stored as sparse, will the allocation/parallelization destroy the >> efficiency of sparse matrix? >> > >> > >> > After the matrix is set up, I would like to use SLEPc4py to solve the >> generalized eigenvalue problem. The example code I got online is like >> > E = SLEPc.EPS(); E.create() >> > E.setOperators(A) >> > E.setProblemType(SLEPc.EPS.ProblemType.GNHEP) >> > E.setFromOptions() >> > E.solve() >> > I'm afraid this script is not designed for the parallel computation >> since in the options there is no indication of parallelization. Do you know >> how to set it up? >> > >> > Thank you very much for your time. I appreciate it very much. >> > Best, >> > Mengqi >> > >> >> Almost all examples under slepc4py/demo are parallel. You can take for >> instance ex1.py as a basis for your own scripts. In these examples, every >> process fills its part of the matrix (locally owned rows), as obtained from >> getOwnershipRange(). See PETSc's documentation here: >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetOwnershipRange.html >> >> The meaning of the preallocation arguments is also explained in PETSc's >> manpages: >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html >> >> petsc4py/slepc4py are just python wrappers to PETSc/SLEPc, so you must >> understand how PETSc and SLEPc work, and dig into their manuals and >> manpages. >> >> Jose >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: matricesEigenvalue.py Type: text/x-python-script Size: 2732 bytes Desc: not available URL: From zonexo at gmail.com Wed Oct 29 22:27:21 2014 From: zonexo at gmail.com (TAY wee-beng) Date: Thu, 30 Oct 2014 11:27:21 +0800 Subject: [petsc-users] Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type In-Reply-To: <1F860363-1461-4A41-A1CA-4B94C88070DB@mcs.anl.gov> References: <545199E6.7040706@gmail.com> <1F860363-1461-4A41-A1CA-4B94C88070DB@mcs.anl.gov> Message-ID: <5451B019.9090601@gmail.com> On 30/10/2014 10:02 AM, Barry Smith wrote: > Hmm, could you send us a (simple) code that reproduces the problem; I don?t think we have a petsc_default_real anywhere. > > Barry Hi Barry, I realised that it's because I was using ver 3.4.4. There's no problem after using ver 3.5.2. Thanks! Regards. > >> On Oct 29, 2014, at 8:52 PM, TAY wee-beng wrote: >> >> Hi, >> >> When compiling using gnu gfortran, I got the following error msg: >> >> Error: Symbol 'petsc_default_real' at (1) has no IMPLICIT type >> >> My code is : >> >> module PETSc_solvers >> >> use set_matrix >> >> use ibm >> >> implicit none >> >> contains >> >> subroutine semi_momentum_simple_xyz(du,dv,dw) >> >> !discretise momentum eqn using semi-implicit mtd >> >> !simple except east,west >> >> #include "finclude/petsc.h90" >> >> ... >> >> call KSPSetTolerances(ksp_semi_xyz,tol,PETSC_DEFAULT_REAL,PETSC_DEFAULT_REAL,PETSC_DEFAULT_INTEGER,ierr) -> error >> >> There's no problem when using ifort though. >> -- >> Thank you >> >> Yours sincerely, >> >> TAY wee-beng >> From jed at jedbrown.org Wed Oct 29 22:52:48 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 29 Oct 2014 21:52:48 -0600 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: References: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> Message-ID: <87ppdaz01b.fsf@jedbrown.org> Mengqi Zhang writes: > It seems that the parallel running doesn't work, the elapsed time is > roughly the same. http://www.mcs.anl.gov/petsc/documentation/faq.html#computers > You see that in the process of assembling the matrix in the parallel > case, the two cores are working according to > > from 0 to 8192 on rank 0 > from 8192 to 16384 on rank 1 > > but in the SLEPc phase, only rank 0 is working (the print command only > works for rank 0, but not for rank 1). That is just printing. If every rank printed, you'd have a jumbled mess of output. Always send the output of -log_summary if you have questions about performance. > I also tried to run the ex1.py in parallel and in serial, the > computation times for a relative large matrix are also roughly the > same for the two runnings, or in some cases, parallel running is even > longer. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From evanum at gmail.com Wed Oct 29 23:30:14 2014 From: evanum at gmail.com (Evan Um) Date: Wed, 29 Oct 2014 21:30:14 -0700 Subject: [petsc-users] Creating a symmetric matrix with the upper triangular elements Message-ID: PETSC users, As memory is at a premium in my problem, I want to compute only the upper triangular elements of a matrix and then construct a parallel sparse symmetric matrix. In this case, do I have to use "MatSetOption (A,MAT_SYMMETRIC,PETSC_TRUE )"? In this case, is the matrix internally stored in the memory saving mode (i.e. the upper triangular elements only) or does this provide us with just the convenience for computing half of matrix elements? In advance, thanks for your comments. Regards, Evan -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Wed Oct 29 23:54:06 2014 From: jed at jedbrown.org (Jed Brown) Date: Wed, 29 Oct 2014 22:54:06 -0600 Subject: [petsc-users] Creating a symmetric matrix with the upper triangular elements In-Reply-To: References: Message-ID: <87a94eyx75.fsf@jedbrown.org> Evan Um writes: > PETSC users, > > As memory is at a premium in my problem, I want to compute only the upper > triangular elements of a matrix and then construct a parallel sparse > symmetric matrix. In this case, do I have to use "MatSetOption > > (A,MAT_SYMMETRIC,PETSC_TRUE This asserts that the matrix is symmetric, but it does not change the data structure at all and doesn't magically make the matrix symmetric. You want MATSBAIJ. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jroman at dsic.upv.es Thu Oct 30 02:10:23 2014 From: jroman at dsic.upv.es (Jose E. Roman) Date: Thu, 30 Oct 2014 08:10:23 +0100 Subject: [petsc-users] How to use PETSc4py/SLEPc4py to solve an eigenvalue problem in parallel In-Reply-To: <87ppdaz01b.fsf@jedbrown.org> References: <77D4A7C4-83E0-4D10-86CD-DE89836956C6@dsic.upv.es> <87ppdaz01b.fsf@jedbrown.org> Message-ID: El 30/10/2014, a las 04:52, Jed Brown escribi?: > Mengqi Zhang writes: >> It seems that the parallel running doesn't work, the elapsed time is >> roughly the same. > > http://www.mcs.anl.gov/petsc/documentation/faq.html#computers > >> You see that in the process of assembling the matrix in the parallel >> case, the two cores are working according to >> >> from 0 to 8192 on rank 0 >> from 8192 to 16384 on rank 1 >> >> but in the SLEPc phase, only rank 0 is working (the print command only >> works for rank 0, but not for rank 1). > > That is just printing. If every rank printed, you'd have a jumbled mess > of output. > > Always send the output of -log_summary if you have questions about performance. > >> I also tried to run the ex1.py in parallel and in serial, the >> computation times for a relative large matrix are also roughly the >> same for the two runnings, or in some cases, parallel running is even >> longer. > As I mentioned in my first email, you need a parallel linear solver as discussed in section 3.4.1. Otherwise you won't see any parallel gain. Jose From mailinglists at xgm.de Thu Oct 30 03:36:55 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Thu, 30 Oct 2014 09:36:55 +0100 Subject: [petsc-users] Problem with Preallocating In-Reply-To: <5E018295-7CE9-4AAF-AA56-F25143BFE462@mcs.anl.gov> References: <2312349.jPYSpgGm9R@asaru> <5E018295-7CE9-4AAF-AA56-F25143BFE462@mcs.anl.gov> Message-ID: <26693280.8WxW8hL8ra@asaru> Am Mittwoch, 29. Oktober 2014, 13:00:17 schrieb Barry Smith: > > MatAssembly flushes out any preallocated space that was not used. Hence when you try to put a diagonal entry in later it is as if you did not preallocate. You should set zeros in the diagonal as part of your initial setting values in the matrix before calling MatAssembly. Even with FLUSH_ASSEMBLY? I had this idea too, but was not able to reproduce the error I have in my application in a small example. Why does this program works? Shouldn't the MatDiagonalSet produce an error/warning that there were mallocs involved? Since the preallocation was lost? Thanks, Florian #include #include "petscmat.h" #include "petscviewer.h" #include "petscksp.h" using namespace std; // Compiling with: mpic++ -g3 -Wall -I ~/software/petsc/include -I ~/software/petsc/arch-linux2-c-debug/include -L ~/software/petsc/arch-linux2-c-debug/lib -lpetsc test.cpp int main(int argc, char **args) { PetscInitialize(&argc, &args, "", NULL); PetscErrorCode ierr = 0; int N = 9; Mat matrix; ierr = MatCreate(PETSC_COMM_SELF, &matrix); ierr = MatSetType(matrix, MATSBAIJ); CHKERRQ(ierr); ierr = MatSetSizes(matrix, PETSC_DECIDE, PETSC_DECIDE, N, N); CHKERRQ(ierr); ierr = MatSetOption(matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); CHKERRQ(ierr); PetscInt nnz[N]; nnz[0] = 2; nnz[1] = 1; nnz[2] = 1; nnz[3] = 1; nnz[4] = 1; nnz[5] = 1; nnz[6] = 1; nnz[7] = 1; nnz[8] = 1; for (int r = 0; r < N; r++) { cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; } MatSeqSBAIJSetPreallocation(matrix, 1, PETSC_DEFAULT, nnz); MatSetValue(matrix, 0, 5, 10, INSERT_VALUES); ierr = MatAssemblyBegin(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); Vec zeros; MatGetVecs(matrix, NULL, &zeros); MatDiagonalSet(matrix, zeros, ADD_VALUES); ierr = MatAssemblyBegin(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatAssemblyEnd(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); ierr = MatView(matrix, PETSC_VIEWER_STDOUT_WORLD); CHKERRQ(ierr); MatDestroy(&matrix); PetscFinalize(); return 0; } > > Barry > > > On Oct 29, 2014, at 8:00 AM, Florian Lindner wrote: > > > > Hello, > > > > I try to preallocate a sparse matrix like it was recommended in another posting, but get an error which kind of surprises me. Somehow I think it might be related to the order of assembly calls... > > > > My code creates the matrix: > > > > MatSetType(_matrixC.matrix, MATSBAIJ); > > MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); > > MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > > > > Than I enter a loop to get the number of elements per row. The entire code is a bit too long, but right before the SetPreallocaiton I do: > > > > for (int r = 0; r < n; r++) { > > cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; > > } > > > > which results in the output: > > > > Matrix Size = 9 > > Preallocated row 0 with 9 elements. > > Preallocated row 1 with 8 elements. > > Preallocated row 2 with 7 elements. > > Preallocated row 3 with 6 elements. > > Preallocated row 4 with 5 elements. > > Preallocated row 5 with 4 elements. > > Preallocated row 6 with 1 elements. > > Preallocated row 7 with 1 elements. > > Preallocated row 8 with 1 elements. > > > > at least in rows 0 to 5 it's a dense upper triangular matrix (incl. main diagonal). > > > > ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, nnz); > > > > Now more or less the same loops as above with MatSetValues call for each row. > > > > Since the KSPSolver requires that the diagonal is set, even if it's merely with zeros I do: > > > > _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm sure you get the point ;-) > > petsc::Vector zeros(_matrixC); > > MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); > > ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); > > [ some more code ] > > ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); > > > > But when I run the code I get an error message: > > > > [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- > > [0]PETSC ERROR: Argument out of range > > [0]PETSC ERROR: New nonzero at (0,0) caused a malloc > > Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn off this check > > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > > [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > > [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by lindnefn Wed Oct 29 13:48:49 2014 > > [0]PETSC ERROR: Configure options --with-debugging=1 > > [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c > > [0]PETSC ERROR: #2 MatSetValues() line 1135 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > > [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > > [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > > > > I'm surprised that I get this error message since I do have preallocated 9 elements and I do not understand why inserting an element at (0, 0) could be a problem / needing a malloc. Has the preallocation got lost somewhere? > > > > Thanks once again.... > > > > Florian > From bsmith at mcs.anl.gov Thu Oct 30 07:25:20 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 30 Oct 2014 07:25:20 -0500 Subject: [petsc-users] Problem with Preallocating In-Reply-To: <26693280.8WxW8hL8ra@asaru> References: <2312349.jPYSpgGm9R@asaru> <5E018295-7CE9-4AAF-AA56-F25143BFE462@mcs.anl.gov> <26693280.8WxW8hL8ra@asaru> Message-ID: > On Oct 30, 2014, at 3:36 AM, Florian Lindner wrote: > > Am Mittwoch, 29. Oktober 2014, 13:00:17 schrieb Barry Smith: >> >> MatAssembly flushes out any preallocated space that was not used. Hence when you try to put a diagonal entry in later it is as if you did not preallocate. You should set zeros in the diagonal as part of your initial setting values in the matrix before calling MatAssembly. > > Even with FLUSH_ASSEMBLY? No, flush assembly does not clean out any unused preallocated space. final assembly does Barry > I had this idea too, but was not able to reproduce the error I have in my application in a small example. > > Why does this program works? Shouldn't the MatDiagonalSet produce an error/warning that there were mallocs involved? Since the preallocation was lost? > > Thanks, > Florian > > #include > #include "petscmat.h" > #include "petscviewer.h" > #include "petscksp.h" > > using namespace std; > > // Compiling with: mpic++ -g3 -Wall -I ~/software/petsc/include -I ~/software/petsc/arch-linux2-c-debug/include -L ~/software/petsc/arch-linux2-c-debug/lib -lpetsc test.cpp > > int main(int argc, char **args) > { > PetscInitialize(&argc, &args, "", NULL); > > PetscErrorCode ierr = 0; > int N = 9; > Mat matrix; > > ierr = MatCreate(PETSC_COMM_SELF, &matrix); > ierr = MatSetType(matrix, MATSBAIJ); CHKERRQ(ierr); > ierr = MatSetSizes(matrix, PETSC_DECIDE, PETSC_DECIDE, N, N); CHKERRQ(ierr); > ierr = MatSetOption(matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); CHKERRQ(ierr); > > PetscInt nnz[N]; > nnz[0] = 2; nnz[1] = 1; nnz[2] = 1; > nnz[3] = 1; nnz[4] = 1; nnz[5] = 1; > nnz[6] = 1; nnz[7] = 1; nnz[8] = 1; > > for (int r = 0; r < N; r++) { > cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; > } > MatSeqSBAIJSetPreallocation(matrix, 1, PETSC_DEFAULT, nnz); > > MatSetValue(matrix, 0, 5, 10, INSERT_VALUES); > > ierr = MatAssemblyBegin(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); > ierr = MatAssemblyEnd(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); > Vec zeros; > MatGetVecs(matrix, NULL, &zeros); > > MatDiagonalSet(matrix, zeros, ADD_VALUES); > ierr = MatAssemblyBegin(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > ierr = MatAssemblyEnd(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > > ierr = MatView(matrix, PETSC_VIEWER_STDOUT_WORLD); CHKERRQ(ierr); > > MatDestroy(&matrix); > PetscFinalize(); > return 0; > } > > > > >> >> Barry >> >>> On Oct 29, 2014, at 8:00 AM, Florian Lindner wrote: >>> >>> Hello, >>> >>> I try to preallocate a sparse matrix like it was recommended in another posting, but get an error which kind of surprises me. Somehow I think it might be related to the order of assembly calls... >>> >>> My code creates the matrix: >>> >>> MatSetType(_matrixC.matrix, MATSBAIJ); >>> MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); >>> MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); >>> >>> Than I enter a loop to get the number of elements per row. The entire code is a bit too long, but right before the SetPreallocaiton I do: >>> >>> for (int r = 0; r < n; r++) { >>> cout << "Preallocated row " << r << " with " << nnz[r] << " elements." << endl; >>> } >>> >>> which results in the output: >>> >>> Matrix Size = 9 >>> Preallocated row 0 with 9 elements. >>> Preallocated row 1 with 8 elements. >>> Preallocated row 2 with 7 elements. >>> Preallocated row 3 with 6 elements. >>> Preallocated row 4 with 5 elements. >>> Preallocated row 5 with 4 elements. >>> Preallocated row 6 with 1 elements. >>> Preallocated row 7 with 1 elements. >>> Preallocated row 8 with 1 elements. >>> >>> at least in rows 0 to 5 it's a dense upper triangular matrix (incl. main diagonal). >>> >>> ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, nnz); >>> >>> Now more or less the same loops as above with MatSetValues call for each row. >>> >>> Since the KSPSolver requires that the diagonal is set, even if it's merely with zeros I do: >>> >>> _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm sure you get the point ;-) >>> petsc::Vector zeros(_matrixC); >>> MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); >>> ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); >>> [ some more code ] >>> ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); CHKERRV(ierr); >>> >>> But when I run the code I get an error message: >>> >>> [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>> [0]PETSC ERROR: Argument out of range >>> [0]PETSC ERROR: New nonzero at (0,0) caused a malloc >>> Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn off this check >>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>> [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by lindnefn Wed Oct 29 13:48:49 2014 >>> [0]PETSC ERROR: Configure options --with-debugging=1 >>> [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c >>> [0]PETSC ERROR: #2 MatSetValues() line 1135 in /data2/scratch/lindner/petsc/src/mat/interface/matrix.c >>> [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c >>> [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in /data2/scratch/lindner/petsc/src/mat/utils/axpy.c >>> >>> I'm surprised that I get this error message since I do have preallocated 9 elements and I do not understand why inserting an element at (0, 0) could be a problem / needing a malloc. Has the preallocation got lost somewhere? >>> >>> Thanks once again.... >>> >>> Florian >> From knepley at gmail.com Thu Oct 30 09:19:38 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 30 Oct 2014 09:19:38 -0500 Subject: [petsc-users] Problem with Preallocating In-Reply-To: References: <2312349.jPYSpgGm9R@asaru> <5E018295-7CE9-4AAF-AA56-F25143BFE462@mcs.anl.gov> <26693280.8WxW8hL8ra@asaru> Message-ID: On Thu, Oct 30, 2014 at 7:25 AM, Barry Smith wrote: > > > On Oct 30, 2014, at 3:36 AM, Florian Lindner > wrote: > > > > Am Mittwoch, 29. Oktober 2014, 13:00:17 schrieb Barry Smith: > >> > >> MatAssembly flushes out any preallocated space that was not used. > Hence when you try to put a diagonal entry in later it is as if you did not > preallocate. You should set zeros in the diagonal as part of your initial > setting values in the matrix before calling MatAssembly. > > > > Even with FLUSH_ASSEMBLY? > > No, flush assembly does not clean out any unused preallocated space. > final assembly does > Something else must be going on in your full code (as shown by the small example). Are you running it in serial or parallel? Thanks, Matt > Barry > > > I had this idea too, but was not able to reproduce the error I have in > my application in a small example. > > > > Why does this program works? Shouldn't the MatDiagonalSet produce an > error/warning that there were mallocs involved? Since the preallocation was > lost? > > > > Thanks, > > Florian > > > > #include > > #include "petscmat.h" > > #include "petscviewer.h" > > #include "petscksp.h" > > > > using namespace std; > > > > // Compiling with: mpic++ -g3 -Wall -I ~/software/petsc/include -I > ~/software/petsc/arch-linux2-c-debug/include -L > ~/software/petsc/arch-linux2-c-debug/lib -lpetsc test.cpp > > > > int main(int argc, char **args) > > { > > PetscInitialize(&argc, &args, "", NULL); > > > > PetscErrorCode ierr = 0; > > int N = 9; > > Mat matrix; > > > > ierr = MatCreate(PETSC_COMM_SELF, &matrix); > > ierr = MatSetType(matrix, MATSBAIJ); CHKERRQ(ierr); > > ierr = MatSetSizes(matrix, PETSC_DECIDE, PETSC_DECIDE, N, N); > CHKERRQ(ierr); > > ierr = MatSetOption(matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > CHKERRQ(ierr); > > > > PetscInt nnz[N]; > > nnz[0] = 2; nnz[1] = 1; nnz[2] = 1; > > nnz[3] = 1; nnz[4] = 1; nnz[5] = 1; > > nnz[6] = 1; nnz[7] = 1; nnz[8] = 1; > > > > for (int r = 0; r < N; r++) { > > cout << "Preallocated row " << r << " with " << nnz[r] << " > elements." << endl; > > } > > MatSeqSBAIJSetPreallocation(matrix, 1, PETSC_DEFAULT, nnz); > > > > MatSetValue(matrix, 0, 5, 10, INSERT_VALUES); > > > > ierr = MatAssemblyBegin(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); > > ierr = MatAssemblyEnd(matrix, MAT_FLUSH_ASSEMBLY); CHKERRQ(ierr); > > Vec zeros; > > MatGetVecs(matrix, NULL, &zeros); > > > > MatDiagonalSet(matrix, zeros, ADD_VALUES); > > ierr = MatAssemblyBegin(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > > ierr = MatAssemblyEnd(matrix, MAT_FINAL_ASSEMBLY); CHKERRQ(ierr); > > > > ierr = MatView(matrix, PETSC_VIEWER_STDOUT_WORLD); CHKERRQ(ierr); > > > > MatDestroy(&matrix); > > PetscFinalize(); > > return 0; > > } > > > > > > > > > >> > >> Barry > >> > >>> On Oct 29, 2014, at 8:00 AM, Florian Lindner > wrote: > >>> > >>> Hello, > >>> > >>> I try to preallocate a sparse matrix like it was recommended in > another posting, but get an error which kind of surprises me. Somehow I > think it might be related to the order of assembly calls... > >>> > >>> My code creates the matrix: > >>> > >>> MatSetType(_matrixC.matrix, MATSBAIJ); > >>> MatSetSizes(_matrixC.matrix, PETSC_DECIDE, PETSC_DECIDE, n, n); > >>> MatSetOption(_matrixC.matrix, MAT_SYMMETRY_ETERNAL, PETSC_TRUE); > >>> > >>> Than I enter a loop to get the number of elements per row. The entire > code is a bit too long, but right before the SetPreallocaiton I do: > >>> > >>> for (int r = 0; r < n; r++) { > >>> cout << "Preallocated row " << r << " with " << nnz[r] << " > elements." << endl; > >>> } > >>> > >>> which results in the output: > >>> > >>> Matrix Size = 9 > >>> Preallocated row 0 with 9 elements. > >>> Preallocated row 1 with 8 elements. > >>> Preallocated row 2 with 7 elements. > >>> Preallocated row 3 with 6 elements. > >>> Preallocated row 4 with 5 elements. > >>> Preallocated row 5 with 4 elements. > >>> Preallocated row 6 with 1 elements. > >>> Preallocated row 7 with 1 elements. > >>> Preallocated row 8 with 1 elements. > >>> > >>> at least in rows 0 to 5 it's a dense upper triangular matrix (incl. > main diagonal). > >>> > >>> ierr = MatSeqSBAIJSetPreallocation(_matrixC.matrix, 1, PETSC_DEFAULT, > nnz); > >>> > >>> Now more or less the same loops as above with MatSetValues call for > each row. > >>> > >>> Since the KSPSolver requires that the diagonal is set, even if it's > merely with zeros I do: > >>> > >>> _matrixC.assemble(MAT_FLUSH_ASSEMBLY) // my helper function, but I'm > sure you get the point ;-) > >>> petsc::Vector zeros(_matrixC); > >>> MatDiagonalSet(_matrixC.matrix, zeros.vector, ADD_VALUES); > >>> ierr = MatAssemblyBegin(_matrixC.matrix, MAT_FINAL_ASSEMBLY); > CHKERRV(ierr); > >>> [ some more code ] > >>> ierr = MatAssemblyEnd(_matrixC.matrix, MAT_FINAL_ASSEMBLY); > CHKERRV(ierr); > >>> > >>> But when I run the code I get an error message: > >>> > >>> [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > >>> [0]PETSC ERROR: Argument out of range > >>> [0]PETSC ERROR: New nonzero at (0,0) caused a malloc > >>> Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to > turn off this check > >>> [0]PETSC ERROR: See > http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. > >>> [0]PETSC ERROR: Petsc Release Version 3.5.2, unknown > >>> [0]PETSC ERROR: ./petBench on a arch-linux2-c-debug named helium by > lindnefn Wed Oct 29 13:48:49 2014 > >>> [0]PETSC ERROR: Configure options --with-debugging=1 > >>> [0]PETSC ERROR: #1 MatSetValues_SeqSBAIJ() line 977 in > /data2/scratch/lindner/petsc/src/mat/impls/sbaij/seq/sbaij.c > >>> [0]PETSC ERROR: #2 MatSetValues() line 1135 in > /data2/scratch/lindner/petsc/src/mat/interface/matrix.c > >>> [0]PETSC ERROR: #3 MatDiagonalSet_Default() line 203 in > /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > >>> [0]PETSC ERROR: #4 MatDiagonalSet() line 241 in > /data2/scratch/lindner/petsc/src/mat/utils/axpy.c > >>> > >>> I'm surprised that I get this error message since I do have > preallocated 9 elements and I do not understand why inserting an element at > (0, 0) could be a problem / needing a malloc. Has the preallocation got > lost somewhere? > >>> > >>> Thanks once again.... > >>> > >>> Florian > >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From lb2653 at columbia.edu Thu Oct 30 09:21:39 2014 From: lb2653 at columbia.edu (Luc Berger-Vergiat) Date: Thu, 30 Oct 2014 10:21:39 -0400 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> <544FE57C.8020303@columbi.edu> <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> Message-ID: <54524973.8010103@columbi.edu> Sorry for the late reply, it took longer than I thought. So a little update on my situation: I have to use a custom version of valgrind on CETUS which has to be linked to my code using -Wl,-e,_start_valgrind at compilation (I also add object file and libraries). After that I can run my code with the following arguments: feap --ignore-ranges=0x4000000000000-0x4063000000000,0x003fdc0000000-0x003fe00000000 --suppressions=/soft/perftools/valgrind/cnk-baseline.supp but I can't use the usual petsc argument (-ksp_type -pc_type ...) since valgrind does not recognize them. So I decided to use the PETSC_OPTIONS='-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale', to pass my arguments to PETSc. However I am a little worried that Petsc still receive the command line arguments that are used by valgrind since I get the following stderr from valgrind: stderr[0]: ==1== by 0x38BEA3F: handle_SCSS_change (m_signals.c:963) stderr[0]: ==1== by 0x38C13A7: vgPlain_do_sys_sigaction (m_signals.c:1114) stderr[0]: ==1== by 0x3962FBF: vgSysWrap_linux_sys_rt_sigaction_before (syswrap-linux.c:3073) stderr[0]: ==1== by 0x3928BF7: vgPlain_client_syscall (syswrap-main.c:1464) stderr[0]: ==1== by 0x3925F4B: vgPlain_scheduler (scheduler.c:1061) stderr[0]: ==1== by 0x3965EB3: run_a_thread_NORETURN (syswrap-linux.c:103) stderr[0]: stderr[0]: sched status: stderr[0]: running_tid=1 stderr[0]: stderr[0]: Thread 1: status = VgTs_Runnable stderr[0]: ==1== at 0x34290E4: __libc_sigaction (sigaction.c:80) stderr[0]: ==1== by 0x3BFF3A7: signal (signal.c:49) stderr[0]: ==1== by 0x1E710DF: PetscPushSignalHandler (in /projects/shearbands/ShearBands/parfeap/feap) stderr[0]: ==1== by 0x18BEA87: PetscOptionsCheckInitial_Private (in /projects/shearbands/ShearBands/parfeap/feap) stderr[0]: ==1== by 0x18E132F: petscinitialize (in /projects/shearbands/ShearBands/parfeap/feap) stderr[0]: ==1== by 0x1027557: pstart (in /projects/shearbands/ShearBands/parfeap/feap) stderr[0]: ==1== by 0x1000B1F: MAIN__ (feap83.f:213) stderr[0]: ==1== by 0x342ABD7: main (fmain.c:21) stderr[0]: stderr[0]: stderr[0]: Note: see also the FAQ in the source distribution. stderr[0]: It contains workarounds to several common problems. stderr[0]: In particular, if Valgrind aborted or crashed after stderr[0]: identifying problems in your program, there's a good chance stderr[0]: that fixing those problems will prevent Valgrind aborting or stderr[0]: crashing, especially if it happened in m_mallocfree.c. stderr[0]: stderr[0]: If that doesn't help, please report this bug to: www.valgrind.org stderr[0]: stderr[0]: In the bug report, send all the above text, the valgrind stderr[0]: version, and what OS and version you are using. Thank stderr[0]: s. stderr[0]: I am only showing the output of rank[0] but it seems that all ranks have about the same error message. Since my problem happens in petscinitialize I have little possibilities to check what's wrong... Any ideas? Best, Luc On 10/28/2014 02:53 PM, Barry Smith wrote: > You don?t care about checking for leaks. I use > > -q --tool=memcheck --num-callers=20 --track-origins=yes > >> On Oct 28, 2014, at 1:50 PM, Luc Berger-Vergiat wrote: >> >> Yes, I am running with --leak-check=full >> Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... >> Hopefully the queue will go up fast and I can give you some details about the issue. >> >> Best, >> Luc >> >> On 10/28/2014 02:25 PM, Barry Smith wrote: >>> You need to pass some options to valgrind telling it to check for memory corruption issues >>> >>> >>>> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat wrote: >>>> >>>> Ok, I'm recompiling PETSc in debug mode then. >>>> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >>>> Would this work for example: >>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >>>> >>>> >>>> Best, >>>> Luc >>>> >>>> On 10/28/2014 12:33 PM, Barry Smith wrote: >>>>> Hmm, this should never happen. In the code >>>>> >>>>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>>>> for (i=0; iB->rmap->n; i++) { >>>>> for (j=0; jilen[i]; j++) { >>>>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>>>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>>>> >>>>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>>>> >>>>> >>>>> Barry >>>>> >>>>> >>>>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>>>> Here is my submission command: >>>>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>>>> >>>>>> Here is "4nodes_32cores_LU": >>>>>> #!/bin/sh >>>>>> >>>>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>>>> echo "Cobalt location args: $LOCARGS" >&2 >>>>>> >>>>>> ################################ >>>>>> # 32 cores on 4 nodes jobs # >>>>>> ################################ >>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>>>> >>>>>> I get the following error message: >>>>>> >>>>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>>> [7]PETSC ERROR: Argument out of range >>>>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>>>> -lblas" - >>>>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>>>> >>>>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>>>> >>>>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>>>> >>>>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>>>> >>>>>> Thanks for the help >>>>>> Best, >>>>>> Luc >>>>>> >>>>>> >>>>>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Thu Oct 30 11:23:17 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Thu, 30 Oct 2014 11:23:17 -0500 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <54524973.8010103@columbi.edu> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> <544FE57C.8020303@columbi.edu> <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> <54524973.8010103@columbi.edu> Message-ID: <5B51768E-D3B7-4FE9-A2D7-A540C9750DC7@mcs.anl.gov> Run with the additional PETSc option -no_signal_handler then PETSc won?t mess with the signals and it may get you past this point. Barry > On Oct 30, 2014, at 9:21 AM, Luc Berger-Vergiat wrote: > > Sorry for the late reply, it took longer than I thought. > So a little update on my situation: > I have to use a custom version of valgrind on CETUS which has to be linked to my code using -Wl,-e,_start_valgrind at compilation (I also add object file and libraries). > After that I can run my code with the following arguments: > feap --ignore-ranges=0x4000000000000-0x4063000000000,0x003fdc0000000-0x003fe00000000 --suppressions=/soft/perftools/valgrind/cnk-baseline.supp > but I can't use the usual petsc argument (-ksp_type -pc_type ...) since valgrind does not recognize them. > So I decided to use the PETSC_OPTIONS='-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale', to pass my arguments to PETSc. > However I am a little worried that Petsc still receive the command line arguments that are used by valgrind since I get the following stderr from valgrind: > > stderr[0]: ==1== by 0x38BEA3F: handle_SCSS_change (m_signals.c:963) > stderr[0]: ==1== by 0x38C13A7: vgPlain_do_sys_sigaction (m_signals.c:1114) > stderr[0]: ==1== by 0x3962FBF: vgSysWrap_linux_sys_rt_sigaction_before (syswrap-linux.c:3073) > stderr[0]: ==1== by 0x3928BF7: vgPlain_client_syscall (syswrap-main.c:1464) > stderr[0]: ==1== by 0x3925F4B: vgPlain_scheduler (scheduler.c:1061) > stderr[0]: ==1== by 0x3965EB3: run_a_thread_NORETURN (syswrap-linux.c:103) > stderr[0]: > stderr[0]: sched status: > stderr[0]: running_tid=1 > stderr[0]: > stderr[0]: Thread 1: status = VgTs_Runnable > stderr[0]: ==1== at 0x34290E4: __libc_sigaction (sigaction.c:80) > stderr[0]: ==1== by 0x3BFF3A7: signal (signal.c:49) > stderr[0]: ==1== by 0x1E710DF: PetscPushSignalHandler (in /projects/shearbands/ShearBands/parfeap/feap) > stderr[0]: ==1== by 0x18BEA87: PetscOptionsCheckInitial_Private (in /projects/shearbands/ShearBands/parfeap/feap) > stderr[0]: ==1== by 0x18E132F: petscinitialize (in /projects/shearbands/ShearBands/parfeap/feap) > stderr[0]: ==1== by 0x1027557: pstart (in /projects/shearbands/ShearBands/parfeap/feap) > stderr[0]: ==1== by 0x1000B1F: MAIN__ (feap83.f:213) > stderr[0]: ==1== by 0x342ABD7: main (fmain.c:21) > stderr[0]: > stderr[0]: > stderr[0]: Note: see also the FAQ in the source distribution. > stderr[0]: It contains workarounds to several common problems. > stderr[0]: In particular, if Valgrind aborted or crashed after > stderr[0]: identifying problems in your program, there's a good chance > stderr[0]: that fixing those problems will prevent Valgrind aborting or > stderr[0]: crashing, especially if it happened in m_mallocfree.c. > stderr[0]: > stderr[0]: If that doesn't help, please report this bug to: www.valgrind.org > stderr[0]: > stderr[0]: In the bug report, send all the above text, the valgrind > stderr[0]: version, and what OS and version you are using. Thank > stderr[0]: s. > stderr[0]: > > I am only showing the output of rank[0] but it seems that all ranks have about the same error message. > Since my problem happens in petscinitialize I have little possibilities to check what's wrong... > Any ideas? > Best, > Luc > > On 10/28/2014 02:53 PM, Barry Smith wrote: >> You don?t care about checking for leaks. I use >> >> -q --tool=memcheck --num-callers=20 --track-origins=yes >> >> >>> On Oct 28, 2014, at 1:50 PM, Luc Berger-Vergiat >>> wrote: >>> >>> Yes, I am running with --leak-check=full >>> Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... >>> Hopefully the queue will go up fast and I can give you some details about the issue. >>> >>> Best, >>> Luc >>> >>> On 10/28/2014 02:25 PM, Barry Smith wrote: >>> >>>> You need to pass some options to valgrind telling it to check for memory corruption issues >>>> >>>> >>>> >>>>> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat >>>>> wrote: >>>>> >>>>> Ok, I'm recompiling PETSc in debug mode then. >>>>> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >>>>> Would this work for example: >>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >>>>> >>>>> >>>>> Best, >>>>> Luc >>>>> >>>>> On 10/28/2014 12:33 PM, Barry Smith wrote: >>>>> >>>>>> Hmm, this should never happen. In the code >>>>>> >>>>>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>>>>> for (i=0; iB->rmap->n; i++) { >>>>>> for (j=0; jilen[i]; j++) { >>>>>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>>>>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>>>>> >>>>>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>>>>> >>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>>>>> >>>>>> >>>>>> >>>>>> Barry >>>>>> >>>>>> >>>>>> >>>>>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>>>>> Here is my submission command: >>>>>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>>>>> >>>>>>> Here is "4nodes_32cores_LU": >>>>>>> #!/bin/sh >>>>>>> >>>>>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>>>>> echo "Cobalt location args: $LOCARGS" >&2 >>>>>>> >>>>>>> ################################ >>>>>>> # 32 cores on 4 nodes jobs # >>>>>>> ################################ >>>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>>>>> >>>>>>> I get the following error message: >>>>>>> >>>>>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>>>> [7]PETSC ERROR: Argument out of range >>>>>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>>>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>>>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>>>>> >>>>>>> -lblas" - >>>>>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>>>>> >>>>>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>>>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>>>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>>>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>>>>> >>>>>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>>>>> >>>>>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>>>>> >>>>>>> Thanks for the help >>>>>>> Best, >>>>>>> Luc >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>> >>> >>> > From jychang48 at gmail.com Thu Oct 30 15:29:27 2014 From: jychang48 at gmail.com (Justin Chang) Date: Thu, 30 Oct 2014 15:29:27 -0500 Subject: [petsc-users] Reading data from a file into a DM created vector Message-ID: Hi all, So I am writing an FEM code where it reads input data (e.g., auxiliary coefficients, source terms, etc) from a text file. I have preprocessed the data text file so that each vertex point has its corresponding data. For instance, if my source term for a diffusion problem has a sin or cos function of the coordinates, then this data is already tabulated and simply needs to be fed into my PETSc program. The data text file containing both the auxiliary data and the coordinate/connectivities will also be used to provide the input arguments for the DMPlexCreateFromDAG() function. Normally, I would hard-code the sin/cos functions into the source code itself, but i want my program to be "universal" in that it can take as input any user-defined function for not only these auxiliaries but for other things like boundary conditions. I see that PETSc has a lot of examples on how to read data into vectors and matrices, but I guess my question is is there a way to project data from a text file into a vector that has already been created with a defined DM structure? Thanks, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 30 15:35:59 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 30 Oct 2014 15:35:59 -0500 Subject: [petsc-users] Reading data from a file into a DM created vector In-Reply-To: References: Message-ID: On Thu, Oct 30, 2014 at 3:29 PM, Justin Chang wrote: > Hi all, > > So I am writing an FEM code where it reads input data (e.g., auxiliary > coefficients, source terms, etc) from a text file. I have preprocessed the > data text file so that each vertex point has its corresponding data. For > instance, if my source term for a diffusion problem has a sin or cos > function of the coordinates, then this data is already tabulated and simply > needs to be fed into my PETSc program. The data text file containing both > the auxiliary data and the coordinate/connectivities will also be used to > provide the input arguments for the DMPlexCreateFromDAG() function. > > Normally, I would hard-code the sin/cos functions into the source code > itself, but i want my program to be "universal" in that it can take as > input any user-defined function for not only these auxiliaries but for > other things like boundary conditions. I see that PETSc has a lot of > examples on how to read data into vectors and matrices, but I guess my > question is is there a way to project data from a text file into a vector > that has already been created with a defined DM structure? > If you have the functions available, I think it is far more universal to use the function itself, since then you can be independent of mesh and discretization when specifying input and BC. However, if you want to read it in, and you guarantee that it matches the mesh and discretization, I think the easiest thing to do is demand that it come in the same order as the vertices and use VecGetArray(V, &a); for (v = vStart, i = 0; v < vEnd; ++v) { PetscSectionGetDof(s, v, &dof); PetscSectionGetOffset(s, v, &off); for (d = 0; d < dof; ++d) a[off+d] = text_data[i++]; } VecRestoreArray(V, &a); Matt > Thanks, > Justin > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From jychang48 at gmail.com Thu Oct 30 16:16:36 2014 From: jychang48 at gmail.com (Justin Chang) Date: Thu, 30 Oct 2014 16:16:36 -0500 Subject: [petsc-users] Reading data from a file into a DM created vector In-Reply-To: References: Message-ID: Matt, thanks for the quick response. What about for (dirichlet) boundary conditions? Would it be possible to do something similar for those, like using those PetscSectionSetConstraint functions? Thanks, Justin On Thu, Oct 30, 2014 at 3:35 PM, Matthew Knepley wrote: > On Thu, Oct 30, 2014 at 3:29 PM, Justin Chang wrote: > >> Hi all, >> >> So I am writing an FEM code where it reads input data (e.g., auxiliary >> coefficients, source terms, etc) from a text file. I have preprocessed the >> data text file so that each vertex point has its corresponding data. For >> instance, if my source term for a diffusion problem has a sin or cos >> function of the coordinates, then this data is already tabulated and simply >> needs to be fed into my PETSc program. The data text file containing both >> the auxiliary data and the coordinate/connectivities will also be used to >> provide the input arguments for the DMPlexCreateFromDAG() function. >> >> Normally, I would hard-code the sin/cos functions into the source code >> itself, but i want my program to be "universal" in that it can take as >> input any user-defined function for not only these auxiliaries but for >> other things like boundary conditions. I see that PETSc has a lot of >> examples on how to read data into vectors and matrices, but I guess my >> question is is there a way to project data from a text file into a vector >> that has already been created with a defined DM structure? >> > > If you have the functions available, I think it is far more universal to > use the function itself, since then you can be independent > of mesh and discretization when specifying input and BC. > > However, if you want to read it in, and you guarantee that it matches the > mesh and discretization, I think the easiest thing to do is > demand that it come in the same order as the vertices and use > > VecGetArray(V, &a); > for (v = vStart, i = 0; v < vEnd; ++v) { > PetscSectionGetDof(s, v, &dof); > PetscSectionGetOffset(s, v, &off); > for (d = 0; d < dof; ++d) a[off+d] = text_data[i++]; > } > VecRestoreArray(V, &a); > > Matt > > >> Thanks, >> Justin >> > > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Thu Oct 30 16:24:48 2014 From: knepley at gmail.com (Matthew Knepley) Date: Thu, 30 Oct 2014 16:24:48 -0500 Subject: [petsc-users] Reading data from a file into a DM created vector In-Reply-To: References: Message-ID: On Thu, Oct 30, 2014 at 4:16 PM, Justin Chang wrote: > Matt, thanks for the quick response. > > What about for (dirichlet) boundary conditions? Would it be possible to do > something similar for those, like using those PetscSectionSetConstraint > functions? > Yes. There are generally two ways of handling Dirichlet conditions: 1) Replace those rows of the Jacobian with the identity and put the boundary value in the rhs. I find this cumbersome for nonlinear solves. 2) Remove these unknowns from the system using PetscSectionSetConstraintDof() and ConstratinIndices(). The DMPlexAddBoundary() functions are automating this processing by marking boundaries using DMLabels, and then constraining dofs on those boundaries. Thanks, Matt > Thanks, > Justin > > On Thu, Oct 30, 2014 at 3:35 PM, Matthew Knepley > wrote: > >> On Thu, Oct 30, 2014 at 3:29 PM, Justin Chang >> wrote: >> >>> Hi all, >>> >>> So I am writing an FEM code where it reads input data (e.g., auxiliary >>> coefficients, source terms, etc) from a text file. I have preprocessed the >>> data text file so that each vertex point has its corresponding data. For >>> instance, if my source term for a diffusion problem has a sin or cos >>> function of the coordinates, then this data is already tabulated and simply >>> needs to be fed into my PETSc program. The data text file containing both >>> the auxiliary data and the coordinate/connectivities will also be used to >>> provide the input arguments for the DMPlexCreateFromDAG() function. >>> >>> Normally, I would hard-code the sin/cos functions into the source code >>> itself, but i want my program to be "universal" in that it can take as >>> input any user-defined function for not only these auxiliaries but for >>> other things like boundary conditions. I see that PETSc has a lot of >>> examples on how to read data into vectors and matrices, but I guess my >>> question is is there a way to project data from a text file into a vector >>> that has already been created with a defined DM structure? >>> >> >> If you have the functions available, I think it is far more universal to >> use the function itself, since then you can be independent >> of mesh and discretization when specifying input and BC. >> >> However, if you want to read it in, and you guarantee that it matches the >> mesh and discretization, I think the easiest thing to do is >> demand that it come in the same order as the vertices and use >> >> VecGetArray(V, &a); >> for (v = vStart, i = 0; v < vEnd; ++v) { >> PetscSectionGetDof(s, v, &dof); >> PetscSectionGetOffset(s, v, &off); >> for (d = 0; d < dof; ++d) a[off+d] = text_data[i++]; >> } >> VecRestoreArray(V, &a); >> >> Matt >> >> >>> Thanks, >>> Justin >>> >> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From gbisht at lbl.gov Thu Oct 30 18:17:23 2014 From: gbisht at lbl.gov (Gautam Bisht) Date: Thu, 30 Oct 2014 16:17:23 -0700 Subject: [petsc-users] TS_SUNDIALS/TS_PSEUDO overshoots the final time Message-ID: All, While running src/ts/examples/tutorials/ex8.c (in PETSc 3.5.1) for TSTYPE sundials and pseudo, I noticed that the solver overshoots the final time. ./ex8 -ts_monitor -ts_type sundials -ts_final_time 0.01 -ts_view 0 TS dt 0.001 time 0 1 TS dt 0.0001 time 0.0001 2 TS dt 0.0001 time 0.0002 3 TS dt 0.0001 time 0.0003 4 TS dt 0.000199571 time 0.000499571 5 TS dt 0.000199571 time 0.000699141 6 TS dt 0.000199571 time 0.000898712 7 TS dt 0.000199571 time 0.00109828 8 TS dt 0.000404911 time 0.00150319 9 TS dt 0.000804269 time 0.00230746 10 TS dt 0.00141825 time 0.00372572 11 TS dt 0.00141825 time 0.00514397 12 TS dt 0.00141825 time 0.00656222 13 TS dt 0.00141825 time 0.00798047 14 TS dt 0.00141825 time 0.00939873 15 TS dt 0.00193698 time 0.0113357 TS Object: 1 MPI processes type: sundials maximum steps=1000 maximum time=0.01 total number of nonlinear solver iterations=18 total number of nonlinear solve failures=0 total number of linear solver iterations=18 total number of rejected steps=0 Sundials integrater does not use SNES! Sundials integrater type BDF: backward differentiation formula Sundials abs tol 1e-06 rel tol 1e-06 Sundials linear solver tolerance factor 0.05 Sundials max dimension of Krylov subspace 5 Sundials using unmodified (classical) Gram-Schmidt for orthogonalization in GMRES Sundials suggested factor for tolerance scaling 1 Sundials cumulative number of internal steps 15 Sundials no. of calls to rhs function 33 Sundials no. of calls to linear solver setup function 0 Sundials no. of error test failures 3 Sundials no. of nonlinear solver iterations 32 Sundials no. of nonlinear convergence failure 0 Sundials no. of linear iterations 18 Sundials no. of linear convergence failures 0 PC Object: 1 MPI processes type: none PC has not been set up so information may be incomplete Sundials no. of preconditioner evaluations 0 Sundials no. of preconditioner solves 0 Sundials no. of Jacobian-vector product evaluations 18 Sundials no. of rhs calls for finite diff. Jacobian-vector evals 18 steps 15 (0 rejected, 0 SNES fails), ftime 0.0113357, nonlinits 18, linits 18 step 15 t=1.13357082e-02 h= 1.94e-03 |x|= 1.07e-01 |x_e|= 1.07e-01 |x-x_e|= 2.57e-06 ./ex8 -ts_monitor -ts_type pseudo -ts_final_time 0.01 0 TS dt 0.001 time 0 1 TS dt 0.0011 time 0.0011 2 TS dt 0.00122331 time 0.00232331 3 TS dt 0.00136211 time 0.00368542 4 TS dt 0.00151873 time 0.00520415 5 TS dt 0.00169599 time 0.00690014 6 TS dt 0.00189725 time 0.00879739 7 TS dt 0.0021266 time 0.010924 TS Object: 1 MPI processes type: pseudo maximum steps=1000 maximum time=0.01 total number of nonlinear solver iterations=7 total number of nonlinear solve failures=0 total number of linear solver iterations=7 total number of rejected steps=0 SNES Object: 1 MPI processes type: ksponly maximum iterations=50, maximum function evaluations=10000 tolerances: relative=1e-08, absolute=1e-50, solution=1e-08 total number of linear solver iterations=1 total number of function evaluations=1 SNESLineSearch Object: 1 MPI processes type: basic maxstep=1.000000e+08, minlambda=1.000000e-12 tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08 maximum iterations=1 KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: ilu ILU: out-of-place factorization 0 levels of fill tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: natural factor fill ratio given 1, needed 1 Factored matrix follows: Mat Object: 1 MPI processes type: seqaij rows=1, cols=1 package used to perform factorization: petsc total: nonzeros=1, allocated nonzeros=1 total number of mallocs used during MatSetValues calls =0 not using I-node routines linear system matrix = precond matrix: Mat Object: 1 MPI processes type: seqaij rows=1, cols=1 total: nonzeros=1, allocated nonzeros=5 total number of mallocs used during MatSetValues calls =0 not using I-node routines steps 7 (0 rejected, 0 SNES fails), ftime 0.010924, nonlinits 7, linits 7 step 7 t=1.09239877e-02 h= 2.13e-03 |x|= 1.03e-01 |x_e|= 1.03e-01 |x-x_e|= 7.93e-04 Is there an option to ensure that the TS doesn't overshoot the final time? Thanks, Gautam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Thu Oct 30 23:22:24 2014 From: jed at jedbrown.org (Jed Brown) Date: Thu, 30 Oct 2014 21:22:24 -0700 Subject: [petsc-users] TS_SUNDIALS/TS_PSEUDO overshoots the final time In-Reply-To: References: Message-ID: <87a94c27i7.fsf@jedbrown.org> Gautam Bisht writes: > All, > > While running src/ts/examples/tutorials/ex8.c (in PETSc 3.5.1) for TSTYPE > sundials and pseudo, I noticed that the solver overshoots the final time. Is this what you're looking for? http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSSetExactFinalTime.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From zonexo at gmail.com Fri Oct 31 09:55:38 2014 From: zonexo at gmail.com (TAY wee-beng) Date: Fri, 31 Oct 2014 22:55:38 +0800 Subject: [petsc-users] ./a.out: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory Message-ID: <5453A2EA.8070107@gmail.com> Hi, I have no problem building PETSc 3.5.2. However, during building of my code, it gives the warning: /ipo: warning #11021: unresolved XauDisposeAuth// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// //ipo: warning #11021: unresolved XauGetBestAuthByAddr// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// //ipo: warning #11021: unresolved XdmcpWrap// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// //ipo: warning #11021: unresolved inflateEnd// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: warning #11021: unresolved deflate// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: warning #11021: unresolved deflateInit_// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: warning #11021: unresolved inflate// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: warning #11021: unresolved deflateEnd// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: warning #11021: unresolved inflateInit_// // Referenced in /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// //ipo: remark #11000: performing multi-file optimizations/ The a.out is still created. However, when running the job, it gives the error: /./a.out: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory/ I checked and found libX11.so.6 in the /usr/lib64 directory. However, using: --with-x-lib=/usr/lib64 --with-x-include=/usr/include/X11 can't work. Btw, I already added --with-x=1 during the building of PETSc. Also, the a.out works if I run interactively. But if I send the job using scheduler, it can't work. It seems that I must statically link the X11 files, is that so? Do you have a solution? -- Thank you. Yours sincerely, TAY wee-beng -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Fri Oct 31 10:05:08 2014 From: mailinglists at xgm.de (Florian Lindner) Date: Fri, 31 Oct 2014 16:05:08 +0100 Subject: [petsc-users] Problem with Preallocating In-Reply-To: References: <2312349.jPYSpgGm9R@asaru> Message-ID: <6710517.N5YgzsWFEb@asaru> Am Donnerstag, 30. Oktober 2014, 09:19:38 schrieb Matthew Knepley: > On Thu, Oct 30, 2014 at 7:25 AM, Barry Smith wrote: > > > > > On Oct 30, 2014, at 3:36 AM, Florian Lindner > > wrote: > > > > > > Am Mittwoch, 29. Oktober 2014, 13:00:17 schrieb Barry Smith: > > >> > > >> MatAssembly flushes out any preallocated space that was not used. > > Hence when you try to put a diagonal entry in later it is as if you did not > > preallocate. You should set zeros in the diagonal as part of your initial > > setting values in the matrix before calling MatAssembly. > > > > > > Even with FLUSH_ASSEMBLY? > > > > No, flush assembly does not clean out any unused preallocated space. > > final assembly does > > > > Something else must be going on in your full code (as shown by the small > example). Are you running it > in serial or parallel? Of course you were right. Something else was going on, that's why I couldn't produce an example. I was not doing a FLUSH_ASSEMBLY but FULL_ASSEMBLY instead because of some bug in my petsc helper functions. I'm so sorry for having bothered you with that! Florian From balay at mcs.anl.gov Fri Oct 31 10:07:02 2014 From: balay at mcs.anl.gov (Satish Balay) Date: Fri, 31 Oct 2014 10:07:02 -0500 Subject: [petsc-users] ./a.out: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory In-Reply-To: <5453A2EA.8070107@gmail.com> References: <5453A2EA.8070107@gmail.com> Message-ID: On Fri, 31 Oct 2014, TAY wee-beng wrote: > Hi, > > I have no problem building PETSc 3.5.2. However, during building of my code, > it gives the warning: > > /ipo: warning #11021: unresolved XauDisposeAuth// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// > //ipo: warning #11021: unresolved XauGetBestAuthByAddr// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// > //ipo: warning #11021: unresolved XdmcpWrap// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libX11.so// > //ipo: warning #11021: unresolved inflateEnd// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: warning #11021: unresolved deflate// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: warning #11021: unresolved deflateInit_// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: warning #11021: unresolved inflate// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: warning #11021: unresolved deflateEnd// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: warning #11021: unresolved inflateInit_// > // Referenced in > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libcrypto.so// > //ipo: remark #11000: performing multi-file optimizations/ IPO is not always useful. You should check if it actually helps or not - and crank down the optimization options a bit. > > The a.out is still created. However, when running the job, it gives the error: > > /./a.out: error while loading shared libraries: libX11.so.6: cannot open > shared object file: No such file or directory/ > > I checked and found libX11.so.6 in the /usr/lib64 directory. However, using: > > --with-x-lib=/usr/lib64 --with-x-include=/usr/include/X11 > > can't work. > > Btw, I already added --with-x=1 during the building of PETSc. > > Also, the a.out works if I run interactively. But if I send the job using > scheduler, it can't work. It means that your compute nodes don't have libX11.so.6 installed [but the front end node has it installed. You can request your sysadmin to make sure the same software is installed on all nodes. Or you can rebuild PETSc with --with-x=0 --with-ssl=0 Satish > > It seems that I must statically link the X11 files, is that so? > > Do you have a solution? > > From olivier.bonnefon at avignon.inra.fr Fri Oct 31 10:43:05 2014 From: olivier.bonnefon at avignon.inra.fr (Olivier Bonnefon) Date: Fri, 31 Oct 2014 16:43:05 +0100 Subject: [petsc-users] petsc-3.5.2: ex12 with Neumann BC In-Reply-To: References: <524430F2.2020507@avignon.inra.fr> <524D3D5D.5090301@avignon.inra.fr> <525804C2.3000101@avignon.inra.fr> Message-ID: <5453AE09.5090806@avignon.inra.fr> Hello, I'm working on the snes/examples/tutorial/ex12 version 3.5.2. I didn't succed to run the simplest case with Neumann BC: ./ex12 -run_type full -refinement_limit 0.0 -bc_type neumann -interpolate 1 -petscspace_order 1 -show_initial -dm_plex_print_fem 1 -show_solution -bd_petscspace_order 1 -snes_linesearch_monitor -snes_monitor -ksp_monitor_true_residual -snes_converged_reason -ksp_converged_reason This leads to dofs negatives values. Do you know the options to get a correct result with Neumann BC ? Regards, Olivier Bonnefon From gbisht at lbl.gov Fri Oct 31 11:35:23 2014 From: gbisht at lbl.gov (Gautam Bisht) Date: Fri, 31 Oct 2014 09:35:23 -0700 Subject: [petsc-users] Error while running src/snes/examples/tutorials/ex70.c Message-ID: Hi, I'm trying to learn how to use Schur complement as a preconditioner. I ran src/snes/examples/tutorials/ex70.c for the two cases: ./ex70 -user_pc -ksp_view KSP Object: 1 MPI processes type: gmres GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement GMRES: happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000 left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 1 MPI processes type: none linear system matrix = precond matrix: Mat Object: 1 MPI processes type: nest rows=72, cols=72 Matrix object: type=nest, rows=2, cols=2 MatNest structure: (0,0) : prefix="a00_", type=mpiaij, rows=48, cols=48 (0,1) : prefix="a01_", type=mpiaij, rows=48, cols=24 (1,0) : prefix="a10_", type=mpiaij, rows=24, cols=48 (1,1) : prefix="a11_", type=mpiaij, rows=24, cols=24 residual u = 1.60628e-06 residual p = 1.2094e-05 residual [u,p] = 1.22002e-05 discretization error u = 0.0106473 discretization error p = 1.85787 discretization error [u,p] = 1.8579 ./ex70 -user_ksp -ksp_view [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: No support for this operation for this object type [0]PETSC ERROR: Cannot locate function PCFieldSplitGetSubKSP_C in object [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.1, unknown [0]PETSC ERROR: ./ex70 on a darwin-gnu-fort-debug named gmbook.dhcp.lbnl.us by gbisht Fri Oct 31 09:28:38 2014 [0]PETSC ERROR: Configure options -with-hdf5-dir=/opt/local/ --with-blas-lapack-lib=/System/Library/Frameworks/Accelerate.framework/Versions/Current/Accelerate --download-parmetis=yes --download-metis=yes --download-sundials=yes --with-mpich=/opt/local --with-cc=/opt/local/bin/mpicc-mpich-gcc48 --with-cxx=/opt/local/bin/mpicxx-mpich-gcc48 --with-fc=/opt/local/bin/mpifort-mpich-gcc48 PETSC_ARCH=darwin-gnu-fort-debug --download-sowing=1 [0]PETSC ERROR: #1 PCFieldSplitGetSubKSP() line 1637 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/ksp/pc/impls/fieldsplit/fieldsplit.c [0]PETSC ERROR: #2 StokesSetupPC() line 113 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/snes/examples/tutorials/ex70.c [0]PETSC ERROR: #3 main() line 686 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/snes/examples/tutorials/ex70.c [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 Is my installation of PETSc incorrect? Would you suggest any particular starting examples that demonstrate how to use Schur complement as a preconditioner? Thanks, Gautam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gbisht at lbl.gov Fri Oct 31 11:49:10 2014 From: gbisht at lbl.gov (Gautam Bisht) Date: Fri, 31 Oct 2014 09:49:10 -0700 Subject: [petsc-users] TS_SUNDIALS/TS_PSEUDO overshoots the final time In-Reply-To: <87a94c27i7.fsf@jedbrown.org> References: <87a94c27i7.fsf@jedbrown.org> Message-ID: Hi Jed, On Thu, Oct 30, 2014 at 9:22 PM, Jed Brown wrote: > Gautam Bisht writes: > > > All, > > > > While running src/ts/examples/tutorials/ex8.c (in PETSc 3.5.1) for TSTYPE > > sundials and pseudo, I noticed that the solver overshoots the final time. > > Is this what you're looking for? > > > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSSetExactFinalTime.html Yes. But it appears only '-ts_exact_final_time INTERPOLATE' for '-ts_type sundials' does not overshoot the final time. ./ex8 -ts_monitor -ts_type pseudo -ts_final_time 0.01 -ts_exact_final_time STEPOVER 0 TS dt 0.001 time 0 1 TS dt 0.0011 time 0.0011 2 TS dt 0.00122331 time 0.00232331 3 TS dt 0.00136211 time 0.00368542 4 TS dt 0.00151873 time 0.00520415 5 TS dt 0.00169599 time 0.00690014 6 TS dt 0.00189725 time 0.00879739 7 TS dt 0.0021266 time 0.010924 steps 7 (0 rejected, 0 SNES fails), ftime 0.010924, nonlinits 7, linits 7 step 7 t=1.09239877e-02 h= 2.13e-03 |x|= 1.03e-01 |x_e|= 1.03e-01 |x-x_e|= 7.93e-04 ./ex8 -ts_monitor -ts_type pseudo -ts_final_time 0.01 -ts_exact_final_time MATCHSTEP 0 TS dt 0.001 time 0 1 TS dt 0.0011 time 0.0011 2 TS dt 0.00122331 time 0.00232331 3 TS dt 0.00136211 time 0.00368542 4 TS dt 0.00151873 time 0.00520415 5 TS dt 0.00169599 time 0.00690014 6 TS dt 0.00189725 time 0.00879739 7 TS dt 0.0021266 time 0.010924 steps 7 (0 rejected, 0 SNES fails), ftime 0.010924, nonlinits 7, linits 7 step 7 t=1.09239877e-02 h= 2.13e-03 |x|= 1.03e-01 |x_e|= 1.03e-01 |x-x_e|= 7.93e-04 ./ex8 -ts_monitor -ts_type pseudo -ts_final_time 0.01 -ts_exact_final_time INTERPOLATE 0 TS dt 0.001 time 0 1 TS dt 0.0011 time 0.0011 2 TS dt 0.00122331 time 0.00232331 3 TS dt 0.00136211 time 0.00368542 4 TS dt 0.00151873 time 0.00520415 5 TS dt 0.00169599 time 0.00690014 6 TS dt 0.00189725 time 0.00879739 [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: No support for this operation for this object type [0]PETSC ERROR: pseudo does not provide interpolation [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Release Version 3.5.1, unknown [0]PETSC ERROR: ./ex8 on a darwin-gnu-fort-debug named gmbook.dhcp.lbnl.us by gbisht Fri Oct 31 09:44:36 2014 [0]PETSC ERROR: Configure options -with-hdf5-dir=/opt/local/ --with-blas-lapack-lib=/System/Library/Frameworks/Accelerate.framework/Versions/Current/Accelerate --download-parmetis=yes --download-metis=yes --download-sundials=yes --with-mpich=/opt/local --with-cc=/opt/local/bin/mpicc-mpich-gcc48 --with-cxx=/opt/local/bin/mpicxx-mpich-gcc48 --with-fc=/opt/local/bin/mpifort-mpich-gcc48 PETSC_ARCH=darwin-gnu-fort-debug --download-sowing=1 [0]PETSC ERROR: #1 TSInterpolate() line 2582 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/ts/interface/ts.c [0]PETSC ERROR: #2 TSSolve() line 2763 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/ts/interface/ts.c [0]PETSC ERROR: #3 main() line 436 in /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/ts/examples/tutorials/ex8.c [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint at mcs.anl.gov---------- application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 ./ex8 -ts_monitor -ts_type sundials -ts_final_time 0.01 -ts_exact_final_time STEPOVER 0 TS dt 0.001 time 0 1 TS dt 0.0001 time 0.0001 2 TS dt 0.0001 time 0.0002 3 TS dt 0.0001 time 0.0003 4 TS dt 0.000199571 time 0.000499571 5 TS dt 0.000199571 time 0.000699141 6 TS dt 0.000199571 time 0.000898712 7 TS dt 0.000199571 time 0.00109828 8 TS dt 0.000404911 time 0.00150319 9 TS dt 0.000804269 time 0.00230746 10 TS dt 0.00141825 time 0.00372572 11 TS dt 0.00141825 time 0.00514397 12 TS dt 0.00141825 time 0.00656222 13 TS dt 0.00141825 time 0.00798047 14 TS dt 0.00141825 time 0.00939873 15 TS dt 0.00193698 time 0.0113357 steps 15 (0 rejected, 0 SNES fails), ftime 0.0113357, nonlinits 18, linits 18 step 15 t=1.13357082e-02 h= 1.94e-03 |x|= 1.07e-01 |x_e|= 1.07e-01 |x-x_e|= 2.57e-06 ./ex8 -ts_monitor -ts_type sundials -ts_final_time 0.01 -ts_exact_final_time MATCHSTEP 0 TS dt 0.001 time 0 1 TS dt 0.0001 time 0.0001 2 TS dt 0.0001 time 0.0002 3 TS dt 0.0001 time 0.0003 4 TS dt 0.000199571 time 0.000499571 5 TS dt 0.000199571 time 0.000699141 6 TS dt 0.000199571 time 0.000898712 7 TS dt 0.000199571 time 0.00109828 8 TS dt 0.000404911 time 0.00150319 9 TS dt 0.000804269 time 0.00230746 10 TS dt 0.00141825 time 0.00372572 11 TS dt 0.00141825 time 0.00514397 12 TS dt 0.00141825 time 0.00656222 13 TS dt 0.00141825 time 0.00798047 14 TS dt 0.00141825 time 0.00939873 15 TS dt 0.00193698 time 0.0113357 steps 15 (0 rejected, 0 SNES fails), ftime 0.0113357, nonlinits 18, linits 18 step 15 t=1.13357082e-02 h= 1.94e-03 |x|= 1.07e-01 |x_e|= 1.07e-01 |x-x_e|= 2.57e-06 ./ex8 -ts_monitor -ts_type sundials -ts_final_time 0.01 -ts_exact_final_time INTERPOLATE 0 TS dt 0.001 time 0 1 TS dt 0.0001 time 0.0001 2 TS dt 0.0001 time 0.0002 3 TS dt 0.0001 time 0.0003 4 TS dt 0.000199571 time 0.000499571 5 TS dt 0.000199571 time 0.000699141 6 TS dt 0.000199571 time 0.000898712 7 TS dt 0.000199571 time 0.00109828 8 TS dt 0.000404911 time 0.00150319 9 TS dt 0.000804269 time 0.00230746 10 TS dt 0.00141825 time 0.00372572 11 TS dt 0.00141825 time 0.00514397 12 TS dt 0.00141825 time 0.00656222 13 TS dt 0.00141825 time 0.00798047 14 TS dt 0.00141825 time 0.00939873 15 TS dt 0.00193698 time 0.01 steps 15 (0 rejected, 0 SNES fails), ftime 0.01, nonlinits 18, linits 18 step 15 t=1.00000000e-02 h= 1.94e-03 |x|= 9.52e-02 |x_e|= 9.52e-02 |x-x_e|= 2.11e-06 -Gautam. -------------- next part -------------- An HTML attachment was scrubbed... URL: From knepley at gmail.com Fri Oct 31 12:50:42 2014 From: knepley at gmail.com (Matthew Knepley) Date: Fri, 31 Oct 2014 12:50:42 -0500 Subject: [petsc-users] petsc-3.5.2: ex12 with Neumann BC In-Reply-To: <5453AE09.5090806@avignon.inra.fr> References: <524430F2.2020507@avignon.inra.fr> <524D3D5D.5090301@avignon.inra.fr> <525804C2.3000101@avignon.inra.fr> <5453AE09.5090806@avignon.inra.fr> Message-ID: On Fri, Oct 31, 2014 at 10:43 AM, Olivier Bonnefon < olivier.bonnefon at avignon.inra.fr> wrote: > Hello, > > I'm working on the snes/examples/tutorial/ex12 version 3.5.2. > > I didn't succed to run the simplest case with Neumann BC: > > ./ex12 -run_type full -refinement_limit 0.0 -bc_type neumann > -interpolate 1 -petscspace_order 1 -show_initial -dm_plex_print_fem 1 > -show_solution -bd_petscspace_order 1 -snes_linesearch_monitor > -snes_monitor -ksp_monitor_true_residual -snes_converged_reason > -ksp_converged_reason > > This leads to dofs negatives values. > I do not understand what you mean. Please always mail the full error message. > Do you know the options to get a correct result with Neumann BC ? > There are some tests here: https://bitbucket.org/petsc/petsc/src/fced3c3f9e703542693913793d15321603e40fe6/config/builder.py?at=master#cl-257 Matt > Regards, > Olivier Bonnefon > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From epscodes at gmail.com Fri Oct 31 13:48:16 2014 From: epscodes at gmail.com (Xiangdong) Date: Fri, 31 Oct 2014 14:48:16 -0400 Subject: [petsc-users] DMDA with runtime dof>1 Message-ID: Hello everyone, For the DMDA with dof>1, it seems that we always define a struct 'Field' with dof doubles. Once we have this struct, it is easy to convert from global vector to multi-dimension arrays (e.g the function DMDAVecGetArray). My question is, if my dof is determined at runtime, how can I define a struct with dof doubles? Thank you. Best, Xiangdong -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Oct 31 14:37:23 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 31 Oct 2014 14:37:23 -0500 Subject: [petsc-users] DMDA with runtime dof>1 In-Reply-To: References: Message-ID: Use DMDAVecGetArrayDOF() Barry > On Oct 31, 2014, at 1:48 PM, Xiangdong wrote: > > Hello everyone, > > For the DMDA with dof>1, it seems that we always define a struct 'Field' with dof doubles. Once we have this struct, it is easy to convert from global vector to multi-dimension arrays (e.g the function DMDAVecGetArray). > > My question is, if my dof is determined at runtime, how can I define a struct with dof doubles? > > Thank you. > > Best, > Xiangdong From lb2653 at columbia.edu Fri Oct 31 15:11:23 2014 From: lb2653 at columbia.edu (Luc Berger-Vergiat) Date: Fri, 31 Oct 2014 16:11:23 -0400 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <5B51768E-D3B7-4FE9-A2D7-A540C9750DC7@mcs.anl.gov> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> <544FE57C.8020303@columbi.edu> <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> <54524973.8010103@columbi.edu> <5B51768E-D3B7-4FE9-A2D7-A540C9750DC7@mcs.anl.gov> Message-ID: <5453ECEB.2090207@columbi.edu> So when I run with -no-signal-handler I get an error from valgrind saying that it received a signal and was aborting... Is there a way to prune a little what PETSc will get from argc/argv in order to let it run correctly? I also ran an example on my local computer with the same parameters and with 4 mpi ranks (that's all my desktop can do) and got the following outputs: luc at euler:~/research/simulations/Petsc_FS/parallel_ISFS/10elem_test/4cores$ mpirun -n 4 /usr/bin/valgrind --leak-check=full --tool=memcheck --track-origins=yes -q /home/luc/research/feap_repo/ShearBands/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale F I N I T E E L E M E N T A N A L Y S I S P R O G R A M FEAP (C) Regents of the University of California All Rights Reserved. VERSION: Release 8.3.19 DATE: 29 March 2011 Files are set as: Status Filename Input (read ) : Exists ILU_0001 Output (write) : Exists OLU_0001 Restart (read ) : New RLU_0001 Restart (write) : New RLU_0001 Plots (write) : New PLU_0001 Caution, existing write files will be overwritten. Are filenames correct? ( y or n; s = stop) :y R U N N I N G F E A P P R O B L E M N O W --> Please report errors by e-mail to: feap at ce.berkeley.edu Saving Parallel data to PLU_000000.pvtu ==7933== Syscall param writev(vector[...]) points to uninitialised byte(s) ==7933== at 0x6C2EF57: writev (writev.c:49) ==7933== by 0x7360F30: MPL_large_writev (mplsock.c:32) ==7933== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) ==7933== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) ==7933== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) ==7933== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) ==7933== by 0x6625309: PMPI_Isend (isend.c:122) ==7933== by 0x656671D: PMPI_ISEND (isendf.c:267) ==7933== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) ==7933== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) ==7933== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) ==7933== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) ==7933== Address 0x8959098 is 8 bytes inside a block of size 336 alloc'd ==7933== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7933== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) ==7933== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) ==7933== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) ==7933== by 0x1B36855: dmumps_ (dmumps_part1.F:665) ==7933== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7933== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7933== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) ==7933== by 0xB58981: MatSolve (matrix.c:3122) ==7933== by 0x152174C: PCApply_LU (lu.c:198) ==7933== by 0x14BAB70: PCApply (precon.c:440) ==7933== by 0x164AB13: KSP_PCApply (kspimpl.h:230) ==7933== Uninitialised value was created by a stack allocation ==7933== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) ==7933== ==7932== Syscall param writev(vector[...]) points to uninitialised byte(s) ==7932== at 0x6C2EF57: writev (writev.c:49) ==7932== by 0x7360F30: MPL_large_writev (mplsock.c:32) ==7932== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) ==7932== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) ==7932== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) ==7932== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) ==7932== by 0x6625309: PMPI_Isend (isend.c:122) ==7932== by 0x656671D: PMPI_ISEND (isendf.c:267) ==7932== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) ==7932== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) ==7932== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) ==7932== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) ==7932== Address 0x86e4448 is 8 bytes inside a block of size 336 alloc'd ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7932== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) ==7932== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) ==7932== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) ==7932== by 0x1B36855: dmumps_ (dmumps_part1.F:665) ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7932== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) ==7932== by 0xB58981: MatSolve (matrix.c:3122) ==7932== by 0x152174C: PCApply_LU (lu.c:198) ==7932== by 0x14BAB70: PCApply (precon.c:440) ==7932== by 0x164AB13: KSP_PCApply (kspimpl.h:230) ==7932== Uninitialised value was created by a stack allocation ==7932== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) ==7932== ==7934== Syscall param writev(vector[...]) points to uninitialised byte(s) ==7934== at 0x6C2EF57: writev (writev.c:49) ==7934== by 0x7360F30: MPL_large_writev (mplsock.c:32) ==7934== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) ==7934== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) ==7934== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) ==7934== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) ==7934== by 0x6625309: PMPI_Isend (isend.c:122) ==7934== by 0x656671D: PMPI_ISEND (isendf.c:267) ==7934== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) ==7934== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) ==7934== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) ==7934== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) ==7934== Address 0x89ec648 is 8 bytes inside a block of size 336 alloc'd ==7934== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7934== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) ==7934== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) ==7934== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) ==7934== by 0x1B36855: dmumps_ (dmumps_part1.F:665) ==7934== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7934== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7934== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) ==7934== by 0xB58981: MatSolve (matrix.c:3122) ==7934== by 0x152174C: PCApply_LU (lu.c:198) ==7934== by 0x14BAB70: PCApply (precon.c:440) ==7934== by 0x164AB13: KSP_PCApply (kspimpl.h:230) ==7934== Uninitialised value was created by a stack allocation ==7934== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) ==7934== ==7935== Syscall param writev(vector[...]) points to uninitialised byte(s) ==7935== at 0x6C2EF57: writev (writev.c:49) ==7935== by 0x7360F30: MPL_large_writev (mplsock.c:32) ==7935== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) ==7935== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) ==7935== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) ==7935== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) ==7935== by 0x6625309: PMPI_Isend (isend.c:122) ==7935== by 0x656671D: PMPI_ISEND (isendf.c:267) ==7935== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) ==7935== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) ==7935== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) ==7935== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) ==7935== Address 0x85d8968 is 8 bytes inside a block of size 336 alloc'd ==7935== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7935== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) ==7935== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) ==7935== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) ==7935== by 0x1B36855: dmumps_ (dmumps_part1.F:665) ==7935== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7935== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7935== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) ==7935== by 0xB58981: MatSolve (matrix.c:3122) ==7935== by 0x152174C: PCApply_LU (lu.c:198) ==7935== by 0x14BAB70: PCApply (precon.c:440) ==7935== by 0x164AB13: KSP_PCApply (kspimpl.h:230) ==7935== Uninitialised value was created by a stack allocation ==7935== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) ==7935== Saving Parallel data to PLU_000001.pvtu Saving Parallel data to PLU_000002.pvtu Saving Parallel data to PLU_000003.pvtu Saving Parallel data to PLU_000004.pvtu Saving Parallel data to PLU_000005.pvtu Saving Parallel data to PLU_000006.pvtu Saving Parallel data to PLU_000007.pvtu Saving Parallel data to PLU_000008.pvtu Saving Parallel data to PLU_000009.pvtu Saving Parallel data to PLU_000010.pvtu ==7932== 13 bytes in 13 blocks are definitely lost in loss record 7 of 51 ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7932== by 0x1BBB3E1: mumps_397.3460 (mumps_static_mapping.F:3746) ==7932== by 0x1BB8283: __mumps_static_mapping_MOD_mumps_369 (mumps_static_mapping.F:302) ==7932== by 0x1A4BBF5: dmumps_537_ (dmumps_part5.F:1706) ==7932== by 0x1A550AC: dmumps_26_ (dmumps_part5.F:447) ==7932== by 0x1B346D6: dmumps_ (dmumps_part1.F:409) ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7932== by 0x13B2F2B: MatLUFactorSymbolic_AIJMUMPS (mumps.c:972) ==7932== by 0xB54FB5: MatLUFactorSymbolic (matrix.c:2842) ==7932== by 0x15205A9: PCSetUp_LU (lu.c:127) ==7932== by 0x14C0685: PCSetUp (precon.c:902) ==7932== ==7932== 13 bytes in 13 blocks are definitely lost in loss record 8 of 51 ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==7932== by 0x1BBB5C7: mumps_397.3460 (mumps_static_mapping.F:3746) ==7932== by 0x1BB8283: __mumps_static_mapping_MOD_mumps_369 (mumps_static_mapping.F:302) ==7932== by 0x1A4BBF5: dmumps_537_ (dmumps_part5.F:1706) ==7932== by 0x1A550AC: dmumps_26_ (dmumps_part5.F:447) ==7932== by 0x1B346D6: dmumps_ (dmumps_part1.F:409) ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) ==7932== by 0x13B2F2B: MatLUFactorSymbolic_AIJMUMPS (mumps.c:972) ==7932== by 0xB54FB5: MatLUFactorSymbolic (matrix.c:2842) ==7932== by 0x15205A9: PCSetUp_LU (lu.c:127) ==7932== by 0x14C0685: PCSetUp (precon.c:902) ==7932== All these mainly come from mumps and I don't think that they would create the memory problem that I described earlier. It really seems that the issue comes with larger problems with more mpi ranks. Best, Luc On 10/30/2014 12:23 PM, Barry Smith wrote: > Run with the additional PETSc option -no_signal_handler then PETSc won?t mess with the signals and it may get you past this point. > > Barry > > > >> On Oct 30, 2014, at 9:21 AM, Luc Berger-Vergiat wrote: >> >> Sorry for the late reply, it took longer than I thought. >> So a little update on my situation: >> I have to use a custom version of valgrind on CETUS which has to be linked to my code using -Wl,-e,_start_valgrind at compilation (I also add object file and libraries). >> After that I can run my code with the following arguments: >> feap --ignore-ranges=0x4000000000000-0x4063000000000,0x003fdc0000000-0x003fe00000000 --suppressions=/soft/perftools/valgrind/cnk-baseline.supp >> but I can't use the usual petsc argument (-ksp_type -pc_type ...) since valgrind does not recognize them. >> So I decided to use the PETSC_OPTIONS='-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale', to pass my arguments to PETSc. >> However I am a little worried that Petsc still receive the command line arguments that are used by valgrind since I get the following stderr from valgrind: >> >> stderr[0]: ==1== by 0x38BEA3F: handle_SCSS_change (m_signals.c:963) >> stderr[0]: ==1== by 0x38C13A7: vgPlain_do_sys_sigaction (m_signals.c:1114) >> stderr[0]: ==1== by 0x3962FBF: vgSysWrap_linux_sys_rt_sigaction_before (syswrap-linux.c:3073) >> stderr[0]: ==1== by 0x3928BF7: vgPlain_client_syscall (syswrap-main.c:1464) >> stderr[0]: ==1== by 0x3925F4B: vgPlain_scheduler (scheduler.c:1061) >> stderr[0]: ==1== by 0x3965EB3: run_a_thread_NORETURN (syswrap-linux.c:103) >> stderr[0]: > >> stderr[0]: sched status: >> stderr[0]: running_tid=1 >> stderr[0]: >> stderr[0]: Thread 1: status = VgTs_Runnable >> stderr[0]: ==1== at 0x34290E4: __libc_sigaction (sigaction.c:80) >> stderr[0]: ==1== by 0x3BFF3A7: signal (signal.c:49) >> stderr[0]: ==1== by 0x1E710DF: PetscPushSignalHandler (in /projects/shearbands/ShearBands/parfeap/feap) >> stderr[0]: ==1== by 0x18BEA87: PetscOptionsCheckInitial_Private (in /projects/shearbands/ShearBands/parfeap/feap) >> stderr[0]: ==1== by 0x18E132F: petscinitialize (in /projects/shearbands/ShearBands/parfeap/feap) >> stderr[0]: ==1== by 0x1027557: pstart (in /projects/shearbands/ShearBands/parfeap/feap) >> stderr[0]: ==1== by 0x1000B1F: MAIN__ (feap83.f:213) >> stderr[0]: ==1== by 0x342ABD7: main (fmain.c:21) >> stderr[0]: >> stderr[0]: >> stderr[0]: Note: see also the FAQ in the source distribution. >> stderr[0]: It contains workarounds to several common problems. >> stderr[0]: In particular, if Valgrind aborted or crashed after >> stderr[0]: identifying problems in your program, there's a good chance >> stderr[0]: that fixing those problems will prevent Valgrind aborting or >> stderr[0]: crashing, especially if it happened in m_mallocfree.c. >> stderr[0]: >> stderr[0]: If that doesn't help, please report this bug to: www.valgrind.org >> stderr[0]: >> stderr[0]: In the bug report, send all the above text, the valgrind >> stderr[0]: version, and what OS and version you are using. Thank >> stderr[0]: s. >> stderr[0]: >> >> I am only showing the output of rank[0] but it seems that all ranks have about the same error message. >> Since my problem happens in petscinitialize I have little possibilities to check what's wrong... >> Any ideas? >> Best, >> Luc >> >> On 10/28/2014 02:53 PM, Barry Smith wrote: >>> You don?t care about checking for leaks. I use >>> >>> -q --tool=memcheck --num-callers=20 --track-origins=yes >>> >>> >>>> On Oct 28, 2014, at 1:50 PM, Luc Berger-Vergiat >>>> wrote: >>>> >>>> Yes, I am running with --leak-check=full >>>> Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... >>>> Hopefully the queue will go up fast and I can give you some details about the issue. >>>> >>>> Best, >>>> Luc >>>> >>>> On 10/28/2014 02:25 PM, Barry Smith wrote: >>>> >>>>> You need to pass some options to valgrind telling it to check for memory corruption issues >>>>> >>>>> >>>>> >>>>>> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat >>>>>> wrote: >>>>>> >>>>>> Ok, I'm recompiling PETSc in debug mode then. >>>>>> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >>>>>> Would this work for example: >>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >>>>>> >>>>>> >>>>>> Best, >>>>>> Luc >>>>>> >>>>>> On 10/28/2014 12:33 PM, Barry Smith wrote: >>>>>> >>>>>>> Hmm, this should never happen. In the code >>>>>>> >>>>>>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>>>>>> for (i=0; iB->rmap->n; i++) { >>>>>>> for (j=0; jilen[i]; j++) { >>>>>>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>>>>>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>>>>>> >>>>>>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>>>>>> >>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>>>>>> >>>>>>> >>>>>>> >>>>>>> Barry >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>>>>>> Here is my submission command: >>>>>>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>>>>>> >>>>>>>> Here is "4nodes_32cores_LU": >>>>>>>> #!/bin/sh >>>>>>>> >>>>>>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>>>>>> echo "Cobalt location args: $LOCARGS" >&2 >>>>>>>> >>>>>>>> ################################ >>>>>>>> # 32 cores on 4 nodes jobs # >>>>>>>> ################################ >>>>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>>>>>> >>>>>>>> I get the following error message: >>>>>>>> >>>>>>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>>>>> [7]PETSC ERROR: Argument out of range >>>>>>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>>>>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>>>>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>>>>>> >>>>>>>> -lblas" - >>>>>>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>>>>>> >>>>>>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>>>>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>>>>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>>>>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>>>>>> >>>>>>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>>>>>> >>>>>>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>>>>>> >>>>>>>> Thanks for the help >>>>>>>> Best, >>>>>>>> Luc >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jed at jedbrown.org Fri Oct 31 16:13:09 2014 From: jed at jedbrown.org (Jed Brown) Date: Fri, 31 Oct 2014 14:13:09 -0700 Subject: [petsc-users] TS_SUNDIALS/TS_PSEUDO overshoots the final time In-Reply-To: References: <87a94c27i7.fsf@jedbrown.org> Message-ID: <8738a4ymca.fsf@jedbrown.org> Gautam Bisht writes: > Hi Jed, > > > On Thu, Oct 30, 2014 at 9:22 PM, Jed Brown wrote: > >> Gautam Bisht writes: >> >> > All, >> > >> > While running src/ts/examples/tutorials/ex8.c (in PETSc 3.5.1) for TSTYPE >> > sundials and pseudo, I noticed that the solver overshoots the final time. >> >> Is this what you're looking for? >> >> >> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/TS/TSSetExactFinalTime.html > > > Yes. But it appears only '-ts_exact_final_time INTERPOLATE' for '-ts_type > sundials' does not overshoot the final time. TSPSEUDO is not time-accurate anyway so there should be no concept of "final time" other than infinity. I think Sundials does only supports STEPOVER and INTERPOLATE, so MATCHSTEP should probably error instead. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 818 bytes Desc: not available URL: From jychang48 at gmail.com Fri Oct 31 16:15:25 2014 From: jychang48 at gmail.com (Justin Chang) Date: Fri, 31 Oct 2014 16:15:25 -0500 Subject: [petsc-users] Reading data from a file into a DM created vector In-Reply-To: References: Message-ID: <491840EA-7604-4E78-B152-4A9C96F3A691@gmail.com> Matt, Thanks for the response. One more question: If I go with approach 2, will manually setting the constraints/indices and its values be compatible with the DMPlexSNESComputeResidual/JacobianFEM routines? When I look at the source code of those routines it seems the constrained values are added to the local solution vectors via DMPlexInsertBoundaryValuesFEM. If I choose not to use DMPlexAddBoundary and wish to manually declare my boundary values, i assume I should call DMPlexProjectField with mode INSERT_BC_VALUES? Thanks, Justin Sent from my iPhone > On Oct 30, 2014, at 4:24 PM, Matthew Knepley wrote: > >> On Thu, Oct 30, 2014 at 4:16 PM, Justin Chang wrote: >> Matt, thanks for the quick response. >> >> What about for (dirichlet) boundary conditions? Would it be possible to do something similar for those, like using those PetscSectionSetConstraint functions? > > Yes. There are generally two ways of handling Dirichlet conditions: > > 1) Replace those rows of the Jacobian with the identity and put the boundary value in the rhs. I find > this cumbersome for nonlinear solves. > > 2) Remove these unknowns from the system using PetscSectionSetConstraintDof() and ConstratinIndices(). > > The DMPlexAddBoundary() functions are automating this processing by marking boundaries using DMLabels, > and then constraining dofs on those boundaries. > > Thanks, > > Matt > >> Thanks, >> Justin >> >>> On Thu, Oct 30, 2014 at 3:35 PM, Matthew Knepley wrote: >>>> On Thu, Oct 30, 2014 at 3:29 PM, Justin Chang wrote: >>>> Hi all, >>>> >>>> So I am writing an FEM code where it reads input data (e.g., auxiliary coefficients, source terms, etc) from a text file. I have preprocessed the data text file so that each vertex point has its corresponding data. For instance, if my source term for a diffusion problem has a sin or cos function of the coordinates, then this data is already tabulated and simply needs to be fed into my PETSc program. The data text file containing both the auxiliary data and the coordinate/connectivities will also be used to provide the input arguments for the DMPlexCreateFromDAG() function. >>>> >>>> Normally, I would hard-code the sin/cos functions into the source code itself, but i want my program to be "universal" in that it can take as input any user-defined function for not only these auxiliaries but for other things like boundary conditions. I see that PETSc has a lot of examples on how to read data into vectors and matrices, but I guess my question is is there a way to project data from a text file into a vector that has already been created with a defined DM structure? >>> >>> If you have the functions available, I think it is far more universal to use the function itself, since then you can be independent >>> of mesh and discretization when specifying input and BC. >>> >>> However, if you want to read it in, and you guarantee that it matches the mesh and discretization, I think the easiest thing to do is >>> demand that it come in the same order as the vertices and use >>> >>> VecGetArray(V, &a); >>> for (v = vStart, i = 0; v < vEnd; ++v) { >>> PetscSectionGetDof(s, v, &dof); >>> PetscSectionGetOffset(s, v, &off); >>> for (d = 0; d < dof; ++d) a[off+d] = text_data[i++]; >>> } >>> VecRestoreArray(V, &a); >>> >>> Matt >>> >>>> Thanks, >>>> Justin >>> >>> >>> >>> -- >>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >>> -- Norbert Wiener > > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsmith at mcs.anl.gov Fri Oct 31 16:48:56 2014 From: bsmith at mcs.anl.gov (Barry Smith) Date: Fri, 31 Oct 2014 16:48:56 -0500 Subject: [petsc-users] Lookup error in PetscTableFind() In-Reply-To: <5453ECEB.2090207@columbi.edu> References: <544E943F.5070007@purdue.edu> <544FB07E.1020502@columbi.edu> <68E7C94C-4D7C-43EC-BC90-CBC00747660D@mcs.anl.gov> <544FD2BB.6070500@columbi.edu> <4D0B23B5-69EB-4E98-993C-90BC15B5A0CE@mcs.anl.gov> <544FE57C.8020303@columbi.edu> <7955DA16-318F-4E9B-B563-B8FBAB623095@mcs.anl.gov> <54524973.8010103@columbi.edu> <5B51768E-D3B7-4FE9-A2D7-A540C9750DC7@mcs.anl.gov> <5453ECEB.2090207@columbi.edu> Message-ID: I would first go back and build another PETSC_ARCH without optimization and run your job that previously crashed with that and see if it still crashes? If it runs then I would one at a time add back the various optimization flags -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction to see which triggers the problem. You could also overload your desktop with the much bigger problem to see if it finds anything else (sure it will run slowly but so what). Barry > On Oct 31, 2014, at 3:11 PM, Luc Berger-Vergiat wrote: > > So when I run with -no-signal-handler I get an error from valgrind saying that it received a signal and was aborting... > Is there a way to prune a little what PETSc will get from argc/argv in order to let it run correctly? > > I also ran an example on my local computer with the same parameters and with 4 mpi ranks (that's all my desktop can do) and got the following outputs: > > luc at euler:~/research/simulations/Petsc_FS/parallel_ISFS/10elem_test/4cores$ mpirun -n 4 /usr/bin/valgrind --leak-check=full --tool=memcheck --track-origins=yes -q /home/luc/research/feap_repo/ShearBands/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale > > > F I N I T E E L E M E N T A N A L Y S I S P R O G R A M > > FEAP (C) Regents of the University of California > All Rights Reserved. > VERSION: Release 8.3.19 > DATE: 29 March 2011 > > Files are set as: Status Filename > > Input (read ) : Exists ILU_0001 > Output (write) : Exists OLU_0001 > Restart (read ) : New RLU_0001 > Restart (write) : New RLU_0001 > Plots (write) : New PLU_0001 > > Caution, existing write files will be overwritten. > > Are filenames correct? ( y or n; s = stop) :y > > R U N N I N G F E A P P R O B L E M N O W > > --> Please report errors by e-mail to: > feap at ce.berkeley.edu > > Saving Parallel data to PLU_000000.pvtu > ==7933== Syscall param writev(vector[...]) points to uninitialised byte(s) > ==7933== at 0x6C2EF57: writev (writev.c:49) > ==7933== by 0x7360F30: MPL_large_writev (mplsock.c:32) > ==7933== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) > ==7933== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) > ==7933== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) > ==7933== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) > ==7933== by 0x6625309: PMPI_Isend (isend.c:122) > ==7933== by 0x656671D: PMPI_ISEND (isendf.c:267) > ==7933== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) > ==7933== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) > ==7933== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) > ==7933== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) > ==7933== Address 0x8959098 is 8 bytes inside a block of size 336 alloc'd > ==7933== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7933== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) > ==7933== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) > ==7933== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) > ==7933== by 0x1B36855: dmumps_ (dmumps_part1.F:665) > ==7933== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7933== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7933== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) > ==7933== by 0xB58981: MatSolve (matrix.c:3122) > ==7933== by 0x152174C: PCApply_LU (lu.c:198) > ==7933== by 0x14BAB70: PCApply (precon.c:440) > ==7933== by 0x164AB13: KSP_PCApply (kspimpl.h:230) > ==7933== Uninitialised value was created by a stack allocation > ==7933== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) > ==7933== > ==7932== Syscall param writev(vector[...]) points to uninitialised byte(s) > ==7932== at 0x6C2EF57: writev (writev.c:49) > ==7932== by 0x7360F30: MPL_large_writev (mplsock.c:32) > ==7932== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) > ==7932== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) > ==7932== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) > ==7932== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) > ==7932== by 0x6625309: PMPI_Isend (isend.c:122) > ==7932== by 0x656671D: PMPI_ISEND (isendf.c:267) > ==7932== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) > ==7932== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) > ==7932== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) > ==7932== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) > ==7932== Address 0x86e4448 is 8 bytes inside a block of size 336 alloc'd > ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7932== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) > ==7932== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) > ==7932== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) > ==7932== by 0x1B36855: dmumps_ (dmumps_part1.F:665) > ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7932== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) > ==7932== by 0xB58981: MatSolve (matrix.c:3122) > ==7932== by 0x152174C: PCApply_LU (lu.c:198) > ==7932== by 0x14BAB70: PCApply (precon.c:440) > ==7932== by 0x164AB13: KSP_PCApply (kspimpl.h:230) > ==7932== Uninitialised value was created by a stack allocation > ==7932== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) > ==7932== > ==7934== Syscall param writev(vector[...]) points to uninitialised byte(s) > ==7934== at 0x6C2EF57: writev (writev.c:49) > ==7934== by 0x7360F30: MPL_large_writev (mplsock.c:32) > ==7934== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) > ==7934== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) > ==7934== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) > ==7934== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) > ==7934== by 0x6625309: PMPI_Isend (isend.c:122) > ==7934== by 0x656671D: PMPI_ISEND (isendf.c:267) > ==7934== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) > ==7934== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) > ==7934== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) > ==7934== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) > ==7934== Address 0x89ec648 is 8 bytes inside a block of size 336 alloc'd > ==7934== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7934== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) > ==7934== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) > ==7934== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) > ==7934== by 0x1B36855: dmumps_ (dmumps_part1.F:665) > ==7934== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7934== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7934== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) > ==7934== by 0xB58981: MatSolve (matrix.c:3122) > ==7934== by 0x152174C: PCApply_LU (lu.c:198) > ==7934== by 0x14BAB70: PCApply (precon.c:440) > ==7934== by 0x164AB13: KSP_PCApply (kspimpl.h:230) > ==7934== Uninitialised value was created by a stack allocation > ==7934== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) > ==7934== > ==7935== Syscall param writev(vector[...]) points to uninitialised byte(s) > ==7935== at 0x6C2EF57: writev (writev.c:49) > ==7935== by 0x7360F30: MPL_large_writev (mplsock.c:32) > ==7935== by 0x65AB588: MPIDU_Sock_writev (sock_immed.i:610) > ==7935== by 0x65940B3: MPIDI_CH3_iSendv (ch3_isendv.c:84) > ==7935== by 0x65862AC: MPIDI_CH3_EagerContigIsend (ch3u_eager.c:550) > ==7935== by 0x658AB5C: MPID_Isend (mpid_isend.c:131) > ==7935== by 0x6625309: PMPI_Isend (isend.c:122) > ==7935== by 0x656671D: PMPI_ISEND (isendf.c:267) > ==7935== by 0x1AE7745: __dmumps_comm_buffer_MOD_dmumps_62 (dmumps_comm_buffer.F:567) > ==7935== by 0x1B6D1FC: dmumps_242_ (dmumps_part2.F:739) > ==7935== by 0x1AB264C: dmumps_249_ (dmumps_part8.F:6541) > ==7935== by 0x1AAB0F6: dmumps_245_ (dmumps_part8.F:3885) > ==7935== Address 0x85d8968 is 8 bytes inside a block of size 336 alloc'd > ==7935== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7935== by 0x1AE9465: __dmumps_comm_buffer_MOD_dmumps_2 (dmumps_comm_buffer.F:175) > ==7935== by 0x1AE9852: __dmumps_comm_buffer_MOD_dmumps_55 (dmumps_comm_buffer.F:123) > ==7935== by 0x1AC5FD5: dmumps_301_ (dmumps_part8.F:989) > ==7935== by 0x1B36855: dmumps_ (dmumps_part1.F:665) > ==7935== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7935== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7935== by 0x13AF869: MatSolve_MUMPS (mumps.c:606) > ==7935== by 0xB58981: MatSolve (matrix.c:3122) > ==7935== by 0x152174C: PCApply_LU (lu.c:198) > ==7935== by 0x14BAB70: PCApply (precon.c:440) > ==7935== by 0x164AB13: KSP_PCApply (kspimpl.h:230) > ==7935== Uninitialised value was created by a stack allocation > ==7935== at 0x1AAE33D: dmumps_249_ (dmumps_part8.F:5817) > ==7935== > Saving Parallel data to PLU_000001.pvtu > Saving Parallel data to PLU_000002.pvtu > Saving Parallel data to PLU_000003.pvtu > Saving Parallel data to PLU_000004.pvtu > Saving Parallel data to PLU_000005.pvtu > Saving Parallel data to PLU_000006.pvtu > Saving Parallel data to PLU_000007.pvtu > Saving Parallel data to PLU_000008.pvtu > Saving Parallel data to PLU_000009.pvtu > Saving Parallel data to PLU_000010.pvtu > ==7932== 13 bytes in 13 blocks are definitely lost in loss record 7 of 51 > ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7932== by 0x1BBB3E1: mumps_397.3460 (mumps_static_mapping.F:3746) > ==7932== by 0x1BB8283: __mumps_static_mapping_MOD_mumps_369 (mumps_static_mapping.F:302) > ==7932== by 0x1A4BBF5: dmumps_537_ (dmumps_part5.F:1706) > ==7932== by 0x1A550AC: dmumps_26_ (dmumps_part5.F:447) > ==7932== by 0x1B346D6: dmumps_ (dmumps_part1.F:409) > ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7932== by 0x13B2F2B: MatLUFactorSymbolic_AIJMUMPS (mumps.c:972) > ==7932== by 0xB54FB5: MatLUFactorSymbolic (matrix.c:2842) > ==7932== by 0x15205A9: PCSetUp_LU (lu.c:127) > ==7932== by 0x14C0685: PCSetUp (precon.c:902) > ==7932== > ==7932== 13 bytes in 13 blocks are definitely lost in loss record 8 of 51 > ==7932== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==7932== by 0x1BBB5C7: mumps_397.3460 (mumps_static_mapping.F:3746) > ==7932== by 0x1BB8283: __mumps_static_mapping_MOD_mumps_369 (mumps_static_mapping.F:302) > ==7932== by 0x1A4BBF5: dmumps_537_ (dmumps_part5.F:1706) > ==7932== by 0x1A550AC: dmumps_26_ (dmumps_part5.F:447) > ==7932== by 0x1B346D6: dmumps_ (dmumps_part1.F:409) > ==7932== by 0x1A170C7: dmumps_f77_ (dmumps_part3.F:6651) > ==7932== by 0x19EDDCA: dmumps_c (mumps_c.c:422) > ==7932== by 0x13B2F2B: MatLUFactorSymbolic_AIJMUMPS (mumps.c:972) > ==7932== by 0xB54FB5: MatLUFactorSymbolic (matrix.c:2842) > ==7932== by 0x15205A9: PCSetUp_LU (lu.c:127) > ==7932== by 0x14C0685: PCSetUp (precon.c:902) > ==7932== > All these mainly come from mumps and I don't think that they would create the memory problem that I described earlier. > It really seems that the issue comes with larger problems with more mpi ranks. > Best, > Luc > > On 10/30/2014 12:23 PM, Barry Smith wrote: >> Run with the additional PETSc option -no_signal_handler then PETSc won?t mess with the signals and it may get you past this point. >> >> Barry >> >> >> >> >>> On Oct 30, 2014, at 9:21 AM, Luc Berger-Vergiat >>> wrote: >>> >>> Sorry for the late reply, it took longer than I thought. >>> So a little update on my situation: >>> I have to use a custom version of valgrind on CETUS which has to be linked to my code using -Wl,-e,_start_valgrind at compilation (I also add object file and libraries). >>> After that I can run my code with the following arguments: >>> feap --ignore-ranges=0x4000000000000-0x4063000000000,0x003fdc0000000-0x003fe00000000 --suppressions=/soft/perftools/valgrind/cnk-baseline.supp >>> but I can't use the usual petsc argument (-ksp_type -pc_type ...) since valgrind does not recognize them. >>> So I decided to use the PETSC_OPTIONS='-ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale', to pass my arguments to PETSc. >>> However I am a little worried that Petsc still receive the command line arguments that are used by valgrind since I get the following stderr from valgrind: >>> >>> stderr[0]: ==1== by 0x38BEA3F: handle_SCSS_change (m_signals.c:963) >>> stderr[0]: ==1== by 0x38C13A7: vgPlain_do_sys_sigaction (m_signals.c:1114) >>> stderr[0]: ==1== by 0x3962FBF: vgSysWrap_linux_sys_rt_sigaction_before (syswrap-linux.c:3073) >>> stderr[0]: ==1== by 0x3928BF7: vgPlain_client_syscall (syswrap-main.c:1464) >>> stderr[0]: ==1== by 0x3925F4B: vgPlain_scheduler (scheduler.c:1061) >>> stderr[0]: ==1== by 0x3965EB3: run_a_thread_NORETURN (syswrap-linux.c:103) >>> stderr[0]: >>> >> >> >>> stderr[0]: sched status: >>> stderr[0]: running_tid=1 >>> stderr[0]: >>> stderr[0]: Thread 1: status = VgTs_Runnable >>> stderr[0]: ==1== at 0x34290E4: __libc_sigaction (sigaction.c:80) >>> stderr[0]: ==1== by 0x3BFF3A7: signal (signal.c:49) >>> stderr[0]: ==1== by 0x1E710DF: PetscPushSignalHandler (in /projects/shearbands/ShearBands/parfeap/feap) >>> stderr[0]: ==1== by 0x18BEA87: PetscOptionsCheckInitial_Private (in /projects/shearbands/ShearBands/parfeap/feap) >>> stderr[0]: ==1== by 0x18E132F: petscinitialize (in /projects/shearbands/ShearBands/parfeap/feap) >>> stderr[0]: ==1== by 0x1027557: pstart (in /projects/shearbands/ShearBands/parfeap/feap) >>> stderr[0]: ==1== by 0x1000B1F: MAIN__ (feap83.f:213) >>> stderr[0]: ==1== by 0x342ABD7: main (fmain.c:21) >>> stderr[0]: >>> stderr[0]: >>> stderr[0]: Note: see also the FAQ in the source distribution. >>> stderr[0]: It contains workarounds to several common problems. >>> stderr[0]: In particular, if Valgrind aborted or crashed after >>> stderr[0]: identifying problems in your program, there's a good chance >>> stderr[0]: that fixing those problems will prevent Valgrind aborting or >>> stderr[0]: crashing, especially if it happened in m_mallocfree.c. >>> stderr[0]: >>> stderr[0]: If that doesn't help, please report this bug to: >>> www.valgrind.org >>> >>> stderr[0]: >>> stderr[0]: In the bug report, send all the above text, the valgrind >>> stderr[0]: version, and what OS and version you are using. Thank >>> stderr[0]: s. >>> stderr[0]: >>> >>> I am only showing the output of rank[0] but it seems that all ranks have about the same error message. >>> Since my problem happens in petscinitialize I have little possibilities to check what's wrong... >>> Any ideas? >>> Best, >>> Luc >>> >>> On 10/28/2014 02:53 PM, Barry Smith wrote: >>> >>>> You don?t care about checking for leaks. I use >>>> >>>> -q --tool=memcheck --num-callers=20 --track-origins=yes >>>> >>>> >>>> >>>>> On Oct 28, 2014, at 1:50 PM, Luc Berger-Vergiat >>>>> >>>>> wrote: >>>>> >>>>> Yes, I am running with --leak-check=full >>>>> Reconfiguring and recompiling the whole library and my code in debug mode does take quite some time on CETUS/MIRA... >>>>> Hopefully the queue will go up fast and I can give you some details about the issue. >>>>> >>>>> Best, >>>>> Luc >>>>> >>>>> On 10/28/2014 02:25 PM, Barry Smith wrote: >>>>> >>>>> >>>>>> You need to pass some options to valgrind telling it to check for memory corruption issues >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On Oct 28, 2014, at 12:30 PM, Luc Berger-Vergiat >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>> Ok, I'm recompiling PETSc in debug mode then. >>>>>>> Do you know what the call sequence should be on CETUS to get valgrind attached to PETSc? >>>>>>> Would this work for example: >>>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /usr/bin/valgrind --log-file=valgrind.log.%p /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale < /projects/shearbands/job1/yesfile >>>>>>> >>>>>>> >>>>>>> Best, >>>>>>> Luc >>>>>>> >>>>>>> On 10/28/2014 12:33 PM, Barry Smith wrote: >>>>>>> >>>>>>> >>>>>>>> Hmm, this should never happen. In the code >>>>>>>> >>>>>>>> ierr = PetscTableCreate(aij->B->rmap->n,mat->cmap->N+1,&gid1_lid1);CHKERRQ(ierr); >>>>>>>> for (i=0; iB->rmap->n; i++) { >>>>>>>> for (j=0; jilen[i]; j++) { >>>>>>>> PetscInt data,gid1 = aj[B->i[i] + j] + 1; >>>>>>>> ierr = PetscTableFind(gid1_lid1,gid1,&data);CHKERRQ(ierr); >>>>>>>> >>>>>>>> Now mat->cmap->N+1 is the total number of columns in the matrix and gid1 are column entries which must always be smaller. Most likely there has been memory corruption somewhere before this point. Can you run with valgrind? >>>>>>>> >>>>>>>> >>>>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Oct 28, 2014, at 10:04 AM, Luc Berger-Vergiat >>>>>>>>> >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> I am running a code on CETUS and I use PETSc for as a linear solver. >>>>>>>>> Here is my submission command: >>>>>>>>> qsub -A shearbands -t 60 -n 4 -O 4nodes_32cores_Mult --mode script 4nodes_32cores_LU >>>>>>>>> >>>>>>>>> Here is "4nodes_32cores_LU": >>>>>>>>> #!/bin/sh >>>>>>>>> >>>>>>>>> LOCARGS="--block $COBALT_PARTNAME ${COBALT_CORNER:+--corner} $COBALT_CORNER ${COBALT_SHAPE:+--shape} $COBALT_SHAPE" >>>>>>>>> echo "Cobalt location args: $LOCARGS" >&2 >>>>>>>>> >>>>>>>>> ################################ >>>>>>>>> # 32 cores on 4 nodes jobs # >>>>>>>>> ################################ >>>>>>>>> runjob --np 32 -p 8 --block $COBALT_PARTNAME --cwd /projects/shearbands/job1/200/4nodes_32cores/LU --verbose=INFO --envs FEAPHOME8_3=/projects/shearbands/ShearBands352 PETSC_DIR=/projects/shearbands/petsc-3.5.2 PETSC_ARCH=arch-linux2-c-opt : /projects/shearbands/ShearBands352/parfeap/feap -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps -ksp_diagonal_scale -malloc_log mlog -log_summary time.log < /projects/shearbands/job1/yesfile >>>>>>>>> >>>>>>>>> I get the following error message: >>>>>>>>> >>>>>>>>> [7]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- >>>>>>>>> [7]PETSC ERROR: Argument out of range >>>>>>>>> [7]PETSC ERROR: Petsc Release Version 3.5.2, unknown >>>>>>>>> [7]PETSC ERROR: key 532150 is greater than largest key allowed 459888 >>>>>>>>> [7]PETSC ERROR: Configure options --known-mpi-int64_t=1 --download-cmake=1 --download-hypre=1 --download-metis=1 --download-parmetis=1 --download-plapack=1 --download-superlu_dist=1 --download-mumps=1 --download-ml=1 --known-bits-per-byte=8 --known-level1-dcache-assoc=0 --known-level1-dcache-linesize=32 --known-level1-dcache-size=32768 --known-memcmp-ok=1 --known-mpi-c-double-complex=1 --known-mpi-long-double=1 --known-mpi-shared-libraries=0 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-sizeof-char=1 --known-sizeof-double=8 --known-sizeof-float=4 --known-sizeof-int=4 --known-sizeof-long-long=8 --known-sizeof-long=8 --known-sizeof-short=2 --known-sizeof-size_t=8 --known-sizeof-void-p=8 --with-batch=1 --with-blacs-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-blacs-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-blas-lapack-lib="-L/soft/libraries/alcf/current/gcc/LAPACK/lib -llapack -L/soft/libraries/alcf/current/gcc/BLAS/lib >>>>>>>>> >>>>>>>>> >>>>>>>>> -lblas" - >>>>>>>>> -with-cc=mpicc --with-cxx=mpicxx --with-debugging=0 --with-fc=mpif90 --with-fortran-kernels=1 --with-is-color-value-type=short --with-scalapack-include=/soft/libraries/alcf/current/gcc/SCALAPACK/ --with-scalapack-lib=/soft/libraries/alcf/current/gcc/SCALAPACK/lib/libscalapack.a --with-shared-libraries=0 --with-x=0 -COPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -CXXOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" -FOPTFLAGS=" -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction" >>>>>>>>> >>>>>>>>> [7]PETSC ERROR: #1 PetscTableFind() line 126 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/include/petscctable.h >>>>>>>>> [7]PETSC ERROR: #2 MatSetUpMultiply_MPIAIJ() line 33 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mmaij.c >>>>>>>>> [7]PETSC ERROR: #3 MatAssemblyEnd_MPIAIJ() line 702 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/impls/aij/mpi/mpiaij.c >>>>>>>>> [7]PETSC ERROR: #4 MatAssemblyEnd() line 4900 in /gpfs/mira-fs1/projects/shearbands/petsc-3.5.2/src/mat/interface/matrix.c >>>>>>>>> >>>>>>>>> Well at least that is what I think comes out after I read all the jammed up messages from my MPI processes... >>>>>>>>> >>>>>>>>> I would guess that I am trying to allocate more memory than I should which seems strange since the same problem runs fine on 2 nodes with 16 cores/node >>>>>>>>> >>>>>>>>> Thanks for the help >>>>>>>>> Best, >>>>>>>>> Luc >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>> >>>>> >>>>> > From gbisht at lbl.gov Fri Oct 31 21:47:25 2014 From: gbisht at lbl.gov (Gautam Bisht) Date: Fri, 31 Oct 2014 19:47:25 -0700 Subject: [petsc-users] Error while running src/snes/examples/tutorials/ex70.c In-Reply-To: References: Message-ID: All, I found an earlier email thread on this topic. So, disregard this email. -Gautam. On Fri, Oct 31, 2014 at 9:35 AM, Gautam Bisht wrote: > Hi, > > I'm trying to learn how to use Schur complement as a preconditioner. I > ran src/snes/examples/tutorials/ex70.c for the two cases: > > ./ex70 -user_pc -ksp_view > KSP Object: 1 MPI processes > type: gmres > GMRES: restart=30, using Classical (unmodified) Gram-Schmidt > Orthogonalization with no iterative refinement > GMRES: happy breakdown tolerance 1e-30 > maximum iterations=10000, initial guess is zero > tolerances: relative=1e-05, absolute=1e-50, divergence=10000 > left preconditioning > using PRECONDITIONED norm type for convergence test > PC Object: 1 MPI processes > type: none > linear system matrix = precond matrix: > Mat Object: 1 MPI processes > type: nest > rows=72, cols=72 > Matrix object: > type=nest, rows=2, cols=2 > MatNest structure: > (0,0) : prefix="a00_", type=mpiaij, rows=48, cols=48 > (0,1) : prefix="a01_", type=mpiaij, rows=48, cols=24 > (1,0) : prefix="a10_", type=mpiaij, rows=24, cols=48 > (1,1) : prefix="a11_", type=mpiaij, rows=24, cols=24 > residual u = 1.60628e-06 > residual p = 1.2094e-05 > residual [u,p] = 1.22002e-05 > discretization error u = 0.0106473 > discretization error p = 1.85787 > discretization error [u,p] = 1.8579 > > > ./ex70 -user_ksp -ksp_view > [0]PETSC ERROR: --------------------- Error Message > -------------------------------------------------------------- > [0]PETSC ERROR: No support for this operation for this object type > [0]PETSC ERROR: Cannot locate function PCFieldSplitGetSubKSP_C in object > [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html > for trouble shooting. > [0]PETSC ERROR: Petsc Release Version 3.5.1, unknown > [0]PETSC ERROR: ./ex70 on a darwin-gnu-fort-debug named > gmbook.dhcp.lbnl.us by gbisht Fri Oct 31 09:28:38 2014 > [0]PETSC ERROR: Configure options -with-hdf5-dir=/opt/local/ > --with-blas-lapack-lib=/System/Library/Frameworks/Accelerate.framework/Versions/Current/Accelerate > --download-parmetis=yes --download-metis=yes --download-sundials=yes > --with-mpich=/opt/local --with-cc=/opt/local/bin/mpicc-mpich-gcc48 > --with-cxx=/opt/local/bin/mpicxx-mpich-gcc48 > --with-fc=/opt/local/bin/mpifort-mpich-gcc48 > PETSC_ARCH=darwin-gnu-fort-debug --download-sowing=1 > [0]PETSC ERROR: #1 PCFieldSplitGetSubKSP() line 1637 in > /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/ksp/pc/impls/fieldsplit/fieldsplit.c > [0]PETSC ERROR: #2 StokesSetupPC() line 113 in > /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/snes/examples/tutorials/ex70.c > [0]PETSC ERROR: #3 main() line 686 in > /Users/gbisht/Research/Models/PETSc/petsc3.5.1/src/snes/examples/tutorials/ex70.c > [0]PETSC ERROR: ----------------End of Error Message -------send entire > error message to petsc-maint at mcs.anl.gov---------- > application called MPI_Abort(MPI_COMM_WORLD, 56) - process 0 > > > Is my installation of PETSc incorrect? Would you suggest any particular > starting examples that demonstrate how to use Schur complement as a > preconditioner? > > Thanks, > Gautam. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: