[petsc-users] (edit GAMG) petsc 3.7.2 memory usage is much higher when compared to 3.6.1
Mark Adams
mfadams at lbl.gov
Thu Jul 7 13:30:32 CDT 2016
>
>
> > GAMG: createProl 6 1.0 1.0036e+01 1.0 3.68e+07 1.2 1.5e+06 2.7e+03
> 1.3e+03 1 0 23 1 31 1 0 23 1 31 1332
> > Graph 12 1.0 6.0783e+00 1.0 4.52e+06 1.1 3.8e+05 9.0e+02
> 2.5e+02 1 0 6 0 6 1 0 6 0 6 279
> > MIS/Agg 6 1.0 9.5831e-02 1.2 0.00e+00 0.0 2.6e+05 1.1e+03
> 4.1e+01 0 0 4 0 1 0 0 4 0 1 0
> > SA: col data 6 1.0 7.7358e-01 1.0 0.00e+00 0.0 6.7e+05 2.9e+03
> 7.8e+02 0 0 10 0 19 0 0 10 0 19 0
> > SA: frmProl0 6 1.0 1.0759e+00 1.0 0.00e+00 0.0 6.2e+04 7.6e+03
> 6.0e+01 0 0 1 0 1 0 0 1 0 1 0
> > GAMG: partLevel 6 1.0 3.8136e+01 1.0 9.09e+08 1.1 3.8e+05 5.0e+04
> 5.4e+02 6 3 6 4 13 6 3 6 4 14 9013
> > repartition 6 1.0 2.7910e+00 1.0 0.00e+00 0.0 4.6e+04 1.3e+02
> 1.6e+02 0 0 1 0 4 0 0 1 0 4 0
> > Invert-Sort 6 1.0 2.5045e+00 2.0 0.00e+00 0.0 0.0e+00 0.0e+00
> 2.4e+01 0 0 0 0 1 0 0 0 0 1 0
> > Move A 6 1.0 1.4832e+01 1.0 0.00e+00 0.0 8.5e+04 1.7e+05
> 1.1e+02 2 0 1 3 3 2 0 1 3 3 0
> > Move P 6 1.0 1.2023e+01 1.0 0.00e+00 0.0 2.4e+04 3.8e+03
> 1.1e+02 2 0 0 0 3 2 0 0 0 3 0
>
THe number of these calls (eg, 6) is the number of grids that are setup.
> > PCSetUp 100 1.0 1.1212e+02 1.0 1.84e+10 1.1 3.2e+06 1.3e+05
> 2.2e+03 17 52 49 84 54 17 52 49 84 54 62052
> > PCSetUpOnBlocks 40 1.0 1.0386e+00 1.2 1.95e+08 1.2 0.0e+00 0.0e+00
> 0.0e+00 0 1 0 0 0 0 1 0 0 0 67368
> > PCApply 380 1.0 2.0034e+01 1.1 8.60e+09 1.1 1.5e+06 9.9e+03
> 6.0e+01 3 24 22 3 1 3 24 22 3 1 161973
> > SFSetGraph 12 1.0 4.9813e-0310.2 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > SFBcastBegin 47 1.0 3.3110e-02 2.6 0.00e+00 0.0 2.6e+05 1.1e+03
> 6.0e+00 0 0 4 0 0 0 0 4 0 0 0
> > SFBcastEnd 47 1.0 1.3497e-02 3.6 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > SFReduceBegin 6 1.0 1.8593e-02 4.2 0.00e+00 0.0 7.2e+04 4.9e+02
> 6.0e+00 0 0 1 0 0 0 0 1 0 0 0
> > SFReduceEnd 6 1.0 7.1628e-0318.5 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > BuildTwoSided 12 1.0 3.5771e-02 2.5 0.00e+00 0.0 5.0e+04 4.0e+00
> 1.2e+01 0 0 1 0 0 0 0 1 0 0 0
> > ----------------------------------------------------------------------
> > --------------------------------------------------
> >
> > Memory usage is given in bytes:
> >
> > Object Type Creations Destructions Memory Descendants'
> Mem.
> > Reports information only for process 0.
> >
> > --- Event Stage 0: Main Stage
> >
> > Matrix 302 299 1992700700 0.
> > Matrix Partitioning 6 6 3888 0.
> > Matrix Coarsen 6 6 3768 0.
> > Vector 600 600 1582204168 0.
> > Vector Scatter 87 87 5614432 0.
> > Krylov Solver 11 11 59472 0.
> > Preconditioner 11 11 11120 0.
> > PetscRandom 1 1 638 0.
> > Viewer 1 0 0 0.
> > Index Set 247 247 9008420 0.
> > Star Forest Bipartite Graph 12 12 10176 0.
> > ======================================================================
> > ==================================================
> >
> > And for petsc 3.6.1:
> >
> > Using Petsc Development GIT revision: v3.6.1-307-g26c82d3 GIT Date:
> > 2015-08-06 11:50:34 -0500
> >
> > Max Max/Min Avg Total
> > Time (sec): 5.515e+02 1.00001 5.515e+02
> > Objects: 1.231e+03 1.00490 1.226e+03
> > Flops: 3.431e+10 1.12609 3.253e+10 1.301e+13
> > Flops/sec: 6.222e+07 1.12609 5.899e+07 2.359e+10
> > MPI Messages: 4.432e+04 7.84165 1.504e+04 6.016e+06
> > MPI Message Lengths: 2.236e+09 12.61261 5.027e+04 3.024e+11
> > MPI Reductions: 4.012e+03 1.00150
> >
> > Flop counting convention: 1 flop = 1 real number operation of type
> (multiply/divide/add/subtract)
> > e.g., VecAXPY() for real vectors of length N
> --> 2N flops
> > and VecAXPY() for complex vectors of
> > length N --> 8N flops
> >
> > Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages
> --- -- Message Lengths -- -- Reductions --
> > Avg %Total Avg %Total counts
> %Total Avg %Total counts %Total
> > 0: Main Stage: 5.5145e+02 100.0% 1.3011e+13 100.0% 6.007e+06
> 99.9% 5.020e+04 99.9% 3.999e+03 99.7%
> >
> > ----------------------------------------------------------------------
> > --------------------------------------------------
> > See the 'Profiling' chapter of the users' manual for details on
> interpreting output.
> > Phase summary info:
> > Count: number of times phase was executed
> > Time and Flops: Max - maximum over all processors
> > Ratio - ratio of maximum to minimum over all
> processors
> > Mess: number of messages sent
> > Avg. len: average message length (bytes)
> > Reduct: number of global reductions
> > Global: entire computation
> > Stage: stages of a computation. Set stages with PetscLogStagePush()
> and PetscLogStagePop().
> > %T - percent time in this phase %F - percent flops in this
> phase
> > %M - percent messages in this phase %L - percent message
> lengths in this phase
> > %R - percent reductions in this phase
> > Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time
> > over all processors)
> >
> ------------------------------------------------------------------------------------------------------------------------
> > Event Count Time (sec) Flops
> --- Global --- --- Stage --- Total
> > Max Ratio Max Ratio Max Ratio Mess Avg len
> Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
> > ----------------------------------------------------------------------
> > --------------------------------------------------
> >
> > --- Event Stage 0: Main Stage
> >
> > MatMult 500 1.0 1.0172e+01 1.2 6.68e+09 1.1 1.9e+06 9.9e+03
> 0.0e+00 2 19 31 6 0 2 19 31 6 0 247182
> > MatMultTranspose 120 1.0 6.9889e-01 1.2 3.56e+08 1.1 2.5e+05 1.4e+04
> 0.0e+00 0 1 4 1 0 0 1 4 1 0 197492
> > MatSolve 380 1.0 3.9310e+00 1.1 1.17e+09 1.1 1.3e+04 5.7e+01
> 6.0e+01 1 3 0 0 1 1 3 0 0 2 112069
> > MatSOR 120 1.0 1.3915e+01 1.1 6.73e+09 1.1 9.5e+05 7.4e+03
> 0.0e+00 2 20 16 2 0 2 20 16 2 0 182405
> > MatLUFactorSym 2 1.0 2.1180e-01 1.8 0.00e+00 0.0 0.0e+00 0.0e+00
> 1.0e+01 0 0 0 0 0 0 0 0 0 0 0
> > MatLUFactorNum 60 1.0 7.9378e+00 1.0 1.95e+08 1.2 0.0e+00 0.0e+00
> 0.0e+00 1 1 0 0 0 1 1 0 0 0 8814
> > MatILUFactorSym 1 1.0 2.3076e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatConvert 6 1.0 3.2693e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00
> 1.8e+01 0 0 0 0 0 0 0 0 0 0 0
> > MatScale 6 1.0 2.1923e-02 1.7 4.50e+06 1.1 2.4e+04 1.5e+03
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 77365
> > MatAssemblyBegin 266 1.0 1.0337e+01 4.4 0.00e+00 0.0 1.8e+05 3.8e+03
> 4.2e+02 1 0 3 0 10 1 0 3 0 10 0
> > MatAssemblyEnd 266 1.0 3.0336e+00 1.0 0.00e+00 0.0 4.1e+05 8.6e+02
> 4.7e+02 1 0 7 0 12 1 0 7 0 12 0
> > MatGetRow 6730366 1.1 8.6473e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatGetRowIJ 3 3.0 5.2931e-035550.2 0.00e+00 0.0 0.0e+00
> 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatGetSubMatrix 12 1.0 2.2689e+01 1.0 0.00e+00 0.0 1.1e+05 1.3e+05
> 1.9e+02 4 0 2 5 5 4 0 2 5 5 0
> > MatGetOrdering 3 3.0 6.5000e-03 5.4 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatPartitioning 6 1.0 2.9801e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00
> 1.4e+01 1 0 0 0 0 1 0 0 0 0 0
> > MatCoarsen 6 1.0 9.5374e-02 1.1 0.00e+00 0.0 2.5e+05 1.1e+03
> 3.8e+01 0 0 4 0 1 0 0 4 0 1 0
> > MatZeroEntries 22 1.0 6.1185e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatTranspose 6 1.0 1.9780e-01 1.1 0.00e+00 0.0 1.9e+05 8.6e+02
> 7.8e+01 0 0 3 0 2 0 0 3 0 2 0
> > MatPtAP 120 1.0 5.2996e+01 1.0 1.70e+10 1.1 9.7e+05 2.1e+05
> 4.2e+02 10 49 16 67 10 10 49 16 67 11 120900
> > MatPtAPSymbolic 12 1.0 5.8209e+00 1.0 0.00e+00 0.0 2.2e+05 3.7e+04
> 8.4e+01 1 0 4 3 2 1 0 4 3 2 0
> > MatPtAPNumeric 120 1.0 4.7185e+01 1.0 1.70e+10 1.1 7.6e+05 2.6e+05
> 3.4e+02 9 49 13 64 8 9 49 13 64 8 135789
> > MatTrnMatMult 3 1.0 1.1679e+00 1.0 3.22e+07 1.2 8.2e+04 8.0e+03
> 5.7e+01 0 0 1 0 1 0 0 1 0 1 9997
> > MatTrnMatMultSym 3 1.0 6.8366e-01 1.0 0.00e+00 0.0 6.9e+04 3.9e+03
> 5.1e+01 0 0 1 0 1 0 0 1 0 1 0
> > MatTrnMatMultNum 3 1.0 4.8513e-01 1.0 3.22e+07 1.2 1.3e+04 3.0e+04
> 6.0e+00 0 0 0 0 0 0 0 0 0 0 24069
> > MatGetLocalMat 126 1.0 1.1939e+00 1.3 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > MatGetBrAoCol 120 1.0 5.9887e-01 2.7 0.00e+00 0.0 5.7e+05 3.3e+04
> 0.0e+00 0 0 9 6 0 0 0 9 6 0 0
> > MatGetSymTrans 24 1.0 1.4878e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > VecDot 320 1.0 1.5860e+00 1.5 2.04e+08 1.1 0.0e+00 0.0e+00
> 3.2e+02 0 1 0 0 8 0 1 0 0 8 49574
> > VecMDot 260 1.0 1.8154e+00 2.5 3.70e+08 1.1 0.0e+00 0.0e+00
> 2.6e+02 0 1 0 0 6 0 1 0 0 7 78497
> > VecNorm 440 1.0 2.8876e+00 1.8 5.88e+08 1.1 0.0e+00 0.0e+00
> 4.4e+02 0 2 0 0 11 0 2 0 0 11 78281
> > VecScale 320 1.0 2.2738e-01 1.2 7.88e+07 1.1 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 133517
> > VecCopy 220 1.0 7.1162e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > VecSet 862 1.0 7.0683e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > VecAXPY 440 1.0 9.0657e-01 1.2 3.83e+08 1.1 0.0e+00 0.0e+00
> 0.0e+00 0 1 0 0 0 0 1 0 0 0 162612
> > VecAYPX 280 1.0 5.8935e-01 1.5 1.92e+08 1.1 0.0e+00 0.0e+00
> 0.0e+00 0 1 0 0 0 0 1 0 0 0 125070
> > VecMAXPY 300 1.0 9.7644e-01 1.2 4.98e+08 1.1 0.0e+00 0.0e+00
> 0.0e+00 0 1 0 0 0 0 1 0 0 0 196269
> > VecAssemblyBegin 234 1.0 5.0308e+00 5.5 0.00e+00 0.0 0.0e+00 0.0e+00
> 6.8e+02 1 0 0 0 17 1 0 0 0 17 0
> > VecAssemblyEnd 234 1.0 1.8253e-03 8.8 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > VecScatterBegin 1083 1.0 2.8195e-01 4.7 0.00e+00 0.0 3.8e+06 8.4e+03
> 2.0e+01 0 0 64 11 0 0 0 64 11 1 0
> > VecScatterEnd 1063 1.0 3.4924e+00 6.9 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > KSPGMRESOrthog 20 1.0 1.5598e+00 3.2 1.28e+08 1.1 0.0e+00 0.0e+00
> 2.0e+01 0 0 0 0 0 0 0 0 0 1 31503
> > KSPSetUp 222 1.0 9.7521e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00
> 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
> > KSPSolve 60 1.0 1.3742e+02 1.0 3.42e+10 1.1 5.7e+06 4.4e+04
> 3.2e+03 25100 95 83 79 25100 95 83 79 94396
> > PCGAMGGraph_AGG 6 1.0 5.7683e+00 1.0 4.50e+06 1.1 3.8e+05 9.1e+02
> 2.5e+02 1 0 6 0 6 1 0 6 0 6 294
> > PCGAMGCoarse_AGG 6 1.0 1.4101e+00 1.0 3.22e+07 1.2 4.0e+05 3.2e+03
> 1.4e+02 0 0 7 0 4 0 0 7 0 4 8280
> > PCGAMGProl_AGG 6 1.0 1.8976e+00 1.0 0.00e+00 0.0 7.2e+05 3.4e+03
> 8.6e+02 0 0 12 1 22 0 0 12 1 22 0
> > PCGAMGPOpt_AGG 6 1.0 5.7220e-06 0.0 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > GAMG: createProl 6 1.0 9.0840e+00 1.0 3.67e+07 1.2 1.5e+06 2.7e+03
> 1.3e+03 2 0 25 1 31 2 0 25 1 31 1472
> > Graph 12 1.0 5.7669e+00 1.0 4.50e+06 1.1 3.8e+05 9.1e+02
> 2.5e+02 1 0 6 0 6 1 0 6 0 6 294
> > MIS/Agg 6 1.0 9.5481e-02 1.1 0.00e+00 0.0 2.5e+05 1.1e+03
> 3.8e+01 0 0 4 0 1 0 0 4 0 1 0
> > SA: col data 6 1.0 8.5414e-01 1.0 0.00e+00 0.0 6.6e+05 3.0e+03
> 7.8e+02 0 0 11 1 19 0 0 11 1 20 0
> > SA: frmProl0 6 1.0 1.0123e+00 1.0 0.00e+00 0.0 6.2e+04 7.6e+03
> 6.0e+01 0 0 1 0 1 0 0 1 0 2 0
> > GAMG: partLevel 6 1.0 3.6150e+01 1.0 8.41e+08 1.1 3.5e+05 5.0e+04
> 5.3e+02 7 2 6 6 13 7 2 6 6 13 8804
> > repartition 6 1.0 3.8351e+00 1.0 0.00e+00 0.0 4.7e+04 1.3e+02
> 1.6e+02 1 0 1 0 4 1 0 1 0 4 0
> > Invert-Sort 6 1.0 4.4953e+00 1.8 0.00e+00 0.0 0.0e+00 0.0e+00
> 2.4e+01 1 0 0 0 1 1 0 0 0 1 0
> > Move A 6 1.0 1.0806e+01 1.0 0.00e+00 0.0 8.5e+04 1.6e+05
> 1.0e+02 2 0 1 5 3 2 0 1 5 3 0
> > Move P 6 1.0 1.1953e+01 1.0 0.00e+00 0.0 2.5e+04 3.6e+03
> 1.0e+02 2 0 0 0 3 2 0 0 0 3 0
> > PCSetUp 100 1.0 1.0166e+02 1.0 1.72e+10 1.1 2.7e+06 8.3e+04
> 2.2e+03 18 50 44 73 54 18 50 44 73 54 63848
> > PCSetUpOnBlocks 40 1.0 1.0812e+00 1.2 1.95e+08 1.2 0.0e+00 0.0e+00
> 0.0e+00 0 1 0 0 0 0 1 0 0 0 64711
> > PCApply 380 1.0 1.9359e+01 1.1 8.58e+09 1.1 1.4e+06 9.6e+03
> 6.0e+01 3 25 24 5 1 3 25 24 5 2 167605
> > SFSetGraph 12 1.0 3.5203e-03 6.8 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > SFBcastBegin 44 1.0 2.4242e-02 3.0 0.00e+00 0.0 2.5e+05 1.1e+03
> 6.0e+00 0 0 4 0 0 0 0 4 0 0 0
> > SFBcastEnd 44 1.0 3.0994e-02 8.6 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > SFReduceBegin 6 1.0 1.6784e-02 3.8 0.00e+00 0.0 7.1e+04 5.0e+02
> 6.0e+00 0 0 1 0 0 0 0 1 0 0 0
> > SFReduceEnd 6 1.0 8.6989e-0332.0 0.00e+00 0.0 0.0e+00 0.0e+00
> 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
> > ----------------------------------------------------------------------
> > --------------------------------------------------
> >
> > Memory usage is given in bytes:
> >
> > Object Type Creations Destructions Memory Descendants'
> Mem.
> > Reports information only for process 0.
> >
> > --- Event Stage 0: Main Stage
> >
> > Matrix 246 243 1730595756 0
> > Matrix Partitioning 6 6 3816 0
> > Matrix Coarsen 6 6 3720 0
> > Vector 602 602 1603749672 0
> > Vector Scatter 87 87 4291136 0
> > Krylov Solver 12 12 60416 0
> > Preconditioner 12 12 12040 0
> > Viewer 1 0 0 0
> > Index Set 247 247 9018060 0
> > Star Forest Bipartite Graph 12 12 10080 0
> > ======================================================================
> > ==================================================
> >
> > Any idea why there are more matrix created with version 3.7.2? I only
> have 2 MatCreate calls and 4 VecCreate calls in my code!, so I assume the
> others are internally created.
> >
> >
> > Thank you,
> >
> >
> > Hassan Raiesi, PhD
> >
> > Advanced Aerodynamics Department
> > Bombardier Aerospace
> >
> > hassan.raiesi at aero.bombardier.com
> >
> > 2351 boul. Alfred-Nobel (BAN1)
> > Ville Saint-Laurent, Québec, H4S 2A9
> >
> >
> >
> > Tél.
> > 514-855-5001 # 62204
> >
> >
> >
> > <image001.png>
> >
> >
> > CONFIDENTIALITY NOTICE - This communication may contain privileged or
> confidential information.
> > If you are not the intended recipient or received this communication
> > by error, please notify the sender and delete the message without
> copying, forwarding and/or disclosing it.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160707/914f1776/attachment-0001.html>
More information about the petsc-users
mailing list