Residual norms (max) for proj_mac_ solve. 0 KSP true resid max norm 1.280000000000e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 1.376614247597e+03 ||r(i)||/||b|| 1.075479880936e+00 2 KSP true resid max norm 1.033178293273e+03 ||r(i)||/||b|| 8.071705416194e-01 3 KSP true resid max norm 3.278697829915e+02 ||r(i)||/||b|| 2.561482679621e-01 4 KSP true resid max norm 1.016986450150e+02 ||r(i)||/||b|| 7.945206641793e-02 5 KSP true resid max norm 2.686955717433e+01 ||r(i)||/||b|| 2.099184154245e-02 6 KSP true resid max norm 7.177649462770e+00 ||r(i)||/||b|| 5.607538642789e-03 7 KSP true resid max norm 2.671473811730e+00 ||r(i)||/||b|| 2.087088915414e-03 8 KSP true resid max norm 6.067054040032e-01 ||r(i)||/||b|| 4.739885968775e-04 9 KSP true resid max norm 1.985646055546e-01 ||r(i)||/||b|| 1.551285980895e-04 10 KSP true resid max norm 6.388958601747e-02 ||r(i)||/||b|| 4.991373907615e-05 11 KSP true resid max norm 1.950029691216e-02 ||r(i)||/||b|| 1.523460696262e-05 12 KSP true resid max norm 5.147888965439e-03 ||r(i)||/||b|| 4.021788254249e-06 13 KSP true resid max norm 1.236600801349e-03 ||r(i)||/||b|| 9.660943760537e-07 14 KSP true resid max norm 2.863780246116e-04 ||r(i)||/||b|| 2.237328317278e-07 15 KSP true resid max norm 5.999300628901e-05 ||r(i)||/||b|| 4.686953616329e-08 16 KSP true resid max norm 1.433806028217e-05 ||r(i)||/||b|| 1.120160959545e-08 17 KSP true resid max norm 3.686873242259e-06 ||r(i)||/||b|| 2.880369720515e-09 18 KSP true resid max norm 7.108319550753e-07 ||r(i)||/||b|| 5.553374649026e-10 19 KSP true resid max norm 1.552980393171e-07 ||r(i)||/||b|| 1.213265932165e-10 20 KSP true resid max norm 4.633329808712e-08 ||r(i)||/||b|| 3.619788913056e-11 21 KSP true resid max norm 1.618172973394e-08 ||r(i)||/||b|| 1.264197635464e-11 22 KSP true resid max norm 5.064066499472e-09 ||r(i)||/||b|| 3.956301952712e-12 Linear proj_mac_ solve converged due to CONVERGED_RTOL iterations 22 KSP Object: (proj_mac_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (proj_mac_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=10017802, allocated nonzeros=10017802 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Residual norms (max) for proj_mac_ solve. 0 KSP true resid max norm 4.257776559457e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 1.091421751744e+03 ||r(i)||/||b|| 2.563360797597e-01 2 KSP true resid max norm 3.124170520167e+02 ||r(i)||/||b|| 7.337563342133e-02 3 KSP true resid max norm 4.851683768275e+01 ||r(i)||/||b|| 1.139487641149e-02 4 KSP true resid max norm 1.122256029749e+01 ||r(i)||/||b|| 2.635779529709e-03 5 KSP true resid max norm 3.785475474413e+00 ||r(i)||/||b|| 8.890733042355e-04 6 KSP true resid max norm 6.582681229079e-01 ||r(i)||/||b|| 1.546037265497e-04 7 KSP true resid max norm 2.151305120929e-01 ||r(i)||/||b|| 5.052649172373e-05 8 KSP true resid max norm 5.745703863991e-02 ||r(i)||/||b|| 1.349461105757e-05 9 KSP true resid max norm 1.334492796316e-02 ||r(i)||/||b|| 3.134248069810e-06 10 KSP true resid max norm 3.572056494818e-03 ||r(i)||/||b|| 8.389487905098e-07 11 KSP true resid max norm 1.245038586745e-03 ||r(i)||/||b|| 2.924152006004e-07 12 KSP true resid max norm 3.564772925984e-04 ||r(i)||/||b|| 8.372381397200e-08 13 KSP true resid max norm 1.060093829892e-04 ||r(i)||/||b|| 2.489782671985e-08 14 KSP true resid max norm 3.294202747384e-05 ||r(i)||/||b|| 7.736908457694e-09 15 KSP true resid max norm 8.774618663665e-06 ||r(i)||/||b|| 2.060845265394e-09 16 KSP true resid max norm 1.732849142400e-06 ||r(i)||/||b|| 4.069845183751e-10 17 KSP true resid max norm 4.846777130751e-07 ||r(i)||/||b|| 1.138335246829e-10 18 KSP true resid max norm 1.253891070974e-07 ||r(i)||/||b|| 2.944943337124e-11 19 KSP true resid max norm 2.457616687934e-08 ||r(i)||/||b|| 5.772065897811e-12 20 KSP true resid max norm 6.133859642432e-09 ||r(i)||/||b|| 1.440625067280e-12 21 KSP true resid max norm 1.504076863057e-09 ||r(i)||/||b|| 3.532540616102e-13 Linear proj_mac_ solve converged due to CONVERGED_RTOL iterations 21 KSP Object: (proj_mac_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (proj_mac_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=10017802, allocated nonzeros=10017802 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Residual norms (max) for visc_ solve. 0 KSP true resid max norm 1.184675832193e+01 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 6.298981006980e-03 ||r(i)||/||b|| 5.317050315207e-04 2 KSP true resid max norm 1.951081556811e-05 ||r(i)||/||b|| 1.646932860274e-06 3 KSP true resid max norm 3.797186076682e-08 ||r(i)||/||b|| 3.205253262956e-09 4 KSP true resid max norm 6.734421353904e-11 ||r(i)||/||b|| 5.684611073258e-12 5 KSP true resid max norm 3.149147609349e-13 ||r(i)||/||b|| 2.658235716280e-14 Linear visc_ solve converged due to CONVERGED_RTOL iterations 5 KSP Object: (visc_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (visc_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=26819942, allocated nonzeros=26819942 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Residual norms (max) for visc_ solve. 0 KSP true resid max norm 4.637166131635e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 2.779219091949e-03 ||r(i)||/||b|| 5.993356746459e-04 2 KSP true resid max norm 1.043985998876e-05 ||r(i)||/||b|| 2.251344828371e-06 3 KSP true resid max norm 3.616724172299e-08 ||r(i)||/||b|| 7.799427645314e-09 4 KSP true resid max norm 4.706063707130e-11 ||r(i)||/||b|| 1.014857689705e-11 Linear visc_ solve converged due to CONVERGED_RTOL iterations 4 KSP Object: (visc_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (visc_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=26819942, allocated nonzeros=26819942 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Residual norms (max) for visc_ solve. 0 KSP true resid max norm 4.821791348018e+00 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 4.519882997516e-03 ||r(i)||/||b|| 9.373866829336e-04 2 KSP true resid max norm 9.036999171291e-06 ||r(i)||/||b|| 1.874199549304e-06 3 KSP true resid max norm 2.546678015691e-08 ||r(i)||/||b|| 5.281601446189e-09 4 KSP true resid max norm 5.899011834565e-11 ||r(i)||/||b|| 1.223406698631e-11 5 KSP true resid max norm 2.255418074526e-13 ||r(i)||/||b|| 4.677552203609e-14 Linear visc_ solve converged due to CONVERGED_RTOL iterations 5 KSP Object: (visc_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (visc_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=26819942, allocated nonzeros=26819942 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines Residual norms (max) for proj_mac_ solve. 0 KSP true resid max norm 2.072026505973e+03 ||r(i)||/||b|| 1.000000000000e+00 1 KSP true resid max norm 1.054233874635e+03 ||r(i)||/||b|| 5.087936238248e-01 2 KSP true resid max norm 2.752712930595e+02 ||r(i)||/||b|| 1.328512411718e-01 3 KSP true resid max norm 4.129283519871e+01 ||r(i)||/||b|| 1.992871957944e-02 4 KSP true resid max norm 1.075825991874e+01 ||r(i)||/||b|| 5.192143965210e-03 5 KSP true resid max norm 4.240903410551e+00 ||r(i)||/||b|| 2.046741872425e-03 6 KSP true resid max norm 7.072058471829e-01 ||r(i)||/||b|| 3.413111971030e-04 7 KSP true resid max norm 2.275399865470e-01 ||r(i)||/||b|| 1.098151910176e-04 8 KSP true resid max norm 6.218704058759e-02 ||r(i)||/||b|| 3.001266654085e-05 9 KSP true resid max norm 1.420347057049e-02 ||r(i)||/||b|| 6.854869148414e-06 10 KSP true resid max norm 3.838023973628e-03 ||r(i)||/||b|| 1.852304477073e-06 11 KSP true resid max norm 1.124743464800e-03 ||r(i)||/||b|| 5.428229134895e-07 12 KSP true resid max norm 3.764715195302e-04 ||r(i)||/||b|| 1.816924245153e-07 13 KSP true resid max norm 1.133137550369e-04 ||r(i)||/||b|| 5.468740612642e-08 14 KSP true resid max norm 3.411420425437e-05 ||r(i)||/||b|| 1.646417367539e-08 15 KSP true resid max norm 8.938209475673e-06 ||r(i)||/||b|| 4.313752478506e-09 16 KSP true resid max norm 1.801537734991e-06 ||r(i)||/||b|| 8.694568963271e-10 17 KSP true resid max norm 5.021408355788e-07 ||r(i)||/||b|| 2.423428629562e-10 18 KSP true resid max norm 1.317764031228e-07 ||r(i)||/||b|| 6.359783658312e-11 19 KSP true resid max norm 2.528548748160e-08 ||r(i)||/||b|| 1.220326448948e-11 20 KSP true resid max norm 6.871459845570e-09 ||r(i)||/||b|| 3.316299200691e-12 21 KSP true resid max norm 1.970022367459e-09 ||r(i)||/||b|| 9.507708331818e-13 Linear proj_mac_ solve converged due to CONVERGED_RTOL iterations 21 KSP Object: (proj_mac_) 64 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=100, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. right preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: (proj_mac_) 64 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 1 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.25 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 1 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 1 Sweeps up 1 Sweeps on coarse 1 Relax down symmetric-SOR/Jacobi Relax up symmetric-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Not using CF-relaxation Not using more complex smoothers. Measure type local Coarsen type HMIS Interpolation type ext+i linear system matrix = precond matrix: Mat Object: 64 MPI processes type: mpiaij rows=1143009, cols=1143009 total: nonzeros=10017802, allocated nonzeros=10017802 total number of mallocs used during MatSetValues calls =0 not using I-node (on process 0) routines ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /global/homes/k/kallemov/svn_treb/EBAMRINS/execBCG/viscousDriver3d.Linux.64.CC.ftn.OPTHIGH.MPI.OPENMPCC.PETSC.ex on a arch-cori-knl-opt64-intel named nid02517 with 64 processors, by kallemov Thu Jan 25 11:48:52 2018 Using Petsc Release Version 3.8.3, unknown Max Max/Min Avg Total Time (sec): 5.293e+01 1.00351 5.282e+01 Objects: 2.420e+02 1.00000 2.420e+02 Flop: 2.353e+08 3.31607 1.568e+08 1.003e+10 Flop/sec: 4.459e+06 3.31480 2.968e+06 1.900e+08 MPI Messages: 2.924e+03 3.09857 1.800e+03 1.152e+05 MPI Message Lengths: 6.851e+06 3.60025 2.209e+03 2.545e+08 MPI Reductions: 3.040e+02 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flop and VecAXPY() for complex vectors of length N --> 8N flop Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 4.9496e+01 93.7% 5.3887e+09 53.7% 6.052e+04 52.5% 1.065e+03 48.2% 1.590e+02 52.3% 1: Projection: 1.9771e+00 3.7% 2.5955e+09 25.9% 2.847e+04 24.7% 4.805e+02 21.7% 7.000e+01 23.0% 2: Viscous: 1.3462e+00 2.5% 2.0503e+09 20.4% 2.621e+04 22.8% 6.638e+02 30.0% 7.400e+01 24.3% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecMDot 43 1.0 1.4662e-01 7.4 2.89e+07 4.0 0.0e+00 0.0e+00 4.3e+01 0 11 0 0 14 0 21 0 0 27 7546 VecNorm 92 1.0 1.6556e-01 7.2 2.69e+06 4.0 0.0e+00 0.0e+00 9.2e+01 0 1 0 0 30 0 2 0 0 58 621 VecScale 45 1.0 4.0824e-03 1.3 1.34e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 12599 VecCopy 94 1.0 4.5540e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 136 1.0 4.9772e-03 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 45 1.0 8.8465e-03 1.6 2.69e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 11628 VecAYPX 45 1.0 2.3110e-03 2.7 1.34e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 22257 VecMAXPY 88 1.0 5.2319e-02 3.5 6.04e+07 4.0 0.0e+00 0.0e+00 0.0e+00 0 23 0 0 0 0 43 0 0 0 44174 VecScatterBegin 88 1.0 9.4392e-03 2.1 0.00e+00 0.0 5.8e+04 1.9e+03 0.0e+00 0 0 51 45 0 0 0 96 92 0 0 VecScatterEnd 88 1.0 2.5504e-0152.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 45 1.0 3.9876e-02 2.1 4.03e+06 4.0 0.0e+00 0.0e+00 4.5e+01 0 2 0 0 15 0 3 0 0 28 3870 KSPGMRESOrthog 43 1.0 1.6332e-01 3.7 5.78e+07 4.0 0.0e+00 0.0e+00 4.3e+01 0 22 0 0 14 0 41 0 0 27 13549 KSPSetUp 1 1.0 1.1142e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 2 1.0 2.6495e+00 1.0 1.35e+08 3.7 5.8e+04 1.9e+03 1.4e+02 5 54 51 45 46 5100 96 92 87 2034 MatMult 88 1.0 3.0218e-01 2.8 3.79e+07 3.2 5.8e+04 1.9e+03 0.0e+00 0 17 51 45 0 0 31 96 92 0 5502 MatConvert 1 1.0 4.4204e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 1 0 0 0 0 3 0 MatAssemblyBegin 2 1.0 9.6836e-0211.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 1 0 0 0 0 1 0 MatAssemblyEnd 2 1.0 9.8383e-02 1.0 0.00e+00 0.0 1.3e+03 9.7e+02 1.2e+01 0 0 1 1 4 0 0 2 1 8 0 MatGetRowIJ 2 1.0 5.8579e-04 4.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 2 1.0 2.8458e-02 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 1 0 0 0 0 1 0 PCSetUp 1 1.0 6.7105e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 1 0 0 0 1 1 0 0 0 3 0 PCApply 88 1.0 1.6766e+00 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 3 0 0 0 0 0 --- Event Stage 1: Projection VecMDot 21 1.0 7.2136e-02 7.5 1.38e+07 4.0 0.0e+00 0.0e+00 2.1e+01 0 5 0 0 7 2 20 0 0 30 7320 VecNorm 45 1.0 7.7528e-0214.0 1.31e+06 4.0 0.0e+00 0.0e+00 4.5e+01 0 1 0 0 15 3 2 0 0 64 649 VecScale 22 1.0 5.5885e-04 2.9 6.57e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 44996 VecCopy 46 1.0 2.2123e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 66 1.0 2.3298e-03 3.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 22 1.0 8.7047e-04 4.7 1.31e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 57776 VecAYPX 22 1.0 9.1076e-04 4.1 6.57e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 27610 VecMAXPY 43 1.0 2.4585e-02 3.5 2.88e+07 4.0 0.0e+00 0.0e+00 0.0e+00 0 11 0 0 0 1 43 0 0 0 44911 VecScatterBegin 43 1.0 4.2541e-03 2.5 0.00e+00 0.0 2.8e+04 1.9e+03 0.0e+00 0 0 25 22 0 0 0100100 0 0 VecScatterEnd 43 1.0 1.2524e-0152.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecNormalize 22 1.0 1.3191e-02 6.0 1.97e+06 4.0 0.0e+00 0.0e+00 2.2e+01 0 1 0 0 7 0 3 0 0 31 5719 KSPGMRESOrthog 21 1.0 7.9416e-02 3.8 2.76e+07 4.0 0.0e+00 0.0e+00 2.1e+01 0 11 0 0 7 3 41 0 0 30 13299 KSPSolve 1 1.0 9.3349e-01 1.0 6.51e+07 3.7 2.8e+04 1.9e+03 6.6e+01 2 26 25 22 22 47100100100 94 2780 MatMult 43 1.0 1.4621e-01 2.9 1.85e+07 3.2 2.8e+04 1.9e+03 0.0e+00 0 8 25 22 0 4 31100100 0 5556 MatView 1 1.0 4.0412e-0360.3 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 0 0 0 0 0 1 0 PCApply 43 1.0 8.1242e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 37 0 0 0 0 0 --- Event Stage 2: Viscous VecMDot 14 1.0 5.0756e-0239.0 2.39e+06 4.0 0.0e+00 0.0e+00 1.4e+01 0 1 0 0 5 2 4 0 0 19 1802 VecNorm 37 1.0 6.4282e-0210.1 1.01e+06 4.0 0.0e+00 0.0e+00 3.7e+01 0 0 0 0 12 2 2 0 0 50 605 VecScale 17 1.0 4.2200e-04 3.4 5.07e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 46045 VecCopy 40 1.0 2.3332e-03 3.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 52 1.0 1.7800e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 17 1.0 8.4066e-04 5.5 1.01e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 2 0 0 0 46228 VecAYPX 17 1.0 8.6427e-04 4.2 5.07e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 22483 VecMAXPY 31 1.0 4.6134e-03 4.1 5.61e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 10 0 0 0 46579 VecScatterBegin 31 1.0 4.0817e-03 2.8 0.00e+00 0.0 2.4e+04 2.8e+03 0.0e+00 0 0 21 26 0 0 0 91 87 0 0 VecScatterEnd 31 1.0 1.0839e-0143.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecNormalize 17 1.0 8.0769e-03 4.7 1.52e+06 4.0 0.0e+00 0.0e+00 1.7e+01 0 1 0 0 6 0 3 0 0 23 7217 KSPGMRESOrthog 14 1.0 5.1428e-0221.2 4.78e+06 4.0 0.0e+00 0.0e+00 1.4e+01 0 2 0 0 5 2 9 0 0 19 3556 KSPSetUp 1 1.0 8.9097e-04 4.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 3 1.0 9.4874e-01 1.0 4.13e+07 2.5 2.4e+04 2.8e+03 5.5e+01 2 20 21 26 18 70100 91 87 74 2161 MatMult 31 1.0 1.3763e-01 2.2 3.46e+07 2.8 2.4e+04 2.8e+03 0.0e+00 0 16 21 26 0 6 79 91 87 0 11824 MatConvert 1 1.0 4.0958e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 1 3 0 0 0 5 0 MatAssemblyBegin 2 1.0 1.2679e-01803.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 1 4 0 0 0 3 0 MatAssemblyEnd 2 1.0 4.9350e-02 1.3 0.00e+00 0.0 1.5e+03 1.4e+03 1.2e+01 0 0 1 1 4 3 0 6 3 16 0 MatGetRowIJ 2 1.0 8.1062e-06 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 1.0 1.4201e-0293.4 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+00 0 0 0 0 1 0 0 0 0 4 0 PCSetUp 1 1.0 2.4881e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 1 18 0 0 0 5 0 PCApply 31 1.0 6.0053e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 40 0 0 0 0 0 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 121 139 11057880 0. Vector Scatter 1 2 2416 0. Index Set 2 2 12624 0. IS L to G Mapping 1 0 0 0. Krylov Solver 1 2 37872 0. Matrix 4 8 6887312 0. Preconditioner 1 2 3296 0. Viewer 2 1 888 0. --- Event Stage 1: Projection Vector 44 44 3618560 0. --- Event Stage 2: Viscous Vector 55 37 2801512 0. Vector Scatter 1 0 0 0. Index Set 2 2 19808 0. IS L to G Mapping 1 0 0 0. Krylov Solver 1 0 0 0. Matrix 4 0 0 0. Preconditioner 1 0 0 0. ======================================================================================================================== Average time to get PetscTime(): 4.05312e-07 Average time for MPI_Barrier(): 1.13964e-05 Average time for zero size MPI_Send(): 6.63847e-06 #PETSc Option Table entries: -ksp_view -log_summary -log_view -options_left -proj_mac_gamg_est_ksp_gmres_restart 50 -proj_mac_gamg_est_ksp_max_it 20 -proj_mac_ksp_converged_reason -proj_mac_ksp_max_it 100 -proj_mac_ksp_monitor_max -proj_mac_ksp_norm_type unpreconditioned -proj_mac_ksp_rtol 1.e-12 -proj_mac_ksp_type gmres -proj_mac_ksp_view -proj_mac_mg_levels_ksp_max_it 1 -proj_mac_mg_levels_ksp_richardson_scale 0.9 -proj_mac_mg_levels_ksp_type richardson -proj_mac_mg_levels_pc_type sor -proj_mac_pc_gamg_agg_nsmooths 1 -proj_mac_pc_gamg_repartition false -proj_mac_pc_gamg_square_graph 20 -proj_mac_pc_gamg_sym_graph true -proj_mac_pc_gamg_threshold .02 -proj_mac_pc_gamg_type agg -proj_mac_pc_gamg_verbose 2 -proj_mac_pc_hypre_boomeramg_agg_nl 1 -proj_mac_pc_hypre_boomeramg_coarsen_type HMIS -proj_mac_pc_hypre_boomeramg_interp_type ext+i -proj_mac_pc_hypre_boomeramg_no_CF true -proj_mac_pc_hypre_type boomeramg -proj_mac_pc_ml_EnergyMinimization 2 -proj_mac_pc_ml_PrintLevel 1 -proj_mac_pc_ml_Threshold 0.01 -proj_mac_pc_type hypre -visc_ksp_converged_reason -visc_ksp_max_it 100 -visc_ksp_monitor_max -visc_ksp_norm_type unpreconditioned -visc_ksp_rtol 1.e-12 -visc_ksp_view -visc_mg_levels_ksp_max_it 1 -visc_mg_levels_ksp_type richardson -visc_mg_levels_pc_type sor -visc_pc_gamg_agg_nsmooths 1 -visc_pc_gamg_repartition false -visc_pc_gamg_square_graph true -visc_pc_gamg_sym_graph true -visc_pc_gamg_threshold .02 -visc_pc_gamg_type agg -visc_pc_hypre_boomeramg_agg_nl 1 -visc_pc_hypre_boomeramg_coarsen_type HMIS -visc_pc_hypre_boomeramg_interp_type ext+i -visc_pc_hypre_boomeramg_no_CF true -visc_pc_hypre_type boomeramg -visc_pc_type hypre #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8 Configure options: --with-cc=cc --with-cxx=CC --with-fc=ftn COPTFLAGS=" -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" --download-metis=1 --download-hypre=1 --download-parmetis=1 --download-fblaslapack --with-debugging=0 --with-mpiexec=srun --with-batch=0 --with-memalign=64 --with-64-bit-indices=1 --known-mpi-shared-libraries=1 PETSC_ARCH=arch-cori-knl-opt64-intel --with-openmp=0 --download-p4est=0 --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel ----------------------------------------- Libraries compiled on Tue Jan 23 07:15:01 2018 on nid04222 Machine characteristics: Linux-4.4.49-92.11.1_3.0-cray_ari_c-x86_64-with-SuSE-12-x86_64 Using PETSc directory: /global/u2/m/madams/petsc_install/petsc Using PETSc arch: arch-cori-knl-opt64-intel ----------------------------------------- Using C compiler: cc -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd ${COPTFLAGS} ${CFLAGS} Using Fortran compiler: ftn -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd ${FOPTFLAGS} ${FFLAGS} ----------------------------------------- Using include paths: -I/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/include -I/global/u2/m/madams/petsc_install/petsc/include -I/global/u2/m/madams/petsc_install/petsc/include -I/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/include -I/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/include ----------------------------------------- Using C linker: cc Using Fortran linker: ftn Using libraries: -Wl,-rpath,/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/lib -L/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/lib -lpetsc -Wl,-rpath,/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/lib -L/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/lib -L/opt/cray/pe/libsci/17.09.1/INTEL/16.0/x86_64/lib -L/opt/cray/pe/hdf5-parallel/1.10.0.3/INTEL/16.0/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/pe/mpt/7.6.2/gni/mpich-intel/16.0/lib -L/opt/cray/udreg/2.3.2-6.0.4.0_12.2__g2f9c3ee.ari/lib64 -L/usr/common/software/darshan/3.1.4/lib -L/opt/cray/rca/2.2.15-6.0.4.1_13.1__g46acb0f.ari/lib64 -L/opt/cray/alps/6.4.1-6.0.4.0_7.2__g86d0f3d.ari/lib64 -L/opt/cray/xpmem/2.2.2-6.0.4.0_3.1__g43b0535.ari/lib64 -L/opt/cray/pe/pmi/5.0.12/lib64 -L/opt/cray/ugni/6.0.14-6.0.4.0_14.1__ge7db4a2.ari/lib64 -L/opt/cray/pe/atp/2.1.1/libApp -L/opt/cray/wlm_detect/1.2.1-6.0.4.0_22.1__gd26a3dc.ari/lib64 -L/usr/common/software/h5part/1.6.6/lib -L/opt/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64 -L/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64 -L/opt/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin -L/usr/lib64/gcc/x86_64-suse-linux/4.8 -L/usr/x86_64-suse-linux/lib -lHYPRE -lflapack -lfblas -lparmetis -lmetis -lhdf5hl_fortran_parallel -lhdf5_fortran_parallel -lmpichcxx_intel -lstdc++ -lmpichcxx_intel -lstdc++ -ldl -lfmpich -lmpichcxx -ldarshan -ldarshan-stubs -lz -lpthread -lhdf5_hl_parallel -lm -lhdf5_parallel -lmpichf90_intel -lrt -lugni -lpmi -lsci_intel_mpi -lsci_intel -lmpich_intel -lalpslli -lwlm_detect -lalpsutil -lrca -lxpmem -ludreg -lhugetlbfs -lAtpSigHandler -lAtpSigHCommData -limf -lifcore -lifport -lsvml -lirng -lipgo -ldecimal -lgcc_eh -lirc -lirc_s -ldl ----------------------------------------- WARNING: -log_summary is being deprecated; switch to -log_view ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- /global/homes/k/kallemov/svn_treb/EBAMRINS/execBCG/viscousDriver3d.Linux.64.CC.ftn.OPTHIGH.MPI.OPENMPCC.PETSC.ex on a arch-cori-knl-opt64-intel named nid02517 with 64 processors, by kallemov Thu Jan 25 11:48:52 2018 Using Petsc Release Version 3.8.3, unknown Max Max/Min Avg Total Time (sec): 5.298e+01 1.00001 5.298e+01 Objects: 2.420e+02 1.00000 2.420e+02 Flop: 2.353e+08 3.31607 1.568e+08 1.003e+10 Flop/sec: 4.442e+06 3.31608 2.959e+06 1.894e+08 MPI Messages: 2.924e+03 3.09635 1.801e+03 1.153e+05 MPI Message Lengths: 6.851e+06 3.60025 2.208e+03 2.545e+08 MPI Reductions: 6.466e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flop and VecAXPY() for complex vectors of length N --> 8N flop Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 4.9496e+01 93.4% 5.3887e+09 53.7% 6.052e+04 52.5% 1.064e+03 48.2% 1.590e+02 2.5% 1: Projection: 1.9771e+00 3.7% 2.5955e+09 25.9% 2.847e+04 24.7% 4.802e+02 21.7% 7.000e+01 1.1% 2: Viscous: 1.3462e+00 2.5% 2.0503e+09 20.4% 2.621e+04 22.7% 6.634e+02 30.0% 7.400e+01 1.1% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage VecMDot 43 1.0 1.4662e-01 7.4 2.89e+07 4.0 0.0e+00 0.0e+00 4.3e+01 0 11 0 0 1 0 21 0 0 27 7546 VecNorm 92 1.0 1.6556e-01 7.2 2.69e+06 4.0 0.0e+00 0.0e+00 9.2e+01 0 1 0 0 1 0 2 0 0 58 621 VecScale 45 1.0 4.0824e-03 1.3 1.34e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 12599 VecCopy 94 1.0 4.5540e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 136 1.0 4.9772e-03 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 45 1.0 8.8465e-03 1.6 2.69e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 11628 VecAYPX 45 1.0 2.3110e-03 2.7 1.34e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 22257 VecMAXPY 88 1.0 5.2319e-02 3.5 6.04e+07 4.0 0.0e+00 0.0e+00 0.0e+00 0 23 0 0 0 0 43 0 0 0 44174 VecScatterBegin 88 1.0 9.4392e-03 2.1 0.00e+00 0.0 5.8e+04 1.9e+03 0.0e+00 0 0 51 45 0 0 0 96 92 0 0 VecScatterEnd 88 1.0 2.5504e-0152.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 45 1.0 3.9876e-02 2.1 4.03e+06 4.0 0.0e+00 0.0e+00 4.5e+01 0 2 0 0 1 0 3 0 0 28 3870 KSPGMRESOrthog 43 1.0 1.6332e-01 3.7 5.78e+07 4.0 0.0e+00 0.0e+00 4.3e+01 0 22 0 0 1 0 41 0 0 27 13549 KSPSetUp 1 1.0 1.1142e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 2 1.0 2.6495e+00 1.0 1.35e+08 3.7 5.8e+04 1.9e+03 1.4e+02 5 54 51 45 2 5100 96 92 87 2034 MatMult 88 1.0 3.0218e-01 2.8 3.79e+07 3.2 5.8e+04 1.9e+03 0.0e+00 0 17 51 45 0 0 31 96 92 0 5502 MatConvert 1 1.0 4.4204e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 3 0 MatAssemblyBegin 2 1.0 9.6836e-0211.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 1 0 MatAssemblyEnd 2 1.0 9.8383e-02 1.0 0.00e+00 0.0 1.3e+03 9.7e+02 1.2e+01 0 0 1 1 0 0 0 2 1 8 0 MatGetRowIJ 2 1.0 5.8579e-04 4.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 2 1.0 2.8458e-02 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 1 0 PCSetUp 1 1.0 6.7105e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 1 0 0 0 0 1 0 0 0 3 0 PCApply 88 1.0 1.6766e+00 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 3 0 0 0 0 3 0 0 0 0 0 --- Event Stage 1: Projection VecMDot 21 1.0 7.2136e-02 7.5 1.38e+07 4.0 0.0e+00 0.0e+00 2.1e+01 0 5 0 0 0 2 20 0 0 30 7320 VecNorm 45 1.0 7.7528e-0214.0 1.31e+06 4.0 0.0e+00 0.0e+00 4.5e+01 0 1 0 0 1 3 2 0 0 64 649 VecScale 22 1.0 5.5885e-04 2.9 6.57e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 44996 VecCopy 46 1.0 2.2123e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 66 1.0 2.3298e-03 3.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 22 1.0 8.7047e-04 4.7 1.31e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 2 0 0 0 57776 VecAYPX 22 1.0 9.1076e-04 4.1 6.57e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 27610 VecMAXPY 43 1.0 2.4585e-02 3.5 2.88e+07 4.0 0.0e+00 0.0e+00 0.0e+00 0 11 0 0 0 1 43 0 0 0 44911 VecScatterBegin 43 1.0 4.2541e-03 2.5 0.00e+00 0.0 2.8e+04 1.9e+03 0.0e+00 0 0 25 22 0 0 0100100 0 0 VecScatterEnd 43 1.0 1.2524e-0152.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecNormalize 22 1.0 1.3191e-02 6.0 1.97e+06 4.0 0.0e+00 0.0e+00 2.2e+01 0 1 0 0 0 0 3 0 0 31 5719 KSPGMRESOrthog 21 1.0 7.9416e-02 3.8 2.76e+07 4.0 0.0e+00 0.0e+00 2.1e+01 0 11 0 0 0 3 41 0 0 30 13299 KSPSolve 1 1.0 9.3349e-01 1.0 6.51e+07 3.7 2.8e+04 1.9e+03 6.6e+01 2 26 25 22 1 47100100100 94 2780 MatMult 43 1.0 1.4621e-01 2.9 1.85e+07 3.2 2.8e+04 1.9e+03 0.0e+00 0 8 25 22 0 4 31100100 0 5556 MatView 1 1.0 4.0412e-0360.3 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 0 0 0 0 0 1 0 PCApply 43 1.0 8.1242e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 37 0 0 0 0 0 --- Event Stage 2: Viscous VecMDot 14 1.0 5.0756e-0239.0 2.39e+06 4.0 0.0e+00 0.0e+00 1.4e+01 0 1 0 0 0 2 4 0 0 19 1802 VecNorm 37 1.0 6.4282e-0210.1 1.01e+06 4.0 0.0e+00 0.0e+00 3.7e+01 0 0 0 0 1 2 2 0 0 50 605 VecScale 17 1.0 4.2200e-04 3.4 5.07e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 46045 VecCopy 40 1.0 2.3332e-03 3.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 52 1.0 1.7800e-03 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 17 1.0 8.4066e-04 5.5 1.01e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 2 0 0 0 46228 VecAYPX 17 1.0 8.6427e-04 4.2 5.07e+05 4.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 22483 VecMAXPY 31 1.0 4.6134e-03 4.1 5.61e+06 4.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 10 0 0 0 46579 VecScatterBegin 31 1.0 4.0817e-03 2.8 0.00e+00 0.0 2.4e+04 2.8e+03 0.0e+00 0 0 21 26 0 0 0 91 87 0 0 VecScatterEnd 31 1.0 1.0839e-0143.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 2 0 0 0 0 0 VecNormalize 17 1.0 8.0769e-03 4.7 1.52e+06 4.0 0.0e+00 0.0e+00 1.7e+01 0 1 0 0 0 0 3 0 0 23 7217 KSPGMRESOrthog 14 1.0 5.1428e-0221.2 4.78e+06 4.0 0.0e+00 0.0e+00 1.4e+01 0 2 0 0 0 2 9 0 0 19 3556 KSPSetUp 1 1.0 8.9097e-04 4.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 3 1.0 9.4874e-01 1.0 4.13e+07 2.5 2.4e+04 2.8e+03 5.5e+01 2 20 21 26 1 70100 91 87 74 2161 MatMult 31 1.0 1.3763e-01 2.2 3.46e+07 2.8 2.4e+04 2.8e+03 0.0e+00 0 16 21 26 0 6 79 91 87 0 11824 MatConvert 1 1.0 4.0958e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 3 0 0 0 5 0 MatAssemblyBegin 2 1.0 1.2679e-01803.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 4 0 0 0 3 0 MatAssemblyEnd 2 1.0 4.9350e-02 1.3 0.00e+00 0.0 1.5e+03 1.4e+03 1.2e+01 0 0 1 1 0 3 0 6 3 16 0 MatGetRowIJ 2 1.0 8.1062e-06 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 3 1.0 1.4201e-0293.4 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+00 0 0 0 0 0 0 0 0 0 4 0 PCSetUp 1 1.0 2.4881e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 18 0 0 0 5 0 PCApply 31 1.0 6.0053e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 40 0 0 0 0 0 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Vector 121 139 11057880 0. Vector Scatter 1 2 2416 0. Index Set 2 2 12624 0. IS L to G Mapping 1 0 0 0. Krylov Solver 1 2 37872 0. Matrix 4 8 6887312 0. Preconditioner 1 2 3296 0. Viewer 2 1 888 0. --- Event Stage 1: Projection Vector 44 44 3618560 0. --- Event Stage 2: Viscous Vector 55 37 2801512 0. Vector Scatter 1 0 0 0. Index Set 2 2 19808 0. IS L to G Mapping 1 0 0 0. Krylov Solver 1 0 0 0. Matrix 4 0 0 0. Preconditioner 1 0 0 0. ======================================================================================================================== Average time to get PetscTime(): 3.09944e-07 Average time for MPI_Barrier(): 6.00815e-06 Average time for zero size MPI_Send(): 5.06267e-06 #PETSc Option Table entries: -ksp_view -log_summary -log_view -options_left -proj_mac_gamg_est_ksp_gmres_restart 50 -proj_mac_gamg_est_ksp_max_it 20 -proj_mac_ksp_converged_reason -proj_mac_ksp_max_it 100 -proj_mac_ksp_monitor_max -proj_mac_ksp_norm_type unpreconditioned -proj_mac_ksp_rtol 1.e-12 -proj_mac_ksp_type gmres -proj_mac_ksp_view -proj_mac_mg_levels_ksp_max_it 1 -proj_mac_mg_levels_ksp_richardson_scale 0.9 -proj_mac_mg_levels_ksp_type richardson -proj_mac_mg_levels_pc_type sor -proj_mac_pc_gamg_agg_nsmooths 1 -proj_mac_pc_gamg_repartition false -proj_mac_pc_gamg_square_graph 20 -proj_mac_pc_gamg_sym_graph true -proj_mac_pc_gamg_threshold .02 -proj_mac_pc_gamg_type agg -proj_mac_pc_gamg_verbose 2 -proj_mac_pc_hypre_boomeramg_agg_nl 1 -proj_mac_pc_hypre_boomeramg_coarsen_type HMIS -proj_mac_pc_hypre_boomeramg_interp_type ext+i -proj_mac_pc_hypre_boomeramg_no_CF true -proj_mac_pc_hypre_type boomeramg -proj_mac_pc_ml_EnergyMinimization 2 -proj_mac_pc_ml_PrintLevel 1 -proj_mac_pc_ml_Threshold 0.01 -proj_mac_pc_type hypre -visc_ksp_converged_reason -visc_ksp_max_it 100 -visc_ksp_monitor_max -visc_ksp_norm_type unpreconditioned -visc_ksp_rtol 1.e-12 -visc_ksp_view -visc_mg_levels_ksp_max_it 1 -visc_mg_levels_ksp_type richardson -visc_mg_levels_pc_type sor -visc_pc_gamg_agg_nsmooths 1 -visc_pc_gamg_repartition false -visc_pc_gamg_square_graph true -visc_pc_gamg_sym_graph true -visc_pc_gamg_threshold .02 -visc_pc_gamg_type agg -visc_pc_hypre_boomeramg_agg_nl 1 -visc_pc_hypre_boomeramg_coarsen_type HMIS -visc_pc_hypre_boomeramg_interp_type ext+i -visc_pc_hypre_boomeramg_no_CF true -visc_pc_hypre_type boomeramg -visc_pc_type hypre #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8 Configure options: --with-cc=cc --with-cxx=CC --with-fc=ftn COPTFLAGS=" -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd" --download-metis=1 --download-hypre=1 --download-parmetis=1 --download-fblaslapack --with-debugging=0 --with-mpiexec=srun --with-batch=0 --with-memalign=64 --with-64-bit-indices=1 --known-mpi-shared-libraries=1 PETSC_ARCH=arch-cori-knl-opt64-intel --with-openmp=0 --download-p4est=0 --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel ----------------------------------------- Libraries compiled on Tue Jan 23 07:15:01 2018 on nid04222 Machine characteristics: Linux-4.4.49-92.11.1_3.0-cray_ari_c-x86_64-with-SuSE-12-x86_64 Using PETSc directory: /global/u2/m/madams/petsc_install/petsc Using PETSc arch: arch-cori-knl-opt64-intel ----------------------------------------- Using C compiler: cc -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd ${COPTFLAGS} ${CFLAGS} Using Fortran compiler: ftn -g -O3 -fp-model fast -xMIC-AVX512 -qopt-report=5 -hcpu=mic-knl -qopenmp-simd ${FOPTFLAGS} ${FFLAGS} ----------------------------------------- Using include paths: -I/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/include -I/global/u2/m/madams/petsc_install/petsc/include -I/global/u2/m/madams/petsc_install/petsc/include -I/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/include -I/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/include ----------------------------------------- Using C linker: cc Using Fortran linker: ftn Using libraries: -Wl,-rpath,/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/lib -L/global/u2/m/madams/petsc_install/petsc/arch-cori-knl-opt64-intel/lib -lpetsc -Wl,-rpath,/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/lib -L/global/homes/m/madams/petsc_install/petsc-cori-knl-opt64-intel/lib -L/opt/cray/pe/libsci/17.09.1/INTEL/16.0/x86_64/lib -L/opt/cray/pe/hdf5-parallel/1.10.0.3/INTEL/16.0/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/pe/mpt/7.6.2/gni/mpich-intel/16.0/lib -L/opt/cray/udreg/2.3.2-6.0.4.0_12.2__g2f9c3ee.ari/lib64 -L/usr/common/software/darshan/3.1.4/lib -L/opt/cray/rca/2.2.15-6.0.4.1_13.1__g46acb0f.ari/lib64 -L/opt/cray/alps/6.4.1-6.0.4.0_7.2__g86d0f3d.ari/lib64 -L/opt/cray/xpmem/2.2.2-6.0.4.0_3.1__g43b0535.ari/lib64 -L/opt/cray/pe/pmi/5.0.12/lib64 -L/opt/cray/ugni/6.0.14-6.0.4.0_14.1__ge7db4a2.ari/lib64 -L/opt/cray/pe/atp/2.1.1/libApp -L/opt/cray/wlm_detect/1.2.1-6.0.4.0_22.1__gd26a3dc.ari/lib64 -L/usr/common/software/h5part/1.6.6/lib -L/opt/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64 -L/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64 -L/opt/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin -L/usr/lib64/gcc/x86_64-suse-linux/4.8 -L/usr/x86_64-suse-linux/lib -lHYPRE -lflapack -lfblas -lparmetis -lmetis -lhdf5hl_fortran_parallel -lhdf5_fortran_parallel -lmpichcxx_intel -lstdc++ -lmpichcxx_intel -lstdc++ -ldl -lfmpich -lmpichcxx -ldarshan -ldarshan-stubs -lz -lpthread -lhdf5_hl_parallel -lm -lhdf5_parallel -lmpichf90_intel -lrt -lugni -lpmi -lsci_intel_mpi -lsci_intel -lmpich_intel -lalpslli -lwlm_detect -lalpsutil -lrca -lxpmem -ludreg -lhugetlbfs -lAtpSigHandler -lAtpSigHCommData -limf -lifcore -lifport -lsvml -lirng -lipgo -ldecimal -lgcc_eh -lirc -lirc_s -ldl ----------------------------------------- #PETSc Option Table entries: -ksp_view -log_summary -log_view -options_left -proj_mac_gamg_est_ksp_gmres_restart 50 -proj_mac_gamg_est_ksp_max_it 20 -proj_mac_ksp_converged_reason -proj_mac_ksp_max_it 100 -proj_mac_ksp_monitor_max -proj_mac_ksp_norm_type unpreconditioned -proj_mac_ksp_rtol 1.e-12 -proj_mac_ksp_type gmres -proj_mac_ksp_view -proj_mac_mg_levels_ksp_max_it 1 -proj_mac_mg_levels_ksp_richardson_scale 0.9 -proj_mac_mg_levels_ksp_type richardson -proj_mac_mg_levels_pc_type sor -proj_mac_pc_gamg_agg_nsmooths 1 -proj_mac_pc_gamg_repartition false -proj_mac_pc_gamg_square_graph 20 -proj_mac_pc_gamg_sym_graph true -proj_mac_pc_gamg_threshold .02 -proj_mac_pc_gamg_type agg -proj_mac_pc_gamg_verbose 2 -proj_mac_pc_hypre_boomeramg_agg_nl 1 -proj_mac_pc_hypre_boomeramg_coarsen_type HMIS -proj_mac_pc_hypre_boomeramg_interp_type ext+i -proj_mac_pc_hypre_boomeramg_no_CF true -proj_mac_pc_hypre_type boomeramg -proj_mac_pc_ml_EnergyMinimization 2 -proj_mac_pc_ml_PrintLevel 1 -proj_mac_pc_ml_Threshold 0.01 -proj_mac_pc_type hypre -visc_ksp_converged_reason -visc_ksp_max_it 100 -visc_ksp_monitor_max -visc_ksp_norm_type unpreconditioned -visc_ksp_rtol 1.e-12 -visc_ksp_view -visc_mg_levels_ksp_max_it 1 -visc_mg_levels_ksp_type richardson -visc_mg_levels_pc_type sor -visc_pc_gamg_agg_nsmooths 1 -visc_pc_gamg_repartition false -visc_pc_gamg_square_graph true -visc_pc_gamg_sym_graph true -visc_pc_gamg_threshold .02 -visc_pc_gamg_type agg -visc_pc_hypre_boomeramg_agg_nl 1 -visc_pc_hypre_boomeramg_coarsen_type HMIS -visc_pc_hypre_boomeramg_interp_type ext+i -visc_pc_hypre_boomeramg_no_CF true -visc_pc_hypre_type boomeramg -visc_pc_type hypre #End of PETSc Option Table entries There are 26 unused database options. They are: Option left: name:-ksp_view (no value) Option left: name:-proj_mac_gamg_est_ksp_gmres_restart value: 50 Option left: name:-proj_mac_gamg_est_ksp_max_it value: 20 Option left: name:-proj_mac_mg_levels_ksp_max_it value: 1 Option left: name:-proj_mac_mg_levels_ksp_richardson_scale value: 0.9 Option left: name:-proj_mac_mg_levels_ksp_type value: richardson Option left: name:-proj_mac_mg_levels_pc_type value: sor Option left: name:-proj_mac_pc_gamg_agg_nsmooths value: 1 Option left: name:-proj_mac_pc_gamg_repartition value: false Option left: name:-proj_mac_pc_gamg_square_graph value: 20 Option left: name:-proj_mac_pc_gamg_sym_graph value: true Option left: name:-proj_mac_pc_gamg_threshold value: .02 Option left: name:-proj_mac_pc_gamg_type value: agg Option left: name:-proj_mac_pc_gamg_verbose value: 2 Option left: name:-proj_mac_pc_ml_EnergyMinimization value: 2 Option left: name:-proj_mac_pc_ml_PrintLevel value: 1 Option left: name:-proj_mac_pc_ml_Threshold value: 0.01 Option left: name:-visc_mg_levels_ksp_max_it value: 1 Option left: name:-visc_mg_levels_ksp_type value: richardson Option left: name:-visc_mg_levels_pc_type value: sor Option left: name:-visc_pc_gamg_agg_nsmooths value: 1 Option left: name:-visc_pc_gamg_repartition value: false Option left: name:-visc_pc_gamg_square_graph value: true Option left: name:-visc_pc_gamg_sym_graph value: true Option left: name:-visc_pc_gamg_threshold value: .02 Option left: name:-visc_pc_gamg_type value: agg