KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 KSP Object: 32 MPI processes type: cg maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using UNPRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: gamg type is MULTIPLICATIVE, levels=6 cycles=v Cycles per PCApply=1 Using externally compute Galerkin coarse grid matrices GAMG specific options Threshold for dropping small values in graph on each level = 0. 0. 0. 0. 0. 0. Threshold scaling factor for each level not specified = 0. AGG specific options Symmetric graph false Number of levels to square graph 1 Number smoothing steps 1 Complexity: grid = 1.65934 Coarse grid solver -- level ------------------------------- KSP Object: (mg_coarse_) 32 MPI processes type: preonly maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_coarse_) 32 MPI processes type: bjacobi number of blocks = 32 Local solver information for first block is in the following KSP and PC objects on rank 0: Use -mg_coarse_ksp_view ::ascii_info_detail to display information for all blocks KSP Object: (mg_coarse_sub_) 1 MPI processes type: preonly maximum iterations=1, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using DEFAULT norm type for convergence test PC Object: (mg_coarse_sub_) 1 MPI processes type: lu PC has not been set up so information may be incomplete out-of-place factorization tolerance for zero pivot 2.22045e-14 using diagonal shift on blocks to prevent zero pivot [INBLOCKS] matrix ordering: nd matrix solver type: (null) matrix not yet factored; no additional information available linear system matrix = precond matrix: Mat Object: (mg_coarse_sub_) 1 MPI processes type: seqaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node routines linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=7, cols=7 total: nonzeros=49, allocated nonzeros=49 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Down solver (pre-smoother) on level 1 ------------------------------- KSP Object: (mg_levels_1_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167469, max = 1.84215 eigenvalues estimate via cg min 0.536382, max 1.67469 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_1_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_1_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=357, cols=357 total: nonzeros=48291, allocated nonzeros=48291 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 2 ------------------------------- KSP Object: (mg_levels_2_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.166577, max = 1.83235 eigenvalues estimate via cg min 0.0779906, max 1.66577 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_2_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_2_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=17942, cols=17942 total: nonzeros=4019896, allocated nonzeros=4019896 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 3 ------------------------------- KSP Object: (mg_levels_3_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.167335, max = 1.84069 eigenvalues estimate via cg min 0.0452015, max 1.67335 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_3_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_3_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=644406, cols=644406 total: nonzeros=89107448, allocated nonzeros=89107448 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 4 ------------------------------- KSP Object: (mg_levels_4_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.151784, max = 1.66962 eigenvalues estimate via cg min 0.037107, max 1.51784 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_4_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_4_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=6548634, cols=6548634 total: nonzeros=215778320, allocated nonzeros=215778320 total number of mallocs used during MatSetValues calls=0 not using I-node (on process 0) routines Up solver (post-smoother) same as down solver (pre-smoother) Down solver (pre-smoother) on level 5 ------------------------------- KSP Object: (mg_levels_5_) 32 MPI processes type: chebyshev eigenvalue estimates used: min = 0.195761, max = 2.15337 eigenvalues estimate via cg min 0.0423986, max 1.95761 eigenvalues estimated using cg with translations [0. 0.1; 0. 1.1] KSP Object: (mg_levels_5_esteig_) 32 MPI processes type: cg maximum iterations=10, initial guess is zero tolerances: relative=1e-12, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test estimating eigenvalues using noisy right hand side maximum iterations=2, nonzero initial guess tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using NONE norm type for convergence test PC Object: (mg_levels_5_) 32 MPI processes type: jacobi type DIAGONAL linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 Up solver (post-smoother) same as down solver (pre-smoother) linear system matrix = precond matrix: Mat Object: 32 MPI processes type: mpiaijcusparse rows=67108864, cols=67108864 total: nonzeros=468582400, allocated nonzeros=468582400 total number of mallocs used during MatSetValues calls=0 **************************************** *********************************************************************************************************************** *** WIDEN YOUR WINDOW TO 160 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** **************************************************************************************************************************************************************** ------------------------------------------------------------------ PETSc Performance Summary: ------------------------------------------------------------------- ########################################################## # # # WARNING!!! # # # # This code was compiled with GPU support and you've # # created PETSc/GPU objects, but you intentionally used # # -use_gpu_aware_mpi 0, such that PETSc had to copy data # # from GPU to CPU for communication. To get meaningfull # # timing results, please use GPU-aware MPI instead. # ########################################################## /global/homes/s/sajid/packages/aclatfd/3D/poisson3d on a named nid002045 with 32 processors, by sajid Thu Feb 10 14:02:54 2022 Using Petsc Development GIT revision: f351d5494b5462f62c419e00645ac2e477b88cae GIT Date: 2022-02-08 15:08:19 +0000 Max Max/Min Avg Total Time (sec): 2.241e+01 1.001 2.240e+01 Objects: 6.780e+02 1.000 6.780e+02 Flop: 3.324e+09 1.023 3.273e+09 1.047e+11 Flop/sec: 1.484e+08 1.022 1.461e+08 4.675e+09 MPI Messages: 2.748e+03 1.844 2.217e+03 7.096e+04 MPI Message Lengths: 1.002e+08 1.344 4.215e+04 2.991e+09 MPI Reductions: 8.650e+02 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flop and VecAXPY() for complex vectors of length N --> 8N flop Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 2.2399e+01 100.0% 1.0406e+11 99.4% 7.065e+04 99.6% 4.234e+04 100.0% 7.720e+02 89.2% 1: linear-solve: 5.2657e-03 0.0% 6.7109e+08 0.6% 3.100e+02 0.4% 4.000e+00 0.0% 7.500e+01 8.7% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) GPU Mflop/s: 10e-6 * (sum of flop on GPU over all processors)/(max GPU time over all processors) CpuToGpu Count: total number of CPU to GPU copies per processor CpuToGpu Size (Mbytes): 10e-6 * (total size of CPU to GPU copies per processor) GpuToCpu Count: total number of GPU to CPU copies per processor GpuToCpu Size (Mbytes): 10e-6 * (total size of GPU to CPU copies per processor) GPU %F: percent flops on GPU in this event ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total GPU - CpuToGpu - - GpuToCpu - GPU Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s Mflop/s Count Size Count Size %F --------------------------------------------------------------------------------------------------------------------------------------------------------------- --- Event Stage 0: Main Stage BuildTwoSided 75 1.0 5.1494e-01 1.5 0.00e+00 0.0 9.2e+03 4.0e+00 7.5e+01 2 0 13 0 9 2 0 13 0 10 0 0 0 0.00e+00 0 0.00e+00 0 BuildTwoSidedF 32 1.0 3.5895e-01 1.9 0.00e+00 0.0 1.3e+03 1.7e+05 3.2e+01 1 0 2 8 4 1 0 2 8 4 0 0 0 0.00e+00 0 0.00e+00 0 MatMult 100 1.0 5.2449e-02 1.2 9.43e+08 1.0 2.4e+04 2.1e+04 5.0e+00 0 28 34 17 1 0 28 34 17 1 564650 2378367 177 2.16e+02 170 3.16e+01 100 MatConvert 15 1.0 1.1410e-01 1.2 0.00e+00 0.0 2.2e+03 5.7e+03 5.0e+00 0 0 3 0 1 0 0 3 0 1 0 0 0 0.00e+00 6 7.72e+01 0 MatScale 15 1.0 1.6903e-01 1.2 6.85e+07 1.0 1.1e+03 2.3e+04 0.0e+00 1 2 2 1 0 1 2 2 1 0 12767 1446116 28 9.57e+01 27 9.63e+01 14 MatAssemblyBegin 43 1.1 3.5768e-01 1.8 0.00e+00 0.0 1.3e+03 1.7e+05 1.7e+01 1 0 2 8 2 1 0 2 8 2 0 0 0 0.00e+00 0 0.00e+00 0 MatAssemblyEnd 43 1.1 1.1025e+00 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 6.4e+01 4 0 0 0 7 4 0 0 0 8 0 0 0 0.00e+00 0 0.00e+00 0 MatCreateSubMat 4 1.0 2.7781e-02 1.0 0.00e+00 0.0 1.4e+02 3.1e+03 5.6e+01 0 0 0 0 6 0 0 0 0 7 0 0 0 0.00e+00 4 1.21e-02 0 MatCoarsen 5 1.0 2.8133e-01 1.1 0.00e+00 0.0 1.1e+04 1.2e+04 3.1e+01 1 0 15 4 4 1 0 15 4 4 0 0 0 0.00e+00 0 0.00e+00 0 MatView 8 1.1 3.6067e-01228.1 0.00e+00 0.0 0.0e+00 0.0e+00 7.0e+00 1 0 0 0 1 1 0 0 0 1 0 0 0 0.00e+00 0 0.00e+00 0 MatAXPY 5 1.0 5.4323e-01 1.0 2.33e+06 1.0 0.0e+00 0.0e+00 5.0e+00 2 0 0 0 1 2 0 0 0 1 137 0 0 0.00e+00 8 7.54e+01 0 MatMatMultSym 5 1.0 5.8937e-01 1.0 4.95e+07 1.0 3.3e+03 1.5e+04 3.0e+01 3 1 5 2 3 3 1 5 2 4 2638 18396 44 3.24e+02 28 1.15e+02 100 MatMatMultNum 5 1.0 1.3230e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 5 0.00e+00 0 0.00e+00 0 MatPtAPSymbolic 5 1.0 1.3952e+00 1.0 6.33e+08 1.0 1.9e+04 7.6e+04 4.0e+01 6 19 27 49 5 6 19 27 49 5 14174 66070 47 6.01e+02 38 2.55e+02 100 MatPtAPNumeric 5 1.0 2.8379e-02 1.1 6.22e+08 1.0 1.2e+03 2.0e+05 0.0e+00 0 18 2 8 0 0 19 2 8 0 681397 2753429 37 6.66e+01 8 1.50e+01 100 MatTrnMatMultSym 1 1.0 5.0540e+00 1.0 0.00e+00 0.0 6.0e+02 4.8e+05 1.2e+01 23 0 1 10 1 23 0 1 10 2 0 0 0 0.00e+00 0 0.00e+00 0 MatGetLocalMat 6 1.0 2.5439e-01 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 0 11 8.46e+01 5 4.70e+01 0 MatGetBrAoCol 10 1.0 9.8112e-02 1.2 0.00e+00 0.0 6.6e+03 4.7e+04 0.0e+00 0 0 9 10 0 0 0 9 10 0 0 0 0 0.00e+00 0 0.00e+00 0 MatCUSPARSCopyTo 74 1.1 1.0839e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 73 8.28e+02 0 0.00e+00 0 MatCUSPARSCopyFr 30 1.2 2.1629e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 27 2.28e+02 0 MatCUSPARSGenT 10 1.1 2.3313e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 MatSetPreallCOO 10 1.0 3.3399e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+01 1 0 0 0 3 1 0 0 0 4 0 0 47 5.69e+02 27 1.03e+02 0 MatSetValuesCOO 10 1.0 2.2480e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 DMCreateMat 1 1.0 4.3235e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 7.0e+00 19 0 0 0 1 19 0 0 0 1 0 0 0 0.00e+00 0 0.00e+00 0 SFSetGraph 60 1.0 1.1555e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 SFSetUp 43 1.0 4.5650e-01 1.2 0.00e+00 0.0 1.7e+04 2.5e+04 4.3e+01 2 0 24 15 5 2 0 24 15 6 0 0 0 0.00e+00 0 0.00e+00 0 SFBcastBegin 41 1.0 1.2121e-02 1.5 0.00e+00 0.0 9.5e+03 2.4e+04 0.0e+00 0 0 13 8 0 0 0 14 8 0 0 0 0 0.00e+00 0 0.00e+00 0 SFBcastEnd 41 1.0 2.7751e-02 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 SFReduceBegin 25 1.0 2.9432e-02 1.5 0.00e+00 0.0 6.1e+03 1.2e+05 0.0e+00 0 0 9 24 0 0 0 9 24 0 0 0 4 3.76e+00 8 1.50e+01 0 SFReduceEnd 25 1.0 9.2771e-02 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 13 1.88e+01 0 0.00e+00 0 SFFetchOpBegin 5 1.0 7.5031e-03 1.5 0.00e+00 0.0 1.2e+03 9.8e+04 0.0e+00 0 0 2 4 0 0 0 2 4 0 0 0 0 0.00e+00 0 0.00e+00 0 SFFetchOpEnd 5 1.0 3.7276e-02 2.5 0.00e+00 0.0 1.2e+03 9.8e+04 0.0e+00 0 0 2 4 0 0 0 2 4 0 0 0 0 0.00e+00 0 0.00e+00 0 SFPack 193 1.0 4.5305e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 13 4.06e+00 0 0.00e+00 0 SFUnpack 198 1.0 3.9856e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 4 3.76e+00 0 0.00e+00 0 VecView 2 1.0 3.8757e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0 0 0 0.00e+00 2 3.36e+01 0 VecMDot 50 1.0 1.3216e-02 1.6 2.56e+08 1.0 0.0e+00 0.0e+00 5.0e+01 0 8 0 0 6 0 8 0 0 6 618636 2273358 0 0.00e+00 0 0.00e+00 100 VecTDot 105 1.0 8.3254e-03 1.4 9.79e+07 1.0 0.0e+00 0.0e+00 1.0e+02 0 3 0 0 12 0 3 0 0 14 374929 931984 0 0.00e+00 0 0.00e+00 100 VecNorm 111 1.0 2.2458e-02 1.7 1.07e+08 1.0 0.0e+00 0.0e+00 1.1e+02 0 3 0 0 13 0 3 0 0 14 151584 623152 0 0.00e+00 0 0.00e+00 100 VecScale 59 1.0 2.7890e-03 1.5 3.40e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 389370 1451008 1 1.68e+01 0 0.00e+00 100 VecCopy 16 1.0 1.6585e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 VecSet 195 1.0 4.4392e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 VecAXPY 105 1.0 2.4867e-03 1.1 9.79e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 1255281 2149663 0 0.00e+00 0 0.00e+00 100 VecAYPX 45 1.0 1.2838e-03 1.2 4.20e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 1042012 1549198 0 0.00e+00 0 0.00e+00 100 VecMAXPY 55 1.0 4.0168e-03 1.1 3.03e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 9 0 0 0 0 9 0 0 0 2405312 2772811 0 0.00e+00 0 0.00e+00 100 VecAssemblyBegin 17 1.0 2.6987e-02 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 1.5e+01 0 0 0 0 2 0 0 0 0 2 0 0 0 0.00e+00 0 0.00e+00 0 VecAssemblyEnd 17 1.0 9.4975e-05 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 VecPointwiseMult 110 1.0 5.5196e-03 1.1 5.13e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 296227 779222 15 5.57e+01 0 0.00e+00 100 VecScatterBegin 122 1.0 6.1036e-02 1.9 0.00e+00 0.0 3.2e+04 2.5e+04 1.3e+01 0 0 45 27 2 0 0 45 27 2 0 0 14 1.89e+01 178 3.32e+01 0 VecScatterEnd 122 1.0 3.9209e-02 4.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 178 3.32e+01 0 0.00e+00 0 VecSetRandom 5 1.0 1.3805e-0310.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 VecNormalize 55 1.0 1.1503e-02 1.9 7.69e+07 1.0 0.0e+00 0.0e+00 5.5e+01 0 2 0 0 6 0 2 0 0 7 213217 725797 0 0.00e+00 0 0.00e+00 100 VecCUDACopyTo 41 1.0 7.0145e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 41 1.65e+02 0 0.00e+00 0 VecCUDACopyFrom 27 1.0 4.9996e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 27 1.09e+02 0 KSPSetUp 12 1.0 2.1270e-01 1.0 7.81e+08 1.0 1.1e+04 2.3e+04 1.8e+02 1 24 16 8 21 1 24 16 8 24 116087 1575121 100 7.15e+01 90 3.44e+01 100 KSPSolve 1 1.0 2.6596e-04 1.3 4.19e+06 1.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 504648 1085763 0 0.00e+00 0 0.00e+00 100 KSPGMRESOrthog 50 1.0 1.6442e-02 1.4 5.13e+08 1.0 0.0e+00 0.0e+00 5.0e+01 0 16 0 0 6 0 16 0 0 6 994509 2488827 0 0.00e+00 0 0.00e+00 100 PCGAMGGraph_AGG 5 1.0 3.5700e+00 1.0 4.95e+07 1.0 3.3e+03 1.1e+04 4.5e+01 16 1 5 1 5 16 1 5 1 6 436 0 23 3.89e+01 24 9.82e+01 0 PCGAMGCoarse_AGG 5 1.0 6.0047e+00 1.0 0.00e+00 0.0 1.3e+04 4.4e+04 5.4e+01 27 0 19 19 6 27 0 19 19 7 0 0 0 0.00e+00 0 0.00e+00 0 PCGAMGProl_AGG 5 1.0 8.2284e-01 1.0 0.00e+00 0.0 5.6e+03 1.7e+04 7.9e+01 4 0 8 3 9 4 0 8 3 10 0 0 0 0.00e+00 0 0.00e+00 0 PCGAMGPOpt_AGG 5 1.0 3.9355e+00 1.0 1.21e+09 1.0 1.7e+04 1.9e+04 1.8e+02 18 37 23 11 21 18 37 23 11 24 9731 367366 166 6.55e+02 140 3.18e+02 99 GAMG: createProl 5 1.0 1.4313e+01 1.0 1.26e+09 1.0 3.9e+04 2.7e+04 3.6e+02 64 38 55 35 42 64 38 55 35 47 2784 366277 188 6.94e+02 165 4.16e+02 95 Graph 10 1.0 3.5472e+00 1.0 4.95e+07 1.0 3.3e+03 1.1e+04 4.5e+01 16 1 5 1 5 16 1 5 1 6 438 0 23 3.89e+01 24 9.82e+01 0 MIS/Agg 5 1.0 2.8142e-01 1.1 0.00e+00 0.0 1.1e+04 1.2e+04 3.1e+01 1 0 15 4 4 1 0 15 4 4 0 0 0 0.00e+00 0 0.00e+00 0 SA: col data 5 1.0 1.8457e-01 1.0 0.00e+00 0.0 4.5e+03 1.9e+04 3.4e+01 1 0 6 3 4 1 0 6 3 4 0 0 0 0.00e+00 0 0.00e+00 0 SA: frmProl0 5 1.0 5.7982e-01 1.0 0.00e+00 0.0 1.1e+03 9.3e+03 2.5e+01 3 0 2 0 3 3 0 2 0 3 0 0 0 0.00e+00 0 0.00e+00 0 SA: smooth 5 1.0 1.2374e+00 1.0 7.08e+07 1.0 3.3e+03 1.5e+04 4.5e+01 6 2 5 2 5 6 2 5 2 6 1804 21689 64 4.18e+02 50 2.84e+02 83 GAMG: partLevel 5 1.0 1.4568e+00 1.0 1.25e+09 1.0 2.1e+04 8.2e+04 1.5e+02 6 37 29 57 17 6 38 29 57 19 26849 127851 84 6.67e+02 51 2.70e+02 100 repartition 2 1.0 3.3668e-02 1.0 0.00e+00 0.0 3.5e+02 1.3e+03 1.1e+02 0 0 0 0 12 0 0 0 0 14 0 0 0 0.00e+00 4 1.21e-02 0 Invert-Sort 2 1.0 4.7750e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 1 0 0 0 0 2 0 0 0 0.00e+00 0 0.00e+00 0 Move A 2 1.0 1.6896e-02 1.0 0.00e+00 0.0 1.4e+02 3.1e+03 3.0e+01 0 0 0 0 3 0 0 0 0 4 0 0 0 0.00e+00 2 1.21e-02 0 Move P 2 1.0 1.4963e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 3.2e+01 0 0 0 0 4 0 0 0 0 4 0 0 0 0.00e+00 2 0.00e+00 0 PCGAMG Squ l00 1 1.0 5.0540e+00 1.0 0.00e+00 0.0 6.0e+02 4.8e+05 1.2e+01 23 0 1 10 1 23 0 1 10 2 0 0 0 0.00e+00 0 0.00e+00 0 PCGAMG Gal l00 1 1.0 6.4353e-01 1.0 4.49e+08 1.0 4.2e+03 1.4e+05 8.0e+00 3 14 6 20 1 3 14 6 20 1 22236 120116 19 4.50e+02 10 1.82e+02 100 PCGAMG Opt l00 1 1.0 3.9678e-01 1.0 2.93e+07 1.0 3.6e+02 8.7e+04 6.0e+00 2 1 1 1 1 2 1 1 1 1 2362 22684 9 2.57e+02 6 9.28e+01 100 PCGAMG Gal l01 1 1.0 5.2267e-01 1.0 5.62e+08 1.1 5.0e+03 1.7e+05 8.0e+00 2 17 7 29 1 2 17 7 29 1 33175 149461 19 1.94e+02 10 7.65e+01 100 PCGAMG Opt l01 1 1.0 1.1455e-01 1.0 1.38e+07 1.0 9.2e+02 1.5e+04 6.0e+00 1 0 1 0 1 1 0 1 0 1 3767 18456 9 5.86e+01 6 1.89e+01 100 PCGAMG Gal l02 1 1.0 1.6174e-01 1.0 2.40e+08 1.1 5.3e+03 4.3e+04 8.0e+00 1 7 8 8 1 1 7 8 8 1 44401 117471 18 2.27e+01 10 1.10e+01 100 PCGAMG Opt l02 1 1.0 4.7131e-02 1.0 6.14e+06 1.1 9.6e+02 5.1e+03 6.0e+00 0 0 1 0 1 0 0 1 0 1 3781 9683 9 8.28e+00 6 2.75e+00 100 PCGAMG Gal l03 1 1.0 5.9585e-02 1.0 1.17e+07 1.6 5.5e+03 4.2e+03 8.0e+00 0 0 8 1 1 0 0 8 1 1 4731 18636 17 9.97e-01 10 4.63e-01 100 PCGAMG Opt l03 1 1.0 1.8623e-02 1.0 2.83e+05 1.2 9.6e+02 7.7e+02 6.0e+00 0 0 1 0 1 0 0 1 0 1 432 1416 9 2.83e-01 6 1.02e-01 100 PCGAMG Gal l04 1 1.0 3.6455e-02 1.0 8.91e+04 0.0 5.0e+02 1.7e+02 8.0e+00 0 0 1 0 1 0 0 1 0 1 15 128 11 3.60e-03 7 1.14e-03 100 PCGAMG Opt l04 1 1.0 1.4690e-02 1.0 1.52e+04 0.0 1.1e+02 1.8e+02 6.0e+00 0 0 0 0 1 0 0 0 0 1 7 43 13 2.65e-03 4 8.54e-04 100 PCSetUp 1 1.0 1.5959e+01 1.0 3.29e+09 1.0 7.1e+04 4.2e+04 7.1e+02 71 99 99100 82 71100100100 92 6495 240981 372 1.43e+03 306 7.21e+02 98 --- Event Stage 1: linear-solve MatView 40 1.1 3.1361e-03 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 3.5e+01 0 0 0 0 4 48 0 0 0 47 0 0 0 0.00e+00 0 0.00e+00 0 VecNorm 5 1.0 5.1959e-04 1.2 2.10e+07 1.0 0.0e+00 0.0e+00 5.0e+00 0 1 0 0 1 9100 0 0 7 1291576 1831094 0 0.00e+00 0 0.00e+00 100 VecCopy 5 1.0 2.2910e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 4 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 VecSet 5 1.0 1.5485e-04 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 3 0 0 0 0 0 0 0 0.00e+00 0 0.00e+00 0 KSPSolve 5 1.0 9.6481e-04 1.0 2.10e+07 1.0 0.0e+00 0.0e+00 1.0e+01 0 1 0 0 1 18100 0 0 13 695565 1055649 0 0.00e+00 0 0.00e+00 100 --------------------------------------------------------------------------------------------------------------------------------------------------------------- Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Distributed Mesh 7 7 36360 0. Matrix 158 158 2260963528 0. Matrix Coarsen 5 5 3120 0. Index Set 92 92 36516616 0. IS L to G Mapping 22 22 17916208 0. Star Forest Graph 69 69 81072 0. Discrete System 7 7 6720 0. Weak Form 7 7 4312 0. Vector 254 254 789426480 0. Krylov Solver 18 18 176848 0. DMKSP interface 1 1 656 0. Preconditioner 18 18 17872 0. Viewer 5 4 3312 0. PetscRandom 10 10 6660 0. --- Event Stage 1: linear-solve Viewer 5 5 4200 0. ======================================================================================================================== Average time to get PetscTime(): 3.3e-08 Average time for MPI_Barrier(): 1.7093e-05 Average time for zero size MPI_Send(): 6.81222e-06 #PETSc Option Table entries: -dm_mat_type aijcusparse -dm_vec_type cuda -ksp_monitor -ksp_norm_type unpreconditioned -ksp_type cg -ksp_view -log_view -mg_levels_esteig_ksp_type cg -mg_levels_ksp_type chebyshev -mg_levels_pc_type jacobi -pc_gamg_agg_nsmooths 1 -pc_gamg_square_graph 1 -pc_gamg_threshold 0.0 -pc_gamg_threshold_scale 0.0 -pc_gamg_type agg -pc_type gamg -use_gpu_aware_mpi 0 #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --prefix=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/petsc-main-mnj56kbexro3fipf6kheyttljzwss7fo --with-ssl=0 --download-c2html=0 --download-sowing=0 --download-hwloc=0 CFLAGS= FFLAGS= CXXFLAGS= --with-cc=/opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpicc --with-cxx=/opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpicxx --with-fc=/opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpif90 --with-precision=double --with-scalar-type=real --with-shared-libraries=1 --with-debugging=0 --with-openmp=0 --with-64-bit-indices=0 COPTFLAGS= FOPTFLAGS= CXXOPTFLAGS= --with-blaslapack-lib=/opt/cray/pe/libsci/21.08.1.2/GNU/9.1/x86_64/lib/libsci_gnu.so --with-x=0 --with-clanguage=C --with-cuda=1 --with-cuda-dir=/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4 --with-hip=0 --with-metis=1 --with-metis-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/metis-5.1.0-lxe5bhakcmkcf7zuqcagulm7tihcav7q/include --with-metis-lib=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/metis-5.1.0-lxe5bhakcmkcf7zuqcagulm7tihcav7q/lib/libmetis.so --with-hypre=1 --with-hypre-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hypre-develop-3gtrobj6ky64qlq4jvi2qzou5mvisy4w/include --with-hypre-lib=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hypre-develop-3gtrobj6ky64qlq4jvi2qzou5mvisy4w/lib/libHYPRE.so --with-parmetis=1 --with-parmetis-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/parmetis-4.0.3-7xhbi6h22ni4fe35vxurnwmr6izbeb7b/include --with-parmetis-lib=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/parmetis-4.0.3-7xhbi6h22ni4fe35vxurnwmr6izbeb7b/lib/libparmetis.so --with-kokkos=1 --with-kokkos-dir=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-3.5.00-65sqphcwz6lwtqectq6yswa6kt3654mb --with-kokkos-kernels=1 --with-kokkos-kernels-dir=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-kernels-3.5.00-zwq3aedpbg7ywpmqiqxmn5nx4w6hdrx6 --with-superlu_dist=1 --with-superlu_dist-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/superlu-dist-develop-l5kc2sttvfqcjlejhgnvygfxwulrujga/include --with-superlu_dist-lib=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/superlu-dist-develop-l5kc2sttvfqcjlejhgnvygfxwulrujga/lib/libsuperlu_dist.so --with-ptscotch=0 --with-suitesparse=0 --with-hdf5=1 --with-hdf5-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/include --with-hdf5-lib="/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/lib/libhdf5_hl.so /global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/lib/libhdf5.so" --with-zlib=1 --with-zlib-include=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.11-ekeupmdcqoimgroigtctln7tqkyh6pdm/include --with-zlib-lib=/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.11-ekeupmdcqoimgroigtctln7tqkyh6pdm/lib/libz.so --with-mumps=0 --with-trilinos=0 --with-fftw=0 --with-valgrind=0 --with-gmp=0 --with-libpng=0 --with-giflib=0 --with-mpfr=0 --with-netcdf=0 --with-pnetcdf=0 --with-moab=0 --with-random123=0 --with-exodusii=0 --with-cgns=0 --with-memkind=0 --with-p4est=0 --with-saws=0 --with-yaml=0 --with-hwloc=0 --with-libjpeg=0 --with-scalapack=1 --with-scalapack-lib=/opt/cray/pe/libsci/21.08.1.2/GNU/9.1/x86_64/lib/libsci_gnu.so --with-strumpack=0 --with-mmg=0 --with-parmmg=0 --with-tetgen=0 --with-cuda-arch=80 ----------------------------------------- Libraries compiled on 2022-02-08 15:44:43 on login22 Machine characteristics: Linux-5.3.18-24.75_10.0.190-cray_shasta_c-x86_64-with-glibc2.26 Using PETSc directory: /global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/petsc-main-mnj56kbexro3fipf6kheyttljzwss7fo Using PETSc arch: ----------------------------------------- Using C compiler: /opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpicc -fPIC Using Fortran compiler: /opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpif90 -fPIC ----------------------------------------- Using include paths: -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/petsc-main-mnj56kbexro3fipf6kheyttljzwss7fo/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hypre-develop-3gtrobj6ky64qlq4jvi2qzou5mvisy4w/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/superlu-dist-develop-l5kc2sttvfqcjlejhgnvygfxwulrujga/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-kernels-3.5.00-zwq3aedpbg7ywpmqiqxmn5nx4w6hdrx6/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-3.5.00-65sqphcwz6lwtqectq6yswa6kt3654mb/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/parmetis-4.0.3-7xhbi6h22ni4fe35vxurnwmr6izbeb7b/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/metis-5.1.0-lxe5bhakcmkcf7zuqcagulm7tihcav7q/include -I/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.11-ekeupmdcqoimgroigtctln7tqkyh6pdm/include -I/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/include -I/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/math_libs/11.4/include ----------------------------------------- Using C linker: /opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpicc Using Fortran linker: /opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/bin/mpif90 Using libraries: -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/petsc-main-mnj56kbexro3fipf6kheyttljzwss7fo/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/petsc-main-mnj56kbexro3fipf6kheyttljzwss7fo/lib -lpetsc -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hypre-develop-3gtrobj6ky64qlq4jvi2qzou5mvisy4w/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hypre-develop-3gtrobj6ky64qlq4jvi2qzou5mvisy4w/lib -Wl,-rpath,/opt/cray/pe/libsci/21.08.1.2/GNU/9.1/x86_64/lib -L/opt/cray/pe/libsci/21.08.1.2/GNU/9.1/x86_64/lib -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/superlu-dist-develop-l5kc2sttvfqcjlejhgnvygfxwulrujga/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/superlu-dist-develop-l5kc2sttvfqcjlejhgnvygfxwulrujga/lib -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-kernels-3.5.00-zwq3aedpbg7ywpmqiqxmn5nx4w6hdrx6/lib64 -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-kernels-3.5.00-zwq3aedpbg7ywpmqiqxmn5nx4w6hdrx6/lib64 -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-3.5.00-65sqphcwz6lwtqectq6yswa6kt3654mb/lib64 -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/kokkos-3.5.00-65sqphcwz6lwtqectq6yswa6kt3654mb/lib64 -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.12.1-7pefaoio5q3hwnzggbnz7mpqw352gtsy/lib -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/parmetis-4.0.3-7xhbi6h22ni4fe35vxurnwmr6izbeb7b/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/parmetis-4.0.3-7xhbi6h22ni4fe35vxurnwmr6izbeb7b/lib -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/metis-5.1.0-lxe5bhakcmkcf7zuqcagulm7tihcav7q/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/metis-5.1.0-lxe5bhakcmkcf7zuqcagulm7tihcav7q/lib -Wl,-rpath,/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.11-ekeupmdcqoimgroigtctln7tqkyh6pdm/lib -L/global/u1/s/sajid/packages/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.11-ekeupmdcqoimgroigtctln7tqkyh6pdm/lib -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/lib64/stubs -Wl,-rpath,/opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/lib -L/opt/cray/pe/mpich/8.1.12/ofi/gnu/9.1/lib -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/lib64/stubs -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/nvvm/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/nvvm/lib64 -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/extras/CUPTI/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/extras/CUPTI/lib64 -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/extras/Debugger/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/cuda/11.4/extras/Debugger/lib64 -Wl,-rpath,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/math_libs/11.4/lib64 -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/math_libs/11.4/lib64 -Wl,-rpath,/opt/cray/pe/mpich/8.1.12/gtl/lib -L/opt/cray/pe/mpich/8.1.12/gtl/lib -Wl,-rpath,/opt/cray/pe/dsmml/0.2.2/dsmml/lib -L/opt/cray/pe/dsmml/0.2.2/dsmml/lib -Wl,-rpath,/opt/cray/xpmem/2.2.40-2.1_3.9__g3cf3325.shasta/lib64 -L/opt/cray/xpmem/2.2.40-2.1_3.9__g3cf3325.shasta/lib64 -Wl,-rpath,/opt/cray/pe/gcc/11.2.0/snos/lib/gcc/x86_64-suse-linux/11.2.0 -L/opt/cray/pe/gcc/11.2.0/snos/lib/gcc/x86_64-suse-linux/11.2.0 -Wl,-rpath,/opt/cray/pe/gcc/11.2.0/snos/lib64 -L/opt/cray/pe/gcc/11.2.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/gcc/11.2.0/snos/lib -L/opt/cray/pe/gcc/11.2.0/snos/lib -lHYPRE -lsci_gnu -lsuperlu_dist -lkokkoskernels -lkokkoscontainers -lkokkoscore -lsci_gnu -lhdf5_hl -lhdf5 -lparmetis -lmetis -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lcuda -lstdc++ -ldl -lmpifort_gnu_91 -lmpi_gnu_91 -lcuda -lmpi_gtl_cuda -lxpmem -lgfortran -lm -lcupti -lcudart -lsci_gnu_82_mpi -lsci_gnu_82 -ldsmml -lgfortran -lquadmath -lpthread -lm -lgcc_s -lquadmath -lstdc++ -ldl ----------------------------------------- ########################################################## # # # WARNING!!! # # # # This code was compiled with GPU support and you've # # created PETSc/GPU objects, but you intentionally used # # -use_gpu_aware_mpi 0, such that PETSc had to copy data # # from GPU to CPU for communication. To get meaningfull # # timing results, please use GPU-aware MPI instead. # ##########################################################