Solving a linear TS problem on 32 processors mx : 1024, my: 1024, energy(in eV) : 1.500000e+04 Assembling matrix Finished assembling matrix 0 TS dt 3.0808e-06 time 0. 0 KSP Residual norm 7.610755342711e+02 1 KSP Residual norm 3.422503042895e+01 2 KSP Residual norm 2.994376541252e+00 3 KSP Residual norm 2.001754024957e-01 4 KSP Residual norm 1.807732428477e-02 5 KSP Residual norm 1.927785938364e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 1 TS dt 3.0808e-06 time 3.0808e-06 0 KSP Residual norm 6.968388920107e+02 1 KSP Residual norm 4.919077350082e+01 2 KSP Residual norm 2.728146144495e+00 3 KSP Residual norm 2.190634209297e-01 4 KSP Residual norm 1.666780063922e-02 5 KSP Residual norm 1.690951204232e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 2 TS dt 3.0808e-06 time 6.1616e-06 0 KSP Residual norm 6.862501002548e+02 1 KSP Residual norm 3.255200154235e+01 2 KSP Residual norm 2.426686401916e+00 3 KSP Residual norm 1.866278590253e-01 4 KSP Residual norm 1.833695361952e-02 5 KSP Residual norm 1.656408615986e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 3 TS dt 3.0808e-06 time 9.2424e-06 0 KSP Residual norm 6.487576804743e+02 1 KSP Residual norm 3.133076135889e+01 2 KSP Residual norm 2.735857223108e+00 3 KSP Residual norm 2.095484678219e-01 4 KSP Residual norm 1.822858513881e-02 5 KSP Residual norm 1.750781558930e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 4 TS dt 3.0808e-06 time 1.23232e-05 0 KSP Residual norm 6.098333757752e+02 1 KSP Residual norm 4.325776999907e+01 2 KSP Residual norm 3.113862863733e+00 3 KSP Residual norm 2.134072035620e-01 4 KSP Residual norm 2.182858378301e-02 5 KSP Residual norm 1.919541098287e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 5 TS dt 3.0808e-06 time 1.5404e-05 0 KSP Residual norm 6.046649357527e+02 1 KSP Residual norm 3.064209337444e+01 2 KSP Residual norm 2.278509506336e+00 3 KSP Residual norm 1.928690729730e-01 4 KSP Residual norm 1.797017185365e-02 5 KSP Residual norm 1.719813105827e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 6 TS dt 3.0808e-06 time 1.84848e-05 0 KSP Residual norm 5.732346882496e+02 1 KSP Residual norm 3.089208548826e+01 2 KSP Residual norm 2.590020047179e+00 3 KSP Residual norm 1.944655898164e-01 4 KSP Residual norm 1.924859671755e-02 5 KSP Residual norm 1.677765033565e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 7 TS dt 3.0808e-06 time 2.15656e-05 0 KSP Residual norm 5.451187365926e+02 1 KSP Residual norm 3.923495267537e+01 2 KSP Residual norm 2.753578409322e+00 3 KSP Residual norm 1.998269238022e-01 4 KSP Residual norm 1.964516214211e-02 5 KSP Residual norm 1.728681503285e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 8 TS dt 3.0808e-06 time 2.46464e-05 0 KSP Residual norm 5.366254232818e+02 1 KSP Residual norm 2.950774075944e+01 2 KSP Residual norm 2.220205627563e+00 3 KSP Residual norm 1.793040307984e-01 4 KSP Residual norm 1.719982440180e-02 5 KSP Residual norm 1.595508314911e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 9 TS dt 3.0808e-06 time 2.77272e-05 0 KSP Residual norm 5.057174290022e+02 1 KSP Residual norm 3.150184752311e+01 2 KSP Residual norm 2.544180038720e+00 3 KSP Residual norm 1.927476908011e-01 4 KSP Residual norm 1.823216047198e-02 5 KSP Residual norm 1.647080965131e-03 KSP Object: 32 MPI processes type: gmres restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement happy breakdown tolerance 1e-30 maximum iterations=10000, initial guess is zero tolerances: relative=1e-05, absolute=1e-50, divergence=10000. left preconditioning using PRECONDITIONED norm type for convergence test PC Object: 32 MPI processes type: hypre HYPRE BoomerAMG preconditioning Cycle type V Maximum number of levels 25 Maximum number of iterations PER hypre call 2 Convergence tolerance PER hypre call 0. Threshold for strong coupling 0.5 Interpolation truncation factor 0. Interpolation: max elements per row 0 Number of levels of aggressive coarsening 2 Number of paths for aggressive coarsening 1 Maximum row sums 0.9 Sweeps down 2 Sweeps up 2 Sweeps on coarse 1 Relax down l1scaled-SOR/Jacobi Relax up l1scaled-SOR/Jacobi Relax on coarse Gaussian-elimination Relax weight (all) 1. Outer relax weight (all) 1. Using CF-relaxation Smooth type Euclid Smooth num levels 25 Euclid ILU(k) levels 2 Euclid ILU(k) drop tolerance 0. Euclid ILU use Block-Jacobi? 0 Measure type local Coarsen type modifiedRuge-Stueben Interpolation type ext+i linear system matrix followed by preconditioner matrix: Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 Mat Object: 32 MPI processes type: mpiaij rows=2097152, cols=2097152, bs=2 total: nonzeros=20971520, allocated nonzeros=20971520 total number of mallocs used during MatSetValues calls=0 using I-node (on process 0) routines: found 32768 nodes, limit used is 5 10 TS dt 3.0808e-06 time 3.0808e-05 ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- ./ex_k1 on a named xrmlite with 32 processors, by sajid Mon May 4 09:12:30 2020 Using Petsc Release Version 3.13.0, Mar 29, 2020 Max Max/Min Avg Total Time (sec): 2.906e+01 1.000 2.905e+01 Objects: 2.870e+02 1.000 2.870e+02 Flop: 1.672e+08 1.000 1.672e+08 5.350e+09 Flop/sec: 5.757e+06 1.000 5.754e+06 1.841e+08 MPI Messages: 4.720e+02 1.000 4.720e+02 1.510e+04 MPI Message Lengths: 1.150e+06 1.000 2.436e+03 3.679e+07 MPI Reductions: 4.260e+02 1.000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flop and VecAXPY() for complex vectors of length N --> 8N flop Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total Count %Total Avg %Total Count %Total 0: Main Stage: 2.9055e+01 100.0% 5.3498e+09 100.0% 1.510e+04 100.0% 2.436e+03 100.0% 4.190e+02 98.4% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flop: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent AvgLen: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flop in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flop --- Global --- --- Stage ---- Total Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage BuildTwoSided 57 1.0 1.0304e-01 1.9 0.00e+00 0.0 3.1e+03 8.0e+00 5.7e+01 0 0 20 0 13 0 0 20 0 14 0 BuildTwoSidedF 33 1.0 1.0124e-01 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 3.3e+01 0 0 0 0 8 0 0 0 0 8 0 MatMult 70 1.0 3.6311e-01 1.1 8.72e+07 1.0 9.0e+03 3.1e+03 0.0e+00 1 52 59 75 0 1 52 59 75 0 7681 MatConvert 12 1.0 8.2637e-01 1.0 0.00e+00 0.0 5.1e+02 1.5e+03 5.2e+01 3 0 3 2 12 3 0 3 2 12 0 MatAssemblyBegin 52 1.0 1.0193e-01 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 3.2e+01 0 0 0 0 8 0 0 0 0 8 0 MatAssemblyEnd 52 1.0 6.6859e-01 1.0 0.00e+00 0.0 5.4e+03 1.5e+03 1.4e+02 2 0 36 23 34 2 0 36 23 35 0 MatGetRowIJ 20 1.0 3.6716e-05 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatZeroEntries 20 1.0 4.1392e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatView 20 1.0 6.7339e-03 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+01 0 0 0 0 5 0 0 0 0 5 0 MatAXPY 20 1.0 3.8299e+00 1.0 1.31e+07 1.0 5.1e+03 1.5e+03 1.6e+02 13 8 34 21 38 13 8 34 21 38 110 DMCreateMat 1 1.0 1.5669e-01 1.0 0.00e+00 0.0 2.6e+02 1.5e+03 8.0e+00 1 0 2 1 2 1 0 2 1 2 0 SFSetGraph 24 1.0 6.4468e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFSetUp 24 1.0 1.1593e-02 1.2 0.00e+00 0.0 6.1e+03 1.5e+03 2.4e+01 0 0 41 25 6 0 0 41 25 6 0 SFBcastOpBegin 70 1.0 4.2777e-03 1.4 0.00e+00 0.0 9.0e+03 3.1e+03 0.0e+00 0 0 59 75 0 0 0 59 75 0 0 SFBcastOpEnd 70 1.0 1.5302e-03 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFPack 70 1.0 2.0413e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 SFUnpack 70 1.0 8.8453e-05 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecView 1 1.0 2.7998e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0 VecMDot 50 1.0 1.2228e-01 2.5 1.97e+07 1.0 0.0e+00 0.0e+00 5.0e+01 0 12 0 0 12 0 12 0 0 12 5145 VecNorm 60 1.0 2.5339e-02 2.9 7.86e+06 1.0 0.0e+00 0.0e+00 6.0e+01 0 5 0 0 14 0 5 0 0 14 9932 VecScale 70 1.0 9.7790e-03 1.1 4.59e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 15012 VecCopy 40 1.0 1.3064e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 113 1.0 1.6201e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 30 1.0 8.1933e-03 1.3 3.93e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 15358 VecAYPX 20 1.0 8.7259e-03 1.1 1.31e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 4807 VecAXPBYCZ 10 1.0 7.2308e-03 1.2 1.97e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 8701 VecMAXPY 60 1.0 5.6753e-02 1.1 2.62e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 16 0 0 0 0 16 0 0 0 14781 VecAssemblyBegin 1 1.0 6.1035e-05 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAssemblyEnd 1 1.0 4.5300e-06 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecLoad 1 1.0 2.6538e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 70 1.0 4.5543e-03 1.4 0.00e+00 0.0 9.0e+03 3.1e+03 0.0e+00 0 0 59 75 0 0 0 59 75 0 0 VecScatterEnd 70 1.0 1.8802e-03 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecNormalize 60 1.0 2.9338e-02 2.3 1.18e+07 1.0 0.0e+00 0.0e+00 6.0e+01 0 7 0 0 14 0 7 0 0 14 12867 TSStep 10 1.0 2.8370e+01 1.0 1.67e+08 1.0 1.4e+04 2.5e+03 3.7e+02 98100 93 96 88 98100 93 96 89 188 TSFunctionEval 20 1.0 1.2298e-01 1.1 2.62e+07 1.0 2.6e+03 3.1e+03 0.0e+00 0 16 17 21 0 0 16 17 21 0 6821 TSJacobianEval 30 1.0 4.1307e+00 1.0 1.44e+07 1.0 5.1e+03 1.5e+03 1.6e+02 14 9 34 21 38 14 9 34 21 38 112 SNESSolve 10 1.0 2.8292e+01 1.0 1.53e+08 1.0 1.3e+04 2.5e+03 3.7e+02 97 92 85 86 87 97 92 85 86 88 173 SNESSetUp 1 1.0 5.2094e-04 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 SNESFunctionEval 10 1.0 6.7383e-02 1.1 1.51e+07 1.0 1.3e+03 3.1e+03 0.0e+00 0 9 8 11 0 0 9 8 11 0 7158 SNESJacobianEval 10 1.0 4.1308e+00 1.0 1.44e+07 1.0 5.1e+03 1.5e+03 1.6e+02 14 9 34 21 38 14 9 34 21 38 112 KSPSetUp 10 1.0 1.2853e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 2 0 0 0 0 2 0 KSPSolve 10 1.0 2.4090e+01 1.0 1.21e+08 1.0 6.4e+03 3.1e+03 1.9e+02 83 73 42 53 45 83 73 42 53 45 161 KSPGMRESOrthog 50 1.0 1.6271e-01 1.8 3.93e+07 1.0 0.0e+00 0.0e+00 5.0e+01 0 24 0 0 12 0 24 0 0 12 7733 PCSetUp 10 1.0 2.1469e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 5.0e+01 74 0 0 0 12 74 0 0 0 12 0 PCApply 60 1.0 2.2729e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 8 0 0 0 0 8 0 0 0 0 0 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Distributed Mesh 2 1 5088 0. Matrix 79 79 360212240 0. Index Set 46 46 1106704 0. IS L to G Mapping 1 0 0 0. Star Forest Graph 28 25 30280 0. Discrete System 2 1 992 0. Vec Scatter 24 23 19872 0. Vector 91 91 12504160 0. Viewer 4 3 2592 0. TSAdapt 1 1 1448 0. TS 1 1 2472 0. DMTS 1 0 0 0. SNES 1 1 1532 0. DMSNES 3 2 1440 0. Krylov Solver 1 1 18936 0. DMKSP interface 1 0 0 0. Preconditioner 1 1 1680 0. ======================================================================================================================== Average time to get PetscTime(): 7.15256e-08 Average time for MPI_Barrier(): 1.22547e-05 Average time for zero size MPI_Send(): 2.80887e-06 #PETSc Option Table entries: -ksp_monitor -ksp_reuse_preconditioner -ksp_type gmres -ksp_view -log_view -pc_hypre_boomeramg_agg_nl 2 -pc_hypre_boomeramg_coarsen_type modifiedRuge-Stueben -pc_hypre_boomeramg_eu_level 2 -pc_hypre_boomeramg_grid_sweeps_all 2 -pc_hypre_boomeramg_interp_type ext+i -pc_hypre_boomeramg_max_iter 2 -pc_hypre_boomeramg_numfunctions 2 -pc_hypre_boomeramg_relax_type_all l1scaled-SOR/Jacobi -pc_hypre_boomeramg_smooth_type Euclid -pc_hypre_boomeramg_strong_threshold 0.5 -pc_hypre_euclid_reuse -pc_hypre_type boomeramg -pc_type hypre -prop_steps 10 -ts_monitor -ts_type cn #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with 64 bit PetscInt Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 8 Configure options: --prefix=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/petsc-3.13.0-ixjmudlfqith3lrxfcttq2f3plvucfrt --with-ssl=0 --download-c2html=0 --download-sowing=0 --download-hwloc=0 CFLAGS= FFLAGS= CXXFLAGS= --with-cc=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpicc --with-cxx=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpic++ --with-fc=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpif90 --with-precision=double --with-scalar-type=real --with-shared-libraries=1 --with-debugging=0 --with-64-bit-indices=1 COPTFLAGS= FOPTFLAGS= CXXOPTFLAGS= --with-blaslapack-lib="/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/intel-mkl-2020.0.166-xcdij7v4hccrboxlwsyrjnarehyaauzt/compilers_and_libraries_2020.0.166/linux/mkl/lib/intel64/libmkl_intel_lp64.so /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/intel-mkl-2020.0.166-xcdij7v4hccrboxlwsyrjnarehyaauzt/compilers_and_libraries_2020.0.166/linux/mkl/lib/intel64/libmkl_sequential.so /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/intel-mkl-2020.0.166-xcdij7v4hccrboxlwsyrjnarehyaauzt/compilers_and_libraries_2020.0.166/linux/mkl/lib/intel64/libmkl_core.so /lib64/libpthread.so /lib64/libm.so /lib64/libdl.so" --with-x=0 --with-clanguage=C --with-scalapack=0 --with-metis=1 --with-metis-dir=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/metis-5.1.0-cc5mnza4r4hdocybr7hgnaa55qdygegv --with-hdf5=1 --with-hdf5-dir=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hdf5-1.10.6-u2yapuygssqkrvo7qcihw66kzlg3ngtw --with-hypre=1 --with-hypre-dir=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hypre-2.18.2-4p4r2ph4zp5hbufbpswitiiij37oovuw --with-parmetis=1 --with-parmetis-dir=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/parmetis-4.0.3-vxj3qtfmtdzyzyg2t3e224gocvgabu4h --with-mumps=0 --with-trilinos=0 --with-fftw=0 --with-valgrind=0 --with-cxx-dialect=C++11 --with-superlu_dist-include=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/superlu-dist-6.3.0-suzf4hdgfgdjpblojcglmp7wc2wcjepk/include --with-superlu_dist-lib=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/superlu-dist-6.3.0-suzf4hdgfgdjpblojcglmp7wc2wcjepk/lib/libsuperlu_dist.a --with-superlu_dist=1 --with-suitesparse=0 --with-zlib-include=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/zlib-1.2.11-fjzlxw5lmcb2y4s6ca2e4su4qteufcm7/include --with-zlib-lib=/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/zlib-1.2.11-fjzlxw5lmcb2y4s6ca2e4su4qteufcm7/lib/libz.so --with-zlib=1 ----------------------------------------- Libraries compiled on 2020-04-14 04:45:47 on xrmlite Machine characteristics: Linux-4.18.0-147.5.1.el8_1.x86_64-x86_64-with-centos-8.1.1911-Core Using PETSc directory: /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/petsc-3.13.0-ixjmudlfqith3lrxfcttq2f3plvucfrt Using PETSc arch: ----------------------------------------- Using C compiler: /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpicc -fPIC Using Fortran compiler: /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpif90 -fPIC ----------------------------------------- Using include paths: -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/petsc-3.13.0-ixjmudlfqith3lrxfcttq2f3plvucfrt/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hypre-2.18.2-4p4r2ph4zp5hbufbpswitiiij37oovuw/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/superlu-dist-6.3.0-suzf4hdgfgdjpblojcglmp7wc2wcjepk/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hdf5-1.10.6-u2yapuygssqkrvo7qcihw66kzlg3ngtw/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/parmetis-4.0.3-vxj3qtfmtdzyzyg2t3e224gocvgabu4h/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/metis-5.1.0-cc5mnza4r4hdocybr7hgnaa55qdygegv/include -I/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/zlib-1.2.11-fjzlxw5lmcb2y4s6ca2e4su4qteufcm7/include ----------------------------------------- Using C linker: /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpicc Using Fortran linker: /home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/bin/mpif90 Using libraries: -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/petsc-3.13.0-ixjmudlfqith3lrxfcttq2f3plvucfrt/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/petsc-3.13.0-ixjmudlfqith3lrxfcttq2f3plvucfrt/lib -lpetsc -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hypre-2.18.2-4p4r2ph4zp5hbufbpswitiiij37oovuw/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hypre-2.18.2-4p4r2ph4zp5hbufbpswitiiij37oovuw/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/superlu-dist-6.3.0-suzf4hdgfgdjpblojcglmp7wc2wcjepk/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/superlu-dist-6.3.0-suzf4hdgfgdjpblojcglmp7wc2wcjepk/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/intel-mkl-2020.0.166-xcdij7v4hccrboxlwsyrjnarehyaauzt/compilers_and_libraries_2020.0.166/linux/mkl/lib/intel64 -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/intel-mkl-2020.0.166-xcdij7v4hccrboxlwsyrjnarehyaauzt/compilers_and_libraries_2020.0.166/linux/mkl/lib/intel64 /lib64/libpthread.so /lib64/libm.so /lib64/libdl.so -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hdf5-1.10.6-u2yapuygssqkrvo7qcihw66kzlg3ngtw/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/hdf5-1.10.6-u2yapuygssqkrvo7qcihw66kzlg3ngtw/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/parmetis-4.0.3-vxj3qtfmtdzyzyg2t3e224gocvgabu4h/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/parmetis-4.0.3-vxj3qtfmtdzyzyg2t3e224gocvgabu4h/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/metis-5.1.0-cc5mnza4r4hdocybr7hgnaa55qdygegv/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/metis-5.1.0-cc5mnza4r4hdocybr7hgnaa55qdygegv/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/zlib-1.2.11-fjzlxw5lmcb2y4s6ca2e4su4qteufcm7/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/zlib-1.2.11-fjzlxw5lmcb2y4s6ca2e4su4qteufcm7/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.3.0/mpich-3.3.2-oxccmmod4vmpmxsz47se5pjxnsyy5kdt/lib -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib:/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib64 -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib/gcc/x86_64-pc-linux-gnu/8.3.0 -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib64 -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib64 -Wl,-rpath,/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib -L/home/sajid/packages/spack/opt/spack/linux-centos8-broadwell/gcc-8.2.1/gcc-8.3.0-j573htph2tblzijltjxvql7hkkzzkpyn/lib -lHYPRE -lsuperlu_dist -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lparmetis -lmetis -lm -lz -lstdc++ -ldl -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -ldl -----------------------------------------