gcc version 6.3.0 for x86_64 architecture loaded. openmpi version 1.10.2-gnu for x86_64 architecture loaded. Residual norm 4.34298e-08 Error norm 2.61494e-05 Error norm 5.11093e-06 Error norm 4.466e-10 ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- ./ex34 on a arch-linux2-c-opt named taurusi4063 with 1536 processors, by incard Fri Jun 16 05:04:30 2017 Using Petsc Release Version 3.6.4, Apr, 12, 2016 Max Max/Min Avg Total Time (sec): 4.341e+01 1.00090 4.339e+01 Objects: 6.900e+01 1.00000 6.900e+01 Flops: 1.190e+10 1.01640 1.184e+10 1.819e+13 Flops/sec: 2.745e+08 1.01689 2.729e+08 4.192e+11 MPI Messages: 4.746e+03 2.00000 4.235e+03 6.505e+06 MPI Message Lengths: 1.160e+08 2.02198 2.450e+04 1.594e+11 MPI Reductions: 2.446e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 4.3384e+01 100.0% 1.8188e+13 100.0% 6.505e+06 100.0% 2.450e+04 100.0% 2.445e+03 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length (bytes) Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ Event Count Time (sec) Flops --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage KSPGMRESOrthog 761 1.0 1.0226e+01 1.1 7.83e+09 1.0 0.0e+00 0.0e+00 7.6e+02 23 66 0 0 31 23 66 0 0 31 1170660 KSPSetUp 2 1.0 7.9459e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 1 1.0 4.0896e+01 1.0 1.19e+10 1.0 6.5e+06 2.5e+04 2.4e+03 94100100100 99 94100100100 99 444621 VecMDot 761 1.0 5.0882e+00 1.0 3.92e+09 1.0 0.0e+00 0.0e+00 7.6e+02 12 33 0 0 31 12 33 0 0 31 1176402 VecNorm 791 1.0 2.4617e+00 1.5 2.64e+08 1.0 0.0e+00 0.0e+00 7.9e+02 5 2 0 0 32 5 2 0 0 32 164205 VecScale 787 1.0 1.3570e-01 1.9 1.32e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 1484645 VecCopy 26 1.0 1.2141e-0124.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 815 1.0 2.3301e-01 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAXPY 52 1.0 9.8753e+00 1.3 1.74e+07 1.0 0.0e+00 0.0e+00 0.0e+00 20 0 0 0 0 20 0 0 0 0 2696 VecMAXPY 787 1.0 5.7940e+00 1.2 4.17e+09 1.0 0.0e+00 0.0e+00 0.0e+00 12 35 0 0 0 12 35 0 0 0 1100354 VecAssemblyBegin 2 1.0 1.6063e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 6.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAssemblyEnd 2 1.0 3.1710e-0511.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecScatterBegin 787 1.0 1.9311e-01 6.1 0.00e+00 0.0 6.5e+06 2.5e+04 0.0e+00 0 0 99100 0 0 0 99100 0 0 VecScatterEnd 787 1.0 2.8719e+00 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 5 0 0 0 0 0 VecNormalize 787 1.0 2.2430e+00 1.5 3.95e+08 1.0 0.0e+00 0.0e+00 7.9e+02 4 3 0 0 32 4 3 0 0 32 269470 MatMult 787 1.0 6.2805e+00 1.4 1.71e+09 1.0 6.5e+06 2.5e+04 0.0e+00 13 14 99100 0 13 14 99100 0 416308 MatSolve 787 1.0 5.8142e+00 1.9 1.68e+09 1.0 0.0e+00 0.0e+00 0.0e+00 8 14 0 0 0 8 14 0 0 0 442850 MatLUFactorNum 1 1.0 2.8629e-02 2.3 3.58e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 191349 MatILUFactorSym 1 1.0 2.7942e-02 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 2 1.0 1.8064e-01 4.8 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyEnd 2 1.0 1.5599e-01 1.0 0.00e+00 0.0 1.6e+04 6.1e+03 8.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetRowIJ 1 1.0 3.4580e-0325.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 9.6290e-03 4.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 PCSetUp 2 1.0 1.9596e-01 1.2 3.58e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 27955 PCSetUpOnBlocks 1 1.0 6.0671e-02 1.9 3.58e+06 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 90293 PCApply 787 1.0 6.0424e+00 1.9 1.68e+09 1.0 0.0e+00 0.0e+00 0.0e+00 8 14 0 0 0 8 14 0 0 0 426121 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. Reports information only for process 0. --- Event Stage 0: Main Stage Krylov Solver 2 2 19504 0 DMKSP interface 1 1 648 0 Vector 43 43 49721800 0 Vector Scatter 2 2 1342176 0 Matrix 4 4 35896172 0 Matrix Null Space 2 2 1184 0 Distributed Mesh 1 1 4960 0 Star Forest Bipartite Graph 2 2 1712 0 Discrete System 1 1 848 0 Index Set 7 7 2089032 0 IS L to G Mapping 1 1 708096 0 Preconditioner 2 2 1896 0 Viewer 1 0 0 0 ======================================================================================================================== Average time to get PetscTime(): 9.53674e-08 Average time for MPI_Barrier(): 3.66211e-05 Average time for zero size MPI_Send(): 2.9363e-06 #PETSc Option Table entries: -log_view #End of PETSc Option Table entries Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4 Configure options: --with-cxx-dialect=C++11 --with-mpi-dir=/sw/taurus/libraries/openmpi/1.10.2-gnu --with-parmetis=yes --with-parmetis-dir=/scratch/p_ppm//PARMETIS/ --with-metis=yes --with-metis-dir=/scratch/p_ppm//METIS --with-boost=yes --with-boost-dir=/scratch/p_ppm//BOOST --with-blas-lib=/scratch/p_ppm//OPENBLAS/lib/libopenblas.a --with-lapack-lib=/scratch/p_ppm//OPENBLAS/lib/libopenblas.a --with-suitesparse=yes --with-suitesparse-dir=/scratch/p_ppm//SUITESPARSE --with-trilinos=yes -with-trilinos-dir=/scratch/p_ppm//TRILINOS --with-scalapack=yes -with-scalapack-dir=/scratch/p_ppm//SCALAPACK --with-mumps=yes --with-mumps-include=/scratch/p_ppm//MUMPS/include --with-superlu_dist=yes --with-superlu_dist-lib=/scratch/p_ppm//SUPERLU_DIST/lib/libsuperlu_dist_4.3.a --with-superlu_dist-include=/scratch/p_ppm//SUPERLU_DIST/include/ --with-hypre=yes -with-hypre-dir=/scratch/p_ppm//HYPRE --with-mumps-lib=""/scratch/p_ppm//MUMPS/lib/libdmumps.a /scratch/p_ppm//MUMPS/lib/libmumps_common.a /scratch/p_ppm//MUMPS/lib/libpord.a"" --prefix=/scratch/p_ppm//PETSC --with-debugging=0 ----------------------------------------- Libraries compiled on Wed Feb 22 17:30:49 2017 on tauruslogin4 Machine characteristics: Linux-2.6.32-642.11.1.el6.Bull.106.x86_64-x86_64-with-redhat-6.8-Santiago Using PETSc directory: /lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4 Using PETSc arch: arch-linux2-c-opt ----------------------------------------- Using C compiler: /sw/taurus/libraries/openmpi/1.10.2-gnu/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O ${COPTFLAGS} ${CFLAGS} Using Fortran compiler: /sw/taurus/libraries/openmpi/1.10.2-gnu/bin/mpif90 -fPIC -Wall -Wno-unused-variable -ffree-line-length-0 -Wno-unused-dummy-argument -O ${FOPTFLAGS} ${FFLAGS} ----------------------------------------- Using include paths: -I/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/arch-linux2-c-opt/include -I/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/include -I/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/include -I/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/arch-linux2-c-opt/include -I/scratch/p_ppm/TRILINOS/include -I/scratch/p_ppm/HYPRE/include -I/scratch/p_ppm/SUPERLU_DIST/include -I/scratch/p_ppm/SUITESPARSE/include -I/scratch/p_ppm/MUMPS/include -I/scratch/p_ppm/PARMETIS/include -I/scratch/p_ppm/METIS/include -I/scratch/p_ppm/BOOST/include -I/sw/taurus/libraries/openmpi/1.10.2-gnu/include ----------------------------------------- Using C linker: /sw/taurus/libraries/openmpi/1.10.2-gnu/bin/mpicc Using Fortran linker: /sw/taurus/libraries/openmpi/1.10.2-gnu/bin/mpif90 Using libraries: -Wl,-rpath,/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/arch-linux2-c-opt/lib -L/lustre/scratch2/p_ppm/jenkins2/workspace/OpenFPM_high_scal_tests/petsc-3.6.4/arch-linux2-c-opt/lib -lpetsc -Wl,-rpath,/scratch/p_ppm/TRILINOS/lib -L/scratch/p_ppm/TRILINOS/lib -lpike-blackbox -ltrilinoscouplings -lmsqutil -lmesquite -lctrilinos -lsundancePdeopt -lsundanceStdFwk -lsundanceStdMesh -lsundanceCore -lsundanceInterop -lsundanceUtils -lsundancePlaya -lpiro -lrol -lstokhos_muelu -lstokhos_ifpack2 -lstokhos_amesos2 -lstokhos_tpetra -lstokhos_sacado -lstokhos -lmoochothyra -lmoocho -lrythmos -lmuelu-adapters -lmuelu-interface -lmuelu -lmoertel -llocathyra -llocaepetra -llocalapack -lloca -lnoxepetra -lnoxlapack -lnox -lphalanx -lstk_mesh_fixtures -lstk_search_util_base -lstk_search -lstk_unit_test_utils -lstk_io_util -lstk_io -lstk_mesh_base -lstk_topology -lstk_util_use_cases -lstk_util_registry -lstk_util_diag -lstk_util_env -lstk_util_util -lstkclassic_search_util -lstkclassic_search -lstkclassic_rebalance_utils -lstkclassic_rebalance -lstkclassic_linsys -lstkclassic_io_util -lstkclassic_io -lstkclassic_expreval -lstkclassic_algsup -lstkclassic_mesh_fem -lstkclassic_mesh_base -lstkclassic_util_use_cases -lstkclassic_util_unit_test_support -lstkclassic_util_parallel -lstkclassic_util_diag -lstkclassic_util_env -lstkclassic_util_util -lstk_mesh_fixtures -lstk_search_util_base -lstk_search -lstk_unit_test_utils -lstk_io_util -lstk_io -lstk_mesh_base -lstk_topology -lstk_util_use_cases -lstk_util_registry -lstk_util_diag -lstk_util_env -lstk_util_util -lstkclassic_search_util -lstkclassic_search -lstkclassic_rebalance_utils -lstkclassic_rebalance -lstkclassic_linsys -lstkclassic_io_util -lstkclassic_io -lstkclassic_expreval -lstkclassic_algsup -lstkclassic_mesh_fem -lstkclassic_mesh_base -lstkclassic_util_use_cases -lstkclassic_util_unit_test_support -lstkclassic_util_parallel -lstkclassic_util_diag -lstkclassic_util_env -lstkclassic_util_util -lintrepid -lteko -lfei_trilinos -lfei_base -lstratimikos -lstratimikosbelos -lstratimikosaztecoo -lstratimikosamesos -lstratimikosml -lstratimikosifpack -lifpack2-adapters -lifpack2 -lanasazitpetra -lModeLaplace -lanasaziepetra -lanasazi -lkomplex -lsupes -laprepro_lib -lchaco -lIonit -lIotr -lIohb -lIogn -lIopg -lIoss -lsupes -laprepro_lib -lchaco -lIonit -lIotr -lIohb -lIogn -lIopg -lIoss -lamesos2 -lshylu -lbelostpetra -lbelosepetra -lbelos -lml -lifpack -lzoltan2 -lpamgen_extras -lpamgen -lamesos -lgaleri-xpetra -lgaleri-epetra -laztecoo -ldpliris -lisorropia -loptipack -lxpetra-sup -lxpetra -lthyratpetra -lthyraepetraext -lthyraepetra -lthyracore -lthyratpetra -lthyraepetraext -lthyraepetra -lthyracore -lepetraext -ltpetraext -ltpetrainout -ltpetra -lkokkostsqr -ltpetrakernels -ltpetraclassiclinalg -ltpetraclassicnodeapi -ltpetraclassic -ltpetraext -ltpetrainout -ltpetra -lkokkostsqr -ltpetrakernels -ltpetraclassiclinalg -ltpetraclassicnodeapi -ltpetraclassic -ltriutils -lglobipack -lshards -lzoltan -lepetra -lsacado -lrtop -lteuchoskokkoscomm -lteuchoskokkoscompat -lteuchosremainder -lteuchosnumerics -lteuchoscomm -lteuchosparameterlist -lteuchoscore -lteuchoskokkoscomm -lteuchoskokkoscompat -lteuchosremainder -lteuchosnumerics -lteuchoscomm -lteuchosparameterlist -lteuchoscore -lkokkosalgorithms -lkokkoscontainers -lkokkoscore -lkokkosalgorithms -lkokkoscontainers -lkokkoscore -ltpi -lgtest -lpthread -Wl,-rpath,/scratch/p_ppm/HYPRE/lib -L/scratch/p_ppm/HYPRE/lib -lHYPRE -Wl,-rpath,/sw/taurus/libraries/openmpi/1.10.2-gnu/lib -L/sw/taurus/libraries/openmpi/1.10.2-gnu/lib -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib64 -L/sw/global/compilers/gcc/5.3.0/lib64 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -L/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 -L/sw/global/compilers/gcc/5.3.0/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/x86_64-unknown-linux-gnu/lib -L/sw/global/compilers/gcc/5.3.0/x86_64-unknown-linux-gnu/lib -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib -L/sw/global/compilers/gcc/5.3.0/lib -lmpi_cxx -lstdc++ -Wl,-rpath,/scratch/p_ppm//SUPERLU_DIST/lib -L/scratch/p_ppm//SUPERLU_DIST/lib -lsuperlu_dist_4.3 -Wl,-rpath,/scratch/p_ppm/SUITESPARSE/lib -L/scratch/p_ppm/SUITESPARSE/lib -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig -lrt -ldmumps -Wl,-rpath,/scratch/p_ppm//MUMPS/lib -L/scratch/p_ppm//MUMPS/lib -lmumps_common -lpord -Wl,-rpath,/scratch/p_ppm/SCALAPACK/lib -L/scratch/p_ppm/SCALAPACK/lib -lscalapack -Wl,-rpath,/scratch/p_ppm//OPENBLAS/lib -L/scratch/p_ppm//OPENBLAS/lib -lopenblas -Wl,-rpath,/scratch/p_ppm/PARMETIS/lib -L/scratch/p_ppm/PARMETIS/lib -lparmetis -Wl,-rpath,/scratch/p_ppm/METIS/lib -L/scratch/p_ppm/METIS/lib -lmetis -lX11 -lhwloc -lssl -lcrypto -lm -lmpi_usempi -lmpi_mpifh -lgfortran -lm -lgfortran -lm -lquadmath -lm -lmpi_cxx -lstdc++ -Wl,-rpath,/sw/taurus/libraries/openmpi/1.10.2-gnu/lib -L/sw/taurus/libraries/openmpi/1.10.2-gnu/lib -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib64 -L/sw/global/compilers/gcc/5.3.0/lib64 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -L/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib64 -L/sw/global/compilers/gcc/5.3.0/lib64 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib64 -L/sw/global/compilers/gcc/5.3.0/lib64 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 -L/sw/global/compilers/gcc/5.3.0/libexec/gcc/x86_64-unknown-linux-gnu/5.3.0 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -L/sw/global/compilers/gcc/5.3.0/lib/gcc/x86_64-unknown-linux-gnu/5.3.0 -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/x86_64-unknown-linux-gnu/lib -L/sw/global/compilers/gcc/5.3.0/x86_64-unknown-linux-gnu/lib -Wl,-rpath,/sw/global/compilers/gcc/5.3.0/lib -L/sw/global/compilers/gcc/5.3.0/lib -ldl -Wl,-rpath,/sw/taurus/libraries/openmpi/1.10.2-gnu/lib -lmpi -lgcc_s -lpthread -ldl -----------------------------------------