Creating /localscratch/pbstmp.159912.l2cu33.ichec.ie Working directory is /ichec/work/staff/tstitt/SolverCode Running with 8 processes Matrix has order 4704 rows by 4704 columns Number of RHS is: 4704 Worker Solve Time is: 6.86795616 Master Solve Time is: 8.66668129 Worker Solve Time is: 8.85465431 ************************************************************************************************************************ *** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document *** ************************************************************************************************************************ ---------------------------------------------- PETSc Performance Summary: ---------------------------------------------- ./solver on a pathscale named h3cu06 with 8 processors, by tstitt Tue Nov 20 20:30:38 2007 Using Petsc Release Version 2.3.3, Patch 7, Fri Oct 26 14:21:35 CDT 2007 HG revision: 2e223033ba960114833e1f9713ab393ec78c056f Max Max/Min Avg Total Worker Solve Time is: 10.0444736 Worker Solve Time is: 13.47995 Worker Solve Time is: 12.8490467 Worker Solve Time is: 16.845438 Worker Solve Time is: 11.9151878 Time (sec): 2.533e+01 1.00016 2.533e+01 Objects: 2.400e+01 1.00000 2.400e+01 Flops: 7.214e+09 1.89167 5.406e+09 4.325e+10 Flops/sec: 2.847e+08 1.89142 2.134e+08 1.707e+09 MPI Messages: 4.000e+00 1.33333 3.500e+00 2.800e+01 MPI Message Lengths: 6.336e+03 1.20824 1.676e+03 4.693e+04 MPI Reductions: 1.766e+03 1.00000 Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract) e.g., VecAXPY() for real vectors of length N --> 2N flops and VecAXPY() for complex vectors of length N --> 8N flops Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions -- Avg %Total Avg %Total counts %Total Avg %Total counts %Total 0: Main Stage: 2.5333e+01 100.0% 4.3245e+10 100.0% 2.800e+01 100.0% 1.676e+03 100.0% 1.413e+04 100.0% ------------------------------------------------------------------------------------------------------------------------ See the 'Profiling' chapter of the users' manual for details on interpreting output. Phase summary info: Count: number of times phase was executed Time and Flops/sec: Max - maximum over all processors Ratio - ratio of maximum to minimum over all processors Mess: number of messages sent Avg. len: average message length Reduct: number of global reductions Global: entire computation Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop(). %T - percent time in this phase %F - percent flops in this phase %M - percent messages in this phase %L - percent message lengths in this phase %R - percent reductions in this phase Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors) ------------------------------------------------------------------------------------------------------------------------ ########################################################## # # # WARNING!!! # # # # The code for various complex numbers numerical # # kernels uses C++, which generally is not well # # optimized. For performance that is about 4-5 times # # faster, specify --with-fortran-kernels=generic # # when running config/configure.py. # # # ########################################################## ########################################################## # # # WARNING!!! # # # # This code was run without the PreLoadBegin() # # macros. To get timing results we always recommend # # preloading. otherwise timing numbers may be # # meaningless. # ########################################################## Event Count Time (sec) Flops/sec --- Global --- --- Stage --- Total Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s ------------------------------------------------------------------------------------------------------------------------ --- Event Stage 0: Main Stage MatSolve 4704 1.0 1.6348e+01 2.6 6.21e+08 1.8 0.0e+00 0.0e+00 0.0e+00 42 98 0 0 0 42 98 0 0 0 2590 MatLUFactorNum 1 1.0 1.6377e-01 4.7 1.46e+09 1.2 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 5479 MatILUFactorSym 1 1.0 4.3809e-03 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyBegin 1 1.0 4.8079e-02101.2 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatAssemblyEnd 1 1.0 4.6219e-02 1.2 0.00e+00 0.0 2.8e+01 1.7e+03 7.0e+00 0 0100100 0 0 0100100 0 0 MatGetRowIJ 1 1.0 5.0068e-06 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 MatGetOrdering 1 1.0 1.0180e-04 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecSet 9408 1.0 3.2616e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 VecAssemblyBegin 4704 1.0 1.2538e+01 5.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.4e+04 32 0 0 0100 32 0 0 0100 0 VecAssemblyEnd 4704 1.0 1.8426e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSetup 2 1.0 4.7684e-06 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0 KSPSolve 4704 1.0 1.6674e+01 2.5 6.19e+08 1.8 0.0e+00 0.0e+00 5.0e+00 43100 0 0 0 43100 0 0 0 2594 PCSetUp 2 1.0 1.6838e-01 4.5 1.36e+09 1.2 0.0e+00 0.0e+00 5.0e+00 0 2 0 0 0 0 2 0 0 0 5329 PCSetUpOnBlocks 4704 1.0 1.8029e-01 3.7 1.14e+09 1.1 0.0e+00 0.0e+00 5.0e+00 0 2 0 0 0 0 2 0 0 0 4977 PCApply 4704 1.0 1.6482e+01 2.6 6.15e+08 1.8 0.0e+00 0.0e+00 0.0e+00 42 98 0 0 0 42 98 0 0 0 2569 ------------------------------------------------------------------------------------------------------------------------ Memory usage is given in bytes: Object Type Creations Destructions Memory Descendants' Mem. --- Event Stage 0: Main Stage Matrix 6 4 29267076 0 Index Set 7 7 17312 0 Vec 6 6 41352 0 Vec Scatter 1 1 0 0 Krylov Solver 2 2 0 0 Preconditioner 2 2 256 0 ======================================================================================================================== Average time to get PetscTime(): 2.40803e-06 Average time for MPI_Barrier(): 0.00820498 Average time for zero size MPI_Send(): 0.000125885 OptionTable: -log_summary OptionTable: -mat_type aijmumps Compiled without FORTRAN kernels Compiled with full precision matrices (default) sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 16 Configure run at: Thu Nov 15 23:52:44 2007 Configure options: --with-cxx=mpiCC --with-cc=mpicc --with-mpi-dir=/usr/local/mpich2/path3.0/ --with-blas-lib=/opt/packages/path-compat/acml/pathscale64/lib/libacml.a --with-lapack-lib=/opt/packages/path-compat/acml/pathscale64/lib/libacml.a --with-timer=mpi --with-fc=mpif77 --download-mumps=1 --download-scalapack=1 --download-superlu_dist=1 --download-superlu=1 --with-shared=0 --CXXOPTFLAGS=-fast --FOPTFLAGS=-fast --COPTFLAGS=-fast --download-blacs=1 --with-scalar-type=complex --with-debugging=0 --download-spooles=1 ----------------------------------------- Libraries compiled on Thu Nov 15 23:52:50 GMT 2007 on l2cu28 Machine characteristics: Linux l2cu28 2.6.5-7.287.3-smp_perfctr #3 SMP Wed Oct 17 21:27:48 BST 2007 x86_64 x86_64 x86_64 GNU/Linux Using PETSc directory: /ichec/work/staff/tstitt/petsc-2.3.3-p7 Using PETSc arch: pathscale_O3 ----------------------------------------- Using C compiler: mpicc -fPIC Using Fortran compiler: mpif77 -fPIC ----------------------------------------- Using include paths: -I/ichec/work/staff/tstitt/petsc-2.3.3-p7 -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/bmake/pathscale_O3 -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/include -I/usr/X11R6/include -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/MUMPS_4.7.3/pathscale_O3/include -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SCALAPACK/pathscale_O3/include -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/blacs-dev/pathscale_O3/include -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_DIST_2.0-Jan_5_2006/pathscale_O3/SRC -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/spooles-2.2/pathscale_O3/ -I/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_3.0-Jan_5_2006/pathscale_O3/SRC -I/usr/local/mpich2/path3.0/include ------------------------------------------ Using C linker: mpicc -fPIC Using Fortran linker: mpif77 -fPIC Using libraries: -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/lib/pathscale_O3 -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/lib/pathscale_O3 -lpetscts -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc -L/usr/X11R6/lib64 -lX11 -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/MUMPS_4.7.3/pathscale_O3/lib -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/MUMPS_4.7.3/pathscale_O3/lib -lcmumps -ldmumps -lsmumps -lzmumps -lpord -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SCALAPACK/pathscale_O3 -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SCALAPACK/pathscale_O3 -lscalapack -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/blacs-dev/pathscale_O3 -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/blacs-dev/pathscale_O3 -lblacs -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_DIST_2.0-Jan_5_2006/pathscale_O3 -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_DIST_2.0-Jan_5_2006/pathscale_O3 -lsuperlu_dist_2.0 /ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/spooles-2.2/pathscale_O3/MPI/src/spoolesMPI.a /ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/spooles-2.2/pathscale_O3/spooles.a -Wl,-rpath,/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_3.0-Jan_5_2006/pathscale_O3 -L/ichec/work/staff/tstitt/petsc-2.3.3-p7/externalpackages/SuperLU_3.0-Jan_5_2006/pathscale_O3 -lsuperlu_3.0 -Wl,-rpath,/opt/packages/path-compat/acml/pathscale64/lib -L/opt/packages/path-compat/acml/pathscale64/lib -lacml -Wl,-rpath,/opt/packages/path-compat/acml/pathscale64/lib -L/opt/packages/path-compat/acml/pathscale64/lib -lacml -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -L/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -L/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -L/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -L/usr/lib/../lib64 -ldl -lpmpich -lmpich -lpthread -lrt -lpscrt -lgcc_eh -lpathfstart -lpathfortran -lmv -lmpath -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -L/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -L/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -L/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -L/usr/lib/../lib64 -ldl -lpmpich -lmpich -lpthread -lrt -lpscrt -lgcc_eh -lpathfstart -lpathfortran -lmv -lmpath -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -lm -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -L/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -L/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -L/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -L/usr/lib/../lib64 -ldl -lpmpich -lmpich -lpthread -lrt -lpscrt -lgcc_eh -lpathfstart -lpathfortran -lmv -lmpath -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -lm -lm -Wl,-rpath,/usr/local/mpich2/path3.0/lib64 -L/usr/local/mpich2/path3.0/lib64 -Wl,-rpath,/opt/packages/pathscale3.0/lib/3.0 -L/opt/packages/pathscale3.0/lib/3.0 -ldl -lpmpich -lmpich -lpthread -lrt -lpscrt -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/lib -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../lib64 -Wl,-rpath,/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -L/usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../.. -Wl,-rpath,/lib/../lib64 -L/lib/../lib64 -Wl,-rpath,/usr/lib/../lib64 -L/usr/lib/../lib64 -lgcc_eh -ldl ------------------------------------------ Deleting /localscratch/pbstmp.159912.l2cu33.ichec.ie