[petsc-users] Investigate parallel code to improve parallelism
TAY Wee Beng
zonexo at gmail.com
Sun Feb 28 19:08:05 CST 2016
Hi,
I've attached the files for x cells running y procs. hypre is called
natively I'm not sure if PETSc catches it.
Thanks
On 29/2/2016 1:11 AM, Barry Smith wrote:
> As I said before, send the -log_summary output for the two processor sizes and we'll look at where it is spending its time and how it could possibly be improved.
>
> Barry
>
>> On Feb 28, 2016, at 10:29 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>
>>
>> On 27/2/2016 12:53 AM, Barry Smith wrote:
>>>> On Feb 26, 2016, at 10:27 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>
>>>>
>>>> On 26/2/2016 11:32 PM, Barry Smith wrote:
>>>>>> On Feb 26, 2016, at 9:28 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have got a 3D code. When I ran with 48 procs and 11 million cells, it runs for 83 min. When I ran with 96 procs and 22 million cells, it ran for 99 min.
>>>>> This is actually pretty good!
>>>> But if I'm not wrong, if I increase the no. of cells, the parallelism will keep on decreasing. I hope it scales up to maybe 300 - 400 procs.
>> Hi,
>>
>> I think I may have mentioned this before, that is, I need to submit a proposal to request for computing nodes. In the proposal, I'm supposed to run some simulations to estimate the time it takes to run my code. Then an excel file will use my input to estimate the efficiency when I run my code with more cells. They use 2 mtds to estimate:
>>
>> 1. strong scaling, whereby I run 2 cases - 1st with n cells and x procs, then with n cells and 2x procs. From there, they can estimate my expected efficiency when I have y procs. The formula is attached in the pdf.
>>
>> 2. weak scaling, whereby I run 2 cases - 1st with n cells and x procs, then with 2n cells and 2x procs. From there, they can estimate my expected efficiency when I have y procs. The formula is attached in the pdf.
>>
>> So if I use 48 and 96 procs and get maybe 80% efficiency, by the time I hit 800 procs, I get 32% efficiency for strong scaling. They expect at least 50% efficiency for my code. To reach that, I need to achieve 89% efficiency when I use 48 and 96 procs.
>>
>> So now my qn is how accurate is this type of calculation, especially wrt to PETSc?
>>
>> Similarly, for weak scaling, is it accurate?
>>
>> Can I argue that this estimation is not suitable for PETSc or hypre?
>>
>> Thanks
>>
>>
>>>>>> So it's not that parallel. I want to find out which part of the code I need to improve. Also if PETsc and hypre is working well in parallel. What's the best way to do it?
>>>>> Run both with -log_summary and send the output for each case. This will show where the time is being spent and which parts are scaling less well.
>>>>>
>>>>> Barry
>>>> That's only for the PETSc part, right? So for other parts of the code, including hypre part, I will not be able to find out. If so, what can I use to check these parts?
>>> You will still be able to see what percentage of the time is spent in hypre and if it increases with the problem size and how much. So the information will still be useful.
>>>
>>> Barry
>>>
>>>>>> I thought of doing profiling but if the code is optimized, I wonder if it still works well.
>>>>>>
>>>>>> --
>>>>>> Thank you.
>>>>>>
>>>>>> Yours sincerely,
>>>>>>
>>>>>> TAY wee-beng
>>>>>>
>> <temp.pdf>
--
Thank you
Yours sincerely,
TAY wee-beng
-------------- next part --------------
0.000000000000000E+000 0.600000000000000 17.5000000000000
120.000000000000 0.000000000000000E+000 0.250000000000000
1.00000000000000 0.400000000000000 0 -400000
AB,AA,BB -2.78150003711926 2.76500003633555
2.78150003711926 2.70650003355695
size_x,size_y,size_z 100 172 171
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 0 1 22
1 29 1 63800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 1 23 44
1 29 63801 127600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 2 45 66
1 29 127601 191400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 3 67 88
1 29 191401 255200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 4 89 109
1 29 255201 316100
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 5 110 130
1 29 316101 377000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 6 131 151
1 29 377001 437900
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 7 152 172
1 29 437901 498800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 8 1 22
30 58 498801 562600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 9 23 44
30 58 562601 626400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 10 45 66
30 58 626401 690200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 11 67 88
30 58 690201 754000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 12 89 109
30 58 754001 814900
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 13 110 130
30 58 814901 875800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 14 131 151
30 58 875801 936700
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 15 152 172
30 58 936701 997600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 16 1 22
59 87 997601 1061400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 17 23 44
59 87 1061401 1125200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 18 45 66
59 87 1125201 1189000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 19 67 88
59 87 1189001 1252800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 20 89 109
59 87 1252801 1313700
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 21 110 130
59 87 1313701 1374600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 22 131 151
59 87 1374601 1435500
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 23 152 172
59 87 1435501 1496400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 24 1 22
88 115 1496401 1558000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 25 23 44
88 115 1558001 1619600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 26 45 66
88 115 1619601 1681200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 27 67 88
88 115 1681201 1742800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 28 89 109
88 115 1742801 1801600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 29 110 130
88 115 1801601 1860400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 30 131 151
88 115 1860401 1919200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 31 152 172
88 115 1919201 1978000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 32 1 22
116 143 1978001 2039600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 33 23 44
116 143 2039601 2101200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 34 45 66
116 143 2101201 2162800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 35 67 88
116 143 2162801 2224400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 36 89 109
116 143 2224401 2283200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 37 110 130
116 143 2283201 2342000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 38 131 151
116 143 2342001 2400800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 39 152 172
116 143 2400801 2459600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 40 1 22
144 171 2459601 2521200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 41 23 44
144 171 2521201 2582800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 42 45 66
144 171 2582801 2644400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 43 67 88
144 171 2644401 2706000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 44 89 109
144 171 2706001 2764800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 45 110 130
144 171 2764801 2823600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 46 131 151
144 171 2823601 2882400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 47 152 172
144 171 2882401 2941200
body_cg_ini 0.850000999999998 9.999999998273846E-007
6.95771875020604
3104 surfaces with wrong vertex ordering
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
7.847540176996057E-002 3.349995610000001E-002 3.500000000000000E-002
maximum ngh_surfaces and ngh_vertics are 28 10
minimum ngh_surfaces and ngh_vertics are 14 5
min IIB_cell_no 0
max IIB_cell_no 160
final initial IIB_cell_no 8000
min I_cell_no 0
max I_cell_no 152
final initial I_cell_no 7600
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u)
8000 7600 8000 7600
IIB_I_cell_no_uvw_total1 2230 2227 2166 1930
1926 1847
1 0.01411765 0.30104754 0.32529731 1.15440698 -0.30539502E+03 -0.29715696E+02 0.29394159E+07
2 0.00973086 0.41244275 0.45087918 1.22116705 -0.34883625E+03 -0.24811937E+02 0.29392112E+07
3 0.00918174 0.45383297 0.51181352 1.27753799 -0.35798168E+03 -0.24574571E+02 0.29391681E+07
4 0.00885756 0.47398509 0.55172076 1.31014211 -0.36230580E+03 -0.25263557E+02 0.29391476E+07
5 0.00872228 0.48832349 0.57971242 1.32671388 -0.36519380E+03 -0.26103818E+02 0.29391332E+07
300 0.00157913 4.12249858 6.52091323 3.82250060 -0.19591967E+04 0.43743742E+05 0.29433413E+07
600 0.00164417 3.72487671 5.92911552 5.37207663 -0.52948695E+03 0.20658440E+05 0.29461349E+07
900 0.00135887 4.43677663 5.89462632 4.98710090 -0.86683472E+04 -0.13481580E+05 0.29508693E+07
1200 0.00164451 4.13807017 6.05251254 4.63339167 -0.98993801E+04 0.17681878E+05 0.29569013E+07
1500 0.00150966 3.40576464 5.20747744 4.68945079 -0.16263794E+05 0.26012616E+05 0.29604137E+07
1800 0.00132562 3.67104160 10.44718173 4.20460908 -0.18839985E+05 -0.18296757E+05 0.29677661E+07
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-0.892081024888283 -1.32621439664773 -5.584506755655451E-003
0.350447346081129 -9.754897341839677E-002 -2.09284777911273
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.6.3_static_rel named n12-04 with 48 processors, by wtay Sat Feb 27 16:15:12 2016
Using Petsc Release Version 3.6.3, Dec, 03, 2015
Max Max/Min Avg Total
Time (sec): 3.252e+03 1.00001 3.252e+03
Objects: 2.008e+04 1.00000 2.008e+04
Flops: 1.590e+11 1.09643 1.523e+11 7.310e+12
Flops/sec: 4.891e+07 1.09643 4.683e+07 2.248e+09
MPI Messages: 8.531e+04 1.93829 7.180e+04 3.446e+06
MPI Message Lengths: 4.573e+09 2.08162 5.352e+04 1.844e+11
MPI Reductions: 4.427e+04 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 3.2521e+03 100.0% 7.3101e+12 100.0% 3.446e+06 100.0% 5.352e+04 100.0% 4.427e+04 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 3998 1.0 2.8182e+01 3.2 1.53e+09 1.1 0.0e+00 0.0e+00 4.0e+03 0 1 0 0 9 0 1 0 0 9 2503
VecDotNorm2 1999 1.0 2.3131e+01 5.3 1.53e+09 1.1 0.0e+00 0.0e+00 2.0e+03 0 1 0 0 5 0 1 0 0 5 3050
VecNorm 3998 1.0 1.6799e+01 3.4 1.53e+09 1.1 0.0e+00 0.0e+00 4.0e+03 0 1 0 0 9 0 1 0 0 9 4200
VecCopy 3998 1.0 4.9803e+00 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 12002 1.0 1.1035e+01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 3998 1.0 9.5659e+00 1.1 3.06e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 14751
VecWAXPY 3998 1.0 1.0318e+01 1.1 1.53e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 6838
VecAssemblyBegin 3998 1.0 1.1847e+01 8.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+04 0 0 0 0 27 0 0 0 0 27 0
VecAssemblyEnd 3998 1.0 1.0632e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 16002 1.0 1.3053e+01 1.4 0.00e+00 0.0 2.6e+06 7.0e+04 0.0e+00 0 0 76100 0 0 0 76100 0 0
VecScatterEnd 16002 1.0 2.4842e+01 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0
MatMult 3998 1.0 1.5690e+02 1.2 5.86e+10 1.1 6.6e+05 1.2e+05 0.0e+00 4 37 19 43 0 4 37 19 43 0 17137
MatSolve 5997 1.0 2.3313e+02 1.1 8.47e+10 1.1 0.0e+00 0.0e+00 0.0e+00 7 53 0 0 0 7 53 0 0 0 16729
MatLUFactorNum 104 1.0 2.2413e+01 1.2 6.54e+09 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 13280
MatILUFactorSym 1 1.0 1.9445e-01 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 5.4134e-02 1.5 7.43e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6293
MatAssemblyBegin 105 1.0 2.0050e+0010.2 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+02 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 105 1.0 2.4443e+00 1.1 0.00e+00 0.0 3.3e+02 3.0e+04 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 8.1062e-06 4.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 8.0609e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 105 1.0 1.2339e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 1999 1.0 4.6014e+02 1.0 1.59e+11 1.1 6.6e+05 1.2e+05 1.0e+04 14100 19 43 23 14100 19 43 23 15886
PCSetUp 208 1.0 2.2611e+01 1.2 6.54e+09 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 13163
PCSetUpOnBlocks 1999 1.0 4.2470e-01 1.2 6.29e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6739
PCApply 5997 1.0 2.5994e+02 1.1 9.11e+10 1.1 0.0e+00 0.0e+00 0.0e+00 8 57 0 0 0 8 57 0 0 0 16138
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 4032 4032 30946112 0
Vector Scatter 2010 15 3596160 0
Matrix 4 4 182686524 0
Distributed Mesh 2003 8 39680 0
Star Forest Bipartite Graph 4006 16 13696 0
Discrete System 2003 8 6784 0
Index Set 4013 4013 15434720 0
IS L to G Mapping 2003 8 2092944 0
Krylov Solver 2 2 2296 0
Preconditioner 2 2 1896 0
Viewer 1 0 0 0
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
Average time for MPI_Barrier(): 1.01566e-05
Average time for zero size MPI_Send(): 7.45555e-06
#PETSc Option Table entries:
-log_summary
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/opt/ud/openmpi-1.8.8/ --with-blas-lapack-dir=/opt/ud/intel_xe_2013sp1/mkl/lib/intel64/ --with-debugging=0 --download-hypre=1 --prefix=/home/wtay/Lib/petsc-3.6.3_static_rel --known-mpi-shared=0 --with-shared-libraries=0 --with-fortran-interfaces=1
-----------------------------------------
Libraries compiled on Thu Jan 7 04:05:35 2016 on hpc12
Machine characteristics: Linux-3.10.0-123.20.1.el7.x86_64-x86_64-with-centos-7.1.1503-Core
Using PETSc directory: /home/wtay/Codes/petsc-3.6.3
Using PETSc arch: petsc-3.6.3_static_rel
-----------------------------------------
Using C compiler: /opt/ud/openmpi-1.8.8/bin/mpicc -wd1572 -O3 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /opt/ud/openmpi-1.8.8/bin/mpif90 -O3 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Lib/petsc-3.6.3_static_rel/include -I/opt/ud/openmpi-1.8.8/include
-----------------------------------------
Using C linker: /opt/ud/openmpi-1.8.8/bin/mpicc
Using Fortran linker: /opt/ud/openmpi-1.8.8/bin/mpif90
Using libraries: -Wl,-rpath,/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -L/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -lpetsc -Wl,-rpath,/home/wtay/Lib/petsc-3.6.3_static_rel/lib -L/home/wtay/Lib/petsc-3.6.3_static_rel/lib -lHYPRE -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -lmpi_cxx -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -Wl,-rpath,/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -L/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lhwloc -lssl -lcrypto -lmpi_usempi -lmpi_mpifh -lifport -lifcore -lm -lmpi_cxx -ldl -L/opt/ud/openmpi-1.8.8/lib -lmpi -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lpthread -lirc_s -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -ldl
-----------------------------------------
-------------- next part --------------
0.000000000000000E+000 0.600000000000000 17.5000000000000
120.000000000000 0.000000000000000E+000 0.250000000000000
1.00000000000000 0.400000000000000 0 -400000
AB,AA,BB -2.78150003711926 2.76500003633555
2.78150003711926 2.70650003355695
size_x,size_y,size_z 100 172 171
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 0 1 29
1 43 1 124700
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 1 30 58
1 43 124701 249400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 2 59 87
1 43 249401 374100
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 3 88 116
1 43 374101 498800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 4 117 144
1 43 498801 619200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 5 145 172
1 43 619201 739600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 6 1 29
44 86 739601 864300
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 7 30 58
44 86 864301 989000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 8 59 87
44 86 989001 1113700
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 9 88 116
44 86 1113701 1238400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 10 117 144
44 86 1238401 1358800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 11 145 172
44 86 1358801 1479200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 12 1 29
87 129 1479201 1603900
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 13 30 58
87 129 1603901 1728600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 14 59 87
87 129 1728601 1853300
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 15 88 116
87 129 1853301 1978000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 16 117 144
87 129 1978001 2098400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 17 145 172
87 129 2098401 2218800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 18 1 29
130 171 2218801 2340600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 19 30 58
130 171 2340601 2462400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 20 59 87
130 171 2462401 2584200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 21 88 116
130 171 2584201 2706000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 22 117 144
130 171 2706001 2823600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 23 145 172
130 171 2823601 2941200
body_cg_ini 0.850000999999998 9.999999998273846E-007
6.95771875020604
3104 surfaces with wrong vertex ordering
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
7.847540176996057E-002 3.349995610000001E-002 3.500000000000000E-002
maximum ngh_surfaces and ngh_vertics are 28 12
minimum ngh_surfaces and ngh_vertics are 14 5
min IIB_cell_no 0
max IIB_cell_no 229
final initial IIB_cell_no 11450
min I_cell_no 0
max I_cell_no 200
final initial I_cell_no 10000
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u)
11450 10000 11450 10000
IIB_I_cell_no_uvw_total1 2230 2227 2166 1930
1926 1847
1 0.01411765 0.30104754 0.32529731 1.15440698 -0.30539502E+03 -0.29715696E+02 0.29394159E+07
2 0.00973086 0.41244573 0.45086899 1.22116550 -0.34890134E+03 -0.25062690E+02 0.29392110E+07
3 0.00918177 0.45383616 0.51179402 1.27757073 -0.35811483E+03 -0.25027396E+02 0.29391677E+07
4 0.00885764 0.47398774 0.55169119 1.31019526 -0.36250500E+03 -0.25910050E+02 0.29391470E+07
5 0.00872241 0.48832538 0.57967282 1.32679047 -0.36545763E+03 -0.26947216E+02 0.29391325E+07
300 0.00163886 4.27898628 6.83028522 3.60837060 -0.19609891E+04 0.43984454E+05 0.29435194E+07
600 0.00160193 3.91014241 4.97460210 5.10461274 -0.61092521E+03 0.18910563E+05 0.29467790E+07
900 0.00150521 3.27352854 5.85427996 4.49166453 -0.89281765E+04 -0.12171584E+05 0.29507471E+07
1200 0.00165280 3.05922213 7.37243530 5.16434634 -0.10954640E+05 0.22049957E+05 0.29575213E+07
1500 0.00153718 3.54908044 5.42918256 4.84940953 -0.16430153E+05 0.24407130E+05 0.29608940E+07
1800 0.00155455 3.30956962 8.35799538 4.50638757 -0.20003619E+05 -0.20349497E+05 0.29676102E+07
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-1.29348921431473 -2.44525665200003 -0.238725356553914
0.644444280391413 -3.056662699041206E-002 -2.91791118488116
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.6.3_static_rel named n12-09 with 24 processors, by wtay Sat Feb 27 16:58:01 2016
Using Petsc Release Version 3.6.3, Dec, 03, 2015
Max Max/Min Avg Total
Time (sec): 5.791e+03 1.00001 5.791e+03
Objects: 2.008e+04 1.00000 2.008e+04
Flops: 3.129e+11 1.06806 3.066e+11 7.360e+12
Flops/sec: 5.402e+07 1.06807 5.295e+07 1.271e+09
MPI Messages: 8.298e+04 1.89703 6.585e+04 1.580e+06
MPI Message Lengths: 6.456e+09 2.05684 7.780e+04 1.229e+11
MPI Reductions: 4.427e+04 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 5.7911e+03 100.0% 7.3595e+12 100.0% 1.580e+06 100.0% 7.780e+04 100.0% 4.427e+04 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 3998 1.0 1.1437e+02 2.3 2.99e+09 1.1 0.0e+00 0.0e+00 4.0e+03 1 1 0 0 9 1 1 0 0 9 617
VecDotNorm2 1999 1.0 1.0442e+02 2.6 2.99e+09 1.1 0.0e+00 0.0e+00 2.0e+03 1 1 0 0 5 1 1 0 0 5 676
VecNorm 3998 1.0 8.5426e+01 2.2 2.99e+09 1.1 0.0e+00 0.0e+00 4.0e+03 1 1 0 0 9 1 1 0 0 9 826
VecCopy 3998 1.0 7.3321e+00 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 12002 1.0 1.2399e+01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 3998 1.0 1.8118e+01 1.4 5.98e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 7788
VecWAXPY 3998 1.0 1.6979e+01 1.3 2.99e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 4155
VecAssemblyBegin 3998 1.0 4.1001e+01 5.6 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+04 0 0 0 0 27 0 0 0 0 27 0
VecAssemblyEnd 3998 1.0 1.4657e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 16002 1.0 1.9519e+01 1.5 0.00e+00 0.0 1.2e+06 1.0e+05 0.0e+00 0 0 77100 0 0 0 77100 0 0
VecScatterEnd 16002 1.0 1.3223e+02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0
MatMult 3998 1.0 3.0904e+02 1.3 1.15e+11 1.1 3.0e+05 1.7e+05 0.0e+00 5 37 19 43 0 5 37 19 43 0 8700
MatSolve 5997 1.0 3.9285e+02 1.4 1.67e+11 1.1 0.0e+00 0.0e+00 0.0e+00 6 54 0 0 0 6 54 0 0 0 10040
MatLUFactorNum 104 1.0 4.2097e+01 1.2 1.30e+10 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 7190
MatILUFactorSym 1 1.0 2.9875e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 1.3492e-01 3.3 1.45e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 2525
MatAssemblyBegin 105 1.0 5.9000e+00 4.7 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+02 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 105 1.0 4.7665e+00 1.1 0.00e+00 0.0 1.5e+02 4.3e+04 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 3.6001e-0518.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 1.6249e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 105 1.0 2.7945e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 1999 1.0 9.1973e+02 1.0 3.13e+11 1.1 3.0e+05 1.7e+05 1.0e+04 16100 19 43 23 16100 19 43 23 8001
PCSetUp 208 1.0 4.2401e+01 1.2 1.30e+10 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 7138
PCSetUpOnBlocks 1999 1.0 7.2389e-01 1.2 1.25e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4020
PCApply 5997 1.0 4.4054e+02 1.3 1.80e+11 1.1 0.0e+00 0.0e+00 0.0e+00 6 58 0 0 0 6 58 0 0 0 9634
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 4032 4032 53827712 0
Vector Scatter 2010 15 7012720 0
Matrix 4 4 359683260 0
Distributed Mesh 2003 8 39680 0
Star Forest Bipartite Graph 4006 16 13696 0
Discrete System 2003 8 6784 0
Index Set 4013 4013 25819112 0
IS L to G Mapping 2003 8 3919440 0
Krylov Solver 2 2 2296 0
Preconditioner 2 2 1896 0
Viewer 1 0 0 0
========================================================================================================================
Average time to get PetscTime(): 1.90735e-07
Average time for MPI_Barrier(): 7.20024e-06
Average time for zero size MPI_Send(): 1.83781e-06
#PETSc Option Table entries:
-log_summary
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/opt/ud/openmpi-1.8.8/ --with-blas-lapack-dir=/opt/ud/intel_xe_2013sp1/mkl/lib/intel64/ --with-debugging=0 --download-hypre=1 --prefix=/home/wtay/Lib/petsc-3.6.3_static_rel --known-mpi-shared=0 --with-shared-libraries=0 --with-fortran-interfaces=1
-----------------------------------------
Libraries compiled on Thu Jan 7 04:05:35 2016 on hpc12
Machine characteristics: Linux-3.10.0-123.20.1.el7.x86_64-x86_64-with-centos-7.1.1503-Core
Using PETSc directory: /home/wtay/Codes/petsc-3.6.3
Using PETSc arch: petsc-3.6.3_static_rel
-----------------------------------------
Using C compiler: /opt/ud/openmpi-1.8.8/bin/mpicc -wd1572 -O3 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /opt/ud/openmpi-1.8.8/bin/mpif90 -O3 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Lib/petsc-3.6.3_static_rel/include -I/opt/ud/openmpi-1.8.8/include
-----------------------------------------
Using C linker: /opt/ud/openmpi-1.8.8/bin/mpicc
Using Fortran linker: /opt/ud/openmpi-1.8.8/bin/mpif90
Using libraries: -Wl,-rpath,/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -L/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -lpetsc -Wl,-rpath,/home/wtay/Lib/petsc-3.6.3_static_rel/lib -L/home/wtay/Lib/petsc-3.6.3_static_rel/lib -lHYPRE -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -lmpi_cxx -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -Wl,-rpath,/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -L/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lhwloc -lssl -lcrypto -lmpi_usempi -lmpi_mpifh -lifport -lifcore -lm -lmpi_cxx -ldl -L/opt/ud/openmpi-1.8.8/lib -lmpi -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lpthread -lirc_s -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -ldl
-----------------------------------------
-------------- next part --------------
0.000000000000000E+000 0.600000000000000 17.5000000000000
120.000000000000 0.000000000000000E+000 0.250000000000000
1.00000000000000 0.400000000000000 0 -400000
AB,AA,BB -3.06400005053729 3.04150004946860
3.06400005053729 2.98650004685624
size_x,size_y,size_z 128 217 210
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 0 1 28
1 35 1 125440
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 1 29 55
1 35 125441 246400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 2 56 82
1 35 246401 367360
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 3 83 109
1 35 367361 488320
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 4 110 136
1 35 488321 609280
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 5 137 163
1 35 609281 730240
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 6 164 190
1 35 730241 851200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 7 191 217
1 35 851201 972160
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 8 1 28
36 70 972161 1097600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 9 29 55
36 70 1097601 1218560
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 10 56 82
36 70 1218561 1339520
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 11 83 109
36 70 1339521 1460480
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 12 110 136
36 70 1460481 1581440
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 13 137 163
36 70 1581441 1702400
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 14 164 190
36 70 1702401 1823360
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 15 191 217
36 70 1823361 1944320
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 16 1 28
71 105 1944321 2069760
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 17 29 55
71 105 2069761 2190720
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 18 56 82
71 105 2190721 2311680
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 19 83 109
71 105 2311681 2432640
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 20 110 136
71 105 2432641 2553600
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 21 137 163
71 105 2553601 2674560
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 22 164 190
71 105 2674561 2795520
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 23 191 217
71 105 2795521 2916480
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 24 1 28
106 140 2916481 3041920
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 25 29 55
106 140 3041921 3162880
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 26 56 82
106 140 3162881 3283840
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 27 83 109
106 140 3283841 3404800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 28 110 136
106 140 3404801 3525760
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 29 137 163
106 140 3525761 3646720
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 30 164 190
106 140 3646721 3767680
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 31 191 217
106 140 3767681 3888640
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 32 1 28
141 175 3888641 4014080
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 33 29 55
141 175 4014081 4135040
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 34 56 82
141 175 4135041 4256000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 35 83 109
141 175 4256001 4376960
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 36 110 136
141 175 4376961 4497920
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 37 137 163
141 175 4497921 4618880
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 38 164 190
141 175 4618881 4739840
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 39 191 217
141 175 4739841 4860800
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 40 1 28
176 210 4860801 4986240
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 41 29 55
176 210 4986241 5107200
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 42 56 82
176 210 5107201 5228160
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 43 83 109
176 210 5228161 5349120
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 44 110 136
176 210 5349121 5470080
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 45 137 163
176 210 5470081 5591040
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 46 164 190
176 210 5591041 5712000
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 47 191 217
176 210 5712001 5832960
body_cg_ini 0.850000999999998 9.999999998273846E-007
6.95771875020604
3104 surfaces with wrong vertex ordering
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
7.847540176996057E-002 3.349995610000001E-002 2.600000000000000E-002
maximum ngh_surfaces and ngh_vertics are 15 5
minimum ngh_surfaces and ngh_vertics are 9 3
min IIB_cell_no 0
max IIB_cell_no 293
final initial IIB_cell_no 14650
min I_cell_no 0
max I_cell_no 330
final initial I_cell_no 16500
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u)
14650 16500 14650 16500
IIB_I_cell_no_uvw_total1 4065 4064 3990 4692
4628 4632
1 0.01043478 0.34272324 0.35569220 1.16440147 -0.68429429E+03 -0.71309583E+02 0.58290411E+07
2 0.00697380 0.42763102 0.50182401 1.25425283 -0.77750642E+03 -0.58636989E+02 0.58286176E+07
3 0.00648381 0.45520992 0.57545722 1.32431608 -0.79650320E+03 -0.55533364E+02 0.58285305E+07
4 0.00611631 0.46806365 0.62117944 1.37204042 -0.80501258E+03 -0.54624483E+02 0.58284901E+07
5 0.00592372 0.47659245 0.65166569 1.40276768 -0.81047260E+03 -0.54497615E+02 0.58284626E+07
300 0.00115066 3.67033285 6.63969480 3.71347160 0.43279356E+04 0.80255072E+05 0.58398256E+07
600 0.00122543 3.75818573 5.23852131 5.46557154 -0.66201226E+04 0.11004125E+06 0.58427867E+07
900 0.00103133 3.87890472 6.22189985 4.38435056 -0.71743226E+03 0.42678349E+05 0.58534173E+07
1200 0.00112141 3.50676323 7.96027994 5.21426706 -0.16564613E+05 -0.28161927E+05 0.58651131E+07
1500 0.00116845 5.55355360 4.80276829 4.70049706 -0.22633631E+05 -0.12833717E+05 0.58692098E+07
1800 0.00106663 3.60921979 9.26223511 4.67749400 -0.30654568E+05 0.62643544E+05 0.58867013E+07
escape_time reached, so abort
cd_cl_cs_mom_implicit1
0.515674932183890 -1.12997638014196 0.113834045912081
7.478720062203850E-002 -1.299985230479712E-002 -0.226394142710332
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.6.3_static_rel named n12-04 with 48 processors, by wtay Sat Feb 27 18:01:18 2016
Using Petsc Release Version 3.6.3, Dec, 03, 2015
Max Max/Min Avg Total
Time (sec): 6.333e+03 1.00000 6.333e+03
Objects: 2.008e+04 1.00000 2.008e+04
Flops: 3.132e+11 1.04090 3.039e+11 1.459e+13
Flops/sec: 4.945e+07 1.04090 4.799e+07 2.303e+09
MPI Messages: 8.482e+04 1.92729 7.176e+04 3.444e+06
MPI Message Lengths: 7.117e+09 1.99989 8.514e+04 2.933e+11
MPI Reductions: 4.427e+04 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 6.3329e+03 100.0% 1.4588e+13 100.0% 3.444e+06 100.0% 8.514e+04 100.0% 4.427e+04 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 3998 1.0 4.4692e+01 2.5 3.01e+09 1.0 0.0e+00 0.0e+00 4.0e+03 0 1 0 0 9 0 1 0 0 9 3131
VecDotNorm2 1999 1.0 3.7771e+01 3.2 3.01e+09 1.0 0.0e+00 0.0e+00 2.0e+03 0 1 0 0 5 0 1 0 0 5 3704
VecNorm 3998 1.0 6.2085e+01 5.2 3.01e+09 1.0 0.0e+00 0.0e+00 4.0e+03 1 1 0 0 9 1 1 0 0 9 2254
VecCopy 3998 1.0 8.7522e+00 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 12002 1.0 1.5554e+01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 3998 1.0 1.9816e+01 1.1 6.02e+09 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 14122
VecWAXPY 3998 1.0 2.0635e+01 1.1 3.01e+09 1.0 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 6781
VecAssemblyBegin 3998 1.0 1.4403e+01 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+04 0 0 0 0 27 0 0 0 0 27 0
VecAssemblyEnd 3998 1.0 1.6043e-02 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 16002 1.0 2.8572e+01 1.3 0.00e+00 0.0 2.6e+06 1.1e+05 0.0e+00 0 0 76100 0 0 0 76100 0 0
VecScatterEnd 16002 1.0 5.2833e+01 2.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0
MatMult 3998 1.0 3.1279e+02 1.1 1.14e+11 1.0 6.6e+05 1.9e+05 0.0e+00 5 37 19 43 0 5 37 19 43 0 17084
MatSolve 5997 1.0 4.8329e+02 1.1 1.68e+11 1.0 0.0e+00 0.0e+00 0.0e+00 7 53 0 0 0 7 53 0 0 0 16147
MatLUFactorNum 104 1.0 9.8655e+01 1.6 1.27e+10 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 6084
MatILUFactorSym 1 1.0 3.9025e-01 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 1.0553e-01 1.4 1.45e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 6415
MatAssemblyBegin 105 1.0 2.1250e+00 5.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+02 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 105 1.0 4.9930e+00 1.1 0.00e+00 0.0 3.3e+02 4.8e+04 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 5.0068e-06 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 2.0131e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 105 1.0 4.2514e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 1999 1.0 9.8702e+02 1.0 3.13e+11 1.0 6.6e+05 1.9e+05 1.0e+04 16100 19 43 23 16100 19 43 23 14779
PCSetUp 208 1.0 9.9050e+01 1.6 1.27e+10 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 6060
PCSetUpOnBlocks 1999 1.0 1.3700e+00 1.4 1.22e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4213
PCApply 5997 1.0 5.8749e+02 1.2 1.81e+11 1.0 0.0e+00 0.0e+00 0.0e+00 8 58 0 0 0 8 58 0 0 0 14295
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 4032 4032 54303424 0
Vector Scatter 2010 15 7054512 0
Matrix 4 4 361394700 0
Distributed Mesh 2003 8 39680 0
Star Forest Bipartite Graph 4006 16 13696 0
Discrete System 2003 8 6784 0
Index Set 4013 4013 27475284 0
IS L to G Mapping 2003 8 3991872 0
Krylov Solver 2 2 2296 0
Preconditioner 2 2 1896 0
Viewer 1 0 0 0
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
Average time for MPI_Barrier(): 6.2561e-05
Average time for zero size MPI_Send(): 1.70867e-06
#PETSc Option Table entries:
-log_summary
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/opt/ud/openmpi-1.8.8/ --with-blas-lapack-dir=/opt/ud/intel_xe_2013sp1/mkl/lib/intel64/ --with-debugging=0 --download-hypre=1 --prefix=/home/wtay/Lib/petsc-3.6.3_static_rel --known-mpi-shared=0 --with-shared-libraries=0 --with-fortran-interfaces=1
-----------------------------------------
Libraries compiled on Thu Jan 7 04:05:35 2016 on hpc12
Machine characteristics: Linux-3.10.0-123.20.1.el7.x86_64-x86_64-with-centos-7.1.1503-Core
Using PETSc directory: /home/wtay/Codes/petsc-3.6.3
Using PETSc arch: petsc-3.6.3_static_rel
-----------------------------------------
Using C compiler: /opt/ud/openmpi-1.8.8/bin/mpicc -wd1572 -O3 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /opt/ud/openmpi-1.8.8/bin/mpif90 -O3 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Lib/petsc-3.6.3_static_rel/include -I/opt/ud/openmpi-1.8.8/include
-----------------------------------------
Using C linker: /opt/ud/openmpi-1.8.8/bin/mpicc
Using Fortran linker: /opt/ud/openmpi-1.8.8/bin/mpif90
Using libraries: -Wl,-rpath,/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -L/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -lpetsc -Wl,-rpath,/home/wtay/Lib/petsc-3.6.3_static_rel/lib -L/home/wtay/Lib/petsc-3.6.3_static_rel/lib -lHYPRE -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -lmpi_cxx -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -Wl,-rpath,/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -L/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lhwloc -lssl -lcrypto -lmpi_usempi -lmpi_mpifh -lifport -lifcore -lm -lmpi_cxx -ldl -L/opt/ud/openmpi-1.8.8/lib -lmpi -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lpthread -lirc_s -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -ldl
-----------------------------------------
-------------- next part --------------
0.000000000000000E+000 0.600000000000000 17.5000000000000
120.000000000000 0.000000000000000E+000 0.250000000000000
1.00000000000000 0.400000000000000 0 -400000
AB,AA,BB -2.51050002424745 2.47300002246629
2.51050002424745 2.43950002087513
size_x,size_y,size_z 79 137 141
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 0 1 35
1 24 1 66360
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 1 36 69
1 24 66361 130824
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 2 70 103
1 24 130825 195288
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 3 104 137
1 24 195289 259752
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 4 1 35
25 48 259753 326112
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 5 36 69
25 48 326113 390576
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 6 70 103
25 48 390577 455040
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 7 104 137
25 48 455041 519504
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 8 1 35
49 72 519505 585864
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 9 36 69
49 72 585865 650328
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 10 70 103
49 72 650329 714792
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 11 104 137
49 72 714793 779256
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 12 1 35
73 95 779257 842851
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 13 36 69
73 95 842852 904629
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 14 70 103
73 95 904630 966407
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 15 104 137
73 95 966408 1028185
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 16 1 35
96 118 1028186 1091780
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 17 36 69
96 118 1091781 1153558
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 18 70 103
96 118 1153559 1215336
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 19 104 137
96 118 1215337 1277114
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 20 1 35
119 141 1277115 1340709
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 21 36 69
119 141 1340710 1402487
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 22 70 103
119 141 1402488 1464265
myid,jsta,jend,ksta,kend,ijk_sta,ijk_end 23 104 137
119 141 1464266 1526043
body_cg_ini 0.850000999999998 9.999999998273846E-007
6.95771875020604
3104 surfaces with wrong vertex ordering
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
7.847540176996057E-002 3.349995610000001E-002 4.700000000000000E-002
maximum ngh_surfaces and ngh_vertics are 47 22
minimum ngh_surfaces and ngh_vertics are 22 9
min IIB_cell_no 0
max IIB_cell_no 112
final initial IIB_cell_no 5600
min I_cell_no 0
max I_cell_no 96
final initial I_cell_no 4800
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u)
5600 4800 5600 4800
IIB_I_cell_no_uvw_total1 1221 1206 1212 775
761 751
1 0.01904762 0.28410536 0.31610359 1.14440147 -0.14430869E+03 -0.13111542E+02 0.15251948E+07
2 0.01348578 0.34638018 0.42392119 1.23447223 -0.16528393E+03 -0.10238827E+02 0.15250907E+07
3 0.01252674 0.38305826 0.49569053 1.27891383 -0.16912542E+03 -0.95950253E+01 0.15250695E+07
4 0.01199639 0.41337279 0.54168038 1.29584768 -0.17048065E+03 -0.94814301E+01 0.15250602E+07
5 0.01165251 0.43544137 0.57347276 1.30255981 -0.17129184E+03 -0.95170304E+01 0.15250538E+07
300 0.00236362 3.56353622 5.06727508 4.03923148 -0.78697893E+03 0.15046453E+05 0.15263125E+07
600 0.00253142 2.94537779 5.74258126 4.71794271 -0.38271069E+04 -0.49150195E+04 0.15289768E+07
900 0.00220341 3.10439489 6.70144317 4.01105348 -0.71943943E+04 0.13728311E+05 0.15320532E+07
1200 0.00245748 3.53496741 7.33163591 4.01935315 -0.85017750E+04 -0.77550358E+04 0.15350351E+07
1500 0.00244299 3.71751725 5.93463559 4.12005108 -0.95364451E+04 0.81223334E+04 0.15373061E+07
1800 0.00237474 3.49908653 5.20866314 4.69712853 -0.10382365E+05 -0.18966840E+04 0.15385160E+07
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-1.03894256791350 -1.53179673343374 6.737940408853320E-002
0.357464909626058 -0.103698436387821 -2.42688484514611
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.6.3_static_rel named n12-09 with 24 processors, by wtay Sat Feb 27 16:09:41 2016
Using Petsc Release Version 3.6.3, Dec, 03, 2015
Max Max/Min Avg Total
Time (sec): 2.922e+03 1.00001 2.922e+03
Objects: 2.008e+04 1.00000 2.008e+04
Flops: 1.651e+11 1.08049 1.582e+11 3.797e+12
Flops/sec: 5.652e+07 1.08049 5.414e+07 1.299e+09
MPI Messages: 8.293e+04 1.89333 6.588e+04 1.581e+06
MPI Message Lengths: 4.109e+09 2.03497 4.964e+04 7.849e+10
MPI Reductions: 4.427e+04 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 2.9219e+03 100.0% 3.7965e+12 100.0% 1.581e+06 100.0% 4.964e+04 100.0% 4.427e+04 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 3998 1.0 4.4655e+01 5.1 1.59e+09 1.1 0.0e+00 0.0e+00 4.0e+03 1 1 0 0 9 1 1 0 0 9 820
VecDotNorm2 1999 1.0 4.0603e+01 7.6 1.59e+09 1.1 0.0e+00 0.0e+00 2.0e+03 1 1 0 0 5 1 1 0 0 5 902
VecNorm 3998 1.0 3.0557e+01 6.2 1.59e+09 1.1 0.0e+00 0.0e+00 4.0e+03 1 1 0 0 9 1 1 0 0 9 1198
VecCopy 3998 1.0 4.4206e+00 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 12002 1.0 9.3725e+00 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 3998 1.0 9.1178e+00 1.5 3.18e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 8030
VecWAXPY 3998 1.0 9.3186e+00 1.5 1.59e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 3928
VecAssemblyBegin 3998 1.0 1.5680e+01 4.2 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+04 0 0 0 0 27 0 0 0 0 27 0
VecAssemblyEnd 3998 1.0 1.1443e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 16002 1.0 9.0984e+00 1.4 0.00e+00 0.0 1.2e+06 6.4e+04 0.0e+00 0 0 77100 0 0 0 77100 0 0
VecScatterEnd 16002 1.0 4.4821e+01 4.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0
MatMult 3998 1.0 1.4268e+02 1.3 6.05e+10 1.1 3.0e+05 1.1e+05 0.0e+00 4 37 19 43 0 4 37 19 43 0 9753
MatSolve 5997 1.0 2.0469e+02 1.4 8.84e+10 1.1 0.0e+00 0.0e+00 0.0e+00 6 53 0 0 0 6 53 0 0 0 9921
MatLUFactorNum 104 1.0 2.2332e+01 1.1 6.70e+09 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 6922
MatILUFactorSym 1 1.0 1.0867e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 3.8305e-02 1.9 7.67e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 4603
MatAssemblyBegin 105 1.0 2.0776e+00 3.6 0.00e+00 0.0 0.0e+00 0.0e+00 2.1e+02 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 105 1.0 2.4702e+00 1.1 0.00e+00 0.0 1.5e+02 2.8e+04 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 4.0531e-06 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 7.1249e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPSetUp 105 1.0 9.8758e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 1999 1.0 4.1857e+02 1.0 1.65e+11 1.1 3.0e+05 1.1e+05 1.0e+04 14100 19 43 23 14100 19 43 23 9070
PCSetUp 208 1.0 2.2440e+01 1.1 6.70e+09 1.1 0.0e+00 0.0e+00 0.0e+00 1 4 0 0 0 1 4 0 0 0 6888
PCSetUpOnBlocks 1999 1.0 2.7087e-01 1.1 6.44e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 5487
PCApply 5997 1.0 2.3123e+02 1.3 9.50e+10 1.1 0.0e+00 0.0e+00 0.0e+00 6 58 0 0 0 6 58 0 0 0 9444
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 4032 4032 31782464 0
Vector Scatter 2010 15 3738624 0
Matrix 4 4 190398024 0
Distributed Mesh 2003 8 39680 0
Star Forest Bipartite Graph 4006 16 13696 0
Discrete System 2003 8 6784 0
Index Set 4013 4013 14715400 0
IS L to G Mapping 2003 8 2137148 0
Krylov Solver 2 2 2296 0
Preconditioner 2 2 1896 0
Viewer 1 0 0 0
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
Average time for MPI_Barrier(): 8.15392e-06
Average time for zero size MPI_Send(): 1.12454e-05
#PETSc Option Table entries:
-log_summary
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/opt/ud/openmpi-1.8.8/ --with-blas-lapack-dir=/opt/ud/intel_xe_2013sp1/mkl/lib/intel64/ --with-debugging=0 --download-hypre=1 --prefix=/home/wtay/Lib/petsc-3.6.3_static_rel --known-mpi-shared=0 --with-shared-libraries=0 --with-fortran-interfaces=1
-----------------------------------------
Libraries compiled on Thu Jan 7 04:05:35 2016 on hpc12
Machine characteristics: Linux-3.10.0-123.20.1.el7.x86_64-x86_64-with-centos-7.1.1503-Core
Using PETSc directory: /home/wtay/Codes/petsc-3.6.3
Using PETSc arch: petsc-3.6.3_static_rel
-----------------------------------------
Using C compiler: /opt/ud/openmpi-1.8.8/bin/mpicc -wd1572 -O3 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /opt/ud/openmpi-1.8.8/bin/mpif90 -O3 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/include -I/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/include -I/home/wtay/Lib/petsc-3.6.3_static_rel/include -I/opt/ud/openmpi-1.8.8/include
-----------------------------------------
Using C linker: /opt/ud/openmpi-1.8.8/bin/mpicc
Using Fortran linker: /opt/ud/openmpi-1.8.8/bin/mpif90
Using libraries: -Wl,-rpath,/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -L/home/wtay/Codes/petsc-3.6.3/petsc-3.6.3_static_rel/lib -lpetsc -Wl,-rpath,/home/wtay/Lib/petsc-3.6.3_static_rel/lib -L/home/wtay/Lib/petsc-3.6.3_static_rel/lib -lHYPRE -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -lmpi_cxx -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -Wl,-rpath,/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -L/opt/ud/intel_xe_2013sp1/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lhwloc -lssl -lcrypto -lmpi_usempi -lmpi_mpifh -lifport -lifcore -lm -lmpi_cxx -ldl -L/opt/ud/openmpi-1.8.8/lib -lmpi -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -Wl,-rpath,/opt/ud/openmpi-1.8.8/lib -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lpthread -lirc_s -L/opt/ud/openmpi-1.8.8/lib -L/opt/ud/intel_xe_2013sp1/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.3 -ldl
-----------------------------------------
More information about the petsc-users
mailing list