[petsc-users] Scaling problem when cores > 600
TAY wee-beng
zonexo at gmail.com
Mon Mar 5 23:19:12 CST 2018
On 5/3/2018 11:43 AM, Smith, Barry F. wrote:
> 360 process
>
> KSPSolve 99 1.0 2.6403e+02 1.0 6.67e+10 1.1 2.7e+05 9.9e+05 5.1e+02 15100 17 42 19 15100 17 42 19 87401
>
> 1920 processes
>
> KSPSolve 99 1.0 2.3184e+01 1.0 1.32e+10 1.2 1.5e+06 4.3e+05 5.1e+02 4100 17 42 19 4100 17 42 19 967717
>
>
> Ratio of number of processes 5.33 ratio of time for KSPSolve 11.388 so the time for the solve is scaling very well (extremely well actually). The problem is
> due to "other time" that is not in KSP solve. Note that the percentage of the total time in KSPSolve went from 15 percent of the runtime to 4 percent. This means something outside of KSPSolve is scaling very poorly. You will need to profile the rest of the code to determine where the time is being spent. PetscLogEventRegister() and PetscLogEventBegin/End() will be needed in your code. Already with 360 processes the linear solver is only taking 15 percent of the time.
>
> Barry
>
Hi,
I have attached the new logging results with the HYPRE Poisson eqn
solver. However, due to some problems, I am now using Intel 2018. Should
be quite similar to 2016 in terms of runtime. Using 360 processes can't
work this time, and I'm not sure why though.
>
>
>> On Mar 4, 2018, at 9:23 PM, TAY wee-beng <zonexo at gmail.com> wrote:
>>
>>
>> On 1/3/2018 12:14 PM, Smith, Barry F. wrote:
>>>> On Feb 28, 2018, at 8:01 PM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>
>>>>
>>>> On 1/3/2018 12:10 AM, Matthew Knepley wrote:
>>>>> On Wed, Feb 28, 2018 at 10:45 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I have a CFD code which uses PETSc and HYPRE. I found that for a certain case with grid size of 192,570,048, I encounter scaling problem when my cores > 600. At 600 cores, the code took 10min for 100 time steps. At 960, 1440 and 2880 cores, it still takes around 10min. At 360 cores, it took 15min.
>>>>>
>>>>> So how can I find the bottleneck? Any recommended steps?
>>>>>
>>>>> For any performance question, we need to see the output of -log_view for all test cases.
>>>> Hi,
>>>>
>>>> To be more specific, I use PETSc KSPBCGS and HYPRE geometric multigrid (entirely based on HYPRE, no PETSc) for the momentum and Poisson eqns in my code.
>>>>
>>>> So can log_view be used in this case to give a meaningful? Since part of the code uses HYPRE?
>>> Yes, just send the logs.
>>>
>> Hi,
>>
>> I have attached the logs, with the number indicating the no. of cores used. Some of the new results are different from the previous runs, although I'm using the same cluster.
>>
>> Thanks for the help.
>>>> I also program another subroutine in the past which uses PETSc to solve the Poisson eqn. It uses either HYPRE's boomeramg, KSPBCGS or KSPGMRES.
>>>>
>>>> If I use boomeramg, can log_view be used in this case?
>>>>
>>>> Or do I have to use KSPBCGS or KSPGMRES, which is directly from PETSc? However, I ran KSPGMRES yesterday with the Poisson eqn and my ans didn't converge.
>>>>
>>>> Thanks.
>>>>> I must also mention that I partition my grid only in the x and y direction. There is no partitioning in the z direction due to limited code development. I wonder if there is a strong effect in this case.
>>>>>
>>>>> Maybe. Usually what happens is you fill up memory with a z-column and cannot scale further.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Matt
>>>>>
>>>>> --
>>>>> Thank you very much
>>>>>
>>>>> Yours sincerely,
>>>>>
>>>>> ================================================
>>>>> TAY Wee-Beng 郑伟明 (Zheng Weiming)
>>>>> Personal research webpage: http://tayweebeng.wixsite.com/website
>>>>> Youtube research showcase: https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
>>>>> linkedin: www.linkedin.com/in/tay-weebeng
>>>>> ================================================
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>> https://www.cse.buffalo.edu/~knepley/
>> <log960.txt><log600.txt><log360.txt><log1920.txt>
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 4 1
min IIB_cell_no 0
max IIB_cell_no 210
IIB_cell_no_sum 88423
min equal_size 576
max equal_size 810
min I_cell_no 0
max I_cell_no 990
I_cell_no_sum 361178
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
210 990 210 990 88423 361178
IIB_equal_cell_no_u1_max 210
I_equal_cell_no_u1_max 990
IIB_I_cell_no_uvw_total1 88423 0 0 361178
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 210 7184
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 990 43594
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
size(IIB_cell_u),IIB_cell_no_max_cur 210 407
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 610
size(I_cell_u),I_cell_no_max_cur 990 2160
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 3240
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 2 88332 0 0
361141 0 0
IIB_equal_cell_no_u1_max 610
size(I_cell_u),I_cell_no_max_cur 3240 3456
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 3 87862 0 0
361081 0 0
size(IIB_cell_u),IIB_cell_no_max_cur 610 764
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 4 87184 0 0
361110 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 5 87458 0 0
361001 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 6 88029 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 7 88405 0 0
361129 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 8 88387 0 0
361154 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 9 88029 0 0
361130 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 361178 361221
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 10 87355 0 0
361221 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 11 87375 0 0
361183 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 12 87922 0 0
361095 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 13 88331 0 0
361102 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
size(IIB_global_cell_u1),IIB_global_cell_no_u1_max_cur 88423 88440
IIB global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 14 88440 0 0
361060 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 15 88169 0 0
361207 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 16 87545 0 0
361053 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 17 87244 0 0
360987 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 18 87775 0 0
361160 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 19 88292 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 20 88499 0 0
361268 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 21 88244 0 0
361100 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 22 87725 0 0
361209 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 23 87181 0 0
361181 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 24 87649 0 0
361235 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 25 88185 0 0
361107 0 0
IIB_equal_cell_no_u1_max 1146
I_equal_cell_no_u1_max 5184
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 10776 65391
time,IIB_I_cell_no_uvw_total1 26 88423 0 0
361178 0 0
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1432 13470 165825 6480 81738 677288
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 4 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
13470 81738 1432 6480 165825 677288
IIB_I_cell_no_uvw_total1 88423 88466 88458 361178
361141 363281
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-4.742031575054313E-002 1.587731906566282E-002 -3.34336552065208
-2.764952501552243E-002 -0.120467059664690 5.36049435517963
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.8.3_intel_rel named std0846 with 1920 processors, by tsltaywb Tue Mar 6 09:34:55 2018
Using Petsc Release Version 3.8.3, Dec, 09, 2017
Max Max/Min Avg Total
Time (sec): 5.911e+02 1.00004 5.910e+02
Objects: 1.391e+03 1.00000 1.391e+03
Flop: 1.326e+10 1.21860 1.170e+10 2.246e+13
Flop/sec: 2.243e+07 1.21862 1.979e+07 3.800e+10
MPI Messages: 4.792e+03 2.00000 4.515e+03 8.668e+06
MPI Message Lengths: 8.540e+08 2.18170 1.737e+05 1.506e+12
MPI Reductions: 2.610e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 5.9104e+02 100.0% 2.2458e+13 100.0% 8.668e+06 100.0% 1.737e+05 100.0% 2.609e+03 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 198 1.0 2.9097e+00 2.8 1.35e+08 1.2 0.0e+00 0.0e+00 2.0e+02 0 1 0 0 8 0 1 0 0 8 78624
VecDotNorm2 99 1.0 2.3226e+00 4.3 1.35e+08 1.2 0.0e+00 0.0e+00 9.9e+01 0 1 0 0 4 0 1 0 0 4 98500
VecNorm 198 1.0 2.0144e+00 2.0 1.35e+08 1.2 0.0e+00 0.0e+00 2.0e+02 0 1 0 0 8 0 1 0 0 8 113571
VecCopy 198 1.0 3.0518e-01 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 603 1.0 5.1531e-01 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 198 1.0 4.5395e-01 1.5 2.69e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 1007927
VecWAXPY 198 1.0 4.4375e-01 1.4 1.35e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 515543
VecAssemblyBegin 198 1.0 4.8822e+00 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 5.9e+02 1 0 0 0 23 1 0 0 0 23 0
VecAssemblyEnd 198 1.0 7.1812e-04 5.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 802 1.0 1.1067e+00 2.6 0.00e+00 0.0 6.0e+06 2.5e+05 0.0e+00 0 0 69100 0 0 0 69100 0 0
VecScatterEnd 802 1.0 3.5382e+00 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 198 1.0 8.2760e+00 1.4 5.18e+09 1.2 1.5e+06 4.3e+05 0.0e+00 1 39 17 42 0 1 39 17 42 0 1061375
MatSolve 297 1.0 1.0776e+01 1.3 7.16e+09 1.2 0.0e+00 0.0e+00 0.0e+00 2 54 0 0 0 2 54 0 0 0 1123019
MatLUFactorNum 1 1.0 2.2423e-01 1.4 1.06e+08 1.4 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 792077
MatILUFactorSym 1 1.0 4.3188e-01 5.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 3.1625e+00 2.3 1.32e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 7105
MatAssemblyBegin 2 1.0 2.8583e-01 4.9 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 3.1411e-01 1.0 0.00e+00 0.0 1.5e+04 1.1e+05 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 4.9114e-0551.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 1.4037e-02 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
Poisson log 100 1.0 3.0957e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 52 0 0 0 0 52 0 0 0 0 0
KSPSetUp 2 1.0 5.5745e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 99 1.0 2.3471e+01 1.0 1.32e+10 1.2 1.5e+06 4.3e+05 5.1e+02 4100 17 42 19 4100 17 42 19 955883
PCSetUp 2 1.0 6.3179e-01 2.6 1.06e+08 1.4 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 281120
PCSetUpOnBlocks 99 1.0 6.3180e-01 2.6 1.06e+08 1.4 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 281113
PCApply 297 1.0 1.0997e+01 1.3 7.16e+09 1.2 0.0e+00 0.0e+00 0.0e+00 2 54 0 0 0 2 54 0 0 0 1100411
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 296 296 48826760 0.
Vector Scatter 141 141 8996640 0.
Matrix 4 4 316139284 0.
Distributed Mesh 134 134 694656 0.
Index Set 275 275 11495424 0.
IS L to G Mapping 134 134 6046316 0.
Star Forest Graph 268 268 229408 0.
Discrete System 134 134 120600 0.
Krylov Solver 2 2 2344 0.
Preconditioner 2 2 1896 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
Average time for MPI_Barrier(): 0.000404978
Average time for zero size MPI_Send(): 3.64383e-06
#PETSc Option Table entries:
-log_view
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/source/git.hypre.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on Fri Mar 2 13:16:05 2018 on nus03
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/source/petsc-3.8.3
Using PETSc arch: petsc-3.8.3_intel_rel
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lifport -lifcoremt -lmpicxx -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lm -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-03-06 09:35:21.959579:
JobId: 6602353.wlm01
Project: 11000550
Exit Status: 0
NCPUs Requested: 1920 NCPUs Used: 1920
CPU Time Used: 03:54:15
Memory Requested: 7680gb Memory Used: 34382100kb
Vmem Used: 67385020kb
Walltime requested: 00:24:00 Walltime Used: 00:10:44
Execution Nodes Used: (std0846:ncpus=24:mem=100663296kb)+(std0848:ncpus=24:mem=100663296kb)+(std0849:ncpus=24:mem=100663296kb)+(std0850:ncpus=24:mem=100663296kb)+(std0852:ncpus=24:mem=100663296kb)+(std0854:ncpus=24:mem=100663296kb)+(std0859:ncpus=24:mem=100663296kb)+(std0860:ncpus=24:mem=100663296kb)+(std0861:ncpus=24:mem=100663296kb)+(std0866:ncpus=24:mem=100663296kb)+(std0867:ncpus=24:mem=100663296kb)+(std0871:ncpus=24:mem=100663296kb)+(std0937:ncpus=24:mem=100663296kb)+(std0938:ncpus=24:mem=100663296kb)+(std0939:ncpus=24:mem=100663296kb)+(std0940:ncpus=24:mem=100663296kb)+(std0941:ncpus=24:mem=100663296kb)+(std0942:ncpus=24:mem=100663296kb)+(std0943:ncpus=24:mem=100663296kb)+(std0944:ncpus=24:mem=100663296kb)+(std0945:ncpus=24:mem=100663296kb)+(std0946:ncpus=24:mem=100663296kb)+(std0947:ncpus=24:mem=100663296kb)+(std0948:ncpus=24:mem=100663296kb)+(std0949:ncpus=24:mem=100663296kb)+(std0950:ncpus=24:mem=100663296kb)+(std0951:ncpus=24:mem=100663296kb)+(std0952:ncpus=24:mem=100663296kb)+(std0953:ncpus=24:mem=100663296kb)+(std0954:ncpus=24:mem=100663296kb)+(std0955:ncpus=24:mem=100663296kb)+(std0956:ncpus=24:mem=100663296kb)+(std0957:ncpus=24:mem=100663296kb)+(std0958:ncpus=24:mem=100663296kb)+(std0959:ncpus=24:mem=100663296kb)+(std0960:ncpus=24:mem=100663296kb)+(std0961:ncpus=24:mem=100663296kb)+(std0962:ncpus=24:mem=100663296kb)+(std0963:ncpus=24:mem=100663296kb)+(std0964:ncpus=24:mem=100663296kb)+(std0965:ncpus=24:mem=100663296kb)+(std0966:ncpus=24:mem=100663296kb)+(std0967:ncpus=24:mem=100663296kb)+(std0968:ncpus=24:mem=100663296kb)+(std0969:ncpus=24:mem=100663296kb)+(std0970:ncpus=24:mem=100663296kb)+(std0971:ncpus=24:mem=100663296kb)+(std0972:ncpus=24:mem=100663296kb)+(std1001:ncpus=24:mem=100663296kb)+(std1002:ncpus=24:mem=100663296kb)+(std1004:ncpus=24:mem=100663296kb)+(std1005:ncpus=24:mem=100663296kb)+(std1006:ncpus=24:mem=100663296kb)+(std1017:ncpus=24:mem=100663296kb)+(std1018:ncpus=24:mem=100663296kb)+(std1019:ncpus=24:mem=100663296kb)+(std1031:ncpus=24:mem=100663296kb)+(std1033:ncpus=24:mem=100663296kb)+(std1034:ncpus=24:mem=100663296kb)+(std1035:ncpus=24:mem=100663296kb)+(std1036:ncpus=24:mem=100663296kb)+(std1037:ncpus=24:mem=100663296kb)+(std1038:ncpus=24:mem=100663296kb)+(std1039:ncpus=24:mem=100663296kb)+(std1040:ncpus=24:mem=100663296kb)+(std1043:ncpus=24:mem=100663296kb)+(std1044:ncpus=24:mem=100663296kb)+(std1045:ncpus=24:mem=100663296kb)+(std1046:ncpus=24:mem=100663296kb)+(std1047:ncpus=24:mem=100663296kb)+(std1048:ncpus=24:mem=100663296kb)+(std1049:ncpus=24:mem=100663296kb)+(std1050:ncpus=24:mem=100663296kb)+(std1051:ncpus=24:mem=100663296kb)+(std1052:ncpus=24:mem=100663296kb)+(std1053:ncpus=24:mem=100663296kb)+(std1054:ncpus=24:mem=100663296kb)+(std1056:ncpus=24:mem=100663296kb)+(std1057:ncpus=24:mem=100663296kb)+(std1058:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
min IIB_cell_no 0
max IIB_cell_no 569
IIB_cell_no_sum 88423
min equal_size 2310
max equal_size 2880
min I_cell_no 0
max I_cell_no 2880
I_cell_no_sum 361178
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
569 2880 569 2880 88423 361178
IIB_equal_cell_no_u1_max 569
I_equal_cell_no_u1_max 2880
IIB_I_cell_no_uvw_total1 88423 0 0 361178
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 569 21090
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 2880 124835
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
size(IIB_cell_u),IIB_cell_no_max_cur 569 1056
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1584
size(I_cell_u),I_cell_no_max_cur 2880 6600
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 9900
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 2 88332 0 0
361141 0 0
IIB_equal_cell_no_u1_max 1584
size(I_cell_u),I_cell_no_max_cur 9900 10163
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 3 87862 0 0
361081 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 4 87184 0 0
361110 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 5 87458 0 0
361001 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 6 88029 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 7 88405 0 0
361129 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 8 88387 0 0
361154 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 9 88029 0 0
361130 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 361178 361221
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 10 87355 0 0
361221 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 11 87375 0 0
361183 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 12 87922 0 0
361095 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 13 88331 0 0
361102 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
size(IIB_global_cell_u1),IIB_global_cell_no_u1_max_cur 88423 88440
IIB global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 14 88440 0 0
361060 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 15 88169 0 0
361207 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 16 87545 0 0
361053 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 17 87244 0 0
360987 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 18 87775 0 0
361160 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 19 88292 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 20 88499 0 0
361268 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 21 88244 0 0
361100 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 22 87725 0 0
361209 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 23 87181 0 0
361181 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 24 87649 0 0
361235 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 25 88185 0 0
361107 0 0
IIB_equal_cell_no_u1_max 1584
I_equal_cell_no_u1_max 15244
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 31635 187252
time,IIB_I_cell_no_uvw_total1 26 88423 0 0
361178 0 0
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1980 39543 165825 19055 234065 677288
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
39543 234065 1980 19055 165825 677288
IIB_I_cell_no_uvw_total1 88423 88466 88458 361178
361141 363281
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-4.729629430920234E-002 1.454279159645171E-002 -3.34069590849895
-2.716548489534213E-002 -0.130629748378504 5.33633167855167
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.8.3_intel_rel named std1101 with 600 processors, by tsltaywb Tue Mar 6 09:24:19 2018
Using Petsc Release Version 3.8.3, Dec, 09, 2017
Max Max/Min Avg Total
Time (sec): 9.310e+02 1.00001 9.309e+02
Objects: 1.391e+03 1.00000 1.391e+03
Flop: 4.082e+10 1.12066 3.822e+10 2.293e+13
Flop/sec: 4.385e+07 1.12067 4.106e+07 2.464e+10
MPI Messages: 4.756e+03 1.98497 4.387e+03 2.632e+06
MPI Message Lengths: 1.530e+09 2.09737 3.238e+05 8.523e+11
MPI Reductions: 2.610e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 9.3095e+02 100.0% 2.2934e+13 100.0% 2.632e+06 100.0% 3.238e+05 100.0% 2.609e+03 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 198 1.0 2.7570e+0196.9 4.06e+08 1.1 0.0e+00 0.0e+00 2.0e+02 3 1 0 0 8 3 1 0 0 8 8298
VecDotNorm2 99 1.0 2.4118e+01151.2 4.06e+08 1.1 0.0e+00 0.0e+00 9.9e+01 2 1 0 0 4 2 1 0 0 4 9486
VecNorm 198 1.0 2.3064e+0188.3 4.06e+08 1.1 0.0e+00 0.0e+00 2.0e+02 2 1 0 0 8 2 1 0 0 8 9919
VecCopy 198 1.0 2.1885e+00 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 603 1.0 3.6030e+00 3.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 198 1.0 3.6540e+00 3.3 8.13e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 125217
VecWAXPY 198 1.0 2.7149e+00 2.4 4.06e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 84267
VecAssemblyBegin 198 1.0 5.8377e+00 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 5.9e+02 0 0 0 0 23 0 0 0 0 23 0
VecAssemblyEnd 198 1.0 3.5024e-04 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 802 1.0 2.6314e+00 2.0 0.00e+00 0.0 1.8e+06 4.6e+05 0.0e+00 0 0 70100 0 0 0 70100 0 0
VecScatterEnd 802 1.0 1.3025e+01 6.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 198 1.0 2.8936e+01 1.5 1.56e+10 1.1 4.6e+05 7.9e+05 0.0e+00 2 38 17 42 0 2 38 17 42 0 303563
MatSolve 297 1.0 9.1478e+01 3.3 2.24e+10 1.1 0.0e+00 0.0e+00 0.0e+00 3 55 0 0 0 3 55 0 0 0 137383
MatLUFactorNum 1 1.0 1.0244e+00 1.8 3.36e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 183350
MatILUFactorSym 1 1.0 1.5719e+00 5.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 1.0685e+00 2.5 4.00e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 21030
MatAssemblyBegin 2 1.0 3.5643e-0146.8 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 2.8034e-01 1.1 0.00e+00 0.0 4.6e+03 2.0e+05 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 1.9073e-0516.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 1.0776e-01 7.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
Poisson log 100 1.0 5.0848e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 55 0 0 0 0 55 0 0 0 0 0
KSPSetUp 2 1.0 9.1731e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 99 1.0 1.2714e+02 1.0 4.08e+10 1.1 4.6e+05 7.9e+05 5.1e+02 14100 17 42 19 14100 17 42 19 180210
PCSetUp 2 1.0 2.6086e+00 3.0 3.36e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 72001
PCSetUpOnBlocks 99 1.0 2.6086e+00 3.0 3.36e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 72000
PCApply 297 1.0 9.2951e+01 3.2 2.24e+10 1.1 0.0e+00 0.0e+00 0.0e+00 3 55 0 0 0 3 55 0 0 0 135206
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 296 296 140331272 0.
Vector Scatter 141 141 26977688 0.
Matrix 4 4 977176612 0.
Distributed Mesh 134 134 694656 0.
Index Set 275 275 32115544 0.
IS L to G Mapping 134 134 16242752 0.
Star Forest Graph 268 268 229408 0.
Discrete System 134 134 120600 0.
Krylov Solver 2 2 2344 0.
Preconditioner 2 2 1896 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 2.19345e-06
Average time for MPI_Barrier(): 0.000237417
Average time for zero size MPI_Send(): 1.66655e-06
#PETSc Option Table entries:
-log_view
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/source/git.hypre.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on Fri Mar 2 13:16:05 2018 on nus03
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/source/petsc-3.8.3
Using PETSc arch: petsc-3.8.3_intel_rel
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lifport -lifcoremt -lmpicxx -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lm -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-03-06 09:24:27.561873:
JobId: 6602351.wlm01
Project: 11000324
Exit Status: 0
NCPUs Requested: 600 NCPUs Used: 600
CPU Time Used: 06:10:22
Memory Requested: 2400gb Memory Used: 59532236kb
Vmem Used: 88920940kb
Walltime requested: 00:29:00 Walltime Used: 00:15:55
Execution Nodes Used: (std1101:ncpus=24:mem=100663296kb)+(std1125:ncpus=24:mem=100663296kb)+(std1131:ncpus=24:mem=100663296kb)+(std1132:ncpus=24:mem=100663296kb)+(std1133:ncpus=24:mem=100663296kb)+(std1134:ncpus=24:mem=100663296kb)+(std1135:ncpus=24:mem=100663296kb)+(std1136:ncpus=24:mem=100663296kb)+(std1137:ncpus=24:mem=100663296kb)+(std1138:ncpus=24:mem=100663296kb)+(std1139:ncpus=24:mem=100663296kb)+(std1140:ncpus=24:mem=100663296kb)+(std1141:ncpus=24:mem=100663296kb)+(std1142:ncpus=24:mem=100663296kb)+(std1143:ncpus=24:mem=100663296kb)+(std1144:ncpus=24:mem=100663296kb)+(std1145:ncpus=24:mem=100663296kb)+(std1146:ncpus=24:mem=100663296kb)+(std1147:ncpus=24:mem=100663296kb)+(std1148:ncpus=24:mem=100663296kb)+(std1149:ncpus=24:mem=100663296kb)+(std1150:ncpus=24:mem=100663296kb)+(std1151:ncpus=24:mem=100663296kb)+(std1152:ncpus=24:mem=100663296kb)+(std1153:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
min IIB_cell_no 0
max IIB_cell_no 403
IIB_cell_no_sum 88423
min equal_size 1320
max equal_size 1728
min I_cell_no 0
max I_cell_no 1872
I_cell_no_sum 361178
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
403 1872 403 1872 88423 361178
IIB_equal_cell_no_u1_max 403
I_equal_cell_no_u1_max 1872
IIB_I_cell_no_uvw_total1 88423 0 0 361178
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 403 14024
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 1872 87639
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
size(IIB_cell_u),IIB_cell_no_max_cur 403 711
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1066
size(I_cell_u),I_cell_no_max_cur 1872 4320
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 6480
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 2 88332 0 0
361141 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 6480
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 3 87862 0 0
361081 0 0
IIB_equal_cell_no_u1_max 1066
size(I_cell_u),I_cell_no_max_cur 6480 7854
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 4 87184 0 0
361110 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 5 87458 0 0
361001 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 6 88029 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 7 88405 0 0
361129 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 8 88387 0 0
361154 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 9 88029 0 0
361130 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 361178 361221
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 10 87355 0 0
361221 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 11 87375 0 0
361183 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 12 87922 0 0
361095 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 13 88331 0 0
361102 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
size(IIB_global_cell_u1),IIB_global_cell_no_u1_max_cur 88423 88440
IIB global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 14 88440 0 0
361060 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 15 88169 0 0
361207 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 16 87545 0 0
361053 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 17 87244 0 0
360987 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 18 87775 0 0
361160 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 19 88292 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 20 88499 0 0
361268 0 0
IIB_equal_cell_no_u1_max 1066
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 21 88244 0 0
361100 0 0
size(IIB_cell_u),IIB_cell_no_max_cur 1066 1204
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1806
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 22 87725 0 0
361209 0 0
IIB_equal_cell_no_u1_max 1806
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 23 87181 0 0
361181 0 0
IIB_equal_cell_no_u1_max 1806
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 24 87649 0 0
361235 0 0
IIB_equal_cell_no_u1_max 1806
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 25 88185 0 0
361107 0 0
IIB_equal_cell_no_u1_max 1806
I_equal_cell_no_u1_max 11781
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 21036 131458
time,IIB_I_cell_no_uvw_total1 26 88423 0 0
361178 0 0
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
2257 26295 165825 14726 164322 677288
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
26295 164322 2257 14726 165825 677288
IIB_I_cell_no_uvw_total1 88423 88466 88458 361178
361141 363281
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-4.766441998809254E-002 1.069513941874686E-002 -3.33756148995123
-2.485778915433470E-002 -0.135236088926291 5.33673399226584
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.8.3_intel_rel named std1101 with 960 processors, by tsltaywb Tue Mar 6 09:08:09 2018
Using Petsc Release Version 3.8.3, Dec, 09, 2017
Max Max/Min Avg Total
Time (sec): 5.890e+02 1.00002 5.890e+02
Objects: 1.391e+03 1.00000 1.391e+03
Flop: 2.555e+10 1.15005 2.374e+10 2.279e+13
Flop/sec: 4.338e+07 1.15007 4.031e+07 3.869e+10
MPI Messages: 4.768e+03 1.98998 4.445e+03 4.268e+06
MPI Message Lengths: 1.174e+09 2.12895 2.458e+05 1.049e+12
MPI Reductions: 2.610e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 5.8901e+02 100.0% 2.2791e+13 100.0% 4.268e+06 100.0% 2.458e+05 100.0% 2.609e+03 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 198 1.0 1.1501e+0123.8 2.56e+08 1.1 0.0e+00 0.0e+00 2.0e+02 2 1 0 0 8 2 1 0 0 8 19892
VecDotNorm2 99 1.0 1.0528e+0185.4 2.56e+08 1.1 0.0e+00 0.0e+00 9.9e+01 2 1 0 0 4 2 1 0 0 4 21731
VecNorm 198 1.0 1.0333e+0129.6 2.56e+08 1.1 0.0e+00 0.0e+00 2.0e+02 2 1 0 0 8 2 1 0 0 8 22140
VecCopy 198 1.0 8.1306e-01 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 603 1.0 1.5303e+00 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 198 1.0 1.3017e+00 1.9 5.12e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 351501
VecWAXPY 198 1.0 1.1356e+00 1.7 2.56e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 201458
VecAssemblyBegin 198 1.0 5.4884e+00 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 5.9e+02 1 0 0 0 23 1 0 0 0 23 0
VecAssemblyEnd 198 1.0 3.5405e-04 2.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 802 1.0 1.8624e+00 2.3 0.00e+00 0.0 3.0e+06 3.5e+05 0.0e+00 0 0 70100 0 0 0 70100 0 0
VecScatterEnd 802 1.0 4.5455e+00 2.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 198 1.0 1.6530e+01 1.4 9.85e+09 1.2 7.3e+05 6.1e+05 0.0e+00 2 39 17 42 0 2 39 17 42 0 531382
MatSolve 297 1.0 4.3417e+01 2.5 1.39e+10 1.1 0.0e+00 0.0e+00 0.0e+00 3 55 0 0 0 3 55 0 0 0 286224
MatLUFactorNum 1 1.0 8.2650e-01 2.4 2.08e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 223518
MatILUFactorSym 1 1.0 9.4145e-01 5.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 2.2974e+00 1.2 2.52e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 9781
MatAssemblyBegin 2 1.0 3.2012e-01 5.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 4.1547e-01 1.0 0.00e+00 0.0 7.4e+03 1.5e+05 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 2.5034e-0526.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 1.9657e-02 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
Poisson log 100 1.0 2.7102e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 46 0 0 0 0 46 0 0 0 0 0
KSPSetUp 2 1.0 4.9580e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 99 1.0 6.4399e+01 1.0 2.55e+10 1.2 7.3e+05 6.1e+05 5.1e+02 11100 17 42 19 11100 17 42 19 353554
PCSetUp 2 1.0 1.5659e+00 3.0 2.08e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 117977
PCSetUpOnBlocks 99 1.0 1.5659e+00 3.0 2.08e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 117978
PCApply 297 1.0 4.4102e+01 2.5 1.39e+10 1.1 0.0e+00 0.0e+00 0.0e+00 3 55 0 0 0 3 55 0 0 0 281781
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 296 296 89769992 0.
Vector Scatter 141 141 17112320 0.
Matrix 4 4 610814692 0.
Distributed Mesh 134 134 694656 0.
Index Set 275 275 20797924 0.
IS L to G Mapping 134 134 10687184 0.
Star Forest Graph 268 268 229408 0.
Discrete System 134 134 120600 0.
Krylov Solver 2 2 2344 0.
Preconditioner 2 2 1896 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 1.19209e-07
Average time for MPI_Barrier(): 5.77927e-05
Average time for zero size MPI_Send(): 1.64484e-06
#PETSc Option Table entries:
-log_view
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/source/git.hypre.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on Fri Mar 2 13:16:05 2018 on nus03
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/source/petsc-3.8.3
Using PETSc arch: petsc-3.8.3_intel_rel
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lifport -lifcoremt -lmpicxx -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lm -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-03-06 09:08:25.025242:
JobId: 6602347.wlm01
Project: 11000550
Exit Status: 0
NCPUs Requested: 960 NCPUs Used: 960
CPU Time Used: 03:53:58
Memory Requested: 3840gb Memory Used: 46271888kb
Vmem Used: 76505608kb
Walltime requested: 00:29:00 Walltime Used: 00:10:28
Execution Nodes Used: (std1101:ncpus=24:mem=100663296kb)+(std1125:ncpus=24:mem=100663296kb)+(std1131:ncpus=24:mem=100663296kb)+(std1132:ncpus=24:mem=100663296kb)+(std1133:ncpus=24:mem=100663296kb)+(std1134:ncpus=24:mem=100663296kb)+(std1135:ncpus=24:mem=100663296kb)+(std1136:ncpus=24:mem=100663296kb)+(std1137:ncpus=24:mem=100663296kb)+(std1138:ncpus=24:mem=100663296kb)+(std1139:ncpus=24:mem=100663296kb)+(std1140:ncpus=24:mem=100663296kb)+(std1141:ncpus=24:mem=100663296kb)+(std1142:ncpus=24:mem=100663296kb)+(std1143:ncpus=24:mem=100663296kb)+(std1144:ncpus=24:mem=100663296kb)+(std1145:ncpus=24:mem=100663296kb)+(std1146:ncpus=24:mem=100663296kb)+(std1147:ncpus=24:mem=100663296kb)+(std1148:ncpus=24:mem=100663296kb)+(std1149:ncpus=24:mem=100663296kb)+(std1150:ncpus=24:mem=100663296kb)+(std1151:ncpus=24:mem=100663296kb)+(std1152:ncpus=24:mem=100663296kb)+(std1153:ncpus=24:mem=100663296kb)+(std1154:ncpus=24:mem=100663296kb)+(std1155:ncpus=24:mem=100663296kb)+(std1156:ncpus=24:mem=100663296kb)+(std1157:ncpus=24:mem=100663296kb)+(std1158:ncpus=24:mem=100663296kb)+(std1159:ncpus=24:mem=100663296kb)+(std1160:ncpus=24:mem=100663296kb)+(std1161:ncpus=24:mem=100663296kb)+(std1162:ncpus=24:mem=100663296kb)+(std1163:ncpus=24:mem=100663296kb)+(std1164:ncpus=24:mem=100663296kb)+(std1165:ncpus=24:mem=100663296kb)+(std1167:ncpus=24:mem=100663296kb)+(std1168:ncpus=24:mem=100663296kb)+(std1169:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
min IIB_cell_no 0
max IIB_cell_no 275
IIB_cell_no_sum 88423
min equal_size 792
max equal_size 1080
min I_cell_no 0
max I_cell_no 1287
I_cell_no_sum 361178
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
275 1287 275 1287 88423 361178
IIB_equal_cell_no_u1_max 275
I_equal_cell_no_u1_max 1287
IIB_I_cell_no_uvw_total1 88423 0 0 361178
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 275 9222
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 1287 56591
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
size(IIB_cell_u),IIB_cell_no_max_cur 275 499
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 748
size(I_cell_u),I_cell_no_max_cur 1287 2880
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 4320
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 2 88332 0 0
361141 0 0
size(IIB_cell_u),IIB_cell_no_max_cur 748 865
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1297
size(I_cell_u),I_cell_no_max_cur 4320 4437
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 3 87862 0 0
361081 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 4 87184 0 0
361110 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 5 87458 0 0
361001 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 6 88029 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 7 88405 0 0
361129 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 8 88387 0 0
361154 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 361178
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 9 88029 0 0
361130 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 361178 361221
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 10 87355 0 0
361221 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 11 87375 0 0
361183 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 12 87922 0 0
361095 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 88423 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 13 88331 0 0
361102 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
size(IIB_global_cell_u1),IIB_global_cell_no_u1_max_cur 88423 88440
IIB global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 14 88440 0 0
361060 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 15 88169 0 0
361207 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 16 87545 0 0
361053 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 17 87244 0 0
360987 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 18 87775 0 0
361160 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 19 88292 0 0
361094 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 20 88499 0 0
361268 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 21 88244 0 0
361100 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 22 87725 0 0
361209 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 23 87181 0 0
361181 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 24 87649 0 0
361235 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 25 88185 0 0
361107 0 0
IIB_equal_cell_no_u1_max 1297
I_equal_cell_no_u1_max 6655
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 132660 541831
IIB_cell_no_u1_max,I_cell_no_u1_max 13833 84886
time,IIB_I_cell_no_uvw_total1 26 88423 0 0
361178 0 0
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1621 17291 165825 8318 106107 677288
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.41000006697141 3.73100008221809
3.32950006314786 3.32950006314786
size_x,size_y,size_z 792x792x307
total grid size = 192570048
recommended cores (50k / core) = 3851.40096000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 3.240000000000000E-004 335.581389622375
ratio bet max_area,min_grid_area not ideal
max element length should be 2.545584412271571E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 1.800000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 1
minimum ngh_surfaces and ngh_vertics are 1 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
17291 106107 1621 8318 165825 677288
IIB_I_cell_no_uvw_total1 88423 88466 88458 361178
361141 363281
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-4.787073078417698E-002 7.245861259577491E-003 -3.34443765634754
-2.324893818188146E-002 -0.158252457962344 5.33003976092268
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a petsc-3.8.3_intel_rel named std0846 with 1440 processors, by tsltaywb Tue Mar 6 09:44:58 2018
Using Petsc Release Version 3.8.3, Dec, 09, 2017
Max Max/Min Avg Total
Time (sec): 5.489e+02 1.00002 5.489e+02
Objects: 1.391e+03 1.00000 1.391e+03
Flop: 1.697e+10 1.18897 1.570e+10 2.261e+13
Flop/sec: 3.091e+07 1.18898 2.861e+07 4.119e+10
MPI Messages: 4.792e+03 2.00000 4.483e+03 6.455e+06
MPI Message Lengths: 9.609e+08 2.16000 2.003e+05 1.293e+12
MPI Reductions: 2.610e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 5.4894e+02 100.0% 2.2613e+13 100.0% 6.455e+06 100.0% 2.003e+05 100.0% 2.609e+03 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
VecDot 198 1.0 3.3593e+00 2.6 1.71e+08 1.2 0.0e+00 0.0e+00 2.0e+02 0 1 0 0 8 0 1 0 0 8 68100
VecDotNorm2 99 1.0 2.9070e+00 3.2 1.71e+08 1.2 0.0e+00 0.0e+00 9.9e+01 0 1 0 0 4 0 1 0 0 4 78697
VecNorm 198 1.0 2.0735e+00 2.1 1.71e+08 1.2 0.0e+00 0.0e+00 2.0e+02 0 1 0 0 8 0 1 0 0 8 110334
VecCopy 198 1.0 3.4745e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 603 1.0 6.1493e-01 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPBYCZ 198 1.0 5.4132e-01 1.3 3.42e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 845249
VecWAXPY 198 1.0 5.5862e-01 1.3 1.71e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 409532
VecAssemblyBegin 198 1.0 4.6988e+00 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 5.9e+02 1 0 0 0 23 1 0 0 0 23 0
VecAssemblyEnd 198 1.0 1.2007e-03 8.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecScatterBegin 802 1.0 1.2880e+00 2.4 0.00e+00 0.0 4.5e+06 2.9e+05 0.0e+00 0 0 69100 0 0 0 69100 0 0
VecScatterEnd 802 1.0 3.3421e+00 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 198 1.0 1.0392e+01 1.3 6.59e+09 1.2 1.1e+06 4.9e+05 0.0e+00 2 39 17 42 0 2 39 17 42 0 845301
MatSolve 297 1.0 1.3647e+01 1.2 9.20e+09 1.2 0.0e+00 0.0e+00 0.0e+00 2 54 0 0 0 2 54 0 0 0 897823
MatLUFactorNum 1 1.0 3.2415e-01 1.5 1.37e+08 1.3 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 558142
MatILUFactorSym 1 1.0 1.5985e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 1 1.0 1.1641e+00 5.3 1.69e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 19303
MatAssemblyBegin 2 1.0 2.4304e-0117.7 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 3.1713e-01 1.0 0.00e+00 0.0 1.1e+04 1.2e+05 8.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 1 1.0 1.9073e-0510.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetOrdering 1 1.0 1.7568e-02 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
Poisson log 100 1.0 2.7163e+02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 49 0 0 0 0 49 0 0 0 0 0
KSPSetUp 2 1.0 5.4460e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 0 0 0 0 0 0 0
KSPSolve 99 1.0 2.9525e+01 1.0 1.70e+10 1.2 1.1e+06 4.9e+05 5.1e+02 5100 17 42 19 5100 17 42 19 765140
PCSetUp 2 1.0 4.9219e-01 1.5 1.37e+08 1.3 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 367585
PCSetUpOnBlocks 99 1.0 4.9221e-01 1.5 1.37e+08 1.3 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 367573
PCApply 297 1.0 1.3938e+01 1.2 9.20e+09 1.2 0.0e+00 0.0e+00 0.0e+00 2 54 0 0 0 2 54 0 0 0 879104
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 296 296 61257992 0.
Vector Scatter 141 141 11488912 0.
Matrix 4 4 405017668 0.
Distributed Mesh 134 134 694656 0.
Index Set 275 275 14353080 0.
IS L to G Mapping 134 134 7489740 0.
Star Forest Graph 268 268 229408 0.
Discrete System 134 134 120600 0.
Krylov Solver 2 2 2344 0.
Preconditioner 2 2 1896 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 0.
Average time for MPI_Barrier(): 0.000998592
Average time for zero size MPI_Send(): 1.81728e-06
#PETSc Option Table entries:
-log_view
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/source/git.hypre.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on Fri Mar 2 13:16:05 2018 on nus03
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/source/petsc-3.8.3
Using PETSc arch: petsc-3.8.3_intel_rel
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2 ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2 ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/include -I/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/include -I/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/source/petsc-3.8.3/petsc-3.8.3_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lifport -lifcoremt -lmpicxx -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lm -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-03-06 09:45:18.640874:
JobId: 6602354.wlm01
Project: 11000550
Exit Status: 0
NCPUs Requested: 1440 NCPUs Used: 1440
CPU Time Used: 03:37:50
Memory Requested: 5760gb Memory Used: 37001920kb
Vmem Used: 68591332kb
Walltime requested: 00:29:00 Walltime Used: 00:09:48
Execution Nodes Used: (std0846:ncpus=24:mem=100663296kb)+(std0848:ncpus=24:mem=100663296kb)+(std0849:ncpus=24:mem=100663296kb)+(std0850:ncpus=24:mem=100663296kb)+(std0852:ncpus=24:mem=100663296kb)+(std0854:ncpus=24:mem=100663296kb)+(std0859:ncpus=24:mem=100663296kb)+(std0860:ncpus=24:mem=100663296kb)+(std0861:ncpus=24:mem=100663296kb)+(std0866:ncpus=24:mem=100663296kb)+(std0867:ncpus=24:mem=100663296kb)+(std0871:ncpus=24:mem=100663296kb)+(std0937:ncpus=24:mem=100663296kb)+(std0938:ncpus=24:mem=100663296kb)+(std0939:ncpus=24:mem=100663296kb)+(std0940:ncpus=24:mem=100663296kb)+(std0941:ncpus=24:mem=100663296kb)+(std0942:ncpus=24:mem=100663296kb)+(std0943:ncpus=24:mem=100663296kb)+(std0944:ncpus=24:mem=100663296kb)+(std0945:ncpus=24:mem=100663296kb)+(std0946:ncpus=24:mem=100663296kb)+(std0947:ncpus=24:mem=100663296kb)+(std0948:ncpus=24:mem=100663296kb)+(std0949:ncpus=24:mem=100663296kb)+(std0950:ncpus=24:mem=100663296kb)+(std0951:ncpus=24:mem=100663296kb)+(std0952:ncpus=24:mem=100663296kb)+(std0953:ncpus=24:mem=100663296kb)+(std0954:ncpus=24:mem=100663296kb)+(std0955:ncpus=24:mem=100663296kb)+(std0956:ncpus=24:mem=100663296kb)+(std0957:ncpus=24:mem=100663296kb)+(std0958:ncpus=24:mem=100663296kb)+(std0959:ncpus=24:mem=100663296kb)+(std0960:ncpus=24:mem=100663296kb)+(std0961:ncpus=24:mem=100663296kb)+(std0962:ncpus=24:mem=100663296kb)+(std0963:ncpus=24:mem=100663296kb)+(std0964:ncpus=24:mem=100663296kb)+(std0965:ncpus=24:mem=100663296kb)+(std0966:ncpus=24:mem=100663296kb)+(std0967:ncpus=24:mem=100663296kb)+(std0968:ncpus=24:mem=100663296kb)+(std0969:ncpus=24:mem=100663296kb)+(std0970:ncpus=24:mem=100663296kb)+(std0971:ncpus=24:mem=100663296kb)+(std0972:ncpus=24:mem=100663296kb)+(std1001:ncpus=24:mem=100663296kb)+(std1002:ncpus=24:mem=100663296kb)+(std1004:ncpus=24:mem=100663296kb)+(std1005:ncpus=24:mem=100663296kb)+(std1006:ncpus=24:mem=100663296kb)+(std1017:ncpus=24:mem=100663296kb)+(std1018:ncpus=24:mem=100663296kb)+(std1019:ncpus=24:mem=100663296kb)+(std1031:ncpus=24:mem=100663296kb)+(std1033:ncpus=24:mem=100663296kb)+(std1034:ncpus=24:mem=100663296kb)+(std1035:ncpus=24:mem=100663296kb)
======================================================================================
More information about the petsc-users
mailing list