[petsc-users] Scaling problem when cores > 600
TAY wee-beng
zonexo at gmail.com
Sat Apr 21 10:34:22 CDT 2018
Hi,
I have found some time to work on this scaling problem again. I am now
using:
mpirun ./a.out -log_view -poisson_pc_type gamg
-poisson_pc_gamg_agg_nsmooths 1
I have attached the log_view output for 288, 600, 960, 1440 procs for
comparison.
Please give some comments.
Thank you very much
Yours sincerely,
================================================
TAY Wee-Beng 郑伟明 (Zheng Weiming)
Personal research webpage: http://tayweebeng.wixsite.com/website
Youtube research showcase: https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
linkedin: www.linkedin.com/in/tay-weebeng
================================================
On 7/3/2018 11:58 PM, Smith, Barry F. wrote:
> What are you using for Poisson log.
>
> If it is a Poisson problem then almost for sure you should be using Hypre BoomerAMG?.
>
> It sounds like your matrix does not change. You will need to discuss the scaling with the hypre people.
>
> Barry
>
>
>> On Mar 7, 2018, at 5:38 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>
>>
>> On 7/3/2018 6:22 AM, Smith, Barry F. wrote:
>>> The speed up for "Poisson log" is 1.6425364214878704 = 5.0848e+02/3.0957e+02
>>>
>>> This is lower than I would expect for Hypre BoomerAMG?
>>>
>>> Are you doing multiple solves with the same matrix with hypre or is each solve a new matrix? If each solve is a new matrix then you may be getting expected behavior since the multigrid AMG construction process does not scale as well as the application of AMG once it is constructed.
>>>
>>> I am forwarding to the hypre team since this is their expertise not ours
>>>
>>> Barry
>>>
>> Hi,
>>
>> My LHS of the eqn does not change. Only the RHS changes at each time step. So should this be expected?
>>
>> So maybe I should change to BoomerAMG and compare?
>>
>> Will PETSc GAMG give better performance?
>>
>> Also, I must add that I only partition in the x and y direction. Will this be a factor?
>>
>> Thanks.
>>
>>>> On Mar 5, 2018, at 11:19 PM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>
>>>>
>>>> On 5/3/2018 11:43 AM, Smith, Barry F. wrote:
>>>>> 360 process
>>>>>
>>>>> KSPSolve 99 1.0 2.6403e+02 1.0 6.67e+10 1.1 2.7e+05 9.9e+05 5.1e+02 15100 17 42 19 15100 17 42 19 87401
>>>>>
>>>>> 1920 processes
>>>>>
>>>>> KSPSolve 99 1.0 2.3184e+01 1.0 1.32e+10 1.2 1.5e+06 4.3e+05 5.1e+02 4100 17 42 19 4100 17 42 19 967717
>>>>>
>>>>>
>>>>> Ratio of number of processes 5.33 ratio of time for KSPSolve 11.388 so the time for the solve is scaling very well (extremely well actually). The problem is
>>>>> due to "other time" that is not in KSP solve. Note that the percentage of the total time in KSPSolve went from 15 percent of the runtime to 4 percent. This means something outside of KSPSolve is scaling very poorly. You will need to profile the rest of the code to determine where the time is being spent. PetscLogEventRegister() and PetscLogEventBegin/End() will be needed in your code. Already with 360 processes the linear solver is only taking 15 percent of the time.
>>>>>
>>>>> Barry
>>>>>
>>>> Hi,
>>>>
>>>> I have attached the new logging results with the HYPRE Poisson eqn solver. However, due to some problems, I am now using Intel 2018. Should be quite similar to 2016 in terms of runtime. Using 360 processes can't work this time, and I'm not sure why though.
>>>>>> On Mar 4, 2018, at 9:23 PM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 1/3/2018 12:14 PM, Smith, Barry F. wrote:
>>>>>>>> On Feb 28, 2018, at 8:01 PM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1/3/2018 12:10 AM, Matthew Knepley wrote:
>>>>>>>>> On Wed, Feb 28, 2018 at 10:45 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I have a CFD code which uses PETSc and HYPRE. I found that for a certain case with grid size of 192,570,048, I encounter scaling problem when my cores > 600. At 600 cores, the code took 10min for 100 time steps. At 960, 1440 and 2880 cores, it still takes around 10min. At 360 cores, it took 15min.
>>>>>>>>>
>>>>>>>>> So how can I find the bottleneck? Any recommended steps?
>>>>>>>>>
>>>>>>>>> For any performance question, we need to see the output of -log_view for all test cases.
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> To be more specific, I use PETSc KSPBCGS and HYPRE geometric multigrid (entirely based on HYPRE, no PETSc) for the momentum and Poisson eqns in my code.
>>>>>>>>
>>>>>>>> So can log_view be used in this case to give a meaningful? Since part of the code uses HYPRE?
>>>>>>> Yes, just send the logs.
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have attached the logs, with the number indicating the no. of cores used. Some of the new results are different from the previous runs, although I'm using the same cluster.
>>>>>>
>>>>>> Thanks for the help.
>>>>>>>> I also program another subroutine in the past which uses PETSc to solve the Poisson eqn. It uses either HYPRE's boomeramg, KSPBCGS or KSPGMRES.
>>>>>>>>
>>>>>>>> If I use boomeramg, can log_view be used in this case?
>>>>>>>>
>>>>>>>> Or do I have to use KSPBCGS or KSPGMRES, which is directly from PETSc? However, I ran KSPGMRES yesterday with the Poisson eqn and my ans didn't converge.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>> I must also mention that I partition my grid only in the x and y direction. There is no partitioning in the z direction due to limited code development. I wonder if there is a strong effect in this case.
>>>>>>>>>
>>>>>>>>> Maybe. Usually what happens is you fill up memory with a z-column and cannot scale further.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Matt
>>>>>>>>> --
>>>>>>>>> Thank you very much
>>>>>>>>>
>>>>>>>>> Yours sincerely,
>>>>>>>>>
>>>>>>>>> ================================================
>>>>>>>>> TAY Wee-Beng 郑伟明 (Zheng Weiming)
>>>>>>>>> Personal research webpage: http://tayweebeng.wixsite.com/website
>>>>>>>>> Youtube research showcase: https://www.youtube.com/channel/UC72ZHtvQNMpNs2uRTSToiLA
>>>>>>>>> linkedin: www.linkedin.com/in/tay-weebeng
>>>>>>>>> ================================================
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>>>>>>> -- Norbert Wiener
>>>>>>>>>
>>>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>> <log960.txt><log600.txt><log360.txt><log1920.txt>
>>>> <log1920_2.txt><log600_2.txt><log960_2.txt><log1440_2.txt>
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 1
min IIB_cell_no 0
max IIB_cell_no 435
IIB_cell_no_sum 71637
min equal_size 1560
max equal_size 1820
min I_cell_no 0
max I_cell_no 2310
I_cell_no_sum 264465
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
435 2310 435 2310 71637 264465
IIB_equal_cell_no_u1_max 435
I_equal_cell_no_u1_max 2310
IIB_I_cell_no_uvw_total1 71637 0 0 264465
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 435 17201
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 2310 84869
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 25801 127303
size(IIB_cell_u),IIB_cell_no_max_cur 435 845
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1267
size(I_cell_u),I_cell_no_max_cur 2310 5060
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 7590
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 264465 264586
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 25801 127303
time,IIB_I_cell_no_uvw_total1 2 71589 0 0
264586 0 0
IIB_equal_cell_no_u1_max 1267
size(I_cell_u),I_cell_no_max_cur 7590 7728
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 11592
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 25801 127303
time,IIB_I_cell_no_uvw_total1 3 71237 0 0
264577 0 0
IIB_equal_cell_no_u1_max 1267
I_equal_cell_no_u1_max 11592
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 25801 127303
time,IIB_I_cell_no_uvw_total1 4 70676 0 0
264504 0 0
IIB_equal_cell_no_u1_max 1267
I_equal_cell_no_u1_max 11592
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 25801 127303
time,IIB_I_cell_no_uvw_total1 5 70909 0 0
264557 0 0
escape_time reached, so abort
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1583 32251 89546 14490 159128 496098
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
32251 159128 1583 14490 89546 496098
IIB_I_cell_no_uvw_total1 71637 71704 71732 264465
264616 264595
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-0.203649100207704 -3.78579022968484 -142.426095309958
3.49125655195658 -18.5255013193465 194.110055674799
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a named std0771 with 600 processors, by tsltaywb Sat Apr 21 00:23:03 2018
Using Petsc Release Version 3.9.0, Apr, 07, 2018
Max Max/Min Avg Total
Time (sec): 1.293e+02 1.00002 1.293e+02
Objects: 8.450e+02 1.00118 8.440e+02
Flop: 2.402e+10 1.16509 2.255e+10 1.353e+13
Flop/sec: 1.858e+08 1.16510 1.744e+08 1.046e+11
MPI Messages: 1.668e+05 3.35829 1.236e+05 7.419e+07
MPI Message Lengths: 1.466e+09 2.16566 1.103e+04 8.180e+11
MPI Reductions: 1.451e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 1.2932e+02 100.0% 1.3533e+13 100.0% 7.419e+07 100.0% 1.103e+04 100.0% 1.442e+03 99.4%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSided 5 1.0 1.1753e-02 5.7 0.00e+00 0.0 2.9e+04 4.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BuildTwoSidedF 67 1.0 1.6764e+00 4.1 0.00e+00 0.0 1.3e+05 1.4e+05 0.0e+00 1 0 0 2 0 1 0 0 2 0 0
VecDot 8 1.0 1.9643e-01 3.4 1.24e+07 1.1 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 35902
VecDotNorm2 4 1.0 1.4356e-0119.8 1.24e+07 1.1 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 49124
VecMDot 100 1.0 3.0183e-01 2.7 6.25e+07 1.1 0.0e+00 0.0e+00 1.0e+02 0 0 0 0 7 0 0 0 0 7 117627
VecTDot 248 1.0 1.0376e+00 2.0 1.28e+08 1.1 0.0e+00 0.0e+00 2.5e+02 1 1 0 0 17 1 1 0 0 17 70231
VecNorm 247 1.0 4.7946e+00 8.8 9.16e+07 1.1 0.0e+00 0.0e+00 2.5e+02 1 0 0 0 17 1 0 0 0 17 10858
VecScale 110 1.0 6.2954e-03 2.2 6.25e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 563963
VecCopy 672 1.0 2.3515e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 2431 1.0 1.4385e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 258 1.0 2.8743e-01 1.2 1.29e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 255784
VecAYPX 5280 1.0 1.2274e+00 2.0 4.28e+08 1.1 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 198344
VecAXPBYCZ 2588 1.0 7.0963e-01 1.5 7.58e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 606609
VecWAXPY 8 1.0 4.5665e-02 1.4 1.24e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 154437
VecMAXPY 110 1.0 9.0745e-02 1.2 7.38e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 462386
VecAssemblyBegin 34 1.0 2.7867e-01 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAssemblyEnd 34 1.0 6.3419e-05 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecPointwiseMult 55 1.0 1.9400e-02 3.0 3.12e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 91507
VecScatterBegin 5448 1.0 1.1685e+00 2.6 0.00e+00 0.0 7.0e+07 1.1e+04 0.0e+00 1 0 94 94 0 1 0 94 94 0 0
VecScatterEnd 5448 1.0 1.1562e+01 3.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 5 0 0 0 0 0
VecSetRandom 5 1.0 7.0169e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecNormalize 110 1.0 1.0416e-01 2.0 1.87e+07 1.1 0.0e+00 0.0e+00 1.1e+02 0 0 0 0 8 0 0 0 0 8 102254
MatMult 4102 1.0 2.2797e+01 1.2 1.15e+10 1.2 6.2e+07 1.1e+04 0.0e+00 16 48 84 86 0 16 48 84 86 0 283196
MatMultAdd 645 1.0 2.0686e+00 1.9 5.99e+08 1.2 3.8e+06 3.7e+03 0.0e+00 1 2 5 2 0 1 2 5 2 0 163213
MatMultTranspose 645 1.0 2.3167e+00 2.2 5.99e+08 1.2 3.8e+06 3.7e+03 0.0e+00 1 2 5 2 0 1 2 5 2 0 145734
MatSolve 14111.8 9.9322e-01 1.2 6.80e+08 1.1 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 388324
MatSOR 3925 1.0 2.3818e+01 1.4 8.20e+09 1.1 0.0e+00 0.0e+00 0.0e+00 17 34 0 0 0 17 34 0 0 0 194060
MatLUFactorSym 1 1.0 3.9697e-0457.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatLUFactorNum 2 1.0 5.6432e-01 1.4 2.52e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 252133
MatILUFactorSym 1 1.0 2.5929e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatConvert 5 1.0 6.1710e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 16 1.0 1.8764e-01 1.3 4.80e+07 1.2 7.8e+04 1.0e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 144625
MatResidual 645 1.0 3.9151e+00 1.5 1.69e+09 1.2 1.0e+07 1.0e+04 0.0e+00 2 7 14 13 0 2 7 14 13 0 243152
MatAssemblyBegin 102 1.0 1.4931e+00 2.5 0.00e+00 0.0 1.3e+05 1.4e+05 0.0e+00 1 0 0 2 0 1 0 0 2 0 0
MatAssemblyEnd 102 1.0 6.1704e+00 1.0 0.00e+00 0.0 6.2e+05 2.7e+03 2.2e+02 5 0 1 0 15 5 0 1 0 15 0
MatGetRow 852087 1.1 1.0557e-01 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 2 2.0 3.6955e-0538.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCreateSubMat 4 1.0 1.3800e+00 1.0 0.00e+00 0.0 1.3e+04 1.6e+02 6.4e+01 1 0 0 0 4 1 0 0 0 4 0
MatGetOrdering 2 2.0 1.5060e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCoarsen 5 1.0 1.3929e-01 1.1 0.00e+00 0.0 2.6e+06 1.3e+03 1.0e+02 0 0 4 0 7 0 0 4 0 7 0
MatZeroEntries 5 1.0 8.9250e-03 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAXPY 5 1.0 7.8287e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMatMult 5 1.0 2.4963e+00 1.0 1.31e+07 1.2 4.6e+05 5.3e+03 6.2e+01 2 0 1 0 4 2 0 1 0 4 2956
MatMatMultSym 5 1.0 2.3848e+00 1.0 0.00e+00 0.0 3.8e+05 4.3e+03 6.0e+01 2 0 1 0 4 2 0 1 0 4 0
MatMatMultNum 5 1.0 1.0086e-01 1.0 1.31e+07 1.2 7.8e+04 1.0e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 73170
MatPtAP 5 1.0 6.8432e+00 1.0 4.11e+08 1.5 8.6e+05 4.6e+04 7.7e+01 5 2 1 5 5 5 2 1 5 5 33013
MatPtAPSymbolic 5 1.0 1.8684e+00 1.0 0.00e+00 0.0 4.5e+05 4.6e+04 3.5e+01 1 0 1 3 2 1 0 1 3 2 0
MatPtAPNumeric 5 1.0 4.9728e+00 1.0 4.11e+08 1.5 4.1e+05 4.6e+04 4.0e+01 4 2 1 2 3 4 2 1 2 3 45430
MatTrnMatMult 1 1.0 9.3095e-01 1.0 2.61e+07 1.2 1.8e+04 1.7e+05 1.6e+01 1 0 0 0 1 1 0 0 0 1 15887
MatTrnMatMultSym 1 1.0 6.6997e-01 1.0 0.00e+00 0.0 1.6e+04 8.7e+04 1.6e+01 1 0 0 0 1 1 0 0 0 1 0
MatTrnMatMultNum 1 1.0 2.6105e-01 1.0 2.61e+07 1.2 2.3e+03 7.7e+05 0.0e+00 0 0 0 0 0 0 0 0 0 0 56658
MatGetLocalMat 17 1.0 7.8735e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetBrAoCol 15 1.0 1.2452e-01 1.8 0.00e+00 0.0 5.5e+05 3.3e+04 0.0e+00 0 0 1 2 0 0 0 1 2 0 0
SFSetGraph 5 1.0 4.0531e-06 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 5 1.0 1.6538e-02 1.9 0.00e+00 0.0 8.6e+04 4.0e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 114 1.0 1.0221e-02 4.0 0.00e+00 0.0 2.5e+06 1.2e+03 0.0e+00 0 0 3 0 0 0 0 3 0 0 0
SFBcastEnd 114 1.0 3.8953e-02 5.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPGMRESOrthog 100 1.0 3.6761e-01 2.0 1.25e+08 1.1 0.0e+00 0.0e+00 1.0e+02 0 1 0 0 7 0 1 0 0 7 193158
KSPSetUp 20 1.0 4.3537e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 0 0 0 0 2 0 0 0 0 2 0
KSPSolve 9 1.0 6.5676e+01 1.0 2.40e+10 1.2 7.4e+07 1.1e+04 1.2e+03 51100100 96 83 51100100 96 83 205797
PCGAMGGraph_AGG 5 1.0 6.6401e-01 1.0 1.31e+07 1.2 1.9e+05 5.4e+03 6.0e+01 1 0 0 0 4 1 0 0 0 4 11114
PCGAMGCoarse_AGG 5 1.0 1.1636e+00 1.0 2.61e+07 1.2 2.7e+06 3.3e+03 1.2e+02 1 0 4 1 9 1 0 4 1 9 12711
PCGAMGProl_AGG 5 1.0 4.3104e-01 1.0 0.00e+00 0.0 1.7e+05 9.3e+03 8.0e+01 0 0 0 0 6 0 0 0 0 6 0
PCGAMGPOpt_AGG 5 1.0 3.0145e+00 1.0 2.28e+08 1.2 1.2e+06 8.4e+03 2.1e+02 2 1 2 1 14 2 1 2 1 14 42573
GAMG: createProl 5 1.0 5.2617e+00 1.0 2.67e+08 1.2 4.3e+06 5.1e+03 4.7e+02 4 1 6 3 32 4 1 6 3 33 28604
Graph 10 1.0 6.5672e-01 1.0 1.31e+07 1.2 1.9e+05 5.4e+03 6.0e+01 1 0 0 0 4 1 0 0 0 4 11237
MIS/Agg 5 1.0 1.3933e-01 1.1 0.00e+00 0.0 2.6e+06 1.3e+03 1.0e+02 0 0 4 0 7 0 0 4 0 7 0
SA: col data 5 1.0 8.9589e-02 1.1 0.00e+00 0.0 1.1e+05 1.2e+04 2.0e+01 0 0 0 0 1 0 0 0 0 1 0
SA: frmProl0 5 1.0 3.3868e-01 1.0 0.00e+00 0.0 5.2e+04 3.5e+03 4.0e+01 0 0 0 0 3 0 0 0 0 3 0
SA: smooth 5 1.0 2.6009e+00 1.0 1.78e+07 1.2 4.6e+05 5.3e+03 7.2e+01 2 0 1 0 5 2 0 1 0 5 3844
GAMG: partLevel 5 1.0 8.3094e+00 1.0 4.11e+08 1.5 8.8e+05 4.5e+04 1.8e+02 6 2 1 5 12 6 2 1 5 12 27188
repartition 3 1.0 2.4631e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 1 0 0 0 0 1 0
Invert-Sort 2 1.0 7.8353e-0291.3 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 0
Move A 2 1.0 1.1592e-02 1.2 0.00e+00 0.0 2.0e+03 8.5e+02 3.4e+01 0 0 0 0 2 0 0 0 0 2 0
Move P 2 1.0 1.3714e+00 1.0 0.00e+00 0.0 1.1e+04 2.8e+01 3.4e+01 1 0 0 0 2 1 0 0 0 2 0
PCSetUp 4 1.0 1.4434e+01 1.0 9.30e+08 1.3 5.1e+06 1.2e+04 6.8e+02 11 4 7 7 47 11 4 7 7 47 35935
PCSetUpOnBlocks 133 1.0 8.3417e-01 1.3 2.52e+08 1.2 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 170569
PCApply 141 1.0 4.7204e+01 1.1 2.13e+10 1.2 6.9e+07 9.5e+03 1.0e+02 36 89 92 80 7 36 89 92 80 7 254450
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 363 363 223074648 0.
Matrix 150 150 1237480116 0.
Matrix Coarsen 5 5 3180 0.
Distributed Mesh 18 18 94176 0.
Index Set 128 128 23259280 0.
IS L to G Mapping 18 18 10397096 0.
Star Forest Graph 41 41 35424 0.
Discrete System 18 18 16632 0.
Vec Scatter 58 58 17003440 0.
Krylov Solver 20 20 314880 0.
Preconditioner 15 15 15212 0.
PetscRandom 10 10 6460 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
Average time for MPI_Barrier(): 1.64032e-05
Average time for zero size MPI_Send(): 5.4582e-06
#PETSc Option Table entries:
-log_view
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_type gamg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/scratch/source/v2.14.0.tar.gz --download-ml=/home/users/nus/tsltaywb/scratch/source/ml-v6.2-p4.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on 2018-04-17 05:50:35 on nus02
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel
Using PETSc arch:
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/lib//include -I/home/users/nus/tsltaywb/lib///include -I/home/users/nus/tsltaywb/lib//include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lml -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lstdc++ -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lstdc++ -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-04-21 00:23:08.128316:
JobId: 6808395.wlm01
Project: 11000324
Exit Status: 0
NCPUs Requested: 600 NCPUs Used: 600
CPU Time Used: 00:50:53
Memory Requested: 2400gb Memory Used: 53653576kb
Vmem Used: 75059520kb
Walltime requested: 00:05:00 Walltime Used: 00:02:25
Execution Nodes Used: (std0771:ncpus=24:mem=100663296kb)+(std0772:ncpus=24:mem=100663296kb)+(std0866:ncpus=24:mem=100663296kb)+(std0867:ncpus=24:mem=100663296kb)+(std0871:ncpus=24:mem=100663296kb)+(std0872:ncpus=24:mem=100663296kb)+(std0901:ncpus=24:mem=100663296kb)+(std0902:ncpus=24:mem=100663296kb)+(std0903:ncpus=24:mem=100663296kb)+(std0904:ncpus=24:mem=100663296kb)+(std0905:ncpus=24:mem=100663296kb)+(std0906:ncpus=24:mem=100663296kb)+(std0907:ncpus=24:mem=100663296kb)+(std0908:ncpus=24:mem=100663296kb)+(std0909:ncpus=24:mem=100663296kb)+(std0910:ncpus=24:mem=100663296kb)+(std0911:ncpus=24:mem=100663296kb)+(std0912:ncpus=24:mem=100663296kb)+(std0913:ncpus=24:mem=100663296kb)+(std0914:ncpus=24:mem=100663296kb)+(std0915:ncpus=24:mem=100663296kb)+(std0916:ncpus=24:mem=100663296kb)+(std0917:ncpus=24:mem=100663296kb)+(std0918:ncpus=24:mem=100663296kb)+(std0919:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 4
minimum ngh_surfaces and ngh_vertics are 1 1
min IIB_cell_no 0
max IIB_cell_no 592
IIB_cell_no_sum 71637
min equal_size 3600
max equal_size 4096
min I_cell_no 0
max I_cell_no 4352
I_cell_no_sum 264465
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
592 4352 592 4352 71637 264465
IIB_equal_cell_no_u1_max 592
I_equal_cell_no_u1_max 4352
IIB_I_cell_no_uvw_total1 71637 0 0 264465
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 592 18100
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 4352 70137
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 27150 105205
size(IIB_cell_u),IIB_cell_no_max_cur 592 1296
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 1944
size(I_cell_u),I_cell_no_max_cur 4352 9324
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 13986
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 264465 264586
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 27150 105205
time,IIB_I_cell_no_uvw_total1 2 71589 0 0
264586 0 0
size(IIB_cell_u),IIB_cell_no_max_cur 1944 2004
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 3006
size(I_cell_u),I_cell_no_max_cur 13986 13992
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 20988
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 27150 105205
time,IIB_I_cell_no_uvw_total1 3 71237 0 0
264577 0 0
IIB_equal_cell_no_u1_max 3006
I_equal_cell_no_u1_max 20988
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 27150 105205
time,IIB_I_cell_no_uvw_total1 4 70676 0 0
264504 0 0
IIB_equal_cell_no_u1_max 3006
I_equal_cell_no_u1_max 20988
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 27150 105205
time,IIB_I_cell_no_uvw_total1 5 70909 0 0
264557 0 0
escape_time reached, so abort
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
3757 33937 89546 26235 131506 496098
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 4
minimum ngh_surfaces and ngh_vertics are 1 1
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
33937 131506 3757 26235 89546 496098
IIB_I_cell_no_uvw_total1 71637 71704 71732 264465
264616 264595
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-0.203707530878565 -3.78586926105787 -142.425572181451
3.49109646861912 -18.5191982260976 194.106345316573
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a named std0442 with 288 processors, by tsltaywb Fri Apr 20 22:40:33 2018
Using Petsc Release Version 3.9.0, Apr, 07, 2018
Max Max/Min Avg Total
Time (sec): 2.523e+02 1.00002 2.523e+02
Objects: 8.450e+02 1.00118 8.440e+02
Flop: 4.819e+10 1.10663 4.730e+10 1.362e+13
Flop/sec: 1.910e+08 1.10663 1.875e+08 5.400e+10
MPI Messages: 9.780e+04 3.18949 7.382e+04 2.126e+07
MPI Message Lengths: 1.951e+09 2.12277 2.441e+04 5.190e+11
MPI Reductions: 1.416e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 2.5229e+02 100.0% 1.3623e+13 100.0% 2.126e+07 100.0% 2.441e+04 100.0% 1.407e+03 99.4%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSided 5 1.0 3.7102e-0213.7 0.00e+00 0.0 8.5e+03 4.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BuildTwoSidedF 67 1.0 8.4729e+00 2.0 0.00e+00 0.0 4.9e+04 2.5e+05 0.0e+00 3 0 0 2 0 3 0 0 2 0 0
VecDot 8 1.0 7.0657e-01 5.3 2.48e+07 1.1 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 9981
VecDotNorm2 4 1.0 4.9834e-0155.8 2.48e+07 1.1 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 14152
VecMDot 100 1.0 1.2717e+00 2.4 1.25e+08 1.1 0.0e+00 0.0e+00 1.0e+02 0 0 0 0 7 0 0 0 0 7 27905
VecTDot 244 1.0 2.3305e+00 3.3 2.52e+08 1.1 0.0e+00 0.0e+00 2.4e+02 1 1 0 0 17 1 1 0 0 17 30766
VecNorm 245 1.0 4.0297e+0117.5 1.81e+08 1.1 0.0e+00 0.0e+00 2.4e+02 9 0 0 0 17 9 0 0 0 17 1277
VecScale 110 1.0 4.8598e-02 2.2 1.25e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 73019
VecCopy 662 1.0 7.8418e-01 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 2395 1.0 6.5509e-01 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 254 1.0 1.0951e+00 2.1 2.55e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 66060
VecAYPX 5198 1.0 2.9301e+00 2.4 8.43e+08 1.1 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 81748
VecAXPBYCZ 2548 1.0 1.9207e+00 2.1 1.49e+09 1.1 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 220650
VecWAXPY 8 1.0 1.9802e-01 2.7 2.48e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 35614
VecMAXPY 110 1.0 2.6582e-01 1.6 1.48e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 157767
VecAssemblyBegin 34 1.0 4.0356e-01 4.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAssemblyEnd 34 1.0 6.3658e-05 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecPointwiseMult 55 1.0 4.0215e-02 1.7 6.24e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 44121
VecScatterBegin 5366 1.0 1.6781e+00 3.0 0.00e+00 0.0 2.0e+07 2.4e+04 0.0e+00 0 0 95 94 0 0 0 95 94 0 0
VecScatterEnd 5366 1.0 7.0576e+01 7.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 17 0 0 0 0 17 0 0 0 0 0
VecSetRandom 5 1.0 1.4006e-02 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecNormalize 110 1.0 1.1545e+00 4.5 3.75e+07 1.1 0.0e+00 0.0e+00 1.1e+02 0 0 0 0 8 0 0 0 0 8 9221
MatMult 4040 1.0 8.7491e+01 1.8 2.25e+10 1.1 1.7e+07 2.6e+04 0.0e+00 25 46 81 86 0 25 46 81 86 0 72363
MatMultAdd 635 1.0 1.3872e+01 7.7 1.17e+09 1.1 1.4e+06 6.2e+03 0.0e+00 2 2 7 2 0 2 2 7 2 0 23814
MatMultTranspose 635 1.0 1.4661e+01 7.5 1.17e+09 1.1 1.4e+06 6.2e+03 0.0e+00 2 2 7 2 0 2 2 7 2 0 22532
MatSolve 13911.6 2.7498e+00 1.6 1.38e+09 1.1 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 142785
MatSOR 3865 1.0 7.5148e+01 2.3 1.71e+10 1.1 0.0e+00 0.0e+00 0.0e+00 18 36 0 0 0 18 36 0 0 0 64645
MatLUFactorSym 1 1.0 5.1308e-0474.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatLUFactorNum 2 1.0 1.8094e+00 2.2 5.16e+08 1.1 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 80741
MatILUFactorSym 1 1.0 6.5920e+0014.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 1 0 0 0 0 0
MatConvert 5 1.0 4.5344e-01 5.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 16 1.0 5.1112e-01 1.5 9.57e+07 1.1 2.2e+04 2.3e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 52984
MatResidual 635 1.0 1.6918e+01 3.2 3.31e+09 1.1 2.8e+06 2.3e+04 0.0e+00 4 7 13 12 0 4 7 13 12 0 55100
MatAssemblyBegin 102 1.0 8.2371e+00 1.8 0.00e+00 0.0 4.9e+04 2.5e+05 0.0e+00 3 0 0 2 0 3 0 0 2 0 0
MatAssemblyEnd 102 1.0 6.0627e+00 1.2 0.00e+00 0.0 2.4e+05 4.6e+03 2.2e+02 2 0 1 0 15 2 0 1 0 15 0
MatGetRow 1702611 1.1 1.6791e-01 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 2 2.0 5.6982e-04597.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCreateSubMat 4 1.0 3.1587e-01 1.0 0.00e+00 0.0 6.8e+03 3.7e+02 6.4e+01 0 0 0 0 5 0 0 0 0 5 0
MatGetOrdering 2 2.0 4.2314e-0117.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCoarsen 5 1.0 1.6996e-01 1.2 0.00e+00 0.0 5.9e+05 3.1e+03 7.5e+01 0 0 3 0 5 0 0 3 0 5 0
MatZeroEntries 5 1.0 1.8760e-0112.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAXPY 5 1.0 2.1740e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMatMult 5 1.0 1.3280e+00 1.0 2.61e+07 1.1 1.4e+05 1.1e+04 6.2e+01 1 0 1 0 4 1 0 1 0 4 5527
MatMatMultSym 5 1.0 1.1175e+00 1.0 0.00e+00 0.0 1.2e+05 8.9e+03 6.0e+01 0 0 1 0 4 0 0 1 0 4 0
MatMatMultNum 5 1.0 2.0309e-01 1.1 2.61e+07 1.1 2.2e+04 2.3e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 36141
MatPtAP 5 1.0 1.2759e+01 1.0 8.09e+08 1.3 3.2e+05 7.9e+04 7.7e+01 5 2 1 5 5 5 2 1 5 5 17465
MatPtAPSymbolic 5 1.0 3.8273e+00 1.0 0.00e+00 0.0 1.4e+05 9.0e+04 3.5e+01 1 0 1 2 2 1 0 1 2 2 0
MatPtAPNumeric 5 1.0 9.0744e+00 1.0 8.09e+08 1.3 1.7e+05 7.0e+04 4.0e+01 4 2 1 2 3 4 2 1 2 3 24556
MatTrnMatMult 1 1.0 4.0050e+00 1.0 5.17e+07 1.1 8.5e+03 2.4e+05 1.6e+01 2 0 0 0 1 2 0 0 0 1 3656
MatTrnMatMultSym 1 1.0 2.7167e+00 1.0 0.00e+00 0.0 7.4e+03 1.2e+05 1.6e+01 1 0 0 0 1 1 0 0 0 1 0
MatTrnMatMultNum 1 1.0 1.2892e+00 1.0 5.17e+07 1.1 1.1e+03 1.1e+06 0.0e+00 1 0 0 0 0 1 0 0 0 0 11357
MatGetLocalMat 17 1.0 2.1107e-01 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetBrAoCol 15 1.0 2.0231e-01 2.1 0.00e+00 0.0 1.5e+05 7.5e+04 0.0e+00 0 0 1 2 0 0 0 1 2 0 0
SFSetGraph 5 1.0 5.2452e-06 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 5 1.0 4.1750e-02 3.9 0.00e+00 0.0 2.5e+04 8.7e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 85 1.0 8.3704e-03 4.0 0.00e+00 0.0 5.6e+05 2.9e+03 0.0e+00 0 0 3 0 0 0 0 3 0 0 0
SFBcastEnd 85 1.0 2.3875e-02 3.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPGMRESOrthog 100 1.0 1.4133e+00 2.0 2.50e+08 1.1 0.0e+00 0.0e+00 1.0e+02 0 1 0 0 7 0 1 0 0 7 50219
KSPSetUp 20 1.0 2.1678e+00 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 1 0 0 0 2 1 0 0 0 2 0
KSPSolve 9 1.0 1.8116e+02 1.0 4.81e+10 1.1 2.1e+07 2.4e+04 1.2e+03 72100100 96 82 72100100 96 83 75103
PCGAMGGraph_AGG 5 1.0 3.1132e+00 1.0 2.61e+07 1.1 5.4e+04 1.2e+04 6.0e+01 1 0 0 0 4 1 0 0 0 4 2358
PCGAMGCoarse_AGG 5 1.0 4.3671e+00 1.0 5.17e+07 1.1 6.1e+05 9.0e+03 9.5e+01 2 0 3 1 7 2 0 3 1 7 3352
PCGAMGProl_AGG 5 1.0 4.7506e-01 1.0 0.00e+00 0.0 5.5e+04 1.8e+04 8.0e+01 0 0 0 0 6 0 0 0 0 6 0
PCGAMGPOpt_AGG 5 1.0 2.6141e+00 1.0 4.53e+08 1.1 3.6e+05 1.9e+04 2.1e+02 1 1 2 1 15 1 1 2 1 15 48912
GAMG: createProl 5 1.0 1.0569e+01 1.0 5.31e+08 1.1 1.1e+06 1.3e+04 4.4e+02 4 1 5 3 31 4 1 5 3 31 14178
Graph 10 1.0 3.1086e+00 1.0 2.61e+07 1.1 5.4e+04 1.2e+04 6.0e+01 1 0 0 0 4 1 0 0 0 4 2361
MIS/Agg 5 1.0 1.7001e-01 1.2 0.00e+00 0.0 5.9e+05 3.1e+03 7.5e+01 0 0 3 0 5 0 0 3 0 5 0
SA: col data 5 1.0 6.0319e-02 1.1 0.00e+00 0.0 3.4e+04 2.6e+04 2.0e+01 0 0 0 0 1 0 0 0 0 1 0
SA: frmProl0 5 1.0 3.9480e-01 1.0 0.00e+00 0.0 2.1e+04 5.6e+03 4.0e+01 0 0 0 0 3 0 0 0 0 3 0
SA: smooth 5 1.0 1.6144e+00 1.0 3.53e+07 1.1 1.4e+05 1.1e+04 7.2e+01 1 0 1 0 5 1 0 1 0 5 6158
GAMG: partLevel 5 1.0 1.3111e+01 1.0 8.09e+08 1.3 3.2e+05 7.7e+04 1.8e+02 5 2 2 5 13 5 2 2 5 13 16996
repartition 2 1.0 1.3661e-03 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 1 0 0 0 0 1 0
Invert-Sort 2 1.0 3.1821e-0235.4 0.00e+00 0.0 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 0
Move A 2 1.0 8.8933e-03 1.1 0.00e+00 0.0 1.5e+03 1.5e+03 3.4e+01 0 0 0 0 2 0 0 0 0 2 0
Move P 2 1.0 3.0897e-01 1.0 0.00e+00 0.0 5.3e+03 3.4e+01 3.4e+01 0 0 0 0 2 0 0 0 0 2 0
PCSetUp 4 1.0 3.1552e+01 1.2 1.86e+09 1.2 1.4e+06 2.8e+04 6.5e+02 11 4 7 7 46 11 4 7 7 46 16441
PCSetUpOnBlocks 131 1.0 7.7258e+00 5.2 5.16e+08 1.1 0.0e+00 0.0e+00 0.0e+00 1 1 0 0 0 1 1 0 0 0 18910
PCApply 139 1.0 1.3946e+02 1.3 4.28e+10 1.1 2.0e+07 2.1e+04 1.0e+02 49 89 92 79 7 49 89 92 79 7 86847
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 363 363 438680928 0.
Matrix 150 150 2506664764 0.
Matrix Coarsen 5 5 3180 0.
Distributed Mesh 18 18 94176 0.
Index Set 128 128 44378656 0.
IS L to G Mapping 18 18 19585328 0.
Star Forest Graph 41 41 35424 0.
Discrete System 18 18 16632 0.
Vec Scatter 58 58 33956736 0.
Krylov Solver 20 20 314880 0.
Preconditioner 15 15 15212 0.
PetscRandom 10 10 6460 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 0.
Average time for MPI_Barrier(): 1.54018e-05
Average time for zero size MPI_Send(): 5.56641e-06
#PETSc Option Table entries:
-log_view
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_type gamg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/scratch/source/v2.14.0.tar.gz --download-ml=/home/users/nus/tsltaywb/scratch/source/ml-v6.2-p4.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on 2018-04-17 05:50:35 on nus02
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel
Using PETSc arch:
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/lib//include -I/home/users/nus/tsltaywb/lib///include -I/home/users/nus/tsltaywb/lib//include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lml -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lstdc++ -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lstdc++ -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-04-20 22:40:36.157718:
JobId: 6800949.wlm01
Project: 11000324
Exit Status: 0
NCPUs Requested: 288 NCPUs Used: 288
CPU Time Used: 01:39:53
Memory Requested: 1152gb Memory Used: 97346692kb
Vmem Used: 117893608kb
Walltime requested: 00:05:00 Walltime Used: 00:04:26
Execution Nodes Used: (std0442:ncpus=24:mem=100663296kb)+(std0116:ncpus=24:mem=100663296kb)+(std0162:ncpus=24:mem=100663296kb)+(std0207:ncpus=24:mem=100663296kb)+(std0215:ncpus=24:mem=100663296kb)+(std0218:ncpus=24:mem=100663296kb)+(std0227:ncpus=24:mem=100663296kb)+(std0254:ncpus=24:mem=100663296kb)+(std0256:ncpus=24:mem=100663296kb)+(std0303:ncpus=24:mem=100663296kb)+(std0313:ncpus=24:mem=100663296kb)+(std0363:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 3
min IIB_cell_no 0
max IIB_cell_no 227
IIB_cell_no_sum 71637
min equal_size 560
max equal_size 792
min I_cell_no 0
max I_cell_no 960
I_cell_no_sum 264465
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
227 960 227 960 71637 264465
IIB_equal_cell_no_u1_max 227
I_equal_cell_no_u1_max 960
IIB_I_cell_no_uvw_total1 71637 0 0 264465
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 227 6585
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 960 36719
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 9877 55078
size(IIB_cell_u),IIB_cell_no_max_cur 227 385
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 577
size(I_cell_u),I_cell_no_max_cur 960 2002
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 3003
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 264465 264586
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 9877 55078
time,IIB_I_cell_no_uvw_total1 2 71589 0 0
264586 0 0
size(IIB_cell_u),IIB_cell_no_max_cur 577 606
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 909
size(I_cell_u),I_cell_no_max_cur 3003 3328
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 4992
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 9877 55078
time,IIB_I_cell_no_uvw_total1 3 71237 0 0
264577 0 0
IIB_equal_cell_no_u1_max 909
I_equal_cell_no_u1_max 4992
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 9877 55078
time,IIB_I_cell_no_uvw_total1 4 70676 0 0
264504 0 0
IIB_equal_cell_no_u1_max 909
I_equal_cell_no_u1_max 4992
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 9877 55078
time,IIB_I_cell_no_uvw_total1 5 70909 0 0
264557 0 0
escape_time reached, so abort
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1136 12346 89546 6240 68847 496098
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 3
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
12346 68847 1136 6240 89546 496098
IIB_I_cell_no_uvw_total1 71637 71704 71732 264465
264616 264595
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-0.203573203414587 -3.78836120073860 -142.428735523722
3.49076187897078 -18.4949348357545 194.047090449719
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a named std0930 with 1440 processors, by tsltaywb Sat Apr 21 20:13:18 2018
Using Petsc Release Version 3.9.0, Apr, 07, 2018
Max Max/Min Avg Total
Time (sec): 1.077e+02 1.00011 1.077e+02
Objects: 8.780e+02 1.00114 8.770e+02
Flop: 9.807e+09 1.27406 9.576e+09 1.379e+13
Flop/sec: 9.107e+07 1.27409 8.893e+07 1.281e+11
MPI Messages: 1.693e+05 8.99238 7.795e+04 1.122e+08
MPI Message Lengths: 1.007e+09 2.35317 1.203e+04 1.351e+12
MPI Reductions: 1.583e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 1.0768e+02 100.0% 1.3789e+13 100.0% 1.122e+08 100.0% 1.203e+04 100.0% 1.574e+03 99.4%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSided 5 1.0 2.0495e-02 4.7 0.00e+00 0.0 4.0e+04 4.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BuildTwoSidedF 69 1.0 1.4551e+00 2.8 0.00e+00 0.0 2.4e+05 1.3e+05 0.0e+00 1 0 0 2 0 1 0 0 2 0 0
VecDot 8 1.0 2.0908e-0121.8 4.96e+06 1.2 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 33730
VecDotNorm2 4 1.0 1.9284e-0193.1 4.96e+06 1.2 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 36571
VecMDot 100 1.0 2.6590e-01 2.0 2.51e+07 1.2 0.0e+00 0.0e+00 1.0e+02 0 0 0 0 6 0 0 0 0 6 133692
VecTDot 260 1.0 7.6737e-01 1.7 5.38e+07 1.2 0.0e+00 0.0e+00 2.6e+02 0 1 0 0 16 0 1 0 0 17 99561
VecNorm 253 1.0 3.7218e+00 3.7 3.79e+07 1.2 0.0e+00 0.0e+00 2.5e+02 2 0 0 0 16 2 0 0 0 16 14464
VecScale 110 1.0 2.4757e-03 2.5 2.51e+06 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1435901
VecCopy 702 1.0 7.9543e-02 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 2542 1.0 5.8362e-02 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 270 1.0 1.0637e-01 1.5 5.42e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 724347
VecAYPX 5526 1.0 3.6515e-01 2.1 1.80e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 698807
VecAXPBYCZ 2708 1.0 3.5396e-01 2.3 3.17e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 1272430
VecWAXPY 8 1.0 2.0493e-02 1.7 4.96e+06 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 344131
VecMAXPY 110 1.0 4.1608e-02 1.9 2.96e+07 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1009728
VecAssemblyBegin 35 1.0 2.0360e-01 2.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAssemblyEnd 35 1.0 6.2466e-05 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecPointwiseMult 55 1.0 7.8759e-03 3.4 1.25e+06 1.2 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 225683
VecScatterBegin 5695 1.0 9.3908e-01 3.8 0.00e+00 0.0 1.0e+08 1.2e+04 0.0e+00 1 0 93 94 0 1 0 93 94 0 0
VecScatterEnd 5695 1.0 8.3244e+00 3.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 4 0 0 0 0 0
VecSetRandom 5 1.0 2.8300e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecNormalize 110 1.0 2.0694e-01 1.6 7.52e+06 1.2 0.0e+00 0.0e+00 1.1e+02 0 0 0 0 7 0 0 0 0 7 51535
MatMult 4288 1.0 1.1565e+01 1.4 4.89e+09 1.4 8.9e+07 1.3e+04 0.0e+00 9 49 79 86 0 9 49 79 86 0 589627
MatMultAdd 675 1.0 2.5995e+00 5.2 2.54e+08 1.3 7.6e+06 3.1e+03 0.0e+00 2 3 7 2 0 2 3 7 2 0 137560
MatMultTranspose 675 1.0 1.8251e+00 4.0 2.54e+08 1.3 7.6e+06 3.1e+03 0.0e+00 1 3 7 2 0 1 3 7 2 0 195931
MatSolve 14712.2 7.5874e-01 2.6 2.64e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 494325
MatSOR 4105 1.0 9.8198e+00 2.0 3.15e+09 1.2 0.0e+00 0.0e+00 0.0e+00 8 32 0 0 0 8 32 0 0 0 452389
MatLUFactorSym 1 1.0 5.6100e-0481.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatLUFactorNum 2 1.0 4.5086e-01 3.0 9.66e+07 1.4 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 302823
MatILUFactorSym 1 1.0 3.1375e-01 4.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatConvert 5 1.0 2.7598e-02 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 16 1.0 7.8282e-02 1.7 1.93e+07 1.3 1.1e+05 1.2e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 348309
MatResidual 675 1.0 2.4201e+00 2.1 7.25e+08 1.4 1.4e+07 1.2e+04 0.0e+00 1 7 13 13 0 1 7 13 13 0 417100
MatAssemblyBegin 106 1.0 1.3056e+00 2.1 0.00e+00 0.0 2.4e+05 1.3e+05 0.0e+00 1 0 0 2 0 1 0 0 2 0 0
MatAssemblyEnd 106 1.0 7.0611e+00 1.0 0.00e+00 0.0 1.5e+06 1.8e+03 2.3e+02 7 0 1 0 15 7 0 1 0 15 0
MatGetRow 341706 1.2 3.7706e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 2 2.0 3.4308e-04359.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCreateSubMat 6 1.0 2.0243e+00 1.0 0.00e+00 0.0 1.6e+05 1.1e+03 9.6e+01 2 0 0 0 6 2 0 0 0 6 0
MatGetOrdering 2 2.0 1.5898e-02 3.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCoarsen 5 1.0 1.1609e-01 1.1 0.00e+00 0.0 4.7e+06 1.8e+03 1.7e+02 0 0 4 1 11 0 0 4 1 11 0
MatZeroEntries 5 1.0 4.3550e-03 2.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAXPY 5 1.0 5.6949e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMatMult 5 1.0 1.1438e+00 1.0 5.37e+06 1.4 6.8e+05 5.7e+03 6.2e+01 1 0 1 0 4 1 0 1 0 4 6537
MatMatMultSym 5 1.0 1.0687e+00 1.0 0.00e+00 0.0 5.7e+05 4.6e+03 6.0e+01 1 0 1 0 4 1 0 1 0 4 0
MatMatMultNum 5 1.0 6.2605e-02 1.1 5.37e+06 1.4 1.1e+05 1.2e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 119436
MatPtAP 5 1.0 5.8088e+00 1.0 1.73e+08 1.9 1.7e+06 3.8e+04 7.7e+01 5 2 2 5 5 5 2 2 5 5 40397
MatPtAPSymbolic 5 1.0 9.7437e-01 1.0 0.00e+00 0.0 7.0e+05 4.9e+04 3.5e+01 1 0 1 3 2 1 0 1 3 2 0
MatPtAPNumeric 5 1.0 4.8295e+00 1.0 1.73e+08 1.9 1.0e+06 3.0e+04 4.0e+01 4 2 1 2 3 4 2 1 2 3 48588
MatTrnMatMult 1 1.0 5.1030e-01 1.0 1.06e+07 1.3 4.4e+04 1.1e+05 1.6e+01 0 0 0 0 1 0 0 0 0 1 29433
MatTrnMatMultSym 1 1.0 3.3892e-01 1.0 0.00e+00 0.0 3.9e+04 5.5e+04 1.6e+01 0 0 0 0 1 0 0 0 0 1 0
MatTrnMatMultNum 1 1.0 1.7566e-01 1.0 1.06e+07 1.3 5.6e+03 4.8e+05 0.0e+00 0 0 0 0 0 0 0 0 0 0 85507
MatGetLocalMat 17 1.0 3.4708e-02 1.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetBrAoCol 15 1.0 9.4534e-02 2.1 0.00e+00 0.0 7.4e+05 4.1e+04 0.0e+00 0 0 1 2 0 0 0 1 2 0 0
SFSetGraph 5 1.0 5.0068e-06 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 5 1.0 2.3274e-02 2.1 0.00e+00 0.0 1.2e+05 4.5e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 177 1.0 9.6838e-03 4.3 0.00e+00 0.0 4.6e+06 1.7e+03 0.0e+00 0 0 4 1 0 0 0 4 1 0 0
SFBcastEnd 177 1.0 2.8875e-02 4.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPGMRESOrthog 100 1.0 2.8578e-01 1.8 5.01e+07 1.2 0.0e+00 0.0e+00 1.0e+02 0 1 0 0 6 0 1 0 0 6 248784
KSPSetUp 20 1.0 4.2640e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 0 0 0 0 2 0 0 0 0 2 0
KSPSolve 9 1.0 3.5276e+01 1.0 9.79e+09 1.3 1.1e+08 1.2e+04 1.3e+03 33100100 97 84 33100100 97 84 390407
PCGAMGGraph_AGG 5 1.0 3.0630e-01 1.0 5.37e+06 1.4 2.5e+05 6.3e+03 6.0e+01 0 0 0 0 4 0 0 0 0 4 24412
PCGAMGCoarse_AGG 5 1.0 6.6444e-01 1.0 1.06e+07 1.3 4.8e+06 3.5e+03 1.9e+02 1 0 4 1 12 1 0 4 1 12 22605
PCGAMGProl_AGG 5 1.0 1.0120e-01 1.0 0.00e+00 0.0 2.7e+05 9.1e+03 8.0e+01 0 0 0 0 5 0 0 0 0 5 0
PCGAMGPOpt_AGG 5 1.0 1.4647e+00 1.0 9.23e+07 1.3 1.7e+06 9.5e+03 2.1e+02 1 1 2 1 13 1 1 2 1 13 88410
GAMG: createProl 5 1.0 2.5337e+00 1.0 1.08e+08 1.3 7.1e+06 5.3e+03 5.3e+02 2 1 6 3 34 2 1 6 3 34 59989
Graph 10 1.0 3.0478e-01 1.0 5.37e+06 1.4 2.5e+05 6.3e+03 6.0e+01 0 0 0 0 4 0 0 0 0 4 24533
MIS/Agg 5 1.0 1.1613e-01 1.1 0.00e+00 0.0 4.7e+06 1.8e+03 1.7e+02 0 0 4 1 11 0 0 4 1 11 0
SA: col data 5 1.0 1.8581e-02 1.1 0.00e+00 0.0 1.6e+05 1.3e+04 2.0e+01 0 0 0 0 1 0 0 0 0 1 0
SA: frmProl0 5 1.0 7.7043e-02 1.0 0.00e+00 0.0 1.1e+05 2.5e+03 4.0e+01 0 0 0 0 3 0 0 0 0 3 0
SA: smooth 5 1.0 1.2110e+00 1.0 7.25e+06 1.4 6.8e+05 5.7e+03 7.2e+01 1 0 1 0 5 1 0 1 0 5 8362
GAMG: partLevel 5 1.0 8.0089e+00 1.0 1.73e+08 1.9 1.9e+06 3.5e+04 2.3e+02 7 2 2 5 15 7 2 2 5 15 29299
repartition 3 1.0 5.4121e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.8e+01 0 0 0 0 1 0 0 0 0 1 0
Invert-Sort 3 1.0 1.5763e-0122.4 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 1 0 0 0 0 1 0
Move A 3 1.0 8.4985e-01 1.0 0.00e+00 0.0 9.2e+04 1.8e+03 5.1e+01 1 0 0 0 3 1 0 0 0 3 0
Move P 3 1.0 1.1894e+00 1.0 0.00e+00 0.0 6.4e+04 6.2e+01 5.1e+01 1 0 0 0 3 1 0 0 0 3 0
PCSetUp 4 1.0 1.1349e+01 1.0 3.78e+08 1.5 9.0e+06 1.1e+04 7.9e+02 10 4 8 8 50 10 4 8 8 50 46098
PCSetUpOnBlocks 139 1.0 7.7229e-01 3.2 9.66e+07 1.4 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 176788
PCApply 147 1.0 2.1472e+01 1.1 8.70e+09 1.3 1.0e+08 1.1e+04 1.0e+02 19 89 91 80 7 19 89 91 80 7 569588
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 369 369 92175624 0.
Matrix 158 158 487572844 0.
Matrix Coarsen 5 5 3180 0.
Distributed Mesh 18 18 94176 0.
Index Set 144 144 10541668 0.
IS L to G Mapping 18 18 4620592 0.
Star Forest Graph 41 41 35424 0.
Discrete System 18 18 16632 0.
Vec Scatter 61 61 6861288 0.
Krylov Solver 20 20 314880 0.
Preconditioner 15 15 15212 0.
PetscRandom 10 10 6460 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 0.
Average time for MPI_Barrier(): 0.000318766
Average time for zero size MPI_Send(): 8.26998e-06
#PETSc Option Table entries:
-log_view
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_type gamg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/scratch/source/v2.14.0.tar.gz --download-ml=/home/users/nus/tsltaywb/scratch/source/ml-v6.2-p4.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on 2018-04-17 05:50:35 on nus02
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel
Using PETSc arch:
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/lib//include -I/home/users/nus/tsltaywb/lib///include -I/home/users/nus/tsltaywb/lib//include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lml -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lstdc++ -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lstdc++ -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-04-21 20:13:23.909068:
JobId: 6809210.wlm01
Project: 11000324
Exit Status: 0
NCPUs Requested: 1440 NCPUs Used: 1440
CPU Time Used: 00:42:12
Memory Requested: 5760gb Memory Used: 29886936kb
Vmem Used: 53296508kb
Walltime requested: 00:05:00 Walltime Used: 00:02:04
Execution Nodes Used: (std0930:ncpus=24:mem=100663296kb)+(std0931:ncpus=24:mem=100663296kb)+(std0932:ncpus=24:mem=100663296kb)+(std0933:ncpus=24:mem=100663296kb)+(std0934:ncpus=24:mem=100663296kb)+(std0935:ncpus=24:mem=100663296kb)+(std0936:ncpus=24:mem=100663296kb)+(std0937:ncpus=24:mem=100663296kb)+(std0938:ncpus=24:mem=100663296kb)+(std0939:ncpus=24:mem=100663296kb)+(std0940:ncpus=24:mem=100663296kb)+(std0941:ncpus=24:mem=100663296kb)+(std0942:ncpus=24:mem=100663296kb)+(std0953:ncpus=24:mem=100663296kb)+(std0954:ncpus=24:mem=100663296kb)+(std0955:ncpus=24:mem=100663296kb)+(std0956:ncpus=24:mem=100663296kb)+(std0957:ncpus=24:mem=100663296kb)+(std0958:ncpus=24:mem=100663296kb)+(std0959:ncpus=24:mem=100663296kb)+(std0960:ncpus=24:mem=100663296kb)+(std0961:ncpus=24:mem=100663296kb)+(std0962:ncpus=24:mem=100663296kb)+(std0963:ncpus=24:mem=100663296kb)+(std1101:ncpus=24:mem=100663296kb)+(std1103:ncpus=24:mem=100663296kb)+(std1105:ncpus=24:mem=100663296kb)+(std1108:ncpus=24:mem=100663296kb)+(std1109:ncpus=24:mem=100663296kb)+(std1111:ncpus=24:mem=100663296kb)+(std1113:ncpus=24:mem=100663296kb)+(std1114:ncpus=24:mem=100663296kb)+(std1116:ncpus=24:mem=100663296kb)+(std1118:ncpus=24:mem=100663296kb)+(std1120:ncpus=24:mem=100663296kb)+(std1121:ncpus=24:mem=100663296kb)+(std1127:ncpus=24:mem=100663296kb)+(std1129:ncpus=24:mem=100663296kb)+(std1131:ncpus=24:mem=100663296kb)+(std1136:ncpus=24:mem=100663296kb)+(std1138:ncpus=24:mem=100663296kb)+(std1139:ncpus=24:mem=100663296kb)+(std1140:ncpus=24:mem=100663296kb)+(std1141:ncpus=24:mem=100663296kb)+(std1154:ncpus=24:mem=100663296kb)+(std1247:ncpus=24:mem=100663296kb)+(std1248:ncpus=24:mem=100663296kb)+(std1250:ncpus=24:mem=100663296kb)+(std1251:ncpus=24:mem=100663296kb)+(std1253:ncpus=24:mem=100663296kb)+(std1255:ncpus=24:mem=100663296kb)+(std1256:ncpus=24:mem=100663296kb)+(std1262:ncpus=24:mem=100663296kb)+(std1263:ncpus=24:mem=100663296kb)+(std1264:ncpus=24:mem=100663296kb)+(std1265:ncpus=24:mem=100663296kb)+(std1268:ncpus=24:mem=100663296kb)+(std1269:ncpus=24:mem=100663296kb)+(std1466:ncpus=24:mem=100663296kb)+(std0630:ncpus=24:mem=100663296kb)
======================================================================================
-------------- next part --------------
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 3
min IIB_cell_no 0
max IIB_cell_no 324
IIB_cell_no_sum 71637
min equal_size 910
max equal_size 1232
min I_cell_no 0
max I_cell_no 1440
I_cell_no_sum 264465
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
324 1440 324 1440 71637 264465
IIB_equal_cell_no_u1_max 324
I_equal_cell_no_u1_max 1440
IIB_I_cell_no_uvw_total1 71637 0 0 264465
0 0
size(IIB_cell_u),IIB_cell_no_max_cur 324 11334
local IIB_cells size exceed, to increase size
size(I_cell_u),I_cell_no_max_cur 1440 62615
local I_cells size exceed, to increase size
IIB_cell_no_u1_max,I_cell_no_u1_max 17001 93922
size(IIB_cell_u),IIB_cell_no_max_cur 324 577
IIB_cells size exceed, to increase size
IIB_equal_cell_no_u1_max 865
size(I_cell_u),I_cell_no_max_cur 1440 3168
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 4752
size(I_global_cell_u1),I_global_cell_no_u1_max_cur 264465 264586
I global cells size exceed, to increase size
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 17001 93922
time,IIB_I_cell_no_uvw_total1 2 71589 0 0
264586 0 0
IIB_equal_cell_no_u1_max 865
size(I_cell_u),I_cell_no_max_cur 4752 4896
I_cells size exceed, to increase size
I_equal_cell_no_u1_max 7344
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 17001 93922
time,IIB_I_cell_no_uvw_total1 3 71237 0 0
264577 0 0
IIB_equal_cell_no_u1_max 865
I_equal_cell_no_u1_max 7344
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 17001 93922
time,IIB_I_cell_no_uvw_total1 4 70676 0 0
264504 0 0
IIB_equal_cell_no_u1_max 865
I_equal_cell_no_u1_max 7344
IIB_global_cell_no_u1_max,I_global_cell_no_u1_max 71637 396879
IIB_cell_no_u1_max,I_cell_no_u1_max 17001 93922
time,IIB_I_cell_no_uvw_total1 5 70909 0 0
264557 0 0
escape_time reached, so abort
Recommended no for final_IIB_equal_no1,final_IIB_no1,final_IIB_global_no1,final
_I_equal_no1,final_I_no1,final_I_global_no1
1081 21251 89546 9180 117402 496098
0.000000000000000E+000 3.700000000000000E-002 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.400000000000000 0 -400000
AB,AA,BB,CC -3.30400006193668 3.62600007723086
3.22400005813688 3.22400005813688
size_x,size_y,size_z 718x718x285
total grid size = 146924340
recommended cores (50k / core) = 2938.48680000000
min_area,max_area,min_grid_area,ratio 9.229464518072358E-005
0.108728370237650 4.000000000000000E-004 271.820925594124
ratio bet max_area,min_grid_area not ideal
max element length should be 2.828427124746190E-002
body_cg_ini -2.750916668409921E-003 2.695116216584833E-004
6.75873529067005
Warning - length difference between element and cell
max_element_length,min_element_length,min_delta
0.558213960965656 4.851715434749100E-003 2.000000000000000E-002
maximum ngh_surfaces and ngh_vertics are 4 3
minimum ngh_surfaces and ngh_vertics are 1 3
size(IIB_cell_u),size(I_cell_u),size(IIB_equal_cell_u),size(I_equal_cell_u),siz
e(IIB_global_cell_u),size(I_global_cell_u)
21251 117402 1081 9180 89546 496098
IIB_I_cell_no_uvw_total1 71637 71704 71732 264465
264616 264595
1 0.00150000 0.00000000 0.00000000 0.00000000 0.00000000E+00 0.00000000E+00 0.00000000E+00
escape_time reached, so abort
cd_cl_cs_mom_implicit1
-0.203534875831958 -3.78723081611176 -142.427697164364
3.49017516030323 -18.5098557775327 194.067816292935
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./a.out on a named std0864 with 960 processors, by tsltaywb Sat Apr 21 01:26:50 2018
Using Petsc Release Version 3.9.0, Apr, 07, 2018
Max Max/Min Avg Total
Time (sec): 1.168e+02 1.00008 1.168e+02
Objects: 8.780e+02 1.00114 8.770e+02
Flop: 1.506e+10 1.20490 1.414e+10 1.358e+13
Flop/sec: 1.289e+08 1.20492 1.211e+08 1.162e+11
MPI Messages: 1.469e+05 9.74171 7.679e+04 7.372e+07
MPI Message Lengths: 1.183e+09 2.27657 1.417e+04 1.044e+12
MPI Reductions: 1.547e+03 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 1.1681e+02 100.0% 1.3576e+13 100.0% 7.372e+07 100.0% 1.417e+04 100.0% 1.538e+03 99.4%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSided 5 1.0 1.2648e-02 7.1 0.00e+00 0.0 3.0e+04 4.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BuildTwoSidedF 69 1.0 1.2455e+00 4.1 0.00e+00 0.0 1.5e+05 1.6e+05 0.0e+00 0 0 0 2 0 0 0 0 2 0 0
VecDot 8 1.0 1.1479e-01 9.0 7.75e+06 1.1 0.0e+00 0.0e+00 8.0e+00 0 0 0 0 1 0 0 0 0 1 61436
VecDotNorm2 4 1.0 1.0597e-0111.8 7.75e+06 1.1 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 66552
VecMDot 100 1.0 2.2530e-01 3.0 3.91e+07 1.1 0.0e+00 0.0e+00 1.0e+02 0 0 0 0 6 0 0 0 0 7 157690
VecTDot 252 1.0 9.9791e-01 1.7 8.14e+07 1.1 0.0e+00 0.0e+00 2.5e+02 1 1 0 0 16 1 1 0 0 16 74204
VecNorm 249 1.0 3.8511e+00 4.9 5.79e+07 1.1 0.0e+00 0.0e+00 2.5e+02 2 0 0 0 16 2 0 0 0 16 13672
VecScale 110 1.0 2.6612e-03 1.8 3.91e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 1334996
VecCopy 682 1.0 1.4146e-01 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecSet 2470 1.0 7.4235e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 262 1.0 1.7395e-01 1.2 8.21e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 429416
VecAYPX 5362 1.0 6.6165e-01 2.3 2.72e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 373910
VecAXPBYCZ 2628 1.0 3.9758e-01 1.5 4.81e+08 1.1 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 1099671
VecWAXPY 8 1.0 3.3081e-02 1.7 7.75e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 213183
VecMAXPY 110 1.0 5.6687e-02 1.3 4.62e+07 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 740673
VecAssemblyBegin 35 1.0 1.7896e-01 3.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAssemblyEnd 35 1.0 6.3896e-05 1.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecPointwiseMult 55 1.0 1.2317e-02 3.1 1.95e+06 1.1 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 144216
VecScatterBegin 5531 1.0 1.0172e+00 2.6 0.00e+00 0.0 6.9e+07 1.4e+04 0.0e+00 1 0 93 94 0 1 0 93 94 0 0
VecScatterEnd 5531 1.0 9.3496e+00 2.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 4 0 0 0 0 0
VecSetRandom 5 1.0 4.4341e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecNormalize 110 1.0 7.5254e-02 1.7 1.17e+07 1.1 0.0e+00 0.0e+00 1.1e+02 0 0 0 0 7 0 0 0 0 7 141630
MatMult 4164 1.0 1.5945e+01 1.4 7.33e+09 1.3 5.9e+07 1.5e+04 0.0e+00 12 49 80 86 0 12 49 80 86 0 413057
MatMultAdd 655 1.0 3.4669e+00 5.3 3.82e+08 1.2 4.8e+06 3.8e+03 0.0e+00 2 3 6 2 0 2 3 6 2 0 99404
MatMultTranspose 655 1.0 1.7476e+00 3.1 3.82e+08 1.2 4.8e+06 3.8e+03 0.0e+00 1 3 6 2 0 1 3 6 2 0 197203
MatSolve 14311.9 6.1919e-01 1.3 4.20e+08 1.2 0.0e+00 0.0e+00 0.0e+00 0 3 0 0 0 0 3 0 0 0 615183
MatSOR 3985 1.0 1.3915e+01 1.5 5.00e+09 1.2 0.0e+00 0.0e+00 0.0e+00 11 33 0 0 0 11 33 0 0 0 324223
MatLUFactorSym 1 1.0 4.4298e-0464.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatLUFactorNum 2 1.0 3.3439e-01 1.4 1.55e+08 1.3 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 417768
MatILUFactorSym 1 1.0 1.6124e-01 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatConvert 5 1.0 3.8676e-02 1.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatScale 16 1.0 1.1365e-01 1.3 3.01e+07 1.2 7.2e+04 1.4e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 239293
MatResidual 655 1.0 2.7324e+00 1.8 1.08e+09 1.3 9.5e+06 1.4e+04 0.0e+00 2 7 13 13 0 2 7 13 13 0 355949
MatAssemblyBegin 106 1.0 1.1344e+00 2.3 0.00e+00 0.0 1.5e+05 1.6e+05 0.0e+00 1 0 0 2 0 1 0 0 2 0 0
MatAssemblyEnd 106 1.0 5.9141e+00 1.0 0.00e+00 0.0 9.2e+05 2.3e+03 2.3e+02 5 0 1 0 15 5 0 1 0 15 0
MatGetRow 533004 1.1 5.9415e-02 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetRowIJ 2 2.0 3.6955e-0538.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCreateSubMat 6 1.0 1.6832e+00 1.0 0.00e+00 0.0 1.3e+05 1.2e+03 9.6e+01 1 0 0 0 6 1 0 0 0 6 0
MatGetOrdering 2 2.0 9.6362e-03 1.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatCoarsen 5 1.0 9.3870e-02 1.1 0.00e+00 0.0 3.1e+06 1.8e+03 1.4e+02 0 0 4 1 9 0 0 4 1 9 0
MatZeroEntries 5 1.0 5.8362e-03 2.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAXPY 5 1.0 7.1930e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMatMult 5 1.0 1.8777e+00 1.0 8.28e+06 1.3 4.5e+05 6.8e+03 6.2e+01 2 0 1 0 4 2 0 1 0 4 3954
MatMatMultSym 5 1.0 1.8023e+00 1.0 0.00e+00 0.0 3.8e+05 5.5e+03 6.0e+01 2 0 1 0 4 2 0 1 0 4 0
MatMatMultNum 5 1.0 6.5588e-02 1.0 8.28e+06 1.3 7.2e+04 1.4e+04 0.0e+00 0 0 0 0 0 0 0 0 0 0 113198
MatPtAP 5 1.0 5.6372e+00 1.0 2.61e+08 1.6 1.0e+06 4.9e+04 7.7e+01 5 2 1 5 5 5 2 1 5 5 40626
MatPtAPSymbolic 5 1.0 1.2509e+00 1.0 0.00e+00 0.0 4.6e+05 5.7e+04 3.5e+01 1 0 1 3 2 1 0 1 3 2 0
MatPtAPNumeric 5 1.0 4.3676e+00 1.0 2.61e+08 1.6 5.7e+05 4.2e+04 4.0e+01 4 2 1 2 3 4 2 1 2 3 52436
MatTrnMatMult 1 1.0 6.8610e-01 1.0 1.64e+07 1.2 2.9e+04 1.3e+05 1.6e+01 1 0 0 0 1 1 0 0 0 1 21707
MatTrnMatMultSym 1 1.0 5.0253e-01 1.0 0.00e+00 0.0 2.6e+04 6.7e+04 1.6e+01 0 0 0 0 1 0 0 0 0 1 0
MatTrnMatMultNum 1 1.0 1.8408e-01 1.0 1.64e+07 1.2 3.7e+03 5.9e+05 0.0e+00 0 0 0 0 0 0 0 0 0 0 80908
MatGetLocalMat 17 1.0 4.8088e-02 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatGetBrAoCol 15 1.0 1.0984e-01 2.0 0.00e+00 0.0 5.1e+05 4.6e+04 0.0e+00 0 0 1 2 0 0 0 1 2 0 0
SFSetGraph 5 1.0 4.5300e-06 0.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFSetUp 5 1.0 1.6025e-02 2.4 0.00e+00 0.0 9.1e+04 4.7e+03 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 153 1.0 9.5887e-03 3.8 0.00e+00 0.0 3.0e+06 1.7e+03 0.0e+00 0 0 4 0 0 0 0 4 0 0 0
SFBcastEnd 153 1.0 1.4909e-02 2.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
KSPGMRESOrthog 100 1.0 2.6370e-01 2.2 7.82e+07 1.1 0.0e+00 0.0e+00 1.0e+02 0 1 0 0 6 0 1 0 0 7 269452
KSPSetUp 20 1.0 2.7623e-02 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 2.8e+01 0 0 0 0 2 0 0 0 0 2 0
KSPSolve 9 1.0 4.4800e+01 1.0 1.50e+10 1.2 7.3e+07 1.4e+04 1.3e+03 38100100 96 84 38100100 96 84 302653
PCGAMGGraph_AGG 5 1.0 3.6210e-01 1.0 8.28e+06 1.3 1.9e+05 6.8e+03 6.0e+01 0 0 0 0 4 0 0 0 0 4 20504
PCGAMGCoarse_AGG 5 1.0 8.4045e-01 1.0 1.64e+07 1.2 3.1e+06 4.0e+03 1.6e+02 1 0 4 1 11 1 0 4 1 11 17721
PCGAMGProl_AGG 5 1.0 1.0871e-01 1.0 0.00e+00 0.0 1.9e+05 9.9e+03 8.0e+01 0 0 0 0 5 0 0 0 0 5 0
PCGAMGPOpt_AGG 5 1.0 2.2764e+00 1.0 1.43e+08 1.2 1.2e+06 1.1e+04 2.1e+02 2 1 2 1 13 2 1 2 1 13 56612
GAMG: createProl 5 1.0 3.5835e+00 1.0 1.68e+08 1.2 4.7e+06 6.1e+03 5.1e+02 3 1 6 3 33 3 1 6 3 33 42190
Graph 10 1.0 3.5973e-01 1.0 8.28e+06 1.3 1.9e+05 6.8e+03 6.0e+01 0 0 0 0 4 0 0 0 0 4 20639
MIS/Agg 5 1.0 9.3908e-02 1.1 0.00e+00 0.0 3.1e+06 1.8e+03 1.4e+02 0 0 4 1 9 0 0 4 1 9 0
SA: col data 5 1.0 1.9782e-02 1.1 0.00e+00 0.0 1.2e+05 1.4e+04 2.0e+01 0 0 0 0 1 0 0 0 0 1 0
SA: frmProl0 5 1.0 8.2424e-02 1.0 0.00e+00 0.0 7.3e+04 3.0e+03 4.0e+01 0 0 0 0 3 0 0 0 0 3 0
SA: smooth 5 1.0 1.9676e+00 1.0 1.12e+07 1.3 4.5e+05 6.8e+03 7.2e+01 2 0 1 0 5 2 0 1 0 5 5110
GAMG: partLevel 5 1.0 7.4519e+00 1.0 2.61e+08 1.6 1.2e+06 4.4e+04 2.3e+02 6 2 2 5 15 6 2 2 5 15 30733
repartition 3 1.0 3.1521e-03 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 1.8e+01 0 0 0 0 1 0 0 0 0 1 0
Invert-Sort 3 1.0 1.2159e-0116.6 0.00e+00 0.0 0.0e+00 0.0e+00 1.2e+01 0 0 0 0 1 0 0 0 0 1 0
Move A 3 1.0 6.3250e-01 1.0 0.00e+00 0.0 8.8e+04 1.8e+03 5.1e+01 1 0 0 0 3 1 0 0 0 3 0
Move P 3 1.0 1.0553e+00 1.0 0.00e+00 0.0 4.4e+04 6.4e+01 5.1e+01 1 0 0 0 3 1 0 0 0 3 0
PCSetUp 4 1.0 1.1538e+01 1.0 5.84e+08 1.4 5.8e+06 1.4e+04 7.7e+02 10 4 8 8 50 10 4 8 8 50 45060
PCSetUpOnBlocks 135 1.0 4.9708e-01 1.3 1.55e+08 1.3 0.0e+00 0.0e+00 0.0e+00 0 1 0 0 0 0 1 0 0 0 281031
PCApply 143 1.0 3.0010e+01 1.1 1.34e+10 1.2 6.7e+07 1.2e+04 1.0e+02 25 89 91 80 7 25 89 91 80 7 401277
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Vector 369 369 141361368 0.
Matrix 158 158 768281328 0.
Matrix Coarsen 5 5 3180 0.
Distributed Mesh 18 18 94176 0.
Index Set 144 144 15496224 0.
IS L to G Mapping 18 18 6803624 0.
Star Forest Graph 41 41 35424 0.
Discrete System 18 18 16632 0.
Vec Scatter 61 61 10664952 0.
Krylov Solver 20 20 314880 0.
Preconditioner 15 15 15212 0.
PetscRandom 10 10 6460 0.
Viewer 1 0 0 0.
========================================================================================================================
Average time to get PetscTime(): 0.
Average time for MPI_Barrier(): 1.81675e-05
Average time for zero size MPI_Send(): 4.80612e-06
#PETSc Option Table entries:
-log_view
-poisson_pc_gamg_agg_nsmooths 1
-poisson_pc_type gamg
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --with-mpi-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64 --with-blaslapack-dir=/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/nus/tsltaywb/scratch/source/v2.14.0.tar.gz --download-ml=/home/users/nus/tsltaywb/scratch/source/ml-v6.2-p4.tar.gz --with-debugging=0 --prefix=/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel --with-shared-libraries=0 --known-mpi-shared-libraries=0 --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g -O2" --FFLAGS="-xHost -g -O2"
-----------------------------------------
Libraries compiled on 2018-04-17 05:50:35 on nus02
Machine characteristics: Linux-2.6.32-696.18.7.el6.x86_64-x86_64-with-redhat-6.9-Santiago
Using PETSc directory: /home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel
Using PETSc arch:
-----------------------------------------
Using C compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc -xHost -g -O2
Using Fortran compiler: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90 -xHost -g -O2
-----------------------------------------
Using include paths: -I/home/users/nus/tsltaywb/lib//include -I/home/users/nus/tsltaywb/lib///include -I/home/users/nus/tsltaywb/lib//include -I/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/include
-----------------------------------------
Using C linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpicc
Using Fortran linker: /app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/bin/mpif90
Using libraries: -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -lpetsc -Wl,-rpath,/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -L/home/users/nus/tsltaywb/lib/petsc-3.9.0_intel_rel/lib -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ippcp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/ipp/lib/intel64 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/compiler/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mkl/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64/gcc4.4 -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/daal/lib/intel64_lin -L/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/tbb/lib/intel64_lin/gcc4.4 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib/debug_mt -Wl,-rpath,/app/intel/xe2018/compilers_and_libraries_2018.0.128/linux/mpi/intel64/lib -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lHYPRE -lml -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lX11 -lstdc++ -ldl -lmpifort -lmpi -lmpigi -lrt -lpthread -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lstdc++ -ldl
-----------------------------------------
======================================================================================
Resource Usage on 2018-04-21 01:26:55.164851:
JobId: 6808435.wlm01
Project: 11000324
Exit Status: 0
NCPUs Requested: 960 NCPUs Used: 960
CPU Time Used: 00:45:58
Memory Requested: 3840gb Memory Used: 38583332kb
Vmem Used: 60953164kb
Walltime requested: 00:05:00 Walltime Used: 00:02:12
Execution Nodes Used: (std0864:ncpus=24:mem=100663296kb)+(std1142:ncpus=24:mem=100663296kb)+(std1143:ncpus=24:mem=100663296kb)+(std1144:ncpus=24:mem=100663296kb)+(std1146:ncpus=24:mem=100663296kb)+(std1147:ncpus=24:mem=100663296kb)+(std1150:ncpus=24:mem=100663296kb)+(std1151:ncpus=24:mem=100663296kb)+(std1152:ncpus=24:mem=100663296kb)+(std1153:ncpus=24:mem=100663296kb)+(std1204:ncpus=24:mem=100663296kb)+(std1206:ncpus=24:mem=100663296kb)+(std1207:ncpus=24:mem=100663296kb)+(std1211:ncpus=24:mem=100663296kb)+(std1213:ncpus=24:mem=100663296kb)+(std1214:ncpus=24:mem=100663296kb)+(std1216:ncpus=24:mem=100663296kb)+(std1217:ncpus=24:mem=100663296kb)+(std1218:ncpus=24:mem=100663296kb)+(std1220:ncpus=24:mem=100663296kb)+(std1222:ncpus=24:mem=100663296kb)+(std1225:ncpus=24:mem=100663296kb)+(std1226:ncpus=24:mem=100663296kb)+(std1227:ncpus=24:mem=100663296kb)+(std1228:ncpus=24:mem=100663296kb)+(std1229:ncpus=24:mem=100663296kb)+(std1230:ncpus=24:mem=100663296kb)+(std1231:ncpus=24:mem=100663296kb)+(std1232:ncpus=24:mem=100663296kb)+(std1233:ncpus=24:mem=100663296kb)+(std1234:ncpus=24:mem=100663296kb)+(std1235:ncpus=24:mem=100663296kb)+(std1236:ncpus=24:mem=100663296kb)+(std1237:ncpus=24:mem=100663296kb)+(std1238:ncpus=24:mem=100663296kb)+(std1270:ncpus=24:mem=100663296kb)+(std1271:ncpus=24:mem=100663296kb)+(std1272:ncpus=24:mem=100663296kb)+(std1319:ncpus=24:mem=100663296kb)+(std1320:ncpus=24:mem=100663296kb)
======================================================================================
More information about the petsc-users
mailing list