[petsc-users] Communication during MatAssemblyEnd
Ale Foggia
amfoggia at gmail.com
Fri Jun 21 09:27:08 CDT 2019
I'm sending one with a bit less time.
I'm timing the functions also with std::chronos and for the case of 180
seconds the program runs out of memory (and crushes) before the PETSc log
gets to be printed, so I know the time only from my function. Anyway, in
every case, the times between std::chronos and the PETSc log match.
(The large times are in part "4b- Building offdiagonal part" or "Event
Stage 5: Offdiag").
El vie., 21 jun. 2019 a las 16:09, Zhang, Junchao (<jczhang at mcs.anl.gov>)
escribió:
>
>
> On Fri, Jun 21, 2019 at 8:07 AM Ale Foggia <amfoggia at gmail.com> wrote:
>
>> Thanks both of you for your answers,
>>
>> El jue., 20 jun. 2019 a las 22:20, Smith, Barry F. (<bsmith at mcs.anl.gov>)
>> escribió:
>>
>>>
>>> Note that this is a one time cost if the nonzero structure of the
>>> matrix stays the same. It will not happen in future MatAssemblies.
>>>
>>> > On Jun 20, 2019, at 3:16 PM, Zhang, Junchao via petsc-users <
>>> petsc-users at mcs.anl.gov> wrote:
>>> >
>>> > Those messages were used to build MatMult communication pattern for
>>> the matrix. They were not part of the matrix entries-passing you imagined,
>>> but indeed happened in MatAssemblyEnd. If you want to make sure processors
>>> do not set remote entries, you can use
>>> MatSetOption(A,MAT_NO_OFF_PROC_ENTRIES,PETSC_TRUE), which will generate an
>>> error when an off-proc entry is set.
>>>
>>
>> I started being concerned about this when I saw that the assembly was
>> taking a few hundreds of seconds in my code, like 180 seconds, which for me
>> it's a considerable time. Do you think (or maybe you need more information
>> to answer this) that this time is "reasonable" for communicating the
>> pattern for the matrix? I already checked that I'm not setting any remote
>> entries.
>>
> It is not reasonable. Could you send log view of that test with 180
> seconds MatAssembly?
>
>
>> Also I see (in my code) that even if there are no messages being passed
>> during MatAssemblyBegin, it is taking time and the "ratio" is very big.
>>
>> >
>>> >
>>> > --Junchao Zhang
>>> >
>>> >
>>> > On Thu, Jun 20, 2019 at 4:13 AM Ale Foggia via petsc-users <
>>> petsc-users at mcs.anl.gov> wrote:
>>> > Hello all!
>>> >
>>> > During the conference I showed you a problem happening during
>>> MatAssemblyEnd in a particular code that I have. Now, I tried the same with
>>> a simple code (a symmetric problem corresponding to the Laplacian operator
>>> in 1D, from the SLEPc Hands-On exercises). As I understand (and please,
>>> correct me if I'm wrong), in this case the elements of the matrix are
>>> computed locally by each process so there should not be any communication
>>> during the assembly. However, in the log I get that there are messages
>>> being passed. Also, the number of messages changes with the number of
>>> processes used and the size of the matrix. Could you please help me
>>> understand this?
>>> >
>>> > I attach the code I used and the log I get for a small problem.
>>> >
>>> > Cheers,
>>> > Ale
>>> >
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190621/0686d080/attachment-0001.html>
-------------- next part --------------
srun -n 32768 home/build/main.x 38 -nn -j1 1.0 -d1 1.0 -eps_type krylovschur -eps_tol 1e-9 -log_view
------ Parameters ------
J1: 1.00000
D1: 1.00000
J2: 0.00000
D2: 0.00000
disorder?: 0
------------------------
1- Creating basis
* Basis: 7.916039e-02 -- 7.916039e-02 -- 1
2- Creating lattice
* Lattice: 4.789100e-05 -- 4.789100e-05 -- 1
3- Creating Hamiltonian
* prealloc-J1: 4.108500e-05 -- 2.892727e+01 -- 1078652
* PetscPreallocation: 6.085745e+01 -- 6.085745e+01 -- 1
* prealloc: 9.314486e+01 -- 9.314486e+01 -- 1
* Miscellaneous: 1.581486e-01 -- 1.581486e-01 -- 1
* MatZeroEntries: 9.806940e-02 -- 9.806940e-02 -- 1
* MatAssembly-init: 2.934797e-01 -- 2.934797e-01 -- 1
* MatInit: 9.369477e+01 -- 9.369477e+01 -- 1
4a- Building diagonal part
* get_coup_elems-d-nn: 3.428000e-06 -- 3.751426e+00 -- 1078652
* MatSetValues-diag-nn: 2.496000e-06 -- 2.741947e+00 -- 1078652
* build_diag: 1.269635e+01 -- 1.269635e+01 -- 1
4b- Building offdiagonal part
* get_coup_elems-o-nn: 4.094500e-05 -- 2.944290e+01 -- 1078652
* MatSetValues-off-nn: 4.119000e-06 -- 4.000079e+00 -- 1078652
* MatAssembly-offdiag: 1.253235e+02 -- 1.253235e+02 -- 1
* build_off_diag: 1.657913e+02 -- 1.657913e+02 -- 1
iter k ||Ax-kx||/||kx||
---- ------------- ------------------
* EPSSolve: 8.045663e+02 -- 8.045663e+02 -- 1
0 -16.861318 4.88671e-10
* Get Eigen: 4.278705e+01 -- 4.278705e+01 -- 1
8- Phys quantities
* EPSSolve: 4.499570e+03 -- 5.304137e+03 -- 2
* CoeffDSF: 1.979352e+01 -- 8.826874e+02 -- 38
* SolverDSF: 4.499571e+03 -- 4.499571e+03 -- 1
* DynStructFactor: 5.404416e+03 -- 5.404416e+03 -- 1
9- Cleaning
===============================================================================
----------------------------- Execution Times ---------------------------------
Function: Current ET Total ET Calls
* Basis: 7.916039e-02 7.916039e-02 1
* CoeffDSF: 1.979352e+01 8.826874e+02 38
* DynStructFactor: 5.404416e+03 5.404416e+03 1
* EPSSolve: 4.499570e+03 5.304137e+03 2
* Get Eigen: 4.278705e+01 4.278705e+01 1
* Lattice: 4.789100e-05 4.789100e-05 1
* MatAssembly-init: 2.934797e-01 2.934797e-01 1
* MatAssembly-offdiag: 1.253235e+02 1.253235e+02 1
* MatInit: 9.369477e+01 9.369477e+01 1
* MatSetValues-diag-nn: 2.496000e-06 2.741947e+00 1078652
* MatSetValues-off-nn: 4.119000e-06 4.000079e+00 1078652
* MatZeroEntries: 9.806940e-02 9.806940e-02 1
* Miscellaneous: 1.581486e-01 1.581486e-01 1
* PetscPreallocation: 6.085745e+01 6.085745e+01 1
* SolverDSF: 4.499571e+03 4.499571e+03 1
* SqzConstructor: 4.669800e-05 2.305247e-03 38
* SqzOpOnStateVector: 5.694926e-01 2.166454e+01 38
* build_diag: 1.269635e+01 1.269635e+01 1
* build_off_diag: 1.657913e+02 1.657913e+02 1
* get_coup_elems-d-nn: 3.428000e-06 3.751426e+00 1078652
* get_coup_elems-o-nn: 4.094500e-05 2.944290e+01 1078652
* prealloc: 9.314486e+01 9.314486e+01 1
* prealloc-J1: 4.108500e-05 2.892727e+01 1078652
===============================================================================
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
home/build/main.x on a named r065c17s01 with 32768 processors, by a Wed May 29 11:31:23 2019
Using Petsc Release Version 3.10.2, Oct, 09, 2018
Max Max/Min Avg Total
Time (sec): 6.527e+03 1.000 6.527e+03
Objects: 5.500e+01 1.000 5.500e+01
Flop: 7.303e+11 1.207 6.692e+11 2.193e+16
Flop/sec: 1.119e+08 1.207 1.025e+08 3.360e+12
MPI Messages: 3.261e+04 6.200 1.907e+04 6.248e+08
MPI Message Lengths: 2.693e+11 10.298 7.757e+06 4.847e+15
MPI Reductions: 1.944e+03 1.000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flop
and VecAXPY() for complex vectors of length N --> 8N flop
Summary of Stages: ----- Time ------ ----- Flop ------ --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total Count %Total Avg %Total Count %Total
0: Main Stage: 8.5045e+02 13.0% 3.0633e+15 14.0% 8.196e+07 13.1% 7.768e+06 13.1% 2.530e+02 13.0%
1: Create Basis: 7.9234e-02 0.0% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0%
2: Create Lattice: 8.3944e-05 0.0% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0%
3: Create Hamilt: 1.0694e+02 1.6% 0.0000e+00 0.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.000e+00 0.1%
5: Offdiag: 1.6525e+02 2.5% 0.0000e+00 0.0% 1.188e+06 0.2% 1.942e+06 0.0% 8.000e+00 0.4%
6: Phys quantities: 5.4045e+03 82.8% 1.8866e+16 86.0% 5.417e+08 86.7% 7.768e+06 86.8% 1.674e+03 86.1%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flop: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
AvgLen: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flop in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flop over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flop --- Global --- --- Stage ---- Total
Max Ratio Max Ratio Max Ratio Mess AvgLen Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
BuildTwoSidedF 1 1.0 1.6722e-0241.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatMult 138 1.0 2.6604e+02 7.4 3.19e+10 2.1 8.2e+07 7.8e+06 0.0e+00 2 4 13 13 0 15 25100100 0 2935476
MatAssemblyBegin 1 1.0 1.6807e-0236.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 1 1.0 3.5680e-01 3.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecNorm 2 1.0 4.4252e+0174.8 1.73e+07 1.0 0.0e+00 0.0e+00 2.0e+00 1 0 0 0 0 5 0 0 0 1 12780
VecCopy 6 1.0 6.5655e-02 2.6 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 2 1.0 1.3793e-02 2.7 1.73e+07 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 41000838
VecScatterBegin 138 1.0 1.1653e+0285.8 0.00e+00 0.0 8.2e+07 7.8e+06 0.0e+00 1 0 13 13 0 4 0100100 0 0
VecScatterEnd 138 1.0 1.3653e+0222.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 4 0 0 0 0 0
VecSetRandom 1 1.0 9.6668e-01 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
EPSSetUp 1 1.0 8.0236e-01 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 2 0
EPSSolve 1 1.0 8.0484e+02 1.0 1.01e+11 1.2 8.1e+07 7.8e+06 2.5e+02 12 14 13 13 13 95100 99 99 98 3790582
STSetUp 1 1.0 7.0133e-022195.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
STApply 136 1.0 2.6264e+02 7.4 3.14e+10 2.1 8.1e+07 7.8e+06 0.0e+00 2 4 13 13 0 15 25 99 99 0 2930393
BVCopy 19 1.0 1.3275e-01 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BVMultVec 243 1.0 1.4706e+01 1.7 2.51e+10 1.0 0.0e+00 0.0e+00 0.0e+00 0 4 0 0 0 1 27 0 0 0 55951391
BVMultInPlace 16 1.0 4.3480e+00 1.6 1.67e+10 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 18 0 0 0 125903565
BVDotVec 243 1.0 7.5430e+02 1.4 2.72e+10 1.0 0.0e+00 0.0e+00 2.4e+02 10 4 0 0 12 78 29 0 0 96 1181949
BVOrthogonalizeV 137 1.0 7.6479e+02 1.4 5.23e+10 1.0 0.0e+00 0.0e+00 2.4e+02 10 8 0 0 12 79 56 0 0 96 2241651
BVScale 137 1.0 7.0904e-01 1.8 5.91e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 27317472
BVSetRandom 1 1.0 9.6671e-01 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSSolve 16 1.0 2.9756e-0115.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSVectors 17 1.0 2.8753e-04 3.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSOther 47 1.0 1.6295e-0270.5 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 1: Create Basis
--- Event Stage 2: Create Lattice
--- Event Stage 3: Create Hamilt
BuildTwoSidedF 1 1.0 2.9318e-01151.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyBegin 1 1.0 2.9336e-01123.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 1 1.0 4.1753e-021018.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatZeroEntries 1 1.0 2.6666e-01 2.8 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 4: Unknown
--- Event Stage 5: Offdiag
BuildTwoSidedF 1 1.0 7.1565e+01148448.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 28 0 0 0 0 0
MatAssemblyBegin 1 1.0 7.1565e+01127783.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 1 0 0 0 0 28 0 0 0 0 0
MatAssemblyEnd 1 1.0 5.3762e+01 1.0 0.00e+00 0.0 1.2e+06 1.9e+06 8.0e+00 1 0 0 0 0 33 0100100100 0
VecSet 1 1.0 7.5533e-02 9.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
--- Event Stage 6: Phys quantities
MatMult 912 1.0 1.5033e+03 6.2 2.11e+11 2.1 5.4e+08 7.8e+06 0.0e+00 13 24 87 87 0 16 27100100 0 3433060
VecDot 380 1.0 1.2952e+01 1.0 3.28e+09 1.0 0.0e+00 0.0e+00 3.8e+02 0 0 0 0 20 0 1 0 0 23 8295974
VecNorm 380 1.0 7.6332e+02 3.0 3.28e+09 1.0 0.0e+00 0.0e+00 3.8e+02 8 0 0 0 20 9 1 0 0 23 140766
VecScale 38 1.0 1.7310e-01 1.5 1.64e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 31036816
VecCopy 1178 1.0 8.6691e+00 1.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 380 1.0 2.3530e+00 2.4 3.28e+09 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 1 0 0 0 45664922
VecScatterBegin 912 1.0 7.4409e+0274.9 0.00e+00 0.0 5.4e+08 7.8e+06 0.0e+00 3 0 87 87 0 4 0100100 0 0
VecScatterEnd 912 1.0 5.7096e+0212.9 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 4 0 0 0 0 0
VecSetRandom 1 1.0 1.0213e+00 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
EPSSetUp 1 1.0 5.4395e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00 0 0 0 0 0 0 0 0 0 0 0
EPSSolve 1 1.0 4.5000e+03 1.0 5.31e+11 1.1 3.2e+08 7.8e+06 9.1e+02 69 75 51 51 47 83 87 58 58 54 3641650
STSetUp 1 1.0 1.0705e-0412.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
STApply 532 1.0 9.1435e+02 6.8 1.23e+11 2.1 3.2e+08 7.8e+06 0.0e+00 8 14 51 51 0 9 16 58 58 0 3292645
BVCopy 811 1.0 5.3793e+00 1.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
BVMultVec 907 1.0 9.5485e+01 1.4 1.56e+11 1.0 0.0e+00 0.0e+00 0.0e+00 1 23 0 0 0 1 27 0 0 0 53641657
BVMultInPlace 52 1.0 2.0483e+01 1.5 8.55e+10 1.0 0.0e+00 0.0e+00 0.0e+00 0 13 0 0 0 0 15 0 0 0 136750118
BVDotVec 907 1.0 4.2638e+03 1.2 1.64e+11 1.0 0.0e+00 0.0e+00 9.1e+02 60 25 0 0 47 72 29 0 0 54 1261411
BVOrthogonalizeV 533 1.0 4.3471e+03 1.2 3.20e+11 1.0 0.0e+00 0.0e+00 9.1e+02 61 48 0 0 47 74 56 0 0 54 2415474
BVScale 533 1.0 2.6414e+00 1.6 2.30e+09 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 28528560
BVSetRandom 1 1.0 1.0213e+00 2.3 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSSolve 52 1.0 1.5004e-0110.1 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSVectors 64 1.0 7.6532e-04 2.4 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DSOther 155 1.0 1.9093e-03 2.7 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Matrix 2 5 446597144 0.
Vector 10 11 404606488 0.
Vec Scatter 0 1 1456 0.
EPS Solver 1 1 2456 0.
Spectral Transform 1 1 904 0.
Basis Vectors 1 1 9440 0.
PetscRandom 1 1 710 0.
Region 1 1 720 0.
Direct Solver 1 1 16176 0.
Krylov Solver 1 1 1520 0.
Preconditioner 1 1 1056 0.
Viewer 1 0 0 0.
--- Event Stage 1: Create Basis
--- Event Stage 2: Create Lattice
--- Event Stage 3: Create Hamilt
Matrix 3 0 0 0.
--- Event Stage 4: Unknown
--- Event Stage 5: Offdiag
Vector 2 1 1792 0.
Index Set 2 2 12450096 0.
Vec Scatter 1 0 0 0.
--- Event Stage 6: Phys quantities
Matrix 7 7 66696 0.
Vector 11 11 552300352 0.
EPS Solver 1 1 2960 0.
Spectral Transform 1 1 904 0.
Basis Vectors 1 1 14560 0.
PetscRandom 1 1 710 0.
Region 1 1 720 0.
Direct Solver 1 1 35184 0.
Krylov Solver 1 1 1520 0.
Preconditioner 1 1 1056 0.
========================================================================================================================
Average time to get PetscTime(): 1.90735e-07
Average time for MPI_Barrier(): 0.000459194
Average time for zero size MPI_Send(): 3.11734e-05
#PETSc Option Table entries:
-d1 1.0
-eps_tol 1e-9
-eps_type krylovschur
-j1 1.0
-log_view
-nn
#End of PETSc Option Table entries
Compiled with FORTRAN kernels
Compiled with 64 bit PetscInt
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 16 sizeof(PetscInt) 8
Configure options: --known-level1-dcache-size=32768 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-int64_t=1 --known-mpi-c-double-complex=1 --known-has-attribute-aligned=1 --prefix=/machine_work/IscrC_SkMagn/opt/lib/petsc/3.10.2/prod-intel-mkl_seq-avx512-64bit-double-complex --with-precision=double --with-scalar-type=complex --with-64-bit-indices=1 --with-shared-libraries=1 --with-avx512-kernels=1 --with-memalign=64 --CC=mpiicc --CXX=mpiicpc --FC=mpiifort --F90=mpiifort --F77=mpiifort --COPTFLAGS="-O3 -g" --CXXOPTFLAGS="-O3 -g" --FOPTFLAGS="-O3 -g" --CFLAGS="-DMKL_ILP64 -I/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/include" --CXXFLAGS="-DMKL_ILP64 -I/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/include" --FFLAGS="-DMKL_ILP64 -I/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/include" --with-debugging=0 --with-mpi=1 --with-mpi-compilers=1 --with-default-arch=0 --with-blaslapack=1 --with-blaslapack-pkg-config=/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/bin/pkgconfig/mkl-dynamic-ilp64-seq.pc --with-valgrind=1 --PETSC_ARCH=prod-intel-mkl_seq-avx512-64bit-double-complex -with-batch=1 --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=1 --CXX_LINKER_FLAGS="-L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -ldl" --CC_LINKER_FLAGS="-L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -ldl" --FC_LINKER_FLAGS="-L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/lib/intel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -ldl" --with-fortran-kernels=1
-----------------------------------------
Libraries compiled on 2018-12-02 13:09:11 on r000u11l06
Machine characteristics: Linux-3.10.0-327.36.3.el7.x86_64-x86_64-with-centos-7.2.1511-Core
Using PETSc directory: /machine_work/IscrC_SkMagn/opt/lib/petsc/3.10.2/prod-intel-mkl_seq-avx512-64bit-double-complex
Using PETSc arch:
-----------------------------------------
Using C compiler: mpiicc -DMKL_ILP64 -I/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/include -fPIC -O3 -g
Using Fortran compiler: mpiifort -DMKL_ILP64 -I/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/include -fPIC -O3 -g
-----------------------------------------
Using include paths: -I/machine_work/IscrC_SkMagn/opt/lib/petsc/3.10.2/prod-intel-mkl_seq-avx512-64bit-double-complex/include
-----------------------------------------
Using C linker: mpiicc
Using Fortran linker: mpiifort
Using libraries: -Wl,-rpath,/machine_work/IscrC_SkMagn/opt/lib/petsc/3.10.2/prod-intel-mkl_seq-avx512-64bit-double-complex/lib -L/machine_work/IscrC_SkMagn/opt/lib/petsc/3.10.2/prod-intel-mkl_seq-avx512-64bit-double-complex/lib -lpetsc -Wl,-rpath,/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/lib/intel64 -L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/mkl/lib/intel64 -Wl,-rpath,/facility/prod/opt/compilers/intel/pe-xe-2018/binary/impi/2018.4.274/intel64/lib/debug_mt -L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/impi/2018.4.274/intel64/lib/debug_mt -Wl,-rpath,/facility/prod/opt/compilers/intel/pe-xe-2018/binary/impi/2018.4.274/intel64/lib -L/facility/prod/opt/compilers/intel/pe-xe-2018/binary/impi/2018.4.274/intel64/lib -Wl,-rpath,/machine/prod/opt/compilers/intel/pe-xe-2018/binary/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin -L/machine/prod/opt/compilers/intel/pe-xe-2018/binary/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib/debug_mt -Wl,-rpath,/opt/intel/mpi-rt/2017.0.0/intel64/lib -lX11 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lpthread -ldl -lstdc++ -lmpifort -lmpi -lmpigi -lrt -lifport -lifcoremt_pic -limf -lsvml -lm -lipgo -lirc -lgcc_s -lirc_s -lstdc++ -ldl
-----------------------------------------
More information about the petsc-users
mailing list