[petsc-users] Fwd: direct solvers on KNL

[Jakub Kruzik]

Hi,

I am looking at a single node performance of MUMPS and SuperLU on KNL 
7230 (on Theta). I am using KSP example ex2 
(http://www.mcs.anl.gov/petsc/petsc-current/src/ksp/ksp/examples/tutorials/ex2.c.html) 
with m X n = 2880 x 2880. KNL runs in cache and quad modes.

Times in seconds for 24 cores:
mumps:   279
superlu:   326
cg:             116

Times in seconds for 64 cores:
mumps:   316
superlu:   410
cg :            49

The performance for 24 cores is OK - both direct solvers are roughly 3.5 
times slower than 2x E5-2680v3. (According to people from Intel, the 
single core performance of KNL is about 3-4 times lower than that of 
E5-2680v3). However, strong scalability is really bad.

I am using cray-petsc/3.7.6.0 module. I tried my own PETSc compilation 
with MKL and MUMPS/SuperLU installed by PETSc configure but the results 
are similar.

Please find attached Theta submission script and logs for KNL and Haswells.

Why the performance of direct solvers on a full node is so bad?

Best,
Jakub
-------------- next part --------------
A non-text attachment was scrubbed...
Name: batch.sub
Type: text/x-microdvd
Size: 2517 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170901/41acc4ac/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: knl.log
Type: text/x-log
Size: 64418 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170901/41acc4ac/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: haswell.log
Type: text/x-log
Size: 53537 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170901/41acc4ac/attachment-0005.bin>