<html><head><title>Fwd: [petsc-users] strange PETSc/KSP GMRES timings for MPI+OMP configuration on KNLs</title>
</head>
<body>
<span style=" font-family:'courier new'; font-size: 9pt;">Hi,<br>
<br>
Regarding my previous post <br>
I looked into both logs of 64MPI/1 OMP vs. 64MPI/2 OMP.<br>
<br>
<br>
<span style=" font-family:'Tahoma'; font-size: 10pt;">What attracted my attention is huge difference in MPI timings in the following places:<br>
<br>
Average time to get PetscTime(): 2.14577e-07<br>
Average time for MPI_Barrier(): 3.9196e-05<br>
Average time for zero size MPI_Send(): 5.45382e-06<br>
<br>
vs. <br>
<br>
Average time to get PetscTime(): 4.05312e-07<br>
Average time for MPI_Barrier(): 0.348399<br>
Average time for zero size MPI_Send(): 0.029937<br>
<br>
<br>
<br>
Isn't something wrong with PETSc library itself?...<br>
<br>
<br>
Best,<br>
Damian<br>
<br>
<span style=" font-family:'courier new'; font-size: 9pt;">Wiadomość przekazana<br>
Od: Damian Kaliszan <damian@man.poznan.pl><br>
Do: PETSc users list <petsc-users@mcs.anl.gov><br>
Data: 16 czerwca 2017, 14:57:10<br>
Temat: [petsc-users] strange PETSc/KSP GMRES timings for MPI+OMP configuration on KNLs<br>
<br>
===8<===============Treść oryginalnej wiadomości===============<br>
<span style=" font-family:'Tahoma'; font-size: 10pt;">Hi,<br>
<br>
For several days I've been trying to figure out what is going wrong<br>
with my python app timings solving Ax=b with KSP (GMRES) solver when trying to run on Intel's KNL 7210/7230.<br>
<br>
I downsized the problem to 1000x1000 A matrix and a single node and<br>
observed the following:<br>
<br>
<img width=482 height=260 alt="" style="padding : 1px;" src="cid:7753EFE8.01D2E8F9.4179B2C6.59CB5BE5_csseditor"><br>
I'm attaching 2 extreme timings where configurations differ only by 1 OMP thread (64MPI/1 OMP vs 64/2 OMPs),<br>
23321 vs 23325 slurm task ids.<br>
<br>
Any help will be appreciated....<br>
<br>
Best,<br>
Damian <br>
<br>
===8<===========Koniec treści oryginalnej wiadomości===========<br>
<br>
<br>
<br>
-------------------------------------------------------<br>
Damian Kaliszan<br>
<br>
Poznan Supercomputing and Networking Center<br>
HPC and Data Centres Technologies<br>
ul. Jana Pawła II 10<br>
61-139 Poznan<br>
POLAND<br>
<br>
phone (+48 61) 858 5109<br>
e-mail </span></span></span></span><a style=" font-family:'courier new'; font-size: 9pt;" href="mailto:damian@man.poznan.pl">damian@man.poznan.pl</a><br>
<span style=" font-family:'courier new'; font-size: 9pt;">www - </span><a style=" font-family:'courier new'; font-size: 9pt;" href="http://www.man.poznan.pl/">http://www.man.poznan.pl/</a><br>
<span style=" font-family:'courier new'; font-size: 9pt;">------------------------------------------------------- </body></html>