[petsc-users] Direct PETSc to use MCDRAM on KNL and other optimizations for KNL

[Zhang, Junchao]

Use srun <srun options> numactl -m 1 ./app OR srun <srun options> numactl -p 1 ./app
See bottom of https://www.nersc.gov/users/computational-systems/cori/configuration/knl-processor-modes/

--Junchao Zhang


On Wed, Feb 27, 2019 at 4:16 PM Sajid Ali via petsc-users <petsc-users at mcs.anl.gov<mailto:petsc-users at mcs.anl.gov>> wrote:
Hi,

I ran a TS integrator for 25 steps on a Broadwell-Xeon and Xeon-Phi (KNL). The problem size is 5000x5000 and I'm using scalar=complex.

The program takes 125 seconds to run on Xeon and 451 seconds on KNL !

The first thing I want to change is to convert the memory access for the program on KNL from DRAM to MCDRAM. I did run the problem in an interactive SLURM job and specified -C quad,flat and yet I see DRAM is being used.

I'm attaching the PETSc log files and Intel APS reports as well. Any help on how I should change my runtime parameters on KNL will be highly appreciated. Thanks in advance.

--
Sajid Ali
Applied Physics
Northwestern University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190227/29b4f01d/attachment-0001.html>