[petsc-users] Direct PETSc to use MCDRAM on KNL and other optimizations for KNL

[Zhang, Junchao]

On Wed, Feb 27, 2019 at 7:03 PM Sajid Ali <sajidsyed2021 at u.northwestern.edu<mailto:sajidsyed2021 at u.northwestern.edu>> wrote:

Hi Junchao,

I’m confused with the syntax. If I submit the following as my job script, I get an error :

#!/bin/bash
#SBATCH --job-name=petsc_test
#SBATCH -N 1
#SBATCH -C knl,quad,flat
#SBATCH -p apsxrmd
#SBATCH --time=1:00:00

module load intel/18.0.3-d6gtsxs
module load intel-parallel-studio/cluster.2018.3-xvnfrfz
module load numactl-2.0.12-intel-18.0.3-wh44iog
srun -n 64 -c 64 --cpu_bind=cores numactl -m 1 aps ./ex_modify -ts_type cn -prop_steps 25 -pc_type gamg -ts_monitor -log_view


The error is :
srun: cluster configuration lacks support for cpu binding

This cluster does not support cpu binding.  You need to remove --cpu_bind=cores. In addition, I don't know what is the 'aps' argument


srun: error: Unable to create step for job 916208: More processors requested than permitted

I remember the product of -n -c has to be 256.  You can try srun -n 64 -c 4 numactl -m 1 ./ex_modify ...


I’m following the advice as given at slide 33 of https://www.nersc.gov/assets/Uploads/02-using-cori-knl-nodes-20170609.pdf

For further info, I’m using LCRC at ANL.

Thank You,
Sajid Ali
Applied Physics
Northwestern University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190228/55b97d1d/attachment.html>