[petsc-dev] error configuring on KNL

Satish Balay balay at mcs.anl.gov
Fri Jun 30 17:27:16 CDT 2017

On Fri, 30 Jun 2017, Richard Tran Mills wrote:

> Hmm. I get the same error when adding "--download-hypre=0" to an otherwise
> functional configure script for Cori KNL. Hypre's configure doesn't like
> something about cross-compiling for KNL. It might be possible to make this
> work by building on the compute nodes, though this is, unfortunately, not
> officially supported on Cori -- NERSC staff say that builds are not
> guaranteed to work on the compute nodes, though I've managed to make it
> work sometimes.

Here is (a part of a prior email) regarding installing hypre on theta (using compute nodes).



The alternative is to run configure on the KNL nodes. But this is very
slow. We could split up the build into 2 parts - only build some
packages on KNL node and the rest on front-end.

Ok - I tried "--download-hypre=1 --prefix=/home/balay/soft/hypre-xsdk-0.2.0-rc2" on the KNL node.
[this way - I can --with-hypre-dir=/home/balay/soft/hypre-xsdk-0.2.0-rc2 for subsequent builds]

This crashed halfway with 'Intel License Server' errors. The crash
happened after hypre configure completed - so I could complete the
build on the frontend.

Anyhow - here is the process"

caveat: PETSC_ARCH/externalpackages should be populated before doing
this - as compute nodes don't have external network access.

1. schedule configure run in the batch queue:

balay at thetalogin5:~/petsc.test> cat reconfigure-arch-theta-opt-cray.py
if __name__ == '__main__':
  import sys
  import os
  sys.path.insert(0, os.path.abspath('config'))
  import configure
  configure_options = [
    'COPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',
    'CXXOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',
    'FOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',

balay at thetalogin5:~/petsc.test> cat x.sh
aprun -n 1 python reconfigure-arch-theta-opt-cray.py

balay at thetalogin5:~/petsc.test> qsub -t 45 -n 1 -A EarlyPerf_theta x.sh


2. wait for job to complete - [or atleast hypre configure to complete].

3. if incomplete - complete the build on frontend - and install hypre libraries.

4. subsequently install PETSc using --with-hypre-dir=/home/balay/soft/hypre-xsdk-0.2.0-rc2
[on the frontend]


More information about the petsc-dev mailing list