[petsc-dev] error configuring on KNL

[Richard Tran Mills]

On Fri, Jun 30, 2017 at 3:27 PM, Balay, Satish <balay at mcs.anl.gov> wrote:

> On Fri, 30 Jun 2017, Richard Tran Mills wrote:
>
> > Hmm. I get the same error when adding "--download-hypre=0" to an
> otherwise
> > functional configure script for Cori KNL. Hypre's configure doesn't like
> > something about cross-compiling for KNL. It might be possible to make
> this
> > work by building on the compute nodes, though this is, unfortunately, not
> > officially supported on Cori -- NERSC staff say that builds are not
> > guaranteed to work on the compute nodes, though I've managed to make it
> > work sometimes.
>
> Here is (a part of a prior email) regarding installing hypre on theta
> (using compute nodes).
>

The approach Satish outlines below sounds like the most reasonable course
of action we ahve. I wonder if it might help a little bit to try to just
run the standalone Hypre configuration (using the incantation in the
configure.log) in the batch script, rather than invoking PETSc's
configure.py.

--Richard


> Satish
>
> --------------------------------
>
> The alternative is to run configure on the KNL nodes. But this is very
> slow. We could split up the build into 2 parts - only build some
> packages on KNL node and the rest on front-end.
>
> Ok - I tried "--download-hypre=1 --prefix=/home/balay/soft/hypre-xsdk-0.2.0-rc2"
> on the KNL node.
> [this way - I can --with-hypre-dir=/home/balay/soft/hypre-xsdk-0.2.0-rc2
> for subsequent builds]
>
> This crashed halfway with 'Intel License Server' errors. The crash
> happened after hypre configure completed - so I could complete the
> build on the frontend.
>
> Anyhow - here is the process"
>
> caveat: PETSC_ARCH/externalpackages should be populated before doing
> this - as compute nodes don't have external network access.
>
> 1. schedule configure run in the batch queue:
>
> balay at thetalogin5:~/petsc.test> cat reconfigure-arch-theta-opt-cray.py
> #!/usr/bin/python
> if __name__ == '__main__':
>   import sys
>   import os
>   sys.path.insert(0, os.path.abspath('config'))
>   import configure
>   configure_options = [
>     '--download-hypre=1',
>     '--prefix=/home/balay/soft/hypre-xsdk-0.2.0-rc2',
>     '--with-cc=cc',
>     '--with-clib-autodetect=0',
>     '--with-cxx=CC',
>     '--with-cxxlib-autodetect=0',
>     '--with-debugging=0',
>     '--with-fc=ftn',
>     '--with-fortranlib-autodetect=0',
>     '--with-mpiexec=mpirun',
>     '--with-memalign=64',
>     'COPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',
>     'CXXOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',
>     'FOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',
>     'LIBS=-lstdc++',
>     'PETSC_ARCH=arch-theta-opt-cray',
>   ]
>   configure.petsc_configure(configure_options)
>
> balay at thetalogin5:~/petsc.test> cat x.sh
> #!/bin/sh
> aprun -n 1 python reconfigure-arch-theta-opt-cray.py
>
> balay at thetalogin5:~/petsc.test> qsub -t 45 -n 1 -A EarlyPerf_theta x.sh
>
> <<<<<<<<<<<<<<<<<<
>
> 2. wait for job to complete - [or atleast hypre configure to complete].
>
> 3. if incomplete - complete the build on frontend - and install hypre
> libraries.
>
> 4. subsequently install PETSc using --with-hypre-dir=/home/balay/
> soft/hypre-xsdk-0.2.0-rc2
> [on the frontend]
>
> Satish
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170630/319a88a8/attachment.html>