<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 30, 2017 at 3:27 PM, Balay, Satish <span dir="ltr"><<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, 30 Jun 2017, Richard Tran Mills wrote:<br>
<br>
> Hmm. I get the same error when adding "--download-hypre=0" to an otherwise<br>
> functional configure script for Cori KNL. Hypre's configure doesn't like<br>
> something about cross-compiling for KNL. It might be possible to make this<br>
> work by building on the compute nodes, though this is, unfortunately, not<br>
> officially supported on Cori -- NERSC staff say that builds are not<br>
> guaranteed to work on the compute nodes, though I've managed to make it<br>
> work sometimes.<br>
<br>
</span>Here is (a part of a prior email) regarding installing hypre on theta (using compute nodes).<br></blockquote><div><br></div><div>The approach Satish outlines below sounds like the most reasonable course of action we ahve. I wonder if it might help a little bit to try to just run the standalone Hypre configuration (using the incantation in the configure.log) in the batch script, rather than invoking PETSc's configure.py.<br><br></div><div>--Richard<br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Satish<br>
<br>
------------------------------<wbr>--<br>
<br>
The alternative is to run configure on the KNL nodes. But this is very<br>
slow. We could split up the build into 2 parts - only build some<br>
packages on KNL node and the rest on front-end.<br>
<br>
Ok - I tried "--download-hypre=1 --prefix=/home/balay/soft/<wbr>hypre-xsdk-0.2.0-rc2" on the KNL node.<br>
[this way - I can --with-hypre-dir=/home/balay/<wbr>soft/hypre-xsdk-0.2.0-rc2 for subsequent builds]<br>
<br>
This crashed halfway with 'Intel License Server' errors. The crash<br>
happened after hypre configure completed - so I could complete the<br>
build on the frontend.<br>
<br>
Anyhow - here is the process"<br>
<br>
caveat: PETSC_ARCH/externalpackages should be populated before doing<br>
this - as compute nodes don't have external network access.<br>
<br>
1. schedule configure run in the batch queue:<br>
<br>
balay@thetalogin5:~/petsc.<wbr>test> cat reconfigure-arch-theta-opt-<wbr>cray.py<br>
#!/usr/bin/python<br>
if __name__ == '__main__':<br>
import sys<br>
import os<br>
sys.path.insert(0, os.path.abspath('config'))<br>
import configure<br>
configure_options = [<br>
'--download-hypre=1',<br>
'--prefix=/home/balay/soft/<wbr>hypre-xsdk-0.2.0-rc2',<br>
'--with-cc=cc',<br>
'--with-clib-autodetect=0',<br>
'--with-cxx=CC',<br>
'--with-cxxlib-autodetect=0',<br>
'--with-debugging=0',<br>
'--with-fc=ftn',<br>
'--with-fortranlib-autodetect=<wbr>0',<br>
'--with-mpiexec=mpirun',<br>
'--with-memalign=64',<br>
'COPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',<br>
'CXXOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',<br>
'FOPTFLAGS=-g -O3 -fp-model fast -xMIC-AVX512',<br>
'LIBS=-lstdc++',<br>
'PETSC_ARCH=arch-theta-opt-<wbr>cray',<br>
]<br>
configure.petsc_configure(<wbr>configure_options)<br>
<br>
balay@thetalogin5:~/petsc.<wbr>test> cat x.sh<br>
#!/bin/sh<br>
aprun -n 1 python reconfigure-arch-theta-opt-<wbr>cray.py<br>
<br>
balay@thetalogin5:~/petsc.<wbr>test> qsub -t 45 -n 1 -A EarlyPerf_theta x.sh<br>
<br>
<<<<<<<<<<<<<<<<<<<br>
<br>
2. wait for job to complete - [or atleast hypre configure to complete].<br>
<br>
3. if incomplete - complete the build on frontend - and install hypre libraries.<br>
<br>
4. subsequently install PETSc using --with-hypre-dir=/home/balay/<wbr>soft/hypre-xsdk-0.2.0-rc2<br>
[on the frontend]<br>
<span class="HOEnZb"><font color="#888888"><br>
Satish<br>
<br>
<br>
</font></span></blockquote></div><br></div></div>