<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sun, Jul 1, 2018 at 4:08 AM Karl Rupp <<a href="mailto:rupp@iue.tuwien.ac.at">rupp@iue.tuwien.ac.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Mark,<br>
<br>
have a look at config/examples/arch-linux-knl.py, which contains on line 20:<br>
'--with-blaslapack-dir='+os.environ['MKLROOT'],<br>
<br>
It's important that you specify the BLAS library *and* the MKL include <br>
directory (either via --with-blaslapack-dir or via a pair of <br>
--with-blaslapack-include and --with-blaslapack-lib), </blockquote><div><br></div><div>I think what I sent was this second option. I nuked, reconfigured and tested again. Same thing:</div><div><br></div><div><div>> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div><div>> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing package: <a href="http://www.mcs.anl.gov/petsc/documentation/installation.html#external">http://www.mcs.anl.gov/petsc/documentation/installation.html#external</a></div><div>> [0]PETSC ERROR: Unknown Mat type given: aijmkl</div><div>> [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.</div><div>> [0]PETSC ERROR: Petsc Release Version 3.9.2, unknown</div><div>> [0]PETSC ERROR: /global/u2/m/madams/petsc_install/petsc/src/snes/examples/tutorials/./ex19 on a named nid02516 by madams Sun Jul 1 07:10:33 2018</div><div>> [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-int64_t=1 --known-mpi-c-double-complex=1 --known-has-attribute-aligned=1 --with-cc=cc --with-cxx=CC --with-fc=ftn COPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O0 -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" --download-metis=1 --with-hypre-dir=/global/homes/m/madams/tmp/hypre-2.14.0 --download-parmetis=1 --with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a --with-blaslapack-include=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/include --with-debugging=1 --with-mpiexec=srun --with-batch=1 --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=0 --with-64-bit-indices=1 PETSC_ARCH=arch-cori-knl-dbg64-intel-omp --with-openmp=1 --download-p4est=0 --with-x=0 --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-dbg64-intel-omp PETSC_DIR=/global/homes/m/madams/petsc_install/petsc</div><div>> [0]PETSC ERROR: #1 MatSetType() line 61 in /global/u2/m/madams/petsc_install/petsc/src/mat/interface/matreg.c</div><div>> [0]PETSC ERROR: #2 MatSetFromOptions() line 229 in /global/u2/m/madams/petsc_install/petsc/src/mat/utils/gcreate.c</div><div>> [0]PETSC ERROR: #3 DMCreateMatrix_DA() line 793 in /global/u2/m/madams/petsc_install/petsc/src/dm/impls/da/fdda.c</div><div>> [0]PETSC ERROR: #4 DMCreateMatrix() line 1262 in /global/u2/m/madams/petsc_install/petsc/src/dm/interface/dm.c</div><div>> [0]PETSC ERROR: #5 SNESSetUpMatrices() line 646 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div><div>> [0]PETSC ERROR: #6 SNESSetUp_NEWTONLS() line 296 in /global/u2/m/madams/petsc_install/petsc/src/snes/impls/ls/ls.c</div><div>> [0]PETSC ERROR: #7 SNESSetUp() line 2908 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div><div>> [0]PETSC ERROR: #8 SNESSolve() line 4300 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div><div>> [0]PETSC ERROR: #9 main() line 161 in /global/homes/m/madams/petsc_install/petsc/src/snes/examples/tutorials/ex19.c</div><div>> [0]PETSC ERROR: PETSc Option Table entries:</div><div>> [0]PETSC ERROR: -da_refine 3</div><div>> [0]PETSC ERROR: -ksp_monitor</div><div>> [0]PETSC ERROR: -mat_type aijmkl</div><div>> [0]PETSC ERROR: -options_left</div><div>> [0]PETSC ERROR: -pc_type gamg</div><div>> [0]PETSC ERROR: -snes_monitor_short</div><div>> [0]PETSC ERROR: -snes_view</div><div>> [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------</div><div>> Rank 0 [Sun Jul 1 07:10:33 2018] [c1-1c0s5n0] application called MPI_Abort(MPI_COMM_WORLD, 86) - process 0</div><div>> srun: error: nid02516: task 0: Aborted</div><div>> srun: Terminating job step 13487457.2</div></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">otherwise it's not <br>
possible to compile the aijmkl code.<br>
<br>
Best regards,<br>
Karli<br>
<br>
<br>
<br>
<br>
On 06/30/2018 09:55 PM, Mark Adams wrote:<br>
> It builds and runs but looks like PETSc does not register aijmkl matrices.<br>
> <br>
> <br>
> ---<br>
> > [0]PETSC ERROR: --------------------- Error Message <br>
> --------------------------------------------------------------<br>
> > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing <br>
> package: <br>
> <a href="http://www.mcs.anl.gov/petsc/documentation/installation.html#external" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/installation.html#external</a><br>
> > [0]PETSC ERROR: Unknown Mat type given: aijmkl<br>
> > [0]PETSC ERROR: See <br>
> <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> > [0]PETSC ERROR: Petsc Release Version 3.9.2, unknown<br>
> > [0]PETSC ERROR: <br>
> /global/u2/m/madams/petsc_install/petsc/src/snes/examples/tutorials/./ex19 <br>
> on a named nid02516 by madams Sat Jun 30 12:48:10 2018<br>
> > [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 <br>
> --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 <br>
> --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 <br>
> --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 <br>
> --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 <br>
> --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 <br>
> --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 <br>
> --known-mpi-int64_t=1 --known-mpi-c-double-complex=1 <br>
> --known-has-attribute-aligned=1 --with-cc=cc --with-cxx=CC --with-fc=ftn <br>
> COPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O0 <br>
> -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O0 -hcpu=mic-knl <br>
> -qopenmp-simd" --download-metis=1 <br>
> --with-hypre-dir=/global/homes/m/madams/tmp/hypre-2.14.0 <br>
> --download-parmetis=1 <br>
> --with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a <br>
> --with-debugging=1 --with-mpiexec=srun --with-batch=1 <br>
> --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=0 <br>
> --with-64-bit-indices=1 PETSC_ARCH=arch-cori-knl-dbg64-intel-omp <br>
> --with-openmp=1 --download-p4est=0 --with-x=0 <br>
> --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-dbg64-intel-omp <br>
> PETSC_DIR=/global/homes/m/madams/petsc_install/petsc<br>
> > [0]PETSC ERROR: #1 MatSetType() line 61 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/mat/interface/matreg.c<br>
> > [0]PETSC ERROR: #2 MatSetFromOptions() line 229 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/mat/utils/gcreate.c<br>
> > [0]PETSC ERROR: #3 DMCreateMatrix_DA() line 793 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/dm/impls/da/fdda.c<br>
> > [0]PETSC ERROR: #4 DMCreateMatrix() line 1262 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/dm/interface/dm.c<br>
> > [0]PETSC ERROR: #5 SNESSetUpMatrices() line 646 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br>
> > [0]PETSC ERROR: #6 SNESSetUp_NEWTONLS() line 296 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/snes/impls/ls/ls.c<br>
> > [0]PETSC ERROR: #7 SNESSetUp() line 2908 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br>
> > [0]PETSC ERROR: #8 SNESSolve() line 4300 in <br>
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br>
> > [0]PETSC ERROR: #9 main() line 161 in <br>
> /global/homes/m/madams/petsc_install/petsc/src/snes/examples/tutorials/ex19.c<br>
> > [0]PETSC ERROR: PETSc Option Table entries:<br>
> > [0]PETSC ERROR: -da_refine 3<br>
> > [0]PETSC ERROR: -ksp_monitor<br>
> > [0]PETSC ERROR: -mat_type aijmkl<br>
> > [0]PETSC ERROR: -options_left<br>
> > [0]PETSC ERROR: -pc_type gamg<br>
> > [0]PETSC ERROR: -snes_monitor_short<br>
> > [0]PETSC ERROR: -snes_view<br>
> > [0]PETSC ERROR: ----------------End of Error Message -------se<br>
> <br>
> On Sat, Jun 30, 2018 at 3:08 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a> <br>
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>> wrote:<br>
> <br>
> OK, that got further.<br>
> <br>
> On Sat, Jun 30, 2018 at 3:03 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a><br>
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>> wrote:<br>
> <br>
> Like this?<br>
> <br>
> <br>
> '--with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a',<br>
> <br>
> <br>
> On Sat, Jun 30, 2018 at 3:00 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a><br>
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>> wrote:<br>
> <br>
> <br>
> Specify either "--with-blaslapack-dir" or<br>
> "--with-blaslapack-lib --with-blaslapack-include".<br>
> But not both!<br>
> <br>
> <br>
> Get rid of the dir option, and give the full path to the<br>
> library.<br>
> <br>
> <br>
> What is the syntax for giving the full path?<br>
> <br>
</blockquote></div></div>