<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
My petsc installation work with
<div class=""><br class="">
</div>
<div class="">--with-blaslapack-dir=/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl</div>
<div class=""><br class="">
</div>
<div class="">FWIW</div>
<div class=""><br class="">
</div>
<div class="">and that seems to be the only thing, but just in case:</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">Starting configure run at Mon, 30 Apr 2018 15:05:30 +0000</div>
<div class="">Configure Options: --configModules=PETSc.Configure --optionsModule=config.compilerOptions --with-x=0 -with-pic --with-make-np=12 --with-packages-dir=/admin/build/admin/rpms/s\</div>
<div class="">tampede2/SOURCES/petsc-packages/externalpackages-3.9 --with-external-packages-dir=/admin/build/admin/rpms/stampede2/SOURCES/petsc-packages/externalpackages-3.9 --with-mpi-com\</div>
<div class="">pilers=1 --with-mpi-include=/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/intel64/include --with-mpi-lib=/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/int\</div>
<div class="">el64/lib/release_mt/libmpi.so --with-scalar-type=real --with-shared-libraries=1 --with-precision=double --with-chaco=1 --download-chaco --with-hypre=1 --download-hypre --with\</div>
<div class="">-parmetis=1 --download-parmetis --with-metis=1 --download-metis --with-plapack=1 --download-plapack --with-spai=1 --download-spai --with-sundials=1 --download-sundials --with\</div>
<div class="">-hdf5=1 --with-hdf5-dir=/opt/apps/intel17/impi17_0/phdf5/1.8.16/x86_64 --with-mumps=1 --download-mumps --with-parmetis=1 --download-parmetis --with-metis=1 --download-metis -\</div>
<div class="">-with-scalapack=1 --download-scalapack --with-blacs=1 --download-blacs --with-spooles=1 --download-spooles --with-suitesparse=1 --download-suitesparse --with-superlu_dist=1 -\</div>
<div class="">-download-superlu_dist --with-superlu=1 --download-superlu --with-parmetis=1 --download-parmetis --with-metis=1 --download-metis --with-zoltan=1 --download-zoltan=1 --downloa\</div>
<div class="">d-ptscotch=1 --with-debugging=no --with-blaslapack-dir=/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl COPTFLAGS="-xCORE-AVX2 -axMIC-AVX512,COMMON-AVX512 -O2 -g" FOPT\</div>
<div class="">FLAGS="-xCORE-AVX2 -axMIC-AVX512,COMMON-AVX512 -O2 -g" CXXOPTFLAGS="-xCORE-AVX2 -axMIC-AVX512,COMMON-AVX512 -O2 -g”</div>
</div>
<div class=""><br class="">
</div>
<div class="">Victor.</div>
<div class=""><br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Jul 1, 2018, at 9:17 AM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<br class="">
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Sun, Jul 1, 2018 at 4:08 AM Karl Rupp <<a href="mailto:rupp@iue.tuwien.ac.at" class="">rupp@iue.tuwien.ac.at</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
Hi Mark,<br class="">
<br class="">
have a look at config/examples/arch-linux-knl.py, which contains on line 20:<br class="">
'--with-blaslapack-dir='+os.environ['MKLROOT'],<br class="">
<br class="">
It's important that you specify the BLAS library *and* the MKL include<span class="Apple-converted-space"> </span><br class="">
directory (either via --with-blaslapack-dir or via a pair of<span class="Apple-converted-space"> </span><br class="">
--with-blaslapack-include and --with-blaslapack-lib),</blockquote>
<div class=""><br class="">
</div>
<div class="">I think what I sent was this second option. I nuked, reconfigured and tested again. Same thing:</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------</div>
<div class="">> [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing package:<span class="Apple-converted-space"> </span><a href="http://www.mcs.anl.gov/petsc/documentation/installation.html#external" class="">http://www.mcs.anl.gov/petsc/documentation/installation.html#external</a></div>
<div class="">> [0]PETSC ERROR: Unknown Mat type given: aijmkl</div>
<div class="">> [0]PETSC ERROR: See<span class="Apple-converted-space"> </span><a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" class="">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><span class="Apple-converted-space"> </span>for trouble
shooting.</div>
<div class="">> [0]PETSC ERROR: Petsc Release Version 3.9.2, unknown</div>
<div class="">> [0]PETSC ERROR: /global/u2/m/madams/petsc_install/petsc/src/snes/examples/tutorials/./ex19 on a named nid02516 by madams Sun Jul 1 07:10:33 2018</div>
<div class="">> [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 --known-sizeof-int=4 --known-sizeof-long=8
--known-sizeof-long-long=8 --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 --known-mpi-int64_t=1 --known-mpi-c-double-complex=1
--known-has-attribute-aligned=1 --with-cc=cc --with-cxx=CC --with-fc=ftn COPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O0 -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" --download-metis=1 --with-hypre-dir=/global/homes/m/madams/tmp/hypre-2.14.0
--download-parmetis=1 --with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a --with-blaslapack-include=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/include --with-debugging=1 --with-mpiexec=srun
--with-batch=1 --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=0 --with-64-bit-indices=1 PETSC_ARCH=arch-cori-knl-dbg64-intel-omp --with-openmp=1 --download-p4est=0 --with-x=0 --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-dbg64-intel-omp
PETSC_DIR=/global/homes/m/madams/petsc_install/petsc</div>
<div class="">> [0]PETSC ERROR: #1 MatSetType() line 61 in /global/u2/m/madams/petsc_install/petsc/src/mat/interface/matreg.c</div>
<div class="">> [0]PETSC ERROR: #2 MatSetFromOptions() line 229 in /global/u2/m/madams/petsc_install/petsc/src/mat/utils/gcreate.c</div>
<div class="">> [0]PETSC ERROR: #3 DMCreateMatrix_DA() line 793 in /global/u2/m/madams/petsc_install/petsc/src/dm/impls/da/fdda.c</div>
<div class="">> [0]PETSC ERROR: #4 DMCreateMatrix() line 1262 in /global/u2/m/madams/petsc_install/petsc/src/dm/interface/dm.c</div>
<div class="">> [0]PETSC ERROR: #5 SNESSetUpMatrices() line 646 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div>
<div class="">> [0]PETSC ERROR: #6 SNESSetUp_NEWTONLS() line 296 in /global/u2/m/madams/petsc_install/petsc/src/snes/impls/ls/ls.c</div>
<div class="">> [0]PETSC ERROR: #7 SNESSetUp() line 2908 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div>
<div class="">> [0]PETSC ERROR: #8 SNESSolve() line 4300 in /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c</div>
<div class="">> [0]PETSC ERROR: #9 main() line 161 in /global/homes/m/madams/petsc_install/petsc/src/snes/examples/tutorials/ex19.c</div>
<div class="">> [0]PETSC ERROR: PETSc Option Table entries:</div>
<div class="">> [0]PETSC ERROR: -da_refine 3</div>
<div class="">> [0]PETSC ERROR: -ksp_monitor</div>
<div class="">> [0]PETSC ERROR: -mat_type aijmkl</div>
<div class="">> [0]PETSC ERROR: -options_left</div>
<div class="">> [0]PETSC ERROR: -pc_type gamg</div>
<div class="">> [0]PETSC ERROR: -snes_monitor_short</div>
<div class="">> [0]PETSC ERROR: -snes_view</div>
<div class="">> [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to<span class="Apple-converted-space"> </span><a href="mailto:petsc-maint@mcs.anl.gov" class="">petsc-maint@mcs.anl.gov</a>----------</div>
<div class="">> Rank 0 [Sun Jul 1 07:10:33 2018] [c1-1c0s5n0] application called MPI_Abort(MPI_COMM_WORLD, 86) - process 0</div>
<div class="">> srun: error: nid02516: task 0: Aborted</div>
<div class="">> srun: Terminating job step 13487457.2</div>
</div>
<div class=""><br class="">
</div>
<div class=""> </div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
otherwise it's not<span class="Apple-converted-space"> </span><br class="">
possible to compile the aijmkl code.<br class="">
<br class="">
Best regards,<br class="">
Karli<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
On 06/30/2018 09:55 PM, Mark Adams wrote:<br class="">
> It builds and runs but looks like PETSc does not register aijmkl matrices.<br class="">
><span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><br class="">
> ---<br class="">
> > [0]PETSC ERROR: --------------------- Error Message<span class="Apple-converted-space"> </span><br class="">
> --------------------------------------------------------------<br class="">
> > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing<span class="Apple-converted-space"> </span><br class="">
> package:<span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><a href="http://www.mcs.anl.gov/petsc/documentation/installation.html#external" rel="noreferrer" target="_blank" class="">http://www.mcs.anl.gov/petsc/documentation/installation.html#external</a><br class="">
> > [0]PETSC ERROR: Unknown Mat type given: aijmkl<br class="">
> > [0]PETSC ERROR: See<span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank" class="">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><span class="Apple-converted-space"> </span>for trouble
shooting.<br class="">
> > [0]PETSC ERROR: Petsc Release Version 3.9.2, unknown<br class="">
> > [0]PETSC ERROR:<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/snes/examples/tutorials/./ex19<span class="Apple-converted-space"> </span><br class="">
> on a named nid02516 by madams Sat Jun 30 12:48:10 2018<br class="">
> > [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768<span class="Apple-converted-space"> </span><br class="">
> --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8<span class="Apple-converted-space"> </span><br class="">
> --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2<span class="Apple-converted-space"> </span><br class="">
> --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8<span class="Apple-converted-space"> </span><br class="">
> --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8<span class="Apple-converted-space"> </span><br class="">
> --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4<span class="Apple-converted-space"> </span><br class="">
> --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1<span class="Apple-converted-space"> </span><br class="">
> --known-mpi-int64_t=1 --known-mpi-c-double-complex=1<span class="Apple-converted-space"> </span><br class="">
> --known-has-attribute-aligned=1 --with-cc=cc --with-cxx=CC --with-fc=ftn<span class="Apple-converted-space"> </span><br class="">
> COPTFLAGS=" -g -O0 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O0<span class="Apple-converted-space"> </span><br class="">
> -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS=" -g -O0 -hcpu=mic-knl<span class="Apple-converted-space"> </span><br class="">
> -qopenmp-simd" --download-metis=1<span class="Apple-converted-space"> </span><br class="">
> --with-hypre-dir=/global/homes/m/madams/tmp/hypre-2.14.0<span class="Apple-converted-space"> </span><br class="">
> --download-parmetis=1<span class="Apple-converted-space"> </span><br class="">
> --with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a<span class="Apple-converted-space"> </span><br class="">
> --with-debugging=1 --with-mpiexec=srun --with-batch=1<span class="Apple-converted-space"> </span><br class="">
> --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=0<span class="Apple-converted-space"> </span><br class="">
> --with-64-bit-indices=1 PETSC_ARCH=arch-cori-knl-dbg64-intel-omp<span class="Apple-converted-space"> </span><br class="">
> --with-openmp=1 --download-p4est=0 --with-x=0<span class="Apple-converted-space"> </span><br class="">
> --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-dbg64-intel-omp<span class="Apple-converted-space"> </span><br class="">
> PETSC_DIR=/global/homes/m/madams/petsc_install/petsc<br class="">
> > [0]PETSC ERROR: #1 MatSetType() line 61 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/mat/interface/matreg.c<br class="">
> > [0]PETSC ERROR: #2 MatSetFromOptions() line 229 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/mat/utils/gcreate.c<br class="">
> > [0]PETSC ERROR: #3 DMCreateMatrix_DA() line 793 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/dm/impls/da/fdda.c<br class="">
> > [0]PETSC ERROR: #4 DMCreateMatrix() line 1262 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/dm/interface/dm.c<br class="">
> > [0]PETSC ERROR: #5 SNESSetUpMatrices() line 646 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br class="">
> > [0]PETSC ERROR: #6 SNESSetUp_NEWTONLS() line 296 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/snes/impls/ls/ls.c<br class="">
> > [0]PETSC ERROR: #7 SNESSetUp() line 2908 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br class="">
> > [0]PETSC ERROR: #8 SNESSolve() line 4300 in<span class="Apple-converted-space"> </span><br class="">
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c<br class="">
> > [0]PETSC ERROR: #9 main() line 161 in<span class="Apple-converted-space"> </span><br class="">
> /global/homes/m/madams/petsc_install/petsc/src/snes/examples/tutorials/ex19.c<br class="">
> > [0]PETSC ERROR: PETSc Option Table entries:<br class="">
> > [0]PETSC ERROR: -da_refine 3<br class="">
> > [0]PETSC ERROR: -ksp_monitor<br class="">
> > [0]PETSC ERROR: -mat_type aijmkl<br class="">
> > [0]PETSC ERROR: -options_left<br class="">
> > [0]PETSC ERROR: -pc_type gamg<br class="">
> > [0]PETSC ERROR: -snes_monitor_short<br class="">
> > [0]PETSC ERROR: -snes_view<br class="">
> > [0]PETSC ERROR: ----------------End of Error Message -------se<br class="">
><span class="Apple-converted-space"> </span><br class="">
> On Sat, Jun 30, 2018 at 3:08 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a><span class="Apple-converted-space"> </span><br class="">
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>>> wrote:<br class="">
><span class="Apple-converted-space"> </span><br class="">
> OK, that got further.<br class="">
><span class="Apple-converted-space"> </span><br class="">
> On Sat, Jun 30, 2018 at 3:03 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a><br class="">
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>>> wrote:<br class="">
><span class="Apple-converted-space"> </span><br class="">
> Like this?<br class="">
><span class="Apple-converted-space"> </span><br class="">
> <br class="">
> '--with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a',<br class="">
><span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><br class="">
> On Sat, Jun 30, 2018 at 3:00 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a><br class="">
> <mailto:<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>>> wrote:<br class="">
><span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><br class="">
> Specify either "--with-blaslapack-dir" or<br class="">
> "--with-blaslapack-lib --with-blaslapack-include".<br class="">
> But not both!<br class="">
><span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><br class="">
> Get rid of the dir option, and give the full path to the<br class="">
> library.<br class="">
><span class="Apple-converted-space"> </span><br class="">
><span class="Apple-converted-space"> </span><br class="">
> What is the syntax for giving the full path?<br class="">
><span class="Apple-converted-space"> </span><br class="">
</blockquote>
</div>
</div>
<span id="cid:f_jj2x5vsm0"><configure.log></span></div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>