[petsc-dev] building with MKL

[Karl Rupp]

Hi Mark,

have a look at config/examples/arch-linux-knl.py, which contains on line 20:
  '--with-blaslapack-dir='+os.environ['MKLROOT'],

It's important that you specify the BLAS library *and* the MKL include 
directory (either via --with-blaslapack-dir or via a pair of 
--with-blaslapack-include and --with-blaslapack-lib), otherwise it's not 
possible to compile the aijmkl code.

Best regards,
Karli




On 06/30/2018 09:55 PM, Mark Adams wrote:
> It builds and runs but looks like PETSc does not register aijmkl matrices.
> 
> 
> ---
>  > [0]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
>  > [0]PETSC ERROR: Unknown type. Check for miss-spelling or missing 
> package: 
> http://www.mcs.anl.gov/petsc/documentation/installation.html#external
>  > [0]PETSC ERROR: Unknown Mat type given: aijmkl
>  > [0]PETSC ERROR: See 
> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
>  > [0]PETSC ERROR: Petsc Release Version 3.9.2, unknown
>  > [0]PETSC ERROR: 
> /global/u2/m/madams/petsc_install/petsc/src/snes/examples/tutorials/./ex19 
> on a  named nid02516 by madams Sat Jun 30 12:48:10 2018
>  > [0]PETSC ERROR: Configure options --known-level1-dcache-size=32768 
> --known-level1-dcache-linesize=64 --known-level1-dcache-assoc=8 
> --known-sizeof-char=1 --known-sizeof-void-p=8 --known-sizeof-short=2 
> --known-sizeof-int=4 --known-sizeof-long=8 --known-sizeof-long-long=8 
> --known-sizeof-float=4 --known-sizeof-double=8 --known-sizeof-size_t=8 
> --known-bits-per-byte=8 --known-memcmp-ok=1 --known-sizeof-MPI_Comm=4 
> --known-sizeof-MPI_Fint=4 --known-mpi-long-double=1 
> --known-mpi-int64_t=1 --known-mpi-c-double-complex=1 
> --known-has-attribute-aligned=1 --with-cc=cc --with-cxx=CC --with-fc=ftn 
> COPTFLAGS="  -g -O0 -hcpu=mic-knl -qopenmp-simd" CXXOPTFLAGS="-g -O0 
> -hcpu=mic-knl -qopenmp-simd" FOPTFLAGS="  -g -O0 -hcpu=mic-knl 
> -qopenmp-simd" --download-metis=1 
> --with-hypre-dir=/global/homes/m/madams/tmp/hypre-2.14.0 
> --download-parmetis=1 
> --with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a 
> --with-debugging=1 --with-mpiexec=srun --with-batch=1 
> --known-mpi-shared-libraries=1 --known-64-bit-blas-indices=0 
> --with-64-bit-indices=1 PETSC_ARCH=arch-cori-knl-dbg64-intel-omp 
> --with-openmp=1 --download-p4est=0 --with-x=0 
> --prefix=/global/homes/m/madams/petsc_install/petsc-cori-knl-dbg64-intel-omp 
> PETSC_DIR=/global/homes/m/madams/petsc_install/petsc
>  > [0]PETSC ERROR: #1 MatSetType() line 61 in 
> /global/u2/m/madams/petsc_install/petsc/src/mat/interface/matreg.c
>  > [0]PETSC ERROR: #2 MatSetFromOptions() line 229 in 
> /global/u2/m/madams/petsc_install/petsc/src/mat/utils/gcreate.c
>  > [0]PETSC ERROR: #3 DMCreateMatrix_DA() line 793 in 
> /global/u2/m/madams/petsc_install/petsc/src/dm/impls/da/fdda.c
>  > [0]PETSC ERROR: #4 DMCreateMatrix() line 1262 in 
> /global/u2/m/madams/petsc_install/petsc/src/dm/interface/dm.c
>  > [0]PETSC ERROR: #5 SNESSetUpMatrices() line 646 in 
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c
>  > [0]PETSC ERROR: #6 SNESSetUp_NEWTONLS() line 296 in 
> /global/u2/m/madams/petsc_install/petsc/src/snes/impls/ls/ls.c
>  > [0]PETSC ERROR: #7 SNESSetUp() line 2908 in 
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c
>  > [0]PETSC ERROR: #8 SNESSolve() line 4300 in 
> /global/u2/m/madams/petsc_install/petsc/src/snes/interface/snes.c
>  > [0]PETSC ERROR: #9 main() line 161 in 
> /global/homes/m/madams/petsc_install/petsc/src/snes/examples/tutorials/ex19.c
>  > [0]PETSC ERROR: PETSc Option Table entries:
>  > [0]PETSC ERROR: -da_refine 3
>  > [0]PETSC ERROR: -ksp_monitor
>  > [0]PETSC ERROR: -mat_type aijmkl
>  > [0]PETSC ERROR: -options_left
>  > [0]PETSC ERROR: -pc_type gamg
>  > [0]PETSC ERROR: -snes_monitor_short
>  > [0]PETSC ERROR: -snes_view
>  > [0]PETSC ERROR: ----------------End of Error Message -------se
> 
> On Sat, Jun 30, 2018 at 3:08 PM Mark Adams <mfadams at lbl.gov 
> <mailto:mfadams at lbl.gov>> wrote:
> 
>     OK, that got further.
> 
>     On Sat, Jun 30, 2018 at 3:03 PM Mark Adams <mfadams at lbl.gov
>     <mailto:mfadams at lbl.gov>> wrote:
> 
>         Like this?
> 
>             
>         '--with-blaslapack-lib=/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a',
> 
> 
>         On Sat, Jun 30, 2018 at 3:00 PM Mark Adams <mfadams at lbl.gov
>         <mailto:mfadams at lbl.gov>> wrote:
> 
> 
>                     Specify either "--with-blaslapack-dir" or
>                     "--with-blaslapack-lib --with-blaslapack-include".
>                     But not both!
> 
> 
>                 Get rid of the dir option, and give the full path to the
>                 library.
> 
> 
>             What is the syntax for giving the full path?
>