[petsc-users] Using multithreaded MKL in PETSc

[Satish Balay]

Ideally petsc configure is smart and automatically determines the correct
mkl library list. [but this is not easy - hence the existance of MKL advisor]

You could use intel mkl advisor - and get the *desired* MKL libary
list for your specific setup [compilers, mkl version, etc..] - and
then specify this library list to PETSc configure using
--with-blas-lapack-lib option.

Alternatively you can edit PETSC_ARCH/lib/conf/petscvariables and change

-lmkl_sequential to -lmkl_intel_thread  [or something suitable for your setup]

and give it a try..

Satish

On Tue, 8 Mar 2016, Vasileios Kalantzis wrote:

> Hi Barry,
> 
> thanks for your note, I am aware of that too. Having only the MKL solver
> being able to see the second level of parallelism is fine with me.
> 
> Thanks for the clarification too! As a sidenote, I set all environmental
> variables correctly but Pardiso prints that is running on 1 OpenMP
> thread only. I can see in the compile line (when I make) that ${CLINKER}
> links with flags -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread
> -lm.
> Could the -lmkl_sequential flag be the reason that MKL sees only a
> single thread?  Can I edit ${CLINKER} or somehow remove the sequential
> flag?
> 
> Thanks again!
> 
> On Tue, Mar 8, 2016 at 2:22 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> >
> > > On Mar 8, 2016, at 2:07 PM, Vasileios Kalantzis <kalan019 at umn.edu>
> > wrote:
> > >
> > > Hi everyone,
> > >
> > > Assuming that I want to use MKL's Pardiso in the
> > > context of PETSc (I need it as a local solver for
> > > block Jacobi type preconditioning), how should I form
> > > my ./configure so that I can take advantage of
> > > multi-threading when calling MKL's Pardiso?
> > >
> > > For example, do I simply have to configure with
> > > --with-openmp=1 --with-pthread=1 and/or similar?
> > > I have tried different combinations and thought
> > > of asking,
> > >
> > > Thank you!
> > >
> > > ps. right now I use flat MPI and works fine, I want
> > > to use threading to add a second level of parallelism,
> >
> >   If you do this then the only part of the code that takes advantage of
> > that "second level of parallelism" will be within the MKL solver. The rest
> > of the code will not use those "extra" threads. You must make sure you run
> > MPI on less cores so there are enough "free" cores to run the threads for
> > the the MKL solver.
> >
> >   To utilize MKL's Pardiso you don't need to configure PETSc with any
> > mention of openmp or pthreads you just need to set appropriate MKL
> > environmental variables so that MKL's Pardiso will use multiple threads. Or
> > you can use MatMkl_PardisoSetCntl() see the manual page for
> > MATSOLVERMKL_PARDISO etc
> >
> >
> >
> >    Barry
> >
> >
> >
> > >
> > >
> >
> >
>