[petsc-users] Using multithreaded MKL in PETSc

[Barry Smith]

  BTW:  There is also MATSOLVERMKL_CPARDISO which is apparently an MPI parallel direct solver.

  Barry

> On Mar 8, 2016, at 4:15 PM, Vasileios Kalantzis <kalan019 at umn.edu> wrote:
> 
> (I accidentally replied to the list only)
> 
> Satish  and Barry,
> 
> thank you very much for your help, problem solved when I edited
> PETSC_ARCH/conf/petscvariables and removed the -lmkl_sequential
> flag from the linker,
> 
> On Tue, Mar 8, 2016 at 3:33 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> Ideally petsc configure is smart and automatically determines the correct
> mkl library list. [but this is not easy - hence the existance of MKL advisor]
> 
> You could use intel mkl advisor - and get the *desired* MKL libary
> list for your specific setup [compilers, mkl version, etc..] - and
> then specify this library list to PETSc configure using
> --with-blas-lapack-lib option.
> 
> Alternatively you can edit PETSC_ARCH/lib/conf/petscvariables and change
> 
> -lmkl_sequential to -lmkl_intel_thread  [or something suitable for your setup]
> 
> and give it a try..
> 
> Satish
> 
> On Tue, 8 Mar 2016, Vasileios Kalantzis wrote:
> 
> > Hi Barry,
> >
> > thanks for your note, I am aware of that too. Having only the MKL solver
> > being able to see the second level of parallelism is fine with me.
> >
> > Thanks for the clarification too! As a sidenote, I set all environmental
> > variables correctly but Pardiso prints that is running on 1 OpenMP
> > thread only. I can see in the compile line (when I make) that ${CLINKER}
> > links with flags -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread
> > -lm.
> > Could the -lmkl_sequential flag be the reason that MKL sees only a
> > single thread?  Can I edit ${CLINKER} or somehow remove the sequential
> > flag?
> >
> > Thanks again!
> >
> > On Tue, Mar 8, 2016 at 2:22 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >
> > >
> > > > On Mar 8, 2016, at 2:07 PM, Vasileios Kalantzis <kalan019 at umn.edu>
> > > wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > Assuming that I want to use MKL's Pardiso in the
> > > > context of PETSc (I need it as a local solver for
> > > > block Jacobi type preconditioning), how should I form
> > > > my ./configure so that I can take advantage of
> > > > multi-threading when calling MKL's Pardiso?
> > > >
> > > > For example, do I simply have to configure with
> > > > --with-openmp=1 --with-pthread=1 and/or similar?
> > > > I have tried different combinations and thought
> > > > of asking,
> > > >
> > > > Thank you!
> > > >
> > > > ps. right now I use flat MPI and works fine, I want
> > > > to use threading to add a second level of parallelism,
> > >
> > >   If you do this then the only part of the code that takes advantage of
> > > that "second level of parallelism" will be within the MKL solver. The rest
> > > of the code will not use those "extra" threads. You must make sure you run
> > > MPI on less cores so there are enough "free" cores to run the threads for
> > > the the MKL solver.
> > >
> > >   To utilize MKL's Pardiso you don't need to configure PETSc with any
> > > mention of openmp or pthreads you just need to set appropriate MKL
> > > environmental variables so that MKL's Pardiso will use multiple threads. Or
> > > you can use MatMkl_PardisoSetCntl() see the manual page for
> > > MATSOLVERMKL_PARDISO etc
> > >
> > >
> > >
> > >    Barry
> > >
> > >
> > >
> > > >
> > > >
> > >
> > >
> >
> 
>