[petsc-dev] Compiling Petsc with Intel mpi safe thread library

[Jose David Bermeol]

many thakns, I didn't know that using -mt_mpi half way wouldn't work. 

Yes cpardiso need this flag if not, it compiles, but when I run it sends the following error:
MPI_THREAD_FUNNELED level is not supported!
Exit...

So there is a way to check in mkl-cpardiso.py that I'm using the flag -mt_mpi

Thanks
----- Original Message -----
From: "Satish Balay" <balay at mcs.anl.gov>
To: "Jose David Bermeol" <jbermeol at purdue.edu>
Cc: petsc-dev at mcs.anl.gov
Sent: Sunday, October 20, 2013 12:04:19 AM
Subject: Re: [petsc-dev] Compiling Petsc with Intel mpi safe thread library

sounds like you are inserting -mt_mpi half way through the configure
process via mkl-cpardiso.py.

This won't work.

is mkl-cpardiso tied to [thread safe variant of] intel mpi?
i.e it won't work with regular intel-mpi or other MPI like mpich?

To use thread safe mpi - you specify thread safe mpi compilers to
petsc configure.

i.e something like:

--with-cc='mpicc -mt_mpi' or --with-cc='mpicc' CFALGS='-mt_mpi'.

Satish

On Sat, 19 Oct 2013, Jose David Bermeol wrote:

> Hi I'm working again with the solver mkl-cpardiso. I did my implementation, the configuration works well, the problem is during compilation. The linking flags petsc is using are the followings:
> 
>     -Wl,-rpath,/home/jbermeol/software/test/arch-linux2-c-opt/lib -L/home/jbermeol/software/test/arch-linux2-c-opt/lib  -lpetsc -Wl,--start-group -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -lpthread -lm -liomp5 -mt_mpi -Wl,--start-group -L/home/jbermeol/testPetscSolvers/intel_mkl_cpardiso/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -lcpardiso_lp64 -lcpardiso_mpi_lp64 -lpthread -lm -liomp5 -lifcore -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lpthread -Wl,-rpath,/apps/rhel6/intel/impi/4.1.0.030/intel64/lib -L/apps/rhel6/intel/impi/4.1.0.030/intel64/lib -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 -Wl,-rpath,/apps/rhel6/intel/compos
 
 er_xe_2013.3.163/ipp/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/ipp/lib/intel64 -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/home/jbermeol/software/test/-Xlinker -Wl,-rpath,/opt/intel/mpi-rt/4.1 -lifport -lifcore -lm -lm -lmpigc4 -ldl -lmpigf -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl
> 
> The problem is with flag "-mt_mpi", this flag is telling to the Intel compiler, please link with the library libmpi_mt.so(this is a thread safe library for intel mpi). However petsc adds the flag "-lmpi" that links with libmpi.so. Because of this I'm gettig the following error:
>    ld: MPIR_Thread: TLS definition in /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi_mt.so section .tbss mismatches non-TLS definition in /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi.so section .bss
> 
> So the first question would be where is this "-lmpi" library flag added??
> Second how can I compile using the thread save mpi library from intel??
> Is there a way to set a dependency such that the solver cpardiso is not installed if petsc is compiled without this thread safe library??
> I don't know yet how to check if petsc is been configure with mkl blas/lapack
> 
> Thanks
>