[petsc-dev] Compiling Petsc with Intel mpi safe thread library

[Jose David Bermeol]

Hi I'm working again with the solver mkl-cpardiso. I did my implementation, the configuration works well, the problem is during compilation. The linking flags petsc is using are the followings:

    -Wl,-rpath,/home/jbermeol/software/test/arch-linux2-c-opt/lib -L/home/jbermeol/software/test/arch-linux2-c-opt/lib  -lpetsc -Wl,--start-group -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -lpthread -lm -liomp5 -mt_mpi -Wl,--start-group -L/home/jbermeol/testPetscSolvers/intel_mkl_cpardiso/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -lcpardiso_lp64 -lcpardiso_mpi_lp64 -lpthread -lm -liomp5 -lifcore -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -lX11 -lpthread -Wl,-rpath,/apps/rhel6/intel/impi/4.1.0.030/intel64/lib -L/apps/rhel6/intel/impi/4.1.0.030/intel64/lib -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/compiler/lib/intel64 -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/ipp/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/ipp/lib/intel64 -Wl,-rpath,/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 -L/apps/rhel6/intel/composer_xe_2013.3.163/tbb/lib/intel64 -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -Wl,-rpath,/home/jbermeol/software/test/-Xlinker -Wl,-rpath,/opt/intel/mpi-rt/4.1 -lifport -lifcore -lm -lm -lmpigc4 -ldl -lmpigf -lmpi -lmpigi -lrt -lpthread -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc -lirc_s -ldl

The problem is with flag "-mt_mpi", this flag is telling to the Intel compiler, please link with the library libmpi_mt.so(this is a thread safe library for intel mpi). However petsc adds the flag "-lmpi" that links with libmpi.so. Because of this I'm gettig the following error:
   ld: MPIR_Thread: TLS definition in /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi_mt.so section .tbss mismatches non-TLS definition in /apps/rhel6/intel/impi/4.1.0.030/intel64/lib/libmpi.so section .bss

So the first question would be where is this "-lmpi" library flag added??
Second how can I compile using the thread save mpi library from intel??
Is there a way to set a dependency such that the solver cpardiso is not installed if petsc is compiled without this thread safe library??
I don't know yet how to check if petsc is been configure with mkl blas/lapack

Thanks