[petsc-dev] broken options handling with intel 16.0 compilers on mac OS

Tue Sep 22 18:00:23 CDT 2015

On Sep 22, 2015, at 3:50 PM, Satish Balay <balay at mcs.anl.gov<mailto:balay at mcs.anl.gov>> wrote:

I'm not sure what to suggest.

Blaise gets a successful build with:

--with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0 LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a"

The next thing Blaise could try is:

--with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0 LIBS=-lifcore

This works.

bourdin at galerkin:tutorials (master)$  otool -l ex5f90 |grep ifor
         name /opt/HPC/mpich-3.1.4-intel16.0/lib/libmpifort.12.dylib (offset 24)

In contrast, with the build command suggested with richard, I get a non-functioning build:

bourdin at galerkin:petsc-dev (master)$ ./configure --with-debugging=1 COPTFLAGS="-g -O0" FOPTFLAGS="-g -O0”
CXXOPTFLAGS="-g -O0" --with-blas-lapack-dir=/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl --with-mpi-dir=$MPI_HOME

bourdin at galerkin:tutorials (master)$  otool -l ex5f90 |grep ifor
         name /opt/HPC/mpich-3.1.4-intel16.0/lib/libmpifort.12.dylib (offset 24)

[and if this fails - see which ifcore the binary is using - via 'otool -l petscbinary |grep ifore']

If not - what I would do - is try compiling an example with a petsc makefile.
[with the broken build]

And then copy/paste the link line - removing stuff - until the binary
created by that link line works. [and identify the linker option is
making the difference]. Again - this would be for Blaise to try.

With the later configuration, it looks like what matters is the order in which  -lmpifort and -lifcore are listed:
This one works:
mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first   -g -O0  -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib  -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -lifcore -lmpifort

This one does not:
mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first   -g -O0  -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib  -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -lmpifort -lifcore

Same goes with the lengthy link line from the petsc makefile:
mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first   -g -O0  -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib  -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl -L/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl -lmkl_intel -lmkl_sequential -lmkl_core -lpthread -lm -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -lX11 -lssl -lcrypto -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.1.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.1.0/lib/darwin -lifcore -lmpifort -lifport  -limf -lsvml -lipgo -lirc -lpthread -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -lclang_rt.osx -limf -lsvml -lirng -lipgo -ldecimal -lirc -lclang_rt.osx -lmpicxx -limf -lsvml -lirng -lipgo -ldecimal -lirc -lclang_rt.osx -ldl -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -lmpi -lpmpi -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -limf -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -lsvml -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -lirng -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -lipgo -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -ldecimal -lc++ -lSystem -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -lirc -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -lclang_rt.osx -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -ldl

this works, but switch the order of libmpifort and libifcore, and it does not anymore. (don’t ask me how I noticed!)

Blaise

Intel compilers (esp on Mac) had workarrounds to system issues encoded
in the link command. [I remember the first generation of intel
compilers on Mac had such issues]. And the way PETSc configure, and
checkFortranLibraries() process/use this 'ifort -v' info can break
these encoded workarrounds.. [Its a hacky piece of code - and not easy
to get right]

Ideally compilers should provide equivalent of 'mpicc -show' - so that
configure tools can grab and use this info for language interoperable
linking.

[Jed suggested we should 'always use FC as linker' and not do any such
detection.  But then we still have to worry about C++. Previously we
had machines where we had to use C++ linker - but perhaps such
machines don't exist anymore]

Satish

On Tue, 22 Sep 2015, Richard Mills wrote:

Blaise and Satish,

I'm a bit slow to pick up on this thread as I was busy traveling, but since
I use a Mac and work for Intel, I thought I should see if I could reproduce
the problems that Blaise is seeing.  I installed the 16.0 compilers and
built a simple configuration ('--with-debugging=1 COPTFLAGS="-g -O0"
FOPTFLAGS="-g -O0" CXXOPTFLAGS="-g -O0"
--with-blas-lapack-dir=/opt/intel/compilers_and_libraries_2016/mac/mkl
--with-mpi-dir=/Users/rtmills/packages/mpich-3.1.4-intel') using a very
recent revision of 'master' (09b4d96fa5749f82a0af9a914729f77a4ef2b2fd, Sun
Sep 20 22:51:31 2015 -0500).  When I try running SNES ex5f and passing
various command-line options, everything appears to work fine.  Any
suggestions for digging deeping into this to try to determine the
difference between what Blaise and I are seeing?

Best regards,
Richard

On Mon, Sep 21, 2015 at 10:21 AM, Blaise A Bourdin <bourdin at lsu.edu<mailto:bourdin at lsu.edu>> wrote:

On Sep 20, 2015, at 9:04 AM, Satish Balay <balay at mcs.anl.gov<mailto:balay at mcs.anl.gov>> wrote:

Hm - I would suggest doing a minimal build build with:

--with-cxx=0
--with-clib-autodetect=0 --with-fortranlib-autodetect=0
LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a"

And see if that makes a difference.
Satish,

It does help.
Turning off auto detect leads to a functional build, turning it back on
leads to a non-functioning one.

The configure.log without auto detect is here:

https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/61967j4XaVp
The one with auto detect there:
  https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/691pPOSRU

The petscconf.h are attached

--
Department of Mathematics and Center for Computation & Technology
Louisiana State University, Baton Rouge, LA 70803, USA
Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150922/8364ea91/attachment.html>