[petsc-dev] petsc/master: unable to link in C++ with last night PETSC_WITH_EXTERNAL_LIB variable changes

Sat Feb 10 11:43:34 CST 2018

  Eric

  Is your main program C, C++ or Fortran?

> On Feb 10, 2018, at 11:37 AM, Smith, Barry F. <bsmith at mcs.anl.gov> wrote:
> 
> 
>  Eric,
> 
>   We need the entire link line.
> 
>   What linker are you using C, or C++?  This is important.
> 
>   Do you have dependencies on MPI C++ symbols? In other words, are you using C++ MPI bindings?
> 
>    I cannot explain why all the libraries you listed would disappear from PETSC_WITH_EXTERNAL_LIB but we did recently make a change to manual pass under some circumstances less system (including MPI libraries) explicitly since they are already usually passed by the linker. This may be causing your difficulties.
> 
>    Eagerly awaiting your reply.
> 
>     Barry
> 
> 
>> On Feb 10, 2018, at 11:29 AM, Éric Chamberland <Eric.Chamberland at giref.ulaval.ca> wrote:
>> 
>> Hi Matthew,
>> 
>> Yes, I heard that the MPI C++ API has been deprecated.
>> 
>> Yes, the mpi_cxx is now missing.  Our link line is formed mainly with PETSC_WITH_EXTERNAL_LIB variable that is now:
>> 
>> PETSC_WITH_EXTERNAL_LIB = -L/opt/petsc-master_debug/lib -Wl,-rpath,/opt/petsc-master_debug/lib -L/opt/petsc-master_debug/lib -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64 -Wl,-rpath,/opt/openmpi-1.10.2/lib -L/opt/openmpi-1.10.2/lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64 -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64 -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -lpetsc -lsuperlu -lsuperlu_dist -lHYPRE -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lml -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread -lmkl_blacs_intelmpi_lp64 -liomp5 -ldl -lpthread -lparmetis -lmetis -lptesmumps -lptscotch -lptscotcherr -lesmumps -lscotch -lscotcherr -lm -lX11 -lstdc++ -ldl -lmpi_usempi -lmpi_mpifh -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lrt -lm -lpthread -lz -lstdc++ -ldl
>> We add some stuff to this (our own compiled libs), but nothing related to MPI or PETSc since we used to rely on PETSC_WITH_EXTERNAL_LIB for all our diffrement environments (think about different petsc version, MPI libs and compilers: clang, icc, g++) and it used to work until yesterday changes...
>> 
>> As you can see in the diff, this "block" of libraries have been removed from PETSC_WITH_EXTERNAL_LIB: (the -- is part of the diff output):
>> --ldl
>> --lmpi_cxx
>> --lmpi
>> --lstdc++
>> --lm
>> --lgcc_s
>> --lpthread
>> 
>> and into this block there was the "-lmpi_cxx" that we need...
>> 
>> I could send you our whole line of link, but the error is into this small change introduced yesterday into master...
>> 
>> Thanks a lot!
>> 
>> Eric
>> 
>> Le 18-02-10 à 10:34, Matthew Knepley a écrit :
>>> On Sat, Feb 10, 2018 at 9:42 AM, Éric Chamberland <Eric.Chamberland at giref.ulaval.ca> wrote:
>>> Hi,
>>> 
>>> we used to link our c++ code with PETSc using PETSC_WITH_EXTERNAL_LIB variable defined in $PETSC_DIR/lib/petsc/conf/petscvariables and everything was fine until this night.
>>> 
>>> It seems some libs have vanished from this variable, see the diff here:
>>> 
>>> -lptscotcherr
>>> -lesmumps
>>> -lscotch
>>> -lscotcherr
>>> -lm
>>> -lX11
>>> +-lstdc++
>>> -ldl
>>> -lmpi_usempi
>>> -lmpi_mpifh
>>> -lmpi
>>> -lgfortran
>>> -lm
>>> -lgfortran
>>> -lm
>>> -lgcc_s
>>> -lquadmath
>>> -lpthread
>>> --ldl
>>> --lmpi_cxx
>>> --lmpi
>>> --lstdc++
>>> --lm
>>> --lgcc_s
>>> --lpthread
>>> -lrt
>>> -lm
>>> -lpthread
>>> -lz
>>> +-lstdc++
>>> -ldl
>>> 
>>> 
>>> causing these errors at link phase for us:
>>> 
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)': 
>>> 
>>> /opt/openmpi-1.10.2/include/
>>> openmpi/ompi/mpi/cxx/op_inln.
>>> h:122: undefined reference to `ompi_mpi_cxx_op_intercept'
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const': 
>>> 
>>> /opt/openmpi-1.10.2/include/
>>> openmpi/ompi/mpi/cxx/
>>> intracomm.h:25: undefined reference to `MPI::Comm::Comm()'
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intercomm::Merge(bool) const': 
>>> 
>>> /opt/openmpi-1.10.2/include/
>>> openmpi/ompi/mpi/cxx/
>>> intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Split(int, int) const': 
>>> 
>>> /opt/openmpi-1.10.2/include/
>>> openmpi/ompi/mpi/cxx/
>>> intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Create(MPI::Group const&) const': 
>>> 
>>> /opt/openmpi-1.10.2/include/
>>> openmpi/ompi/mpi/cxx/
>>> intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
>>> 
>>> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Clone() const': 
>>> Hi Eric,
>>> 
>>> These symbols are all coming from -lmpi_cxx. I would note that I believe the MPI Forum has deprecated the C++ interface, so it will
>>> eventually go away. However, lets fix this. In the configure log, I see that mpicxx -show has that library in it, so it seems that the link
>>> is not being done with the C++ compiler. Can you send the whole link line?
>>> 
>>>  Thanks,
>>> 
>>>     Matt
>>> 
>>> Is this a normal and definitive change or an unwanted/unobserved bug?
>>> 
>>> Thanks,
>>> 
>>> Eric
>>> 
>>> ps: here are the logs:
>>> 
>>> this night:
>>> 
>>> ---------
>>> 
>>> http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.10.02h00m01s_configure.log
>>> http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.10.02h00m01s_make.log
>>> 
>>> 
>>> 
>>> a day before:
>>> ------------
>>> 
>>> 
>>> http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.09.02h00m02s_configure.log
>>> http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.09.02h00m02s_make.log
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>> -- Norbert Wiener
>>> 
>>> https://www.cse.buffalo.edu/~knepley/
>> 
> 