[mpich-discuss] mpicxx linking problem - related to -Wl, -rpath - g++ works

Dave Goodell goodell at mcs.anl.gov
Thu May 20 00:21:41 CDT 2010 

What version of MPICH2 is this?  I think we removed the rpath bit from the shared lib link line in r6336 [1], which probably only showed up as of mpich2-1.3a2.  Rpaths cause problems on a lot of systems because you can't easily override them, which is why we got rid of them.

What library actually defines the missing symbols in the rpath case?  Is it libqhull.so (not "++")?  You can check with "nm some.so | grep qh_clear_outputflags" for any relevant shared library files as "some.so".

Are there conflicting library versions for any libraries (esp. a "qhull" lib) in /usr/lib?

You may be able to get some more info by passing "-Wl,-M" on the command line.  This will print a linker map to the standard output, which may tell you which shared libs are being loaded in each case (with and without rpath).

-Dave

[1] https://trac.mcs.anl.gov/projects/mpich2/changeset/6336

On May 19, 2010, at 8:10 PM, Sebastian Steiger wrote:

> Hello everybody,
> 
> I'm encountering a weird linking problem. I'm trying to link against a library "libnemo.so" which itself has a lot of dependencies. When I use g++ as a linker, it works:
> 
> 
> % g++ -o hello_world.bin hello_world.cpp -L../../lib -lnemo
> %
> 
> 
> When I use mpicxx, it doesn't:
> 
> % mpicxx -o hello_world.bin hello_world.cpp -L../../lib -lnemo
> 
> (...)/libqhull++.so: undefined reference to `qh_clear_outputflags()'
> (...)/libqhull++.so: undefined reference to `qh_getcentrum(facetT*)'
> (etc.)
> 
> 
> libqhull++ is a dependency of libnemo. This is the actual linking command that mpicxx substitutes:
> 
> % mpicxx -link_info -o hello_world.bin hello_world.cpp -L../../lib -lnemo
> c++ -g -O2 -g -Wall -O2 -Wl,-Bsymbolic-functions -o my_test.bin my_test.cpp -L../../lib -lnemo -I/usr/include/mpich2 -L/usr/lib -L/usr/lib -Wl,-rpath -Wl,/usr/lib -lmpichcxx -lmpich -lopa -lpthread -lrt
> 
> 
> So mpicxx attaches a couple of flags and some libraries at the end. FYI, when I use mpicxx -cxx=g++ the g++ linker is invoked but the same problem remains.
> 
> When I take away the "-Wl,-rpath -Wl,/usr/lib" part, it works:
> 
> % c++ -g -O2 -g -Wall -O2 -Wl,-Bsymbolic-functions -o hello_world.bin hello_world.cpp -L../../lib -lnemo -I/usr/include/mpich2 -L/usr/lib -L/usr/lib -lmpichcxx -lmpich -lopa -lpthread -lrt
> %
> 
> 
> Also, mpicxx works by itself if I don't link to my library:
> 
> % mpicxx -o hello_world.bin hello_world.cpp
> %
> 
> 
> So it's both connected to mpicxx and my library. Unfortunately this is just one notch beyond my level of understanding of the linking process. Does anybody have a clue?
> 
> 
> At the bottom you find the libraries against which I link libnemo.so.
> 
> 
> Thanks for your help
> Sebastian
> 
> 
> 
> ====================================================================
> 
> steiger at ncnlnx07:~/src/app-nemo/NEMO/prototype/tests/test_material_db$ ldd ../../lib/libnemo.so
>        linux-vdso.so.1 =>  (0x00007fff1bdfa000)
>        libdl.so.2 => /lib/libdl.so.2 (0x00007f3362749000)
>        liblapack.so.3gf => /usr/lib/liblapack.so.3gf (0x00007f3361b68000)
>        libblas.so.3gf => /usr/lib/libblas.so.3gf (0x00007f33618cd000)
>        libtensor.so => /home/steiger/src/app-nemo/NEMO/libs/tensor3D/libtensor.so (0x00007f33616b1000)
>        libNEMat-1.00.so => /home/steiger/src/app-nemo/NEMO/libs/../MaterialDB/NEMat/libNEMat-1.00.so (0x00007f336142c000)
>        libnemo_petsc_double.so => /home/steiger/src/app-nemo/NEMO/prototype/src/math/libnemo_petsc_double/libnemo_petsc_double.so (0x00007f33607e8000)
>        libnemo_petsc_complex.so => /home/steiger/src/app-nemo/NEMO/prototype/src/math/libnemo_petsc_complex/libnemo_petsc_complex.so (0x00007f335fc2a000)
>        libqhull++.so => /home/steiger/src/app-nemo/NEMO/libs/qhull-2010.1/lib/libqhull++.so (0x00007f335fa09000)
>        libqhull.so.5 => /usr/lib/libqhull.so.5 (0x00007f335f7b8000)
>        libtet.so => /home/steiger/src/app-nemo/NEMO/libs/tetgen1.4.3/libtet.so (0x00007f335f4d8000)
>        libumfpack.so.5.4.0 => /usr/lib/libumfpack.so.5.4.0 (0x00007f335f218000)
>        libamd.so.2.2.0 => /usr/lib/libamd.so.2.2.0 (0x00007f335f00e000)
>        libarpack.so => /home/steiger/src/app-nemo/NEMO/libs/ARPACK/libarpack.so (0x00007f335ed72000)
>        libboost_filesystem.so.1.40.0 => /usr/lib/libboost_filesystem.so.1.40.0 (0x00007f335eb5d000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/lib/x86_64-unknown-linux-gnu_opt/libmesh.so (0x00007f335e36a000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/liblaspack.so (0x00007f335e149000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libmetis.so (0x00007f335ded2000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libsfcurves.so (0x00007f335dcce000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libgzstream.so (0x00007f335daca000)
>        libz.so.1 => /lib/libz.so.1 (0x00007f335d8b2000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libgmv.so (0x00007f335d697000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libtetgen.so (0x00007f335d419000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libtriangle.so (0x00007f335d1f1000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libexodusii.so (0x00007f335cfad000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libnetcdf.so (0x00007f335cd75000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libnemesis.so (0x00007f335cb58000)
> /home/steiger/src/app-nemo/NEMO/libs/libmesh-0.6.4/libmesh/contrib/lib/x86_64-unknown-linux-gnu_opt/libHilbert.so (0x00007f335c940000)
>        libmpich.so.1.2 => /usr/lib/libmpich.so.1.2 (0x00007f335c575000)
>        libmpichcxx.so.1.2 => /usr/lib/libmpichcxx.so.1.2 (0x00007f335c351000)
>        libpthread.so.0 => /lib/libpthread.so.0 (0x00007f335c133000)
>        librt.so.1 => /lib/librt.so.1 (0x00007f335bf2b000)
>        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f335bc17000)
>        libm.so.6 => /lib/libm.so.6 (0x00007f335b993000)
>        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f335b77c000)
>        libc.so.6 => /lib/libc.so.6 (0x00007f335b3fa000)
>        /lib64/ld-linux-x86-64.so.2 (0x00007f3362c94000)
>        libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0x00007f335b10c000)
>        libnml.so => /home/steiger/src/app-nemo/NEMO/MaterialDB/nml/lib/libnml.so (0x00007f335aee5000)
>        libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f335abae000)
>        libqhull.so => /home/steiger/src/app-nemo/NEMO/libs/qhull-2010.1/cpp/../lib/libqhull.so (0x00007f335a93b000)
>        libboost_system.so.1.40.0 => /usr/lib/libboost_system.so.1.40.0 (0x00007f335a736000)
>        libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007f335a519000)
>        libXau.so.6 => /usr/lib/libXau.so.6 (0x00007f335a315000)
>        libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007f335a10f000)
> _______________________________________________
> mpich-discuss mailing list
> mpich-discuss at mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

More information about the mpich-discuss mailing list