[petsc-dev] Investigation about 'unused direct shared library dependencies'

Jed Brown jed at jedbrown.org
Thu Oct 6 12:43:33 CDT 2016 

PETSc's configure does not distinguish library dependencies for shared
versus static libraries.  I've brought this issue up a few times over
the years, but nobody else seems to have been bothered by overlinking.
The configure problem is actually much easier if shared libraries are
linked correctly.

Satish Balay <balay at mcs.anl.gov> writes:

>>>>>>>>
> + ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info CFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic CXXFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic FCFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic FFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic LDFLAGS=-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld --with-default-arch=0 --with-make=1 --with-single-library=1 --enable-debug=0 --enable-shared=1 --with-precision=double --with-index-size=32 --with-petsc-arch=x86_64 --with-clanguage=C --with-shared-libraries=1 --with-64-bit-indices=0 --with-fortran-interfaces=1 --with-windows-graphics=0 --CC=gcc --FC=gfortran --with-gnu-compilers=1 --with-vendor-compilers=0 --with-shared-ld=ld --with-pic=1 --sharedLibraryFlags= --dynamicLibraryFlags= --with-debugging=0 'COPTFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -O3 -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' 'CXXOPTFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -O3' 'FOPTFLAGS=-O2 -g -pipe -Wall -Werror=format-security =2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -O3 -I/usr/lib64/gfortran/modules' --with-blas-lib=/usr/lib64/libopenblas.so --with-lapack-lib=/usr/lib64/libopenblas.so --with-blas-lapack-suffix= --known-64-bit-blas-indices=0 --with-hwloc=1 --with-hwloc-pkg-config=/usr/lib64/pkgconfig --with-hwloc-include=/usr/include/hwloc --with-hwloc-lib=/usr/lib64/libhwloc.so --with-metis=1 --with-metis-include=/usr/include --with-metis-lib=/usr/lib64/libmetis.so --with-openmp=1 --with-mpi=0 --with-mpiuni-fortran-binding=1 --with-fc=gfortran --with-libjpeg=1 --with-libpng=1 --with-papi=1 --with-papi-include=/usr/include --with-papi-lib=/usr/lib64/libpapi.so --with-hwloc=1 --with-hwloc-include=/usr/include --with-hwloc-lib=/usr/lib64/libhwloc.so --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse '--with-suitesparse-lib=[/usr/lib64/libumfpack.so,/usr/lib64/libklu.so,/usr/lib64/libcholmod.so,/usr/lib64/libamd.so]' --with-cgns=1 --with-pthread=1 --with-valgrind=1 --with-valgrind-dir=/usr
> <<<<<<<<<
>
> Thats a lot of options to configure. Ok - lets look at the summary
>
>>>>>>>>>
> Compilers:
>   C Compiler:         gcc -O2 -fPIC  -fopenmp 
>   C++ Compiler:       g++ -O2  -fopenmp  -fPIC   
>   Fortran Compiler:   gfortran -O2 -fPIC  -fopenmp 
> Linkers:
>   Shared linker:   gcc -Wl,-z,relro -shared -O2 -fPIC  -fopenmp
>   Dynamic linker:   gcc -Wl,-z,relro -shared -O2 -fPIC  -fopenmp
> make:
> BLAS/LAPACK: -L/usr/lib64 -L/usr/lib64 -lopenblas -L/usr/lib64 -L/usr/lib64 -lopenblas
>   Arch:     
> hwloc:
>   Includes: -I/usr/include
>   Library:  -lhwloc
> metis:
>   Includes: -I/usr/include
>   Library:  -L/usr/lib64 -lmetis
> openmp:
> papi:
>   Includes: -I/usr/include
>   Library:  -L/usr/lib64 -lpapi
> pthread:
>   Library:  -lpthread
> cgns:
>   Library:  -lcgns -lhdf5
> libjpeg:
>   Library:  -ljpeg
> libpng:
>   Library:  -lpng
> valgrind:
>   Includes: -I/usr/include
> SuiteSparse:
>   Includes: -I/usr/include/suitesparse
>   Library:  -L/usr/lib64 -lumfpack -lklu -lcholmod -lamd
> PETSc:
>   PETSC_ARCH: x86_64
>   PETSC_DIR: /builddir/build/BUILD/petsc-3.7.3/petsc-3.7.3
>   Scalar type: real
>   Precision: double
>   Clanguage: C
>   shared libraries: enabled
>   Integer size: 32
>   Memory alignment: 16
> <<<<<<<
>
>>       /lib64/libmetis.so.0
>
> As Matt mentioned - metis is a dependency for parmetis [so its of no use in a sequential build].
> You could use --with-metis=0
>
>>       /lib64/libhwloc.so.5
>
> And PETSc does not make library calls to hwloc. [Its a useful tool to
> have - so packaging part of configure provides support for it - but
> sneaks in as a library? Its also a dependency for Trilinos]
>
> You could use --with-hwloc=0 [similarly --with-ssl=0]
>
> PAPI? Hm - that doesn't get tested much.. similarly libjpeg libpng
> However you might want to use --with-x=1
>
>>       /lib64/libhdf5.so.10
>
> hdf5 is sneaking in as a dependency for cgns. Not sure what you can do about it.
>
>>       /lib64/libquadmath.so.0
>>       /lib64/libstdc++.so.6
>>       /lib64/libgomp.so.1
>
> PETSc configure attempts to detect compiler libraries - and provide a link command that
> would work with all languages [c,c++,fortran]
>
> -lstdc++ is coming from g++, -lquadmath from gfortran and -lgomp  must be coming from all 3 with '-fopenmp'
>
> To workarround this - you could do force-specify the compatibility libraries [since the compilers are fixed for fedora the libs are also fixed]:
> --with-clib-autodetect=0 --with-fortranlib-autodetect=0 --with-cxxlib-autodetect=0 LIBS=-lgfortran
>
> Note: for a mpi build with hypre - you might need -lstdc++ aswell.
>
>
> BTW: for stuff in compiler search path - its best to not specify these paths.
>
> For eg:
> --with-metis-include=/usr/include could be --with-metis-include="" or --with-metis-include=[]
>
> --with-metis-lib=/usr/lib64/libmetis.so could be --with-metis-lib=-lmetis
>
> "--with-valgrind=1 --with-valgrind-dir=/usr" could be "--with-valgrind=1"
>
> Satish
>
> On Thu, 6 Oct 2016, Antonio Trande wrote:
>
>> Hi all.
>> 
>> I'm compiling PETSc 3.7.3 on Fedora; the libraries are built in 4
>> different stages, each stage starting from a *clean PETSc source* directory:
>> 
>> petsc src
>>        |
>>        v
>>      petsc src1 |--> serial libraries (link1)
>>      petsc src2 |--> serial (64bit INTEGER) libraries (link2)
>>      petsc src3 |--> OpenMPI libraries (link3)
>>      petsc src4 |--> MPICH libraries (link4)
>> 
>> link1:
>> http://copr-dist-git.fedorainfracloud.org/cgit/sagitter/petsc/petsc.git/tree/petsc.spec?id=ea7abab121859c66de05f70863950b4a6c5f7035#n402
>> 
>> link2:
>> http://copr-dist-git.fedorainfracloud.org/cgit/sagitter/petsc/petsc.git/tree/petsc.spec?id=ea7abab121859c66de05f70863950b4a6c5f7035#n559
>> 
>> link3:
>> http://copr-dist-git.fedorainfracloud.org/cgit/sagitter/petsc/petsc.git/tree/petsc.spec?id=ea7abab121859c66de05f70863950b4a6c5f7035#n682
>> 
>> link4:
>> http://copr-dist-git.fedorainfracloud.org/cgit/sagitter/petsc/petsc.git/tree/petsc.spec?id=ea7abab121859c66de05f70863950b4a6c5f7035#n842
>> 
>> The resulting libraries look contain 'unused (strange) direct shared
>> library dependencies'. Like for serial libpetsc.so:
>> 
>> $ ldd -u /usr/lib64/libpetsc.so.3.7.3
>> Unused direct dependencies:
>> 	/lib64/libmetis.so.0
>> 	/lib64/libhwloc.so.5
>> 	/lib64/libhdf5.so.10
>> 	/lib64/libquadmath.so.0
>> 	/lib64/libstdc++.so.6
>> 	/lib64/libgomp.so.1
>> 
>> libmetis.so is (correctly?) activated with *--with-metis=1* option but
>> results unused.
>> libhwloc.so.5, libhdf5.so.10 and libstdc++.so shouldn't be linked.
>> 
>> Here you can find a full build log on Fedora 24 64bit:
>> https://copr-be.cloud.fedoraproject.org/results/sagitter/petsc/fedora-24-x86_64/00461492-petsc/build.log.gz
>> 
>> RPMs built are:
>> https://copr-be.cloud.fedoraproject.org/results/sagitter/petsc/fedora-24-x86_64/00461492-petsc/
>> 
>> Do you have idea why this happens?
>> 
>> Regards.
>> ---
>> Antonio Trande
>> mailto: sagitter 'at' fedoraproject 'dot' org
>> http://fedoraos.wordpress.com/
>> https://fedoraproject.org/wiki/User:Sagitter
>> GPG Key: 0x6CE6D08A
>> Check on https://keys.fedoraproject.org/
>> 
>> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20161006/edc6c1c2/attachment.sig>

More information about the petsc-dev mailing list