[petsc-users] Configure with IBM XL toolchain quietly ignores request to build shared libraries

Wed Jul 30 21:30:05 CDT 2014

Compiling manually did the trick. Thanks!
——————————————————
Alp Dener

> On Jul 30, 2014, at 9:48 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> 
> https://www.alcf.anl.gov/user-guides/compiling-and-linking-faq#how-do-i-compile-and-link-a-shared-library?
> 
> Looks like I need to specify '-Wl,-dy' for the linker to find the shared library.
> 
> [balay at vestalac1 tutorials]$ make ex2 CFLAGS="-Wl,-dy -Wl,-rpath,/home/balay/petsc/arch-xl-opt-nopkg/lib"
> mpixlc_r -o ex2.o -c -qpic -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction  -Wl,-dy -Wl,-rpath,/home/balay/petsc/arch-xl-opt-nop
> kg/lib -I/home/balay/petsc/include -I/home/balay/petsc/arch-xl-opt-nopkg/include -I/bgsys/drivers/V1R2M1/ppc64/comm/include -I/bgsys/drivers/V1R2M1/pp
> c64/comm/lib/gnu -I/bgsys/drivers/V1R2M1/ppc64 -I/bgsys/drivers/V1R2M1/ppc64/comm/sys/include -I/bgsys/drivers/V1R2M1/ppc64/spi/include -I/bgsys/drivers/V1R2M1/ppc64/spi/include/kernel/cnk    `pwd`/ex2.c
> mpixlc_r -qpic -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction -Wl,-dy -Wl,-rpath,/home/balay/petsc/arch-xl-opt-nopkg/lib -o ex2 ex2.o  -L/home/balay/petsc/arch-xl-opt-nopkg/lib  -lpetsc -L/soft/libraries/alcf/current/xl/LAPACK/lib -llapack -L/soft/libraries/alcf/current/xl/BLAS/lib -lblas -lpthread -L/bgsys/drivers/V1R2M1/ppc64/comm/lib -L/bgsys/drivers/V1R2M1/ppc64/comm/lib64 -L/bgsys/drivers/V1R2M1/ppc64/spi/lib -L/bgsys/drivers/V1R2M1/ppc64/comm/sys/lib -L/soft/compilers/ibmcmp-may2014/xlsmp/bg/3.1/bglib64 -L/soft/compilers/ibmcmp-may2014/xlmass/bg/7.3/bglib64 -L/soft/compilers/ibmcmp-may2014/vac/bg/12.1/bglib64 -L/soft/compilers/ibmcmp-may2014/vacpp/bg/12.1/bglib64 -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6 -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/lib/gcc -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/powerpc64-bgq-linux/lib -L/soft/compilers/ibmcmp-may2014/xlf/bg/14.1/bglib64 -Wl,-rpath,/soft/compilers/ibmcmp-may2014/lib64/bg/bglib64 -lmpichf90-gcc -lxlf90_r -lxlomp_ser -lxlfmath -lm -lmpichcxx-gcc -libmc++ -ldl -lmpich-gcc -lopa-gcc -lmpl-gcc -lpami-gcc -lSPI -lSPI_cnk -lrt -lpthread -lstdc++ -lxlopt -lxl -lnss_files -lnss_dns -lresolv -lgcc_eh -ldl
> /bin/rm -f ex2.o
> [balay at vestalac1 tutorials]$ ldd ex2
>        linux-vdso64.so.1 =>  (0x00000fff8d020000)
>        libpetsc.so => /home/balay/petsc/arch-xl-opt-nopkg/lib/libpetsc.so (0x00000fff8bfb0000)
>        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000fff8bf50000)
>        libmpichf90-gcc.so.8 => /bgsys/drivers/ppcfloor/comm/lib/libmpichf90-gcc.so.8 (0x00000fff8be40000)
>        libxlf90_r.so.1 => /soft/compilers/ibmcmp-may2014/lib64/bg/bglib64/libxlf90_r.so.1 (0x00000fff8b7a0000)
>        libxlomp_ser.so.1 => /soft/compilers/ibmcmp-may2014/lib64/bg/bglib64/libxlomp_ser.so.1 (0x00000fff8b690000)
>        libxlfmath.so.1 => /soft/compilers/ibmcmp-may2014/lib64/bg/bglib64/libxlfmath.so.1 (0x00000fff8b580000)
>        libm.so.6 => /lib64/libm.so.6 (0x00000fff8b480000)
>        libmpichcxx-gcc.so.8 => /bgsys/drivers/ppcfloor/comm/lib/libmpichcxx-gcc.so.8 (0x00000fff8b370000)
>        libibmc++.so.1 => /soft/compilers/ibmcmp-may2014/lib64/bg/bglib64/libibmc++.so.1 (0x00000fff8b260000)
>        libdl.so.2 => /lib64/libdl.so.2 (0x00000fff8b230000)
>        libmpich-gcc.so.8 => /bgsys/drivers/ppcfloor/comm/lib/libmpich-gcc.so.8 (0x00000fff8ae90000)
>        libopa-gcc.so.0 => /bgsys/drivers/ppcfloor/comm/lib/libopa-gcc.so.0 (0x00000fff8ad80000)
>        libmpl-gcc.so.1 => /bgsys/drivers/ppcfloor/comm/lib/libmpl-gcc.so.1 (0x00000fff8ac70000)
>        libpami-gcc.so => /bgsys/drivers/ppcfloor/comm/lib/libpami-gcc.so (0x00000fff8a700000)
>        librt.so.1 => /lib64/librt.so.1 (0x00000fff8a6d0000)
>        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000080ea910000)
>        libnss_files.so.2 => /lib64/libnss_files.so.2 (0x00000fff8a6a0000)
>        libnss_dns.so.2 => /lib64/libnss_dns.so.2 (0x00000fff8a670000)
>        libresolv.so.2 => /lib64/libresolv.so.2 (0x00000fff8a630000)
>        libc.so.6 => /lib64/libc.so.6 (0x00000fff8a440000)
>        /lib64/bgq/ld64.so.1 => /lib64/ld64.so.1 (0x00000000264e0000)
>        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000080ea720000)
>        libgfortran.so.3 => /bgsys/drivers/toolchain/V1R2M2_base/gnu-linux/powerpc64-bgq-linux/lib/libgfortran.so.3 (0x00000fff8a220000)
> 
> And I could run this binary successfully..
> 
> Satish
> 
> On Wed, 30 Jul 2014, Satish Balay wrote:
> 
>> On Wed, 30 Jul 2014, Alp Dener wrote:
>> 
>>> Hi,
>>> 
>>> I’m trying to build PETSc 3.4.5 on an IBM BlueGene/Q at the moment using the XL toolchain. I need to build the PETSc shared library and to accomplish that, I use the following configure options:
>>> 
>>> ./configure \
>>>    --with-shared-libraries=1 \
>>>    --with-debugging=1 \
>>>    --with-fortran-interfaces=1 \
>>>    --with-mpi=/bgsys/drivers/ppcfloor/comm \
>>>    --with-blas-lib="-L/gpfs/u/software/bgq/lapack/xl/3.5.0/lib -lblas" \
>>>    --with-lapack-lib="-L/gpfs/u/software/bgq/lapack/xl/3.5.0/lib -llapack" \
>>>    --with-metis-include=/gpfs/u/software/bgq/parmetis/xl/4.0.2/include \
>>>    --with-metis-lib="-L/gpfs/u/software/bgq/parmetis/xl/4.0.2/lib -lmetis" \
>>>    --with-parmetis-include=/gpfs/u/software/bgq/parmetis/xl/4.0.2/include \
>>>    --with-parmetis-lib="-L/gpfs/u/software/bgq/parmetis/xl/4.0.2/lib -lparmetis" \
>>>    --download-superlu=/gpfs/u/barn/ODLC/shared/src/PETSc/superlu_4.3.tar.gz \
>>>    --download-superlu_dist=/gpfs/u/barn/ODLC/shared/src/PETSc/superlu_dist_3.3.tar.gz \
>>>    --prefix=/gpfs/u/barn/ODLC/shared/xl/petsc/3.4.5 \
>>>    --PETSC_ARCH=xl-dbg \
>>>    FFLAGS="-qextname”
>>> 
>>> Despite specifying the “—with-shared-libraries” option, the configuration report shows “shared libraries : disabled”. The configure log doesn’t appear to contain any warnings or errors about this (or maybe I couldn’t find it).
>>> 
>>> Normally, compiling shared objects with XL compilers require a -qmkshrobj flag. I’ve attempted passing this to PETSc using LDFLAGS but this causes the configuration to fail with an error, stating that it cannot run executables created with the C compiler.
>>> 
>>> This problem seems to be persistent across a number of PETSc versions. I need to use 3.4.5 for compatibility reasons, but I’ve attempted the same build with 3.5, 3.3 and 3.0 with the exact same results.
>>> 
>>> Would really appreciate it if anyone can offer guidance on how to force PETSc to build its shared library. The configure log from the above options is provided in the attachment.
>> 
>> Shared libraries with IBM compilers is something I've not tired. If its just a matter
>> of using '-qmkshrobj' - you could try to build it manually - and see if it works.
>> 
>> [when I try it I get errors.]
>> 
>> Satish
>> 
>> ----------
>> 
>> [balay at vestalac1 ~]$ cd petsc/arch-xl-opt-nopkg/lib/
>> [balay at vestalac1 lib]$ ar x libpetsc.a 
>> [balay at vestalac1 lib]$ mpixlc -qmkshrobj -o libpetsc.so *.o
>> [balay at vestalac1 lib]$ file libpetsc.so
>> libpetsc.so: ELF 64-bit MSB shared object, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, not stripped
>> [balay at vestalac1 lib]$ mv libpetsc.a libpetsc-bkp.a
>> [balay at vestalac1 lib]$ cd $PETSC_DIR
>> [balay at vestalac1 petsc]$ cd src/ksp/ksp/examples/tutorials/
>> [balay at vestalac1 tutorials]$ make ex2
>> mpixlc_r -o ex2.o -c -qpic -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction   -I/home/balay/petsc/include -I/home/balay/petsc/arch-xl-opt-nopkg/include -I/bgsys/drivers/V1R2M1/ppc64/comm/include -I/bgsys/drivers/V1R2M1/ppc64/comm/lib/gnu -I/bgsys/drivers/V1R2M1/ppc64 -I/bgsys/drivers/V1R2M1/ppc64/comm/sys/include -I/bgsys/drivers/V1R2M1/ppc64/spi/include -I/bgsys/drivers/V1R2M1/ppc64/spi/include/kernel/cnk    `pwd`/ex2.c
>> mpixlc_r -qpic -O3 -qhot=level=0 -qsimd=auto -qmaxmem=-1 -qstrict -qstrict_induction  -o ex2 ex2.o  -L/home/balay/petsc/arch-xl-opt-nopkg/lib  -lpetsc -L/soft/libraries/alcf/current/xl/LAPACK/lib -llapack -L/soft/libraries/alcf/current/xl/BLAS/lib -lblas -lpthread -L/bgsys/drivers/V1R2M1/ppc64/comm/lib -L/bgsys/drivers/V1R2M1/ppc64/comm/lib64 -L/bgsys/drivers/V1R2M1/ppc64/spi/lib -L/bgsys/drivers/V1R2M1/ppc64/comm/sys/lib -L/soft/compilers/ibmcmp-may2014/xlsmp/bg/3.1/bglib64 -L/soft/compilers/ibmcmp-may2014/xlmass/bg/7.3/bglib64 -L/soft/compilers/ibmcmp-may2014/vac/bg/12.1/bglib64 -L/soft/compilers/ibmcmp-may2014/vacpp/bg/12.1/bglib64 -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6 -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/lib/gcc -L/bgsys/drivers/toolchain/V1R2M1_base-efix033/gnu-linux/powerpc64-bgq-linux/lib -L/soft/compilers/ibmcmp-may2014/xlf/bg/14.1/bglib64 -Wl,-rpath,/soft/compilers/ibmcmp-may2014/lib64/bg/bglib64 -lmpichf90-gcc -lxlf90_r -lxlomp_ser -lxlfmath -lm -lmpichcxx-gcc -libmc++ -ldl -lmpich-gcc -lopa-gcc -lmpl-gcc -lpami-gcc -lSPI -lSPI_cnk -lrt -lpthread -lstdc++ -lxlopt -lxl -lnss_files -lnss_dns -lresolv -lgcc_eh -ldl 
>> /bgsys/drivers/ppcfloor/gnu-linux/powerpc64-bgq-linux/bin/ld: cannot find -lpetsc
>> make: [ex2] Error 1 (ignored)
>> /bin/rm -f ex2.o
>> [balay at vestalac1 tutorials]$ 