[petsc-users] --download-fblaslapack libraries cannot be used

[Fande Kong]

A PR here https://gitlab.com/petsc/petsc/-/merge_requests/2612

On Wed, Mar 18, 2020 at 3:35 PM Fande Kong <fdkong.jd at gmail.com> wrote:

> Thanks, Satish,
>
> I keep investigating into this issue. Now, I have more insights. The
> fundamental reason is that: Conda-compilers (installed by: conda install -c
> conda-forge compilers) have a bunch of system libs in */sysroot/lib and
> */sysroot/usr/lib. Most of them are related to glibc. These libs may or may
> not be compatible to the OS system you are using.
>
> PETSc will find these libs, and think they are just regular user libs, and
> then hard code with "-rpath".
>
> If I make some changes to ignore sysroot/lib*, and then everything
> runs smoothly ( do not need to install glibc because OS will have a right
> one).
>
>  git diff
> diff --git a/config/BuildSystem/config/compilers.py
> b/config/BuildSystem/config/compilers.py
> index 5367383141..6b26594b4e 100644
> --- a/config/BuildSystem/config/compilers.py
> +++ b/config/BuildSystem/config/compilers.py
> @@ -1132,6 +1132,7 @@ Otherwise you need a different combination of C,
> C++, and Fortran compilers")
>          if m:
>            arg = '-L'+os.path.abspath(arg[2:])
>            if arg in ['-L/usr/lib','-L/lib','-L/usr/lib64','-L/lib64']:
> continue
> +          if 'sysroot/usr/lib' in arg or 'sysroot/lib' in arg: continue
>            if not arg in lflags:
>              lflags.append(arg)
>              self.logPrint('Found library directory: '+arg, 4, 'compilers')
>
>
> PETSc should treat these libs as system-level libs???
>
> Have a branch here: Fande-Kong/skip_sysroot_libs_maint
>
> Any suggestion are appreciated,
>
> Thanks,
>
> Fande,
>
>
> On Tue, Mar 17, 2020 at 2:17 PM Satish Balay <balay at mcs.anl.gov> wrote:
>
>> Thanks for the update.
>>
>> Hopefully Matt can check on the issue with missing stuff in configure.log.
>>
>> The MR is at https://gitlab.com/petsc/petsc/-/merge_requests/2606
>>
>> Satish
>>
>>
>> On Tue, 17 Mar 2020, Fande Kong wrote:
>>
>> > On Tue, Mar 17, 2020 at 9:24 AM Satish Balay <balay at mcs.anl.gov> wrote:
>> >
>> > > So what was the initial problem? Did conda install gcc without glibc?
>> Or
>> > > was it using the wrong glibc?
>> > >
>> >
>> > Looks like GCC installed by conda uses an old version of glibc (2.12).
>> >
>> >
>> > > Because the compiler appeared partly functional [well the build worked
>> > > with just LIBS="-lmpifort -lgfortran"]
>> > >
>> > > And after the correct glibc was installed - did current maint still
>> fail
>> > > to build?
>> > >
>> >
>> > Still failed because PETSc claimed that: there were no needed fortran
>> > libraries when using mpicc as the linker. But in fact, we need these
>> > fortran stuffs when linking blaslapack and mumps.
>> >
>> >
>> > >
>> > > Can you send configure.log for this?
>> > >
>> > > And its not clear to me why balay/fix-checkFortranLibraries/maint
>> broke
>> > > before this fix. [for one configure.log was incomplete]
>> > >
>> >
>> > I am not 100% sure, but I think the complied and linked executable can
>> not
>> > run because  of "glibc_2.14' not found". The version of glibc was too
>> low.
>> >
>> >
>> > So current solution for me is that: your branch + a new version of glibc
>> > (2.18).
>> >
>> > Thanks,
>> >
>> > Fande,
>> >
>> >
>> >
>> > >
>> > > Satish
>> > >
>> > > On Tue, 17 Mar 2020, Fande Kong wrote:
>> > >
>> > > > Hi Satish,
>> > > >
>> > > > Could you merge your branch, balay/fix-checkFortranLibraries/maint,
>> into
>> > > > maint?
>> > > >
>> > > > I added glibc to my conda environment (conda install -c
>> dan_blanchard
>> > > > glibc), and your branch ran well.
>> > > >
>> > > > If you are interested, I attached the successful log file here.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Fande
>> > > >
>> > > > On Sat, Mar 14, 2020 at 5:01 PM Fande Kong <fdkong.jd at gmail.com>
>> wrote:
>> > > >
>> > > > > Without touching the configuration file, the
>> > > > > option: --download-hypre-configure-arguments='LIBS="-lmpifort
>> > > -lgfortran"',
>> > > > > also works.
>> > > > >
>> > > > >
>> > > > > Thanks, Satish,
>> > > > >
>> > > > >
>> > > > > Fande,
>> > > > >
>> > > > > On Sat, Mar 14, 2020 at 4:37 PM Fande Kong <fdkong.jd at gmail.com>
>> > > wrote:
>> > > > >
>> > > > >> OK. I finally got PETSc complied.
>> > > > >>
>> > > > >> "-lgfortran" was required by fblaslapack
>> > > > >> "-lmpifort" was required by mumps.
>> > > > >>
>> > > > >> However, I had to manually add the same thing for hypre as well:
>> > > > >>
>> > > > >> git diff
>> > > > >> diff --git a/config/BuildSystem/config/packages/hypre.py
>> > > > >> b/config/BuildSystem/config/packages/hypre.py
>> > > > >> index 4d915c312f..f4300230a6 100644
>> > > > >> --- a/config/BuildSystem/config/packages/hypre.py
>> > > > >> +++ b/config/BuildSystem/config/packages/hypre.py
>> > > > >> @@ -66,6 +66,7 @@ class Configure(config.package.GNUPackage):
>> > > > >>      args.append('--with-lapack-lib=" "')
>> > > > >>      args.append('--with-blas=no')
>> > > > >>      args.append('--with-lapack=no')
>> > > > >> +    args.append('LIBS="-lmpifort -lgfortran"')
>> > > > >>      if self.openmp.found:
>> > > > >>        args.append('--with-openmp')
>> > > > >>        self.usesopenmp = 'yes'
>> > > > >>
>> > > > >>
>> > > > >> Why hypre could not pick up LIBS options automatically?
>> > > > >>
>> > > > >>
>> > > > >> Thanks,
>> > > > >>
>> > > > >> Fande,
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> On Sat, Mar 14, 2020 at 2:49 PM Satish Balay via petsc-users <
>> > > > >> petsc-users at mcs.anl.gov> wrote:
>> > > > >>
>> > > > >>> Configure Options: --configModules=PETSc.Configure
>> > > > >>> --optionsModule=config.compilerOptions --download-hypre=1
>> > > > >>> --with-debugging=no --with-shared-libraries=1
>> > > --download-fblaslapack=1
>> > > > >>> --download-metis=1 --download-ptscotch=1 --download-parmetis=1
>> > > > >>> --download-superlu_dist=1 --download-mumps=1
>> --download-scalapack=1
>> > > > >>> --download-slepc=git://https://gitlab.com/slepc/slepc.git
>> > > > >>> --download-slepc-commit= 59ff81b --with-mpi=1
>> > > --with-cxx-dialect=C++11
>> > > > >>> --with-fortran-bindings=0 --with-sowing=0 CFLAGS=-march=nocona
>> > > > >>> -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong
>> > > -fno-plt -O2
>> > > > >>> -ffunction-sections -pipe -isystem
>> > > > >>> /home/kongf/workhome/rod/miniconda3/include CXXFLAGS=
>> LDFLAGS=-Wl,-O2
>> > > > >>> -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
>> > > > >>> -Wl,--with-new-dtags=0 -Wl,--gc-sections
>> > > > >>> -Wl,-rpath,/home/kongf/workhome/rod/miniconda3/lib
>> > > > >>> -Wl,-rpath-link,/home/kongf/workhome/rod/miniconda3/lib
>> > > > >>> -L/home/kongf/workhome/rod/miniconda3/lib
>> > > > >>>
>> > >
>> AR=/home/kongf/workhome/rod/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ar
>> > > > >>> --with-mpi-dir=/home/kongf/workhome/rod/mpich LIBS=-lgfortran
>> > > -lmpifort
>> > > > >>>
>> > > > >>> You are missing quotes with LIBS option - and likely the
>> libraries in
>> > > > >>> the wrong order.
>> > > > >>>
>> > > > >>> Suggest using:
>> > > > >>>
>> > > > >>> LIBS="-lmpifort -lgfortran"
>> > > > >>> or
>> > > > >>> 'LIBS=-lmpifort -lgfortran'
>> > > > >>>
>> > > > >>> Assuming you are invoking configure from shell.
>> > > > >>>
>> > > > >>> Satish
>> > > > >>>
>> > > > >>> On Sat, 14 Mar 2020, Satish Balay via petsc-users wrote:
>> > > > >>>
>> > > > >>> > to work around - you can try:
>> > > > >>> >
>> > > > >>> > LIBS="-lmpifort -lgfortran"
>> > > > >>> >
>> > > > >>> > Satish
>> > > > >>> >
>> > > > >>> > On Sat, 14 Mar 2020, Satish Balay via petsc-users wrote:
>> > > > >>> >
>> > > > >>> > > Its the same location as before. For some reason configure
>> is not
>> > > > >>> saving the relevant logs.
>> > > > >>> > >
>> > > > >>> > > I don't understand saveLog() restoreLog() stuff. Matt, can
>> you
>> > > check
>> > > > >>> on this?
>> > > > >>> > >
>> > > > >>> > > Satish
>> > > > >>> > >
>> > > > >>> > > On Sat, 14 Mar 2020, Fande Kong wrote:
>> > > > >>> > >
>> > > > >>> > > > The configuration crashed earlier than before with your
>> > > changes.
>> > > > >>> > > >
>> > > > >>> > > > Please see the attached log file when using your branch.
>> The
>> > > > >>> trouble lines
>> > > > >>> > > > should be:
>> > > > >>> > > >
>> > > > >>> > > >  "    asub=self.mangleFortranFunction("asub")
>> > > > >>> > > >     cbody = "extern void "+asub+"(void);\nint main(int
>> > > argc,char
>> > > > >>> > > > **args)\n{\n  "+asub+"();\n  return 0;\n}\n";
>> > > > >>> > > > "
>> > > > >>> > > >
>> > > > >>> > > > Thanks,
>> > > > >>> > > >
>> > > > >>> > > > Fande,
>> > > > >>> > > >
>> > > > >>> > > > On Thu, Mar 12, 2020 at 7:06 PM Satish Balay <
>> > > balay at mcs.anl.gov>
>> > > > >>> wrote:
>> > > > >>> > > >
>> > > > >>> > > > > I can't figure out what the stack in the attached
>> > > configure.log.
>> > > > >>> [likely
>> > > > >>> > > > > some stuff isn't getting logged in it]
>> > > > >>> > > > >
>> > > > >>> > > > > Can you retry with branch
>> > > > >>> 'balay/fix-checkFortranLibraries/maint'?
>> > > > >>> > > > >
>> > > > >>> > > > > Satish
>> > > > >>> > > > >
>> > > > >>> > > > > On Thu, 12 Mar 2020, Fande Kong wrote:
>> > > > >>> > > > >
>> > > > >>> > > > > > Thanks, Satish,
>> > > > >>> > > > > >
>> > > > >>> > > > > > But still have the problem. Please see the attached
>> log
>> > > file.
>> > > > >>> > > > > >
>> > > > >>> > > > > > Thanks,
>> > > > >>> > > > > >
>> > > > >>> > > > > > Fande.
>> > > > >>> > > > > >
>> > > > >>> > > > > > On Thu, Mar 12, 2020 at 3:42 PM Satish Balay <
>> > > > >>> balay at mcs.anl.gov> wrote:
>> > > > >>> > > > > >
>> > > > >>> > > > > > > Can you retry with the attached patch?
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > BTW: Its best to use the latest patched version -
>> i.e
>> > > > >>> > > > > petsc-3.12.4.tar.gz
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Satish
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > On Thu, 12 Mar 2020, Fande Kong wrote:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > > This fixed the fblaslapack issue. Now have another
>> > > issue
>> > > > >>> about mumps.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Please see the log file attached.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Thanks,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Fande,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > On Thu, Mar 12, 2020 at 1:38 PM Satish Balay <
>> > > > >>> balay at mcs.anl.gov>
>> > > > >>> > > > > wrote:
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > > For some reason - the fortran compiler libraries
>> > > check
>> > > > >>> worked fine
>> > > > >>> > > > > > > without
>> > > > >>> > > > > > > > > -lgfortran.
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > But now - flbaslapack check is failing without
>> it.
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > To work arround - you can use option
>> LIBS=-lgfortran
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Satish
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > On Thu, 12 Mar 2020, Fande Kong wrote:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > > Hi All,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > I had an issue when configuring petsc on a
>> linux
>> > > > >>> machine. I have
>> > > > >>> > > > > the
>> > > > >>> > > > > > > > > > following error message:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >      Compiling FBLASLAPACK; this may take
>> several
>> > > > >>> minutes
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > >
>> > > > >>>
>> > >
>> ===============================================================================
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >    TESTING: checkLib from
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > >
>> > > > >>>
>> > >
>> config.packages.BlasLapack(config/BuildSystem/config/packages/BlasLapack.py:120)
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > >
>> > > > >>>
>> > >
>> *******************************************************************************
>> > > > >>> > > > > > > > > >          UNABLE to CONFIGURE with GIVEN
>> OPTIONS
>> > > (see
>> > > > >>> > > > > > > configure.log for
>> > > > >>> > > > > > > > > > details):
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > >
>> > > > >>>
>> > >
>> -------------------------------------------------------------------------------
>> > > > >>> > > > > > > > > > --download-fblaslapack libraries cannot be
>> used
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > >
>> > > > >>>
>> > >
>> *******************************************************************************
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > The configuration log was attached.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Thanks,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Fande,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > > >>
>> > > >
>> > >
>> > >
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200318/32c41826/attachment-0001.html>