<div dir="ltr"><div dir="ltr">A PR here <a href="https://gitlab.com/petsc/petsc/-/merge_requests/2612">https://gitlab.com/petsc/petsc/-/merge_requests/2612</a></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 18, 2020 at 3:35 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com">fdkong.jd@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Thanks, Satish,<div><br></div><div>I keep investigating into this issue. Now, I have more insights. The fundamental reason is that: Conda-compilers (installed by: conda install -c conda-forge compilers) have a bunch of system libs in */sysroot/lib and */sysroot/usr/lib. Most of them are related to glibc. These libs may or may not be compatible to the OS system you are using. </div><div><br></div><div>PETSc will find these libs, and think they are just regular user libs, and then hard code with "-rpath". </div><div><br></div><div>If I make some changes to ignore sysroot/lib*, and then everything runs smoothly ( do not need to install glibc because OS will have a right one).</div><div><br></div><div><div> git diff </div><div>diff --git a/config/BuildSystem/config/compilers.py b/config/BuildSystem/config/compilers.py</div><div>index 5367383141..6b26594b4e 100644</div><div>--- a/config/BuildSystem/config/compilers.py</div><div>+++ b/config/BuildSystem/config/compilers.py</div><div>@@ -1132,6 +1132,7 @@ Otherwise you need a different combination of C, C++, and Fortran compilers")</div><div> if m:</div><div> arg = '-L'+os.path.abspath(arg[2:])</div><div> if arg in ['-L/usr/lib','-L/lib','-L/usr/lib64','-L/lib64']: continue</div><div>+ if 'sysroot/usr/lib' in arg or 'sysroot/lib' in arg: continue</div><div> if not arg in lflags:</div><div> lflags.append(arg)</div><div> self.logPrint('Found library directory: '+arg, 4, 'compilers')</div></div><div><br></div><div><br></div><div>PETSc should treat these libs as system-level libs???</div><div><br></div><div>Have a branch here: Fande-Kong/skip_sysroot_libs_maint</div><div><br></div><div>Any suggestion are appreciated,</div><div><br></div><div>Thanks,</div><div><br></div><div>Fande,</div><div><br></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 17, 2020 at 2:17 PM Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Thanks for the update.<br>
<br>
Hopefully Matt can check on the issue with missing stuff in configure.log.<br>
<br>
The MR is at <a href="https://gitlab.com/petsc/petsc/-/merge_requests/2606" rel="noreferrer" target="_blank">https://gitlab.com/petsc/petsc/-/merge_requests/2606</a><br>
<br>
Satish<br>
<br>
<br>
On Tue, 17 Mar 2020, Fande Kong wrote:<br>
<br>
> On Tue, Mar 17, 2020 at 9:24 AM Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
> <br>
> > So what was the initial problem? Did conda install gcc without glibc? Or<br>
> > was it using the wrong glibc?<br>
> ><br>
> <br>
> Looks like GCC installed by conda uses an old version of glibc (2.12).<br>
> <br>
> <br>
> > Because the compiler appeared partly functional [well the build worked<br>
> > with just LIBS="-lmpifort -lgfortran"]<br>
> ><br>
> > And after the correct glibc was installed - did current maint still fail<br>
> > to build?<br>
> ><br>
> <br>
> Still failed because PETSc claimed that: there were no needed fortran<br>
> libraries when using mpicc as the linker. But in fact, we need these<br>
> fortran stuffs when linking blaslapack and mumps.<br>
> <br>
> <br>
> ><br>
> > Can you send configure.log for this?<br>
> ><br>
> > And its not clear to me why balay/fix-checkFortranLibraries/maint broke<br>
> > before this fix. [for one configure.log was incomplete]<br>
> ><br>
> <br>
> I am not 100% sure, but I think the complied and linked executable can not<br>
> run because of "glibc_2.14' not found". The version of glibc was too low.<br>
> <br>
> <br>
> So current solution for me is that: your branch + a new version of glibc<br>
> (2.18).<br>
> <br>
> Thanks,<br>
> <br>
> Fande,<br>
> <br>
> <br>
> <br>
> ><br>
> > Satish<br>
> ><br>
> > On Tue, 17 Mar 2020, Fande Kong wrote:<br>
> ><br>
> > > Hi Satish,<br>
> > ><br>
> > > Could you merge your branch, balay/fix-checkFortranLibraries/maint, into<br>
> > > maint?<br>
> > ><br>
> > > I added glibc to my conda environment (conda install -c dan_blanchard<br>
> > > glibc), and your branch ran well.<br>
> > ><br>
> > > If you are interested, I attached the successful log file here.<br>
> > ><br>
> > > Thanks,<br>
> > ><br>
> > > Fande<br>
> > ><br>
> > > On Sat, Mar 14, 2020 at 5:01 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>> wrote:<br>
> > ><br>
> > > > Without touching the configuration file, the<br>
> > > > option: --download-hypre-configure-arguments='LIBS="-lmpifort<br>
> > -lgfortran"',<br>
> > > > also works.<br>
> > > ><br>
> > > ><br>
> > > > Thanks, Satish,<br>
> > > ><br>
> > > ><br>
> > > > Fande,<br>
> > > ><br>
> > > > On Sat, Mar 14, 2020 at 4:37 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>><br>
> > wrote:<br>
> > > ><br>
> > > >> OK. I finally got PETSc complied.<br>
> > > >><br>
> > > >> "-lgfortran" was required by fblaslapack<br>
> > > >> "-lmpifort" was required by mumps.<br>
> > > >><br>
> > > >> However, I had to manually add the same thing for hypre as well:<br>
> > > >><br>
> > > >> git diff<br>
> > > >> diff --git a/config/BuildSystem/config/packages/hypre.py<br>
> > > >> b/config/BuildSystem/config/packages/hypre.py<br>
> > > >> index 4d915c312f..f4300230a6 100644<br>
> > > >> --- a/config/BuildSystem/config/packages/hypre.py<br>
> > > >> +++ b/config/BuildSystem/config/packages/hypre.py<br>
> > > >> @@ -66,6 +66,7 @@ class Configure(config.package.GNUPackage):<br>
> > > >> args.append('--with-lapack-lib=" "')<br>
> > > >> args.append('--with-blas=no')<br>
> > > >> args.append('--with-lapack=no')<br>
> > > >> + args.append('LIBS="-lmpifort -lgfortran"')<br>
> > > >> if self.openmp.found:<br>
> > > >> args.append('--with-openmp')<br>
> > > >> self.usesopenmp = 'yes'<br>
> > > >><br>
> > > >><br>
> > > >> Why hypre could not pick up LIBS options automatically?<br>
> > > >><br>
> > > >><br>
> > > >> Thanks,<br>
> > > >><br>
> > > >> Fande,<br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > >><br>
> > > >> On Sat, Mar 14, 2020 at 2:49 PM Satish Balay via petsc-users <<br>
> > > >> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
> > > >><br>
> > > >>> Configure Options: --configModules=PETSc.Configure<br>
> > > >>> --optionsModule=config.compilerOptions --download-hypre=1<br>
> > > >>> --with-debugging=no --with-shared-libraries=1<br>
> > --download-fblaslapack=1<br>
> > > >>> --download-metis=1 --download-ptscotch=1 --download-parmetis=1<br>
> > > >>> --download-superlu_dist=1 --download-mumps=1 --download-scalapack=1<br>
> > > >>> --download-slepc=git://<a href="https://gitlab.com/slepc/slepc.git" rel="noreferrer" target="_blank">https://gitlab.com/slepc/slepc.git</a><br>
> > > >>> --download-slepc-commit= 59ff81b --with-mpi=1<br>
> > --with-cxx-dialect=C++11<br>
> > > >>> --with-fortran-bindings=0 --with-sowing=0 CFLAGS=-march=nocona<br>
> > > >>> -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong<br>
> > -fno-plt -O2<br>
> > > >>> -ffunction-sections -pipe -isystem<br>
> > > >>> /home/kongf/workhome/rod/miniconda3/include CXXFLAGS= LDFLAGS=-Wl,-O2<br>
> > > >>> -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now<br>
> > > >>> -Wl,--with-new-dtags=0 -Wl,--gc-sections<br>
> > > >>> -Wl,-rpath,/home/kongf/workhome/rod/miniconda3/lib<br>
> > > >>> -Wl,-rpath-link,/home/kongf/workhome/rod/miniconda3/lib<br>
> > > >>> -L/home/kongf/workhome/rod/miniconda3/lib<br>
> > > >>><br>
> > AR=/home/kongf/workhome/rod/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ar<br>
> > > >>> --with-mpi-dir=/home/kongf/workhome/rod/mpich LIBS=-lgfortran<br>
> > -lmpifort<br>
> > > >>><br>
> > > >>> You are missing quotes with LIBS option - and likely the libraries in<br>
> > > >>> the wrong order.<br>
> > > >>><br>
> > > >>> Suggest using:<br>
> > > >>><br>
> > > >>> LIBS="-lmpifort -lgfortran"<br>
> > > >>> or<br>
> > > >>> 'LIBS=-lmpifort -lgfortran'<br>
> > > >>><br>
> > > >>> Assuming you are invoking configure from shell.<br>
> > > >>><br>
> > > >>> Satish<br>
> > > >>><br>
> > > >>> On Sat, 14 Mar 2020, Satish Balay via petsc-users wrote:<br>
> > > >>><br>
> > > >>> > to work around - you can try:<br>
> > > >>> ><br>
> > > >>> > LIBS="-lmpifort -lgfortran"<br>
> > > >>> ><br>
> > > >>> > Satish<br>
> > > >>> ><br>
> > > >>> > On Sat, 14 Mar 2020, Satish Balay via petsc-users wrote:<br>
> > > >>> ><br>
> > > >>> > > Its the same location as before. For some reason configure is not<br>
> > > >>> saving the relevant logs.<br>
> > > >>> > ><br>
> > > >>> > > I don't understand saveLog() restoreLog() stuff. Matt, can you<br>
> > check<br>
> > > >>> on this?<br>
> > > >>> > ><br>
> > > >>> > > Satish<br>
> > > >>> > ><br>
> > > >>> > > On Sat, 14 Mar 2020, Fande Kong wrote:<br>
> > > >>> > ><br>
> > > >>> > > > The configuration crashed earlier than before with your<br>
> > changes.<br>
> > > >>> > > ><br>
> > > >>> > > > Please see the attached log file when using your branch. The<br>
> > > >>> trouble lines<br>
> > > >>> > > > should be:<br>
> > > >>> > > ><br>
> > > >>> > > > " asub=self.mangleFortranFunction("asub")<br>
> > > >>> > > > cbody = "extern void "+asub+"(void);\nint main(int<br>
> > argc,char<br>
> > > >>> > > > **args)\n{\n "+asub+"();\n return 0;\n}\n";<br>
> > > >>> > > > "<br>
> > > >>> > > ><br>
> > > >>> > > > Thanks,<br>
> > > >>> > > ><br>
> > > >>> > > > Fande,<br>
> > > >>> > > ><br>
> > > >>> > > > On Thu, Mar 12, 2020 at 7:06 PM Satish Balay <<br>
> > <a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>><br>
> > > >>> wrote:<br>
> > > >>> > > ><br>
> > > >>> > > > > I can't figure out what the stack in the attached<br>
> > configure.log.<br>
> > > >>> [likely<br>
> > > >>> > > > > some stuff isn't getting logged in it]<br>
> > > >>> > > > ><br>
> > > >>> > > > > Can you retry with branch<br>
> > > >>> 'balay/fix-checkFortranLibraries/maint'?<br>
> > > >>> > > > ><br>
> > > >>> > > > > Satish<br>
> > > >>> > > > ><br>
> > > >>> > > > > On Thu, 12 Mar 2020, Fande Kong wrote:<br>
> > > >>> > > > ><br>
> > > >>> > > > > > Thanks, Satish,<br>
> > > >>> > > > > ><br>
> > > >>> > > > > > But still have the problem. Please see the attached log<br>
> > file.<br>
> > > >>> > > > > ><br>
> > > >>> > > > > > Thanks,<br>
> > > >>> > > > > ><br>
> > > >>> > > > > > Fande.<br>
> > > >>> > > > > ><br>
> > > >>> > > > > > On Thu, Mar 12, 2020 at 3:42 PM Satish Balay <<br>
> > > >>> <a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
> > > >>> > > > > ><br>
> > > >>> > > > > > > Can you retry with the attached patch?<br>
> > > >>> > > > > > ><br>
> > > >>> > > > > > > BTW: Its best to use the latest patched version - i.e<br>
> > > >>> > > > > petsc-3.12.4.tar.gz<br>
> > > >>> > > > > > ><br>
> > > >>> > > > > > > Satish<br>
> > > >>> > > > > > ><br>
> > > >>> > > > > > > On Thu, 12 Mar 2020, Fande Kong wrote:<br>
> > > >>> > > > > > ><br>
> > > >>> > > > > > > > This fixed the fblaslapack issue. Now have another<br>
> > issue<br>
> > > >>> about mumps.<br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > > > Please see the log file attached.<br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > > > Thanks,<br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > > > Fande,<br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > > > On Thu, Mar 12, 2020 at 1:38 PM Satish Balay <<br>
> > > >>> <a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>><br>
> > > >>> > > > > wrote:<br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > > > > For some reason - the fortran compiler libraries<br>
> > check<br>
> > > >>> worked fine<br>
> > > >>> > > > > > > without<br>
> > > >>> > > > > > > > > -lgfortran.<br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > > But now - flbaslapack check is failing without it.<br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > > To work arround - you can use option LIBS=-lgfortran<br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > > Satish<br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > > On Thu, 12 Mar 2020, Fande Kong wrote:<br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > > > Hi All,<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > I had an issue when configuring petsc on a linux<br>
> > > >>> machine. I have<br>
> > > >>> > > > > the<br>
> > > >>> > > > > > > > > > following error message:<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > Compiling FBLASLAPACK; this may take several<br>
> > > >>> minutes<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > ><br>
> > > >>><br>
> > ===============================================================================<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > TESTING: checkLib from<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > ><br>
> > > >>><br>
> > config.packages.BlasLapack(config/BuildSystem/config/packages/BlasLapack.py:120)<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > ><br>
> > > >>><br>
> > *******************************************************************************<br>
> > > >>> > > > > > > > > > UNABLE to CONFIGURE with GIVEN OPTIONS<br>
> > (see<br>
> > > >>> > > > > > > configure.log for<br>
> > > >>> > > > > > > > > > details):<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > ><br>
> > > >>><br>
> > -------------------------------------------------------------------------------<br>
> > > >>> > > > > > > > > > --download-fblaslapack libraries cannot be used<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > ><br>
> > > >>><br>
> > *******************************************************************************<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > The configuration log was attached.<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > Thanks,<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > > > Fande,<br>
> > > >>> > > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > > ><br>
> > > >>> > > > > > > ><br>
> > > >>> > > > > > ><br>
> > > >>> > > > > ><br>
> > > >>> > > > ><br>
> > > >>> > > > ><br>
> > > >>> > > ><br>
> > > >>> > ><br>
> > > >>> ><br>
> > > >>><br>
> > > >><br>
> > ><br>
> ><br>
> ><br>
> <br>
<br>
</blockquote></div>
</blockquote></div>