[petsc-users] 3.16.4 amd blis libflame linking problem

[Anton Popov]

On 04.02.22 22:18, Satish Balay wrote:
> On Fri, 4 Feb 2022, Anton Popov wrote:
>
>> Am 04.02.2022 um 18:54 schrieb Satish Balay:
>>> On Fri, 4 Feb 2022, Anton Popov wrote:
>>>
>>>> Thanks Matt, Barry and Satish, for  your suggestions.
>>>>
>>>> The problem was indeed in mismatch between the gfortran library version
>>>> (system has libgfortran.so.4 libflame needs libgfortran.so.5)
>>>>
>>>> 3.9.4 did not detect this during configure, but only gave error later
>>>> during
>>>> test.
>>>>
>>>> 3.16.4 detected this immediately.
>>>>
>>>> After installing libgfortran.so.5, both PETSc versions install just fine,
>>>> however I get the warning mentioned by Satish.
>>> Hm - you could install libgfortran.so.5 without the corresponding compiler?
>> Yes, apparently it is possible on Ubuntu 18.04.
> Presumably it also comes with the corresponding gcc/gfortran [gcc-8, gfortran-8?]. Perhaps these compilers would avoid this issue.
>
>>>>> /opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.so
>>> Ah - this is a binary download - not something compiled locally?
>> Yes it is the latest binary from AMD website.
>>> I wonder if the compiler gave both an error [return code] and a warning
>>> previously [with the missing libgfortran.so.5] - as Barry was suggesting.
>> It was an error at runtime, configure was just fine.
>>>
>>> As you say - you still get that warning [which is expected - as you are
>>> using a different version of gfortran than what libflame.so was built with]
>> Now since libgfortran.so.5 coexists with libgfortran.so.4 it links against the
>> proper one, but gives a warning that both versions are available.
>>
>> I think I will just compile BLIS and LibFLAME from sources on my system to
>> avoid these problems altogether.
> Yeah - generally building everything with the same version of the compiler avoids these issues.

Actually this is the fastest way to go and it works: 
--download-f2cblaslapack --download-blis

Are there plans to also support  --download-libflame?

I know that most of performance boost comes from BLIS in this case. But 
maybe LibFLAME would also run a bit faster than a reference LAPACK.

Anton

>
> Satish
>
>> Best,
>>
>> Anton
>>
>>
>>> Satish
>>>
>>>> Maybe it is indeed worth upgrading everything to the compatible versions.
>>>>
>>>> Best,
>>>>
>>>> Anton
>>>>
>>>>
>>>> On 04.02.22 18:33, Barry Smith wrote:
>>>>>> On Feb 4, 2022, at 12:27 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>>>>
>>>>>> Probably best if you can use the same version of gfortran to build both
>>>>>> petsc and libflame/blis
>>>>>>
>>>>>>> /usr/bin/ld: warning: libgfortran.so.5, needed by
>>>>>>> /opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.so, not found (try using
>>>>>>> -rpath or -rpath-link):
>>>>>> This is probably an ignore-able warning - but configure defaults to
>>>>>> -Werror
>>>>>> mode here.
>>>>>      Hmm, if it needs libgfortran.so.5 then it needs it and it cannot be
>>>>>      ignored since a link cannot succeed. Flame presumably contains a lot
>>>>>      of
>>>>>      old Fortran code from Lapack so would normally need the fortran
>>>>>      libraries.
>>>>>
>>>>>> Wrt forcing link with static libraries - you can try:
>>>>>>
>>>>>> LIBS="/opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.a
>>>>>> /opt/amd/amd-blis-3.1.0/lib/lp64/libblis.a"
>>>>>>
>>>>>> [instead of --with-blas-lib= --with-lapack-lib= options].
>>>>>>
>>>>>> Satish
>>>>>>
>>>>>>
>>>>>> On Fri, 4 Feb 2022, Barry Smith wrote:
>>>>>>
>>>>>>> Please do
>>>>>>>
>>>>>>>     ldd -O /opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.so
>>>>>>>
>>>>>>> You may need to list the gfortran library directory of libgfortran.so.5
>>>>>>> it
>>>>>>> needs to use in LDFLAGS passed to PETSc configure
>>>>>>>
>>>>>>> Barry
>>>>>>>
>>>>>>> Note: Even though you explicitly listed a static library of libflame to
>>>>>>> use our configure is goofy and loses that information and wants to link
>>>>>>> with the shared version
>>>>>>>
>>>>>>>
>>>>>>>> On Feb 4, 2022, at 12:00 PM, Anton Popov <popov at uni-mainz.de> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 04.02.22 17:39, Matthew Knepley wrote:
>>>>>>>>> On Fri, Feb 4, 2022 at 11:35 AM Anton Popov <popov at uni-mainz.de
>>>>>>>>> <mailto:popov at uni-mainz.de>> wrote:
>>>>>>>>> Hi Satish,
>>>>>>>>>
>>>>>>>>> I just discovered that PETSc 3.16.4 fails to link against the latest
>>>>>>>>> AMD
>>>>>>>>> BLIS and LibFLAME libraries on a Linux box.
>>>>>>>>>
>>>>>>>>> -------------------------------------------------------------------------------
>>>>>>>>> You set a value for --with-blas-lib=<lib> and --with-lapack-lib=<lib>,
>>>>>>>>> but ['/opt/amd/amd-blis-3.1.0/lib/lp64/libblis.a'] and
>>>>>>>>> ['/opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.a'] cannot be used
>>>>>>>>> *******************************************************************************
>>>>>>>>>
>>>>>>>>> My previous experience with 3.9.4 on the same system was fully
>>>>>>>>> successful. Looking in the configure logs (attached) reveals small
>>>>>>>>> difference in the linking compared to 3.9.4
>>>>>>>>>
>>>>>>>>> Could you please make a guess what went wrong?
>>>>>>>>>
>>>>>>>>> Down in the log I see:
>>>>>>>>>
>>>>>>>>> /usr/bin/ld: warning: libgfortran.so.5, needed by
>>>>>>>>> /opt/amd/amd-libflame-3.1.0/lib/lp64/libflame.so, not found (try using
>>>>>>>>> -rpath or -rpath-link):
>>>>>>>> Thanks Matt, I'll try.
>>>>>>>>
>>>>>>>>> Did the gfortran library move or get upgraded?
>>>>>>>> Not at all. I have configured 3.9.4 just now to make a test, and it
>>>>>>>> perfectly finds all the libraries. So there must be something that
>>>>>>>> 3.16.4
>>>>>>>> does differently.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>>
>>>>>>>> Anton
>>>>>>>>
>>>>>>>>>     Thanks,
>>>>>>>>>
>>>>>>>>>        Matt
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> Anton
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>>>>> their experiments lead.
>>>>>>>>> -- Norbert Wiener
>>>>>>>>>
>>>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>>>> <http://www.cse.buffalo.edu/~knepley/>