[petsc-users] Exhausted all shared linker guesses. Could not determine how to create a shared library!

Barry Smith bsmith at petsc.dev
Wed Mar 10 23:19:25 CST 2021 

  Excellent, thanks for letting me know. It is a bit miraculous since I could not reproduce your problem (and was not willing to install Conda myself).

  https://gitlab.com/petsc/petsc/-/merge_requests/3703



> On Mar 10, 2021, at 8:46 PM, Fande Kong <fdkong.jd at gmail.com> wrote:
> 
> Thanks Barry,
> 
> Your branch works very well. Thanks for your help!!!
> 
> Could you merge it to upstream?
> 
> Fande
> 
> On Wed, Mar 10, 2021 at 6:30 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
> 
>   Fande,
> 
>      Before send the files I requested in my last email could you try with the branch barry/2021-03-10/handle-pie-flag-conda/release and send its configure.log if it fails.
> 
>    Thanks
> 
>     Barry
> 
> 
>> On Mar 10, 2021, at 5:59 PM, Fande Kong <fdkong.jd at gmail.com <mailto:fdkong.jd at gmail.com>> wrote:
>> 
>> Do not know what the fix should look like, but this works for me
>> 
>> 
>>  @staticmethod
>> @@ -1194,7 +1194,6 @@ class Configure(config.base.Configure):
>>      output.find('unrecognized command line option') >= 0 or output.find('unrecognized option') >= 0 or output.find('unrecognised option') >= 0 or
>>      output.find('not recognized') >= 0 or output.find('not recognised') >= 0 or
>>      output.find('unknown option') >= 0 or output.find('unknown flag') >= 0 or output.find('Unknown switch') >= 0 or
>> -    output.find('ignoring option') >= 0 or output.find('ignored') >= 0 or
>>      output.find('argument unused') >= 0 or output.find('not supported') >= 0 or
>>      # When checking for the existence of 'attribute'
>>      output.find('is unsupported and will be skipped') >= 0 or
>> 
>> 
>> 
>> Thanks,
>> 
>> Fande
>> 
>> On Wed, Mar 10, 2021 at 4:21 PM Fande Kong <fdkong.jd at gmail.com <mailto:fdkong.jd at gmail.com>> wrote:
>> 
>> 
>> On Wed, Mar 10, 2021 at 1:36 PM Satish Balay <balay at mcs.anl.gov <mailto:balay at mcs.anl.gov>> wrote:
>> Can you use a different MPI for this conda install?
>> 
>> We control how to build MPI. If I take "-pie" options out of LDFLAGS, conda can not compile mpich.
>> 
>> 
>>  
>> 
>> Alternative:
>> 
>> ./configure CC=x86_64-apple-darwin13.4.0-clang COPTFLAGS="-march=core2 -mtune=haswell" CPPFLAGS=-I/Users/kongf/miniconda3/envs/testpetsc/include
>>  LDFLAGS="-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-commons,use_dylibs" LIBS="-Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi"
>> 
>> MPI can not generate an executable because we took out "-pie". 
>> 
>> Thanks,
>> 
>> Fande
>>  
>> 
>> etc.. [don't know if you really need LDFLAGS options]
>> 
>> Satish
>> 
>> On Wed, 10 Mar 2021, Fande Kong wrote:
>> 
>> > I guess it was encoded in mpicc
>> > 
>> > petsc % mpicc -show
>> > x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -Wl,-pie
>> > -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs
>> > -Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib
>> > -L/Users/kongf/miniconda3/envs/testpetsc/lib -Wl,-commons,use_dylibs
>> > -I/Users/kongf/miniconda3/envs/testpetsc/include
>> > -L/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi
>> > 
>> > 
>> > Thanks,
>> > 
>> > Fande
>> > 
>> > On Wed, Mar 10, 2021 at 12:51 PM Satish Balay <balay at mcs.anl.gov <mailto:balay at mcs.anl.gov>> wrote:
>> > 
>> > > > LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs -rpath
>> > > /Users/kongf/miniconda3/envs/testpetsc/lib
>> > > -L/Users/kongf/miniconda3/envs/testpetsc/lib
>> > >
>> > > Does conda compiler pick up '-pie' from this env variable? If so - perhaps
>> > > its easier to just modify it?
>> > >
>> > > Or is it encoded in mpicc wrapper? [mpicc -show]
>> > >
>> > > Satish
>> > >
>> > > On Wed, 10 Mar 2021, Fande Kong wrote:
>> > >
>> > > > Thanks Barry,
>> > > >
>> > > > Got the same result, but  "-pie" was not filtered out somehow.
>> > > >
>> > > > I did changes like this:
>> > > >
>> > > > kongf at x86_64-apple-darwin13 petsc % git diff
>> > > > diff --git a/config/BuildSystem/config/framework.py
>> > > > b/config/BuildSystem/config/framework.py
>> > > > index beefe82956..c31fbeb95e 100644
>> > > > --- a/config/BuildSystem/config/framework.py
>> > > > +++ b/config/BuildSystem/config/framework.py
>> > > > @@ -504,6 +504,8 @@ class Framework(config.base.Configure,
>> > > > script.LanguageProcessor):
>> > > >     lines = [s for s in lines if s.find('Load a valid targeting module or
>> > > > set CRAY_CPU_TARGET') < 0]
>> > > >     # pgi dumps filename on stderr - but returns 0 errorcode'
>> > > >     lines = [s for s in lines if lines != 'conftest.c:']
>> > > > +   # in case -pie is always being passed to linker
>> > > > +   lines = [s for s in lines if s.find('-pie being ignored. It is only
>> > > > used when linking a main executable') < 0]
>> > > >     if lines: output = reduce(lambda s, t: s+t, lines, '\n')
>> > > >     else: output = ''
>> > > >     log.write("Linker stderr after filtering:\n"+output+":\n")
>> > > >
>> > > > The log was attached again.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Fande
>> > > >
>> > > >
>> > > > On Wed, Mar 10, 2021 at 12:05 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>> > > >
>> > > > >  Fande,
>> > > > >
>> > > > >    Please add in config/BuildSystem/config/framework.py line 528 two
>> > > new
>> > > > > lines
>> > > > >
>> > > > >       # pgi dumps filename on stderr - but returns 0 errorcode'
>> > > > >       lines = [s for s in lines if lines != 'conftest.c:']
>> > > > >       # in case -pie is always being passed to linker
>> > > > >       lines = [s for s in lines if s.find('-pie being ignored. It is
>> > > only
>> > > > > used when linking a main executable') < 0]
>> > > > >
>> > > > >    Barry
>> > > > >
>> > > > >    You have (another of Conda's "take over the world my way" approach)
>> > > > >
>> > > > >    LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs
>> > > -rpath
>> > > > > /Users/kongf/miniconda3/envs/testpetsc/lib
>> > > > > -L/Users/kongf/miniconda3/envs/testpetsc/lib
>> > > > >
>> > > > > Executing: mpicc  -o
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest
>> > > > >  -dynamiclib -single_module
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o
>> > > > > Possible ERROR while running linker:
>> > > > > stderr:
>> > > > > ld: warning: -pie being ignored. It is only used when linking a main
>> > > > > executable
>> > > > >             Rejecting C linker flag -dynamiclib -single_module due to
>> > > > >
>> > > > > ld: warning: -pie being ignored. It is only used when linking a main
>> > > > > executable
>> > > > >
>> > > > > This is the correct link command for the Mac but it is being rejected
>> > > due
>> > > > > to the warning message.
>> > > > >
>> > > > >
>> > > > > On Mar 10, 2021, at 10:11 AM, Fande Kong <fdkong.jd at gmail.com <mailto:fdkong.jd at gmail.com>> wrote:
>> > > > >
>> > > > > Thanks, Barry,
>> > > > >
>> > > > > It seems PETSc works fine with manually built compilers. We are pretty
>> > > > > much sure that the issue is related to conda. Conda might introduce
>> > > extra
>> > > > > flags.
>> > > > >
>> > > > > We still need to make it work with conda because we deliver our package
>> > > > > via conda for users.
>> > > > >
>> > > > >
>> > > > > I unset all flags from conda, and got slightly different results this
>> > > > > time.  The log was attached. Anyone could  explain the motivation that
>> > > we
>> > > > > try to build executable without a main function?
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Fande
>> > > > >
>> > > > > Executing: mpicc -c -o
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o
>> > > > >
>> > > -I/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers
>> > > > >  -fPIC
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.c
>> > > > >
>> > > > > Successful compile:
>> > > > > Source:
>> > > > > #include "confdefs.h"
>> > > > > #include "conffix.h"
>> > > > > #include <stdio.h>
>> > > > > int (*fprintf_ptr)(FILE*,const char*,...) = fprintf;
>> > > > > void  foo(void){
>> > > > >   fprintf_ptr(stdout,"hello");
>> > > > >   return;
>> > > > > }
>> > > > > void bar(void){foo();}
>> > > > > Running Executable WITHOUT threads to time it out
>> > > > > Executing: mpicc  -o
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/libconftest.so
>> > > > >  -dynamic  -fPIC
>> > > > >
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o
>> > > > >
>> > > > > Possible ERROR while running linker: exit code 1
>> > > > > stderr:
>> > > > > Undefined symbols for architecture x86_64:
>> > > > >   "_main", referenced from:
>> > > > >      implicit entry/start for main executable
>> > > > > ld: symbol(s) not found for architecture x86_64
>> > > > > clang-11: error: linker command failed with exit code 1 (use -v to see
>> > > > > invocation)
>> > > > >           Rejected C compiler flag -fPIC because it was not compatible
>> > > > > with shared linker mpicc using flags ['-dynamic']
>> > > > >
>> > > > >
>> > > > > On Mon, Mar 8, 2021 at 7:28 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>> > > > >
>> > > > >>
>> > > > >>   Fande,
>> > > > >>
>> > > > >>      I see you are using CONDA, this can cause issues since it sticks
>> > > all
>> > > > >> kinds of things into the environment. PETSc tries to remove some of
>> > > them
>> > > > >> but perhaps not enough. If you run printenv you will see all the mess
>> > > it is
>> > > > >> dumping in.
>> > > > >>
>> > > > >>     Can you trying the same build without CONDA environment?
>> > > > >>
>> > > > >>   Barry
>> > > > >>
>> > > > >>
>> > > > >> On Mar 8, 2021, at 7:31 PM, Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>
>> > > wrote:
>> > > > >>
>> > > > >> On Mon, Mar 8, 2021 at 8:23 PM Fande Kong <fdkong.jd at gmail.com <mailto:fdkong.jd at gmail.com>>
>> > > wrote:
>> > > > >>
>> > > > >>> Thanks Matthew,
>> > > > >>>
>> > > > >>> Hmm, we still have the same issue after shutting off all unknown
>> > > flags.
>> > > > >>>
>> > > > >>
>> > > > >> Oh, I was misinterpreting the error message:
>> > > > >>
>> > > > >>   ld: can't link with a main executable file
>> > > > >>
>> > > '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib'
>> > > > >>
>> > > > >> So clang did not _actually_ make a shared library, it made an
>> > > executable.
>> > > > >> Did clang-11 change the options it uses to build a shared library?
>> > > > >>
>> > > > >> Satish, do we test with clang-11?
>> > > > >>
>> > > > >>   Thanks,
>> > > > >>
>> > > > >>       Matt
>> > > > >>
>> > > > >> Thanks,
>> > > > >>>
>> > > > >>> Fande
>> > > > >>>
>> > > > >>> On Mon, Mar 8, 2021 at 6:07 PM Matthew Knepley <knepley at gmail.com <mailto:knepley at gmail.com>>
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>>> On Mon, Mar 8, 2021 at 7:55 PM Fande Kong <fdkong.jd at gmail.com <mailto:fdkong.jd at gmail.com>>
>> > > wrote:
>> > > > >>>>
>> > > > >>>>> Hi All,
>> > > > >>>>>
>> > > > >>>>> mpicc rejected "-fPIC". Anyone has a clue how to work around this
>> > > > >>>>> issue?
>> > > > >>>>>
>> > > > >>>>
>> > > > >>>> The failure is at the last step
>> > > > >>>>
>> > > > >>>> Executing: mpicc  -o
>> > > > >>>>
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest
>> > > > >>>>   -fPIC
>> > > > >>>>
>> > > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest.o
>> > > > >>>>
>> > > -L/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers
>> > > > >>>> -lconftest
>> > > > >>>> Possible ERROR while running linker: exit code 1
>> > > > >>>> stderr:
>> > > > >>>> ld: can't link with a main executable file
>> > > > >>>>
>> > > '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib'
>> > > > >>>> for architecture x86_64
>> > > > >>>> clang-11: error: linker command failed with exit code 1 (use -v to
>> > > see
>> > > > >>>> invocation)
>> > > > >>>>
>> > > > >>>> but you have some flags stuck in which may or may not affect this. I
>> > > > >>>> would try shutting them off:
>> > > > >>>>
>> > > > >>>> LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs
>> > > -rpath
>> > > > >>>> /Users/kongf/miniconda3/envs/moose/lib
>> > > > >>>> -L/Users/kongf/miniconda3/envs/moose/lib
>> > > > >>>>
>> > > > >>>> I cannot tell exactly why clang is failing because it does not
>> > > report a
>> > > > >>>> specific error.
>> > > > >>>>
>> > > > >>>>   Thanks,
>> > > > >>>>
>> > > > >>>>      Matt
>> > > > >>>>
>> > > > >>>> The log was attached.
>> > > > >>>>>
>> > > > >>>>> Thanks so much,
>> > > > >>>>>
>> > > > >>>>> Fande
>> > > > >>>>>
>> > > > >>>>
>> > > > >>>>
>> > > > >>>> --
>> > > > >>>> What most experimenters take for granted before they begin their
>> > > > >>>> experiments is infinitely more interesting than any results to
>> > > which their
>> > > > >>>> experiments lead.
>> > > > >>>> -- Norbert Wiener
>> > > > >>>>
>> > > > >>>> https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
>> > > > >>>> <http://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>>
>> > > > >>>>
>> > > > >>>
>> > > > >>
>> > > > >> --
>> > > > >> What most experimenters take for granted before they begin their
>> > > > >> experiments is infinitely more interesting than any results to which
>> > > their
>> > > > >> experiments lead.
>> > > > >> -- Norbert Wiener
>> > > > >>
>> > > > >> https://www.cse.buffalo.edu/~knepley/ <https://www.cse.buffalo.edu/~knepley/>
>> > > > >> <http://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>>
>> > > > >>
>> > > > >>
>> > > > >> <configure.log>
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210310/3c8e4d93/attachment-0001.html>

More information about the petsc-users mailing list