<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 10, 2021 at 1:36 PM Satish Balay <<a href="mailto:balay@mcs.anl.gov">balay@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Can you use a different MPI for this conda install?<br></blockquote><div><br></div><div>We control how to build MPI. If I take "-pie" options out of LDFLAGS, conda can not compile mpich.<br><br><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Alternative:<br>
<br>
./configure CC=x86_64-apple-darwin13.4.0-clang COPTFLAGS="-march=core2 -mtune=haswell" CPPFLAGS=-I/Users/kongf/miniconda3/envs/testpetsc/include<br>
LDFLAGS="-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-commons,use_dylibs" LIBS="-Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi"<br></blockquote><div><br></div><div>MPI can not generate an executable because we took out "-pie". <br><br></div><div>Thanks,<br><br></div><div>Fande<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
etc.. [don't know if you really need LDFLAGS options]<br>
<br>
Satish<br>
<br>
On Wed, 10 Mar 2021, Fande Kong wrote:<br>
<br>
> I guess it was encoded in mpicc<br>
> <br>
> petsc % mpicc -show<br>
> x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -Wl,-pie<br>
> -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs<br>
> -Wl,-rpath,/Users/kongf/miniconda3/envs/testpetsc/lib<br>
> -L/Users/kongf/miniconda3/envs/testpetsc/lib -Wl,-commons,use_dylibs<br>
> -I/Users/kongf/miniconda3/envs/testpetsc/include<br>
> -L/Users/kongf/miniconda3/envs/testpetsc/lib -lmpi -lpmpi<br>
> <br>
> <br>
> Thanks,<br>
> <br>
> Fande<br>
> <br>
> On Wed, Mar 10, 2021 at 12:51 PM Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
> <br>
> > > LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs -rpath<br>
> > /Users/kongf/miniconda3/envs/testpetsc/lib<br>
> > -L/Users/kongf/miniconda3/envs/testpetsc/lib<br>
> ><br>
> > Does conda compiler pick up '-pie' from this env variable? If so - perhaps<br>
> > its easier to just modify it?<br>
> ><br>
> > Or is it encoded in mpicc wrapper? [mpicc -show]<br>
> ><br>
> > Satish<br>
> ><br>
> > On Wed, 10 Mar 2021, Fande Kong wrote:<br>
> ><br>
> > > Thanks Barry,<br>
> > ><br>
> > > Got the same result, but "-pie" was not filtered out somehow.<br>
> > ><br>
> > > I did changes like this:<br>
> > ><br>
> > > kongf@x86_64-apple-darwin13 petsc % git diff<br>
> > > diff --git a/config/BuildSystem/config/framework.py<br>
> > > b/config/BuildSystem/config/framework.py<br>
> > > index beefe82956..c31fbeb95e 100644<br>
> > > --- a/config/BuildSystem/config/framework.py<br>
> > > +++ b/config/BuildSystem/config/framework.py<br>
> > > @@ -504,6 +504,8 @@ class Framework(config.base.Configure,<br>
> > > script.LanguageProcessor):<br>
> > > lines = [s for s in lines if s.find('Load a valid targeting module or<br>
> > > set CRAY_CPU_TARGET') < 0]<br>
> > > # pgi dumps filename on stderr - but returns 0 errorcode'<br>
> > > lines = [s for s in lines if lines != 'conftest.c:']<br>
> > > + # in case -pie is always being passed to linker<br>
> > > + lines = [s for s in lines if s.find('-pie being ignored. It is only<br>
> > > used when linking a main executable') < 0]<br>
> > > if lines: output = reduce(lambda s, t: s+t, lines, '\n')<br>
> > > else: output = ''<br>
> > > log.write("Linker stderr after filtering:\n"+output+":\n")<br>
> > ><br>
> > > The log was attached again.<br>
> > ><br>
> > > Thanks,<br>
> > ><br>
> > > Fande<br>
> > ><br>
> > ><br>
> > > On Wed, Mar 10, 2021 at 12:05 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br>
> > ><br>
> > > > Fande,<br>
> > > ><br>
> > > > Please add in config/BuildSystem/config/framework.py line 528 two<br>
> > new<br>
> > > > lines<br>
> > > ><br>
> > > > # pgi dumps filename on stderr - but returns 0 errorcode'<br>
> > > > lines = [s for s in lines if lines != 'conftest.c:']<br>
> > > > # in case -pie is always being passed to linker<br>
> > > > lines = [s for s in lines if s.find('-pie being ignored. It is<br>
> > only<br>
> > > > used when linking a main executable') < 0]<br>
> > > ><br>
> > > > Barry<br>
> > > ><br>
> > > > You have (another of Conda's "take over the world my way" approach)<br>
> > > ><br>
> > > > LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs<br>
> > -rpath<br>
> > > > /Users/kongf/miniconda3/envs/testpetsc/lib<br>
> > > > -L/Users/kongf/miniconda3/envs/testpetsc/lib<br>
> > > ><br>
> > > > Executing: mpicc -o<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest<br>
> > > > -dynamiclib -single_module<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o<br>
> > > > Possible ERROR while running linker:<br>
> > > > stderr:<br>
> > > > ld: warning: -pie being ignored. It is only used when linking a main<br>
> > > > executable<br>
> > > > Rejecting C linker flag -dynamiclib -single_module due to<br>
> > > ><br>
> > > > ld: warning: -pie being ignored. It is only used when linking a main<br>
> > > > executable<br>
> > > ><br>
> > > > This is the correct link command for the Mac but it is being rejected<br>
> > due<br>
> > > > to the warning message.<br>
> > > ><br>
> > > ><br>
> > > > On Mar 10, 2021, at 10:11 AM, Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>> wrote:<br>
> > > ><br>
> > > > Thanks, Barry,<br>
> > > ><br>
> > > > It seems PETSc works fine with manually built compilers. We are pretty<br>
> > > > much sure that the issue is related to conda. Conda might introduce<br>
> > extra<br>
> > > > flags.<br>
> > > ><br>
> > > > We still need to make it work with conda because we deliver our package<br>
> > > > via conda for users.<br>
> > > ><br>
> > > ><br>
> > > > I unset all flags from conda, and got slightly different results this<br>
> > > > time. The log was attached. Anyone could explain the motivation that<br>
> > we<br>
> > > > try to build executable without a main function?<br>
> > > ><br>
> > > > Thanks,<br>
> > > ><br>
> > > > Fande<br>
> > > ><br>
> > > > Executing: mpicc -c -o<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o<br>
> > > ><br>
> > -I/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers<br>
> > > > -fPIC<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.c<br>
> > > ><br>
> > > > Successful compile:<br>
> > > > Source:<br>
> > > > #include "confdefs.h"<br>
> > > > #include "conffix.h"<br>
> > > > #include <stdio.h><br>
> > > > int (*fprintf_ptr)(FILE*,const char*,...) = fprintf;<br>
> > > > void foo(void){<br>
> > > > fprintf_ptr(stdout,"hello");<br>
> > > > return;<br>
> > > > }<br>
> > > > void bar(void){foo();}<br>
> > > > Running Executable WITHOUT threads to time it out<br>
> > > > Executing: mpicc -o<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/libconftest.so<br>
> > > > -dynamic -fPIC<br>
> > > ><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-pkset22y/config.setCompilers/conftest.o<br>
> > > ><br>
> > > > Possible ERROR while running linker: exit code 1<br>
> > > > stderr:<br>
> > > > Undefined symbols for architecture x86_64:<br>
> > > > "_main", referenced from:<br>
> > > > implicit entry/start for main executable<br>
> > > > ld: symbol(s) not found for architecture x86_64<br>
> > > > clang-11: error: linker command failed with exit code 1 (use -v to see<br>
> > > > invocation)<br>
> > > > Rejected C compiler flag -fPIC because it was not compatible<br>
> > > > with shared linker mpicc using flags ['-dynamic']<br>
> > > ><br>
> > > ><br>
> > > > On Mon, Mar 8, 2021 at 7:28 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br>
> > > ><br>
> > > >><br>
> > > >> Fande,<br>
> > > >><br>
> > > >> I see you are using CONDA, this can cause issues since it sticks<br>
> > all<br>
> > > >> kinds of things into the environment. PETSc tries to remove some of<br>
> > them<br>
> > > >> but perhaps not enough. If you run printenv you will see all the mess<br>
> > it is<br>
> > > >> dumping in.<br>
> > > >><br>
> > > >> Can you trying the same build without CONDA environment?<br>
> > > >><br>
> > > >> Barry<br>
> > > >><br>
> > > >><br>
> > > >> On Mar 8, 2021, at 7:31 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
> > wrote:<br>
> > > >><br>
> > > >> On Mon, Mar 8, 2021 at 8:23 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>><br>
> > wrote:<br>
> > > >><br>
> > > >>> Thanks Matthew,<br>
> > > >>><br>
> > > >>> Hmm, we still have the same issue after shutting off all unknown<br>
> > flags.<br>
> > > >>><br>
> > > >><br>
> > > >> Oh, I was misinterpreting the error message:<br>
> > > >><br>
> > > >> ld: can't link with a main executable file<br>
> > > >><br>
> > '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib'<br>
> > > >><br>
> > > >> So clang did not _actually_ make a shared library, it made an<br>
> > executable.<br>
> > > >> Did clang-11 change the options it uses to build a shared library?<br>
> > > >><br>
> > > >> Satish, do we test with clang-11?<br>
> > > >><br>
> > > >> Thanks,<br>
> > > >><br>
> > > >> Matt<br>
> > > >><br>
> > > >> Thanks,<br>
> > > >>><br>
> > > >>> Fande<br>
> > > >>><br>
> > > >>> On Mon, Mar 8, 2021 at 6:07 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
> > > >>> wrote:<br>
> > > >>><br>
> > > >>>> On Mon, Mar 8, 2021 at 7:55 PM Fande Kong <<a href="mailto:fdkong.jd@gmail.com" target="_blank">fdkong.jd@gmail.com</a>><br>
> > wrote:<br>
> > > >>>><br>
> > > >>>>> Hi All,<br>
> > > >>>>><br>
> > > >>>>> mpicc rejected "-fPIC". Anyone has a clue how to work around this<br>
> > > >>>>> issue?<br>
> > > >>>>><br>
> > > >>>><br>
> > > >>>> The failure is at the last step<br>
> > > >>>><br>
> > > >>>> Executing: mpicc -o<br>
> > > >>>><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest<br>
> > > >>>> -fPIC<br>
> > > >>>><br>
> > /var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/conftest.o<br>
> > > >>>><br>
> > -L/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers<br>
> > > >>>> -lconftest<br>
> > > >>>> Possible ERROR while running linker: exit code 1<br>
> > > >>>> stderr:<br>
> > > >>>> ld: can't link with a main executable file<br>
> > > >>>><br>
> > '/var/folders/tv/ljnkj46x3nq45cp9tbkc000c0000gn/T/petsc-6v1w4q4u/config.setCompilers/libconftest.dylib'<br>
> > > >>>> for architecture x86_64<br>
> > > >>>> clang-11: error: linker command failed with exit code 1 (use -v to<br>
> > see<br>
> > > >>>> invocation)<br>
> > > >>>><br>
> > > >>>> but you have some flags stuck in which may or may not affect this. I<br>
> > > >>>> would try shutting them off:<br>
> > > >>>><br>
> > > >>>> LDFLAGS_LD=-pie -headerpad_max_install_names -dead_strip_dylibs<br>
> > -rpath<br>
> > > >>>> /Users/kongf/miniconda3/envs/moose/lib<br>
> > > >>>> -L/Users/kongf/miniconda3/envs/moose/lib<br>
> > > >>>><br>
> > > >>>> I cannot tell exactly why clang is failing because it does not<br>
> > report a<br>
> > > >>>> specific error.<br>
> > > >>>><br>
> > > >>>> Thanks,<br>
> > > >>>><br>
> > > >>>> Matt<br>
> > > >>>><br>
> > > >>>> The log was attached.<br>
> > > >>>>><br>
> > > >>>>> Thanks so much,<br>
> > > >>>>><br>
> > > >>>>> Fande<br>
> > > >>>>><br>
> > > >>>><br>
> > > >>>><br>
> > > >>>> --<br>
> > > >>>> What most experimenters take for granted before they begin their<br>
> > > >>>> experiments is infinitely more interesting than any results to<br>
> > which their<br>
> > > >>>> experiments lead.<br>
> > > >>>> -- Norbert Wiener<br>
> > > >>>><br>
> > > >>>> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
> > > >>>> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
> > > >>>><br>
> > > >>><br>
> > > >><br>
> > > >> --<br>
> > > >> What most experimenters take for granted before they begin their<br>
> > > >> experiments is infinitely more interesting than any results to which<br>
> > their<br>
> > > >> experiments lead.<br>
> > > >> -- Norbert Wiener<br>
> > > >><br>
> > > >> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
> > > >> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
> > > >><br>
> > > >><br>
> > > >> <configure.log><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > ><br>
> ><br>
> ><br>
> <br>
<br>
</blockquote></div></div>