<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div><div class=""> This comes from the persistent problem with PETSc's make system using too many flags for compiling CUDA that have not been tested by configure. See below the -mp=gpu is provided probably from the CPPFLAGS or CXXPPFLAGS(sp) that is improperly used by the PETSc makefiles to compile CUDA code! </div><div class=""><br class=""></div><div class=""><br class=""></div>Using CUDA compile: /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc -o .o -I/opt/cray/pe/mpich/8.1.10/ofi/nvidia/20.7/include -I/opt/cray/pe/libsci/21.08.1.2/NVIDIA/20.7/x86_64/include -I/opt/cray/pe/pmi/6.0.14/include -I/opt/cray/pe/dsmml/0.2.2/dsmml//include -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include -g -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -std=c++17 -gencode arch=compute_80,code=sm_80 -Wno-deprecated-gpu-targets -c --compiler-options=-I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include -mp=gpu -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-nvidia-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include <div class=""><br class=""></div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Nov 4, 2021, at 7:31 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" class="">mfadams@lbl.gov</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">OK, configure is done.<div class="">Maybe I have too many -mp=gpu</div><div class=""><br class=""></div><div class=""> CUDAC arch-perlmutter-opt-nvidia-cuda/obj/sys/classes/random/impls/curand/curand2.o<br class="">gcc: error: unrecognized command line option ‘-mp=gpu’; did you mean ‘-mpku’?<br class=""></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 4, 2021 at 5:51 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;" class=""><div class=""><br class=""></div> Need the same thing for the C++ preprocessor flag<div class=""><br class=""></div><div class=""><br class=""><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Nov 4, 2021, at 5:44 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">It gets a lot further.</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 4, 2021 at 5:32 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">OK, sorry I missed the CPPFLAGS. It is running now.<div class="">Thanks,</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 4, 2021 at 4:43 PM Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank" class="">balay@mcs.anl.gov</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Multiple e-mail threads on the same issue (:<br class="">
<br class="">
As suggested in my earlier thread - add -mp=gpu to both CPPFLAGS and CFLAGS [or LDFLAGS]<br class="">
<br class="">
Satish<br class="">
<br class="">
-------<br class="">
Executing: cc -o /tmp/petsc-Vvs8_T/config.setCompilers/conftest -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 /tmp/petsc-Vvs8_T/config.setCompilers/conftest.o<br class="">
Possible ERROR while running linker:<br class="">
stderr:<br class="">
nvc-Warning-The -gpu option has no effect unless a language-specific option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar, -cuda)<br class="">
<br class="">
<br class="">
On Thu, 4 Nov 2021, Mark Adams wrote:<br class="">
<br class="">
> It is CPPFLAGS. I seem to get the same behavior.<br class="">
> <br class="">
> FWIW, I did get this response from NERSc but I don't know how to interpret<br class="">
> it.<br class="">
> <br class="">
> He seems to be saying that I don't need -mp=gpu for the device compiler<br class="">
> (nvcc). He seems to think that I am adding -gpu.<br class="">
> <br class="">
> nvcc -- The device compiler does not need any of those flags because it<br class="">
> already knows that it's being fed cuda code. The warning you're seeing is<br class="">
> coming from nvc (which is the host / CPU side compiler) if you're in the<br class="">
> PrgEnv-nvidia environment. You should not need to add -mp=gpu and -cuda,<br class="">
> please just add the -cuda flag (to your host code) not to the device code.<br class="">
> <br class="">
> I will try to talk with this guy again.<br class="">
> <br class="">
> Thanks,<br class="">
> <br class="">
> <br class="">
> On Thu, Nov 4, 2021 at 4:11 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank" class="">bsmith@petsc.dev</a>> wrote:<br class="">
> <br class="">
> ><br class="">
> > Yes, you need to use the CPPFLAGS which maybe called CPPCFLAGS I am not<br class="">
> > sure<br class="">
> ><br class="">
> ><br class="">
> > On Nov 4, 2021, at 3:23 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>> wrote:<br class="">
> ><br class="">
> > Ah, CCFLAGS does not seem to work.<br class="">
> ><br class="">
> > On Thu, Nov 4, 2021 at 3:07 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank" class="">bsmith@petsc.dev</a>> wrote:<br class="">
> ><br class="">
> >><br class="">
> >> You have to pass in the flag to turn off the bitching about -gpu to the<br class="">
> >> C preprocessor, not the C compiler.<br class="">
> >><br class="">
> >><br class="">
> >> stderr:<br class="">
> >> nvc-Warning-The -gpu option has no effect unless a language-specific<br class="">
> >> option to enable GPU code generation is used (e.g.: -acc, -mp=gpu, -stdpar,<br class="">
> >> -cuda)<br class="">
> >> Source:<br class="">
> >> #include "confdefs.h"<br class="">
> >> #include "conffix.h"<br class="">
> >> #include <stdlib.h><br class="">
> >><br class="">
> >><br class="">
> >><br class="">
> >> > On Nov 4, 2021, at 2:49 PM, Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank" class="">mfadams@lbl.gov</a>> wrote:<br class="">
> >> ><br class="">
> >> > on Perlmutter with nvhpc:<br class="">
> >> ><br class="">
> >> > Defined make macro "CPP" to "cc --use cpp32"<br class="">
> >> > Preprocessing source:<br class="">
> >> > #include "confdefs.h"<br class="">
> >> > #include "conffix.h"<br class="">
> >> > #include <stdlib.h><br class="">
> >> ><br class="">
> >> > Executing: cc --use cpp32 -I/tmp/petsc-jV9U1b/config.setCompilers<br class="">
> >> /tmp/petsc-jV9U1b/config.setCompilers/conftest.c<br class="">
> >> > Possible ERROR while running preprocessor: exit code 1<br class="">
> >> > stderr:<br class="">
> >> > nvc-Error-Unknown switch: --use<br class="">
> >> > Source:<br class="">
> >> > <configure.log><br class="">
> >><br class="">
> >> <configure.log><br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> <br class="">
<br class="">
</blockquote></div>
</blockquote></div>
<span id="gmail-m_-4702671338611196325cid:f_kvlh8gl80" class=""><configure.log></span></div></blockquote></div><br class=""></div></div></div></blockquote></div>
<span id="cid:f_kvll01990"><make.log></span><span id="cid:f_kvll019e1"><configure.log></span></div></blockquote></div><br class=""></div></body></html>