[petsc-users] kokkos and include flags

[Satish Balay]

I'm glad you have it working now! Thanks for the update!

Satish

On Thu, 20 Feb 2025, Steven Dargaville wrote:

> My apologies, that does seem to have fixed the problem, the build rules in
> my tests were overriding the new variables.
> 
> Thanks for your help in sorting that out!
> Steven
> 
> On Thu, 20 Feb 2025 at 11:36, Steven Dargaville <dargaville.steven at gmail.com>
> wrote:
> 
> > Thanks for the reply! I've tried that and again it doesn't seem to work
> > for the kokkos files. I went a bit overboard and set every variable I could
> > find but it doesn't seem to change the kokkos compilation, despite some of
> > those flags definitely being present in the kokkos compile targets.
> >
> > CPPFLAGS = $(INCLUDE)
> > FPPFLAGS = $(INCLUDE)
> > CPPFLAGS = $(INCLUDE)
> > CXXPPFLAGS = $(INCLUDE)
> > CXXCPPFLAGS = $(INCLUDE)
> > CUDAC_FLAGS = $(INCLUDE)
> > HIPC_FLAGS = $(INCLUDE)
> > SYCLC_FLAGS = $(INCLUDE)
> > PETSC_CXXCPPFLAGS = $(INCLUDE)
> > PETSC_CCPPFLAGS = $(INCLUDE)
> > PETSC_FCPPFLAGS = $(INCLUDE)
> > PETSC_CUDACPPFLAGS = $(INCLUDE)
> > MPICXX_INCLUDES = $(INCLUDE)
> >
> > # Read in the petsc compile/linking variables and makefile rules
> > include ${PETSC_DIR}/lib/petsc/conf/variables
> > include ${PETSC_DIR}/lib/petsc/conf/rules
> >
> > The strangest thing is if I echo the value of PETSC_KOKKOSCOMPILE_SINGLE
> > before building, it seems to have the correct flags in it.
> >
> > # Build the tests
> > build_tests: $(OUT)
> > echo $(PETSC_KOKKOSCOMPILE_SINGLE)
> > @for t in $(TEST_TARGETS); do \
> > $(MAKE) -C tests $$t; \
> > done
> >
> > for example the echo gives (where I've bolded the flags I need added):
> >
> > mpicxx -o .o -c -Wall -Wwrite-strings -Wno-strict-aliasing
> > -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
> > -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> > -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
> > -fPIC *-I/home/sdargavi/projects/PFLARE -Iinclude*
> >
> > but the actual command that is called when the build is happening is
> > (which doesn't have the includes I need):
> >
> > mpicxx -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> > -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -Wall -Wwrite-strings
> > -Wno-strict-aliasing -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi
> > -fstack-protector -g -O0 -Wall -Wwrite-strings -Wno-strict-aliasing
> > -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
> > -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> > -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
> > -fPIC -I/home/sdargavi/projects/dependencies/petsc-3.22.0/include
> > -I/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/include
> > adv_1dk.kokkos.cxx  -L/home/sdargavi/projects/PFLARE/lib -lpflare
> > -Wl,-rpath,/home/sdargavi/projects/PFLARE/lib:-Wl,-rpath,/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
> > -L/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
> > -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
> > -L/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
> > -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/11
> > -L/usr/lib/gcc/x86_64-linux-gnu/11 -lpetsc -lkokkoskernels
> > -lkokkoscontainers -lkokkoscore -lkokkossimd -lflapack -lfblas -lparmetis
> > -lmetis -lm -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
> > -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lgfortran -lm
> > -lz -lgfortran -lm -lgfortran -lgcc_s -lquadmath -lstdc++ -lquadmath -o
> > adv_1dk
> >
> >
> >
> > On Thu, 20 Feb 2025 at 00:32, Satish Balay <balay.anl at fastmail.org> wrote:
> >
> >> Try setting CPPFLAGS, FPPFLAGS, CXXPPFLAGS [and not via
> >> PETSC_FC_INCLUDES].
> >>
> >> I think kokkos compile targets [for *.kokkos.cxx sources]  should pick up
> >> one of them.
> >>
> >> for ex:
> >>
> >> >>>
> >> CPPFLAGS = -Wall
> >> FPPFLAGS = -Wall
> >> CXXPPFLAGS = -Wall
> >>
> >> include ${PETSC_DIR}/lib/petsc/conf/variables
> >> include ${PETSC_DIR}/lib/petsc/conf/rules
> >>
> >> ...
> >> <<<
> >>
> >> Satish
> >>
> >> On Wed, 19 Feb 2025, Steven Dargaville wrote:
> >>
> >> > Hi
> >> >
> >> > I'm trying to build my application code (which includes C and Fortran
> >> > files) with a Makefile based off
> >> $PETSC_DIR/share/petsc/Makefile.basic.user
> >> > by using the variables and rules defined in
> >> > ${PETSC_DIR}/lib/petsc/conf/variables. My application uses petsc as
> >> well as
> >> > another library, and hence I have to add some extra include statements
> >> > pointing at the other library during compilation. Currently I have been
> >> > doing:
> >> >
> >> > # Read in the petsc compile/linking variables and makefile rules
> >> > include ${PETSC_DIR}/lib/petsc/conf/variables
> >> > include ${PETSC_DIR}/lib/petsc/conf/rules
> >> >
> >> > # Add the extra include files
> >> > PETSC_FC_INCLUDES += $(INCLUDE_OTHER_LIB)
> >> > PETSC_CC_INCLUDES += $(INCLUDE_OTHER_LIB)
> >> >
> >> >
> >> > which works very well, with the correct include flags from
> >> > INCLUDE_OTHER_LIBS being added to the compilation of both fortran and C
> >> > files.
> >> >
> >> > If however I try and compile a kokkos file, named adv_1dk.kokkos.cxx (by
> >> > calling "make adv_1dk"), the extra flags are not included. If I instead
> >> > call "make adv_1dk.kokkos", the rule for cxx files is instead triggered
> >> and
> >> > correctly includes the include flags, but this just calls the c++
> >> wrapper,
> >> > rather than the nvcc_wrapper and therefore breaks when kokkos has been
> >> > built with cuda (or hip, etc).
> >> >
> >> > Just wondering if there is something I have missed, from what I can tell
> >> > the kokkos rules don't use the PETSC_CC_INCLUDES during compilation.
> >> >
> >> > Thanks for all your help
> >> > Steven
> >> >
> >>
> >>
>