[petsc-users] kokkos and include flags

[Steven Dargaville]

Thanks for the reply! I've tried that and again it doesn't seem to work for
the kokkos files. I went a bit overboard and set every variable I could
find but it doesn't seem to change the kokkos compilation, despite some of
those flags definitely being present in the kokkos compile targets.

CPPFLAGS = $(INCLUDE)
FPPFLAGS = $(INCLUDE)
CPPFLAGS = $(INCLUDE)
CXXPPFLAGS = $(INCLUDE)
CXXCPPFLAGS = $(INCLUDE)
CUDAC_FLAGS = $(INCLUDE)
HIPC_FLAGS = $(INCLUDE)
SYCLC_FLAGS = $(INCLUDE)
PETSC_CXXCPPFLAGS = $(INCLUDE)
PETSC_CCPPFLAGS = $(INCLUDE)
PETSC_FCPPFLAGS = $(INCLUDE)
PETSC_CUDACPPFLAGS = $(INCLUDE)
MPICXX_INCLUDES = $(INCLUDE)

# Read in the petsc compile/linking variables and makefile rules
include ${PETSC_DIR}/lib/petsc/conf/variables
include ${PETSC_DIR}/lib/petsc/conf/rules

The strangest thing is if I echo the value of PETSC_KOKKOSCOMPILE_SINGLE
before building, it seems to have the correct flags in it.

# Build the tests
build_tests: $(OUT)
echo $(PETSC_KOKKOSCOMPILE_SINGLE)
@for t in $(TEST_TARGETS); do \
$(MAKE) -C tests $$t; \
done

for example the echo gives (where I've bolded the flags I need added):

mpicxx -o .o -c -Wall -Wwrite-strings -Wno-strict-aliasing
-Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
-Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
-fPIC *-I/home/sdargavi/projects/PFLARE -Iinclude*

but the actual command that is called when the build is happening is (which
doesn't have the includes I need):

mpicxx -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-Wno-lto-type-mismatch -Wno-psabi -fstack-protector -Wall -Wwrite-strings
-Wno-strict-aliasing -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi
-fstack-protector -g -O0 -Wall -Wwrite-strings -Wno-strict-aliasing
-Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
-Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
-Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
-fPIC -I/home/sdargavi/projects/dependencies/petsc-3.22.0/include
-I/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/include
adv_1dk.kokkos.cxx  -L/home/sdargavi/projects/PFLARE/lib -lpflare
-Wl,-rpath,/home/sdargavi/projects/PFLARE/lib:-Wl,-rpath,/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
-L/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
-Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
-L/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
-Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/11
-L/usr/lib/gcc/x86_64-linux-gnu/11 -lpetsc -lkokkoskernels
-lkokkoscontainers -lkokkoscore -lkokkossimd -lflapack -lfblas -lparmetis
-lmetis -lm -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
-lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lgfortran -lm
-lz -lgfortran -lm -lgfortran -lgcc_s -lquadmath -lstdc++ -lquadmath -o
adv_1dk



On Thu, 20 Feb 2025 at 00:32, Satish Balay <balay.anl at fastmail.org> wrote:

> Try setting CPPFLAGS, FPPFLAGS, CXXPPFLAGS [and not via PETSC_FC_INCLUDES].
>
> I think kokkos compile targets [for *.kokkos.cxx sources]  should pick up
> one of them.
>
> for ex:
>
> >>>
> CPPFLAGS = -Wall
> FPPFLAGS = -Wall
> CXXPPFLAGS = -Wall
>
> include ${PETSC_DIR}/lib/petsc/conf/variables
> include ${PETSC_DIR}/lib/petsc/conf/rules
>
> ...
> <<<
>
> Satish
>
> On Wed, 19 Feb 2025, Steven Dargaville wrote:
>
> > Hi
> >
> > I'm trying to build my application code (which includes C and Fortran
> > files) with a Makefile based off
> $PETSC_DIR/share/petsc/Makefile.basic.user
> > by using the variables and rules defined in
> > ${PETSC_DIR}/lib/petsc/conf/variables. My application uses petsc as well
> as
> > another library, and hence I have to add some extra include statements
> > pointing at the other library during compilation. Currently I have been
> > doing:
> >
> > # Read in the petsc compile/linking variables and makefile rules
> > include ${PETSC_DIR}/lib/petsc/conf/variables
> > include ${PETSC_DIR}/lib/petsc/conf/rules
> >
> > # Add the extra include files
> > PETSC_FC_INCLUDES += $(INCLUDE_OTHER_LIB)
> > PETSC_CC_INCLUDES += $(INCLUDE_OTHER_LIB)
> >
> >
> > which works very well, with the correct include flags from
> > INCLUDE_OTHER_LIBS being added to the compilation of both fortran and C
> > files.
> >
> > If however I try and compile a kokkos file, named adv_1dk.kokkos.cxx (by
> > calling "make adv_1dk"), the extra flags are not included. If I instead
> > call "make adv_1dk.kokkos", the rule for cxx files is instead triggered
> and
> > correctly includes the include flags, but this just calls the c++
> wrapper,
> > rather than the nvcc_wrapper and therefore breaks when kokkos has been
> > built with cuda (or hip, etc).
> >
> > Just wondering if there is something I have missed, from what I can tell
> > the kokkos rules don't use the PETSC_CC_INCLUDES during compilation.
> >
> > Thanks for all your help
> > Steven
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250220/d82610bb/attachment-0001.html>