[petsc-users] kokkos and include flags

[Steven Dargaville]

My apologies, that does seem to have fixed the problem, the build rules in
my tests were overriding the new variables.

Thanks for your help in sorting that out!
Steven

On Thu, 20 Feb 2025 at 11:36, Steven Dargaville <dargaville.steven at gmail.com>
wrote:

> Thanks for the reply! I've tried that and again it doesn't seem to work
> for the kokkos files. I went a bit overboard and set every variable I could
> find but it doesn't seem to change the kokkos compilation, despite some of
> those flags definitely being present in the kokkos compile targets.
>
> CPPFLAGS = $(INCLUDE)
> FPPFLAGS = $(INCLUDE)
> CPPFLAGS = $(INCLUDE)
> CXXPPFLAGS = $(INCLUDE)
> CXXCPPFLAGS = $(INCLUDE)
> CUDAC_FLAGS = $(INCLUDE)
> HIPC_FLAGS = $(INCLUDE)
> SYCLC_FLAGS = $(INCLUDE)
> PETSC_CXXCPPFLAGS = $(INCLUDE)
> PETSC_CCPPFLAGS = $(INCLUDE)
> PETSC_FCPPFLAGS = $(INCLUDE)
> PETSC_CUDACPPFLAGS = $(INCLUDE)
> MPICXX_INCLUDES = $(INCLUDE)
>
> # Read in the petsc compile/linking variables and makefile rules
> include ${PETSC_DIR}/lib/petsc/conf/variables
> include ${PETSC_DIR}/lib/petsc/conf/rules
>
> The strangest thing is if I echo the value of PETSC_KOKKOSCOMPILE_SINGLE
> before building, it seems to have the correct flags in it.
>
> # Build the tests
> build_tests: $(OUT)
> echo $(PETSC_KOKKOSCOMPILE_SINGLE)
> @for t in $(TEST_TARGETS); do \
> $(MAKE) -C tests $$t; \
> done
>
> for example the echo gives (where I've bolded the flags I need added):
>
> mpicxx -o .o -c -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
> -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
> -fPIC *-I/home/sdargavi/projects/PFLARE -Iinclude*
>
> but the actual command that is called when the build is happening is
> (which doesn't have the includes I need):
>
> mpicxx -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -Wall -Wwrite-strings
> -Wno-strict-aliasing -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi
> -fstack-protector -g -O0 -Wall -Wwrite-strings -Wno-strict-aliasing
> -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-psabi -fstack-protector
> -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas
> -Wno-lto-type-mismatch -Wno-psabi -fstack-protector -g -O0 -std=gnu++17
> -fPIC -I/home/sdargavi/projects/dependencies/petsc-3.22.0/include
> -I/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/include
> adv_1dk.kokkos.cxx  -L/home/sdargavi/projects/PFLARE/lib -lpflare
> -Wl,-rpath,/home/sdargavi/projects/PFLARE/lib:-Wl,-rpath,/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
> -L/home/sdargavi/projects/dependencies/petsc-3.22.0/arch-linux-c-debug/lib
> -Wl,-rpath,/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
> -L/usr/lib/x86_64-linux-gnu/openmpi/lib/fortran/gfortran
> -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/11
> -L/usr/lib/gcc/x86_64-linux-gnu/11 -lpetsc -lkokkoskernels
> -lkokkoscontainers -lkokkoscore -lkokkossimd -lflapack -lfblas -lparmetis
> -lmetis -lm -lX11 -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
> -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lgfortran -lm
> -lz -lgfortran -lm -lgfortran -lgcc_s -lquadmath -lstdc++ -lquadmath -o
> adv_1dk
>
>
>
> On Thu, 20 Feb 2025 at 00:32, Satish Balay <balay.anl at fastmail.org> wrote:
>
>> Try setting CPPFLAGS, FPPFLAGS, CXXPPFLAGS [and not via
>> PETSC_FC_INCLUDES].
>>
>> I think kokkos compile targets [for *.kokkos.cxx sources]  should pick up
>> one of them.
>>
>> for ex:
>>
>> >>>
>> CPPFLAGS = -Wall
>> FPPFLAGS = -Wall
>> CXXPPFLAGS = -Wall
>>
>> include ${PETSC_DIR}/lib/petsc/conf/variables
>> include ${PETSC_DIR}/lib/petsc/conf/rules
>>
>> ...
>> <<<
>>
>> Satish
>>
>> On Wed, 19 Feb 2025, Steven Dargaville wrote:
>>
>> > Hi
>> >
>> > I'm trying to build my application code (which includes C and Fortran
>> > files) with a Makefile based off
>> $PETSC_DIR/share/petsc/Makefile.basic.user
>> > by using the variables and rules defined in
>> > ${PETSC_DIR}/lib/petsc/conf/variables. My application uses petsc as
>> well as
>> > another library, and hence I have to add some extra include statements
>> > pointing at the other library during compilation. Currently I have been
>> > doing:
>> >
>> > # Read in the petsc compile/linking variables and makefile rules
>> > include ${PETSC_DIR}/lib/petsc/conf/variables
>> > include ${PETSC_DIR}/lib/petsc/conf/rules
>> >
>> > # Add the extra include files
>> > PETSC_FC_INCLUDES += $(INCLUDE_OTHER_LIB)
>> > PETSC_CC_INCLUDES += $(INCLUDE_OTHER_LIB)
>> >
>> >
>> > which works very well, with the correct include flags from
>> > INCLUDE_OTHER_LIBS being added to the compilation of both fortran and C
>> > files.
>> >
>> > If however I try and compile a kokkos file, named adv_1dk.kokkos.cxx (by
>> > calling "make adv_1dk"), the extra flags are not included. If I instead
>> > call "make adv_1dk.kokkos", the rule for cxx files is instead triggered
>> and
>> > correctly includes the include flags, but this just calls the c++
>> wrapper,
>> > rather than the nvcc_wrapper and therefore breaks when kokkos has been
>> > built with cuda (or hip, etc).
>> >
>> > Just wondering if there is something I have missed, from what I can tell
>> > the kokkos rules don't use the PETSC_CC_INCLUDES during compilation.
>> >
>> > Thanks for all your help
>> > Steven
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250220/e4c2db64/attachment.html>