<div dir="ltr">Any idea what is going wrong with this?<div><br></div><div>Using PETSC_DIR=/global/homes/m/madams/petsc and PETSC_ARCH=arch-perlmutter-opt-gcc-kokkos-cuda<br>C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process<br>C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes<br>C/C++ example src/snes/tutorials/ex19 run successfully with cuda<br>gmake[3]: [/global/homes/m/madams/petsc/lib/petsc/conf/rules:301: ex3k.PETSc] Error 2 (ignored)<br>*******************Error detected during compile or link!*******************<br>See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a><br>/global/homes/m/madams/petsc/src/snes/tutorials ex3k<br>*********************************************************************************<br>PATH=/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib:/global/common/software/nersc/shasta2105/arm-forge/21.0.2-linux-x86_64/bin:/global/common/software/nersc/cos1.3/cuda/11.3.0/bin:/opt/cray/pe/perftools/21.10.0/bin:/opt/cray/pe/papi/<a href="http://6.0.0.10/bin:/opt/cray/pe/gcc/9.3.0/bin:/opt/cray/pe/craype/2.7.11/bin:/global/common/software/nersc/shasta2105/python/3.8-anaconda-2021.05/bin:/global/homes/m/madams/.local/perlmutter/3.8-anaconda-2021.05/bin:/global/common/software/nersc/shasta2105/cmake/git-master/bin:/global/common/software/nersc/bin:/opt/cray/libfabric/1.11.0.4.79/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/opt/cray/pe/bin:`dirname">6.0.0.10/bin:/opt/cray/pe/gcc/9.3.0/bin:/opt/cray/pe/craype/2.7.11/bin:/global/common/software/nersc/shasta2105/python/3.8-anaconda-2021.05/bin:/global/homes/m/madams/.local/perlmutter/3.8-anaconda-2021.05/bin:/global/common/software/nersc/shasta2105/cmake/git-master/bin:/global/common/software/nersc/bin:/opt/cray/libfabric/1.11.0.4.79/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/opt/cray/pe/bin:`dirname</a> /global/common/software/nersc/cos1.3/cuda/11.3.0/bin/nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=CC /global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/bin/nvcc_wrapper --expt-extended-lambda -g -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -ccbin CC -std=c++17 -gencode arch=compute_80,code=sm_80  -Wno-deprecated-gpu-targets -I/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/include -I/opt/cray/pe/libsci/<a href="http://21.08.1.2/GNU/9.1/x86_64/include">21.08.1.2/GNU/9.1/x86_64/include</a> -I/opt/cray/pe/pmi/6.0.14/include -I/opt/cray/pe/dsmml/0.2.2/dsmml//include -I/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/include  -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/include -I/global/common/software/nersc/cos1.3/cuda/11.3.0/include    -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -fPIC -O3 -g -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DLANDAU_MAX_Q=4 -O3   -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -L/opt/cray/pe/mpich/8.1.10/gtl/lib -L/opt/cray/pe/libsci/<a href="http://21.08.1.2/GNU/9.1/x86_64/lib">21.08.1.2/GNU/9.1/x86_64/lib</a> -L/opt/cray/pe/pmi/6.0.14/lib -L/opt/cray/pe/dsmml/0.2.2/dsmml//lib -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -Wl,--as-needed,-lmpi_gnu_91,--no-as-needed -Wl,--as-needed,-lsci_gnu_82_mpi,--no-as-needed -Wl,--as-needed,-lsci_gnu_82,--no-as-needed -Wl,--as-needed,-ldsmml,--no-as-needed  -lpmi -lpmi2 -lmpi_gtl_cuda -ldl -lxpmem   ex3k.kokkos.cxx  -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -Wl,-rpath,/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -L/global/homes/m/madams/petsc/arch-perlmutter-opt-gcc-kokkos-cuda/lib -Wl,-rpath,/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64 -L/global/common/software/nersc/cos1.3/cuda/11.3.0/lib64/stubs -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -L/opt/cray/pe/mpich/8.1.10/ofi/gnu/9.1/lib -Wl,-rpath,/opt/cray/pe/mpich/8.1.10/gtl/lib -L/opt/cray/pe/mpich/8.1.10/gtl/lib -Wl,-rpath,/opt/cray/pe/libsci/<a href="http://21.08.1.2/GNU/9.1/x86_64/lib">21.08.1.2/GNU/9.1/x86_64/lib</a> -L/opt/cray/pe/libsci/<a href="http://21.08.1.2/GNU/9.1/x86_64/lib">21.08.1.2/GNU/9.1/x86_64/lib</a> -Wl,-rpath,/opt/cray/pe/pmi/6.0.14/lib -L/opt/cray/pe/pmi/6.0.14/lib -Wl,-rpath,/opt/cray/pe/dsmml/0.2.2/dsmml/lib -L/opt/cray/pe/dsmml/0.2.2/dsmml/lib -Wl,-rpath,/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -L/opt/cray/xpmem/2.2.40-7.0.1.0_3.1__g1d7a24d.shasta/lib64 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 -L/opt/cray/pe/gcc/9.3.0/snos/lib/gcc/x86_64-suse-linux/9.3.0 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib64 -L/opt/cray/pe/gcc/9.3.0/snos/lib64 -Wl,-rpath,/opt/cray/pe/gcc/9.3.0/snos/lib -L/opt/cray/pe/gcc/9.3.0/snos/lib -lpetsc -lkokkoskernels -lkokkoscontainers -lkokkoscore -lp4est -lsc -lparmetis -lmetis -lz -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -lcuda -lX11 -lstdc++ -ldl -lpmi -lpmi2 -lmpi_gtl_cuda -lxpmem -lgfortran -lm -lmpifort_gnu_91 -lmpi_gnu_91 -lsci_gnu_82_mpi -lsci_gnu_82 -ldsmml -lgfortran -lquadmath -lpthread -lm -lgcc_s -lquadmath -lstdc++ -ldl -o ex3k<br>nvcc_wrapper - *warning* you have set multiple optimization flags (-O*), only the last is used because nvcc can only accept a single optimization setting.<br><b><font color="#ff0000">g++: error: unrecognized command line option '-rpath'<br></font></b>gmake[4]: *** [/global/homes/m/madams/petsc/lib/petsc/conf/test:32: ex3k] Error 1<br>gmake[3]: [makefile:115: runex3k_kokkos] Error 2 (ignored)<br>1,25c1,4<br>< atol=1e-50, rtol=1e-08, stol=1e-08, maxit=50, maxf=10000<br>< Vec Object: Exact Solution 2 MPI processes<br><   type: mpikokkos<br></div></div>