<div dir="ltr">Even with <div>CXXFLAGS=-Xcompiler -fopenmp -std=c++17<br>LDFLAGS= -Xcompiler -fopenmp<br>CXXPPFLAGS=-I/u/qiyuelu1/cuda/cuda-samples/Common<br>include ${PETSC_DIR}/lib/petsc/conf/variables<br>include ${PETSC_DIR}/lib/petsc/conf/rules<br></div><div><br></div><div>won't work. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 25, 2023 at 11:06 AM Qiyue Lu <<a href="mailto:qiyuelu1@gmail.com">qiyuelu1@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks for your reply, using this configurations: <div><b>--with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --download-f2cblaslapack=1 --with-cudac=nvcc --with-cuda=1 --with-openmp=1 --with-threadsafety=1<br></b></div><div>However, I got an error like:</div><div><b>nvcc fatal   : Unknown option '-fopenmp'</b><br></div><div>Previously, when I don't have --with-openmp for the configuration, the PETSc make system can build my *.cu code using nvcc and g++, of course, OpenMP doesn't work. Now with this --with-openmp option, it cannot even build. The interesting thing is, I got this error even after removing the <b>-fopenmp</b> from <b>CXXFLAGS</b> contents:</div><div>CXXFLAGS=-std=c++17<br>LDFLAGS=<br>CXXPPFLAGS=-I/u/qiyuelu1/cuda/cuda-samples/Common<br></div><div>include ${PETSC_DIR}/lib/petsc/conf/variables<br>include ${PETSC_DIR}/lib/petsc/conf/rules<br></div><div><br></div><div><br></div><div><br></div><div>Thanks,</div><div>Qiyue Lu</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 25, 2023 at 10:54 AM Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
On Wed, 25 Oct 2023, Qiyue Lu wrote:<br>
<br>
> Hello,<br>
> I have an in-house code enabled OpenMP and it works. Now I am trying to<br>
> incorporate PETSc as the linear solver and build together using the<br>
> building rules in $PETSC_HOME/lib/petsc/conf/rules. However, I found the<br>
> OpenMP part doesn't work anymore.<br>
<br>
If you are looking at building only your sources with openmp - using petsc formatted makefile [using petsc build rules],<br>
you can specify it via CFLAGS - either in makefile - or on command line.<br>
<br>
>>>>>>><br>
For ex: [this example is using src/ksp/ksp/tutorials/makefile - with the corresponding make fules]<br>
<br>
[balay@pj01 tutorials]$ make ex2<br>
mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-stringop-overflow -fstack-protector -fvisibility=hidden -g3 -O0  -I/home/balay/petsc/include -I/home/balay/petsc/arch-linux-c-debug/include     -Wl,-export-dynamic ex2.c  -Wl,-rpath,/home/balay/petsc/arch-linux-c-debug/lib -L/home/balay/petsc/arch-linux-c-debug/lib -Wl,-rpath,/software/mpich-4.1.1/lib -L/software/mpich-4.1.1/lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/13 -L/usr/lib/gcc/x86_64-redhat-linux/13 -lpetsc -llapack -lblas -lm -lX11 -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath -o ex2<br>
[balay@pj01 tutorials]$ make clean<br>
[balay@pj01 tutorials]$ make ex2 CFLAGS=-fopenmp<br>
mpicc -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -Wno-stringop-overflow -fstack-protector -fvisibility=hidden -g3 -O0 -fopenmp -I/home/balay/petsc/include -I/home/balay/petsc/arch-linux-c-debug/include     -Wl,-export-dynamic ex2.c  -Wl,-rpath,/home/balay/petsc/arch-linux-c-debug/lib -L/home/balay/petsc/arch-linux-c-debug/lib -Wl,-rpath,/software/mpich-4.1.1/lib -L/software/mpich-4.1.1/lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/13 -L/usr/lib/gcc/x86_64-redhat-linux/13 -lpetsc -llapack -lblas -lm -lX11 -lmpifort -lmpi -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath -o ex2<br>
[balay@pj01 tutorials]$ <br>
<<<<<<br>
<br>
Satish<br>
<br>
<br>
> Should I re-configure the petsc installation with --with-openmp=1 option? I<br>
> wonder are the building rules affected by this missing option?<br>
> <br>
> Thanks,<br>
> Qiyue Lu<br>
> <br>
<br>
</blockquote></div>
</blockquote></div>