[petsc-users] Combining SUPERLU_DIST with CUDA fails
Barry Smith
bsmith at petsc.dev
Tue Dec 21 12:04:24 CST 2021
I think the problem comes from the following:
> -DCUDA_ARCH_FLAGS="-I/usr/local/cuda-11.5/include -arch=sm_61 -DDEBUGlevel=0 -DPRNTlevel=0"
>
which is ok, the location of the CUDA include files is passed in correctly but when the compile is done inside SuperLU_DIST's make
cd ~/Downloads/git-files/petsc/mpich-complex-linux-gcc-demo/externalpackages/git.superlu_dist/petsc-build/SRC && /usr/local/cuda-11.5/bin/nvcc -forward-unknown-to-host-compiler -DSUPERLU_DIST_EXPORTS -Dsuperlu_dist_EXPORTS -I~/Downloads/git-files/petsc/mpich-complex-linux-gcc-demo/externalpackages/git.superlu_dist/petsc-build/SRC -isystem=/usr/local/cuda-11.5/include -DUSE_VENDOR_BLAS -allow-unsupported-compiler -Xcompiler -fPIC -O3 -ccbin /opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx -std=c++17 -gencode arch=compute_61,code=sm_61 -Wno-deprecated-gpu-targets -I/opt/intel/oneapi/mpi/2021.5.0/include -O3 -DNDEBUG --generate-code=arch=compute_61,code=[compute_61,sm_61] -Xcompiler=-fPIC -std=c++11 -MD -MT SRC/CMakeFiles/superlu_dist.dir/superlu_gpu_utils.cu.o -MF CMakeFiles/superlu_dist.dir/superlu_gpu_utils.cu.o.d -x cu -dc ~/Downloads/git-files/petsc/mpich-complex-linux-gcc-demo/externalpackages/git.superlu_dist/SRC/superlu_gpu_utils.cu -o CMakeFiles/superlu_dist.dir/superlu_gpu_utils.cu.o
The
-I/usr/local/cuda-11.5/include
has been replaced with
-isystem=/usr/local/cuda-11.5/include
and I suspect the nvcc doesn't understand it so it ignores it. I assume somewhere inside the CMAKE processing it is replacing the -I with the -isystem=
I don't know how one could fix this. I suggest you try installing SuperLU_DIST yourself directly. But I suspect the same problem will appear.
It is possible that if you do not use the Intel compilers but use for example the GNU compilers the problem may not occur because cmake may not make the transformation.
Barry
Googling cmake isystem produces many messages about issues related to isystem and cmake but it would take a life-time to understand them.
> On Dec 21, 2021, at 2:24 AM, Roland Richter <roland.richter at ntnu.no> wrote:
>
> I added/replaced all six lines which are in that commit, getting
>
> Executing: /usr/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/opt/petsc -DCMAKE_INSTALL_NAME_DIR:STRING="/opt/petsc/lib" -DCMAKE_INSTALL_LIBDIR:STRING="lib" -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicc" -DMPI_C_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicc" -DCMAKE_AR=/usr/bin/ar -DCMAKE_RANLIB=/usr/bin/ranlib -DCMAKE_C_FLAGS:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_C_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_C_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_CXX_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx" -DMPI_CXX_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx" -DCMAKE_CXX_FLAGS:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_CXX_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_CXX_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_Fortran_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpifc" -DMPI_Fortran_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpifc" -DCMAKE_Fortran_FLAGS:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_Fortran_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_EXE_LINKER_FLAGS:STRING=" -fopenmp -fopenmp" -DBUILD_SHARED_LIBS:BOOL=ON -DTPL_ENABLE_CUDALIB=TRUE -DTPL_CUDA_LIBRARIES="-Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda" -DCUDA_ARCH_FLAGS="-I/usr/local/cuda-11.5/include -arch=sm_61 -DDEBUGlevel=0 -DPRNTlevel=0" -DCMAKE_CUDA_FLAGS="-allow-unsupported-compiler -Xcompiler -fPIC -O3 -ccbin /opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx -std=c++17 -gencode arch=compute_61,code=sm_61 -Wno-deprecated-gpu-targets -I"/opt/intel/oneapi/mpi/2021.5.0/include" " -DUSE_XSDK_DEFAULTS=YES -DTPL_BLAS_LIBRARIES="-lopenblas -Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda -lm -lstdc++ -ldl -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -L/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -L/opt/intel/oneapi/mpi/2021.5.0/lib -lmpifort -lmpi -lrt -lpthread -lgfortran -lm -Wl,-rpath,/usr/lib64/gcc/x86_64-suse-linux/11 -L/usr/lib64/gcc/x86_64-suse-linux/11 -Wl,-rpath,/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -Wl,-rpath,/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -L/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -Wl,-rpath,/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -Wl,-rpath,/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -Wl,-rpath,/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -Wl,-rpath,/usr/x86_64-suse-linux/lib -L/usr/x86_64-suse-linux/lib -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -lgfortran -lm -lgcc_s -lquadmath" -DTPL_LAPACK_LIBRARIES="-lopenblas -Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda -lm -lstdc++ -ldl -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -L/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -L/opt/intel/oneapi/mpi/2021.5.0/lib -lmpifort -lmpi -lrt -lpthread -lgfortran -lm -Wl,-rpath,/usr/lib64/gcc/x86_64-suse-linux/11 -L/usr/lib64/gcc/x86_64-suse-linux/11 -Wl,-rpath,/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -Wl,-rpath,/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -L/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -Wl,-rpath,/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -Wl,-rpath,/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -Wl,-rpath,/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -Wl,-rpath,/usr/x86_64-suse-linux/lib -L/usr/x86_64-suse-linux/lib -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -lgfortran -lm -lgcc_s -lquadmath" -Denable_parmetislib=FALSE -DTPL_ENABLE_PARMETISLIB=FALSE -DXSDK_ENABLE_Fortran=ON -Denable_tests=0 -Denable_examples=0 -DMPI_C_COMPILE_FLAGS:STRING="" -DMPI_C_INCLUDE_PATH:STRING="" -DMPI_C_HEADER_DIR:STRING="" -DMPI_C_LIBRARIES:STRING=""
>
> but the compilation still fails with the same error.
>
> Am 20.12.21 um 22:46 schrieb Roland Richter:
>> Yes, just checked, I only included the changes above the comment...
>>
>> Will test tomorrow, thanks for the help!
>>
>> Regards,
>>
>> Roland
>>
>> Am 20.12.2021 um 21:46 schrieb Barry Smith:
>>> Hmm, the fix should now be supplying a -DCMAKE_CUDA_FLAGS to this cmake command (line 48 at https://gitlab.com/petsc/petsc/-/merge_requests/4635/diffs <https://gitlab.com/petsc/petsc/-/merge_requests/4635/diffs>) I do not see that flag being set inside the configure.log so I am guessing you didn't get the complete fix.
>>>
>>>
>>> Executing: /usr/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/opt/petsc -DCMAKE_INSTALL_NAME_DIR:STRING="/opt/petsc/lib" -DCMAKE_INSTALL_LIBDIR:STRING="lib" -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicc" -DMPI_C_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicc" -DCMAKE_AR=/usr/bin/ar -DCMAKE_RANLIB=/usr/bin/ranlib -DCMAKE_C_FLAGS:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_C_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_C_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp" -DCMAKE_CXX_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx" -DMPI_CXX_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx" -DCMAKE_CXX_FLAGS:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_CXX_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_CXX_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fopenmp -fPIC -std=gnu++17 -fopenmp" -DCMAKE_Fortran_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpifc" -DMPI_Fortran_COMPILER="/opt/intel/oneapi/mpi/2021.5.0/bin/mpifc" -DCMAKE_Fortran_FLAGS:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_Fortran_FLAGS_DEBUG:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-mavx2 -march=native -O3 -fPIC -fopenmp -fallow-argument-mismatch" -DCMAKE_EXE_LINKER_FLAGS:STRING=" -fopenmp -fopenmp" -DBUILD_SHARED_LIBS:BOOL=ON -DTPL_ENABLE_CUDALIB=TRUE -DTPL_CUDA_LIBRARIES="-Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda" -DCUDA_ARCH_FLAGS="-I/usr/local/cuda-11.5/include -arch=sm_61 -DDEBUGlevel=0 -DPRNTlevel=0" -DUSE_XSDK_DEFAULTS=YES -DTPL_BLAS_LIBRARIES="-lopenblas -Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda -lm -lstdc++ -ldl -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -L/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -L/opt/intel/oneapi/mpi/2021.5.0/lib -lmpifort -lmpi -lrt -lpthread -lgfortran -lm -Wl,-rpath,/usr/lib64/gcc/x86_64-suse-linux/11 -L/usr/lib64/gcc/x86_64-suse-linux/11 -Wl,-rpath,/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -Wl,-rpath,/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -L/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -Wl,-rpath,/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -Wl,-rpath,/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -Wl,-rpath,/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -Wl,-rpath,/usr/x86_64-suse-linux/lib -L/usr/x86_64-suse-linux/lib -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -lgfortran -lm -lgcc_s -lquadmath" -DTPL_LAPACK_LIBRARIES="-lopenblas -Wl,-rpath,/usr/local/cuda-11.5/lib64 -L/usr/local/cuda-11.5/lib64 -lcudart -lcufft -lcublas -lcusparse -lcusolver -lcurand -L/usr/local/cuda-11.5/lib64/stubs -lcuda -lm -lstdc++ -ldl -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -L/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -L/opt/intel/oneapi/mpi/2021.5.0/lib -lmpifort -lmpi -lrt -lpthread -lgfortran -lm -Wl,-rpath,/usr/lib64/gcc/x86_64-suse-linux/11 -L/usr/lib64/gcc/x86_64-suse-linux/11 -Wl,-rpath,/opt/intel/oneapi/vpl/2022.0.0/lib -L/opt/intel/oneapi/vpl/2022.0.0/lib -Wl,-rpath,/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -L/opt/intel/oneapi/tbb/2021.5.0/lib/intel64/gcc4.8 -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -L/opt/intel/oneapi/mpi/2021.5.0/libfabric/lib -Wl,-rpath,/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -L/opt/intel/oneapi/mkl/2022.0.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -L/opt/intel/oneapi/ipp/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -L/opt/intel/oneapi/ippcp/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -L/opt/intel/oneapi/dnnl/2022.0.1/cpu_dpcpp_gpu_dpcpp/lib -Wl,-rpath,/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -L/opt/intel/oneapi/dal/2021.5.1/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -L/opt/intel/oneapi/compiler/2022.0.1/linux/compiler/lib/intel64_lin -Wl,-rpath,/opt/intel/oneapi/compiler/2022.0.1/linux/lib -L/opt/intel/oneapi/compiler/2022.0.1/linux/lib -Wl,-rpath,/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -L/opt/intel/oneapi/clck/2021.5.0/lib/intel64 -Wl,-rpath,/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -L/opt/intel/oneapi/ccl/2021.5.0/lib/cpu_gpu_dpcpp -Wl,-rpath,/usr/x86_64-suse-linux/lib -L/usr/x86_64-suse-linux/lib -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib/release -Wl,-rpath,/opt/intel/oneapi/mpi/2021.5.0/lib -lgfortran -lm -lgcc_s -lquadmath" -Denable_parmetislib=FALSE -DTPL_ENABLE_PARMETISLIB=FALSE -DXSDK_ENABLE_Fortran=ON -Denable_tests=0 -Denable_examples=0 -DMPI_C_COMPILE_FLAGS:STRING="" -DMPI_C_INCLUDE_PATH:STRING="" -DMPI_C_HEADER_DIR:STRING="" -DMPI_C_LIBRARIES:STRING=""
>>>
>>>
>>>
>>>
>>>> On Dec 20, 2021, at 2:59 PM, Roland Richter <roland.richter at ntnu.no <mailto:roland.richter at ntnu.no>> wrote:
>>>>
>>>> I introduced the changes from that patch directly, without checking out. Is that insufficient?
>>>>
>>>> Regards,
>>>>
>>>> Roland
>>>>
>>>> Am 20.12.2021 um 20:38 schrieb Barry Smith:
>>>>>
>>>>> Are you sure you have the correct PETSc branch? From configure.log it has
>>>>>
>>>>> Defined "VERSION_GIT" to ""v3.16.2-466-g959e1fce86""
>>>>> Defined "VERSION_DATE_GIT" to ""2021-12-18 11:17:24 -0600""
>>>>> Defined "VERSION_BRANCH_GIT" to ""master""
>>>>>
>>>>> It should have balay/slu-without-omp-3 for the branch.
>>>>>
>>>>>
>>>>>
>>>>>> On Dec 20, 2021, at 10:50 AM, Roland Richter <roland.richter at ntnu.no <mailto:roland.richter at ntnu.no>> wrote:
>>>>>>
>>>>>> In that case it fails with
>>>>>>
>>>>>> ~/Downloads/git-files/petsc/mpich-complex-linux-gcc-demo/externalpackages/git.superlu_dist/SRC/cublas_utils.h:22:10: fatal error: cublas_v2.h: No such file or directory
>>>>>>
>>>>>> even though this header is available. I assume some header paths are not set correctly?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> regards,
>>>>>>
>>>>>> Roland
>>>>>>
>>>>>> Am 20.12.21 um 16:29 schrieb Barry Smith:
>>>>>>>
>>>>>>> Please try the branch balay/slu-without-omp-3 It is in MR https://gitlab.com/petsc/petsc/-/merge_requests/4635 <https://gitlab.com/petsc/petsc/-/merge_requests/4635>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Dec 20, 2021, at 8:14 AM, Roland Richter <roland.richter at ntnu.no <mailto:roland.richter at ntnu.no>> wrote:
>>>>>>>>
>>>>>>>> Hei,
>>>>>>>>
>>>>>>>> I tried to combine CUDA with superlu_dist in petsc using the following configure-line:
>>>>>>>>
>>>>>>>> ./configure PETSC_ARCH=mpich-complex-linux-gcc-demo --CC=/opt/intel/oneapi/mpi/2021.5.0/bin/mpicc --CXX=/opt/intel/oneapi/mpi/2021.5.0/bin/mpicxx --FC=/opt/intel/oneapi/mpi/2021.5.0/bin/mpifc --CFLAGS="-mavx2 -march=native -O3" --CXXFLAGS="-mavx2 -march=native -O3" --FFLAGS="-mavx2 -march=native -O3" --CUDAFLAGS=-allow-unsupported-compiler --CUDA-CXX=g++ --prefix=/opt/petsc --with-blaslapack=1 --with-mpi=1 --with-scalar-type=complex --download-suitesparse=1 --with-cuda --with-debugging=0 --with-openmp --download-superlu_dist --force
>>>>>>>>
>>>>>>>> but the configure-step fails with several errors correlated with CUDA and superlu_dist, the first one being
>>>>>>>>
>>>>>>>> cublas_utils.c:21:37: error: ‘CUDART_VERSION’ undeclared (first use in this function); did you mean ‘CUDA_VERSION’?
>>>>>>>> 21 | printf("CUDA version: v %d\n",CUDART_VERSION);
>>>>>>>> | ^~~~~~~~~~~~~~
>>>>>>>> | CUDA_VERSION
>>>>>>>>
>>>>>>>> Compiling superlu_dist separately works, though (including CUDA).
>>>>>>>>
>>>>>>>> Is there a bug somewhere in the configure-routine? I attached the full configure-log.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Roland
>>>>>>>>
>>>>>>>> <configure.log>
>>>>>>>
>>>>>> <configure.log>
>>>>>
>>>
> <configure.log>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211221/32957809/attachment-0001.html>
More information about the petsc-users
mailing list