[petsc-dev] Question on PETSc + CUDA configuration with MPI on cluster

[岳新海]

Dear PETSc Team,
 
I am encountering an issue when running PETSc with CUDA support on a cluster. When I set the vector type to VECCUDA, PETSc reports that my MPI is not GPU-aware. However, the MPI library (OpenMPI 4.1.5) I used to configure PETSc was built with the --with-cuda option enabled.


Here are some details:
PETSc version: 3.20.6
MPI: OpenMPI 4.1.5, configured with --with-cuda
GPU: RTX3090
CUDA version: 12.1 
I have attached both my PETSc configure command and OpenMPI configure command for reference.

My questions are:

 
 
 
Even though I enabled --with-cuda in OpenMPI, why does PETSc still report that MPI is not GPU-aware?
 
 
 
Are there additional steps or specific configuration flags required (either in OpenMPI or PETSc) to ensure GPU-aware MPI is correctly detected?


Any guidance or suggestions would be greatly appreciated.

 

Best regards,

Xinhai Yue



 







岳新海



南方科技大学/学生/研究生/2023级研究生



广东省深圳市南山区学苑大道1088号




 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20250922/632fcd9b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: openmpi_conf.sh
Type: application/octet-stream
Size: 551 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20250922/632fcd9b/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: petsc_conf.sh
Type: application/octet-stream
Size: 957 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20250922/632fcd9b/attachment-0001.obj>