[petsc-dev] Question on PETSc + CUDA configuration with MPI on cluster

[Matthew Knepley]

On Mon, Sep 22, 2025 at 1:55 PM 岳新海 <12332508 at mail.sustech.edu.cn> wrote:

> Dear PETSc Team,
>
> I am encountering an issue when running PETSc with CUDA support on a
> cluster. When I set the vector type to VECCUDA, PETSc reports that my MPI
> is not GPU-aware. However, the MPI library (OpenMPI 4.1.5) I used to
> configure PETSc was built with the --with-cuda option enabled.
>
> Here are some details:
>
> PETSc version: 3.20.6
> MPI: OpenMPI 4.1.5, configured with --with-cuda
> GPU: RTX3090
> CUDA version: 12.1
>
> I have attached both my PETSc configure command and OpenMPI configure
> command for reference.
>
> My questions are:
>
>
>    1.
>
>    Even though I enabled --with-cuda in OpenMPI, why does PETSc still
>    report that MPI is not GPU-aware?
>
>
You need to send us the configure.log from this build.

  Thanks,

     Matt


>
>    1.
>    2.
>
>    Are there additional steps or specific configuration flags required
>    (either in OpenMPI or PETSc) to ensure GPU-aware MPI is correctly detected?
>
> Any guidance or suggestions would be greatly appreciated.
>
> Best regards,
>
> Xinhai Yue
>
>
>
> 岳新海
>
> 南方科技大学/学生/研究生/2023级研究生
>
> 广东省深圳市南山区学苑大道1088号
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!e4FQQzSZ5GghIY-rygAc20LL68HLqH4yeKRZvOHu4yZLV1R1lbmJ-6tUY3LUyZOH6FJWH3BM59unaJwiuHGv$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!e4FQQzSZ5GghIY-rygAc20LL68HLqH4yeKRZvOHu4yZLV1R1lbmJ-6tUY3LUyZOH6FJWH3BM59unaEscvOrg$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20250922/da8d864f/attachment.html>