[petsc-users] Error compiling with cuda

Stefano Zampini stefano.zampini at gmail.com
Sat Aug 2 12:26:34 CDT 2025 

NVIDIA ships OpenMPI. You can select the FORTRAN compiler by using the
OMPI_FC environment variable, see
https://urldefense.us/v3/__https://docs.open-mpi.org/en/v5.0.x/building-apps/customizing-wrappers.html__;!!G_uCfscf7eWS!dVGG_CfOVumpO5kAMWVbjzj4d6nRX1gwuaGnbv41cwClu5AMW_lX90wMUOZkeZwnbrCat53pFti0ckjLxs7nQJzi0CZkLW4$ 

Il giorno sab 2 ago 2025 alle ore 19:12 Renato E Poli <rpoli at utexas.edu> ha
scritto:

> Understood.
>
> I believe I am running in circles.
>
> I managed to create the container, but it ended up with multiple MPI
> implementations.
> It seems that CUDA SDK has its own MPI, and polutes the PATH.
> Is there any docker container PETSC+CUDA you would recommend?
> This is a snippet of my Dockerfile, but I am not 100% happy with it.
>
> <<<<<<
> FROM docker.io/ubuntu:24.04
> <https://urldefense.us/v3/__http://docker.io/ubuntu:24.04__;!!G_uCfscf7eWS!ft2AAfoKdKf-j-6E6nTqtzXHqQEmnbsH7as3hylOIpz7OiONASpD-qNuRNcnDapAFok1YeTP2lLQfH7gQBL7Ow$>
> # Install necessary build tools and dependencies
> RUN apt-get update && apt-get install -y  libmpich-dev   gfortran
> libomp-dev  (...)
> (...)
>
> ## NVIDIA HPC SDK
> RUN wget
> https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!dVGG_CfOVumpO5kAMWVbjzj4d6nRX1gwuaGnbv41cwClu5AMW_lX90wMUOZkeZwnbrCat53pFti0ckjLxs7nQJziY4A4vmo$ 
> <https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!ft2AAfoKdKf-j-6E6nTqtzXHqQEmnbsH7as3hylOIpz7OiONASpD-qNuRNcnDapAFok1YeTP2lLQfH7ne1cFSw$>
> &&
>     tar xpzfv nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz &&
>     nvhpc_2025_255_Linux_x86_64_cuda_12.9/install --silent
> --accept-license --instal-dir /opt/nvidia/hpc_sdk
>
> RUN cd petsc-3.23.5 &&\
>     ./configure --prefix=/usr/local/petsc  --PETSC_ARCH=arch-linux-c-opt
> --with-debugging=0 \
>         --with-mpi --download-hypre --with-blaslapack --with-cuda
>         --with-cuda-dir=/opt/nvidia/hpc_sdk/Linux_x86_64/25.5/cuda  && \
>     make all install && \
>     cd ..
>
> (...)
> ENV
> PATH=${NVHPC_ROOT}/compilers/bin:${NVHPC_ROOT}/comm_libs/mpi/bin:${PATH}
> ENV MANPATH=${NVHPC_ROOT}/compilers/man:${MANPATH}
> ENV LD_LIBRARY_PATH=${NVHPC_ROOT}/compilers/lib:${LD_LIBRARY_PATH}
> >>>>>>
>
> Renato
>
> On Sat, Aug 2, 2025 at 11:11 AM Satish Balay <balay.anl at fastmail.org>
> wrote:
>
>> Please send configure.log
>>
>> presumably you are using a pre-installed mpi from hpc-sdk - that uses NV
>> compilers. [and PETSc configure is defaulting to mpicc/mpif90 in PATH]
>>
>> If you do not have MPI with gcc/gfortran - you can do something like:
>>
>> ./configure --with-cuda=1 --with-cc=gcc --with-cxx=g++ --with-fc=gfortran
>> --download-mpich [other options]
>>
>> When no options are specified - configure guesses - and picks up what it
>> can find - and sometimes that might not be what you intend to use..
>>
>> Satish
>>
>> On Sat, 2 Aug 2025, Renato E Poli wrote:
>>
>> > It seems that the installer is the one choosing to use nvfortran
>> instead of
>> > gfortran.
>> >
>> > I am building a docker container (clean environment).
>> > I simply install CUDA SDK from
>> >
>> https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!cBTymZpXcpFZ7BHpCGxn0ooq_STD1f9CioiFgRypIYNWg9lpwnmLpddrWk0-J-wQoeTw9Ipu2FPm037mQg_C2w$
>> > , set paths and call petsc configure.
>> >
>> > RUN cd petsc-3.23.5 &&\
>> >     ./configure --prefix=/usr/local/petsc  --with-debugging=0
>> --with-mpi
>> > --download-hdf5   --download-hypre  --with-blaslapack   --with-cuda && \
>> >     make all && make install
>> >
>> > Do you see any reason for using the wrong compiler?
>> >
>> > Renato
>> >
>> > On Sat, Aug 2, 2025 at 12:28 AM Satish Balay <balay.anl at fastmail.org>
>> wrote:
>> >
>> > > NVFORTRAN (based on flang-17) does not support some F2008 features
>> that
>> > > are required here (flang-19+ should work)
>> > >
>> > > Suggest using gfortran instead.
>> > >
>> > > Satish
>> > >
>> > > On Fri, 1 Aug 2025, Renato E Poli wrote:
>> > >
>> > > >  Hi,
>> > > >
>> > > > I see the error below when compiling with --with-cuda.
>> > > >
>> > > > Any suggestion?
>> > > >
>> > > >
>> > > >          CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/inode2.o
>> > > >          CC
>> arch-linux-c-opt/obj/src/mat/impls/aij/seq/aijperm/aijperm.o
>> > > >          CC
>> > > arch-linux-c-opt/obj/src/mat/impls/aij/seq/mattransposematmult.o
>> > > >          CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/matptap.o
>> > > >          CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/matrart.o
>> > > >          FC arch-linux-c-opt/obj/src/sys/ftn-mod/petscsysmod.o
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h: 6)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 18)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 30)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 42)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 54)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 65)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 76)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 88)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 100)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 112)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 124)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 136)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 148)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:
>> > > 160)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscviewer.h: 6)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscviewer.h:
>> > > > 17)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:
>> > > > 6)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:
>> > > > 18)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:
>> > > > 30)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h: 5)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h:
>> > > 17)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h:
>> > > 29)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscdevice.h: 6)
>> > > > NVFORTRAN-S-0034-Syntax error at or near end of line
>> > > >
>> (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscdevice.h:
>> > > > 17)
>> > > >   0 inform,   0 warnings,  24 severes, 0 fatal for petscsysdef
>> > > >
>> > >
>> > >
>> >
>>
>

-- 
Stefano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20250802/6a030053/attachment-0001.html>

More information about the petsc-users mailing list