<div dir="ltr">
<div id="gmail-:up" class="gmail-Am gmail-aiL gmail-aO9 gmail-Al editable gmail-LW-avf gmail-tS-tW gmail-tS-tY" aria-label="Message Body" role="textbox" aria-multiline="true" tabindex="1" style="direction:ltr;min-height:85px" aria-controls=":x4" aria-expanded="false">Understood.<div><br></div><div>I believe I am running in circles.</div><div><br></div><div>I managed to create the container, but it ended up with multiple MPI implementations.</div><div>It seems that CUDA SDK has its own MPI, and polutes the PATH.</div><div></div><div>Is there any docker container PETSC+CUDA you would recommend?</div><div></div><div></div><div>This is a snippet of my Dockerfile, but I am not 100% happy with it.</div><div><br></div><div><<<<<<<br></div><div>FROM <a href="https://urldefense.us/v3/__http://docker.io/ubuntu:24.04__;!!G_uCfscf7eWS!ft2AAfoKdKf-j-6E6nTqtzXHqQEmnbsH7as3hylOIpz7OiONASpD-qNuRNcnDapAFok1YeTP2lLQfH7gQBL7Ow$">docker.io/ubuntu:24.04</a><br># Install necessary build tools and dependencies<br>RUN apt-get update && apt-get install -y libmpich-dev gfortran libomp-dev (...)<br></div><div>(...)</div><div><br></div><div>## NVIDIA HPC SDK<br>RUN wget <a href="https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!ft2AAfoKdKf-j-6E6nTqtzXHqQEmnbsH7as3hylOIpz7OiONASpD-qNuRNcnDapAFok1YeTP2lLQfH7ne1cFSw$">https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz</a> && <br> tar xpzfv nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz && <br> nvhpc_2025_255_Linux_x86_64_cuda_12.9/install --silent --accept-license --instal-dir /opt/nvidia/hpc_sdk<br></div><div><br></div><div>RUN cd petsc-3.23.5 &&\<br> ./configure --prefix=/usr/local/petsc --PETSC_ARCH=arch-linux-c-opt --with-debugging=0 \<br> --with-mpi --download-hypre --with-blaslapack --with-cuda <br> --with-cuda-dir=/opt/nvidia/hpc_sdk/Linux_x86_64/25.5/cuda && \<br> make all install && \<br> cd ..<br><br>(...)<br></div><div>ENV PATH=${NVHPC_ROOT}/compilers/bin:${NVHPC_ROOT}/comm_libs/mpi/bin:${PATH}<br>ENV MANPATH=${NVHPC_ROOT}/compilers/man:${MANPATH}<br>ENV LD_LIBRARY_PATH=${NVHPC_ROOT}/compilers/lib:${LD_LIBRARY_PATH}</div><div>>>>>>></div><div><br></div><div>Renato</div></div>
</div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sat, Aug 2, 2025 at 11:11 AM Satish Balay <<a href="mailto:balay.anl@fastmail.org">balay.anl@fastmail.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Please send configure.log<br>
<br>
presumably you are using a pre-installed mpi from hpc-sdk - that uses NV compilers. [and PETSc configure is defaulting to mpicc/mpif90 in PATH]<br>
<br>
If you do not have MPI with gcc/gfortran - you can do something like:<br>
<br>
./configure --with-cuda=1 --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich [other options]<br>
<br>
When no options are specified - configure guesses - and picks up what it can find - and sometimes that might not be what you intend to use..<br>
<br>
Satish<br>
<br>
On Sat, 2 Aug 2025, Renato E Poli wrote:<br>
<br>
> It seems that the installer is the one choosing to use nvfortran instead of<br>
> gfortran.<br>
> <br>
> I am building a docker container (clean environment).<br>
> I simply install CUDA SDK from<br>
> <a href="https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!cBTymZpXcpFZ7BHpCGxn0ooq_STD1f9CioiFgRypIYNWg9lpwnmLpddrWk0-J-wQoeTw9Ipu2FPm037mQg_C2w$" rel="noreferrer" target="_blank">https://urldefense.us/v3/__https://developer.download.nvidia.com/hpc-sdk/25.5/nvhpc_2025_255_Linux_x86_64_cuda_12.9.tar.gz__;!!G_uCfscf7eWS!cBTymZpXcpFZ7BHpCGxn0ooq_STD1f9CioiFgRypIYNWg9lpwnmLpddrWk0-J-wQoeTw9Ipu2FPm037mQg_C2w$</a> <br>
> , set paths and call petsc configure.<br>
> <br>
> RUN cd petsc-3.23.5 &&\<br>
> ./configure --prefix=/usr/local/petsc --with-debugging=0 --with-mpi<br>
> --download-hdf5 --download-hypre --with-blaslapack --with-cuda && \<br>
> make all && make install<br>
> <br>
> Do you see any reason for using the wrong compiler?<br>
> <br>
> Renato<br>
> <br>
> On Sat, Aug 2, 2025 at 12:28 AM Satish Balay <<a href="mailto:balay.anl@fastmail.org" target="_blank">balay.anl@fastmail.org</a>> wrote:<br>
> <br>
> > NVFORTRAN (based on flang-17) does not support some F2008 features that<br>
> > are required here (flang-19+ should work)<br>
> ><br>
> > Suggest using gfortran instead.<br>
> ><br>
> > Satish<br>
> ><br>
> > On Fri, 1 Aug 2025, Renato E Poli wrote:<br>
> ><br>
> > > Hi,<br>
> > ><br>
> > > I see the error below when compiling with --with-cuda.<br>
> > ><br>
> > > Any suggestion?<br>
> > ><br>
> > ><br>
> > > CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/inode2.o<br>
> > > CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/aijperm/aijperm.o<br>
> > > CC<br>
> > arch-linux-c-opt/obj/src/mat/impls/aij/seq/mattransposematmult.o<br>
> > > CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/matptap.o<br>
> > > CC arch-linux-c-opt/obj/src/mat/impls/aij/seq/matrart.o<br>
> > > FC arch-linux-c-opt/obj/src/sys/ftn-mod/petscsysmod.o<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h: 6)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 18)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 30)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 42)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 54)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 65)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 76)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 88)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 100)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 112)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 124)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 136)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 148)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscsys.h:<br>
> > 160)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > ><br>
> > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscviewer.h: 6)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscviewer.h:<br>
> > > 17)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > ><br>
> > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:<br>
> > > 6)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > ><br>
> > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:<br>
> > > 18)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > ><br>
> > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscoptions.h:<br>
> > > 30)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h: 5)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h:<br>
> > 17)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petsclog.h:<br>
> > 29)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > ><br>
> > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscdevice.h: 6)<br>
> > > NVFORTRAN-S-0034-Syntax error at or near end of line<br>
> > > (/opt/src/petsc-3.23.5/arch-linux-c-opt/include/../ftn/sys/petscdevice.h:<br>
> > > 17)<br>
> > > 0 inform, 0 warnings, 24 severes, 0 fatal for petscsysdef<br>
> > ><br>
> ><br>
> ><br>
> <br>
</blockquote></div>