[petsc-users] Running PETSc with a Kokkos backend on OLCF Frontier

[Sircar, Arpan]

Hi,

We have been working with OpenFOAM (an open-source CFD software) which can transfer its matrices to PETSc to use its linear solvers. This has been tested and is working well on OCLF's Frontier machine. Next we are trying to use the Kokkos backend to run it on Frontier GPUs. While the OpenFOAM+PETSc+Kokkos environment built correctly on Frontier using the modules sourced (attached file bash_petsc4foam_gpu) and configuring PETSc correctly (attached file config_gpu), the GPU solve seems to take more time than the CPU solve.

The PETSc run-time options we are using are attached to this email (file fvSolution_petsc_pKok_Uof). Could you please take a look  and let us know if this combination of options is fine? In this approach we are trying to solve the pressure equation only on the GPUs.

Thanks,
Arpan

Arpan Sircar
R&D Associate Staff
Thermal Hydraulics Group
Nuclear Energy and Fuel Cycle Division
Oak Ridge National Laboratory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240614/29fada4d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fvSolution_petsc_pKok_Uof
Type: application/octet-stream
Size: 2812 bytes
Desc: fvSolution_petsc_pKok_Uof
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240614/29fada4d/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bash_petsc4foam_gpu
Type: application/octet-stream
Size: 1725 bytes
Desc: bash_petsc4foam_gpu
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240614/29fada4d/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config_gpu
Type: application/octet-stream
Size: 895 bytes
Desc: config_gpu
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240614/29fada4d/attachment-0002.obj>