[petsc-users] Preliminaries to use gpu capabilities

[Barry Smith]

> On Aug 25, 2024, at 7:49 AM, Edoardo alinovi <edoardo.alinovi at gmail.com> wrote:
> 
> Hello Petsc friends,
> 
> As many people is doing,  I would like to explore a bit gpu capabilities (cuda) in petsc. 
> 
> Before attemping any coding effort I would like to hear from you if all of this make sense:
> - compile mpi with cuda support

      This is commonly called CUDA aware MPI, but actually only means that the MPI can send and receive messages from memory addresses directly on the GPU, 
> - compile petsc with cuda support
> - build matrix and vectors as MATAIJCUSPARSE and VECMPICUDA to tell petsc using gpu.

   As presented, this will compute the vector and matrix entries on the CPU, and then PETSc will automatically move the values to the GPU for the linear solver. Which is a good start.  You can run with -log_view -log_view_gpu_time to see the timings, how much data is moved between the CPU and GPU, and where the computation happens.

 If all goes well, then you will find almost all the compute time is in building the vectors and matrices and copying the values to the GPU. At that point you will need to think about moving your computation to the GPU. This is problem-dependent, but you can look at VecSetPreallocationCOO() and MatSetPreallocationCOO() for how you can efficiently provide the values to PETSc on the GPU.

   As always, feel free to ask questions; the process is not trivial or as simple as we would like it to be,

  Barry



> 
> That's really it or do I need to take care of something else? 
> 
> I have seen that there is an amgXWrapper library aroud, but not sure if it is still relevant now or not.
> 
> Thank you for the suggestions!
> 
>