[petsc-users] CUDA GPU supported KSPs and PCs

[Barry Smith]

   It is less a question of what KSP and PC support running with CUDA and more a question of what parts of each KSP and PC run with CUDA (and which parts don't causing memory traffic back and forth between the CPU and GPU).  

     Generally speaking, all the PETSc Vec operations run on CUDA. Thus "all" the KSP "support CUDA". For Mat operations, it is more complicated; triangular solves do not run well (or at all) on CUDA but much of the other operations do run on CUDA. Since setting and and solving with some PC involved rather complicated Mat operations (like PCGAMG and PCFIELDSPLIT) parts may work on CUDA and parts may not. 

     The best way to determine how the GPU is being utilized is to run with -log_view and look at the columns that present the amount of memory traffic between the CPU and GPU and the percentage of floating point that is done on the GPU. Feel free to ask specific questions about the output. In some cases, given the output, we may be able to add additional CUDA support that is missing to decrease the memory traffic between the CPU and GPU and increase the flops done on the GPU.

    We cannot produce a table of what is "supported" and what is not supported or even how much is supported since there are so many combinations of possible, hence it is best to run to determine the problematic places.

> On Apr 24, 2024, at 10:22 AM, Giyantha Binu Amaratunga Mukadange <ge75rud at mytum.de> wrote:
> 
> This Message Is From an External Sender
> This message came from outside your organization.
> Hi, 
> 
> Is it possible to know which KSPs and PCs currently support running on Nvidia GPUs with CUDA, or a source that has this information?
> The following page doesn't provide details about the supported KSPs and PCs. 
> https://urldefense.us/v3/__https://petsc.org/main/overview/gpu_roadmap/__;!!G_uCfscf7eWS!Y4JoiNMqvhSlcSuHnANYAK0LByq3ybAxBp_7_NTmzTcV2gBNyzA0D08G7lhPAZnQZsdtwo3zvr0Dlh3jIGpKCnM$  <https://urldefense.us/v3/__https://petsc.org/main/overview/gpu_roadmap/__;!!G_uCfscf7eWS!Yi-yp6M1qurQgY1iD0qI5XhdMGGfQqSpZpHdvfnh7DxMH4BL7V-0_6HaM47ifrfJqtWBWCBbGPCdqN29E8KNZTc$>
> 
> Thank you very much!
> 
> Best regards, 
> Binu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240424/6e73f190/attachment-0001.html>