[petsc-dev] Additive Schwarz Method + ILU on GPU platforms

[Junchao Zhang]

Hi, Justin,
  I don't know ASM well enough. I just browsed its code. It seems it has a
lot of matrix partitioning and indexing operations, which definitely are
not done on GPUs currently.
  But you could still try that, as petsc will copy data from the device to
the host as needed to perform host-only operations.  You can profile with
-log_vew -log_view_gpu_time so that we can see how expensive these
operations are.

   Barry should know more about ASM.

--Junchao Zhang


On Tue, Apr 15, 2025 at 10:43 AM Angus, Justin Ray <angus1 at llnl.gov> wrote:

> Hi Junchao,
>
>
>
> Thanks for the reply.
>
>
>
> Does ASM work the same on GPU systems as it does on CPU systems?
>
>
>
> *From: *Junchao Zhang <junchao.zhang at gmail.com>
> *Date: *Monday, April 14, 2025 at 7:35 PM
> *To: *Angus, Justin Ray <angus1 at llnl.gov>
> *Cc: *petsc-dev at mcs.anl.gov <petsc-dev at mcs.anl.gov>, Ghosh, Debojyoti <
> ghosh5 at llnl.gov>
> *Subject: *Re: [petsc-dev] Additive Schwarz Method + ILU on GPU platforms
>
> Petsc supports ILU0/ICC0 numeric factorization (without reordering) and
> then triangular solve on GPUs. It is done by calling vendor libraries (ex.
> cusparse).
>
> We have options -pc_factor_mat_factor_on_host <bool>
> -pc_factor_mat_solve_on_host <bool> to force doing the factorization and
> MatSolve on the host for device matrix types.
>
>
>
> You can try to see if it works for your case.
>
>
>
> --Junchao Zhang
>
>
>
>
>
> On Mon, Apr 14, 2025 at 4:39 PM Angus, Justin Ray via petsc-dev <
> petsc-dev at mcs.anl.gov> wrote:
>
> Hello,
>
>
>
> A project I work on uses GMRES via PETSc. In particular, we have had good
> successes using the Additive Schwarz Method + ILU preconditioner setup
> using a CPU-based code. I found online where it is stated that “Parts of
> most preconditioners run directly on the GPU” (
> https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!Z8bOHrIJpUl9O8A9Z3SKLCa5vOhktluqMMJsYNsnLi92XNwl0gLCRaWy6TPoNDZns8N8wpd1UmnSEVkpvySgQkVMSsIT$ 
> <https://urldefense.us/v3/__https:/petsc.org/release/faq/__;!!G_uCfscf7eWS!bw6qeKcY7MKSvlEgcogdKR7fpjZSOFvka6zfDprUZ_sJHdE-YZmRD6UTqWQW3_uGVBII4P-AG0zaGTLbI67_fQ$>).
> Is ASM + ILU also available for GPU platforms?
>
>
>
> -Justin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20250415/21350152/attachment.html>