[petsc-users] Using CPR with PETSC and GPU

[Renato E Poli]

> First what GPU are using using NVIDIA?
Yes, NVIDIA RTX 5090.

> And MatType of MATAIJCUSPARSE?
Correct.

On Tue, Jun 16, 2026 at 9:36 AM Junchao Zhang <junchao.zhang at gmail.com>
wrote:

>
> Add petsc-users.
>
> On Tue, Jun 16, 2026 at 8:14 AM Barry Smith <bsmith at petsc.dev> wrote:
>
>>
>>     PCFIELDSPLIT should be utilizing the GPU though it may take some
>> tinkering to get everything on the GPU. First what GPU are using using
>> NVIDIA? And MatType of MATAIJCUSPARSE?
>>
>>     PCFIELDSPLIT uses MatCreateSubMatrix() to create the parts of the
>> matrix needed for the subsolve.
>>
>>     If you are comfortable with a debugger the easiest way to track what
>> is going on is run with one MPI process in the debugger
>> (-start_in_debugger) and watch what kind of vectors and matrices (CPU or
>> GPU) are created especially in PCSetUp_FieldSplit().
>>
>>    Barry
>>
>>   Side note to PETSc developers. Are there/could there be better ways to
>> make it easier to track the GPU usage of objects in a complicated run?
>>
>> On Jun 16, 2026, at 12:00 AM, Junchao Zhang <junchao.zhang at gmail.com>
>> wrote:
>>
>> Barry,  how does petsc handle a matrix in fieldsplit?  Will it extract
>> entries on the host?
>>
>> --Junchao Zhang
>>
>> On Fri, Jun 12, 2026 at 5:04 PM Renato E Poli <rpoli at utexas.edu> wrote:
>>
>>> Hi
>>>
>>> We are trying to use a CPR preconditioner for a multiphase flow solver
>>> using the fieldsplit option. It seems to work. We see a reduced number of
>>> iterations.
>>> However, it is not using the GPU effectively.
>>>
>>> Is there any documentation that could help?
>>> Is the fieldsplit+GPU supported? (v. 3.25.2)
>>>
>>> These are the options we are using (after a bit of trial-and-error):
>>>
>>> -ksp_type fgmres
>>> -ksp_gmres_restart 40
>>> -pc_fieldsplit_type multiplicative
>>> -fieldsplit_pressure_ksp_type preonly
>>> -fieldsplit_pressure_pc_type hypre
>>> -fieldsplit_pressure_pc_hypre_type boomeramg
>>> -fieldsplit_other_ksp_type gmres
>>> -fieldsplit_other_ksp_gmres_restart 5
>>> -fieldsplit_other_ksp_max_it 5
>>> -fieldsplit_other_ksp_rtol 1e-1
>>> -fieldsplit_other_pc_type bjacobi
>>> -fieldsplit_other_sub_pc_type ilu
>>> -fieldsplit_other_sub_pc_factor_reuse_ordering true
>>> -fieldsplit_other_sub_pc_factor_reuse_fill true
>>>
>>> Thank you,
>>> Renato
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20260616/1c91c318/attachment.html>