[petsc-users] Nesting splits based on IS

Sun Apr 25 11:20:34 CDT 2021

Dear Petsc Team,

I have begun to write a small test to illustrate the use of a DMShell in
order to nest splits from the command line.
I am writing it in Fortran because the targeted application is also in
Fortran.
=> I am afraid that DMShellSetCreateSubDM lacks a Fortran interface.
Am I wrong ? Can't the interface be generated by bfort ?

Thanks,
Nicolas

Le lun. 19 avr. 2021 à 12:28, Matthew Knepley <knepley at gmail.com> a écrit :

> On Sun, Apr 18, 2021 at 11:54 PM Barry Smith <bsmith at petsc.dev> wrote:
>
>>
>>   Matt,
>>
>>    I agree it can be done with DMs, but I am not convinced that is
>> necessarily a superior way when one has to make DMSHELLS and mess around
>> with their behavior.
>>
>>    One way to look at PCFIELDSPLIT is that its basic splitting tool is IS
>> and DM is a way of generating IS that can be used for PCFIELDSPLIT. In this
>> approach PCFIELDSPLIT needs a bit more support for IS and nesting. To also
>> help using multiple levels of DM.
>>
>>    The other way is to view DM and subDM as the basic splitting tool for
>> PCFIELDSPLIT but then one has to ask the question how does DM communicate
>> its splits to PCFIELDSPLIT?   I am too lazy to look at the code but
>> presumably with IS, hence the approach above.
>>
>>    In the traditional Unix software stack approach to code, each layer
>> uses the simpler layer below to communicate; so DM uses IS to communicate
>> to the layer below (the linear algebra and preconditioners).  DM is a
>> hugely complicated layer and making it communicate directly with the vector
>> level is complex; I like having the IS level in between to simplify the
>> software stack and programming model.
>>
>>    PetscSection makes live more complicated since it is a bit disjoint
>> from IS. I've never resolved in my mind what role PetscSection plays, is it
>> a level above IS in the software stack that generates IS, does it sometimes
>> "skip over" IS to directly talk to linear algebra?
>>
>>    If you cannot make a cartoon picture of the software stack, with all
>> the objects, then I think the software stack is not well designed or
>> defined. I fear we cannot make such a cartoon currently.
>>
>
> DM _does_ communicate with PCFIELDSPLIT using IS. I agree with you that IS
> is a good object for communication. In PETSc, IS is just a nice way to pass
> a list of integers.
>
> I don't think DM is hugely complicated. It does a few simple jobs. Here
> it's job is to remember which field each dof belongs to. That is all we
> have to worry about.
>
> PetscSection semantically is a linear space with some structure. We
> already know we want some structure like this, since we break all linear
> spaces in PETSc into processes. Section allows you to break it down a
> little finer into the pieces of the space for each "point", where you can
> use a point to mark anything you want, like a process, cell, edge, another
> dof, etc. Sections can make an IS when asked a question, such as "which
> dofs lie in the closure of this cell", or "which dofs are in this field",
> or "which dofs are owned by this process". I have written this in the
> manual years ago.
>
>     Matt
>
>
>>  Barry
>>
>>
>>
>>
>> On Apr 18, 2021, at 8:54 AM, Matthew Knepley <knepley at gmail.com> wrote:
>>
>> On Sat, Apr 17, 2021 at 6:13 PM Barry Smith <bsmith at petsc.dev> wrote:
>>
>>>
>>>   So you would like to be able to create three IS in your code and
>>> attach them with names to the PC.  Then have -pc_fieldsplit_XXX_fields be
>>> able to utilize the attached IS by name and use them to define the blocks.
>>>
>>>
>>>   This is all doable and could be added to PCFIELDSPLIT without too much
>>> code, new code. The code would be largely like
>>> PCFieldSplitSetRuntimeSplits_Private.
>>>
>>>    The recursive part may also be doable but I think your syntax below
>>> is not exactly right. You would need something like
>>>
>>> -fieldsplit_0_pc_type fieldsplit   // split the first PC into a
>>> fieldsplit
>>>      -fieldsplit_0_pc_fieldsplit_0_fields xxx
>>> -fieldsplit_0_fieldsplit_0_pc_type jacobi
>>>      -fieldsplit_0_pc_fieldsplit_1_fields yyy
>>>      etc
>>>
>>> this would split the first field into two fields and use jacobi on the
>>> first field.
>>>
>>> The problem is what to use for labelling the xxx and the yyy?
>>>
>>> I think one could achieve this by having the PCFIELDPLIT attach to each
>>> of its split PCs the appropriate modified IS with names attached to them.
>>> There are two cases,
>>>
>>>   when building the split uses first all the entries from fieldsplit_v_,
>>> then from fieldsplit_p_ then the new ISs it needs to attach to the first
>>> split PC are two sets of integers the first from 0 to the len(v)-1 and the
>>> second from len(v) to len(v)+len(p)-1.
>>>
>>>   when building the split it interlaces the indices from v and p
>>> (interlacing only make sense if the size of v and p is the same). Then the
>>> new v would be {0,2,4,...} and p would be {1,3,...}.
>>>
>>>   If you are ambitious and would like to add this to fieldsplit.c we'd
>>> be very happy to receive an MR. It might even lead to allowing us to simply
>>> how the PCFIELDPLIT interacts with DMs. If all the split type, stride,
>>> named, etc are handle in a single consistent manner.
>>>
>>
>> Barry, this is already working with DMs, which I think is the right way
>> to do this.
>>
>> Here is the code:
>>
>>
>> https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L420
>>
>> The DM must respond to DMCreateSubDM(). The interface is that the call
>> provides a list of fields [f_0, f_1, ...]
>> and the DM returns an IS for that combination and a subdm for it. The
>> subdm part is what allows you to do this
>> recursively, since you make the same call on the subdm.
>>
>> If you use a PetscSection to keep track of the data layout, the code to
>> do field selection is already done:
>>
>>   https://gitlab.com/petsc/petsc/-/blob/main/src/dm/interface/dmi.c#L105
>>
>> which can just be called from the DMCreateSubDM() implementation.
>>
>>   Thanks,
>>
>>      Matt
>>
>>
>>>   Barry
>>>
>>>
>>>
>>> > On Apr 17, 2021, at 11:53 AM, Karin&NiKo <niko.karin at gmail.com> wrote:
>>> >
>>> > Dear PETSc users,
>>> >
>>> > I use the fieldsplit PC in an application where the splits are
>>> programmatically defined by IS using PCFieldSplitSetIS. Then the user can
>>> specify its own PC at runtime using PETSc options.
>>> > My question : is it possible to define nested splits in this case as
>>> it can be done with strided splits (see snes/examples/tutorials/ex19.c with
>>> test suffix fieldsplit_4).
>>> >
>>> > In order to be perfectly clear : let's say I have a 3 fields problem :
>>> velocity (v split), pressure (p split) and temperature (t split).
>>> > What I would like to do is something like the following but it fails :
>>> >
>>> > -ksp_type fgmres
>>> > -pc_fieldsplit_type multiplicative
>>> > -pc_type fieldsplit    -pc_fieldsplit_0_fields fieldsplit_v_,
>>> fieldsplit_p_    -pc_fieldsplit_1_fields fieldsplit_t_
>>> >
>>> > -prefix_push  fieldsplit_0_
>>> > -ksp_type fgmres
>>> > -pc_fieldsplit_schur_factorization_type upper
>>> > -pc_type fieldsplit
>>> > -pc_fieldsplit_type schur
>>> > -pc_fieldsplit_schur_precondition a11
>>> > -prefix_pop
>>> >
>>> > -prefix_push  fieldsplit_1_
>>> > -ksp_type fgmres
>>> > -pc_type jacobi
>>> > -prefix_pop
>>> >
>>> > -prefix_push  fieldsplit_v_
>>> > -ksp_type fgmres
>>> > -pc_type gamg
>>> > -prefix_pop
>>> >
>>> > -prefix_push  fieldsplit_p_
>>> > -ksp_type fgmres
>>> > -pc_type jacobi
>>> > -prefix_pop
>>> >
>>> > I thank you for your help,
>>> > Nicolas
>>> >
>>> >
>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210425/214a6172/attachment-0001.html>