[petsc-users] Nesting splits based on IS

[Barry Smith]

  Matt,

   I agree it can be done with DMs, but I am not convinced that is necessarily a superior way when one has to make DMSHELLS and mess around with their behavior.

   One way to look at PCFIELDSPLIT is that its basic splitting tool is IS and DM is a way of generating IS that can be used for PCFIELDSPLIT. In this approach PCFIELDSPLIT needs a bit more support for IS and nesting. To also help using multiple levels of DM.

   The other way is to view DM and subDM as the basic splitting tool for PCFIELDSPLIT but then one has to ask the question how does DM communicate its splits to PCFIELDSPLIT?   I am too lazy to look at the code but presumably with IS, hence the approach above.

   In the traditional Unix software stack approach to code, each layer uses the simpler layer below to communicate; so DM uses IS to communicate to the layer below (the linear algebra and preconditioners).  DM is a hugely complicated layer and making it communicate directly with the vector level is complex; I like having the IS level in between to simplify the software stack and programming model. 

   PetscSection makes live more complicated since it is a bit disjoint from IS. I've never resolved in my mind what role PetscSection plays, is it a level above IS in the software stack that generates IS, does it sometimes "skip over" IS to directly talk to linear algebra? 

   If you cannot make a cartoon picture of the software stack, with all the objects, then I think the software stack is not well designed or defined. I fear we cannot make such a cartoon currently.

 Barry




> On Apr 18, 2021, at 8:54 AM, Matthew Knepley <knepley at gmail.com> wrote:
> 
> On Sat, Apr 17, 2021 at 6:13 PM Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
> 
>   So you would like to be able to create three IS in your code and attach them with names to the PC.  Then have -pc_fieldsplit_XXX_fields be able to utilize the attached IS by name and use them to define the blocks.   
> 
>   This is all doable and could be added to PCFIELDSPLIT without too much code, new code. The code would be largely like PCFieldSplitSetRuntimeSplits_Private.
> 
>    The recursive part may also be doable but I think your syntax below is not exactly right. You would need something like 
> 
> -fieldsplit_0_pc_type fieldsplit   // split the first PC into a fieldsplit
>      -fieldsplit_0_pc_fieldsplit_0_fields xxx        -fieldsplit_0_fieldsplit_0_pc_type jacobi
>      -fieldsplit_0_pc_fieldsplit_1_fields yyy 
>      etc
> 
> this would split the first field into two fields and use jacobi on the first field.
> 
> The problem is what to use for labelling the xxx and the yyy? 
> 
> I think one could achieve this by having the PCFIELDPLIT attach to each of its split PCs the appropriate modified IS with names attached to them.  There are two cases, 
> 
>   when building the split uses first all the entries from fieldsplit_v_, then from fieldsplit_p_ then the new ISs it needs to attach to the first split PC are two sets of integers the first from 0 to the len(v)-1 and the second from len(v) to len(v)+len(p)-1. 
> 
>   when building the split it interlaces the indices from v and p (interlacing only make sense if the size of v and p is the same). Then the new v would be {0,2,4,...} and p would be {1,3,...}. 
> 
>   If you are ambitious and would like to add this to fieldsplit.c we'd be very happy to receive an MR. It might even lead to allowing us to simply how the PCFIELDPLIT interacts with DMs. If all the split type, stride, named, etc are handle in a single consistent manner.
> 
> Barry, this is already working with DMs, which I think is the right way to do this.
> 
> Here is the code:
> 
>    https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L420 <https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L420>
> 
> The DM must respond to DMCreateSubDM(). The interface is that the call provides a list of fields [f_0, f_1, ...]
> and the DM returns an IS for that combination and a subdm for it. The subdm part is what allows you to do this
> recursively, since you make the same call on the subdm.
> 
> If you use a PetscSection to keep track of the data layout, the code to do field selection is already done:
> 
>   https://gitlab.com/petsc/petsc/-/blob/main/src/dm/interface/dmi.c#L105 <https://gitlab.com/petsc/petsc/-/blob/main/src/dm/interface/dmi.c#L105>
> 
> which can just be called from the DMCreateSubDM() implementation.
> 
>   Thanks,
> 
>      Matt
>  
>   Barry
> 
> 
> 
> > On Apr 17, 2021, at 11:53 AM, Karin&NiKo <niko.karin at gmail.com <mailto:niko.karin at gmail.com>> wrote:
> > 
> > Dear PETSc users,
> > 
> > I use the fieldsplit PC in an application where the splits are programmatically defined by IS using PCFieldSplitSetIS. Then the user can specify its own PC at runtime using PETSc options.
> > My question : is it possible to define nested splits in this case as it can be done with strided splits (see snes/examples/tutorials/ex19.c with test suffix fieldsplit_4).
> > 
> > In order to be perfectly clear : let's say I have a 3 fields problem : velocity (v split), pressure (p split) and temperature (t split).
> > What I would like to do is something like the following but it fails : 
> > 
> > -ksp_type fgmres
> > -pc_fieldsplit_type multiplicative 
> > -pc_type fieldsplit    -pc_fieldsplit_0_fields fieldsplit_v_, fieldsplit_p_    -pc_fieldsplit_1_fields fieldsplit_t_ 
> > 
> > -prefix_push  fieldsplit_0_  
> > -ksp_type fgmres
> > -pc_fieldsplit_schur_factorization_type upper 
> > -pc_type fieldsplit 
> > -pc_fieldsplit_type schur 
> > -pc_fieldsplit_schur_precondition a11 
> > -prefix_pop
> > 
> > -prefix_push  fieldsplit_1_  
> > -ksp_type fgmres
> > -pc_type jacobi
> > -prefix_pop
> > 
> > -prefix_push  fieldsplit_v_  
> > -ksp_type fgmres
> > -pc_type gamg
> > -prefix_pop
> > 
> > -prefix_push  fieldsplit_p_  
> > -ksp_type fgmres
> > -pc_type jacobi
> > -prefix_pop
> > 
> > I thank you for your help,
> > Nicolas
> >  
> > 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
> 
> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210418/adca3e0d/attachment-0001.html>