<div dir="ltr"><div dir="ltr">On Tue, Apr 27, 2021 at 9:58 AM Karin&NiKo <<a href="mailto:niko.karin@gmail.com">niko.karin@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I will have a look at PetscSection (that I have never used before). <div>The FE I use are P2-P1-P1-P1 Lagrange elements. The point is that the numbering of the unknowns is done by the application (I do not have the hand on it). At the moment, I build IS in order to define the different fields and I nest them programmatically in order to set some recursive preconditioner.</div><div>To enable the use of the command line options in order to nest the splits is a matter of elegance and smoothness since the nesting of splits I have programmed are fully fixed.</div></div></blockquote><div><br></div><div>Okay, PetscSection should be completely adequate for that. You just</div><div><br></div><div>1) Create a Section with 4 fields and give 1 dof/vertex for each field, and 1dof/edge for the first field (have to also set the cumulative dofs per point)</div><div><br></div><div>2) SectionSetUp()</div><div><br></div><div>3) Add a permutation if necessary to the points to get the dof ordering you want</div><div><br></div><div>Does this make sense?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Nicolas</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le mar. 27 avr. 2021 à 14:15, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Tue, Apr 27, 2021 at 2:51 AM Karin&NiKo <<a href="mailto:niko.karin@gmail.com" target="_blank">niko.karin@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear PETSc Team,<div><br></div><div>I am coming back to you to get some clue on the (missing?) Fortran interface to the DMShellSetCreateSubDM routine.</div></div></blockquote><div><br></div><div>Yes, we will have to put that in. Fortran bindings for callbacks are somewhat involved.</div><div><br></div><div>However, it is easier if you layout can be described by a PetscSection (which has all the Fortran bindings). Then it will</div><div>work automatically from the DMShell (I think). What kind of layout were you looking for?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thanks,</div><div>Nicolas</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le dim. 25 avr. 2021 à 18:20, Karin&NiKo <<a href="mailto:niko.karin@gmail.com" target="_blank">niko.karin@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Petsc Team, <br><br>I have begun to write a small test to illustrate the use of a DMShell in order to nest splits from the command line.<br>I am writing it in Fortran because the targeted application is also in Fortran.<div>=> I am afraid that DMShellSetCreateSubDM lacks a Fortran interface.<br></div><div>Am I wrong ? Can't the interface be generated by bfort ?</div><div><br></div><div>Thanks,</div><div>Nicolas</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le lun. 19 avr. 2021 à 12:28, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Sun, Apr 18, 2021 at 11:54 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><br></div> Matt,<div><br></div><div> I agree it can be done with DMs, but I am not convinced that is necessarily a superior way when one has to make DMSHELLS and mess around with their behavior.</div><div><br></div><div> One way to look at PCFIELDSPLIT is that its basic splitting tool is IS and DM is a way of generating IS that can be used for PCFIELDSPLIT. In this approach PCFIELDSPLIT needs a bit more support for IS and nesting. To also help using multiple levels of DM.</div><div><br></div><div> The other way is to view DM and subDM as the basic splitting tool for PCFIELDSPLIT but then one has to ask the question how does DM communicate its splits to PCFIELDSPLIT? I am too lazy to look at the code but presumably with IS, hence the approach above.</div><div><br></div><div> In the traditional Unix software stack approach to code, each layer uses the simpler layer below to communicate; so DM uses IS to communicate to the layer below (the linear algebra and preconditioners). DM is a hugely complicated layer and making it communicate directly with the vector level is complex; I like having the IS level in between to simplify the software stack and programming model. </div><div><br></div><div> PetscSection makes live more complicated since it is a bit disjoint from IS. I've never resolved in my mind what role PetscSection plays, is it a level above IS in the software stack that generates IS, does it sometimes "skip over" IS to directly talk to linear algebra? </div><div><br></div><div> If you cannot make a cartoon picture of the software stack, with all the objects, then I think the software stack is not well designed or defined. I fear we cannot make such a cartoon currently.</div></div></blockquote><div><br></div><div>DM _does_ communicate with PCFIELDSPLIT using IS. I agree with you that IS is a good object for communication. In PETSc, IS is just a nice way to pass a list of integers.</div><div><br></div><div>I don't think DM is hugely complicated. It does a few simple jobs. Here it's job is to remember which field each dof belongs to. That is all we have to worry about.</div><div><br></div><div>PetscSection semantically is a linear space with some structure. We already know we want some structure like this, since we break all linear spaces in PETSc into processes. Section allows you to break it down a little finer into the pieces of the space for each "point", where you can use a point to mark anything you want, like a process, cell, edge, another dof, etc. Sections can make an IS when asked a question, such as "which dofs lie in the closure of this cell", or "which dofs are in this field", or "which dofs are owned by this process". I have written this in the manual years ago.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div> Barry</div><div><br></div><div><br></div><div><br><div><br><blockquote type="cite"><div>On Apr 18, 2021, at 8:54 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:</div><br><div><div dir="ltr"><div dir="ltr">On Sat, Apr 17, 2021 at 6:13 PM Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
So you would like to be able to create three IS in your code and attach them with names to the PC. Then have -pc_fieldsplit_XXX_fields be able to utilize the attached IS by name and use them to define the blocks. <br>
<br>
This is all doable and could be added to PCFIELDSPLIT without too much code, new code. The code would be largely like PCFieldSplitSetRuntimeSplits_Private.<br>
<br>
The recursive part may also be doable but I think your syntax below is not exactly right. You would need something like <br>
<br>
-fieldsplit_0_pc_type fieldsplit // split the first PC into a fieldsplit<br>
-fieldsplit_0_pc_fieldsplit_0_fields xxx -fieldsplit_0_fieldsplit_0_pc_type jacobi<br>
-fieldsplit_0_pc_fieldsplit_1_fields yyy <br>
etc<br>
<br>
this would split the first field into two fields and use jacobi on the first field.<br>
<br>
The problem is what to use for labelling the xxx and the yyy? <br>
<br>
I think one could achieve this by having the PCFIELDPLIT attach to each of its split PCs the appropriate modified IS with names attached to them. There are two cases, <br>
<br>
when building the split uses first all the entries from fieldsplit_v_, then from fieldsplit_p_ then the new ISs it needs to attach to the first split PC are two sets of integers the first from 0 to the len(v)-1 and the second from len(v) to len(v)+len(p)-1. <br>
<br>
when building the split it interlaces the indices from v and p (interlacing only make sense if the size of v and p is the same). Then the new v would be {0,2,4,...} and p would be {1,3,...}. <br>
<br>
If you are ambitious and would like to add this to fieldsplit.c we'd be very happy to receive an MR. It might even lead to allowing us to simply how the PCFIELDPLIT interacts with DMs. If all the split type, stride, named, etc are handle in a single consistent manner.<br></blockquote><div><br></div><div>Barry, this is already working with DMs, which I think is the right way to do this.</div><div><br></div><div>Here is the code:</div><div><br></div><div> <a href="https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L420" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/ksp/pc/impls/fieldsplit/fieldsplit.c#L420</a><br></div><div><br></div><div>The DM must respond to DMCreateSubDM(). The interface is that the call provides a list of fields [f_0, f_1, ...]</div><div>and the DM returns an IS for that combination and a subdm for it. The subdm part is what allows you to do this</div><div>recursively, since you make the same call on the subdm.</div><div><br></div><div>If you use a PetscSection to keep track of the data layout, the code to do field selection is already done:</div><div><br></div><div> <a href="https://gitlab.com/petsc/petsc/-/blob/main/src/dm/interface/dmi.c#L105" target="_blank">https://gitlab.com/petsc/petsc/-/blob/main/src/dm/interface/dmi.c#L105</a></div><div><br></div><div>which can just be called from the DMCreateSubDM() implementation.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Barry<br>
<br>
<br>
<br>
> On Apr 17, 2021, at 11:53 AM, Karin&NiKo <<a href="mailto:niko.karin@gmail.com" target="_blank">niko.karin@gmail.com</a>> wrote:<br>
> <br>
> Dear PETSc users,<br>
> <br>
> I use the fieldsplit PC in an application where the splits are programmatically defined by IS using PCFieldSplitSetIS. Then the user can specify its own PC at runtime using PETSc options.<br>
> My question : is it possible to define nested splits in this case as it can be done with strided splits (see snes/examples/tutorials/ex19.c with test suffix fieldsplit_4).<br>
> <br>
> In order to be perfectly clear : let's say I have a 3 fields problem : velocity (v split), pressure (p split) and temperature (t split).<br>
> What I would like to do is something like the following but it fails : <br>
> <br>
> -ksp_type fgmres<br>
> -pc_fieldsplit_type multiplicative <br>
> -pc_type fieldsplit -pc_fieldsplit_0_fields fieldsplit_v_, fieldsplit_p_ -pc_fieldsplit_1_fields fieldsplit_t_ <br>
> <br>
> -prefix_push fieldsplit_0_ <br>
> -ksp_type fgmres<br>
> -pc_fieldsplit_schur_factorization_type upper <br>
> -pc_type fieldsplit <br>
> -pc_fieldsplit_type schur <br>
> -pc_fieldsplit_schur_precondition a11 <br>
> -prefix_pop<br>
> <br>
> -prefix_push fieldsplit_1_ <br>
> -ksp_type fgmres<br>
> -pc_type jacobi<br>
> -prefix_pop<br>
> <br>
> -prefix_push fieldsplit_v_ <br>
> -ksp_type fgmres<br>
> -pc_type gamg<br>
> -prefix_pop<br>
> <br>
> -prefix_push fieldsplit_p_ <br>
> -ksp_type fgmres<br>
> -pc_type jacobi<br>
> -prefix_pop<br>
> <br>
> I thank you for your help,<br>
> Nicolas<br>
> <br>
> <br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</div></blockquote></div><br></div></div></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>