[petsc-users] DMCloning from a DMPlex has changed in Petsc-3.17.0?

[Matthew Knepley]

On Wed, Apr 6, 2022 at 9:41 AM Berend van Wachem <berend.vanwachem at ovgu.de>
wrote:

> Dear Matt,
>
> I have made a small working example of cloning a DM, illustrating the
> problem we have.
> In the attached code, I wrote a function 'CloneDMWithNewSection', which
> clones a section and puts a different number of fields on it.
>
> The code itself prints the number of local cells of the DM, which
> changes as the DM is cloned.
> In our code, we assume that the cloned DM should have exactly the same
> partitioning - this was the behaviour in PETSc versions prior to 3.17.
>
> If I run the attached code on 2 processors, I get:
>
> First DM: Processor 1 reports Start: 0, End 4000 giving number of local
> cells: 4000
> First DM: Processor 0 reports Start: 0, End 4000 giving number of local
> cells: 4000
>
> Cloned DM: Processor 1 reports Start: 0, End 3984 giving number of local
> cells: 3984
> Cloned DM: Processor 0 reports Start: 0, End 4016 giving number of local
> cells: 4016
>
> Maybe we are doing something wrong in the function CloneDMWithNewSection?
>

I apologize for taking so long on this. Jed persuaded me to change the
default. Now, when you
call DMSetFromOptions() it distributes by default, rather than requiring
you to explicitly call it.
You can shut this behavior off, so that if you add

  ierr = DMPlexDistributeSetDefault(*NewDM, PETSC_FALSE);CHKERRQ(ierr);

right after DMClone(), you will preserve the layout you have.

Also, the call to DMPlexDistribute() here (and the Partitioner calls) are
now superfluous.

  Thanks,

     Matt


> Many thanks for looking into this, best regards,
> Berend.
>
>
>
> On 4/4/22 23:05, Matthew Knepley wrote:
> > On Mon, Apr 4, 2022 at 3:36 PM Berend van Wachem
> > <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>> wrote:
> >
> >     Dear Petsc team,
> >
> >     Since about 2 years we have been using Petsc with DMPlex, but since
> >     upgrading our code to Petsc-3.17.0 something has broken.
> >
> >     First we generate a DM from a DMPlex with  DMPlexCreateFromFile or
> >     creating one with DMPlexCreateBoxMesh. Then the DM is distributed
> with
> >     DMPlexDistribute. This DM works fine and we set a numer of fields and
> >     set a section to it.
> >     However, on the same mesh we also want to solve a problem with a
> >     different number of fields, and therefore we create a clone of this
> >     original DM, using the code:
> >
> >     DMClone(OriginalDM, NewDM);
> >     DMClearDS(*NewDM);
> >     PetscCalloc2(1, &NumComp, 4, &NumDof);
> >     NumComp[0] = 1;
> >     NumDof[3] = NFields;
> >     DMSetNumFields(*NewDM, 1);
> >     DMSetFromOptions(*NewDM);
> >     DMPlexCreateSection(*NewDM, NULL, NumComp, NumDof, 0, NULL, NULL,
> NULL,
> >     NULL, &section);
> >     DMSetLocalSection(*NewDM, section);
> >     PetscFree2(NumComp, NumDof);
> >     PetscSectionDestroy(&section);
> >
> >     However, with Petsc-3.17.0, the *NewDM is corrupt - When I call
> >     DMGlobalToLocalBegin with a Global and Local vector created with this
> >     NewDM, the code crashes. Indeed, the cloned DM seems to be
> partitioned
> >     differently than the original DM, as it these two DMs have a
> different
> >     number of local cells.
> >
> >
> > The cloned DM will have exactly the same topology and distribution. This
> > must be a misinterpretation
> > of what is happening. We can do a few things:
> >
> > 1) Make a small example to show what you are talking about
> >
> > 2) Look at a PETSc example that does something similar
> >
> > 3) Look directly at your code if I can somehow run it here
> >
> > 4) Start doing diagnostics on your code to see what is going on
> >
> > Which one do you prefer?
> >
> >     This worked fine in Petsc releases before 3.17 (e.g. 3.16.5). So my
> >     question is: what has changed? Am I doing something wrong, which
> should
> >     be changed for using with Petsc-3.17?
> >
> >
> > I don't think any of this should have changed, so this should be
> > something simple.
> >
> >    Thanks,
> >
> >       Matt
> >
> >     Thanks, best regards,
> >
> >     Berend.
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which
> > their experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/ <
> http://www.cse.buffalo.edu/~knepley/>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220411/a87c1a32/attachment.html>