[petsc-users] DMCloning from a DMPlex has changed in Petsc-3.17.0?
Matthew Knepley
knepley at gmail.com
Tue Apr 12 05:49:32 CDT 2022
On Tue, Apr 12, 2022 at 2:50 AM Berend van Wachem <berend.vanwachem at ovgu.de>
wrote:
> Dear Matt,
>
> Thank you very much - I can confirm that that works.
>
> I have one question about your remark:
>
> > Also, the call to DMPlexDistribute() here (and the Partitioner calls)
> > are now superfluous.
>
> If I shouldn't call DMPlexDistribute(),
I just meant that DMPlexDistribute() was being called automatically from
within DMSetFromOptions()
> how should I set the overlap of
> the DM from within the code (our code determines the overlap required)?
>
-dm_distribute_overlap <n> - The size of the overlap halo
from https://petsc.org/main/docs/manualpages/DM/DMSetFromOptions.html
Thanks,
Matt
> Many thanks, best regards,
>
> Berend.
>
>
>
>
> On 4/11/22 16:23, Matthew Knepley wrote:
> > On Wed, Apr 6, 2022 at 9:41 AM Berend van Wachem
> > <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>> wrote:
> >
> > Dear Matt,
> >
> > I have made a small working example of cloning a DM, illustrating the
> > problem we have.
> > In the attached code, I wrote a function 'CloneDMWithNewSection',
> which
> > clones a section and puts a different number of fields on it.
> >
> > The code itself prints the number of local cells of the DM, which
> > changes as the DM is cloned.
> > In our code, we assume that the cloned DM should have exactly the
> same
> > partitioning - this was the behaviour in PETSc versions prior to
> 3.17.
> >
> > If I run the attached code on 2 processors, I get:
> >
> > First DM: Processor 1 reports Start: 0, End 4000 giving number of
> local
> > cells: 4000
> > First DM: Processor 0 reports Start: 0, End 4000 giving number of
> local
> > cells: 4000
> >
> > Cloned DM: Processor 1 reports Start: 0, End 3984 giving number of
> > local
> > cells: 3984
> > Cloned DM: Processor 0 reports Start: 0, End 4016 giving number of
> > local
> > cells: 4016
> >
> > Maybe we are doing something wrong in the function
> > CloneDMWithNewSection?
> >
> >
> > I apologize for taking so long on this. Jed persuaded me to change the
> > default. Now, when you
> > call DMSetFromOptions() it distributes by default, rather than requiring
> > you to explicitly call it.
> > You can shut this behavior off, so that if you add
> >
> > ierr = DMPlexDistributeSetDefault(*NewDM, PETSC_FALSE);CHKERRQ(ierr);
> >
> > right after DMClone(), you will preserve the layout you have.
> >
> > Also, the call to DMPlexDistribute() here (and the Partitioner calls)
> > are now superfluous.
> >
> > Thanks,
> >
> > Matt
> >
> > Many thanks for looking into this, best regards,
> > Berend.
> >
> >
> >
> > On 4/4/22 23:05, Matthew Knepley wrote:
> > > On Mon, Apr 4, 2022 at 3:36 PM Berend van Wachem
> > > <berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>
> > <mailto:berend.vanwachem at ovgu.de <mailto:berend.vanwachem at ovgu.de>>>
> > wrote:
> > >
> > > Dear Petsc team,
> > >
> > > Since about 2 years we have been using Petsc with DMPlex, but
> > since
> > > upgrading our code to Petsc-3.17.0 something has broken.
> > >
> > > First we generate a DM from a DMPlex with
> > DMPlexCreateFromFile or
> > > creating one with DMPlexCreateBoxMesh. Then the DM is
> > distributed with
> > > DMPlexDistribute. This DM works fine and we set a numer of
> > fields and
> > > set a section to it.
> > > However, on the same mesh we also want to solve a problem
> with a
> > > different number of fields, and therefore we create a clone
> > of this
> > > original DM, using the code:
> > >
> > > DMClone(OriginalDM, NewDM);
> > > DMClearDS(*NewDM);
> > > PetscCalloc2(1, &NumComp, 4, &NumDof);
> > > NumComp[0] = 1;
> > > NumDof[3] = NFields;
> > > DMSetNumFields(*NewDM, 1);
> > > DMSetFromOptions(*NewDM);
> > > DMPlexCreateSection(*NewDM, NULL, NumComp, NumDof, 0, NULL,
> > NULL, NULL,
> > > NULL, §ion);
> > > DMSetLocalSection(*NewDM, section);
> > > PetscFree2(NumComp, NumDof);
> > > PetscSectionDestroy(§ion);
> > >
> > > However, with Petsc-3.17.0, the *NewDM is corrupt - When I
> call
> > > DMGlobalToLocalBegin with a Global and Local vector created
> > with this
> > > NewDM, the code crashes. Indeed, the cloned DM seems to be
> > partitioned
> > > differently than the original DM, as it these two DMs have a
> > different
> > > number of local cells.
> > >
> > >
> > > The cloned DM will have exactly the same topology and
> > distribution. This
> > > must be a misinterpretation
> > > of what is happening. We can do a few things:
> > >
> > > 1) Make a small example to show what you are talking about
> > >
> > > 2) Look at a PETSc example that does something similar
> > >
> > > 3) Look directly at your code if I can somehow run it here
> > >
> > > 4) Start doing diagnostics on your code to see what is going on
> > >
> > > Which one do you prefer?
> > >
> > > This worked fine in Petsc releases before 3.17 (e.g. 3.16.5).
> > So my
> > > question is: what has changed? Am I doing something wrong,
> > which should
> > > be changed for using with Petsc-3.17?
> > >
> > >
> > > I don't think any of this should have changed, so this should be
> > > something simple.
> > >
> > > Thanks,
> > >
> > > Matt
> > >
> > > Thanks, best regards,
> > >
> > > Berend.
> > >
> > >
> > >
> > > --
> > > What most experimenters take for granted before they begin their
> > > experiments is infinitely more interesting than any results to
> which
> > > their experiments lead.
> > > -- Norbert Wiener
> > >
> > > https://www.cse.buffalo.edu/~knepley/
> > <https://www.cse.buffalo.edu/~knepley/>
> > <http://www.cse.buffalo.edu/~knepley/
> > <http://www.cse.buffalo.edu/~knepley/>>
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which
> > their experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/ <
> http://www.cse.buffalo.edu/~knepley/>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220412/169510b2/attachment.html>
More information about the petsc-users
mailing list