[petsc-dev] Petsc "make test" have more failures for --with-openmp=1

[Matthew Knepley]

On Fri, Mar 19, 2021 at 2:55 PM Jed Brown <jed at jedbrown.org> wrote:

> Matthew Knepley <knepley at gmail.com> writes:
>
> > On Fri, Mar 19, 2021 at 2:21 PM Jed Brown <jed at jedbrown.org> wrote:
> >
> >> Matthew Knepley <knepley at gmail.com> writes:
> >>
> >> >> Notice how the permutations are contained within the vertices {0,
> ...,
> >> 8},
> >> >> edges {9, ..., 24}, and cells {25, ..., 32}. I would like to get rid
> of
> >> >> that restriction, but you've said it would have significant non-local
> >> >> consequences so I haven't tried.
> >> >>
> >> >
> >> > Yes, that is not a good idea, even in theory. It wrecks the nice
> >> contiguity
> >> > we use for topological operations.
> >> >
> >> > As Lawrence points out, this need not affect your smoother because dof
> >> > permutations can break this stratification. It has been that way for a
> >> > decade.
> >>
> >> I only want to break dof stratification, not point stratification. The
> >> points produced by the stratified RCM are okay (at least as Lawrence
> >> described it; not sure how it's done in DMPlexGetOrdering, but can't be
> >> anything hard).
> >>
> >> What code needs to be written to get unstratified RCM-like dofs?
> Lawrence
> >> evidently has this code in Firedrake, but I want it in DMPlex, tested
> and
> >> probably done by default because most users would benefit from it. I
> could
> >> swear you said it would break some other assumptions and thus not
> >> everything would work, but perhaps we weren't talking about the same
> thing.
> >>
> >
> > It is possible that I misunderstood what you were asking for. What
> Section
> > requires for everything to work is that dofs follow the point ordering.
>
> I think what you mean is that the dofs for any point are contiguous, not
> that the dofs for point p come before the dofs for point p+1.
>

Yes.


> > The point ordering can be anything we want since we can use a
> permutation from the mesh point ordering. However, some stuff depends on
> all dofs associated with a point being contiguous. Not everything does,
> since I wrote something for LibMesh which allows them to order in a field
> major way, instead of point major, but it is probably not possible to
> individually permute unknowns. Usually, this is not what we want anyway I
> would assume.
>
> I don't want to break up the dofs for a point, just an unstratified
> locality-preserving ordering of dofs (not points).
>

Good.


> PetscSectionSetUp currently calculates offsets by walking the points in
> order. I'd like to be able to walk them in a different ordering, perhaps
> specified via an IS permutation, which could be computed using
> MATORDERINGRCM or directly via BFS of a closure (saves building a Mat).
>

You can do this using


https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PetscSection/PetscSectionSetPermutation.html

This is what Lawrence is doing. He permutes the points so that interior
points come first, then points that are not
shared but touch shared points, then shared points.

  Thanks,

      Matt


> 1. Would this break anything you can think of?
> 2. How would you like it implemented?
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20210319/ef9e0dd1/attachment.html>