[petsc-users] DMPlexDistributeField

[Matthew Knepley]

On Fri, Jun 21, 2019 at 12:49 AM Adrian Croucher via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> I have been trying to get this FVM geometry data re-distribution to work
> using DMPlexDistributeField().
>
> It seems to be working OK for the cell geometry data (cell volumes and
> centroids). But it is making a mess of the face geometry data (face
> normals and centroids).
>
> Should I even expect DMPlexDistributeField() to work for redistributing
> a vector of data defined on mesh faces? Or is there some reason I
> haven't thought of, which means that will never work?
>

Sorry this took a long time. The place I was in France did not have
internet.
Here is how this stuff works:

  1) You start with a Section and local vector. The Section describes
layout of data in
       the local vector by mapping mesh points to {# dof, offset}. For a
small example,
       suppose I had two triangles sharing an edge on a sequential mesh for
2 procs.
       The mesh points would be

         [0, 1]:  Cells
         [2, 5]:  Vertices, where 3,4 are shared
         [6, 10]: Edges, where 8 is shared

       A Section for face normals would then look like

        Process 0
        [0, 5]: {0, 0}   Meaning no variables lie on cells or vertices
        6:       {2, 0}   One vector per face
        7:       {2, 2}
        8:       {2, 4}
        9:       {2, 6}
        10      {2, 8}
        Process 1
        empty

     The vector would just have the face normal values in the canonical
order. You can use
     PetscSectionView() to check that yours looks similar.

  2) Now we add a PetscSF describing the redistribution. An SF is a map
from a given set of
      integers (leaves) to pairs (int, rank) called roots. Many leaves can
point to one root. To begin,
      we provide an SF mapping mesh points to the new distribution

        Process 0
        0 -> {0, 0}
        1 -> {2, 0}
        2 -> {3, 0}
        3 -> {4, 0}
        4 -> {6, 0}
        5 -> {7, 0}
        6 -> {8, 0}
        Process 1
        0 -> {1, 0}
        1 -> {4, 0}
        2 -> {3, 0}
        3 -> {5, 0}
        4 -> {8, 0}
        5 -> {9, 0}
        6 -> {10, 0}

     This tells Petsc how to move the points to the parallel distribution.
You can use PetscSFView() to check
     that your point SF is similar. We first use the SF to move the Section
data to make a new parallel Section,

       Process 0
       [0, 3]: {0, 0}
       4:       {2, 0}
       5:       {2, 2}
       6:       {2, 4}
       Process 1
       [0, 3]: {0, 0}
       4:       {2, 0}
       5:       {2, 2}
       6:       {2, 4}

     and then we create a new SF that maps dofs, instead of points, using
this new Section

     Process 0:
     0 -> {0, 0} First face normal
     1 -> {1, 0}
     2 -> {2, 0} Second face normal
     3 -> {3, 0}
     4 -> {4, 0} Third face normal
     5 -> {5, 0}
     Process 1:
     0 -> {4, 0} First face normal
     1 -> {5, 0}
     2 -> {6, 0} Second face normal
     3 -> {7, 0}
     4 -> {8, 0} Third face normal
     5 -> {9, 0}

    and then this moves the face normal data to the new layout just by
broadcasting.

You can View the structures you get back to see what Petsc thought it
should do.

Does this make sense?

  Thanks,

     Matt


> The only example I could find anywhere of DMPlexDistributeField() being
> used is in DMPlexDistributeCoordinates() so I've been basing what I'm
> doing on that.
>
> (I think I have answered my own questions below by experiment- 1) local
> vectors should work (they do in DMPlexDistributeCoordinates); 2)
> probably doesn't matter; 3) yes.)
>
> - Adrian
>
> On 6/06/19 1:42 PM, Adrian Croucher wrote:
> > hi
> >
> > I have some questions about using the DMPlexDistributeField()
> > function. I have finite volume mesh geometry data stored in two local
> > vectors created using DMPlexComputeGeometryFVM(), and I need to
> > redistribute these after calling DMPlexDistribute() to redistribute my
> > mesh. (I need the geometry data before redistribution, so I can't just
> > wait until after redistribution to create them.)
> >
> > So I figured DMPlexDistributeField() looks like the thing to use for
> > that, using the SF that comes out of DMPlexDistribute().
> >
> > 1) Does DMPlexDistributeField() work on local vectors or do they have
> > to be global ones?
> >
> > 2) It takes a 'dm' parameter and the documentation says this is "The
> > DMPlex object", but is that the original DM (before redistribution) or
> > the redistributed one, or does it not matter? It looks like it only
> > uses the DM to get the vector type.
> >
> > 3) It looks like you need to manually create the newSection and newVec
> > output parameters before passing them in to this routine, is that
> > correct?
> >
> > - Adrian
> >
> --
> Dr Adrian Croucher
> Senior Research Fellow
> Department of Engineering Science
> University of Auckland, New Zealand
> email: a.croucher at auckland.ac.nz
> tel: +64 (0)9 923 4611
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190623/c1e320e2/attachment.html>