[petsc-users] creation of parallel dmplex from a partitioned mesh

[Cameron Smith]

Hello,

We have a partitioned mesh that we want to create a DMPlex from that has 
the same distribution of elements (i.e., assignment of elements to 
processes) and vertices 'interior' to a process (i.e., vertices not on 
the inter-process boundary).

We were trying to use DMPlexCreateFromCellListParallelPetsc() or 
DMPlexBuildFromCellListParallel() and found that the vertex ownership 
(roots in the returned Vertex SF) appears to be sequentially assigned to 
processes based on global vertex id.  In general, this will not match 
our mesh distribution.  As we understand, to subsequently set vertex 
coordinates (or other vertex data) we would have to utilize a star 
forest (SF) communication API to send data to the correct process. Is 
that correct?

Alternatively, if we create a dmplex object from the elements that exist 
on each process using DMCreateFromCellList(), and then create a SF from 
mesh vertices on inter-process boundaries (using the mapping from local 
to global vertex ids provided by our mesh library), could we then 
associate the dmplex objects with the SF?  Is it as simple as calling 
DMSetPointSF()?

If manually defining the PointSF is a way forward, we would like some 
help understanding its definition; i.e., which entities become roots and 
which become leaves.  In DMPlexBuildFromCellListParallel()

https://gitlab.com/petsc/petsc/-/blob/753428fdb0644bc4cb7be6429ce8776c05405d40/src/dm/impls/plex/plexcreate.c#L2875-2899

the PointSF appears to contain roots for elements and vertices and 
leaves for owned vertices on the inter-process boundary.  Is that correct?

Thank-you,
Cameron