[petsc-users] DMPlex ghost cell indexing

[Nicholas Arnold-Medabalimi]

Dear PETSc users,

I have some general novice questions regarding how to properly access a
distributed DMPLEX vector that has ghost cells for a finite volume use case.

My process for setup
1) generated a simple box mesh using DMPlexCreateBoxMesh
2) distribute the mesh using DMPlexDistribute (at this point everything
looks fine when I View (as paraview file) and I can see the partitions)
3) I generate a section with 5 variables located at the cells. (I don't
think it matters if I use 5 fields or 1 field with 5 components?) I can
alternatively do this using the PetscFV variable with 5 components.
4)  I then have the dm setup properly (with no ghost cells at this point)
and I use DMCreateGlobalVector and DMCreateLocalVector to get the Vectors
to work with
5) I'm just setting a quadratic IC. The overall loop over the cells is
constructed via getting the cell stratum which has the Cell point list and
then using the SectionOffset to assign each of the components at each cell
using DMPlexComputeCellGeometryFVM to get the cell centroid.

As far as I can tell this is an adequate solution but now I'm moving into
learning how to access neighboring cells for flux calculations which isn't
just the local cell but requires access to neighboring cells which at some
point will be located on neighboring processors and synched to be
accessible. When I did this for a DMDA it was pretty straightforward
insofar as I just needed to call DMDAGetCorners and the LocalVector
indexing would allow the stencil to extend gracefully into the ghost
values.

My general question is how do I achieve a similar loop structure for DMPlex
as far as the indexing over the owned cells while being able to access the
neighbor halo cells. I have been looking at example 52 and 11 in the ts
tutorial examples but I'm struggling to extract exactly what I need.

To boil it down into 3 starting questions
1) What is the preferred way to setup the cells and then access the
internal cells for say N cell halos for the boundaries between processors
in a DMPlex?
2) What is the distinction between the overlap argument in DMPlexDistribute
and DMPlexConstructGhostCells (which I see used in the examples)? (at least
based on my tests it seems GhostCells is generating cells at the overall
boundaries not the internal partition boundaries?)
3) Once I do have a DMPlex object that has the appropriate halos, how would
I get the access range for those cells? Based on my current assumption the
Stratum range for the Cells would include the ghost cells as well which I
would like to avoid? (Ultimately in the examples I see cases where there is
the synch step for Local to Global and vice versa but I'm trying to just
get to the point where I have the DM setup properly to handle the internal
mesh boundaries and have the correct indexing from the StratumGet commands.)

Any advice and corrections would be welcome.

Sincerely
Nicholas

-- 
Nicholas Arnold-Medabalimi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220702/4ff01f79/attachment.html>