[petsc-users] DMPlex/PetscSF How to determine if local topology is other rank's ghost?

[Toby Isaac]

A leaf point is attached to a root point (in a star forest there are only
leaves and roots), so that means that a root point would be the point that
owns a degree of freedom and a leaf point would have a ghost value.

For a "point SF" of a DMPlex:

- Each process has a local numbering of mesh points (cells + edges + faces
+ vertices): they are all potential roots, so the number of these is what
is returned by `nroots`.

- The number of ghost mesh points is `nleaves`.

- `ilocal` would be a list of the mesh points that are leaves (using the
local numbering).

- For each leaf in `ilocal`, `iremote` describes the root it is attached
to: which process it belongs to, and its id in *that* process's local
numbering.

If you're trying to create dof numberings on your own, please consider
PetscSectionCreateGlobalSection: <
https://petsc.org/main/docs/manualpages/PetscSection/PetscSectionCreateGlobalSection/>.
You supply the PetscSF and a PetscSection which says how many dofs there
are for each point and whether any have essential boundary conditions, and
it computes a global PetscSection that tells you what the global id is for
each dof on this process.


On Tue, May 17, 2022 at 7:26 PM Ferrand, Jesus A. <FERRANJ2 at my.erau.edu>
wrote:

> Dear PETSc team:
>
> I am working with a non-overlapping distributed plex (i.e., when I call
> DMPlexDistribute(), I input overlap = 0), so only vertices and edges appear
> as ghosts to the local ranks.
> For preallocation of a parallel global stiffness matrix for FEA, I want to
> determine which locally owned vertices are ghosts to another rank.
>
> From reading the paper on PetscSF (
> https://ieeexplore.ieee.org/document/9442258) I think I can answer my
> question by inspecting the PetscSF returned by DMPlexDistribute() with
> PetscSFGetGraph(). I am just confused by the root/leaf and ilocal/iremote
> terminology.
>
> I read the manual page on PetscSFGetGraph() (
> https://petsc.org/release/docs/manualpages/PetscSF/PetscSFGetGraph.html)
> and that gave me the impression that I need to PetscSFBcast() the point IDs
> from foreign ranks to the local ones.
>
> Is this correct?
>
>
> <https://ieeexplore.ieee.org/document/9442258>
> The PetscSF Scalable Communication Layer | IEEE Journals & Magazine | IEEE
> Xplore <https://ieeexplore.ieee.org/document/9442258>
> PetscSF, the communication component of the Portable, Extensible Toolkit
> for Scientific Computation (PETSc), is designed to provide PETSc's
> communication infrastructure suitable for exascale computers that utilize
> GPUs and other accelerators. PetscSF provides a simple application
> programming interface (API) for managing common communication patterns in
> scientific computations by using a star ...
> ieeexplore.ieee.org
> **
>
>
>
> Sincerely:
>
> *J.A. Ferrand*
>
> Embry-Riddle Aeronautical University - Daytona Beach FL
>
> M.Sc. Aerospace Engineering | May 2022
>
> B.Sc. Aerospace Engineering
>
> B.Sc. Computational Mathematics
>
>
>
> Sigma Gamma Tau
>
> Tau Beta Pi
>
>
>
> *Phone:* (386)-843-1829
>
> *Email(s):* ferranj2 at my.erau.edu
>
>     jesus.ferrand at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220518/6c80dffd/attachment-0001.html>