[petsc-users] DMPlex/PetscSF How to determine if local topology is other rank's ghost?

[Matthew Knepley]

On Tue, May 17, 2022 at 6:47 PM Toby Isaac <toby.isaac at gmail.com> wrote:

> A leaf point is attached to a root point (in a star forest there are only
> leaves and roots), so that means that a root point would be the point that
> owns a degree of freedom and a leaf point would have a ghost value.
>
> For a "point SF" of a DMPlex:
>
> - Each process has a local numbering of mesh points (cells + edges + faces
> + vertices): they are all potential roots, so the number of these is what
> is returned by `nroots`.
>
> - The number of ghost mesh points is `nleaves`.
>
> - `ilocal` would be a list of the mesh points that are leaves (using the
> local numbering).
>
> - For each leaf in `ilocal`, `iremote` describes the root it is attached
> to: which process it belongs to, and its id in *that* process's local
> numbering.
>
> If you're trying to create dof numberings on your own, please consider
> PetscSectionCreateGlobalSection: <
> https://petsc.org/main/docs/manualpages/PetscSection/PetscSectionCreateGlobalSection/>.
> You supply the PetscSF and a PetscSection which says how many dofs there
> are for each point and whether any have essential boundary conditions, and
> it computes a global PetscSection that tells you what the global id is for
> each dof on this process.
>

Toby is exactly right. Also, if you want global numbering of points you can
use

  https://petsc.org/main/docs/manualpages/DMPLEX/DMPlexCreatePointNumbering/

and there is a similar thing for jsut cells or vertices.

  Thanks,

    Matt


> On Tue, May 17, 2022 at 7:26 PM Ferrand, Jesus A. <FERRANJ2 at my.erau.edu>
> wrote:
>
>> Dear PETSc team:
>>
>> I am working with a non-overlapping distributed plex (i.e., when I call
>> DMPlexDistribute(), I input overlap = 0), so only vertices and edges appear
>> as ghosts to the local ranks.
>> For preallocation of a parallel global stiffness matrix for FEA, I want
>> to determine which locally owned vertices are ghosts to another rank.
>>
>> From reading the paper on PetscSF (
>> https://ieeexplore.ieee.org/document/9442258) I think I can answer my
>> question by inspecting the PetscSF returned by DMPlexDistribute() with
>> PetscSFGetGraph(). I am just confused by the root/leaf and ilocal/iremote
>> terminology.
>>
>> I read the manual page on PetscSFGetGraph() (
>> https://petsc.org/release/docs/manualpages/PetscSF/PetscSFGetGraph.html)
>> and that gave me the impression that I need to PetscSFBcast() the point IDs
>> from foreign ranks to the local ones.
>>
>> Is this correct?
>>
>>
>> <https://ieeexplore.ieee.org/document/9442258>
>> The PetscSF Scalable Communication Layer | IEEE Journals & Magazine |
>> IEEE Xplore <https://ieeexplore.ieee.org/document/9442258>
>> PetscSF, the communication component of the Portable, Extensible Toolkit
>> for Scientific Computation (PETSc), is designed to provide PETSc's
>> communication infrastructure suitable for exascale computers that utilize
>> GPUs and other accelerators. PetscSF provides a simple application
>> programming interface (API) for managing common communication patterns in
>> scientific computations by using a star ...
>> ieeexplore.ieee.org
>> **
>>
>>
>>
>> Sincerely:
>>
>> *J.A. Ferrand*
>>
>> Embry-Riddle Aeronautical University - Daytona Beach FL
>>
>> M.Sc. Aerospace Engineering | May 2022
>>
>> B.Sc. Aerospace Engineering
>>
>> B.Sc. Computational Mathematics
>>
>>
>>
>> Sigma Gamma Tau
>>
>> Tau Beta Pi
>>
>>
>>
>> *Phone:* (386)-843-1829
>>
>> *Email(s):* ferranj2 at my.erau.edu
>>
>>     jesus.ferrand at gmail.com
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220517/49dbf5fb/attachment.html>