[petsc-users] What exactly is the GlobalToNatural PetscSF of DMPlex/DM?

[Matthew Knepley]

On Mon, Jul 8, 2024 at 10:28 PM Ferrand, Jesus A. <FERRANJ2 at my.erau.edu>
wrote:

> Dear PETSc team: Greetings. I keep working on mesh I/O utilities using
> DMPlex. Specifically for the output stage, I need a solid grasp on the
> global numbers and ideally how to set them into the DMPlex during an input
> operation and carrying
> ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
>
> ZjQcmQRYFpfptBannerEnd
> Dear PETSc team:
>
> Greetings.
> I keep working on mesh I/O utilities using DMPlex.
> Specifically for the output stage, I need a solid grasp on the global
> numbers and ideally how to set them into the DMPlex during an input
> operation and carrying the global numbers through API calls to
> DMPlexDistribute() or DMPlexMigrate() and hopefully also through some of
> the mesh adaption APIs. I was wondering if the GlobalToNatural PetscSF
> manages these global numbers. The next most useful object is the PointSF,
> but to me, it seems to only help establish DAG point ownership, not DAG
> point global indices.
>

This is a good question, and gets at a design point of Plex. I don't
believe global numbers are the "right" way to talk about mesh points, or
even a very useful way to do it, for several reasons. Plex is designed to
run just fine without any global numbers. It can, of course, produce
them on command, as many people remain committed to their existence.

Thus, the first idea is that global numbers should not be stored, since
they can always be created on command very cheaply. It is much more
costly to write global numbers to disk, or pull them through memory, than
compute them.

The second idea is that we use a combination of local numbers, namely
(rank, point num) pairs, and PetscSF objects to establish sharing relations
for parallel meshes. Global numbering is a particular traversal of a mesh,
running over the locally owned parts of each mesh in local order. Thus an
SF + a local order = a global order, and the local order is provided by the
point numbering.

The third idea is that a "natural" order is just the global order in which
a mesh is first fed to Plex. When I redistribute and reorder for good
performance, I keep track of a PetscSF that can map the mesh back to the
original order in which it was provided. I see this as an unneeded expense,
but many many people want output written in the original order (mostly
because processing tools are so poor). This management is what we mean by
GlobalToNatural.


> Otherwise, I have been working with the IS obtained from
> DMPlexGetPointNumbering() and manually determining global stratum sizes,
> offsets, and numbers by looking at the signs of the involuted index list
> that comes with that IS. It's working for now (I can monolithically write
> meshes to CGNS in parallel), but it is resulting in repetitive code that I
> will need for another mesh format that I want to support.
>

What is repetitive? It should be able to be automated.

  Thanks,

    Matt


> Sincerely:
>
> *J.A. Ferrand*
>
> Embry-Riddle Aeronautical University - Daytona Beach - FL
> Ph.D. Candidate, Aerospace Engineering
>
> M.Sc. Aerospace Engineering
>
> B.Sc. Aerospace Engineering
>
> B.Sc. Computational Mathematics
>
>
> *Phone:* (386)-843-1829
>
> *Email(s):* ferranj2 at my.erau.edu
>
>     jesus.ferrand at gmail.com
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://urldefense.us/v3/__https://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zcybs3rxgbG35ciZiIHB3TY07Qnjd1sD0HzJVDWwr-OuDyXtVjDJ8WbIMS4LRixsMUZwLGtwsznQ8MeOalw3$  <https://urldefense.us/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!G_uCfscf7eWS!Zcybs3rxgbG35ciZiIHB3TY07Qnjd1sD0HzJVDWwr-OuDyXtVjDJ8WbIMS4LRixsMUZwLGtwsznQ8PkYHNH4$ >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240711/34eeea51/attachment.html>