[MOAB-dev] ghost/halo elements

[Vijay S. Mahadevan]

Hi Ryan,

Glad the example helped.

> The example shows the ghost exchange happening between shared owned
entities. In my experiments I've been unable to make the ghost exchange
work, and I think that might be because my entities are not shared.

You need to call resolve_shared_entities on the entities prior to
doing ghost exchange. When we load the mesh from file, this happens
automatically based on the read options but if you are forming a mesh
in memory, you need to make sure that the shared vertices have the
same GLOBAL_ID numbering consistently across processes. That is shared
vertices are unique. Once that is set, shared entities resolution will
work correctly out of the box and you will have shared edges/entities
query working correctly. A call to get ghosted layers once this is
done would be the way to go.

I assume ESMF has a unique global numbering for the vertices ? Use
that to set the GLOBAL_ID tag. Let us know if you are still having
issues.

Vijay
On Thu, Oct 25, 2018 at 11:17 AM Ryan O'Kuinghttons
<ryan.okuinghttons at noaa.gov> wrote:
>
> Hi Vijay,
>
> I've had some time to play with this now, and I have another question.
> Thank you for sending along the example code, it has been extremely
> helpful.
>
> The example shows the ghost exchange happening between shared owned
> entities. In my experiments I've been unable to make the ghost exchange
> work, and I think that might be because my entities are not shared. The
> situation I have is entities that are owned wholly on a single
> processor, which need to be communicated to other processors which
> require them as part of a halo region for mesh based computation. In
> this situation would I need to "share" my entities across the whole
> processor space, before requesting a ghost exchange? I'm not even really
> sure what shared entities mean, is there a good place to look in the
> documentation to learn more about the terminology? Thanks,
>
> Ryan
>
> On 10/8/18 12:17, Vijay S. Mahadevan wrote:
> > Ryan,
> >
> > You need to use the ParallelComm object in MOAB to call
> > exchange_ghost_cells [1] with appropriate levels of ghost layers for
> > your mesh. This needs to be done after the mesh is loaded, or you can
> > pass this information also as part of the options when loading a file
> > and MOAB will internally load the file with ghosted layers. Here's an
> > example [2].
> >
> > Vijay
> >
> > [1] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a55dfa308f56fd368319bfb4244428878
> > [2] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/HelloParMOAB_8cpp-example.html
> > On Mon, Oct 8, 2018 at 1:23 PM Ryan O'Kuinghttons
> > <ryan.okuinghttons at noaa.gov> wrote:
> >> Hi, I am wondering if there is a way to create ghost elements in MOAB.
> >> By this I mean a list of copies of elements surrounding a specific MOAB
> >> element, ghost elements may exist on a different processor than the
> >> source element. I see a ghost_elems class in the appData namespace, but
> >> there is not much documentation on how to use it.. Thank you,
> >>
> >> --
> >> Ryan O'Kuinghttons
> >> Cherokee Nation Management and Consulting
> >> NESII/NOAA/Earth System Research Laboratory
> >> ryan.okuinghttons at noaa.gov
> >> https://www.esrl.noaa.gov/gsd/nesii/
> >>