[MOAB-dev] ghost/halo elements

[Vijay S. Mahadevan]

You can try either the first or the second variant instead [1] with
following arguments.

ErrorCode moab::ParallelComm::resolve_shared_ents(EntityHandle this_set,
int resolve_dim = 2,
int shared_dim = -1,
const Tag * id_tag = 0
)

That should resolve 2-dim entities with shared edges across
partitions. You can leave id_tag pointer as zero since the default is
GLOBAL_ID.

> This brings up a more general question I've had about the moab
documentation for a while. In the doc for this routine, it only lists 2
parameters, proc_ents and shared_dim, even though in the function
signature above it clearly shows more. I've had trouble understanding
which parameters are relevant in the past, or what they do because I'm
not quite sure how to read the documentation.

This is an oversight. We will go through and rectify some of the
inconsistencies in the documentation. We are preparing for an upcoming
release and I'll make sure that routines in Core/ParallelComm have
updated documentation that match the interfaces. Meanwhile, if you
have questions, feel free to shoot an email to the list.

Hope that helps.

Vijay

[1] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a29a3b834b3fc3b4ddb3a5d8a78a37c8a
On Thu, Oct 25, 2018 at 1:58 PM Ryan O'Kuinghttons
<ryan.okuinghttons at noaa.gov> wrote:
>
> Hi Vijay,
>
> Thanks again for that explanation.
>
> ESMF does use unique global ids for the vertices and set them to the
> GLOBAL_ID in the moab mesh. So I think we are good there.
>
> I can't quite figure out how to use resolve_shared_entities though.
> There are three versions of the call in the documentation, and I assume
> that I should use the first and pass in a Range containing all entities
> for proc_ents:
>
> http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a59e35d9906f2e33fe010138a144a5cb6
>
> However, I'm not sure what the this_set EntityHandle should be.
>
> This brings up a more general question I've had about the moab
> documentation for a while. In the doc for this routine, it only lists 2
> parameters, proc_ents and shared_dim, even though in the function
> signature above it clearly shows more. I've had trouble understanding
> which parameters are relevant in the past, or what they do because I'm
> not quite sure how to read the documentation.
>
> Any example or explanation you give me for resolve_shared_entities is
> much appreciated!
>
> Ryan
>
> On 10/25/18 09:40, Vijay S. Mahadevan wrote:
> > Hi Ryan,
> >
> > Glad the example helped.
> >
> >> The example shows the ghost exchange happening between shared owned
> > entities. In my experiments I've been unable to make the ghost exchange
> > work, and I think that might be because my entities are not shared.
> >
> > You need to call resolve_shared_entities on the entities prior to
> > doing ghost exchange. When we load the mesh from file, this happens
> > automatically based on the read options but if you are forming a mesh
> > in memory, you need to make sure that the shared vertices have the
> > same GLOBAL_ID numbering consistently across processes. That is shared
> > vertices are unique. Once that is set, shared entities resolution will
> > work correctly out of the box and you will have shared edges/entities
> > query working correctly. A call to get ghosted layers once this is
> > done would be the way to go.
> >
> > I assume ESMF has a unique global numbering for the vertices ? Use
> > that to set the GLOBAL_ID tag. Let us know if you are still having
> > issues.
> >
> > Vijay
> > On Thu, Oct 25, 2018 at 11:17 AM Ryan O'Kuinghttons
> > <ryan.okuinghttons at noaa.gov> wrote:
> >> Hi Vijay,
> >>
> >> I've had some time to play with this now, and I have another question.
> >> Thank you for sending along the example code, it has been extremely
> >> helpful.
> >>
> >> The example shows the ghost exchange happening between shared owned
> >> entities. In my experiments I've been unable to make the ghost exchange
> >> work, and I think that might be because my entities are not shared. The
> >> situation I have is entities that are owned wholly on a single
> >> processor, which need to be communicated to other processors which
> >> require them as part of a halo region for mesh based computation. In
> >> this situation would I need to "share" my entities across the whole
> >> processor space, before requesting a ghost exchange? I'm not even really
> >> sure what shared entities mean, is there a good place to look in the
> >> documentation to learn more about the terminology? Thanks,
> >>
> >> Ryan
> >>
> >> On 10/8/18 12:17, Vijay S. Mahadevan wrote:
> >>> Ryan,
> >>>
> >>> You need to use the ParallelComm object in MOAB to call
> >>> exchange_ghost_cells [1] with appropriate levels of ghost layers for
> >>> your mesh. This needs to be done after the mesh is loaded, or you can
> >>> pass this information also as part of the options when loading a file
> >>> and MOAB will internally load the file with ghosted layers. Here's an
> >>> example [2].
> >>>
> >>> Vijay
> >>>
> >>> [1] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a55dfa308f56fd368319bfb4244428878
> >>> [2] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/HelloParMOAB_8cpp-example.html
> >>> On Mon, Oct 8, 2018 at 1:23 PM Ryan O'Kuinghttons
> >>> <ryan.okuinghttons at noaa.gov> wrote:
> >>>> Hi, I am wondering if there is a way to create ghost elements in MOAB.
> >>>> By this I mean a list of copies of elements surrounding a specific MOAB
> >>>> element, ghost elements may exist on a different processor than the
> >>>> source element. I see a ghost_elems class in the appData namespace, but
> >>>> there is not much documentation on how to use it.. Thank you,
> >>>>
> >>>> --
> >>>> Ryan O'Kuinghttons
> >>>> Cherokee Nation Management and Consulting
> >>>> NESII/NOAA/Earth System Research Laboratory
> >>>> ryan.okuinghttons at noaa.gov
> >>>> https://www.esrl.noaa.gov/gsd/nesii/
> >>>>