[MOAB-dev] ghost/halo elements
Ryan O'Kuinghttons
ryan.okuinghttons at noaa.gov
Thu Oct 25 14:41:42 CDT 2018
OK, thanks again for that explanation, I never would have been able to
figure out that this_set was the root set. I apologize for all the dumb
questions, hopefully they are all getting me closer to self sufficiency..
I'm getting an error that doesn't make sense to me. I specifying the
resolve_dim and shared_dim in the call:
// Get the ParallelComm instance
ParallelComm* pcomm = new ParallelComm(mb, mpi_comm);
int nprocs = pcomm->proc_config().proc_size();
int rank = pcomm->proc_config().proc_rank();
// get the root set handle
EntityHandle root_set = mb->get_root_set();
merr = pcomm->resolve_shared_ents(root_set, 2, -1);
but I'm getting this message:
[0]MOAB ERROR: Unable to guess shared_dim or resolve_dim!
what am i missing?
On 10/25/18 12:51, Vijay S. Mahadevan wrote:
> It is your root set or fileset if you added entities to it. You can
> get the root set handle from Core object.
>
> Vijay
>
> On Thu, Oct 25, 2018, 14:38 Ryan O'Kuinghttons
> <ryan.okuinghttons at noaa.gov <mailto:ryan.okuinghttons at noaa.gov>> wrote:
>
> Thanks again, Vijay. However, I still don't understand what this_set
> should be, is it an output maybe?
>
> Ryan
>
> On 10/25/18 12:27, Vijay S. Mahadevan wrote:
> > You can try either the first or the second variant instead [1] with
> > following arguments.
> >
> > ErrorCode moab::ParallelComm::resolve_shared_ents(EntityHandle
> this_set,
> > int resolve_dim = 2,
> > int shared_dim = -1,
> > const Tag * id_tag = 0
> > )
> >
> > That should resolve 2-dim entities with shared edges across
> > partitions. You can leave id_tag pointer as zero since the
> default is
> > GLOBAL_ID.
> >
> >> This brings up a more general question I've had about the moab
> > documentation for a while. In the doc for this routine, it only
> lists 2
> > parameters, proc_ents and shared_dim, even though in the function
> > signature above it clearly shows more. I've had trouble
> understanding
> > which parameters are relevant in the past, or what they do
> because I'm
> > not quite sure how to read the documentation.
> >
> > This is an oversight. We will go through and rectify some of the
> > inconsistencies in the documentation. We are preparing for an
> upcoming
> > release and I'll make sure that routines in Core/ParallelComm have
> > updated documentation that match the interfaces. Meanwhile, if you
> > have questions, feel free to shoot an email to the list.
> >
> > Hope that helps.
> >
> > Vijay
> >
> > [1]
> http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a29a3b834b3fc3b4ddb3a5d8a78a37c8a
> > On Thu, Oct 25, 2018 at 1:58 PM Ryan O'Kuinghttons
> > <ryan.okuinghttons at noaa.gov <mailto:ryan.okuinghttons at noaa.gov>>
> wrote:
> >> Hi Vijay,
> >>
> >> Thanks again for that explanation.
> >>
> >> ESMF does use unique global ids for the vertices and set them
> to the
> >> GLOBAL_ID in the moab mesh. So I think we are good there.
> >>
> >> I can't quite figure out how to use resolve_shared_entities though.
> >> There are three versions of the call in the documentation, and
> I assume
> >> that I should use the first and pass in a Range containing all
> entities
> >> for proc_ents:
> >>
> >>
> http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a59e35d9906f2e33fe010138a144a5cb6
> >>
> >> However, I'm not sure what the this_set EntityHandle should be.
> >>
> >> This brings up a more general question I've had about the moab
> >> documentation for a while. In the doc for this routine, it only
> lists 2
> >> parameters, proc_ents and shared_dim, even though in the function
> >> signature above it clearly shows more. I've had trouble
> understanding
> >> which parameters are relevant in the past, or what they do
> because I'm
> >> not quite sure how to read the documentation.
> >>
> >> Any example or explanation you give me for
> resolve_shared_entities is
> >> much appreciated!
> >>
> >> Ryan
> >>
> >> On 10/25/18 09:40, Vijay S. Mahadevan wrote:
> >>> Hi Ryan,
> >>>
> >>> Glad the example helped.
> >>>
> >>>> The example shows the ghost exchange happening between shared
> owned
> >>> entities. In my experiments I've been unable to make the ghost
> exchange
> >>> work, and I think that might be because my entities are not
> shared.
> >>>
> >>> You need to call resolve_shared_entities on the entities prior to
> >>> doing ghost exchange. When we load the mesh from file, this
> happens
> >>> automatically based on the read options but if you are forming
> a mesh
> >>> in memory, you need to make sure that the shared vertices have the
> >>> same GLOBAL_ID numbering consistently across processes. That
> is shared
> >>> vertices are unique. Once that is set, shared entities
> resolution will
> >>> work correctly out of the box and you will have shared
> edges/entities
> >>> query working correctly. A call to get ghosted layers once this is
> >>> done would be the way to go.
> >>>
> >>> I assume ESMF has a unique global numbering for the vertices ? Use
> >>> that to set the GLOBAL_ID tag. Let us know if you are still having
> >>> issues.
> >>>
> >>> Vijay
> >>> On Thu, Oct 25, 2018 at 11:17 AM Ryan O'Kuinghttons
> >>> <ryan.okuinghttons at noaa.gov
> <mailto:ryan.okuinghttons at noaa.gov>> wrote:
> >>>> Hi Vijay,
> >>>>
> >>>> I've had some time to play with this now, and I have another
> question.
> >>>> Thank you for sending along the example code, it has been
> extremely
> >>>> helpful.
> >>>>
> >>>> The example shows the ghost exchange happening between shared
> owned
> >>>> entities. In my experiments I've been unable to make the
> ghost exchange
> >>>> work, and I think that might be because my entities are not
> shared. The
> >>>> situation I have is entities that are owned wholly on a single
> >>>> processor, which need to be communicated to other processors
> which
> >>>> require them as part of a halo region for mesh based
> computation. In
> >>>> this situation would I need to "share" my entities across the
> whole
> >>>> processor space, before requesting a ghost exchange? I'm not
> even really
> >>>> sure what shared entities mean, is there a good place to look
> in the
> >>>> documentation to learn more about the terminology? Thanks,
> >>>>
> >>>> Ryan
> >>>>
> >>>> On 10/8/18 12:17, Vijay S. Mahadevan wrote:
> >>>>> Ryan,
> >>>>>
> >>>>> You need to use the ParallelComm object in MOAB to call
> >>>>> exchange_ghost_cells [1] with appropriate levels of ghost
> layers for
> >>>>> your mesh. This needs to be done after the mesh is loaded,
> or you can
> >>>>> pass this information also as part of the options when
> loading a file
> >>>>> and MOAB will internally load the file with ghosted layers.
> Here's an
> >>>>> example [2].
> >>>>>
> >>>>> Vijay
> >>>>>
> >>>>> [1]
> http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/classmoab_1_1ParallelComm.html#a55dfa308f56fd368319bfb4244428878
> >>>>> [2]
> http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/HelloParMOAB_8cpp-example.html
> >>>>> On Mon, Oct 8, 2018 at 1:23 PM Ryan O'Kuinghttons
> >>>>> <ryan.okuinghttons at noaa.gov
> <mailto:ryan.okuinghttons at noaa.gov>> wrote:
> >>>>>> Hi, I am wondering if there is a way to create ghost
> elements in MOAB.
> >>>>>> By this I mean a list of copies of elements surrounding a
> specific MOAB
> >>>>>> element, ghost elements may exist on a different processor
> than the
> >>>>>> source element. I see a ghost_elems class in the appData
> namespace, but
> >>>>>> there is not much documentation on how to use it.. Thank you,
> >>>>>>
> >>>>>> --
> >>>>>> Ryan O'Kuinghttons
> >>>>>> Cherokee Nation Management and Consulting
> >>>>>> NESII/NOAA/Earth System Research Laboratory
> >>>>>> ryan.okuinghttons at noaa.gov <mailto:ryan.okuinghttons at noaa.gov>
> >>>>>> https://www.esrl.noaa.gov/gsd/nesii/
> >>>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/moab-dev/attachments/20181025/9003e0b6/attachment.html>
More information about the moab-dev
mailing list