[MOAB-dev] Parallel MOAB usage

[Vijay S. Mahadevan]

> I am performing calculations per mesh element that requires
> information about its immediate neighbours.

This is exactly where the ghosted layers help. What you are trying to
do is very typical in many FV and FE calculations and when you load
the mesh, you can impose a layer of ghosting so that you have access
to shared elements on the partition boundary from other processes.
Take a look at the HelloParMOAB example [1] to familiarize yourself
with the workflow to create ghosted layers.

> information about its immediate neighbours. As a first approach, I am
> trying to send the elements at the interface of a mesh partition to its
> respective neighbour so that it has all elements necessary before it starts
> the calculations. At this moment, I am achieving it by calling MPI_Send
> and MPI_Recv directly.

Unless I understood your use case wrongly, I do not recommend using
the advanced packing + send/receive of entities since what you need
can be handled by MOAB directly. Let us know if you have any other
questions.

Vijay

[1] http://ftp.mcs.anl.gov/pub/fathom/moab-docs-develop/HelloParMOAB_8cpp-example.html

On Fri, Oct 26, 2018 at 8:59 AM Filipe Antonio Cumaru Silva Alves
<facsa at cin.ufpe.br> wrote:
>
> Hi Vijay,
>
> > Unless the elements that you are trying to access are ghosted
> a-priori, there is no way to access entities on a different partition
> directly. Remember that the strength of MOAB comes from the fact that
> we maintain a fully parallel view of the mesh. We do not replicate the
> mesh on every process. None of the adjacency calls will work unless
> the element is available in the local partition (either as a owned or
> ghosted element).
>
> Thank you for the clarification.
>
> > Yes these methods in ParallelComm can help you exchange entities but
> these are advanced methods and yes, we do utilize ISend/IRecv methods
> underneath to exchange the buffers. The API documentation is available
> here [1].
>
> I took a look at the API documentation and the source code and it helped,
> although I still have some questions. What are entprocs and recv_remoteh_reqs
> used for? Does incoming1 and incoming2 must have the same value?
>
> > If you can explain your use case, perhaps we can suggest a
> better way to approach it.
>
> I am performing calculations per mesh element that requires
> information about its immediate neighbours. As a first approach, I am
> trying to send the elements at the interface of a mesh partition to its
> respective neighbour so that it has all elements necessary before it starts
> the calculations. At this moment, I am achieving it by calling MPI_Send
> and MPI_Recv directly.
>
> Em qui, 18 de out de 2018 às 18:36, Vijay S. Mahadevan <vijay.m at gmail.com> escreveu:
>>
>> Hi Filipe,
>>
>> > 1. Is there any method to access an element in a mesh partition from another partition? E.g., get the elements surrounding another element which could possibly belong to another processor. I've tried to use the  get_bridge_adjacencies method from moab::MeshTopoUtil but it does not seem to work this way.
>>
>> Unless the elements that you are trying to access are ghosted
>> a-priori, there is no way to access entities on a different partition
>> directly. Remember that the strength of MOAB comes from the fact that
>> we maintain a fully parallel view of the mesh. We do not replicate the
>> mesh on every process. None of the adjacency calls will work unless
>> the element is available in the local partition (either as a owned or
>> ghosted element).
>>
>> > 2. How the send_entities and recv_entities methods from ParallelComm class work? Is it similar to MPI_Send and MPI_Receive? I would like to know more about those methods's arguments.
>>
>> Yes these methods in ParallelComm can help you exchange entities but
>> these are advanced methods and yes, we do utilize ISend/IRecv methods
>> underneath to exchange the buffers. The API documentation is available
>> here [1]. If you can explain your use case, perhaps we can suggest a
>> better way to approach it.
>>
>> Vijay
>>
>> [1] http://ftp.mcs.anl.gov/pub/fathom/moab-docs/classmoab_1_1ParallelComm.html#aa921ffb8f68a405ff88aba80f7597a2a
>>
>> On Wed, Oct 17, 2018 at 3:31 PM Filipe Antonio Cumaru Silva Alves
>> <facsa at cin.ufpe.br> wrote:
>> >
>> > Hello,
>> > I am using MOAB to accomplish some tasks in parallel computing of meshes and I've got some doubts about it.
>> >
>> > 1. Is there any method to access an element in a mesh partition from another partition? E.g., get the elements surrounding another element which could possibly belong to another processor. I've tried to use the  get_bridge_adjacencies method from moab::MeshTopoUtil but it does not seem to work this way.
>> >
>> > 2. How the send_entities and recv_entities methods from ParallelComm class work? Is it similar to MPI_Send and MPI_Receive? I would like to know more about those methods's arguments.
>> >
>> > Thank you.
>> >
>> > --
>> > Atenciosamente,
>> > Filipe Antônio Cumaru Silva Alves
>> >
>> > Graduando em Engenharia da Computação (2016.1)
>> > Monitor de Interface Hardware-Software (IF817)
>> > Bolsista de iniciação científica CNPq no PADMEC (Processamento de Alto Desempenho na Mecânica Computacional)
>
>
>
> --
> Atenciosamente,
> Filipe Antônio Cumaru Silva Alves
>
> Graduando em Engenharia da Computação (2016.1)
> Monitor de Interface Hardware-Software (IF817)
> Bolsista de iniciação científica CNPq no PADMEC (Processamento de Alto Desempenho na Mecânica Computacional)