[petsc-users] DMPlexCreateFromDAG in parallel

[Asitav Mishra]

Matt,

Thanks for the example, it make it very clear. I understand this example
assumes only one process (here Process 1) 'owns' the shared vertices  on
the shared face. Specifically, for Process 1 can I also have the SF:
numRoots    = 4
numLeaves  = 2
local indices = {1, 2}
remote indices = {{3,0}, {2,0}}
?

Also, I'm still not sure how DMPlexCreateFromDAG uses PetscSF. For ex, is
the DM creation after SF creation for each process or before? Are there any
examples available in Petsc distributions?

Thanks,
Asitav



On Tue, Oct 1, 2019 at 11:54 AM Matthew Knepley <knepley at gmail.com> wrote:

> On Tue, Oct 1, 2019 at 10:08 AM Asitav Mishra <asitav at gmail.com> wrote:
>
>> Matt,
>>
>> Thank you very much for your prompt response. Between the two solutions
>> suggested by you, solution (1) would be tough since it would be difficult
>> to ensure non-repeating vertices between processors.
>> Would you be able to provide an example for the solution (2), that is
>> using CreateFromDAG() on multiple processes.
>> For example, how can we set up PetscSF for the simple case of two
>> triangles (cells: 0 and 1) sharing a face (with vertices: 1, 2) with cell 0
>> in proc 0 and cell 1 in proc 1?
>>
>>         2
>>       / | \
>>     /   |   \
>>    /    |    \
>>  0  0 | 1  3
>>    \    |    /
>>      \  |   /
>>       \ | /
>>         1
>>
>
> Okay, on 2 processes, you would have something like:
>
> Process 0      Process 1
>      3      <->          1
>     / |                     | \
>    /  |                     |  \
>  /    |                     |   \
> 1 0 |                     | 0  3
>  \    |                     |    /
>   \   |                     |  /
>    \  |                     | /
>      2     <->          2
>
> where we number all points sequentially. We will refer to points as (p, r)
> where p is the local point number and r is the rank.
> So you need to match up
>
>     (3, 0) <-> (1, 1)
>     (2, 0) <-> (2, 1)
>
> Lets say that Process 1 owns both points. Then making the SF for Process 0
> is
>
>   numRoots        = 4
>   numLeaves      = 2
>   local indices     = {2, 3}
>   remote indices = {{2, 1}, {1, 1}}
>
> The SF for Process 1 is
>
>   numRoots   = 4
>   numLeaves = 0
>
>   Thanks,
>
>     Matt
>
> Any help would be great. Thanks again.
>> Asitav
>>
>> On Sat, Sep 28, 2019 at 1:20 AM Matthew Knepley <knepley at gmail.com>
>> wrote:
>>
>>> On Wed, Sep 25, 2019 at 5:56 PM Asitav Mishra via petsc-users <
>>> petsc-users at mcs.anl.gov> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have a native distributed mesh graph across multiple processors,
>>>> using which I would want to create DMPlex mesh using DMPlexCreateFromDAG. I
>>>> see in Petsc plex/examples that DMPlexCreateFromDAG creates DM only from
>>>> master processor and then the DM is distributed across multiple
>>>> (one-to-many) processors. My question is: is it possible to create DAG
>>>> locally in each processor and then build the global DM? If yes, are there
>>>> any such examples?
>>>>
>>>
>>> 1) If you do not mind us redistributing the mesh on input, then you can
>>> probably do what you want using
>>>
>>>
>>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/DMPlexCreateFromCellListParallel.html
>>>
>>> Note that the input to this function wants a unique set of vertices from
>>> each process, so each vertex must come from only one processes.
>>>
>>>  2) If that does not work, you can certainly call CreateFromDAG() on
>>> multiple processes. However, then you must manually create the PetscSF
>>>      which describes how the mesh is connected in parallel. If this is
>>> what you need to do, I can give you instructions but at that point we should
>>>      probably make an example that does it.
>>>
>>>   Thanks,
>>>
>>>     Matt
>>>
>>>
>>>> Best,
>>>> Asitav
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>>> <http://www.cse.buffalo.edu/~knepley/>
>>>
>>
>>
>> --
>> Asitav Mishra, PhD
>> Research Engineer II, NIA
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>


-- 
Asitav Mishra, PhD
Research Engineer II, NIA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20191001/ffefc0be/attachment-0001.html>