[petsc-users] DMPlexCreateFromDAG in parallel

[Matthew Knepley]

On Tue, Oct 1, 2019 at 10:08 AM Asitav Mishra <asitav at gmail.com> wrote:

> Matt,
>
> Thank you very much for your prompt response. Between the two solutions
> suggested by you, solution (1) would be tough since it would be difficult
> to ensure non-repeating vertices between processors.
> Would you be able to provide an example for the solution (2), that is
> using CreateFromDAG() on multiple processes.
> For example, how can we set up PetscSF for the simple case of two
> triangles (cells: 0 and 1) sharing a face (with vertices: 1, 2) with cell 0
> in proc 0 and cell 1 in proc 1?
>
>         2
>       / | \
>     /   |   \
>    /    |    \
>  0  0 | 1  3
>    \    |    /
>      \  |   /
>       \ | /
>         1
>

Okay, on 2 processes, you would have something like:

Process 0      Process 1
     3      <->          1
    / |                     | \
   /  |                     |  \
 /    |                     |   \
1 0 |                     | 0  3
 \    |                     |    /
  \   |                     |  /
   \  |                     | /
     2     <->          2

where we number all points sequentially. We will refer to points as (p, r)
where p is the local point number and r is the rank.
So you need to match up

    (3, 0) <-> (1, 1)
    (2, 0) <-> (2, 1)

Lets say that Process 1 owns both points. Then making the SF for Process 0
is

  numRoots        = 4
  numLeaves      = 2
  local indices     = {2, 3}
  remote indices = {{2, 1}, {1, 1}}

The SF for Process 1 is

  numRoots   = 4
  numLeaves = 0

  Thanks,

    Matt

Any help would be great. Thanks again.
> Asitav
>
> On Sat, Sep 28, 2019 at 1:20 AM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Wed, Sep 25, 2019 at 5:56 PM Asitav Mishra via petsc-users <
>> petsc-users at mcs.anl.gov> wrote:
>>
>>> Hi,
>>>
>>> I have a native distributed mesh graph across multiple processors, using
>>> which I would want to create DMPlex mesh using DMPlexCreateFromDAG. I see
>>> in Petsc plex/examples that DMPlexCreateFromDAG creates DM only from master
>>> processor and then the DM is distributed across multiple (one-to-many)
>>> processors. My question is: is it possible to create DAG locally in each
>>> processor and then build the global DM? If yes, are there any such examples?
>>>
>>
>> 1) If you do not mind us redistributing the mesh on input, then you can
>> probably do what you want using
>>
>>
>> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/DMPlexCreateFromCellListParallel.html
>>
>> Note that the input to this function wants a unique set of vertices from
>> each process, so each vertex must come from only one processes.
>>
>>  2) If that does not work, you can certainly call CreateFromDAG() on
>> multiple processes. However, then you must manually create the PetscSF
>>      which describes how the mesh is connected in parallel. If this is
>> what you need to do, I can give you instructions but at that point we should
>>      probably make an example that does it.
>>
>>   Thanks,
>>
>>     Matt
>>
>>
>>> Best,
>>> Asitav
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>
>
> --
> Asitav Mishra, PhD
> Research Engineer II, NIA
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20191001/827d2faf/attachment.html>