<div dir="ltr"><div dir="ltr">On Tue, Oct 1, 2019 at 12:25 PM Asitav Mishra <<a href="mailto:asitav@gmail.com">asitav@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Matt,<div><br></div><div>Thanks for the example, it make it very clear. I understand this example assumes only one process (here Process 1) 'owns' the shared vertices on the shared face. Specifically, for Process 1 can I also have the SF:</div><div>numRoots = 4</div><div>numLeaves = 2</div><div>local indices = {1, 2}</div><div>remote indices = {{3,0}, {2,0}} </div><div>?</div></div></blockquote><div><br></div><div>You could choose to have Process 0 own the vertices instead of Process 1, but you cannot specify the info on both Processes since SF</div><div>takes one-sided information.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Also, I'm still not sure how DMPlexCreateFromDAG uses PetscSF. For ex, is the DM creation after SF creation for each process or before? Are there any examples available in Petsc distributions?</div></div></blockquote><div><br></div><div>No, we don't have any examples of creating things in parallel by hand. The SF would be built after DMPlexCreateFromDAG()</div><div>and you set it using DMPlexSetPointSF().</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thanks,</div><div>Asitav</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 1, 2019 at 11:54 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Tue, Oct 1, 2019 at 10:08 AM Asitav Mishra <<a href="mailto:asitav@gmail.com" target="_blank">asitav@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Matt,<div><br></div><div>Thank you very much for your prompt response. Between the two solutions suggested by you, solution (1) would be tough since it would be difficult to ensure non-repeating vertices between processors. </div><div>Would you be able to provide an example for the solution (2), that is using CreateFromDAG() on multiple processes.</div><div>For example, how can we set up PetscSF for the simple case of two triangles (cells: 0 and 1) sharing a face (with vertices: 1, 2) with cell 0 in proc 0 and cell 1 in proc 1?</div><div><br></div><div> 2<br> / | \<br> / | \<br> / | \<br> 0 0 | 1 3<br> \ | /<br> \ | /<br> \ | /<br> 1</div></div></blockquote><div><br></div><div>Okay, on 2 processes, you would have something like:</div><div><br></div><div>Process 0 Process 1</div><div> 3 <-> 1</div><div> / | | \</div><div> / | | \</div><div> / | | \</div><div>1 0 | | 0 3</div><div> \ | | /</div><div> \ | | /</div><div> \ | | /</div><div> 2 <-> 2</div><div> </div><div>where we number all points sequentially. We will refer to points as (p, r) where p is the local point number and r is the rank.</div><div>So you need to match up</div><div><br></div><div> (3, 0) <-> (1, 1)</div><div> (2, 0) <-> (2, 1)</div><div><br></div><div>Lets say that Process 1 owns both points. Then making the SF for Process 0 is</div><div><br></div><div> numRoots = 4</div><div> numLeaves = 2</div><div> local indices = {2, 3}</div><div> remote indices = {{2, 1}, {1, 1}}</div><div><br></div><div>The SF for Process 1 is</div><div><br></div><div> numRoots = 4</div><div> numLeaves = 0</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Any help would be great. Thanks again.</div><div>Asitav</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Sep 28, 2019 at 1:20 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Sep 25, 2019 at 5:56 PM Asitav Mishra via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I have a native distributed mesh graph across multiple processors, using which I would want to create DMPlex mesh using DMPlexCreateFromDAG. I see in Petsc plex/examples that DMPlexCreateFromDAG creates DM only from master processor and then the DM is distributed across multiple (one-to-many) processors. My question is: is it possible to create DAG locally in each processor and then build the global DM? If yes, are there any such examples?</div></div></blockquote><div><br></div><div>1) If you do not mind us redistributing the mesh on input, then you can probably do what you want using</div><div><br></div><div> <a href="https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/DMPlexCreateFromCellListParallel.html" target="_blank">https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/DMPLEX/DMPlexCreateFromCellListParallel.html</a></div><div><br></div><div>Note that the input to this function wants a unique set of vertices from each process, so each vertex must come from only one processes.</div><div><br></div><div> 2) If that does not work, you can certainly call CreateFromDAG() on multiple processes. However, then you must manually create the PetscSF</div><div> which describes how the mesh is connected in parallel. If this is what you need to do, I can give you instructions but at that point we should</div><div> probably make an example that does it.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Best,</div><div>Asitav</div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Asitav Mishra, PhD <div><div>Research Engineer II, NIA</div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Asitav Mishra, PhD <div><div>Research Engineer II, NIA</div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>