<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Dec 22, 2015 at 7:17 AM, Alejandro D Otero <span dir="ltr"><<a href="mailto:aotero@fi.uba.ar" target="_blank">aotero@fi.uba.ar</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Thanks for your answer.<br></div>In principle, this is what I want, although I know that this doesn't scale. At this point I am reading the mesh in only one proc so I have all the properties there, that is why after that I need to distribute the mesh along with the data.<br></div>Is there a better solution within this scheme (only one proc reads mesh data)?<br></div></div></div></blockquote><div><br></div><div>Most physical problems that I deal with decouple properties from the mesh. This seems reasonable since the mesh is</div><div>just a crutch to solve the continuum problem. So, for example, in PyLith (<a href="https://geodynamics.org/cig/software/pylith/">https://geodynamics.org/cig/software/pylith/</a>),</div><div>we read in/generate a mesh in serial, manipulate it to add faults etc., mark the volumes/boundaries, distribute it, and</div><div>then in parallel for each local marked section we queries a spatial database to get the properties and boundary data.</div><div><br></div><div>By summer, we will have parallel read for the initial mesh.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Regards,<br><br></div>Alejandro<br></div><div class=""><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 22, 2015 at 1:11 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Thu, Dec 17, 2015 at 1:27 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">So you would use something like <a href="http://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/DM/DMPlexDistributeField.html" target="_blank">DMPlexDistributeField()</a> in that case. You have your original/global DM, PetscSection, and Vec of values. Then using the PetscSF that was created during DMPlexDistribute, you map the global stuff into the local stuff.</div></blockquote><div><br></div></span><div>Justin is correct. However, I will note that this strategy is not scalable. Usually the data is much larger than the mesh,</div><div>so this runs out of memory much sooner. Are you sure that this is what you want?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div class="gmail_extra"><div class="gmail_quote">On Thu, Dec 17, 2015 at 11:21 AM, Alejandro D Otero <span dir="ltr"><<a href="mailto:aotero@fi.uba.ar" target="_blank">aotero@fi.uba.ar</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Thanks, <br>The problem then is that after
DMPlexDistribute the DMPlex 'points' are renumbered. So if the values
are related to each point in the original numbering how do I set the
values after the distribution. I know the property stored in the vector
related to the entities with the numbering of the original mesh which I
use to create the first DMPlex.<br><br></div>Ideally for me, I would
like to set the values in the vector before DMPlexDistribute and get the
vector components renumbered and redistributed accordingly in a global
vector. And then, get the local vector. <br><br></div>Hope it could be more clear now.<br></div>Regards,<br><br></div>Alejandro<br><br><br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 16, 2015 at 7:01 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">I think you would follow this order:<div><br></div><div>1*) create a DMPlex (depth, chart, etc) on rank 0. Other ranks have an empty DM</div><div><br></div><div>2) DMPlexDistribute()</div><div><br></div><div>3*) Create the PetscSection</div><div><br></div><div>4) DMCreateGlobalVector() </div><div><br></div><div>5) DMCreateLocalVector()</div><div><br></div><div>Now you have a global vector and a local vector for your distributed DMPlex. The mapping/ghosting/etc of dofs is already taken care of.</div><div><br></div><div>* if you're using standard Galerkin FE then in SNES examples 12 and 62 (and maybe others?) the first step is handled through the mesh generation functions and step 3 is handled through step 4</div><div><br></div><div>Thanks,</div><div>Justin</div><div><div><div><br>On Wednesday, December 16, 2015, Alejandro D Otero <<a href="mailto:aotero@fi.uba.ar" target="_blank">aotero@fi.uba.ar</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi, I need some help understanding how to distribute data together with a dmplex representing a FE mesh. <div>At the beginning I define the structure of the dmplex assigning certain number of DoF to cells, edges and vertexes, in one process (the dmplex in the rest is empty) </div><div>I create a petscsecton and I create an associated global vector with the quantities I want to store.</div><div>Then I distribute the dmplex over all the processes.</div><div>* Although this does not perform well it is just a starting point. I know it has to be improved. </div><div><br></div><div>I would like to have the global vector distributed accordingly so that each process has access to the corresponding local part with its DoF (possibly adding some ghost values corresponding to the shared DoF not taken care by it).</div><div><br></div><div>Is there any 'correct' way to do that in PETSc? </div><div><br></div><div>Thanks in advance,</div><div><br></div><div>Alejandro</div></div>
</blockquote></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div></div></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>