<div dir="ltr"><div><div><div>Matt, Okay that makes a lot of sense now, thank you so much. Sorry I have one more question then I will leave you all alone :)<br><br></div>From what I understand about creating DMPlex, it seems that if there are multiple processes, only the root rank populates the DM structure. I see that DMPlexCreateBoxMesh as well as the GMSH/Exodus/CGNS readers have rank 0 read the mesh files and populate the sieve/mesh points within the DMPlex (assuming that the DM was created with PETSC_COMM_WORLD).<br><br></div>However, when I look at the DMPlexCreateFromDAG and DMPlexCreateFromCellList functions, I don't see anywhere where only rank 0 handles the creation of the DM structure. I do notice that DMPlexCreateFromDAG already requires the DM structure to be created. So I guess my question is, is it possible to simply have rank 0 invoke the DMPlexCreateFromDAG function? Because that way only the root process reads the mesh data file and creates/distributes the DMPlex among the remaining processors.<br><br></div>Thanks,<br>Justin<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Nov 1, 2014 at 1:30 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Sat, Nov 1, 2014 at 12:20 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="auto"><div>Thanks for the response. On a related note,</div><div><br></div><div>In the source code of SNES ex12, where is the PetscSection actually being created? I can't seem to find anywhere in the code or its routines where the creation has been explicitly called. Because when the discretization/problem is define, I imagine a PetscSection for the fields and constraints has to be invoked somewhere. It seems to me as soon as DMPlexAddBoundary() is called it creates the PetscSectionConstraints but I can't seem to really confirm this within the source of that function. The reason I want to know this is because if I really were to declare my own constraints, I want to make sure that the PetscSection for the field variables has already been set up.</div></div></blockquote><div><br></div></span><div>Much of the mechanism has been moved into the library. The reason I did this was to make things like</div><div>nonlinear multigrid automatic, rather than making the user define a different section on each level. Let</div><div>me explain what is happening. When we call DMGetDefaultSection(), if the section is not present it</div><div>will call the createdefaultsection() member function:</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/interface/dm.c?at=master#cl-2977" target="_blank">https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/interface/dm.c?at=master#cl-2977</a></div><div><br></div><div>which for Plex is here</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-5601" target="_blank">https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-5601</a></div><div><br></div><div>It uses the discretization information from PetscDS and the BC information from DMPlexAddBoundary() to build the inputs for</div><div>DMPlexCreateSection()</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-5739" target="_blank">https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-5739</a></div><div><br></div><div>I realize that call is still limited. For example, it constrains all dofs on a point, but sometimes you only want to constrain some. In</div><div>the most general case, you would just build the Section manually, using the same tools I use in DMPlexCreateSection():</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-3204" target="_blank">https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plex.c?at=master#cl-3204</a></div><div><div class="h5"><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="auto"><div>Thanks,</div><div>Justin</div><div><br></div><div><br>Sent from my iPhone</div><div><br>On Nov 1, 2014, at 11:40 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Oct 31, 2014 at 4:15 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="auto"><div><span></span></div><div><div>Matt,</div><div><br></div><div>Thanks for the response. One more question:</div><div><br></div><div>If I go with approach 2, will manually setting the constraints/indices and its values be compatible with the DMPlexSNESComputeResidual/JacobianFEM routines? When I look at the source code of those routines it seems the constrained values are added to the local solution vectors via DMPlexInsertBoundaryValuesFEM. If I choose not to use DMPlexAddBoundary and wish to manually declare my boundary values, i assume I should call DMPlexProjectField with mode INSERT_BC_VALUES?</div></div></div></blockquote><div><br></div><div>Yes, exactly. I use DMPlexProjectFunctionLabelLocal(),</div><div><br></div><div><a href="https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plexfem.c?at=master#cl-576" target="_blank">https://bitbucket.org/petsc/petsc/src/ea1a9a653238fa98fb68e87b49145608ab6e5301/src/dm/impls/plex/plexfem.c?at=master#cl-576</a><br></div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="auto"><div><div>Thanks,</div><div>Justin</div><div><br>Sent from my iPhone<div><br>On Oct 30, 2014, at 4:24 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 30, 2014 at 4:16 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div>Matt, thanks for the quick response.<br><br></div>What about for (dirichlet) boundary conditions? Would it be possible to do something similar for those, like using those PetscSectionSetConstraint functions? <br></div></div></blockquote><div><br></div><div>Yes. There are generally two ways of handling Dirichlet conditions:</div><div><br></div><div> 1) Replace those rows of the Jacobian with the identity and put the boundary value in the rhs. I find</div><div> this cumbersome for nonlinear solves.</div><div><br></div><div> 2) Remove these unknowns from the system using PetscSectionSetConstraintDof() and ConstratinIndices().</div><div><br></div><div>The DMPlexAddBoundary() functions are automating this processing by marking boundaries using DMLabels,</div><div>and then constraining dofs on those boundaries.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div></div>Thanks,<br>Justin<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 30, 2014 at 3:35 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div>On Thu, Oct 30, 2014 at 3:29 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi all,<br><br></div>So I am writing an FEM code where it reads input data (e.g., auxiliary coefficients, source terms, etc) from a text file. I have preprocessed the data text file so that each vertex point has its corresponding data. For instance, if my source term for a diffusion problem has a sin or cos function of the coordinates, then this data is already tabulated and simply needs to be fed into my PETSc program. The data text file containing both the auxiliary data and the coordinate/connectivities will also be used to provide the input arguments for the DMPlexCreateFromDAG() function. <br><br></div>Normally, I would hard-code the sin/cos functions into the source code itself, but i want my program to be "universal" in that it can take as input any user-defined function for not only these auxiliaries but for other things like boundary conditions. I see that PETSc has a lot of examples on how to read data into vectors and matrices, but I guess my question is is there a way to project data from a text file into a vector that has already been created with a defined DM structure? <br></div></div></blockquote><div><br></div></div></div><div>If you have the functions available, I think it is far more universal to use the function itself, since then you can be independent</div><div>of mesh and discretization when specifying input and BC.</div><div><br></div><div>However, if you want to read it in, and you guarantee that it matches the mesh and discretization, I think the easiest thing to do is</div><div>demand that it come in the same order as the vertices and use</div><div><br></div><div>VecGetArray(V, &a);</div><div>for (v = vStart, i = 0; v < vEnd; ++v) {</div><div> PetscSectionGetDof(s, v, &dof);</div><div> PetscSectionGetOffset(s, v, &off);</div><div> for (d = 0; d < dof; ++d) a[off+d] = text_data[i++];</div><div>}</div><div>VecRestoreArray(V, &a);</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div></div><div>Thanks,<br>Justin<span><font color="#888888"><br></font></span></div></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><span><font color="#888888"><span><font color="#888888"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</font></span></font></span></font></span></div></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br></font></span></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</font></span></font></span></div></div><span><font color="#888888">
</font></span></div></blockquote></div></div></div></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</font></span></div></div>
</div></blockquote></div></blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</div></div></div></div>
</blockquote></div><br></div>