<div dir="ltr"><div><div><div><div><div>Sorry I am a little confused here. I understand that each refinement of a 3D simplex element yields 8 subcells, but give the expression <span>c_f \in [C c_c, C (c_c + 1) ) </span>what exactly are these and/or what do they mean:<br><br></div>c_f \in [... )<br></div>C<br></div>c_c<br></div><div>(c_c + 1)<br></div><div><br></div>Originally I was thinking the functions DMPlexGetTransitiveClosure(...) and/or DMPlexGetCoarseDM(...) need to be invoked somehow. Do they?<br><br></div>Again thanks for your help<br clear="all"><div><div><div><br>-- <br><div><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br><a href="tel:%28512%29%20963-3262" value="+15129633262" target="_blank">(512) 963-3262</a><br></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 10, 2015 at 11:38 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Fri, Apr 10, 2015 at 1:01 AM, Justin Chang <span dir="ltr"><<a href="mailto:jchang27@uh.edu" target="_blank">jchang27@uh.edu</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Matt,<br><br></div><span class="">I am not sure I follow what you're saying here. Currently I am working with 3D simplex elements so if the variable numSubcells is what you're referring to in CellRefinerGetAffineTransforms_Internal(), there's no example for REFINER_SIMPLEX_3D. What would it be in that case, 7?</span></div></blockquote><div><br></div><div>Its 8. There are 8 fine cells for every coarse cell.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>What I am looking for is a function and/or a set of algorithms where given a sieve point m from the original coarse mesh, I can obtain the corresponding set of (fine mesh) sieve points. I am assuming c_f \in [C c_c, C (c_c + 1) ) describes just that? <br></div></div></blockquote><div><br></div></span><div>Yes, C = 8.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Is there an existing DMPlex related function that implements this directly? Because I don't see how CellRefinerGetAffineTransforms_Internal() will help.<br></div></div></blockquote><div><br></div></span><div>I would just give you C. Yes, I would just write it. It should be 1 line.</div><div><div class="h5"><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>Thanks,<br></div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 9, 2015 at 8:20 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Thu, Apr 9, 2015 at 8:14 PM, Justin Chang <span dir="ltr"><<a href="mailto:jchang27@uh.edu" target="_blank">jchang27@uh.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div>Matt, thank you for the clarification. One more question somewhat on a related note.<br><br>I have auxiliary data corresponding to the permeability of a reservoir
(for instance, the SPE10 benchmark problem). The data is cell-wise and
read from a binary file. I want to refine both the DM and this data. Is there a way to obtain the mapping from the coarse mesh to the fine mesh? What I mean is, given a coarse cell, can I somehow get the transitive closure of the fine cells it created during the refinement process? This would allow me to copy the value of the coarse cell data onto it's children.<br></div></div></div></blockquote><div><br></div></span><div>If you are doing regular refinement, there is a simple formula:</div><div><br></div><div> c_f \in [ C c_c, C (c_c +1) )</div><div><br></div><div>where C is the number of cells created from each cell. You can get C from</div><div><br></div><div> CellRefinerGetAffineTransforms_Internal()</div><div><br></div><div>which I could promote to a real interface function.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div>Thanks,<br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 9, 2015 at 7:52 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Thu, Apr 9, 2015 at 7:21 PM, Justin Chang <span dir="ltr"><<a href="mailto:jchang27@uh.edu" target="_blank">jchang27@uh.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div>1) I did what you had suggested and it works very well thanks.<br></div></div></div></div></blockquote><div><br></div></span><div>Cool.</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div></div>2) In my case, calling uninterpolate does make a difference, just not necessarily in the solver step (I am using Tao's BLMVM). I attached two output files, one containing the timings and summaries with uninterpolating and one without uninterpolating. And the source code if you wanted to verify the "correctness" of my implementation. Namely, the biggest difference I see is in setting up the PetscSection. When uninterpolating the data this steps takes 8.64039 seconds but when I do not uninterpolate the data it takes 61.2977 seconds. Is this "significant" enough, or is it unimportant given the fact that this step occurs only once during any simulation?<br></div></div></div></blockquote><div><br></div></span><div>That is significant. Most of that time is in the operator preallocation routine. It was cleaner to first gather the point</div><div>adjacency information, and then push the dof information on top. If I aggressively merged the dof data, I could have</div><div>pruned a bunch of the work here and gotten much closer to the first time, at the expense of more complicated code.</div><div>I guess this is the right pattern if you know that you will only have unknowns on cells and vertices.</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div>3) Also, about DMPlexDistribute(...). Right now it still seems extremely slow, hence why I am distributing a coarse mesh and having each processor refine its local portion of the mesh. In my current implementation, I have rank 0 distribute the mesh via ParMETIS. Will there eventually be a methodology to do parallel I/O then redistribution for load balancing or does it already exist?<br></div></div></blockquote><div><br></div></span><div>Redistribution for load balancing already exists. The stumbling block right now is parallel mesh import. However,</div><div>the group at ICL (Michael Lange and Gerard Gorman) is working on this, and we expect to have it working by</div><div>the end of summer. Until then, I recommend exactly what you are doing.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div></div>Thanks,<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 7, 2015 at 8:37 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div>On Tue, Apr 7, 2015 at 7:18 PM, Justin Chang <span dir="ltr"><<a href="mailto:jchang27@uh.edu" target="_blank">jchang27@uh.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Hi all,<br><br></div>So I am interpolating/uninterpolating my DMPlex mesh. This is the mesh information for a cell-vertex mesh only:<br><br>DM Object: 1 MPI processes<br> type: plex<br>DM_0x84000000_0 in 3 dimensions:<br> 0-cells: 159<br> 3-cells: 592<br>Labels:<br> marker: 1 strata of sizes (100)<br> depth: 2 strata of sizes (159, 592)<br><br></div>I am interpolating the mesh with the following lines:<br><br> ierr = DMPlexInterpolate(*dm, &idm);CHKERRQ(ierr);<br> ierr = DMPlexCopyCoordinates(*dm, idm);CHKERRQ(ierr);<br> ierr = DMPlexCopyLabels(*dm, idm);CHKERRQ(ierr);<br> ierr = DMPlexGetLabel(idm, "marker", &label);CHKERRQ(ierr);<br> ierr = DMPlexMarkBoundaryFaces(idm,label);CHKERRQ(ierr);<br> ierr = DMDestroy(dm);CHKERRQ(ierr);<br> *dm = idm;<br><br></div>And the resulting mesh:<br><br>DM Object: 1 MPI processes<br> type: plex<br>DM_0x84000000_1 in 3 dimensions:<br> 0-cells: 159<br> 1-cells: 845<br> 2-cells: 1280<br> 3-cells: 592<br>Labels:<br> marker: 1 strata of sizes (292)<br> depth: 4 strata of sizes (159, 845, 1280, 592)<br><br></div>The reason I did this is so that incase I decide to refine the mesh with "-dm_refine <number>" the DM will be sure to include the "marker" points in the refinement process. Now, if I decide to uninterpolate the mesh with the following lines:<br><br> ierr = DMPlexUninterpolate(*dm, &idm);CHKERRQ(ierr);<br> ierr = DMPlexCopyCoordinates(*dm, idm);CHKERRQ(ierr);<br> ierr = DMPlexCopyLabels(*dm, idm);CHKERRQ(ierr);<br> ierr = DMDestroy(dm);CHKERRQ(ierr);<br> *dm = idm;<br><br></div>I end up with this mesh:<br><br>DM Object: 1 MPI processes<br> type: plex<br>DM_0x84000000_2 in 3 dimensions:<br> 0-cells: 159<br> 3-cells: 592<br>Labels:<br> marker: 1 strata of sizes (292)<br> depth: 2 strata of sizes (159, 592)<br><br><div><div><div>The problem is that although the cells and vertices have gone back to the original number, the "marker" label still includes face/edge points. This gives me an error once I invoke DMCreateGobalVector(...).<br><br>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: Argument out of range<br>[0]PETSC ERROR: Section point 755 should be in [0, 751)<br>[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Development GIT revision: v3.5.3-2528-gbee642f GIT Date: 2015-03-29 20:36:38 -0500<br>[0]PETSC ERROR: ./cube_with_hole on a arch-linux2-c-debug named pacotaco by justin Tue Apr 7 19:09:12 2015<br>[0]PETSC ERROR: Configure options --download-chaco --download-exodusii --download-fblaslapack --download-hdf5 --download-metis --download-mpich --download-mumps --download-netcdf --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cmake=cmake --with-cxx=g++ --with-debugging=1 --with-fc=gfortran --with-valgrind=1 PETSC_ARCH=arch-linux2-c-debug<br>[0]PETSC ERROR: #1 PetscSectionGetDof() line 499 in /home/justin/petsc-dev/src/vec/is/utils/vsectionis.c<br>[0]PETSC ERROR: #2 DMPlexGetConeSize() line 841 in /home/justin/petsc-dev/src/dm/impls/plex/plex.c<br>[0]PETSC ERROR: #3 DMPlexGetTransitiveClosure() line 1342 in /home/justin/petsc-dev/src/dm/impls/plex/plex.c<br>[0]PETSC ERROR: #4 DMPlexLabelComplete() line 88 in /home/justin/petsc-dev/src/dm/impls/plex/plexsubmesh.c<br>[0]PETSC ERROR: #5 DMCreateDefaultSection_Plex() line 5605 in /home/justin/petsc-dev/src/dm/impls/plex/plex.c<br>[0]PETSC ERROR: #6 DMGetDefaultSection() line 3035 in /home/justin/petsc-dev/src/dm/interface/dm.c<br>[0]PETSC ERROR: #7 DMGetDefaultGlobalSection() line 3266 in /home/justin/petsc-dev/src/dm/interface/dm.c<br>[0]PETSC ERROR: #8 DMCreateGlobalVector_Section_Private() line 13 in /home/justin/petsc-dev/src/dm/interface/dmi.c<br>[0]PETSC ERROR: #9 DMCreateGlobalVector_Plex() line 1170 in /home/justin/petsc-dev/src/dm/impls/plex/plexcreate.c<br>[0]PETSC ERROR: #10 DMCreateGlobalVector() line 698 in /home/justin/petsc-dev/src/dm/interface/dm.c<br>[0]PETSC ERROR: #11 main() line 363 in /home/justin/Dropbox/Research_Topics/Code_PETSc/Nonneg/cube_with_hole.c<br><br><br></div><div>>From this error, it seems it's trying to access sieve points that non longer exist. And I think it has to do with the label "marker" still contains data from the interpolated mesh. Is there a way to "uninterpolate" or remove such points? Or is there a better way of approaching this whole thing?<br></div></div></div></div></blockquote><div><br></div></div></div><div>1) It would be easy to write a small function to throw out the points in the label that do not exist in the DM. You</div><div> would just extract each stratumIS and then Clear each invalid point.</div><div><br></div><div>2) However, I do not understand the rationale for this above. You could just call MarkBoundaryFaces on the final mesh.</div><div> If you are really trying to preserve a label across regular refinement AND you really do not want an interpolated mesh,</div><div> then code up 1). I have yet to see a case where the extra memory for edges and faces makes a difference, but it may exist.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div></div><div>Thanks,<span><font color="#888888"><br></font></span></div><span><font color="#888888"><div><br><br></div></font></span></div></div><span><font color="#888888"><br>-- <br><div><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br><a href="tel:%28512%29%20963-3262" value="+15129633262" target="_blank">(512) 963-3262</a><br></div></div></font></span></div>
</blockquote></span></div><span><font color="#888888"><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></font></span></div></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br></font></span></div><span><font color="#888888"><br clear="all"><br>-- <br><div><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br><a href="tel:%28512%29%20963-3262" value="+15129633262" target="_blank">(512) 963-3262</a><br></div></div>
</font></span></blockquote></div></div></div><div><div><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div></div></div>
</blockquote></div><br></div><br clear="all"><br>-- <br><div><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br><a href="tel:%28512%29%20963-3262" value="+15129633262" target="_blank">(512) 963-3262</a><br></div></div>
</div></div></blockquote></div></div></div><div><div><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div></div></div>
</blockquote></div><br></div><br clear="all"><br>-- <br><div><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br><a href="tel:%28512%29%20963-3262" value="+15129633262" target="_blank">(512) 963-3262</a><br></div></div>
</div></div></blockquote></div></div></div><div><div class="h5"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div><div><div>Justin Chang<br></div>PhD Candidate, Civil Engineering - Computational Sciences<br></div>University of Houston, Department of Civil and Environmental Engineering<br></div>Houston, TX 77004<br>(512) 963-3262<br></div></div>
</div>