<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Jun 29, 2017 at 8:59 AM, Hassan Raiesi <span dir="ltr"><<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.bombardier.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US">
<div class="gmail-m_-5733350705985167537WordSection1">
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Matthew,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Not sure if I understood completely, I have the ordering according to the CGNS standard for all cells in the mesh, or I can change it if needed, but I don’t understand
what you mean by reporting all faces as ordered vertices? Do you mean I just define the face connectivity?Could you explain a bit more?</span></p></div></div></blockquote><div><br></div><div>For a triangular prism, you would add the case in GetRawFaces() for dim 3, coneSize 6. Lets look at a triangle (dim 2, coneSize 3)</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/5ebd2cd705b0ffa5a27aee8a08f86795d413b548/src/dm/impls/plex/plexinterpolate.c?at=master&fileviewer=file-view-default#plexinterpolate.c-63">https://bitbucket.org/petsc/petsc/src/5ebd2cd705b0ffa5a27aee8a08f86795d413b548/src/dm/impls/plex/plexinterpolate.c?at=master&fileviewer=file-view-default#plexinterpolate.c-63</a><br></div><div><br></div><div>You see that it returns 3 groups of 2 vertices, which are the edges. For a tetrahedron</div><div><br></div><div> <a href="https://bitbucket.org/petsc/petsc/src/5ebd2cd705b0ffa5a27aee8a08f86795d413b548/src/dm/impls/plex/plexinterpolate.c?at=master&fileviewer=file-view-default#plexinterpolate.c-101">https://bitbucket.org/petsc/petsc/src/5ebd2cd705b0ffa5a27aee8a08f86795d413b548/src/dm/impls/plex/plexinterpolate.c?at=master&fileviewer=file-view-default#plexinterpolate.c-101</a></div><div><br></div><div>it returns 4 groups of 3 vertices, which are the faces. Note that all faces should be oriented such that the normal is outward.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><div class="gmail-m_-5733350705985167537WordSection1"><p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thank you<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">-Hassan<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span style="font-size:11pt;font-family:Calibri,sans-serif"> Matthew Knepley [mailto:<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, June 27, 2017 1:54 PM<br>
<b>To:</b> Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParall<wbr>el<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 27, 2017 at 11:30 AM, Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">I meant the interpolation,
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">DMPlex supports those element shapes, however, there are two problems, one, “DMPlexBuildFromCellList_<wbr>Parallel”
does not take mixed elements (numCorners must be constant for all elements in current implementation), that was easy to fix,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">I already extended DMPlexBuildFromCellList_<wbr>Parallel_Private to take elements with different shapes,
then I realized the interpolation does not work when the mesh has elements other than tets and hex.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Okay, here is what is needed. You need to<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> a) prescribe an order for the vertices in a prism/pyramid (all input cells must have this order)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> b) report all faces as sets of ordered vertices<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> - You have to order them matching the Plex vertex order for the lower dimensional shape<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> - They should be oriented as to have outward facing normal<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Regarding the interpolation, I also noticed that the memory requirement is huge if I load the whole
mesh on one core and interpolate (I cannot interpolate a mesh with 9M tets on a machine with 128GB of memory, it ran out of memory, I’ll try to run with petsc memory logs and send).
</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I am not sure what the upper limit is, but in a 3D mesh you could have many times the number of cells in faces and edges. Note that<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">you would need --with-64-bit-indices to go beyond 4GB.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">is there anyways to interpolate the mesh after DMPlexdistribute? The code crashes if I move DMPlexInterpolate
to after calling DMPlexdistribute., I guess what is needed is a DMPlexInterpolate on already distributed meshes.</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This should work. Maybe try a small example? If that crashes, just send it.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> Matt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thank you</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">-Hassan</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span style="font-size:11pt;font-family:Calibri,sans-serif"> Matthew Knepley [mailto:<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, June 27, 2017 11:53 AM<br>
<b>To:</b> Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParall<wbr>el</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 27, 2017 at 10:08 AM, Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Great, It’s clear now
</span><span style="font-size:11pt;font-family:Wingdings;color:rgb(31,73,125)">J</span><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">,
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">One more question, any plan to support other element shapes (prism and pyramid) in 3D?, </span>DMPlexGetRawFaces_Internal
o<span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">nly supports tets and hexs in 3D, can prisms and pyramids be used as degenerate hexahedrons?</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">It depends on what you mean "support". Right now, we can represent these shapes in Plex. However, if you<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">want mesh interpolation to work, then yes you need to extend GetRawFaces() to understand that shape. If<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">you want them read out of a file format, other than Gmsh, we would likely have to extend that as well. These<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">are straightforward once I understand what exactly you want to do.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Matt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thank you</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">-Hassan</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<pre><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></pre>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span style="font-size:11pt;font-family:Calibri,sans-serif"> Matthew Knepley [mailto:<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, June 27, 2017 10:17 AM<br>
<b>To:</b> Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParall<wbr>el</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 27, 2017 at 9:12 AM, Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Thanks for your reply,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Is there any example where each rank owns more than 1 element, i.e for the simple mesh here(attached
png file), how should I pack and pass the coordinates of the vertices owned by rank0, rank1</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="FR-CA" style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">Rank0:numcells = 2; num nodes=4, cells=[154 245] , nodes=[1 5 2 4]</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">nodes = 1245</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">vertex coords: in what node order ?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">[coords_n1 coords_n2 coords_n4 coords_n5] or</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">[coords_n2 coords_n4 coords_n1 coords_n5] or …..?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="FR-CA" style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">rank1: numcells = 2; num nodes=2, cells=[532 635], nodes=[6 3]</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">vertexcoords [how to pack the nodes coords here?]</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">should it be [x6y6 x3y3] or [x3y3 x6y6]? In what order?</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I think there is a misunderstanding here.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">There is NO connection between the cell order and the vertex order. Each process gets a contiguous<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">set of cells (in the global numbering) and a contiguous set of vertices (in the global numbering). These<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">two are NOT related. We then move the vertices to the correct processes. In this way, we can load<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">completely in parallel, without requiring any setup in the mesh file.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">If you are worried, you can always arrange the order of vertices to "match" the order of cells.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Matt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)">thanks</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:11pt;font-family:Calibri,sans-serif">From:</span></b><span style="font-size:11pt;font-family:Calibri,sans-serif"> Matthew Knepley [mailto:<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>]
<br>
<b>Sent:</b> Sunday, June 25, 2017 1:04 PM<br>
<b>To:</b> Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>><br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] specifying vertex coordinates using DMPlexCreateFromCellListParall<wbr>el</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Apr 11, 2017 at 9:21 AM, Hassan Raiesi <<a href="mailto:Hassan.Raiesi@aero.bombardier.com" target="_blank">Hassan.Raiesi@aero.<wbr>bombardier.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal">Hello,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I’m trying to use
<span style="font-family:Consolas">DMPlexCreateFromCellListParall<wbr>el</span> to create a DM from an already partitioned mesh,
<u></u><u></u></p>
<p class="MsoNormal">It requires an array of numVertices*spaceDim numbers, but how should one order the coordinates of the vertices?<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Global order. Here is the idea. You must read the file in chunks so that each proc can read its own chunk in parallel<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">without talking to anyone else.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal">we only pass the global vertex numbers using ‘const int cells[]’ to define the cell-connectivity, so passing the vertex coordinates in local ordering wouldn’t make sense?<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Yes.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal">If it needs to be in global ordering, should I sort the global index of the node numbers owned by each rank (as they wont be continuous).<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Nope.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> Matt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</blockquote>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt">
<div>
<div>
<p class="MsoNormal">Thank you<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Hassan Raiesi,<u></u><u></u></p>
<p class="MsoNormal">Bombardier Aerospace
<u></u><u></u></p>
<p class="MsoNormal"><cite><span style="font-family:Calibri,sans-serif"><a href="http://www.bombardier.com" target="_blank">www.bombardier.com</a></span></cite><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">--
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~<wbr>mk51/</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">--
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~<wbr>mk51/</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">--
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~<wbr>mk51/</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~<wbr>mk51/</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">http://www.caam.rice.edu/~mk51/</a><br></div></div></div>
</div></div>