<div dir="ltr"><div>Given the format I described in the previous post, I have the following mesh:<br><br></div>8<br><div><div>9<br>3<br>2<br>0 1 3<br>1 4 3<br>1 2 4<br>2 5 4<br>3 4 6<br>4 7 6<br>4 5 7<br>5 8 7<br>0.0 0.0<br>0.5 0.0<br>1.0 0.0<br>0.0 0.5<br>0.5 0.5<br>1.0 0.5<br>0.0 1.0<br>0.5 1.0<br>1.0 1.0<br>8<br></div><div>0 1 2 3 5 6 7 8<br><br></div><div>If I If i want to partition across 2 MPI processes, I would:<br><br></div><div>a) have all processes open the file and read the first four entires. <br></div><div><br></div><div>b) declare that rank 0 holds cells 0-3 and rank 1 holds cell 4-7. A special algorithm would be needed to handle cases where the number of cells is not divisible by the number of MPI processes<br><br></div><div>c) rank 0 stores 0 1 3 1 4 3 1 2 4 2 5 4, rank 1 reads 3 4 6 4 7 6 4 5 7 5 8 7<br><br></div><div>d) traverse the respective arrays and read in the coordinates associated with the vertices. I am guessing this part is "tricky" because I cannot simply do a naive partitioning of the coordinates the way I could with the connectivity?<br><br></div><div>e) Wont worry about exterior nodes for now<br><br></div><div>f) With the above I invoke DMPlexCreateFromDAG(), (of course making sure that the numbering is of right format)<br><br></div><div>g) When creating a SF for shared boundary, does this mean determining which nodes are ghosted and which ones are local? For a structured looking grid this would be easy since I could employ some sort of round-robin scheme.<br><br></div><div>h) Does the DMPlexDistribute() also take care of the repartitioning and redistribution?<br><br></div><div>Thanks,<br></div><div>Justin<br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jun 27, 2015 at 6:01 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5">On Sat, Jun 27, 2015 at 2:05 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div><div>Hi everyone,<br><br></div>I see that parallel IO of various input mesh formats for DMPlex is not currently supported. However, is there a way to read a custom mesh file in parallel?<br><br></div>For instance, I have a binary data file formatted like this:<br><br></div><No. of elements><br></div><No. of vertices><br></div><No. of nodes per element><br></div><Spatial dimension><br></div><Connectivity array ...<br>....><br></div><Coordinates array ...<br>....><br></div><No. of exterior nodes><br></div><List of exterior nodes ...<br><div>....><br><br></div><div>Reading this will allow me to create a DMPlex with DMPlexCreateFromDAG(). Currently, only the root process reads this binary mesh file and creates the DMPlex whereas the other processes create an empty DMPlex. Then all processes invoke DMPlexDistribute(). This one-to-all distribution seems to be a bottleneck, and I want to know if it's possible to have each process read a local portion of the connectivity and coordinates arrays and let DMPlex/METIS/ParMETIS handle the load balancing and redistribution. Intuitively this would be easy to write, but again I want to know how to do this through leveraging the functions and routines within DMPlex.<br></div></div></blockquote><div><br></div></div></div><div>This is on our agenda for the fall, but I can describe the process:</div><div><br></div><div> a) Do a naive partition of the cells (easy)</div><div><br></div><div> b) Read the connectivity for "your" cells (easy)</div><div><br></div><div> c) Read "your" coordinates (tricky)</div><div><br></div><div> d) Read "your" exterior (tricky)</div><div><br></div><div> e) Create local DAGs (easy)</div><div><br></div><div> f) Create SF for shared boundary (hard)</div><div><br></div><div> g) Repartition and redistribute (easy)</div><div><br></div><div>You could start writing this for your format and we could help. I probably will not get to the generic one until late in the year.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>Thanks,<br></div><div>Justin<span class="HOEnZb"><font color="#888888"><br></font></span></div></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div>