<div dir="ltr"><div dir="ltr">On Mon, Jan 22, 2024 at 2:26 PM Berend van Wachem <<a href="mailto:berend.vanwachem@ovgu.de">berend.vanwachem@ovgu.de</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear Matt,<br>
<br>
The problem is that I haven't figured out how to write a polyhedral DMplex in parallel. So, currently, I can write the Vec data <br>
in parallel, but the cones for the cells/faces/edges/nodes for the mesh from just one process to a file (after gathering the <br>
DMplex to a single process).<br></blockquote><div><br></div><div>Ah shoot. Can you send me a polyhedral mesh (or code to generate one) so I can fix the parallel write problem? Or maybe it is already an issue and I forgot?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
From the restart, I can then read the cone information from one process from the file, recreate the DMPlex, and then <br>
redistribute it. In this scenario, the Vec data I read in (in parallel) will not match the correct cells of the DMplex. Hence, I <br>
need to put it in the right place afterwards.<br></blockquote><div><br></div><div>Yes, then searching makes sense. You could call DMLocatePoints(), but maybe you are doing that.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Best, Berend.<br>
<br>
On 1/22/24 20:03, Matthew Knepley wrote:<br>
> On Mon, Jan 22, 2024 at 1:57 PM Berend van Wachem <<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a> <mailto:<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a>>> wrote:<br>
> <br>
> Dear Matt,<br>
> <br>
> Thanks for your quick response.<br>
> I have a DMPlex with a polyhedral mesh, and have defined a number of vectors with data at the cell center. I have generated<br>
> data<br>
> for a number of timesteps, and I write the data for each point to a file together with the (x,y,z) co-ordinate of the cell<br>
> center.<br>
> <br>
> When I want to do a restart from the DMPlex, I recreate the DMplex with the polyhedral mesh, redistribute it, and for each cell<br>
> center find the corresponding (x,y,z) co-ordinate and insert the data that corresponds to it. This is quite expensive, as it<br>
> means I need to compare doubles very often.<br>
> <br>
> But reading your response, this may not be a bad way of doing it?<br>
> <br>
> <br>
> It always seems to be a game of "what do you want to assume?". I tend to assume that I wrote the DM and Vec in the same order, <br>
> so when I load them they match. This is how Firedrake I/O works, so that you can load up on a different number of processes <br>
> (<a href="https://arxiv.org/abs/2401.05868" rel="noreferrer" target="_blank">https://arxiv.org/abs/2401.05868</a> <<a href="https://arxiv.org/abs/2401.05868" rel="noreferrer" target="_blank">https://arxiv.org/abs/2401.05868</a>>).<br>
> <br>
> So, are you writing a Vec, and then redistributing and writing another Vec? In the scheme above, you would have to write both <br>
> DMs. Are you trying to avoid this?<br>
> <br>
> Thanks,<br>
> <br>
> Matt<br>
> <br>
> Thanks,<br>
> <br>
> Berend.<br>
> <br>
> On 1/22/24 18:58, Matthew Knepley wrote:<br>
> > On Mon, Jan 22, 2024 at 10:49 AM Berend van Wachem <<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a> <mailto:<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a>><br>
> <mailto:<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a> <mailto:<a href="mailto:berend.vanwachem@ovgu.de" target="_blank">berend.vanwachem@ovgu.de</a>>>> wrote:<br>
> ><br>
> > Dear Petsc-Team,<br>
> ><br>
> > Is there a good way to define a unique integer number in each element<br>
> > (e.g. a cell) of a DMPlex mesh, which is in the same location,<br>
> > regardless of the number of processors or the distribution of the mesh<br>
> > over the processors?<br>
> ><br>
> > So, for instance, if I have a DMPlex box mesh, the top-right-front<br>
> > corner element (e.g. cell) will always have the same unique number,<br>
> > regardless of the number of processors the mesh is distributed over?<br>
> ><br>
> > I want to be able to link the results I have achieved with a mesh from<br>
> > DMPlex on a certain number of cores to the same mesh from a DMPlex on a<br>
> > different number of cores.<br>
> ><br>
> > Of course, I could make a tree based on the distance of each element to<br>
> > a certain point (based on the X,Y,Z co-ordinates of the element), and go<br>
> > through this tree in the same way and define an integer based on this,<br>
> > but that seems rather cumbersome.<br>
> ><br>
> ><br>
> > I think this is harder than it sounds. The distance will not work because it can be very degenerate.<br>
> > You could lexicographically sort the coordinates, but this is hard in parallel. It is fine if you are willing<br>
> > to gather everything on one process. You could put down a p4est, use the Morton order to number them since this is stable<br>
> for a<br>
> > given refinement. And then within each box lexicographically sort the centroids. This is definitely cumbersome, but I cannot<br>
> > think of anything else. This also might have parallel problems since you need to know how much overlap you need to fill<br>
> each box.<br>
> ><br>
> > Thanks,<br>
> ><br>
> > Matt<br>
> ><br>
> > Thanks and best regards, Berend.<br>
> ><br>
> > --<br>
> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any<br>
> results to<br>
> > which their experiments lead.<br>
> > -- Norbert Wiener<br>
> ><br>
> > <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a> <<a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a>> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a><br>
> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>>><br>
> <br>
> <br>
> <br>
> -- <br>
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to <br>
> which their experiments lead.<br>
> -- Norbert Wiener<br>
> <br>
> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>