<div><div dir="auto">Hi,</div></div><div><br><div class="gmail_quote"><div dir="ltr">On Wed 23. Mar 2022 at 18:52, Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Mar 23, 2022 at 11:09 AM Joauma Marichal <<a href="mailto:joauma.marichal@uclouvain.be" target="_blank">joauma.marichal@uclouvain.be</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hello, <br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I sent an email last week about an issue I had with DMSwarm but did not get an answer yet. If there is any other information needed or anything I could try to solve it, I would be happy to do them...</div></div></blockquote><div><br></div><div>I got a chance to run the code. I believe this undercovered a bug in our implementation of point location with DMDA. I will make an Issue.</div><div><br></div><div>Your example runs correctly for me if you replace DM_BOUNDARY_GHOSTED with DM_BOUNDARY_NONE in the DMDACreate3d.</div><div>Can you try that?</div></div></div></blockquote><div dir="auto"><br></div><div dir="auto">The PIC support in place between DMSwarm and DMDA only works when the DA points define the vertices of a set of quads / hexes AND if the mesh is uniform, ie you defined the coordinates using SetUniformCoordinates. The point location routine is very simple. </div><div dir="auto"><br></div><div dir="auto">There is no way for the DA infrastructure to know what the points in the DA physically represent (Ie vertices, cell centroid or face centroids). The DA just defines a set of logically order points which can be indexed in an i,j,k manner.</div><div dir="auto"><br></div><div dir="auto">So if you are using the DA to represent cell centered data then the point location routine will give incorrect results. Also, the coordinates from SetUniformCoordinates won’t give you what you expect either if the x0,x1 you provide define the start,end coordinates of the physical boundary, but you interpret the DA points to be cell centers.</div><div dir="auto"><br></div><div dir="auto">There are several options you can pursue. </div><div dir="auto">1/ Make an independent DMDA which represents the vertices of your mesh. Use this DA with your DMSwarm.</div><div dir="auto">2/ Provide your own point location routine for your collocated DA representation.</div><div dir="auto"><br></div><div dir="auto">Thanks,</div><div dir="auto">Dave</div><div dir="auto"><br></div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div></div></div><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Thanks a lot for your help.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Best regards,
<div>Joauma</div>
<br>
</div>
<div id="m_6472318984381978196gmail-m_6285434010030868936appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="m_6472318984381978196gmail-m_6285434010030868936divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Joauma Marichal<br>
<b>Sent:</b> Friday, March 18, 2022 4:02 PM<br>
<b>To:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a> <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
<b>Subject:</b> DMSwarm</font>
<div> </div>
</div>
<div dir="ltr">
<div>Hello, </div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<div><br>
</div>
<div>I am writing to you as I am trying to implement a Lagrangian Particle Tracking method to my eulerian solver that relies on a 3D collocated DMDA.</div>
<div><br>
</div>
<span>I have been using examples to develop a first basic code. The latter creates particles on rank 0 with random coordinates on the whole domain and then migrates them to the rank corresponding to these coordinates. Unfortunately, as I migrate I am loosing
some particles. I came to understand that when I create a DMDA with 6 grid points in each 3 directions and then set coordinates in between 0 and 1 using ,DMDASetUniformCoordinates and running on 2 processors, I obtain the following coordinates values on each
proc:</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>[Proc 0] X = 0.000000 0.200000 0.400000 0.600000 0.800000 1.000000
<div>[Proc 0] Y = 0.000000 0.200000 0.400000 0.600000 0.800000 1.000000 </div>
<div>[Proc 0] Z = 0.000000 0.200000 0.400000 </div>
<div>[Proc 1] X = 0.000000 0.200000 0.400000 0.600000 0.800000 1.000000 </div>
<div>[Proc 1] Y = 0.000000 0.200000 0.400000 0.600000 0.800000 1.000000 </div>
<span>[Proc 1] Z = 0.600000 0.800000 1.000000 </span>.</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>Furthermore, it appears that the particles that I am losing are (in the case of 2 processors) located in between z = 0.4 and z = 0.6. How can this be avoided?
<br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>I attach my code to this email (I run it using mpirun -np 2 ./cobpor).</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>Furthermore, my actual code relies on a collocated 3D DMDA, however the <span>
DMDASetUniformCoordinates</span> seems to be working for staggered grids only... How would you advice to deal with particles in this case?
<br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>Thanks a lot for your help. <br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span>Best regards,
<div>Joauma</div>
<br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
<div style="color:black;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<span><br>
</span></div>
</div>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div></div>