<div dir="ltr"><div dir="ltr">My apologies for the late follow-up. There was a time conflict. <br><br>A simple example code related to the issue I mentioned is attached here. The sample code does: (1) load grid on dm, (2) compute vertex-wise control volume for each node in a median-dual way, (3) halo exchange among procs to have complete control volume values, and (4) print out its field as a .vtu file. To make sure, the computed control volume is also compared with PETSc-computed control volume via DMPlexComputeCellGeometryFVM() (see lines 771-793). <br><br>Back to the original problem, I can get a proper control volume field with PETSc 3.18.4, which is the latest stable release. However, if I use PETSc from the main repo, it gives a strange control volume field. Something is certainly strange around the parallel boundaries, thus I think something went wrong with halo communication. To help understand, a comparing snapshot is also attached. I guess a certain part of my code is no longer compatible with PETSc unless there is a bug in the library. Could I get comments on it? <br><br>Thanks,<br>Mike<br></div><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br>On Mon, Feb 20, 2023 at 12:05 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Sat, Feb 18, 2023 at 12:00 PM Mike Michell <<a href="mailto:mi.mike1021@gmail.com" target="_blank">mi.mike1021@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">As a follow-up, I tested: <div><br></div><div>(1) Download tar for v3.18.4 from petsc gitlab (<a href="https://gitlab.com/petsc/petsc/-/tree/v3.18.4" target="_blank">https://gitlab.com/petsc/petsc/-/tree/v3.18.4</a>) has no issue on DMPlex halo exchange. This version works as I expect.</div><div>(2) Clone main branch (git clone <a href="https://gitlab.com/petsc/petsc.git" target="_blank">https://gitlab.com/petsc/petsc.git</a>) has issues with DMPlex halo exchange. Something is suspicious about this main branch, related to DMPlex halo. The solution field I got is not correct. But it works okay with 1-proc. </div><div><br></div><div>Does anyone have any comments on this issue? I am curious if other DMPlex users have no problem regarding halo exchange. FYI, I do not declare ghost layers for halo exchange. </div></div></div></blockquote><div><br></div><div>There should not have been any changes there and there are definitely tests for this.</div><div><br></div><div>It would be great if you could send something that failed. I could fix it and add it as a test.</div></div></div></blockquote><div><br></div><div>Just to follow up, we have tests of the low-level communication (Plex tests ex1, ex12, ex18, ex29, ex31), and then we have</div><div>tests that use halo exchange for PDE calculations, for example SNES tutorial ex12, ex13, ex62. THe convergence rates</div><div>should be off if the halo exchange were wrong. Is there any example similar to your code that is failing on your installation?</div><div>Or is there a way to run your code?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Thanks,</div><div>Mike</div></div><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br>Dear PETSc team,<div><br></div><div>I am using PETSc for Fortran with DMPlex. I have been using this version of PETSc:</div><div>>>git rev-parse origin</div><div>>>995ec06f924a86c4d28df68d1fdd6572768b0de1</div><div>>>git rev-parse FETCH_HEAD</div><div>>>9a04a86bf40bf893fb82f466a1bc8943d9bc2a6b</div><div><br></div><div>There has been no issue, before the one with VTK viewer, which Jed fixed today (<a href="https://gitlab.com/petsc/petsc/-/merge_requests/6081/diffs?commit_id=27ba695b8b62ee2bef0e5776c33883276a2a1735" rel="noreferrer" target="_blank">https://gitlab.com/petsc/petsc/-/merge_requests/6081/diffs?commit_id=27ba695b8b62ee2bef0e5776c33883276a2a1735</a>). </div><div><br></div><div>Since that MR has been merged into the main repo, I pulled the latest version of PETSc (basically I cloned it from scratch). But if I use the same configure option with before, and run my code, then there is an issue with halo exchange. The code runs without error message, but it gives wrong solution field. I guess the issue I have is related to graph partitioner or halo exchange part. This is because if I run the code with 1-proc, the solution is correct. I only updated the version of PETSc and there was no change in my own code. Could I get any comments on the issue? I was wondering if there have been many changes in halo exchange or graph partitioning & distributing part related to DMPlex.</div><div><br></div><div>Thanks,</div><div>Mike</div></div>
</blockquote></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div></div>