<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div>Sounds good, thanks.</div><div><br></div><div>I’ve also been looking into Elemental, but the documentation seems outdated and I can’t find good examples on how to use it. I have the LLNL fork installed.</div><div><br></div><div>Thanks,</div><div>-Damyn</div><div>
</div>
<div><br><blockquote type="cite"><div>On Oct 28, 2023, at 8:56 AM, Matthew Knepley <knepley@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div dir="ltr">On Fri, Oct 27, 2023 at 3:54 PM Damyn Chipman <<a href="mailto:damynchipman@u.boisestate.edu">damynchipman@u.boisestate.edu</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div>Yeah, I’ll make an issue and use a modified version of this test routine.</div><div><br></div><div>Does anything change if I will be using MATSCALAPACK matrices instead of the built in MATDENSE?</div></div></blockquote><div><br></div><div>No, that is likely worse.</div><div> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div>Like I said, I will be computing Schur complements and need to use a parallel and dense matrix format.</div></div></blockquote><div><br></div><div>I do not understand the communication pattern, but it is possible that Elemental would be slightly faster since it has some cool built-in communication operations, however it might be more programming.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div>-Damyn</div><div></div><div><br><blockquote type="cite"><div>On Oct 26, 2023, at 10:01 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:</div><br><div><div dir="ltr" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;"><div dir="ltr">On Wed, Oct 25, 2023 at 11:55 PM Damyn Chipman <<a href="mailto:damynchipman@u.boisestate.edu" target="_blank">damynchipman@u.boisestate.edu</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div>Great thanks, that seemed to work well. This is something my algorithm will do fairly often (“elevating” a node’s communicator to a communicator that includes siblings). The matrices formed are dense but low rank. With MatCreateSubMatrix, it appears I do a lot of copying from one Mat to another. Is there a way to do it with array copying or pointer movement instead of copying entries?</div></div></blockquote><div><br></div><div>We could make a fast path for dense that avoids MatSetValues(). Can you make an issue for this? The number one thing that would make this faster is to contribute a small test. Then we could run it continually when putting in the fast path to make sure we are preserving correctness.</div><div><br></div><div> <span class="Apple-converted-space"> </span>Thanks,</div><div><br></div><div> <span class="Apple-converted-space"> </span>Matt</div><div> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div>-Damyn</div><div></div><div><br><blockquote type="cite"><div>On Oct 24, 2023, at 9:51 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:</div><br><div><div>You can place it in a parallel Mat (that has rows or columns on only one rank or a subset of ranks) and then MatCreateSubMatrix with all new rows/columns on a different rank or subset of ranks.<br><br>That said, you usually have a function that assembles the matrix and you can just call that on the new communicator.<br><br>Damyn Chipman <<a href="mailto:damynchipman@u.boisestate.edu" target="_blank">damynchipman@u.boisestate.edu</a>> writes:<br><br><blockquote type="cite">Hi PETSc developers,<br><br>In short, my question is this: Does PETSc provide a way to move or copy an object (say a Mat) from one communicator to another?<br><br>The more detailed scenario is this: I’m working on a linear algebra solver on quadtree meshes (i.e., p4est). I use communicator subsets in order to facilitate communication between siblings or nearby neighbors. When performing linear algebra across siblings (a group of 4), I need to copy a node’s data (i.e., a Mat object) from a sibling’s communicator to the communicator that includes the four siblings. From what I can tell, I can only copy a PETSc object onto the same communicator.<br><br>My current approach will be to copy the raw data from the Mat on one communicator to a new Mat on the new communicator, but I wanted to see if there is a more “elegant” approach within PETSc.<br><br>Thanks in advance,<br><br>Damyn Chipman<br>Boise State University<br>PhD Candidate<br>Computational Sciences and Engineering<br><a href="mailto:damynchipman@u.boisestate.edu" target="_blank">damynchipman@u.boisestate.edu</a><br></blockquote></div></div></blockquote></div><br></div></blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">--<span> </span></span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></div></div></blockquote></div><br></div></blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">--<span class="Apple-converted-space"> </span></span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></div></div></blockquote></div><br></body></html>