[petsc-users] Copying PETSc Objects Across MPI Communicators

[Barry Smith]

  If the matrices are stored as dense it is likely new code is the best way to go. 

  What pieces live on the sub communicator? Is it an m by N matrix where m is the number of rows (on that rank) and N is the total number of columns in the final matrix? Or are they smaller "chunks" that need to be combined together?

  Barry


> On Oct 25, 2023, at 3:38 PM, Damyn Chipman <damynchipman at u.boisestate.edu> wrote:
> 
> Great thanks, that seemed to work well. This is something my algorithm will do fairly often (“elevating” a node’s communicator to a communicator that includes siblings). The matrices formed are dense but low rank. With MatCreateSubMatrix, it appears I do a lot of copying from one Mat to another. Is there a way to do it with array copying or pointer movement instead of copying entries?
> 
> -Damyn
> 
>> On Oct 24, 2023, at 9:51 PM, Jed Brown <jed at jedbrown.org> wrote:
>> 
>> You can place it in a parallel Mat (that has rows or columns on only one rank or a subset of ranks) and then MatCreateSubMatrix with all new rows/columns on a different rank or subset of ranks.
>> 
>> That said, you usually have a function that assembles the matrix and you can just call that on the new communicator.
>> 
>> Damyn Chipman <damynchipman at u.boisestate.edu> writes:
>> 
>>> Hi PETSc developers,
>>> 
>>> In short, my question is this: Does PETSc provide a way to move or copy an object (say a Mat) from one communicator to another?
>>> 
>>> The more detailed scenario is this: I’m working on a linear algebra solver on quadtree meshes (i.e., p4est). I use communicator subsets in order to facilitate communication between siblings or nearby neighbors. When performing linear algebra across siblings (a group of 4), I need to copy a node’s data (i.e., a Mat object) from a sibling’s communicator to the communicator that includes the four siblings. From what I can tell, I can only copy a PETSc object onto the same communicator.
>>> 
>>> My current approach will be to copy the raw data from the Mat on one communicator to a new Mat on the new communicator, but I wanted to see if there is a more “elegant” approach within PETSc.
>>> 
>>> Thanks in advance,
>>> 
>>> Damyn Chipman
>>> Boise State University
>>> PhD Candidate
>>> Computational Sciences and Engineering
>>> damynchipman at u.boisestate.edu
>