[petsc-users] Copying PETSc Objects Across MPI Communicators

[Matthew Knepley]

On Fri, Oct 27, 2023 at 3:54 PM Damyn Chipman <damynchipman at u.boisestate.edu>
wrote:

> Yeah, I’ll make an issue and use a modified version of this test routine.
>
> Does anything change if I will be using MATSCALAPACK matrices instead of
> the built in MATDENSE?
>

No, that is likely worse.


> Like I said, I will be computing Schur complements and need to use a
> parallel and dense matrix format.
>

I do not understand the communication pattern, but it is possible that
Elemental would be slightly faster since it has some cool built-in
communication operations, however it might be more programming.

  Thanks,

     Matt


> -Damyn
>
> On Oct 26, 2023, at 10:01 AM, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Wed, Oct 25, 2023 at 11:55 PM Damyn Chipman <
> damynchipman at u.boisestate.edu> wrote:
>
>> Great thanks, that seemed to work well. This is something my algorithm
>> will do fairly often (“elevating” a node’s communicator to a communicator
>> that includes siblings). The matrices formed are dense but low rank. With
>> MatCreateSubMatrix, it appears I do a lot of copying from one Mat to
>> another. Is there a way to do it with array copying or pointer movement
>> instead of copying entries?
>>
>
> We could make a fast path for dense that avoids MatSetValues(). Can you
> make an issue for this? The number one thing that would make this faster is
> to contribute a small test. Then we could run it continually when putting
> in the fast path to make sure we are preserving correctness.
>
>   Thanks,
>
>     Matt
>
>
>> -Damyn
>>
>> On Oct 24, 2023, at 9:51 PM, Jed Brown <jed at jedbrown.org> wrote:
>>
>> You can place it in a parallel Mat (that has rows or columns on only one
>> rank or a subset of ranks) and then MatCreateSubMatrix with all new
>> rows/columns on a different rank or subset of ranks.
>>
>> That said, you usually have a function that assembles the matrix and you
>> can just call that on the new communicator.
>>
>> Damyn Chipman <damynchipman at u.boisestate.edu> writes:
>>
>> Hi PETSc developers,
>>
>> In short, my question is this: Does PETSc provide a way to move or copy
>> an object (say a Mat) from one communicator to another?
>>
>> The more detailed scenario is this: I’m working on a linear algebra
>> solver on quadtree meshes (i.e., p4est). I use communicator subsets in
>> order to facilitate communication between siblings or nearby neighbors.
>> When performing linear algebra across siblings (a group of 4), I need to
>> copy a node’s data (i.e., a Mat object) from a sibling’s communicator to
>> the communicator that includes the four siblings. From what I can tell, I
>> can only copy a PETSc object onto the same communicator.
>>
>> My current approach will be to copy the raw data from the Mat on one
>> communicator to a new Mat on the new communicator, but I wanted to see if
>> there is a more “elegant” approach within PETSc.
>>
>> Thanks in advance,
>>
>> Damyn Chipman
>> Boise State University
>> PhD Candidate
>> Computational Sciences and Engineering
>> damynchipman at u.boisestate.edu
>>
>>
>>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20231028/8b3b09ee/attachment.html>