[petsc-users] Reading in the full matrix in one process and then trying to solve in parallel with PETSc

[Jed Brown]

Matthew Knepley via petsc-users <petsc-users at mcs.anl.gov> writes:

> On Fri, Sep 20, 2019 at 7:54 AM Bao Kai via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
>
>> Hi,
>>
>> I understand that PETSc is not designed to be used this way, while I
>> am wondering if someone have done something similar to this.
>>
>> We have the full matrix from a simulation and rhs vector. We would
>> like to read in through PETSc in one process, then we use some
>> partition functions to partition the matrix.
>>
>> Based on the partition information, we redistribute the matrix among
>> the processes. Then we solve it in parallel.  It is for testing the
>> performance of some parallel linear solver and preconditions.
>>
>> We are not in the position to develop a full parallel implementation
>> of the simulator yet.

An alternative is to assemble a Mat living on a parallel communicator,
but with all entries on rank 0 (so just call your serial code to build
the matrix).  You can do the same for your vector, then KSPSolve.  To
make the solver parallel, just use run-time options:

 -ksp_type preonly -pc_type redistribute

will redistribute automatically inside the solver and return the
solution vector to you on rank 0.  You can control the inner solver via
prefix

 -redistribute_ksp_type gmres -redistribute_pc_type gamg -redistibute_ksp_monitor