[petsc-users] Use parallel PETSc linear solver in a sequential program

[Qin Lu]

Jed,

The program is sequential so the CSR matrix is only constructed in the master processor. You said: "You should really partition the *mesh*, then assemble the local parts in parallel", do you mean I should parallelize the whole program (using domain decomposition, for instance)? That will be my next step. But for now I need to make the parallel solver work in a sequential program.

In the context of partitioning/distributing the matrix, the manual says the first n0 number of rows should be on the first process, the second n1 rows should be on the second process, and so on. But when calling MatSetValues, each coefficient should use GLOBAL (instead of local) row/column index (as showed in sample code ex2.c), right?

Thanks,
Qin



On Thursday, October 31, 2013 4:07 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

Qin Lu <lu_qin_2000 at yahoo.com> writes:

> Jed,
>  
> What about MatCreateSeqAIJWithArrays? Is it also implemented by looping over
> the rows calling MatSetValues?

No, it uses the arrays directly because that format makes sense in
serial.

> My CRS matrix is constructed in the master processor when using a
> parallel solver. 

This will absolutely cripple your scalability.  If you're just trying to
use a few cores, then fine, but if you care about scalability, you need
to rethink your design.


> Do I have to manually partition it (using metis, for instance) and
> distribute it to all processors using MPI, or PETSc has any
> subroutines to do this job?

You should really partition the *mesh*, then assemble the local parts in
parallel.  The alternative you can use is to partition the matrix
entries (renumbering from your "native" ordering) and then call
MatSetValues (mapping the indices) from rank 0.  This part is not
scalable and may only make sense if you have a difficult problem or many
systems to solve.  Better to do it right and assemble in parallel.  It's
not difficult.