[petsc-users] 1D parallel split of dense matrix

[Barry Smith]

  Yann,

    I'm sorry I don't understand what you are explaining but I can explain my understanding of multiply right hand sides with linear solves.

    Usually when people want to solve linear systems in PETSc with multiple right hand sides thye use MatMatSolve(), this efficiently solves for multiple right hand sides "at the same time."  MatMatSolve(A,B,X). B is a dense matrix with the same row partitioning as A and one column for each right hand side b and right hand side solution x.
You don't do anything with the local column sizes of B, they are meaningless for MatMatSolve(). 

   PETSc does not provide an interface for solving with multiply right hand sides with iterative solvers (using KSP) "all at once).

  Barry

Note you can use MatDenseGetArray() and VecPlace array to share memory between a column of the dense matrix and a vector; both the Mat and Vec need to have the same row layout across the processes.



> On Sep 17, 2020, at 9:48 AM, Yann Jobic <yann.jobic at univ-amu.fr> wrote:
> 
> Hi all,
> 
> I want to decompose a matrix in parallel for only one direction, the columns, for a dense matrix, loaded from a binary file. I tried to use :
> MatCreateDense(PETSC_COMM_WORLD,NumberOfRhs,matLocNbCols,NumberOfRhs,matNbCols,NULL,&NRhs);
> with matLocNbCols the number of local columns of a linear system (matrix A).
> However i've got an error when i use MatLoad : Read from file failed, read past end of file.
> Maybe because the number of local rows is equal to the number of global ones (variable NumberOfRhs) ?
> The sequential version runs fine.
> 
> Is it possible ?
> 
> I'm trying to set up a specific rhs matrix in order to solve the same linear system (ksp) with different right hand sides (rhs).
> In order to do so, i transposed the matrix containing all the rhs in a file, which i would like to load in parallel. The idea is to use MatGetRow, in order to create the rhs Vec, and then call KSPSolve (the inverse matrix of A should have been kept).
> 
> However, i would like a particular setup (1D parallel split) for the parallel layout of the rhs matrix : The number of columns of A is equal to the number of column of the matrix rhs, and the number of rows of rhs is equal to the number of rhs that i want to solve. I want this rhs to be split in the same way of the linear system, and i want each processors to have the whole global number of rows, that is to say, i want a 1D decomposition of my rhs matrix (corresponding to MatCreateVecs).
> 
> I then have to copy a row of rhs in a vec, and solve, for all the rows of rhs.
> 
> Is it a bad way to handle a N rhs problem with KSPSolve ?
> 
> Thanks,
> 
> Yann
> 
> ps : the error :
> Reading ComplexMatrix.bin Matrix...
> Matrix ComplexMatrix.bin read
> global size of A matrix (583335,583335)
> PROC : 1 Number of local  columns of the matrix A   : 291667
> PROC : 1 Number of local  columns of the vector     : 291667
> global size of NRhs Matrix : (15,583335)
> PROC : 0 Number of local  columns of the matrix A   : 291668
> PROC : 0 Number of local  columns of the vector     : 291668
> Reading NRhs ComplexRhs.bin Matrix...
> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
> [0]PETSC ERROR: Read from file failed
> [0]PETSC ERROR: Read past end of file