[petsc-dev] Using PETSc MatIS, how to matmult a global IS matrix and a global vector ?

[Franck Houssen]

The first thing I did was to put 3, not 4 : I got an error thrown in MatCreateIS (see the git diff + stack below). As the error said I used globalSize = numberOfMPIProcessus * localSize : my understanding is that, when using MatIS, the global size needs to be the sum of all local sizes. Correct ? 

I have a 3x3 global matrix made of two overlapping 2x2 local matrix (= diagonal with 1.). Each local matrix correspond to one domain (each domain is delegated to one MPI proc, so, I have 2 MPI procs because I have 2 domains). 
This is the simplest possible example: I have two 2x2 (local) diag matrix that overlap so that the global matrix built from them is 1, 2, 1 on the diagonal (local contributions add up in the middle). 
I need to MatMult this global matrix with a global vector filled with 1. 

Franck 

Git diff : 

--- a/matISLocalMat.cpp 
+++ b/matISLocalMat.cpp 
@@ -16,7 +16,7 @@ int main(int argc,char **argv) { 
int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size); if (size != 2) return 1; 
int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank); 

- PetscInt localSize = 2, globalSize = localSize*2 /*2 MPI*/; 
+ PetscInt localSize = 2, globalSize = 3; 
PetscInt localIdx[2] = {0, 0}; 
if (rank == 0) {localIdx[0] = 0; localIdx[1] = 1;} 
else {localIdx[0] = 1; localIdx[1] = 2;} 

Stack error: 

[0]PETSC ERROR: Nonconforming object sizes 
[0]PETSC ERROR: Sum of local lengths 4 does not equal global length 3, my local length 2 
[0]PETSC ERROR: [0] ISG2LMapApply line 17 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/vec/is/utils/isltog.c 
[0]PETSC ERROR: [0] MatSetValues_IS line 692 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c 
[0]PETSC ERROR: [0] MatSetValues line 1157 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c 
[0]PETSC ERROR: [0] MatISSetPreallocation_IS line 95 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c 
[0]PETSC ERROR: [0] MatISSetPreallocation line 80 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c 
[0]PETSC ERROR: [0] PetscSplitOwnership line 80 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/sys/utils/psplit.c 
[0]PETSC ERROR: [0] PetscLayoutSetUp line 129 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/vec/is/utils/pmap.c 
[0]PETSC ERROR: [0] MatSetLocalToGlobalMapping_IS line 628 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c 
[0]PETSC ERROR: [0] MatSetLocalToGlobalMapping line 1899 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c 
[0]PETSC ERROR: [0] MatCreateIS line 986 /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c 

----- Mail original -----

> De: "Stefano Zampini" <stefano.zampini at gmail.com>
> À: "Matthew Knepley" <knepley at gmail.com>
> Cc: "Franck Houssen" <franck.houssen at inria.fr>, "PETSc"
> <petsc-users at mcs.anl.gov>, "PETSc" <petsc-dev at mcs.anl.gov>
> Envoyé: Dimanche 21 Mai 2017 23:02:37
> Objet: Re: [petsc-dev] Using PETSc MatIS, how to matmult a global IS matrix
> and a global vector ?

> Franck,

> PETSc takes care of doing the matrix-vector multiplication properly using
> MatIS. As Matt said, the layout of the vectors is the usual parallel layout.
> The local sizes of the MatIS matrix (i.e. the local size of the left and
> right vectors used in MatMult) are not the sizes of the local subdomain
> matrices in MatIS.

> > On May 21, 2017, at 6:47 PM, Matthew Knepley < knepley at gmail.com > wrote:
> 

> > On Sun, May 21, 2017 at 11:26 AM, Franck Houssen < franck.houssen at inria.fr
> > >
> > wrote:
> 

> > > Using PETSc MatIS, how to matmult a global IS matrix and a global vector
> > > ?
> > > Example is attached : I don't get what I expect that is a vector such
> > > that
> > > proc0 = [1, 2] and proc1 = [2, 1]
> > 
> 

> > 1) I think the global size of your matrix is wrong. You seem to want 3, not
> > 4
> 

> > 2) Global vectors have a non-overlapping row partition. You might be
> > thinking
> > of local vectors
> 

> > Thanks,
> 

> > Matt
> 

> > --
> 
> > What most experimenters take for granted before they begin their
> > experiments
> > is infinitely more interesting than any results to which their experiments
> > lead.
> 
> > -- Norbert Wiener
> 

> > http://www.caam.rice.edu/~mk51/
> 

----- Mail original -----

> De: "Stefano Zampini" <stefano.zampini at gmail.com>
> À: "Matthew Knepley" <knepley at gmail.com>
> Cc: "Franck Houssen" <franck.houssen at inria.fr>, "PETSc"
> <petsc-users at mcs.anl.gov>, "PETSc" <petsc-dev at mcs.anl.gov>
> Envoyé: Dimanche 21 Mai 2017 23:02:37
> Objet: Re: [petsc-dev] Using PETSc MatIS, how to matmult a global IS matrix
> and a global vector ?

> Franck,

> PETSc takes care of doing the matrix-vector multiplication properly using
> MatIS. As Matt said, the layout of the vectors is the usual parallel layout.
> The local sizes of the MatIS matrix (i.e. the local size of the left and
> right vectors used in MatMult) are not the sizes of the local subdomain
> matrices in MatIS.

> > On May 21, 2017, at 6:47 PM, Matthew Knepley < knepley at gmail.com > wrote:
> 

> > On Sun, May 21, 2017 at 11:26 AM, Franck Houssen < franck.houssen at inria.fr
> > >
> > wrote:
> 

> > > Using PETSc MatIS, how to matmult a global IS matrix and a global vector
> > > ?
> > > Example is attached : I don't get what I expect that is a vector such
> > > that
> > > proc0 = [1, 2] and proc1 = [2, 1]
> > 
> 

> > 1) I think the global size of your matrix is wrong. You seem to want 3, not
> > 4
> 

> > 2) Global vectors have a non-overlapping row partition. You might be
> > thinking
> > of local vectors
> 

> > Thanks,
> 

> > Matt
> 

> > > Franck
> > 
> 

> > --
> 
> > What most experimenters take for granted before they begin their
> > experiments
> > is infinitely more interesting than any results to which their experiments
> > lead.
> 
> > -- Norbert Wiener
> 

> > http://www.caam.rice.edu/~mk51/
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20170523/68244d32/attachment.html>