[petsc-users] Building MatIS with dense local matrix ?

Tue Jun 20 10:31:19 CDT 2017

OK. 

Franck 

----- Mail original -----

> De: "Stefano Zampini" <stefano.zampini at gmail.com>
> À: "Franck Houssen" <franck.houssen at inria.fr>
> Cc: "PETSc users list" <petsc-users at mcs.anl.gov>
> Envoyé: Mardi 20 Juin 2017 17:08:29
> Objet: Re: [petsc-users] Building MatIS with dense local matrix ?

> It should be fixed right now, both in master and in maint.
> Again, sorry for this ping-pong of fixes, my brain it's not fully functional
> these days....

> https://bitbucket.org/petsc/petsc/commits/c6f20c4fa7817632f09219574920bd3bd922f6f1

> 2017-06-20 16:30 GMT+02:00 Franck Houssen < franck.houssen at inria.fr > :

> > OK. I moved from petsc-3.7.6 to development version (git clone bitbucket).
> 
> > The very first version of the dummy example (=
> > matISCSRDenseSquare/Rect.cpp)
> > works with this fix:
> > https://bitbucket.org/petsc/petsc/commits/4c8dd594d1988a0cbe282f8a37d9916f61e0c445
> > .
> 
> > The second version of the dummy example works too with the fix if one moves
> > to petsc bitbucket (master).
> 

> > But, the code still breaks in "my" initial "real" case (using now master
> > from
> > petsc bitbucket)... With another error "SEGV under
> > MatISSetMPIXAIJPreallocation_Private" (note: this is not a "new non zero"
> > message, this seems to be another problem).
> 

> > Here is a third version of the dummy example that breaks with "SEGV under
> > MatISSetMPIXAIJPreallocation_Private" (using master from petsc bitbucket) :
> > the idea is the same but with N procs (not only 2) and a rectangular matrix
> > of size N*(N+1).
> 
> > With 2 procs, it works (all cases).
> 
> > With 4 procs, new problems occur:
> 
> > >> mpirun -n 4 ./matISCSRDenseSquare.exe csr; mpirun -n 4
> > >> ./matISCSRDenseSquare.exe dense => OK
> 
> > >> mpirun -n 4 ./matISCSRDenseRect.exe csr => OK
> 
> > but
> 
> > >> mpirun -n 4 ./matISCSRDenseRect.exe dense;
> 
> > dense
> 
> > dense
> 
> > dense
> 
> > dense
> 
> > [3]PETSC ERROR:
> > ------------------------------------------------------------------------
> 
> > [3]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> > probably memory access out of range
> 
> > [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> 
> > [3]PETSC ERROR: or see
> > http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> 
> > [3]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X
> > to
> > find memory corruption errors
> 
> > [3]PETSC ERROR: likely location of problem given in stack below
> 
> > [3]PETSC ERROR: --------------------- Stack Frames
> > ------------------------------------
> 
> > [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not
> > available,
> 
> > [3]PETSC ERROR: INSTEAD the line number of the start of the function
> 
> > [3]PETSC ERROR: is given.
> 
> > [3]PETSC ERROR: [3] MatISSetMPIXAIJPreallocation_Private line 1055
> > /home/fghoussen/Documents/INRIA/petsc/src/mat/impls/is/matis.c
> 
> > [3]PETSC ERROR: [3] MatISGetMPIXAIJ_IS line 1230
> > /home/fghoussen/Documents/INRIA/petsc/src/mat/impls/is/matis.c
> 
> > [3]PETSC ERROR: [3] MatISGetMPIXAIJ line 1384
> > /home/fghoussen/Documents/INRIA/petsc/src/mat/impls/is/matis.c
> 
> > [3]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> 

> > I tried to go through this with caution... But I have to say I feel messy.
> > Can you reproduce this problem at your side ?
> 

> > Franck
> 

> > >> git diff .
> 
> > --- a/matISCSRDenseRect.cpp
> 
> > +++ b/matISCSRDenseRect.cpp
> 
> > @@ -14,19 +14,17 @@ int main(int argc,char **argv) {
> 
> > if (matType != "csr" && matType != "dense") {cout << "error: need arg = csr
> > or dense" << endl; return 1;}
> 

> > PetscInitialize(&argc, &argv, NULL, NULL);
> 
> > - int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size); if (size != 2) {cout
> > <<
> > "error: mpi != 2" << endl; return 1;}
> 
> > + int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
> 
> > int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> 

> > - PetscInt localIdx[2] = {0, 0};
> 
> > - if (rank == 0) {localIdx[0] = 0; localIdx[1] = 1;}
> 
> > - else {localIdx[0] = 1; localIdx[1] = 2;}
> 
> > + PetscInt localIdx[2] = {rank, rank+1};
> 
> > ISLocalToGlobalMapping rmap;
> 
> > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 1, &rank,
> > PETSC_COPY_VALUES, &rmap);
> 
> > ISLocalToGlobalMapping cmap;
> 
> > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 2, localIdx,
> > PETSC_COPY_VALUES, &cmap);
> 

> > Mat A;
> 
> > - MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, 2, 3, rmap,
> > cmap, &A);
> 
> > + MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, size,
> > size+1,
> > rmap, cmap, &A);
> 

> > Mat Aloc;
> 
> > if (matType == "csr") {cout << matType << endl;
> > MatCreateSeqAIJ(PETSC_COMM_SELF, 1, 2, 2, NULL, &Aloc);}
> 

> > --- a/matISCSRDenseSquare.cpp
> 
> > +++ b/matISCSRDenseSquare.cpp
> 
> > @@ -14,19 +14,17 @@ int main(int argc,char **argv) {
> 
> > if (matType != "csr" && matType != "dense") {cout << "error: need arg = csr
> > or dense" << endl; return 1;}
> 

> > PetscInitialize(&argc, &argv, NULL, NULL);
> 
> > - int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size); if (size != 2) {cout
> > <<
> > "error: mpi != 2" << endl; return 1;}
> 
> > + int size = 0; MPI_Comm_size(MPI_COMM_WORLD, &size);
> 
> > int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> 

> > - PetscInt localIdx[2] = {0, 0};
> 
> > - if (rank == 0) {localIdx[0] = 0; localIdx[1] = 1;}
> 
> > - else {localIdx[0] = 1; localIdx[1] = 2;}
> 
> > + PetscInt localIdx[2] = {rank, rank+1};
> 
> > ISLocalToGlobalMapping rmap;
> 
> > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 2, localIdx,
> > PETSC_COPY_VALUES, &rmap);
> 
> > ISLocalToGlobalMapping cmap;
> 
> > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 2, localIdx,
> > PETSC_COPY_VALUES, &cmap);
> 

> > Mat A;
> 
> > - MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, 3, 3, rmap,
> > cmap, &A);
> 
> > + MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, size+1,
> > size+1, rmap, cmap, &A);
> 

> > Mat Aloc;
> 
> > if (matType == "csr") {cout << matType << endl;
> > MatCreateSeqAIJ(PETSC_COMM_SELF, 2, 2, 2, NULL, &Aloc);}
> 

> > > De: "Stefano Zampini" < stefano.zampini at gmail.com >
> > 
> 
> > > À: "Franck Houssen" < franck.houssen at inria.fr >
> > 
> 
> > > Cc: "PETSc users list" < petsc-users at mcs.anl.gov >
> > 
> 
> > > Envoyé: Mardi 20 Juin 2017 13:23:27
> > 
> 

> > > Objet: Re: [petsc-users] Building MatIS with dense local matrix ?
> > 
> 

> > > Franck
> > 
> 

> > > I tested your new example with master and it works. However, It doesn't
> > > work
> > > with maint. I fixed the rectangular case a while ago in master and forgot
> > > to
> > > add the change to maint too. Sorry for that.
> > 
> 

> > > This should fix the problem with maint:
> > > https://bitbucket.org/petsc/petsc/commits/0ea065fb06d751599c4157d36bfe1a1b41348e0b
> > 
> 

> > > Test your real case and let me know.
> > 
> 
> > > If you could, it would be good to test against master too.
> > 
> 
> > > Thanks,
> > 
> 
> > > Stefano
> > 
> 

> > > 2017-06-20 12:58 GMT+02:00 Franck Houssen < franck.houssen at inria.fr > :
> > 
> 

> > > > As I said, it is often difficult to reduce the "real" problem: it turns
> > > > out
> > > > that your fix solves the "matISCSRDenseSquare/Rect.cpp" dummy example I
> > > > sent, but, it's still not working in "my real" situation.
> > > 
> > 
> 

> > > > I changed a bit the "matISCSRDenseSquare/Rect.cpp" dummy example (see
> > > > git
> > > > diff below - I just changed the point that overlaps) : the dummy
> > > > example
> > > > is
> > > > still failing
> > > 
> > 
> 

> > > > "mpirun -n 2 ./matISCSRDenseSquare.exe csr" and "mpirun -n 2
> > > > ./matISCSRDenseSquare.exe dense" : OK
> > > 
> > 
> 
> > > > but
> > > 
> > 
> 
> > > > "mpirun -n 2 ./matISCSRDenseRect.exe csr" and "mpirun -n 2
> > > > ./matISCSRDenseRect.exe dense": KO with error "Argument out of range -
> > > > New
> > > > nonzero at (0,2) caused a malloc"
> > > 
> > 
> 

> > > > I would say, the problem (I am concerned with the "real" case) is
> > > > around
> > > > lines 360-380 of /src/mat/impls/is/matis.c (not around 181 : this fixes
> > > > a
> > > > valid problem, but, this problem is another one)
> > > 
> > 
> 

> > > > Franck
> > > 
> > 
> 

> > > > --- a/matISCSRDenseRect.cpp
> > > 
> > 
> 
> > > > +++ b/matISCSRDenseRect.cpp
> > > 
> > 
> 
> > > > @@ -18,7 +18,7 @@ int main(int argc,char **argv) {
> > > 
> > 
> 
> > > > int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> > > 
> > 
> 

> > > > PetscInt localIdx[2] = {0, 0};
> > > 
> > 
> 
> > > > - if (rank == 0) {localIdx[0] = 0; localIdx[1] = 1;}
> > > 
> > 
> 
> > > > + if (rank == 0) {localIdx[0] = 0; localIdx[1] = 2;}
> > > 
> > 
> 
> > > > else {localIdx[0] = 1; localIdx[1] = 2;}
> > > 
> > 
> 
> > > > ISLocalToGlobalMapping rmap;
> > > 
> > 
> 
> > > > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 1, &rank,
> > > > PETSC_COPY_VALUES, &rmap);
> > > 
> > 
> 
> > > > diff --git
> > > > a/Graphs/Franck/03.petscDDM/02.petscMailList/matISCSRDenseSquare.cpp
> > > > b/Graphs/Franck/03.petscDDM/02.petscMailList/matISCSRDenseSquare.cpp
> > > 
> > 
> 
> > > > index 4bc6190..4a6ea41 100644
> > > 
> > 
> 
> > > > --- a/matISCSRDenseSquare.cpp
> > > 
> > 
> 
> > > > +++ b/matISCSRDenseSquare.cpp
> > > 
> > 
> 
> > > > @@ -18,7 +18,7 @@ int main(int argc,char **argv) {
> > > 
> > 
> 
> > > > int rank = 0; MPI_Comm_rank(MPI_COMM_WORLD, &rank);
> > > 
> > 
> 

> > > > PetscInt localIdx[2] = {0, 0};
> > > 
> > 
> 
> > > > - if (rank == 0) {localIdx[0] = 0; localIdx[1] = 1;}
> > > 
> > 
> 
> > > > + if (rank == 0) {localIdx[0] = 0; localIdx[1] = 2;}
> > > 
> > 
> 
> > > > else {localIdx[0] = 1; localIdx[1] = 2;}
> > > 
> > 
> 
> > > > ISLocalToGlobalMapping rmap;
> > > 
> > 
> 
> > > > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 2, localIdx,
> > > > PETSC_COPY_VALUES, &rmap);
> > > 
> > 
> 

> > > > > De: "Stefano Zampini" < stefano.zampini at gmail.com >
> > > > 
> > > 
> > 
> 
> > > > > À: "Franck Houssen" < franck.houssen at inria.fr >
> > > > 
> > > 
> > 
> 
> > > > > Cc: "PETSc users list" < petsc-users at mcs.anl.gov >
> > > > 
> > > 
> > 
> 
> > > > > Envoyé: Mardi 20 Juin 2017 00:23:24
> > > > 
> > > 
> > 
> 

> > > > > Objet: Re: [petsc-users] Building MatIS with dense local matrix ?
> > > > 
> > > 
> > 
> 

> > > > > It should be fixed now in maint and master
> > > > 
> > > 
> > 
> 

> > > > > https://bitbucket.org/petsc/petsc/commits/4c8dd594d1988a0cbe282f8a37d9916f61e0c445
> > > > 
> > > 
> > 
> 

> > > > > Thanks for reporting the problem,
> > > > 
> > > 
> > 
> 
> > > > > Stefano
> > > > 
> > > 
> > 
> 

> > > > > > On Jun 19, 2017, at 10:46 PM, Stefano Zampini <
> > > > > > stefano.zampini at gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > Franck,
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > Thanks. I'll get back soon with a fix.
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > Stefano
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > Il 19 Giu 2017 18:17, "Franck Houssen" < franck.houssen at inria.fr >
> > > > > > ha
> > > > > > scritto:
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > The problem was difficult to reduce as reducing make things
> > > > > > > disappear...
> > > > > > > Luckily, I believe I got it (or at least, it looks "like" the one
> > > > > > > I
> > > > > > > "really"
> > > > > > > have...).
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > Seems that for square matrix, it works fine for csr and dense
> > > > > > > matrix.
> > > > > > > But,
> > > > > > > If
> > > > > > > I am not mistaken, it does not for dense rectangular matrix
> > > > > > > (still
> > > > > > > OK
> > > > > > > for
> > > > > > > csr).
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > matISCSRDenseSquare.cpp: 2 procs, global 3x3 matrix, each proc
> > > > > > > adds
> > > > > > > a
> > > > > > > 2x2
> > > > > > > local matrix in the global matrix.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > matISCSRDenseRect.cpp: 2 procs, global 2 x3 matrix, each proc
> > > > > > > adds
> > > > > > > a
> > > > > > > 1
> > > > > > > x2
> > > > > > > local vector in the global matrix.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > reminder: running debian/testing with gcc-6.3 + petsc-3.7.6
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > Franck
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > >> mpirun -n 2 ./matISCSRDenseSquare.exe csr; mpirun -n 2
> > > > > > > >> ./matISCSRDenseSquare.exe dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > csr
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > csr
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 2 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: is
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqaij
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 0: (0, 1.) (1, 0.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 1: (0, 0.) (1, 1.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqaij
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 0: (0, 1.) (1, 0.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 1: (0, 0.) (1, 1.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 2 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: is
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqdense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 1.0000000000000000e+00 0.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 0.0000000000000000e+00 1.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqdense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 1.0000000000000000e+00 0.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 0.0000000000000000e+00 1.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > >> mpirun -n 2 ./matISCSRDenseRect.exe csr; mpirun -n 2
> > > > > > > >> ./matISCSRDenseRect.exe dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > csr
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > csr
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 2 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: is
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqaij
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 0: (0, 1.) (1, 0.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqaij
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > row 0: (0, 1.) (1, 0.)
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > dense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: --------------------- Error Message
> > > > > > > --------------------------------------------------------------
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: Argument out of range
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error
> > > > > > > Message
> > > > > > > --------------------------------------------------------------
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: Argument out of range
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: New nonzero at (0,1) caused a malloc
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE)
> > > > > > > to
> > > > > > > turn
> > > > > > > off
> > > > > > > this check
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: See
> > > > > > > http://www.mcs.anl.gov/petsc/documentation/faq.html
> > > > > > > for
> > > > > > > trouble shooting.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: Petsc Release Version 3.7.6, Apr, 24, 2017
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: ./matISCSRDenseRect.exe on a arch-linux2-c-debug
> > > > > > > named
> > > > > > > yoda
> > > > > > > by fghoussen Mon Jun 19 18:08:58 2017
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > New nonzero at (0,1) caused a malloc
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE)
> > > > > > > to
> > > > > > > turn
> > > > > > > off
> > > > > > > this check
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: See
> > > > > > > http://www.mcs.anl.gov/petsc/documentation/faq.html
> > > > > > > for
> > > > > > > trouble shooting.
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: Petsc Release Version 3.7.6, Apr, 24, 2017
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: Configure options
> > > > > > > --prefix=/home/fghoussen/Documents/INRIA/petsc-3.7.6/local
> > > > > > > --with-mpi=1
> > > > > > > --with-pthread=1 --download-f2cblaslapack=yes
> > > > > > > --download-mumps=yes
> > > > > > > --download-scalapack=yes --download-superlu=yes
> > > > > > > --download-suitesparse=yes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 616 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/aij/mpi/mpiaij.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: #2 MatSetValues() line 1190 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: #3 MatSetValuesLocal() line 2053 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: #4 MatISGetMPIXAIJ_IS() line 365 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [1]PETSC ERROR: #5 MatISGetMPIXAIJ() line 437 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: ./matISCSRDenseRect.exe on a arch-linux2-c-debug
> > > > > > > named
> > > > > > > yoda
> > > > > > > by fghoussen Mon Jun 19 18:08:58 2017
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: Configure options
> > > > > > > --prefix=/home/fghoussen/Documents/INRIA/petsc-3.7.6/local
> > > > > > > --with-mpi=1
> > > > > > > --with-pthread=1 --download-f2cblaslapack=yes
> > > > > > > --download-mumps=yes
> > > > > > > --download-scalapack=yes --download-superlu=yes
> > > > > > > --download-suitesparse=yes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: #1 MatSetValues_MPIAIJ() line 582 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/aij/mpi/mpiaij.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: #2 MatSetValues() line 1190 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: #3 MatSetValuesLocal() line 2053 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/interface/matrix.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: #4 MatISGetMPIXAIJ_IS() line 365 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > [0]PETSC ERROR: #5 MatISGetMPIXAIJ() line 437 in
> > > > > > > /home/fghoussen/Documents/INRIA/petsc-3.7.6/src/mat/impls/is/matis.c
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 2 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: is
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqdense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 1.0000000000000000e+00 0.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Mat Object: 1 MPI processes
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > type: seqdense
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 1.0000000000000000e+00 0.0000000000000000e+00
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > >> diff matISCSRDenseSquare.cpp matISCSRDenseRect.cpp
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 3c3
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < // ~> g++ -o matISCSRDenseSquare.exe matISCSRDenseSquare.cpp
> > > > > > > -lpetsc
> > > > > > > -lm;
> > > > > > > mpirun -n 2 matISCSRDenseSquare.exe
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > ---
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > // ~> g++ -o matISCSRDenseRect.exe matISCSRDenseRect.cpp
> > > > > > > > -lpetsc
> > > > > > > > -lm;
> > > > > > > > mpirun -n 2 matISCSRDenseRect.exe
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 24c24
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 2, localIdx,
> > > > > > > PETSC_COPY_VALUES, &rmap);
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > ---
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > ISLocalToGlobalMappingCreate(PETSC_COMM_WORLD, 1, 1, &rank,
> > > > > > > > PETSC_COPY_VALUES, &rmap);
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 29c29
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, 3,
> > > > > > > 3,
> > > > > > > rmap,
> > > > > > > cmap, &A);
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > ---
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > MatCreateIS(PETSC_COMM_WORLD, 1, PETSC_DECIDE, PETSC_DECIDE, 2,
> > > > > > > > 3,
> > > > > > > > rmap,
> > > > > > > > cmap, &A);
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 32,33c32,33
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < if (matType == "csr") {cout << matType << endl;
> > > > > > > MatCreateSeqAIJ(PETSC_COMM_SELF, 2, 2, 2, NULL, &Aloc);}
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < else {cout << matType << endl;
> > > > > > > MatCreateSeqDense(PETSC_COMM_SELF,
> > > > > > > 2,
> > > > > > > 2,
> > > > > > > NULL, &Aloc);}
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > ---
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > if (matType == "csr") {cout << matType << endl;
> > > > > > > > MatCreateSeqAIJ(PETSC_COMM_SELF, 1, 2, 2, NULL, &Aloc);}
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > else {cout << matType << endl;
> > > > > > > > MatCreateSeqDense(PETSC_COMM_SELF,
> > > > > > > > 1,
> > > > > > > > 2,
> > > > > > > > NULL, &Aloc);}
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > 35,36c35,36
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < PetscScalar localVal[4] = {1., 0., 0., 1.};
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > < MatSetValues(Aloc, 2, localIdx, 2, localIdx, localVal,
> > > > > > > ADD_VALUES);
> > > > > > > //
> > > > > > > Add
> > > > > > > local 2x2 matrix
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > ---
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > PetscScalar localVal[2] = {1., 0.}; PetscInt oneLocalRow = 0;
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > MatSetValues(Aloc, 1, &oneLocalRow, 2, localIdx, localVal,
> > > > > > > > ADD_VALUES);
> > > > > > > > //
> > > > > > > > Add local row
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > De: "Stefano Zampini" < stefano.zampini at gmail.com >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > À: "Franck Houssen" < franck.houssen at inria.fr >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Cc: "PETSc users list" < petsc-users at mcs.anl.gov >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Envoyé: Lundi 19 Juin 2017 15:25:35
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Objet: Re: [petsc-users] Building MatIS with dense local matrix
> > > > > > > > ?
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > Can you send a minimal working example so that I can fix the
> > > > > > > > code?
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > Thanks
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Stefano
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > Il 19 Giu 2017 15:20, "Franck Houssen" <
> > > > > > > > franck.houssen at inria.fr
> > > > > > > > >
> > > > > > > > ha
> > > > > > > > scritto:
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > Hi,
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > I try to call MatISGetMPIXAIJ on a MatIS (A) that has been
> > > > > > > > > feed
> > > > > > > > > locally
> > > > > > > > > by
> > > > > > > > > sequential (Aloc) dense matrix.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > Seems this ends up with this error: [0]PETSC ERROR: New
> > > > > > > > > nonzero
> > > > > > > > > at
> > > > > > > > > (0,1)
> > > > > > > > > caused a malloc. Is this a known error / limitation ? (not
> > > > > > > > > supposed
> > > > > > > > > to
> > > > > > > > > work
> > > > > > > > > with dense matrix ?)
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > This (pseudo code) works fine:
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > MatCreateIS(..., A)
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > MatCreateSeqAIJ(..., Aloc)
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > MatISSetLocalMat(pcA, pcALoc)
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > MatISGetMPIXAIJ(A, ...) // OK !
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > When I try to replace MatCreateSeqAIJ(..., Aloc) with
> > > > > > > > > MatCreateSeqDense(...,
> > > > > > > > > Aloc), it does no more work.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > Franck
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > PS: running debian/testing with gcc-6.3 + petsc-3.7.6
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > --
> > 
> 
> > > Stefano
> > 
> 

> --
> Stefano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170620/c1a92fb8/attachment-0001.html>