[petsc-dev] Fwd: [petsc-users] Local size 7 not compatible with block size 13

Wed Jan 15 16:36:29 CST 2014

   This problem comes up for MPIAIJ matrices when the “off-diagona” matrix is compressed to only the  columns that have non zeros in them. When a MatDuplicate() is done on this it detects the inconsistency that the number of local columns may no longer be divisible by the block size. We could handle this by

1) when the matrix is compressed (removing empty columns) we just mark the block size to 1 if the new number of local columns is not divisible by the block size 

but then if the more values are put into the matrix when the matrix is uncompressed it no longer has the block size > 1 (possibly hurting efficiency) so we could

2) stash the old block size in the matrix and put it back when we uncompress it.

What do you think? Should I do 1)? 2)?

   Barry 

Request-assigned: Barry   fix MatDuplicate() problem on MPIAIJ matrices where block size of off diagonal matrix is no longer valid when compressed

Begin forwarded message:

> From: "Jay J. Billings" <billingsjj at ornl.gov>
> Subject: [petsc-users] Local size 7 not compatible with block size 13
> Date: January 15, 2014 at 1:14:02 PM CST
> To: <petsc-users at mcs.anl.gov>
> 
> Everyone,
> 
> I'm trying to run PETSc in parallel and I'm getting an error I haven't seen before that the local and block size arguments don't match. Can anyone point me in the right direction on this? It happens right after I call TSSolve and, according to the backtrace, when TSSolve is trying to form the Jacobian. It doesn't look like it is actually calling my Jacobian routine, although I could be wrong about that.
> 
> Jay
> 
> [1]PETSC ERROR: --------------------- Error Message ------------------------------------
> [1]PETSC ERROR: [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> [0]PETSC ERROR: Arguments are incompatible!
> [1]PETSC ERROR: Arguments are incompatible!
> [0]PETSC ERROR: Local size 7 not compatible with block size 13!
> [1]PETSC ERROR: Local size 7 not compatible with block size 13!
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: Petsc Development GIT revision: v3.4.3-2305-g52cd893  GIT Date: 2014-01-13 09:37:41 -0500
> [1]PETSC ERROR: See docs/changes/index.html for recent updates.
> [1]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [1]PETSC ERROR: See docs/index.html for manual pages.
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: ./xolotl on a arch-linux2-c-opt named box.ornl.gov by jay Wed Jan 15 14:12:23 2014
> [1]PETSC ERROR: Libraries linked from /opt/petsc-latest_mpich-3.0.1/lib
> [1]PETSC ERROR: Configure run at Mon Jan 13 11:11:18 2014
> [1]PETSC ERROR: Configure options --prefix=/opt/petsc-latest_mpich-3.0.1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif77 --with-debugging=no --download-f-blas-lapack=1 --FOPTFLAGS= --with-shared-libraries=1 --download-hypre=yes --with-debugging=0
> [1]PETSC ERROR: ------------------------------------------------------------------------
> [1]PETSC ERROR: PetscLayoutSetBlockSize() line 464 in /home/jay/Programs/petsc/src/vec/is/utils/pmap.c
> [1]PETSC ERROR: MatSetBlockSizes() line 6772 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [1]PETSC ERROR: MatDuplicate_SeqAIJ() line 4158 in /home/jay/Programs/petsc/src/mat/impls/aij/seq/aij.c
> [1]PETSC ERROR: MatDuplicate() line 4116 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [1]PETSC ERROR: MatDuplicate_MPIAIJ() line 3395 in /home/jay/Programs/petsc/src/mat/impls/aij/mpi/mpiaij.c
> [1]PETSC ERROR: MatDuplicate() line 4116 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [1]PETSC ERROR: TSGetRHSMats_Private() line 605 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [1]PETSC ERROR: TSComputeIJacobian() line 782 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [1]PETSC ERROR: SNESTSFormJacobian_ARKIMEX() line 1056 in /home/jay/Programs/petsc/src/ts/impls/arkimex/arkimex.c
> [1]PETSC ERROR: SNESTSFormJacobian() line 3541 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [1]PETSC ERROR: SNESComputeJacobian() line 2253 in /home/jay/Programs/petsc/src/snes/interface/snes.c
> [1]PETSC ERROR: SNESSolve_NEWTONLS() line 231 in /home/jay/Programs/petsc/src/snes/impls/ls/ls.c
> [1]PETSC ERROR: SNESSolve() line 3812 in /home/jay/Programs/petsc/src/snes/interface/snes.c
> [1]PETSC ERROR: TSStep_ARKIMEX() line 776 in /home/jay/Programs/petsc/src/ts/impls/arkimex/arkimex.c
> [1]PETSC ERROR: TSStep() line 2625 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [1]PETSC ERROR: TSSolve() line 2741 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [1]PETSC ERROR: checkPetscError() line 65 in /home/jay/research/xolotl/xolotl_workspace/xolotl-Source at xolotl/xolotlSolver/PetscSolver.cpp
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development GIT revision: v3.4.3-2305-g52cd893  GIT Date: 2014-01-13 09:37:41 -0500
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: ./xolotl on a arch-linux2-c-opt named box.ornl.gov by jay Wed Jan 15 14:12:23 2014
> [0]PETSC ERROR: Libraries linked from /opt/petsc-latest_mpich-3.0.1/lib
> [0]PETSC ERROR: Configure run at Mon Jan 13 11:11:18 2014
> [0]PETSC ERROR: Configure options --prefix=/opt/petsc-latest_mpich-3.0.1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif77 --with-debugging=no --download-f-blas-lapack=1 --FOPTFLAGS= --with-shared-libraries=1 --download-hypre=yes --with-debugging=0
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: PetscLayoutSetBlockSize() line 464 in /home/jay/Programs/petsc/src/vec/is/utils/pmap.c
> [0]PETSC ERROR: MatSetBlockSizes() line 6772 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [0]PETSC ERROR: MatDuplicate_SeqAIJ() line 4158 in /home/jay/Programs/petsc/src/mat/impls/aij/seq/aij.c
> [0]PETSC ERROR: MatDuplicate() line 4116 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [0]PETSC ERROR: MatDuplicate_MPIAIJ() line 3395 in /home/jay/Programs/petsc/src/mat/impls/aij/mpi/mpiaij.c
> [0]PETSC ERROR: MatDuplicate() line 4116 in /home/jay/Programs/petsc/src/mat/interface/matrix.c
> [0]PETSC ERROR: TSGetRHSMats_Private() line 605 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [0]PETSC ERROR: TSComputeIJacobian() line 782 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [0]PETSC ERROR: SNESTSFormJacobian_ARKIMEX() line 1056 in /home/jay/Programs/petsc/src/ts/impls/arkimex/arkimex.c
> [0]PETSC ERROR: SNESTSFormJacobian() line 3541 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [0]PETSC ERROR: SNESComputeJacobian() line 2253 in /home/jay/Programs/petsc/src/snes/interface/snes.c
> [0]PETSC ERROR: SNESSolve_NEWTONLS() line 231 in /home/jay/Programs/petsc/src/snes/impls/ls/ls.c
> [0]PETSC ERROR: SNESSolve() line 3812 in /home/jay/Programs/petsc/src/snes/interface/snes.c
> [0]PETSC ERROR: TSStep_ARKIMEX() line 776 in /home/jay/Programs/petsc/src/ts/impls/arkimex/arkimex.c
> [0]PETSC ERROR: TSStep() line 2625 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [0]PETSC ERROR: TSSolve() line 2741 in /home/jay/Programs/petsc/src/ts/interface/ts.c
> [0]PETSC ERROR: checkPetscError() line 65 in /home/jay/research/xolotl/xolotl_workspace/xolotl-Source at xolotl/xolotlSolver/PetscSolver.cpp
> 
> -- 
> Jay Jay Billings
> Oak Ridge National Laboratory
> Twitter Handle: @jayjaybillings
> 