[petsc-dev] horrid bug

[Mark F. Adams]

I've got a bug with block sizes in multigrid that is getting harry and wanted to see if I could get some help.

The crux of the problem seems to be that different versions of VecScatterEnd are being called.  I've added a print statement in:

PetscErrorCode PETSCMAP1(VecScatterEnd)(VecScatter ctx,Vec xin,Vec yin,InsertMode addv,ScatterMode mode)

in vpscat.h. and with a two processor run see this:

*****[0]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1
*****[1]VecScatterEnd_3 start, N=27 Xin.bs=3 yin.bs=1
*****[0]VecScatterEnd_2 start, N=27 Xin.bs=3 yin.bs=1
*****[1]VecScatterEnd_1 start, N=27 Xin.bs=3 yin.bs=1
[1]PETSC ERROR: VecScatterEnd_1() line 217 in /Users/markadams/Codes/petsc-dev/include/../src/vec/vec/utils/vpscat.h
[1]PETSC ERROR: VecScatterEnd() line 1620 in /Users/markadams/Codes/petsc-dev/src/vec/vec/utils/vscat.c
[1]PETSC ERROR: MatMultAdd_MPIAIJ() line 1118 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c
[1]PETSC ERROR: MatMultAdd() line 2372 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c
[1]PETSC ERROR: MatInterpolateAdd() line 7439 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c
[1]PETSC ERROR: PCMGMCycle_Private() line 52 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/mg/mg.c


This is a two level solve and the coarse grid is all on proc 0 and has a block size of 3. The fine grid has a block size of 2.  THis error, as you can see, comes from the interpolation from coarse to fine.  You can see from this output that the code called VecScatterEnd_3 successfully (the coarse grid) but when it does the interpolation proc 0 calls VecScatterEnd_2 (the destination is block size 2) and proc 1 calls VecScatterEnd_1 -- clearly wrong.

Anyway, I'll keep digging, but any ideas would be welcome.

Mark