<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Thanks, I think I found it and pushed to solution. The problem was that the row block sizes where used to set the scatter maps in MatMult where column block size is needed. This splitting the row and column block sizes will probably cause more problems. This bug was only a fatal problem when the two block sizes where not multiples of each other (so my 3D problems worked fine).<div><br></div><div><div>Mark</div><div><div><br></div><div><div><div>On May 18, 2012, at 1:50 PM, Dmitry Karpeev wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">The problem is most likely in the creation routine or, rather, in the arguments passed into it:<div>is it possible that the Vecs fed into VecScatterCreate have different block sizes on different procs?</div><div><br></div>
<div>Dmitry.<br><br><div class="gmail_quote">On Fri, May 18, 2012 at 7:20 AM, Mark F. Adams <span dir="ltr"><<a href="mailto:mark.adams@columbia.edu" target="_blank">mark.adams@columbia.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I've got a bug with block sizes in multigrid that is getting harry and wanted to see if I could get some help.<br>
<br>
The crux of the problem seems to be that different versions of VecScatterEnd are being called. I've added a print statement in:<br>
<br>
PetscErrorCode PETSCMAP1(VecScatterEnd)(VecScatter ctx,Vec xin,Vec yin,InsertMode addv,ScatterMode mode)<br>
<br>
in vpscat.h. and with a two processor run see this:<br>
<br>
*****[0]VecScatterEnd_3 start, N=27 Xin.bs=3 <a href="http://yin.bs/" target="_blank">yin.bs</a>=1<br>
*****[1]VecScatterEnd_3 start, N=27 Xin.bs=3 <a href="http://yin.bs/" target="_blank">yin.bs</a>=1<br>
*****[0]VecScatterEnd_2 start, N=27 Xin.bs=3 <a href="http://yin.bs/" target="_blank">yin.bs</a>=1<br>
*****[1]VecScatterEnd_1 start, N=27 Xin.bs=3 <a href="http://yin.bs/" target="_blank">yin.bs</a>=1<br>
[1]PETSC ERROR: VecScatterEnd_1() line 217 in /Users/markadams/Codes/petsc-dev/include/../src/vec/vec/utils/vpscat.h<br>
[1]PETSC ERROR: VecScatterEnd() line 1620 in /Users/markadams/Codes/petsc-dev/src/vec/vec/utils/vscat.c<br>
[1]PETSC ERROR: MatMultAdd_MPIAIJ() line 1118 in /Users/markadams/Codes/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c<br>
[1]PETSC ERROR: MatMultAdd() line 2372 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c<br>
[1]PETSC ERROR: MatInterpolateAdd() line 7439 in /Users/markadams/Codes/petsc-dev/src/mat/interface/matrix.c<br>
[1]PETSC ERROR: PCMGMCycle_Private() line 52 in /Users/markadams/Codes/petsc-dev/src/ksp/pc/impls/mg/mg.c<br>
<br>
<br>
This is a two level solve and the coarse grid is all on proc 0 and has a block size of 3. The fine grid has a block size of 2. THis error, as you can see, comes from the interpolation from coarse to fine. You can see from this output that the code called VecScatterEnd_3 successfully (the coarse grid) but when it does the interpolation proc 0 calls VecScatterEnd_2 (the destination is block size 2) and proc 1 calls VecScatterEnd_1 -- clearly wrong.<br>
<br>
Anyway, I'll keep digging, but any ideas would be welcome.<br>
<span class="HOEnZb"><font color="#888888"><br>
Mark<br>
<br>
</font></span></blockquote></div><br></div>
</blockquote></div><br></div></div></div></body></html>