[petsc-dev] petsc-dev: ASM with sbaij matrix and block size=1 broken?

[Brad Aagaard]

Matt and Jed,

The error is in our inner solve (associated with computing the 
adjustment to the solution based on the friction criterion) for cases 
where not all processors have points on the fault. On processors without 
points on the fault the block size is -1 and the local size is 0. For 
processors with points on the fault, the block size is 3.

I tried altering DMMeshCreateMatrix so that it uses a block size of 1 if 
bs==-1, but I still get the error. Does the block size need to be 
consistent across all processors, even those a local size of 0? 
Alternatively, is the local size of 0 causing problems?

Matt- You can reproduce this problem using the pylith-stable branch:
cd tests/3d/cyclicfriction
pylith --nodes=3

Brad


On 11/5/11 5:26 AM, Matthew Knepley wrote:
> On Sat, Nov 5, 2011 at 5:20 AM, Brad Aagaard <baagaard at usgs.gov
> <mailto:baagaard at usgs.gov>> wrote:
>
>     Jed,
>
>     I assume you mean src/ksp/ksp/examples/__tutorials/ex2
>     I have not yet been able to duplicate the error with this example.
>     Do you know of any other test cases that might be appropriate?
>
>     My PyLith simulation with -malloc and -malloc_debug flags chokes on
>     line 933 of src/mat/impls/baij/mpi/__baijjob.c with a EXEC_BAD_ACCESS.
>
>
>     933               for (l=0; l<nzA; l++)   cols[imark+l] = cstart +
>     cworkA[l];
>
>     When the memory error happens at this line, the debugger shows
>     l = 1400100
>     imark = 0
>     cstart = 0
>     cworkA[l] = 0
>     cols[imark+l] = -17958193
>
>     The address of the bad memory access matches the address of
>     cols[imark+l].
>
>
> Brad, can you check C->rmap->bs when this happens? Since this is the
> only thing that changes, I am guessing that
> it might be 0, and then you can get the wrong 'proc' which cascades. Can
> I run this here?
>
>    Thanks,
>
>        Matt
>
>     Brad
>
>
>
>     On 11/4/11 8:39 PM, Jed Brown wrote:
>
>         On Fri, Nov 4, 2011 at 21:30, Brad Aagaard <baagaard at usgs.gov
>         <mailto:baagaard at usgs.gov>
>         <mailto:baagaard at usgs.gov <mailto:baagaard at usgs.gov>>> wrote:
>
>             Barry,
>
>             Sorry for the hg goof. Here is the changeset that I think is
>         related
>             to the problem.
>
>         http://petsc.cs.iit.edu/petsc/____petsc-dev/rev/7782710e3f96
>         <http://petsc.cs.iit.edu/petsc/__petsc-dev/rev/7782710e3f96>
>
>         <http://petsc.cs.iit.edu/__petsc/petsc-dev/rev/__7782710e3f96
>         <http://petsc.cs.iit.edu/petsc/petsc-dev/rev/7782710e3f96>>
>
>
>         If bs=1, then this doesn't change anything. As far as I know, it
>         is not
>         possible to create an SBAIJ(1) with map->bs != 1. Can you find a
>         way for
>         me to reproduce this? I see no problem with this, for example:
>
>         mpiexec -n 4 ./ex2 -mat_type sbaij -pc_type asm
>
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener