On Sat, Nov 5, 2011 at 12:10 AM, Brad Aagaard <span dir="ltr"><<a href="mailto:baagaard@usgs.gov">baagaard@usgs.gov</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Jed,<br>
<br>
It looks like your changeset 20985 may have introduced a bug. I just pulled and rebuilt petsc-dev. I am trying to run a PyLith simulation that worked in parallel with ASM, sbaij matrix, and bs=1 a week ago but now fails. I get a memory allocation error (huge allocation) or a memory corruption error. Using an aij matrix or running in serial works fine. My simulation runs fine with 2 processes but not 3 or 4. The stack traces are<br>
</blockquote><div><br></div><div>I am trying to understand, but having problems.</div><div><br></div><div>1) 20985 does not change anything here. Do you mean 20989?</div><div><br></div><div>2) The changes that 20989 makes are only to multiply in two place by C->rmap->bs. Now maybe that is screwed up.</div>
<div><br></div><div>3) I can't see memory allocation anywhere near here:</div><div><br></div><div>line 929</div><div><div> if ((ctmp = bmap[cworkB[l]]) < cstart) cols[l] = ctmp;</div></div><div>line 933</div>
<div><div> for (l=0; l<nzA; l++) cols[imark+l] = cstart + cworkA[l];</div></div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
NPROC=4 (memory corruption)<br>
#0 0x00007f014a93bb00 in MatGetSubMatrices_MPIBAIJ_<u></u>local (C=0x29560a0,<br>
ismax=1, isrow=0x2a06520, iscol=0x29ff6f0, scall=MAT_INITIAL_MATRIX,<br>
allrows=0x2a05db0, allcolumns=0x2a00ea0, submats=0x2a0ab00)<br>
at /tools/common/petsc-dev/src/<u></u>mat/impls/baij/mpi/baijov.c:<u></u>933<br>
#1 0x00007f014aacd73d in MatIncreaseOverlap_MPISBAIJ (C=0x29560a0, is_max=1,<br>
is=0x29fb110, ov=1)<br>
at /tools/common/petsc-dev/src/<u></u>mat/impls/sbaij/mpi/sbaijov.c:<u></u>85<br>
#2 0x00007f014ac476db in MatIncreaseOverlap (mat=0x29560a0, n=1,<br>
is=0x29fb110, ov=1)<br>
at /tools/common/petsc-dev/src/<u></u>mat/interface/matrix.c:6669<br>
#3 0x00007f014b016df6 in PCSetUp_ASM (pc=0x299d2f0)<br>
at /tools/common/petsc-dev/src/<u></u>ksp/pc/impls/asm/asm.c:199<br>
<br>
NPROC=3 (request for huge amount of memory allocation)<br>
#0 0x00007f67d30bca75 in MatGetSubMatrices_MPIBAIJ_<u></u>local (C=0x2cc8430,<br>
ismax=1, isrow=0x2eb4b70, iscol=0x2eadaf0, scall=MAT_INITIAL_MATRIX,<br>
allrows=0x2eb4400, allcolumns=0x2eaf2a0, submats=0x2eb9150)<br>
at /tools/common/petsc-dev/src/<u></u>mat/impls/baij/mpi/baijov.c:<u></u>929<br>
#1 0x00007f67d324e73d in MatIncreaseOverlap_MPISBAIJ (C=0x2cc8430, is_max=1,<br>
is=0x2ea9510, ov=1)<br>
at /tools/common/petsc-dev/src/<u></u>mat/impls/sbaij/mpi/sbaijov.c:<u></u>85<br>
#2 0x00007f67d33c86db in MatIncreaseOverlap (mat=0x2cc8430, n=1,<br>
is=0x2ea9510, ov=1)<br>
at /tools/common/petsc-dev/src/<u></u>mat/interface/matrix.c:6669<br>
#3 0x00007f67d3797df6 in PCSetUp_ASM (pc=0x2d0cdf0)<br>
at /tools/common/petsc-dev/src/<u></u>ksp/pc/impls/asm/asm.c:199<br>
#4 0x00007f67d36c43a5 in PCSetUp (pc=0x2d0cdf0)<br>
<br>
Thanks,<br><font color="#888888">
Brad<br>
</font></blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>