On Mon, Jan 18, 2010 at 7:53 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="h5">On Mon, Jan 18, 2010 at 7:45 AM, Jed Brown <span dir="ltr"><<a href="mailto:jed@59a2.org" target="_blank">jed@59a2.org</a>></span> wrote:<br></div></div><div class="gmail_quote"><div>
<div></div><div class="h5"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I thought this used to work:<br>
<br>
$ cd petsc/src/ksp/ksp/examples/tutorials/<br>
$ ./ex2 -pc_type asm -pc_asm_blocks 2<br>
[0]PETSC ERROR: --------------------- Error Message ------------------------------------<br>
[0]PETSC ERROR: Petsc has generated inconsistent data!<br>
[0]PETSC ERROR: Specified ASM subdomain sizes were invalid: 112 != 56!<br>
[0]PETSC ERROR: ------------------------------------------------------------------------<br>
[0]PETSC ERROR: Petsc Development HG revision: c48026146083ef288c8218540e6f61b678c1c226 HG Date: Sun Jan 17 17:16:57 2010 +0100<br>
[0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
[0]PETSC ERROR: See docs/index.html for manual pages.<br>
[0]PETSC ERROR: ------------------------------------------------------------------------<br>
[0]PETSC ERROR: ./ex2 on a ompi named kunyang by jed Mon Jan 18 14:39:15 2010<br>
[0]PETSC ERROR: Libraries linked from /home/jed/petsc/ompi/lib<br>
[0]PETSC ERROR: Configure run at Sun Jan 17 17:20:32 2010<br>
[0]PETSC ERROR: Configure options --with-zoltan-dir=/usr --download-ml --with-blas-lapack-dir=/usr --download-blacs --download-chaco --with-hdf5-dir=/usr --download-mumps --download-superlu_dist --download-spooles --download-sundials --download-hypre --with-c2html --with-mpi-dir=/usr --with-umfpack-dir=/usr --with-parmetis-dir=/usr --download-scalapack --with-lgrind --with-shared --with-sowing -PETSC_ARCH=ompi --download-superlu --download-spai<br>
[0]PETSC ERROR: ------------------------------------------------------------------------<br>
[0]PETSC ERROR: PCSetUp_ASM() line 239 in src/ksp/pc/impls/asm/asm.c<br>
[0]PETSC ERROR: PCSetUp() line 795 in src/ksp/pc/interface/precon.c<br>
[0]PETSC ERROR: KSPSetUp() line 237 in src/ksp/ksp/interface/itfunc.c<br>
[0]PETSC ERROR: KSPSolve() line 353 in src/ksp/ksp/interface/itfunc.c<br>
[0]PETSC ERROR: main() line 196 in src/ksp/ksp/examples/tutorials/ex2.c<br>
--------------------------------------------------------------------------<br>
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD<br>
with errorcode 77.<br>
<br>
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.<br>
You may or may not see output from other processes, depending on<br>
exactly when Open MPI kills them.<br>
--------------------------------------------------------------------------<br>
<br>
<br>
Looking at the loop that is generating this inconsistency, osm->is_local<br>
is NULL, so we take the second branch for each block:<br>
<br>
if (osm->is_local) {<br>
...<br>
} else {<br>
ierr = VecGetLocalSize(vec,&m_local);CHKERRQ(ierr);<br>
osm->y_local[i] = osm->y[i];<br>
ierr = PetscObjectReference((PetscObject) osm->y[i]);CHKERRQ(ierr);<br>
osm->prolongation[i] = osm->restriction[i];<br>
ierr = PetscObjectReference((PetscObject) osm->restriction[i]);CHKERRQ(ierr);<br>
}<br>
<br>
But vec is everything owned by that process so m_local is too big unless<br>
there is only one subdomain. What was this code supposed to be doing?<font color="#888888"><br></font></blockquote></div></div><div><br>I had to change ASM back in July? to get multiple overlapping blocks on a given parallel subdomain.<br>
<br>Why?<br><br> We used a particularly simple mechanism to implement RASM, namely dropping off-process values.<br>That does not work when you have 2+ domains on 1 process.<br><br> I needed this functionality for our RBF preconditioner<br>
<br>How?<br><br> I added another IS that gave the local part of each block, since we already know the blocks themselves.<br>It is taking the right branch here since no extra IS is specified.<br><br>What is wrong?<br><br>
I do not understand the comment. The local size should be right.<br></div></div></blockquote><div><br>Okay, I just tried it and I understand the problem: This never really worked. RASM was not doing the proper thing.<br>
Now it does, but we need to correct the code to produce is_local automatically with this argument.<br><br> Matt<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="gmail_quote"><div> Matt<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font color="#888888">
Jed<br>
</font></blockquote></div><div><div></div><div class="h5">-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>