Reuse matrix and vector

[Jed Brown]

Matthew Knepley wrote:
> On Tue, Nov 10, 2009 at 4:51 AM, Jed Brown <jed at 59a2.org
> <mailto:jed at 59a2.org>> wrote:
> 
>     jarunan at ascomp.ch <mailto:jarunan at ascomp.ch> wrote:
>     > Total number of cells is 744872, divided into 40 blocks. In one
>     > processor, MatCreateMPIAIJWithArrays() takes 0.097 sec but 280 sec
>     with
>     > 4 processors. Usually, this routine has no problem with small test
>     case.
>     > It works the same for one or more than one processors.
> 
>     This sounds like incorrect preallocation.  Is your PETSc built with
>     debugging?  Debug does some extra integrity checks that don't add
>     significantly to the time (although other Debug checks do), but it would
>     be useful to know that they pass.  In particular, it checks that your
>     rows are sorted.  If they are not sorted then PETSc's preallocation
>     would be wrong.  (I actually don't think this requirement enables
>     significantly faster implementation, so I'm tempted to change it to work
>     correctly with unsorted rows.)
> 
> 
> I do not think its preallocation per se, since 1 proc is fast. I think
> that your partition of rows fed to the MatCreate() call does not match
> what you provide to MatSetValues() and thus you do a lot of
> communication in MatAssemblyEnd().  There are 2 ways to debug this:

Matt, he says MatSetValues() is fast, but MatCreateMPIAIJWithArrays() is
slow.  Look how preallocation is done (mpiaij.c:3263).  This would do
the correct thing in serial, but be under-allocate the diagonal part
when the rows are not sorted.  It looks like a silly "optimization" to
me.

Jed

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20091110/c2355211/attachment.pgp>