Reuse matrix and vector

[Jed Brown]

jarunan at ascomp.ch wrote:
> Total number of cells is 744872, divided into 40 blocks. In one
> processor, MatCreateMPIAIJWithArrays() takes 0.097 sec but 280 sec with
> 4 processors. Usually, this routine has no problem with small test case.
> It works the same for one or more than one processors.

This sounds like incorrect preallocation.  Is your PETSc built with
debugging?  Debug does some extra integrity checks that don't add
significantly to the time (although other Debug checks do), but it would
be useful to know that they pass.  In particular, it checks that your
rows are sorted.  If they are not sorted then PETSc's preallocation
would be wrong.  (I actually don't think this requirement enables
significantly faster implementation, so I'm tempted to change it to work
correctly with unsorted rows.)

You can also run with -info |grep malloc, there should be no mallocs in
MatSetValues().

> in the first iteration.
>     Mat Ap
> 
>     call MatCreateMPIAIJWithArrays(PETSC_COMM_WORLD, istorf_no_ovcell,         &
>       istorf_no_ovcell, PETSC_DETERMINE, PETSC_DETERMINE, rowind, columnind,   &
>       A, Ap, ierr)
>
>     call MatAssemblyBegin(Ap,MAT_FINAL_ASSEMBLY,ierr)
>     call MatAssemblyEnd(Ap,MAT_FINAL_ASSEMBLY,ierr)

This assembly is superfluous (but harmless).

> Does the communication of MatCreateMPIAIJWithArrays() in parallel
> computation cost a lot? What could be the cause that
> MatCreateMPIAIJWithArrays() so slow in the first iteration?

There is no significant communication, it has to be preallocation.

Jed

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20091110/1f558f0e/attachment.pgp>