Reuse matrix and vector

Matthew Knepley knepley at
Tue Nov 10 05:50:22 CST 2009

On Tue, Nov 10, 2009 at 4:51 AM, Jed Brown <jed at> wrote:

> jarunan at wrote:
> > Total number of cells is 744872, divided into 40 blocks. In one
> > processor, MatCreateMPIAIJWithArrays() takes 0.097 sec but 280 sec with
> > 4 processors. Usually, this routine has no problem with small test case.
> > It works the same for one or more than one processors.
> This sounds like incorrect preallocation.  Is your PETSc built with
> debugging?  Debug does some extra integrity checks that don't add
> significantly to the time (although other Debug checks do), but it would
> be useful to know that they pass.  In particular, it checks that your
> rows are sorted.  If they are not sorted then PETSc's preallocation
> would be wrong.  (I actually don't think this requirement enables
> significantly faster implementation, so I'm tempted to change it to work
> correctly with unsorted rows.)

I do not think its preallocation per se, since 1 proc is fast. I think that
partition of rows fed to the MatCreate() call does not match what you
to MatSetValues() and thus you do a lot of communication in
There are 2 ways to debug this:

  1)  -log_summary to see where the time is spent



You can also run with -info |grep malloc, there should be no mallocs in
> MatSetValues().
> > in the first iteration.
> >     Mat Ap
> >
> >     call MatCreateMPIAIJWithArrays(PETSC_COMM_WORLD, istorf_no_ovcell,
>       &
> >       istorf_no_ovcell, PETSC_DETERMINE, PETSC_DETERMINE, rowind,
> columnind,   &
> >       A, Ap, ierr)
> >
> >     call MatAssemblyBegin(Ap,MAT_FINAL_ASSEMBLY,ierr)
> >     call MatAssemblyEnd(Ap,MAT_FINAL_ASSEMBLY,ierr)
> This assembly is superfluous (but harmless).
> > Does the communication of MatCreateMPIAIJWithArrays() in parallel
> > computation cost a lot? What could be the cause that
> > MatCreateMPIAIJWithArrays() so slow in the first iteration?
> There is no significant communication, it has to be preallocation.
> Jed

What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the petsc-users mailing list