[petsc-users] MatSetValues is expensive

[Matthew Knepley]

On Thu, Feb 24, 2011 at 9:33 AM, M. Scot Breitenfeld <brtnfld at uiuc.edu>wrote:

> Hi,
>
> I'm working on a particle type method and I'm using MatSetValues, to
> insert values (add_values) into my matrix. Currently I:
>
> do  i, loop over number of particles
>     do j, loop over particles in i's family
>         ...
>          in row i's dof; insert values in columns of j's (x,y,z) dofs
> (3 calls to MatSetValues for i's x,y,z dof)
>

You can set this whole row with a single call.


>          in row j's dof; insert values in columns of i's (x,y,z) dofs
> (3 calls to MatSetValues for j's x,y,z dof)
>         ...
>     enddo
> enddo
>
> Running serially, using MatSetValues it takes 294.8 sec. to assemble the
> matrix, if I remove the calls to MatSetValues it takes 29.5 sec. to run
> through the same loops, so the MatSetValues calls take up 90% of the
> assembling time. I'm preallocating the A matrix specifying d_nnz and o_nnz.
>

Its hard to believe that the preallocation is correct. In order to check,
use

  MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR)

before your MatSetValues() calls.


> I guess I need to add extra storage so I can call the MatSetValues with
> more values so that I can call it less, or just do a lot of
> recalculating of values so that I can add an entire row at once. I just
>

Recalculating?


> want to make sure this is expected behavior and not something that I'm
> doing wrong before I start to rewrite my assembling routine. Probably a
> hash table would be better but I don't want to store that and then
> convert that to a CRS matrix, I'm already running into memory issues as
> it is.
>
> Just out of curiosity, wouldn't a finite element code have a similar
> situation, in that case you would form the local stiffness matrix and
> then insert that into the global stiffness matrix, so you would be
> calling MatSetValues "number of elements" times.
>

No, you call it once per element matrix.

   Matt

-- 
What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20110224/90dad69a/attachment.htm>