[petsc-users] MatSetValues is expensive

[M. Scot Breitenfeld]

On 02/24/2011 04:59 PM, Jed Brown wrote:
> On Thu, Feb 24, 2011 at 23:49, M. Scot Breitenfeld <brtnfld at uiuc.edu
> <mailto:brtnfld at uiuc.edu>> wrote:
>
>     I would not say it's zero to compute the entries (I guess it takes
>     about
>     3.5ms per particle for the calculations). This is a fairly small case,
>     only 8000 particles.
>
>
> With 300 to 900 interactions per particle, times 3 for each component,
> times two for lower and upper triangular piece. So we're looking at
> half a microsecond per insertion. That still seems like a lot, but
> perhaps the access pattern is very irregular because the particles
> have an essentially random ordering. Did you build --with-debugging=0?
> That should make a reasonable difference.
>
I have not because on the machine I'm running on the compilation fails
with error: cast to type "__m64" is not allowed, I've reported it back
in September (I'm going to upgrade my compiler and OS soon, so hopefully
that will fix the problem). I'll recompile it on another machine and see
if that helps.


> Also, since the matrix is symmetric, you might consider using the
> SBAIJ matrix format. That will cut your storage costs almost in half
> and should speed up insertion because all interactions for a given
> particle will be in the same block-row, thus nearby in memory.

That would be great! But I don't see in the manual a function for
creating a parallel SBAIJ matrix, only  a sequential SBAIJ.