[petsc-users] Read in sequential, solve in parallel

[Jed Brown]

On Wed, Sep 29, 2010 at 14:51, Moinier, Pierre (UK)
<Pierre.Moinier at baesystems.com> wrote:
> Jed,
>
> The matrix is 1000000x1000000 and I have 4996000 non zeros

These statistics look like perhaps this matrix comes from a 5-point
discretization of an elliptic operator.  Is that true?

> Here is the output for a single proc:

You'll want to compare the time in each event when run in parallel:

> MatMult             1633 1.0 1.6247e+01 1.0 1.47e+10 1.0 0.0e+00 0.0e+00
> 0.0e+00 36 43  0  0  0  36 43  0  0  0   904

16 seconds in this event, 904 Mflop/s is good for a problem like this
on a single core.

> VecDot              3266 1.0 4.4834e+00 1.0 6.53e+09 1.0 0.0e+00 0.0e+00
> 0.0e+00 10 19  0  0  0  10 19  0  0  0  1457
> VecNorm             1634 1.0 1.2968e+01 1.0 3.27e+09 1.0 0.0e+00 0.0e+00
> 0.0e+00 28 10  0  0  0  28 10  0  0  0   252

This is confusing, the norms should take about a tenth of this (they
should have almost double the Mflop/s of VecDot).  Is there something
else running on this machine?  Anyone have other ideas?

> VecAXPY             3266 1.0 5.5580e+00 1.0 6.53e+09 1.0 0.0e+00 0.0e+00
> 0.0e+00 12 19  0  0  0  12 19  0  0  0  1175
> VecAYPX             1632 1.0 2.5961e+00 1.0 3.26e+09 1.0 0.0e+00 0.0e+00
> 0.0e+00  6 10  0  0  0   6 10  0  0  0  1257

These look normal.

> KSPSolve               1 1.0 4.4828e+01 1.0 3.43e+10 1.0 0.0e+00 0.0e+00
> 0.0e+00 98100  0  0  0  98100  0  0  0   765

And here's your solve total, the aggregate numbers look fine.

Jed