[petsc-users] GAMG Parallel Performance

[Matthew Knepley]

On Thu, Nov 15, 2018 at 11:52 AM Karin&NiKo via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> Dear PETSc team,
>
> I am solving a linear transient dynamic problem, based on a discretization
> with finite elements. To do that, I am using FGMRES with GAMG as a
> preconditioner. I consider here 10 time steps.
> The problem has round to 118e6 dof and I am running on 1000, 1500 and 2000
> procs. So I have something like 100e3, 78e3 and 50e3 dof/proc.
> I notice that the performance deteriorates when I increase the number of
> processes.
> You can find as attached file the log_view of the execution and the
> detailled definition of the KSP.
>
> Is the problem too small to run on that number of processes or is there
> something wrong with my use of GAMG?
>

I am having a hard time understanding the data. Just to be clear, I
understand you to be running the exact same problem on 1000, 1500, and 2000
processes, so looking for strong speedup. The PCSetUp time actually sped up
a little, which is great, and its still a small percentage (notice that
your whole solve is only half the runtime). Lets just look at a big time
component, MatMult,

P = 1000

MatMult             7342 1.0 4.4956e+01 1.4 4.09e+10 1.2 9.6e+07
4.3e+03 0.0e+00 23 53 81 86  0  23 53 81 86  0 859939


P = 2000

MatMult             7470 1.0 4.7611e+01 1.9 2.11e+10 1.2 2.0e+08
2.9e+03 0.0e+00 11 53 81 86  0  11 53 81 86  0 827107


So there was no speedup at all. It is doing 1/2 the flops per process, but
taking almost exactly the same time. This looks like your 2000 process run
is on exactly the same number of nodes as your 1000 process run, but you
just use more processes. Your 1000 process run was maxing out the bandwidth
of those nodes, and thus 2000 runs no faster. Is this true? Otherwise, I am
misunderstanding the run.

  Thanks,

    Matt


> I thank you in advance for your help,
> Nicolas
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20181115/a0df0d47/attachment.html>