[petsc-users] GAMG setup scalability

[John Fettig]

Mark,

Thanks for your response, comments inline:

On Fri, Aug 17, 2012 at 10:50 AM, Mark F. Adams <mark.adams at columbia.edu> wrote:
> Yes John, the GAMG setup is not great.  So we know there are problems.  We do have a new staff person that will be working on matrix product methods soon.  Matrix products are hard to make fast and GAMG setup relies on them quite a bit.
>
> A few things to note:
>
> 1) This is in the mesh setup phase, so many applications can amortize this cost.  (MatPtAP is a "matrix" setup cost and so it is amortized for linear problems or non-full Newton solves, but not for full nonlinear solves).

I'd be willing to live with the MatPtAP time, it is the
MatTransposeMatMult time that is really poor.  Any idea why the latter
is so slow?

> 3) This test is getting less than 2x speedup in KSPSolve for 8x processors.  So this problem looks hard: small or poorly partitioned, and not in the range of where we want people to run to get good performance.

It's about 725k unknowns, and it was run with ex10 from the ksp
tutorials (i.e. loaded from a file and solved).  It is possible the
partitioning isn't great.

> 4) I have found that the setup times are about twice that of ML, which uses a similar algorithm, and about 5x slower than hypre, which uses a very different algorithm.  So if you can not amortize theses setup costs then ML or hypre would probably work better for you.

I think you actually have this backwards, or your experience is
different from mine.  In my experience ML's setup is much faster than
that of hypre.

John