[petsc-users] GAMG parameters for ideal coarsening ratio

[Mark Adams]

On Mon, Mar 16, 2020 at 8:31 PM Sajid Ali <sajidsyed2021 at u.northwestern.edu>
wrote:

> Hi PETSc-developers,
>
> As per the manual, the ideal gamg parameters are those which result in
> MatPtAP time being roughly similar to (or just slightly larger) than KSP
> solve times. The way to adjust this is by changing the threshold for
> coarsening and/or squaring the graph. I was working with a grid of size
> 2^14 by 2^14 in a linear & time-independent TS with the following params :
>
> #PETSc Option Table entries:
> -ksp_monitor
> -ksp_rtol 1e-5
> -ksp_type fgmres
> -ksp_view
> -log_view
> -mg_levels_ksp_type gmres
> -mg_levels_pc_type jacobi
> -pc_gamg_coarse_eq_limit 1000
> -pc_gamg_reuse_interpolation true
> -pc_gamg_square_graph 10
> -pc_gamg_threshold -0.04
> -pc_gamg_type agg
> -pc_gamg_use_parallel_coarse_grid_solver
> -pc_mg_monitor
> -pc_type gamg
> -prop_steps 8
> -ts_monitor
> -ts_type cn
> #End of PETSc Option Table entries
>
> With this I get a grid complexity of 1.33047, 6 multigrid levels,
> MatPtAP/KSPSolve ratio of 0.24, and the linear solve at each TS step takes
> 5 iterations (with approx one order of magnitude reduction in residual per
> step for iterations 2 through 5 and two orders for the first). The
> convergence and grid complexity look good, but the ratio of grid coarsening
> time to ksp-solve time is far from ideal. I've attached the log file from
> this set of base parameters as well.
>

Just a few notes,

* a negative threshold makes no sense, but I use it as a flag to keep all
matrix entries, even 0.0, when the graph, used for coarsening, is created.
A threshold of zero says drop only edges with 0 weight. All edge weights
are non-negative by construction. A threshold >= 0 says drop edges that are
<= the threshold. A threshold of 0.1 is very very high. So you probably
want to scan 0-0.1.

* -pc_gamg_square_graph N, says square the graph on the first N levels. So
"10" is basically infinity, although for 2D problems you can hit 10.


>
> To investigate the effect of coarsening rates, I ran a parameter sweep
> over the coarsening parameters (threshold and sq. graph) and I'm confused
> by the results. For some reason either the number of gamg levels turns out
> to be too high or it is set to 1.
>

Are you solving a mass matrix? A lot of coarsening parameters can
decimate all the edges of a FE mass matrix, in which case GAMG just gives
you a one level solver. Which is a good choice for a mass matrix.


> When I try to manually set the number of levels to 4 (with pc_mg_levels 4
> and thres. -0.04/ squaring of 10) I see performance much worse than the
> base parameters. Any advice as to what I'm missing in my search for a set
> of params where MatPtAP to KSPSolve is ~ 1 ?
>

If you are not using a Laplacian as the test then use that. That will help
to get us on the same page. If you are using a Laplacian then we need to
get some more data from you (ie, run with -info and grep on GAMG).

Mark


>
> [image: image.png]
>
> Thanks in advance for the help and hope everyone is staying safe from the
> virus!
>
>
> --
> Sajid Ali | PhD Candidate
> Applied Physics
> Northwestern University
> s-sajid-ali.github.io
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200316/acc8ee1d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 108837 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200316/acc8ee1d/attachment-0001.png>