[petsc-dev] GAMG coarsening in-place

[Jed Brown]

Mark Adams <mfadams at lbl.gov> writes:

> The only reason that I can think of that you would want to use more than
> one proc on the coarse grid is if you want to use a non-default coarse grid
> solver (bjacobi/lu).  Perhaps I could *not* go to one proc if the coarse
> grid solver pc type is set (more below).  I don't like adding yet another
> switch for this if possible.

What about checking whether the coarse grid has fewer dofs than the
requested max per process?  If the problem is that small, it should be
no harm to ensure that it ends up on one process, otherwise it's
probably a bad idea.  The problems I care about have Green's functions
that decay sufficiently rapidly that I don't want to continue
coarsening.

> I now see there is another problem.  I recently took out code
> (mark/gamg-coarseksp) to remove setting the coarse grid KSP:
>
>       ierr = PCSetUp_MG(pc);CHKERRQ(ierr);
>
>       /* PCSetUp_MG seems to insists on setting this to GMRES */
>       ierr = KSPSetType(mglevels[0]->smoothd, KSPPREONLY);CHKERRQ(ierr);
>
> The problem is that GMRES is going to do a few reductions even if it has an
> exact solver - this is bad on a large problem because it uses
> PETSC_COMM_WORLD.

I'd prefer that we configure the coarse solvers on suitable
subcommunicators (and this shouldn't be difficult), but vendors' broken
implementations of MPI-1 subcommunicators has me worried if we don't
have a fallback.

> I don't know if PCSetUp_MG still sets the coarse grid solver to GMRES, but
> we should fix this now as well.
>
> One might have '-mg_coarse_pc_type lu' but have a parallel LU so just
> checking this would be fragile.
>
> Perhaps I could turn off this hard switch to one processor if
> -mg_coarse_pc_type or -mg_coarse_ksp_type or
>       -mg_coarse_pc_factor_mat_solver_package is set?  And also put this
> KSPPREONLY back in in that case?

That's all too fragile and confusing.  The decision needs to be
independent of the selected coarse KSP/PC implementation.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150106/4980ea75/attachment.sig>