[petsc-dev] [petsc-users] Problem with AMG packages

[Mark F. Adams]

As I said before your setup times are terrible and I _was_ thinking that they could not get better with more equation but they actually could.  Higher order stencils with tiny subdomains can have processors "reaching across" other processors subdomains resulting in communicating with a lot more processor than what you might expect from a good partitioning.  You definitely need to increase your eq/proc by at least 100x.


On Oct 9, 2013, at 6:50 PM, Matthew Knepley <knepley at gmail.com> wrote:

> On Wed, Oct 9, 2013 at 5:47 PM, Mark F. Adams <mfadams at lbl.gov> wrote:
> >
> > (referring to your previous email about the kind of discretization) I'm
> > using P_2 finite elements on tetras. The mesh is so small that it might as
> > well be partitioned lexicographically (but in this case it was done by
> > Metis)
> >
> 
> Well, This seems fine.  Higher order elements can confuse AMG sometimes but the math is decent in hypre and ML and I think it would be about the same with GAMG with the parameter that I gave you.  Something is going wrong.  We are trying to find it but not having any luck.
> 
> There are many parameters, including stupid things like solver parameters and stupider things like bugs, that can kill an iterative solver and we are trying to figure out what is wrong here.  You should be running with about 1/10 or 1/100, or even 1/1000, the number of processors that you are using here.  If you eventually want to run in this regime then that is fine but for debugging it helps to be in a more normal regime, because then you (or us) can start to see secondary things like super slow flop rates that can provide a hint of what is going wrong.  It is also better to start in serial so there are less moving parts -- I have to think that these terrible ML and hypre setup times would go away in serial … not that that is a solution but it is knowledge.
> 
> Also, keep in mind that MG is better at scaling so for small problems direct solvers or simple solvers can be faster.  Also AMG has significant setup costs (but they should not be as high as you are seeing) and this pushes the "cross over" point higher if you are just looking at one solve.  These costs are amortized for multiple solves but these are just the basic complexity issues with solvers.  MG may not always win on Poisson but it should, just about always.
> 
> We can run this exact case with SNES ex12.
> 
>    Matt
>  
> 
> Mark
> 
> > Now that at least either GAMG or BoomerAMG are not stuck for small
> > matrices I'll try to go bigger, and hopefully won't have to bother you
> > again, thanks a lot for all your pointers.
> >
> > Pierre
> >
> > PS: about src/ksp/ksp/examples/tutorials/ex56.c, Dirichelet (line 6)
> > should read Dirichlet.
> 
> Thanks,
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20131009/78f73ab0/attachment.html>