HYPRE with multiple variables
Barry Smith
bsmith at mcs.anl.gov
Mon May 7 10:28:03 CDT 2007
> AMG SOLUTION INFO:
> relative
> residual factor residual
> -------- ------ --------
> Initial 4.430431e-03 1.000000e+00
> Cycle 1 4.540275e-03 1.024793 1.024793e+00
> Cycle 2 4.539148e-03 0.999752 1.024539e+00
> Cycle 3 4.620010e-03 1.017814 1.042790e+00
> Cycle 4 5.196532e-03 1.124788 1.172918e+00
> Cycle 5 6.243043e-03 1.201386 1.409128e+00
> Cycle 6 7.310008e-03 1.170905 1.649954e+00
>
Hmmm. This is not converging.
> Even though I get a satisfactory result, I can not explain why hypre is
> performing what looks like several outer loops (see below for an abstract
> of the output from -pc_hypre_boomeramg_print_statistics).
What are you using for PETSc KSP? Unless you use -ksp_max_it 1 or
-ksp_type preonly PETSc will run the Krylov method until convergence.
If you run with -ksp_monitor you can watch the residuals and -ksp_view
for the solver options.
Barry
On Mon, 7 May 2007, Nicolas Bathfield wrote:
> Hi Allison, Barry, and PETSc users,
>
> The nodal relaxation leads to a nice solution! Thanks a lot for your help
> on this!
>
> Even though I get a satisfactory result, I can not explain why hypre is
> performing what looks like several outer loops (see below for an abstract
> of the output from -pc_hypre_boomeramg_print_statistics).
>
> Here is the list of options I used:
>
> -pc_hypre_boomeramg_tol 1e-5
> -pc_type hypre -pc_hypre_type boomeramg
> -pc_hypre_boomeramg_print_statistics
> -pc_hypre_boomeramg_grid_sweeps_all 20
> -pc_hypre_boomeramg_max_iter 6
> -pc_hypre_boomeramg_relax_weight_all 0.6
> -pc_hypre_boomeramg_outer_relax_weight_all 0.6
> -pc_hypre_boomeramg_nodal_coarsen 1
>
> I would expect the preconditioner to be executed once only (in my case
> this includes 6 V cycles).
> As you can see from the print_statistic option, boomeramg is exectuted
> several times (many more times than displyed here, it was too long to copy
> everything). Is this normal? If yes, why is it so?
>
> Best regards,
>
> Nicolas
>
> BoomerAMG SOLVER PARAMETERS:
>
> Maximum number of cycles: 6
> Stopping Tolerance: 1.000000e-05
> Cycle type (1 = V, 2 = W, etc.): 1
>
> Relaxation Parameters:
> Visiting Grid: down up coarse
> Visiting Grid: down up coarse
> Number of partial sweeps: 20 20 1
> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> Point types, partial sweeps (1=C, -1=F):
> Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1
> Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1
> Coarsest grid: 0
>
> Relaxation Weight 0.600000 level 0
> Outer relaxation weight 0.600000 level 0
> Output flag (print_level): 3
>
>
> AMG SOLUTION INFO:
> relative
> residual factor residual
> -------- ------ --------
> Initial 4.430431e-03 1.000000e+00
> Cycle 1 4.540275e-03 1.024793 1.024793e+00
> Cycle 2 4.539148e-03 0.999752 1.024539e+00
> Cycle 3 4.620010e-03 1.017814 1.042790e+00
> Cycle 4 5.196532e-03 1.124788 1.172918e+00
> Cycle 5 6.243043e-03 1.201386 1.409128e+00
> Cycle 6 7.310008e-03 1.170905 1.649954e+00
>
>
> ==============================================
> NOTE: Convergence tolerance was not achieved
> within the allowed 6 V-cycles
> ==============================================
>
> Average Convergence Factor = 1.087039
>
> Complexity: grid = 1.000000
> operator = 1.000000
> cycle = 1.000000
>
>
>
>
>
> BoomerAMG SOLVER PARAMETERS:
>
> Maximum number of cycles: 6
> Stopping Tolerance: 1.000000e-05
> Cycle type (1 = V, 2 = W, etc.): 1
>
> Relaxation Parameters:
> Visiting Grid: down up coarse
> Visiting Grid: down up coarse
> Number of partial sweeps: 20 20 1
> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> Point types, partial sweeps (1=C, -1=F):
> Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1
> Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1
> Coarsest grid: 0
>
> Relaxation Weight 0.600000 level 0
> Outer relaxation weight 0.600000 level 0
> Output flag (print_level): 3
>
>
> AMG SOLUTION INFO:
> relative
> residual factor residual
> -------- ------ --------
> Initial 7.914114e-03 1.000000e+00
> Cycle 1 8.320104e-03 1.051300 1.051300e+00
> Cycle 2 7.767803e-03 0.933618 9.815126e-01
> Cycle 3 6.570752e-03 0.845896 8.302575e-01
> Cycle 4 5.836166e-03 0.888204 7.374377e-01
> Cycle 5 6.593214e-03 1.129717 8.330956e-01
> Cycle 6 8.191117e-03 1.242356 1.035001e+00
>
>
> ==============================================
> NOTE: Convergence tolerance was not achieved
> within the allowed 6 V-cycles
> ==============================================
>
> Average Convergence Factor = 1.005750
>
> Complexity: grid = 1.000000
> operator = 1.000000
> cycle = 1.000000
>
>
>
>
>
> BoomerAMG SOLVER PARAMETERS:
>
> Maximum number of cycles: 6
> Stopping Tolerance: 1.000000e-05
> Cycle type (1 = V, 2 = W, etc.): 1
>
> Relaxation Parameters:
> Visiting Grid: down up coarse
> Visiting Grid: down up coarse
> Number of partial sweeps: 20 20 1
> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> Point types, partial sweeps (1=C, -1=F):
> Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1
> Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1
> Coarsest grid: 0
>
> Relaxation Weight 0.600000 level 0
> Outer relaxation weight 0.600000 level 0
> Output flag (print_level): 3
>
>
> AMG SOLUTION INFO:
> relative
> residual factor residual
> -------- ------ --------
> Initial 5.553016e-03 1.000000e+00
> Cycle 1 6.417482e-03 1.155675 1.155675e+00
> Cycle 2 7.101926e-03 1.106653 1.278931e+00
> Cycle 3 7.077471e-03 0.996556 1.274527e+00
> Cycle 4 6.382835e-03 0.901853 1.149436e+00
> Cycle 5 5.392392e-03 0.844827 9.710744e-01
> Cycle 6 4.674173e-03 0.866809 8.417359e-01
>
>
> ==============================================
> NOTE: Convergence tolerance was not achieved
> within the allowed 6 V-cycles
> ==============================================
>
> Average Convergence Factor = 0.971694
>
> Complexity: grid = 1.000000
> operator = 1.000000
> cycle = 1.000000
>
>
>
>
>
> BoomerAMG SOLVER PARAMETERS:
>
> Maximum number of cycles: 6
> Stopping Tolerance: 1.000000e-05
> Cycle type (1 = V, 2 = W, etc.): 1
>
> Relaxation Parameters:
> Visiting Grid: down up coarse
> Visiting Grid: down up coarse
> Number of partial sweeps: 20 20 1
> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> Point types, partial sweeps (1=C, -1=F):
> Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> -1 1 -1 1 -1
> Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> 1 -1 1 -1 1
> Coarsest grid: 0
>
> Relaxation Weight 0.600000 level 0
> Outer relaxation weight 0.600000 level 0
> Output flag (print_level): 3
>
>
> AMG SOLUTION INFO:
> relative
> residual factor residual
> -------- ------ --------
> Initial 3.846663e-03 1.000000e+00
> Cycle 1 4.136662e-03 1.075390 1.075390e+00
> Cycle 2 4.463861e-03 1.079097 1.160450e+00
> Cycle 3 4.302262e-03 0.963798 1.118440e+00
> Cycle 4 4.175328e-03 0.970496 1.085441e+00
> Cycle 5 4.735871e-03 1.134251 1.231163e+00
> Cycle 6 5.809054e-03 1.226607 1.510154e+00
>
>
> ==============================================
> NOTE: Convergence tolerance was not achieved
> within the allowed 6 V-cycles
> ==============================================
>
> Average Convergence Factor = 1.071117
>
> Complexity: grid = 1.000000
> operator = 1.000000
> cycle = 1.000000
>
>
> >
> > Nicolas,
> >
> > I have added support for both 1 and/or 2 to petsc-dev
> > http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html
> >
> > The two new options are -pc_hypre_boomeramg_nodal_coarsen and
> > -pc_hypre_boomeramg_nodal_relaxation <optional:n) where the optional
> > argument n indicates the levels which the SmoothNumLevels() sets.
> >
> > I have not tested the code so please let me know what problems you have.
> >
> > Allison,
> >
> > Thank you very much for the clarifications.
> >
> > Barry
> >
> > On Wed, 25 Apr 2007, Allison Baker wrote:
> >
> >> Hi Barry and Nicolas,
> >>
> >> To clarify,
> >>
> >> HYPRE_BoomerAMGSetNumFunctions(solver, int num_functions) tells AMG to
> >> solve a
> >> system of equations with the specified number of functions/unknowns. The
> >> default AMG scheme to solve a PDE system is the "unknown" approach. (The
> >> coarsening and interpolation are determined by looking at each
> >> unknown/function independently - therefore you can imagine that the
> >> coarse
> >> grids are generally not the same for each variable. This approach
> >> generally
> >> works well unless you have strong coupling between unknowns.)
> >>
> >> HYPRE_BOOMERAMGSetNodal(solver, int nodal ) tells AMG to coarsen such
> >> that
> >> each variable has the same coarse grid - sometimes this is more
> >> "physical" for
> >> a particular problem. The value chosen here for nodal determines how
> >> strength
> >> of connection is determined between the coupled system. I suggest
> >> setting
> >> nodal = 1, which uses a Frobenius norm. This does NOT tell AMG to use
> >> nodal
> >> relaxation.
> >>
> >> If you want to use nodal relaxation in hypre there are two choices:
> >>
> >> (1) If you call HYPRE_BOOMERAMGSetNodal, then you can additionally do
> >> nodal
> >> relaxation via the schwarz smoother option in hypre. I did not
> >> implement this
> >> in the Petsc interface, but it could be done easy enough. The following
> >> four
> >> functions need to be called:
> >>
> >> HYPRE_BoomerAMGSetSmoothType(solver, 6);
> >> HYPRE_BoomerAMGSetDomainType(solver, 1);
> >> HYPRE_BoomerAMGSetOverlap(solver, 0);
> >> HYPRE_BoomerAMGSetSmoothNumLevels(solver, num_levels); (Set
> >> num_levels
> >> to number of levels on which you want nodal smoothing, i.e. 1=just the
> >> fine
> >> grid, 2= fine grid and the grid below, etc. I find that doing nodal
> >> relaxation on just the finest level is generally sufficient.) Note that
> >> the
> >> interpolation scheme used will be the same as in the unknown approach -
> >> so
> >> this is what we call a hybrid systems method.
> >>
> >> (2) You can do both nodal smoothing and a nodal interpolation scheme.
> >> While
> >> this is currently implemented in 2.0.0, it is not advertised (i.e.,
> >> mentioned
> >> at all in the user's manual) because it is not yet implemented very
> >> efficiently (the fine grid matrix is converted to a block matrix - and
> >> both
> >> are stored), and we have not found it to be as effective as advertised
> >> elsewhere (this is an area of current research for us)..... If you want
> >> to try
> >> it anyway, let me know and I will provide more info.
> >>
> >> Hope this helps,
> >> Allison
> >>
> >>
> >> Barry Smith wrote:
> >> > Nicolas,
> >> >
> >> > On Wed, 25 Apr 2007, Nicolas Bathfield wrote:
> >> >
> >> >
> >> > > Dear Barry,
> >> > >
> >> > > Using MatSetBlockSize(A,5) improved my results greatly. Boomemramg
> >> is now
> >> > > solving the system of equations.
> >> > >
> >> >
> >> > Good
> >> >
> >> >
> >> > > Still, the equations I solve are coupled, and my discretization
> >> scheme is
> >> > > meant for a non-segregated solver. As a consequence (I believe),
> >> boomeramg
> >> > > still diverges.
> >> > >
> >> >
> >> > How can "Boomeramg be now solving the system of equations" but also
> >> > diverge? I am so confused.
> >> >
> >> >
> >> > > I would therefore like to use the nodal relaxation in
> >> > > boomeramg (the hypre command is HYPRE_BOOMERAMGSetNodal) in order to
> >> > > couple the coarse grid choice for all my variables.
> >> > >
> >> >
> >> > I can add this this afternoon.
> >> >
> >> > I have to admit I do not understand the difference between
> >> > HYPRE_BOOMERAMGSetNodal() and hypre_BoomerAMGSetNumFunctions(). Do
> >> you?
> >> >
> >> > Barry
> >> >
> >> > > How can I achieve this from PETSc?
> >> > >
> >> > > Best regards,
> >> > >
> >> > > Nicolas
> >> > >
> >> > >
> >> > > > From PETSc MPIAIJ matrices you need to set the block size of the
> >> > > > matrix
> >> > > > with MatSetBlockSize(A,5) after you have called MatSetType() or
> >> > > > MatCreateMPIAIJ(). Then HYPRE_BoomerAMGSetNumFunctions() is
> >> > > > automatically
> >> > > > called by PETSc.
> >> > > >
> >> > > > Barry
> >> > > >
> >> > > > The reason this is done this way instead of as
> >> > > > -pc_hypre_boomeramg_block_size <bs> is the idea that hypre will
> >> use the
> >> > > > properties of the matrix it is given in building the
> >> preconditioner so
> >> > > > the user does not have to pass those properties in seperately
> >> directly
> >> > > > to hypre.
> >> > > >
> >> > > >
> >> > > > On Fri, 13 Apr 2007, Shaman Mahmoudi wrote:
> >> > > >
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > int HYPRE_BoomerAMGSetNumFunctions (.....)
> >> > > > >
> >> > > > > sets the size of the system of PDEs.
> >> > > > >
> >> > > > > With best regards, Shaman Mahmoudi
> >> > > > >
> >> > > > > On Apr 13, 2007, at 2:04 PM, Shaman Mahmoudi wrote:
> >> > > > >
> >> > > > >
> >> > > > > > Hi Nicolas,
> >> > > > > >
> >> > > > > > You are right. hypre has changed a lot since the version I
> >> used.
> >> > > > > >
> >> > > > > > I found this interesting information:
> >> > > > > >
> >> > > > > > int HYPRE_BOOMERAMGSetNodal(....)
> >> > > > > >
> >> > > > > > Sets whether to use the nodal systems version. Default is 0.
> >> > > > > >
> >> > > > > > Then information about smoothers:
> >> > > > > >
> >> > > > > > One interesting thing there is this,
> >> > > > > >
> >> > > > > > HYPRE_BoomerAMGSetDomainType(....)
> >> > > > > >
> >> > > > > > 0 - each point is a domain (default)
> >> > > > > > 1 each node is a domain (only of interest in systems AMG)
> >> > > > > > 2 ....
> >> > > > > >
> >> > > > > > I could not find how you define the nodal displacement
> >> ordering. But
> >> > > > > >
> >> > > > > it
> >> > > > >
> >> > > > > > should be there somewhere.
> >> > > > > >
> >> > > > > > I read the reference manual for hypre 2.0
> >> > > > > >
> >> > > > > > With best regards, Shaman Mahmoudi
> >> > > > > >
> >> > > > > >
> >> > > > > > On Apr 13, 2007, at 1:40 PM, Nicolas Bathfield wrote:
> >> > > > > >
> >> > > > > >
> >> > > > > > > Dear Shaman,
> >> > > > > > >
> >> > > > > > > As far as I could understand, there is a BoomerAMG?s systems
> >> AMG
> >> > > > > > >
> >> > > > > version
> >> > > > >
> >> > > > > > > available. This seems to be exactly what I am looking for,
> >> but I
> >> > > > > > >
> >> > > > > just
> >> > > > >
> >> > > > > > > don't know how to access it, either through PETSc or
> >> directly.
> >> > > > > > >
> >> > > > > > > Best regards,
> >> > > > > > >
> >> > > > > > > Nicolas
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > > Hi,
> >> > > > > > > >
> >> > > > > > > > You want to exploit the structure of the model?
> >> > > > > > > > As far as I know, boomeramg can not treat a set of rows or
> >> > > > > > > > blocks
> >> > > > > > > >
> >> > > > > as
> >> > > > >
> >> > > > > > > > a molecule, a so called block-smoother?
> >> > > > > > > > ML 2.0 smoothed aggregation does support it.
> >> > > > > > > >
> >> > > > > > > > With best regards, Shaman Mahmoudi
> >> > > > > > > >
> >> > > > > > > > On Apr 13, 2007, at 10:45 AM, Nicolas Bathfield wrote:
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > > Hi,
> >> > > > > > > > >
> >> > > > > > > > > I am solving the Navier-stokes equations and try to use
> >> Hypre
> >> > > > > > > > > as
> >> > > > > > > > > preconditioner.
> >> > > > > > > > > Until now, I used PETSc as non-segregated solver and it
> >> worked
> >> > > > > > > > > perfectly.
> >> > > > > > > > > Things got worse when I decided to use Boomeramg
> >> (Hypre).
> >> > > > > > > > > As I solve a system of PDEs, each cell is represented
> >> by 5
> >> > > > > > > > > rows
> >> > > > > > > > >
> >> > > > > in my
> >> > > > >
> >> > > > > > > > > matrix (I solve for 5 variables). PETSc handles that
> >> without
> >> > > > > > > > >
> >> > > > > problem
> >> > > > >
> >> > > > > > > > > apparently, but the coarsening scheme of Boomeramg needs
> >> more
> >> > > > > > > > >
> >> > > > > input in
> >> > > > >
> >> > > > > > > > > order to work properly.
> >> > > > > > > > >
> >> > > > > > > > > Is there an option in PETSc to tell HYPRE that we are
> >> dealing
> >> > > > > > > > >
> >> > > > > with a
> >> > > > >
> >> > > > > > > > > system of PDEs? (something like:
> >> -pc_hypre_boomeramg_...)
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Thanks for your help.
> >> > > > > > > > >
> >> > > > > > > > > Best regards,
> >> > > > > > > > >
> >> > > > > > > > > Nicolas
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Nicolas BATHFIELD
> >> > > > > > > > > Chalmers University of Technology
> >> > > > > > > > > Shipping and Marine Technology
> >> > > > > > > > > phone: +46 (0)31 772 1476
> >> > > > > > > > > fax: +46 (0)31 772 3699
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > --
> >> > > > > > > Nicolas BATHFIELD
> >> > > > > > > Chalmers University of Technology
> >> > > > > > > Shipping and Marine Technology
> >> > > > > > > phone: +46 (0)31 772 1476
> >> > > > > > > fax: +46 (0)31 772 3699
> >> > > > > > >
> >> > > > > > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >
> >
>
>
>
More information about the petsc-users
mailing list