HYPRE with multiple variables
Nicolas Bathfield
nicolas.bathfield at chalmers.se
Mon May 7 09:50:56 CDT 2007
Hi Allison, Barry, and PETSc users,
The nodal relaxation leads to a nice solution! Thanks a lot for your help
on this!
Even though I get a satisfactory result, I can not explain why hypre is
performing what looks like several outer loops (see below for an abstract
of the output from -pc_hypre_boomeramg_print_statistics).
Here is the list of options I used:
-pc_hypre_boomeramg_tol 1e-5
-pc_type hypre -pc_hypre_type boomeramg
-pc_hypre_boomeramg_print_statistics
-pc_hypre_boomeramg_grid_sweeps_all 20
-pc_hypre_boomeramg_max_iter 6
-pc_hypre_boomeramg_relax_weight_all 0.6
-pc_hypre_boomeramg_outer_relax_weight_all 0.6
-pc_hypre_boomeramg_nodal_coarsen 1
I would expect the preconditioner to be executed once only (in my case
this includes 6 V cycles).
As you can see from the print_statistic option, boomeramg is exectuted
several times (many more times than displyed here, it was too long to copy
everything). Is this normal? If yes, why is it so?
Best regards,
Nicolas
BoomerAMG SOLVER PARAMETERS:
Maximum number of cycles: 6
Stopping Tolerance: 1.000000e-05
Cycle type (1 = V, 2 = W, etc.): 1
Relaxation Parameters:
Visiting Grid: down up coarse
Visiting Grid: down up coarse
Number of partial sweeps: 20 20 1
Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
Point types, partial sweeps (1=C, -1=F):
Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1
Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1
Coarsest grid: 0
Relaxation Weight 0.600000 level 0
Outer relaxation weight 0.600000 level 0
Output flag (print_level): 3
AMG SOLUTION INFO:
relative
residual factor residual
-------- ------ --------
Initial 4.430431e-03 1.000000e+00
Cycle 1 4.540275e-03 1.024793 1.024793e+00
Cycle 2 4.539148e-03 0.999752 1.024539e+00
Cycle 3 4.620010e-03 1.017814 1.042790e+00
Cycle 4 5.196532e-03 1.124788 1.172918e+00
Cycle 5 6.243043e-03 1.201386 1.409128e+00
Cycle 6 7.310008e-03 1.170905 1.649954e+00
==============================================
NOTE: Convergence tolerance was not achieved
within the allowed 6 V-cycles
==============================================
Average Convergence Factor = 1.087039
Complexity: grid = 1.000000
operator = 1.000000
cycle = 1.000000
BoomerAMG SOLVER PARAMETERS:
Maximum number of cycles: 6
Stopping Tolerance: 1.000000e-05
Cycle type (1 = V, 2 = W, etc.): 1
Relaxation Parameters:
Visiting Grid: down up coarse
Visiting Grid: down up coarse
Number of partial sweeps: 20 20 1
Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
Point types, partial sweeps (1=C, -1=F):
Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1
Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1
Coarsest grid: 0
Relaxation Weight 0.600000 level 0
Outer relaxation weight 0.600000 level 0
Output flag (print_level): 3
AMG SOLUTION INFO:
relative
residual factor residual
-------- ------ --------
Initial 7.914114e-03 1.000000e+00
Cycle 1 8.320104e-03 1.051300 1.051300e+00
Cycle 2 7.767803e-03 0.933618 9.815126e-01
Cycle 3 6.570752e-03 0.845896 8.302575e-01
Cycle 4 5.836166e-03 0.888204 7.374377e-01
Cycle 5 6.593214e-03 1.129717 8.330956e-01
Cycle 6 8.191117e-03 1.242356 1.035001e+00
==============================================
NOTE: Convergence tolerance was not achieved
within the allowed 6 V-cycles
==============================================
Average Convergence Factor = 1.005750
Complexity: grid = 1.000000
operator = 1.000000
cycle = 1.000000
BoomerAMG SOLVER PARAMETERS:
Maximum number of cycles: 6
Stopping Tolerance: 1.000000e-05
Cycle type (1 = V, 2 = W, etc.): 1
Relaxation Parameters:
Visiting Grid: down up coarse
Visiting Grid: down up coarse
Number of partial sweeps: 20 20 1
Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
Point types, partial sweeps (1=C, -1=F):
Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1
Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1
Coarsest grid: 0
Relaxation Weight 0.600000 level 0
Outer relaxation weight 0.600000 level 0
Output flag (print_level): 3
AMG SOLUTION INFO:
relative
residual factor residual
-------- ------ --------
Initial 5.553016e-03 1.000000e+00
Cycle 1 6.417482e-03 1.155675 1.155675e+00
Cycle 2 7.101926e-03 1.106653 1.278931e+00
Cycle 3 7.077471e-03 0.996556 1.274527e+00
Cycle 4 6.382835e-03 0.901853 1.149436e+00
Cycle 5 5.392392e-03 0.844827 9.710744e-01
Cycle 6 4.674173e-03 0.866809 8.417359e-01
==============================================
NOTE: Convergence tolerance was not achieved
within the allowed 6 V-cycles
==============================================
Average Convergence Factor = 0.971694
Complexity: grid = 1.000000
operator = 1.000000
cycle = 1.000000
BoomerAMG SOLVER PARAMETERS:
Maximum number of cycles: 6
Stopping Tolerance: 1.000000e-05
Cycle type (1 = V, 2 = W, etc.): 1
Relaxation Parameters:
Visiting Grid: down up coarse
Visiting Grid: down up coarse
Number of partial sweeps: 20 20 1
Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
Point types, partial sweeps (1=C, -1=F):
Pre-CG relaxation (down): 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
-1 1 -1 1 -1
Post-CG relaxation (up): -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
1 -1 1 -1 1
Coarsest grid: 0
Relaxation Weight 0.600000 level 0
Outer relaxation weight 0.600000 level 0
Output flag (print_level): 3
AMG SOLUTION INFO:
relative
residual factor residual
-------- ------ --------
Initial 3.846663e-03 1.000000e+00
Cycle 1 4.136662e-03 1.075390 1.075390e+00
Cycle 2 4.463861e-03 1.079097 1.160450e+00
Cycle 3 4.302262e-03 0.963798 1.118440e+00
Cycle 4 4.175328e-03 0.970496 1.085441e+00
Cycle 5 4.735871e-03 1.134251 1.231163e+00
Cycle 6 5.809054e-03 1.226607 1.510154e+00
==============================================
NOTE: Convergence tolerance was not achieved
within the allowed 6 V-cycles
==============================================
Average Convergence Factor = 1.071117
Complexity: grid = 1.000000
operator = 1.000000
cycle = 1.000000
>
> Nicolas,
>
> I have added support for both 1 and/or 2 to petsc-dev
> http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html
>
> The two new options are -pc_hypre_boomeramg_nodal_coarsen and
> -pc_hypre_boomeramg_nodal_relaxation <optional:n) where the optional
> argument n indicates the levels which the SmoothNumLevels() sets.
>
> I have not tested the code so please let me know what problems you have.
>
> Allison,
>
> Thank you very much for the clarifications.
>
> Barry
>
> On Wed, 25 Apr 2007, Allison Baker wrote:
>
>> Hi Barry and Nicolas,
>>
>> To clarify,
>>
>> HYPRE_BoomerAMGSetNumFunctions(solver, int num_functions) tells AMG to
>> solve a
>> system of equations with the specified number of functions/unknowns. The
>> default AMG scheme to solve a PDE system is the "unknown" approach. (The
>> coarsening and interpolation are determined by looking at each
>> unknown/function independently - therefore you can imagine that the
>> coarse
>> grids are generally not the same for each variable. This approach
>> generally
>> works well unless you have strong coupling between unknowns.)
>>
>> HYPRE_BOOMERAMGSetNodal(solver, int nodal ) tells AMG to coarsen such
>> that
>> each variable has the same coarse grid - sometimes this is more
>> "physical" for
>> a particular problem. The value chosen here for nodal determines how
>> strength
>> of connection is determined between the coupled system. I suggest
>> setting
>> nodal = 1, which uses a Frobenius norm. This does NOT tell AMG to use
>> nodal
>> relaxation.
>>
>> If you want to use nodal relaxation in hypre there are two choices:
>>
>> (1) If you call HYPRE_BOOMERAMGSetNodal, then you can additionally do
>> nodal
>> relaxation via the schwarz smoother option in hypre. I did not
>> implement this
>> in the Petsc interface, but it could be done easy enough. The following
>> four
>> functions need to be called:
>>
>> HYPRE_BoomerAMGSetSmoothType(solver, 6);
>> HYPRE_BoomerAMGSetDomainType(solver, 1);
>> HYPRE_BoomerAMGSetOverlap(solver, 0);
>> HYPRE_BoomerAMGSetSmoothNumLevels(solver, num_levels); (Set
>> num_levels
>> to number of levels on which you want nodal smoothing, i.e. 1=just the
>> fine
>> grid, 2= fine grid and the grid below, etc. I find that doing nodal
>> relaxation on just the finest level is generally sufficient.) Note that
>> the
>> interpolation scheme used will be the same as in the unknown approach -
>> so
>> this is what we call a hybrid systems method.
>>
>> (2) You can do both nodal smoothing and a nodal interpolation scheme.
>> While
>> this is currently implemented in 2.0.0, it is not advertised (i.e.,
>> mentioned
>> at all in the user's manual) because it is not yet implemented very
>> efficiently (the fine grid matrix is converted to a block matrix - and
>> both
>> are stored), and we have not found it to be as effective as advertised
>> elsewhere (this is an area of current research for us)..... If you want
>> to try
>> it anyway, let me know and I will provide more info.
>>
>> Hope this helps,
>> Allison
>>
>>
>> Barry Smith wrote:
>> > Nicolas,
>> >
>> > On Wed, 25 Apr 2007, Nicolas Bathfield wrote:
>> >
>> >
>> > > Dear Barry,
>> > >
>> > > Using MatSetBlockSize(A,5) improved my results greatly. Boomemramg
>> is now
>> > > solving the system of equations.
>> > >
>> >
>> > Good
>> >
>> >
>> > > Still, the equations I solve are coupled, and my discretization
>> scheme is
>> > > meant for a non-segregated solver. As a consequence (I believe),
>> boomeramg
>> > > still diverges.
>> > >
>> >
>> > How can "Boomeramg be now solving the system of equations" but also
>> > diverge? I am so confused.
>> >
>> >
>> > > I would therefore like to use the nodal relaxation in
>> > > boomeramg (the hypre command is HYPRE_BOOMERAMGSetNodal) in order to
>> > > couple the coarse grid choice for all my variables.
>> > >
>> >
>> > I can add this this afternoon.
>> >
>> > I have to admit I do not understand the difference between
>> > HYPRE_BOOMERAMGSetNodal() and hypre_BoomerAMGSetNumFunctions(). Do
>> you?
>> >
>> > Barry
>> >
>> > > How can I achieve this from PETSc?
>> > >
>> > > Best regards,
>> > >
>> > > Nicolas
>> > >
>> > >
>> > > > From PETSc MPIAIJ matrices you need to set the block size of the
>> > > > matrix
>> > > > with MatSetBlockSize(A,5) after you have called MatSetType() or
>> > > > MatCreateMPIAIJ(). Then HYPRE_BoomerAMGSetNumFunctions() is
>> > > > automatically
>> > > > called by PETSc.
>> > > >
>> > > > Barry
>> > > >
>> > > > The reason this is done this way instead of as
>> > > > -pc_hypre_boomeramg_block_size <bs> is the idea that hypre will
>> use the
>> > > > properties of the matrix it is given in building the
>> preconditioner so
>> > > > the user does not have to pass those properties in seperately
>> directly
>> > > > to hypre.
>> > > >
>> > > >
>> > > > On Fri, 13 Apr 2007, Shaman Mahmoudi wrote:
>> > > >
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > int HYPRE_BoomerAMGSetNumFunctions (.....)
>> > > > >
>> > > > > sets the size of the system of PDEs.
>> > > > >
>> > > > > With best regards, Shaman Mahmoudi
>> > > > >
>> > > > > On Apr 13, 2007, at 2:04 PM, Shaman Mahmoudi wrote:
>> > > > >
>> > > > >
>> > > > > > Hi Nicolas,
>> > > > > >
>> > > > > > You are right. hypre has changed a lot since the version I
>> used.
>> > > > > >
>> > > > > > I found this interesting information:
>> > > > > >
>> > > > > > int HYPRE_BOOMERAMGSetNodal(....)
>> > > > > >
>> > > > > > Sets whether to use the nodal systems version. Default is 0.
>> > > > > >
>> > > > > > Then information about smoothers:
>> > > > > >
>> > > > > > One interesting thing there is this,
>> > > > > >
>> > > > > > HYPRE_BoomerAMGSetDomainType(....)
>> > > > > >
>> > > > > > 0 - each point is a domain (default)
>> > > > > > 1 each node is a domain (only of interest in systems AMG)
>> > > > > > 2 ....
>> > > > > >
>> > > > > > I could not find how you define the nodal displacement
>> ordering. But
>> > > > > >
>> > > > > it
>> > > > >
>> > > > > > should be there somewhere.
>> > > > > >
>> > > > > > I read the reference manual for hypre 2.0
>> > > > > >
>> > > > > > With best regards, Shaman Mahmoudi
>> > > > > >
>> > > > > >
>> > > > > > On Apr 13, 2007, at 1:40 PM, Nicolas Bathfield wrote:
>> > > > > >
>> > > > > >
>> > > > > > > Dear Shaman,
>> > > > > > >
>> > > > > > > As far as I could understand, there is a BoomerAMG?s systems
>> AMG
>> > > > > > >
>> > > > > version
>> > > > >
>> > > > > > > available. This seems to be exactly what I am looking for,
>> but I
>> > > > > > >
>> > > > > just
>> > > > >
>> > > > > > > don't know how to access it, either through PETSc or
>> directly.
>> > > > > > >
>> > > > > > > Best regards,
>> > > > > > >
>> > > > > > > Nicolas
>> > > > > > >
>> > > > > > >
>> > > > > > > > Hi,
>> > > > > > > >
>> > > > > > > > You want to exploit the structure of the model?
>> > > > > > > > As far as I know, boomeramg can not treat a set of rows or
>> > > > > > > > blocks
>> > > > > > > >
>> > > > > as
>> > > > >
>> > > > > > > > a molecule, a so called block-smoother?
>> > > > > > > > ML 2.0 smoothed aggregation does support it.
>> > > > > > > >
>> > > > > > > > With best regards, Shaman Mahmoudi
>> > > > > > > >
>> > > > > > > > On Apr 13, 2007, at 10:45 AM, Nicolas Bathfield wrote:
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > > Hi,
>> > > > > > > > >
>> > > > > > > > > I am solving the Navier-stokes equations and try to use
>> Hypre
>> > > > > > > > > as
>> > > > > > > > > preconditioner.
>> > > > > > > > > Until now, I used PETSc as non-segregated solver and it
>> worked
>> > > > > > > > > perfectly.
>> > > > > > > > > Things got worse when I decided to use Boomeramg
>> (Hypre).
>> > > > > > > > > As I solve a system of PDEs, each cell is represented
>> by 5
>> > > > > > > > > rows
>> > > > > > > > >
>> > > > > in my
>> > > > >
>> > > > > > > > > matrix (I solve for 5 variables). PETSc handles that
>> without
>> > > > > > > > >
>> > > > > problem
>> > > > >
>> > > > > > > > > apparently, but the coarsening scheme of Boomeramg needs
>> more
>> > > > > > > > >
>> > > > > input in
>> > > > >
>> > > > > > > > > order to work properly.
>> > > > > > > > >
>> > > > > > > > > Is there an option in PETSc to tell HYPRE that we are
>> dealing
>> > > > > > > > >
>> > > > > with a
>> > > > >
>> > > > > > > > > system of PDEs? (something like:
>> -pc_hypre_boomeramg_...)
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Thanks for your help.
>> > > > > > > > >
>> > > > > > > > > Best regards,
>> > > > > > > > >
>> > > > > > > > > Nicolas
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Nicolas BATHFIELD
>> > > > > > > > > Chalmers University of Technology
>> > > > > > > > > Shipping and Marine Technology
>> > > > > > > > > phone: +46 (0)31 772 1476
>> > > > > > > > > fax: +46 (0)31 772 3699
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > --
>> > > > > > > Nicolas BATHFIELD
>> > > > > > > Chalmers University of Technology
>> > > > > > > Shipping and Marine Technology
>> > > > > > > phone: +46 (0)31 772 1476
>> > > > > > > fax: +46 (0)31 772 3699
>> > > > > > >
>> > > > > > >
>> > >
>> > >
>> >
>> >
>>
>>
>
>
--
Nicolas BATHFIELD
Chalmers University of Technology
Shipping and Marine Technology
phone: +46 (0)31 772 1476
fax: +46 (0)31 772 3699
More information about the petsc-users
mailing list