HYPRE with multiple variables
Barry Smith
bsmith at mcs.anl.gov
Wed May 9 10:11:56 CDT 2007
Yes. Make sure you give BoomerAMG enough iterations to
solve the system as accurately as you need with
-pc_hypre_boomeramg_max_iter and -pc_hypre_boomeramg_rtol
Barry
On Tue, 8 May 2007, Nicolas Bathfield wrote:
> Hi,
>
>
> Boomeramg is used as preconditioner, and I use default gmres to solve the
> equations. I did not expect the preconditioner to be used at every
> iteration, but I now understand better what's going on.
> I solve a spase matrix with 5 diagonals.
> I tried to use boomeramg as a solver only. To achieve this, I had to
> comment out the command ierr = KSPSetInitialGuessNonzero(ksp,PETSC_TRUE),
> and I used the option -ksp_preonly. Is it the right way to do?
>
> Best regards,
>
> Nicolas
>
> > Hi,
> >
> > You mentioned that your preconditioner is boomerAMG. What solver are
> > you using?
> > What type of matrix are you trying to solve? Is it SPD and sparse?
> > Either way, if your solver is not AMG and you are using AMG as a
> > preconditioner, I would suggest that you use a stopping criteria for
> > AMG such as a relative residual convergence of say 1e-03 for the
> > preconditioner, and the solvers relative convergence 1e-05 as in your
> > example, and disregard a stopping criteria which involves maximum
> > number of iterations for AMG, set that to something high. Maybe you
> > will get better results that way. In the end, I guess it is a lot of
> > testing and fine-tuning.
> >
> > With best regards, Shaman Mahmoudi
> >
> > On May 7, 2007, at 4:50 PM, Nicolas Bathfield wrote:
> >
> >> Hi Allison, Barry, and PETSc users,
> >>
> >> The nodal relaxation leads to a nice solution! Thanks a lot for
> >> your help
> >> on this!
> >>
> >> Even though I get a satisfactory result, I can not explain why
> >> hypre is
> >> performing what looks like several outer loops (see below for an
> >> abstract
> >> of the output from -pc_hypre_boomeramg_print_statistics).
> >>
> >> Here is the list of options I used:
> >>
> >> -pc_hypre_boomeramg_tol 1e-5
> >> -pc_type hypre -pc_hypre_type boomeramg
> >> -pc_hypre_boomeramg_print_statistics
> >> -pc_hypre_boomeramg_grid_sweeps_all 20
> >> -pc_hypre_boomeramg_max_iter 6
> >> -pc_hypre_boomeramg_relax_weight_all 0.6
> >> -pc_hypre_boomeramg_outer_relax_weight_all 0.6
> >> -pc_hypre_boomeramg_nodal_coarsen 1
> >>
> >> I would expect the preconditioner to be executed once only (in my case
> >> this includes 6 V cycles).
> >> As you can see from the print_statistic option, boomeramg is exectuted
> >> several times (many more times than displyed here, it was too long
> >> to copy
> >> everything). Is this normal? If yes, why is it so?
> >>
> >> Best regards,
> >>
> >> Nicolas
> >>
> >> BoomerAMG SOLVER PARAMETERS:
> >>
> >> Maximum number of cycles: 6
> >> Stopping Tolerance: 1.000000e-05
> >> Cycle type (1 = V, 2 = W, etc.): 1
> >>
> >> Relaxation Parameters:
> >> Visiting Grid: down up coarse
> >> Visiting Grid: down up coarse
> >> Number of partial sweeps: 20 20 1
> >> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> >> Point types, partial sweeps (1=C, -1=F):
> >> Pre-CG relaxation (down): 1 -1 1 -1 1
> >> -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1
> >> Post-CG relaxation (up): -1 1 -1 1 -1
> >> 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1
> >> Coarsest grid: 0
> >>
> >> Relaxation Weight 0.600000 level 0
> >> Outer relaxation weight 0.600000 level 0
> >> Output flag (print_level): 3
> >>
> >>
> >> AMG SOLUTION INFO:
> >> relative
> >> residual factor residual
> >> -------- ------ --------
> >> Initial 4.430431e-03 1.000000e+00
> >> Cycle 1 4.540275e-03 1.024793 1.024793e+00
> >> Cycle 2 4.539148e-03 0.999752 1.024539e+00
> >> Cycle 3 4.620010e-03 1.017814 1.042790e+00
> >> Cycle 4 5.196532e-03 1.124788 1.172918e+00
> >> Cycle 5 6.243043e-03 1.201386 1.409128e+00
> >> Cycle 6 7.310008e-03 1.170905 1.649954e+00
> >>
> >>
> >> ==============================================
> >> NOTE: Convergence tolerance was not achieved
> >> within the allowed 6 V-cycles
> >> ==============================================
> >>
> >> Average Convergence Factor = 1.087039
> >>
> >> Complexity: grid = 1.000000
> >> operator = 1.000000
> >> cycle = 1.000000
> >>
> >>
> >>
> >>
> >>
> >> BoomerAMG SOLVER PARAMETERS:
> >>
> >> Maximum number of cycles: 6
> >> Stopping Tolerance: 1.000000e-05
> >> Cycle type (1 = V, 2 = W, etc.): 1
> >>
> >> Relaxation Parameters:
> >> Visiting Grid: down up coarse
> >> Visiting Grid: down up coarse
> >> Number of partial sweeps: 20 20 1
> >> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> >> Point types, partial sweeps (1=C, -1=F):
> >> Pre-CG relaxation (down): 1 -1 1 -1 1
> >> -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1
> >> Post-CG relaxation (up): -1 1 -1 1 -1
> >> 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1
> >> Coarsest grid: 0
> >>
> >> Relaxation Weight 0.600000 level 0
> >> Outer relaxation weight 0.600000 level 0
> >> Output flag (print_level): 3
> >>
> >>
> >> AMG SOLUTION INFO:
> >> relative
> >> residual factor residual
> >> -------- ------ --------
> >> Initial 7.914114e-03 1.000000e+00
> >> Cycle 1 8.320104e-03 1.051300 1.051300e+00
> >> Cycle 2 7.767803e-03 0.933618 9.815126e-01
> >> Cycle 3 6.570752e-03 0.845896 8.302575e-01
> >> Cycle 4 5.836166e-03 0.888204 7.374377e-01
> >> Cycle 5 6.593214e-03 1.129717 8.330956e-01
> >> Cycle 6 8.191117e-03 1.242356 1.035001e+00
> >>
> >>
> >> ==============================================
> >> NOTE: Convergence tolerance was not achieved
> >> within the allowed 6 V-cycles
> >> ==============================================
> >>
> >> Average Convergence Factor = 1.005750
> >>
> >> Complexity: grid = 1.000000
> >> operator = 1.000000
> >> cycle = 1.000000
> >>
> >>
> >>
> >>
> >>
> >> BoomerAMG SOLVER PARAMETERS:
> >>
> >> Maximum number of cycles: 6
> >> Stopping Tolerance: 1.000000e-05
> >> Cycle type (1 = V, 2 = W, etc.): 1
> >>
> >> Relaxation Parameters:
> >> Visiting Grid: down up coarse
> >> Visiting Grid: down up coarse
> >> Number of partial sweeps: 20 20 1
> >> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> >> Point types, partial sweeps (1=C, -1=F):
> >> Pre-CG relaxation (down): 1 -1 1 -1 1
> >> -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1
> >> Post-CG relaxation (up): -1 1 -1 1 -1
> >> 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1
> >> Coarsest grid: 0
> >>
> >> Relaxation Weight 0.600000 level 0
> >> Outer relaxation weight 0.600000 level 0
> >> Output flag (print_level): 3
> >>
> >>
> >> AMG SOLUTION INFO:
> >> relative
> >> residual factor residual
> >> -------- ------ --------
> >> Initial 5.553016e-03 1.000000e+00
> >> Cycle 1 6.417482e-03 1.155675 1.155675e+00
> >> Cycle 2 7.101926e-03 1.106653 1.278931e+00
> >> Cycle 3 7.077471e-03 0.996556 1.274527e+00
> >> Cycle 4 6.382835e-03 0.901853 1.149436e+00
> >> Cycle 5 5.392392e-03 0.844827 9.710744e-01
> >> Cycle 6 4.674173e-03 0.866809 8.417359e-01
> >>
> >>
> >> ==============================================
> >> NOTE: Convergence tolerance was not achieved
> >> within the allowed 6 V-cycles
> >> ==============================================
> >>
> >> Average Convergence Factor = 0.971694
> >>
> >> Complexity: grid = 1.000000
> >> operator = 1.000000
> >> cycle = 1.000000
> >>
> >>
> >>
> >>
> >>
> >> BoomerAMG SOLVER PARAMETERS:
> >>
> >> Maximum number of cycles: 6
> >> Stopping Tolerance: 1.000000e-05
> >> Cycle type (1 = V, 2 = W, etc.): 1
> >>
> >> Relaxation Parameters:
> >> Visiting Grid: down up coarse
> >> Visiting Grid: down up coarse
> >> Number of partial sweeps: 20 20 1
> >> Type 0=Jac, 3=hGS, 6=hSGS, 9=GE: 6 6 6
> >> Point types, partial sweeps (1=C, -1=F):
> >> Pre-CG relaxation (down): 1 -1 1 -1 1
> >> -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1
> >> -1 1 -1 1 -1
> >> Post-CG relaxation (up): -1 1 -1 1 -1
> >> 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
> >> 1 -1 1 -1 1
> >> Coarsest grid: 0
> >>
> >> Relaxation Weight 0.600000 level 0
> >> Outer relaxation weight 0.600000 level 0
> >> Output flag (print_level): 3
> >>
> >>
> >> AMG SOLUTION INFO:
> >> relative
> >> residual factor residual
> >> -------- ------ --------
> >> Initial 3.846663e-03 1.000000e+00
> >> Cycle 1 4.136662e-03 1.075390 1.075390e+00
> >> Cycle 2 4.463861e-03 1.079097 1.160450e+00
> >> Cycle 3 4.302262e-03 0.963798 1.118440e+00
> >> Cycle 4 4.175328e-03 0.970496 1.085441e+00
> >> Cycle 5 4.735871e-03 1.134251 1.231163e+00
> >> Cycle 6 5.809054e-03 1.226607 1.510154e+00
> >>
> >>
> >> ==============================================
> >> NOTE: Convergence tolerance was not achieved
> >> within the allowed 6 V-cycles
> >> ==============================================
> >>
> >> Average Convergence Factor = 1.071117
> >>
> >> Complexity: grid = 1.000000
> >> operator = 1.000000
> >> cycle = 1.000000
> >>
> >>
> >>>
> >>> Nicolas,
> >>>
> >>> I have added support for both 1 and/or 2 to petsc-dev
> >>> http://www-unix.mcs.anl.gov/petsc/petsc-as/developers/index.html
> >>>
> >>> The two new options are -pc_hypre_boomeramg_nodal_coarsen and
> >>> -pc_hypre_boomeramg_nodal_relaxation <optional:n) where the optional
> >>> argument n indicates the levels which the SmoothNumLevels() sets.
> >>>
> >>> I have not tested the code so please let me know what problems you
> >>> have.
> >>>
> >>> Allison,
> >>>
> >>> Thank you very much for the clarifications.
> >>>
> >>> Barry
> >>>
> >>> On Wed, 25 Apr 2007, Allison Baker wrote:
> >>>
> >>>> Hi Barry and Nicolas,
> >>>>
> >>>> To clarify,
> >>>>
> >>>> HYPRE_BoomerAMGSetNumFunctions(solver, int num_functions) tells
> >>>> AMG to
> >>>> solve a
> >>>> system of equations with the specified number of functions/
> >>>> unknowns. The
> >>>> default AMG scheme to solve a PDE system is the "unknown"
> >>>> approach. (The
> >>>> coarsening and interpolation are determined by looking at each
> >>>> unknown/function independently - therefore you can imagine that the
> >>>> coarse
> >>>> grids are generally not the same for each variable. This approach
> >>>> generally
> >>>> works well unless you have strong coupling between unknowns.)
> >>>>
> >>>> HYPRE_BOOMERAMGSetNodal(solver, int nodal ) tells AMG to coarsen
> >>>> such
> >>>> that
> >>>> each variable has the same coarse grid - sometimes this is more
> >>>> "physical" for
> >>>> a particular problem. The value chosen here for nodal determines how
> >>>> strength
> >>>> of connection is determined between the coupled system. I suggest
> >>>> setting
> >>>> nodal = 1, which uses a Frobenius norm. This does NOT tell AMG
> >>>> to use
> >>>> nodal
> >>>> relaxation.
> >>>>
> >>>> If you want to use nodal relaxation in hypre there are two choices:
> >>>>
> >>>> (1) If you call HYPRE_BOOMERAMGSetNodal, then you can
> >>>> additionally do
> >>>> nodal
> >>>> relaxation via the schwarz smoother option in hypre. I did not
> >>>> implement this
> >>>> in the Petsc interface, but it could be done easy enough. The
> >>>> following
> >>>> four
> >>>> functions need to be called:
> >>>>
> >>>> HYPRE_BoomerAMGSetSmoothType(solver, 6);
> >>>> HYPRE_BoomerAMGSetDomainType(solver, 1);
> >>>> HYPRE_BoomerAMGSetOverlap(solver, 0);
> >>>> HYPRE_BoomerAMGSetSmoothNumLevels(solver, num_levels); (Set
> >>>> num_levels
> >>>> to number of levels on which you want nodal smoothing, i.e.
> >>>> 1=just the
> >>>> fine
> >>>> grid, 2= fine grid and the grid below, etc. I find that doing nodal
> >>>> relaxation on just the finest level is generally sufficient.)
> >>>> Note that
> >>>> the
> >>>> interpolation scheme used will be the same as in the unknown
> >>>> approach -
> >>>> so
> >>>> this is what we call a hybrid systems method.
> >>>>
> >>>> (2) You can do both nodal smoothing and a nodal interpolation
> >>>> scheme.
> >>>> While
> >>>> this is currently implemented in 2.0.0, it is not advertised (i.e.,
> >>>> mentioned
> >>>> at all in the user's manual) because it is not yet implemented very
> >>>> efficiently (the fine grid matrix is converted to a block matrix
> >>>> - and
> >>>> both
> >>>> are stored), and we have not found it to be as effective as
> >>>> advertised
> >>>> elsewhere (this is an area of current research for us)..... If
> >>>> you want
> >>>> to try
> >>>> it anyway, let me know and I will provide more info.
> >>>>
> >>>> Hope this helps,
> >>>> Allison
> >>>>
> >>>>
> >>>> Barry Smith wrote:
> >>>>> Nicolas,
> >>>>>
> >>>>> On Wed, 25 Apr 2007, Nicolas Bathfield wrote:
> >>>>>
> >>>>>
> >>>>>> Dear Barry,
> >>>>>>
> >>>>>> Using MatSetBlockSize(A,5) improved my results greatly. Boomemramg
> >>>> is now
> >>>>>> solving the system of equations.
> >>>>>>
> >>>>>
> >>>>> Good
> >>>>>
> >>>>>
> >>>>>> Still, the equations I solve are coupled, and my discretization
> >>>> scheme is
> >>>>>> meant for a non-segregated solver. As a consequence (I believe),
> >>>> boomeramg
> >>>>>> still diverges.
> >>>>>>
> >>>>>
> >>>>> How can "Boomeramg be now solving the system of equations" but
> >>>>> also
> >>>>> diverge? I am so confused.
> >>>>>
> >>>>>
> >>>>>> I would therefore like to use the nodal relaxation in
> >>>>>> boomeramg (the hypre command is HYPRE_BOOMERAMGSetNodal) in
> >>>>>> order to
> >>>>>> couple the coarse grid choice for all my variables.
> >>>>>>
> >>>>>
> >>>>> I can add this this afternoon.
> >>>>>
> >>>>> I have to admit I do not understand the difference between
> >>>>> HYPRE_BOOMERAMGSetNodal() and hypre_BoomerAMGSetNumFunctions(). Do
> >>>> you?
> >>>>>
> >>>>> Barry
> >>>>>
> >>>>>> How can I achieve this from PETSc?
> >>>>>>
> >>>>>> Best regards,
> >>>>>>
> >>>>>> Nicolas
> >>>>>>
> >>>>>>
> >>>>>>> From PETSc MPIAIJ matrices you need to set the block size of
> >>>>>>> the
> >>>>>>> matrix
> >>>>>>> with MatSetBlockSize(A,5) after you have called MatSetType() or
> >>>>>>> MatCreateMPIAIJ(). Then HYPRE_BoomerAMGSetNumFunctions() is
> >>>>>>> automatically
> >>>>>>> called by PETSc.
> >>>>>>>
> >>>>>>> Barry
> >>>>>>>
> >>>>>>> The reason this is done this way instead of as
> >>>>>>> -pc_hypre_boomeramg_block_size <bs> is the idea that hypre will
> >>>> use the
> >>>>>>> properties of the matrix it is given in building the
> >>>> preconditioner so
> >>>>>>> the user does not have to pass those properties in seperately
> >>>> directly
> >>>>>>> to hypre.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, 13 Apr 2007, Shaman Mahmoudi wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> int HYPRE_BoomerAMGSetNumFunctions (.....)
> >>>>>>>>
> >>>>>>>> sets the size of the system of PDEs.
> >>>>>>>>
> >>>>>>>> With best regards, Shaman Mahmoudi
> >>>>>>>>
> >>>>>>>> On Apr 13, 2007, at 2:04 PM, Shaman Mahmoudi wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Hi Nicolas,
> >>>>>>>>>
> >>>>>>>>> You are right. hypre has changed a lot since the version I
> >>>> used.
> >>>>>>>>>
> >>>>>>>>> I found this interesting information:
> >>>>>>>>>
> >>>>>>>>> int HYPRE_BOOMERAMGSetNodal(....)
> >>>>>>>>>
> >>>>>>>>> Sets whether to use the nodal systems version. Default is 0.
> >>>>>>>>>
> >>>>>>>>> Then information about smoothers:
> >>>>>>>>>
> >>>>>>>>> One interesting thing there is this,
> >>>>>>>>>
> >>>>>>>>> HYPRE_BoomerAMGSetDomainType(....)
> >>>>>>>>>
> >>>>>>>>> 0 - each point is a domain (default)
> >>>>>>>>> 1 each node is a domain (only of interest in systems AMG)
> >>>>>>>>> 2 ....
> >>>>>>>>>
> >>>>>>>>> I could not find how you define the nodal displacement
> >>>> ordering. But
> >>>>>>>>>
> >>>>>>>> it
> >>>>>>>>
> >>>>>>>>> should be there somewhere.
> >>>>>>>>>
> >>>>>>>>> I read the reference manual for hypre 2.0
> >>>>>>>>>
> >>>>>>>>> With best regards, Shaman Mahmoudi
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Apr 13, 2007, at 1:40 PM, Nicolas Bathfield wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Dear Shaman,
> >>>>>>>>>>
> >>>>>>>>>> As far as I could understand, there is a BoomerAMG?s systems
> >>>> AMG
> >>>>>>>>>>
> >>>>>>>> version
> >>>>>>>>
> >>>>>>>>>> available. This seems to be exactly what I am looking for,
> >>>> but I
> >>>>>>>>>>
> >>>>>>>> just
> >>>>>>>>
> >>>>>>>>>> don't know how to access it, either through PETSc or
> >>>> directly.
> >>>>>>>>>>
> >>>>>>>>>> Best regards,
> >>>>>>>>>>
> >>>>>>>>>> Nicolas
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> You want to exploit the structure of the model?
> >>>>>>>>>>> As far as I know, boomeramg can not treat a set of rows or
> >>>>>>>>>>> blocks
> >>>>>>>>>>>
> >>>>>>>> as
> >>>>>>>>
> >>>>>>>>>>> a molecule, a so called block-smoother?
> >>>>>>>>>>> ML 2.0 smoothed aggregation does support it.
> >>>>>>>>>>>
> >>>>>>>>>>> With best regards, Shaman Mahmoudi
> >>>>>>>>>>>
> >>>>>>>>>>> On Apr 13, 2007, at 10:45 AM, Nicolas Bathfield wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am solving the Navier-stokes equations and try to use
> >>>> Hypre
> >>>>>>>>>>>> as
> >>>>>>>>>>>> preconditioner.
> >>>>>>>>>>>> Until now, I used PETSc as non-segregated solver and it
> >>>> worked
> >>>>>>>>>>>> perfectly.
> >>>>>>>>>>>> Things got worse when I decided to use Boomeramg
> >>>> (Hypre).
> >>>>>>>>>>>> As I solve a system of PDEs, each cell is represented
> >>>> by 5
> >>>>>>>>>>>> rows
> >>>>>>>>>>>>
> >>>>>>>> in my
> >>>>>>>>
> >>>>>>>>>>>> matrix (I solve for 5 variables). PETSc handles that
> >>>> without
> >>>>>>>>>>>>
> >>>>>>>> problem
> >>>>>>>>
> >>>>>>>>>>>> apparently, but the coarsening scheme of Boomeramg needs
> >>>> more
> >>>>>>>>>>>>
> >>>>>>>> input in
> >>>>>>>>
> >>>>>>>>>>>> order to work properly.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is there an option in PETSc to tell HYPRE that we are
> >>>> dealing
> >>>>>>>>>>>>
> >>>>>>>> with a
> >>>>>>>>
> >>>>>>>>>>>> system of PDEs? (something like:
> >>>> -pc_hypre_boomeramg_...)
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for your help.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Nicolas
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Nicolas BATHFIELD
> >>>>>>>>>>>> Chalmers University of Technology
> >>>>>>>>>>>> Shipping and Marine Technology
> >>>>>>>>>>>> phone: +46 (0)31 772 1476
> >>>>>>>>>>>> fax: +46 (0)31 772 3699
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Nicolas BATHFIELD
> >>>>>>>>>> Chalmers University of Technology
> >>>>>>>>>> Shipping and Marine Technology
> >>>>>>>>>> phone: +46 (0)31 772 1476
> >>>>>>>>>> fax: +46 (0)31 772 3699
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Nicolas BATHFIELD
> >> Chalmers University of Technology
> >> Shipping and Marine Technology
> >> phone: +46 (0)31 772 1476
> >> fax: +46 (0)31 772 3699
> >>
> >
> >
>
>
>
More information about the petsc-users
mailing list