Setting tolerances for multigrid

Sat Jun 16 16:56:59 CDT 2007

On Sat, 16 Jun 2007, Knut Erik Teigen wrote:

> On Fri, 2007-06-15 at 14:30 -0500, Barry Smith wrote:
> >   What happens if you run the two level with -ksp_rtol 1.e-12
> 
> The exact same thing happens. At the third time step, the solution with
> two levels begin to deviate from the one with one level.
> 
> > 
> >   BTW: when you just change -dmmg_nlevels from 1 to 2 you are now
> > solving a NEW problem that has twice as many points in each direction,
> > this can also effect the time-stepping. You can use -dmmg_nlevels 2
> > -pc_type lu to compare with the -dmmg_nlevels 2
> 
> I forgot to mention that I adjust the coarsest level according to the
> number of levels, so that the finest level always contains the same
> amount of points. I.e. with one level it is 65x129, with two 33x65 and
> so on.
> 
> > 
> >   Always verify things with a direct solver before using multigrid
> > or any other iterative linear solver. Multigrid is ONLY an optimization
> > (faster, less memory), debug BEFORE turning on the optimizations.
> 
> Yes, I've already done this. With a direct solver, I get the same
> solution as with one level. Since multigrid is only an optimization as
> you mention, I don't understand why the solution should be different
> when running with two or more levels as opposed to only one.
> Here's the output when running with -pc_type lu. As you can see, with
> two levels, suddenly more iterations are needed for the third step.
> 
> -dmmg_nlevels 1
>  Start of time loop:
>  L-s reinit converged after   10 steps with norm  1.716E-09
>   0 KSP preconditioned resid norm 1.366287028032e+04 true resid norm 3.168341703825e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.325862387561e+02 true resid norm 8.783238319618e-07 ||Ae||/||Ax|| 2.772187832207e-15
>   2 KSP preconditioned resid norm 4.237324220037e-10 true resid norm 9.365968491320e-07 ||Ae||/||Ax|| 2.956110598807e-15
> Linear solve converged due to CONVERGED_RTOL
    0 KSP preconditioned resid norm 1.366743784848e+04 true resid norm 3.168385551957e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.578541063222e+02 true resid norm 3.047639475331e-06 ||Ae||/||Ax|| 9.618903461571e-15
>   2 KSP preconditioned resid norm 2.490853654289e-10 true resid norm 2.047397542219e-06 ||Ae||/||Ax|| 6.461958333808e-15
> Linear solve converged due to CONVERGED_RTOL
>   0 KSP preconditioned resid norm 1.373641039770e+04 true resid norm 4.083933364397e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 2.756300642422e+02 true resid norm 1.333942531534e-06 ||Ae||/||Ax|| 3.266318062786e-15
>   2 KSP preconditioned resid norm 1.967864218276e-09 true resid norm 4.708778143639e-06 ||Ae||/||Ax|| 1.153000728339e-14
> Linear solve converged due to CONVERGED_RTOL
> 
> -dmmg_nlevels 2
>  Start of time loop:
>  L-s reinit converged after   10 steps with norm  1.716E-09
>   0 KSP preconditioned resid norm 1.366287028032e+04 true resid norm 3.168341703825e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.325862387561e+02 true resid norm 8.783238319618e-07 ||Ae||/||Ax|| 2.772187832207e-15
>   2 KSP preconditioned resid norm 4.237324220037e-10 true resid norm 9.365968491320e-07 ||Ae||/||Ax|| 2.956110598807e-15
> Linear solve converged due to CONVERGED_RTOL
>   0 KSP preconditioned resid norm 1.366743784848e+04 true resid norm 3.168385551957e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.578541063222e+02 true resid norm 3.047639475331e-06 ||Ae||/||Ax|| 9.618903461571e-15
>   2 KSP preconditioned resid norm 2.490853654289e-10 true resid norm 2.047397542219e-06 ||Ae||/||Ax|| 6.461958333808e-15
> Linear solve converged due to CONVERGED_RTOL
>   0 KSP preconditioned resid norm 1.373641039770e+04 true resid norm 4.083933364397e+08 ||Ae||/||Ax|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 2.826990181167e+02 true resid norm 3.621921471554e+07 ||Ae||/||Ax|| 8.868708542429e-02
>   2 KSP preconditioned resid norm 2.151728459050e+02 true resid norm 5.126312558070e+05 ||Ae||/||Ax|| 1.255239030774e-03
>   3 KSP preconditioned resid norm 9.420491033127e+01 true resid norm 7.221320422298e+02 ||Ae||/||Ax|| 1.768226799500e-06
> (...)
> 118 KSP preconditioned resid norm 2.011348229632e-04 true resid norm 1.933530019765e-05 ||Ae||/||Ax|| 4.734479843920e-14
> 119 KSP preconditioned resid norm 1.002113384828e-04 true resid norm 1.933774400466e-05 ||Ae||/||Ax|| 4.735078239337e-14
> 120 KSP preconditioned resid norm 3.976002405523e-07 true resid norm 1.947236185243e-05 ||Ae||/||Ax|| 4.768041031762e-14
> Linear solve converged due to CONVERGED_RTOL
> 
> 

  The true residuals both me, they are very big at "convergence" 1.e-5? Are you using KSPSetInitialGuessNonzero() and
using the previous solution as initial guess?

  Run with -ksp_type gmres -pc_type lu -ksp_monitor_true_residual. I bet the converged true residual is
much better. Send the output.

   Barry

> 
> > 
> > 
> >    Barry
> > 
> > On Fri, 15 Jun 2007, Knut Erik Teigen wrote:
> > 
> > > When running with those options I get this after the first timestep for
> > > both one and two levels:
> > > 159 KSP preconditioned resid norm 1.397494864058e-07 true resid norm
> > > 4.996893774970e-02 ||Ae||/||Ax|| 1.577132216812e-10
> > > Linear solve converged due to CONVERGED_RTOL
> > > 
> > > However, after the second timestep they start to get different.
> > > With two levels the norm is actually lower, but still, the computation
> > > diverges after about 6 time steps. With one level the computation runs
> > > smoothly. Here are the numbers:
> > > 
> > > -dmmg_nlevels 1
> > >   0 KSP preconditioned resid norm 1.674785901465e+02 
> > > true resid norm 4.083933362528e+08 ||Ae||/||Ax|| 1.000000000000e+00
> > >   1 KSP preconditioned resid norm 6.256839347623e+01 
> > > true resid norm 4.220478752135e+07 ||Ae||/||Ax|| 1.033434774147e-01
> > > (...)
> > > 159 KSP preconditioned resid norm 1.318896741223e-07 
> > > true resid norm 5.104451971417e-02 ||Ae||/||Ax|| 1.249886204866e-10
> > > Linear solve converged due to CONVERGED_RTOL
> > > 
> > > -dmmg_nlevels 2
> > >   0 KSP preconditioned resid norm 1.674785901465e+02 
> > > true resid norm 4.083933362528e+08 ||Ae||/||Ax|| 1.000000000000e+00
> > >   1 KSP preconditioned resid norm 6.252455641723e+01 
> > > true resid norm 5.561545924132e+07 ||Ae||/||Ax|| 1.361811134129e-01
> > > (...) 
> > > 159 KSP preconditioned resid norm 1.305901659826e-07 
> > > true resid norm 4.297164043386e-02 ||Ae||/||Ax|| 1.052212086224e-10
> > > Linear solve converged due to CONVERGED_RTOL
> > > 
> > > -Knut Erik-
> > > 
> > > On Fri, 2007-06-15 at 10:10 -0500, Barry Smith wrote:
> > > >   You need to make sure the linear solvers are converging properly
> > > > before even worrying about time-stepping. Run with -ksp_monitor_true_residual
> > > > (-ksp_truemonitor on older versions of PETSc) and -ksp_converged_reason
> > > > 
> > > >    Barry
> > > > 
> > > > 
> > > > On Fri, 15 Jun 2007, Knut Erik Teigen wrote:
> > > > 
> > > > > The computation runs just fine with -dmmg_nlevels 1.
> > > > > I use a grid of 65x129, with three levels, so that the coarsest grid is
> > > > > 17x33. 
> > > > > I've diffed the solutions, and they are equal for the first two time
> > > > > steps, but then they start to differ somewhat. Here is an excerpt:
> > > > > -dmmg_nlevels 1:
> > > > > 65.0427492820557        65.0216231814457      64.9793945321386        
> > > > > -dmmg_nlevels 3:
> > > > > 65.0920864319067        65.0711253401585      65.0292265582817
> > > > > 
> > > > > Using 2 or 4 levels produce the same result as 3 levels.
> > > > > 
> > > > > -Knut Erik-
> > > > > 
> > > > > 
> > > > > 
> > > > > On Fri, 2007-06-15 at 10:21 +0200, Matthew Knepley wrote:
> > > > > > This does not sound right to me. I think something must be
> > > > > > configured incorrectly. All MG does is accelerate the solution
> > > > > > of the fine grid problem. I think something must be wrong with
> > > > > > the specification here. It is easy to check. Run with -dmmg_nlevels 1
> > > > > > which just solves on the fine grid.
> > > > > > 
> > > > > >   Matt
> > > > > > 
> > > > > > On 6/15/07, Knut Erik Teigen <knutert at stud.ntnu.no> wrote:
> > > > > > > Thanks, Matthew. It seems like the tolerances weren't my problem,
> > > > > > > though. Even with a tolerance of 1e-10, the computation diverges after
> > > > > > > only a few time steps when using multigrid. With ordinary linear
> > > > > > > solvers, I can use a tolerance of 1e-4 and still get a satisfactory
> > > > > > > solution.
> > > > > > > Could it be that the Galerkin approximation for the coarser levels is
> > > > > > > too inaccurate for my problem?
> > > > > > >
> > > > > > > -Knut Erik-
> > > > > > >
> > > > > > > On Thu, 2007-06-14 at 11:32 -0500, Matthew Knepley wrote:
> > > > > > > > On 6/14/07, Knut Erik Teigen <knutert at stud.ntnu.no> wrote:
> > > > > > > > > Hello,
> > > > > > > > >
> > > > > > > > > How do I set the tolerances for the KSP solvers when using the
> > > > > > > > > DMMG routines without using command line arguments? It seems that
> > > > > > > > > I can use the argument -ksp_rtol on the command line, but in the code I
> > > > > > > > > can't use KSPSetTolerances since I don't have access to the KSP solver
> > > > > > > > > context?
> > > > > > > >
> > > > > > > > You can use DMMGGetKSP() for the finest level, and dmmg[level]->ksp for
> > > > > > > > any others.
> > > > > > > >
> > > > > > > > > Also, I see there is a reference to the function DMMGSetUseGalerkin
> > > > > > > > > in the source code, but I can't find the documentation for it.
> > > > > > > > > Does this function not exist yet? Again I would like to use
> > > > > > > > > -dmmg_galerkin, but setting it in the code instead of on the command
> > > > > > > > > line.
> > > > > > > >
> > > > > > > > I think you want this:
> > > > > > > >
> > > > > > > > http://www-unix.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/DA/DMMGSetUseGalerkinCoarse.html
> > > > > > > >
> > > > > > > >   Matt
> > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Knut Erik Teigen
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> 
> 