[petsc-users] amg from hypre

[Jed Brown]

On 06 Mar 2010 15:41:12 -0600, hxie at umn.edu wrote:
> Hi,
> 
> I am solving incompressible navier-stokes equations. The code implements 
> the Newton iteration and just uses the KSP solver for each iteration. It 
> takes 14 nonlinear iterations (~8min) to converge using the default solver 
> in PETSc. But it takes 6 nonlinear iterations (~30min) using boomeramg from 
> hypre.

Can you confirm that these are the number of NONLINEAR iterations?
Changing the preconditioner should not change this if it behaving
correctly.  Are you solving incompressible NS in a coupled manner,
sending the indefinite (momentum + continuity equations) to the
preconditioner?  In this case, BoomerAMG can be giving you completely
the wrong answer.  For example, on a different Navier-Stokes example, it
produces very nice preconditioned residuals

   0 KSP Residual norm 9.526419214320e+00 
   1 KSP Residual norm 9.406988599189e-02 
   2 KSP Residual norm 2.767040414824e-03 
   3 KSP Residual norm 1.649604812054e-03 
   4 KSP Residual norm 1.611023301144e-03 
   5 KSP Residual norm 7.073431543229e-04 
   6 KSP Residual norm 1.404066303578e-04 
   7 KSP Residual norm 1.347821147393e-04 
   8 KSP Residual norm 7.713640141302e-05 
   9 KSP Residual norm 5.121198361232e-05 
  10 KSP Residual norm 4.790100151952e-05 
  11 KSP Residual norm 1.769376148248e-05 
  12 KSP Residual norm 1.671836687758e-05 
  13 KSP Residual norm 9.561298137614e-06 
  14 KSP Residual norm 6.509746067580e-07 
  15 KSP Residual norm 5.863323408081e-07 
  16 KSP Residual norm 5.120192651612e-07 
  17 KSP Residual norm 3.403122131501e-07 
  18 KSP Residual norm 3.342529361191e-07 
  19 KSP Residual norm 9.178974981883e-08 

but converges to completely the WRONG answer.  This is because the
preconditioner is singular, and the unpreconditioned residuals (using
right-preconditioned GMRES) look like

   0 KSP Residual norm 9.899494936612e+05
   1 KSP Residual norm 3.460941545811e-02
   2 KSP Residual norm 3.411670842883e-02
   3 KSP Residual norm 3.406431296292e-02
   4 KSP Residual norm 3.376198186394e-02
   5 KSP Residual norm 3.374434209905e-02
   6 KSP Residual norm 3.370086274150e-02
   7 KSP Residual norm 3.334190783058e-02
   8 KSP Residual norm 3.321057363881e-02
   9 KSP Residual norm 3.321055343672e-02
  10 KSP Residual norm 3.318392045928e-02

which is NO CONVERGENCE at all after the penalty boundary conditions
were enforced.  So CPU time aside, the nonlinear iteration count could
occur because Newton is failing due to an incorrect search direction or
a vanishing step size, but the answer could be completely wrong.

You have to clear up subtleties like getting the wrong answer before
looking any further at the amount of time the various preconditioners
are taking.  Always run with

  -snes_converged_reason -ksp_converged_reason

and check the unpreconditioned residuals, especially when dealing with
indefinite problems.

Jed