[petsc-users] Using nonzero -pc_hypre_boomeramg_restriction_type in field split

[Pierre Jolivet]

> On 13 Apr 2023, at 10:33 PM, Alexander Lindsay <alexlindsay239 at gmail.com> wrote:
> 
> Hi, I'm trying to solve steady Navier-Stokes for different Reynolds numbers. My options table
> 
> -dm_moose_fieldsplit_names u,p
> -dm_moose_nfieldsplits 2
> -fieldsplit_p_dm_moose_vars pressure
> -fieldsplit_p_ksp_type preonly
> -fieldsplit_p_pc_type jacobi
> -fieldsplit_u_dm_moose_vars vel_x,vel_y
> -fieldsplit_u_ksp_type preonly
> -fieldsplit_u_pc_hypre_type boomeramg
> -fieldsplit_u_pc_type hypre
> -pc_fieldsplit_schur_fact_type full
> -pc_fieldsplit_schur_precondition selfp
> -pc_fieldsplit_type schur
> -pc_type fieldsplit
> 
> works wonderfully for a low Reynolds number of 2.2. The solver performance crushes LU as I scale up the problem. However, not surprisingly this options table struggles when I bump the Reynolds number to 220. I've read that use of AIR (approximate ideal restriction) can improve performance for advection dominated problems. I've tried setting -pc_hypre_boomeramg_restriction_type 1 for a simple diffusion problem and the option works fine. However, when applying it to my field-split preconditioned Navier-Stokes system, I get immediate non-convergence:
> 
>  0 Nonlinear |R| = 1.033077e+03
>       0 Linear |R| = 1.033077e+03
>   Linear solve did not converge due to DIVERGED_NANORINF iterations 0
> Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0
> 
> Does anyone have an idea as to why this might be happening?

Do not use this option, even when not part of PCFIELDSPLIT.
There is some missing plumbing in PETSc which makes it unusable, see Ben’s comment here https://github.com/hypre-space/hypre/issues/764#issuecomment-1353452417.
In fact, it’s quite easy to make HYPRE generate NaN with a very simple stabilized convection—diffusion problem near the pure convection limit (something that ℓAIR is supposed to handle).
Even worse, you can make HYPRE fill your terminal with printf-style debugging messages https://github.com/hypre-space/hypre/blob/5546cc22d46b3dba253849f258786da47c9a7b21/src/parcsr_ls/par_lr_restr.c#L1416 with this option turned on.
As a result, I have been unable to reproduce any of the ℓAIR results.
This also explains why I have been using plain BoomerAMG instead of ℓAIR for the comparison in page 9 of https://arxiv.org/pdf/2201.02250.pdf (if you would like to try the PC we are using, I could send you the command line options).

Thanks,
Pierre

> If not, I'd take a suggestion on where to set a breakpoint to start my own investigation. Alternatively, I welcome other preconditioning suggestions for an advection dominated problem.
> 
> Alex

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230414/b61d6941/attachment.html>