[petsc-users] Efficient Use of GAMG for Poisson Equation with Full Neumann Boundary Conditions

[Fabian Gabel]

Hong,

you find attached the stand-alone code. It comes with a small
script (runtest.sh) for compilation and execution. If you are using the
gnu compiler or want compilation with debugging flags just uncomment the
corresponding line. Compilation should terminate successfully if the
environment variable PETSC_DIR is set properly (root directory of PETSc
installation).

Fabian

On Do, 2015-02-19 at 11:45 -0600, Hong wrote:
> Fabian,
> Too much time was spent on the matrix operations during setup phase,
> which has plenty room for optimization. 
> Can you provide us a stand-alone code used in your experiment so we
> can investigate how to make our gamg more efficient?
> 
> 
> Hong
> 
> On Wed, Feb 18, 2015 at 12:20 PM, Barry Smith <bsmith at mcs.anl.gov>
> wrote:
>         
>           Fabian,
>         
>            CG requires that the preconditioner be symmetric positive
>         definite. ICC even if given a symmetric positive definite
>         matrix can generate an indefinite preconditioner.
>         
>           Similarly if an algebraic multigrid application is not
>         "strong enough" it can also result in a preconditioner that is
>         indefinite.
>         
>           You never want to use ICC for pressure type problems it
>         cannot compete with multigrid for large problems so let's
>         forget about ICC and focus on the GAMG.
>         
>         > -pressure_mg_coarse_sub_pc_type svd
>         > -pressure_mg_levels_ksp_rtol 1e-4
>         > -pressure_mg_levels_ksp_type richardson
>         > -pressure_mg_levels_pc_type sor
>         > -pressure_pc_gamg_agg_nsmooths 1
>         > -pressure_pc_type gamg
>         
>           There are many many tuning parameters for MG.
>         
>            First, is your pressure problem changing dramatically at
>         each new solver? That is, for example, is the mesh moving or
>         are there very different numerical values in the matrix?  Is
>         the nonzero structure of the pressure matrix changing?
>         Currently the entire GAMG process is done for each new solve,
>         if you use the flag
>         
>         -pressure_pc_gamg_reuse_interpolation true
>         
>         it will create the interpolation needed for GAMG once and
>         reuse it for all the solves. Please try that and see what
>         happens.
>         
>          Then I will have many more suggestions.
>         
>         
>           Barry
>         
>         
>         
>         > On Feb 17, 2015, at 9:14 AM, Fabian Gabel
>         <gabel.fabian at gmail.com> wrote:
>         >
>         > Dear PETSc team,
>         >
>         > I am trying to optimize the solver parameters for the linear
>         system I
>         > get, when I discretize the pressure correction equation
>         Poisson equation
>         > with Neumann boundary conditions) in a SIMPLE-type algorithm
>         using a
>         > finite volume method.
>         >
>         > The resulting system is symmetric and positive
>         semi-definite. A basis to
>         > the associated nullspace has been provided to the KSP
>         object.
>         >
>         > Using a CG solver with ICC preconditioning the solver needs
>         a lot of
>         > inner iterations to converge (-ksp_monitor -ksp_view output
>         attached for
>         > a case with approx. 2e6 unknowns; the lines beginning with
>         000XXXX show
>         > the relative residual regarding the initial residual in the
>         outer
>         > iteration no. 1 for the variables u,v,w,p). Furthermore I
>         don't quite
>         > understand, why the solver reports
>         >
>         > Linear solve did not converge due to DIVERGED_INDEFINITE_PC
>         >
>         > at the later stages of my Picard iteration process
>         (iteration 0001519).
>         >
>         > I then tried out CG+GAMG preconditioning with success
>         regarding the
>         > number of inner iterations, but without advantages regarding
>         wall time
>         > (output attached). Also the DIVERGED_INDEFINITE_PC reason
>         shows up
>         > repeatedly after iteration 0001487. I used the following
>         options
>         >
>         > -pressure_mg_coarse_sub_pc_type svd
>         > -pressure_mg_levels_ksp_rtol 1e-4
>         > -pressure_mg_levels_ksp_type richardson
>         > -pressure_mg_levels_pc_type sor
>         > -pressure_pc_gamg_agg_nsmooths 1
>         > -pressure_pc_type gamg
>         >
>         > I would like to get an opinion on how the solver performance
>         could be
>         > increased further. -log_summary shows that my code spends
>         80% of the
>         > time solving the linear systems for the pressure correction
>         (STAGE 2:
>         > PRESSCORR). Furthermore, do you know what could be causing
>         the
>         > DIVERGED_INDEFINITE_PC converged reason?
>         >
>         > Regards,
>         > Fabian Gabel
>         > <gamg.128.out.531314><icc.128.out.429762>
>         
> 
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: 128_1_1_seg_selfcontained.tgz
Type: application/x-compressed-tar
Size: 1895410 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150220/2fe4a59f/attachment-0001.bin>