[petsc-users] Convergence on Axisymmetric Poisson matrix

[Barry Smith]

> On Oct 25, 2021, at 4:14 PM, Lionel CHENG <cheng at cerfacs.fr> wrote:
> 
> The number of iterations at initialization (with rtol=1e-10) for both cg is 11 for gmres and 12 for cg so roughly the same. Switching to bcgs the number of iterations goes down to 6. So CG does not seem to fake it. 

   Yes, this sounds reasonable enough, maybe a touch of loss of orthogonality with the CG.
> 
> Going back to the number of multi grid levels: how should I choose the number of multi grid level depending on the problem at hand? How does GAMG evaluate the number of grid levels required?

   Usually we just stick with the heuristic that GAMG comes up with. Maybe Mark has some better advice.

  Barry

> 
> Lionel
> 
> De: "Barry Smith" <bsmith at petsc.dev>
> À: "cheng" <cheng at cerfacs.fr>
> Cc: "petsc-users" <petsc-users at mcs.anl.gov>
> Envoyé: Lundi 25 Octobre 2021 19:52:26
> Objet: Re: [petsc-users] Convergence on Axisymmetric Poisson matrix
> 
> 
> 
> On Oct 25, 2021, at 1:01 PM, Lionel CHENG <cheng at cerfacs.fr <mailto:cheng at cerfacs.fr>> wrote:
> 
> We are running with the -ksp_norm_type unpreconditioned so the convergence is done with the true residual for all the previous tests. I have a case with 800 000 nodes that I have run for 200 iterations on 36 CPU cores (Intel Xeon Gold 6140 - Skylake) and the Poisson solver gives me
> 
> | Krylov Solver | Poisson running time [s] |
> | `cg` | 3.9150E+00 |
> | `gmres` | 4.6527E+00 |
> | `bcgs` | 5.4416E+00 |
> 
> Only the ksp_type has been changed in the following line:
> mpirun -np $nb_cpu $exec -ksp_initial_guess_nonzero true \
> -ksp_type bcgs -ksp_norm_type unpreconditioned \
> -ksp_rtol 1e-10 \
> -pc_type gamg -mg_levels_pc_type sor -mg_levels_ksp_type richardson \
> 
> So CG is better than gmres (I have included the BiCGStab runs as well as I have talked about them earlier).
> 
>    I was not interested in the runtime, I was interested in the convergence behavior of CG vs GMRES for this problem. If CG is "faking it" then one would see the GMRES converging faster (its residual would get smaller with fewer iterations).
> 
>   Barry
> 
> 
> I find it really weird that it behaves well with the preconditioner gamg I can't really find an explanation why, it is really against my intuition. 
> 
> Apart from that I have also played around with the number of multi-grid levels (-pc_mg_levels):
> 
> | Number of MG levels | Poisson running time [s] |
> | ------------------------------- | ------------------------ |
> | 2 | 1.0385E+01 |
> | 3 | 5.0015E+00 |
> | 4 | 3.9150E+00 |
> | 5 | 4.5015E+00 |
> | 6 (default petsc for this case) | 4.5510E+00 |
> 
> So that I find an optimum for 4 and not 6 as in the default PETSc configuration and not specifying anything. How should I choose the number of multi grid level depending on my problem? How does GAMG evaluate the number of grid levels required?
> 
> Lionel
> 
> De: "Barry Smith" <bsmith at petsc.dev <mailto:bsmith at petsc.dev>>
> À: "cheng" <cheng at cerfacs.fr <mailto:cheng at cerfacs.fr>>
> Cc: "petsc-users" <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>>
> Envoyé: Lundi 25 Octobre 2021 15:33:50
> Objet: Re: [petsc-users] Convergence on Axisymmetric Poisson matrix
> 
> 
>    Are you running with -ksp_monitor_true_residual to track the b - A*x residual instead of just the preconditioned residual? 
> 
>    GAMG definitely does not symmetrize the system but it is possible the preconditioner results in the solve "not seeing" the unsymmetry during the solution process and hence CG still converging; it would be dangerous to rely on this in general I think. You could also run this case with GMRES to see if that is better than the CG iterations.
> 
>    Barry
> 
> On Oct 24, 2021, at 7:00 PM, Lionel CHENG <cheng at cerfacs.fr <mailto:cheng at cerfacs.fr>> wrote:
> 
> Hello everyone,
> 
> I have some questions regarding a linear system that I am solving in my plasma simulations. We have in this case a strongly non-symmetric matrix due to the cylindrical coordinates for which the Laplacian cell is given by Fig. 2 for two kinds of triangles. The different unstructured grids have from 300 000 nodes to 7 000 000 nodes.
> 
> To my understanding, CG should not work properly on this matrix but BiCGStab(1) should. When using SOR preconditioner it is indeed the case: -ksp_type cg -pc_type sor yields solutions in 10 to 20 times more iterations than -ksp_type bcgs -pc_type sor.
> 
> However, when switching to -ksp_type cg -pc_type gamg the convergence is great and even slightly better than -ksp_type bcgs. I do not understand how CG is able to make the system converge when using GAMG although the matrix is non-symmetric ? Is GAMG able to somehow symmetrize the system? I have the impression that when using -pc_type gamg the Krylov solver is actually the Pre-relaxation and post-relaxation of the initial grid, is that right?
> 
> For GAMG since the matrix is non-symmetric -mg_levels_pc_type sor for and -mg_levels_ksp_type richardson have been used and yields better results than the original chebychev solver.
> 
> Sincerely yours,
> 
> Lionel Cheng
> <main.pdf>
> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211025/8125a3f2/attachment-0001.html>