[petsc-users] CG with right preconditioning supports NONE norm type only
Barry Smith
bsmith at mcs.anl.gov
Wed Mar 8 20:57:22 CST 2017
Patrick,
Thanks, this is interesting, we should try to add material to the KSPCG page to capture some of the subtleties that I admit after 28 years I still don't really understand. The paper A TAXONOMY FOR CONJUGATE GRADIENT METHODS (attached) has some interesting discussion (particularly page 1548 "Therefore, only left preconditioning need be considered: Right preconditioning may be effected by incorporating it into the left preconditioner and inner product." I don't know exactly what this means in practical terms in respect to code. (In PETSc KSP we don't explicitly talk about, or use a separate "inner product" in the Krylov methods, we only have the concept of operator and preconditioner operator.)
Remembering vaguely, perhaps incorrectly, from a real long time ago "left preconditioning with a spd M is just the unpreconditioned cg in the M inner product" while "right preconditioning with M is unpreconditioned cg in a M^-1 inner product". If this is correct then it implies (I think) that right preconditioned CG would produce a different set of iterates than left preconditioning and hence is "missing" in terms of completeness from PETSc KSP.
Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0727091.pdf
Type: application/pdf
Size: 3382522 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170308/725fdbe0/attachment-0001.pdf>
-------------- next part --------------
> On Mar 8, 2017, at 7:37 PM, Patrick Sanan <patrick.sanan at gmail.com> wrote:
>
> On Wed, Mar 8, 2017 at 11:12 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>
>>> On Mar 8, 2017, at 10:47 AM, Kong, Fande <fande.kong at inl.gov> wrote:
>>>
>>> Thanks Barry,
>>>
>>> We are using "KSPSetPCSide(ksp, pcside)" in the code. I just tried "-ksp_pc_side right", and petsc did not error out.
>>>
>>> I like to understand why CG does not work with right preconditioning? Mathematically, the right preconditioning does not make sense?
>>
>> No, mathematically it makes sense to do it on the right. It is just that the PETSc code was never written to support it on the right. One reason is that CG is interesting that you can run with the true residual or the preconditioned residual with left preconditioning, hence less incentive to ever bother writing it to support right preconditioning. For completeness we should support right as well as symmetric.
>
> For standard CG preconditioning, which PETSc calls left
> preconditioning, you use a s.p.d. preconditioner M to define an inner
> product in the algorithm, and end up finding iterates x_k in K_k(MA;
> Mb). That isn't quite the same as left-preconditioned GMRES, where you
> apply standard GMRES to the equivalent system MAx=Mb, and also end up
> finding iterates in K_k(MA,Mb). This wouldn't work for CG because MA
> isn't s.p.d. in general, even if M and A are.
>
> Standard CG preconditioning is often motivated as a clever way to do
> symmetric preconditioning, E^TAEy = E^Tb, x=Ey, without ever needing E
> explicitly, using only M=EE^T . y_k lives in K_k(E^TAE,E^Tb) and thus
> x_k again lives in K_k(MA;Mb).
>
> Thus, it's not clear that there is an candidate for a
> right-preconditioned CG variant, as what PETSc calls "left"
> preconditioning doesn't arise in the same way that it does for other
> Krylov methods, namely using the standard algorithm on MAx=Mb. For
> GMRES you would get a right-preconditioned variant by looking at the
> transformed system AMy=b, x = My. This means that y_k lives in
> K_k(AM,b), so x lives in K_k(MA,Mb), as before. For CG, AM wouldn't be
> spd in general so this approach wouldn't make sense.
>
> Another way to look at the difference in "left" preconditioning
> between GMRES and CG is that
>
> - introducing left preconditioning for GMRES alters both the Krylov
> subspaces *and* the optimality condition: you go from minimizing || b
> - Ax_k ||_2 over K_k(A;b) to minimizing || M (b-Ax_k) ||_2 over
> K_k(MA;Mb).
>
> - introducing "left" preconditioning for CG alters *only* the Krylov
> subspaces: you always minimize || x - x_k ||_A , but change the space
> from K_k(A;b) to K_k(MA;Mb).
>
> Thus, my impression is that it's misleading to call standard CG
> preconditioning "left" preconditioning in PETSc - someone might think
> of GMRES and naturally ask why there is no right preconditioning.
>
> One might define a new entry in PCSide to be used with CG and friends.
> I can't think of any slam dunk suggestions yet, but something in the
> genre of PC_INNERPRODUCT, PC_METRIC, PC_CG, or PC_IMPLICITSYMMETRIC,
> perhaps.
>
>
>>
>> Barry
>>
>>>
>>> Fande,
>>>
>>> On Wed, Mar 8, 2017 at 9:33 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>
>>> Please tell us how you got this output.
>>>
>>> PETSc CG doesn't even implement right preconditioning. If you ask for it it should error out. CG supports no norm computation with left preconditioning.
>>>
>>> Barry
>>>
>>>> On Mar 8, 2017, at 10:26 AM, Kong, Fande <fande.kong at inl.gov> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> The NONE norm type is supported only when CG is used with a right preconditioner. Any reason for this?
>>>>
>>>>
>>>>
>>>> 0 Nonlinear |R| = 1.732051e+00
>>>> 0 Linear |R| = 0.000000e+00
>>>> 1 Linear |R| = 0.000000e+00
>>>> 2 Linear |R| = 0.000000e+00
>>>> 3 Linear |R| = 0.000000e+00
>>>> 4 Linear |R| = 0.000000e+00
>>>> 5 Linear |R| = 0.000000e+00
>>>> 6 Linear |R| = 0.000000e+00
>>>> 1 Nonlinear |R| = 1.769225e-08
>>>> 0 Linear |R| = 0.000000e+00
>>>> 1 Linear |R| = 0.000000e+00
>>>> 2 Linear |R| = 0.000000e+00
>>>> 3 Linear |R| = 0.000000e+00
>>>> 4 Linear |R| = 0.000000e+00
>>>> 5 Linear |R| = 0.000000e+00
>>>> 6 Linear |R| = 0.000000e+00
>>>> 7 Linear |R| = 0.000000e+00
>>>> 8 Linear |R| = 0.000000e+00
>>>> 9 Linear |R| = 0.000000e+00
>>>> 10 Linear |R| = 0.000000e+00
>>>> 2 Nonlinear |R| = 0.000000e+00
>>>> SNES Object: 1 MPI processes
>>>> type: newtonls
>>>> maximum iterations=50, maximum function evaluations=10000
>>>> tolerances: relative=1e-08, absolute=1e-50, solution=1e-50
>>>> total number of linear solver iterations=18
>>>> total number of function evaluations=23
>>>> norm schedule ALWAYS
>>>> SNESLineSearch Object: 1 MPI processes
>>>> type: bt
>>>> interpolation: cubic
>>>> alpha=1.000000e-04
>>>> maxstep=1.000000e+08, minlambda=1.000000e-12
>>>> tolerances: relative=1.000000e-08, absolute=1.000000e-15, lambda=1.000000e-08
>>>> maximum iterations=40
>>>> KSP Object: 1 MPI processes
>>>> type: cg
>>>> maximum iterations=10000, initial guess is zero
>>>> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.
>>>> right preconditioning
>>>> using NONE norm type for convergence test
>>>> PC Object: 1 MPI processes
>>>> type: hypre
>>>> HYPRE BoomerAMG preconditioning
>>>> HYPRE BoomerAMG: Cycle type V
>>>> HYPRE BoomerAMG: Maximum number of levels 25
>>>> HYPRE BoomerAMG: Maximum number of iterations PER hypre call 1
>>>> HYPRE BoomerAMG: Convergence tolerance PER hypre call 0.
>>>> HYPRE BoomerAMG: Threshold for strong coupling 0.25
>>>> HYPRE BoomerAMG: Interpolation truncation factor 0.
>>>> HYPRE BoomerAMG: Interpolation: max elements per row 0
>>>> HYPRE BoomerAMG: Number of levels of aggressive coarsening 0
>>>> HYPRE BoomerAMG: Number of paths for aggressive coarsening 1
>>>> HYPRE BoomerAMG: Maximum row sums 0.9
>>>> HYPRE BoomerAMG: Sweeps down 1
>>>> HYPRE BoomerAMG: Sweeps up 1
>>>> HYPRE BoomerAMG: Sweeps on coarse 1
>>>> HYPRE BoomerAMG: Relax down symmetric-SOR/Jacobi
>>>> HYPRE BoomerAMG: Relax up symmetric-SOR/Jacobi
>>>> HYPRE BoomerAMG: Relax on coarse Gaussian-elimination
>>>> HYPRE BoomerAMG: Relax weight (all) 1.
>>>> HYPRE BoomerAMG: Outer relax weight (all) 1.
>>>> HYPRE BoomerAMG: Using CF-relaxation
>>>> HYPRE BoomerAMG: Not using more complex smoothers.
>>>> HYPRE BoomerAMG: Measure type local
>>>> HYPRE BoomerAMG: Coarsen type Falgout
>>>> HYPRE BoomerAMG: Interpolation type classical
>>>> linear system matrix followed by preconditioner matrix:
>>>> Mat Object: 1 MPI processes
>>>> type: mffd
>>>> rows=9, cols=9
>>>> Matrix-free approximation:
>>>> err=1.49012e-08 (relative error in function evaluation)
>>>> Using wp compute h routine
>>>> Does not compute normU
>>>> Mat Object: () 1 MPI processes
>>>> type: seqaij
>>>> rows=9, cols=9
>>>> total: nonzeros=49, allocated nonzeros=49
>>>> total number of mallocs used during MatSetValues calls =0
>>>> not using I-node routines
>>>>
>>>> Fande,
More information about the petsc-users
mailing list