[petsc-users] Monotonic convergence in FGMRES.

Tue Dec 21 15:34:30 CST 2010

On Dec 21, 2010, at 3:16 PM, Vijay S. Mahadevan wrote:

> Also GCR seems to use and allocate comparatively more vectors,
> translating to lot more memory.

  Yes

> This does make FGMRES more attractive.
> I will look at the preconditioner and try to find the true cause of
> the issue.
> 
> Cheers,
> Vijay
> 
> On Tue, Dec 21, 2010 at 2:52 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>> 
>>  The GCR algorithm computes the residual and hence the residual norm EXPLICITLY as part of the solution process, it does not use the recursive formula that FGMES uses. I "think" the use of the recursive formula is why FGMRES is cheaper than GCR (and hence much more commonly used).
>> 
>>   Barry
>> 
>> 
>> On Dec 21, 2010, at 2:46 PM, Vijay S. Mahadevan wrote:
>> 
>>>> Yes but look at the true residual norm it is huge and indicates the residual is not really getting small.
>>> Ah yes. I was reading the output wrongly. Thanks for pointing that
>>> out. So then it is quite possible that my preconditioner is terrible
>>> for this problem.
>>> 
>>> Curiously with GCR, the true residual does converge.
>>> 
>>> 62 KSP Residual norm 6.845396874593e-10
>>>   62 KSP preconditioned resid norm 6.845396874593e-10 true resid norm
>>> 6.845396874593e-10 ||Ae||/||Ax|| 1.063128003731e+00
>>>   63 KSP preconditioned resid norm 4.617426258215e-10 true resid norm
>>> 4.617426258215e-10 ||Ae||/||Ax|| 9.425403350509e-01
>>> 63 KSP Residual norm 4.617426258215e-10
>>>   63 KSP preconditioned resid norm 4.617426258215e-10 true resid norm
>>> 4.617426258215e-10 ||Ae||/||Ax|| 9.425403350509e-01
>>>   64 KSP preconditioned resid norm 3.659090331422e-10 true resid norm
>>> 3.659090331422e-10 ||Ae||/||Ax|| 1.044433624917e+00
>>> 64 KSP Residual norm 3.659090331422e-10
>>>   64 KSP preconditioned resid norm 3.659090331422e-10 true resid norm
>>> 3.659090331422e-10 ||Ae||/||Ax|| 1.044433624917e+00
>>>   65 KSP preconditioned resid norm 2.457005532004e-10 true resid norm
>>> 2.457005532004e-10 ||Ae||/||Ax|| 9.250757590415e-01
>>> 65 KSP Residual norm 2.457005532004e-10
>>>   65 KSP preconditioned resid norm 2.457005532004e-10 true resid norm
>>> 2.457005532004e-10 ||Ae||/||Ax|| 9.250757590415e-01
>>>   66 KSP preconditioned resid norm 1.765446010945e-10 true resid norm
>>> 1.765446010945e-10 ||Ae||/||Ax|| 9.880804659179e-01
>>> 66 KSP Residual norm 1.765446010945e-10
>>>   66 KSP preconditioned resid norm 1.765446010945e-10 true resid norm
>>> 1.765446010945e-10 ||Ae||/||Ax|| 9.880804659179e-01
>>> 
>>> Jed, with modified gram schmidt procedure, fgmres yields the
>>> following, which looks like the same as before:
>>> 
>>> 49 KSP Residual norm 2.426160176080e-08
>>>   49 KSP preconditioned resid norm 2.426160176080e-08 true resid norm
>>> 1.864897210364e+02 ||Ae||/||Ax|| 2.696456942624e+02
>>>   50 KSP preconditioned resid norm 1.864914790828e+02 true resid norm
>>> 1.864914790828e+02 ||Ae||/||Ax|| 2.798875072987e+02
>>> 50 KSP Residual norm 1.864914790828e+02
>>>   50 KSP preconditioned resid norm 1.864914790828e+02 true resid norm
>>> 1.864914790828e+02 ||Ae||/||Ax|| 2.798875072987e+02
>>>   51 KSP preconditioned resid norm 6.741080961009e+01 true resid norm
>>> 6.759768469363e+01 ||Ae||/||Ax|| 1.666964983874e+02
>>> 51 KSP Residual norm 6.741080961009e+01
>>>   51 KSP preconditioned resid norm 6.741080961009e+01 true resid norm
>>> 6.759768469363e+01 ||Ae||/||Ax|| 1.666964983874e+02
>>>   52 KSP preconditioned resid norm 5.191621875736e+01 true resid norm
>>> 5.146342142561e+01 ||Ae||/||Ax|| 7.225409161988e+01
>>> 
>>> But I generally see that the true residual of GCR seems to converge to
>>> desired tolerance but for GMRES, the convergence stagnates with
>>> different options on my MG preconditioner. This is puzzling to me
>>> since I spent enough time making sure that the preconditioner was
>>> working correctly but I will look more into this now. Thanks for all
>>> the helpful comments guys ! I will post here if I find any other
>>> curious behavior.
>>> 
>>> Vijay
>>> 
>>> On Tue, Dec 21, 2010 at 2:30 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>> 
>>>>  Yes but look at the true residual norm it is huge and indicates the residual is not really getting small.
>>>> 
>>>>  Barry
>>>> 
>>>> On Dec 21, 2010, at 2:26 PM, Vijay S. Mahadevan wrote:
>>>> 
>>>>> Barry, I tried with the true_residual_norm option and it gives me the
>>>>> exact same convergence as the one I have shown before.
>>>>> 
>>>>> 45 KSP Residual norm 2.511364148934e-07
>>>>>   45 KSP preconditioned resid norm 2.511364148934e-07 true resid norm
>>>>> 1.865039278877e+02 ||Ae||/||Ax|| 2.699481989705e+02
>>>>>   46 KSP preconditioned resid norm 1.307034672896e-07 true resid norm
>>>>> 1.864478183180e+02 ||Ae||/||Ax|| 2.724877015479e+02
>>>>> 46 KSP Residual norm 1.307034672896e-07
>>>>>   46 KSP preconditioned resid norm 1.307034672896e-07 true resid norm
>>>>> 1.864478183180e+02 ||Ae||/||Ax|| 2.724877015479e+02
>>>>>   47 KSP preconditioned resid norm 7.105770015635e-08 true resid norm
>>>>> 1.864563163311e+02 ||Ae||/||Ax|| 2.722662760395e+02
>>>>> 47 KSP Residual norm 7.105770015635e-08
>>>>>   47 KSP preconditioned resid norm 7.105770015635e-08 true resid norm
>>>>> 1.864563163311e+02 ||Ae||/||Ax|| 2.722662760395e+02
>>>>>   48 KSP preconditioned resid norm 4.098578230710e-08 true resid norm
>>>>> 1.864560351328e+02 ||Ae||/||Ax|| 2.690284539995e+02
>>>>> 48 KSP Residual norm 4.098578230710e-08
>>>>>   48 KSP preconditioned resid norm 4.098578230710e-08 true resid norm
>>>>> 1.864560351328e+02 ||Ae||/||Ax|| 2.690284539995e+02
>>>>>   49 KSP preconditioned resid norm 2.426160176080e-08 true resid norm
>>>>> 1.864897210364e+02 ||Ae||/||Ax|| 2.696456942624e+02
>>>>> 49 KSP Residual norm 2.426160176080e-08
>>>>>   49 KSP preconditioned resid norm 2.426160176080e-08 true resid norm
>>>>> 1.864897210364e+02 ||Ae||/||Ax|| 2.696456942624e+02
>>>>>   50 KSP preconditioned resid norm 1.864914790828e+02 true resid norm
>>>>> 1.864914790828e+02 ||Ae||/||Ax|| 2.798875072987e+02
>>>>> 50 KSP Residual norm 1.864914790828e+02
>>>>>   50 KSP preconditioned resid norm 1.864914790828e+02 true resid norm
>>>>> 1.864914790828e+02 ||Ae||/||Ax|| 2.798875072987e+02
>>>>>   51 KSP preconditioned resid norm 6.741080961009e+01 true resid norm
>>>>> 6.759768469363e+01 ||Ae||/||Ax|| 1.666964983874e+02
>>>>> 51 KSP Residual norm 6.741080961009e+01
>>>>>   51 KSP preconditioned resid norm 6.741080961009e+01 true resid norm
>>>>> 6.759768469363e+01 ||Ae||/||Ax|| 1.666964983874e+02
>>>>>   52 KSP preconditioned resid norm 5.191621875736e+01 true resid norm
>>>>> 5.146342142561e+01 ||Ae||/||Ax|| 7.225409161988e+01
>>>>> 52 KSP Residual norm 5.191621875736e+01
>>>>>   52 KSP preconditioned resid norm 5.191621875736e+01 true resid norm
>>>>> 5.146342142561e+01 ||Ae||/||Ax|| 7.225409161988e+01
>>>>>   53 KSP preconditioned resid norm 4.513782866249e+01 true resid norm
>>>>> 4.546883708687e+01 ||Ae||/||Ax|| 7.426476446334e+01
>>>>> 53 KSP Residual norm 4.513782866249e+01
>>>>>   53 KSP preconditioned resid norm 4.513782866249e+01 true resid norm
>>>>> 4.546883708687e+01 ||Ae||/||Ax|| 7.426476446334e+01
>>>>>   54 KSP preconditioned resid norm 3.320195603375e+01 true resid norm
>>>>> 3.297361634749e+01 ||Ae||/||Ax|| 5.285029509147e+01
>>>>> 
>>>>> 
>>>>> Vijay
>>>>> 
>>>>> On Tue, Dec 21, 2010 at 2:23 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>>>> 
>>>>>> On Dec 21, 2010, at 2:08 PM, Jed Brown wrote:
>>>>>> 
>>>>>>> On Tue, Dec 21, 2010 at 21:04, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>>>>> This is a sign that the preconditioner is seriously messed up and should not be used in its current form.  It can happen if the matrix is nearly singular and for example you use an incomplete factorization for a preconditioner that just screws up the scaling like totally. Run with -ksp_monitor_true_residual and you'll see that the solver is not really solving the problem even though it thinks it is converging fine.
>>>>>>> 
>>>>>>> FGMRES only does right preconditioning so it should be showing the true residual.
>>>>>> 
>>>>>>  No because it uses a recursive formula for "computing" the residual norm it does not compute it explicitly. So in extreme circumstances the recursively compute one generates "garbage".
>>>>>> 
>>>>>>   Barry
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>>