DMMG solver convergence with single precision?

Wed Dec 3 19:59:54 CST 2008

Thanks Barry.  Initial indications are that fgmres is converging better 
than richardson on a custom test setup, with cg somewhere in between.  
The solver is a little different to ex34 as well though.  Thanks for the 
pointers, the convergence is much better now.  My main concern was why 
the multigrid seemed to be stalling with single precision, but that 
doesn't appear to be an issue now.

cheers,
Chris

Barry Smith wrote:
>
>   I have reproduced this. I expect that you would not be able to 
> decrease the residual by more than a factor
> of 10^6 but the results below are worse than I expected.
>
>   After first running it I thought I would have to say "I have no 
> idea". But then I ran
>
> barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 
> -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres 
> -ksp_gmres_restart 4
>     0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm 
> 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00
>     1 KSP preconditioned resid norm 2.803983807098e-04 true resid norm 
> 2.803510287777e-04 ||Ae||/||Ax|| 6.297569256276e-03
>     2 KSP preconditioned resid norm 7.431745416397e-06 true resid norm 
> 7.503178949264e-06 ||Ae||/||Ax|| 1.685450988589e-04
>     3 KSP preconditioned resid norm 4.979479399481e-06 true resid norm 
> 6.953502179385e-06 ||Ae||/||Ax|| 1.561976241646e-04
>     4 KSP preconditioned resid norm 6.971503353270e-06 true resid norm 
> 6.971503353270e-06 ||Ae||/||Ax|| 1.566019782331e-04
>     5 KSP preconditioned resid norm 4.069146442021e-08 true resid norm 
> 8.315442414641e-07 ||Ae||/||Ax|| 1.867910941655e-05
> Residual norm 8.31544e-07
> Error norm 0.000372035
> Error norm 9.54604e-05
> Error norm 1.38928e-07
>
>   Part of the problem is coming from the gmres not generating a good 
> orthonormal basis of directions, so I ran
> [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 
> -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type fgmres 
> -ksp_gmres_modifiedgramschmidt
>     0 KSP preconditioned resid norm 4.451733827591e-02 true resid norm 
> 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00
>     1 KSP preconditioned resid norm 2.803631359711e-04 true resid norm 
> 2.803224197123e-04 ||Ae||/||Ax|| 6.296926643699e-03
>     2 KSP preconditioned resid norm 6.318659416138e-06 true resid norm 
> 6.395913715096e-06 ||Ae||/||Ax|| 1.436724269297e-04
>     3 KSP preconditioned resid norm 5.441804660222e-06 true resid norm 
> 5.559954843193e-06 ||Ae||/||Ax|| 1.248941407539e-04
>     4 KSP preconditioned resid norm 5.441734174383e-06 true resid norm 
> 5.580118795478e-06 ||Ae||/||Ax|| 1.253470836673e-04
>     5 KSP preconditioned resid norm 4.763217020809e-06 true resid norm 
> 4.965124844603e-06 ||Ae||/||Ax|| 1.115323830163e-04
>     6 KSP preconditioned resid norm 1.149661784439e-06 true resid norm 
> 1.742098334034e-06 ||Ae||/||Ax|| 3.913302862202e-05
>     7 KSP preconditioned resid norm 1.129046836468e-06 true resid norm 
> 1.607488229638e-06 ||Ae||/||Ax|| 3.610926069086e-05
>     8 KSP preconditioned resid norm 1.129005681832e-06 true resid norm 
> 1.682757897470e-06 ||Ae||/||Ax|| 3.780005499721e-05
>     9 KSP preconditioned resid norm 9.863339300864e-07 true resid norm 
> 1.657332745708e-06 ||Ae||/||Ax|| 3.722892870428e-05
>    10 KSP preconditioned resid norm 3.080494650476e-07 true resid norm 
> 1.298029587815e-06 ||Ae||/||Ax|| 2.915784352808e-05
> Residual norm 1.29803e-06
> Error norm 0.00100342
> Error norm 0.000631685
> Error norm 6.85996e-07
>
> It looks like using the modified gram-schmidt orthogonalization helps 
> (that is it takes less iterations than with the classical (default) BUT
> looking more closely we see that it actually terminated too early and 
> the answer is crappy.
>
> So lets turn off ALL the GMRES stuff
> [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 
> -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson 
> -mg_levels_ksp_type richardson
>     0 KSP preconditioned resid norm 3.698072814941e+02 true resid norm 
> 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00
>     1 KSP preconditioned resid norm 4.145919799805e+01 true resid norm 
> 9.632258675992e-03 ||Ae||/||Ax|| 2.163709551096e-01
>     2 KSP preconditioned resid norm 4.848244190216e+00 true resid norm 
> 1.894293935038e-03 ||Ae||/||Ax|| 4.255182296038e-02
>     3 KSP preconditioned resid norm 5.902504324913e-01 true resid norm 
> 3.106203221250e-04 ||Ae||/||Ax|| 6.977513432503e-03
>     4 KSP preconditioned resid norm 7.403665781021e-02 true resid norm 
> 4.669937698054e-05 ||Ae||/||Ax|| 1.049015438184e-03
>     5 KSP preconditioned resid norm 9.483236819506e-03 true resid norm 
> 6.756537914043e-06 ||Ae||/||Ax|| 1.517731725471e-04
>     6 KSP preconditioned resid norm 1.249413820915e-03 true resid norm 
> 1.256137807104e-06 ||Ae||/||Ax|| 2.821682210197e-05
> Residual norm 1.25614e-06
> Error norm 0.000369181
> Error norm 9.46438e-05
> Error norm 1.37756e-07
>
> The final residual doesn't look that great but if you look at the 
> error norms you see that the solution has fully converged.
>
> This final run is actually classical multigrid. So it looks like that 
> is the way to go; BUT you have to be very careful with how
> you decide when it has converged. Obviously you want to stop as soon 
> as the algebraic error is smaller than the truncation error.
> But, sadly we don't know what the algebraic error is (we only know the 
> residual) and we do not know what the truncation error is.
>
> My hack convergence criteria would be to stop as soon as the residual 
> norm DOES NOT decrease by at least (say) 20%
>
> To do a little better we can use SOR as the smoother instead of ILU
> [barry-smiths-macbook-pro-17:ksp/examples/tutorials] bsmith% ./ex34 
> -ksp_monitor_true_residual -dmmg_nlevels 6 -ksp_type richardson  
> -mg_levels_ksp_type richardson  -mg_levels_pc_type sor
>     0 KSP preconditioned resid norm 3.202437744141e+02 true resid norm 
> 4.451733827591e-02 ||Ae||/||Ax|| 1.000000000000e+00
>     1 KSP preconditioned resid norm 1.194643592834e+01 true resid norm 
> 2.088646404445e-03 ||Ae||/||Ax|| 4.691759496927e-02
>     2 KSP preconditioned resid norm 4.755175709724e-01 true resid norm 
> 1.131777826231e-04 ||Ae||/||Ax|| 2.542330417782e-03
>     3 KSP preconditioned resid norm 1.988304778934e-02 true resid norm 
> 6.019943612046e-06 ||Ae||/||Ax|| 1.352269464405e-04
>     4 KSP preconditioned resid norm 8.689521928318e-04 true resid norm 
> 8.886481737136e-07 ||Ae||/||Ax|| 1.996184437303e-05
> Residual norm 8.88648e-07
> Error norm 0.000368883
> Error norm 9.4803e-05
> Error norm 1.37996e-07
>
> Hope this helps,
>
>   Barry
>
>
>
>
>
> On Dec 2, 2008, at 7:52 PM, Chris Cooper wrote:
>
>> Hi,
>>
>> Just wondering if there are any techniques for improving DMMG 
>> convergence on a linear problem when operating in single precision?
>>
>> eg using ksp example 34 in Petsc 2.3.3 with multigrid levels changed 
>> to 6...
>>
>> double precision:
>> 0 KSP Residual norm 4.451733404502e-02
>> 1 KSP Residual norm 2.801676364873e-04
>> 2 KSP Residual norm 3.215426166316e-06
>> 3 KSP Residual norm 5.446277176355e-08
>> Residual norm 5.44628e-08
>> Error norm 0.000356798
>> Error norm 9.21724e-05
>> Error norm 1.34208e-07
>>
>> single precision:
>> 0 KSP Residual norm 4.450287297368e-02
>> 1 KSP Residual norm 2.799208450597e-04
>> 2 KSP Residual norm 2.232483529951e-05
>> 3 KSP Residual norm 1.575733222126e-05
>> 4 KSP Residual norm 1.286925271415e-05
>> ...
>> 27 KSP Residual norm 4.369261660031e-06
>> 28 KSP Residual norm 4.287577212381e-06
>> 29 KSP Residual norm 4.210308816255e-06
>> 30 KSP Residual norm 2.235170904896e-05
>> 31 KSP Residual norm 1.709007477757e-07
>> Residual norm 8.45584e-07
>> Error norm 0.000370784
>> Error norm 9.54664e-05
>> Error norm 1.38935e-07
>>
>> thanks,
>> Chris Cooper
>> Animal Logic
>>
>> Animal Logic
>> http://www.animallogic.com
>>
>> Please think of the environment before printing this email.
>>
>> This email and any attachments may be confidential and/or privileged. 
>> If you are not the intended recipient of this email, you must not 
>> disclose or use the information contained in it. Please notify the 
>> sender immediately and delete this document if you have received it 
>> in error. We do not guarantee this email is error or virus free.
>>
>>
>>
>

Animal Logic
http://www.animallogic.com

Please think of the environment before printing this email.

This email and any attachments may be confidential and/or privileged. If you are not the intended recipient of this email, you must not disclose or use the information contained in it. Please notify the sender immediately and delete this document if you have received it in error. We do not guarantee this email is error or virus free.