[petsc-users] Solving Linear Systems with Scalar Real and Complex

Wed Aug 1 08:41:51 CDT 2018

On Wed, Aug 1, 2018 at 9:02 AM Pierpaolo Minelli <pierpaolo.minelli at cnr.it>
wrote:

> Il giorno 27 lug 2018, alle ore 18:32, Smith, Barry F. <bsmith at mcs.anl.gov>
> ha scritto:
>
> On Jul 27, 2018, at 3:26 AM, Pierpaolo Minelli <pierpaolo.minelli at cnr.it>
> wrote:
>
>
> Finally, I have a question. In my simulation I solve the two systems at
> each step of the calculation, and it was my habit to use the following
> option after the first resolution and before solving the system in the
> second time step:
>
> call KSPSetInitialGuessNonzero(ksp,PETSC_TRUE,ierr)
>
> Since this option was incompatible with the use of MUMPS or SuperLU_Dist,
> I commented on it and noticed, to my surprise, that iterative methods were
> not affected by this comment, rather they were slightly faster. Is this
> normal? Or do I use this command incorrectly?
>
>
>   Presuming this is a linear problem?
>
>
>   You should run with -ksp_monitor_true_residual and compare the final
> residual norms (at linear system convergence). Since KSP uses a relative
> decrease in the residual norm to declare convergence what you have told us
> isn't enough to determine if one is converging "faster" to the solution
> then the other.
>
>   Barry
>
>
> Yes, there are two linear problems. The first in the field of complex
> numbers (which I divided into two problems in the field of real numbers as
> suggested by Matthew) and the second in the field of real numbers.
>
> These are the results obtained using the option -ksp_monitor_true_residual:
>
> *-pc_type gamg -pc_gamg_agg_nsmooths 1 -pc_gamg_reuse_interpolation true
> -pc_gamg_square_graph 1 -pc_gamg_threshold 0. -ksp_rtol 1.e-7
> -ksp_monitor_true_residual*
>
>   0 KSP preconditioned resid norm 1.324344286254e-02 true resid norm
> 2.979865703850e-03 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.399266083153e-03 true resid norm
> 1.196951495628e+05 ||r(i)||/||b|| 4.016796777391e+07
>   2 KSP preconditioned resid norm 6.287919852336e-04 true resid norm
> 4.429876706490e+04 ||r(i)||/||b|| 1.486602802524e+07
>   3 KSP preconditioned resid norm 8.836547690039e-05 true resid norm
> 1.182383826632e+04 ||r(i)||/||b|| 3.967909778970e+06
>   4 KSP preconditioned resid norm 1.291370058561e-05 true resid norm
> 2.801893692950e+03 ||r(i)||/||b|| 9.402751571421e+05
>   5 KSP preconditioned resid norm 2.073922228951e-06 true resid norm
> 6.312112895919e+02 ||r(i)||/||b|| 2.118254150770e+05
>   6 KSP preconditioned resid norm 3.283811876800e-07 true resid norm
> 8.777865833701e+01 ||r(i)||/||b|| 2.945725313178e+04
>   7 KSP preconditioned resid norm 5.414680273500e-08 true resid norm
> 1.610127004050e+01 ||r(i)||/||b|| 5.403354258448e+03
>   8 KSP preconditioned resid norm 9.645834363683e-09 true resid norm
> 3.006444251909e+00 ||r(i)||/||b|| 1.008919377818e+03
>   9 KSP preconditioned resid norm 1.915420455785e-09 true resid norm
> 6.672996533262e-01 ||r(i)||/||b|| 2.239361500298e+02
>  10 KSP preconditioned resid norm 3.334928638696e-10 true resid norm
> 1.185976397497e-01 ||r(i)||/||b|| 3.979965929219e+01
>

You can see from the "true resid norm" that your complex problem is not
actually being solved. ML and Hypre have similar results.
I would guess that your coarse grid is near singular.

The real problem is being solved correctly, although you are still 100x off
the true residual so the preconditioner is not great.
Have you tested with a manufactured solution?

  Thanks,

     Matt

>   0 KSP preconditioned resid norm 1.416256896904e+04 true resid norm
> 6.623626291881e+05 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.781264980774e+03 true resid norm
> 1.283809201228e+11 ||r(i)||/||b|| 1.938227104995e+05
>   2 KSP preconditioned resid norm 7.392413714593e+02 true resid norm
> 4.990467508600e+10 ||r(i)||/||b|| 7.534343407504e+04
>   3 KSP preconditioned resid norm 1.087641767696e+02 true resid norm
> 1.367263690390e+10 ||r(i)||/||b|| 2.064222270610e+04
>   4 KSP preconditioned resid norm 1.627726591174e+01 true resid norm
> 3.451534923895e+09 ||r(i)||/||b|| 5.210944536719e+03
>   5 KSP preconditioned resid norm 2.564636460142e+00 true resid norm
> 7.980525032990e+08 ||r(i)||/||b|| 1.204857381941e+03
>   6 KSP preconditioned resid norm 4.252626820180e-01 true resid norm
> 1.151817528111e+08 ||r(i)||/||b|| 1.738953070953e+02
>   7 KSP preconditioned resid norm 6.758292325957e-02 true resid norm
> 2.065701091779e+07 ||r(i)||/||b|| 3.118686050134e+01
>   8 KSP preconditioned resid norm 1.099617201063e-02 true resid norm
> 3.470561062696e+06 ||r(i)||/||b|| 5.239669192917e+00
>   9 KSP preconditioned resid norm 2.195352537111e-03 true resid norm
> 7.056823048483e+05 ||r(i)||/||b|| 1.065401750871e+00
>  10 KSP preconditioned resid norm 4.380752631896e-04 true resid norm
> 1.440377395627e+05 ||r(i)||/||b|| 2.174605468598e-01
>
>   0 KSP preconditioned resid norm 4.970534641714e+02 true resid norm
> 1.879478105170e+02 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 9.825390294838e+01 true resid norm
> 6.304087699909e+03 ||r(i)||/||b|| 3.354169267824e+01
>   2 KSP preconditioned resid norm 1.317425630977e+01 true resid norm
> 1.659038483292e+03 ||r(i)||/||b|| 8.827123224947e+00
>   3 KSP preconditioned resid norm 1.267331175258e+00 true resid norm
> 3.723773819327e+02 ||r(i)||/||b|| 1.981280765700e+00
>   4 KSP preconditioned resid norm 1.451198865319e-01 true resid norm
> 6.113281457366e+01 ||r(i)||/||b|| 3.252648403059e-01
>   5 KSP preconditioned resid norm 2.428012832580e-02 true resid norm
> 9.525516489005e+00 ||r(i)||/||b|| 5.068171032587e-02
>   6 KSP preconditioned resid norm 3.273131599046e-03 true resid norm
> 1.693121214500e+00 ||r(i)||/||b|| 9.008464689439e-03
>   7 KSP preconditioned resid norm 3.629020564958e-04 true resid norm
> 1.937921370699e-01 ||r(i)||/||b|| 1.031095475584e-03
>   8 KSP preconditioned resid norm 4.303742583350e-05 true resid norm
> 2.733022203972e-02 ||r(i)||/||b|| 1.454138889117e-04
>
> *-pc_type ml -ksp_rtol 1.e-7 -ksp_monitor_true_residual*
>
>   0 KSP preconditioned resid norm 1.825767020537e-02 true resid norm
> 2.979865703850e-03 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 6.495628259378e-04 true resid norm
> 1.373525306349e+04 ||r(i)||/||b|| 4.609353047603e+06
>   2 KSP preconditioned resid norm 4.971875712010e-05 true resid norm
> 2.154150315236e+03 ||r(i)||/||b|| 7.229018114651e+05
>   3 KSP preconditioned resid norm 3.726806462903e-06 true resid norm
> 3.000693510070e+02 ||r(i)||/||b|| 1.006989511706e+05
>   4 KSP preconditioned resid norm 2.496898447113e-07 true resid norm
> 3.004338446358e+01 ||r(i)||/||b|| 1.008212699813e+04
>   5 KSP preconditioned resid norm 1.541814409235e-08 true resid norm
> 2.256927674575e+00 ||r(i)||/||b|| 7.573924125705e+02
>   6 KSP preconditioned resid norm 1.153890845088e-09 true resid norm
> 2.007294077611e-01 ||r(i)||/||b|| 6.736189738408e+01
>   0 KSP preconditioned resid norm 1.961448048638e+04 true resid norm
> 6.623626291881e+05 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 7.940694187362e+02 true resid norm
> 1.483756468402e+10 ||r(i)||/||b|| 2.240096894085e+04
>   2 KSP preconditioned resid norm 6.141876183549e+01 true resid norm
> 2.581042149104e+09 ||r(i)||/||b|| 3.896720671376e+03
>   3 KSP preconditioned resid norm 4.872887149903e+00 true resid norm
> 3.544664743366e+08 ||r(i)||/||b|| 5.351547003355e+02
>   4 KSP preconditioned resid norm 3.088357668912e-01 true resid norm
> 3.689655474675e+07 ||r(i)||/||b|| 5.570446326655e+01
>   5 KSP preconditioned resid norm 2.030016543101e-02 true resid norm
> 2.847019997470e+06 ||r(i)||/||b|| 4.298279933093e+00
>   6 KSP preconditioned resid norm 1.458276940703e-03 true resid norm
> 2.567051860886e+05 ||r(i)||/||b|| 3.875598875547e-01
>
>   0 KSP preconditioned resid norm 6.230592591466e+02 true resid norm
> 1.879478105170e+02 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 3.825349376322e+01 true resid norm
> 7.755266613308e+02 ||r(i)||/||b|| 4.126287287931e+00
>   2 KSP preconditioned resid norm 4.321561767174e+00 true resid norm
> 2.170558998883e+02 ||r(i)||/||b|| 1.154873255992e+00
>   3 KSP preconditioned resid norm 2.806349660119e-01 true resid norm
> 2.931496742303e+01 ||r(i)||/||b|| 1.559739767247e-01
>   4 KSP preconditioned resid norm 1.707567397460e-02 true resid norm
> 2.684913551118e+00 ||r(i)||/||b|| 1.428542074383e-02
>   5 KSP preconditioned resid norm 1.741069717192e-03 true resid norm
> 2.914639372698e-01 ||r(i)||/||b|| 1.550770591410e-03
>   6 KSP preconditioned resid norm 1.460913620440e-04 true resid norm
> 3.460791330512e-02 ||r(i)||/||b|| 1.841357619965e-04
>   7 KSP preconditioned resid norm 1.741167177387e-05 true resid norm
> 3.396573365236e-03 ||r(i)||/||b|| 1.807189642642e-05
>
> *-pc_type hypre -ksp_rtol 1.e-7 -ksp_monitor_true_residual*
>
> 0 KSP preconditioned resid norm 1.984853668903e-02 true resid norm
> 2.979865703850e-03 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 1.924446712660e-04 true resid norm
> 8.461145171168e+03 ||r(i)||/||b|| 2.839438421750e+06
>   2 KSP preconditioned resid norm 5.161509100761e-06 true resid norm
> 4.501007519328e+02 ||r(i)||/||b|| 1.510473278548e+05
>   3 KSP preconditioned resid norm 9.297326931235e-08 true resid norm
> 1.474595580807e+01 ||r(i)||/||b|| 4.948530327732e+03
>   4 KSP preconditioned resid norm 1.910271882618e-09 true resid norm
> 3.603243780988e-01 ||r(i)||/||b|| 1.209196701829e+02
>   0 KSP preconditioned resid norm 2.157687745804e+04 true resid norm
> 6.623626291881e+05 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 1.949268476385e+02 true resid norm
> 8.974916757495e+09 ||r(i)||/||b|| 1.354985375382e+04
>   2 KSP preconditioned resid norm 5.078054475786e+00 true resid norm
> 4.502684440161e+08 ||r(i)||/||b|| 6.797914377626e+02
>   3 KSP preconditioned resid norm 8.663802743525e-02 true resid norm
> 1.432117031381e+07 ||r(i)||/||b|| 2.162134408362e+01
>   4 KSP preconditioned resid norm 1.795605563041e-03 true resid norm
> 3.389868548833e+05 ||r(i)||/||b|| 5.117843911255e-01
>
>   0 KSP preconditioned resid norm 6.728304961395e+02 true resid norm
> 1.879478105170e+02 ||r(i)||/||b|| 1.000000000000e+00
>   1 KSP preconditioned resid norm 2.190497539532e+01 true resid norm
> 4.630095820203e+02 ||r(i)||/||b|| 2.463500802413e+00
>   2 KSP preconditioned resid norm 8.425561564252e-01 true resid norm
> 7.012565302251e+01 ||r(i)||/||b|| 3.731123700223e-01
>   3 KSP preconditioned resid norm 3.029848345705e-02 true resid norm
> 4.379018464663e+00 ||r(i)||/||b|| 2.329911932795e-02
>   4 KSP preconditioned resid norm 7.374025528575e-04 true resid norm
> 1.337183702137e-01 ||r(i)||/||b|| 7.114654320570e-04
>   5 KSP preconditioned resid norm 3.009400175162e-05 true resid norm
> 7.731135032616e-03 ||r(i)||/||b|| 4.113447776459e-05
>
> Thanks
>
> Pierpaolo
>
>
> Thanks
>
> Pierpaolo
>
> Il giorno 23 lug 2018, alle ore 15:43, Mark Adams <mfadams at lbl.gov> ha
> scritto:
>
> Note, as Barry said, GAMG works with native complex numbers. You can start
> with your original complex build and use '-pc_type gamg'.
> Mark
>
> On Mon, Jul 23, 2018 at 6:52 AM Pierpaolo Minelli <
> pierpaolo.minelli at cnr.it> wrote:
>
>
> Il giorno 20 lug 2018, alle ore 19:58, Smith, Barry F. <bsmith at mcs.anl.gov>
> ha scritto:
>
>
>
> On Jul 20, 2018, at 7:01 AM, Pierpaolo Minelli <pierpaolo.minelli at cnr.it>
> wrote:
>
> Hi,
>
> in my code I have to solve both a system in the field of real numbers and
> in the field of complex numbers.
> My approach has been as follows.
> First I configured PETSc with the --with-scalar-type=complex option.
> Using this option I have unfortunately lost the possibility to use the two
> preconditioners ML and Hypre.
>
>
>   You should still be able to use the PETSc PCGAMG algebraic multigrid
> solver. Have you tried that? If there are issues let us know because we
> would like to continue to improve the performance of PCGAMG to get it to be
> closer to as good as ML and hypre.
>
>  Barry
>
>
> I will try to convert, as suggested by Matthew, my complex system in a
> system twice as large in real numbers. When i finish, i will try to use ML,
> Hypre and GAMG and i let you know if there are any performance differences.
>
> Thanks
>
> Pierpaolo
>
>
> I later created two subspaces of Krylov and tried to solve the two systems
> as I used to when solving the only equation in the real numbers field.
> In order to correctly solve the system in the field of real numbers I had
> to transform the coefficients from real to complex with an imaginary part
> equal to zero.
>
> Is there a possibility to use a different and better approach to solve my
> problem?
>
> Perhaps an approach where you can continue to use ML and Hypre for system
> solving in the real numbers field or where you don't need to use complex
> numbers when real numbers would actually suffice?
>
> Thanks in advance
>
> Pierpaolo
>
>
>
>
>
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180801/40cc12e0/attachment.html>