<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, May 8, 2023 at 2:32 AM Sebastian Blauth <<a href="mailto:sebastian.blauth@itwm.fraunhofer.de">sebastian.blauth@itwm.fraunhofer.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello everyone,<br>
<br>
I wanted to briefly follow up on my question (see my last reply).<br>
Does anyone know / have an idea why the LSC preconditioner in PETSc does <br>
not seem to scale well with the problem size (the outer fgmres solver I <br>
am using nearly scale nearly linearly with the problem size in my example).<br>
I have also already tried using -ksp_diagonal_scale but the results are <br>
identical.<br>
Any help is really appreciated.<br></blockquote><div><br></div><div>I would start by finding results in the literature that you like, in that they are on a problem similar to yours and you like the performance, and reproduce them in your code.</div><div>If you can do that then you have a research problem to see how to get your problem to work well.</div><div>If your solver does not reproduce published results then we might be able to provide some advice.</div><div><br></div><div>Mark</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thanks a lot,<br>
Sebastian<br>
<br>
On 03.05.2023 09:07, Sebastian Blauth wrote:<br>
> First of all, yes you are correct that I am trying to solve the <br>
> stationary incompressible Navier Stokes equations.<br>
> <br>
> On 02.05.2023 21:33, Matthew Knepley wrote:<br>
>> On Tue, May 2, 2023 at 2:29 PM Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a> <br>
>> <mailto:<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>>> wrote:<br>
>><br>
>> Sebastian Blauth <<a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a><br>
>> <mailto:<a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a>>> writes:<br>
>><br>
>> > I agree with your comment for the Stokes equations - for these, I<br>
>> have<br>
>> > already tried and used the pressure mass matrix as part of a<br>
>> (additive)<br>
>> > block preconditioner and it gave mesh independent results.<br>
>> ><br>
>> > However, for the Navier Stokes equations, is the Schur complement<br>
>> really<br>
>> > spectrally equivalent to the pressure mass matrix?<br>
>><br>
>> No, it's not. You'd want something like PCD (better, but not<br>
>> algebraic) or LSC.<br>
>><br>
> <br>
> I would like to take a look at the LSC preconditioner. For this, I did <br>
> also not achieve mesh-independent results. I am using the following <br>
> options (I know that the tolerances are too high at the moment, but it <br>
> should just illustrate the behavior w.r.t. mesh refinement). Again, I am <br>
> using a simple 2D channel problem for testing purposes.<br>
> <br>
> I am using the following options<br>
> <br>
> -ksp_type fgmres<br>
> -ksp_gmres_restart 100<br>
> -ksp_gmres_cgs_refinement_type refine_ifneeded<br>
> -ksp_max_it 1000<br>
> -ksp_rtol 1e-10<br>
> -ksp_atol 1e-30<br>
> -pc_type fieldsplit<br>
> -pc_fieldsplit_type schur<br>
> -pc_fieldsplit_schur_fact_type full<br>
> -pc_fieldsplit_schur_precondition self<br>
> -fieldsplit_0_ksp_type preonly<br>
> -fieldsplit_0_pc_type lu<br>
> -fieldsplit_1_ksp_type gmres<br>
> -fieldsplit_1_ksp_pc_side right<br>
> -fieldsplit_1_ksp_gmres_restart 100<br>
> -fieldsplit_1_ksp_gmres_cgs_refinement_type refine_ifneeded<br>
> -fieldsplit_1_ksp_max_it 1000<br>
> -fieldsplit_1_ksp_rtol 1e-10<br>
> -fieldsplit_1_ksp_atol 1e-30<br>
> -fieldsplit_1_pc_type lsc<br>
> -fieldsplit_1_lsc_ksp_type preonly<br>
> -fieldsplit_1_lsc_pc_type lu<br>
> -fieldsplit_1_ksp_converged_reason<br>
> <br>
> Again, the direct solvers are used so that only the influence of the LSC <br>
> preconditioner is seen. I have suitable preconditioners for all of these <br>
> available (using boomeramg).<br>
> <br>
> At the bottom, I attach the output for different discretizations. As you <br>
> can see there, the number of iterations increases nearly linearly with <br>
> the problem size.<br>
> <br>
> I think that the problem could occur due to a wrong scaling. In your <br>
> docs <a href="https://petsc.org/release/manualpages/PC/PCLSC/" rel="noreferrer" target="_blank">https://petsc.org/release/manualpages/PC/PCLSC/</a> , you write that <br>
> the LSC preconditioner is implemented as<br>
> <br>
> inv(S) \approx inv(A10 A01) A10 A00 A01 inv(A10 A01)<br>
> <br>
> However, in the book of Elman, Sylvester and Wathen (Finite Elements and <br>
> Fast Iterative Solvers), the LSC preconditioner is defined as<br>
> <br>
> inv(S) \approx inv(A10 inv(T) A01) A10 inv(T) A00 inv(T) A01 <br>
> inv(A10 inv(T) A01)<br>
> <br>
> where T = diag(Q) and Q is the velocity mass matrix.<br>
> <br>
> There is an options -pc_lsc_scale_diag, which states that it uses the <br>
> diagonal of A for scaling. I suppose, that this means, that the diagonal <br>
> of the A00 block is used for scaling - however, this is not the right <br>
> scaling, is it? Even in the source code for the LSC preconditioner, in <br>
> /src/ksp/pc/impls/lsc/lsc.c it is mentioned, that a mass matrix should <br>
> be used...<br>
> Is there any way to implement this in PETSc? Maybe by supplying the mass <br>
> matrix as Pmat?<br>
> <br>
> Thanks a lot in advance,<br>
> Sebastian<br>
> <br>
>><br>
>> I think you can do a better job than that using something like<br>
>><br>
>> <a href="https://arxiv.org/abs/1810.03315" rel="noreferrer" target="_blank">https://arxiv.org/abs/1810.03315</a> <<a href="https://arxiv.org/abs/1810.03315" rel="noreferrer" target="_blank">https://arxiv.org/abs/1810.03315</a>><br>
>><br>
>> Basically, you use an augmented Lagrangian thing to make the Schur <br>
>> complement well-conditioned,<br>
>> and then use a special smoother to handle that perturbation.<br>
>><br>
>> > And even if it is, the convergence is only good for small<br>
>> Reynolds numbers, for moderately high ones the convergence really<br>
>> deteriorates. This is why I am trying to make<br>
>> fieldsplit_schur_precondition selfp work better (this is, if I<br>
>> understand it correctly, a SIMPLE type preconditioner).<br>
>><br>
>> SIMPLE is for short time steps (not too far from resolving CFL) and<br>
>> bad for steady. This taxonomy is useful, though the problems are<br>
>> super academic and they don't use high aspect ratio.<br>
>><br>
> <br>
> Okay, I get that I cannot expect the SIMPLE preconditioning <br>
> (schur_precondition selfp) to work efficiently. I guess the reason it <br>
> works for small time steps (for the instationary equations) is that the <br>
> velocity block becomes diagonally dominant in this case, so that diag(A) <br>
> is a good approximation of A.<br>
> <br>
> <br>
>> <a href="https://doi.org/10.1016/j.jcp.2007.09.026" rel="noreferrer" target="_blank">https://doi.org/10.1016/j.jcp.2007.09.026</a><br>
>> <<a href="https://doi.org/10.1016/j.jcp.2007.09.026" rel="noreferrer" target="_blank">https://doi.org/10.1016/j.jcp.2007.09.026</a>><br>
>><br>
>><br>
>> Thanks,<br>
>><br>
>> Matt<br>
>><br>
>> -- <br>
>> What most experimenters take for granted before they begin their <br>
>> experiments is infinitely more interesting than any results to which <br>
>> their experiments lead.<br>
>> -- Norbert Wiener<br>
>><br>
>> <a href="https://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">https://www.cse.buffalo.edu/~knepley/</a> <br>
>> <<a href="http://www.cse.buffalo.edu/~knepley/" rel="noreferrer" target="_blank">http://www.cse.buffalo.edu/~knepley/</a>><br>
> <br>
> <br>
> And here is the output of my scaling tests<br>
> <br>
> 8x8 discretization<br>
> <br>
> Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. tol)<br>
> <br>
> Newton solver: 0, 1.023e+03 (1.00e-30), 1.000e+00 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 38<br>
> Newton solver: 1, 1.313e+03 (1.00e-30), 1.283e+00 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 76<br>
> Newton solver: 2, 1.198e+02 (1.00e-30), 1.171e-01 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 74<br>
> Newton solver: 3, 7.249e-01 (1.00e-30), 7.084e-04 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 74<br>
> Newton solver: 4, 3.883e-05 (1.00e-30), 3.795e-08 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 74<br>
> Newton solver: 5, 2.778e-12 (1.00e-30), 2.714e-15 (1.00e-10)<br>
> <br>
> <br>
> <br>
> 16x16 discretization<br>
> <br>
> Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. tol)<br>
> <br>
> Newton solver: 0, 1.113e+03 (1.00e-30), 1.000e+00 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 62<br>
> Newton solver: 1, 8.316e+02 (1.00e-30), 7.475e-01 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 141<br>
> Newton solver: 2, 5.806e+01 (1.00e-30), 5.218e-02 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 119<br>
> Newton solver: 3, 3.309e-01 (1.00e-30), 2.974e-04 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 118<br>
> Newton solver: 4, 9.085e-06 (1.00e-30), 8.166e-09 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 120<br>
> Newton solver: 5, 3.475e-12 (1.00e-30), 3.124e-15 (1.00e-10)<br>
> <br>
> <br>
> <br>
> 32x32 discretization<br>
> <br>
> Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. tol)<br>
> <br>
> Newton solver: 0, 1.330e+03 (1.00e-30), 1.000e+00 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations 98<br>
> Newton solver: 1, 5.913e+02 (1.00e-30), 4.445e-01 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 183<br>
> Newton solver: 2, 3.214e+01 (1.00e-30), 2.416e-02 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 152<br>
> Newton solver: 3, 2.059e-01 (1.00e-30), 1.547e-04 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 151<br>
> Newton solver: 4, 6.949e-06 (1.00e-30), 5.223e-09 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 149<br>
> Newton solver: 5, 5.300e-12 (1.00e-30), 3.983e-15 (1.00e-10)<br>
> <br>
> <br>
> <br>
> 64x64 discretization<br>
> <br>
> Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. tol)<br>
> <br>
> Newton solver: 0, 1.707e+03 (1.00e-30), 1.000e+00 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 198<br>
> Newton solver: 1, 4.259e+02 (1.00e-30), 2.494e-01 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 357<br>
> Newton solver: 2, 1.706e+01 (1.00e-30), 9.993e-03 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 266<br>
> Newton solver: 3, 1.134e-01 (1.00e-30), 6.639e-05 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 261<br>
> Newton solver: 4, 4.285e-06 (1.00e-30), 2.510e-09 (1.00e-10)<br>
> Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations <br>
> 263<br>
> Newton solver: 5, 9.650e-12 (1.00e-30), 5.652e-15 (1.00e-10)<br>
> <br>
<br>
-- <br>
Dr. Sebastian Blauth<br>
Fraunhofer-Institut für<br>
Techno- und Wirtschaftsmathematik ITWM<br>
Abteilung Transportvorgänge<br>
Fraunhofer-Platz 1, 67663 Kaiserslautern<br>
Telefon: +49 631 31600-4968<br>
<a href="mailto:sebastian.blauth@itwm.fraunhofer.de" target="_blank">sebastian.blauth@itwm.fraunhofer.de</a><br>
<a href="http://www.itwm.fraunhofer.de" rel="noreferrer" target="_blank">www.itwm.fraunhofer.de</a><br>
</blockquote></div></div>