<div dir="ltr">Hi Barry,<div><br></div><div>I understand and perfectly agree with you that the behavior increase after the release due to better tuning.</div><div><br></div><div>In my case, the difference in the solution is negligible, but the runtime increases up to +70% (with the same number of ksp_iterations).</div><div>So I was wondering if maybe there were just some flags related to memory preallocation or re-usage of intermediate solution that before was defaulted.</div><div><br></div><div>Thank you,</div><div>Federico</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 13 September 2017 at 17:29, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
There will likely always be slight differences in convergence over that many releases. Lots of little defaults etc get changed over time as we learn from users and increase the robustness of the defaults.<br>
<br>
So in your case do the differences matter?<br>
<br>
1) What is the time to solution in both cases, is it a few percent different or now much slower?<br>
<br>
2) What about number of iterations? Almost identical (say 1 or 2 different) or does it now take 30 iterations when it use to take 5?<br>
<span class="HOEnZb"><font color="#888888"><br>
Barry<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> On Sep 13, 2017, at 10:25 AM, Federico Golfrè Andreasi <<a href="mailto:federico.golfre@gmail.com">federico.golfre@gmail.com</a>> wrote:<br>
><br>
> Dear PETSc users/developers,<br>
><br>
> I recently switched from PETSc-3.4 to PETSc-3.7 and found that some default setup for the "mg" (mutigrid) preconditioner have changed.<br>
><br>
> We were solving a linear system passing, throug command line, the following options:<br>
> -ksp_type fgmres<br>
> -ksp_max_it 100000<br>
> -ksp_rtol 0.000001<br>
> -pc_type mg<br>
> -ksp_view<br>
><br>
> The output of the KSP view is as follow:<br>
><br>
> KSP Object: 128 MPI processes<br>
> type: fgmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=100000, initial guess is zero<br>
> tolerances: relative=1e-06, absolute=1e-50, divergence=10000<br>
> right preconditioning<br>
> using UNPRECONDITIONED norm type for convergence test<br>
> PC Object: 128 MPI processes<br>
> type: mg<br>
> MG: type is MULTIPLICATIVE, levels=1 cycles=v<br>
> Cycles per PCApply=1<br>
> Not using Galerkin computed coarse grid matrices<br>
> Coarse grid solver -- level ------------------------------<wbr>-<br>
> KSP Object: (mg_levels_0_) 128 MPI processes<br>
> type: chebyshev<br>
> Chebyshev: eigenvalue estimates: min = 0.223549, max = 2.45903<br>
> Chebyshev: estimated using: [0 0.1; 0 1.1]<br>
> KSP Object: (mg_levels_0_est_) 128 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using NONE norm type for convergence test<br>
> PC Object: (mg_levels_0_) 128 MPI processes<br>
> type: sor<br>
> SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1<br>
> linear system matrix followed by preconditioner matrix:<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> maximum iterations=1, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000<br>
> left preconditioning<br>
> using NONE norm type for convergence test<br>
> PC Object: (mg_levels_0_) 128 MPI processes<br>
> type: sor<br>
> SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1<br>
> linear system matrix followed by preconditioner matrix:<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> linear system matrix followed by preconditioner matrix:<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> Matrix Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
><br>
> When I build the same program using PETSc-3.7 and run it with the same options we observe that the runtime increases and the convergence is slightly different. The output of the KSP view is:<br>
><br>
> KSP Object: 128 MPI processes<br>
> type: fgmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=100000, initial guess is zero<br>
> tolerances: relative=1e-06, absolute=1e-50, divergence=10000.<br>
> right preconditioning<br>
> using UNPRECONDITIONED norm type for convergence test<br>
> PC Object: 128 MPI processes<br>
> type: mg<br>
> MG: type is MULTIPLICATIVE, levels=1 cycles=v<br>
> Cycles per PCApply=1<br>
> Not using Galerkin computed coarse grid matrices<br>
> Coarse grid solver -- level ------------------------------<wbr>-<br>
> KSP Object: (mg_levels_0_) 128 MPI processes<br>
> type: chebyshev<br>
> Chebyshev: eigenvalue estimates: min = 0.223549, max = 2.45903<br>
> Chebyshev: eigenvalues estimated using gmres with translations [0. 0.1; 0. 1.1]<br>
> KSP Object: (mg_levels_0_esteig_) 128 MPI processes<br>
> type: gmres<br>
> GMRES: restart=30, using Classical (unmodified) Gram-Schmidt Orthogonalization with no iterative refinement<br>
> GMRES: happy breakdown tolerance 1e-30<br>
> maximum iterations=10, initial guess is zero<br>
> tolerances: relative=1e-12, absolute=1e-50, divergence=10000.<br>
> left preconditioning<br>
> using PRECONDITIONED norm type for convergence test<br>
> maximum iterations=2, initial guess is zero<br>
> tolerances: relative=1e-05, absolute=1e-50, divergence=10000.<br>
> left preconditioning<br>
> using NONE norm type for convergence test<br>
> PC Object: (mg_levels_0_) 128 MPI processes<br>
> type: sor<br>
> SOR: type = local_symmetric, iterations = 1, local iterations = 1, omega = 1.<br>
> linear system matrix followed by preconditioner matrix:<br>
> Mat Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> Mat Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> linear system matrix followed by preconditioner matrix:<br>
> Mat Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
> Mat Object: 128 MPI processes<br>
> type: mpiaij<br>
> rows=279669, cols=279669<br>
> total: nonzeros=6427943, allocated nonzeros=6427943<br>
> total number of mallocs used during MatSetValues calls =0<br>
> not using I-node (on process 0) routines<br>
><br>
> I was able to get a closer solution adding the following options:<br>
> -mg_levels_0_esteig_ksp_norm_<wbr>type none<br>
> -mg_levels_0_esteig_ksp_rtol 1.0e-5<br>
> -mg_levels_ksp_max_it 1<br>
><br>
> But I still can reach the same runtime we were observing with PETSc-3.4, could you please advice me if I should specify any other options?<br>
><br>
> Thank you very much for your support,<br>
> Federico Golfre' Andreasi<br>
><br>
<br>
</div></div></blockquote></div><br></div>