<div dir="ltr">GANG has failed here. Compare the real data. <div>Run with -ksp_converged_reason to start getting an idea what is going on.</div><div>GAMG/AGG failures are often from bad eigen estimates in the Chebyshev smoother.</div><div>Is your problem symmetric?</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 7, 2022 at 9:19 AM Karabelas, Elias (<a href="mailto:elias.karabelas@uni-graz.at">elias.karabelas@uni-graz.at</a>) <<a href="mailto:elias.karabelas@uni-graz.at">elias.karabelas@uni-graz.at</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Am 07.07.22 um 15:05 schrieb Mark Adams:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jul 7, 2022 at 7:03 AM Karabelas, Elias (<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>) <<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>OK got it, well the "classical" option of GAMG removes this issue, and also HYPRE does that out-of-the box.</div>
</div>
</blockquote>
<div><br>
</div>
<div>Humm, so you are using hypre in PETSc with a Krylov KSP or using hypre directly.</div>
<div><br>
</div>
<div>GAMG uses a polynomial smoother by default and hypre does not, I believe. </div>
<div>That could be the difference, but I would think the outer Krylov in PETSc would still pollute this.</div>
</div>
</div>
</blockquote>
<p>So the outer Krylov doesn't seem to pollute either for hypre or using gamg classical</p>
<p>this is an example output from those two (vtu file) (gmres as krylov and either hypre or gamg classical as pctype)<br>
</p>
<p><DataArray type="Float32" format="ascii" NumberOfComponents="3" Name="displacement"><br>
0 0 0<br>
0.182029 0 0<br>
0.223618 0 0<br>
0.250511 0 0<br>
0.209325 0 0<br>
0 0.182029 0<br>
0.1305 0.1305 0<br>
</p>
<p><br>
</p>
<p>and this for gamg with smoothed aggregation (gmres as krylov and gamg agg)</p>
<p><DataArray type="Float32" format="ascii" NumberOfComponents="3" Name="displacement"><br>
0 0 0<br>
0.0905508 -5.2019e-05 6.4198e-05<br>
0.173765 -4.34019e-05 5.86626e-05<br>
0.256981 -4.14403e-05 2.78863e-05<br>
0.233807 -3.94787e-05 -2.88998e-06<br>
-4.78464e-05 0.0905198 6.21817e-05<br>
0.0819423 0.0820083 4.89063e-05<br>
</p>
<p>So the difference is noticeable.<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>If you are using hypre directly, I could imagine that they take the residual that they compute to check convergence and do a quick update of the solution, with the diagonal that they have access to, with u = u + D^-1 r</div>
<div>(this is the sort of clever thing they would do) </div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div><br>
</div>
<div>I would slightly disagree with non-exact DirichletBCs, especially in mechanics where I really assume a clamped node is clamped.</div>
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I'm not saying exactness is not important (to you), I'm just saying many people seem to live with it.</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div></div>
<div>Best</div>
<div>Elias<br>
</div>
<div><br>
</div>
<div>Am 07.07.22 um 13:00 schrieb Mark Adams:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I think PCCOMPOSITE is overkill here.
<div><br>
</div>
<div>First, I would only bother with this if this error is a problem. People use your method all the time and accept error at the scale of the solver tolerance.</div>
<div><br>
</div>
<div>But if you want the exact solution, well, you could just clobber the solution values after the solve if you have access to the Diri BC data on hand.</div>
<div><br>
</div>
<div>I was suggesting just creating a "Richardson/jacobi" solver, hardwire one iteration, use an initial guess and solve it.</div>
<div>Not great, because this would have to grab the diagonal. If you had the diagonal (eg, from the AMG smoother that does exist) then this would be pretty cheap (a residual calculation mostly).</div>
<div><br>
</div>
<div>Mark</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jul 7, 2022 at 3:14 AM Karabelas, Elias (<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>) <<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Hi Mark,</div>
<div>thanks for the answer, but I'm struggling to translate your suggestion into solver options.
<br>
</div>
<div>From scrolling through the user manual I think this points towards PCCOMPOSITE.<br>
</div>
<div>However, the User Manual is not very precise with composite PCs, so how would I achieve this on top of my existing solver options?</div>
<div>Best regards</div>
<div>Elias<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Am 06.07.22 um 16:41 schrieb Mark Adams:<br>
</div>
<blockquote type="cite">
<div dir="ltr">And one iteration of undamped Jacobi after the solve should fix this.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jul 6, 2022 at 8:42 AM Karabelas, Elias (<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>) <<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Dear Matt,</div>
<div><br>
</div>
<div>thanks for the fast response. That makes perfect sense to me. <br>
</div>
<div><br>
</div>
<div>Best regards</div>
<div>Elias<br>
</div>
<div><br>
</div>
<div>Am 06.07.22 um 14:35 schrieb Matthew Knepley:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">On Wed, Jul 6, 2022 at 7:46 AM Karabelas, Elias (<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>) <<a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>>
wrote:<br>
</div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><font size="4"><font face="monospace">Dear all,</font></font></p>
<p><font size="4"><font face="monospace">I don't know if this is a bug, but I observed that when using GMRES with AGG-PCGAMG as preconditioner Dirichlet boundary conditions don't seem to be exactly fulfilled.</font></font></p>
<p><font size="4"><font face="monospace">My Matrix has zero rows and cols with 1 on the diagonal where I have dirichlet-bcs in my FE-mesh and I would expect that the eqs in this rows can be exactly fulfilled (as u_i = g_i) there.</font></font></p>
</div>
</blockquote>
<div>I would not expect aggregation to be exact here, but only within the iteration tolerance. If instead you eliminate those variables, you can maintain algebraic exactness.</div>
<div>This is what we do in examples, like SNES ex56.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p><font size="4"><font face="monospace">However, when I solve A*x = b with the above solver I only get u_i = g_i + error in that part of the vector. Switching from pc_gamg_type agg to pc_gamg_type classical cures this problem, but the classical is not advertised
in the user manual.</font></font></p>
<p>These are the options I'm currently using:</p>
<p>-ksp_type gmres<br>
-ksp_pc_side right<br>
-pc_type gamg<br>
-pc_gamg_type agg [or classical]<br>
-pc_gamg_sym_graph 1<br>
-pc_gamg_square_graph 1<br>
-pc_gamg_agg_nsmooths 1<br>
-pc_gamg_threshold 0.01<br>
-pc_mg_cycles v<br>
</p>
<p>Iteration counts are basically the same.</p>
<p>Best regards</p>
<p>Elias<br>
</p>
<pre cols="72">--
Dr. Elias Karabelas
Research Associate
University of Graz
Institute of Mathematics and Scientific Computing
Heinrichstraße 36
A-8010 Graz
Austria
Phone: +43 316 380 8546
Email: <a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>
Web: <a href="https://ccl.medunigraz.at/" target="_blank">https://ccl.medunigraz.at/</a></pre>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
<div><br>
</div>
<div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
<pre cols="72">--
Dr. Elias Karabelas
Research Associate
University of Graz
Institute of Mathematics and Scientific Computing
Heinrichstraße 36
A-8010 Graz
Austria
Phone: +43 316 380 8546
Email: <a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>
Web: <a href="https://ccl.medunigraz.at/" target="_blank">https://ccl.medunigraz.at/</a></pre>
</div>
</blockquote>
</div>
</blockquote>
<p><br>
</p>
<pre cols="72">--
Dr. Elias Karabelas
Research Associate
University of Graz
Institute of Mathematics and Scientific Computing
Heinrichstraße 36
A-8010 Graz
Austria
Phone: +43 316 380 8546
Email: <a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>
Web: <a href="https://ccl.medunigraz.at/" target="_blank">https://ccl.medunigraz.at/</a></pre>
</div>
</blockquote>
</div>
</blockquote>
<p><br>
</p>
<pre cols="72">--
Dr. Elias Karabelas
Research Associate
University of Graz
Institute of Mathematics and Scientific Computing
Heinrichstraße 36
A-8010 Graz
Austria
Phone: +43 316 380 8546
Email: <a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>
Web: <a href="https://ccl.medunigraz.at/" target="_blank">https://ccl.medunigraz.at/</a></pre>
</div>
</blockquote>
</div>
</div>
</blockquote>
<p><br>
</p>
<pre cols="72">--
Dr. Elias Karabelas
Research Associate
University of Graz
Institute of Mathematics and Scientific Computing
Heinrichstraße 36
A-8010 Graz
Austria
Phone: +43 316 380 8546
Email: <a href="mailto:elias.karabelas@uni-graz.at" target="_blank">elias.karabelas@uni-graz.at</a>
Web: <a href="https://ccl.medunigraz.at/" target="_blank">https://ccl.medunigraz.at/</a></pre>
</div>
</blockquote></div>