<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hello Mark,</p>
<p><br>
</p>
<p>Thanks for your answer. Indeed, I didn't see the information that classical AMG was not really supported:
<br>
</p>
<p><br>
</p>
<p></p>
<div> -solver2_pc_gamg_type <now classical : formerly agg>: Type of AMG method (<span style="color: rgb(255, 0, 0);">only 'agg' supported and useful</span>) (one of)
<span style="color: rgb(255, 0, 0);">classical geo agg</span> (PCGAMGSetType)<br>
<br>
</div>
<div>We switched very recently from GAMG("agg") to GAMG("classical") for a weak scaling test up to 32000 cores, where we saw very good scalability with
<span>GAMG("classical") </span>compared to <span>GAMG("agg")</span>. But it was with PETSc 3.14...</div>
<div><br>
</div>
<div>So today, we are going to upgrade to 3.20 and focus on <span>GAMG("agg") or Hypre Classical AMG.
<span>We will see</span> how it compares.<br>
</span></div>
<div><span><br>
</span></div>
<div><span>May I ask you what is your point of view of the current state of the GPU versions of GAMG("agg") versus Hypre AMG Classical ?</span></div>
<div><span><br>
</span></div>
<div><span>In fact, the reason of our move from 3.14 to 3.20 is to take advantage of all the progress in PETSc and Hypre on accelerated solvers/preconditioners during the last 2 years.<br>
</span></div>
<div><span><br>
</span></div>
<div><span>Greatly appreciate your help,<br>
</span></div>
<div><span><br>
</span></div>
<p></p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div style="font-family:Tahoma; font-size:13px">
<div class="BodyFragment"><font size="2"><span style="font-size:10pt">
<div class="PlainText">Pierre LEDAC<br>
Commissariat à l’énergie atomique et aux énergies alternatives<br>
Centre de SACLAY<br>
DES/ISAS/DM2S/SGLS/LCAN<br>
Bâtiment 451 – point courrier n°43<br>
F-91191 Gif-sur-Yvette<br>
+33 1 69 08 04 03<br>
+33 6 83 42 05 79</div>
</span></font></div>
</div>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>De :</b> Mark Adams <mfadams@lbl.gov><br>
<b>Envoyé :</b> mercredi 13 décembre 2023 20:54:17<br>
<b>À :</b> LEDAC Pierre<br>
<b>Cc :</b> petsc-users@mcs.anl.gov; BRUNETON Adrien<br>
<b>Objet :</b> Re: [petsc-users] PETSc 3.14 to PETSc 3.20: Different (slower) convergence for classical AMG (sequential and especially in parallel)</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi Pierre,
<div><br>
</div>
<div>Sorry I missed this post and your issues were brought to my attention today.</div>
<div><br>
</div>
<div>First, the classic version is not supported well. The postdoc that wrote the code is long gone and I don't know the code at all.</div>
<div>It is really a reference implementation that someone could build on and is not meant for production.</div>
<div>In 10 years you are the first user that has connected us.</div>
<div><br>
</div>
<div>The hypre package is a very good AMG solver and it uses classical AMG as the main solver.</div>
<div>I wrote GAMG ("agg") which is a smoothed aggregation AMG solver and is very different from classical.</div>
<div>I would suggest you move to hypre or '-pc_gamg_type agg'.</div>
<div><br>
</div>
<div>The coarsening was developed in this time frame and there was a lot of churn as a new strategy for aggressive coarsening did not work well for some users and I had to add the old method in and then made it the default (again).</div>
<div>This change missed v3.20, but you can get the old aggressive strategy with '-pc_gamg_aggressive_square_graph'.</div>
<div>Check with -options_left to check that it is being used.</div>
<div><br>
</div>
<div>As far as your output (nice formatting, thank you), the coarse grid is smaller in the new code.</div>
<div> rows=41, cols=41 | rows=30, cols=30<br>
</div>
<div>"square graph" should fix this.</div>
<div><br>
</div>
<div>You can also try not using aggressive coarsening with:</div>
<div>You could try '-pc_gamg_aggressive_coarsening 0'</div>
<div><br>
</div>
<div>Let me know how it goes and let's try to get you into a more sustainable state ... I really try not to change this code but sometimes need to.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Mark</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Oct 9, 2023 at 10:43 AM LEDAC Pierre <<a href="mailto:Pierre.LEDAC@cea.fr">Pierre.LEDAC@cea.fr</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="msg2203274150731110762">
<div dir="ltr">
<div id="m_4559298248948725813divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p>Hello all,</p>
<p><br>
</p>
<p>I am struggling to find the same convergence in iterations when using classical algebric multigrid in my code with PETSc 3.20 compared to PETSc 3.14.</p>
<p><br>
</p>
<p>I am using in order to solve a Poisson system:</p>
<p><b>-ksp_type cg -pc_type gamg -<span>pc_gamg_type</span> classical</b><br>
</p>
<p><br>
</p>
<p>I read the different releases notes between 3.15 and 3.20:</p>
<p><a href="https://petsc.org/release/changes/317" id="m_4559298248948725813LPlnk566993" target="_blank">https://petsc.org/release/changes/317</a>
<br>
</p>
<p><a href="https://petsc.org/main/manualpages/PC/PCGAMGSetThreshold/" target="_blank">https://petsc.org/main/manualpages/PC/PCGAMGSetThreshold/</a><br>
</p>
<p><br>
</p>
<p>And have a look at the archive mailing list (especially this one: <a href="https://www.mail-archive.com/petsc-users@mcs.anl.gov/msg46688.html" target="_blank">
https://www.mail-archive.com/petsc-users@mcs.anl.gov/msg46688.html</a>) <br>
</p>
<p>so I added some other options to try to have the same behaviour than PETSc 3.14:</p>
<p><br>
</p>
<p></p>
<p><b>-ksp_type cg -pc_type gamg -<span>pc_gamg_type</span> classical </b><span style="color:rgb(255,0,0)">-</span><span><span style="color:rgb(255,0,0)">mg_levels_pc_type sor -</span><span style="color:rgb(255,0,0)">pc_gamg_threshold</span><span style="color:rgb(255,0,0)">
0.</span></span></p>
<p></p>
<p><br>
</p>
<p>It improves the convergence but there still a different convergence though (26 vs 18 iterations).</p>
<p>On another of my test case, the number of levels is different (e.g. 6 vs 4) also, and here it is the same, but with a different coarsening according to the output from the -ksp_view option</p>
<br>
<p><b></b>The main point is that the convergence dramatically degrades in parallel on a third test case, so I can't upgrade to PETSc 3.20 for now unhappily.<br>
</p>
<br>
<p>I send you the partial report (petsc_314_vs_petsc_320.ksp_view) with -ksp_view (left PETSc 3.14, right PETSc 3.20) and the configure/command line options used (in petsc_XXX_petsc.TU files).</p>
<p><br>
</p>
<p>Could my issue related to the following 3.18 change ? I have not tried the first one.</p>
<p></p>
<ul style="box-sizing:border-box;margin-bottom:1rem;margin-top:0px;color:rgb(50,50,50);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-size:15px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;word-spacing:0px;white-space:normal;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">
<li style="box-sizing:border-box">
<p style="box-sizing:border-box;margin-bottom:0.25rem;margin-top:0px;font-size:1em">
Remove<span> </span><code class="gmail-notranslate" style="box-sizing:border-box;font-size:13.125px;border-radius:0.25rem;padding:0.1rem 0.25rem"><span style="box-sizing:border-box;white-space:nowrap">PCGAMGSetSymGraph()</span></code><span> </span>and<span> </span><code class="gmail-notranslate" style="box-sizing:border-box;font-size:13.125px;border-radius:0.25rem;padding:0.1rem 0.25rem"><span style="box-sizing:border-box;white-space:nowrap">-pc_gamg_sym_graph</span></code>.
<span style="color:rgb(255,0,0)">The user should now indicate symmetry and structural symmetry using</span><span style="color:rgb(255,0,0)"> </span><code class="gmail-notranslate" style="box-sizing:border-box;font-size:13.125px;border-radius:0.25rem;padding:0.1rem 0.25rem"><span style="box-sizing:border-box;white-space:nowrap"><a href="https://petsc.org/release/manualpages/Mat/MatSetOption/" style="box-sizing:border-box;background-color:transparent;text-decoration:none" target="_blank"><span style="color:rgb(255,0,0)">MatSetOption</span></a><span style="color:rgb(255,0,0)">()</span></span></code><span style="color:rgb(255,0,0)"> </span><span style="color:rgb(255,0,0)">and
GAMG will symmetrize the graph if a symmetric options is not set</span></p>
</li><li style="box-sizing:border-box">
<p style="box-sizing:border-box;margin-bottom:0px;margin-top:0px;font-size:1em">Change<span> </span><code class="gmail-notranslate" style="box-sizing:border-box;font-size:13.125px;border-radius:0.25rem;padding:0.1rem 0.25rem"><span style="box-sizing:border-box;white-space:nowrap">-pc_gamg_reuse_interpolation</span></code><span style="color:rgb(255,0,0)"> </span><span style="color:rgb(255,0,0)">default
from false to true.</span></p>
</li></ul>
<br>
<p></p>
<p>Any advice would be greatly appreciated,<br>
</p>
<p><br>
</p>
<div id="m_4559298248948725813Signature">
<div id="m_4559298248948725813divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div style="font-family:Tahoma;font-size:13px">
<div><font size="2"><span style="font-size:10pt">
<div>Pierre LEDAC<br>
Commissariat à l’énergie atomique et aux énergies alternatives<br>
Centre de SACLAY<br>
DES/ISAS/DM2S/SGLS/LCAN<br>
Bâtiment 451 – point courrier n°43<br>
F-91191 Gif-sur-Yvette<br>
+33 1 69 08 04 03<br>
+33 6 83 42 05 79</div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>