<div dir="ltr"><div>Manav,</div><div><br></div><div>How are interpolated the rotational degrees of freedom? AFAIK, when using smoothed aggregation, the interpolation process tries to mimic linear interpolation, which can be OK for the displacement DOF but is not for the rotational DOF using some plate and shell formulations. <br></div><div>This can explain poor convergence of a multilevel approach, which needs to restrict and extrapolate the unknowns. In order to check this hypothesis, you can try a test case with zero rotations.</div><div><br></div><div>Nicolas<br></div></div><br><div class="gmail_quote"><div dir="ltr">Le lun. 29 oct. 2018 à 22:13, Mark Adams via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">* the two level results tell us that MG is not doing well on the coarse grids. So the coarse grids are the problem.<div><br></div><div>* Do not worry about timing now. Get the math correct. The two level solve is not meant to be a solution just a diagnostic so don't try to optimize it by squaring the graph. Use -pc_gamg_square_graph 0.</div><div><br></div><div>* It looks like you don't need 4 smoothing steps but lets keep it and we can dial it back later.</div><div><br></div><div>* This table is interesting. First, you had about 12 iterations earlier and I think your rtol was tighter than the default (so the iteration could should go down not up). Do you know what change here?</div><div><br></div><div>Note, even though -mg_levels_ksp_max_it is not in the ksp_view it does work. It is syntactic sugar to just add it to all levels like you did manually.</div><div><br></div><div>Anyway, these number look reasonable. It is interesting that 3 levels ran well but the 4th level ran poorly. This implies we want to slow down coarsening on these levels, but ...</div><div><br></div><div>First can you please rerun this experiment with -pc_gamg_square_graph 0.</div><div><br></div><div>Also, please run with -info. This is very noisy but you can grep on "GAMG" and send that output to us (about 15 lines).</div><div><br></div><div>Thanks,</div><div>Mark</div><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Oct 29, 2018 at 3:34 PM Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Barry, <div><br></div><div> Here are some quick numbers with the following options on 4 CPUs and 543,606 dofs: </div><div><br></div><div><div style="margin:0px;font-stretch:normal;font-size:10px;line-height:normal;font-family:Monaco;color:rgb(242,242,242);background-color:rgba(0,0,0,0.85098)"><span style="font-variant-ligatures:no-common-ligatures">-mg_levels_ksp_max_it 4 -pc_gamg_square_graph 1 -pc_gamg_threshold 0.</span></div></div><div><br></div><div> #levels | #KSP Iters</div><div>———————————</div><div> 2 | 18</div><div> 3 | 18</div><div><div> 4 | 40</div><div><div> 5 | 59</div></div><div><br></div><div>-Manav</div><div><br></div><div><br><blockquote type="cite"><div>On Oct 29, 2018, at 2:06 PM, Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:</div><br class="m_-924147432417557279m_7661141021920953809m_-8340025827211607414m_-3681596211813809245Apple-interchange-newline"><div><div><br> Exactly how much does it increase with number of levels? Send a chart number of levels and number of iterations. With say -mg_levels_ksp_maxit 4<br><br> Thanks<br><br> Barry<br><br><br><br><br><blockquote type="cite">On Oct 29, 2018, at 12:59 PM, Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> wrote:<br><br>Thanks for the clarification. <br><br>I also observed that the number of KSP iterations increases with an increase in the levels of AMG. Is this true, in general, for all/most applications? <br><br>-Manav<br><br><blockquote type="cite">On Oct 29, 2018, at 12:53 PM, Jed Brown <<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>> wrote:<br><br>Manav Bhatia <<a href="mailto:bhatiamanav@gmail.com" target="_blank">bhatiamanav@gmail.com</a>> writes:<br><br><blockquote type="cite">Thanks, Jed. <br><br>The description says: “ Square the graph, ie. compute A'*A before aggregating it"<br><br>What is A here? <br></blockquote><br>The original matrix, or its "graph" (your 6x6 blocks condensed to scalars).<br><br><blockquote type="cite">What is the impact of setting this to 0, which led to a very significant increase in the CPU time in my case? <br></blockquote><br>The aggregates are formed on the connectivity of your original matrix,<br>so root nodes are aggregated only with their first neighbors, resulting<br>in slower coarsening.<br></blockquote><br></blockquote><br></div></div></blockquote></div><br></div></div></blockquote></div>
</blockquote></div>