<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Jun 4, 2015 at 10:31 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Yeah I saw his recommendation and am trying it out. But I am not sure what most of those parameters mean. For instance:<br><br></div>1) What does -pc_gamg_agg_nsmooths refer to?<br></div></div></div></div></div></blockquote><div><br></div><div>This is always 1 (its the definition of smoothed aggregation). Mark allows 0 to support unsmoothed aggregation, which may be</div><div>better for easy problems on extremely large machines.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div></div>2) Does increase in the threshold of -pc_gamg_threshold translate to making the coarsening faster?<br></div></div></div></div></blockquote><div><br></div><div>Yes, I believe so (easy to check).</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>3) What does -mg_levels_ksp_max_it refer to?<br></div></div></div></div></blockquote><div><br></div><div>This sets the maximum number of iterations for the smoother on each level.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div></div>I am not too worried about "bad" solutions from anisotropy (because it is driven by the velocity field, not the actual material property or mesh orientations) and we have a work-around for it (via optimization). I am more concerned about the time to solution especially for bigger problems.</div></div></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div></div>Thanks,<br></div>Justin<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jun 4, 2015 at 9:04 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div>On Thu, Jun 4, 2015 at 8:12 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>Hello everyone,<br><br></div>Apologies if this sounds like a newbie question, but I am attempting to play around with the gamg preconditioner for my anisotropic diffusion problem, but I have no idea how to "fine tune" the parameters such that I get better performance. I understand that this depends on both the material properties and the type of mesh used, but I guess for starters, here is what I am doing and some stuff I have noticed:<br><br></div>- I am solving a unit cube with a small hole in the middle. The outside BC conditions are 0 and the inside is unity. I have a tensorial dispersion diffusivity (with constant velocity). I have 6 different sized grids to solve this problem on. The problem sizes range from 36K dofs to 1M dofs. I was able to solve all of them using the CG and Jacobi solver and preconditioner combination. <br><br>- When I try to solve them using CG and GAMG (I did not set any other command line options) I seem to get slower wall clock times but with much fewer KSP iterations. I also notice that the FLOPS/s metric is much smaller.<br></div><br></div>- For certain meshes, my CG/GAMG solver fails to converge after 2 iterations due to DIVERGED_INDEFINITE_PC. This does not happen when I solve this on one processor or with the CG/Jacobi solver.<br><br></div>From what I have read online and through these petsc-mailing lists, it sounds to me the gamg preconditioner will give better performance for nice elliptic problems like the one I am solving. When I saw the SNES ex12 test case 39 from builder.py, it only had -pc_type gamg. I am guessing that I need to set additional command line options? If so, where should I start?<br></div></div></div></blockquote><div><br></div></div></div><div>Mark recommends starting with</div><div><br></div><div><div style="font-size:12.8000001907349px"><div>-ksp_type cg</div><div>-pc_type <span>gamg</span></div><div>-pc_gamg_agg_nsmooths 1<br></div><div>-pc_gamg_threshold 0.02 # [0 - 0.1]</div><div>#-mg_levels_ksp_type richardson</div><div>-mg_levels_ksp_type chebyshev<br></div><div>-mg_levels_pc_type sor</div><div>#-mg_levels_pc_type jacobi<br></div><div>-mg_levels_ksp_max_it 2 # [1-8]</div><div><br></div><div>and messing with the # stuff.</div><div><br></div><div>1) GAMG should definitely be more scalable than CG/ILU. You should be able to see this by plotting the time/iterates as a function of problem size. GAMG</div><div> should have a constant number of iterates, whereas CG should grow. GAMG will have higher overheads, so for smaller problem sizes CG can be better.</div><div><br></div><div>2) The <span style="font-size:small">DIVERGED_INDEFINITE_PC can happen if the GAMG is non-symmetric due to different number of iterates or the subsolver choice. Try out</span></div><div><span style="font-size:small"> richardson instead of chebyshev.</span></div><div><span style="font-size:small"><br></span></div><div><span style="font-size:small">3) If your coefficient is highly anisotropic, GAMG may need help making good coarse basis vectors. Mark has some experimental stuff for this. Adjusting the</span></div><div><span style="font-size:small"> thresholding parameter determines how fast you coarsen (you can see the coarse problem sizes in -ksp_view). If its too fast, convergence deteriorates,</span></div><div><span style="font-size:small"> too slow and the coarse problems are expensive.</span></div><div><span style="font-size:small"><br></span></div><div><span style="font-size:small"> Mark, how do you turn on heavy-edge matching?</span></div><div><span style="font-size:small"><br></span></div><div><span style="font-size:small"> Thanks,</span></div><div><span style="font-size:small"><br></span></div><div><span style="font-size:small"> Matt</span></div></div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div>Thanks,<br></div>Justin<span><font color="#888888"><br></font></span></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>