<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Oct 14, 2015 at 9:32 PM, Timothée Nicolas <span dir="ltr"><<a href="mailto:timothee.nicolas@gmail.com" target="_blank">timothee.nicolas@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Thank you Barry for pointing this out. Indeed on a system with no debugging the Jacobian evaluations no longer dominate the time (less than 10%). However the rest is similar, except the improvement from 2 to 3 levels is much better. Still it saturates after levels=3. I understand it in terms of CPU time thanks to Matthew's explanations, however what surprises me more is that KSP iterations are not more efficient. At the least, even if it takes more time to have more levels because of memory issues, I would expect KSP iterations to converge more rapidly with more levels, but it is not the case as you can see. Probably there is also a rationale behind this but I cannot see easily. <br></div></div></div></div></blockquote><div><br></div><div>That conclusion makes no sense to me. Thought experiment:</div><div><br></div><div> I have K levels, which means that on the coarsest level K I do a direct solve. Now I add a level</div><div> so that I have K+1. On level K-1, now instead using the result of a direct solve as a starting guess,</div><div> I use some iterative result. I cannot imagine that the iterates would go down.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div></div>I send the new outputs<br><br></div>Best<br><br></div>Timothee<br></div><div class="gmail_extra"><br><div class="gmail_quote">2015-10-15 3:02 GMT+09:00 Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">1) Your timings are meaningless! You cannot compare timings when built with all debugging on, PERIOD!<br>
<br>
##########################################################<br>
# #<br>
# WARNING!!! #<br>
# #<br>
# This code was compiled with a debugging option, #<br>
# To get timing results run ./configure #<br>
# using --with-debugging=no, the performance will #<br>
# be generally two or three times faster. #<br>
# #<br>
##########################################################<br>
<br>
2) Please run with -snes_view .<br>
<br>
3) Note that with 7 levels<br>
<br>
SNESJacobianEval 21 1.0 2.4364e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 54 0 0 0 0 54 0 0 0 0 0<br>
<br>
with 2 levels<br>
<br>
SNESJacobianEval 6 1.0 2.2441e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 34 0 0 0 0 34 0 0 0 0 0<br>
<br>
<br>
The Jacobian evaluation is dominating the time! Likely if you fix the debugging this will be less the case<br>
<br>
Barry<br>
<div><div><br>
> On Oct 13, 2015, at 9:23 PM, Timothée Nicolas <<a href="mailto:timothee.nicolas@gmail.com" target="_blank">timothee.nicolas@gmail.com</a>> wrote:<br>
><br>
> Dear all,<br>
><br>
> I have been playing around with multigrid recently, namely with /ksp/ksp/examples/tutorials/ex42.c, with /snes/examples/tutorial/ex5.c and with my own implementation of a laplacian type problem. In all cases, I have noted no improvement whatsoever in the performance, whether in CPU time or KSP iteration, by varying the number of levels of the multigrid solver. As an example, I have attached the log_summary for ex5.c with nlevels = 2 to 7, launched by<br>
><br>
> mpiexec -n 1 ./ex5 -da_grid_x 21 -da_grid_y 21 -ksp_rtol 1.0e-9 -da_refine 6 -pc_type mg -pc_mg_levels # -snes_monitor -ksp_monitor -log_summary<br>
><br>
> where -pc_mg_levels is set to a number between 2 and 7.<br>
><br>
> So there is a noticeable CPU time improvement from 2 levels to 3 levels (30%), and then no improvement whatsoever. I am surprised because with 6 levels of refinement of the DMDA the fine grid has more than 1200 points so with 3 levels the coarse grid still has more than 300 points which is still pretty large (I assume the ratio between grids is 2). I am wondering how the coarse solver efficiently solves the problem on the coarse grid with such a large number of points ? Given the principle of multigrid which is to erase the smooth part of the error with relaxation methods, which are usually efficient only for high frequency, I would expect optimal performance when the coarse grid is basically just a few points in each direction. Does anyone know why the performance saturates at low number of levels ? Basically what happens internally seems to be quite different from what I would expect...<br>
><br>
> Best<br>
><br>
> Timothee<br>
</div></div>> <ex5_2_levels_of_multigrid.log><ex5_3_levels_of_multigrid.log><ex5_4_levels_of_multigrid.log><ex5_5_levels_of_multigrid.log><ex5_6_levels_of_multigrid.log><ex5_7_levels_of_multigrid.log><br>
<br>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>