<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, May 6, 2015 at 10:29 AM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Perhaps I should explain what my ultimate goal is.<br><br></div>I have two sets of code. First is the classical FEM which gives numerically "incorrect answers" and the classical FEM employing convex optimization through TAO which fixes these "incorrect answers". We have shown numerically that the latter should always be the way to solve these types of problems but now I want to compare the computational performance between these two methodologies for large problems.<br><br></div>I already have speedup/strong scaling results that essentially depict the difference between the KSPSolve() and TaoSolve(). However, I have been told by someone that strong-scaling isn't enough - that I should somehow include something to show the "efficiency" of these two methodologies. That is, how much of the wall-clock time reported by these two very different solvers is spent doing useful work.<br><br></div><div>Is such an "efficiency" metric necessary to report in addition to strong-scaling results? The overall computational framework is the same for both problems, the only difference being one uses a linear solver and the other uses an optimization solver. My first thought was to use PAPI to include hardware counters, but these are notoriously inaccurate. Then I thought about simply reporting the manual FLOPS and FLOPS/s via PETSc, but these metrics ignore memory bandwidth. And so here I am looking at the idea of implementing the Roofline model, but now I am wondering if any of this is worth the trouble.</div></div></div></div></blockquote><div><br></div><div>I think its always best to start with a goal, and then explain how to achieve that goal.</div><div><br></div><div>If your goal is "Run a particular problem as fast as possible", then strong scaling is a good metric. It shows</div><div>you how much time improvement you get by adding more processors. However, suppose I just put a TON</div><div>of useless work into my solver which is perfectly parallel, like factoring a bunch of large numbers. Now I can</div><div>show perfect strong scaling, but this is not the most _efficient_ way to implement this solver.</div><div><br></div><div>Now there are at least two kinds of efficiency, hardware efficiency and algorithmic efficiency. For both you have</div><div>to have some conceptual model of the computation. Then I can say "This is how long I expect things to take in</div><div>a perfect world", and "this is how long your algorithm takes". Also, I can say "This is how fast it should run on</div><div>this hardware in a perfect world", and "this is how fast it runs on my machine".</div><div><br></div><div>Both measures are interesting and useful. As Jed points out, you must have some kind of model for the computation</div><div>in order to make claims about "efficiency". For example, I expect VecDot() for length N vectors to use 2N flops and</div><div>pull 2N data, and do one reduction. You can imagine algorithms which do this slightly differently. You can imagine</div><div>hardware where this is flop limited and where this is memory bandwidth limited. I can talk about these tradeoffs once</div><div>I have the simple computational model.</div><div><br></div><div>Does this help?</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Thanks,<br></div><div>Justin<br></div></div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 6, 2015 at 8:34 AM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Wed, May 6, 2015 at 8:29 AM, Jed Brown <span dir="ltr"><<a href="mailto:jed@jedbrown.org" target="_blank">jed@jedbrown.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> writes:<br>
> I think the idea is to be explicit. I would just use something like<br>
><br>
> # Vecs * Vec size * 8 bytes/double + <same for Mat><br>
><br>
> and forget about small stuff. This model is uncached,<br>
<br>
</span>This is for a perfect cache model -- each byte of the data structures<br>
needs to be fetched from DRAM only once.<br></blockquote><div><br></div></span><div>I meant uncached, in which you count # Vecs for any operation you are</div><div>doing. If you count # Vecs for the whole program, then you have perfect cache.</div><div><br></div><div> Matt</div><span><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Make sure that "# Vecs" actually means the number that are accessed.<br>
For example, GMRES accesses an increasing number of vectors on each<br>
iteration (until a restart).<br>
<div><div><br>
> and you just state that up front. Someone else can take the data and<br>
> produce a more nuanced model.<br>
><br>
> You can use the Flop counts (not Flop/s) from PETSc, and get a rough<br>
> estimate<br>
> of flop/byte ratio.<br>
<br>
</div></div></blockquote></span></div><br><br clear="all"><span><div><br></div>-- <br><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>