<br>Matt, Jed, thanks a lot for the discussions. Since the ordering could minimizing the bandwidth, I think it is really worth to have a try with the matrix partitioning / ordering. If there is a factor two of increase in the flop rate, that's quite promising!<br>
<br><br><div class="gmail_quote">On Thu, Dec 23, 2010 at 3:32 AM, Jed Brown <span dir="ltr"><<a href="mailto:jed@59a2.org">jed@59a2.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<p>I disagree, there is easily a factor of two in flop/s between a naive ordering (e.g. hierarchical by node type in a finite element method) and a good low-bandwidth ordering.</p>
<p>This is in the FUN3D papers and still true today, in my experience.</p>
<p>Incomplete factorization is also very order dependent, as you note.</p>
<p>Jed</p>
<p></p><blockquote type="cite"><div class="im">On Dec 22, 2010 5:03 PM, "Matthew Knepley" <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br><br><p><font color="#500050">On Wed, Dec 22, 2010 at 10:11 AM, Yongjun Chen <<a href="mailto:yjxd.chen@gmail.com" target="_blank">yjxd.chen@gmail.com</a>> wrote:</font></p>
</div><div class="gmail_quote"><p><font color="#500050"><div class="im">><br>> On Wed, Dec 22, 2010 at 6:53 PM, Satish Balay <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>> wrote:<br>
>><br></div>>> On Wed, 22 De...</font></p><div class="im">
<div>1) To see a large gain, the ordering you start with would have to be very bad. Maybe it is. These</div>
<div> orderings try to minimize bandwidth, which means minimize communication in the MatMult.</div><div><br></div><div>2) If you use incomplete facotrization, the ordering can have a large effect on conditioning, so</div>
<div> number of iterations, which does not improve scalability. This would impact scalability if you</div><div> use a parallel IC, however all those packages reorder your matrix already.</div><div><br></div><div>In short, I suspect this will not help a lot, except maybe with conditioning, which is what I was refering to in the quote.</div>
<div><br></div><div> Matt</div><div> <br></div></div></div><p><font color="#500050">-- <br>What most experimenters take for granted before they begin their experiments is infinitely more...</font></p></blockquote>
</blockquote></div><br><br>