On Fri, Jul 29, 2011 at 7:56 AM, John Chludzinski <span dir="ltr"><<a href="mailto:jchludzinski@gmail.com">jchludzinski@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<font face="courier new,monospace">These are the resulting stats from decomposing a 4002x4002 (dense matrices) generalized eigenvalue problem into 2 MPI processes. Note the amount of message traffic:</font></blockquote><div>
<br></div><div>The way to solve this problem is to use Elemental (<a href="http://code.google.com/p/elemental/">http://code.google.com/p/elemental/</a>), which solves the dense parallel eigenproblem.</div><div><br></div><div>
Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><font face="courier new,monospace"><br></font></div><div><div><font face="courier new,monospace"> Max Max/Min Avg Total</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04</font></div><div><font face="courier new,monospace">MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08</font></div>
<font face="courier new,monospace">
</font></div><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Total # of messages: </font><span style="font-family:'courier new', monospace"><font face="courier new,monospace">1.602e+04 with an average message length: </font></span><span style="font-family:'courier new', monospace"><font face="courier new,monospace">2.799e+0. </font></span></div>
<font face="courier new,monospace">
</font><div><span style="font-family:'courier new', monospace"><font face="courier new,monospace"><br></font></span></div><div><span style="font-family:'courier new', monospace"><font face="courier new,monospace"><br>
</font></span></div><div><span style="font-family:'courier new', monospace"><font face="courier new,monospace">With the number of MPI set to 1, you get (not surprisingly):</font></span></div><font face="courier new,monospace">
</font><div><span style="font-family:'courier new', monospace"><font face="courier new,monospace"><br></font></span></div><div><div><font face="courier new,monospace"> Max Max/Min Avg Total</font></div>
<font face="courier new,monospace">
</font><span style="font-family:'courier new', monospace"><div><font face="courier new,monospace">MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00</font></div><div><font face="courier new,monospace">MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00</font></div>
<font face="courier new,monospace">
</font></span><font face="courier new,monospace"><br></font><div><font face="courier new,monospace">In the end the time require to solve my 4002x4002 eigenvalue problem for 2 MPI processes: 1.821e+03 vs. 1.312e+03 for 1 MPI process. </font></div>
<div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Am I reading this correctly?</font></div><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace"><br>
</font></div><div><font face="courier new,monospace">---John<br></font></div><div><font face="courier new,monospace"><br></font></div><font face="courier new,monospace">
</font></div><div><font face="courier new,monospace"><br></font></div><div><b><font face="courier new,monospace"><font size="4">Complete stats for 2 MPI process run:</font></font></b></div><font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Using Petsc Release Version 3.1.0, Patch 7, Mon Dec 20 14:26:37 CST 2010</font></div><font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace"> Max Max/Min Avg Total</font></div><font face="courier new,monospace">
</font><div><font face="courier new,monospace">Time (sec): 1.821e+03 1.00064 1.820e+03</font></div><div><font face="courier new,monospace">Objects: 2.005e+04 1.00000 2.005e+04</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">Flops: 1.282e+11 1.00000 1.282e+11 2.564e+11</font></div><div><font face="courier new,monospace">Flops/sec: 7.046e+07 1.00064 7.044e+07 1.409e+08</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">Memory: 1.286e+09 1.00000 2.571e+09</font></div><div><font face="courier new,monospace">MPI Messages: 8.011e+03 1.00000 8.011e+03 1.602e+04</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">MPI Message Lengths: 2.242e+08 1.00000 2.799e+04 4.485e+08</font></div><div><font face="courier new,monospace">MPI Reductions: 2.412e+04 1.00000</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)</font></div><font face="courier new,monospace">
</font><div><font face="courier new,monospace"> e.g., VecAXPY() for real vectors of length N --> 2N flops</font></div><div><font face="courier new,monospace"> and VecAXPY() for complex vectors of length N --> 8N flops</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"> Avg %Total Avg %Total counts %Total Avg %Total counts %Total</font></div><div><font face="courier new,monospace"> 0: Main Stage: 1.8203e+03 100.0% 2.5645e+11 100.0% 1.602e+04 100.0% 2.799e+04 100.0% 2.007e+04 83.2%</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace"><br></font></div><div><span style="font-family:'courier new', monospace"><font size="4"><b><font face="courier new,monospace">Complete stats for 1 MPI process run:</font></b></font></span></div>
<font face="courier new,monospace">
</font><div><span style="font-family:'courier new', monospace"><font face="courier new,monospace"><br></font></span></div><div><div><font face="courier new,monospace"> Max Max/Min Avg Total</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">Time (sec): 1.312e+03 1.00000 1.312e+03</font></div><div><font face="courier new,monospace">Objects: 2.003e+04 1.00000 2.003e+04</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">Flops: 2.564e+11 1.00000 2.564e+11 2.564e+11</font></div><div><font face="courier new,monospace">Flops/sec: 1.955e+08 1.00000 1.955e+08 1.955e+08</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">Memory: 1.029e+09 1.00000 1.029e+09</font></div><div><font face="courier new,monospace">MPI Messages: 0.000e+00 0.00000 0.000e+00 0.000e+00</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace">MPI Message Lengths: 0.000e+00 0.00000 0.000e+00 0.000e+00</font></div><div><font face="courier new,monospace">MPI Reductions: 2.404e+04 1.00000</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)</font></div><font face="courier new,monospace">
</font><div><font face="courier new,monospace"> e.g., VecAXPY() for real vectors of length N --> 2N flops</font></div><div><font face="courier new,monospace"> and VecAXPY() for complex vectors of length N --> 8N flops</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"><br></font></div><div><font face="courier new,monospace">Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --</font></div>
<font face="courier new,monospace">
</font><div><font face="courier new,monospace"> Avg %Total Avg %Total counts %Total Avg %Total counts %Total</font></div><div><font face="courier new,monospace"> 0: Main Stage: 1.3119e+03 100.0% 2.5645e+11 100.0% 0.000e+00 0.0% 0.000e+00 0.0% 2.002e+04 83.3%</font></div>
</div>
<div class="gmail_quote"><br>
</div><br>
</blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<br>