<div dir="ltr"><div>Thanks Karl! Yes, please send your preprint, I'd like to learn more about this.<br></div><div>Best regards<br><br></div>Andrea<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jan 17, 2014 at 10:33 PM, Karl Rupp <span dir="ltr"><<a href="mailto:rupp@mcs.anl.gov" target="_blank">rupp@mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Andrea,<div class="im"><br>
<br>
> Well, I have 9 equations, so 9x9 I guess...<br>
<br></div>
Ok, this is just in the range where it is technically meaningful (register sizes) but getting challenging implementation-wise (explicit inversion formulas vs. Gauss with pivoting)<div class="im"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I hope the one you are mentioning was a major bug, because what I get is<br>
seriously wrong: while on single GPU (KSPGMRES+PCASM) I get a residual<br>
of +0.72, on 8-cores/GPU I get -1.00 at the first time step, just to<br>
make an example. Can this be due to the bug you are saying or you can<br>
suspect something more?<br>
</blockquote>
<br></div>
Yes, this was a major bug, breaking the matrix-vector product when using multiple MPI ranks with GPUs.<div class="im"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
What should I do then? wait for the valgrind fix which is underway and<br>
then update? Can you please notify me when this is fixed? I'm writing a<br>
final report for a project and I would like to include this feature<br>
fully fixed if possible.<br>
</blockquote>
<br></div>
I will merge the fix to master tomorrow when I'm back on my main GPU machine (there do not seem to be any problems in 'next' with the patch) and fix the valgrind complaints separately. The second issue is not directly related to the first, it only happens in the same module.<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Another question, what do you exactly mean by "order the unknowns<br>
properly" in this case?<br>
</blockquote>
<br></div>
If you build the elimination graph for the triangular factors of ILU preconditioners, then the ordering of the unknowns (i.e. the way you assign the degrees of freedoms (DOFs) on your mesh) can have a considerable influence on the amount of parallelism. The Cuthill-McKee algorithm for example is quite good for reducing the bandwidth of a sparse matrix, but it may also reduce the amount of parallelism for ILU0 factors compared to e.g. a red-black ordering of the DOFs. I can send you a preprint if you're interested.<br>
<br>
Best regards,<br>
Karli<br>
<br>
</blockquote></div><br><br clear="all"><br>-- <br><span style="color:rgb(102,102,102)">Dr. Andrea Lani </span><br style="color:rgb(102,102,102)">
<span style="color:rgb(102,102,102)">Senior Research Engineer, PhD</span><br style="color:rgb(102,102,102)"><span style="color:rgb(102,102,102)">Aeronautics & Aerospace dept., CFD group </span><span style="color:rgb(102,102,102)"><br>
Von Karman Institute for Fluid Dynamics </span><br style="color:rgb(102,102,102)"><span style="color:rgb(102,102,102)">
Chausse de Waterloo 72, </span><br style="color:rgb(102,102,102)"><span style="color:rgb(102,102,102)">B-1640, Rhode-Saint-Genese, Belgium</span><span style="color:rgb(102,102,102)"><br>fax : +32-2-3599600 </span><br style="color:rgb(102,102,102)">
<span style="color:rgb(153,153,153)"><span style="color:rgb(102,102,102)">
work : +32-2-3599769 </span> <span style="background-color:rgb(255,255,255)"></span></span><u style="background-color:rgb(51,51,255);color:rgb(153,153,153)"><br><span style="background-color:rgb(255,255,255);color:rgb(0,0,153)"><a href="mailto:lani@vki.ac.be" target="_blank">lani@vki.ac.be</a></span></u>
</div>