On Fri, Oct 28, 2011 at 10:24 AM, Fredrik Heffer Valdmanis <span dir="ltr"><<a href="mailto:fredva@ifi.uio.no">fredva@ifi.uio.no</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi,<div><br></div><div>I am working on integrating the new GPU based vectors and matrices into FEniCS. Now, I'm looking at the possibility for getting some speedup during finite element assembly, specifically when inserting the local element matrix into the global element matrix. In that regard, I have a few questions I hope you can help me out with:</div>
<div><br></div><div>- When calling MatSetValues with a MATSEQAIJCUSP matrix as parameter, what exactly is it that happens? As far as I can see, MatSetValues is not implemented for GPU based matrices, neither is the mat->ops->setvalues set to point at any function for this Mat type. </div>
</blockquote><div><br></div><div>Yes, MatSetValues always operates on the CPU side. It would not make sense to do individual operations on the GPU.</div><div><br></div><div>I have written batched of assembly for element matrices that are all the same size:</div>
<div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatSetValuesBatch.html">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatSetValuesBatch.html</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>- Is it such that matrices are assembled in their entirety on the CPU, and then copied over to the GPU (after calling MatAssemblyBegin)? Or are values copied over to the GPU each time you call MatSetValues?</div>
</blockquote><div><br></div><div>That function assembles the matrix on the GPU and then copies to the CPU. The only time you do not want this copy is when</div><div>you are running in serial and never touch the matrix afterwards, so I left it in.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>- Can we expect to see any speedup from using MatSetValuesBatch over MatSetValues, or is the batch version simply a utility function? This question goes for both CPU- and GPU-based matrices.</div>
</blockquote><div><br></div><div>CPU: no</div><div><br></div><div>GPU: yes, I see about the memory bandwidth ratio</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>Thanks,</div><div><br></div><div>Fredrik V</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>