<br><div class="gmail_extra">On Sun, Nov 4, 2012 at 8:51 PM, Karl Rupp <span dir="ltr"><<a href="mailto:rupp@mcs.anl.gov" target="_blank">rupp@mcs.anl.gov</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi guys,<br>
<br>
I've made quite some progress with my unification approach for GPUs and threading. Here's how my toy framework is set up:<br></blockquote><div><br></div><div>Very cool. I did not expect the large AMD overheads, but the results below make the current CUDA strategy look</div>
<div>pretty good, and an OpenCL strategy look fine for large problems.</div><div><br></div><div>I have a more basic question. I saw the previous implementation in very simple terms:</div><div><br></div><div> 1) Replication of CPU vectors on the GPU</div>
<div><br></div><div> 2) A coherence policy</div><div><br></div><div>The nice thing here is how robust it is. We really do not have to commit to any implementation because</div><div>the CPU part can always pick up the slack. From what is written below, I cannot understand how the</div>
<div>"coherence policy" works.</div><div><br></div><div>Lets use an example to explain it to me. Say that you have a CUDA vector, but you want to execute</div><div>VecPointwiseMult() with another Vec, but that operation is not part of your CUDA implementation.</div>
<div>What happens?</div><div><br></div><div> Thanks, </div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Each (linear algebra) object is equipped with a memory handle, e.g.<br>
<br>
struct Vector {<br>
/* other stuff here */<br>
<br>
memory_handle my_handle;<br>
};<br>
<br>
The handle itself, however, is not a-priori bound to a particular domain, but rather holds a collection of raw handles. Ignoring implementation details, this looks as follows:<br>
<br>
struct memory_handle {<br>
void * cpu_data;<br>
cl_mem opencl_data;<br>
void * cuda_data;<br>
size_t active_handle;<br>
};<br>
<br>
Right now, active_handle is used to indicate which of the handles holds the current data. This could be further refined to holding a valid-flag for each of the raw pointers. Adding all the management logic, kernels, etc., one then obtains user code comparable to<br>
<br>
Vector v1, v2, v3;<br>
// ...<br>
GenericVectorAdd(v1, v2, v3); //C++: v1 = v2 + v3;<br>
<br>
Now, GenericVectorAdd() can dispatch into the respective memory region (maybe using a delegator here):<br>
<br>
switch(v1.my_handle.active_<u></u>handle){<br>
case MAIN_MEMORY: perform_add_in_main_memory(); break;<br>
case OPENCL_MEMORY: perform_add_in_opencl_memory()<u></u>; break;<br>
case CUDA_MEMORY: perform_add_in_cuda_memory(); break;<br>
default: error_handling(); break;<br>
}<br>
<br>
For the cases where multiple handles hold current data, appropriate priorizations can be applied. Also, for each memory region one can further dispatch into other libraries:<br>
<br>
void perform_add_in_main_memory(...<u></u>) {<br>
if (with_thread_pool) { ... }<br>
if (with_blas_lib1) { ... }<br>
...<br>
}<br>
<br>
and similarly for OpenCL and CUDA. Perhaps most important for PETSc, bindings to CUSparse can be applied in a straightforward manner without duplicating code from the CUSP bindings if memory handles are not entirely hidden in each of the libraries.<br>
<br>
----------<br>
<br>
And now for some benchmarks I've run within the multi-backend framework using double precision. An AMD Radeon HD 7970 is now operational, competing with an NVIDIA GTX 285 and an Intel Xeon X5550 using a single thread. The NVIDIA GPU is run both with OpenCL and CUDA, leading to quite interesting differences.<br>
<br>
The first benchmark (vector.png) is a simple vector addition of the form x = ay + bz for various vector sizes. The AMD Radeon is expected to outperform NVIDIA GPUs for large problem sizes due to higher memory bandwidth (264 GB/sec vs. 159 GB/sec). However, it suffers from large overhead at smaller vector sizes (the kink at about 10k is due to the higher number of repetitions at smaller sizes). CUDA code at small vector sizes is about a factor 2 faster than an OpenCL implementation on the same hardware. The CPU is the best choice for anything below ~10k entries.<br>
<br>
Next, sparse matrix-vector multiplications are compared in spmv.png. The matrix is obtained from a finite difference discretization of the unit square in 2d, and the unit cube in 3d, respectively, using lexicographical ordering (no bandwidth reductions such as Cuthill-McKee). Timings for the sparse formats (CSR, vectorized CSR, COO, ELL, HYB) were recorded and the fastest among these considered in the graph for each case. The AMD GPU ultimately gives the best performance at large problem sizes, but shows some weird performance characteristics in 2d. CUDA again has a lower overhead at smaller problem sizes, but otherwise shows the same performance as OpenCL. The 2d-OpenCL performance on the NVIDIA-GPU is qualitatively the same as in 3d, I just forgot to transfer the data from the lab. The CPU implementation suffers from heavy cache misses in 3d at larger problem sizes.<br>
<br>
Finally, something more practical, yet still a bit synthetic: Timings for 50 CG iterations with standard BLAS kernels (no fusing of vector operations). Surprisingly, the overhead of OpenCL on the AMD GPU now even becomes an order of magnitude at small problem sizes. This is mostly due to the transfer of vector norms for error checking, so the AMD driver seems to have quite some problems with that. Also, the difference between OpenCL and CUDA is now more pronounced at smaller systems, even though asymptotically the performance is the same. The CUDA implementation failed at large problem sizes, I still need to investigate that. Still, the trend of matching with OpenCL at larger problem sizes is readily visible.<br>
<br>
That's it for now, after some more refining I'll start with a careful migration of the code/concepts into PETSc. Comments are, of course, always welcome.<br>
<br>
Best regards,<br>
Karli<br>
<br>
<br>
PS: Am I the only one who finds the explicit requirement of nvcc for CUDA-code awful? Sure, it is convenient to write CUDA device code inline, but I wasn't able to get around nvcc by using precompiled PTX code with any reasonable effort. Is there anybody with more luck/skill on petsc-dev?<br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div>