[petsc-dev] PETSc GPU capabilities

[Paul Mullowney]

One other point, my code supports
(1) CUDA 4.1, Thrust 1.5, CUSP 0.3
or
(2) CUDA 4.0, Thurst 1.4, CUSP 0.2

While it may work with mercurial versions of Thrust and CUSP, I make no 
guarantees.

-Paul
> Hi All,
>
> I've been developing GPU capabilities for PETSc. The development has 
> focused mostly on
> (1) An efficient multi-GPU SpMV, i.e. MatMult. This is working well.
> (2) Triangular Solve used in ILU preconditioners; i.e. MatSolve. The 
> performance of this ... is what it is :|
> This code is in beta mode. Keep that in mind, if you decide to use it. 
> It supports single and double precision, real numbers only! Complex 
> will be supported at some point in the future, but not any time soon.
>
> To build with these capabilities, add the following to your configure 
> line.
> --download-txpetscgpu=yes
>
> The capabilities of the SpMV code are accessed with the following 2 
> command line flags
> -cusp_storage_format csr (other options are coo (coordinate), ell 
> (ellpack), dia (diagonal). hyb (hybrid) is not yet supported)
> -use_cusparse (this is a boolean and at the moment is only supported 
> with csr format matrices. In the future, cusparse will work with ell, 
> coo, and hyb formats).
>
> Regarding the number of GPUs to run on:
> Imagine a system with P nodes, N cores per node, and M GPUs per node. 
> Then, to use only the GPUs, I would run with M ranks per node over P 
> nodes.  As an example, I have a system with 2 nodes. Each node has 8 
> cores, and 4 GPUs attached to each node (P=2, N=8, M=4). In a PBS 
> queue script, one would use 2 nodes at 4 processors per node. Each mpi 
> rank (CPU processor) will be attached to a GPU.
>
> You do not need to explicitly manage the GPUs, apart from 
> understanding what type of system you are running on. To learn how 
> many devices are available per node, use the command line flag:
> -cuda_show_devices
>
> -Paul