[petsc-dev] SNES ex19 not using GPU despite passing the options

[Mani Chandra]

Hi Karli,

Thanks for the reply. That fixed it. I get only a 10% speed up using the
cusp options. Is the residual evaluation at each iteration happening on the
CPU or the GPU? Is there anyway one can do the residual evaluation on the
GPU too, after the data has been transferred? Ex42 shows how it can be done
using cusp but it looks really ugly and I want to use OpenCL. Basically can
I do something like this?

DMGetLocalVector(da, &localX); //Vector is now in GPU.
DMDAVecGetArray(da, localX, &x); //Array is on GPU.

//Create buffers for OpenCL
buffer = cl::Buffer(context, CL_MEM_USE_HOST_PTR |
                                                CL_MEM_READ_WRITE,
                                  sizeofarray, &x[X2Start-Ng][X1Start-Ng]
                                   , &clErr);

(I'm hoping that here CL_MEM_USE_HOST_PTR will give a pointer to the data
already on the GPU)

// Launch OpenCL kernels and now map the buffers to read off the data.

DMDAVecRestoreArray(da, localX, &x);
DMRestoreLocalVector(da, &localX);

I think the question is whether DMDAVecGetArray will return a pointer to
the data on the GPU or not.

Cheers,
Mani


On Tue, Jan 14, 2014 at 3:34 AM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Hi Mani,
>
> the options are misspelled, you want to use
> -dm_vec_type cusp -dm_mat_type aijcusp
> (you can omit the 'mpi' to support both the MPI case and the sequential
> case).
>
> I also suggest doing all development in debug mode, in which case you
> would have received the following hints:
>
> WARNING! There are options you set that were not used!
> WARNING! could be spelling mistake, etc!
> Option left: name:-da_mat_type value: mpiaijcusp
> Option left: name:-da_vec_type value: mpicusp
> Option left: name:-dmmg_nlevels value: 1
> Option left: name:-preload value: off
>
> Best regards,
> Karli
>
>
>
>
> On 01/14/2014 04:28 AM, Mani Chandra wrote:
>
>> Hi,
>>
>> I tried to run SNES ex19 with the following options but it does not seem
>> to use my GPU. See the log summary attached. Am I interpretting the log
>> summary wrong? I don't see any CUSP calls to copy data from the CPU to
>> the GPU.
>>
>> /ex19 -da_vec_type mpicusp -da_mat_type mpiaijcusp -pc_type none
>> -dmmg_nlevels 1 -da_grid_x 300 -da_grid_y 300 -log_summary -mat_no_inode
>> -preload off  -cusp_synchronize -cuda_s
>> et_device 0
>>
>> I get the following output when I do -cuda_show_devices
>> CUDA device 0: Quadro FX 1800M
>>
>> Cheers,
>> Mani
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140114/5d9e71e4/attachment.html>