[petsc-dev] Backend independent VecGetArray for GPUs

[Barry Smith]

> On Oct 17, 2014, at 4:13 PM, Dominic Meiser <dmeiser at txcorp.com> wrote:
> 
> On 10/17/2014 02:42 PM, Ashwin Srinath wrote:
>> > Why would we want this? The packages themselves (CUDA/ViennaCL) only expose
>> memory using these specific types. What use is it to wrap these up in a void * if you
>> just have to caste back down to use them. Isn't it better to maintain type-specific, and
>> type safe, interfaces for this stuff?
>> 
>> The biggest problem I faced was that there was no way to access device memory using petsc4py - since there is no equivalent for VecCUSPGetArray. So returning a raw pointer may not be very helpful for C++/CUSP users (they already have a nice way to access device memory) but it would definitely make things a lot easier for Python users.
> 
> To me it sounds like something that should be dealt with in the library that does the python bindings,

   Yup. That library is petsc4py :-)  We want to try to have as much as possible the same API in petsc4py and PETSc so the question is if we provide the API all the way in PETSc or just petsc4py. For consistency across libraries it might belong in PETSc, plus it might actually be useful directly in C/C++ so it may belong there.

   Barry


> not in PETSc itself. For example, it would be easy to write a wrapper functions like the following:
> 
> void *getDevicePtr(Vec v)
> {
>   void *ptr;
> 
>   /* Use VecCUSPGetArray* or VecViennaCLGetArray*  to obtain device ptr */
> 
>   return ptr;
> }
> 
> void releaseDevicePtr(Vec v, void *ptr)
> {
>   /* Vec type specific code to release the pointer */
> }
> 
> What do we gain by having these wrappers in PETSc rather than in a separate library?
> 
> Cheers,
> Dominic
> 
>> 
>> Thanks!
>> 
>> On Fri, Oct 17, 2014 at 12:09 PM, Dominic Meiser <dmeiser at txcorp.com> wrote:
>> On 10/17/2014 09:45 AM, Lisandro Dalcin wrote:
>> On 17 October 2014 17:54, Dominic Meiser <dmeiser at txcorp.com> wrote:
>> Hi Ashwin,
>> 
>> Are you suggesting that `VecGetGPUArray*`  be added to the `Vec` interface?
>> That might be problematic because these methods only makes sense for GPU
>> vectors. The interface of `Vec` would then be tied to the PETSc
>> configuration.
>> 
>> These methods could be always available simply generate an error for
>> non-GPU vector types.
>> That's a possible approach. Barry, Jed, what's the preferred way of doing this? Make base class interface wider or subtyping via compose and query?
>> 
>> An alternative might be to compose the various `VecCUSPGetArray` and related
>> methods with `Vec` objects. You could then query these methods and use them
>> if available (and handle absence of the methods if needed, e.g. throw an
>> exception). This would be easy to add.
>> 
>> Yes, however note that what we really need is a backend-independent
>> way to get the RAW pointer to the GPU buffers. Right now we have calls
>> to get the CUSP or ViennaCL array, to handle them you need C++ and
>> depend on CUDA/OpenCL at compile-time.
>> 
>> 
>> In what sense should it be backend-independent? The method name should be independent of the backend? Or the presence of the method should be backend independent?
>> 
>> I don't see what your gaining by having the methods in Vec. You still need to handle the error that occurs if you're dealing with a non-GPU vector and you still have to know the vector type in order to interpret the return value (device pointer vs OpenCL buffer).
>> 
>> Dominic
>> 
>> 
>> -- 
>> Dominic Meiser
>> Tech-X Corporation
>> 5621 Arapahoe Avenue
>> Boulder, CO 80303
>> USA
>> Telephone: 303-996-2036
>> Fax: 303-448-7756
>> www.txcorp.com
>> 
>> 
> 
> 
> -- 
> Dominic Meiser
> Tech-X Corporation
> 5621 Arapahoe Avenue
> Boulder, CO 80303
> USA
> Telephone: 303-996-2036
> Fax: 303-448-7756
> 
> www.txcorp.com