[petsc-dev] Not possible to do a VecPlaceArray for veccusp

[Dominic Meiser]

On Fri, Feb 26, 2016 at 02:49:39PM +0100, Karl Rupp wrote:
> 
> >>The alternative would be to use raw cuda pointers instead of cusp
> >>arrays for GPU memory in VecCUSP.  That would be a fairly
> >>significant undertaking (certainly more than the 2-3 weeks Karli
> >>is estimating for getting the ViennaCL cuda backend in).
> >
> >Do you mean creating a new class VECCUDA in addition to VECCUSP and VECVIENNACL? This could be a solution for us. It would mean maybe refactoring MATAIJCUSPARSE to work with these new Vecs?
> 
> I prefer to replace VECCUSP with e.g. VECCUDA (and eventually also
> rename VECCUSPARSE to VECCUDA to have a unified naming for all the
> things provided natively with the CUDA SDK) in order to reduce
> external dependencies. CUSP will provide matrices, preconditioners,
> etc. as before, but is only optional and thus less likely to cause
> installation troubles. Supporting VECCUSP and VECCUDA next to each
> other is going to be too much code duplication without any benefit.

That makes sense.  At the Vec level we should be using a low
level construct (i.e. cuda raw pointers) because clients can
always provide raw pointers and they know how to consume them
(e.g. if they want to use cusp vectors on their end).

> 
> Even if we do provide VECCUDA, I still dislike the fact that we
> would have to maintain essentially the same code twice: One for
> CUDA, one for OpenCL/ViennaCL. With the ViennaCL bindings providing
> OpenMP and CUDA support soon, this also duplicates functionality. A
> possible 'fix' is to just use ViennaCL for CUDA+OpenCL+OpenMP and
> thus only maintain a single PETSc plugin for all three. However, I'm
> certainly too biased to be taken seriously here.

I agree with this in principle.  Perhaps it's time to consolidate
the cuda/cusp/cusparse/opencl efforts.  Note however that
MATAIJCUSPARSE provides capabilities that won't be available
right away with ViennaCL (e.g. multi-GPU block Jacobi and ASM
preconditioners).

Cheers,
Dominic


> 
> >
> > If there is interest we can help in adding this stuff.
> 
> What are your time constraints?
> 
> Best regards,
> Karli
> 
> 

-- 
Dominic Meiser
Tech-X Corporation - 5621 Arapahoe Avenue - Boulder, CO 80303