[petsc-dev] refactoring petsccusp.h needed

[Barry Smith]

On Mar 8, 2013, at 3:48 PM, Paul Mullowney <paulm at txcorp.com> wrote:

> Yes. I can work on this.
>>   Paul,
>> 
>>     We are thinking of a PETSc release around March 21. Before that time petsccusp.h
>> 
>> #if !defined(__PETSCCUSP_H)
>> #define __PETSCCUSP_H
>> /*
>>     This should only be included in user code that uses CUSP directly and hence the file name ends with .cu
>> */
>> #include<../src/vec/vec/impls/dvecimpl.h>
>> #include<../src/vec/vec/impls/seq/seqcusp/cuspvecimpl.h>
>> #endif
>> 
>> so that ONLY the public interface of the stuff (what needs to be known to user code) is available and the rest is kept in the cuspvecimpl.h file. Maybe pets ccu
>> sp.h could be empty, I don't know).
> Wish list:
> 1) I would like a public method that exposes the GPU device pointers to the user. For example, we have VecGetArrayRead(). But, this gives a CPU pointer (which in turn causes a memcpy from CPU to GPU).  We need something that returns the GPU device pointer. This is useful for embedded PETSc solves in external apps that want to use the GPU.

  Yup. Are these not just 
#undef __FUNCT__
#define __FUNCT__ "VecCUSPGetArrayReadWrite"
PETSC_STATIC_INLINE PetscErrorCode VecCUSPGetArrayReadWrite(Vec v, CUSPARRAY **a)
{
  PetscErrorCode ierr;

  PetscFunctionBegin;
  *a   = 0;
  ierr = VecCUSPCopyToGPU(v);CHKERRQ(ierr);
  *a   = ((Vec_CUSP*)v->spptr)->GPUarray;
  PetscFunctionReturn(0);
}

and friends? Just make them not static inline and put the prototypes for them in petsccusp.h  The function call cost can be lived with.

(You would make them not static inline because we don't want Vec_CUSP to be public.)


Thanks 

    Barry

   Barry

> 
> 2) As for other things. I have a patch for ICC that I'm nearly finished with. After that, I'm going to try to harden the other CUSP preconditioners so that they work with the CUSP and CUSPARSE classes.
> 
> -Paul
>>    Do you think you can make that change and anything else that needs doing before the release?
>> 
>>    Thanks
>>     Barry
>> 
>