[petsc-dev] code review request : txpetscgpu package removal

[Karl Rupp]

Hi Paul,

 > I've been refactoring some pieces of PETSc to remove the txpetscgpu
> dependence by bringing that code directly into petsc.  All of the
> capability in that package only depends on CUDA (and thrust), thus there
> is no need for an external package. I think removing this package is an
> improvement to the PETSc GPU capabilities as all the code is now easily
> visible (i.e. not hidden in some package).

Thanks a lot! Really. I merged the ViennaCL-stuff into master a little 
less than two weeks back and now I'm just about to start my big 
PETSc-GPU cleanup/unification/documentation project. I would have 
addressed the txpetscgpu package sooner or later, so I'm really happy to 
see that you are taking care of this.


> The new code is in the fork:
> pmullowney/txpetscgpu-package-removal.git
>
> This branch builds and run on a machine with CUDA/Thrust/CUSP (i.e. no
> need for --download-txpetscgpu). However, txpetscgpu has not been
> entirely removed yet as there are still some dependencies in the aijcusp
> matrix and cusp preconditioner classes.

How 'heavy' is this dependency? Is there some 'blocker' which prevents a 
complete integration, or is it just not finished yet?


> I would like to start a discussion on the changes I have made which
> primarily affect aijcusparse, mpiaijcusparse, and veccusp. There are two
> commits that need to be reviewed:


Alright, I'll comment there.


> I think (1) should be reviewed first as this adds most of the serial GPU
> capability to aijcusparse. The second commit (2) adds changes to veccusp
> and mpiaijcusparse to get an efficient multi-GPU SpMV.

I assume that this only works on a single node in order to enumerate and 
initialize the GPUs correctly?

Best regards,
Karli