[petsc-dev] GPU based preconditioner and PCSHELL

[Dave Nystrom]

I have a 2d resistive mhd code interfaced to petsc.  The code has seven
different linear solves per timestep and these linear solves consume around
95 percent of the run time for a reasonably small grid of 100x301.  The run
time is dominated by the solution of one of the linear solves that is
particularly difficult to solve.  This particular linear solve actually takes
about 80 percent of the total run time.  All of these linear systems are
symmetric.  I am actually able to run a simulation where I am using jacobi
preconditioning with conjugate gradient on a gpu and get a solution and this
is the fastest solution which I currently get but the iteration count ranges
from a minimum of 771 to a max of 47300 for the difficult linear system.

We also have a native cg solver in the code which uses a full cholesky band
solve of the inner set of bands and that solver has a much better iteration
count although it takes a bit over 2x the run time of petsc w/ jacobi and
cusp.  So, I have been interested in making a custom petsc preconditioner
that does this cholesky solve using the PCSHELL capability of petsc.  And I'm
interested in trying to do this cholesky solve on the GPU using a CULA SPARSE
band solver.  However, I'm wondering if the petsc PCSHELL capability only
runs on the cpu and if I would need a GPU analog such as a PCGPUSHELL
capability in petsc.  I'm wondering if this is the case and if so, whether
there is a possibility that a PCGPUSHELL capability might be added to petsc
in the near term.

Thanks,

Dave

-- 
Dave Nystrom

phone: 505-661-9943 (home office)
       505-662-6893 (home)
skype: dave.nystrom76
email: dnystrom1 at comcast.net
smail: 219 Loma del Escolar
       Los Alamos, NM 87544