[petsc-dev] A more positive thread

[Paul Mullowney]

I'll write up something about the implementation this weekend and pass 
it on. The key is that everything is done asynchronously and that 
communication costs are pretty well hidden for large matrices. The code 
is in our add on package, txpetscgpu. If many people are working on this 
code, it may be worthwhile to move this code into the main PETSc source.

The example I've been using to test is : 
src/ksp/ksp/examples/tutorials/ex10.c

The matrices that I get
1) 40+ GFlops is af_shell4 from the UFL collection.
2) cfd.2.10 gets around 18 GFlops on 4 GPUs in CSR format. It gets 22 
GFlops on 2 GPUs in ELL format.

My system has 4 Nvidia 2070s attached to a single node. There are 2 nodes.

-Paul
>
> On Fri, Feb 10, 2012 at 10:11 AM, Paul Mullowney <paulm at txcorp.com 
> <mailto:paulm at txcorp.com>> wrote:
>
>     Congrats! Given that I have a DoE phase II to continue Petsc GPU
>     development, we should probably all get on the same page.
>
>     If you're looking to do multi-GPU computing, I've already
>     effectively redesigned the MatMult to get very good strong scaling
>     and some excellent performance.
>     I have some matrices getting 40+ GFlops on 4 GPUs. 20 GFlops on 2
>     GPUs, ...
>
>
> Great. Can you check in a benchmark code to src/benchmarks so that I 
> can run it and reproduce these results? It would
> be great to start this sort of thing to make papers easier. I can help 
> with any problems doing that.
>
> I will look over the code so I get a better idea what is happening.
>
>   Thanks,
>
>      Matt
>
>
>     -Paul
>
>
>         Pramod (who did the work) was my student at EPCC last year. We
>         looked at porting Fluidity (or bits of it) to GPU and as a
>         result decided that working on PETSc's GPU support was going
>         to be the most promising angle for the time we had available.
>         We have never got round to benchmarking Fluidity with Pramod's
>         extended sparse matrix format support, but it shouldn't be too
>         much effort.
>
>         If you have any questions about it all, feel free to ask.
>
>         cheers,
>         Michele
>         On 10 Feb 2012, at 15:57, Matthew Knepley wrote:
>
>             Has anyone taken a look at this:
>
>             http://www.epcc.ed.ac.uk/wp-content/uploads/2011/11/PramodKumbhar.pdf
>
>             It is the Fluidity people I think. I have just gotten an
>             NSF award with Dan Negrut
>             in Wisconsin and Ahmed Sameh at Purdue to port Ahmed's
>             SPIKE preconditioner
>             (you may have heard Olaf Schenk talk about this) to PETSc,
>             and in particular to
>             PETSc's GPU backend so that Dan can run it on his GPU
>             cluster. Thus, we will
>             be seriously stressing this feature in the near future.
>
>                 Matt
>
>             -- 
>             What most experimenters take for granted before they begin
>             their experiments is infinitely more interesting than any
>             results to which their experiments lead.
>             -- Norbert Wiener
>
>
>
>
>
> -- 
> What most experimenters take for granted before they begin their 
> experiments is infinitely more interesting than any results to which 
> their experiments lead.
> -- Norbert Wiener

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120210/9d6375fa/attachment.html>