<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I'll write up something about the implementation this weekend and
pass it on. The key is that everything is done asynchronously and
that communication costs are pretty well hidden for large matrices.
The code is in our add on package, txpetscgpu. If many people are
working on this code, it may be worthwhile to move this code into
the main PETSc source.<br>
<br>
The example I've been using to test is :
src/ksp/ksp/examples/tutorials/ex10.c<br>
<br>
The matrices that I get <br>
1) 40+ GFlops is af_shell4 from the UFL collection.<br>
2) cfd.2.10 gets around 18 GFlops on 4 GPUs in CSR format. It gets
22 GFlops on 2 GPUs in ELL format.<br>
<br>
My system has 4 Nvidia 2070s attached to a single node. There are 2
nodes.<br>
<br>
-Paul<br>
<blockquote
cite="mid:CAMYG4GmxSa9qWti3RAW_0SBVZg6AK7MyuKLHPYmhVYLaq-3htw@mail.gmail.com"
type="cite"><br>
<div class="gmail_quote">On Fri, Feb 10, 2012 at 10:11 AM, Paul
Mullowney <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:paulm@txcorp.com">paulm@txcorp.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
Congrats! Given that I have a DoE phase II to continue Petsc
GPU development, we should probably all get on the same page.<br>
<br>
If you're looking to do multi-GPU computing, I've already
effectively redesigned the MatMult to get very good strong
scaling and some excellent performance.<br>
I have some matrices getting 40+ GFlops on 4 GPUs. 20 GFlops
on 2 GPUs, ...</blockquote>
<div><br>
</div>
<div>Great. Can you check in a benchmark code to src/benchmarks
so that I can run it and reproduce these results? It would</div>
<div>be great to start this sort of thing to make papers easier.
I can help with any problems doing that.</div>
<div><br>
</div>
<div>I will look over the code so I get a better idea what is
happening.</div>
<div><br>
</div>
<div>
Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;"><span class="HOEnZb"><font color="#888888"><br>
-Paul</font></span>
<div class="HOEnZb">
<div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
Pramod (who did the work) was my student at EPCC last
year. We looked at porting Fluidity (or bits of it) to
GPU and as a result decided that working on PETSc's GPU
support was going to be the most promising angle for the
time we had available. We have never got round to
benchmarking Fluidity with Pramod's extended sparse
matrix format support, but it shouldn't be too much
effort.<br>
<br>
If you have any questions about it all, feel free to
ask.<br>
<br>
cheers,<br>
Michele<br>
On 10 Feb 2012, at 15:57, Matthew Knepley wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt
0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
Has anyone taken a look at this:<br>
<br>
<a moz-do-not-send="true"
href="http://www.epcc.ed.ac.uk/wp-content/uploads/2011/11/PramodKumbhar.pdf"
target="_blank">http://www.epcc.ed.ac.uk/wp-content/uploads/2011/11/PramodKumbhar.pdf</a><br>
<br>
It is the Fluidity people I think. I have just gotten
an NSF award with Dan Negrut<br>
in Wisconsin and Ahmed Sameh at Purdue to port Ahmed's
SPIKE preconditioner<br>
(you may have heard Olaf Schenk talk about this) to
PETSc, and in particular to<br>
PETSc's GPU backend so that Dan can run it on his GPU
cluster. Thus, we will<br>
be seriously stressing this feature in the near
future.<br>
<br>
Matt<br>
<br>
-- <br>
What most experimenters take for granted before they
begin their experiments is infinitely more interesting
than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</blockquote>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to
which their experiments lead.<br>
-- Norbert Wiener<br>
</blockquote>
<br>
</body>
</html>