<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 14, 2017 at 3:29 PM, Karl Rupp <span dir="ltr"><<a href="mailto:rupp@iue.tuwien.ac.at" target="_blank">rupp@iue.tuwien.ac.at</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Mark,<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
A lot of this is driven by desires of DOE programs -- not your monkey not your circus -- but I think that we need to have a story for how to use GPUs, or whatever apps in our funding community want to do, and tell it dispassionately. We don't commit resources nor commit to design changes but just say this how one could do it.<br>
</blockquote>
<br></span>
it will nonetheless require a lot of convincing that at best they get moderate speed-ups, not the 580+x claimed in some of those early GPU papers...<span class=""><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The fusion folks that I work with, and I assume other DOE offices, are just looking at their codes, subroutine by subroutine, and having postdocs look at GPUising them. We just need intelligent answers to their questions. Even if we as sentient and passionate human being have opinions on the approach that is implied by their questions, it is part of my job to just give them a professional answer.<br>
</blockquote>
<br></span>
In the past ~18 months I've worked with applications that wanted to use GPUs in just that manner. Needless to say that you end up with touching almost everything to actually beat an existing (efficient) CPU-based application by less than a factor of 2. This involves MPI-parallel applications; it's much easier to get higher speedups if you don't need to communicate across ranks.<span class=""><br></span></blockquote><div><br></div><div>The viewpoint I'm currently working from is as follow. As Karl says, there is a fairly modest upper bound on the best you can hope to beat an efficient CPU implementation by (for our applications, which are trying to speed up MPI-parallelized applications of sparse operators). That being said, there are available machines which have GPUs, and there is some performance (maybe even energy efficiency) to be gained if the software sufficiently reduces the amount of implementation time, so the key is to be able to do it either<br></div><div>i) (ideally) from the command line. This should be possible in some cases, like the one Matt mentions: AIJ ViennaCL matrices, Chebyshev-Jacobi smoothing. <br></div><div>ii) by providing your own optimized matrix-free operator which "drops in" easily enough. This is something that we're working toward atm!<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I have enough now (thanks Jed and Lorena, et al!) to answer the AMGx question sufficiently, and if you could give me a quick assessment of where we are with hypre's GPU solver that would be great.<br>
</blockquote>
<br></span>
does "work in progress" suffice? ;-)<br>
<br>
Best regards,<br>
Karli<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
On Thu, Jul 13, 2017 at 11:16 PM, Karl Rupp <<a href="mailto:rupp@iue.tuwien.ac.at" target="_blank">rupp@iue.tuwien.ac.at</a> <mailto:<a href="mailto:rupp@iue.tuwien.ac.at" target="_blank">rupp@iue.tuwien.ac.at</a>><wbr>> wrote:<br>
<br>
Hi Mark,<br>
<br>
I hear Hypre has support for GPUs in a May release. Any word on<br>
the status of using it in PETSc?<br>
<br>
<br>
as far as I know, it is currently not supported in PETSc. I'll have<br>
a look at it and see what needs to be done to enable it.<br>
<br>
<br>
And we discussed interfacing to AMGx, which is complicated<br>
(precluded?) by not releasing source. Anything on the potential<br>
of interfacing to AMGx? I think it would be great to make this<br>
available. It is on a lot of checkboxes. I would love to be able<br>
to say, yea you can use it.<br>
<br>
<br>
Lorena Barba's group actually interfaced PETSc to AMGx at some point<br>
(presented at GTC 2016 if I'm not mistaken). I'll reach out to them,<br>
maybe they have something to contribute.<br>
<br>
Best regards,<br>
Karli<br>
<br>
<br>
</blockquote>
</div></div></blockquote></div><br></div></div>