[petsc-dev] Dev release GPU solvers

[Matthew Knepley]

On Tue, Sep 6, 2011 at 11:25 AM, Randall Mackie <rlmackie862 at gmail.com>wrote:

>
> On Sep 6, 2011, at 9:16 AM, Matthew Knepley wrote:
>
> On Tue, Sep 6, 2011 at 11:12 AM, Randall Mackie <rlmackie862 at gmail.com>wrote:
>
>>
>> On Sep 6, 2011, at 9:08 AM, Barry Smith wrote:
>>
>> >
>> >
>> > On Sep 6, 2011, at 10:59 AM, Daniel Lowell wrote:
>> >
>> >> On this page: http://www.mcs.anl.gov/petsc/petsc-as/features/gpus.html
>> >>
>> >> It states: "All of the Krylov methods except KSPIBCGS run on the GPU."
>> >> I actually could only find ksp/pc/bicgstabcusp for any ksp
>> implementation on the GPU.
>> >
>> >  Daniel,
>> >
>> >     All the vector operations and the matrix-vector product used by the
>> Krylov methods implemented in PETSc (except KSPIBCGS) run on the GPU without
>> copying up or down vector or matrix values, in the same way as
>> ksp/pc/bicgstabcusp does, hence we say they run on the GPU. In fact, in our
>> tests our BiGCStab runs just as fast as the Nvidia CUSP code
>> ksp/pc/bicgstabcusp.
>> >
>> >    In other words, higher level algorithms such as Krylov methods and
>> Newton methods do not need to be "written for the GPU", so long as the
>> kernel operations run on the GPU.
>> >
>> >
>> >
>> >   Barry
>>
>> Barry,
>>
>> But is this still only for real data types? Last time I checked, earlier
>> this year, complex valued data were not
>> supported under PETSc, even though complex data types are supported in
>> CUSP.
>>
>
> I have not tried, but I do not understand why it would not work. We just
> use PetscScalar. Can you send
> the problem you found?
>
>    Matt
>
>
>
> Matt,
>
> Here is the message that I received earlier this year, but I never did have
> time to follow up on it then:
>

Ah, so I would characterize this as "CUSP does not work with std::complex",
which is a big problem.

    Matt


>
> *From: *Barry Smith <bsmith at mcs.anl.gov>
> *Date: *May 24, 2011 5:04:51 AM PDT
> *To: *petsc-maint at mcs.anl.gov, Victor Minden <victorminden at gmail.com>
> *Cc: *Randall Mackie <rlmackie862 at gmail.com>
> *Subject: **Re: [petsc-maint #71751] error during compile*
>
>
>  Victor,
>
>     Thanks for the info. I've forgotten about the blas.
>
>  Randall,
>
>      Looks like it is not so trivial as I had made it out to be. Perhaps if
> you ask on petsc-dev at mcs.anl.gov there may be people who want this also
> and are willing to share the work?
>
>   Barry
>
> On May 23, 2011, at 6:06 PM, Victor Minden wrote:
>
> Barry,
>
>
> Currently there should be two things going on--one is that some of the
>
> CUBLAS routines used for basic vector operations (which we've been moving
>
> away from) have 4 different forms depending on single/double and
>
> real/complex--currently those that we still use are hard-coded as real and
>
> #ifdeffed to be single/double, and I haven't looked in depth but I think
>
> those are the errors that Randy sent.
>
>
> Those should be pretty quick to change, but they won't make complex work,
>
> because once you get past veccusp.cu there are a bunch more errors in the
>
> cusp matrix routines which work with complex if you use the relatively new
>
> (Oct?  Dec?) cusp complex type that they've added to the cusp library.  So,
>
> we need to make the GPU routines work with the standard complex type or we
>
> need to convert all the GPU code to use the cusp type.  I don't know how
>
> much work this is.
>
>
> As it stands, I'm just starting my internship this week so I probably won't
>
> be able to look at this in-depth until the weekend, just FYI.
>
>
> Cheers,
>
>
> Victor
>
> ---
>
> Victor L. Minden
>
>
> Tufts University
>
> School of Engineering
>
> Class of 2012
>
>
>
>
>
>
>
>> Randy
>>
>>
>> >
>> >>
>> >> CUSP does have everything mentioned in the above PETSc link:
>> >>
>> http://code.google.com/p/cusp-library/wiki/QuickStartGuide#Iterative_Solvers
>> >>
>> >> However I don't see where any of it is implemented in the latest Dev
>> release, Krylov solvers or preconditioners.
>> >>
>> >> Can anyone clear this up for me?
>> >>
>> >> Thanks,
>> >>
>> >> Daniel Lowell
>> >
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>


-- 
What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20110906/e4d4b895/attachment.html>