[petsc-dev] GAMG error with MKL

[Dave May]

On 10 July 2018 at 13:06, Mark Adams <mfadams at lbl.gov> wrote:

>
> If PETSc was an application, it could do whatever it wanted, but it's
>> not.  If PETSc is a library that intends to meet the needs of HPC
>> applications, it needs to support the programming models the applications
>> are using.
>>
>
> To repeat, PETSc supports threads, currently with MKL kernels and third
> party solvers (hypre), and GPUs with native solvers and again through
> hypre. I'm sure we will add more mechanism in the future. (I really don't
> know how to say this any more clearly.)
>

And let's not forgot the "E" for extensible within PETSc.

Despite the notion held by many in the committee that "PETSc does not
support threads" there is absolutely nothing which prohibits a
knowledgeable user from using threads (or any other form of "X") within
their own custom implementation of KSP, PC, Mat, Vec etc which then
naturally then plugs into the rest of the PETSc ecosystem. In fact, it's
often far simpler to incorporate "X" via a custom implementation since the
knowledgeable user is responsible for compiling their custom thing and does
not have to make it natively work with PETSc's configure.


>
>> Or I suppose you will continue to disparage everyone who doesn't bow down
>> and worship flat MPI on homogeneous big-core machines as a divine execution
>> model until your users abandon you for otherwise inferior software that is
>> willing to embrace user requirements.
>>
>
> I agree, and I suspect we have a consensus that we should not disparage
> threads publicly as much (or at all) in the future as we have.
>

This would be a great thing - and I'm happy to hear Barry will accept such
branches.

I have been in the situation where a project in review proposing to do
things with GPUs and OpenMP within PETSc was almost killed because the
reviewers truly believed that "PETSc does not support threads". The
consensus held was that the project would only be possible if we used
Trilinos. I don't know how many times the reviewer wrote "PETSc does not
support threads". The reviewer did not understand what, or how, the "E" in
PETSc can be easily exploited to incorporate accelerators "within" the
PETSc ecosystem. It was explained in the project proposal, but the strong
(and loud) stance adopted regarding PETSc and threads prevented them from
even thinking about the proposal content. A reviewer should not even have
to understand how the extensibility works - but it would be great if in
future, reviewers (I'm assuming they are representative of the larger
community) simply didn't reject projects out of hand because they used the
words "PETSc" and "threads" in the same sentence.


> It just does not do any good. Those that get it get it and those that
> don't don't.
>

Exactly. And sometimes those that don't get it are reviewing our projects.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20180710/737fd572/attachment.html>