[petsc-dev] Generality of VecScatter

[Barry Smith]

On Nov 24, 2011, at 2:30 PM, Jed Brown wrote:

> On Thu, Nov 24, 2011 at 14:17, Barry Smith <bsmith at mcs.anl.gov> wrote:
>     Come on, stop being silly; it has nothing to do with the fact that it is MPI+something.  It is because it is too complicated and hard to use and both MPI and pthreads have bad limitations (for pthreads it is the fact that pthreads does not have a good way of waking up many pthreads together to do something, for MPI it is due to point to point and one sided do not allow proper scheduling of communication at the low level).
> 
> This is why they added neighborhood collectives.
>  
> 
> > What about a library that was implemented in terms of MPI and pthreads, but gave you a more consistent abstraction?
> 
>   That would be fine, even great. I just don't believe that either of them are the right base to build on.
> 
> I think of MPI as an abstraction for the network and pthreads as an abstraction for kernel management of threads. If you don't want to build on them, we need to justify why they are bad abstractions (and have different network or kernel-level implementations). The abstraction being good has nothing to do with whether it has a nice API for users, it's whether it provides a suitable model for the lower level (network hardware or kernel) upon which useful abstractions can be written.

   I totally agree with you on this.  Where we disagree is thatI don't believe that MPI or pthreads have what is needed. For example since pthreads sequentializes the waking up threads it cannot be used in a way scalable to hundreds of threads. I'fe already explained my concerns about MPI. 

>  
>   You sound a bit like Jack telling me in 1994 that I should write PETSc on top of Scalapack.
> 
> Or maybe like Bill suggesting that you use C and make it run on operating systems that exist? ;-)

   Bill didn't suggest this, it was a given.

   Barry