[petsc-dev] IMP prototype

[Victor Eijkhout]

On Jan 2, 2014, at 10:12 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> the execution model is BSP

No it's not. There are no barriers or syncs.

> it's not clear what is gained by switching
> to callbacks.

The callbacks are there just to make the code uniform. I've edited the document to reflect that in the MPI case you can dispense with them.

> your transformation is recognizing a common
> pattern of communication into temporary buffers, followed by
> computation, followed by post-communication and putting a declarative
> syntax on it

Somewhat simplified, but not wrong. I'm kind of interested in the question what practically relevant algorithms do not conform to that model.

> (with callbacks for the computation)

They are incidental, at least in the MPI case. Ignore them.

> Your abstraction is not uniform
> if you need to index into owned parts of shared data structures or
> perform optimizations like cooperative prefetch.

Not sure what you're saying here. In case you dug into my code deeply, at the moment gathering the halo region involves one send-to-self that should be optimized away in the future. Look, it's a prototype, all right?

>  If you're going to use
> separate buffers, why is a system like MPI not sufficient?

1. No idea why you're so fixated on buffers. I've just been going for the simplest implementation that makes it work on this one example. It can all be optimized.
2. Why is MPI not sufficient: because you have to spell out too much. Calculating a halo in the case of a block-cyclically distributed vector is way too much work. An extension of IMP would make this very easy to specify, and all the hard work is done under the covers.

> What semantic does your abstraction provide for hybrid
> distributed/shared memory that imperative communication systems cannot?

How about having the exact same code that I've shown you, except that you specify that the processes are organized as N nodes times C cores? Right now I've not implemented hybrid programming, but it shouldn't be hard.

Victor.
-- 
Victor Eijkhout, 512 471 5809 (w)
Texas Advanced Computing Center, 
The University of Texas at Austin