[petsc-dev] Generality of VecScatter

[Dmitry Karpeev]

On Wed, Nov 23, 2011 at 6:28 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> I had a useful conversation with the MPICH guys today about the
> unstructured communication primitives that Mark, Matt, Barry, and I have
> been discussing. It's looking like we can do a very thin communication
> library that has the nice semantics I was looking for, namely that the
> local-to-global is stored only as indices in the local space. To implement
> the communication primitives, the owner of a global node never needs to
> know how many or which processes have that point in their local space.
> There is an MPI-3 feature that we can use to elegantly avoid two-way
> knowledge for pointwise gather/scatter. (We can always implement the same
> interface using vanilla MPI-1, but the MPI-2/3 implementation would have
> less overhead.)
>
> If I/we write this layer, VecScatter (with arbitrary data types, etc)
> could be implemented very easily on top of it. I'm in no rush to do this,
> but I could imagine getting there eventually. The VecScatter interface
> allows some things that might not be important. For example, any process
> can specify an edge in the communication graph, even if it does not own the
> source or the destination vertex. Also, both source and destination
> vertices can have degree higher than one. We can even put a link between
> the same points twice (effectively multiplying that contribution by 2 when
> using ADD_VALUES). I don't know if this ever makes semantic sense, but the
> interface allows it.
>
> I wonder if we can express all useful communication with a more
> restrictive interface similar to a local-to-global map (but mapping between
> any vectors) where one side (typically called "local", but doesn't need to
> actually be a local Vec) has at most one edge.
>
What does this mean: "where one side has at most one edge"?

>
> What weird things do people use VecScatter for that might break with this
> more restrictive model?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20111123/42876ebe/attachment.html>