[petsc-dev] Implementing longer pipelines with VecDotBegin and VecDotEnd

[Barry Smith]

> On Mar 25, 2017, at 2:50 AM, Wim Vanroose <wim at vanroo.se> wrote:
> 
> Thanks for the feedback.  We will explore the different options and come back with an update.

   After all the sound and fury it looks like the best approach is that VecNorm/DotBegin/End be refactored to support multiple "independent phases". This will change the plumbing of VecNorm/DotBegin/End a bit but need not change the public API.

   I'm sorry but I don't have time to take on replumbing the VecNorm/DotBegin/End now and I doubt that Jed has so you would need to do that yourself (make it in its own pull request) and once you have the replumbing I think it should be straightforward for you to write your new pipelined methods. Are you able to consider doing the replumbing? We, of course, can answer questions that come up in the implementation.

   Barry

> 
> Wim
> 
> On Fri, Mar 24, 2017 at 5:45 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
> > On Mar 23, 2017, at 11:23 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> >
> > Barry Smith <bsmith at mcs.anl.gov> writes:
> >
> >>   Jed,
> >>
> >>      Ok I buy what you are saying as a possible solution but now I am
> >>      wondering why you need the request argument. Instead, as soon as
> >>      you hit the first End or PetscCommSplitReductionBegin why don't
> >>      you start a new batch (internally) that all new Begin() calls
> >>      feed into? Then each End call just checks through the previous
> >>      (still incompleted) batches to find its begin partner
> >
> > How is the "begin partner" identified?
> 
>    The current model has always been first in, first out. If you do anything else currently it would/should error out.
> 
> >
> >>      (or we can require always first in, first out). Thus your first
> >>      VecNormEnd(Y, would match the first begin with Y posted and your
> >>      second would match the second.
> >
> > VecNormBegin(X,&norm);
> > function(X);
> > VecNormEnd(X,&norm);
> >
> > would be broken if function() uses split reductions.
> 
>    Do we need to support this far more general model? I think the proposed use case doesn't need it. But you are right, who is to stop someone from writing that code.
> 
>    If we match on both X and &norm then your case will work correctly but if you changed it to
> 
> > VecNormBegin(X,&norm);
> > function(X,&norm);
> > VecNormEnd(X,&norm);
> 
> and function() used split reduction with the &norm argument it would get a false match so yes your request argument would be required. But we could eliminate that possibly by checking for each new begin if its arguments exactly match an outstanding begin and erroring in that case. Thus eliminating the need for the publicly visible request? Does this still support all "reasonable" use cases?
> 
> 
>   Barry
> 
> 
> 
> 
>